Chen Chen | HXI - Human-centered eXtended Intelligence

Simulated Patients for Clinical Communication Training

Mon, 14 Sep 2026 00:00:00 +0000

Overview

How a clinician talks to a patient is part of the care, not a skill layered on top of it. Communication quality determines whether trust is built, whether a patient discloses what matters, and whether bias is enacted in the room. While that much is established, the opportunity to practice a difficult conversation, and to learn afterwards how it went, is scarce for clinicians and trainees alike.

Our UnBIASED work established that these dynamics are measurable: dominance, warmth, engagement, interactivity, and turn-taking can be extracted from real clinical conversations and modeled against how the interaction was experienced. This project turns that measurement into a training platform. Large language models and expressive speech synthesis drive simulated patients that behave like people rather than assistants: they hold information back, and their affect shifts in response to how they are treated, so trust has to be earned before a patient’s underlying needs surface. Each rehearsed conversation is analyzed with the UnBIASED pipeline and returned through ConverSense feedback, pairing time-aligned visualizations of the interaction with prompts that ask the learner to interpret their own behavior.

ConversHIVe applies the platform to HIV care teams, where implicit bias linked to sexual orientation and race affects access to care for people who have already faced stigma, and where clinics and community-based organizations work under limited time, staffing shortages, and turnover. Its second phase, ConversHIVe-Live, embeds the training in routine HIV service delivery.

EmpathIQ applies it to pre-clinical medical students, where empathy declines during training precisely as students meet the emotional load of clinical work, and where standardized patients and faculty-led workshops are effective but too resource-intensive to scale. EmpathIQ identifies the markers that distinguish empathic communication, varies patient affect within a single encounter, and evaluates whether the resulting feedback changes measured empathy and compassion.

Funding and External Collaborations

ConversHIVe is funded by the California HIV/AIDS Research Program (CHRP) under its Low Barrier Technology Interventions for HIV Prevention and Care program, with a second phase supporting real-world implementation. It is a collaboration between the HXI Lab and the Owen Clinic at UC San Diego, the AntiViral Research Center (AVRC) Community Advisory Board, and San Diego community-based organizations including Christie’s Place, San Ysidro Health Center, and Father Joe’s Villages.

EmpathIQ is supported by a Sanford Research Fellowship from the T. Denny Sanford Institute for Empathy and Compassion at UC San Diego, awarded to Manas Bedmutha, and is conducted with Drs. Lisa Eyler and Federica Klaus of the Sanford Institute and with Dr. Charles Goldberg, Associate Dean for Graduate Medical Education.

CSE 118

Thu, 23 Sep 2021 00:00:00 +0000

Applications in Ubiquitous Computing (Fall 2021)

Background

“The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it” (M. Weiser)

Ubiquitous computing looks at how we can use technology everywhere and in everything that we do. Examples include Augmented and Virtual Reality, smart sticky notes, table-top displays, home security, everything IoT, wearables, medical technology, HUDs, even your AC’s thermometer control.

While scrolling through memes is fun, this only exploits a tip of what is possible with current technology. Today, even simple web browsers can read your brain, transform your webcam into a smart sensor listen to you (and speak back), create digital instruments, connect people through audio and video in real-time, and so much more.

This class will show how to leverage the power of ubiquitous technologies to create experiences that extend human intelligence and capabilities. This might take the form of new modes of communication, sensor-feedback mechanisms for extra-sensory phenomena, disability technologies, and so much more!

This course will give you hands-on experience with a human-centered workflow that focuses on problems, design, and finally development. Students will be grouped in teams and will design, create, and deliver low-fi prototypes, as well as a working final prototype alongside a final presentation/demo. Based on both the feasibility and refinement of their class work, students may be offered the opportunity to continue their work in a collaborative research project after the course concludes. Independent research credit can be provided in subsequent quarters as CSE 198 / CSE 199.

Course Description

Ubiquitous Computing is a field that considers how technology can fade into the background, permeating space in such an intuitive way as to become unnoticed by its users.

This is a project-based course that focuses on using cheap sensors and networked multimodal devices to create novel and thoughtful technologies. Students will work in groups to identify problems, design technological solutions, and implement those solutions.

Successful students in this class often follow up on their design projects with actual development and deployment of their technological intervention.

More information

See CSE 118 (Ubiquitous Computing) on Canvas: https://canvas.ucsd.edu

CSE 165

Mon, 30 Mar 2026 00:00:00 +0000

3D User Interaction (Spring Quarter)

Background

From video games to mobile augmented reality, 3D interaction is everywhere. Choosing to use 3D input or 3D displays is not by itself enough: 3D user interfaces (3D UIs) have to be designed carefully for the experience to work.

This course covers the full spectrum of emerging applications for 3D UIs and presents an array of pioneering techniques and technologies. It works through the theoretical foundations, analyses devices and techniques, sets out empirically validated design guidelines, and illustrates the key concepts with running case studies.

Classes are held in the XR Lab (CSE 3219), with lectures twice a week and a dedicated lab session on Fridays.

Goals

By the end of the quarter, students will have gained an understanding of:

Essentials of HCI and human factors centered around 3D UIs
How 3D UIs have evolved, and the hard problems that remain
3D UIs in console gaming, VR, augmented reality (AR), mobile and wearable computers, and remote collaboration
How 3D input and output let users perceive and act much as they do in the physical world
3D output devices, including visual, auditory, haptic, and tactile displays
3D input devices, including traditional, special purpose, and direct human input
3D interaction techniques for common tasks such as selection, manipulation, navigation, and system control
Strategies for designing and developing 3D UIs
Evaluating existing 3D UIs, and previewing their future

Course Structure

The course combines team assignments with a final group project. An introductory project gets students working with XR and Unity, followed by three projects built on the material delivered in class, each completed in teams of two using the XR hardware made available for it:

Project 0: Intro to XR and Unity
Project 1: Selection and Manipulation
Project 2: Travel and Wayfinding
Project 3: Embodied Agent and AR

Projects are demonstrated both as a video submission and in person to the course staff. A midterm in Week 7 covers the material up to that point, and the quarter closes with a final group project.

Textbook

Lectures loosely follow LaViola, Kruijff, McMahan, Bowman, and Poupyrev, 3D User Interfaces: Theory and Practice (2nd Edition), Addison-Wesley Professional, 2017. The textbook is recommended rather than required, and an electronic version is available through the UC San Diego library.

Course Description

CSE 165 focuses on the design of three-dimensional user interfaces, VR devices, and interaction techniques. The course consists of lectures, student presentations, and programming assignments, and students create VR applications to learn how VR displays and 3D interactions work.

Prerequisites: CSE 167 or CSE 167R or MATH 155A.

Contouring for Radiation Oncology: Interactive Training and Feedback

Thu, 23 Sep 2021 00:00:00 +0000

Overview

Treating patients with safe and effective radiation therapy depends critically on the precise identification of tumor and nearby normal tissues. This process, known as contouring, refers to identifying and outlining cancer and normal tissues in medical images.

Poor radiation planning has detrimental consequences on patient well-being. Radiation plans that deviate from protocol specifications have substantially decreased survival compared to patients with compliant radiation plans. Given the impact of contouring on patient outcomes, many contouring resources exist. However, practice guidelines rarely translate into real-world clinical practices, primarily due to ineffective methods of development, delivery, and access.

In collaboration with the department of Radiation Medicine at UCSD, we work to replace the traditional one-to-one apprenticeship model with training that is personalized, timely, and available whenever a resident is ready to practice. Three systems carry that agenda.

iContour

iContour is a web-based contouring platform that presents anonymized DICOM cases and gives residents immediate, structured feedback as they delineate. It grew out of needfinding with residents and attendings about when and how contouring feedback actually reaches a trainee, and out of measurements showing that everyday touch devices are accurate enough for real delineation work.

A randomized international trial of iContour has now completed. Secondary analyses of that trial characterize the specific, site-dependent mistakes residents make before and after their clinical rotations, showing that contouring errors are predictable rather than idiosyncratic.

VRContour

VRContour asks what contouring becomes when the anatomy is no longer flat. Residents and attendings work slice by slice on 2D displays even though the structures they are outlining are volumetric. VRContour brings delineation into virtual reality, with sketching techniques designed for 3D exploration and annotation of medical images, and studies which parts of the task actually benefit from immersion.

iConTutor

iConTutor is the next stage. Where iContour delivered practice and feedback, iConTutor aims to turn that into personalized tutoring: using the errors a trainee actually makes, at the disease site they are actually rotating through, to decide what feedback they should see and when.

The completed trial and the mistake analyses are the evidence base. Because errors are predictable and site-dependent, targeted automated tutoring becomes plausible where generic feedback would not be.

Funding and External Collaborations

iContour is funded by Agency for Healthcare Research and Quality (AHRQ). It is a collaboration between the HXI Lab and a number of radiation oncology faculty and residents at UCSD school of medicine.

Designing Smart and Autonomous Vehicles

Thu, 25 May 2023 00:00:00 +0000

Overview

Adoption of autonomous vehicles is held back less by the driving than by the passenger’s judgment of it. Trust has to be calibrated: too little and the technology goes unused, too much and it is relied on where it should not be. We study that calibration at the level of the individual, using virtual reality, biometric measurement, human-centered design, and data science, rather than treating drivers as a single population.

Trust is personal. A survey of 1,457 young adults, modeled with machine learning and SHAP, showed that perceptions of AV risks and benefits, attitudes toward feasibility and usability, institutional trust, prior experience, and mental models predict trust, while psychosocial traits and driving style contribute far less.

Explanations carry that trust, and they can fail. In a simulated driving study with 232 participants, errors in an AV’s explanations reduced comfort in relying on the vehicle, confidence in its ability, and explanation satisfaction, even though the driving itself was identical. Perceived harm and driving difficulty amplified the damage, so the contexts where an explanation matters most are where getting it wrong costs most.

Studying any of this requires observing the driver. DriveSimQuest is a VR driving simulator and research platform on the Meta Quest Pro, capturing gaze, facial expression, hand activity, and full-body gesture in real time, so that studying a driver’s affective state is a matter of designing the study rather than building the rig.

The same setting looks different when the driver is a person. Fleet drivers are involved in collisions that impose severe financial costs and endanger lives, and fleet companies rely on one-to-one coaching to prepare them. We characterize that coaching from both sides, surveying coaches and interviewing drivers, and find that manager-led coaching outperforms self-coaching across experiential outcomes.

Funding and External Collaborations

The autonomous vehicle trust work is a collaboration between the UCSD departments of Cognitive Science, Computer Science and Engineering, and external industry partners. The fleet driver coaching research is conducted with Lytx.

WES 237a

Fri, 08 Jan 2021 00:00:00 +0000

Introduction to Embedded System Design (Winter Quarter)

Course Description

WES 237A - Introduction to Embedded System Design - is a Masters of Advanced Study course in the Wireless and Embedded Systems (WES) program.

The course provides an introduction to embedded systems that stresses practical, hands-on experience with wirelessly connected embedded systems. Students utilize state of the art tools to create novel embedded systems utilizing the PYNQ Z-2 (Links to an external site.) board, and a range of Internet of Things (IOT) sensors and wireless devices.

The course teaches aspects of hardware and software architectures, peripherals and on-board communication protocols, hardware and software input/output interfaces, and real time constraints, combining these concept with a close coupling with the ARM architecture and various synthesized hardware blocks.

Students build their knowledge and experience through five labs that culminate in a final IOT project that will allow them to build a working system based on a variety of sensors and I/O components.

Learning Objectives

After successfully completing this course, a student will have:

Knowledge of embedded systems, peripherals, and communication protocols.
Expertise in C-based coding of ARM architectures utilizing a variety of soft-IP cores and interfaces.
Familiarity with the Xilinx Zynq SoC architecture and tools necessary for development of its embedded core, including the Python-based Jupyter interface
Experience working with, and coding for, wirelessly connected embedded systems
An understanding of the real-time tradeoffs of implementing functionality in software vs hardware.

More information

See WES 237a (Introduction to Embedded System Design) Canvas Page here: https://canvas.ucsd.edu/courses/23874

Designing and Studying eXtended Reality

Wed, 16 Sep 2026 00:00:00 +0000

Overview

The lab’s applied XR work lives in its own projects: surgical telementoring in ARTEMIS, holographic stroke assessment in HoloStroke, delineation training in Contouring, remote guidance in eXtended Collaboration. This project holds the work where the headset itself is the object of study.

One line concerns XR as a design medium. Immersive interfaces raise questions that flat displays do not: how precisely a person can draw a non-planar curve in mid-air and whether a pen, a controller, or bare hands support that best; where a step of an instruction should sit so it is readable without occluding the object it refers to; how a user interface should be laid out when the interface occupies the room. PintAR lets designers sketch spatial experiences in the space where they will be used, and HoloCPR places resuscitation guidance on the patient. AcuVR takes the same approach to acupuncture training, where students currently learn from 2D atlases and practice on peers: it simulates customized scenarios from medical imaging and standardized anatomy models, including the sensitive areas that make practice on a person risky.

A second line uses XR as a research instrument. A headset reproduces the same situation for every participant, under experimental control, which makes it a measurement tool. A VR paradigm presents alcohol cues in three dimensions and measures attentional bias toward them, which flat pictorial cues cannot elicit. Wheelchair users explore high-fidelity replicas of physical spaces as embodied avatars, assessing accessibility themselves before travelling. Both studies obtain observations that the real setting would make impractical or unsafe.

Collaborations

This work spans collaborations with UC San Diego Anesthesiology and the School of Medicine, the Department of Psychiatry, Adobe Research, UCLA, and Carnegie Mellon University.

VOLI: Voice Assistant for Quality of Life and Healthcare Improvement in Aging Populations

Thu, 23 Sep 2021 00:00:00 +0000

Overview

According to the latest US Census Bureau predictions, by 2035 older people are projected to outnumber children for the first time in US history. This brings significant societal challenges based on their unique living and health-related conditions stemming from reduced sensory, motor, and cognitive capabilities, as well as multiple chronic conditions. Technology can play a pivotal role in meeting the needs of older adults in ways that preserve their independence. Voice represents a natural choice for interaction between an aging individual and their caregivers, social networks, and healthcare providers, and it becomes key for those with visual or mobility impairment.

We are working on a personalized and context-aware voice-based digital assistant to improve the quality of life and the healthcare of older adults, and consequently, to reduce caregiving burden and optimize the interactions with healthcare and service providers.

We strive for innovations in natural language understanding, deep learning, and human-computer interfaces that leverage information from EHRs, clinical ontologies, and novel patient-level terminologies to support among others the clinical use case of detecting symptom changes and medication side effects.

More Info here: http://voli.ucsd.edu

Funding and External Collaborations

VOLI is a NIH/NSF Smart and Connected Health (SCH) funded by the National Institute of Aging (NIA) at NIH. It is a collaboration between the HXI Lab and a number of experts at UC San Diego’s Qualcomm Instititute, Computer Science and Engineering, and School of Medicine in healthcare, expert systems in clinical care, EHR integration, the aging population, patient monitoring, patient self-report, machine learning for natural language processing and understanding, experimental prototyping, field studies, and software engineering of large-scale systems.

Mixed-Dimensional Information Spaces

Sun, 29 Oct 2023 00:00:00 +0000

Overview

Information that describes physical work is still authored and consumed in flat documents. A 3D model is reviewed by typing comments into a text box, a repair procedure that plays out across a kitchen arrives as a numbered list on a phone, and a designer using a screen reader has no way to ask what a model looks like. While the task is three-dimensional, its representation is not, and the user absorbs the cost of the translation.

This project builds interfaces that carry content across that boundary, using generative AI and spatial computing. MemoVis creates companion reference images while a reviewer types design feedback, suggesting viewpoints on the 3D model that match the comment and composing an image from them. PaperToPlace transforms existing instruction documents into spatialized mixed-reality experiences, placing each step where it can be read without occluding the object it refers to. SweeperBot opens 3D model repositories to screen reader users, answering natural-language questions about a model by selecting informative viewpoints and describing their content.

Collaborations

This work was carried out largely with Adobe Research, with collaborators including Cuong Nguyen, Jane Hoffswell, Jennifer Healey, Trung Bui, Thibault Groueix, and Alexa Siu.

AffAdapt: AFFect-driven ADAPTive AI Personas for Seamless Conversations

Mon, 02 Nov 2026 00:00:00 +0000

SweeperBot: Making 3D Browsing Accessible Through View Analysis and Visual Question Answering

Thu, 01 Jan 2026 00:00:00 +0000

DriveSimQuest: A VR Driving Simulator and Research Platform on Meta Quest with Unity

Sat, 27 Sep 2025 00:00:00 +0000

Seamless and Efficient Interactions within a Mixed-Dimensional Information Space

Wed, 01 Jan 2025 00:00:00 +0000

MemoVis: A GenAI-Powered Tool for Creating Companion Reference Images for 3D Design Feedback

Wed, 04 Sep 2024 00:00:00 +0000

AcuVR: Enhancing Acupuncture Training Workflow with Virtual Reality

Tue, 02 Jul 2024 00:00:00 +0000

Enhancing Accuracy, Time Spent, and Ubiquity in Critical Healthcare Delineation via Cross-Device Contouring

Sun, 09 Jun 2024 00:00:00 +0000

“I’d be watching him contour till 10 o’clock at night”: Understanding Tensions between Teaching Methods and Learning Needs in Healthcare Apprenticeship

Fri, 23 Feb 2024 00:00:00 +0000

PaperToPlace - Transforming Instruction Documents into Spatialized and Context-Aware Mixed Reality Experiences

Sun, 29 Oct 2023 00:00:00 +0000

Embodied Exploration - Facilitating Remote Accessibility Assessment for Wheelchair Users with Virtual Reality

Tue, 18 Jul 2023 00:00:00 +0000

Screen or No Screen? Lessons Learnt from a Real-World Deployment Study of Using Voice Assistants With and Without Touchscreen for Older Adults

Tue, 18 Jul 2023 00:00:00 +0000

How do Older Adults Set Up Voice Assistants? Lessons Learned from a Deployment Experience for Older Adults to Set Up Standalone Voice Assistants

Mon, 10 Jul 2023 00:00:00 +0000

Design and Development of a Training and Immediate Feedback Tool to Support Healthcare Apprenticeship

Wed, 19 Apr 2023 00:00:00 +0000

Mites - Design and Deployment of a General-Purpose Sensing Infrastructure for Buildings

Tue, 28 Mar 2023 00:00:00 +0000

Investigating Input Modality and Task Geometry on Precision–first 3D Drawing in Virtual Reality

Fri, 14 Oct 2022 00:00:00 +0000

VRContour - Bringing Contour Delineations of Medical Structures Into Virtual Reality

Fri, 14 Oct 2022 00:00:00 +0000

Towards Visualization of Time–Series Ecological Momentary Assessment (EMA) Data on Standalone Voice–First Virtual Assistants

Wed, 20 Jul 2022 00:00:00 +0000

Exploring Needs and Design Opportunities for Virtual Reality-based Contour Delineations of Medical Structures

Tue, 21 Jun 2022 00:00:00 +0000

Evaluating Accuracy, Completion Time and Usability of Everyday Touch Devices for Contouring

Sat, 01 Jan 2022 00:00:00 +0000

Toward a Unified Metadata Schema for Ecological Momentary Assessment with Voice-First Virtual Assistants

Tue, 27 Jul 2021 00:00:00 +0000

Facilitating Remote Design Thinking Workshops in Healthcare: the Case of Contouring in Radiation Oncology

Fri, 01 Jan 2021 00:00:00 +0000

How might an intelligent voice assistant address older adults' health-related needs?

Fri, 01 Jan 2021 00:00:00 +0000

Understanding Barriers and Design Opportunities to Improve Healthcare and QOL for Older Adults through Voice Assistants

Fri, 01 Jan 2021 00:00:00 +0000