Steven Rick | HXI - Human-centered eXtended Intelligence

Simulated Patients for Clinical Communication Training

Mon, 14 Sep 2026 00:00:00 +0000

Overview

How a clinician talks to a patient is part of the care, not a skill layered on top of it. Communication quality determines whether trust is built, whether a patient discloses what matters, and whether bias is enacted in the room. While that much is established, the opportunity to practice a difficult conversation, and to learn afterwards how it went, is scarce for clinicians and trainees alike.

Our UnBIASED work established that these dynamics are measurable: dominance, warmth, engagement, interactivity, and turn-taking can be extracted from real clinical conversations and modeled against how the interaction was experienced. This project turns that measurement into a training platform. Large language models and expressive speech synthesis drive simulated patients that behave like people rather than assistants: they hold information back, and their affect shifts in response to how they are treated, so trust has to be earned before a patient’s underlying needs surface. Each rehearsed conversation is analyzed with the UnBIASED pipeline and returned through ConverSense feedback, pairing time-aligned visualizations of the interaction with prompts that ask the learner to interpret their own behavior.

ConversHIVe applies the platform to HIV care teams, where implicit bias linked to sexual orientation and race affects access to care for people who have already faced stigma, and where clinics and community-based organizations work under limited time, staffing shortages, and turnover. Its second phase, ConversHIVe-Live, embeds the training in routine HIV service delivery.

EmpathIQ applies it to pre-clinical medical students, where empathy declines during training precisely as students meet the emotional load of clinical work, and where standardized patients and faculty-led workshops are effective but too resource-intensive to scale. EmpathIQ identifies the markers that distinguish empathic communication, varies patient affect within a single encounter, and evaluates whether the resulting feedback changes measured empathy and compassion.

Funding and External Collaborations

ConversHIVe is funded by the California HIV/AIDS Research Program (CHRP) under its Low Barrier Technology Interventions for HIV Prevention and Care program, with a second phase supporting real-world implementation. It is a collaboration between the HXI Lab and the Owen Clinic at UC San Diego, the AntiViral Research Center (AVRC) Community Advisory Board, and San Diego community-based organizations including Christie’s Place, San Ysidro Health Center, and Father Joe’s Villages.

EmpathIQ is supported by a Sanford Research Fellowship from the T. Denny Sanford Institute for Empathy and Compassion at UC San Diego, awarded to Manas Bedmutha, and is conducted with Drs. Lisa Eyler and Federica Klaus of the Sanford Institute and with Dr. Charles Goldberg, Associate Dean for Graduate Medical Education.

UnBIASED: Understanding Biased patient-provider Interactions And Supporting Enhanced Discourse

Thu, 23 Sep 2021 00:00:00 +0000

Overview

Healthcare bias, based on patients’ race, gender, sexual orientation, and other factors, leads to health disparities, such as lack of appropriate treatment and inadequate pain support. Such biases are often unintentional and “hidden” in communication between patients and doctors.

Existing approaches to address hidden bias are limited because they are removed from actual patient-doctor interactions in which bias hides. Technology offers an opportunity to design new approaches that can make den bias more visible and thus addressable.

We are investigating a new approach to address hidden healthcare bias by improving patient-doctor communication in primary care. This approach monitors body language for signs of bias and provides feedback to raise awareness of patients and doctors for opportunities to adjust their communication style.

We are partnering closely with patients and doctors to ensure this approach is guided by their experiences and needs. Through this collaborative effort, we expect to gain a deep understanding of how hidden bias is experienced and how we can address it better in the future.

More Info here: http://unbiased.health

ConverSense

Measurement is only useful if a provider can see it. ConverSense turns the pipeline’s output into feedback: a web application that visualizes how dominance, interactivity, engagement and warmth moved through a visit, and across visits over time. A study with five clinicians across ten patient visits showed the feedback lands only when it is tied to the specific interaction it came from, which is why the visualizations are time-aligned to the conversation rather than reported as scores.

SocialLM

Assessing communication at scale is the bottleneck. SocialLM asks whether large language models can track social behaviors directly from clinical transcripts without fine-tuning, and finds that they can, but unevenly: performance varies by patient race and by segment of the visit. Because that variability is itself an equity problem, the work introduces an agreement-weighted ensemble that improves both accuracy and stability, giving a practical route to social signal tracking at scale.

Funding and External Collaborations

UnBIASED ia a 5-year project, funded by the National Library of Medicine (NLMR01LM013301), and it is a collaboration between the University of Washington and the HXI Lab at UC San Diego. Our ultimate goal is to create tools to support patients and the next generation of doctors to have bias-free interactions that promote healthcare access, quality, and equity.

ARTEMIS: Augmented Reality Technology-Enabled reMote Integrated Surgery

Thu, 23 Sep 2021 00:00:00 +0000

Artistic rendering of ARTEMIS and its features. Left: a Novice Surgeon in Augmented Reality receiving help from a remote expert. Right: a Remote Expert Surgeon in VR interacting with a 3D point-cloud of the patient, and engaging with the novice on a surgical procedure.

Overview

Specialized surgeries are often needed in places where there is no access to surgeons, and timely access to telemedicine helps novice surgeons save lives.

While modern telesurgery is limited to just audio/video, ARTEMIS enables skilled surgeons and novices to share the same virtual space.

In ARTEMIS, expert surgeons in remote sites use Virtual Reality to access a 3D reconstruction of a patient’s body, and instruct novice surgeons on complex procedures as if they were together in the operating room. Novice surgeons in the field can focus on saving the patient’s live while being guided through an intuitive Augmented Reality interface.

More Info here: http://artemis.surgery

Funding and External Collaborations

ARTEMIS is the result of a collaboration between the HXI Lab and the ARC Lab at UC San Diego, the Contextual Roboticsd Institute, and San Diego’s Naval Medical Center.

This project has been funded by the Naval Medical Center San Diego.

ChronoSense and Lab-in-a-Box

Mon, 27 Sep 2021 00:00:00 +0000

Screenshot of the ChronoSense UI. This tool enables a user to simultaneously capture multiple streams of multimodal data for use at a later point in time. It also allows for real-time and near-time analysis through it’s extensible plugin interface.

Overview

ChronoSense was born to serve researchers of all kinds helping them more easily conduct observational research. Temporally aware and aligned data capture can be incredibly difficult when multiple software has to be used to collect disparate data from cameras, microphones, screen capture software, eye trackers, and more. Post-hoc alignment can be time-consuming as well as technically challenging, requiring special hardware, environment artifacts, or even special computational tooling. By providing a unified environment for the capture of multiple data streams, ChronoSense massively simplifies the issue, giving observational researchers a tool to easily and quickly record data

Moving beyond observational research we enter into interventional studies, where the goal is to not just study humans but to also generate some sort of artifact that may provide some sort of feedback, support behavior modification, or otherwise prompt reflection that facilitates changes. On this front ChronoSense seeks to empower researchers with a data capture tool that can be extended to also perform inference as well as inform intervention design.

GenAI and Synthetic Data in Healthcare: Exploring the Design and Use of AI-Generated Data for Interactive Health Systems

Thu, 01 Jan 2026 00:00:00 +0000

Sensing, Understanding, and Augmenting - Methods for Enhancing Human Behavior Analysis using Computational Sensing and Social Signal Processing

Wed, 01 Jan 2025 00:00:00 +0000

Developing a Healthcare Equity Tool: Combining Technology and Stakeholder Engagement to Uncover Hidden Bias in Patient-Provider Interactions

Fri, 01 Jan 2021 00:00:00 +0000

Cognitive Bias in Patient-Provider Communication: Sensing and Design to Reduce Health Disparities

Wed, 01 Apr 2020 00:00:00 +0000

NeuroPose: geriatric rehabilitation in the home using a webcam and pose estimation

Tue, 01 Jan 2019 00:00:00 +0000

SleepBot: encouraging sleep hygiene using an intelligent chatbot

Tue, 01 Jan 2019 00:00:00 +0000

Keystrokes, mouse clicks, and gazing at the computer: how physician interaction with the EHR affects patient participation

Mon, 01 Jan 2018 00:00:00 +0000

Exploring Mixed Reality in Specialized Surgical Environments

Sat, 06 May 2017 00:00:00 +0000

Hands That Speak: An Integrated Approach to Studying Complex Human Communicative Body Movements

Fri, 01 Jan 2016 00:00:00 +0000

Eyes on the clinic: accelerating meaningful interface analysis through unobtrusive eye tracking

Thu, 01 Jan 2015 00:00:00 +0000

LAB-IN-A-BOX: semi-automatic tracking of activity in the medical office

Thu, 01 Jan 2015 00:00:00 +0000

Validating free-text order entry for a note-centric EHR

Thu, 01 Jan 2015 00:00:00 +0000