Weichen Liu | HXI - Human-centered eXtended Intelligence

CSE 291A / DSC 266R

Thu, 17 Oct 2024 00:00:00 +0000

Human-Centered AI

Background

Artificial intelligence permeates the fabric of human society, and its ability to generate data-driven insights and recommendations has potential benefits at an unprecedented scale, particularly for underserved and marginalized communities. Together with that promise, AI brings new risks: it can deepen existing inequalities, reinforce injustices, and create deeper disconnects within societies. Many of those risks stem from a growing gap between a technology-centric approach to building AI and the complex socio-cultural contexts the technology ends up embedded in.

This course takes a human-first approach instead. A human-first approach means creating AI systems where human perspective and needs drive the technological innovations throughout every stage of a system’s design: data collection, learning models, inference strategies, interaction paradigms, validation, deployment, evaluation, and maintenance.

Course Description

The course teaches future data scientists how to carry that human-centered approach through the decisions they make across the Data Science Pipeline. One question runs through the whole quarter:

How can we use design thinking to ensure data science and machine learning models are more transparent, approachable, and equitable?

Each week focuses on a specific stage of the pipeline, from use case development and the design phase onward, and pairs it with the human-centered topics and tools that belong at that stage. Students work through case studies and engage directly with the socio-technical questions every data scientist should be able to reason about when designing, building, and deploying AI systems.

The course also brings in outside voices on the themes that cut across the whole pipeline, including sessions on ethics, user-centered design, and safety.

Enrollment

The course is offered jointly as CSE 291A (Topics in AI) and DSC 266R, and is open to graduate students across computer science and data science.

CSE 165

Mon, 30 Mar 2026 00:00:00 +0000

3D User Interaction (Spring Quarter)

Background

From video games to mobile augmented reality, 3D interaction is everywhere. Choosing to use 3D input or 3D displays is not by itself enough: 3D user interfaces (3D UIs) have to be designed carefully for the experience to work.

This course covers the full spectrum of emerging applications for 3D UIs and presents an array of pioneering techniques and technologies. It works through the theoretical foundations, analyses devices and techniques, sets out empirically validated design guidelines, and illustrates the key concepts with running case studies.

Classes are held in the XR Lab (CSE 3219), with lectures twice a week and a dedicated lab session on Fridays.

Goals

By the end of the quarter, students will have gained an understanding of:

Essentials of HCI and human factors centered around 3D UIs
How 3D UIs have evolved, and the hard problems that remain
3D UIs in console gaming, VR, augmented reality (AR), mobile and wearable computers, and remote collaboration
How 3D input and output let users perceive and act much as they do in the physical world
3D output devices, including visual, auditory, haptic, and tactile displays
3D input devices, including traditional, special purpose, and direct human input
3D interaction techniques for common tasks such as selection, manipulation, navigation, and system control
Strategies for designing and developing 3D UIs
Evaluating existing 3D UIs, and previewing their future

Course Structure

The course combines team assignments with a final group project. An introductory project gets students working with XR and Unity, followed by three projects built on the material delivered in class, each completed in teams of two using the XR hardware made available for it:

Project 0: Intro to XR and Unity
Project 1: Selection and Manipulation
Project 2: Travel and Wayfinding
Project 3: Embodied Agent and AR

Projects are demonstrated both as a video submission and in person to the course staff. A midterm in Week 7 covers the material up to that point, and the quarter closes with a final group project.

Textbook

Lectures loosely follow LaViola, Kruijff, McMahan, Bowman, and Poupyrev, 3D User Interfaces: Theory and Practice (2nd Edition), Addison-Wesley Professional, 2017. The textbook is recommended rather than required, and an electronic version is available through the UC San Diego library.

Course Description

CSE 165 focuses on the design of three-dimensional user interfaces, VR devices, and interaction techniques. The course consists of lectures, student presentations, and programming assignments, and students create VR applications to learn how VR displays and 3D interactions work.

Prerequisites: CSE 167 or CSE 167R or MATH 155A.

HoloStroke: Holographic Telestroke and Remote Assessment

Thu, 01 Jan 2026 00:00:00 +0000

Overview

Stroke treatment runs against a clock. Thrombolysis reaches fewer than 5% of acute strokes, because the decision has to be made within hours of symptom onset, and it needs both a stroke specialist and adjunctive imaging. Neither is reliably present where and when the patient arrives, and rural and community hospitals carry most of that shortfall.

HoloStroke moves the expertise rather than the patient. Holograms of a patient, or of their imaging, are rendered in three-dimensional space on a HoloLens and transmitted to a distant site, so that a remote stroke neurologist assesses and interacts as if present at the bedside.

Four studies have tested the approach. Holo-Stroke established that patients found remote holographic visits with a stroke provider satisfying and immersive. Holo-Stroke-CTA compared providers reading computed tomographic angiography for large vessel occlusion in holographic space against a conventional DICOM viewer, and found higher reliability and markedly higher satisfaction. Holo-Stroke-ii extended the approach to inpatient immersion. Holo-PACS widened it from stroke to radiographic scans more generally, testing whether holograms improve provider acumen when reading a scan.

Funding and External Collaborations

HoloStroke is a collaboration with the UCSD Stroke Center and HomniHealth.

Designing Smart and Autonomous Vehicles

Thu, 25 May 2023 00:00:00 +0000

Overview

Adoption of autonomous vehicles is held back less by the driving than by the passenger’s judgment of it. Trust has to be calibrated: too little and the technology goes unused, too much and it is relied on where it should not be. We study that calibration at the level of the individual, using virtual reality, biometric measurement, human-centered design, and data science, rather than treating drivers as a single population.

Trust is personal. A survey of 1,457 young adults, modeled with machine learning and SHAP, showed that perceptions of AV risks and benefits, attitudes toward feasibility and usability, institutional trust, prior experience, and mental models predict trust, while psychosocial traits and driving style contribute far less.

Explanations carry that trust, and they can fail. In a simulated driving study with 232 participants, errors in an AV’s explanations reduced comfort in relying on the vehicle, confidence in its ability, and explanation satisfaction, even though the driving itself was identical. Perceived harm and driving difficulty amplified the damage, so the contexts where an explanation matters most are where getting it wrong costs most.

Studying any of this requires observing the driver. DriveSimQuest is a VR driving simulator and research platform on the Meta Quest Pro, capturing gaze, facial expression, hand activity, and full-body gesture in real time, so that studying a driver’s affective state is a matter of designing the study rather than building the rig.

The same setting looks different when the driver is a person. Fleet drivers are involved in collisions that impose severe financial costs and endanger lives, and fleet companies rely on one-to-one coaching to prepare them. We characterize that coaching from both sides, surveying coaches and interviewing drivers, and find that manager-led coaching outperforms self-coaching across experiential outcomes.

Funding and External Collaborations

The autonomous vehicle trust work is a collaboration between the UCSD departments of Cognitive Science, Computer Science and Engineering, and external industry partners. The fleet driver coaching research is conducted with Lytx.

Designing and Studying eXtended Reality

Wed, 16 Sep 2026 00:00:00 +0000

Overview

The lab’s applied XR work lives in its own projects: surgical telementoring in ARTEMIS, holographic stroke assessment in HoloStroke, delineation training in Contouring, remote guidance in eXtended Collaboration. This project holds the work where the headset itself is the object of study.

One line concerns XR as a design medium. Immersive interfaces raise questions that flat displays do not: how precisely a person can draw a non-planar curve in mid-air and whether a pen, a controller, or bare hands support that best; where a step of an instruction should sit so it is readable without occluding the object it refers to; how a user interface should be laid out when the interface occupies the room. PintAR lets designers sketch spatial experiences in the space where they will be used, and HoloCPR places resuscitation guidance on the patient. AcuVR takes the same approach to acupuncture training, where students currently learn from 2D atlases and practice on peers: it simulates customized scenarios from medical imaging and standardized anatomy models, including the sensitive areas that make practice on a person risky.

A second line uses XR as a research instrument. A headset reproduces the same situation for every participant, under experimental control, which makes it a measurement tool. A VR paradigm presents alcohol cues in three dimensions and measures attentional bias toward them, which flat pictorial cues cannot elicit. Wheelchair users explore high-fidelity replicas of physical spaces as embodied avatars, assessing accessibility themselves before travelling. Both studies obtain observations that the real setting would make impractical or unsafe.

Collaborations

This work spans collaborations with UC San Diego Anesthesiology and the School of Medicine, the Department of Psychiatry, Adobe Research, UCLA, and Carnegie Mellon University.

(Holo-PACS): use of holograms to improve provider acumen when reading clinical radiographic scans

Thu, 01 Jan 2026 00:00:00 +0000

Attentional Bias During a Novel Alcohol Cue Virtual Reality Paradigm

Thu, 01 Jan 2026 00:00:00 +0000

Holo-Stroke-ii: Stroke care through stroke hologram teleportation- inpatient immersion

Thu, 01 Jan 2026 00:00:00 +0000

DriveSimQuest: A VR Driving Simulator and Research Platform on Meta Quest with Unity

Sat, 27 Sep 2025 00:00:00 +0000

Holo-Stroke-CTA: Stroke Hologram Teleportation for CTA Large Vessel Occlusion Assessments

Wed, 01 Jan 2025 00:00:00 +0000

Holo-Stroke: Assessing for Immersive Stroke Care Through Stroke Hologram Teleportation

Tue, 01 Oct 2024 00:00:00 +0000