Earlier | HXI - Human-centered eXtended Intelligence

VOLI: Voice Assistant for Quality of Life and Healthcare Improvement in Aging Populations

Thu, 23 Sep 2021 00:00:00 +0000

Overview

According to the latest US Census Bureau predictions, by 2035 older people are projected to outnumber children for the first time in US history. This brings significant societal challenges based on their unique living and health-related conditions stemming from reduced sensory, motor, and cognitive capabilities, as well as multiple chronic conditions. Technology can play a pivotal role in meeting the needs of older adults in ways that preserve their independence. Voice represents a natural choice for interaction between an aging individual and their caregivers, social networks, and healthcare providers, and it becomes key for those with visual or mobility impairment.

We are working on a personalized and context-aware voice-based digital assistant to improve the quality of life and the healthcare of older adults, and consequently, to reduce caregiving burden and optimize the interactions with healthcare and service providers.

We strive for innovations in natural language understanding, deep learning, and human-computer interfaces that leverage information from EHRs, clinical ontologies, and novel patient-level terminologies to support among others the clinical use case of detecting symptom changes and medication side effects.

More Info here: http://voli.ucsd.edu

Funding and External Collaborations

VOLI is a NIH/NSF Smart and Connected Health (SCH) funded by the National Institute of Aging (NIA) at NIH. It is a collaboration between the HXI Lab and a number of experts at UC San Diego’s Qualcomm Instititute, Computer Science and Engineering, and School of Medicine in healthcare, expert systems in clinical care, EHR integration, the aging population, patient monitoring, patient self-report, machine learning for natural language processing and understanding, experimental prototyping, field studies, and software engineering of large-scale systems.

Mitigating Uncertainty in Online Learning at Scale

Wed, 29 Sep 2021 00:00:00 +0000

Overview

Online learners frequently encounter uncertainity (e.g., confusion) during their study sessions. Unlike in-person educational settings in which learners can conveniently access support from instructors, online students are usually left to address uncertainty on their own. Suboptimal methods of resolving uncertainty (e.g., navigating the sea of information on the web or asking questions in discussion fora with delayed responses) can lead to learning inefficiency, frustration, and lack of productivity.

In this line of works, we first identify and contextualize the breakdowns of remote learning in different settings, such as formal undergraduate education, MOOC (Massive Open Online Courses) users, and healthcare training. We then aim to improve remote learning by leveraging minimally-invasive user context and providing timely AI and peer support.

Our works in this space are well-recognized in the media: UC San Diego News Center, ACM TechNews, Engineering.com, CSE Alumni Magazine

Funding and External Collaborations

This project is a collaboration between Computer Science and Engineering (CSE) and Cognitive Science (COGS) departments at UC San Diego and Carnegie Mellon University.

PrototipAR/PintAR: Rapid Prototyping in Augmented Reality

Mon, 27 Sep 2021 00:00:00 +0000

Overview

Over the last few years, we have seen a growth of developer-oriented tools for Augmented Reality (AR) such as Microsoft’s Mixed Reality toolkit as well as Application Programming Interfaces (APIs) exposed by game engines like Unity3D and Unreal Engine that attempt to ease the entry to AR authoring.

However, these toolkits still require a high degree of programming proficiency, presenting multiple barriers and a steep learning curve for designers or non-programmers, which results in slow iteration cycles that impede creativity and leave designers fatigued.

Low-fidelity prototyping methods, especially sketching, facilitate ideation and experimentation and are natural to users from different backgrounds. In this project we combine the advantages of sketching with the unique affordances of head-mounted AR displays by enabling direct manipulation of digital content in AR while relying on a digital pen user interface for sketching on a tablet. PrototipAR and PintAR allow users to specify interactive behaviors without writing code and provide an implicit state machine to express different stages of an experiences. They also provide users with an aerial overview of digital content to facilitate debugging through a world-in-miniature view.

ChronoSense and Lab-in-a-Box

Mon, 27 Sep 2021 00:00:00 +0000

Screenshot of the ChronoSense UI. This tool enables a user to simultaneously capture multiple streams of multimodal data for use at a later point in time. It also allows for real-time and near-time analysis through it’s extensible plugin interface.

Overview

ChronoSense was born to serve researchers of all kinds helping them more easily conduct observational research. Temporally aware and aligned data capture can be incredibly difficult when multiple software has to be used to collect disparate data from cameras, microphones, screen capture software, eye trackers, and more. Post-hoc alignment can be time-consuming as well as technically challenging, requiring special hardware, environment artifacts, or even special computational tooling. By providing a unified environment for the capture of multiple data streams, ChronoSense massively simplifies the issue, giving observational researchers a tool to easily and quickly record data

Moving beyond observational research we enter into interventional studies, where the goal is to not just study humans but to also generate some sort of artifact that may provide some sort of feedback, support behavior modification, or otherwise prompt reflection that facilitates changes. On this front ChronoSense seeks to empower researchers with a data capture tool that can be extended to also perform inference as well as inform intervention design.

HoloCPR: Designing and Evaluating a Mixed Reality Interface for Time-Critical Emergencies

Thu, 23 Sep 2021 00:00:00 +0000

Overview

Performing time-critical procedures such as Cardiopulmonary Resuscitation (CPR) usually requires trained individuals on the scene. Even with aids available, most bystanders do not attempt resuscitation due to panic or fear of failing, often at the cost of the victim’s life.

To better understand the use of mixed reality for resuscitation guidance, we investigate if spatially localized instructions are better than those on a 2D screen. We propose Mixed Reality (MR) as a compelling medium to support time-critical emergencies, and study its use in this context through an iterative user-centered design process.

Our research outlines a number of key considerations for the design of time-critical emergency interfaces that led to the creation of HoloCPR, an MR application providing real-time instructions for resuscitation for novices through a combination of visual and spatial cues.

Embodied Coding

Thu, 23 Sep 2021 00:00:00 +0000

Overview

This project creates an environment in which students can practice pair programming in an AR environment. Pair programming is a software development technique in which two programmers work together at one workstation, on the same piece of code. Wearing AR headsets, students manipulate code in three-dimensional space, using familiar gestures such as pointing and grabbing. The use of an AR programming environment that immerses students in their code, forms a more intuitive and collaborative computing experience that engages learners with low confidence in programming, while supporting growth in their computational thinking abilities.

This human-centered AR coding platform is being developed for the creation of three-dimensional assets, artwork, and computational logic. This platform will be a merged digital/physical workspace where spatial representations of code, interactive outputs, and user editing activities are simultaneously located. While wearing AR headsets, learners will manipulate virtual code blocks in real space to assemble programs, and they will debug their code by evaluating the representations that they create. The approach will build on the accessibility and sense of play in successful visual learning technologies (e.g., Scratch), but will leverage regular patterns of perception, action, and social interaction in the three-dimensional physical world. The goal is to increase participation and interest in groups traditionally underrepresented in the educational and career pathways of computer science, including females and some minority students, who often exhibit lower confidence in STEM-related abilities relative to other students.

More Info here: https://xrdesign.github.io/

Funding and External Collaborations

The Embodied Coding project is an NSF Education & Human Resource funded project (2017042). It is a collaboration between the HXI Lab and the Arthur C. Clarke Center for Human Imagination.

DIY Masks

Thu, 23 Sep 2021 00:00:00 +0000

Overview

These projects created N95 half masks that were manufactured and donated to hospitals in the San Diego - Tijuana region. The masks' filtration abilities were tested and instructions made for the DIY creation of these masks. Check out the instructions and extra info for detailed testing results, materials, tips, and more.

Silicone Mask

More Info + Instructions

This is a silicone mask. The silicone allows for a much better seal around the face, allows for resuablitiy, replaceable filters, and easy cleaning.

To make the mask, a mold is 3D printed. 2 part silicone (available in popular hardware stores) is poured into the mold and allowed to set. Once set, the silicone is removed, trimmed, and is ready to go. The mold can be re-used an infinite number of times and shared with others.

Triton Mask

More Info + Instructions

This is a polypropelene mask similar to the everyday N95 and surgical masks people wear. It is extremely cheap and easy to manufacture on both individual and massive scales, with a metal nose piece to make the mask form to the face.

To make the mask, cut or laser cut the form from a polypropelene or fabric sheet (dimmensions can be found in More Info). Cut the aluminum strip to size, use a hair straightener, bag sealer, or other hot device to melt the mask edges and nose piece into place. You now have a mask!

Earlier | HXI - Human-centered eXtended Intelligence

VOLI: Voice Assistant for Quality of Life and Healthcare Improvement in Aging Populations

Overview

Funding and External Collaborations

Mitigating Uncertainty in Online Learning at Scale

Overview

Funding and External Collaborations

PrototipAR/PintAR: Rapid Prototyping in Augmented Reality

Overview

ChronoSense and Lab-in-a-Box

Overview

HoloCPR: Designing and Evaluating a Mixed Reality Interface for Time-Critical Emergencies

Overview

Embodied Coding

Overview

​

Funding and External Collaborations

DIY Masks

Overview