Human-Centered AI (Winter Quarter)

Background

We use a specific Data Science Pipeline to drive this course, and discuss specific Human-Centered AI topics and tools related to the different steps of this pipeline. The goal is to be able to use human-centered strategies at every step of the way when developing a Data Science solution. We approach this goal on a week-by-week basis, with every week of the course focusing on a specific part of the pipeline as we outline it below.

Use Case Development
The Use Case Development is the first step of our data science pipeline. We focus on understanding the problem. This means talking to the different stakeholders involved and considering diverse perspectives. The goal is to establish a broad view of the problem and possible solutions, thinking about both end users and also others who might be affected by these solutions.

Design Phase
In the Design Phase, we will focus on clarifying the problem and solution, and ensuring that you are really solving the problem you set out to solve. This will include questioning if the problem that you’re solving is even a problem. The Design Phase will include exploring user needs, prototyping initial solutions, and engaging stakeholders in the process.

Data Procurement
During the Data Procurement Phase, we start considering what data is realistically attainable, and what privacy risks, copyright issues, and potentially sensitive data we need to collect. This phase reviews inputs and model features, and investigates/identifies bias in the data. Ultimately, the goal is to only use high quality data from the start, and only gather the data needed.

Model Building
While starting to think about Model Building, we need to think about how powerful the model is, how long it will take to train, and any unexpected repercussions of the solution. We want to choose the right features and models, aligning the model to the user, while keep checking for biases. The end goal is to develop the most cost-effective, safe, and efficient model.

Testing & Deployment
In the Test & Deployment Phase, we need to think about what metrics to use to evaluate our model, and how to make the model available and potentially open source. This includes defining who should have access to the model, and what the worst case scenarios are in case of failure. Overall, it is important to implement ways to explain the model to users, and test with stakeholders in mind.

Monitoring
The Monitoring Phase is continuous and needs to ensure that the Data Science system is always performing in a human-centered way. In this phase, we need to check how it performs in real-world settings, and ensure that there is a way for users to communicate any issues with the model. This phase also focuses on identifying and explaining errors, and developing good fallback plans.

Course Description

This course will helps consider AI through a “human-first” approach. A “human-first” approach means creating AI systems where human perspective and needs drive technological innovations throughout all stages of the systems’ design (data collection, learning models, inference strategies, interaction paradigms, validation, deployment, evaluation, and maintenance).

With this in mind, this course is meant to teach future Data Scientists how to take this human-centered approach to making decisions throughout the Data Science Pipeline. Throughout the course, we ask one fundamental question: “How can we use design thinking to ensure Data Science and machine learning models are more transparent, approachable, and equitable?”

More information

See DSC 266R (Human-Centered AI) on Canvas: https://canvas.ucsd.edu