A multimodal interface combines printed worksheets with digital tablets and speech recognition to support speech-language therapy exercises. Users write or tap on paper prompts to trigger audio feedback, while the system provides real-time pronunciation guidance, enhancing patient engagement and clinician efficiency. Qualitative evaluation shows improved therapy outcomes by blending tactile and audiovisual modalities.">
After a stroke or brain injury, it may be more difficult to understand language and communicate with others. Speech-language therapy may help an individual regain language and cope with changes in their communication abilities. Our research examines the process of speech-language therapy with an emphasis on the practices of therapists working with adults with aphasia and apraxia of speech. This paper presents findings from field work undertaken to inform the design of a mixed paper-digital interface prototype using multimodal digital pens. We describe and analyze therapists' initial reactions to the system and present two case studies of use by older adults undergoing speech-language therapy. We discuss the utility of multimodal paper-digital interfaces to assist therapy and describe our vision of a system to help therapists independently create custom interactive paper materials for their clients.