MultiMemoHome: Multimodal Reminders Within the Home

Lead Research Organisation: University of Edinburgh
Department Name: Sch of Informatics

Abstract

Reminder applications deliver alarms, warnings, and other contextually relevant information to users. There is already a significant body of research on the design of alarms and warnings for workplaces such as aircraft cockpits [5, 6] and hospitals [7]. This work has been dis-tilled into guidelines that lay out many of the features required for effective alarms. However, the home is a very different interaction space to the workplace [8]. Home users are not highly trained experts, tasks are much less tightly controlled and there are many subtle social aspects that are very different to those formalised contexts. These challenges are particularly prominent in home care applications, where reminders can be both vitally important (e.g. medication reminders) and sensi-tive (e.g. toileting). Therefore, many existing guidelines are not applicable in home care settings. Basic HCI re-search is needed to help designers create effective re-minders for this key environment.There are many uses for reminders in the home, from reminders to take medication, to calendar, event and appointment reminders. Most previous work on reminder systems has focused on aspects such as sched-uling the reminders at the most convenient time [9] and ensuring that the infrastructure of the reminder system is fast, reliable, and secure [10]. However, the existing research does not adequately address the three key is-sues of sensory impairment, social context and user preference. Users of home care systems are far more likely to have one or more sensory impairments, such as sight or hearing loss. Due to a lack of appropriate com-mercially available reminder solutions for this user group, many have to rely on friends, family or technologies such as automatic pill dispensers and cooking timers. Making reminders accessible via different modalities is crucial to ensure users will be able to receive the reminder despite their impairments. For example, if a user has moderate hearing loss, standard auditory alarms could be reinforced by olfactory alarms. Although the complex social context of the home as an interaction space impacts the design and presentation of reminders, this has received little attention. Currently, reminders are presented with no awareness of the social setting, often resulting in reminders being switched off to avoid annoyance or embarrassment. Instead, systems should be able to present sensitive reminders in a more subtle way. For example, when visitors are present, toileting reminders could be given as vibration to the skin or a non-speech sound that only the intended recipient can decode. Finally, traditional reminders do not always fully take into account user preferences. In current commercial products the range of reminder types is often limited (with a focus on auditory alarms), as is the ability to easily switch between them. A reminder system will be inef-fective if the user switches it off because the reminders are annoying. Allowing users to adapt the reminders to their own preferences using an alternative modality would make them more acceptable. With an increase in the ageing population, and a drive to promote self management of care in the home [11], there is an increased need for configurable multimodal reminder systems that can support people staying in their own homes independently for longer. There are many potential stakeholders who might want to have input into the configuration of the reminders, for example, end users, carers, family, manufacturers and health and social care professionals. Providing tools to support configuration is important to allow the creation of re-minders that will be acceptable to all involved.The key research question in this project is: How can we design reminder systems for the home that are effective, adaptable, acceptable and accessible to users?

Publications


10 25 50
Wolters M (2012) Hold that thought
Wolters MK (2015) Can older people remember medication reminders presented using synthetic speech? in Journal of the American Medical Informatics Association : JAMIA
Wolters, M (2010) Designing Usable and Acceptable Reminders for the Home in Proc. AAATE Workshop AT Technology Transfer, Sheffield, UK
Wolters, M (2010) Evaluating speech synthesis intelligibility using Amazon Mechanical Turk in Proc. 7th Speech Synthesis Workshop (SSW7)
 
Description 1. Design of speech reminders: Spoken reminders should use familiar words, take into account a person's habits, and be played at a convenient time without violating privacy. If those rules are followed, a computer-generated voice can be used without jeopardising intelligibility. It is even feasible to use compressed speech (spearcons) to provide reminders.

2. Evaluating speech reminders: Crowdsourcing platforms such as Amazon Mechanical Turk are valid tools for investigating the intelligibility of synthetic speech. The Matrix sentences, simple meaningful sentences designed for use in audiology, may be too simple and learned too easily to be useful for evaluating synthetic speech. Synthetic speech is highly intelligible given background noise types typically found in the home such as music and chat, and it is also highly intelligible in mildly reverberant rooms such as kitchens and living rooms.

3. Dual tasking: We found that spoken reminders do not appear to unduly disrupt background tasks that require serial recall. However, more work is needed to identify appropriate cognitive models for predicting the impact of different kinds of reminders on tasks.
Exploitation Route The project has developed a methodology for the evaluation of synthetic speech in clean, noisy, and reverberant conditions. This work has been incorporated into the Blizzard Challenge, http://www.synsig.org/index.php/Blizzard_Challenge, the annual international evaluation of speech synthesis systems. This work also influenced the EC FP7 FET project, LISTA.

Our work in speech driven Reminder Systems forms a basis for future work in dialogue systems, such as in the EU project Help4Mood.

Our work is influencing current and future work in assistive technology. For example, Steve Renals was invited to talk to the assistive technology group at INESC-ID, Lisbon, who are following up on work done in this project.
Sectors Creative Economy,Digital/Communication/Information Technologies (including Software),Healthcare,Culture, Heritage, Museums and Collections
URL http://www.multimemohome.org
 
Description Our findings have been used and disseminated through a variety of routes: 1. We have formed a collaboration with Action on Hearing Loss to increase support for people with hearing loss with low mood and/or cognitive problems through the health services. 2. Maria Wolters disseminated relevant project findings through her research blog, mariawolters.wordpress.com, and through tweeting about relevant research. 3. There was a final project workshop that brought together participants from social care, health care, and academia.
First Year Of Impact 2011
Sector Digital/Communication/Information Technologies (including Software),Healthcare
Impact Types Cultural,Societal,Economic,Policy & public services
 
Description Can computer-generated voices be used to deliver telecare services to people with hearing problems?
Amount £58,036 (GBP)
Funding ID CZG/2/495 
Organisation Scottish Government 
Department Chief Scientist Office (CSO)
Sector Public
Country United Kingdom of Great Britain & Northern Ireland (UK)
Start 12/2011 
End 12/2012
 
Description KAIST seminar 2012 Wolters 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Seminar at Korea Advanced Institute of Science & Technology (KAIST), Dept Math Sci - "Designing Reminder Systems for Older People - What is the Context?". Invited talk has sparked a long-term collaboration
Year(s) Of Engagement Activity 2012
 
Description SICSA talk Wolters 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach Regional
Primary Audience Postgraduate students
Results and Impact Talk at the SICSA Workshop on Technology for Health and Wellbeing by Maria Wolters, "The Minimal effective dose of reminder technology"
Year(s) Of Engagement Activity 2014
 
Description seminar at INESC 2012 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Talk entitled "Assistive Speech Technology" at INESC-ID, Lisbon
Year(s) Of Engagement Activity 2012
 
Description talk at Rijksuniversiteit Groningen 2010 Wolters 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Talk entitled "Monitoring and Reminding: How Much is too much?" at Rijksuniversiteit Groningen
Year(s) Of Engagement Activity 2010
 
Description talk at TU Berlin 2010 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Postgraduate students
Results and Impact Invited talk, "Usability of Spoken Dialogue Systems" at TU Berlin
Year(s) Of Engagement Activity 2010