Assistive Technology

Instructor: Jennifer Mankoffjmankoff@cs.cmu.edu
Spring 2005

HCII, 3601 NSH, (W)+1 (412) 268-1295
Office hours: By Appointment & 1-2pm Thurs

Course Description

This class will focus on computer accessibility, including web and desktop computing, and research in the area of assistive technology.

The major learning goals from this course include:

  • Develop an understanding of the relationship between disability policy, the disability rights movement, and your role as a technologist. For example, we will discuss we will discuss the pros and cons and infrastructure involved in supporting mainstream computer applications rather than creating new ones from scratch.
  • Develop a skill set for basic design and evaluation of accessible web pages and desktop applications.
  • Develop familiarity with technologies and research relating to accessibility including a study of optimal font size and color for people with dyslexia, word-prediction aids, a blind-accessible drawing program,
  • Develop familiarity with assistive technologies that use computation to increase the accessibility of the world in general. Examples include memory aids, sign-language recognition, and so on.

Requirements

Students will be expected to do service work with non-profits serving the local disabled community during one to two weekends of the start of the semester. This course has a project component, where students will design, implement, and test software for people with disabilities. Additionally, students will read and report on research papers pertinent to the domain.

Grading will be based on service work (10%); the project (60%); and class participation, including your reading summary and the lecture you lead (30%).

Other relevant documents

Course CalendarAssignmentsBibliography

Prerequisites

Prerequisites for this class are: Familiarity with basic Human Computer Interaction material or consent of the instructor (for undergraduate students)

It is recommended that you contact the instructor if you are interested in taking this class.

James Gan

James Gan is a M.S. Technology Innovation student at the Global Innovation Exchange program at the University of Washington. He is working with Megan Hofmann on a project expanding on the work of her paper “PARTs: Expressing and Reusing Design Intent in 3D Models”, particularly towards allowing the system to create Advanced Tactile Maps. He pursues numerous personal projects, and is an avid Hackathon attendee, having won prizes from Google, BlackRock, and Bloomberg. He hopes to grow his Computer Science skills as much as possible while a student, to help him pursue becoming a Product Manager and potentially pursuing a Ph.D. in the future.
Previously, James was a Program Manager and Consultant at srnd.org, working with Microsoft Philanthropies and managing CodeDay, a series of 24 hour events to promote CS education. Through this role, he was able to promote equality in CS education and get hundreds of students from underrepresented backgrounds to pursue CS studies. He graduated from Cornell University in 2018 with a B.A. in Economics with minors in Computer Science, Information Science, and Asian American Studies.
You can find more information about him at https://bellevue.tech

Automatically Tracking and Executing Green Actions

We believe that self-reporting is a limiting factor in the original vision of StepGreen.org, and this component of our research has begun to explore alternatives. For example, we showed that financial data can be used to extract footprint information [1], and in collaboration with researchers at Intel and University of Washington, we used a mobile device to track and visualize green transportation behavior in the Ubigreen project (published at CHI 2009 [2]). We have also worked on algorithms to predict the indoor location of work and home arrival time of residential building occupants so as to automatically minimize thermostat use [3, 4]. Finally, we moved away from individual behavioral remedies to structural remedies by exploring tools that could help tenants to pick greener apartments [5]

[1] J. Schwartz, J. Mankoff, H. Scott Matthews. Reflections of everyday activity in spending data. In Proceedings of CHI 2009.  (Note). (pdf)

[2] J. Froehlich, T. Dillahunt, P. Klasnja, J. Mankoff, S. Consolvo, B. Harrison, J. A. Landay, UbiGreen: Investigating a Mobile Tool for Tracking and Supporting Green Transportation Habits. In Proceedings of CHI 2009. (Full paper) (pdf)

[3] Indoor-ALPS: an adaptive indoor location prediction system Christian Koehler, Nikola Banovic, Ian Oakley, Jennifer Mankoff, Anind K. Dey
UbiComp ’14 Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2014

[4] TherML: occupancy prediction for thermostat control Christian Koehler, Brian D. Ziebart, Jennifer Mankoff, Anind K. Dey UbiComp ’13 Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, 2013

[5] Jennifer Mankoff, Dimeji Onafuwa, Kirstin Early, Nidhi Vyas, Vikram Kamath Cannanure: Understanding the Needs of Prospective Tenants. COMPASS 2018: 36:1-36:10

Lyme Disease’s Heterogeneous Impact

An ongoing, and very personal thread of research that our group engages in (due to my own journey with Lyme Disease, which I occasionally blog about here) is research into the impacts of Lyme Disease and opportunities for helping to support patients with Lyme Disease. From a patient perspective, Lyme disease is as tough to deal with as many other more well known conditions [1].

Lyme disease can be difficult to navigate because of the disagreements about its diagnosis and the disease process. In addition, it is woefully underfunded and understudied, given that the CDC estimates around 300,000 new cases occur per year (similar to the rate of breast cancer) [2].

Bar chart showing that Lyme disease is woefully under studied.

As an HCI researcher, I started out trying to understand the relationship that Lyme Disease patients have with digital technologies. For example, we studied the impact of conflicting information online on patients [3] and how patients self-mediate the accessibility of online content [4]. It is my hope to eventually begin exploring technologies that can improve quality of life as well.

However, one thing patients need right away is peer reviewed evidence about the impact that Lyme disease has on patients (e.g. [3]) and the value of treatment for patients (e.g. [4]). Here, as a technologist, the opportunity is to work with big data (thousands of patient reports) to unpack trends and model outcomes in new ways. That research is still in the formative stages, but in our most recent publication [4] we use straightforward subgroup analysis to demonstrate that treatment effectiveness is not adequately captured simply by looking at averages.

This chart shows that there is a large subgroup (about a third) of respondents to our survey who reported positive response to treatment, even though the average response was not positive.

There are many opportunities and much need for further data analysis here, including documenting the impact of differences such as gender on treatment (and access to treatment), developing interventions that can help patients to track symptoms, manage interaction within and between doctors, and navigate accessibility and access issues.

[1] Johnson, L., Wilcox, S., Mankoff, J., & Stricker, R. B. (2014). Severity of chronic Lyme disease compared to other chronic conditions: a quality of life survey. PeerJ2, e322.

[2] Johnson, L., Shapiro, M. & Mankoff, J. Removing the mask of average treatment effects in chronic Lyme Disease research using big data and subgroup analysis.

[3] Mankoff, J., Kuksenok, K., Kiesler, S., Rode, J. A., & Waldman, K. (2011, May). Competing online viewpoints and models of chronic illness. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 589-598). ACM.

[4] Kuksenok, K., Brooks, M., & Mankoff, J. (2013, April). Accessible online content creation by end users. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 59-68). ACM.

 

Understanding gender equity in author order assignment

Academic success and promotion are heavily influenced by publication record. In many fields, including computer science, multi-author papers are the norm. Evidence from other fields shows that norms for ordering author names can influence the assignment of credit. We interviewed 38 students and faculty in human- computer interaction (HCI) and machine learning (ML) at two institutions to determine factors related to assignment of author order in collaborative publication in the field of computer science. We found that women were concerned with author order earlier in the process:

Our female interviews reported raising author order in discussion earlier in the process than men.

Interview outcomes informed metrics for our bibliometric analysis of gender and collaboration in papers published between 1996 and 2016 in three top HCI and ML conferences. We found expected results overall — being the most junior author increased the likelihood of first authorship, while being the most senior author increased the likelihood of last authorship. However, these effects disappeared or even reversed for women authors:

Comparison of regression weights for author rank (blue) with author rank crossed with gender (orange). Regression was predicting author position (first, middle, last)

Based on our findings, we make recommendations for assignment of credit in multi-author papers and interpretation of author order, particularly with respect to how these factors affect women.

3D Printed Wireless Analytics

Wireless Analytics for 3D Printed Objects: Vikram Iyer, Justin Chan, Ian Culhane, Jennifer Mankoff, Shyam Gollakota UIST, Oct. 2018 [PDF]

We created a wireless physical analytics system works with commonly available conductive plastic filaments. Our design can enable various data capture and wireless physical analytics capabilities for 3D printed objects, without the need for electronics.

We make three key contributions:

(1) We demonstrate room scale backscatter communication and sensing using conductive plastic filaments.

(2) We introduce the first backscatter designs that detect a variety of bi-directional motions and support linear and rotational movements. An example is shown below

(3) As shown in the image below, we enable data capture and storage for later retrieval when outside the range of the wireless coverage, using a ratchet and gear system.

We validate our approach by wirelessly detecting the opening and closing of a pill bottle, capturing the joint angles of a 3D printed e-NABLE prosthetic hand, and an insulin pen that can store information to track its use outside the range of a wireless receiver.

Selected Media

6 of the most amazing things that were 3D-printed in 2018 (Erin Winick, MIT Technology Review, 12/24/2018)

Researchers develop 3D printed objects that can track and store how they are used (Sarah McQuate), UW Press release. 10/9/2018

Assistive Objects Can Track Their Own Use (Elizabeth Montalbano), Design News. 11/14/2018

People

Students

Vikram Iyer
Justin Chan
Ian Culhane

Faculty

Jennifer Mankoff
Shyam Gollakota

Contact: printedanalytics@cs.washington.edu

Minxuan Gao

Hi, I’m Minxuan Gao and I’m a senior in Tsinghua University majoring in Software Engineering. I’m always passionate about creating new and innovative way of people interacting with every day objects by seeing, touching, listening using data-driven methods. My research focus lies in Human Computer Interaction and I am currently working on the SPRITEs project.

Yasaman Sefigar

Headshot of Yasaman wearing a green scarf smiling

I am a PhD student at the University of Washington’s Paul G. Allen School of Computer Science and Engineering. My current research is focused on human behavior modeling. More specifically, I model and study routine behaviors and the impact of external events on them in the context of wellbeing and mobility. I am also interested in end-user tools and interfaces to improve data collection, exploration, and analysis processes.

My past research spans from designing interfaces for end-user robot programming, to modeling human-object interactions in realistic videos, to studying affective haptic human-robot interaction for psychological enrichment.

My Google Scholar page is https://goo.gl/D1QbSJ

Some recent projects (see more)

Venkatesh Potluri

Venkatesh Potluri is a Ph.D. student at the Paul G. Allen Center for Computer Science & Engineering at University of Washington. He is advised by Prof Jennifer Mankoff and Prof Jon Froehlich. Venkatesh believes that technology, when designed right, empowers everybody to fulfill their goals and aspirations. His broad research goals are to upgrade accessibility to the ever-changing ways of our interactions with technology, and, improve the independence and quality of life of people with disabilities. These goals stem from his personal experience as a researcher with a visual impairment. His research focus is to enable developers with visual impairments perform a variety of programming tasks efficiently. Previously, he was a Research Fellow at Microsoft Research India, where his team was responsible for building CodeTalk, an accessibility framework and a plugin for better IDE accessibility. Venkatesh earned a master’s degree in Computer Science at International Institute of Information Technology Hyderabad, where his research was on audio rendering of mathematical content.

You can find more information about him at https://venkateshpotluri.me

Xin Liu

Xin is a first-year Ph.D. student with Jennifer Mankoff and Shwetak Patel in the Paul G. Allen School of Computer Science & Engineering at the University of Washington – Seattle. Prior to joining UW, he obtained a Bachelor’s degree in computer science from the University of Massachusetts Amherst in 2018. While at UMass Amherst, he received a 21st Century Leaders Award, Rising Researcher Award, and Outstanding Undergraduate Achievements Award. He is interested in using wearable sensing, human-computer interaction and machine learning to advancing healthcare.

Website: https://homes.cs.washington.edu/~xliu0/