Research Projects

Accessibility

accessibility|fabrication|inclusion |data science|all

overview
To revolutionize the creation of assistive technology, our work is divided into three complementary activities

One in 5 Americans has a disability, and the needs of people with disabilities are extremely heterogeneous. Yet accessibility is not a given, and assistive technology today is mostly homogenous and very expensive. Our group is focused on the scientific challenges that can address accessibility across a broad range of domains.

Infant Oxygen Monitoring

Improving healthcare decision making with better workflow and information flow.

Shows someone placing a pen in a cap with two different types of adaptations.

Printable Adaptations

Reprise: A Design Tool for Specifying, Generating, and Customizing 3D Printable Adaptations on Everyday Objects Reprise is a tool for creating custom adaptive 3D printable designs for making it easier to manipulate everything from tools to zipper pulls. Reprise’s library is based on a survey of about 3,000 assistive technology and life hacks drawn from …

Tactile Interfaces to Appliances

Facade is a crowdsourced fabrication pipeline to automatically generate tactile interfaces to appliances for blind people.

Volunteer AT Fabricators

Perry-Hill, J., Shi, P., Mankoff, J. & Ashbrook, D. Understanding Volunteer AT Fabricators: Opportunities and Challenges in DIY-AT for Others in e-NABLE. Accepted to CHI 2017 We present the results of a study of e-NABLE, a distributed, collaborative volunteer effort to design and fabricate upper-limb assistive technology devices for limb-different users. Informed by interviews with …

Making the field of computing more inclusive for people with disabilities

Lazar, J., Churchill, E. F., Grossman, T., Van der Veer, G., Palanque, P., Morris, J. S., & Mankoff, J. (2017). Making the field of computing more inclusive. Communications of the ACM, 60(3), 50-59. More accessible conferences, digital resources, and ACM SIGs will lead to greater participation by more people with disabilities. Improving conference and online material accessibility …

The Tangible Desktop

Mark S. Baldwin, Gillian R. Hayes, Oliver L. Haimson, Jennifer Mankoff, Scott E. Hudson: The Tangible Desktop: A Multimodal Approach to Nonvisual Computing. TACCESS 10(3): 9:1-9:28 (2017) Audio-only interfaces, facilitated through text-to-speech screen reading software, have been the primary mode of computer interaction for blind and low-vision computer users for more than four decades. During this time, the advances that have made visual …

Inclusion

accessibility|fabrication|inclusion |data science|all
Our group also works to address inclusion of other under represented groups. Some of our recent work has looked at gender in academia and other settings.

Picture of potted plants and a bench with the word Africatown in the background, painted in bright red and green colors

Who Gets to Future? Race, Representation, and Design Methods in Africatown

Who Gets to Future? Race, Representation, and Design Methods in Africatown Jasper Tran O’Leary, Sara Zewde, Jennifer Mankoff , Daniela K. Rosner CHI 2019 This paper draws on a collaborative project called the Africatown Activation to examine the role design practices play in contributing to (or conspiring against) the flourishing of the Black community in …

Fabrication

accessibility|fabrication|inclusion |data science|all
President Obama once said ‘3D printing […] has the potential to revolutionize the way we make almost everything” [Obama 2013].’ One might say that computers have the same ability. Together, computational advances and fabrication can revolutionize personal problem solving.

Our group is embarking on an effort to enhance fabrication technology, both through the use of new and better materials and in its ease of program. Much of our effort here is on embedding standard programming capabilities such as testability, modularity, iterability, and version control into the design and fabrication process.

Layered Fabric Printing

A Layered Fabric 3D Printer for Soft Interactive Objects. Huaishu Peng, Jennifer Mankoff, Scott E. Hudson, James McCann. CHI ’15 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015. In work done collaboratively with Disney Research and led by Disney Intern Huaishu Peng (of Cornell), we have begun to explore alternative material options for …

Helping Hands

Prosthetic limbs and assistive technology (AT) require customization and modification over time to effectively meet the needs of end users. Yet, this process is typically costly and, as a result, abandonment rates are very high. Rapid prototyping technologies such as 3D printing have begun to alleviate this issue by making it possible to inexpensively, and …

3D printed attachments

Encore: 3D printed attachments What happens when you want to 3D print something that must interact with the real world? The Encore project makes it possible to 3D print objects that must attach to things in the real world. Encore provides an interface that, given an imported object and a chosen attachment method, visualizes metrics relating the goodness …

Threadsteading

  In work done collaboratively with Disney Research Pittsburgh and led by Gillian Smith of Northeastern we explored a multi-player game that can be embedded into a quilting and/or embroidery machine interface. Gameplay is constrained by the fact that only a single thread of fabric can be drawn over time. Players compete to ‘scout’ over a …

Sketchup Interface with plugin for RapID being used to create a Tic Tac Toe board.

RapID — interactive RFID

RapID – A framework for fabricating low-latency interactive objects with RFID tags RFID tags can be used to add inexpensive, wireless, batteryless sensing to objects. However, quickly and accurately estimating the state of an RFID tag is difficult. In this work, we show how to achieve low-latency manipulation and movement sensing with off-the-shelf RFID tags …

A teddy bear wearing a knit hat, scarf (with pocket) and sweater

A Knitting Machine Compiler

  Although industrial knitting machines can automatically produce a wide range of garments, they are programmed through onerous means such as pixel level image manipulation. This limits the potential for automation of knitted object design, re-use of object components, and narrows the audience able to design for these machines. Our contribution is a visual design …

Data Science

accessibility|fabrication|inclusion |data science|all
An ability to process, understand, and share data is critical to many of today’s computational tasks. Our group’s work takes a human centered view on data analytic problems. Some of our advances include a machine learning algorithm that tries to minimize the burden on people answering questions needed to support prediction and decreasing the impact of uncertainty by carrying it forward into interactive systems rather than guessing the correct interpretation without any knowledge of the user who is interacting with a system.

Exiting the cleanroom: On ecological validity and ubiquitous computing

Carter, Scott, Jennifer Mankoff, Scott R. Klemmer, and Tara Matthews. “Exiting the cleanroom: On ecological validity and ubiquitous computing.” Human–Computer Interaction 23, no. 1 (2008): 47-99. Over the past decade and a half, corporations and academies have invested considerable time and money in the realization of ubiquitous computing. Yet design approaches that yield ecologically valid understandings of …

Images from a variety of projects supporting green behavior

StepGreen

The goal of the Stepgreen project is to leverage Internet scale technologies to create opportunities for reduced energy consumption. The original vision of the project was to leverage existing online social networks to encourage individual change. Since then the project has broadened to include a number of other ideas. We have explored the impact of …

Infant Oxygen Monitoring

Improving healthcare decision making with better workflow and information flow.

Diagram of FOCUS method for dynamic, cost-sensitive feature selection

Dynamic question ordering

In recent years, surveys have been shifting online, offering the possibility for adaptive questions, where later questions depend on responses to earlier questions. We present a general framework for dynamically ordering questions, based on previous responses, to engage respondents, improving survey completion and imputation of unknown items. Our work considers two scenarios for data collection …

Modeling Human Routines

Modeling and Understanding Human Routine Behavior Human routines are blueprints of behavior, which allow people to accomplish their purposeful repetitive tasks and activities. People express their routines through actions that they perform in the particular situations that triggered those actions. An ability to model routines and understand the situations in which they are likely to …

Watch-ya-doin

Watch-ya-doin is an innovative experienced based sampling framework for longitudinal data collection and analysis. Our system consists of a smartwatch and an android device working unobtrusively to track data. Our goal is to train on and recognize a specific activity over time. We use a simple wrist-worn accelerometer to predict eating behavior and other activities. These are inexpensive …

All recent publications

News and Updates

Welcoming new group members and celebrating the new year

 A wonderful photo of the make4all group members who were able to celebrate the start of the new year and CHI together at my home. Thanks to everyone for coming, Megan for organizing and cooking, and all of the dishes and sweets that everyone shared. Happy birthday to Minxuan as well!

Fostering Science Education and Positive Technology Use at Home

I have been asked in a couple of different forums about how I have gone about parenting my children with respect ot technology. What came out of that thinking lacks any formal reference to literature, but it does represent my experience. I’ve always felt that despite many potential negatives (and despite being mostly a homesteading …

Teaching

Current Grad Courses | Current Undergrad Courses | Middle School Curricula| Past Courses

Current Grad Courses

Assistive Technology (Current Grad)

Posts relating to Assistive Technology Teaching

Project 2: Build a Better Button

Learning Goals for the Project Learn about Circuit design Learn how to communicate between an Arduino and your phone Apply your knowledge of physical design (3D printing/laser cutting) to support your goals Build a simple circuit that is enhanced by its connection to your phone Basic Requirements for Project Your project should demonstrate your ability …

The Future of Access Technologies

Access technology (AT) has the potential to increase autonomy, and improve millions of people’s ability to live independently. This potential is currently under-realized because the expertise needed to create the right AT is in short supply and the custom nature of AT makes it difficult to deliver inexpensively. Yet computers’ flexibility and exponentially increasing power …

Fabrication and Society (Grad)

Rapid Fabrication / Prototyping

Required Readings (videos for these and others found below) Mueller, S., Im, S., Gurevich, S., Teibrich, A., Pfisterer, L., Guimbretière, F., & Baudisch, P. (2014, October). WirePrint: 3D printed previews for fast prototyping. In Proceedings of the 27th annual ACM symposium on User interface software and technology (pp. 273-280). ACM. Interactive design space exploration and …

Metamaterials

Pick one to read (or read both!) Ion, A., Frohnhofen, J., Wall, L., Kovacs, R., Alistar, M., Lindsay, J., … & Baudisch, P. (2016, October). Metamaterial mechanisms. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 529-539). ACM. Ion, A., Wall, L., Kovacs, R., & Baudisch, P. (2017, May). Digital …

Current Undergrad Course

Interaction Programming

Middle School Curricula

Receipt Printing Robot

This document is based on a draft curriculum to be used for the 2016 Fall Tech Club (at the Waldorf School of Pittsburgh). During this club, students will work together to create a robot that can print quotes out using a receipt printer when a button is pressed, and blink its eyes. Learning Goals; Materials needed; Setup; Curricular …

Some Past Courses

Interactive Data Science

People



Undergrads


Current Interns


Graduated PhD Students, Post Docs & Visitors

chrisi_smiling

Christian Koehler (PhD) (co-advised with Anind Dey)

Samsung

Julia Schwarz

Julia Schwarz (PhD) (co-advised with Scott Hudson)

Qeexo, Microsoft

Sunyoung Kim

Sunyoung Kim (PhD)

Harvard post doc (co-advised with Eric Paulos)

Scott Carter

Scott Carter (PhD)

FX Pal

Tara Matthews

Tara Matthews (PhD)

Google

Michael Terry (Post Doc)

 

Other Alumni

Marty McGuire (Masters)

Masters Project, CMU VLIS, Stepgreen Social Website 2007-08

Taylor Raack (Masters)

Masters Project, CMU VLIS, Stepgreen NILM support 2011-12

Miriam Walker (Masters)

2003 MS Thesis “High-Fidelity or Low-fidelity, paper or computer? Choosing attributes when testing web prototypes,” UC Berkeley, Masters Report, 2003

Lauren Chapman (Masters)

MS in Design, Spring 2011, co-advised with Suguru Ishizaki: “Design for Chronic Illness: Exploring service systems & new technologies for patients with type 2 diabetes.”

Catherine Grevet (Undergraduate)

Undergraduate Honors Thesis, 2009: “Motivating Community-Oriented Behavior through an Online Social visualization,” Wellesley College.

PhD student at Georgia Tech

Devin BLais (Undergraduate)

Undergraduate Honors Thesis, 2009: “Green Facebook Applications: A competitive Analysis,” Carnegie Mellon University.

Ana Ramirez (Masters)

2005 MS Thesis “Designing systems that direct human action.” UC Berkeley, co-advised with Mark Davis, SIMS; NSF Fellowship

Scott Lederer (Masters)

MS, Fall 2003, co-advised with Anind Dey: “Designing disclosure: Interactive personal privacy at the dawn of ubiquitous computing.”

Holly Fait (Masters)

2003 MS Thesis “Simulation of user interaction experiences to improve evaluation for accessibility.”

Wai-Ling Ho-Ching (Masters)

MS, Fall 2003, co-advised with James Landy: “Can you see what I hear? The design and evaluation of a peripheral sound display for the deaf.”

Ruth Wylie (Undergraduate)

Undergraduate Honors Thesis, 2003 “The Effects of Computers on Cognitive Assessment,” UC Berkeley.

PhD at CMU; currently faculty at ASU (next position)