Research Projects

Accessibility

accessibility|fabrication|inclusion |data science|all

overview
To revolutionize the creation of assistive technology, our work is divided into three complementary activities

One in 5 Americans has a disability, and the needs of people with disabilities are extremely heterogeneous. Yet accessibility is not a given, and assistive technology today is mostly homogenous and very expensive. Our group is focused on the scientific challenges that can address accessibility across a broad range of domains.

Infant Oxygen Monitoring

Improving healthcare decision making with better workflow and information flow.

Shows someone placing a pen in a cap with two different types of adaptations.

Printable Adaptations

Reprise: A Design Tool for Specifying, Generating, and Customizing 3D Printable Adaptations on Everyday Objects Reprise is a tool for creating custom adaptive 3D printable designs for making it easier to manipulate everything from tools to zipper pulls. Reprise’s library is based on a survey of about 3,000 assistive technology and life hacks drawn from …

Tactile Interfaces to Appliances

Facade is a crowdsourced fabrication pipeline to automatically generate tactile interfaces to appliances for blind people.

Volunteer AT Fabricators

Perry-Hill, J., Shi, P., Mankoff, J. & Ashbrook, D. Understanding Volunteer AT Fabricators: Opportunities and Challenges in DIY-AT for Others in e-NABLE. Accepted to CHI 2017 We present the results of a study of e-NABLE, a distributed, collaborative volunteer effort to design and fabricate upper-limb assistive technology devices for limb-different users. Informed by interviews with …

Making the field of computing more inclusive for people with disabilities

Lazar, J., Churchill, E. F., Grossman, T., Van der Veer, G., Palanque, P., Morris, J. S., & Mankoff, J. (2017). Making the field of computing more inclusive. Communications of the ACM, 60(3), 50-59. More accessible conferences, digital resources, and ACM SIGs will lead to greater participation by more people with disabilities. Improving conference and online material accessibility …

The Tangible Desktop

Mark S. Baldwin, Gillian R. Hayes, Oliver L. Haimson, Jennifer Mankoff, Scott E. Hudson: The Tangible Desktop: A Multimodal Approach to Nonvisual Computing. TACCESS 10(3): 9:1-9:28 (2017) Audio-only interfaces, facilitated through text-to-speech screen reading software, have been the primary mode of computer interaction for blind and low-vision computer users for more than four decades. During this time, the advances that have made visual …

Inclusion

accessibility|fabrication|inclusion |data science|all
Our group also works to address inclusion of other under represented groups. Some of our recent work has looked at gender in academia and other settings.

Understanding gender equity in author order assignment

Academic success and promotion are heavily influenced by publication record. In many fields, including computer science, multi-author papers are the norm. Evidence from other fields shows that norms for ordering author names can influence the assignment of credit. We interviewed 38 students and faculty in human- computer interaction (HCI) and machine learning (ML) at two …

Picture of potted plants and a bench with the word Africatown in the background, painted in bright red and green colors

Who Gets to Future? Race, Representation, and Design Methods in Africatown

Who Gets to Future? Race, Representation, and Design Methods in Africatown Jasper Tran O’Leary, Sara Zewde, Jennifer Mankoff , Daniela K. Rosner CHI 2019 This paper draws on a collaborative project called the Africatown Activation to examine the role design practices play in contributing to (or conspiring against) the flourishing of the Black community in …

Fabrication

accessibility|fabrication|inclusion |data science|all
President Obama once said ‘3D printing […] has the potential to revolutionize the way we make almost everything” [Obama 2013].’ One might say that computers have the same ability. Together, computational advances and fabrication can revolutionize personal problem solving.

Our group is embarking on an effort to enhance fabrication technology, both through the use of new and better materials and in its ease of program. Much of our effort here is on embedding standard programming capabilities such as testability, modularity, iterability, and version control into the design and fabrication process.

Layered Fabric Printing

A Layered Fabric 3D Printer for Soft Interactive Objects. Huaishu Peng, Jennifer Mankoff, Scott E. Hudson, James McCann. CHI ’15 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015. In work done collaboratively with Disney Research and led by Disney Intern Huaishu Peng (of Cornell), we have begun to explore alternative material options for …

Helping Hands

Prosthetic limbs and assistive technology (AT) require customization and modification over time to effectively meet the needs of end users. Yet, this process is typically costly and, as a result, abandonment rates are very high. Rapid prototyping technologies such as 3D printing have begun to alleviate this issue by making it possible to inexpensively, and …

3D printed attachments

Encore: 3D printed attachments What happens when you want to 3D print something that must interact with the real world? The Encore project makes it possible to 3D print objects that must attach to things in the real world. Encore provides an interface that, given an imported object and a chosen attachment method, visualizes metrics relating the goodness …

Threadsteading

  In work done collaboratively with Disney Research Pittsburgh and led by Gillian Smith of Northeastern we explored a multi-player game that can be embedded into a quilting and/or embroidery machine interface. Gameplay is constrained by the fact that only a single thread of fabric can be drawn over time. Players compete to ‘scout’ over a …

Sketchup Interface with plugin for RapID being used to create a Tic Tac Toe board.

RapID — interactive RFID

RapID – A framework for fabricating low-latency interactive objects with RFID tags RFID tags can be used to add inexpensive, wireless, batteryless sensing to objects. However, quickly and accurately estimating the state of an RFID tag is difficult. In this work, we show how to achieve low-latency manipulation and movement sensing with off-the-shelf RFID tags …

A teddy bear wearing a knit hat, scarf (with pocket) and sweater

A Knitting Machine Compiler

  Although industrial knitting machines can automatically produce a wide range of garments, they are programmed through onerous means such as pixel level image manipulation. This limits the potential for automation of knitted object design, re-use of object components, and narrows the audience able to design for these machines. Our contribution is a visual design …

Data Science

accessibility|fabrication|inclusion |data science|all
An ability to process, understand, and share data is critical to many of today’s computational tasks. Our group’s work takes a human centered view on data analytic problems. Some of our advances include a machine learning algorithm that tries to minimize the burden on people answering questions needed to support prediction and decreasing the impact of uncertainty by carrying it forward into interactive systems rather than guessing the correct interpretation without any knowledge of the user who is interacting with a system.

Exiting the cleanroom: On ecological validity and ubiquitous computing

Carter, Scott, Jennifer Mankoff, Scott R. Klemmer, and Tara Matthews. “Exiting the cleanroom: On ecological validity and ubiquitous computing.” Human–Computer Interaction 23, no. 1 (2008): 47-99. Over the past decade and a half, corporations and academies have invested considerable time and money in the realization of ubiquitous computing. Yet design approaches that yield ecologically valid understandings of …

Images from a variety of projects supporting green behavior

StepGreen

The goal of the Stepgreen project is to leverage Internet scale technologies to create opportunities for reduced energy consumption. The original vision of the project was to leverage existing online social networks to encourage individual change. Since then the project has broadened to include a number of other ideas. We have explored the impact of …

Infant Oxygen Monitoring

Improving healthcare decision making with better workflow and information flow.

Diagram of FOCUS method for dynamic, cost-sensitive feature selection

Dynamic question ordering

In recent years, surveys have been shifting online, offering the possibility for adaptive questions, where later questions depend on responses to earlier questions. We present a general framework for dynamically ordering questions, based on previous responses, to engage respondents, improving survey completion and imputation of unknown items. Our work considers two scenarios for data collection …

Modeling Human Routines

Modeling and Understanding Human Routine Behavior Human routines are blueprints of behavior, which allow people to accomplish their purposeful repetitive tasks and activities. People express their routines through actions that they perform in the particular situations that triggered those actions. An ability to model routines and understand the situations in which they are likely to …

Watch-ya-doin

Watch-ya-doin is an innovative experienced based sampling framework for longitudinal data collection and analysis. Our system consists of a smartwatch and an android device working unobtrusively to track data. Our goal is to train on and recognize a specific activity over time. We use a simple wrist-worn accelerometer to predict eating behavior and other activities. These are inexpensive …

All recent publications