Archives

The Tangible Desktop

Leave a comment

Mark S. Baldwin, Gillian R. Hayes, Oliver L. Haimson, Jennifer Mankoff, Scott E. Hudson: The Tangible Desktop: A Multimodal Approach to Nonvisual Computing. TACCESS 10(3): 9:1-9:28 (2017)

Audio-only interfaces, facilitated through text-to-speech screen reading software, have been the primary mode of computer interaction for blind and low-vision computer users for more than four decades. During this time, the advances that have made visual interfaces faster and easier to use, from direct manipulation to skeuomorphic design, have not been paralleled in nonvisual computing environments. The screen reader–dependent community is left with no alternatives to engage with our rapidly advancing technological infrastructure. In this article, we describe our efforts to understand the problems that exist with audio-only interfaces. Based on observing screen reader use for 4 months at a computer training school for blind and low-vision adults, we identify three problem areas within audio-only interfaces: ephemerality, linear interaction, and unidirectional communication. We then evaluated a multimodal approach to computer interaction called the Tangible Desktop that addresses these problems by moving semantic information from the auditory to the tactile channel. Our evaluation demonstrated that among novice screen reader users, Tangible Desktop improved task completion times by an average of 6 minutes when compared to traditional audio-only computer systems.

Also see: Mark S. Baldwin, Jennifer Mankoff, Bonnie A. Nardi, Gillian R. Hayes: An Activity Centered Approach to Nonvisual Computer Interaction. ACM Trans. Comput. Hum. Interact. 27(2): 12:1-12:27 (2020)

Average task completion time comparison between the participant system and experimental system grouped by technology
Task completion time comparison between the participant system and experimental system for screen reader users. Study system is faster in all cases. Completion times are total elapsed time, so we have not included error bars.
The physical icons used in the Tangible Desktop. Each icon is a small cube that has an RFID tag embedded inside and a tactilely distinct rubber crown.
A picture of the Tangible Desktop in its standard arrangement. The Tangible Taskbar sits to the left of the laptop while a user engages with the thumb of the Tangible Scrollbar.
Examples of the different hand positions used by study participants.

Uncertainty in Measurement

Leave a comment

Kim, J., Guo, A., Yeh, T., Hudson, S. E., & Mankoff, J. (2017, June). Understanding Uncertainty in Measurement and Accommodating its Impact in 3D Modeling and Printing. In Proceedings of the 2017 Conference on Designing Interactive Systems (pp. 1067-1078). ACM.

3D printing enables everyday users to augment objects around them with personalized adaptations. There has been a proliferation of 3D models available on sharing platforms supporting this. If a model is parametric, a novice modeler can obtain a custom model simply by entering a few parameters (e.g., in the Customizer tool on Thingiverse.com). In theory, such custom models could fit any real world object one intends to augment. But in practice, a printed model seldom fits on the first try; multiple iterations are often necessary, wasting a considerable amount of time and material. We argue that parameterization or scaling alone is not sufficient for customizability, because users must correctly measure an object to specify parameters.

In a study of attempts to measure length, angle, and diameter, we demonstrate measurement errors as a significant (yet often overlooked) factor that adversely impacts the adaptation of 3D models to existing objects, requiring increased iteration. Images taken from our study are shown below.

Ruler is bent, introducing error
Bulb is not correctly lined up with ruler
Edge of phone is curved, and length difficult to measure

We argue for a new design principle—accommodating measurement uncertainty—that designers as well as novices should begin to consider. We offer two strategies—modular joint and, buffer insertion—to help designers to build models that are robust to measurement uncertainty. Examples shown below.

 

 

3D Printing with Embedded Textiles

Leave a comment

screen-shot-2017-01-09-at-9-03-14-pm

Stretching the Bounds of 3D Printing with Embedded Textiles

Textiles are an old and well developed technology that have many desirable characteristics. They can be easily folded, twisted, deformed, or cut; some can be stretched; many are soft. Textiles can maintain their shape when placed under tension and can even be engineered with variable stretching ability.

When combined, textiles and 3D printing open up new opportunities for rapidly creating rigid objects with embedded flexibility as well as soft materials imbued with additional functionality. We introduce a suite of techniques for integrating the two and demonstrate how the malleability, stretchability and aesthetic qualities of textiles can enhance rigid printed objects, and how textiles can be augmented with functional properties enabled by 3D printing.

Click images below to see more detail:

Citation

Rivera, M.L., Moukperian, M., Ashbrook, D., Mankoff, J., Hudson, S.E. 2017. Stretching the Bounds of 3D Printing with Embedded Textiles. To appear in to the annual ACM conference on Human Factors in Computing Systems. CHI ‘17. [Paper]

Layered Fabric Printing

Leave a comment

A Layered Fabric 3D Printer for Soft Interactive Objects. Huaishu Peng, Jennifer Mankoff, Scott E. Hudson, James McCann. CHI ’15 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015.

In work done collaboratively with Disney Research and led by Disney Intern Huaishu Peng (of Cornell), we have begun to explore alternative material options for fabrication. Unlike traditional 3D printing, which uses hard plastic, this project made use of cloth (in the video shown above, felt). In addition to its aesthetic properties, fabric is deformable, and the degree of deformability can be controlled. Our printer, which works by gluing layers of laser-cut fabric to each other also allows for dual material printing, meaning that layers of conductive fabric can be inserted. This allows fabric objects to also easily support embedded electronics. This work has been in the news recently, and was featured at Adafruit; Futurity; Gizmodo; Geek.com and TechCrunch, among others.

Jennifer Mankoff

Leave a comment

Research | Students | Teaching | Bio | CV | Advice | Fun | Contact

Research

My work is focused on giving people with disabilities the voice, tools and agency to advocate for themselves. I take a multifaceted approach that includes machine learning, 3D printing, and tool building. At a high level, my goal is to tackle the technical challenges necessary for everyday individuals and communities to solve real-world problems (see all the Make4all projects).

Some of my writing about my disability experience and my Lyme blog

Some of my most recent projects (All):

14 drawings of participant gestures organized as 2 rows and 7 columns. The top row represents high agreement gestures, and the bottom row represents unique gestures that participants designed. The columns are functions (from left to right): move/pan, select, rotate, delete, open/close, zoom-in/-out, duplicate. For move/pan, the high agreement gesture was moving hand from left to right in mid-air and the unique gesture was flexing fingers from left to right. For select, the high agreement gesture was pointing with hand or finger, and the unique gesture was squeezing hand into fist. For rotate, the high agreement gesture was grabbing object and rotating about the person's wrist and the unique gesture was rotating shoulders. For delete, the high agreement gesture was swiping away and the unique gesture was looking down. For open/close, the high agreement gesture was swiping down, and the unique gesture was shrugging shoulders. For zoom-in/-out, the high agreement gesture was moving both hands towards and away from body, and the unique gesture was flexing shoulders in and out. For duplicate, the high agreement gesture was tapping twice in mid-air in two separate locations with the hand and the unique gesture was tapping chin twice.

How Do People with Limited Movement Personalize Upper-Body Gestures? Considerations for the Design of Personalized and Accessible Gesture Interfaces

Artwork by Lydia X.Z. Brown,(they/them) 2020. Watercolor and ink drawing of four disabled people. The first is wearing a bipap respirator and a shirt that says disabled and proud! and has brown skin and dark brown hair in a bob. The second is wearing a green shirt that says Access is Love and has short green hair and green lips. They are in a wheelchair and have brown skin. Next is a person holding a white cain wearing a blue shirt saying "my other disability is a bad attitude" with shoulder-length brown hair and lighter skin. Finally, there is a person with dreadlocks in an orange shirt. Words like "disability" and "genius" are visible behind them. The bottom right of the image states "By Lydia X Brown, 2020. Honor crip, mad, disabled & neurodivergent wisdom & genius. Disability Justice Now! Downloaded from community.amplifier.org

Race, Disability and Accessibility Technology

A picture of part of a dashboard visualization showing sales by state, using a US map, bar charts, a pie chart, and total sales. To the side, part of a textual dashboard description is visible summarizing the same information visible in the charts. Above, a filter menu is visible where the user can select which region to show sales for.

Azimuth: Designing Accessible Dashboards for Screen Reader Users

A picture of two mock heads showing how color and position can be used to visualize phonemes around the head. A green circle at bottom left shows the wh phoneme in the left image. A yellow blob at the top right shows the f phoneme.

Understanding and Enhancing the Role of Speechreading in Online d/DHH Communication Accessibility

GLOBEM logo, blue text with a world for the O.

Cross-Dataset Generalization for Human Behavior Modeling

PSST: Enabling Blind or Visually Impaired Developers to Author Sonifications of Streaming Sensor Data

A picture of two people wearing 3D printed PPE. On the left is a face shield (42.2% of submissions) and on the right a facemask (28.4% of submissions).

Rapid Convergence: The Outcomes of Making PPE during a Healthcare Crisis

A complex diagram showing the flow of information from sensors through feature extraction, formation of a user behavior profile, and then final prediction.

Personalized behavior modeling: depression detection

A diagram showing how we build stress & resilience frameworks for student mental health. The framework has three components -- social characteristics, cumulative stress, and resilience resources. At the center where all three overlap is perceived stress, anxiety and depression.

Distress and resilience among marginalized undergraduates

Students

Current PhD students:

Aashaka Desai

Han Zhang

Kelly Mack

Venkatesh Potluri

Former PhD Students:

3D printed prosthetic bow holder with bow and velcro for attaching to the arm.

Megan Hofmann

Orson (Xuhai) Xu (PhD, co-advised with Anind Dey)

Head shot of Amy Hurst

Amy Hurst (co-advised with Scott Hudson)

Kirstin Early (co-advised with Steven E. Feinberg)

Nikola Banovic (PhD, co-advised with Anind Dey)

Tawanna Dillahunt

Also those with no make4all page: Mark Baldwin (co-advised with Gillian Hayes); Christian Koehler; Sunyoung Kim; Scott Carter; Tara Matthews; Julia Schwarz 

I love to work with undergraduate and masters students and have mentored more than I can count. My mentorship always tries to include career advice as well as project advice, whether students are going on to research or not. Many undergraduate students I advised have gone on to careers in research, however, including some current faculty (Julie Kientz, Gary Hsieh, Ruth Wylie). There are at least 50 other students who are alumni of my group who are not currently listed on this page but who all made important contributions to my work over the years. Some current mentees:

Your Portfolio Archive currently has no entries. You can start creating them on your dashboard.

Recent Alumni I mentored/advised:

Taylor Gotfrid

Yasaman Sefigar

Xin Liu

Aleks Tapinsh

Gary Yang

Nidhi Vyas

Additional alumni can be found on the People page.

Teaching

I love to teach, and have put significant time into curriculum development over the years.

CLASSES DEVELOPED FOR AND TAUGHT AT CMU
  • I am currently developing a new course on data centric computing, called The Data Pipeline. The course is accessible to novice programmers and includes a series of tutorials that can support independent online learning.
  • I helped to redesign the HCI Masters course User Centered Research and Evaluation, specifically bringing a real world focus to our skills teaching around contextual inquiry
  • I developed an online course specifically for folks who want to know enough program to be able to prototype simple interfaces (targeted at our incoming masters students). The course is available free online at CMU’s Open Learning Initiative under “Media Programming”
  • I developed and taught the Environment and Society course over the last five years. This was a project oriented course that took a very multifaceted look at the role of technology in solving environmental problems.
  • I helped to develop a reading course that is required for our PhD students to ensure that they have depth in technical HCI: CS Mini
  • Assistive Technology: I developed and taught one of the first Assistive Technology courses in the country (specifically from an HCI perspective), and I used a service learning model to do so. Original class
  • I have helped to revamp Process and Theory over the years, a skills course intended for our first year PhD students.

Bio

My Bachelor’s of Arts was done at Oberlin College, where I was a member of two great societies — FOO and ACM. I received my Ph.D. as a member of the Future Computing Environments research group in the College of Computing at Georgia Tech , Gregory Abowd and Scott Hudson were my advisors. I then spent three formative years at UC Berkeley as an Assistant Professor working with the I/O group and 12 years at CMU before joining the faculty of the University of Washington. This is my “Academic genealogy” on the Abowd side. I am also disabled, with an invisible chronic illness, and I am happy to talk about my experience of navigating both medical and social barriers in academia and provide mentorship. Please reach out if I can help. 

Bio:  Jennifer Mankoff is the Richard E. Ladner Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington. Her research is focused on accessibility through giving people the voice, tools and agency to advocate for themselves.  She strives to bring both structural and personal perspectives to her work. For example, her recent work in fabrication of accessible technologies considers not only innovative tools that can enable individual makers but also the larger clinical and sociological challenges to disseminating and sharing designs. Similarly,  her work in the intersection of mental health and discrimination uses sensed data to explore how external risks and pressures interact with people’s responses to challenging moments. Jennifer received her PhD at Georgia Tech, advised by Gregory Abowd and Scott Hudson, and her B.A. from Oberlin College.

Her previous faculty positions include UC Berkeley’s EECS department and Carnegie Mellon’s HCI Institute. Jennifer is a CHI Academy member and has been recognized with an Alfred P. Sloan Fellowship, IBM Faculty Fellowship and Best Paper awards from ASSETS, CHI and Mobile HCI. Some supporters of her research include Autodesk, Google Inc., the Intel Corporation, IBM, Hewlett-Packard, Microsoft Corporation and the National Science Foundation.

Other Thoughts and Links

  • Advice about searching through literature, doing reviews, etc.
  • Please email me if you need information or help regarding RSI (or are experiencing any computer-related pain).
  • I have chronic lyme disease. Lyme disease is the most common vector born disease in America today. I write about my experiences on A Lyme Disease Journal
 Best conference experience ever: The CHI2009 Straggler’s Seder

This slideshow requires JavaScript.

Things I love (below)

kids

This slideshow requires JavaScript.
Picture of my children Artwork I’ve done
lupa-small My Husband, Anind

This slideshow requires JavaScript.
My Viola My Husband My dogs: Demi, Nugget, Gryffin

Contact Information

Jennifer Mankoff
jmankoff [at] acm.org
​206-685-3035
Paul G. Allen School of Computer Science & Engineering
University of Washington
Paul G. Allen Center
185 Stevens Way
Campus Box 352350
Seattle, WA 98195