A table (shown on screen). Columns are mapped to the number row of the keyboard and rows to the leftmost column of keys, and (1) By default the top left cell is selected. (2) The right hand presses the ‘2’ key, selecting the second column (3) The left hand selects the next row (4) The left hand selects the third row. In each case, the position of the cell and its content are read out aloud.
Web user interfaces today leverage many common GUI design patterns, including navigation bars and menus (hierarchical structure), tabular content presentation, and scrolling. These visual-spatial cues enhance the interaction experience of sighted users. However, the linear nature of screen translation tools currently available to blind users make it difficult to understand or navigate these structures. We introduce Spatial Region Interaction Techniques (SPRITEs) for nonvisual access: a novel method for navigating two-dimensional structures using the keyboard surface. SPRITEs 1) preserve spatial layout, 2) enable bimanual interaction, and 3) improve the end user experience. We used a series of design probes to explore different methods for keyboard surface interaction. Our evaluation of SPRITEs shows that three times as many participants were able to complete spatial tasks with SPRITEs than with their preferred current technology.
Graph showing task completion rates for different kinds of tasks in our user studyA user is searching a table (shown on screen) for the word ‘Jill’. Columns are mapped to the number row of the keyboard and rows to the leftmost column of keys. (1) By default the top left cell is selected. (2) The right hand presses the ‘2’ key, selecting the second column (3) The left hand selects the next row (4) The left hand selects the third row. In each case, the number of occurrences of the search query in the respective column or row are read aloud. When the query is found, the position and content of the cell are read out aloud.
Audio-only interfaces, facilitated through text-to-speech screen reading software, have been the primary mode of computer interaction for blind and low-vision computer users for more than four decades. During this time, the advances that have made visual interfaces faster and easier to use, from direct manipulation to skeuomorphic design, have not been paralleled in nonvisual computing environments. The screen reader–dependent community is left with no alternatives to engage with our rapidly advancing technological infrastructure. In this article, we describe our efforts to understand the problems that exist with audio-only interfaces. Based on observing screen reader use for 4 months at a computer training school for blind and low-vision adults, we identify three problem areas within audio-only interfaces: ephemerality, linear interaction, and unidirectional communication. We then evaluated a multimodal approach to computer interaction called the Tangible Desktop that addresses these problems by moving semantic information from the auditory to the tactile channel. Our evaluation demonstrated that among novice screen reader users, Tangible Desktop improved task completion times by an average of 6 minutes when compared to traditional audio-only computer systems.
Average task completion time comparison between the participant system and experimental system grouped by technology
Task completion time comparison between the participant system and experimental system for screen reader users. Study system is faster in all cases. Completion times are total elapsed time, so we have not included error bars.
The physical icons used in the Tangible Desktop. Each icon is a small cube that has an RFID tag embedded inside and a tactilely distinct rubber crown.
A picture of the Tangible Desktop in its standard arrangement. The Tangible Taskbar sits to the left of the laptop while a user engages with the thumb of the Tangible Scrollbar.
Examples of the different hand positions used by study participants.
Kim, J., Guo, A., Yeh, T., Hudson, S. E., & Mankoff, J. (2017, June). Understanding Uncertainty in Measurement and Accommodating its Impact in 3D Modeling and Printing. In Proceedings of the 2017 Conference on Designing Interactive Systems (pp. 1067-1078). ACM.
3D printing enables everyday users to augment objects around them with personalized adaptations. There has been a proliferation of 3D models available on sharing platforms supporting this. If a model is parametric, a novice modeler can obtain a custom model simply by entering a few parameters (e.g., in the Customizer tool on Thingiverse.com). In theory, such custom models could fit any real world object one intends to augment. But in practice, a printed model seldom fits on the first try; multiple iterations are often necessary, wasting a considerable amount of time and material. We argue that parameterization or scaling alone is not sufficient for customizability, because users must correctly measure an object to specify parameters.
In a study of attempts to measure length, angle, and diameter, we demonstrate measurement errors as a significant (yet often overlooked) factor that adversely impacts the adaptation of 3D models to existing objects, requiring increased iteration. Images taken from our study are shown below.
Edge of phone is curved, and length difficult to measure
Ruler is bent, introducing error
Bulb is not correctly lined up with ruler
We argue for a new design principle—accommodating measurement uncertainty—that designers as well as novices should begin to consider. We offer two strategies—modular joint and, buffer insertion—to help designers to build models that are robust to measurement uncertainty. Examples shown below.
Stretching the Bounds of 3D Printing with Embedded Textiles
Textiles are an old and well developed technology that have many desirable characteristics. They can be easily folded, twisted, deformed, or cut; some can be stretched; many are soft. Textiles can maintain their shape when placed under tension and can even be engineered with variable stretching ability.
When combined, textiles and 3D printing open up new opportunities for rapidly creating rigid objects with embedded flexibility as well as soft materials imbued with additional functionality. We introduce a suite of techniques for integrating the two and demonstrate how the malleability, stretchability and aesthetic qualities of textiles can enhance rigid printed objects, and how textiles can be augmented with functional properties enabled by 3D printing.
Click images below to see more detail:
Citation
Rivera, M.L., Moukperian, M., Ashbrook, D., Mankoff, J., Hudson, S.E. 2017. Stretching the Bounds of 3D Printing with Embedded Textiles. To appear in to the annual ACM conference on Human Factors in Computing Systems. CHI ‘17. [Paper]
In work done collaboratively with Disney Research and led by Disney Intern Huaishu Peng (of Cornell), we have begun to explore alternative material options for fabrication. Unlike traditional 3D printing, which uses hard plastic, this project made use of cloth (in the video shown above, felt). In addition to its aesthetic properties, fabric is deformable, and the degree of deformability can be controlled. Our printer, which works by gluing layers of laser-cut fabric to each other also allows for dual material printing, meaning that layers of conductive fabric can be inserted. This allows fabric objects to also easily support embedded electronics. This work has been in the news recently, and was featured at Adafruit; Futurity; Gizmodo; Geek.com and TechCrunch, among others.
My work is focused on giving people with disabilities the voice, tools and agency to advocate for themselves. I take a multifaceted approach that includes machine learning, 3D printing, and tool building. At a high level, my goal is to tackle the technical challenges necessary for individuals and communities of people with disabilities to solve real-world problems (see all the Make4all projects). I aim to translate my research to policy and open source settings, and am proud that my work has had some policy impact.
I love to work with undergraduate and masters students and have mentored more than I can count. My mentorship always tries to include career advice as well as project advice, whether students are going on to research or not. Many undergraduate students I advised have gone on to careers in research, however, including some current faculty (Julie Kientz, Gary Hsieh, Ruth Wylie). There are at least 50 other students who are alumni of my group who are not currently listed on this page but who all made important contributions to my work over the years. Some current mentees:
Your Portfolio Archive currently has no entries. You can start creating them on your dashboard.
Additional alumni can be found on the People page.
Teaching
I love to teach, and have put significant time into curriculum development over the years.
CLASSES DEVELOPED FOR AND TAUGHT AT CMU
I am currently developing a new course on data centric computing, called The Data Pipeline. The course is accessible to novice programmers and includes a series of tutorials that can support independent online learning.
I helped to redesign the HCI Masters course User Centered Research and Evaluation, specifically bringing a real world focus to our skills teaching around contextual inquiry
I developed an online course specifically for folks who want to know enough program to be able to prototype simple interfaces (targeted at our incoming masters students). The course is available free online at CMU’s Open Learning Initiative under “Media Programming”
I developed and taught the Environment and Society course over the last five years. This was a project oriented course that took a very multifaceted look at the role of technology in solving environmental problems.
I helped to develop a reading course that is required for our PhD students to ensure that they have depth in technical HCI: CS Mini
Assistive Technology: I developed and taught one of the first Assistive Technology courses in the country (specifically from an HCI perspective), and I used a service learning model to do so. Original class
I have helped to revamp Process and Theory over the years, a skills course intended for our first year PhD students.
Bio
My Bachelor’s of Arts was done at Oberlin College, where I was a member of two great societies — FOO and ACM. I received my Ph.D. as a member of the Future Computing Environments research group in the College of Computing at Georgia Tech , Gregory Abowd and Scott Hudson were my advisors. I then spent three formative years at UC Berkeley as an Assistant Professor working with the I/O group and 12 years at CMU before joining the faculty of the University of Washington. This is my “Academic genealogy” on the Abowd side. I am also disabled, with an invisible chronic illness, and I am happy to talk about my experience of navigating both medical and social barriers in academia and provide mentorship. Please reach out if I can help.
CREATE Bio: Dr. J. Mankoff directs the Center for Research and Education on Accessible Technology and Experiences (CREATE) at the University of Washington. CREATE’s mission is to make technology accessible, and make the world accessible through technology. Mankoff’s own research uses technologies including generative AI, data science, 3D printing, and computational knitting to solve accessibility problems. They strive to bring both structural and personal perspectives to their work. Dr. Mankoff received a PhD at Georgia Tech, advised by Gregory Abowd and Scott Hudson, and a B.A. from Oberlin College. They have identified as disabled since graduate school.
Individual Bio:
Dr. J. Mankoff is the Richard E. Ladner Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, where they direct the Center for Research and Education on Accessible Technology and Experiences (CREATE). Their research is focused on accessibility through giving disabled people the voice, tools and agency to advocate for themselves. They strive to bring both structural and personal perspectives to their work. Dr. Mankoff’s work tackles the technical challenges necessary for individuals and communities to solve real-world problems in accessibility, including in domains such as higher education, health, and DIY solutions. They use technologies including generative AI, data science, 3D printing, and computational knitting in their work. Dr. Mankoff is a SIGCHI Academy member, a SIGCHI Social Impact Awardee and has received the ASSETS IMPACT award, and AccessComputing Capacity Building award. They have published over 180 papers, received their PhD at Georgia Tech, advised by Gregory Abowd and Scott Hudson, and received a B.A. from Oberlin College. Dr. Mankoff has identified as disabled since graduate school.
Dr. Mankoff’s previous faculty positions include UC Berkeley’s EECS department and Carnegie Mellon’s HCI Institute. Dr. Mankoff is a CHI Academy member and has been recognized with a SIGCHI Social Impact Award, an Alfred P. Sloan Fellowship and IBM Faculty Fellowship, and an ASSETS 10 year impact award.
Other Thoughts and Links
Advice about searching through literature, doing reviews, etc.
Please email me if you need information or help regarding RSI (or are experiencing any computer-related pain).
I have chronic lyme disease. Lyme disease is the most common vector born disease in America today. I write about my experiences on A Lyme Disease Journal
Best Conference Experience Ever: The CHI Straggles Seder
This slideshow requires JavaScript.
Contact Information
Jennifer Mankoff
jmankoff [at] acm.org
206-685-3035 Paul G. Allen School of Computer Science & Engineering University of Washington Paul G. Allen Center 185 Stevens Way
Campus Box 352350
Seattle, WA 98195
