Archives

Disabled Innovators & Innovations

Leave a comment

Disability culture, disability innovation, and DIY efforts have long been an important part of the work that disabled leaders and resesearch values, studies, and contributes to. Over the last year, we’ve been exploring this through a couple of perspectives:

First, we’ve been developing A11yhood.org, a website that automates the collection of open source accessibility solutions across a wide variety of media from 3D printing to knitting to software, and supports search and exploration. We searched for words like “accessibility” and found over 1440 repositories/things. This project was recently launched at the Open Source and Accessibility Summit and is supported by the GitHub Tides Foundation, NIDILRR’s RERC program (90REGE0026) and in collaboration with other open-source and accessibility focused organizations and leaders including GitHub’s Ed Summers, CAOS and GOAT.

Next, we have been talking to disabled innovators. We presented a paper at ASSETS 2025 that highlights cultural processes of finding community and building solidarity, valuing disabled agency and knowledge, and rejecting ableist norms. To see how these cultural aspects might inform accessibility technology design, we studied accessibility technologies made by disabled people for disabled people – interviewing disabled innovators who had created and disseminated accessibility technologies. We asked these innovators to share their stories and reflect on goals and values they imbued in their innovations. We analyzed how cultural themes of belonging, knowledge, and creativity influenced their work. Our work highlights the potential of a cultural lens in aligning accessibility technology with disabled people’s values as well as unearthing new directions for inquiry for the field: Exploring Disability Culture Through Accounts of Disabled Innovators of Accessibility Technology, by Aashaka Desai, Jennifer Mankoff, Richard E. Ladner
(ASSETS ’25)

Embroidering Tactile Graphics

Leave a comment

Beyond Beautiful: Embroidering Legible and Expressive Tactile Graphics:
Margaret Ellen Seehorn, Claris Winston, Bo Liu, Gene S-H Kim, Emily White, Nupur Gorkar, Kate S Glazko, Aashaka Desai, Jerry Cao, Megan Hofmann, Jennifer Mankoff. ASSETS 2025

Tactile graphics present visual information to blind and visually-impaired individuals in an accessible way, through touch. Current methods for producing tactile graphics, such as embossing or swell-paper printing, have limitations such as durability – and the tools required to produce them are limited in expressiveness. In this project, we explore embroidery as a medium for producing tactile graphics. Embroidery, traditionally known for its variety and visual beauty, offers not just improved durability and ease of production – but the ability to convey information through a broad range of stitch types. Following an exploration of the design space of embroidered tactile graphics, we identify key perceptual properties that impact how embroidered textures are differentiated. Based on these differences, we introduce an optimization algorithm for assigning textures to regions of tactile graphics in a way that makes them diverse and legible. We implement an end-to-end pipeline for producing embroidered tactile graphics and evaluate the comprehensibility and legibility of our design with 6 blind participants. Our findings showed that embroidered tactile graphics present information accurately and comprehensively, and that measurable properties, such as the use of spacing and distinctiveness, were an important factor of expressive and legible design.

Photograph of two embroidered graphics. On the left is a map, with filled areas for sidewalks and buildings, with different textures indicating which is which. Braille is visible along the top. On the right is a diagram of layers of Saturn, shaped like a pie slice with different textures for the central are, middle, and outer area of the slice, each labeled.

MatPlotAlt

Leave a comment

MatplotAlt is an open-source Python package for easily adding alternative text to matplotlib figures. MatplotAlt equips Jupyter notebook authors to automatically generate and surface chart descriptions with a single line of code or command, and supports a range of options that allow users to customize the generation and display of captions based on their preferences and accessibility needs.

Our evaluation indicates that MatplotAlt’s heuristic and LLM-based methods to generate alt text can create accurate long-form descriptions of both simple univariate and complex Matplotlib figures. We find that state-of-the-art LLMs still struggle with factual errors when describing charts, and improve the accuracy of our descriptions by prompting GPT4-turbo with heuristic-based alt text or data tables parsed from the Matplotlib figure.

Here is some example ALT text generated for the pie chart shown below. A variety of examples can be found in the MatPlotAlt documentation.

A pie chart titled ’percentage of annual sunshine’. There are 12 slices: jan (3.19%), feb (4.993%), mar (8.229%), apr (9.57%), may (11.7%), june (12.39%), july (14.42%), aug (12.99%), sep (10.22%), oct (6.565%), nov (3.329%), and dec (2.404%). The data has a standard deviation of x=4.006, an average of x=8.333, a maximum value of x=14.42, and a minimum value of x=2.404. The data strictly increase up to their max at x=14.42, then strictly decrease.

A pie chart titled ’percentage of annual sunshine’. There are 12 slices: jan (3.19%), feb (4.993%), mar (8.229%), apr (9.57%), may (11.7%), june (12.39%), july (14.42%), aug (12.99%), sep (10.22%), oct (6.565%), nov (3.329%), and dec (2.404%). The data has a standard deviation of x=4.006, an average of x=8.333, a maximum value of x=14.42, and a minimum value of x=2.404. The data strictly increase up to their max at x=14.42, then strictly decrease.

Kai Nylund, Jennifer Mankoff, Venkatesh Potluri: MatplotAlt: A Python Library for Adding Alt Text to Matplotlib Figures in Computational Notebooks. Comput. Graph. Forum 44(3) (2025)

“A Tool for Freedom”

Leave a comment

Jerry Cao, Krish Jain, Julie Zhang, Yuecheng Peng, Shwetak Patel, and Jennifer Mankof. 2025. “A Tool for Freedom”: Co-Designing Mobility Aid Improvements Using Personal Fabrication and Physical Interface Modules with Primarily Young Adults. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26–May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/3706598.3713366

Mobility aids (e.g., canes, crutches, and wheelchairs) are crucial for people with mobility disabilities; however, pervasive dissatisfaction with these aids keeps usage rates low. Through semi-structured interviews with 17 mobility aid users, mostly under the age of 30, we identified specific sources of dissatisfaction among younger users of mobility aids, uncovered community-based solutions for these dissatisfactions, and explored ways these younger users wanted to improve mobility aids. We found that users sought customizable, reconfigurable, multifunctional, and more aesthetically pleasing mobility aids. Participants’ feedback guided our prototyping of tools/accessories, such as laser cut decorative sleeves, hot-swappable physical interface modules, and modular canes with custom 3D-printed handles. These prototypes were then the focus of additional co-design sessions where six returning participants offered suggestions for improvements and provided feedback on their usefulness and usability. Our findings highlight that many mobility aid users have the desire, ability, and need to customize and improve their aids in different ways compared to older adults. We propose various solutions and design guidelines to facilitate the modifications of mobility aids.

Toward Language Justice

Leave a comment

Aashaka Desai, Rahaf Alharbi, Stacy Hsueh, Richard E. Ladner, and Jennifer Mankoff. 2025. Toward Language Justice: Exploring Multilingual Captioning for Accessibility. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26–May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3706598.3713622

A growing body of research investigates how to make captioning experiences more accessible and enjoyable to disabled people. However, prior work has focused largely on English captioning, neglecting the majority of people who are multilingual (i.e., understand or express themselves in more than one language). To address this gap, we conducted semi-structured interviews and diary logs with 13 participants who used multilingual captions for accessibility. Our findings highlight the linguistic and cultural dimensions of captioning, detailing how language features (scripts and orthography) and the inclusion/negation of cultural context shape the accessibility of captions. Despite lack of quality and availability, participants emphasized the importance of multilingual captioning to learn a new language, build community, and preserve cultural heritage. Moving toward a future where all ways of communicating are celebrated, we present ways to orient captioning research to a language justice agenda that decenters English and engages with varied levels of fluency.

Shaping Lace

Leave a comment

Glazko, K., Portnova-Fahreeva, A., Mankoff-Dey, A., Psarra, A., & Mankoff, J. (2024, July). Shaping Lace: Machine embroidered metamaterials. In Proceedings of the 9th ACM Symposium on Computational Fabrication (pp. 1-12).

The ability to easily create embroidered lace textile objects that can be manipulated in structured ways, i.e., metamaterials, could enable a variety of applications from interactive tactile graphics to physical therapy devices. However, while machine embroidery has been used to create sensors and digitally enhanced fabrics, its use for creating metamaterials is an understudied area. This article reviews recent advances in metamaterial textiles and conducts a design space exploration of metamaterial freestanding lace embroidery. We demonstrate that freestanding lace embroidery can be used to create out-of-plane kirigami and auxetic effects. We provide examples of applications of these effects to create a variety of prototypes and demonstrations.

Identifying and improving disability bias in GPT-based resume screening

Leave a comment

Glazko, K., Mohammed, Y., Kosa, B., Potluri, V., & Mankoff, J. (2024, June). Identifying and improving disability bias in GPT-based resume screening. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (pp. 687-700).

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability-related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.

Notably Inaccessible

Leave a comment

Venkatesh Potluri, Sudheesh Singanamalla, Nussara Tieanklin, Jennifer Mankoff: Notably Inaccessible – Data Driven Understanding of Data Science Notebook (In)Accessibility. ASSETS 2023: 13:1-13:19

Computational notebooks are tools that help people explore, analyze data, and create stories about that data. They are the most popular choice for data scientists. People use software like Jupyter, Datalore, and Google Colab to work with these notebooks in universities and companies.

There is a lot of research on how data scientists use these notebooks and how to help them work together better. But there is not much information about the problems faced by blind and visually impaired (BVI) users. BVI users have difficulty using these notebooks because:

  • The interfaces are not accessible.
  • The way data is shown is not user-friendly for them.
  • Popular libraries do not provide outputs they can use.

We analyzed 100,000 Jupyter notebooks to find accessibility problems. We looked for issues that affect how these notebooks are created and read. From our study, we give advice on how to make notebooks more accessible. We suggest ways for people to write better notebooks and changes to make the notebook software work better for everyone.

Domain Specific Metaheuristic Optimization

Leave a comment

For non-technical domain experts and designers it can be a substantial challenge to create designs that meet domain specific goals. This presents an opportunity to create specialized tools that produce optimized designs in the domain. However, implementing domain specific optimization methods requires a rare combination of programming and domain expertise. Creating flexible design tools with re-configurable optimizers that can tackle a variety of problems in a domain requires even more domain and programming expertise. We present OPTIMISM, a toolkit which enables programmers and domain experts to collaboratively implement an optimization component of design tools. OPTIMISM supports the implementation of metaheuristic optimization methods by factoring them into easy to implement and reuse components: objectives that measure desirable qualities in the domain, modifiers which make useful changes to designs, design and modifier selectors which determine how the optimizer steps through the search space, and stopping criteria that determine when to return results. Implementing optimizers with OPTIMISM shifts the burden of domain expertise from programmers to domain experts.

Megan Hofmann, Nayha Auradkar, Jessica Birchfield, Jerry Cao, Autumn G. Hughes, Gene S.-H. Kim, Shriya Kurpad, Kathryn J. Lum, Kelly Mack, Anisha Nilakantan, Margaret Ellen Seehorn, Emily Warnock, Jennifer Mankoff, Scott E. Hudson: OPTIMISM: Enabling Collaborative Implementation of Domain Specific Metaheuristic Optimization. CHI 2023: 709:1-709:19

https://youtube.com/watch?v=wjQrFeLbOiw%3Fsi%3DkMTxEkEBjoUrQDJ3

COVID-19 Risk Negotation

Leave a comment

During the COVID-19 pandemic, risk negotiation became an important precursor to in-person contact. For young adults, social planning generally occurs through computer-mediated communication. Given the importance of social connectedness for mental health and academic engagement, we sought to understand how young adults plan in-person meetups over computer-mediated communication in the context of the pandemic. We present a qualitative study that explores young adults’ risk negotiation during the COVID-19 pandemic, a period of conflicting public health guidance. Inspired by cultural probe studies, we invited participants to express their preferred precautions for one week as they planned in-person meetups. We interviewed and surveyed participants about their experiences. Through qualitative analysis, we identify strategies for risk negotiation, social complexities that impede risk negotiation, and emotional consequences of risk negotiation. Our findings have implications for AI-mediated support for risk negotiation and assertive communication more generally. We explore tensions between risks and potential benefits of such systems.

Margaret E. Morris, Jennifer Brown, Paula S. Nurius, Savanna Yee, Jennifer Mankoff, Sunny Consolvo:
“I Just Wanted to Triple Check… They were all Vaccinated”: Supporting Risk Negotiation in the Context of COVID-19.ACM Trans. Comput. Hum. Interact. 30(4): 60:1-60:31 (2023)