Archives

“A Tool for Freedom”

Leave a comment

Jerry Cao, Krish Jain, Julie Zhang, Yuecheng Peng, Shwetak Patel, and Jennifer Mankof. 2025. “A Tool for Freedom”: Co-Designing Mobility Aid Improvements Using Personal Fabrication and Physical Interface Modules with Primarily Young Adults. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26–May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/3706598.3713366

Mobility aids (e.g., canes, crutches, and wheelchairs) are crucial for people with mobility disabilities; however, pervasive dissatisfaction with these aids keeps usage rates low. Through semi-structured interviews with 17 mobility aid users, mostly under the age of 30, we identified specific sources of dissatisfaction among younger users of mobility aids, uncovered community-based solutions for these dissatisfactions, and explored ways these younger users wanted to improve mobility aids. We found that users sought customizable, reconfigurable, multifunctional, and more aesthetically pleasing mobility aids. Participants’ feedback guided our prototyping of tools/accessories, such as laser cut decorative sleeves, hot-swappable physical interface modules, and modular canes with custom 3D-printed handles. These prototypes were then the focus of additional co-design sessions where six returning participants offered suggestions for improvements and provided feedback on their usefulness and usability. Our findings highlight that many mobility aid users have the desire, ability, and need to customize and improve their aids in different ways compared to older adults. We propose various solutions and design guidelines to facilitate the modifications of mobility aids.

Autoethnographic Insights from Neurodivergent GAI “Power Users”

Leave a comment

Kate Glazko, JunHyeok Cha, Aaleyah Lewis, Ben Kosa, Brianna Wimer, Andrew Zheng, Yiwei Zheng, and Jennifer Mankoff. 2025. Autoethnographic Insights from Neurodivergent GAI “Power Users”. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan. ACM, New York, NY, USA, 20 pages. https://doi.org/10.1145/ 3706598.3713670

Generative AI has become ubiquitous in both daily and professional life, with emerging research demonstrating its potential as a tool for accessibility. Neurodivergent people, often left out by existing accessibility technologies, develop their own ways of navigating normative expectations. GAI offers new opportunities for access, but it is important to understand how neurodivergent “power users”—successful early adopters—engage with it and the challenges they face. Further, we must understand how marginalization and intersectional identities influence their interactions with GAI. Our autoethnography, enhanced by privacy-preserving GAI-based diaries and interviews, reveals the intricacies of using GAI to navigate normative environments and expectations. Our findings demonstrate how GAI can both support and complicate tasks like code-switching, emotional regulation, and accessing information. We show that GAI can help neurodivergent users to reclaim their agency in systems that diminish their autonomy and self-determination. However, challenges such as balancing authentic self-expression with societal conformity, alongside other risks, create barriers to realizing GAI’s full potential for accessibility.

Toward Language Justice

Leave a comment

Aashaka Desai, Rahaf Alharbi, Stacy Hsueh, Richard E. Ladner, and Jennifer Mankoff. 2025. Toward Language Justice: Exploring Multilingual Captioning for Accessibility. In CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26–May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3706598.3713622

A growing body of research investigates how to make captioning experiences more accessible and enjoyable to disabled people. However, prior work has focused largely on English captioning, neglecting the majority of people who are multilingual (i.e., understand or express themselves in more than one language). To address this gap, we conducted semi-structured interviews and diary logs with 13 participants who used multilingual captions for accessibility. Our findings highlight the linguistic and cultural dimensions of captioning, detailing how language features (scripts and orthography) and the inclusion/negation of cultural context shape the accessibility of captions. Despite lack of quality and availability, participants emphasized the importance of multilingual captioning to learn a new language, build community, and preserve cultural heritage. Moving toward a future where all ways of communicating are celebrated, we present ways to orient captioning research to a language justice agenda that decenters English and engages with varied levels of fluency.

Shaping Lace

Leave a comment

Glazko, K., Portnova-Fahreeva, A., Mankoff-Dey, A., Psarra, A., & Mankoff, J. (2024, July). Shaping Lace: Machine embroidered metamaterials. In Proceedings of the 9th ACM Symposium on Computational Fabrication (pp. 1-12).

The ability to easily create embroidered lace textile objects that can be manipulated in structured ways, i.e., metamaterials, could enable a variety of applications from interactive tactile graphics to physical therapy devices. However, while machine embroidery has been used to create sensors and digitally enhanced fabrics, its use for creating metamaterials is an understudied area. This article reviews recent advances in metamaterial textiles and conducts a design space exploration of metamaterial freestanding lace embroidery. We demonstrate that freestanding lace embroidery can be used to create out-of-plane kirigami and auxetic effects. We provide examples of applications of these effects to create a variety of prototypes and demonstrations.

Identifying and improving disability bias in GPT-based resume screening

Leave a comment

Glazko, K., Mohammed, Y., Kosa, B., Potluri, V., & Mankoff, J. (2024, June). Identifying and improving disability bias in GPT-based resume screening. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (pp. 687-700).

As Generative AI rises in adoption, its use has expanded to include domains such as hiring and recruiting. However, without examining the potential of bias, this may negatively impact marginalized populations, including people with disabilities. To address this important concern, we present a resume audit study, in which we ask ChatGPT (specifically, GPT-4) to rank a resume against the same resume enhanced with an additional leadership award, scholarship, panel presentation, and membership that are disability-related. We find that GPT-4 exhibits prejudice towards these enhanced CVs. Further, we show that this prejudice can be quantifiably reduced by training a custom GPTs on principles of DEI and disability justice. Our study also includes a unique qualitative analysis of the types of direct and indirect ableism GPT-4 uses to justify its biased decisions and suggest directions for additional bias mitigation work. Additionally, since these justifications are presumably drawn from training data containing real-world biased statements made by humans, our analysis suggests additional avenues for understanding and addressing human bias.

Notably Inaccessible

Leave a comment

Venkatesh Potluri, Sudheesh Singanamalla, Nussara Tieanklin, Jennifer Mankoff: Notably Inaccessible – Data Driven Understanding of Data Science Notebook (In)Accessibility. ASSETS 2023: 13:1-13:19

Computational notebooks are tools that help people explore, analyze data, and create stories about that data. They are the most popular choice for data scientists. People use software like Jupyter, Datalore, and Google Colab to work with these notebooks in universities and companies.

There is a lot of research on how data scientists use these notebooks and how to help them work together better. But there is not much information about the problems faced by blind and visually impaired (BVI) users. BVI users have difficulty using these notebooks because:

  • The interfaces are not accessible.
  • The way data is shown is not user-friendly for them.
  • Popular libraries do not provide outputs they can use.

We analyzed 100,000 Jupyter notebooks to find accessibility problems. We looked for issues that affect how these notebooks are created and read. From our study, we give advice on how to make notebooks more accessible. We suggest ways for people to write better notebooks and changes to make the notebook software work better for everyone.

Domain Specific Metaheuristic Optimization

Leave a comment

For non-technical domain experts and designers it can be a substantial challenge to create designs that meet domain specific goals. This presents an opportunity to create specialized tools that produce optimized designs in the domain. However, implementing domain specific optimization methods requires a rare combination of programming and domain expertise. Creating flexible design tools with re-configurable optimizers that can tackle a variety of problems in a domain requires even more domain and programming expertise. We present OPTIMISM, a toolkit which enables programmers and domain experts to collaboratively implement an optimization component of design tools. OPTIMISM supports the implementation of metaheuristic optimization methods by factoring them into easy to implement and reuse components: objectives that measure desirable qualities in the domain, modifiers which make useful changes to designs, design and modifier selectors which determine how the optimizer steps through the search space, and stopping criteria that determine when to return results. Implementing optimizers with OPTIMISM shifts the burden of domain expertise from programmers to domain experts.

Megan Hofmann, Nayha Auradkar, Jessica Birchfield, Jerry Cao, Autumn G. Hughes, Gene S.-H. Kim, Shriya Kurpad, Kathryn J. Lum, Kelly Mack, Anisha Nilakantan, Margaret Ellen Seehorn, Emily Warnock, Jennifer Mankoff, Scott E. Hudson: OPTIMISM: Enabling Collaborative Implementation of Domain Specific Metaheuristic Optimization. CHI 2023: 709:1-709:19

https://youtube.com/watch?v=wjQrFeLbOiw%3Fsi%3DkMTxEkEBjoUrQDJ3

COVID-19 Risk Negotation

Leave a comment

During the COVID-19 pandemic, risk negotiation became an important precursor to in-person contact. For young adults, social planning generally occurs through computer-mediated communication. Given the importance of social connectedness for mental health and academic engagement, we sought to understand how young adults plan in-person meetups over computer-mediated communication in the context of the pandemic. We present a qualitative study that explores young adults’ risk negotiation during the COVID-19 pandemic, a period of conflicting public health guidance. Inspired by cultural probe studies, we invited participants to express their preferred precautions for one week as they planned in-person meetups. We interviewed and surveyed participants about their experiences. Through qualitative analysis, we identify strategies for risk negotiation, social complexities that impede risk negotiation, and emotional consequences of risk negotiation. Our findings have implications for AI-mediated support for risk negotiation and assertive communication more generally. We explore tensions between risks and potential benefits of such systems.

Margaret E. Morris, Jennifer Brown, Paula S. Nurius, Savanna Yee, Jennifer Mankoff, Sunny Consolvo:
“I Just Wanted to Triple Check… They were all Vaccinated”: Supporting Risk Negotiation in the Context of COVID-19.ACM Trans. Comput. Hum. Interact. 30(4): 60:1-60:31 (2023)

Generative Artificial Intelligence’s Utility for Accessibility

Leave a comment

With the recent rapid rise in Generative Artificial Intelligence (GAI) tools, it is imperative that we understand their impact on people with disabilities, both positive and negative. However, although we know that AI in general poses both risks and opportunities for people with disabilities, little is known specifically about GAI in particular.

To address this, we conducted a three-month autoethnography of our use of GAI to meet personal and professional needs as a team of researchers with and without disabilities. Our findings demonstrate a wide variety of potential accessibility-related uses for GAI while also highlighting concerns around verifiability, training data, ableism, and false promises.

Glazko, K. S., Yamagami, M., Desai, A., Mack, K. A., Potluri, V., Xu, X., & Mankoff, J. An Autoethnographic Case Study of Generative Artificial Intelligence’s Utility for Accessibility. ASSETS 2023. https://dl.acm.org/doi/abs/10.1145/3597638.3614548

News: Can AI help boost accessibility? These researchers tested it for themselves

Presentation (starts at about 20mins)

https://youtube.com/watch?v=S40-jPBH820%3Fsi%3DCm17oTaMaDnoQGvK%3F%23t%3D20m26s

How Do People with Limited Movement Personalize Upper-Body Gestures?

Leave a comment

Personalized upper-body gestures that can enable input from diverse body parts (e.g., head, neck, shoulders, arms, hands, and fingers), and match the abilities of each user, might make gesture systems more accessible for people with upper-body motor disabilities. Static gesture sets that make ability assumptions about the user (e.g., touch thumb and index finger together in midair) may not be accessible. In our work, we characterize the personalized gesture sets designed by 25 participants with upper-body motor disabilities. We found that the personalized gesture sets that participants designed were specific to their abilities and needs. Six participants mentioned that their inspiration for designing the gestures was based on “how I would do [the gesture] with the abilities that I have”. We suggest three considerations when designing accessible upper-body gesture interfaces: 

1) Track the whole upper body. Our participants used their whole upper-body to perform the gestures, and some switched back and forth from the left to the right hand to combat fatigue.

2) Use sensing mechanisms that are agnostic to the location and orientation of the body. About half of our participants kept their hand on or barely took their hand off of the armrest to decrease arm movement and fatigue.

3) Use sensors that can sense muscle activations without movement. Our participants activated their muscles but did not visibly move in 10% of the personalized gestures.   

Our work highlights the need for personalized upper-body gesture interfaces supported by multimodal biosignal sensors (e.g., accelerometers, sensors that can sense muscle activity like EMG).