BLV Understanding of Visual Semantics


Venkatesh Potluri
Tadashi E. GrindelandJon E. Froehlich, Jennifer Mankoff: Examining Visual Semantic Understanding in Blind and Low-Vision Technology Users. CHI 2021: 35:1-35:14

Visual semantics provide spatial information like size, shape, and position, which are necessary to understand and efficiently use interfaces and documents. Yet little is known about whether blind and low-vision (BLV) technology users want to interact with visual affordances, and, if so, for which task scenarios. In this work, through semi-structured and task-based interviews, we explore preferences, interest levels, and use of visual semantics among BLV technology users across two device platforms (smartphones and laptops), and information seeking and interactions common in apps and web browsing. Findings show that participants could benefit from access to visual semantics for collaboration, navigation, and design. To learn this information, our participants used trial and error, sighted assistance, and features in existing screen reading technology like touch exploration. Finally, we found that missing information and inconsistent screen reader representations of user interfaces hinder learning. We discuss potential applications and future work to equip BLV users with necessary information to engage with visual semantics.

Understanding Disabled Knitters


Taylor Gotfrid
Kelly MackKathryn J. LumEvelyn YangJessica K. HodginsScott E. Hudson, Jennifer Mankoff: Stitching Together the Experiences of Disabled Knitters. CHI 2021: 488:1-488:14

Knitting is a popular craft that can be used to create customized fabric objects such as household items, clothing and toys. Additionally, many knitters find knitting to be a relaxing and calming exercise. Little is known about how disabled knitters use and benefit from knitting, and what accessibility solutions and challenges they create and encounter. We conducted interviews with 16 experienced, disabled knitters and analyzed 20 threads from six forums that discussed accessible knitting to identify how and why disabled knitters knit, and what accessibility concerns remain. We additionally conducted an iterative design case study developing knitting tools for a knitter who found existing solutions insufficient. Our innovations improved the range of stitches she could produce. We conclude by arguing for the importance of improving tools for both pattern generation and modification as well as adaptations or modifications to existing tools such as looms to make it easier to track progress

Living Disability Theory

A picture of a carved wooden cane in greens and blues

It was my honor this year to participate in an auto-ethnographic effort to explore accessibility research from a combination of personal and theoretical perspectives. In the process, and thanks to my amazing co-authors, I learned so much about myself, disability studies, ableism and accessibility.

Best Paper Award Hoffman, M., Kasnitz, D., Mankoff, J. and Bennett, C. l. (2020) Living Disability Theory: Reflections on Access, Research, and Design. In Proceedings of ASSETS 2020, 4:1-4:13

Abstract: Accessibility research and disability studies are intertwined fields focused on, respectively, building a world more inclusive of people with disability and understanding and elevating the lived experiences of disabled people. Accessibility research tends to focus on creating technology related to impairment, while disability studies focuses on understanding disability and advocating against ableist systems. Our paper presents a reflexive analysis of the experiences of three accessibility researchers and one disability studies scholar. We focus on moments when our disability was misunderstood and causes such as expecting clearly defined impairments. We derive three themes: ableism in research, oversimplification of disability, and human relationships around disability. From these themes, we suggest paths toward more strongly integrating disability studies perspectives and disabled people into accessibility research.

Make your Webpage More Accessible

This is an individual project to make your webpage more accessible. Learning goals include

  • Some of the basic rules for web accessibility
  • How to use an accessibility checker to assess whether a web page is accessible
  • How to fix accessibility problems
  • How to work within the constraints of end-user content editing tools and still make something accessible

This project has two phases.

Phase one: Assess problems

In phase one, you will assess problems with the web page you choose.

Picking a webpage

You can assess your own website, if you have one. If not, a next best option would be your personal social media site (such as your linked-in page). If you don’t have one, but use social media such as facebook and twitter you can assess your posts on one of those sites. Finally, if none of those are options, just pick any site you think makes sense.

Running an accessibility checker

Once you have selected a web page, you should run it through an accessibility checker. The WebAim accessibility checker, WAVE, is a great choice for many sites. However, if the site requires that you log in, you may need an alternative. A great choice is the Chrome plugin Axe.

What to bring to class from Phase one

You should not change anything about the website you selected before class. You should bring your accessibility checking results to class and have read them over. We will work together in class on addressing the problems you found.

Phase two: Fixing problems

In phase two, you will fix problems on the website you chose. We will talk about how to write alt text, set up proper header structures, simplify language, and what else is possible within the constraints of the technology you are using.

Final Project

The goal of your final project is to explore an accessibility issue in more depth than you’ve been able to do in our projects so far. In choosing this project, you may want to draw from personal expertise, literature, or user data should you have access to it.

Your final project will have three phases:

Proposal

  • Proposal: Your proposal be a slide deck with 5 slides that describe your
    • promise: How the world will be better based on your project
    • obstacle: Why we don’t have this already.
    • solution: How you will achieve the promise. This will most likely be primarily technical, such as a novel device.
    • related work: It should also include a related work section with at least 5 references showing some evidence for the importance of this problem.
    • timeline: Finally, it should include a timeline showing that this is feasible.
  • Development: We will check in on projects in part of class and/or office hours on a weekly basis to help provide guidance about progress on the milestones laid out in your timeline

Midterm Writeup

Midway through the project you will turn in a brief update to your project. This should included an up-to-date written version of your promise, obstacle and solution (1-3 paragraphs) and a related work section, also updated based on feedback (3-4 paragraphs). The total should be less than a page long.

Final Project Writeup

The final 2-page report should be in the 2-column CHI template format: https://chi2020.acm.org/authors/chi-proceedings-format/

Requirements for this are below. In addition you should follow the writing guidelines put out by SIGACCESS for writing about disability.

In addition, you will participate in a poster session.

Poster

Your poster should cover the same basic items as your report, but in much less depth. It should have a section highlighting the key goals of the project, images of what you did and/or pictures that convey study results if you did one, and some explanation of how you accomplished things, as well as mentioning how a disability studies perspective informed your project.

It does not need a related work section, and you will want to put your names on it and a big title.

Written Document

The report should cover these main sections:

  • Introduction — 1-3 paragraphs: Present the promise/ obstacle/ solution for your project — what is the problem are you solving and why is it important to solve it? This can re-use text from your midterm report.
  • Related Work  — 3-4 paragraphs: Talk about relevant work that closely connects with your project. This can re-use text from your midterm report.
  • Methodology — about 1 page: What did you do in your project – If you worked with participants: how many people, what did they do. If you implemented a system, or designed something, what did you design?
  • Disability Studies Perspective – 1 Paragraph: How did a disability studies perspective inform your project
  • Conclusions — 1-2 paragraphs: describe what you learnt and how can this be extended/built on in the future
  • Personal reflection — 1-2 paragraphs, individual and handed in separately: describe what you personally learned from this project, and what your individual contributions were to the team.

Important notes and considerations

  • Language: You will be expected to use best practices in language and presentation. Here is the SIGACCESS guide on this.
  • The things we have emphasized in this class, namely a disability studies perspective and physical building, should be featured in your project as much as possible.
    • With respect to disability studies, you should think critically about whether and how your project empowers and gives agency to people with disabilities, as well as the extent to which it expects/engages the larger structural issues around the problem you’re trying to solve
    • With respect to physical computing, this is not required, but you should get approval from the instructor if you go in a different direction, and have a rationale
  • If you don’t have personal experience justifying the choice of problem, it is important to find studies that involved people with disabilities that help justify the sense of your proposed work. It is not feasible to do a full iterative design cycle in this project (and not necessarily an ethical use of the time of people with disabilities), but equally important not to come in with a ‘hero complex’ and simply believe you know what people need.
  • Your project can include designing and piloting a study, but only if you have significant experience already in this domain since we haven’t really taught that aspect of accessibility in this class. Better to spend time on skills you learned here! In addition, given the number of weeks available, be careful not to overcommit (e.g. creating a significant novel device and a lengthy study!)

Use 3D printing to make something Accessible (Due 10/16)

The goal of this assignment is for you to develop basic familiarity with OpenSCAD. Your goal is to create a model of something that makes something more accessible for you or someone else. To keep this problem within reason for a first assignment, you should focus on things that are fairly simple to model. You should work in pairs on this assignment.

Examples would be a tactile label for something (such as a luggage tag), a guide (to make moving something along a path easier) or a lever (to make rotating something easier.

  • Your solution should be correctly sized (i.e. try to measure the thing you are modifying and to make sure that your printed object is appropriately sized).
  • You should use a simple method to attach things such as a zip-tie (simply requires small holes), or glue.
  • Your object should be small (be printable in 20 minutes to 2 hours)

You should create a Thingiverse “thing” which represents your object with a picture of your final object in use, your OpenSCAD file, and a picture of your model, along with a brief explanation of what problem it solves, how to correctly size it, how it attaches to or interacts with the real world. If you remixed something else on Thingiverse be sure to correctly attribute it (by creating a remix).

You should submit the link to your Thingiverse “thing” on Canvas.

You should also print it out to demo in class. Here is a page with information about using the Ultimaker printers. This slide deck about 3D printing also has lots of in.

The grading rubric for this assignment is as follows. When points are 1 or 0, this is pass fail (no nuance). When points are 0-3, use the following scale: 0 – Not done; 1 – Short shallow solution; 2 – Good solution; 3 – Outstanding solution.

Points Description Comments (by grader)
0-3 Create a 3D model that solves a problem
1 or 0Learn how to correctly size a model
1 or 0Apply an appropriate attachment method
1 or 0Learn the pipeline: Create a 3D printed object from your model
0-3Describe how a model should be used

Digital Fabrication in Medical Practice

Maker culture in health care is on the rise with the rapid adoption of consumer-grade fabrication technologies. However, little is known about the activity and resources involved in prototyping medical devices to improve patient care. In this paper, we characterize medical making based on a qualitative study of medical stakeholder engagement in physical prototyping (making) experiences. We examine perspectives from diverse stakeholders including clinicians, engineers, administrators, and medical researchers. Through 18 semi-structured interviews with medical-makers in US and Canada, we analyze making activity in medical settings. We find that medical-makers share strategies to address risks, define labor roles, and acquire resources by adapting traditional structures or creating new infrastructures. Our findings outline how medical-makers mitigate risks for patient safety, collaborate with local and global stakeholder networks, and overcome constraints of co-location and material practices. We recommend a clinician-aided software system, partially-open repositories, and a collaborative skill-share social network to extend their strategies in support of medical making.

“Point-of-Care Manufacturing”: Maker Perspectives onDigital Fabrication in Medical Practice. Udaya Lakshmi, Megan Hofmann, Stephanie Valencia, Lauren Wilcox, Jennifer Mankoff and Rosa Arriaga. CSCW 2019. To Appear.

A venn diagram showing the domains of expertise of those we interviewed including people from hospitals, universities, non-profits, va networks, private practices, and government. We interviewed clinicians and facilitators in each of these domains and there was a great deal of overlap with participants falling into multiple categories. For example, one participant was in a VA network and in private practice, while another was at a university and also a non-profit.

Project 2: Build a Better Button

Learning Goals for the Project

  • Learn about Circuit design
  • Learn how to communicate between an Arduino and your phone
  • Build a simple circuit that is enhanced by its connection to your phone

Basic Requirements for Project

Your project should demonstrate your ability to either:

  • Take input from at least one button (or other sensor), and connect it to some interesting service
  • Your focus should be on circuit design and Arduino programming. You don’t need to create a custom phone app. You can if you want create a custom case or button using 3d printing.

You should make a case for why this is an assistive technology of some sort. For example, you could build a door opening sensor (using a button or proximity sensor) that causes your phone to announce the door was opened, or a single switch control for scrolling or tabbing through a web page, or a capacitive sensor that captures a log of how often a cane is used.

There is some great software that con be connected to the Arduino including 1Shield, AppInventor, Blynk and IFTTT. Some work only for Android, others for both Android and iPhone.

There are lots of really great examples online of arduino based projects, arduino projects that involve smartphones, and arduino projects that involve 3D printing or laser cutting. Many of them are too complex for the expectations of this project, though they might help to inspire final projects, or give you ideas for something simple you can do in a week. Here is a sample:

Hand In

Create a Thingiverse or Instructables page for your project with a brief description of the project, a video, any 3D printed files, and a schematic for your circuit. Turn the URL in by email with the subject: Project 2. Be prepared to demo your project in class.

PointsDescription
1 or 0 Project uses physical computing to solve an accessibility problem
1 or 0Project communicates with your phone in some way
1 or 0Project includes a working circuit that you designed
1 or 0Project includes at least one button
1 or 0 Project includes some kind of response to the button
1 or 0Thingiverse or Instructables page describes project in a reproducable fashion.

“Occupational Therapy is Making”

Assistive Technology

Instructor: Jennifer Mankoffjmankoff@cs.cmu.edu
Spring 2005

HCII, 3601 NSH, (W)+1 (412) 268-1295
Office hours: By Appointment & 1-2pm Thurs

Course Description

This class will focus on computer accessibility, including web and desktop computing, and research in the area of assistive technology.

The major learning goals from this course include:

  • Develop an understanding of the relationship between disability policy, the disability rights movement, and your role as a technologist. For example, we will discuss we will discuss the pros and cons and infrastructure involved in supporting mainstream computer applications rather than creating new ones from scratch.
  • Develop a skill set for basic design and evaluation of accessible web pages and desktop applications.
  • Develop familiarity with technologies and research relating to accessibility including a study of optimal font size and color for people with dyslexia, word-prediction aids, a blind-accessible drawing program,
  • Develop familiarity with assistive technologies that use computation to increase the accessibility of the world in general. Examples include memory aids, sign-language recognition, and so on.

Requirements

Students will be expected to do service work with non-profits serving the local disabled community during one to two weekends of the start of the semester. This course has a project component, where students will design, implement, and test software for people with disabilities. Additionally, students will read and report on research papers pertinent to the domain.

Grading will be based on service work (10%); the project (60%); and class participation, including your reading summary and the lecture you lead (30%).

Other relevant documents

Course CalendarAssignmentsBibliography

Prerequisites

Prerequisites for this class are: Familiarity with basic Human Computer Interaction material or consent of the instructor (for undergraduate students)

It is recommended that you contact the instructor if you are interested in taking this class.