Tag Archives: teaching

When Accessibility and Learning Are Given Equal Weight

Last year, I wrote about my efforts to design an accessibility course that centers disabled voices. This fall was the first offering of that course at the undergraduate level. Every decision we made about the course was made with accessibility as the most important priority, on equal footing with the goal of educating students.

The result was an absolute joy to teach, and in the process, we refined our emphasis on competency-based grading and the importance of intersectionality in disability-based work. The course also has a service learning component — we take suggestion and requests from the broader community for websites and apps that they want assessed for accessibility concerns, and the students learn about how to use common accessibility technologies and automated tools to test for accessibility violations, and how to communicate those back.

The final projects that the students developed deeply reflect the impact of these teaching decisions. They pushed the boundaries of what is typically published in the field of accessibility to include topics such as content warnings (project 1 and project 2) and food restrictions. Other projects dove deeply into high expertise concerns such as scientific alt text and matplotlib accessibility. And projects demonstrated deep consideration of intersectionality, such as Complexion Cupid, an accessible solution for foundation colormatching for people with a variety of skin colors; and PadMap, an accessible map for finding menstrual products.

The quotes below are from students who were asked to describe how the class embodies one or more of the ten principles of disability justice that we studied, during week two of the quarter.

Inclusive Language and Content

Throughout the class, we worked to emphasize that disability is something that can be present in any identity group and that we should consider the disability experience in all the same places and spaces that it exists, as well as the unique concepts that disability culture brings to those experiences.

  • “by recognizing and acknowledging the diverse experiences of the participants, [the class] creates an inclusive environment that respects and values various aspects of identity, such as race, class, sexuality, age, and more.”
  • “Often, people and curriculums unintentionally forget to do this. The way they describe situations and the systems around us sometimes makes it seem like they aren’t addressing the wholeness of people with disabilities (again unintentionally). The way sentences are phrased makes it seem like they’re ‘different.’ But this class does a really good job of making sure we’re using the right terminology and viewing the world from the perspective of everyone.

Competency-based Grading

The class grade is primarily based on four competencies each, in three categories: disability justice, accessibility compliance assessment, and creation of accessible documents and media. The course syllabus describes this in more depth, but in brief, competency-based grading separates out how you learn a skill from whether you know it. Many of these competencies are assessed repeatedly. It doesn’t matter whether students get them wrong at first, as long as they eventually learn them.

  • “The grading system is designed around sustainability and anti-capitalistic principles. [it] allows students to pace their learning and create a deeper sense of understanding and mastery. […] It removes the urgency of understanding accessibility right on the first attempt, and instead teaches us that we are all slowly moving towards a shared goal. It provides a space for collaboration over competition where students learn from each others’ success.”
  • “competency-based grading […] rewards students based on their ability to be able to learn and apply the content and skills taught in the class instead of rewarding students based on ‘normative’ levels of productivity. I feel that competency-based grading also promotes the principle of sustainability, as it prioritizes someone’s ability to learn a skill throughout the entirety of the class, and not just immediately after it is taught.”
  • “Typical grading techniques focus on a person’s ability to produce a certain quantity of work of a certain quality in a certain amount of time, and evaluate this productivity. Curve-based grading encourages competition. […] In contrast, in this class, we have multiple chances to demonstrate competency (placing less of an emphasis on productivity) and grading is not competitive.”

Discuss, summarize, and post

In-class discussions are designed to include remote and asynchronous learners. This is done through a share, pair, and post method: All discussions are summarized by a student involved and posted to the class discussion board at the end of the pair time. All students, even those who can not attend a class in person or synchronously, are expected to post as well.

  • “[Interdependence] is emphasized a lot in class because we are all expected to participate in discussions and share ideas with one another. Since everyone has different backgrounds and perspectives, group discussion makes up a big part of what we learn in this class. We are, in a way, responsible for each other’s learning. Respect is also very important, as we are able to work better when we feel comfortable sharing our ideas. Everyone has the responsibility of creating a respectful environment for group discussions. We are often able to learn better from firsthand experiences as they tell us specific details that could be missed by others. Students have a wide variety of experiences that can be beneficial for learning new perspectives. If we skip class without a good reason, we could deprive someone else of a valuable learning opportunity.”

Online access to lectures

Lectures are recorded and made available to students asynchronously. As mentioned, students still are asked to participate in discussions through posting; additionally, asynchronous remote students are asked to summarize a reading to help ensure engagement in the material.

  • “[the class] addresses collective liberation [through the] ability to participate online, providing students an opportunity to structure their work and participation in a flexible way.”
  • “Recognizing wholeness is a big part of the class, it’s also like a theme for the course, we respect all kinds of circumstances, and we choose to understand. Such as […] options to attend the class. It makes everyone feel welcomed and included.”
  • “This class has different ways to attend and participate, such as attending synchronously in person or virtually and doing make-up work asynchronously. I think having these options helps students to pace themselves and to learn collectively for the long term. As it allows students to catch up in case they can’t make it to class, this component creates flexibility as well.”

Flexible Due Dates & Regrades

We allow students up to two days of “late turn in” with no penalty on every assignment and also allow unlimited regrades for the competencies.

  • “Sustainability is present in several formats such as the given late days [2 for every single assignment] and other things which prioritize our mental health and ability to continue.”
  • “collective liberation has to do with ‘a vision that leaves no bodymind behind.’ By integrating free late days into every assignment, students who are late in submitting assignments can still proceed along with the rest of the group. No one, including those with and those without disabilities, gets left behind.”
  • “With the flexible late-submission policy, students facing diverse challenges that interfere with timely submissions are still given the opportunity to turn in their work. This ensures that all students, regardless of disability status, are not disadvantaged by rigid deadlines, which highlights the principle that you should leave no one behind.”

Everything accessible by default

The class is designed, if at all possible, to be “accessible by default” meaning that anyone, whether they have accommodations or not, will find that their needs are met. Some examples of what we did are mentioned in the quotes below.

  • “There are lots of accommodations in this class in order to make sure that everyone in the class is moving along together. By allowing for remote participation, students who are unable to attend in-person classes due to their own circumstances will still be able to participate and learn from the class. This class also provides closed captioning, which helps students with hearing impairments, students whose first language is not English and may need more help understanding the lecture, students with attention deficit disorders, and many more.”
  • “In class, our slides always have alt text for images and we also went over how to present in a more accessible way. The fact that this class even addresses these things shows that the class is trying to come from the perspective of people who are impacted negatively by the systems around us”
  • “There are many opportunities for students to share with the staff any access needs they may need to get the most out of this class. Additionally, there are accessibility tools and methods that are used in class every day, like captioning, recording, and visual descriptions when needed.”
  • “[…] by handing out masks at the beginning of the class, in order to protect those around us who are more vulnerable to falling ill, we’re ensuring collective access.”

Accessibility Course: Centering Disabled Voices

Over the last six months, I led a team of students* in (re) designing an accessibility course focused on bringing together disability studies, disability justice, and advanced technologies and settings for accessibility technologies.

The course focused on three categories of learning goals: (1) Creating Accessible Media and Documents (2) Assessing and Addressing Accessibility Compliance in Apps and Websites and (3) Disability Awareness and Justice

It also was designed to itself center disability justice through its structure, by providing the following without requiring official accommodations: hybrid engagement options; competency based learning with unlimited revision allowed; and project deadlines with built in flexibility.

Throughout the class, we also tried to center themes and readings that did not just reflect or default to majority experiences of disability; and two assignments provided opportunities for direct service to the disability community.

Finally, the course explicitly included both traditional accessibility knowledge such as GUI and App focused accessibility work; and opportunities to explore newer settings for access work both from a technological perspective (such as VR/AR and Fabrication) and from a contextual perspective (such as accessibility and sustainability/disaster response and accessibility in the context of carceral (in)justice).

The class had four homework assignments and a final project, along with a number of in-class exercises. They included a disability justice analysis of a current events topic; finding and presenting first person accounts of accessibility technology; creating a website or app based accessibility evaluation (with the option to do this for a specific nonprofit hoping to improve the accessibility of their website or app) and creating a plain language summary highlighting some of the accessibility concerns and existing available accessibility solutions of a technology such as educational games; ar/vr; or information visualization tools.

As mentioned, this is the second iteration of this course. During the first one (which also included working on contributions to an open source screen reader), we learned that we needed to scale back the implementation requirements in the first half of the course and focus on learning best practices and practical skills; while increasing the length of time and depth of the final project. We still wanted to have some service featured in the course design, and the website or app accessibility evaluations served that role extremely well. In addition, the addition of disability justice analysis, along with discussions of topics like disability dongles, helped us to provide a better structure to students’ understanding of what we were looking for in their final projects. A learning from our second offering is to provide much more structure around plain language writing, which is very difficult to do properly, especially for longer articles. While there is still some iteration to be done, students reported taking lessons from the course back into their workplace, and engaged deeply with concepts such as disability advocacy and the importance of centering disabled voices.

The final projects were as varied as the students in the course, from browser extensions to contributions to the cache structure of an open source screen reader, Odilia. Here are some other examples of what students built:

Multiple projects focused on the accessibility of documents and other creative artifacts such as markdown, code tours and visualizations. These looked at a variety of issues from color contrast to header structure, proper labeling, and screen reader access. A special category I want to highlight in this group of projects is plain language. The Plain Language Checker highlights where plain language rules may be violated if you paste text into it; while Jargon(e) annotates jargon in documents based on community created dictionaries that provide definitions.

A couple of projects focused more on physical world settings. SafeRide is an accessible app that uses Apple Watch’s nonspeech audio recognition to alert a bicyclist about environmental sounds of interest, such as another cyclist ringing a bell. A project that bridges both documentation and physical world interaction focuses on tactile schematic production for people who are blind or low vision, based on the documentation at tactileschematics.com. The project provides a tutorial and code for extending Fritzing to support tactile schematic

For better or for worse, I have made the decision in my accessibility courses to neither guarantee nor require that students work directly with individuals who are disabled in their projects. While I think it is incredibly important to ensure that project ideas are driven by real needs, I teach accessibility often enough, and to enough people, that it would not be sustainable to ask the disability community to support these student learners at the scale that would be necessary. To balance this, I have put a lot of thought into how to ensure that these projects center disabled needs and preferences. That is why we teach and require disability justice, and how to seek out first person accounts through blogs, vlogs, and in research papers in multiple assignments before the final project; and require that these are carried through into the final project.

In the end, four of these projects ended up having disabled customers; one other was based directly on careful work co-led by a disabled person and a researcher; and others were based on more remote information. In addition, as mentioned earlier, multiple students contributed to web and/or app assessments that were shared back to nonprofits associated with CREATE (one with a disability mission; one with a goal of increasing access for disabled people among their constituents).

The first two offerings of this course were taught in our professional masters program, which has mature students already in the workforce who can directly see the value of the content we are teaching. My next iteration will be for undergraduate computer science majors. I hope that students at all levels will benefit from this overall structure but I am sure it will evolve to meet the needs of each group in new ways.

*Thanks to the evolving cast of students who helped with this course including Dhruv Jain; Aashaka Desai; Ather Sharif; Venkatesh Potluri; Orson Xu and Kelly Mack.

The Data Pipeline

I’ve been teaching a course on using data for three years now, and it feels ‘finished’ enough that it is perhaps worth writing about. When I began the course, I had the goal of (1) exploring the human side of data, meaning the ways in which what we know about people impact our ability to make, process, interpret, and use data (2) make the course accessible to a broad variety of people (not just programmers) and (3) organize the course around modules that would produce nice portfolio items. The result is the course at data.cmubi.org.  While it has evolved over the years, it’s always included at least a few beginner programmers, and the projects have been interesting opportunities for students to explore issues like interactive machine learning, data visualization, and topic areas they care a great deal about.

A big emphasis on the course is on data cleaning –understanding deeply the flaws in your data, from bias in data collection to missing values in data files. Many (hopefully most) of the projects below have significant sections documenting their sources and efforts / decision making around this topic.

Another big emphasis in the course is on understanding what the data will be used for, and by whom. Tied to this, we talk extensively about intelligibility in machine learning, the importance of narrative in visualization (and visualization in general), and the importance of defining the question you are answering.

Here are some of the highlights over the last three years:

Screen Shot 2016-05-09 at 12.51.04 PM

Bus bunching is a phenomenon that can impact bus wait times. One of my 2016 students has been collecting data and extensively studying the phenomenon. His final project in the class drew on this data set and explores visual representations of the phenomenon.


Screen Shot 2016-05-09 at 12.56.20 PMYelp data is always an area of interest. In 2014 … In 2015 students explored which state has the best pizza :). In 2016, the ‘Bon Yinzers‘ developed a wonderful series of visualizations of factors that affect popularity of Pittsburgh restaurants. They uncovered some interesting phenomena such as the unexpectedly off-cycle checkin times of the most active Yelp users in Pittsburgh.

Screen Shot 2016-05-09 at 1.00.13 PMSan Francisco Crime Alert explores the likelihood of different types of  Crime in different SF area neighborhoods. Their prediction algorithm gives you a way to explore the prevelance of major and minor crime in terms of time of year, time of day, and location.

Screen Shot 2016-05-09 at 1.03.50 PMIn 2015, a group collected and analyzed data set of tweets by potential ISIS supporters, with the goal of ultimately engaging others in helping to label such data and understand how ISIS supporter accounts differ from other accounts with sometimes similar tweets (e.g. news accounts or bloggers).

Screen Shot 2016-05-09 at 1.12.00 PMOften, a goal of class students is more about policy than about end users. In 2015 Healt$care explores the quality of healthcare and its relationship to dollars spent across the U.S. in a highly visual fashion.


Screen Shot 2016-05-09 at 1.17.31 PMIn 2014, a group asked what jobs are popular in what parts of the US?. Again a combination of data visualization and prediction supports exploration of the question. A similar approach was explored by a 2014 group that collected data about movie piracy and its relationship to DVD release strategies.

Sadly, not all of the older projects still work (web standards change so fast!). I wish I could provide links to work such as the Reddit AMA visualization pictured here.

Screen Shot 2015-05-04 at 4.08.56 PM


Learning languages

I’ve mentioned before that one of my sabbatical goals was to learn a new language (Hindi). I am not fluent, but I think I came a fair way with it, and I want to comment on the role of different technologies and approaches in our successes (and failures) as a family to learn the three languages that we tackled on this trip.

One of the most useful technologies we employed was the Rosetta Stone software. The kids loved Rosetta Stone, which we started using almost as soon as the sabbatical was approved to get them familiar with Hindi. They spent about 30 minutes at a time on it at the beginning. At our peak, this happened almost daily (after we left Pittsburgh but before we were settled in India. Eventually we hired a tutor (a wonderful friend now) to come for about an hour most days instead. The kids were far more resistant to being tutored than they were to using the software, but I feel we covered much more ground in those hours. We made up all sorts of games, retold fairy tales, played shop, and generally did our best to make it child friendly.

Hindi was a relatively hard language to learn (new alphabet, different sentence structures, and so on). Once we got past the vocabulary phase,  progress was slowish. Still, by the end of the fall we could have whole conversations in Hindi as a family. The kids were not alone in learning the language: Anind and I were trying very hard to learn it as well and we tried to speak it at meals, with our Indian driver, and so on. So between the tutoring and the daily practice opportunities, they used Rosetta Stone less and less.

The Rosetta Stone was not a pure success. It required the right context to be used — enough motivation, and not too much other support. We almost never used the German Rosetta Stone I bought, and of course the kids are far more fluent (they are immersed, unlike with Hindi, and it is a much easier language for them to learn). Use of Rosetta Stone is rare at this point, and mostly me.

You get free tutoring through the online package with Rosetta Stone, along with access to online games. The games are a fun way to practice but slow. When possible, I sign up and have a session with an online tutor. It is based on the material I’m currently covering in the software. However, they only let the kids do it when there’s no other remote participants, which is sometimes hard to find at popular times in the early stages of learning a language.

As a computer scientist, I cannot help but be impressed by the software. It is dedicated to learning language through immersion, and the authors have done an excellent job of maintaining that throughout the software and the tutoring sessions. It uses speech recognition to check pronunciation, and provides multi-media support for learning. And it works, if you put the time in with it, you learn. To my mind, it’s a success as an educational tool and an interactive tool. It supposedly has a social side as well, though as a Hindi learner I was one of few and could not take advantage of it. I’d be curious to see what it’s like.

It has always seemed such a shame to me, that learning multiple languages is not a norm in the United States. During our travels we met 10 year olds who spoke 5 or 6 languages, all with fluent ease. They never needed to touch a piece of software or a tutor. The world has become so small, yet so many of us in the United States fail to give our children the gift of understanding and the mastery of complexity that comes with learning multiple languages. Most of my swiss cousins have raised children who are bi- or tri- lingual, and without the errors that plague even my immersed children.  There is no substitute for that level of early exposure.