Archives

Interaction via Wireless Earbuds

Leave a comment

Xuhai Xu, Haitian Shi, Xin Yi, Wenjia Liu, Yukang Yan, Yuanchun Shi, Alex Mariakakis, Jennifer Mankoff, Anind K. Dey:
EarBuddy: Enabling On-Face Interaction via Wireless Earbuds. CHI 2020: 1-14

Past research regarding on-body interaction typically requires custom sensors, limiting their scalability and generalizability. We propose EarBuddy, a real-time system that leverages the microphone in commercial wireless earbuds to detect tapping and sliding gestures near the face and ears. We develop a design space to generate 27 valid gestures and conducted a user study (N=16) to select the eight gestures that were optimal for both human preference and microphone detectability. We collected a dataset on those eight gestures (N=20) and trained deep learning models for gesture detection and classification. Our optimized classifier achieved an accuracy of 95.3%. Finally, we conducted a user study (N=12) to evaluate EarBuddy’s usability. Our results show that EarBuddy can facilitate novel interaction and that users feel very positively about the system. EarBuddy provides a new eyes-free, socially acceptable input method that is compatible with commercial wireless earbuds and has the potential for scalability and generalizability

  • A user is tapping on their cheek. The tapping is sensed by a microphone in the earphone.
  • Plot of Test Accuracy versus number of left-out user's additional samples
  • Two barplots from the EarBuddy user study comparing EarBuddy, Table and Pocket.
  • Gesture design of EarBuddy.

Project 2: Build a Better Button

Leave a comment

Learning Goals for the Project

  • Learn about Circuit design
  • Learn how to communicate between an Arduino and your phone
  • Build a simple circuit that is enhanced by its connection to your phone

Basic Requirements for Project

Your project should demonstrate your ability to either:

  • Take input from at least one button (or other sensor), and connect it to some interesting service
  • Your focus should be on circuit design and Arduino programming. You don’t need to create a custom phone app. You can if you want create a custom case or button using 3d printing.

You should make a case for why this is an assistive technology of some sort. For example, you could build a door opening sensor (using a button or proximity sensor) that causes your phone to announce the door was opened, or a single switch control for scrolling or tabbing through a web page, or a capacitive sensor that captures a log of how often a cane is used.

There is some great software that con be connected to the Arduino including 1Shield, AppInventor, Blynk and IFTTT. Some work only for Android, others for both Android and iPhone.

There are lots of really great examples online of arduino based projects, arduino projects that involve smartphones, and arduino projects that involve 3D printing or laser cutting. Many of them are too complex for the expectations of this project, though they might help to inspire final projects, or give you ideas for something simple you can do in a week. Here is a sample:

Hand In

Create a Thingiverse or Instructables page for your project with a brief description of the project, a video, any 3D printed files, and a schematic for your circuit. Turn the URL in by email with the subject: Project 2. Be prepared to demo your project in class.

PointsDescription
1 or 0 Project uses physical computing to solve an accessibility problem
1 or 0Project communicates with your phone in some way
1 or 0Project includes a working circuit that you designed
1 or 0Project includes at least one button
1 or 0 Project includes some kind of response to the button
1 or 0Thingiverse or Instructables page describes project in a reproducable fashion.