A user is tapping on their cheek. The tapping is sensed by a microphone in the earphone.

Interaction via Wireless Earbuds

Xuhai XuHaitian ShiXin YiWenjia LiuYukang YanYuanchun ShiAlex Mariakakis, Jennifer Mankoff, Anind K. Dey:
EarBuddy: Enabling On-Face Interaction via Wireless Earbuds. CHI 2020: 1-14

Past research regarding on-body interaction typically requires custom sensors, limiting their scalability and generalizability. We propose EarBuddy, a real-time system that leverages the microphone in commercial wireless earbuds to detect tapping and sliding gestures near the face and ears. We develop a design space to generate 27 valid gestures and conducted a user study (N=16) to select the eight gestures that were optimal for both human preference and microphone detectability. We collected a dataset on those eight gestures (N=20) and trained deep learning models for gesture detection and classification. Our optimized classifier achieved an accuracy of 95.3%. Finally, we conducted a user study (N=12) to evaluate EarBuddy’s usability. Our results show that EarBuddy can facilitate novel interaction and that users feel very positively about the system. EarBuddy provides a new eyes-free, socially acceptable input method that is compatible with commercial wireless earbuds and has the potential for scalability and generalizability

Leave a Reply

Your email address will not be published. Required fields are marked *

− 6 = 2