Archives

3D Printed Wireless Analytics

Leave a comment

Wireless Analytics for 3D Printed Objects: Vikram Iyer, Justin Chan, Ian Culhane, Jennifer Mankoff, Shyam Gollakota UIST, Oct. 2018 [PDF]

We created a wireless physical analytics system works with commonly available conductive plastic filaments. Our design can enable various data capture and wireless physical analytics capabilities for 3D printed objects, without the need for electronics.

We make three key contributions:

(1) We demonstrate room scale backscatter communication and sensing using conductive plastic filaments.

(2) We introduce the first backscatter designs that detect a variety of bi-directional motions and support linear and rotational movements. An example is shown below

(3) As shown in the image below, we enable data capture and storage for later retrieval when outside the range of the wireless coverage, using a ratchet and gear system.

We validate our approach by wirelessly detecting the opening and closing of a pill bottle, capturing the joint angles of a 3D printed e-NABLE prosthetic hand, and an insulin pen that can store information to track its use outside the range of a wireless receiver.

Selected Media

6 of the most amazing things that were 3D-printed in 2018 (Erin Winick, MIT Technology Review, 12/24/2018)

Researchers develop 3D printed objects that can track and store how they are used (Sarah McQuate), UW Press release. 10/9/2018

Assistive Objects Can Track Their Own Use (Elizabeth Montalbano), Design News. 11/14/2018

People

Students

Vikram Iyer
Justin Chan
Ian Culhane

Faculty

Jennifer Mankoff
Shyam Gollakota

Contact: printedanalytics@cs.washington.edu

Minxuan Gao

Leave a comment

Hi, I’m Minxuan Gao and I’m a senior in Tsinghua University majoring in Software Engineering. I’m always passionate about creating new and innovative way of people interacting with every day objects by seeing, touching, listening using data-driven methods. My research focus lies in Human Computer Interaction and I am currently working on the SPRITEs project.

Yasaman Sefigar

Leave a comment
Headshot of Yasaman wearing a green scarf smiling

I am a PhD student at the University of Washington’s Paul G. Allen School of Computer Science and Engineering. My current research is focused on human behavior modeling. More specifically, I model and study routine behaviors and the impact of external events on them in the context of wellbeing and mobility. I am also interested in end-user tools and interfaces to improve data collection, exploration, and analysis processes.

My past research spans from designing interfaces for end-user robot programming, to modeling human-object interactions in realistic videos, to studying affective haptic human-robot interaction for psychological enrichment.

My Google Scholar page is https://goo.gl/D1QbSJ

Some recent projects (see more)

Passively-sensing Discrimination

A lineplot showing anxiousness (Y axis, varying from 0 to 4) over time (X axis). Each student in the study is plotted as a different line over each day of the quarter. The plot overall looks very messy, but two things are clear; Every student has a very different trajectory from every other, with all of them going up and down multiple times. And the average, overall, shown is a fit line, is fairly low and slightly increasing (from about .75 to just under 1).

College during COVID

A diagram showing how we build stress & resilience frameworks for student mental health. The framework has three components -- social characteristics, cumulative stress, and resilience resources. At the center where all three overlap is perceived stress, anxiety and depression.

Distress and resilience among marginalized undergraduates

A complex diagram showing the flow of information from sensors through feature extraction, formation of a user behavior profile, and then final prediction.

Personalized behavior modeling: depression detection

Venkatesh Potluri

Leave a comment

Venkatesh Potluri is a Ph.D. student at the Paul G. Allen Center for Computer Science & Engineering at University of Washington. He is advised by Prof Jennifer Mankoff and Prof Jon Froehlich. Venkatesh believes that technology, when designed right, empowers everybody to fulfill their goals and aspirations. His broad research goals are to upgrade accessibility to the ever-changing ways of our interactions with technology, and, improve the independence and quality of life of people with disabilities. These goals stem from his personal experience as a researcher with a visual impairment. His research focus is to enable developers with visual impairments perform a variety of programming tasks efficiently. Previously, he was a Research Fellow at Microsoft Research India, where his team was responsible for building CodeTalk, an accessibility framework and a plugin for better IDE accessibility. Venkatesh earned a master’s degree in Computer Science at International Institute of Information Technology Hyderabad, where his research was on audio rendering of mathematical content.

You can find more information about him at https://venkateshpotluri.me

ALT text generated for two charts includes (at left): A radial line plot titled ’avg. Monthly hours of sunshine in seattle’. The x-axis ranges from jan to dec using a datetime scale and the y-axis ranges from 0 to 312 using a linear scale. Avg. Hours of sunshine is plotted in orange. Avg. Hours of sunshine has a minimum value of y=52 at x=dec, a maximum value of y=312 at x=july, and an average of y=180.2. (at right) A 7x6 heatmap titled ’number of points from combined gaussians’. The x-axis ranges from -8 to 6, the y-axis ranges from -4 to 8, and number of gaussian points is plotted on the z-axis from 0 to 61, all using linear scales. The data has a minimum value of z=0 at (-8, -4), a maximum value of z=61 at (0, -4), and an average of z=14.19.

MatPlotAlt

An AI generated decorative image of a person standing alone outside of a corporate boardroom, excluded.

Identifying and improving disability bias in GPT-based resume screening

Wordcloud of alt text assigned to images in notebooks prominently featuring useless text such as "png" "image" "alt" and "Colab"

Notably Inaccessible

Left: A hand is shown moving back and forth on a laptop touchpad. Right: A TED video is playing and the slider is highlighted to indicate that the hands motion controls the video position.

Touchpad Mapper

Two AI-generated images show different interpretations of a doll-sized “crocheted lavender husky wearing ski goggles,” including one pictured outdoors and one against a wooden background.

Generative Artificial Intelligence’s Utility for Accessibility

PSST: Enabling Blind or Visually Impaired Developers to Author Sonifications of Streaming Sensor Data

Mixed Abilities and Varied Experiences in a Virtual Summer Internship

Anticipate and Adjust: Cultivating Access in Human-Centered Methods

BLV Understanding of Visual Semantics

Xin Liu

Leave a comment

Xin is a first-year Ph.D. student with Jennifer Mankoff and Shwetak Patel in the Paul G. Allen School of Computer Science & Engineering at the University of Washington – Seattle. Prior to joining UW, he obtained a Bachelor’s degree in computer science from the University of Massachusetts Amherst in 2018. While at UMass Amherst, he received a 21st Century Leaders Award, Rising Researcher Award, and Outstanding Undergraduate Achievements Award. He is interested in using wearable sensing, human-computer interaction and machine learning to advancing healthcare.

Website: https://homes.cs.washington.edu/~xliu0/

Orson (Xuhai) Xu (PhD, co-advised with Anind Dey)

Leave a comment

Orson is a Ph.D. student working with Jennifer Mankoff  and Anind K. Dey in the Information School at the University of Washington – Seattle. Prior to joining UW, he obtained his Bachelor’s degrees in Industrial Engineering (major) and Computer Science (minor) from Tsinghua University in 2018. While at Tsinghua, he received Best Paper Honorable Mentioned Award (CHI 2018), Person of the Year Award and Outstanding Undergraduate Awards. His research focuses on two aspects in the intersection of human-computer interaction, ubiquitous computing and machine learning: 1) the modeling of human behavior such as routine behavior and 2) novel interaction techniques.

Visit Orson’s homepage at : orsonxu.com

Some recent projects (see more)

Interactiles

Leave a comment

The absence of tactile cues such as keys and buttons makes touchscreens difficult to navigate for people with visual impairments. Increasing tactile feedback and tangible interaction on touchscreens can improve their accessibility. However, prior solutions have either required hardware customization or provided limited functionality with static overlays. In addition, the investigation of tactile solutions for large touchscreens may not address the challenges on mobile devices. We therefore present Interactiles, a low-cost, portable, and unpowered system that enhances tactile interaction on Android touchscreen phones. Interactiles consists of 3D-printed hardware interfaces and software that maps interaction with that hardware to manipulation of a mobile app. The system is compatible with the built-in screen reader without requiring modification of existing mobile apps. We describe the design and implementation of Interactiles, and we evaluate its improvement in task performance and the user experience it enables with people who are blind or have low vision.

XiaoyiZhang, TracyTran, YuqianSun, IanCulhane, ShobhitJain, JamesFogarty, JenniferMankoff: Interactiles: 3D Printed Tactile Interfaces to Enhance Mobile Touchscreen Accessibility. ASSETS 2018: To Appear [PDF]

Figure 2. Floating windows created for number pad (left), scrollbar (right) and control button (right bottom). The windows can be transparent; we use colors for demonstration.

Figure 4. Average task completion times of all tasks in the study.

EDigs

Leave a comment

eDigs logoJennifer Mankoff, Dimeji Onafuwa, Kirstin Early, Nidhi Vyas, Vikram Kamath:
Understanding the Needs of Prospective Tenants. COMPASS 2018: 36:1-36:10

EDigs is a research project group in Carnegie Mellon University working on sustainability. Our research is focused on helping people find a perfect rental through machine learning and user research.

We sometimes study how our members use EDigs in order to learn how to build software support for successful social communities.

eDigs websiteScreenshot of edigs.org showing a mobile app, facebook and twitter feeds, and information about it.

Nonvisual Interaction Techniques at the Keyboard Surface

Leave a comment

Rushil Khurana,Duncan McIsaac, Elliot Lockerman,Jennifer Mankoff Nonvisual Interaction Techniques at the Keyboard Surface, CHI 2018, To Appear

A table (shown on screen). Columns are mapped to the number row of the keyboard and rows to the leftmost column of keys, and (1) By default the top left cell is selected. (2) The right hand presses the ‘2’ key, selecting the second column (3) The left hand selects the next row (4) The left hand selects the third row. In each case, the position of the cell and its content are read out aloud.

Web user interfaces today leverage many common GUI design patterns, including navigation bars and menus (hierarchical structure), tabular content presentation, and scrolling. These visual-spatial cues enhance the interaction experience of sighted users. However, the linear nature of screen translation tools currently available to blind users make it difficult to understand or navigate these structures. We introduce Spatial Region Interaction Techniques (SPRITEs) for nonvisual access: a novel method for navigating two-dimensional structures using the keyboard surface. SPRITEs 1) preserve spatial layout, 2) enable bimanual interaction, and 3) improve the end user experience. We used a series of design probes to explore different methods for keyboard surface interaction. Our evaluation of SPRITEs shows that three times as many participants were able to complete spatial tasks with SPRITEs than with their preferred current technology.

Talk [Slides]:

Sample Press:

KOMO Radio | New screen reader method helps blind, low-vision users browse complex web pages

Device helps blind, low-vision users better browse web pages. Allen Cone

Graph showing task completion rates for different kinds of tasks in our user study

A user is searching a table (shown on screen) for the word ‘Jill’. Columns are mapped to the number row of the keyboard and rows to the leftmost column of keys. (1) By default the top left cell is selected. (2) The right hand presses the ‘2’ key, selecting the second column (3) The left hand selects the next row (4) The left hand selects the third row. In each case, the number of occurrences of the search query in the respective column or row are read aloud. When the query is found, the position and content of the cell are read out aloud.