Archives

Modeling Human Routines

Leave a comment

Modeling and Understanding Human Routine Behavior

Human routines are blueprints of behavior, which allow people to accomplish their purposeful repetitive tasks and activities. People express their routines through actions that they perform in the particular situations that triggered those actions. An ability to model routines and understand the situations in which they are likely to occur could allow technology to help people improve their bad habits, inexpert behavior, and other suboptimal routines. In this project we explore generalizable routine modeling approaches that encode patterns of routine behavior in ways that allow systems, such as smart agents, to classify, predict, and reason about human actions under the inherent uncertainty present in human behavior. Such technologies can have a positive effect on society by making people healthier, safer, and more efficient in their routine tasks.

Routines_Viz_Tool

Modeling and Understanding Human Routine Behavior
Nikola Banovic, Tofi Buzali, Fanny Chevalier, Jennifer Mankoff, and Anind K. Dey
In Proceedings of the 2016 ACM annual conference on Human Factors in Computing Systems(CHI ’16). ACM, New York, NY, USA.
Honorable Mention Award

Threadsteading

Leave a comment

 

In work done collaboratively with Disney Research Pittsburgh and led by Gillian Smith of Northeastern we explored a multi-player game that can be embedded into a quilting and/or embroidery machine interface. Gameplay is constrained by the fact that only a single thread of fabric can be drawn over time. Players compete to ‘scout’ over a map (a hex grid), where different hexes have different costs to explore.

Threadsteading map and custom control panel for quilting machine (light sunder fabric)

Threadsteading was accepted to Alt.CTRL.GDC.

G. Smith, A. Grow, C. Liu, L. Albaugh, J. Mankoff and J. McCann. Threadsteading: A single-line, two-player, territory-control game for quilting and embroidery machines. alt.ctrl.GDC 2016.

Dynamic question ordering

Leave a comment

In recent years, surveys have been shifting online, offering the possibility for adaptive questions, where later questions depend on responses to earlier questions. We present a general framework for dynamically ordering questions, based on previous responses, to engage respondents, improving survey completion and imputation of unknown items. Our work considers two scenarios for data collection from survey-takers. In the first, we want to maximize survey completion (and the quality of necessary imputations) and so we focus on ordering questions to engage the respondent and collect hopefully all the information we seek, or at least the information that most characterizes the respondent so imputed values will be accurate. In the second scenario, our goal is to give the respondent a personalized prediction, based on information they provide. Since it is possible to give a reasonable prediction with only a subset of questions, we are not concerned with motivating the user to answer all questions. Instead, we want to order questions so that the user provides information that most reduces the uncertainty of our prediction, while not being too burdensome to answer.

Publications
Kirstin Early, Stephen E. Fienberg, Jennifer Mankoff. (2016). Test time feature ordering with FOCUS: Interactive predictions with minimal user burden. In Proceedings of 2016 ACM Conference on Pervasive and Ubiquitous Computing. Honorable Mention: Top 5% of submissions. Talk slides.

3D printed attachments

Leave a comment

Encore: 3D printed attachments

What happens when you want to 3D print something that must interact with the real world? The Encore project makes it possible to 3D print objects that must attach to things in the real world. Encore provides an interface that, given an imported object and a chosen attachment method, visualizes metrics relating the goodness of the attachment. In addition, once an attachment type and location is chosen, Encore helps to produce the necessary support structure for attachment. Encore supports three main types of attachment: print-over, print-to-affix, and print-through.

Print-Over

Print-over attachments are printed directly on the existing object. This works well if the object is flat enough that the print head won’t encounter obstacles as it moves, and the object is made of a material that the printed material will easily adhere to. Encore helps by finding a rotation of the existing object that minimizes obstacles, and generating support material to hold the existing object in place.

Printing a magnet holder over a Teddy bear toy.
 
Left: printing an LED casing on a battery to make a simple torch; right: printing a handle to an espresso cup.
 

Print-to-Affix

An alternative that is useful when the existing object does not fit on the print bed is print-to-affix. In this approach, the attachment is designed to fit snugly against the existing object. It may be glued in place, or can include holes for a strap, such as a zip tie.

Left: printing a structure to make a glue gun stand; right: printing a reusable four-pack holder.
 

Print-Through

Finally, sometimes the attachment should be interlocked more loosely with the existing object. In this case, the process is to begin printing and stop the print partway through so that the existing object can be inserted. Encore can compute when this stopping point should be (and
whether it is possible)

A name tag printed through a pair of scissors
 
A bracelet printed through a charm
 

Encore the Design Tool

Encore is implemented in WebGL. It supports importation of an existing object, selection of an attachment, and then lets the user click to indicate where the attachment will go. Given this information, it uses geometric analysis to compute metrics for goodness of attachment, such as attachability and strength. Encore visualizes them using a heat map so that the user can adjust the attachment point.


Encore visualizes which parts of a wrench are more attachable when printing over a handle.

More Examples


Using print-to-affix to make a trophy from an egg holder
 

Using print-over to make a minion keychain
 

Using print-over to add a hanger to a screwdriver handle
 

Using print-through to make a key ring.
 

Using print-to-affix to make a battery case
 
Xiang ‘Anthony’ Chen, Stelian Coros, Jennifer Mankoff, Scott Hudson (2015). Encore: 3D Printed Augmentation of Everyday Objects with Printed-Over, Affixed and Interlocked Attachments. Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST 2015)

Helping Hands

Leave a comment

Prosthetic limbs and assistive technology (AT) require customization and modification over time to effectively meet the needs of end users. Yet, this process is typically costly and, as a result, abandonment rates are very high. Rapid prototyping technologies such as 3D printing have begun to alleviate this issue by making it possible to inexpensively, and iteratively create general AT designs and prosthetics. However for effective use, technology must be applied using design methods that support physical rapid prototyping and can accommodate the unique needs of a specific user. While most research has focused on the tools for creating fitted assistive devices, we focus on the requirements of a design process that engages the user and designer in the rapid iterative prototyping of prosthetic devices.

We present a case study of three participants with upper-limb amputations working with researchers to design prosthetic devices for specific tasks. Kevin wanted to play the cello, Ellen wanted to ride a hand-cycle (a bicycle for people with lower limb mobility impairments), and Bret wanted to use a table knife. Our goal was to identify requirements for a design process that can engage the assistive technology user in rapidly prototyping assistive devices that fill needs not easily met by traditional assistive technology. Our study made use of 3D printing and other playful and practical prototyping materials. We discuss materials that support on-the-spot design and iteration, dimensions along which in-person iteration is most important (such as length and angle) and the value of a supportive social network for users who prototype their own assistive technology. From these findings we argue for the importance of extensions in supporting modularity, community engagement, and relatable prototyping materials in the iterative design of prosthetics

Prosthetic limbs and assistive technology (AT) require customization and modification over time to effectively meet the needs of end users. Yet, this process is typically costly and, as a result, abandonment rates are very high. Rapid prototyping technologies such as 3D printing have begun to alleviate this issue by making it possible to inexpensively, and iteratively create general AT designs and prosthetics. However for effective use, technology must be applied using design methods that support physical rapid prototyping and can accommodate the unique needs of a specific user. While most research has focused on the tools for creating fitted assistive devices, we focus on the requirements of a design process that engages the user and designer in the rapid iterative prototyping of prosthetic devices.

We present a case study of three participants with upper-limb amputations working with researchers to design prosthetic devices for specific tasks. Kevin wanted to play the cello, Ellen wanted to ride a hand-cycle (a bicycle for people with lower limb mobility impairments), and Bret wanted to use a table knife. Our goal was to identify requirements for a design process that can engage the assistive technology user in rapidly prototyping assistive devices that fill needs not easily met by traditional assistive technology. Our study made use of 3D printing and other playful and practical prototyping materials. We discuss materials that support on-the-spot design and iteration, dimensions along which in-person iteration is most important (such as length and angle) and the value of a supportive social network for users who prototype their own assistive technology. From these findings we argue for the importance of extensions in supporting modularity, community engagement, and relatable prototyping materials in the iterative design of prosthetics

Photos

The length of his arm, significantly impacted Kevin’s ability to play the cello. During the prototyping process he was able to test a variety of lengths by adding on layers of Lego to his prototyping prosthetic.
Kevin enjoys his latest cello bow holder so much that, according to his mother, she’s not allowed to touch or adjust it in case it gets messed up.
“Life changing technology can come in many forms; it can come in the form of Styrofoam and pipe cleaners,” Said one participant while working on adjusting the angle of her prosthetic to maximize her comfort while gripping a hand-cycle.
This 3D printed prosthetic demonstrates the value of creating prosthetics for a very specific task (in this case control of a hand-bike), and the importance of manipulating factors such as angle.
Brett M. Fadgen MSN, CRNA. is a practicing Nurse Anesthetist with an amputation below his left elbow. He has customized his profesional grade prosthetics to help him perform nursing tasks.
Despite having a wide variety of professionaly designed prosthetic devices, Bret never found a good solution for using a common table knife. Using 3D scanning technology and modeling clay we were able to design a custom grip that could stabilize the knife in his myoelectric hand.

Project Files

https://www.thingiverse.com/thing:2365703

Project Publications

Helping Hands: Requirements for a Prototyping Methodology for Upper-limb Prosthetics Users

Reference:

Megan Kelly Hofmann, Jeffery Harris, Scott E Hudson, Jennifer Mankoff. 2016.Helping Hands: Requirements for a Prototyping Methodology for Upper-limb Prosthetics Users. InProceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems (CHI ’16). ACM, New York, NY, USA, 525-534.

Making Connections: Modular 3D Printing for Designing Assistive Attachments to Prosthetic Devices

Reference:

Megan Kelly Hofmann. 2015. Making Connections: Modular 3D Printing for Designing Assistive Attachments to Prosthetic Devices. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility (ASSETS ’15). ACM, New York, NY, USA, 353-354. DOI=http://dx.doi.org/10.1145/2700648.2811323

Supporting Navigation in the Wild for the Blind

Leave a comment
uncovering_thumbnailSighted individuals often develop significant knowledge about their environment through what they can visually observe. In contrast, individuals who are visually impaired mostly acquire such knowledge about their environment through information that is explicitly related to them. Our work examines the practices that visually impaired individuals use to learn about their environments and the associated challenges. In the first of our two studies, we uncover four types of information needed to master and navigate the environment. We detail how individuals’ context impacts their ability to learn this information, and outline requirements for independent spatial learning. In a second study, we explore how individuals learn about places and activities in their environment. Our findings show that users not only learn information to satisfy their immediate needs, but also to enable future opportunities – something existing technologies do not fully support. From these findings, we discuss future research and design opportunities to assist the visually impaired in independent spatial learning.

Uncovering information needs for independent spatial learning for users who are visually impaired. Nikola Banovic, Rachel L. Franz, Khai N. Truong, Jennifer Mankoff, and Anind K. DeyIn Proceedings of the 15th international ACM SIGACCESS conference on Computers and accessibility (ASSETS ’13). ACM, New York, NY, USA, Article 24, 8 pages. (pdf)

Layered Fabric Printing

Leave a comment

A Layered Fabric 3D Printer for Soft Interactive Objects. Huaishu Peng, Jennifer Mankoff, Scott E. Hudson, James McCann. CHI ’15 Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 2015.

In work done collaboratively with Disney Research and led by Disney Intern Huaishu Peng (of Cornell), we have begun to explore alternative material options for fabrication. Unlike traditional 3D printing, which uses hard plastic, this project made use of cloth (in the video shown above, felt). In addition to its aesthetic properties, fabric is deformable, and the degree of deformability can be controlled. Our printer, which works by gluing layers of laser-cut fabric to each other also allows for dual material printing, meaning that layers of conductive fabric can be inserted. This allows fabric objects to also easily support embedded electronics. This work has been in the news recently, and was featured at Adafruit; Futurity; Gizmodo; Geek.com and TechCrunch, among others.

Infant Oxygen Monitoring

Leave a comment

Hospitalized children on continuous oxygen monitors generate >40,000 data points per patient each day. These data do not show context or reveal trends over time, techniques proven to improve comprehension and use. Management of oxygen in hospitalized patients is suboptimal—premature infants spend >40% of each day outside of evidence-based oxygen saturation ranges and weaning oxygen is delayed in infants with bronchiolitis who are physiologically ready. Data visualizations may improve user knowledge of data trends and inform better decisions in managing supplemental oxygen delivery.

First, we studied the workflows and breakdowns for nurses and respiratory therapists (RTs) in the supplemental oxygen delivery of infants with respiratory disease. Secondly, using end-user design we developed a data display that informed decision-making in this context. Our ultimate goal is to improve the overall work process using a combination of visualization and machine learning.

Visualization mockup for displaying O2 saturation over time to nurses.
Visualization mockup for displaying O2 saturation over time to nurses.

3D Printed Prosthetics: Case Study

Leave a comment

Readings

  • Megan Hofmann, Julie Burke, Jon Pearlman, Goeran Fiedler, Andrea Hess, Jon Schull, Scott E. Hudson, Jennifer Mankoff: Clinical and Maker Perspectives on the Design of Assistive Technology with Rapid Prototyping Technologies. ASSETS 2016: 251-256
  • Cynthia L. Bennett, Keting Cen, Katherine Muterspaugh Steele, Daniela K. Rosner: An Intimate Laboratory?: Prostheses as a Tool for Experimenting with Identity and Normalcy. CHI 2016: 1745-1756

Optional: 

  • Jeremiah Parry-Hill, Patrick C. Shih, Jennifer Mankoff, Daniel Ashbrook: Understanding Volunteer AT Fabricators: Opportunities and Challenges in DIY-AT for Others in e-NABLE. CHI 2017: 6184-6194

 

StepGreen

Leave a comment

The goal of the Stepgreen project is to leverage Internet scale technologies to create opportunities for reduced energy consumption. The original vision of the project was to leverage existing online social networks to encourage individual change. Since then the project has broadened to include a number of other ideas. We have explored the impact of demographics on energy use practices; studied the value of empathetic figures such as a polar bear for motivation and exploredorganizational-level planning. We have also developed mobile technologies that can provide feedback about green actions on the go.

StepGreen.org Website

StepGreen.org Website

Try StepGreen.org out: The Stepgreen.org website provides a mechanism for allowing individuals to report on and track their environmental impact. It includes a visualization that can be displayed on an individual’s social networking web page. Go to Stepgreen.organd see for yourself how we leverage social networks to engage individuals in green behaviors.

Learn about our software products. Stepgreen  is a service that we are hoping to share with non-profits that are encouraging behavior change,  such as an open API you can use to build your own clients for encouraging green behavior. Please contact us at stepgreen@cs.cmu.edu if you are interested in collaborating with us. 

Learn about our research and our publications

Keep in touch with us through our Facebook page  and Twitter account.

Sample Publications

JOURNAL PAPERS & MAGAZINE ARTICLES

  1. J. Mankoff. “HCI and Sustainability: A Tale of Two Motivations,” Interactions.
  2. Dillahunt, T. & Mankoff, J. (2011) In the dark, out in the cold. ACM Crossroads 17(4):39-41
  3. Jennifer Mankoff, Robin Kravets, Eli Blevis, Some Computer Science Issues in Creating a Sustainable World, IEEE Computer 41(8):102-105. (pdf)
    1. Reprinted as: Jennifer Mankoff, Robin Kravets and Eli Blevis, Some Computer Science Issues in Creating a Sustainable World. Posted on November 17th, 2008 in Articles, Climate, OpEd, Technology http://www.earthzine.org/2008/11/17/some-computer-science-issues-in-creating-a-sustainable-world/

CONFERENCE PAPERS

  1. Tawanna Dillahunt, Jennifer Mankoff, Eric Paulos. Understanding Conflict Between Landlords and Tenants: Implications for Energy Sensing and Feedback. Ubicomp ’10.  (full paper)(pdf)
  2. Jennifer Mankoff, Susan R. Fussell, Tawanna Dillahunt, Rachel Glaves, Catherine Grevet, Michael Johnson, Deanna Matthews, H. Scott Matthews, Robert McGuire, Robert Thompson. StepGreen.org: Increasing Energy Saving Behaviors via Social Networks. ICWSM’10.  (full paper) (pdf, video of talk)
  3. C. Grevet, J. Mankoff, S. D. Anderson Design and Evaluation of a Social Visualization aimed at Encouraging Sustainable Behavior. In Proceedings of HICSS 2010.  (full paper) (pdf)
  4. T. Dillahunt, J. Mankoff, E. Paulos, S. Fussell It’s Not All About “Green”: Energy Use in Low-Income Communities. In Proceedings of Ubicomp 2009. (Full paper) (pdf)
  5. J. Froehlich, T. Dillahunt, P. Klasnja, J. Mankoff, S. Consolvo, B. Harrison, J. A. Landay, UbiGreen: Investigating a Mobile Tool for Tracking and Supporting Green Transportation Habits. In Proceedings of CHI 2009. (Full paper) (pdf)
  6. J. Schwartz, J. Mankoff, H. Scott Matthews. Reflections of everyday activity in spending data. In Proceedings of CHI 2009.  (Note). (pdf)
  7. Jennifer Mankoff, Deanna Matthews, Susan R. Fussell and Michael Johnson. Leveraging Social Networks to Motivate Individuals to Reduce their Ecological Footprints. HICSS 2007. (pdf)

OTHER

  1. Rachael Nealer, Christopher Weber, H. Scott Matthews and Chris Hendrickson. Energy and Environmental Impacts of Consumer Purchases: A Case Study on Grocery Purchases. ISSST 2010
  2. Dillahunt, T., Becker, G., Mankoff, J. and Kraut, R. Motivating Environmentally Sustainable Behavior Changes with a Virtual Polar Bear.” Pervasive 2008 workshop on Pervasive Persuasive Technology and Environmental Sustainability. (pdf)
  3. Johnson, M., Fussell, S. Mankoff, J., Matthwes, D., and Setlock, L. “When Users Pledge to Take Green Actions, Are They Solving a Decision Problem?” INFORMS Fall 2008 Conference. (ppt)
  4. Johnson, M., Fussell, S. Mankoff, J. and Matthwes, D. “How Does Problem Representation Influence Decision Performance and Attitudes?” INFORMS Fall 2007 Conference. Abstract
  5.  Johnson, M.P. 2006. “Public Participation and Decision Support Systems: Theory, Requirements, and Applications.” For presentation at Association of Public Policy Analysis and Management Fall Conference, Madison, WI, November 3, 2006. (pdf)