Personalized behavior modeling: depression detection d

Xuhai Xu, Prerna Chikersal, Janine M. Dutcher, Yasaman S. Sefidgar, Seo Woosuk, Michael J. Tumminia, Daniella K. Villalba, Sheldon Cohen,
Kasey G. Creswell, Creswell, J. David, Afsaneh Doryab, Paula S. Nurius, Eve Riskin, Anind K. Dey, & Jennifer Mankoff. Leveraging Collaborative-Filtering for Personalized Behavior Modeling: A Case Study of Depression Detection among College Students. Proc. ACM interact. mob. wearable ubiquitous technol., Article 41. (March 21) 27pages.

The prevalence of mobile phones and wearable devices enables the passive capturing and modeling of human behavior at an unprecedented resolution and scale. Past research has demonstrated the capability of mobile sensing to model aspects of physical health, mental health, education, and work performance, etc. However, most of the algorithms and models proposed in previous work follow a one-size-fits-all (i.e., population modeling) approach that looks for common behaviors amongst all users, disregarding the fact that individuals can behave very differently, resulting in reduced model performance. Further, black-box models are often used that do not allow for interpretability and human behavior understanding. We present a new method to address the problems of personalized behavior classification and interoperability, and apply it to depression detection among college students. Inspired by the idea of collaborative-filtering, our method is a type of memory-based learning algorithm. It leverages the relevance of mobile-sensed behavior features among individuals to calculate personalized relevance weights, which are used to impute missing data and select features according to a specific modeling goal (e.g., whether the student has depressive symptoms) in different time epochs, i.e., times of the day and days of the week. It then compiles features from epochs using majority voting to obtain the final prediction. We apply our algorithm on a depression detection dataset collected from first-year college students with low data-missing rates and show that our method outperforms the state-of the-art machine learning model by 5.1% in accuracy and 5.5% in F1 score. We further verify the pipeline-level generalizability of our approach by achieving similar results on a second dataset, with an average improvement of 3.4% across performance metrics. Beyond achieving better classification performance, our novel approach is further able to generate personalized interpretations of the models for each individual. These interpretations are supported by existing depression-related literature and can potentially inspire automated and personalized depression intervention design in the future.

Distress and resilience among marginalized undergraduates

Nurius, P. S., Sefidgar, Y. S., Kuehn, K. S, Jake, X, Zhang, H., Browning, A., Riskin, E., Dey, A. K., & Mankoff, J.  Distress among undergraduates: Marginality, stressors and resilience supports. Journal of American College Health, 1-9.

Stress and related mental health struggles are of growing concern at colleges and universities across the country and internationally, with some evidence of levels higher than general population peers. The university experience can pose considerable strain on students, in some cases adding to early and current life stressors, and, if not mitigated, can lead to impaired well-being and academic success/retention.

This study provides a 2019 data snapshot of multiple stressor effects on early-stage students, resilience resources (or the lack thereof) that can mitigate these effects, and sociodemographic characteristics reflecting minoritized identities. Participants were 253 first- and second-year undergraduate students (age =18.76; 49.80% male, 69% students of color) enrolled at the University of Washington.

Multivariate analysis demonstrated significant associations between greater stress exposures and lower levels of resilience resources with each of three mental health indicators—perceived stress (intensity of experienced stress), depression, and anxiety. Stressors such as poor physical health, discrimination exposure, experiencing one or more marginalizing status (e.g., first generation student, having disabilities, sexual minority), and using maladaptive coping strategies (e.g, denial, self-blame) significantly accounted for each of the mental health indicators. Prior stressors such as adverse childhood experiences and other life and academic adversities were also significantly correlated with the mental health variables.

Race/ethnicity was less clearly patterned, although students of Asian descent reported significantly greater depression and anxiety, and females reported higher levels on all distress forms. In terms of resilience supports, those reporting greater social support and perception of oneself as a “bounce back” kind of person reported lesser psychological distress and these variables reduced the effects of stressors. Assessment of student well-being from this same project during the 2020 COVID-19 context indicated that students entering the pandemic with mental health vulnerabilities experienced significantly greater psychological distress and academic strain as the university pivoted toward remote instruction, signaling highly consequential differences (Morris et al., 2021)

These results support the value of “poly-strengths” –multiple forms of resilience- fostering resources–for mitigating the effects of stressors on psychological distress. College leaders are noting increases in the severity of students’ mental health concerns and demand for services, changing the roles of campus counseling centers, and requiring new institutional responses. Better understanding cumulative stress/resilience resource profiles, particularly among marginalized students and those experiencing discrimination, can help universities in prioritizing institutional support responses toward prevention, strengthening resilience, and mitigating psychological distress.

COVID-19 and Remote Learning for Students with Disabilities

Han Zhang, Margaret E. Morris, Paula S. Nurius, Kelly Mack, Jennifer Brown, Kevin S. Kuehn, Yasaman S. Sefidgar, Xuhai Xu, Eve A. Riskin, Anind K. Dey and Jennifer Mankoff. Impact of Online Learning in the Context of COVID-19 on Undergraduates with Disabilities and Mental Health Concerns. Transactions on Accessible Computing. Accepted, April 12th, 2022.

The COVID-19 pandemic upended college education and the experiences of students due to the rapid and uneven shift to online learning. This study examined the experiences of students with disabilities with online learning, with a consideration of surrounding stressors such as financial pressures. In a mixed method approach, we compared 28 undergraduate students with disabilities(including mental health concerns) to their peers during 2020, to assess differences and similarities in their educational concerns, stress levels and COVID-19 related adversities. We found that students with disabilities entered the Spring quarter of 2020 with significantly higher concerns about classes going online, and reported more recent negative life events than other students. These differences between the two groups diminished three months later with the exception of recent negative life events. For a fuller understanding of students’ experiences, we conducted qualitative analysis of open ended interviews. We examined both positive and negative experiences with online learning among students with disabilities and mental health concerns. Online learning enabled greater access – e.g., reducing the need for travel to campus–alongside ways in which online learning impeded academic engagement–e.g., reducing interpersonal interaction. Learning systems need to continue to meet the diverse and dynamic needs of students with disabilities.

Anticipate and Adjust

Kelly MackEmma McDonnellVenkatesh PotluriMaggie XuJailyn ZabalaJeffrey Bigham, Jennifer Mankoff, Cynthia L. Bennett:
Anticipate and Adjust: Cultivating Access in Human-Centered Methods. CHI 2022: 603:1-603:18 [pdf] [Plain Language Summary]

“Human-centered research” must make sure that research practices are accessible for both participants and researchers with disabilities. Yet, people rarely discuss how to make common methods accessible. We interviewed 17 accessibility experts who were researchers or community organizers about their practices. Our findings emphasize the importance of considering accessibility at all stages of the research process and across different dimensions of studies like communication, materials, time, and space. We explore how technology or processes could reflect a norm of accessibility and offer a practical structure for planning accessible research.

Maptimizer

Megan HofmannKelly MackJessica BirchfieldJerry CaoAutumn G. HughesShriya KurpadKathryn J. LumEmily WarnockAnat CaspiScott E. Hudson, Jennifer Mankoff:
Maptimizer: Using Optimization to Tailor Tactile Maps to Users Needs. CHI 2022: 592:1-592:15 [pdf]

Tactile maps can help people who are blind or have low vision navigate and familiarize themselves with unfamiliar locations. Ideally, tactile maps are created by considering an individual’s unique needs and abilities because of their limited space for representation. However, significant customization is not supported by existing tools for generating tactile maps. We present the Maptimizer system which generates tactile maps that are customized to a user’s preferences and requirements, while making simplified and easy to read tactile maps. Maptimizer uses a two stage optimization process to pair representations with geographic information and tune those representations to present that information more clearly. In a user study with six blind/low-vision participants, Maptimizer helped participants more successfully and efficiently identify locations of interest in unknown areas. These results demonstrate the utility of optimization techniques and generative design in complex accessibility domains that require significant customization by the end user.

A system diagram showing the maptimizer data flow setup. The inputs are geography sets, representations options, and user preferences. Geography types and representation options are paired and tuned using an optimizer. The output is a tactile map.

Computational Design of Knit Templates

We present an interactive design system for knitting that allows users to create template patterns that can be fabricated using an industrial knitting machine. Our interactive design tool is novel in that it allows direct control of key knitting design axes we have identified in our formative study and does so consistently across the variations of an input parametric template geometry. This is achieved with two key technical advances. First, we present an interactive meshing tool that lets users build a coarse quadrilateral mesh that adheres to their knit design guidelines. This solution ensures consistency across the parameter space for further customization over shape variations and avoids helices, promoting knittability. Second, we lift and formalize low-level machine knitting constraints to the level of this coarse quad mesh. This enables us to not only guarantee hand- and machine-knittability, but also provides automatic design assistance through auto-completion and suggestions. We show the capabilities through a set of fabricated examples that illustrate the effectiveness of our approach in creating a wide variety of objects and interactively exploring the space of design variations.

Benjamin JonesYuxuan MeiHaisen ZhaoTaylor Gotfrid, Jennifer Mankoff, Adriana Schulz:
Computational Design of Knit Templates. ACM Trans. Graph. 41(2): 16:1-16:16 (2022)

Four pink knit dresses mounted on four mannekins. each showing different styles of neckline and skirt. Behind each dress is the pattern used to create that dress. The shape of the quads in the pattern demonstrate their relationship to typical knitting patterns -- for example a collar knit in the round has quads that narrow as they go up.

Our interactive design system helps users explore key design axes for knitting to generate highly customized patterns from input shape templates; e.g., a seamless yoke dress with princess-cut apparent seams (a), and drop shoulder dresses with textures on the arms and skirt (b–d). The output of our system is a knit pattern template that lets users vary the shape while preserving the design, for example, creating a child’s dress with short sleeves (d) that matches an adult dress (b), or varying skirt texture and angle, and sleeve knitting direction (c). The system guarantees that all results and variations are machine knittable.

A diagram showing four differently shaped duck faces (a) which all have the same mesh, which can react easily to different shapes by adjusting quad shapes. The final product of a duck with a short, and a long, snout, is shown knitted in lavendar at the right.

Overview of our framework. (a) Triangle meshes from a parametric template (the system deals with a single mesh at a time). (b) Input triangle mesh with user annotations of composition, layout, and direction guidelines. (c) Generated quad mesh patches, which are consistent across template variations. (d) Quad mesh annotated for knitting the body tube in the round using short rows to curve the tube. Blue lines indicate seams. The same design applies to all template variations (two shown here). (e) Duck knit with short rows. (f ) Quad mesh annotated with different textures and orientations; the body is knit as seamed sheets with decreases. (g) Duck knit with textures and a large head from template (f ).

TypeOut: Just-in-Time Self-Affirmation for Reducing Phone Use

Smartphone overuse is related to a variety of issues such as lack of sleep and anxiety. We explore the application of Self-Affirmation Theory on smartphone overuse intervention in a just-in-time manner. We present TypeOut, a just-in-time intervention technique that integrates two components: an in-situ typing-based unlock process to improve user engagement, and self-affirmation-based typing content to enhance effectiveness. We hypothesize that the integration of typing and self-affirmation content can better reduce smartphone overuse. We conducted a 10-week within-subject field experiment (N=54) and compared TypeOut against two baselines: one only showing the self-affirmation content (a common notification-based intervention), and one only requiring typing non-semantic content (a state-of-the-art method). TypeOut reduces app usage by over 50%, and both app opening frequency and usage duration by over 25%, all significantly outperforming baselines. TypeOut can potentially be used in other domains where an intervention may benefit from integrating self-affirmation exercises with an engaging just-in-time mechanism.

Typeout: Leveraging just-in-time self-affirmation for smartphone overuse reduction. Xuhai Xu, Tianyuan Zou, Xiao Han, Yanzhang Li, Ruolin Wang, Tianyi Yuan, Yuntao Wang, Yuanchun Shi, Jennifer Mankoff,and Anind K. Dey. 2022. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22). ACM, New York, NY, USA.

Anticipate and Adjust: Cultivating Access in Human-Centered Methods

In order for “human-centered research” to include all humans, we need to make sure that research practices are accessible for both participants and researchers with disabilities. Yet, people rarely discuss how to make common methods accessible. We interviewed 17 accessibility experts who were researchers or community organizers about their practices. Our findings emphasize the importance of considering accessibility at all stages of the research process and across different dimensions of studies like communication, materials, time, and space. We explore how technology or processes could reflect a norm of accessibility and offer a practical structure for planning accessible research.

Anticipate and Adjust: Cultivating Access in Human-Centered Methods. Kelly Mac, Emma J. McDonnell, Venkatesh Potluri, Maggie Xu, Jailyn Zabala, Jeffrey P. Bigham, Jennifer Mankoff, and Cynthia L. Bennett. CHI 2022

Practices and Needs of Mobile Sensing Researchers

Passive mobile sensing for the purpose of human state modeling is a fast-growing area. It has been applied to solve a wide range of behavior-related problems, including physical and mental health monitoring, affective computing, activity recognition, routine modeling, etc. However, in spite of the emerging literature that has investigated a wide range of application scenarios, there is little work focusing on the lessons learned by researchers, and on guidance for researchers to this approach. How do researchers conduct these types of research studies? Is there any established common practice when applying mobile sensing across different application areas? What are the pain points and needs that they frequently encounter? Answering these questions is an important step in the maturing of this growing sub-field of ubiquitous computing, and can benefit a wide range of audiences. It can serve to educate researchers who have growing interests in this area but have little to no previous experience. Intermediate researchers may also find the results interesting and helpful for reference to improve their skills. Moreover, it can further shed light on the design guidelines for a future toolkit that could facilitate research processes being used. In this paper, we fill this gap and answer these questions by conducting semi-structured interviews with ten experienced researchers from four countries to understand their practices and pain points when conducting their research. Our results reveal a common pipeline that researchers have adopted, and identify major challenges that do not appear in published work but that researchers often encounter. Based on the results of our interviews, we discuss practical suggestions for novice researchers and high-level design principles for a toolkit that can accelerate passive mobile sensing research.

Understanding practices and needs of researchers in human state modeling by passive mobile sensing. Xu, Xuhai, Jennifer Mankoff, and Anind K. Dey. CCF Transactions on Pervasive Computing and Interaction (2021): 1-23.

College during COVID

Mental health of UW students during Spring 2020 varied tremendously: the challenges of online learning during the pandemic were entwined with social isolation, family demands and socioeconomic pressures. In this context, individual differences in coping mechanisms had a big impact. The findings of this paper underline the need for interventions oriented towards problem-focused coping and suggest opportunities for peer role modeling.

College from home during COVID-19: A mixed-methods study of heterogeneous experiences. Morris ME, Kuehn KS, Brown J, Nurius PS, Zhang H, Sefidgar YS, Xuhai X, Riskin EA, Dey A, Consolvo S, Mankoff JC. (2021) PLoS ONE 16(6): e0251580. (reported in UW News and the Hechtinger Report)

A lineplot showing anxiousness (Y axis, varying from 0 to 4) over time (X axis). Each student in the study is plotted as a different line over each day of the quarter. The plot overall looks very messy, but two things are clear; Every student has a very different trajectory from every other, with all of them going up and down multiple times. And the average, overall, shown is a fit line, is fairly low and slightly increasing (from about .75 to just under 1).
Heterogeneity in individuals’ levels of anxiety (reported in ESM). Individual trajectories of anxiety are shown in different line types and colors (dotted versus solid lines represent different participants). Although the mean level of anxiety is 1 on a scale of 0–4, the significant variation in responses invites examination of individuals and subgroups.

This mixed-method study examined the experiences of college students during the COVID-19 pandemic through surveys, experience sampling data collected over two academic quarters (Spring 2019 n1 = 253; Spring 2020 n2 = 147), and semi-structured interviews with 27 undergraduate students. 

There were no marked changes in mean levels of depressive symptoms, anxiety, stress, or loneliness between 2019 and 2020, or over the course of the Spring 2020 term. Students in both the 2019 and 2020 cohort who indicated psychosocial vulnerability at the initial assessment showed worse psychosocial functioning throughout the entire Spring term relative to other students. However, rates of distress increased faster in 2020 than in 2019 for these individuals. Across individuals, homogeneity of variance tests and multi-level models revealed significant heterogeneity, suggesting the need to examine not just means but the variations in individuals’ experiences. 

Thematic analysis of interviews characterizes these varied experiences, describing the contexts for students’ challenges and strategies. This analysis highlights the interweaving of psychosocial and academic distress: Challenges such as isolation from peers, lack of interactivity with instructors, and difficulty adjusting to family needs had both an emotional and academic toll. Strategies for adjusting to this new context included initiating remote study and hangout sessions with peers, as well as self-learning. In these and other strategies, students used technologies in different ways and for different purposes than they had previously. Supporting qualitative insight about adaptive responses were quantitative findings that students who used more problem-focused forms of coping reported fewer mental health symptoms over the course of the pandemic, even though they perceived their stress as more severe. 

Example quotes:

I like to build things and stuff like that. I like to see it in person and feel it. So the fact that everything was online…. I’m just basically reading all the time. I just couldn’t learn that way

Insomnia has been pretty hard for me . . .  I would spend a lot of time lying in bed not doing anything when I had a lot of homework to do the next day. So then I would become stressed about whether I’ll be able to finish that homework or not.”

“It was challenging … being independent and then being pushed back home. It’s a huge change because now you have more rules again”

For a few of my classes I feel like actually [I] was self-learning because sometimes it’s hard to sit through hours of lectures and watch it.”

I would initiate… we have a study group chat and every day I would be like ‘Hey I’m going to be on at this time starting at this time.’ So then I gave them time to all have the room open for Zoom and stuff. Okay and then any time after that they can join and then said I [would] wait like maybe 30 minutes or even an hour…. And then people join and then we work maybe … till midnight, a little bit past midnight