Archives

Cross-Dataset Generalization for Human Behavior Modeling

Leave a comment

Overview; Data; Code

Overview of The Contributions of This Work. We systematically evaluate cross-dataset generalizability of 19 algorithms: 9 prior behavior modeling algorithm for depression detection, 8 recent domain generalization algorithms, and 2 two new algorithms proposed in this paper. Our open-source platform GLOBEM consolidates these 19 algorithms and support using, developing, evaluating various algorithms.

There is a growing body of research revealing that longitudinal passive sensing data from smartphones and wearable devices can capture daily behavior signals for human behavior modeling, such as depression detection. Most prior studies build and evaluate machine learning models using data collected from a single population. However, to ensure that a behavior model can work for a larger group of users, its generalizability needs to be verified on multiple datasets from different populations. We present the first work evaluating cross-dataset generalizability of longitudinal behavior models, using depression detection as an application. We collect multiple longitudinal passive mobile sensing datasets with over 500 users from two institutes over a two-year span, leading to four institute-year datasets. Using the datasets, we closely re-implement and evaluated nine prior depression detection algorithms. Our experiment reveals the lack of model generalizability of these methods. We also implement eight recently popular domain generalization algorithms from the machine learning community. Our results indicate that these methods also do not generalize well on our datasets, with barely any advantage over the naive baseline of guessing the majority. We then present two new algorithms with better generalizability. Our new algorithm, Reorder, significantly and consistently outperforms existing methods on most cross-dataset generalization setups. However, the overall advantage is incremental and still has great room for improvement. Our analysis reveals that the individual differences (both within and between populations) may play the most important role in the cross-dataset generalization challenge. Finally, we provide an open-source benchmark platform GLOBEM – short for Generalization of LOngitudinal BEhavior Modeling – to consolidate all 19 algorithms. GLOBEM can support researchers in using, developing, and evaluating different longitudinal behavior modeling methods. We call for researchers’ attention to model generalizability evaluation for future longitudinal human behavior modeling studies.

Xuhai Xu, Xin Liu, Han Zhang, Weichen Wang, Subigya Nepal, Yasaman S. Sefidgar, Woosuk Seo, Kevin S. Kuehn, Jeremy F. Huckins, Margaret E. Morris, Paula S. Nurius, Eve A. Riskin, Shwetak N. Patel, Tim Althoff, Andrew Campbell, Anind K. Dey, and Jennifer Mankoff. GlOBEM: Cross-Dataset Generalization of Longitudinal Human Behavior Modeling. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6(4): 190:1-190:34 (2022).

Xuhai Xu, Han Zhang, Yasaman S. Sefidgar, Yiyi Ren, Xin Liu, Woosuk Seo, Jennifer Brown, Kevin S. Kuehn, Mike A. Merrill, Paula S. Nurius, Shwetak N. Patel, Tim Althoff, Margaret Morris, Eve A. Riskin, Jennifer Mankoff, Anind K. Dey:
GLOBEM Dataset: Multi-Year Datasets for Longitudinal Human Behavior Modeling Generalization. NeurIPS 2022

Distress and resilience among marginalized undergraduates

Leave a comment

Nurius, P. S., Sefidgar, Y. S., Kuehn, K. S, Jake, X, Zhang, H., Browning, A., Riskin, E., Dey, A. K., & Mankoff, J.  Distress among undergraduates: Marginality, stressors and resilience supports. Journal of American College Health, 1-9.

Stress and related mental health struggles are of growing concern at colleges and universities across the country and internationally, with some evidence of levels higher than general population peers. The university experience can pose considerable strain on students, in some cases adding to early and current life stressors, and, if not mitigated, can lead to impaired well-being and academic success/retention.

This study provides a 2019 data snapshot of multiple stressor effects on early-stage students, resilience resources (or the lack thereof) that can mitigate these effects, and sociodemographic characteristics reflecting minoritized identities. Participants were 253 first- and second-year undergraduate students (age =18.76; 49.80% male, 69% students of color) enrolled at the University of Washington.

Multivariate analysis demonstrated significant associations between greater stress exposures and lower levels of resilience resources with each of three mental health indicators—perceived stress (intensity of experienced stress), depression, and anxiety. Stressors such as poor physical health, discrimination exposure, experiencing one or more marginalizing status (e.g., first generation student, having disabilities, sexual minority), and using maladaptive coping strategies (e.g, denial, self-blame) significantly accounted for each of the mental health indicators. Prior stressors such as adverse childhood experiences and other life and academic adversities were also significantly correlated with the mental health variables.

Race/ethnicity was less clearly patterned, although students of Asian descent reported significantly greater depression and anxiety, and females reported higher levels on all distress forms. In terms of resilience supports, those reporting greater social support and perception of oneself as a “bounce back” kind of person reported lesser psychological distress and these variables reduced the effects of stressors. Assessment of student well-being from this same project during the 2020 COVID-19 context indicated that students entering the pandemic with mental health vulnerabilities experienced significantly greater psychological distress and academic strain as the university pivoted toward remote instruction, signaling highly consequential differences (Morris et al., 2021)

These results support the value of “poly-strengths” –multiple forms of resilience- fostering resources–for mitigating the effects of stressors on psychological distress. College leaders are noting increases in the severity of students’ mental health concerns and demand for services, changing the roles of campus counseling centers, and requiring new institutional responses. Better understanding cumulative stress/resilience resource profiles, particularly among marginalized students and those experiencing discrimination, can help universities in prioritizing institutional support responses toward prevention, strengthening resilience, and mitigating psychological distress.

COVID-19 and Remote Learning for Students with Disabilities

Leave a comment

Han Zhang, Margaret E. Morris, Paula S. Nurius, Kelly Mack, Jennifer Brown, Kevin S. Kuehn, Yasaman S. Sefidgar, Xuhai Xu, Eve A. Riskin, Anind K. Dey and Jennifer Mankoff. Impact of Online Learning in the Context of COVID-19 on Undergraduates with Disabilities and Mental Health Concerns. ACM Transactions on Accessible Computing (TACCESS).

The COVID-19 pandemic upended college education and the experiences of students due to the rapid and uneven shift to online learning. This study examined the experiences of students with disabilities with online learning, with a consideration of surrounding stressors such as financial pressures. In a mixed method approach, we compared 28 undergraduate students with disabilities(including mental health concerns) to their peers during 2020, to assess differences and similarities in their educational concerns, stress levels and COVID-19 related adversities. We found that students with disabilities entered the Spring quarter of 2020 with significantly higher concerns about classes going online, and reported more recent negative life events than other students. These differences between the two groups diminished three months later with the exception of recent negative life events. For a fuller understanding of students’ experiences, we conducted qualitative analysis of open ended interviews. We examined both positive and negative experiences with online learning among students with disabilities and mental health concerns. Online learning enabled greater access – e.g., reducing the need for travel to campus–alongside ways in which online learning impeded academic engagement–e.g., reducing interpersonal interaction. Learning systems need to continue to meet the diverse and dynamic needs of students with disabilities.

College during COVID

Leave a comment

Mental health of UW students during Spring 2020 varied tremendously: the challenges of online learning during the pandemic were entwined with social isolation, family demands and socioeconomic pressures. In this context, individual differences in coping mechanisms had a big impact. The findings of this paper underline the need for interventions oriented towards problem-focused coping and suggest opportunities for peer role modeling.

College from home during COVID-19: A mixed-methods study of heterogeneous experiences. Morris ME, Kuehn KS, Brown J, Nurius PS, Zhang H, Sefidgar YS, Xuhai X, Riskin EA, Dey A, Consolvo S, Mankoff JC. (2021) PLoS ONE 16(6): e0251580. (reported in UW News and the Hechtinger Report)

A lineplot showing anxiousness (Y axis, varying from 0 to 4) over time (X axis). Each student in the study is plotted as a different line over each day of the quarter. The plot overall looks very messy, but two things are clear; Every student has a very different trajectory from every other, with all of them going up and down multiple times. And the average, overall, shown is a fit line, is fairly low and slightly increasing (from about .75 to just under 1).
Heterogeneity in individuals’ levels of anxiety (reported in ESM). Individual trajectories of anxiety are shown in different line types and colors (dotted versus solid lines represent different participants). Although the mean level of anxiety is 1 on a scale of 0–4, the significant variation in responses invites examination of individuals and subgroups.

This mixed-method study examined the experiences of college students during the COVID-19 pandemic through surveys, experience sampling data collected over two academic quarters (Spring 2019 n1 = 253; Spring 2020 n2 = 147), and semi-structured interviews with 27 undergraduate students. 

There were no marked changes in mean levels of depressive symptoms, anxiety, stress, or loneliness between 2019 and 2020, or over the course of the Spring 2020 term. Students in both the 2019 and 2020 cohort who indicated psychosocial vulnerability at the initial assessment showed worse psychosocial functioning throughout the entire Spring term relative to other students. However, rates of distress increased faster in 2020 than in 2019 for these individuals. Across individuals, homogeneity of variance tests and multi-level models revealed significant heterogeneity, suggesting the need to examine not just means but the variations in individuals’ experiences. 

Thematic analysis of interviews characterizes these varied experiences, describing the contexts for students’ challenges and strategies. This analysis highlights the interweaving of psychosocial and academic distress: Challenges such as isolation from peers, lack of interactivity with instructors, and difficulty adjusting to family needs had both an emotional and academic toll. Strategies for adjusting to this new context included initiating remote study and hangout sessions with peers, as well as self-learning. In these and other strategies, students used technologies in different ways and for different purposes than they had previously. Supporting qualitative insight about adaptive responses were quantitative findings that students who used more problem-focused forms of coping reported fewer mental health symptoms over the course of the pandemic, even though they perceived their stress as more severe. 

Example quotes:

“I like to build things and stuff like that. I like to see it in person and feel it. So the fact that everything was online…. I’m just basically reading all the time. I just couldn’t learn that way”

Insomnia has been pretty hard for me . . .  I would spend a lot of time lying in bed not doing anything when I had a lot of homework to do the next day. So then I would become stressed about whether I’ll be able to finish that homework or not.”

“It was challenging … being independent and then being pushed back home. It’s a huge change because now you have more rules again”

“For a few of my classes I feel like actually [I] was self-learning because sometimes it’s hard to sit through hours of lectures and watch it.”

“I would initiate… we have a study group chat and every day I would be like ‘Hey I’m going to be on at this time starting at this time.’ So then I gave them time to all have the room open for Zoom and stuff. Okay and then any time after that they can join and then said I [would] wait like maybe 30 minutes or even an hour…. And then people join and then we work maybe … till midnight, a little bit past midnight”