Jingping Nie

Despite the global mental health crisis, access to screenings, professionals, and treatments remains high. In collaboration with licensed psychotherapists, we propose a C onversational AI T herapist with psychotherapeutic I nterventions (CaiTI), a platform that leverages large language models (LLMs) and smart devices to enable better mental health self-care. CaiTI can screen the day-to-day functioning using natural and psychotherapeutic conversations. CaiTI leverages reinforcement learning to provide personalized conversation flow. CaiTI can accurately understand and interpret user responses. When the user needs further attention during the conversation, CaiTI can provide conversational psychotherapeutic interventions, including cognitive behavioral therapy and motivational interviewing. Leveraging the datasets prepared by the licensed psychotherapists, we experiment and microbenchmark various LLMs’ performance in tasks along CaiTI’s conversation flow and discuss their strengths and weaknesses. With the psychotherapists, we implement CaiTI and conduct 14-day and 24-week studies. The study results, validated by therapists, demonstrate that CaiTI can converse with users naturally, accurately understand and interpret user responses, and provide psychotherapeutic interventions appropriately and effectively. We showcase the potential of CaiTI LLMs to assist the mental therapy diagnosis and treatment and improve day-to-day functioning screening and precautionary psychotherapeutic intervention systems.

Mental healthcare is increasingly shaped by the convergence of ubiquitous sensing technologies and large language models (LLMs), enabling smart systems that extend screening and psychotherapeutic support beyond traditional care settings. While prior surveys examine sensing-based technologies or LLM-driven conversational agents in isolation, there remains a lack of clinically aligned, cross-modality synthesis that situates these approaches side by side across mental health screening and intervention contexts. This survey systematically organizes Sensor-based, LLM-based, and hybrid Sensor & LLM approaches into a common taxonomy grounded in clinical functions and pairs this structure with a comprehensive set of evaluation metrics spanning across three complementary axes, including system performance and technical reliability, usability, and clinical effectiveness. We further annotate these systems by device form factors, signal modalities, interaction modalities, psychotherapeutic endpoints, and pipeline design, and incorporate continuous input from licensed psychotherapists to ensure alignment with clinical norms. Finally, the perspective of psychotherapists outlines a research agenda for trustworthy and equitable autonomous mental health systems that integrate seamlessly with human clinicians, emphasizing transparent, multimodal evaluation and responsible real-world deployment, enabling seamless integration with human clinicians to augment rather than replace human care.

Running metrics like cadence and ground contact time (GCT) are crucial for both novice and experienced runners to optimize performance and prevent injuries. We present SoundTrack, a contactless mobile solution that estimates these metrics by analyzing treadmill running sounds using on-device machine learning. Our main contributions are: (i) SoundTrackDB - a comprehensive 40-hour dataset of treadmill running sounds collected from 61 subjects across 363 sessions in 13 public gyms, created in collaboration with a licensed running coach; and (ii) SoundTrack - an on-device mobile system capturing treadmill running sounds, mitigating noise, estimating cadence and GCT with a custom multi-layer perceptron (MLP) model, and providing real-time feedback. Microbenchmarks and evaluations show that SoundTrack effectively mitigates real-world noise challenges in public gyms and adapts to individual variations among runners and treadmill models. It achieves mean absolute percentage errors (MAPEs) of 1.62% for cadence and 6.05% for GCT on the test set of unseen running sessions, yielding results that are superior or comparable to commercial sports wearables. SoundTrack offers an accessible solution for treadmill metrics on mobile platforms, reducing reliance on specialized wearables and broadening accessibility. SoundTrackDB, SoundTrack, and the demonstration video are available at: https://github.com/Columbia-ICSL/SoundTrackDB.

Auscultation, particularly heart sound, is a non-invasive technique that provides essential vital sign information. Recently, self-supervised acoustic representation foundation models (FMs) have been proposed to offer insights into acoustics-based vital signs. However, there has been little exploration of the extent to which auscultation is encoded in these pre-trained FM representations. In this work, using a publicly available phonocardiogram (PCG) dataset and a heart rate (HR) estimation model, we conduct a layer-wise investigation of six acoustic representation FMs: HuBERT, wav2vec2, wavLM, Whisper, Contrastive Language-Audio Pretraining (CLAP), and an in-house CLAP model. Additionally, we implement the baseline method from Nie et al., 2024 (which relies on acoustic features) and show that overall, representation vectors from pre-trained foundation models (FMs) offer comparable performance to the baseline. Notably, HR estimation using the representations from the audio encoder of the in-house CLAP model outperforms the results obtained from the baseline, achieving a lower mean absolute error (MAE) across various train/validation/test splits despite the domain mismatch.