Stats, Optimization, and Machine Learning Seminar - Sidney D'Mello
Project Tesserae: Longitudinal Multimodal Modeling of Individuals in Naturalistic Contexts
I will describe our team鈥檚 efforts on a two-year Intelligence Advanced Research Projects Activity (IARPA) program called MOSAIC - Multimodal Objective Sensing to Assess Individuals with Context. The program鈥檚 ambitious aims are to 鈥渁dvance multimodal sensing to measure personnel and their environment unobtrusively, passively, and persistently both at work and outside of work, reduce the time and manpower required to process and integrate such data, and construct personalized and adaptive assessments of an individual that are accurate throughout the individual鈥檚 career.鈥
The premise of our team (called Project Tesserae) was to fuse information from low-cost mobile devices which individuals already use in their daily lives with accurate and robust machine learning techniques to develop generalizable models of psychological, health, and job performance measures. Towards this end, we conducted a year-long study of over 750 working professionals from across the US to explore the extent to which wearables, smartphones, Bluetooth beacons, social media, and other sensing streams can offer insights into individuals embedded in their social contexts. In this talk, I will present an overview of the Project Tesserae study design, offer insight on lessons learned, share experiences on various sensing technologies, and describe several early insights gained.