PROJECT 1

Federated Learning for Human-centered Experience and Perception Modeling with Bio-behavioral Data

Project Leader: Professor Shrikanth Narayanan

Website: https://sail.usc.edu/people/shri.html

Abstract: Human bio-behavior sensing and inference technologies such as for predicting physical and psychological states, alongside individual details related to human identity and other demographic variables, pose potential privacy risks in data acquisition, storage, modeling and learning. The proposed project aims at developing federated learning (FL) methods that can handle the multifaceted information contained in human bio-behavior data in seeking critical application specific details (e.g., health status, stress level) while at the same time addressing privacy needs (e.g., about protected variables such as gender/race/age; location). The proposed framework in particular is cognizant of the diversity and subjectivity inherent in the generation and processing of human of bio-behavior data.The human-centric tasks of interest in this work can be categorized based on how reference/ground truth for modeling is obtained: Experience modeling, where reference is from self (expressed) or perception modeling where judgments are provided by external observers; both these tasks need to contend with inherent subjectivity. We propose a framework to incorporate FL in addition to modeling the specific type of label heterogeneity these tasks come with. In an FL setup for human experience modeling, the goal of the server model would be that of a regularizer figure, right), capturing common modes of variability across different clients so that a new client can personalize as and when new data come in (the client models are what we want for deployment). In modeling perception (figure, left), we aim to learn across the different views, that is have a model that captures as much information as possible and can serve as a general model; at the same time one that does not overfit to individual models on the client side (the server model is the one we want to learn/ deploy). The proposed models will be evaluated on two unique real-world human bio-behavioral wearable sensor datasets.