PROJECT 1

Private Labeling and Learning for voice assistants with Cameras

Project Leader: Konstantinos Psounis, Professor Electrical Engineering and Computer Science

Website: https://sites.usc.edu/kpsounis/

Abstract: Voice assistants are found in the majority of US households already and their penetration is forecasted to increase further. Advanced voice assistants are equipped with cameras to improve user experience. Privacy and security challenges are exacerbated in the presence of cameras recording everyday life inside people’s home, prompting major voice assistant service providers to withhold sharing video feeds and even video metadata with cloud servers. Motivated on one hand by the potential benefits of camera data to user experience and on the other hand the substantial privacy concerns from sharing such data or metadata with human labelers and/or servers, we propose to architect a system that allows private labeling and learning for video data in the context of voice assistants. To achieve meaningfully privacy user videos cannot be shared, including for labeling purposes. Instead, we propose to use synthetic videos, which may be based on user videos, for labeling purposes. To further strengthen privacy, we also propose a system where neither user videos nor any video embeddings leave the user device. We also propose to use federated learning in combination with generative adversarial networks to improve the accuracy and privacy of labeling and learning by creating synthetic videos which achieve a good privacy-utility tradeoff. (note that this is the executive summary of the proposal since it does not mention Amazon or any Amazon products or any other companies)