SIGSEC talk: 23 Nov - Privacy Side Channels in Machine Learning Systems
22 views
Skip to first unread message
ACL SIGSEC
Nov 21, 2023, 3:30:40 PM11/21/23
to ACL SIGSEC
Privacy Side Channels in Machine Learning Systems
2023, November 23rd, 11.00 ET / 17.00 CET (30 mins + questions)
Edoardo Debenedetti
Most current approaches for protecting privacy in machine learning (ML) assume that models exist in a vacuum, when in reality, ML models are part of larger systems that include components for training data filtering, output monitoring, and more. In this work, we introduce privacy side channels: attacks that exploit these system-level components to extract private information at far higher rates than is otherwise possible for standalone models. We propose four categories of side channels that span the entire ML lifecycle (training data filtering, input preprocessing, output post-processing, and query filtering) and allow for either enhanced membership inference attacks or even novel threats such as extracting users’ test queries. For example, we show that deduplicating training data before applying differentially-private training creates a side-channel that completely invalidates any provable privacy guarantees. Moreover, we show that systems which block language models from regenerating training data can be exploited to allow exact reconstruction of private keys contained in the training set – even if the model did not memorize these keys. Taken together, our results demonstrate the need for a holistic, end-to-end privacy analysis of machine learning.
Zoom link: https://itucph.zoom.us/j/3319000227
arXiv: https://arxiv.org/abs/2309.05610
Calendar link at https://sig.llmsecurity.net/talks/
See you there!!
2023, November 23rd, 11.00 ET / 17.00 CET (30 mins + questions)
Edoardo Debenedetti
Most current approaches for protecting privacy in machine learning (ML) assume that models exist in a vacuum, when in reality, ML models are part of larger systems that include components for training data filtering, output monitoring, and more. In this work, we introduce privacy side channels: attacks that exploit these system-level components to extract private information at far higher rates than is otherwise possible for standalone models. We propose four categories of side channels that span the entire ML lifecycle (training data filtering, input preprocessing, output post-processing, and query filtering) and allow for either enhanced membership inference attacks or even novel threats such as extracting users’ test queries. For example, we show that deduplicating training data before applying differentially-private training creates a side-channel that completely invalidates any provable privacy guarantees. Moreover, we show that systems which block language models from regenerating training data can be exploited to allow exact reconstruction of private keys contained in the training set – even if the model did not memorize these keys. Taken together, our results demonstrate the need for a holistic, end-to-end privacy analysis of machine learning.
Zoom link: https://itucph.zoom.us/j/3319000227
arXiv: https://arxiv.org/abs/2309.05610
Calendar link at https://sig.llmsecurity.net/talks/
See you there!!
Reply all
Reply to author
Forward
0 new messages