Utility–Privacy Trade-Off in Distributed Machine Learning Systems
In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mec...
Published in: | Entropy |
---|---|
Main Authors: | , , |
Format: | Article in Journal/Newspaper |
Language: | English |
Published: |
MDPI AG
2022
|
Subjects: | |
Online Access: | https://doi.org/10.3390/e24091299 https://doaj.org/article/891ae7052f0d4de5a69bf460644127dc |
id |
ftdoajarticles:oai:doaj.org/article:891ae7052f0d4de5a69bf460644127dc |
---|---|
record_format |
openpolar |
spelling |
ftdoajarticles:oai:doaj.org/article:891ae7052f0d4de5a69bf460644127dc 2023-05-15T16:01:37+02:00 Utility–Privacy Trade-Off in Distributed Machine Learning Systems Xia Zeng Chuanchuan Yang Bin Dai 2022-09-01T00:00:00Z https://doi.org/10.3390/e24091299 https://doaj.org/article/891ae7052f0d4de5a69bf460644127dc EN eng MDPI AG https://www.mdpi.com/1099-4300/24/9/1299 https://doaj.org/toc/1099-4300 doi:10.3390/e24091299 1099-4300 https://doaj.org/article/891ae7052f0d4de5a69bf460644127dc Entropy, Vol 24, Iss 1299, p 1299 (2022) differential privacy distributed machine learning mutual information Gaussian noise trade-off Science Q Astrophysics QB460-466 Physics QC1-999 article 2022 ftdoajarticles https://doi.org/10.3390/e24091299 2022-12-30T22:01:45Z In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mechanism to protect the clients’ local parameters. In this paper, from an information-theoretic point of view, we study the utility–privacy trade-off in DML with the help of the DP mechanism. Specifically, three cases including independent clients’ local parameters with independent DP noise, dependent clients’ local parameters with independent/dependent DP noise are considered. Mutual information and conditional mutual information are used to characterize utility and privacy, respectively. First, we show the relationship between utility and privacy for the three cases. Then, we show the optimal noise variance that achieves the maximal utility under a certain level of privacy. Finally, the results of this paper are further illustrated by numerical results. Article in Journal/Newspaper DML Directory of Open Access Journals: DOAJ Articles Entropy 24 9 1299 |
institution |
Open Polar |
collection |
Directory of Open Access Journals: DOAJ Articles |
op_collection_id |
ftdoajarticles |
language |
English |
topic |
differential privacy distributed machine learning mutual information Gaussian noise trade-off Science Q Astrophysics QB460-466 Physics QC1-999 |
spellingShingle |
differential privacy distributed machine learning mutual information Gaussian noise trade-off Science Q Astrophysics QB460-466 Physics QC1-999 Xia Zeng Chuanchuan Yang Bin Dai Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
topic_facet |
differential privacy distributed machine learning mutual information Gaussian noise trade-off Science Q Astrophysics QB460-466 Physics QC1-999 |
description |
In distributed machine learning (DML), though clients’ data are not directly transmitted to the server for model training, attackers can obtain the sensitive information of clients by analyzing the local gradient parameters uploaded by clients. For this case, we use the differential privacy (DP) mechanism to protect the clients’ local parameters. In this paper, from an information-theoretic point of view, we study the utility–privacy trade-off in DML with the help of the DP mechanism. Specifically, three cases including independent clients’ local parameters with independent DP noise, dependent clients’ local parameters with independent/dependent DP noise are considered. Mutual information and conditional mutual information are used to characterize utility and privacy, respectively. First, we show the relationship between utility and privacy for the three cases. Then, we show the optimal noise variance that achieves the maximal utility under a certain level of privacy. Finally, the results of this paper are further illustrated by numerical results. |
format |
Article in Journal/Newspaper |
author |
Xia Zeng Chuanchuan Yang Bin Dai |
author_facet |
Xia Zeng Chuanchuan Yang Bin Dai |
author_sort |
Xia Zeng |
title |
Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
title_short |
Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
title_full |
Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
title_fullStr |
Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
title_full_unstemmed |
Utility–Privacy Trade-Off in Distributed Machine Learning Systems |
title_sort |
utility–privacy trade-off in distributed machine learning systems |
publisher |
MDPI AG |
publishDate |
2022 |
url |
https://doi.org/10.3390/e24091299 https://doaj.org/article/891ae7052f0d4de5a69bf460644127dc |
genre |
DML |
genre_facet |
DML |
op_source |
Entropy, Vol 24, Iss 1299, p 1299 (2022) |
op_relation |
https://www.mdpi.com/1099-4300/24/9/1299 https://doaj.org/toc/1099-4300 doi:10.3390/e24091299 1099-4300 https://doaj.org/article/891ae7052f0d4de5a69bf460644127dc |
op_doi |
https://doi.org/10.3390/e24091299 |
container_title |
Entropy |
container_volume |
24 |
container_issue |
9 |
container_start_page |
1299 |
_version_ |
1766397401606651904 |