Can I read Proactive DP: A Multple Target Optimization Framework for DP-SGD on EtoBox?

Proactive DP: A Multple Target Optimization Framework for DP-SGD by van Dijk, Marten; Nguyen, Nhuong V.; Nguyen, Toan N.; Nguyen, Lam M.; Nguyen, Phuong Ha is a scholarly article available to read on EtoBox.

What is Proactive DP: A Multple Target Optimization Framework for DP-SGD about?

We introduce a multiple target optimization framework for DP-SGD referred to as pro-active DP. In contrast to traditional DP accountants, which are used to track the expenditure of privacy budgets, the pro-active DP scheme allows one to a-priori select parameters of DP-SGD based on a fixed privacy budget (in terms of $\epsilon$ and $\delta$) in such a way to optimize the anticipated utility (test accuracy) the most. To achieve this objective, we first propose significant improvements to the moment account method, presenting a closed-form $(\epsilon,\delta)$-DP guarantee that connects all parameters in the DP-SGD setup. We show that DP-SGD is $(\epsilon<0.5,\delta=1/N)$-DP if $\sigma=\sqrt{2(\epsilon +\ln(1/\delta))/\epsilon}$ with $T$ at least $\approx 2k^2/\epsilon$ and $(2/e)^2k^2-1/2\geq \ln(N)$, where $T$ is the total number of rounds, and $K=kN$ is the total number of gradient computations where $k$ measures $K$ in number of epochs of size $N$ of the local data set. We prove that our expression is close to tight in that if $T$ is more than a constant factor $\approx 4$ smaller than the lower bound $\approx 2k^2/\epsilon$, then the $(\epsilon,\delta)$-DP guarantee is violated.

Author

van Dijk, Marten; Nguyen, Nhuong V.; Nguyen, Toan N.; Nguyen, Lam M.; Nguyen, Phuong Ha

Published

2021

Language

EN