Can I read Entropic characterization of optimal rates for learning Gaussian mixtures on EtoBox?

Entropic characterization of optimal rates for learning Gaussian mixtures by Jia, Zeyu; Polyanskiy, Yury; Wu, Yihong is a scholarly article available to read on EtoBox.

What is Entropic characterization of optimal rates for learning Gaussian mixtures about?

We consider the question of estimating multi-dimensional Gaussian mixtures (GM) with compactly supported or subgaussian mixing distributions. Minimax estimation rate for this class (under Hellinger, TV and KL divergences) is a long-standing open question, even in one dimension. In this paper we characterize this rate (for all constant dimensions) in terms of the metric entropy of the class. Such characterizations originate from seminal works of Le Cam (1973); Birge (1983); Haussler and Opper (1997); Yang and Barron (1999). However, for GMs a key ingredient missing from earlier work (and widely sought-after) is a comparison result showing that the KL and the squared Hellinger distance are within a constant multiple of each other uniformly over the class. Our main technical contribution is in showing this fact, from which we derive entropy characterization for estimation rate under Hellinger and KL. Interestingly, the sequential (online learning) estimation rate is characterized by the global entropy, while the single-step (batch) rate corresponds to local entropy, paralleling a similar result for the Gaussian sequence model recently discovered by Neykov (2022) and Mourtada (2023). A

Author

Jia, Zeyu; Polyanskiy, Yury; Wu, Yihong

Published

2023

Language

EN