Can I read $\beta$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search on EtoBox?

$\beta$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search by Ye, Peng; He, Tong; Li, Baopu; Chen, Tao; Bai, Lei; Ouyang, Wanli is a scholarly article available to read on EtoBox.

What is $\beta$-DARTS++: Bi-level Regularization for Proxy-robust Differentiable Architecture Search about?

Neural Architecture Search has attracted increasing attention in recent years. Among them, differential NAS approaches such as DARTS, have gained popularity for the search efficiency. However, they still suffer from three main issues, that are, the weak stability due to the performance collapse, the poor generalization ability of the searched architectures, and the inferior robustness to different kinds of proxies. To solve the stability and generalization problems, a simple-but-effective regularization method, termed as Beta-Decay, is proposed to regularize the DARTS-based NAS searching process (i.e., $\beta$-DARTS). Specifically, Beta-Decay regularization can impose constraints to keep the value and variance of activated architecture parameters from being too large, thereby ensuring fair competition among architecture parameters and making the supernet less sensitive to the impact of input on the operation set. In-depth theoretical analyses on how it works and why it works are provided. Comprehensive experiments validate that Beta-Decay regularization can help to stabilize the searching process and makes the searched network more transferable across different datasets. To address

Author

Ye, Peng; He, Tong; Li, Baopu; Chen, Tao; Bai, Lei; Ouyang, Wanli

Published

2023

Language

EN