Can I read Spectral Regularization in Deep MoE Routing on EtoBox?

Spectral Regularization in Deep MoE Routing by scp6 is a document available to read on EtoBox.

What is Spectral Regularization in Deep MoE Routing about?

The paper introduces Spectrally-Regularized Mixture of Experts (SR-MoE), a framework designed to enhance modularity and stability in deep neural networks by applying geometric constraints on the routing manifold. By implementing a dual-objective penalty that regulates the spectral norm and stable rank, SR-MoE effectively prevents expert collapse and catastrophic interference during adaptation. Experimental results demonstrate that SR-MoE outperforms traditional linear gating mechanisms, maintaining structur

Author

scp6

Language

EN