Can I read Bias in BERT: Scale and Data Effects on EtoBox?

Bias in BERT: Scale and Data Effects by Swastika bera is a document available to read on EtoBox.

What is Bias in BERT: Scale and Data Effects about?

This study investigates the relationship between model scale, pre-training data, and social biases in language models, specifically focusing on BERT. It finds that while larger models generally exhibit increased upstream biases, downstream biases tend to decrease with scale, highlighting the significant role of pre-training data in shaping these biases. The research underscores the complexity of bias dynamics in language models and emphasizes the need for careful consideration of training data in bias mitig

Author

Swastika bera

Language

EN