Can I read Compact Transformers for Small Datasets on EtoBox?

Compact Transformers for Small Datasets by opekkhasu is a document available to read on EtoBox.

What is Compact Transformers for Small Datasets about?

1) The document introduces Compact Transformers, a new approach for small-scale learning using transformers that can avoid overfitting and outperform CNNs on small datasets. 2) Compact Transformers use convolutional tokenization and are much smaller in size than previous transformer models, with the best model achieving competitive results on CIFAR-10 with only 3.7M parameters, over 10x smaller than other transformers. 3) The simple and compact design of Compact Transformers makes them well-suited for app

Author

opekkhasu

Language

EN