Can I read Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit on EtoBox?

Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit by Lee, Kin Long Kelvin; Gonzales, Carmelo; Spellings, Matthew; Galkin, Mikhail; Miret, Santiago; Kumar, Nalini is a scholarly article available to read on EtoBox.

What is Towards Foundation Models for Materials Science: The Open MatSci ML Toolkit about?

Artificial intelligence and machine learning have shown great promise in their ability to accelerate novel materials discovery. As researchers and domain scientists seek to unify and consolidate chemical knowledge, the case for models with potential to generalize across different tasks within materials science - so-called "foundation models" - grows with ambitions. This manuscript reviews our recent progress with development of Open MatSci ML Toolkit, and details experiments that lay the groundwork for foundation model research and development with our framework. First, we describe and characterize a new pretraining task that uses synthetic data generated from symmetry operations, and reveal complex training dynamics at large scales. Using the pretrained model, we discuss a number of use cases relevant to foundation model development: semantic architecture of datasets, and fine-tuning for property prediction and classification. Our key results show that for simple applications, pretraining appears to provide worse modeling performance than training models from random initialization. However, for more complex instances, such as when a model is required to learn across multiple datas

Author

Lee, Kin Long Kelvin; Gonzales, Carmelo; Spellings, Matthew; Galkin, Mikhail; Miret, Santiago; Kumar, Nalini

Published

2023

Language

EN