A volume of data used to train an AI model. There are many bodies of data that are combined to train small and large language models. For example, the RedPajama AI small model dataset comprises 300 billion data items from books, GitHub, Wikipedia and other sources. The Dolma (Data to feed OLMo's Appetite) dataset contains three trillion items of data from sources such as Reddit, Project Gutenberg (books), Wikipedia and Wikibooks (see
OLMo).
Validation and Testing
There are millions of training datasets worldwide; for example, Hugging Face has more than a million alone (see
Hugging Face). Although the bulk are for training AI models, datasets are also created for validation and testing.