What happened this week in AI by Louie This week in AI has seen some exciting developments in the world of open-source language models together with further discussion on the legal standing of LLM training data and AI-generated content. Stability AI, the company behind the AI-powered Stable Diffusion image generator, has released a suite of open-source large language models (LLMs) called StableLM. These models are currently available between 3 billion and 7 billion parameters, with larger models arriving later. Similarly, Together has announced RedPajama, an open-source project in collaboration with other AI organizations to create large language models. RedPajama has released a 1.2 trillion token dataset that replicates the LLaMA recipe, enabling organizations to pre-train models that can be permissively licensed. RedPajama has three key components: pre-training data, base models, and instruction tuning data and models. RedPajama and StableLM follow the recent release of Dolly 2.0 and together should give a lot more flexibility for individuals or groups to train or fine-tune their own custom models for use in research or commercial products.
This AI newsletter is all you need #44
This AI newsletter is all you need #44
This AI newsletter is all you need #44
What happened this week in AI by Louie This week in AI has seen some exciting developments in the world of open-source language models together with further discussion on the legal standing of LLM training data and AI-generated content. Stability AI, the company behind the AI-powered Stable Diffusion image generator, has released a suite of open-source large language models (LLMs) called StableLM. These models are currently available between 3 billion and 7 billion parameters, with larger models arriving later. Similarly, Together has announced RedPajama, an open-source project in collaboration with other AI organizations to create large language models. RedPajama has released a 1.2 trillion token dataset that replicates the LLaMA recipe, enabling organizations to pre-train models that can be permissively licensed. RedPajama has three key components: pre-training data, base models, and instruction tuning data and models. RedPajama and StableLM follow the recent release of Dolly 2.0 and together should give a lot more flexibility for individuals or groups to train or fine-tune their own custom models for use in research or commercial products.