Another truly open-source instruction-following LLM was recently released - Dolly 2.0 by @databricks!
I wrote a lot about LLaMA over the previous period, but there is a BIG caveat with LLaMA: the license is not really permissive...
Blog: databricks.com
1/ 👇🧵
I wrote a lot about LLaMA over the previous period, but there is a BIG caveat with LLaMA: the license is not really permissive...
Blog: databricks.com
1/ 👇🧵
(not open source) and you thus can't use the model for commercial purposes.
All of that has changed now with Dolly 2.0 (& OpenAssistant), everything is open-sourced including training code, dataset (15k human-generated high-quality samples!), and model weights.
2/
All of that has changed now with Dolly 2.0 (& OpenAssistant), everything is open-sourced including training code, dataset (15k human-generated high-quality samples!), and model weights.
2/
Dolly is a 12B parameter language model based on EleutherAI's Pythia model family. Thus important to note: the model will likely not be as powerful as other available models, but it's a step in the right direction.
3/
3/
DATA:
Fun fact: the data was collected by 5k+ Databricks employees by creating an internal 1 week-long contest with proper incentives set in place! Kudos to the team!
Why was it created? Here is an excerpt from their blog:
4/
Fun fact: the data was collected by 5k+ Databricks employees by creating an internal 1 week-long contest with proper incentives set in place! Kudos to the team!
Why was it created? Here is an excerpt from their blog:
4/
Dolly 1.0 was trained for $30 using a dataset that the Alpaca team had created using the OpenAI API. That dataset contained output from ChatGPT, and as the Stanford team pointed out, the terms of service seek to prevent anyone from creating a model that competes with OpenAI
5/
5/
Thus Alpaca, Koala, GPT4All, Vicuna et al, all suffer from this and can't be used for commercial purposes - while Dolly 2.0 and its future derivatives can!
6/6
6/6
Not huge news at this point but in case you missed it ;)
Loading suggestions...