ChatGPT is great but a lot of folks are concerned about data privacy.
Here's an alternative: Install a local LLM (Large Language Model) on your computer. You can it use offline and keep your data private.
Best part — you can do it for free!
Here's how to do it:
Here's an alternative: Install a local LLM (Large Language Model) on your computer. You can it use offline and keep your data private.
Best part — you can do it for free!
Here's how to do it:
1. Go to lmstudio(dot)ai and download the version of the studio compatible with your device.
Open the setup file to install LM Studio on your computer.
Open the setup file to install LM Studio on your computer.
2. Open LM Studio after you have installed it.
In the search bar, type in the name of an LLM you would like to use and click on "Go."
For the purposes of this tutorial, I am going with LLaMa, an open source model by Meta AI.
On the left side of your screen, you will see different version of LLaMa. On the right side, you will "Quantization" of different models.
On the top of the Quantization column, you will see a check mark with "Should Work." It shows that your computer's hardware fulfills the requirements of this model.
If your hardware does not fulfill the requirements of a given model, you won't see "Should Work."
I looked up Falcon and it told me I would need 150+ GBs of RAM to run this model.
In the search bar, type in the name of an LLM you would like to use and click on "Go."
For the purposes of this tutorial, I am going with LLaMa, an open source model by Meta AI.
On the left side of your screen, you will see different version of LLaMa. On the right side, you will "Quantization" of different models.
On the top of the Quantization column, you will see a check mark with "Should Work." It shows that your computer's hardware fulfills the requirements of this model.
If your hardware does not fulfill the requirements of a given model, you won't see "Should Work."
I looked up Falcon and it told me I would need 150+ GBs of RAM to run this model.
3. Choose a model and version you want to download.
Click on the "Download" button and wait for the download to complete.
If you are starting out, I would recommend downloading a model that is less that 10GBs.
Bigger models would take a lot of time to download and will slow your computer down.
Click on the "Download" button and wait for the download to complete.
If you are starting out, I would recommend downloading a model that is less that 10GBs.
Bigger models would take a lot of time to download and will slow your computer down.
4. After you have downloaded a model, click on the chat bubble in the left task bar.
Click on the bar at the top and load the model. Once your model is loaded, you can customize its "Inference Parameters" by going to "Advanced Configuration."
You can customize the randomness and length of the responses.
Click on the bar at the top and load the model. Once your model is loaded, you can customize its "Inference Parameters" by going to "Advanced Configuration."
You can customize the randomness and length of the responses.
5. Now you can start using the model. Type your question in the chat bar and it will answer it.
Quality of responses: I used LLaMa 2 7B for this chat and it started hallucinating right from the start.
I asked if it knew about Mikhail Bakhtin. It said that Bakhtin was born in Ukraine in 1891, which is not true. He was born in Oryol in 1895.
I asked it about Bakhtin's idea of "chronotopes" and the answer was way off the mark. It looked like the model was trying but falling short.
That said, having a local chat bot on your computer is still very useful. It can certainly help you brainstorm ideas and create outlines for journal articles and grant proposals.
Quality of responses: I used LLaMa 2 7B for this chat and it started hallucinating right from the start.
I asked if it knew about Mikhail Bakhtin. It said that Bakhtin was born in Ukraine in 1891, which is not true. He was born in Oryol in 1895.
I asked it about Bakhtin's idea of "chronotopes" and the answer was way off the mark. It looked like the model was trying but falling short.
That said, having a local chat bot on your computer is still very useful. It can certainly help you brainstorm ideas and create outlines for journal articles and grant proposals.
Compare LLaMa 2 7B with ChatGPT 3.5 and you can see the difference quite clearly.
Without any custom instructions, ChatGPT 3.5 gave a correct, detailed, and useful answer to my question about chronotopes.
If you haven't tried, I'd suggest download an LLM compatible with your computer and start experimenting.
Without any custom instructions, ChatGPT 3.5 gave a correct, detailed, and useful answer to my question about chronotopes.
If you haven't tried, I'd suggest download an LLM compatible with your computer and start experimenting.
Found this thread on using local LLMs helpful?
1. Scroll to the top and hit Like on the first tweet.
2. Bookmark it so you can come back to it later.
3. Follow me for more threads on how to use AI apps for academic writing.
1. Scroll to the top and hit Like on the first tweet.
2. Bookmark it so you can come back to it later.
3. Follow me for more threads on how to use AI apps for academic writing.
Loading suggestions...