Technology
Artificial Intelligence
Machine Learning
Software Development
Natural Language Processing
I just added @nomic_ai new GPT4All Embeddings to @LangChainAI. Here's a new doc on running local / private retrieval QA (e.g., on your laptop) w/ GPT4All embeddings + @trychroma + GPT4All LLM. Easy setup, great work from @nomic_ai ...
python.langchain.com
python.langchain.com
Impressively easy UX from @nomic_ai. Embeddings "just worked" (auto-downloads locally) ...
python.langchain.com
python.langchain.com
... example in docs shows retrievalQA on @lilianweng great blog post on agents. Load the blog text from website, split it, and create a local @trychroma vectorDB w/ local GPT4All embeddings. Ask any question, get back relevant splits ...
... w/ local DB done, download LLM from @nomic_ai. Their Model Explorer is very helpful to browse different models. Download and then supply the local model path in notebook to create a local LLM ...
gpt4all.io
gpt4all.io
Docs show running locally at 3 levels of abstraction: 1) LLM chain. Simplest / lowest level of abstraction. Input retrieved docs (we fetched above), a prompt, the llm we initialized (above). Summarization of the 4 retrieved docs runs locally ~100s for me (Apple M2 Max, 32GB):
2) QA chain. Input retrieved docs (above), an optional prompt, the llm (above), and a question. QA on the retrieved docs runs locally ~30s (Apple M2 Max, 32GB):
3) Retrieval QA chain. Input vectorDB, an optional prompt, the llm (above), and a question. This takes care of retrieval from the vectorDB (whereas above in QA chain we needed to do it ourselves and pass the docs in). Runs locally ~30s (Apple M2 Max, 32GB):
Loading suggestions...