is it possible to run autogen locally with GPT4ALL model without API ?!

[Mana-84]

Hello,
Kindly I need your guidance in my project, I want to run Autogen locally on my machine with out API, using GPT4ALL, is it possible ?!
MODEL_TYPE=GPT4All
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
this is what I am using in .env file,

could you help :)

[sonichi]

related issue: #46
related PR: #95

[Mana-84]

@sonichi thank you for your answer...
to be more specific ,I am interested in Autogen (agents) AssistantAgent, UserProxyAgent in my project:
config_list to the initiate_chat function.
your help is highly appreciated

[sonichi]

It might already work with https://microsoft.github.io/autogen/blog/2023/07/14/Local-LLMs

[Mana-84]

@sonichi i will check it, and i will let you know the results, thanks a lot

[basharourabi]

@sonichi cant this be done like the Langchain method by simply placing a model folder and placing a path instead of an api call ?

[sonichi]

Could you ask it in https://discord.com/channels/1153072414184452236/1158548894209282078 ?