App Crash When Loading Models

[Vinayak0-0]

Description

I encountered crashes in my application when attempting to load the gemma-2b-it.gguf and Phi-3-mini-4k-instruct-q4.gguf models. Below are the error messages and details for each model:

For gemma-2b-it.gguf:

error loading model: unknown model architecture: 'gemma'
llama_load_model_from_file: failed to load model
flutter: [AubAi] llama_new_context_with_model(model, cparams)
Lost connection to device.

For Phi-3-mini-4k-instruct-q4.gguf:

error loading model: unknown model architecture: 'phi3'
llama_load_model_from_file: failed to load model
flutter: [AubAi] llama_new_context_with_model(model, cparams)
Lost connection to device.

Expected Behavior

The models should load without causing the app to crash.

Actual Behavior

The app crashes immediately after attempting to load the models, with an error indicating "unknown model architecture" for both gemma and phi3.

Additional Information

• Platform: Linux Ubuntu
• Flutter Version: 3.22.2

[ArachnaDigitalLtd]

Yea same here. Unfortunately, its looks like the repo isn't active anymore. If you find something similar that actually works, let me know!

[snehccurry]

I believe you first have to load the model to the os, there's a limitiation ig as to how much data u can transfer while building the app, ig it's 1 gb, also there is a package path_provider that has its own compatibility issues too ig. idk what's wrong tho, but this shouldn't have been an issue in the first place, like ppl will have to download the model manually. or after the app is installed it will have to check if the model is on device or not, if not then download it from the cloud and then use it. try this once!