Skip to content

Conversation

@Ualas
Copy link

@Ualas Ualas commented Apr 23, 2024

This pull request introduces support for Llama 3 8B model with GGUF format. The high-level changes are:

  1. Updated llama-cpp-python to v0.2.63. The previous version was causing the API to crash.
  2. Changed DEFAULT_MODEL to Llama 3 8B.
  3. Added API and UI compatibility to the Llama 3 8B model.

@Ualas Ualas mentioned this pull request Apr 23, 2024
@gyzerok
Copy link

gyzerok commented Apr 26, 2024

@Ualas would it make sense to add 70b as well?

Copy link

@derek-palmer derek-palmer May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend updating the echo to include llama3-8b with the intro of the model to reflect the current availability of models. :)

derek-palmer@f64fc2f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants