Skip to content
This repository was archived by the owner on Sep 20, 2025. It is now read-only.

feat: add ollama and llama.cpp engines #5

Merged
merged 1 commit into from
Feb 19, 2025
Merged

feat: add ollama and llama.cpp engines #5

merged 1 commit into from
Feb 19, 2025

Conversation

11zhouxuan
Copy link
Collaborator

@11zhouxuan 11zhouxuan commented Feb 18, 2025 •
edited
Loading

  1. Add ollama and llama.cpp engines.
  2. Support DeepSeek R1 / V3 series models.
  3. Support TGI engine with Inf2 instance.
  4. Support customize model files path.
  5. Add supported models doc.

@11zhouxuan 11zhouxuan changed the title Dev feat: add ollama and llama.cpp engines Feb 18, 2025
@AoyuQC
Copy link
Contributor

AoyuQC commented Feb 18, 2025

@11zhouxuan please reset commits to one commit after @outstandingcandy finishes reviewing, thx

@11zhouxuan
add ollama engine
03bfce2 
support multiple model files prepare methods

modify local deploy

support llama.cpp engine

add generate_supported_models_doc_cli.py; add docs/supported_models.md

fix concurrency bug

modify tgi backend
@AoyuQC AoyuQC self-requested a review February 19, 2025 00:36
@AoyuQC AoyuQC removed the request for review from outstandingcandy February 19, 2025 00:41
@11zhouxuan 11zhouxuan merged commit 80e62ee into main Feb 19, 2025
1 check passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@AoyuQC AoyuQC AoyuQC approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@11zhouxuan @AoyuQC @outstandingcandy