Skip to content

Conversation

@Wauplin
Copy link
Contributor

@Wauplin Wauplin commented Sep 16, 2024

Follow-up after https://github.com/huggingface-internal/moon-landing/pull/11060 and https://github.com/huggingface-internal/moon-landing/pull/11112 (internal links).

This PR adds support for the gguf key in ModelInfo. Also documenting it as supported in expand[]= parameter. Note that it was supported as soon as the server-side change is made, this PR simply makes it better documented.

Example:

>>> from huggingface_hub import model_info
>>> model = model_info("MaziyarPanahi/solar-pro-preview-instruct-GGUF", expand="gguf")
>>> model.gguf
{'total': 22140032000, 'architecture': 'llama', 'context_length': 4096, 'quantize_imatrix_file': './solar-pro-preview-instruct-GGUF_imatrix.dat', 'chat_template': "{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}"}

@hanouticelina
Copy link
Contributor

Seems like a concurrent PR with @hlky's #2544 😄

Copy link
Contributor

@hanouticelina hanouticelina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 thanks!

@Wauplin Wauplin merged commit a9adf8f into main Sep 16, 2024
@Wauplin Wauplin deleted the add-gguf-as-expand-parameter branch September 16, 2024 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants