Skip to content

[QEff]: Add OpenAI Oss Models (gpt_oss) #534

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 23 commits into from
Closed

[QEff]: Add OpenAI Oss Models (gpt_oss) #534

wants to merge 23 commits into from

Conversation

@vbaddi
Copy link
Contributor

@vbaddi vbaddi commented Aug 6, 2025

No description provided.

@vbaddi
[QEff]: Add gpt_oss
485308a 
Signed-off-by: vbaddi <[email protected]>
@vbaddi vbaddi self-assigned this Aug 6, 2025
quic-rishinr
quic-rishinr requested changes Aug 7, 2025
View reviewed changes
@vbaddi vbaddi added the enhancement New feature or request label Aug 7, 2025
ochougul and others added 10 commits August 7, 2025 15:23
@ochougul
apirunner change
8e5783e 
Signed-off-by: Onkar Chougule <[email protected]>
@ochougul
added test along with simplified Hybridcache
5c3c971 
Signed-off-by: Onkar Chougule <[email protected]>
@ochougul
added test assert
ce53d3c 
Signed-off-by: Onkar Chougule <[email protected]>
@vbaddi
nit: update modeling and make transform uniform
e0bd90f 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: add changes from gpt_oss_swa branch
18795f2 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: update test gpt file
30ed222 
Signed-off-by: vbaddi <[email protected]>
@ochougul
MOE optimized
14afedb 
Signed-off-by: Onkar Chougule <[email protected]>
@vbaddi
nit: update modeling with new decode moe forward
99f4795 
Signed-off-by: vbaddi <[email protected]>
@ochougul
simplified slidingwindow KV gather and attention is permutation invar…
8637f1f 
…ient for columns of KV

Signed-off-by: Onkar Chougule <[email protected]>
@vbaddi
nit: seperate gate, up projections for MoE
87414a2 
Signed-off-by: vbaddi <[email protected]>
abukhoy
abukhoy reviewed Sep 2, 2025
View reviewed changes
ochougul and others added 8 commits October 8, 2025 11:52
@ochougul
added MXFP4 quantizer support to directly load GPT-OSS models via QEF…
3adccf6 
…FAutoModelForCausalLM (#577)

* added mxfp4 quantizer to match weights keys
* added transform to dequantize mxfp4 to float32

---------

Signed-off-by: Onkar Chougule <[email protected]>
@vbaddi
nit: rebase to mainline and resolve conflicts
890eed7 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: add license details to mxfp4 quantizer
402f8cb 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: rebase to mainline
e557472
@vbaddi
nit: remove test file and add sample test in config
05c599d 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: remove streamer from .generate() api in example file
dfddd55 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: device_ids typo in example script
04ae7d8 
Signed-off-by: vbaddi <[email protected]>
@vbaddi
nit: fix model_name in tests
3cadfba 
Signed-off-by: vbaddi <[email protected]>
@MT-Paugurs
Copy link

MT-Paugurs commented Oct 28, 2025

Good day, Qefficient dev team,
we are using 8x AI100Ultra cards locally and are very interested into trying out this model. Any estimates for the release date?

@mamtsing
Enable CB for GptOssModel
21a6620 
Signed-off-by: Mamta Singh <[email protected]>
@ochougul
Copy link
Contributor

ochougul commented Nov 3, 2025

Good day, Qefficient dev team, we are using 8x AI100Ultra cards locally and are very interested into trying out this model. Any estimates for the release date?

Hi @MT-Paugurs , Thanks for showing interest.
Feel free to use this branch to give a quick run using examples/gpt_oss.py script.
Currently the model is optimized for decode-only mode.
We are working on enabling the prefill+decode mode. And will be released soon.

Can you please tell, which sdk version you are using?

@ochougul ochougul force-pushed the add_gpt_oss branch 2 times, most recently from 084412a to 21a6620 Compare November 3, 2025 16:47
@quic-mamta quic-mamta marked this pull request as draft November 3, 2025 17:23
@PaugursA
Copy link

PaugursA commented Nov 3, 2025

Good day, Qefficient dev team, we are using 8x AI100Ultra cards locally and are very interested into trying out this model. Any estimates for the release date?

Hi @MT-Paugurs , Thanks for showing interest.

Feel free to use this branch to give a quick run using examples/gpt_oss.py script.

Currently the model is optimized for decode-only mode.

We are working on enabling the prefill+decode mode. And will be released soon.

Can you please tell, which sdk version you are using?

Hi! Currently using Platform/Apps SDKs 1.20.4.0.

Thanks for the info!

Paugurs

@quic-mamta quic-mamta closed this Nov 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@quic-rishinr quic-rishinr quic-rishinr requested changes

@ochougul ochougul Awaiting requested review from ochougul ochougul is a code owner

@quic-hemagnih quic-hemagnih Awaiting requested review from quic-hemagnih quic-hemagnih is a code owner

@quic-amitraj quic-amitraj Awaiting requested review from quic-amitraj quic-amitraj is a code owner

+1 more reviewer

@abukhoy abukhoy abukhoy left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@vbaddi vbaddi

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@vbaddi @MT-Paugurs @ochougul @PaugursA @quic-rishinr @abukhoy @quic-mamta @mamtsing