Skip to content

add example for mix-precison recipes (MXFP4+MXFP8) #2289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 24 commits into from
Sep 29, 2025
Merged

add example for mix-precison recipes (MXFP4+MXFP8) #2289

merged 24 commits into from
Sep 29, 2025

Conversation

xin3he
Copy link
Contributor

@xin3he xin3he commented Sep 19, 2025

Type of Change

example

@xinhe3
add example
cce026c 
Signed-off-by: xinhe3 <[email protected]>
@xinhe3
fix tp device issue
37804c1 
Signed-off-by: xinhe3 <[email protected]>
@xinhe3
update readme
5c20e1f 
Signed-off-by: xinhe3 <[email protected]>
@xin3he xin3he marked this pull request as draft September 19, 2025 07:05
xinhe3 and others added 10 commits September 19, 2025 16:29
@xinhe3
remove num_fewshot
01e8266 
Signed-off-by: xinhe3 <[email protected]>
@xinhe3
change recipe
e453ac1 
Signed-off-by: xinhe3 <[email protected]>
@xinhe3
add llama8b 7bits recipe
b31c359 
Signed-off-by: xinhe3 <[email protected]>
@xinhe3
fix gsm8k accuracy issue
afd41a2 
Signed-off-by: xinhe3 <[email protected]>
@xin3he
change batch_size as default 8
ea9ce69 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he
update script
a13eef1 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he
update example
dc5f7cb 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he
update for reproducibility
458ffab 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he
remove print
7bed59c 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he
update recipe of 8b and remove qwen
8600f2f 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he xin3he marked this pull request as ready for review September 23, 2025 07:16
@xin3he xin3he requested review from thuang6 and chensuyue September 23, 2025 07:18
@xin3he xin3he force-pushed the xinhe/mx_recipe branch 3 times, most recently from a2de5c0 to e1be060 Compare September 24, 2025 09:17
thuang6
thuang6 reviewed Sep 25, 2025
View reviewed changes
@xin3he xin3he requested a review from thuang6 September 25, 2025 06:52
@xin3he
Copy link
Contributor Author

xin3he commented Sep 25, 2025

@thuang6, Saving is added.

@thuang6
Copy link
Contributor

thuang6 commented Sep 25, 2025

@thuang6, Saving is added.

@yiliu30，Xin added mixed-bits model saving, please enable loading as well

@yiliu30
Copy link
Contributor

yiliu30 commented Sep 26, 2025

@thuang6, Saving is added.

@yiliu30，Xin added mixed-bits model saving, please enable loading as well

@thuang6 Verified the mixed MXFP4–MXFP8 llama 8b loading. It needed a small fix, which is included in intel/auto-round#848

@chensuyue chensuyue added this to the 3.6 milestone Sep 26, 2025
@xin3he xin3he requested a review from chensuyue September 29, 2025 05:50
@xin3he
update requirement
ef3221f 
Signed-off-by: He, Xin3 <[email protected]>
@xin3he xin3he requested a review from chensuyue September 29, 2025 07:10
@xin3he xin3he merged commit 7f91341 into master Sep 29, 2025
12 checks passed
@xin3he xin3he deleted the xinhe/mx_recipe branch September 29, 2025 07:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@thuang6 thuang6 thuang6 approved these changes

@chensuyue chensuyue chensuyue approved these changes

@Kaihui-intel Kaihui-intel Awaiting requested review from Kaihui-intel

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.6
Development

Successfully merging this pull request may close these issues.
6 participants
@xin3he @thuang6 @yiliu30 @chensuyue @Kaihui-intel @xinhe3