Skip to content

Commit 7322b91

Browse files
mlcommons-botmlcommons-bot
committed
Updated results summary [skip ci]
1 parent 248e788 commit 7322b91

File tree

6 files changed

+22
-22
lines changed

6 files changed

+22
-22
lines changed

‎README.md‎

Lines changed: 15 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,21 @@
11
Please download [summary.xlsx](summary.xlsx) to view the most recent results.
22
```
3-
[2025-09-14 16:27:04,903 submission_checker1.py:3492 INFO] Results=33, NoResults=0, Power Results=0
4-
[2025-09-14 16:27:04,903 submission_checker1.py:3499 INFO] ---
5-
[2025-09-14 16:27:04,903 submission_checker1.py:3500 INFO] Closed Results=0, Closed Power Results=0
3+
[2025-09-14 18:13:35,945 submission_checker1.py:3492 INFO] Results=33, NoResults=0, Power Results=0
4+
[2025-09-14 18:13:35,945 submission_checker1.py:3499 INFO] ---
5+
[2025-09-14 18:13:35,945 submission_checker1.py:3500 INFO] Closed Results=0, Closed Power Results=0
66
7-
[2025-09-14 16:27:04,903 submission_checker1.py:3505 INFO] Open Results=33, Open Power Results=0
7+
[2025-09-14 18:13:35,945 submission_checker1.py:3505 INFO] Open Results=33, Open Power Results=0
88
9-
[2025-09-14 16:27:04,903 submission_checker1.py:3510 INFO] Network Results=0, Network Power Results=0
9+
[2025-09-14 18:13:35,945 submission_checker1.py:3510 INFO] Network Results=0, Network Power Results=0
1010
11-
[2025-09-14 16:27:04,903 submission_checker1.py:3515 INFO] ---
12-
[2025-09-14 16:27:04,904 submission_checker1.py:3517 INFO] Systems=26, Power Systems=0
13-
[2025-09-14 16:27:04,904 submission_checker1.py:3521 INFO] Closed Systems=0, Closed Power Systems=0
14-
[2025-09-14 16:27:04,904 submission_checker1.py:3526 INFO] Open Systems=26, Open Power Systems=0
15-
[2025-09-14 16:27:04,904 submission_checker1.py:3531 INFO] Network Systems=0, Network Power Systems=0
16-
[2025-09-14 16:27:04,904 submission_checker1.py:3536 INFO] ---
17-
[2025-09-14 16:27:04,904 submission_checker1.py:3541 INFO] SUMMARY: submission looks OK
18-
[2025-09-14 16:27:06,012 module.py:5392 INFO] - ! call "postprocess" from /home/runner/MLC/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
11+
[2025-09-14 18:13:35,945 submission_checker1.py:3515 INFO] ---
12+
[2025-09-14 18:13:35,945 submission_checker1.py:3517 INFO] Systems=26, Power Systems=0
13+
[2025-09-14 18:13:35,945 submission_checker1.py:3521 INFO] Closed Systems=0, Closed Power Systems=0
14+
[2025-09-14 18:13:35,945 submission_checker1.py:3526 INFO] Open Systems=26, Open Power Systems=0
15+
[2025-09-14 18:13:35,945 submission_checker1.py:3531 INFO] Network Systems=0, Network Power Systems=0
16+
[2025-09-14 18:13:35,945 submission_checker1.py:3536 INFO] ---
17+
[2025-09-14 18:13:35,945 submission_checker1.py:3541 INFO] SUMMARY: submission looks OK
18+
[2025-09-14 18:13:37,207 module.py:5401 INFO] - ! call "postprocess" from /home/runner/MLC/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
1919
2020
```
2121

@@ -39,12 +39,12 @@ Please download [summary.xlsx](summary.xlsx) to view the most recent results.
3939
| 15 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config | retinanet | retinanet | Offline | 0.437938 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config/retinanet/offline | onnxruntime v1.21.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4040
| 16 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config | resnet50 | resnet | Offline | 22.7609 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline | onnxruntime v1.22.1 | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4141
| 17 | MLCommons | available | open | edge | gh_macos-latest_x86 | gh_macos-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config | bert-99 | bert-99 | Offline | 4.48923 | F1: 80.0 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config/bert-99/offline | deepsparse v1.8.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
42-
| 18 | MLCommons | available | open | edge | default | default-mlcommons_cpp-cpu-onnxruntime-default_config | retinanet | retinanet | Offline | 0.423324 | mAP: 49.593 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
42+
| 18 | MLCommons | available | open | edge | default | default-mlcommons_cpp-cpu-onnxruntime-default_config | retinanet | retinanet | Offline | 0.421802 | mAP: 49.593 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4343
| 19 | MLCommons | available | open | datacenter | gh_action | gh_action-reference-cpu-pytorch_v2.6.0-default_config | mixtral-8x7b | mixtral-8x7b | Offline | 0.0148696 | ROUGE1: 31.4286 ROUGE2: 5.8824 ROUGEL: 28.5714 TOKENS_PER_SAMPLE: 152.0 gsm8k_accuracy: 100.0 mbxp_accuracy: 0.0 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | nan | 0 | open/MLCommons/results/gh_action-reference-cpu-pytorch_v2.6.0-default_config/mixtral-8x7b/offline | pytorch v2.6.0 | Ubuntu 22.04 (linux-6.8.0-52-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Tokens/s | fp32 |
4444
| 20 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config | resnet50 | resnet | Offline | 23.0274 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/resnet50/offline | onnxruntime v1.21.1 | Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4545
| 21 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config | retinanet | retinanet | Offline | 0.431769 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/retinanet/offline | onnxruntime v1.21.1 | Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4646
| 22 | MLCommons | available | open | datacenter | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config | rgat | rgat | Offline | 10.1918 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline | pytorch v2.4.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
47-
| 23 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config | resnet50 | resnet | Offline | 14.5769 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline | onnxruntime v1.22.1 | (darwin-24.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
47+
| 23 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config | resnet50 | resnet | Offline | 18.4787 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline | onnxruntime v1.22.1 | (darwin-24.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4848
| 24 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.6.0-default_config | retinanet | retinanet | Offline | 0.405224 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.6.0-default_config/retinanet/offline | pytorch v2.6.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
4949
| 25 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config | resnet50 | resnet | Offline | 15.9553 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/resnet50/offline | onnxruntime v1.22.0 | (darwin-23.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
5050
| 26 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config | resnet50 | resnet | Offline | 9.24205 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline | onnxruntime | (darwin-24.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |

‎dbversion‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1532
1+
1533

‎summary.csv‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,12 @@ MLCommons,available,open,"edge","gh_ubuntu-latest_x86",gh_ubuntu-latest_x86-refe
1717
MLCommons,available,open,"edge","gh_ubuntu-latest_x86",gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config,retinanet,retinanet,Offline,0.437938,mAP: 76.951,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.0-default_config/retinanet/offline,"onnxruntime v1.21.0","Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
1818
MLCommons,available,open,"edge","gh_ubuntu-latest x86",gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config,resnet50,resnet,Offline,22.7609,acc: 76.000,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline,"onnxruntime v1.22.1","Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
1919
MLCommons,available,open,"edge","gh_macos-latest_x86",gh_macos-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config,bert-99,bert-99,Offline,4.48923,F1: 80.0,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-reference-cpu-deepsparse_v1.8.0-default_config/bert-99/offline,"deepsparse v1.8.0"," (darwin-23.6.0)","",1,0,v5.0,0,False,Samples/s,"fp32"
20-
MLCommons,available,open,"edge","default",default-mlcommons_cpp-cpu-onnxruntime-default_config,retinanet,retinanet,Offline,0.423324,mAP: 49.593,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline,"onnxruntime","Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
20+
MLCommons,available,open,"edge","default",default-mlcommons_cpp-cpu-onnxruntime-default_config,retinanet,retinanet,Offline,0.421802,mAP: 49.593,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline,"onnxruntime","Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
2121
MLCommons,available,open,"datacenter","gh_action",gh_action-reference-cpu-pytorch_v2.6.0-default_config,mixtral-8x7b,mixtral-8x7b,Offline,0.0148696,ROUGE1: 31.4286 ROUGE2: 5.8824 ROUGEL: 28.5714 TOKENS_PER_SAMPLE: 152.0 gsm8k_accuracy: 100.0 mbxp_accuracy: 0.0,1,"Intel(R) Xeon(R) w7-2495X",1,24,"N/A","0",open/MLCommons/results/gh_action-reference-cpu-pytorch_v2.6.0-default_config/mixtral-8x7b/offline,"pytorch v2.6.0","Ubuntu 22.04 (linux-6.8.0-52-generic-glibc2.35)","",1,0,v5.0,0,False,Tokens/s,"fp32"
2222
MLCommons,available,open,"edge","gh_ubuntu-latest x86",gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config,resnet50,resnet,Offline,23.0274,acc: 76.000,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/resnet50/offline,"onnxruntime v1.21.1","Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
2323
MLCommons,available,open,"edge","gh_ubuntu-latest x86",gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config,retinanet,retinanet,Offline,0.431769,mAP: 76.951,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.21.1-default_config/retinanet/offline,"onnxruntime v1.21.1","Ubuntu 24.04 (linux-6.11.0-1013-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
2424
MLCommons,available,open,"datacenter","gh_ubuntu-latest_x86",gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config,rgat,rgat,Offline,10.1918,acc: 76.000,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.4.0-default_config/rgat/offline,"pytorch v2.4.0","Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
25-
MLCommons,available,open,"edge","gh_macos-latest x86",gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config,resnet50,resnet,Offline,14.5769,acc: 76.000,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline,"onnxruntime v1.22.1"," (darwin-24.6.0)","",1,0,v5.0,0,False,Samples/s,"fp32"
25+
MLCommons,available,open,"edge","gh_macos-latest x86",gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config,resnet50,resnet,Offline,18.4787,acc: 76.000,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline,"onnxruntime v1.22.1"," (darwin-24.6.0)","",1,0,v5.0,0,False,Samples/s,"fp32"
2626
MLCommons,available,open,"edge","gh_ubuntu-latest_x86",gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.6.0-default_config,retinanet,retinanet,Offline,0.405224,mAP: 76.951,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-pytorch_v2.6.0-default_config/retinanet/offline,"pytorch v2.6.0","Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
2727
MLCommons,available,open,"edge","gh_macos-latest x86",gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config,resnet50,resnet,Offline,15.9553,acc: 76.000,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.0-default_config/resnet50/offline,"onnxruntime v1.22.0"," (darwin-23.6.0)","",1,0,v5.0,0,False,Samples/s,"fp32"
2828
MLCommons,available,open,"edge","gh_macos-latest x86",gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config,resnet50,resnet,Offline,9.24205,acc: 76.000,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-mlcommons_cpp-cpu-onnxruntime-default_config/resnet50/offline,"onnxruntime"," (darwin-24.6.0)","",1,0,v5.0,0,False,Samples/s,"fp32"

0 commit comments

Comments
 (0)