Skip to content

Commit cc53a8f

Browse files
mlcommons-botmlcommons-bot
committed
Updated results summary [skip ci]
1 parent 75b976e commit cc53a8f

File tree

6 files changed

+18
-18
lines changed

6 files changed

+18
-18
lines changed

‎README.md‎

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,29 @@
11
Please download [summary.xlsx](summary.xlsx) to view the most recent results.
22
```
3-
[2025-10-16 14:33:41,495 submission_checker1.py:3492 INFO] Results=37, NoResults=0, Power Results=0
4-
[2025-10-16 14:33:41,495 submission_checker1.py:3499 INFO] ---
5-
[2025-10-16 14:33:41,495 submission_checker1.py:3500 INFO] Closed Results=0, Closed Power Results=0
3+
[2025-10-16 15:08:25,410 submission_checker1.py:3492 INFO] Results=37, NoResults=0, Power Results=0
4+
[2025-10-16 15:08:25,410 submission_checker1.py:3499 INFO] ---
5+
[2025-10-16 15:08:25,410 submission_checker1.py:3500 INFO] Closed Results=0, Closed Power Results=0
66
7-
[2025-10-16 14:33:41,495 submission_checker1.py:3505 INFO] Open Results=37, Open Power Results=0
7+
[2025-10-16 15:08:25,410 submission_checker1.py:3505 INFO] Open Results=37, Open Power Results=0
88
9-
[2025-10-16 14:33:41,495 submission_checker1.py:3510 INFO] Network Results=0, Network Power Results=0
9+
[2025-10-16 15:08:25,410 submission_checker1.py:3510 INFO] Network Results=0, Network Power Results=0
1010
11-
[2025-10-16 14:33:41,495 submission_checker1.py:3515 INFO] ---
12-
[2025-10-16 14:33:41,495 submission_checker1.py:3517 INFO] Systems=30, Power Systems=0
13-
[2025-10-16 14:33:41,495 submission_checker1.py:3521 INFO] Closed Systems=0, Closed Power Systems=0
14-
[2025-10-16 14:33:41,495 submission_checker1.py:3526 INFO] Open Systems=30, Open Power Systems=0
15-
[2025-10-16 14:33:41,495 submission_checker1.py:3531 INFO] Network Systems=0, Network Power Systems=0
16-
[2025-10-16 14:33:41,495 submission_checker1.py:3536 INFO] ---
17-
[2025-10-16 14:33:41,495 submission_checker1.py:3541 INFO] SUMMARY: submission looks OK
18-
[2025-10-16 14:33:42,926 module.py:5457 INFO] - ! call "postprocess" from /home/runner/MLC/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
11+
[2025-10-16 15:08:25,410 submission_checker1.py:3515 INFO] ---
12+
[2025-10-16 15:08:25,410 submission_checker1.py:3517 INFO] Systems=30, Power Systems=0
13+
[2025-10-16 15:08:25,410 submission_checker1.py:3521 INFO] Closed Systems=0, Closed Power Systems=0
14+
[2025-10-16 15:08:25,410 submission_checker1.py:3526 INFO] Open Systems=30, Open Power Systems=0
15+
[2025-10-16 15:08:25,410 submission_checker1.py:3531 INFO] Network Systems=0, Network Power Systems=0
16+
[2025-10-16 15:08:25,410 submission_checker1.py:3536 INFO] ---
17+
[2025-10-16 15:08:25,410 submission_checker1.py:3541 INFO] SUMMARY: submission looks OK
18+
[2025-10-16 15:08:26,468 module.py:5457 INFO] - ! call "postprocess" from /home/runner/MLC/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
1919
2020
```
2121

2222
| | Organization | Availability | Division | SystemType | SystemName | Platform | Model | MlperfModel | Scenario | Result | Accuracy | number_of_nodes | host_processor_model_name | host_processors_per_node | host_processor_core_count | accelerator_model_name | accelerators_per_node | Location | framework | operating_system | notes | compliance | errors | version | inferred | has_power | Units | weight_data_types |
2323
|---:|:---------------|:---------------|:-----------|:-------------|:---------------------|:----------------------------------------------------------------------|:--------------------|:--------------------|:-----------|-----------:|:----------------------------------------------------------------------------------------------------------------------|------------------:|:--------------------------------|---------------------------:|----------------------------:|:-------------------------|------------------------:|:---------------------------------------------------------------------------------------------------------------|:--------------------|:-------------------------------------------------|--------:|-------------:|---------:|:----------|-----------:|:------------|:----------|:--------------------|
2424
| 0 | MLCommons | available | open | edge | gh_ubuntu-latest x86 | gh_ubuntu-latest_x86-reference-cpu-tf_v2.18.0-default_config | resnet50 | resnet | Offline | 20.9828 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-tf_v2.18.0-default_config/resnet50/offline | tf v2.18.0 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
2525
| 1 | MLCommons | available | open | edge | gh_macos-latest x86 | gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config | resnet50 | resnet | Offline | 17.5501 | acc: 76.000 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline | onnxruntime v1.22.1 | (darwin-24.6.0) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
26-
| 2 | MLCommons | available | open | edge | default | default-mlcommons_cpp-cpu-onnxruntime-default_config | retinanet | retinanet | Offline | 0.421782 | mAP: 49.593 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
26+
| 2 | MLCommons | available | open | edge | default | default-mlcommons_cpp-cpu-onnxruntime-default_config | retinanet | retinanet | Offline | 0.417492 | mAP: 49.593 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline | onnxruntime | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
2727
| 3 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config | resnet50 | resnet | Offline | 22.8483 | acc: 76.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline | onnxruntime v1.20.1 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
2828
| 4 | MLCommons | available | open | edge | gh_ubuntu-latest_x86 | gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config | retinanet | retinanet | Offline | 0.436959 | mAP: 76.951 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/retinanet/offline | onnxruntime v1.20.1 | Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
2929
| 5 | MLCommons | available | open | edge | gh_ubuntu-latest | gh_ubuntu-latest-reference-cpu-tvm-onnx_v1.19.2-default_config | resnet50 | resnet | Offline | 1.6929 | acc: 80.000 | 1 | AMD EPYC 7763 64-Core Processor | 1 | 2 | nan | 0 | open/MLCommons/results/gh_ubuntu-latest-reference-cpu-tvm-onnx_v1.19.2-default_config/resnet50/offline | tvm-onnx v1.19.2 | Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |

‎dbversion‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
1594
1+
1595

‎summary.csv‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
Organization,Availability,Division,SystemType,SystemName,Platform,Model,MlperfModel,Scenario,Result,Accuracy,number_of_nodes,host_processor_model_name,host_processors_per_node,host_processor_core_count,accelerator_model_name,accelerators_per_node,Location,framework,operating_system,notes,compliance,errors,version,inferred,has_power,Units,weight_data_types
22
MLCommons,available,open,"edge","gh_ubuntu-latest x86",gh_ubuntu-latest_x86-reference-cpu-tf_v2.18.0-default_config,resnet50,resnet,Offline,20.9828,acc: 76.000,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-tf_v2.18.0-default_config/resnet50/offline,"tf v2.18.0","Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
33
MLCommons,available,open,"edge","gh_macos-latest x86",gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config,resnet50,resnet,Offline,17.5501,acc: 76.000,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline,"onnxruntime v1.22.1"," (darwin-24.6.0)","",1,0,v5.0,0,False,Samples/s,"fp32"
4-
MLCommons,available,open,"edge","default",default-mlcommons_cpp-cpu-onnxruntime-default_config,retinanet,retinanet,Offline,0.421782,mAP: 49.593,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline,"onnxruntime","Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
4+
MLCommons,available,open,"edge","default",default-mlcommons_cpp-cpu-onnxruntime-default_config,retinanet,retinanet,Offline,0.417492,mAP: 49.593,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet/offline,"onnxruntime","Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
55
MLCommons,available,open,"edge","gh_ubuntu-latest_x86",gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config,resnet50,resnet,Offline,22.8483,acc: 76.000,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/resnet50/offline,"onnxruntime v1.20.1","Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
66
MLCommons,available,open,"edge","gh_ubuntu-latest_x86",gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config,retinanet,retinanet,Offline,0.436959,mAP: 76.951,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest_x86-reference-cpu-onnxruntime_v1.20.1-default_config/retinanet/offline,"onnxruntime v1.20.1","Ubuntu 24.04 (linux-6.8.0-1021-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"
77
MLCommons,available,open,"edge","gh_ubuntu-latest",gh_ubuntu-latest-reference-cpu-tvm-onnx_v1.19.2-default_config,resnet50,resnet,Offline,1.6929,acc: 80.000,1,"AMD EPYC 7763 64-Core Processor",1,2,"N/A","0",open/MLCommons/results/gh_ubuntu-latest-reference-cpu-tvm-onnx_v1.19.2-default_config/resnet50/offline,"tvm-onnx v1.19.2","Ubuntu 24.04 (linux-6.11.0-1018-azure-glibc2.39)","",1,0,v5.0,0,False,Samples/s,"fp32"

‎summary.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@
6969
"Model":"retinanet",
7070
"MlperfModel":"retinanet",
7171
"Scenario":"Offline",
72-
"Result":0.421782,
72+
"Result":0.417492,
7373
"Accuracy":"mAP: 49.593",
7474
"number_of_nodes":1,
7575
"host_processor_model_name":"AMD EPYC 7763 64-Core Processor",

‎summary.xlsx‎

0 Bytes
Binary file not shown.

‎summary_results.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@
104104
"Code": "https://github.com/mlcommons/mlperf_inference_test_submissions_v5.0/tree/main/open/MLCommons/code",
105105
"Unique ID (e.g. for Audit)": "edge/open/MLCommons/default-mlcommons_cpp-cpu-onnxruntime-default_config/retinanet",
106106
"ColorKey": "availableMLCommons",
107-
"Performance_Result": 0.421782,
107+
"Performance_Result": 0.417492,
108108
"Performance_Units": "Samples/s"
109109
},
110110
{

0 commit comments

Comments
 (0)