Skip to content

Commit aa502f5

Browse files
gateoverflow-admingateoverflow-admin
committed
Updated results summary
1 parent 76416d7 commit aa502f5

File tree

6 files changed

+17
-17
lines changed

6 files changed

+17
-17
lines changed

‎README.md‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,20 @@
11
Please download [summary.xlsx](summary.xlsx) to view the most recent results.
22
```
3-
[2025-01-07 05:42:28,043 submission_checker1.py:3183 INFO] Results=38, NoResults=0, Power Results=0
4-
[2025-01-07 05:42:28,043 submission_checker1.py:3190 INFO] ---
5-
[2025-01-07 05:42:28,043 submission_checker1.py:3191 INFO] Closed Results=2, Closed Power Results=0
3+
[2025-01-07 17:39:49,122 submission_checker1.py:3183 INFO] Results=38, NoResults=0, Power Results=0
4+
[2025-01-07 17:39:49,122 submission_checker1.py:3190 INFO] ---
5+
[2025-01-07 17:39:49,122 submission_checker1.py:3191 INFO] Closed Results=2, Closed Power Results=0
66
7-
[2025-01-07 05:42:28,043 submission_checker1.py:3196 INFO] Open Results=36, Open Power Results=0
7+
[2025-01-07 17:39:49,123 submission_checker1.py:3196 INFO] Open Results=36, Open Power Results=0
88
9-
[2025-01-07 05:42:28,043 submission_checker1.py:3201 INFO] Network Results=0, Network Power Results=0
9+
[2025-01-07 17:39:49,123 submission_checker1.py:3201 INFO] Network Results=0, Network Power Results=0
1010
11-
[2025-01-07 05:42:28,043 submission_checker1.py:3206 INFO] ---
12-
[2025-01-07 05:42:28,044 submission_checker1.py:3208 INFO] Systems=28, Power Systems=0
13-
[2025-01-07 05:42:28,044 submission_checker1.py:3212 INFO] Closed Systems=1, Closed Power Systems=0
14-
[2025-01-07 05:42:28,044 submission_checker1.py:3217 INFO] Open Systems=28, Open Power Systems=0
15-
[2025-01-07 05:42:28,044 submission_checker1.py:3222 INFO] Network Systems=0, Network Power Systems=0
16-
[2025-01-07 05:42:28,044 submission_checker1.py:3227 INFO] ---
17-
[2025-01-07 05:42:28,044 submission_checker1.py:3232 INFO] SUMMARY: submission looks OK
11+
[2025-01-07 17:39:49,123 submission_checker1.py:3206 INFO] ---
12+
[2025-01-07 17:39:49,123 submission_checker1.py:3208 INFO] Systems=28, Power Systems=0
13+
[2025-01-07 17:39:49,123 submission_checker1.py:3212 INFO] Closed Systems=1, Closed Power Systems=0
14+
[2025-01-07 17:39:49,123 submission_checker1.py:3217 INFO] Open Systems=28, Open Power Systems=0
15+
[2025-01-07 17:39:49,123 submission_checker1.py:3222 INFO] Network Systems=0, Network Power Systems=0
16+
[2025-01-07 17:39:49,123 submission_checker1.py:3227 INFO] ---
17+
[2025-01-07 17:39:49,123 submission_checker1.py:3232 INFO] SUMMARY: submission looks OK
1818
INFO:root: ! call "postprocess" from /home/runner/CM/repos/mlcommons@mlperf-automations/script/run-mlperf-inference-submission-checker/customize.py
1919
2020
```
@@ -52,7 +52,7 @@ INFO:root: ! call "postprocess" from /home/runner/CM/repos/mlcommons@mlper
5252
| 28 | MLCommons | available | open | edge | gh_windows-latest | gh_windows-latest-reference-cpu-pytorch_v2.4.1-default_config | retinanet | retinanet | Offline | 0.327316 | mAP: 76.951 | 1 | undefined | 1 | 1 | nan | 0 | open/MLCommons/results/gh_windows-latest-reference-cpu-pytorch_v2.4.1-default_config/retinanet/offline | pytorch v2.4.1 | Windows-2022Server-10.0.20348-SP0 | Automated by MLCommons CM v3.2.3. | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
5353
| 29 | MLCommons | available | open | edge | gh_action | gh_action-reference-gpu-pytorch_v2.4.1-default_config | gptj-99 | gptj-99 | Offline | 50.3446 | ROUGE1: 32.2581 ROUGE2: 6.6667 ROUGEL: 22.5806 GEN_LEN: 264 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 1 | open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.4.1-default_config/gptj-99/offline | pytorch v2.4.1 | Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35) | Automated by MLCommons CM v2.3.4. | 1 | 0 | v5.0 | 0 | False | Tokens/s | fp32 |
5454
| 30 | MLCommons | available | open | edge | gh_action | gh_action-reference-gpu-pytorch_v2.5.1-cu124 | gptj-99 | gptj-99 | Offline | 48.2391 | GEN_LEN: 264 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/gptj-99/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-6.8.0-49-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Tokens/s | fp32 |
55-
| 31 | MLCommons | available | open | edge | gh_action | gh_action-reference-gpu-pytorch_v2.5.1-cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.35248 | nan | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/stable-diffusion-xl/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-6.8.0-49-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
55+
| 31 | MLCommons | available | open | edge | gh_action | gh_action-reference-gpu-pytorch_v2.5.1-cu124 | stable-diffusion-xl | stable-diffusion-xl | Offline | 0.352373 | nan | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/stable-diffusion-xl/offline | pytorch v2.5.1 | Ubuntu 22.04 (linux-6.8.0-49-generic-glibc2.35) | nan | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |
5656
| 32 | MLCommons | available | open | datacenter | gh_ubuntu_x86 | gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config | resnet50 | resnet | Server | 75016.1 | acc: 76.078 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config/resnet50/server | TensorRT | Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) | Automated by MLCommons CM v3.2.4. | 1 | 0 | v5.0 | 0 | False | Queries/s | int8 |
5757
| 33 | MLCommons | available | open | datacenter | gh_ubuntu_x86 | gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config | resnet50 | resnet | Offline | 80229.7 | acc: 76.078 | 1 | Intel(R) Xeon(R) w7-2495X | 1 | 24 | NVIDIA GeForce RTX 4090 | 2 | open/MLCommons/results/gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config/resnet50/offline | TensorRT | Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31) | Automated by MLCommons CM v3.2.4. | 1 | 0 | v5.0 | 0 | False | Samples/s | int8 |
5858
| 34 | MLCommons | available | open | edge | gh_macos-latest_x86 | gh_macos-latest_x86-reference-cpu-pytorch_v2.5.0-default_config | retinanet | retinanet | Offline | 0.277839 | mAP: 76.951 | 1 | undefined | 1 | 3 | nan | 0 | open/MLCommons/results/gh_macos-latest_x86-reference-cpu-pytorch_v2.5.0-default_config/retinanet/offline | pytorch v2.5.0 | (darwin-23.6.0) | Automated by MLCommons CM v3.2.6. | 1 | 0 | v5.0 | 0 | False | Samples/s | fp32 |

‎dbversion‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
302
1+
303

‎summary.csv‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ MLCommons,available,open,"edge","gh_action",gh_action-reference-gpu-pytorch_v2.4
3030
MLCommons,available,open,"edge","gh_windows-latest",gh_windows-latest-reference-cpu-pytorch_v2.4.1-default_config,retinanet,retinanet,Offline,0.327316,mAP: 76.951,1,"undefined",1,1,"N/A","0",open/MLCommons/results/gh_windows-latest-reference-cpu-pytorch_v2.4.1-default_config/retinanet/offline,"pytorch v2.4.1","Windows-2022Server-10.0.20348-SP0","Automated by MLCommons CM v3.2.3. ",1,0,v5.0,0,False,Samples/s,"fp32"
3131
MLCommons,available,open,"edge","gh_action",gh_action-reference-gpu-pytorch_v2.4.1-default_config,gptj-99,gptj-99,Offline,50.3446,ROUGE1: 32.2581 ROUGE2: 6.6667 ROUGEL: 22.5806 GEN_LEN: 264,1,"Intel(R) Xeon(R) w7-2495X",1,24,"NVIDIA GeForce RTX 4090","1",open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.4.1-default_config/gptj-99/offline,"pytorch v2.4.1","Ubuntu 22.04 (linux-6.2.0-39-generic-glibc2.35)","Automated by MLCommons CM v2.3.4. ",1,0,v5.0,0,False,Tokens/s,"fp32"
3232
MLCommons,available,open,"edge","gh_action",gh_action-reference-gpu-pytorch_v2.5.1-cu124,gptj-99,gptj-99,Offline,48.2391,GEN_LEN: 264,1,"Intel(R) Xeon(R) w7-2495X",1,24,"NVIDIA GeForce RTX 4090","2",open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/gptj-99/offline,"pytorch v2.5.1","Ubuntu 22.04 (linux-6.8.0-49-generic-glibc2.35)","",1,0,v5.0,0,False,Tokens/s,"fp32"
33-
MLCommons,available,open,"edge","gh_action",gh_action-reference-gpu-pytorch_v2.5.1-cu124,stable-diffusion-xl,stable-diffusion-xl,Offline,0.35248,,1,"Intel(R) Xeon(R) w7-2495X",1,24,"NVIDIA GeForce RTX 4090","2",open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/stable-diffusion-xl/offline,"pytorch v2.5.1","Ubuntu 22.04 (linux-6.8.0-49-generic-glibc2.35)","",1,0,v5.0,0,False,Samples/s,"fp32"
33+
MLCommons,available,open,"edge","gh_action",gh_action-reference-gpu-pytorch_v2.5.1-cu124,stable-diffusion-xl,stable-diffusion-xl,Offline,0.352373,,1,"Intel(R) Xeon(R) w7-2495X",1,24,"NVIDIA GeForce RTX 4090","2",open/MLCommons/results/gh_action-reference-gpu-pytorch_v2.5.1-cu124/stable-diffusion-xl/offline,"pytorch v2.5.1","Ubuntu 22.04 (linux-6.8.0-49-generic-glibc2.35)","",1,0,v5.0,0,False,Samples/s,"fp32"
3434
MLCommons,available,open,"datacenter","gh_ubuntu_x86",gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config,resnet50,resnet,Server,75016.1,acc: 76.078,1,"Intel(R) Xeon(R) w7-2495X",1,24,"NVIDIA GeForce RTX 4090","2",open/MLCommons/results/gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config/resnet50/server,"TensorRT","Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31)","Automated by MLCommons CM v3.2.4. ",1,0,v5.0,0,False,Queries/s,"int8"
3535
MLCommons,available,open,"datacenter","gh_ubuntu_x86",gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config,resnet50,resnet,Offline,80229.7,acc: 76.078,1,"Intel(R) Xeon(R) w7-2495X",1,24,"NVIDIA GeForce RTX 4090","2",open/MLCommons/results/gh_ubuntu_x86-nvidia-gpu-TensorRT-default_config/resnet50/offline,"TensorRT","Ubuntu 20.04 (linux-6.2.0-39-generic-glibc2.31)","Automated by MLCommons CM v3.2.4. ",1,0,v5.0,0,False,Samples/s,"int8"
3636
MLCommons,available,open,"edge","gh_macos-latest_x86",gh_macos-latest_x86-reference-cpu-pytorch_v2.5.0-default_config,retinanet,retinanet,Offline,0.277839,mAP: 76.951,1,"undefined",1,3,"N/A","0",open/MLCommons/results/gh_macos-latest_x86-reference-cpu-pytorch_v2.5.0-default_config/retinanet/offline,"pytorch v2.5.0"," (darwin-23.6.0)","Automated by MLCommons CM v3.2.6. ",1,0,v5.0,0,False,Samples/s,"fp32"

‎summary.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -939,7 +939,7 @@
939939
"Model":"stable-diffusion-xl",
940940
"MlperfModel":"stable-diffusion-xl",
941941
"Scenario":"Offline",
942-
"Result":0.35248,
942+
"Result":0.352373,
943943
"Accuracy":null,
944944
"number_of_nodes":1,
945945
"host_processor_model_name":"Intel(R) Xeon(R) w7-2495X",

‎summary.xlsx‎

2 Bytes
Binary file not shown.

‎summary_results.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -608,7 +608,7 @@
608608
"Code": "https://github.com/mlcommons/mlperf_inference_test_submissions_v5.0/tree/auto-update/open/MLCommons/code",
609609
"Unique ID (e.g. for Audit)": "edge/open/MLCommons/gh_action-reference-gpu-pytorch_v2.5.1-cu124/stable-diffusion-xl",
610610
"ColorKey": "availableMLCommons",
611-
"Performance_Result": 0.35248,
611+
"Performance_Result": 0.352373,
612612
"Performance_Units": "Samples/s"
613613
},
614614
{

0 commit comments

Comments
 (0)