Skip to content

Commit 0cb382a

Browse files
mlcommons-botmlcommons-bot
committed
Auto-merge updates from auto-update branch
2 parents 3cc8bfe + 0301251 commit 0cb382a

File tree

14 files changed

+236
-236
lines changed

14 files changed

+236
-236
lines changed

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
| Model | Scenario | Accuracy | Throughput | Latency (in ms) |
22
|----------|------------|------------|--------------|-------------------|
3-
| resnet50 | offline | 76 | 15.773 | - |
3+
| resnet50 | offline | 76 | 20.579 | - |

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ pip install -U mlcflow
1616

1717
mlc rm cache -f
1818

19-
mlc pull repo mlcommons@mlperf-automations --checkout=5d633b6c3c768c0cf89d801aeff49a90435c25c5
19+
mlc pull repo mlcommons@mlperf-automations --checkout=02b832a68f9239e713b901e3ea6a4248e2073eb5
2020

2121

2222
```
@@ -40,4 +40,4 @@ Model Precision: fp32
4040
`acc`: `76.0`, Required accuracy for closed division `>= 75.6954`
4141

4242
### Performance Results
43-
`Samples per second`: `15.7727`
43+
`Samples per second`: `20.5794`

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/accuracy_console.out‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
1-
python3 python/main.py --profile resnet50-onnxruntime --model "/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_0dc193d9/resnet50_v1.onnx" --dataset-path /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0 --output "/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2f702b85/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 3 --user_conf /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/6e3406c5e7fa40f78cb7b2c80c81068b.conf --accuracy --use_preprocessed_dataset --cache_dir /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0 --dataset-list /Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_35f77049/val.txt
2-
INFO:main:Namespace(dataset='imagenet', dataset_path='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0', dataset_list='/Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_35f77049/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_0dc193d9/resnet50_v1.onnx', output='/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2f702b85/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=3, qps=None, cache=0, cache_dir='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/6e3406c5e7fa40f78cb7b2c80c81068b.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8)
1+
python3 python/main.py --profile resnet50-onnxruntime --model "/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_4296fd39/resnet50_v1.onnx" --dataset-path /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56 --output "/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_86b024b4/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/accuracy" --scenario Offline --count 500 --threads 3 --user_conf /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/ebed675f99424f229cac9af87ee3dde6.conf --accuracy --use_preprocessed_dataset --cache_dir /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56 --dataset-list /Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_9b59c6b6/val.txt
2+
INFO:main:Namespace(dataset='imagenet', dataset_path='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56', dataset_list='/Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_9b59c6b6/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_4296fd39/resnet50_v1.onnx', output='/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_86b024b4/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/accuracy', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=3, qps=None, cache=0, cache_dir='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=True, find_peak_performance=False, debug=False, user_conf='/Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/ebed675f99424f229cac9af87ee3dde6.conf', audit_conf='audit.config', time=None, count=500, performance_sample_count=None, max_latency=None, samples_per_query=8)
33
INFO:imagenet:Loading 500 preprocessed images using 3 threads
44
INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=0.0sec
55
INFO:main:starting TestScenario.Offline
6-
TestScenario.Offline qps=0.56, mean=16.5269, time=28.548, acc=76.000%, queries=16, tiles=50.0:16.1568,80.0:25.6048,90.0:25.8762,95.0:26.6198,99.0:27.9656,99.9:28.2684
6+
TestScenario.Offline qps=0.72, mean=12.8767, time=22.343, acc=76.000%, queries=16, tiles=50.0:12.6992,80.0:19.8876,90.0:20.2367,95.0:20.8748,99.0:21.9924,99.9:22.2438

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/mlc-deps.mmd‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,9 @@ graph TD
1515
get-generic-sys-util,bb0393afa8404a11_(_md5sha1sum_) --> get,generic-sys-util,_install,_md5sha1sum
1616
download-file,9cdc8dc41aae437e_(_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> get,generic-sys-util,_md5sha1sum
1717
download-and-extract,c67e81a4ce2649f5_(_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> download,file,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz
18-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz_) --> detect,os
19-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz_) --> get,generic-sys-util,_md5sha1sum
20-
download-and-extract,c67e81a4ce2649f5_(_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz
18+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz_) --> detect,os
19+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz_) --> get,generic-sys-util,_md5sha1sum
20+
download-and-extract,c67e81a4ce2649f5_(_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz
2121
get-dataset-imagenet-aux,bb2c6dd8c8c64217 --> download-and-extract,_extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz
2222
app-mlperf-inference,d775cac873ee4231_(_python,_resnet50,_onnxruntime,_cpu,_test,_r5.1-dev_default,_offline_) --> get,dataset-aux,imagenet-aux
2323
app-mlperf-inference-mlcommons-python,ff149e9781fc4b65_(_cpu,_onnxruntime,_offline,_resnet50,_fp32_) --> detect,os
@@ -45,9 +45,9 @@ graph TD
4545
download-file,9cdc8dc41aae437e_(_cmutil,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> detect,os
4646
download-file,9cdc8dc41aae437e_(_cmutil,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> get,generic-sys-util,_md5sha1sum
4747
download-and-extract,c67e81a4ce2649f5_(_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> download,file,_cmutil,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar
48-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar_) --> detect,os
49-
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar_) --> get,generic-sys-util,_md5sha1sum
50-
download-and-extract,c67e81a4ce2649f5_(_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar
48+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar_) --> detect,os
49+
extract-file,3f0b76219d004817_(_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar_) --> get,generic-sys-util,_md5sha1sum
50+
download-and-extract,c67e81a4ce2649f5_(_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar_) --> extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar
5151
get-dataset-imagenet-val,7afd58d287fe4f11_(_size.500,_2012_) --> download-and-extract,file,_extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar
5252
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_-for.mobilenet,_NCHW_) --> get,dataset,image-classification,original,_size.500,_2012
5353
get-preprocessed-dataset-imagenet,f259d490bbaf45f5_(_-for.mobilenet,_NCHW_) --> get,dataset-aux,image-classification,imagenet-aux

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/mlc-deps.png‎

-937 Bytes
Loading

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/mlc-version-info.json‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -167,7 +167,7 @@
167167
"script_tags": "detect-os,detect,os,info",
168168
"script_variations": "",
169169
"version": "",
170-
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz )"
170+
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz )"
171171
}
172172
},
173173
{
@@ -177,15 +177,15 @@
177177
"script_tags": "get,sys-util,generic,generic-sys-util",
178178
"script_variations": "md5sha1sum",
179179
"version": "0.9.5",
180-
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz )"
180+
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz )"
181181
}
182182
},
183183
{
184-
"extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz": {
184+
"extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz": {
185185
"script_uid": "3f0b76219d004817",
186186
"script_alias": "extract-file",
187187
"script_tags": "extract,file",
188-
"script_variations": "keep,path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_6a06435c/caffe_ilsvrc12.tar.gz",
188+
"script_variations": "keep,path./Users/runner/MLC/repos/local/cache/download-file_imagenet-aux-da_63ac3373/caffe_ilsvrc12.tar.gz",
189189
"version": "",
190190
"parent": "download-and-extract,c67e81a4ce2649f5 ( extract,_wget,_url.https://armi.in/files/caffe_ilsvrc12.tar.gz )"
191191
}
@@ -467,7 +467,7 @@
467467
"script_tags": "detect-os,detect,os,info",
468468
"script_variations": "",
469469
"version": "",
470-
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar )"
470+
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar )"
471471
}
472472
},
473473
{
@@ -477,15 +477,15 @@
477477
"script_tags": "get,sys-util,generic,generic-sys-util",
478478
"script_variations": "md5sha1sum",
479479
"version": "0.9.5",
480-
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar )"
480+
"parent": "extract-file,3f0b76219d004817 ( keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar )"
481481
}
482482
},
483483
{
484-
"extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar": {
484+
"extract,file,_keep,_path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar": {
485485
"script_uid": "3f0b76219d004817",
486486
"script_alias": "extract-file",
487487
"script_tags": "extract,file",
488-
"script_variations": "keep,path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_18a00ef7/ILSVRC2012_img_val_500.tar",
488+
"script_variations": "keep,path./Users/runner/MLC/repos/local/cache/get-dataset-imagenet-val_39de4f6b/ILSVRC2012_img_val_500.tar",
489489
"version": "",
490490
"parent": "download-and-extract,c67e81a4ce2649f5 ( extract,_url.http://cKnowledge.org/ai/data/ILSVRC2012_img_val_500.tar )"
491491
}

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/os_info.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,5 +10,5 @@
1010
"MLC_HOST_OS_PACKAGE_MANAGER_UPDATE_CMD": "brew update",
1111
"MLC_HOST_PLATFORM_FLAVOR": "arm64",
1212
"MLC_HOST_PYTHON_BITS": "64",
13-
"MLC_HOST_SYSTEM_NAME": "Mac-1757621999359.local"
13+
"MLC_HOST_SYSTEM_NAME": "sjc22-bt152-38976eac-55f9-4675-9457-7b15ec4d1ca2-2628A2CEA82D.local"
1414
}

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/performance_console.out‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
1-
python3 python/main.py --profile resnet50-onnxruntime --model "/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_0dc193d9/resnet50_v1.onnx" --dataset-path /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0 --output "/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2f702b85/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/performance/run_1" --scenario Offline --threads 3 --user_conf /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/8a8b18d2e92f4a6f8dd130bf5d34bd6c.conf --use_preprocessed_dataset --cache_dir /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0 --dataset-list /Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_35f77049/val.txt
2-
INFO:main:Namespace(dataset='imagenet', dataset_path='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0', dataset_list='/Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_35f77049/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_0dc193d9/resnet50_v1.onnx', output='/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_2f702b85/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/performance/run_1', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=3, qps=None, cache=0, cache_dir='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_b25edbc0', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=False, find_peak_performance=False, debug=False, user_conf='/Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/8a8b18d2e92f4a6f8dd130bf5d34bd6c.conf', audit_conf='audit.config', time=None, count=None, performance_sample_count=None, max_latency=None, samples_per_query=8)
1+
python3 python/main.py --profile resnet50-onnxruntime --model "/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_4296fd39/resnet50_v1.onnx" --dataset-path /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56 --output "/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_86b024b4/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/performance/run_1" --scenario Offline --threads 3 --user_conf /Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/b5697f05a1e94bbf8b5d7c980c7ce1b0.conf --use_preprocessed_dataset --cache_dir /Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56 --dataset-list /Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_9b59c6b6/val.txt
2+
INFO:main:Namespace(dataset='imagenet', dataset_path='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56', dataset_list='/Users/runner/MLC/repos/local/cache/extract-file_imagenet-aux-da_9b59c6b6/val.txt', data_format=None, profile='resnet50-onnxruntime', scenario='Offline', max_batchsize=32, model='/Users/runner/MLC/repos/local/cache/download-file_ml-model-resnet_4296fd39/resnet50_v1.onnx', output='/Users/runner/MLC/repos/local/cache/get-mlperf-inference-results-dir_86b024b4/test_results/gh_macos-latest x86-reference-cpu-onnxruntime-v1.22.1-default_config/resnet50/offline/performance/run_1', inputs=None, outputs=['ArgMax:0'], backend='onnxruntime', device=None, model_name='resnet50', threads=3, qps=None, cache=0, cache_dir='/Users/runner/MLC/repos/local/cache/get-preprocessed-dataset-imagenet_49c7ca56', preprocessed_dir=None, use_preprocessed_dataset=True, accuracy=False, find_peak_performance=False, debug=False, user_conf='/Users/runner/MLC/repos/mlcommons@mlperf-automations/script/generate-mlperf-inference-user-conf/tmp/b5697f05a1e94bbf8b5d7c980c7ce1b0.conf', audit_conf='audit.config', time=None, count=None, performance_sample_count=None, max_latency=None, samples_per_query=8)
33
INFO:imagenet:Loading 50000 preprocessed images using 3 threads
44
INFO:imagenet:reduced image list, 49500 images not found
5-
INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=1.0sec
5+
INFO:imagenet:loaded 500 images, cache=0, already_preprocessed=True, took=0.4sec
66
INFO:main:starting TestScenario.Offline
7-
TestScenario.Offline qps=0.50, mean=18.0951, time=31.879, queries=16, tiles=50.0:17.3061,80.0:26.6066,90.0:26.7166,95.0:27.3343,99.0:28.8117,99.9:29.1441
7+
TestScenario.Offline qps=0.66, mean=14.3992, time=24.390, queries=16, tiles=50.0:13.3295,80.0:21.6620,90.0:21.6966,95.0:22.1904,99.0:23.3467,99.9:23.6068

‎open/MLCommons/measurements/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/pip_freeze.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
"packaging": "25.0",
1818
"pillow": "11.3.0",
1919
"pip": "25.2",
20-
"protobuf": "6.32.0",
20+
"protobuf": "6.32.1",
2121
"psutil": "7.0.0",
2222
"pybind11": "3.0.1",
2323
"pycocotools": "2.0.10",

‎open/MLCommons/results/gh_macos-latest_x86-reference-cpu-onnxruntime_v1.22.1-default_config/resnet50/offline/accuracy/accuracy.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
accuracy=76.000%, good=380, total=500
22

3-
hash=111bffa28f36d3d1bb65c8721106e3a1c03234454b70ae4e967bd5b75c8bc6fd
3+
hash=ea91f80979e43b00f541c72d1e4929d0f7bac3c33ccb34198906e4695525fc7b

0 commit comments

Comments
 (0)