Skip to content

Commit 072d88d

Browse files
Add README + format script
1 parent b917d71 commit 072d88d

File tree

2 files changed

+43
-4
lines changed

2 files changed

+43
-4
lines changed
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# MLPerf compute standalone score
2+
3+
MLPerf compute standalone score
4+
5+
## Usage
6+
7+
To compute the scores of a single benchmark. All the results files are assumed to be in the same folder:
8+
9+
```sh
10+
python3 -m mlperf_logging.result_summarizer.compute_score --benchmark BENCHMARK \
11+
--system SYSTEM_NAME --benchmark_folder BENCHMARK_FOLDER --usage USAGE --ruleset RULESET \
12+
[--is_weak_scaling] [--scale] [--has_power]
13+
```
14+
15+
16+
**BENCHMARK:** Name of the benchmark to compute the score such as rgat, llama31_8b, etc.
17+
**SYSTEM_NAME:** The name of the system, it can be set to None.
18+
**BENCHMARK_FOLDER:** Folder containing all the results files of the benchmark.
19+
**USAGE:** Either "training" or "hpc",
20+
**RULESET:** Version of the rules that applies one of "1.0.0", "1.1.0", "2.0.0", "2.1.0", "3.0.0", "3.1.0", "4.0.0", "4.1.0", "5.0.0", "5.1.0".
21+
**[--is_weak_scaling]:** Is the benchmark weak scaling (only applies to HPC).
22+
**[--scale]:** Compute the scaling.json file (only if the folder does not contain it already).
23+
**[--has_power]:** Have the results power measurements .
24+
25+
26+
27+
## Tested software versions
28+
Tested and confirmed working using the following software versions:
29+
30+
Python 3.9.18

mlperf_logging/result_summarizer/compute_score/__main__.py

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,13 +9,21 @@ def get_compute_args():
99
prog="mlperf_logging.result_summarizer.compute_score",
1010
description="Compute the score of a single benchmark",
1111
)
12-
parser.add_argument("--benchmark", type=str, help="TODO:", required=True)
12+
parser.add_argument(
13+
"--benchmark",
14+
type=str,
15+
help="Benchmark to compute the score such as rgat, llama31_8b, etc.",
16+
required=True,
17+
)
1318
parser.add_argument("--system", type=str, help="System name", default=None)
1419
parser.add_argument(
1520
"--has_power", action="store_true", help="Compute power score as well"
1621
)
1722
parser.add_argument(
18-
"--benchmark_folder", type=str, help="Folder containing all the result files", required=True
23+
"--benchmark_folder",
24+
type=str,
25+
help="Folder containing all the result files",
26+
required=True,
1927
)
2028
parser.add_argument(
2129
"--usage",
@@ -49,17 +57,18 @@ def print_benchmark_info(args):
4957
print(f"System: {args.system}")
5058
print(f"Benchmark: {args.benchmark}")
5159

60+
5261
args = get_compute_args()
5362

5463
if args.scale:
5564
rcp_checker.check_directory(
56-
args.benchmark_folder,
65+
args.benchmark_folder,
5766
args.usage,
5867
args.ruleset,
5968
False,
6069
False,
6170
rcp_file=None,
62-
rcp_pass='pruned_rcps',
71+
rcp_pass="pruned_rcps",
6372
rcp_bypass=False,
6473
set_scaling=True,
6574
)

0 commit comments

Comments
 (0)