Skip to content

Commit 38ab226

Browse files
authored
Merge pull request #436 from mmarcinkiewicz/michalm/fix-rcp-check-before-v5.1
DRAFT: Fix submission checkers before v5.1
2 parents 54cb0bb + e5e1b55 commit 38ab226

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

mlperf_logging/rcp_checker/rcp_checker.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,7 @@ def read_submission_file(result_file, ruleset, use_train_samples):
8484
eval_metric = json.loads(eval_accuracy_str)["metadata"]["metric"]
8585
eval_score = json.loads(eval_accuracy_str)["value"]
8686
stable_diffusion_eval_results[eval_step][eval_metric] = eval_score
87-
elif benchmark in {"llama2_70b_lora", "flux1", "llama31_405b"} and ("eval_error" in str or "eval_accuracy" in str):
87+
elif benchmark in {"llama2_70b_lora", "flux1", "llama31_405b", "llama31_8b"} and ("eval_error" in str or "eval_accuracy" in str):
8888
eval_accuracy_str = str
8989
conv_epoch = json.loads(eval_accuracy_str)["metadata"]["samples_count"]
9090
eval_score = json.loads(eval_accuracy_str)["value"]

0 commit comments

Comments
 (0)