Skip to content

Conversation

mickqian
Copy link
Collaborator

@mickqian mickqian commented Oct 5, 2025

Motivation

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

@mickqian
Copy link
Collaborator Author

mickqian commented Oct 5, 2025

Guide for adding models to nightly-ci

  • for text models: extend global model lists variables prefixed with DEFAULT_MODEL_NAME_FOR_NIGHTLY_EVAL_ in test_utils.py, or add more model lists
  • for vlms: extend global variables containing ModelLaunchSettings in test_nightly_vlms_.*.py, see here

@mickqian mickqian force-pushed the nightly-ci branch 2 times, most recently from 687b640 to db90b57 Compare October 5, 2025 11:19
@mickqian mickqian marked this pull request as ready for review October 6, 2025 04:12
Comment on lines 65 to 66
"zai-org/GLM-4.1V-9B-Thinking", extra_args=VLM_EXTRA_ARGS
): ModelEvalMetrics(0.280, 30.4),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use

VLMModelDeploySetup = partial(ModelLaunchSettings, extra_args=VLM_EXTRA_ARGS)
VLMModelLaunchSettings("zai-org/GLM-4.1V-9B-Thinking"): ModelEvalMetrics(0.280, 30.4),

to avoid code duplication

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants