Vz/image count limit

[vincentzed]

Motivation

Modifications

Accuracy Test

Benchmark & Profiling

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[cubic-dev-ai]

cubic analysis

1 issue found across 2 files • Review in cubic

_{React with 👍 or 👎 to teach cubic. You can also tag @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[cubic-dev-ai]

Naming mismatch between dataclass field max_num_image and CLI dest max_num_images will raise AttributeError during ServerArgs.from_cli_args.

Prompt for AI agents

Address the following comment on python/sglang/srt/server_args.py at line 273:

<comment>Naming mismatch between dataclass field `max_num_image` and CLI dest `max_num_images` will raise AttributeError during `ServerArgs.from_cli_args`.</comment>

<file context>
@@ -269,6 +269,9 @@ class ServerArgs:
     enable_pdmux: bool = False
     sm_group_num: int = 3
 
+    # Multimodal settings
+    max_num_image: Optional[int] = None
+
     def __post_init__(self):
</file context>

Suggested change

	max_num_image: Optional[int] = None
	max_num_images: Optional[int] = None

[cubic-dev-ai]

cubic analysis

2 issues found across 2 files • Review in cubic

_{React with 👍 or 👎 to teach cubic. You can also tag @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[cubic-dev-ai]

Comparing num_images to self.IMAGE_NUM_LIMITATION will raise TypeError when the limit is not set (None). Guard the comparison with a None check.

Prompt for AI agents

Address the following comment on python/sglang/srt/multimodal/processors/base_processor.py at line 257:

<comment>Comparing num_images to self.IMAGE_NUM_LIMITATION will raise TypeError when the limit is not set (None). Guard the comparison with a None check.</comment>

<file context>
@@ -249,8 +251,15 @@ def get_estimated_frames_list(self, image_data):
         from decord import VideoReader, cpu
 
         # Before processing inputs
-        if not image_data or len(image_data) == 0:
+        num_images = len(image_data) if image_data else 0
+        if not image_data or num_images == 0:
             return []
+        if num_images &gt; self.IMAGE_NUM_LIMITATION:
+            raise ValueError(
</file context>

Suggested change

	if num_images > self.IMAGE_NUM_LIMITATION:
	if self.IMAGE_NUM_LIMITATION is not None and num_images > self.IMAGE_NUM_LIMITATION:

[cubic-dev-ai]

The CLI option produces an argparse destination name (max_num_images) that does not match the dataclass field max_num_image, causing ServerArgs.from_cli_args to raise AttributeError at runtime.

Prompt for AI agents

Address the following comment on python/sglang/srt/server_args.py at line 1817:

<comment>The CLI option produces an argparse destination name (`max_num_images`) that does not match the dataclass field `max_num_image`, causing `ServerArgs.from_cli_args` to raise `AttributeError` at runtime.</comment>

<file context>
@@ -1810,6 +1813,12 @@ def add_cli_args(parser: argparse.ArgumentParser):
             action=&quot;store_true&quot;,
             help=&quot;Disable mmap while loading weight using safetensors.&quot;,
         )
+        parser.add_argument(
+            &quot;--max-num-images&quot;,
+            type=int,
+            default=None,
</file context>

Suggested change

	"--max-num-images",
	"--max-num-images", dest="max_num_image",

[cubic-dev-ai]

@cubic-dev-ai please re-review

@vincentzed I've started the AI code review. It'll take a few minutes to complete.

[cubic-dev-ai]

@cubic-dev-ai please re-review

@vincentzed I've started the AI code review. It'll take a few minutes to complete.

[cubic-dev-ai]

cubic analysis

No issues found across 3 files. Review in cubic

[cubic-dev-ai]

cubic analysis

1 issue found across 3 files • Review in cubic

_{React with 👍 or 👎 to teach cubic. You can also tag @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[cubic-dev-ai]

Raising ValueError here causes the client to receive a generic 500 InternalServerError instead of a clear 4xx validation response; raise HTTPException(400) or return a validation message instead

Prompt for AI agents

Address the following comment on python/sglang/srt/entrypoints/openai/serving_chat.py at line 80:

<comment>Raising ValueError here causes the client to receive a generic 500 InternalServerError instead of a clear 4xx validation response; raise HTTPException(400) or return a validation message instead</comment>

<file context>
@@ -69,6 +69,20 @@ def _validate_request(self, request: ChatCompletionRequest) -&gt; Optional[str]:
 
         return None
 
+    def _validate_image_count(self, image_data: List[Any]) -&gt; None:
+        &quot;&quot;&quot;Validate that the number of images doesn&#39;t exceed the configured limit&quot;&quot;&quot;
+        max_num_images = self.tokenizer_manager.server_args.max_num_images
+        if max_num_images is None:
+            return
+        
</file context>

[vincentzed]

@cubic-dev-ai review the new function more in depth

[cubic-dev-ai]

@cubic-dev-ai review the new function more in depth

@vincentzed I've started the AI code review. It'll take a few minutes to complete.

[cubic-dev-ai]

cubic analysis

No issues found across 2 files. Review in cubic