[NVIDIA] Adding default args for DSR1 on Blackwell

[kushanam]

Optimized performance on Blackwell is achieved with these defaults:
--enable-flashinfer-allreduce-fusion
--attention-backend=trtllm_mla
--enable-flashinfer-trtllm-moe
This change sets those as the default behaviors.

Update server arguments to apply DSv3 default backends for attention, allreduce fusion, and MoE

N/A

N/A

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Summary of Changes

Hello @kushanam, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the performance of DeepSeek V3 models on NVIDIA Blackwell architecture by automatically configuring optimal backend settings. It introduces default activations for trtllm_mla attention, FlashInfer TRTLLM MoE, and FlashInfer AllReduce Fusion, ensuring that DeepSeek V3 leverages these performance-critical features out-of-the-box on compatible hardware for improved efficiency.

Highlights

• DeepSeek V3 Optimization: Introduces default performance optimizations specifically for DeepSeek V3 models when running on NVIDIA Blackwell (SM100) GPUs.
• Attention Backend: Sets the attention_backend to trtllm_mla by default for DeepSeek V3 on Blackwell, if it hasn't been explicitly configured otherwise.
• FlashInfer TRTLLM MoE: Enables flashinfer_trtllm_moe by default for DeepSeek V3 models when operating on Blackwell architecture.
• FlashInfer AllReduce Fusion: Activates flashinfer_allreduce_fusion by default for DeepSeek V3 on Blackwell, provided that enable_dp_attention is not already active.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request adds default server arguments to optimize performance for DeepSeek V3 models on Blackwell GPUs. The changes correctly set the attention backend and enable FlashInfer all-reduce fusion. However, there is a critical issue with how FlashInfer TRTLLM MoE is enabled, as it uses a deprecated and non-existent attribute, which will cause a runtime error. My review includes a suggestion to fix this by using the current moe_runner_backend argument.

[gemini-code-assist]

The code attempts to use self.enable_flashinfer_trtllm_moe, which is not an attribute of the ServerArgs class. This will cause an AttributeError at runtime.

Additionally, the --enable-flashinfer-trtllm-moe command-line argument is deprecated. The recommended way to enable this feature is by setting moe_runner_backend to 'flashinfer_trtllm'. The help message for the deprecated argument states: NOTE: --enable-flashinfer-trtllm-moe is deprecated. Please set --moe-runner-backend to 'flashinfer_trtllm' instead.

The suggested change fixes the error and uses the current recommended approach. It also checks if a user has already specified a moe_runner_backend to avoid overriding their choice, which is consistent with how attention_backend is handled.

if self.moe_runner_backend == "auto":
    self.moe_runner_backend = "flashinfer_trtllm"
    logger.info(
        f"Set moe_runner_backend to 'flashinfer_trtllm' on sm100 for {model_arch}"
    )