Skip to content

[V1][Spec Decode][Perf] Add fused Triton kernel to reduce overhead in EAGLE spec decoding #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 35 commits into from
Closed

[V1][Spec Decode][Perf] Add fused Triton kernel to reduce overhead in EAGLE spec decoding #4

wants to merge 35 commits into from

Conversation

@leo-cf-tian
Copy link

@leo-cf-tian leo-cf-tian commented May 15, 2025

Parallel PR to vllm-project#18221

bnellnm and others added 30 commits May 14, 2025 13:11
@bnellnm
Modularize fused experts and integrate PPLX kernels (vllm-project#15956)
f9c069c
@robertgshaw2-redhat
[CI] Disable Failing Tests (vllm-project#18165)
8568650
@davidxia @aarnphm
[Frontend] decrease import time of vllm.multimodal (vllm-project#18031)
749f792 
Co-authored-by: Aaron Pham <[email protected]>
@LucasWilkinson
[Kernel] Have rotary embeddings support tensors (vllm-project#18046)
d93c976 
Signed-off-by: Lucas Wilkinson <[email protected]>
@aarnphm @russellb
[V1] Structured Outputs + Thinking compatibility (vllm-project#16577)
2fc9075 
Signed-off-by: Aaron Pham <[email protected]>
Co-authored-by: Russell Bryant <[email protected]>
@jerryzh168
Add support for loading torchao models with AOPerModuleConfig (vllm…
7974736 
…-project#17826)

Signed-off-by: Jerry Zhang <[email protected]>
@russellb
[CI] Fix race condition in test_kv_cache_events test (vllm-project#18169
78aa341 
)

Signed-off-by: Russell Bryant <[email protected]>
@mgoin @njhill
[V1] Support multiple kv connectors (vllm-project#17564)
2142035 
Signed-off-by: mgoin <[email protected]>
Signed-off-by: Nick Hill <[email protected]>
Co-authored-by: Nick Hill <[email protected]>
@atalman
Upload vllm index for the rc builds (vllm-project#18173)
09f106a
@davidxia
[Bugfix]: make most of test_openai_schema.py pass (vllm-project#17664)
f25e0d1
@heheda12345
[v1] Support multiple KV cache groups in GPU model runner (vllm-proje…
e60f550 
…ct#17945)

Signed-off-by: Chen Zhang <[email protected]>
@markmc
[V1][Metrics] Remove unused code (vllm-project#18158)
65334ef 
Signed-off-by: Mark McLoughlin <[email protected]>
@aarnphm
[Chore] astral's ty (vllm-project#18116)
afe3236 
Signed-off-by: Aaron Pham <[email protected]>
@reidliu41
[Misc] add lobe-chat support (vllm-project#18177)
2dff093 
Signed-off-by: reidliu41 <[email protected]>
Co-authored-by: reidliu41 <[email protected]>
@ProExpertProg
[Fix][ROCm] Enforce eager for all encoder-decoder models on ROCm (vll…
83f74c6 
…m-project#18154)

Signed-off-by: Luka Govedič <[email protected]>
@hmellor
Update deprecated type hinting in models (vllm-project#18132)
26d0419 
Signed-off-by: Harry Mellor <[email protected]>
@tdoublep @LucasWilkinson
[Bugfix] Fix fp8 tests for triton_unified_attention for Triton 3.3 (v…
e6b8e65 
…llm-project#18013)

Signed-off-by: Thomas Parnell <[email protected]>
Co-authored-by: Lucas Wilkinson <[email protected]>
@huachenheli
Support custom implementations of VideoLoader backends. (vllm-project…
4f07a64 
…#18091)
@andyxning
[UT] Add ut for none hash (vllm-project#17892)
420caf7 
Signed-off-by: Andy Xie <[email protected]>
@inkcherry
[Model] Allow the use of sliding window in Qwen2 (vllm-project#17772)
dd2a945 
Signed-off-by: inkcherry <[email protected]>
@MengqingCao
[Bugfix] Fix FusedMoEPrepareAndFinalize for cuda-disalike backends (v…
70f8b96 
…llm-project#18178)

Signed-off-by: Mengqing Cao <[email protected]>
@davidxia
[CI] don't skip fixed test_kv_cache_events() (vllm-project#18183)
de71fec 
Signed-off-by: David Xia <[email protected]>
@russellb
[V1] Update zmq socket creation in nixl connector (vllm-project#18148)
a8f5aec 
Signed-off-by: Russell Bryant <[email protected]>
@omahs
fix: typos (vllm-project#18151)
a9944aa 
Signed-off-by: omahs <[email protected]>
@hmellor
Update deprecated type hinting in model_loader (vllm-project#18130)
07ad271 
Signed-off-by: Harry Mellor <[email protected]>
@hustxiayang
add tools into TokenizeChatRequest (vllm-project#18187)
451da4b 
Signed-off-by: yangxia <[email protected]>
@tdoublep @LucasWilkinson
[Kernel] [V1] Fix performance regression for triton unified attention (…
01c2233 
…vllm-project#18161)

Signed-off-by: Thomas Parnell <[email protected]>
Co-authored-by: Lucas Wilkinson <[email protected]>
@Alexei-V-Ivanov-AMD @DarkLight1337
Adding "Basic Models Test" and "Multi-Modal Models Test (Extended) 3"…
566ec04 
… in AMD Pipeline (vllm-project#18106)

Signed-off-by: Alexei V. Ivanov <[email protected]>
Co-authored-by: Cyrus Leung <[email protected]>
@hmellor
Improve examples rendering in docs and GitHub (vllm-project#18203)
51ff154 
Signed-off-by: Harry Mellor <[email protected]>
@schoennenbeck
[Frontend] Fix chat template content format detection (vllm-project#1…
2aa5470 
…8190)

Signed-off-by: Sebastian Schönnenbeck <[email protected]>
@github-actions
Copy link

github-actions bot commented May 15, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.