Skip to content

cleanup BaseAttentionCache #2401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

cleanup BaseAttentionCache #2401

wants to merge 7 commits into from

Conversation

lisaliu1
Copy link
Contributor

@lisaliu1 lisaliu1 commented Oct 1, 2025 •
edited
Loading

This PR completes the following work.

  • clean up BaseAttentionCache APIs
  • remove cache related functions and attributes from LlmInferenceExecRequest
  • add APIs in decoder to organize the cache across different requests and beams.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 1, 2025

Coverage report

This PR does not seem to contain any modification to coverable code.

@lisaliu1 lisaliu1 marked this pull request as ready for review October 1, 2025 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@stbaione stbaione Awaiting requested review from stbaione

@zeeshanhaque21 zeeshanhaque21 Awaiting requested review from zeeshanhaque21

@rsuderman rsuderman Awaiting requested review from rsuderman

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@lisaliu1