Skip to content

[BUG] Negative bias is not allowed in DeepSeek routing for TRT LLM MoE #2168

@divchenko

Description

@divchenko

GLM uses DeepSeek-style router w/ num groups = 1 and topk groups = 1. It has negative bias, which breaks current routing logic (it has non-negativity assumption). Example fix is here: fw-ai/flashinfer#8

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions