Skip to content

vllm.models.kimi_k3.amd.specialized_decode

Kimi-K3 specialized decode for ROCm (MI355X, TP8), layer mode: FlyDSL MonoKernels per MoE decoder layer on pure spec-verify steps, inside the stock layer loop. Enabled by additional_config={"decode_backend": "latency_specialized"}.

hooks.py holds the model-side hooks (SpecializedDecodeMixin) and model_state.py the model runner state that builds the executor. runtime/ is the host side: the config check (runtime/__init__.py), the executor (runtime/decode.py), the layer kernels, the per-rank launch state and the later refusals. mono/ is the kernel source. Tests live in tests/models/kimi_k3/amd_specialized_decode/.

Modules:

  • hooks –

    The specialized decode's hooks on KimiLinearForCausalLM, kept here so the

  • model_state –

    Kimi-K3's model runner state on ROCm: the hybrid (KDA + MLA) state, plus the

  • mono –
  • runtime –

    Kimi-K3 specialized decode on MI355X (layer mode), enabled by