Skip to content

vllm.models.kimi_k3.nvidia.kv_balanced_mla

Kimi K3 KV-balanced MLA decode: TokenSpeed's FP8 MLA decode kernel with each request's KV split planned on the device so that every cluster gets about the same KV, as the KIMI_K3_KV_BALANCED_MLA attention backend (backend.py).

Modules:

  • backend –

    KIMI_K3_KV_BALANCED_MLA: an FP8 MLA decode backend on the KV-balanced kernel.

  • core –

    FP8 MLA decode over the paged FP8 cache with a device-planned split-KV work list.

  • decode –

    The decode of the KIMI_K3_KV_BALANCED_MLA attention backend (backend.py) on

  • helpers –
  • kernel –