vllm.model_executor.layers.fused_moe.router.router_factory

Functions:

• create_fused_moe_router –

  Factory function to create the appropriate FusedMoERouter subclass based on

create_fused_moe_router(top_k, global_num_experts, renormalize=True, use_grouped_topk=False, num_expert_group=None, topk_group=None, scoring_func='softmax', num_fused_shared_experts=0, shared_expert_weight=1.0, routed_scaling_factor=1.0, e_score_correction_bias=None, custom_routing_function=None, eplb_state=None, zero_expert_type=None, num_logical_experts=None, hash_indices_table=None, bias_vl=None, image_sentinel_lo=0, skip_padding=False)

Factory function to create the appropriate FusedMoERouter subclass based on the provided parameters.

The selection logic follows this priority order: 1. ZeroExpertRouter - if zero_expert_type is not None 2. CPURouter - on CPU, before scheme-specific routers 3. GroupedTopKRouter - if use_grouped_topk is True and the grouped routing cannot fall through while preserving routing and scaling semantics 4. CustomRoutingRouter - if custom_routing_function is not None 5. FusedTopKBiasRouter - if e_score_correction_bias or hash_indices_table is not None 6. AiterSharedRoutedFusedMoERouter - if num_fused_shared_experts > 0, scoring_func is softmax, and AITER shared-expert fusion is enabled 7. FusedTopKRouter - default fallback

When VLLM_MOE_ROUTING_SIMULATION_STRATEGY is set, the selected router supplies the kernel routing method and is replaced by a RoutingSimulatorRouter.

Common arguments

top_k: Number of experts to select per token global_num_experts: Total number of experts in the model renormalize: Whether to renormalize the routing weights routing_method_type: Optional explicit routing method type

Grouped topk arguments

use_grouped_topk: Whether to use grouped top-k routing num_expert_group: Number of expert groups (for grouped routing) topk_group: Top-k within each group (for grouped routing) scoring_func: Scoring function to use ("softmax" or "sigmoid") num_fused_shared_experts: Number of fused shared experts (for ROCm AITER) skip_padding: Whether to invalidate routes for padding rows

Grouped topk and fused topk bias arguments

routed_scaling_factor: Scaling factor for routed weights e_score_correction_bias: Optional bias correction for expert scores

Custom routing arguments

custom_routing_function: Optional custom routing function

EPLB arguments

eplb_state: Optional EplbLayerState, None when EPLB is disabled.

Zero expert arguments

zero_expert_type: Type of zero expert (e.g. identity). If not None, creates a ZeroExpertRouter. num_logical_experts: Number of real (non-zero) experts. Required when zero_expert_type is not None.

Hash Indices Table

Used to map input_ids to experts, need for Deepseek V4

Returns:

• FusedMoERouter –

  An instance of the appropriate FusedMoERouter subclass

Source code in vllm/model_executor/layers/fused_moe/router/router_factory.py

def create_fused_moe_router(
    # common parameters
    top_k: int,
    global_num_experts: int,
    renormalize: bool = True,
    # grouped topk parameters
    use_grouped_topk: bool = False,
    num_expert_group: int | None = None,
    topk_group: int | None = None,
    scoring_func: str = "softmax",
    num_fused_shared_experts: int = 0,
    shared_expert_weight: float = 1.0,
    # grouped topk + fused topk bias parameters
    routed_scaling_factor: float = 1.0,
    e_score_correction_bias: torch.Tensor | None = None,
    # custom routing parameters
    custom_routing_function: Callable | None = None,
    # eplb parameters
    eplb_state: EplbLayerState | None = None,
    # zero expert parameters
    zero_expert_type: str | None = None,
    num_logical_experts: int | None = None,
    hash_indices_table: torch.Tensor | None = None,
    # Deepseek V4 vision routing bias parameters
    bias_vl: torch.Tensor | None = None,
    image_sentinel_lo: int = 0,
    skip_padding: bool = False,
) -> FusedMoERouter:
    """Factory function to create the appropriate FusedMoERouter subclass based on
    the provided parameters.

    The selection logic follows this priority order:
    1. ZeroExpertRouter - if zero_expert_type is not None
    2. CPURouter - on CPU, before scheme-specific routers
    3. GroupedTopKRouter - if use_grouped_topk is True and the grouped routing
       cannot fall through while preserving routing and scaling semantics
    4. CustomRoutingRouter - if custom_routing_function is not None
    5. FusedTopKBiasRouter - if e_score_correction_bias or hash_indices_table
       is not None
    6. AiterSharedRoutedFusedMoERouter - if num_fused_shared_experts > 0,
       scoring_func is softmax, and AITER shared-expert fusion is enabled
    7. FusedTopKRouter - default fallback

    When VLLM_MOE_ROUTING_SIMULATION_STRATEGY is set, the selected router
    supplies the kernel routing method and is replaced by a
    RoutingSimulatorRouter.

    Common arguments:
        top_k: Number of experts to select per token
        global_num_experts: Total number of experts in the model
        renormalize: Whether to renormalize the routing weights
        routing_method_type: Optional explicit routing method type

    Grouped topk arguments:
        use_grouped_topk: Whether to use grouped top-k routing
        num_expert_group: Number of expert groups (for grouped routing)
        topk_group: Top-k within each group (for grouped routing)
        scoring_func: Scoring function to use ("softmax" or "sigmoid")
        num_fused_shared_experts: Number of fused shared experts (for ROCm AITER)
        skip_padding: Whether to invalidate routes for padding rows

    Grouped topk and fused topk bias arguments:
        routed_scaling_factor: Scaling factor for routed weights
        e_score_correction_bias: Optional bias correction for expert scores

    Custom routing arguments:
        custom_routing_function: Optional custom routing function

    EPLB arguments:
        eplb_state: Optional EplbLayerState, None when EPLB is disabled.

    Zero expert arguments:
        zero_expert_type: Type of zero expert (e.g. identity). If not None,
            creates a ZeroExpertRouter.
        num_logical_experts: Number of real (non-zero) experts. Required when
            zero_expert_type is not None.

    Hash Indices Table:
        Used to map input_ids to experts, need for Deepseek V4

    Returns:
        An instance of the appropriate FusedMoERouter subclass

    """
    router: FusedMoERouter | None = None
    if zero_expert_type is not None:
        assert num_logical_experts is not None, (
            "num_logical_experts is required when zero_expert_type is set"
        )
        assert e_score_correction_bias is not None, (
            "e_score_correction_bias is required when zero_expert_type is set"
        )
        router = ZeroExpertRouter(
            top_k=top_k,
            global_num_experts=global_num_experts,
            eplb_state=eplb_state,
            e_score_correction_bias=e_score_correction_bias,
            num_logical_experts=num_logical_experts,
            zero_expert_type=zero_expert_type,
            scoring_func=scoring_func,
            renormalize=renormalize,
            routed_scaling_factor=routed_scaling_factor,
        )

    elif current_platform.is_cpu():
        # CPURouter covers every routing scheme CPU MoE experts support
        # (plain softmax, grouped topk, custom routing functions, and
        # DeepSeek V4's sqrtsoftplus/hash routing) in a single class, so it
        # takes priority over the scheme-specific routers below.
        router = CPURouter(
            top_k=top_k,
            global_num_experts=global_num_experts,
            use_grouped_topk=use_grouped_topk,
            num_expert_group=num_expert_group,
            topk_group=topk_group,
            renormalize=renormalize,
            scoring_func=scoring_func,
            routed_scaling_factor=routed_scaling_factor,
            e_score_correction_bias=e_score_correction_bias,
            custom_routing_function=custom_routing_function,
            hash_indices_table=hash_indices_table,
            eplb_state=eplb_state,
        )

    elif use_grouped_topk:
        assert custom_routing_function is None
        if num_expert_group is None or topk_group is None:
            raise ValueError(
                "num_expert_group and topk_group must be provided when "
                "use_grouped_topk is True"
            )

        # For topk_group <= 1, grouped implementation is pure overhead.
        degenerate_grouping = num_expert_group <= 1 and topk_group <= 1
        # FusedTopKRouter cannot apply routed_scaling_factor, FusedTopKBiasRouter can.
        scaling_handled_downstream = (
            routed_scaling_factor == 1.0 or e_score_correction_bias is not None
        )

        # Degenerating must not change the advertised routing method, which drives
        # kernel selection. num_expert_group only affects it for biased routing.
        def advertised_routing_method(groups: int | None) -> RoutingMethodType:
            return get_routing_method_type(
                scoring_func=scoring_func,
                top_k=top_k,
                renormalize=renormalize,
                num_expert_group=groups,
                has_e_score_bias=e_score_correction_bias is not None,
            )

        routing_method_preserved = advertised_routing_method(
            num_expert_group
        ) == advertised_routing_method(None)

        if not (
            degenerate_grouping
            and scaling_handled_downstream
            and routing_method_preserved
        ):
            router = GroupedTopKRouter(
                top_k=top_k,
                global_num_experts=global_num_experts,
                eplb_state=eplb_state,
                num_expert_group=num_expert_group,
                topk_group=topk_group,
                renormalize=renormalize,
                scoring_func=scoring_func,
                routed_scaling_factor=routed_scaling_factor,
                e_score_correction_bias=e_score_correction_bias,
                num_fused_shared_experts=num_fused_shared_experts,
                skip_padding=skip_padding,
            )
        # Otherwise fall through to the non-grouped chain below.

    if router is None:
        if custom_routing_function is None:
            assert scoring_func in ["sigmoid", "softmax", "sqrtsoftplus"]

        if custom_routing_function is not None:
            router = CustomRoutingRouter(
                top_k=top_k,
                global_num_experts=global_num_experts,
                eplb_state=eplb_state,
                custom_routing_function=custom_routing_function,
                renormalize=renormalize,
            )

        elif e_score_correction_bias is not None or hash_indices_table is not None:
            router = FusedTopKBiasRouter(
                top_k=top_k,
                global_num_experts=global_num_experts,
                eplb_state=eplb_state,
                e_score_correction_bias=e_score_correction_bias,
                renormalize=renormalize,
                routed_scaling_factor=routed_scaling_factor,
                scoring_func=scoring_func,
                hash_indices_table=hash_indices_table,
                num_fused_shared_experts=num_fused_shared_experts,
                shared_expert_weight=shared_expert_weight,
                bias_vl=bias_vl,
                image_sentinel_lo=image_sentinel_lo,
            )

        elif (
            num_fused_shared_experts > 0
            and scoring_func == "softmax"
            and rocm_aiter_ops.is_fusion_moe_shared_experts_enabled()
        ):
            router = AiterSharedRoutedFusedMoERouter(
                top_k=top_k,
                global_num_experts=global_num_experts,
                eplb_state=eplb_state,
                num_fused_shared_experts=num_fused_shared_experts,
                renormalize=renormalize,
                scoring_func=scoring_func,
            )

        else:
            router = FusedTopKRouter(
                top_k=top_k,
                global_num_experts=global_num_experts,
                eplb_state=eplb_state,
                renormalize=renormalize,
                scoring_func=scoring_func,
            )

    if envs.VLLM_MOE_ROUTING_SIMULATION_STRATEGY != "":
        router = RoutingSimulatorRouter(
            top_k=top_k,
            global_num_experts=global_num_experts,
            kernel_routing_method_type=router.routing_method_type,
            eplb_state=eplb_state,
        )
    return router