The PR #291 files failed `ruff check` (ANN001 on cake_moe.expert_major_views,
userbuffer.register_cake_moe and the test helpers; E741 on the tests' `I`)
and `ruff format --check`. Annotate the arguments (CakeMoE under
TYPE_CHECKING in userbuffer), rename the intermediate size to INTER in the
tests, and apply ruff format. No behavior change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One persistent kernel per MoE call dispatches the tokens, runs the routed
MXFP4 experts and the FP8 shared expert, and combines, moving data over
PCIe peer load/store between reachable ranks and NCCL GIN elsewhere. The
DeepEP path keeps all peer traffic on the NICs (NCCL_LSA_TEAM_SIZE=1).
- cake_moe.py: the routed and shared expert weights convert to Cake's
operands (bit-exact against DeepInfer's dequantized layouts) and its
expert-major kernel layout, owned by DeepInfer so EPLB moves them in
place. Every call is bucketed with one row bound on all EP ranks:
decode one call bounded by the decode capacity (graph-capturable;
the bound does not change the cost), prefill chunks of
DEEPINFER_SM120_CAKE_CHUNK_TOKENS rows (default 2048, which bounds the
workspace) covering the EP group's largest batch, found once per step
by a CPU all-reduce. Warmup at userbuffer allocation creates the pooled
workspace and checks the kernel's protocol flag.
- EPLB: Cake does not report what each rank received, so its layers count
the routes they send and the load balancer all-reduces them once per
step into the received counts (count_on_sender).
- Runtime: the cake backend leaves the LSA team to NCCL and warns without
NCCL_TOPO_FILE; is_within_nvl_domain is False on SM120 (no NVLink).
- build.sh: SM120 installs NCCL 2.30.7, which Cake requires exactly.
Tests: tests/kernels/test_sm120_cake_moe.py (operand conversion, EPLB
locality of the kernel layout) and tests/distributed/test_sm120_cake_moe.py
(8 GPUs: against a reference with Cake's quantization steps, chunked
calls bitwise equal to one call, uneven and empty ranks, CUDA graph
replay). examples/offline_inference.py runs end to end on 8 GPUs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>