mirror of
https://github.com/deepseek-ai/DeepInfer.git
synced 2026-11-20 04:22:27 +00:00
The first Cake MoE layer found the EP group's largest prefill row count with a gloo all-reduce and stored it as an ad-hoc attribute on the metadata. The YoCo consumer view is a shallow copy (get_consumer_view), so it inherited the full-token maximum: under compact truncation every consumer layer made as many collective Cake calls as a full-token layer, the extra ones empty. - DeepseekModel.forward reduces the full-token and, under compaction, the consumer row counts in one CPU all-reduce before the layer loop (like the per-layer YoCo dispatch bound) and sets each on its own metadata, so consumer layers make calls for their own rows. - InputMetadata declares cake_moe_max_tokens_per_rank. Layers reached with metadata the forward did not prepare (e.g. draft layers) still compute it once on the first layer. Tests: tests/models/test_cake_moe_prefill_calls.py (CPU). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>