* ci(release): prepare source builds from four main snapshots
* docs(release): add verified Kimi RAWINT4 LoRA smoke configs
* ci(release): gate four-main PyPI promotion on immutable wheels and model E2E
* ci: add isolated model E2E acceptance for PR wheels and PyPI
* ci: recheck GPU occupancy before each model starts
* ci: disable inherited container healthchecks during acceptance
* ci: wait for free GPUs before native import validation
* docs: record qj5090 acceptance infrastructure validation
* ci: add immutable candidate and public PyPI release acceptance gates
Advance third_party/sglang to main@3424f35d, including all-target KT MoE adapter serving, the MXFP4 routed-scaling fix, and the GLM-5.3 MTP metadata fix.
The AVX2 MXFP4 MoE kernel decoded FP4 weights with 128-bit SSSE3 PSHUFB
and converted the BF16 activations per row, costing roughly 28
shuffle-port uops per 32 weights. On Zen 2, whose two shuffle pipes are
the binding resource here, that left the kernel compute-bound at about
39-45 GB/s per socket, well under the 83.6 GB/s STREAM ceiling of the
same socket.
Add a fast path for the common group_size == 32 layout:
- decode a whole 32-value k-group with one 256-bit VPSHUFB pair,
- fuse the BF16->FP32 widen and shift into unpack(zero, u16), which is a
single shuffle,
- drop the reordering shuffles entirely by converting each activation
row once into the decoder's natural in-lane emission order
({0,2,4,6,1,3,5,7,...} per 32-group) instead of permuting the weights,
- process tokens in blocks of 4 so each decoded group is amortized over
4 accumulators, with a single-row remainder loop that also serves
decode (m == 1).
This is about 9 shuffle uops per 32 weights, which makes the kernel
memory-bound. Large per-expert batches fall back to the generic path so
the per-thread FP32 staging buffer stays bounded.
Measured on 2x EPYC 7452 (AVX2, 2x30 threads) with DeepSeek-V4-Flash
MXFP4:
- microbench per layer-token: m=1 989 -> 440 us (2.25x),
m=4 1713 -> 1002 us, m=16 6567 -> 3829 us, m=64 25651 -> 19361 us
- end-to-end 512-token decode: 18.4 -> 29.5 tok/s at 65K context, and
18.3 -> 32.3 tok/s on a 1M-context serving profile
- NUMA pool scaling verified unchanged (1937 us 1 pool -> 968 us 2 pools)
Output stays coherent in generation smoke tests on both profiles.
Co-authored-by: 05yuki <ceu.azul.samu@gmail.com>
* Add full FT development snapshot
* [fix](kt-kernel): fix Full FT TP base weight gradients
* [fix]: bug fix of 2d81e86
* [fix](kt-kernel): fix AMX BF16 full-weight gradients
* [docs](kt-kernel): document Full FT fork changes and debug history
* [docs](kt-kernel): align fork remote terminology
* [chore](kt-kernel): keep agent notes local
* [fix](kt-kernel): configure SFT OpenMP threads
* [perf](kt-kernel): optimize AMX Full-FT weight gradients
Coarsen base-weight gradient work from individual output tiles to fixed-intermediate strips so each task reuses packed panels across the hidden dimension.
Keep aligned thread-local BF16 panels across tasks and retain FP32 AMX accumulator tiles for the full K reduction. Gate and up run separate K passes while sharing the packed input panel.
On the matched Qwen3-30B-A3B 1-GPU test, stable Full-FT backward drops from 9.281s to 6.792s (-26.82%), step time drops from 19.252s to 16.140s, and TPS rises from 212.76 to 253.78. The LoRA-only backward control changes by -2.74%.
Validated with clang-format, the Release AMX/CUDA extension build, TP1/TP2 reference gradients across boundary token counts, and the 15-step Full-then-LoRA performance run.
* [feat](kt-kernel): add staged SFT profiling
* [fix](kt-kernel): reuse inference BF16 kernel for SFT
* [perf](kt-kernel): add fine-grained Full-FT profiling
* [perf](kt-kernel): batch BF16 Full-FT weight gradients
Use one expert-aggregated tile driver for AVX512-BF16 and AMX base-weight gradients, and pack updated full-precision weights directly into TP BufferB layouts without temporary partitions. Add worker-local profiling and focused dWeight/strided-repack coverage.
* [test](kt-kernel): benchmark BF16 dWeight AMX driver
* [fix](kt-kernel): label dWeight store as worker CPU time
* [docs](sft): record BF16 Full-FT performance
* [docs](sft): remove Qwen3 Full-FT performance report
* [perf](kt-kernel): reduce BF16 Full-FT checkpoint overhead
Retain the first CPU MoE forward state across non-reentrant checkpoint recomputation, write BF16 activations directly into the backward cache, and reduce dWeight packing and gradient-clear traffic. Extend staged profiling and cover checkpoint reuse plus AMX/AVX dWeight paths.
* [perf](kt-kernel): make SFT optimizer gradients authoritative
Bind Full-FT and LoRA Parameter.grad directly to the KT-managed BF16 gradient buffers, avoiding PyTorch duplicate accumulation.
Accumulate microbatch gradients in C++, lazily clear expert buffers between optimizer windows, preserve rank-0 distributed ownership, and add lifecycle and AMX dWeight coverage.
* [perf](kt-kernel): avoid eager Full-FT gradient zeroing
Allocate authoritative Full-FT gradient buffers with torch.empty. The C++ state machine performs the mandatory full clear before first use, avoiding redundant Python-side first touch.
* perf(sft): enable checkpoint forward reuse for LoRA
* fix(sft): serialize checkpoint recompute with async repack
* fix(sft): persist authoritative full weights
* fix(sft): normalize legacy distributed gradients
* [fix](sft): train gated shared experts
* fix(sft): preserve expert placeholders across state dict loads
* ci: publish ktransformers sdist in release workflow
* fix(sft): preserve router autograd for LoRA training
Determine routing graph tracking from the router's trainable parameters instead of the Full-FT mode flag, so PEFT LoRA adapters on MoE gates receive routing-weight gradients. Fail fast when a trainable router returns detached weights and cover frozen, TopK, checkpoint-reuse, and two-step optimizer behavior.
---------
Co-authored-by: illu <wubowen03@foxmail.com>
* docs(DSV4-Flash): AMXINT4 on Skylake (no AMX), dual-Xeon tuning + single-stream negative results
Adds a section based on a validated single-workstation deployment (1x RTX 5090 +
2x Xeon Gold 6138 / Skylake-SP): AMXINT4 runs on AVX512F+BW without an AMX tile;
dual-socket tuning notes; and a table of configs that reduced single-stream
throughput here (notably EAGLE going net-negative on a CPU-bandwidth-bound box,
vs the 8x-RTX-5090 speedup the guide cites).
* Fix command syntax in DeepSeek-V4-Flash.md
---------
Co-authored-by: yyj <159887351+yyj6666667@users.noreply.github.com>
yyj