* [feat](kt-kernel): add AVX2 RAWINT4 routed-expert LoRA SFT backend
* [fix](kt-kernel): fix RAWINT4 SFT wrapper and clamp gradients
Allow the RAWINT4 production wrapper on AVX2 runtimes and reject shared backward pools before construction. Preserve FP32 clamp decisions across BF16 caches, remove unused configuration state, and cover rectangular dimensions, multiblock dX, clamp boundaries, and production wrapper tasks with tighter LoRA gradient checks.
* [fix](kt-kernel): plumb swiglu_limit through the RAWINT4 SFT python path
The production wrapper hard-zeroed swiglu_limit for every format except MXFP4, and both SFT python gates rejected non-zero values for RAWINT4 even though MOESFTConfig carries the field and the AVX2 kernel implements the clamp in forward plus the gradient mask in backward. RAWINT4_SFT now accepts a finite non-negative swiglu_limit end to end, while the AMX K2 part (which computes plain SiLU) fails fast instead of silently dropping the clamp. The production-wrapper contract test is parametrized over swiglu_limit 0 and 2, a model-config plumb test covers plain and nested text_config shapes, the __main__ entry exercises both limits, and bench_rawint4_sft.py gains --swiglu-limit.
* fix(kt-kernel): size per-NUMA thread counters by physical NUMA id
WorkerPool::init() and NumaJobDistributor::init() indexed the per-NUMA
worker counter by the *physical* NUMA ID while sizing it by the number
of subpools. A subpool mapped to a NUMA node whose ID is >= the subpool
count (e.g. --kt-threadpool-count 1 --kt-numa-nodes 1) therefore read
and wrote out of bounds, shifting the per-node core offset and logging
"Core ... not found" (or corrupting the heap and aborting).
Size the counter vector to cover every NUMA ID present in the subpool
map. Add a CPU regression test that builds two subpools on a
high-numbered NUMA node and verifies the worker thread offsets.
* test(kt-kernel): add single-subpool regression case for #2221
The existing fixture always builds two subpools, so on a dual-socket host
(NUMA IDs [0, 1]) the two-element counter stays in bounds and the old code
passes there. Add a single subpool mapped to a nonzero NUMA node: the size-1
counter is then indexed at an ID >= 1, reproducing #2221 on any multi-NUMA
host. Verified that this case fails/aborts on the unpatched code and passes
with the fix.
Suggested by @f-wunderlich in review.
* feat(sft): add native MXFP4 routed-expert LoRA kernels
* test(sft): validate DeepSeek V4 MXFP4 routed experts
* feat(sft): AVX2 tier for MXFP4 routed-expert LoRA SFT
MXFP4_SFT_MOE on non-AVX512 builds, same Python contract as the AMX /
AVX512-BF16 kernel from #2185 (KTMoEWrapper(method="MXFP4_SFT")), for
AVX2+FMA+F16C hosts (Zen 2/3 EPYC, older Xeons).
* operators/avx2/sft_mxfp4-moe.hpp: AVX2_SFT_MXFP4_MOE_TP part on top of
AVX2_MXFP4_MOE_TP (frozen packed E2M1/UE8M0 base, LoRA gate/up/down,
V4 swiglu clamp, transpose-free packed-weight dX, authoritative LoRA grads).
* operators/moe-sft-tp.hpp: AVX2 result merge; AVX-512 merge loops and the
block-FP8 staging guarded so the wrapper builds without AVX-512.
* ext_bindings.cpp: bind MXFP4_SFT_MOE -> TP_MOE_SFT<AVX2_SFT_MXFP4_MOE_TP>
when USE_AMX_AVX_KERNEL is off; MOESFTBindings no longer AMX-only.
* python/sft/backend.py: runtime reports cpu_variant=avx2, kernel avx2-mxfp4-g32.
* test_mxfp4_sft_numerical.py: inference regression uses AVX2MXFP4_MOE when
AMXFP4_KGroup_MOE is absent.
Validated on 2x EPYC 7452 (AVX2, no AVX-512): test_mxfp4_sft_numerical.py
TP1/TP2 all stages rel-L2 <= 0.4% (limits 3-5%); validate_dsv4_mxfp4_sft_layer.py
on DeepSeek-V4-Flash-0731 layer 3, TP1 and TP2: passed, forward/grad_input/
LoRA grads rel-L2 0.10-0.29%, base weights unchanged.
Note: the build uses -ffast-math, so exp() past the float range is undefined;
the scalar sigmoid tails clamp the argument like the vector path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Mgsduzz2fx1gSvfHbR8RZ
* fix(sft): clamp MXFP4 scalar-tail sigmoid exponent
* feat(sft): integrate native V4 experts with LoRA training and export
Wrap only routed experts while preserving V4 routing and shared branches. Validate native checkpoint provenance and stream a BF16 non-expert cache without dequantizing routed experts.
Add audited static SGLang export and reproducible training, resume and generation acceptance helpers. CPU retain/GPU recompute GAS1/GAS4 smoke and 265 KT regression tests pass; long-run quality and resume acceptance remain in progress.
* fix(sft): retain async inference buffers until CUDA work completes
* test(sft): isolate V4 adapter effects with matched-kernel controls
Allow baseline and non-expert-only exports to retain a zero-B expert adapter so all ablations execute the same native forward kernel. Add paired held-out generation reporting with explicitly diagnostic lexical metrics.
* test(sft): compare PEFT targets semantically during V4 resume audits
* test(sft): cancel queued V4 inference checks after a failed request
* test(sft): preserve raw V4 responses before finite-output validation
* feat: prepare V4 cache atomically and package static adapter export
* build: preserve accepted CUDA carriers across Python ABIs
* test: retain FP8 loading contracts after V4 wrapper integration
* test: exclude historical concatenated-sample smoke from delivery
* test(sft): audit unpacked batches and single-GPU resume state
* docs(sft): align April V4 recipe with approved unpacked delivery
* test: keep frozen-parameter audit indices exact for large tensors
* test: record the loaded native backend in acceptance evidence
* test: timestamp training audit events for startup measurements
* docs: pin V4 serving dependencies and account for reserved token slots
* [refactor](sft): keep V4 follow-up focused on runtime integration
---------
Co-authored-by: 05yuki <ceu.azul.samu@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* [docs]: add reproducible Kimi K2.5 PyPI LoRA tutorial
* [docs]: highlight Kimi K2.5 and K2.6 LoRA support
* Update README to simplify LoRA fine-tuning entry
* Add updates section to README.md
* [docs]: clarify clean Kimi setup and held-out evaluation
* [docs]: explain pinned model downloads on offline training hosts
* [docs]: isolate Kimi caches in the tutorial work directory
* [docs]: simplify Kimi LoRA installation and training tutorial
* ci(release): prepare source builds from four main snapshots
* docs(release): add verified Kimi RAWINT4 LoRA smoke configs
* ci(release): gate four-main PyPI promotion on immutable wheels and model E2E
* ci: add isolated model E2E acceptance for PR wheels and PyPI
* ci: recheck GPU occupancy before each model starts
* ci: disable inherited container healthchecks during acceptance
* ci: wait for free GPUs before native import validation
* docs: record qj5090 acceptance infrastructure validation
* ci: add immutable candidate and public PyPI release acceptance gates
Advance third_party/sglang to main@3424f35d, including all-target KT MoE adapter serving, the MXFP4 routed-scaling fix, and the GLM-5.3 MTP metadata fix.
The AVX2 MXFP4 MoE kernel decoded FP4 weights with 128-bit SSSE3 PSHUFB
and converted the BF16 activations per row, costing roughly 28
shuffle-port uops per 32 weights. On Zen 2, whose two shuffle pipes are
the binding resource here, that left the kernel compute-bound at about
39-45 GB/s per socket, well under the 83.6 GB/s STREAM ceiling of the
same socket.
Add a fast path for the common group_size == 32 layout:
- decode a whole 32-value k-group with one 256-bit VPSHUFB pair,
- fuse the BF16->FP32 widen and shift into unpack(zero, u16), which is a
single shuffle,
- drop the reordering shuffles entirely by converting each activation
row once into the decoder's natural in-lane emission order
({0,2,4,6,1,3,5,7,...} per 32-group) instead of permuting the weights,
- process tokens in blocks of 4 so each decoded group is amortized over
4 accumulators, with a single-row remainder loop that also serves
decode (m == 1).
This is about 9 shuffle uops per 32 weights, which makes the kernel
memory-bound. Large per-expert batches fall back to the generic path so
the per-thread FP32 staging buffer stays bounded.
Measured on 2x EPYC 7452 (AVX2, 2x30 threads) with DeepSeek-V4-Flash
MXFP4:
- microbench per layer-token: m=1 989 -> 440 us (2.25x),
m=4 1713 -> 1002 us, m=16 6567 -> 3829 us, m=64 25651 -> 19361 us
- end-to-end 512-token decode: 18.4 -> 29.5 tok/s at 65K context, and
18.3 -> 32.3 tok/s on a 1M-context serving profile
- NUMA pool scaling verified unchanged (1937 us 1 pool -> 968 us 2 pools)
Output stays coherent in generation smoke tests on both profiles.
Co-authored-by: 05yuki <ceu.azul.samu@gmail.com>
yyj