mirror of
https://github.com/kvcache-ai/ktransformers.git
synced 2026-10-17 05:11:19 +08:00
yyj
a5d7ad9047
* feat(sft): add native MXFP4 routed-expert LoRA kernels * test(sft): validate DeepSeek V4 MXFP4 routed experts * feat(sft): AVX2 tier for MXFP4 routed-expert LoRA SFT MXFP4_SFT_MOE on non-AVX512 builds, same Python contract as the AMX / AVX512-BF16 kernel from #2185 (KTMoEWrapper(method="MXFP4_SFT")), for AVX2+FMA+F16C hosts (Zen 2/3 EPYC, older Xeons). * operators/avx2/sft_mxfp4-moe.hpp: AVX2_SFT_MXFP4_MOE_TP part on top of AVX2_MXFP4_MOE_TP (frozen packed E2M1/UE8M0 base, LoRA gate/up/down, V4 swiglu clamp, transpose-free packed-weight dX, authoritative LoRA grads). * operators/moe-sft-tp.hpp: AVX2 result merge; AVX-512 merge loops and the block-FP8 staging guarded so the wrapper builds without AVX-512. * ext_bindings.cpp: bind MXFP4_SFT_MOE -> TP_MOE_SFT<AVX2_SFT_MXFP4_MOE_TP> when USE_AMX_AVX_KERNEL is off; MOESFTBindings no longer AMX-only. * python/sft/backend.py: runtime reports cpu_variant=avx2, kernel avx2-mxfp4-g32. * test_mxfp4_sft_numerical.py: inference regression uses AVX2MXFP4_MOE when AMXFP4_KGroup_MOE is absent. Validated on 2x EPYC 7452 (AVX2, no AVX-512): test_mxfp4_sft_numerical.py TP1/TP2 all stages rel-L2 <= 0.4% (limits 3-5%); validate_dsv4_mxfp4_sft_layer.py on DeepSeek-V4-Flash-0731 layer 3, TP1 and TP2: passed, forward/grad_input/ LoRA grads rel-L2 0.10-0.29%, base weights unchanged. Note: the build uses -ffast-math, so exp() past the float range is undefined; the scalar sigmoid tails clamp the argument like the vector path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011Mgsduzz2fx1gSvfHbR8RZ * fix(sft): clamp MXFP4 scalar-tail sigmoid exponent * feat(sft): integrate native V4 experts with LoRA training and export Wrap only routed experts while preserving V4 routing and shared branches. Validate native checkpoint provenance and stream a BF16 non-expert cache without dequantizing routed experts. Add audited static SGLang export and reproducible training, resume and generation acceptance helpers. CPU retain/GPU recompute GAS1/GAS4 smoke and 265 KT regression tests pass; long-run quality and resume acceptance remain in progress. * fix(sft): retain async inference buffers until CUDA work completes * test(sft): isolate V4 adapter effects with matched-kernel controls Allow baseline and non-expert-only exports to retain a zero-B expert adapter so all ablations execute the same native forward kernel. Add paired held-out generation reporting with explicitly diagnostic lexical metrics. * test(sft): compare PEFT targets semantically during V4 resume audits * test(sft): cancel queued V4 inference checks after a failed request * test(sft): preserve raw V4 responses before finite-output validation * feat: prepare V4 cache atomically and package static adapter export * build: preserve accepted CUDA carriers across Python ABIs * test: retain FP8 loading contracts after V4 wrapper integration * test: exclude historical concatenated-sample smoke from delivery * test(sft): audit unpacked batches and single-GPU resume state * docs(sft): align April V4 recipe with approved unpacked delivery * test: keep frozen-parameter audit indices exact for large tensors * test: record the loaded native backend in acceptance evidence * test: timestamp training audit events for startup measurements * docs: pin V4 serving dependencies and account for reserved token slots * [refactor](sft): keep V4 follow-up focused on runtime integration --------- Co-authored-by: 05yuki <ceu.azul.samu@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
9 lines
278 B
Python
9 lines
278 B
Python
#!/usr/bin/env python3
|
|
"""Compatibility entry point; the converter is included in the kt-kernel wheel."""
|
|
|
|
from kt_kernel.sft.export_dsv4_sglang_adapter import * # noqa: F401,F403
|
|
from kt_kernel.sft.export_dsv4_sglang_adapter import main
|
|
|
|
if __name__ == "__main__":
|
|
main()
|