Commit Graph
22 Commits
Author SHA1 Message Date
yyj 41613064c7 feat(GLM): add native GLM-5.3-flash support (#2173)
* feat: support SwiGLU limit for block FP8 experts

* perf(fp8): add native layerwise batch transport

* feat(fp8): support layerwise transport for TP1/2/4/8

* release: prepare GLM-5.3-flash support

* docs(glm): refine GLM-5.3-flash documentation

* docs(glm): simplify release announcement
2026-08-26 22:45:12 +08:00
Benjamin 50d9434800 [docs](kt-kernel): add MiniMax-M3 SGLang + KT-Kernel tutorial (#2051) 2026-06-18 10:40:51 +08:00
Jianwei Dong 0e2ff783d8 update glm52 tutorial (#2046) 2026-06-16 17:15:23 +08:00
Aliez RenandJiaheng Dai 02be2bf53f [feat](kt-kernel): add AVX2/AVX-VNNI RAWINT4 MoE backend (#1942)
* [feat](kt-kernel): add AVX2/AVX-VNNI RAWINT4 MoE backend

* Update AVX2 tutorial with AVX2 compilation instructions

Added instructions for forcing AVX2 compilation on AVX512 or AMX machines.

* Add instructions for AVX2 compilation

---------

Co-authored-by: Jiaheng Dai <108478605+jdai0@users.noreply.github.com>
2026-04-30 17:16:49 +08:00
Peilin Li 0656e01ac1 [docs]: refresh KT install commands (#1958) 2026-04-27 00:45:43 +08:00
Benjamin F 06ee9f62f3 [doc]: add prerequisite note for GLM-5.1 tutorial (#1932) 2026-04-14 15:07:08 +08:00
Jianwei Dong 891c5c0a13 Support glm5.1 (#1916) 2026-04-07 11:29:32 +08:00
mrhaoxxandBenjamin F 7a9daf0cd4 [feat](kt-kernel): support avx2 only inference for bf16 fp8 and gptq int4 (#1892)
* feat: support avx2 bf16 fp8 inference

* feat: support avx2 gptq int4 inference

* fix: numeric issues in fp8 dequant

* Tutorial avx2 (#1900)

* fix: prevent injecting -DLLAMA_AVX512=ON on AVX2-only machines

* docs: add AVX2 tutorial for running KTransformers on AVX2-only CPUs

* Tutorial avx2 (#1901)

* fix: prevent injecting -DLLAMA_AVX512=ON on AVX2-only machines

* docs: add AVX2 tutorial for running KTransformers on AVX2-only CPUs

* docs: update README.md

---------

Co-authored-by: Benjamin F <159887351+yyj6666667@users.noreply.github.com>
2026-03-27 14:45:02 +08:00
Jianwei DongandClaude Opus 4.6 15c624dcae Fix/sglang kt detection (#1875)
* [feat]: simplify sglang installation with submodule, auto-sync CI, and version alignment

- Add kvcache-ai/sglang as git submodule at third_party/sglang (branch = main)
- Add top-level install.sh for one-click source installation (sglang + kt-kernel)
- Add sglang-kt as hard dependency in kt-kernel/pyproject.toml
- Add CI workflow to auto-sync sglang submodule daily and create PR
- Add CI workflow to build and publish sglang-kt to PyPI
- Integrate sglang-kt build into release-pypi.yml (version.py bump publishes both packages)
- Align sglang-kt version with ktransformers via SGLANG_KT_VERSION env var injection
- Update Dockerfile to use submodule and inject aligned version
- Update all 13 doc files, CLI hints, and i18n strings to reference new install methods

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [build]: bump version to 0.5.2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [build]: rename PyPI package from kt-kernel to ktransformers

Users can now `pip install ktransformers` to get everything
(sglang-kt is auto-installed as a dependency).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Revert "[build]: rename PyPI package from kt-kernel to ktransformers"

This reverts commit e0cbbf6364.

* [build]: add ktransformers meta-package for PyPI

`pip install ktransformers` now works as a single install command.
It pulls kt-kernel (which in turn pulls sglang-kt).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [fix]: show sglang-kt package version in kt version command

- Prioritize sglang-kt package version (aligned with ktransformers)
  over sglang internal __version__
- Update display name from "sglang" to "sglang-kt"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [fix]: improve sglang-kt detection in kt doctor and kt version

Recognize sglang-kt package name as proof of kvcache-ai fork installation.
Previously both commands fell through to "PyPI (not recommended)" for
non-editable local source installs. Now version.py reuses the centralized
check_sglang_installation() logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [build]: bump version to 0.5.2.post1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-04 16:54:48 +08:00
Oql 1c72b3f5bd fix glm5 docs (#1845) 2026-02-12 02:33:37 +08:00
Oql 7f7aeaeff6 support glm 5 (#1844) 2026-02-12 02:03:32 +08:00
Oql 4f64665758 [docs]: add Qwen3 Coder Next Tutorial (#1833) 2026-02-04 16:27:10 +08:00
Oql bf4c8a690b Add Native Precision Tutorial, update worker strategy and README.md (#1807) 2026-01-23 18:00:13 +08:00
Jianwei Dong 8652346e69 [fix]: doc (#1805) 2026-01-23 11:17:08 +08:00
Jianwei Dongandouqingliang 779bf14556 [doc]: add Experts sched tutorial (#1802)
* Change num gpu experts to gpu expert masks and add eplb statistics

* [feat]: update examples

* [fix]: fix fp8 perchannel

* Delete useless tests

* Delete useless tests

* add experts_sched tutorial

---------

Co-authored-by: ouqingliang <1692110604@qq.com>
2026-01-22 15:40:07 +08:00
ErvinXie d2305538f7 Modify installation steps in Kimi-K2-Thinking-Native.md (#1800)
Updated installation instructions for sglang repository.
2026-01-21 15:46:21 +08:00
ZiWei Yuan b096b01fbc [docs]: add kt-cli doc and update corresponding website (#1768) 2025-12-29 23:06:22 +08:00
Oql 63796374c1 [docs]: fix and add MiniMax-M2 tutorial images. (#1752) 2025-12-25 20:14:35 +08:00
ZiWei Yuan 3315335fb1 [docs]: update docs to kt-kernel & add amd_blis doc (#1744) 2025-12-24 17:55:15 +08:00
Jianwei Dong 670c488155 [docs]: Add deepseek-v3.2 run tutorial (#1659) 2025-12-02 20:04:10 +08:00
ZiWei Yuan be6db6f46b [docs]: improve structure for kt-kernel (#1624)
* [docs]: improve structure for kt-kernel

* Update doc/en/kt-kernel/README.md
2025-11-16 13:21:41 +08:00
ZiWei Yuan 133eea037c [docs]: improve docs structure (#1623) 2025-11-16 12:40:59 +08:00