Commit Graph

Select branches

Hide Pull Requests

ErvinXie-patch-1

JimmyPeilinLi-patch-1

JimmyPeilinLi-patch-2

add-maintainers

archive-kt-sft

auto/sync-sglang

avx2

bump-sglang-pr31

chore/kt-layerwise-prefill-label

chore/kt-layerwise-prefill-main

chore/kt-run-numa-nodes-unified

ci/four-main-pypi-release

ci/model-e2e-acceptance

cli-dev-ljq

cpu-fix

debug_llama

doc-reorg-entrypoints-support-matrix

docs-v061-refresh

docs/deepseek-v4-flash-docker

docs/gosim-2026-and-roadmap-link

docs/kt-ft-figure2-single-column-20261010

docs/kt-ft-paper-20261008

experts_sched

feat/mistral-loader-support

fix-deploy-djw

fix-docker-build

fix-import

fix/avx2-wheel-feature-isolation

fix/glm53-index-share-sync

fix/release-self-hosted-runner

fix/sglang-kt-detection

flatten-ktransformers-shim

fp4-moe-amx

gh-pages

glm47_fp8

hotfix/0.7.0.post2-pypi-runtime

kimi_k2.5

ksft-llama

ksft-sglang

kt-kernel-torch-range-v061

layerwise-prefill-opt

ljq-cli-dev

ljq-cli-dev-new

ljq-dev-1

lut-nvfp4

main

minimax-m2.5

mtp

oql-dev

oql-doc

ovowei-patch-1

ovowei-patch-2

ovowei-patch-3

perf/bf16-grad-norm-20261010

qwen3.5

refactor_sft

refactor_sft_int4_debug

release/0.7.0.post2-frozen-pypi

release/0.7.0.post3-frozen-pypi

release/20260911-carrier-repair

release/20260911-ci-python

release/20260911-pypi

release/automate-kt-sft-pypi

release/v0.5.3

revert-1918-fix/rocm-cpuinfer-stream-bridge

route-moe

sft

sft-rel

sft-whl

sosp25-ae

support-glm5-next-djw

support-glm51-djw

support-glm52-djw

#1005

#1013

#1015

#1024

#1027

#1029

#1030

#1031

#1033

#1034

#1042

#1043

#1044

#1049

#106

#1060

#1063

#1064

#1069

#1079

#1084

#1085

#1090

#1091

#1095

#1096

#1097

#1098

#1099

#110

#1100

#1101

#1103

#1108

#1109

#1116

#1132

#1134

#1135

#1137

#1143

#115

#1154

#1157

#1158

#1159

#1160

#1163

#1165

#1167

#1168

#1170

#1172

#1177

#118

#1180

#1182

#1183

#1184

#1186

#1188

#1193

#1197

#1198

#1202

#1204

#1210

#1211

#1212

#1215

#1216

#1217

#1218

#1219

#122

#1220

#1221

#1222

#1223

#1224

#1225

#1227

#123

#124

#1241

#1244

#1245

#1246

#1247

#126

#1266

#127

#1270

#1271

#1276

#1277

#128

#1280

#1289

#1295

#1296

#1297

#1298

#1300

#1301

#1303

#1304

#1305

#1306

#1307

#1311

#1313

#1320

#1323

#1328

#1331

#1337

#1342

#1343

#135

#1350

#1351

#1358

#1361

#1366

#1372

#1379

#1398

#1399

#14

#1405

#1407

#1409

#141

#1410

#142

#1422

#1428

#143

#1431

#1432

#1434

#1435

#1436

#1437

#1438

#1439

#144

#1449

#1452

#1456

#1457

#1458

#1458

#1466

#1487

#1493

#1494

#1495

#1498

#1500

#1505

#1506

#151

#152

#1520

#1521

#1522

#1523

#1525

#1527

#1529

#153

#1530

#1531

#1535

#1538

#1540

#1542

#1543

#1545

#1547

#1548

#1549

#1550

#1551

#1552

#1554

#1557

#1558

#1559

#1560

#1561

#1562

#1564

#1568

#1569

#1571

#1572

#1573

#1574

#1576

#158

#1581

#1583

#1584

#1585

#1586

#1587

#1588

#1589

#1590

#1592

#1594

#1595

#1596

#1597

#1599

#160

#1600

#1602

#1603

#1604

#1606

#1607

#1609

#1613

#1614

#1615

#1616

#1617

#1619

#1621

#1622

#1623

#1624

#1625

#1626

#1629

#163

#1630

#1631

#1632

#1634

#1636

#1638

#1640

#1642

#1643

#1644

#1646

#1647

#1648

#165

#1653

#1656

#1659

#166

#1660

#1662

#1663

#1668

#1669

#1670

#1671

#1672

#1673

#1674

#1675

#1683

#1684

#1685

#1686

#1687

#1691

#1695

#1696

#1698

#1699

#17

#170

#1702

#1703

#1704

#1706

#1707

#1708

#1712

#1716

#1717

#1718

#1719

#1720

#1721

#1722

#1723

#1724

#1725

#1726

#1728

#1729

#1731

#1733

#1734

#1735

#1737

#1738

#1739

#1740

#1742

#1743

#1744

#1745

#1746

#1748

#1749

#1752

#1763

#1764

#1765

#1766

#1768

#1769

#1770

#1771

#1772

#178

#1780

#1783

#1784

#1788

#1789

#1791

#1796

#1797

#1798

#1799

#18

#180

#1800

#1801

#1802

#1805

#1807

#1808

#1810

#1811

#1812

#1814

#1819

#1821

#1822

#1823

#1824

#1825

#1828

#183

#1830

#1833

#1834

#1837

#1838

#1840

#1844

#1845

#1846

#1848

#1849

#1850

#1851

#1853

#1857

#1862

#1872

#1873

#1875

#1876

#1886

#1887

#189

#1890

#1891

#1892

#1899

#190

#1900

#1901

#1902

#1903

#1904

#1906

#1908

#1909

#1916

#1918

#1919

#1920

#1923

#1925

#1926

#1927

#1929

#1932

#1934

#1935

#1936

#1937

#1941

#1942

#1944

#1945

#1948

#1950

#1952

#1953

#1954

#1955

#1957

#1958

#1959

#1962

#1963

#1964

#1965

#1966

#1967

#1970

#1972

#1973

#1974

#1975

#1976

#1979

#1980

#1984

#1986

#1987

#1991

#1993

#1994

#1995

#1997

#2

#200

#2003

#2003

#2005

#2006

#2010

#2014

#2015

#2018

#2020

#2020

#2021

#2022

#2025

#2026

#2028

#2029

#2031

#2032

#2033

#2037

#2041

#2043

#2043

#2044

#2046

#2047

#2048

#2049

#2051

#2052

#2054

#2055

#2057

#2059

#2062

#2063

#2064

#2064

#2068

#2069

#2070

#2071

#2074

#2075

#2079

#2080

#2082

#2085

#2085

#2086

#2088

#2089

#2091

#2092

#2093

#2093

#2094

#2095

#2095

#2096

#2096

#2097

#2097

#2101

#2101

#2102

#2102

#2103

#2103

#2104

#2105

#2111

#2111

#2112

#2114

#2116

#2117

#2119

#2119

#2120

#2120

#2121

#2121

#2122

#2122

#2123

#2123

#2124

#2124

#2125

#2125

#2126

#2126

#2129

#2129

#2130

#2130

#2131

#2131

#2132

#2132

#2133

#2133

#2134

#2134

#2135

#2135

#2136

#2137

#2137

#2138

#2138

#2140

#2141

#2142

#2142

#2152

#2153

#2153

#2154

#2155

#2155

#2156

#2157

#2158

#2159

#2160

#2161

#2161

#2162

#2163

#2164

#2165

#2166

#2167

#2168

#2168

#2169

#2170

#2171

#2173

#2174

#2174

#2175

#2176

#2177

#2180

#2181

#2182

#2185

#2186

#2187

#2188

#2190

#2190

#2191

#2193

#2193

#2194

#2195

#2196

#2197

#2198

#2199

#220

#2200

#2201

#2201

#2202

#2203

#2204

#2205

#2206

#2207

#2209

#2210

#2211

#2212

#2212

#2215

#2215

#2216

#2217

#2217

#2219

#2219

#2220

#2223

#2224

#2225

#2228

#2229

#2229

#2231

#2231

#2232

#2233

#2234

#2235

#2236

#2236

#2237

#2237

#2238

#2238

#2239

#2239

#224

#2240

#2240

#2241

#2241

#2242

#2242

#2243

#2246

#2246

#2247

#2248

#2248

#2250

#227

#229

#230

#232

#237

#238

#24

#247

#25

#254

#255

#26

#27

#280

#284

#29

#290

#294

#295

#30

#301

#305

#306

#307

#313

#315

#316

#317

#324

#327

#329

#330

#333

#34

#341

#35

#354

#355

#357

#36

#38

#382

#384

#39

#40

#401

#406

#41

#412

#413

#421

#43

#442

#445

#450

#476

#48

#481

#487

#501

#506

#51

#52

#523

#532

#550

#56

#57

#577

#58

#582

#583

#597

#60

#608

#612

#62

#621

#622

#626

#638

#64

#640

#643

#644

#645

#650

#651

#654

#657

#658

#659

#662

#663

#664

#665

#667

#668

#67

#670

#684

#685

#691

#697

#699

#7

#702

#71

#717

#719

#72

#721

#722

#730

#731

#732

#743

#750

#751

#753

#754

#759

#761

#77

#770

#775

#778

#781

#787

#791

#795

#798

#799

#803

#808

#809

#81

#810

#813

#814

#815

#819

#820

#83

#835

#839

#840

#842

#847

#848

#852

#86

#88

#880

#886

#889

#891

#892

#893

#897

#898

#899

#902

#904

#906

#911

#913

#915

#916

#925

#926

#927

#931

#935

#936

#940

#943

#954

#956

#970

#974

#978

#99

#998

paper-ktft-v1

paper-ktft-v1.1

v0.1.0

v0.1.1

v0.1.2

v0.1.3

v0.1.4

v0.2.0

v0.2.1

v0.2.1.post1

v0.2.2

v0.2.2rc1

v0.2.2rc2

v0.2.3

v0.2.3post1

v0.2.3post2

v0.2.4

v0.2.4post1

v0.3

v0.3.1

v0.3.2

v0.4.1

v0.4.2

v0.4.3

v0.4.4

v0.5.0

v0.5.0.post1

v0.5.1

v0.5.2.post1

v0.5.3

v0.6.1

v0.6.1.post1

v0.6.2

v0.6.2.post1

v0.6.2.post2

v0.6.2.post3

v0.6.3

v0.6.3.post1

v0.6.4

v0.7.0

v0.7.0.post1

v0.7.0.post4

v0.7.1

v0.7.2

Mono Color

• 050b745a6e Merge pull request #643 from Azure-Tang/support-fp8 Azure 2025-02-25 16:22:12 +08:00
• 36fbeee341 Update doc Azure 2025-02-25 08:21:18 +00:00
• f639fbc19e feat: basic api key support ceerrep 2025-02-25 14:11:39 +08:00
• 4dc5518e4d update fp8 kernel tutorial Azure 2025-02-24 15:37:01 +00:00
• 7b2a6690ab Merge pull request #608 from makllama/fix_musa_ext Atream 2025-02-24 23:12:54 +08:00
• 6f9ea689a9 Merge pull request #645 from makllama/torch2.2 Atream 2025-02-24 23:12:33 +08:00
• f88c05a6f1 Ensure backward compatibility with Torch 2.2 Xiaodong Ye 2025-02-24 21:55:30 +08:00
• 07eb712a73 Left out lazymio 2025-02-24 21:51:14 +08:00
• 91062a834f Default values lazymio 2025-02-24 21:38:01 +08:00
• 76487c4dcb Revert repetition_penalty as it is not in API spec lazymio 2025-02-24 21:30:03 +08:00
• 05ad288453 Also /chat/completions lazymio 2025-02-24 21:08:36 +08:00
• bf36547f98 Also allow repetition_penalty lazymio 2025-02-24 21:07:35 +08:00
• 8704c09192 Allow temperature and top_p from requests lazymio 2025-02-24 21:01:33 +08:00
• ca7366d2db Merge remote-tracking branch 'upstream/develop-0.2.2' into support-fp8 Azure 2025-02-24 11:58:10 +00:00
• 581a524f65 Add data loader to read special weights for fp8; Add special weight process script Azure 2025-02-24 11:16:23 +00:00
• e9b1216a9a Merge branch 'main' into feat-absorb-for-long-prefill Atream 2025-02-24 09:44:17 +00:00
• 4b5991e77e Merge pull request #638 from kvcache-ai/feat-moonlight Atream 2025-02-24 17:32:05 +08:00
• f327695079 fix KExpertsMarlin on GPU with out CUDA Graph Atream 2025-02-24 09:30:54 +00:00
• cea07d1998 Feat: Clear cache during weight loading to prevent OOM on GPUs with <=8GB VRAM Yuhao Tsui 2025-02-24 10:09:42 +08:00
• 706e69f4fc Fix missing macro definition for KTRANSFORMERS_USE_CUDA and <chrono> includes on MSVC akemimadoka 2025-02-24 01:37:50 +08:00
• eb039b723d Merge pull request #621 from kvcache-ai/feat-moonlight Atream 2025-02-23 22:39:08 +08:00
• f5f6c6b95d update yaml Atream 2025-02-23 14:33:58 +00:00
• e8e02e5ccc support Moonlight Atream 2025-02-23 14:21:18 +00:00
• 95d937c51d tmp DDong Jianwei 2025-02-23 18:51:42 +08:00
• 006e8c6abc remove causal mask Atream 2025-02-23 07:40:47 +00:00
• cdb6f896bb Merge pull request #612 from kvcache-ai/fix-bf16-load Atream 2025-02-23 15:37:23 +08:00
• 036ae25a89 fix bf16 load, TODO: refactor cpu dequant Atream 2025-02-23 15:37:09 +08:00
• 18b1d18367 musa: support bf16 Xiaodong Ye 2025-02-23 10:19:19 +08:00
• 7b7c6a657d Add fp8 linear kernel;\n Add empty cache to fit in 16G VRAM; By 'wkGCaSS - 知乎 https://zhuanlan.zhihu.com/p/25491611225' Azure 2025-02-22 13:05:08 +00:00
• 94ab2de3b9 Merge pull request #523 from miaooo0000OOOO/main Atream 2025-02-22 17:38:18 +08:00
• 72d09f3f6e Merge pull request #597 from kvcache-ai/feat-more-context Atream 2025-02-22 17:17:09 +08:00
• f7f1059873 fix merge bug, this branch also padding Marlin Atream 2025-02-22 09:00:09 +00:00
• 4c568e04bb deploy: e90896314c Atream 2025-02-22 08:46:01 +00:00
• e90896314c Merge pull request #577 from JiamingMai/dev Atream 2025-02-22 16:45:41 +08:00
• 954796123c Merge pull request #582 from twobob/patch-1 Atream 2025-02-22 16:44:48 +08:00
• 024009675e Merge branch 'main' into feat-more-context Atream 2025-02-22 06:17:39 +00:00
• 5ec33d046d optimize gguf dequant, save mem, support Q2_K use marlin for lm_head, lm_head only calc last token for prefill extend context window to 19K for DeepSeek-V3/R1 within 24GB VRAM Atream 2025-02-22 06:13:01 +00:00
• 5ed441a0f5 Update README.md _ 2025-02-21 14:15:50 +00:00
• 45faddf668 fix the link addresses JiamingMai 2025-02-21 17:53:20 +08:00
• 7e1fe256c8 optimize GPU Atream 2025-02-21 05:06:57 +00:00
• 25c5bddd08 Merge pull request #506 from makllama/musa Azure 2025-02-20 22:50:31 +08:00
• 1dd84b4a5b Merge pull request #550 from kvcache-ai/docker_dev ZiWei Yuan 2025-02-20 22:29:56 +08:00
• ed74f91a25 ⚡ docker-iamge workflow liam 2025-02-20 22:27:51 +08:00
• a6aee1c54a Merge branch 'main' of https://github.com/kvcache-ai/ktransformers into main liam 2025-02-20 22:00:55 +08:00
• 64a00651b1 Merge pull request #232 from HandSonic/patch-1 ZiWei Yuan 2025-02-20 22:00:16 +08:00
• 97e1dc97f6 ⚡ fix .so bug liam 2025-02-20 21:24:46 +08:00
• 574b13390b ⚡ fin docker-image liam 2025-02-20 12:23:11 +08:00
• b05cc5685d fix: fix SSE formatting xv44586 2025-02-20 15:01:35 +08:00
• 0cdc446a19 默认并行编译 miaooo0000OOOO 2025-02-20 11:09:16 +08:00
• 2207f6cd14 feat: Support Moore Threads GPU Xiaodong Ye 2025-02-19 18:26:55 +08:00
• b7db4d3ad3 deploy: b4fb633991 Atream 2025-02-19 08:47:03 +00:00
• b4fb633991 Merge pull request #501 from Zhoneym/main Atream 2025-02-19 16:46:48 +08:00
• c92a300bd8 Necessary tips for Node.js related issues Zhoneym 2025-02-19 16:37:18 +08:00
• c8d11c431f deploy: c9eb1f76f6 Atream 2025-02-19 05:47:02 +00:00
• c9eb1f76f6 Merge pull request #384 from ubergarm/add-tutorial-notes Atream 2025-02-19 13:46:49 +08:00
• 89f8218a2a Merge pull request #487 from kvcache-ai/clean_pr Atream 2025-02-19 12:46:48 +08:00
• a529518346 clean PR code and disable flashinfer Atream 2025-02-19 04:42:47 +00:00
• cf4da5fd47 Merge pull request #382 from ceerRep/server-prefix-cache Atream 2025-02-19 12:33:36 +08:00
• be801a3645 deploy: ea75849d03 Azure-Tang 2025-02-19 03:19:30 +00:00
• ea75849d03 Merge pull request #481 from Zhoneym/main Azure 2025-02-19 11:19:11 +08:00
• c72291832d Update install.md Zhoneym 2025-02-19 10:39:43 +08:00
• 3015e9dfa7 Modify and add any incorrect or missing content in the install.md file. Zhoneym 2025-02-19 10:01:16 +08:00
• 584c7d5639 fix: object type for non-streaming response ceerrep 2025-02-18 23:49:51 +08:00
• 6d45871de8 fix: workaround return dummy usage ceerrep 2025-02-18 22:39:49 +08:00
• 09f5c5e77e Merge pull request #476 from kvcache-ai/V0.2.1.post1 v0.2.1.post1 ZiWei Yuan 2025-02-18 21:48:30 +08:00
• 75d7d423b0 🔖 release v0.2.1.post1 liam 2025-02-18 20:45:48 +08:00
• a435e1bbcd Merge pull request #406 from hoshinohikari/main Azure 2025-02-18 20:09:59 +08:00
• db61375f1a Merge remote-tracking branch 'kvai/main' into server-prefix-cache ceerrep 2025-02-18 17:51:22 +08:00
• 19fd24f461 Merge pull request #412 from haojiubujian1985/delete-unused-link Azure 2025-02-18 14:58:10 +08:00
• 1c1769a579 Merge pull request #413 from kvcache-ai/fix_precision_MLA Atream 2025-02-18 14:44:50 +08:00
• 592e13d453 ⚡ add mmlu_pro test liam 2025-02-18 14:43:38 +08:00
• 1054ee3ae4 Merge pull request #450 from kvcache-ai/update-WeChatGroup Atream 2025-02-18 14:34:07 +08:00
• 261d5d393f Add files via upload Atream 2025-02-18 14:28:29 +08:00
• 0c1fcac2ea deploy: 2671fc7f4b Azure-Tang 2025-02-18 06:23:58 +00:00
• 2671fc7f4b Merge pull request #445 from dedfaf/main Azure 2025-02-18 14:23:29 +08:00
• 2259ff6749 deploy: ccf0f1910f UnicornChan 2025-02-18 05:59:00 +00:00
• ccf0f1910f Merge pull request #442 from Zhoneym/patch-1 UnicornChan 2025-02-18 13:58:44 +08:00
• 07a0555016 ⚡ fix device and add test liam 2025-02-18 12:52:17 +08:00
• 73d072f609 Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache ceerrep 2025-02-18 11:44:28 +08:00
• 2ffb43f974 fix: adapt prefix cache in forward_linux_flashinfer ceerrep 2025-02-18 11:40:15 +08:00
• f029588b61 fix server warmup Xie Weiyu 2025-02-18 11:39:45 +08:00
• 8b221f532a Fix `GLIBCXX_3.4.30' not found in Anaconda Zhoneym 2025-02-18 11:32:42 +08:00
• c70b6f4d5b fix: use 'cuda:0' by default if torch_device is 'cuda' ceerrep 2025-02-18 11:15:17 +08:00
• 3e5c5a2f4a Fix typo in DeepseekR1_V3_tutorial.md Line 174 dedfaf 2025-02-18 10:57:12 +08:00
• 9f1da18630 update install.sh Xie Weiyu 2025-02-17 20:50:19 +08:00
• c176e516b5 server mix mla Xie Weiyu 2025-02-17 20:40:28 +08:00
• ee24eb8dc3 fix: fix server for triton kernel ceerrep 2025-02-17 18:08:45 +08:00
• bb1cadfff3 Merge branch 'fix_precision_MLA' of https://github.com/kvcache-ai/ktransformers into server-prefix-cache ceerrep 2025-02-17 18:08:04 +08:00
• 038bc30888 fix precision bug imported by position_ids in 0.2.0 Atream 2025-02-17 09:23:14 +00:00
• 49c6e2fc04 Update README_ZH.md xubo 2025-02-17 16:56:44 +08:00
• 6d9182795c Fix cmake error caused by lack of environment variables in Windows environment hoshinohikari 2025-02-17 15:56:56 +08:00
• cd9f7f8f34 fix: server: drop <think> tag in chat template ceerrep 2025-02-17 14:14:10 +08:00
• bb39eeb005 Add notes to DeepSeek-R1 tutorial documentation John W. Leimgruber III 2025-02-16 13:40:27 -05:00
• ca2090d89b feat: use model name in openai endpoint ceerrep 2025-02-17 00:11:54 +08:00
• 5ac266085e fix: use flash_attn for faster prefill ceerrep 2025-02-17 00:11:24 +08:00
• bb0ccc7b1a feat: add prefix cache for server ceerrep 2025-02-17 00:10:55 +08:00
• 41a0b63068 deploy: 24de607be4 qiyuxinlin 2025-02-16 07:06:13 +00:00
• 2f26a7efd6 deploy: c515cc49a5 qiyuxinlin 2025-02-16 07:05:55 +00:00
• c515cc49a5 Merge pull request #355 from kvcache-ai/qiyuxinlin-patch-1 wang jiahao 2025-02-16 15:05:43 +08:00
• 1a03698c94 deploy: 24de607be4 qiyuxinlin 2025-02-16 07:05:32 +00:00