wang jiahao
47a89ae752
Merge pull request #1031 from wangkuigang-yewu-cmss/doc-update
...
文档更新：model_path名字要求以及在示例中添加force_think
2025-04-03 15:58:12 +08:00
wangkuigang-yewu-cmss
c590583262
doc upgrade: model_path requirements and reasoning
...
* add documentations about `--model_path` requirements
* add `--force_think` in doc (most users would run R1 and would want it to provide reasoning process)
2025-04-03 15:16:56 +08:00
wang jiahao
72e8e16fa4
Merge pull request #1029 from kvcache-ai/mian-update-doc
...
fix local_chat bug and update doc
2025-04-03 12:44:59 +08:00
dongjw
1b7672937b
update install doc and fix local_chat bug
2025-04-03 12:42:41 +08:00
dongjw
ab0b0f4ea1
fix local_chat and update balance-serve and SUMMARY doc
2025-04-03 12:19:43 +08:00
ZiWei Yuan
9654bc1c5b
Merge pull request #1027 from kvcache-ai/KMSorSMS-patch-1
...
Update SUMMARY.md
2025-04-03 12:02:18 +08:00
ZiWei Yuan
a0ce48ee21
Update SUMMARY.md
2025-04-03 12:00:34 +08:00
wang jiahao
f7a8a91f46
Merge pull request #1024 from kvcache-ai/mian-update-doc
...
delete sudo install
2025-04-03 10:51:54 +08:00
dongjw
8acb270c90
delete sudo install
2025-04-03 10:46:52 +08:00
Atream
795524cacc
Merge pull request #954 from aubreyli/yaml_marlin_fix
...
yaml: fix Marlin AssertionError
2025-04-02 15:00:37 +08:00
Atream
ec12429c46
Merge pull request #1005 from fishingfly/improve/backend-error-msg
...
fix: refine backend error message to include ROCM_HOME
2025-04-02 14:54:23 +08:00
wang jiahao
ac95b6c710
Merge pull request #1015 from kvcache-ai/qiyuxinlin-patch-1
...
Update balance-serve.md
2025-04-02 14:22:30 +08:00
wang jiahao
ee179c2ad0
Update balance-serve.md
2025-04-02 14:22:15 +08:00
wang jiahao
a41d216393
Merge pull request #1013 from kvcache-ai/work-concurrent
...
In v0.2.4 version, we’ve added highly desired multi-concurrency support to the community through a major refactor of the whole architecture.
2025-04-02 14:09:10 +08:00
dongjw
4ed9744ebb
update readme
2025-04-02 14:02:57 +08:00
dongjw
b62cefaec9
update readme
2025-04-02 13:11:01 +08:00
dongjw
d41dd23b14
update Dockerfile
2025-04-02 12:10:58 +08:00
dongjw
65798994cb
update Dockerfile
2025-04-02 12:04:09 +08:00
dongjw
56a18ad02c
change tag v0.2.4
2025-04-01 21:07:13 +08:00
Azure-Tang
d98433c2d1
update git action env, add USE_BALANCE_SERVE=1
2025-04-01 12:58:28 +00:00
dongjw
5c7ed7b579
fix top_p = 0 bug
2025-04-01 20:38:33 +08:00
Azure-Tang
aeabd783b0
update git action env, add BALANCE_SERVE=1
2025-04-01 11:21:55 +00:00
Azure-Tang
31677181c3
Fix ktransformers-server flashinfer wrapper position arg issue;
...
Fix db position issue
2025-04-01 07:30:23 +00:00
Azure-Tang
203b853c75
rm KMoEGateDeepSeekV3, fall back to KMoEGate
2025-04-01 07:13:05 +00:00
Azure-Tang
3a5330b215
Merge branch 'main' into work-concurrent
2025-04-01 06:48:19 +00:00
fishingfly
7549ff335a
fix: refine backend error message to include ROCM_HOME
...
Signed-off-by: fishingfly <zhoyuzf@163.com >
2025-04-01 10:50:38 +08:00
Atream
80c5cbecdd
add nlohmann
2025-04-01 10:38:45 +08:00
Atream
9360d1e3c8
add submodules
2025-03-31 23:20:29 +08:00
Atream
25cee5810e
add balance-serve, support concurrence
2025-03-31 22:55:32 +08:00
Atream
8d0292aa44
refactor folders
2025-03-31 22:45:37 +08:00
Atream
f142f4dff3
Merge pull request #956 from kvcache-ai/Atream-patch-7
...
Update README.md
2025-03-22 12:14:48 +08:00
Atream
d4c6c2bb02
Update README.md
2025-03-22 12:14:36 +08:00
Aubrey Li
a12e8ab46e
yaml: fix Marlin AssertionError
...
Marlin quantized linear only supports GPU device, when change generate_op
to "KLinearMarlin", generate_device need to be changed to "cuda" accordingly.
Fixes: e5b001d76f
2025-03-21 23:58:20 +08:00
Jiaqi Liao
05f6cede37
Merge pull request #943 from SkqLiao/main
...
fix benchmark params for human eval benchmark
2025-03-20 18:49:34 +08:00
SkqLiao
6d4626a5d9
fix params
2025-03-20 18:48:51 +08:00
Atream
ddd35d5be9
Merge pull request #940 from kvcache-ai/Atream-patch-6
...
Update gate.py
2025-03-20 14:54:20 +08:00
Atream
633af5d235
Update gate.py
2025-03-20 14:54:01 +08:00
SkqLiao
8cc4df980e
use DeepSeek V3 instead of R1 for benchmarking
2025-03-20 11:59:03 +08:00
Jiaqi Liao
32a91c78c1
Merge pull request #935 from SkqLiao/main
...
Fix benchmarking slow issue on self-hosted actions
2025-03-20 10:14:37 +08:00
SkqLiao
e7d7d2705c
rename CI/CD
2025-03-20 10:11:24 +08:00
SkqLiao
19c824f9d0
change cpu-infer due to actual cpu cores on self-hosted server.
2025-03-20 10:10:52 +08:00
Jiaqi Liao
649489dc67
Merge pull request #931 from SkqLiao/main
...
Add Human Eval Benchmark Test for CI/CD
2025-03-19 21:35:24 +08:00
SkqLiao
bad334fa5b
fix path
2025-03-19 21:28:58 +08:00
SkqLiao
bc369b256c
add CI/CD for human eval score benchmarking
2025-03-19 21:25:21 +08:00
Atream
8be56a0190
Merge pull request #927 from kvcache-ai/fix-gate-precision
...
Update gate.py
2025-03-19 16:16:31 +08:00
Atream
b453333f60
Update gate.py
2025-03-19 16:14:54 +08:00
Atream
6ca233cca3
Merge pull request #926 from kvcache-ai/Atream-patch-5
...
Update gate.py
2025-03-19 12:17:09 +08:00
Atream
44599229cd
Update gate.py
2025-03-19 12:16:48 +08:00
Atream
aa8f985f85
Merge pull request #925 from kvcache-ai/fix-gate-compile
...
fix-gate-compile
2025-03-19 11:44:41 +08:00
Atream
114995355b
fix-gate-compile
2025-03-19 11:27:18 +08:00
wang jiahao