Atream
b84524622e
support bf16 read
2025-02-16 06:43:27 +00:00
Azure
9f9c3738f9
Merge pull request #354 from Azure-Tang/fix-mockTritonMLA
...
[fix] Mock triton mla due to precision issue
2025-02-16 14:06:54 +08:00
Azure
ff6b265e53
Mock triton mla due to precision issue
2025-02-16 06:03:12 +00:00
Atream
c5f036e8a4
Merge pull request #333 from kvcache-ai/feat_experts_gpu
...
toy support for experts on GPU, no CUDA Graph
2025-02-15 23:30:24 +08:00
Atream
8ed8eb2a9e
Update FAQ.md
2025-02-15 23:27:35 +08:00
Atream
c189d55bd1
toy support for experts on GPU, no CUDA Graph
2025-02-15 15:16:00 +00:00
wang jiahao
ae8da019c1
Merge pull request #330 from hrz6976/fix-nonetype
...
thanks..., I was about to submit and found that you had already modified it. Thank you for your contribution
2025-02-15 22:50:51 +08:00
Azure
56382aa869
Merge pull request #290 from ZhangShuaiyi/dev/check_gguf_path
...
ensure that gguf_path argument is a directory.
2025-02-15 22:48:57 +08:00
Azure
c8bf250104
Merge pull request #324 from BiFangKNT/patch-2
...
Update DeepseekR1_V3_tutorial.md
2025-02-15 22:23:09 +08:00
12f23eddde
4516282ccc
Fix NoneType object has no attribute zero_
2025-02-15 22:04:45 +08:00
ZiWei Yuan
3c6035aa8a
Merge pull request #329 from kvcache-ai/fix_doc
...
📝 fix typo
2025-02-15 21:45:19 +08:00
liam
69b00753c2
📝 fix typo
2025-02-15 21:44:40 +08:00
彼方
bd693b698f
Update DeepseekR1_V3_tutorial.md
2025-02-15 19:43:18 +08:00
UnicornChan
65d73ea3f8
Merge pull request #317 from kvcache-ai/develop-0.2.1
...
[feature] update docker image and entrypoint
2025-02-15 16:19:18 +08:00
ZiWei Yuan
718a71b3a9
Merge pull request #316 from KMSorSMS/main
...
📝 update V0.2.1 Doc
2025-02-15 16:19:00 +08:00
liam
13382f88ab
📝 update V0.2.1 Doc
2025-02-15 16:17:05 +08:00
chenxl
0e4b7a3929
[feature] update docker image and entrypoint
2025-02-15 07:55:33 +00:00
Atream
f9f9f746c0
Merge pull request #315 from kvcache-ai/Atream-add-adapted
...
Atream add adapted
2025-02-15 15:45:34 +08:00
Atream
92399283b6
Update attention.py
2025-02-15 15:43:35 +08:00
Atream
d90749d35d
Update triton_attention.py
2025-02-15 15:41:01 +08:00
Shuaiyi
22280bf17f
get dirname if gguf_path is a file
2025-02-15 07:08:22 +00:00
Azure
1548c99234
Merge pull request #307 from Azure-Tang/main
...
[update] Reorganize documentation/README
2025-02-15 10:06:56 +08:00
Azure
227e81b0d3
update zh readme
2025-02-15 01:54:01 +00:00
Azure
ef89b1520b
* Reorganize documentation/README
...
* Consolidate the installation section, as it's currently too cluttered
* Move the Multi-GPU section to the top-level structure
* Add a **detailed** tutorial on registering extra GPU memory with Marlin
2025-02-14 19:58:26 +00:00
Azure
b0b90270d8
Merge pull request #306 from kvcache-ai/revert-305-main
...
Revert "[update] Reorganize documentation/README"
2025-02-15 03:44:42 +08:00
Azure
4f4ed36442
Revert "[update] Reorganize documentation/README"
2025-02-15 03:43:48 +08:00
Azure
19d4a50b1c
[update] Reorganize documentation/README
...
[update] Reorganize documentation/README
2025-02-15 03:41:43 +08:00
Azure
483182fc3a
fix typo and detail
2025-02-14 19:40:15 +00:00
Azure
823b25eec9
Reorganize documentation/README
2025-02-14 19:08:17 +00:00
Atream
cc8d627e32
Merge pull request #301 from kvcache-ai/fix-cuda-graph-bug
...
warm_up before capture
2025-02-14 23:54:43 +08:00
Atream
1946493f2d
warm_up before capture
2025-02-14 15:52:21 +00:00
Atream
cadd55078f
Merge pull request #295 from kvcache-ai/update-wechatgroup
...
Update wechatgroup
2025-02-14 19:52:05 +08:00
Atream
e153d78227
Add files via upload
2025-02-14 19:51:11 +08:00
Atream
96e6dff7ac
Delete WeChatGrouop.png
2025-02-14 19:49:14 +08:00
Atream
885a91e7db
Merge pull request #294 from kvcache-ai/feat-fast-MLA
...
Feat fast mla
2025-02-14 19:40:36 +08:00
Atream
1084d4e4b4
linux support triton MLA kernel
2025-02-14 11:38:55 +00:00
Azure
6738908699
Merge pull request #280 from Azure-Tang/main
...
[fix] Fix incorrect image content in the document
2025-02-14 17:12:14 +08:00
Azure
1b1f417267
Fix incorrect image content in the document
2025-02-14 09:04:22 +00:00
Azure
f4bb374eaf
Merge pull request #254 from Azure-Tang/main
...
[Update] Add V3/R1 8 gpu yaml example
2025-02-14 11:54:14 +08:00
Azure
95c81eaf01
Merge branch 'kvcache-ai:main' into main
2025-02-14 11:53:52 +08:00
Atream
0a9c59922a
Merge pull request #255 from kvcache-ai/update-wechatgroup
...
Add files via upload
2025-02-14 11:08:59 +08:00
Atream
ce7210321a
Add files via upload
2025-02-14 11:06:56 +08:00
Azure
b7653b9c4f
add V3/R1 8 gpu yaml example
2025-02-14 02:56:13 +00:00
Azure
e612b14739
Merge pull request #247 from liugddx/patch-1
...
[Doc]Fix dead link problem
2025-02-14 10:37:32 +08:00
Azure
ae5d9e11a9
Merge pull request #227 from hrz6976/main
...
Add a lock to server inference()
2025-02-14 10:35:11 +08:00
Guangdong Liu
e65be580ab
Fix dead link problem
2025-02-14 09:57:57 +08:00
Atream
bb35dc5b0d
init support for MLA using Attention kernel
2025-02-13 15:01:14 +00:00
ZiWei Yuan
a456e25a54
Merge pull request #200 from devin2255/main
...
add README_ZH.md
2025-02-13 22:22:25 +08:00
dhliu
d04b570fb5
edit README_ZH.md && add DeepseekR1_V3_tutorial_zh.md
2025-02-13 21:14:44 +08:00
Atream
aa21edd2fe
Merge pull request #230 from kvcache-ai/updata-wechatgroup-1
...
Updata wechatgroup 1
2025-02-13 19:33:51 +08:00
Atream