風的彷徨
ab3790faca
fix MHA singleton mask broadcast across backends ( #6869 )
...
Keep persistent KV caches on the blob allocator, broadcast single-channel attention masks across heads on CPU and Vulkan paths, and add long multi-token cache regressions.
2026-10-10 19:47:13 +08:00
nihui
60bfa40044
sdpa flash attention x86 optimization, llm models enable gqa, dispatch for avx2 bfloat2float ( #6923 )
...
* enable gqa
* update perf test
2026-09-01 17:35:42 +08:00
nihui
e665c48470
sdpa vulkan flash attention with cooperative matrix optimization ( #6528 )
...
* disable sm pad on old driver
2026-02-04 14:21:02 +08:00
nihui
ce2c5659b9
sdpa vulkan ( #6514 )
...
* sdpa vulkan
* unroll 2x2
* local memory opt
2026-01-16 17:41:29 +08:00
風的彷徨
9d294b2162
spda x86 optimization (using gemm & softmax) ( #6421 )
2025-11-26 10:43:13 +08:00
nihui
11fb99795b
sdpa kvcache ( #6405 )
2025-11-18 11:05:27 +08:00
風的彷徨