ab3790faca
fix MHA singleton mask broadcast across backends ( #6869 )
...
Keep persistent KV caches on the blob allocator, broadcast single-channel attention masks across heads on CPU and Vulkan paths, and add long multi-token cache regressions.
2026-10-10 19:47:13 +08:00
9f9d4ec815
reduce gpu test cases ( #7017 )
2026-10-01 23:33:05 +08:00
60bfa40044
sdpa flash attention x86 optimization, llm models enable gqa, dispatch for avx2 bfloat2float ( #6923 )
...
* enable gqa
* update perf test
2026-09-01 17:35:42 +08:00
7f162381ab
kvcache allocator ( #6901 )
...
* decode benchmark
* reject kvcache batch
* enable mha vulkan gemm out_elempack 1
2026-08-17 15:44:30 +08:00
92b73d6665
x86/arm/mips/loongarch/riscv gemm block quantization optimization ( #6831 )
...
* benchncnn_llm int8 models
2026-08-04 14:34:22 +08:00
f74639deeb
gemm block quantization infrastructure for llm decoder style ( #6814 )
...
* another mha oob read
2026-07-13 10:13:39 +08:00
風的彷徨