f74639deeb
gemm block quantization infrastructure for llm decoder style ( #6814 )
...
* another mha oob read
2026-07-13 10:13:39 +08:00
f6f734f44d
vulkan int8 kernels ( #6751 )
...
* vulkan: add int8 kernels for core layers
Implement Vulkan int8 paths for convolution, convolutiondepthwise, innerproduct, and gemm with scalar fallback and integer dot product shader branches. Add goal.md and enable existing int8 unit tests for Vulkan coverage.
* enable int8 gpu benchmark
2026-06-17 17:39:49 +08:00
706c4d1964
support 4d mat in more operators ( #6737 )
...
Add 4d Mat handling across generic and optimized backends, extend operator tests, and document supported input and output Mat dimensions.
2026-05-22 15:31:01 +08:00
8c18666821
feat(vulkan): persist pipeline cache ( #6702 )
...
Add single-file Vulkan pipeline cache save/load support with strict device, driver, shader, and payload validation. Cache compiled SPIR-V per shader hash alongside the driver pipeline cache blob, add focused cache tests, and document the current Vulkan pipeline cache architecture and usage.
Limit Vulkan layer test cache growth with a shared test-only pipeline cache window so repeated test_layer() calls can still exercise cache reuse.
Co-authored-by: futz12 <56149058+futz12@users.noreply.github.com >
Co-authored-by: CLV-Iclucia <89170890+CLV-Iclucia@users.noreply.github.com >
2026-05-09 09:37:06 +08:00
526e5e65ef
unroll 4x4 for gemm and sdpa vulkan, vectorize a and b loading, avoid bank conflict ( #6524 )
...
* unroll 4x4 for vulkan gemm, vectorize loading a and b
* unroll 4x4 for vulkan sdpa, vectorize loading a and b, avoid bank conflict
* Update glsl-extension.md
* Update glsl-extension.zh.md
* gemm vulkan subgroup optimization
* apply code-format changes
* enable it
---------
Co-authored-by: nihui <171016+nihui@users.noreply.github.com >
2026-01-28 13:55:23 +08:00
fe7c0cd2ec
dynamic vulkan loading on apple platforms, honor VK_DRIVER_FILES env ( #6499 )
2026-01-11 19:54:54 +08:00
41187d42b0
simplevk: add qcvkarm64xum.dll from qualcomm windows driver ( #6502 )
2026-01-11 17:36:28 +08:00
6499fc0348
vulkan bf16 infrastructure ( #6469 )
...
* skip gpu bf16 winograd
* update doc
2026-01-04 19:34:50 +08:00
559a8e8cb0
rotaryembed layer ( #6407 )
...
* add deepseek_v3 attention test
* fuse non interleaved rotary embed
* fuse t5 layernorm without gamma
* test qwen2 attention
* sdpa pattern ++
2025-11-19 17:28:08 +08:00
11fb99795b
sdpa kvcache ( #6405 )
2025-11-18 11:05:27 +08:00
641a3e5be8
ncnn sdpa layer ( #6397 )
...
* fuse gqa
2025-11-11 11:14:04 +08:00
63a7a8a590
optional layer support any packing ( #6392 )
...
* split gemm tests, enable gemm vulkan any packing
2025-11-07 14:23:53 +08:00
3ab4b58d1d
optional layer support vulkan packing ( #6389 )
...
* pyncnn expose layer support vulkan packing
* doc layer support behavior
2025-11-05 16:27:23 +08:00
adee6a0a53
multiheadattention kvcache ( #6333 )
...
* skip input layer forward
* unified kvcache layout
* mha kvcache test
2025-10-15 14:19:57 +08:00
a6eda68f23
disable onnx2ncnn build ( #6324 )
...
Migrate most of onnx2ncnn functions to pnnx
2025-09-23 19:06:07 +08:00
b5bc05e2e8
drop vulkan shader pack8 ( #6270 )
2025-08-22 11:05:59 +08:00
9b91fe5153
implement flip layer and pnnx torch.flip conversion ( #6233 )
...
Co-authored-by: 佰阅 <43716063+Baiyuetribe@users.noreply.github.com >
2025-08-06 14:49:16 +08:00
4644540ea4
Add Windows XP support merging PRs #6176 and #6177 ( #6204 )
...
Co-authored-by: Sugar-Baby <87747602+Sugar-Baby@users.noreply.github.com >
Co-authored-by: AtomAlpaca <66774326+AtomAlpaca@users.noreply.github.com >
2025-07-24 15:45:02 +08:00
44982d0d23
About the update to the GLSL documentation after the image functions are deprecated ( #6173 )
2025-07-12 14:22:30 +08:00
4c72f52954
docs: update Chinese glsl-extension documentation ( #6162 )
...
- synchronize the latest English content to the Chinese documentation
- correct spelling errors in the English version of glsl-extention
- Fix spelling 'enable_validation_layer' in src/gpu.cpp
2025-07-10 14:21:34 +08:00
bd0b111775
vulkan tight fp16p pack1 ( #6127 )
2025-06-16 23:09:35 +08:00
80da741307
glsl define ncnn_glsl_version macro ( #6003 )
2025-04-18 18:49:33 +08:00
ef0b0e631c
interp output size expression ( #5994 )
2025-04-17 14:24:57 +08:00
39c055d7f2
crop axes starts ends expression ( #5976 )
...
* skip dynamic tensor index
* handle clone oom
2025-04-14 19:43:50 +08:00
84970eed4d
vulkan validation layer enables NCNN_LOGE in shader source ( #5963 )
...
* NCNN_LOGE in glsl
* Update glsl-extension.md
2025-03-31 16:17:16 +08:00
6396a732ef
reshape shape expression, drop reshape permute, test reshape oom ( #5918 )
2025-03-10 15:31:34 +08:00
1e3fcb9dda
paramdict value string type, natural array representation ( #5915 )
2025-02-20 19:04:16 +08:00
0734b657d9
spectrogram and inverse spectrogram ( #5779 )
...
* only supports hann, hamming and all-one window
* inverse spectrogram does not support length parameter
* spectrogram always returns torch.view_as_real(out) as ncnn does not support complex typed mat yet
* inverse spectrogram always accepts torch.view_as_complex(in) as ncnn does not support complex typed mat yet
2024-11-22 16:14:54 +08:00
66b54cbea2
multiheadattention int8 quantization ( #5733 )
...
* x86 vulkan fallback
* comment about bf16s
2024-10-15 16:38:00 +08:00
1c7af00499
gemm int8 quantization ( #5706 )
...
* quantize gemm
* write gemm quantize scales
* update doc
* less openmp args
* x86 riscv fallback
* skip gemm vulkan int8
* fix noint8 test, fix arm bf16 test
* enable vfpv4 on neon build only
* fix gemm vulkan without C
* fp16 pack8 output
* enable elempack=8 only for asimdhp+
* tiled gemm int8 test
* opt arm64 tiles, fix asimdhp dispatch
2024-10-12 19:02:55 +08:00
5df5413c81
embed int8 quantization and add embed test ( #5667 )
2024-09-02 18:48:01 +08:00
fdf0df3079
RMSNorm ( #5630 )
2024-08-15 16:52:06 +08:00
4c3debae2d
multiheadattention scale param ( #5526 )
...
* update swiftshader
* skip vs2017 swiftshader
2024-06-23 23:51:02 +08:00
db035d602d
update ncnnoptimize layers, lightmode=false keeps original weight ( #5414 )
2024-04-09 23:53:54 +08:00
8ddc85f4dd
fix doc dual issue ( #5342 )
2024-02-22 19:13:31 +08:00
f35eb4b3b8
chore: Fix multiple typos ( #5301 )
...
Signed-off-by: hugo-syn <hugo.vincent@synacktiv.com >
2024-01-20 21:54:27 +08:00
5329d32e74
check vulkan fp16 uniform support and implement lfp conversion without fp16u ( #5287 )
2024-01-17 15:53:34 +08:00
c222208cc9
feat mask for disable threading, make some extractor setter no-op, update doc ( #5270 )
2024-01-08 17:42:08 +08:00
2ecaf37a3e
Fix find GPU driver dll path in windows ( #5141 )
2023-11-10 19:07:24 +08:00
b4f26237cb
in-house vulkan loader ( #5130 )
...
* vulkan-driver-loader.md
* static vulkan on apple
2023-11-09 16:58:23 +08:00
4494aadd74
deconvolution dynamic weight ( #5119 )
2023-10-31 17:37:24 +08:00
14e14a9ae8
slice with indices ( #5103 )
2023-10-25 17:12:03 +08:00
3f437d3f3d
Grid sample op ( #4373 )
...
* pnnx support grid_sample op
* complete the permute and gridsample operator fusion
* spilt calculation into two stages and support permute fusion
2023-10-20 19:09:59 +08:00
7e5fa3ade3
shrink operator ( #5022 )
2023-09-20 14:08:31 +08:00
bcfec1da33
Celu layer and export to ncnn ( #5019 )
2023-09-12 10:28:22 +08:00
c851231832
add diag layer and its converter ( #4935 )
2023-09-08 22:01:37 +08:00
4abadd2ffb
binaryop implicit broadcast B with 1 dimension rank for outer axis ( #4930 )
2023-08-10 21:29:49 +08:00
1e0d70af8c
Add translated document: glsl-extension.zh.md ( #4818 )
2023-07-02 11:47:44 +08:00
43aba6badb
Update glsl-extension.md
2023-06-26 11:48:14 +08:00
172b748c74
add ncnn glsl extension doc ( #4817 )
2023-06-26 11:13:41 +08:00
nihui