8bc95a1e7f
pnnx: store bool MemoryData attributes as raw int8 ( #7006 )
...
* pnnx: store bool MemoryData attributes as raw int8
Keep bool constants as one byte per element and select MemoryData
load_type=3. Implement the reserved raw int8 ModelBin loading mode and
pad exported bool data to four bytes so subsequent weights stay aligned.
This preserves the byte representation used by the proposed logical ops.
Add regression coverage for scalar and padded bool exports in both fp16
modes, multidimensional MemoryData, raw int8 loading through reference
and copy readers, trailing fp32 weights, and missing padding. Document
the format.
Based on the bool attribute serialization fixes proposed in:
https://github.com/Tencent/ncnn/pull/6956
https://github.com/Tencent/ncnn/pull/6866
Fixes #6855
* tools: preserve byte MemoryData when optimizing models
Save MemoryData load_type so raw int8 constants retain their byte
representation after ModelWriter saves the model. Skip fusion paths that
interpret MemoryData as float32 when its elements are not four bytes.
This change does not add bool arithmetic support or automatic type
promotion. Byte bool storage is intended for future logical, comparison
and selection operators.
Validated ncnnoptimize save and reload for scalar and one-to-five-element
bool constants in fp32 and fp16 storage modes, preserving byte values
and element size in all 12 cases.
---------
Co-authored-by: ktz03 <2484593937@qq.com >
Co-authored-by: Giulio Corradi <53930027+giuliocorradi@users.noreply.github.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
2026-09-17 10:09:00 +08:00
60bfa40044
sdpa flash attention x86 optimization, llm models enable gqa, dispatch for avx2 bfloat2float ( #6923 )
...
* enable gqa
* update perf test
2026-09-01 17:35:42 +08:00
92b73d6665
x86/arm/mips/loongarch/riscv gemm block quantization optimization ( #6831 )
...
* benchncnn_llm int8 models
2026-08-04 14:34:22 +08:00
f74639deeb
gemm block quantization infrastructure for llm decoder style ( #6814 )
...
* another mha oob read
2026-07-13 10:13:39 +08:00
706c4d1964
support 4d mat in more operators ( #6737 )
...
Add 4d Mat handling across generic and optimized backends, extend operator tests, and document supported input and output Mat dimensions.
2026-05-22 15:31:01 +08:00
559a8e8cb0
rotaryembed layer ( #6407 )
...
* add deepseek_v3 attention test
* fuse non interleaved rotary embed
* fuse t5 layernorm without gamma
* test qwen2 attention
* sdpa pattern ++
2025-11-19 17:28:08 +08:00
11fb99795b
sdpa kvcache ( #6405 )
2025-11-18 11:05:27 +08:00
641a3e5be8
ncnn sdpa layer ( #6397 )
...
* fuse gqa
2025-11-11 11:14:04 +08:00
adee6a0a53
multiheadattention kvcache ( #6333 )
...
* skip input layer forward
* unified kvcache layout
* mha kvcache test
2025-10-15 14:19:57 +08:00
a6eda68f23
disable onnx2ncnn build ( #6324 )
...
Migrate most of onnx2ncnn functions to pnnx
2025-09-23 19:06:07 +08:00
9b91fe5153
implement flip layer and pnnx torch.flip conversion ( #6233 )
...
Co-authored-by: 佰阅 <43716063+Baiyuetribe@users.noreply.github.com >
2025-08-06 14:49:16 +08:00
ef0b0e631c
interp output size expression ( #5994 )
2025-04-17 14:24:57 +08:00
39c055d7f2
crop axes starts ends expression ( #5976 )
...
* skip dynamic tensor index
* handle clone oom
2025-04-14 19:43:50 +08:00
6396a732ef
reshape shape expression, drop reshape permute, test reshape oom ( #5918 )
2025-03-10 15:31:34 +08:00
0734b657d9
spectrogram and inverse spectrogram ( #5779 )
...
* only supports hann, hamming and all-one window
* inverse spectrogram does not support length parameter
* spectrogram always returns torch.view_as_real(out) as ncnn does not support complex typed mat yet
* inverse spectrogram always accepts torch.view_as_complex(in) as ncnn does not support complex typed mat yet
2024-11-22 16:14:54 +08:00
66b54cbea2
multiheadattention int8 quantization ( #5733 )
...
* x86 vulkan fallback
* comment about bf16s
2024-10-15 16:38:00 +08:00
1c7af00499
gemm int8 quantization ( #5706 )
...
* quantize gemm
* write gemm quantize scales
* update doc
* less openmp args
* x86 riscv fallback
* skip gemm vulkan int8
* fix noint8 test, fix arm bf16 test
* enable vfpv4 on neon build only
* fix gemm vulkan without C
* fp16 pack8 output
* enable elempack=8 only for asimdhp+
* tiled gemm int8 test
* opt arm64 tiles, fix asimdhp dispatch
2024-10-12 19:02:55 +08:00
5df5413c81
embed int8 quantization and add embed test ( #5667 )
2024-09-02 18:48:01 +08:00
fdf0df3079
RMSNorm ( #5630 )
2024-08-15 16:52:06 +08:00
4c3debae2d
multiheadattention scale param ( #5526 )
...
* update swiftshader
* skip vs2017 swiftshader
2024-06-23 23:51:02 +08:00
db035d602d
update ncnnoptimize layers, lightmode=false keeps original weight ( #5414 )
2024-04-09 23:53:54 +08:00
4494aadd74
deconvolution dynamic weight ( #5119 )
2023-10-31 17:37:24 +08:00
14e14a9ae8
slice with indices ( #5103 )
2023-10-25 17:12:03 +08:00
3f437d3f3d
Grid sample op ( #4373 )
...
* pnnx support grid_sample op
* complete the permute and gridsample operator fusion
* spilt calculation into two stages and support permute fusion
2023-10-20 19:09:59 +08:00
7e5fa3ade3
shrink operator ( #5022 )
2023-09-20 14:08:31 +08:00
bcfec1da33
Celu layer and export to ncnn ( #5019 )
2023-09-12 10:28:22 +08:00
c851231832
add diag layer and its converter ( #4935 )
2023-09-08 22:01:37 +08:00
172b748c74
add ncnn glsl extension doc ( #4817 )
2023-06-26 11:13:41 +08:00
c28c8c04a1
multiheadattention attn mask ( #4668 )
2023-04-28 12:28:38 +08:00
b640574b88
rough vulkan gemm and multiheadattention ( #4618 )
2023-04-26 15:13:27 +08:00
afc9310c62
update new operators for modelwriter ( #4540 )
2023-02-21 13:30:05 +08:00
fc6ce4a641
copyto operator ( #4522 )
2023-02-14 15:22:38 +08:00
242e775d21
pnnx convert torch log10, pow 2 as square ( #4518 )
2023-02-09 19:28:18 +08:00
246e71c526
implement atan2 ( #4516 )
2023-02-09 00:29:18 +08:00
92e75105c9
Support torch.cumsum ( #4505 )
2023-02-08 10:50:50 +08:00
ab4cfbf5b0
enrich ncnn binary broadcast rules ( #4513 )
2023-02-08 10:22:30 +08:00
fed99fd35b
gemm output transpose, prepack c ( #4479 )
...
* mha is now permute and reshape free
* gemm user defined tile mnk param
2023-01-27 13:03:04 +08:00
10e9d91576
Add x86 MultiHeadAttention ( #4443 )
...
* fix doc, sync x86 gemm fix
Co-authored-by: EdVince <EdVince@users.noreply.github.com >
Co-authored-by: nihuini <nihuini@tencent.com >
2023-01-04 17:28:20 +08:00
fd1ac3c7a0
x86 optimization for gemm unified elempack ( #4387 )
2022-12-10 17:36:50 +08:00
eceac35a7f
implement MultiheadAttention kdim vdim ( #4347 )
2022-11-14 12:46:51 +08:00
6a47f8d15c
gridsample op support ( #4288 )
...
Co-authored-by: LRY89757 <LRY89757@users.noreply.github.com >
Co-authored-by: nihuini <nihuini@tencent.com >
Co-authored-by: nihui <shuizhuyuanluo@126.com >
2022-11-11 20:55:57 +08:00
5281d51535
implement GLU and pnnx conversion ( #4283 )
2022-10-19 12:41:23 +08:00
77eda4c19f
implement lstm proj_size ( #4263 )
2022-10-14 10:54:03 +08:00
720f3c9aab
Add DeformableConv2D ( #4070 )
...
* Add DeformableConv2D
* add unittest and docs
* pnnx torchvision deformconv2d conversion
Co-authored-by: miemie2013 <miemie2013@users.noreply.github.com >
Co-authored-by: nihui <shuizhuyuanluo@126.com >
2022-07-28 11:45:09 +08:00
2880eff264
deconv1d deconv3d ( #3584 )
...
* fix sigmoid returns nan with very large input
2022-03-01 21:50:17 +08:00
bc188ece58
update modelwriter for new operators
2022-02-25 11:27:01 +08:00
c5d7f963b9
layer tile ( #3491 )
2022-01-10 15:05:07 +08:00
014387dfae
update operators doc
2021-12-31 17:47:23 +08:00
e8d1d40398
update operators doc
2021-12-07 11:10:22 +08:00
f10cc6dd93
initial data structure changes for 3dcnn, conv3d, pooling3d ( #3378 )
...
Co-authored-by: ElvisYu <elvisyuovo@gmail.com >
Co-authored-by: 余浩文 <m18107220188@163.com >
Co-authored-by: Zr2223 <67497651+Zr2223@users.noreply.github.com >
2021-12-02 16:50:55 +08:00
nihui