ab3790faca
fix MHA singleton mask broadcast across backends ( #6869 )
...
Keep persistent KV caches on the blob allocator, broadcast single-channel attention masks across heads on CPU and Vulkan paths, and add long multi-token cache regressions.
2026-10-10 19:47:13 +08:00
b9af93c74b
Add percentile PTQ method in ncnn2table ( #6854 )
2026-10-09 10:33:34 +08:00
7f162381ab
kvcache allocator ( #6901 )
...
* decode benchmark
* reject kvcache batch
* enable mha vulkan gemm out_elempack 1
2026-08-17 15:44:30 +08:00
f74639deeb
gemm block quantization infrastructure for llm decoder style ( #6814 )
...
* another mha oob read
2026-07-13 10:13:39 +08:00
3724d10fc8
tools:Move Embed/MHA/RNN/LSTM/GRU weight scale generation to ncnn2table ( #6688 )
...
Co-authored-by: nihui <nihuini@tencent.com >
2026-05-18 19:38:07 +08:00
4699b26933
Using snprintf instead of sprintf for a tools ( #6554 )
...
Signed-off-by: Evgeny Proydakov <e.proydakov@gmail.com >
2026-02-26 10:29:28 +08:00
641a3e5be8
ncnn sdpa layer ( #6397 )
...
* fuse gqa
2025-11-11 11:14:04 +08:00
171b9d1bba
use spdx license header, copyright Tencent ( #6152 )
2025-07-08 11:01:29 +08:00
701f53794d
add read npy for ncnn2table ( #5930 )
...
* add read npy for ncnn2table
* optimize npy to mat to avoid non-contiguous memory data
* add create the calibration table from npy
* add catch exception
2025-03-18 16:50:36 +08:00
9afeb17cd4
ncnn2int8 use fp16 weight type by default ( #5907 )
2025-02-17 17:26:25 +08:00
66b54cbea2
multiheadattention int8 quantization ( #5733 )
...
* x86 vulkan fallback
* comment about bf16s
2024-10-15 16:38:00 +08:00
1c7af00499
gemm int8 quantization ( #5706 )
...
* quantize gemm
* write gemm quantize scales
* update doc
* less openmp args
* x86 riscv fallback
* skip gemm vulkan int8
* fix noint8 test, fix arm bf16 test
* enable vfpv4 on neon build only
* fix gemm vulkan without C
* fp16 pack8 output
* enable elempack=8 only for asimdhp+
* tiled gemm int8 test
* opt arm64 tiles, fix asimdhp dispatch
2024-10-12 19:02:55 +08:00
5df5413c81
embed int8 quantization and add embed test ( #5667 )
2024-09-02 18:48:01 +08:00
08b7d99a75
rnn/lstm/gru dynamic quantization ( #5435 )
2024-05-08 19:25:51 +08:00
1e4daff075
fix ncnn tools crash with compound layer creation, write gemm weight data ( #5416 )
...
* fix ncnn tools crash with compound layer creation
* write gemm weight data
2024-04-10 15:09:08 +08:00
1c8f1ba7c6
Fixed LGTM warnings ( #3891 )
...
src/stb_image.h
src/layer/detectionoutput.cpp
tools/quantize/imreadwrite.cpp
2022-06-09 16:07:56 +08:00
5400028e12
fix ncnn2table crash in lightmode ( #3898 )
...
* fix crash in ncnn2table for missing weight data in lightmode
* Update modelwriter.h
2022-06-08 19:26:44 +08:00
a2e25d3fd3
fix Multiplication result may overflow 'int' before it is converted to 'size_t'. ( #3757 )
2022-05-05 09:49:03 +08:00
6ef0f51f40
fix: do not find opencv when simpleocv specified for examples and quantize tool ( #3393 )
2021-12-01 22:20:54 +08:00
e24a2e6c40
[fix] fix typo ( #3377 )
2021-11-23 21:44:36 +08:00
0b12f5ee07
update readme of quantize ( #3283 )
2021-10-15 16:42:06 +08:00
52c6502ae7
Feature/ncnntool/add install ( #3212 )
2021-09-10 14:08:10 +08:00
fd1d5bb2ba
Update ncnn2table.cpp ( #3081 )
...
修正eq 校准中imagelists错误的索引
2021-07-14 16:24:00 +08:00
8fad97cc68
[ncnn2table]do not always resize image ( #3023 )
...
* it's possible not to resize image
2021-06-24 19:48:47 +08:00
e47ab30cf5
fix ncnn2table specification ( #3006 )
2021-06-17 10:38:20 +08:00
71bc617a05
better ncnn2table multithreading, print parsed parameters, print progress
2021-05-29 21:43:29 +08:00
312c7372c1
fix norm parsing for long decimal
2021-05-28 17:19:27 +08:00
1ad37fae1e
generate random weight when null bin passed
2021-05-24 15:54:12 +08:00
00b0094b2f
copy-pasta makes msvc happy
2021-05-20 19:23:36 +08:00
9a12597dff
build ncnn quantization tool even opencv not found
2021-05-20 17:51:56 +08:00
34bd5ef161
update eq quant info
2021-04-29 10:44:08 +08:00
d021079cd3
implement ncnn2table eq method
2021-04-28 15:51:58 +08:00
7d10e5593b
fix build
2021-04-27 23:22:16 +08:00
45e548e342
rewrite ncnn2table tool, with better multithreading processing
2021-04-27 19:35:46 +08:00
79927a9eeb
update quantized int8 inference doc
2021-04-16 11:47:57 +08:00
56d74580a0
fix uninitialized input and weight
2021-04-15 17:45:12 +08:00
2b1f8bea25
fix float locale issue in ncnn2mem, fix long line parsing in ncnn2int8
2021-04-14 18:59:28 +08:00
5fe75f19ef
architecture changes for int8 packing ( #2771 )
...
* quantize and dequantize tests
* unify activation and usability function
* drop NCNN_REQUANT cmake option, test dequantize requantize pack8, fix webassembly build
* benchmark use requantize int8 model
2021-03-27 18:49:13 +08:00
f3c48cd063
Fix bug of adaptive pool in int8 quantization model ( #2637 )
2021-01-28 12:48:31 +08:00
30a4482506
fix typo in comment in ncnn2int8.cpp ( #2627 )
2021-01-26 21:48:01 +08:00
54c0a13b9f
build shared library ( #2525 )
...
* build shared lib and enable lto
* reserved for layer and option
* allocator pimpl
* datareader pimpl
* paramdict pimpl, disable copy assign for allocator and datareader
* modelbin pimpl
* net extractor pimpl
* gpu pimple
* disable copy assign vulkandevice, code format
* command pimpl, dummy image readonly
* pipeline pipelinecache pimpl, export platform class
* code format, export simple family
* update ci
* disable lto on android armv7, merge webassembly ci
* link libgcc, fix macos dylib version
* pipeline pimpl, gpu info pimpl
* destroy gpu info after vulkan device
* ignore msvc stl class warning
* fix ncnn_paramdict_get_float return type
* fix vktransfer upload fp16 without flatten, add command test
2021-01-01 22:55:10 +08:00
cab255f107
Fix compile failure when NCNN_PIXEL off ( #2260 )
...
* fix compiling failure on cxx14
* fix compile failure when NCNN_PIXEL off
* do not export pixel api when NCNN_PIXEL off
* disable squeezenet test when NCNN_PIXEL off
* do not build ncnn2table and examples when NCNN_PIXEL off
* make cmakedefine header independent
* restore platform.h.in
Co-authored-by: nihui <shuizhuyuanluo@126.com >
Co-authored-by: nihuini <nihuini@tencent.com >
2020-11-13 19:39:06 +08:00
266382a832
fix cmake 3.5.1 config error ( #2067 )
...
* fix cmake 3.5.1 config error
* fix ubuntu16.04 build error
2020-09-04 08:58:19 +08:00
3b1b41ec0b
Add some compile options, add vulkan dependency export ( #2062 )
...
* vulkan cmake export templete
* 1) vulkan cmake dependency export. 2) support opencv_world import. 3) add BUILD_WITH_STATIC_CRT option
* Threads dependency
* NCNN_BUILD_WITH_STATIC_CRT option
* we do not support cmake before version 3.15 for option NCNN_BUILD_WITH_STATIC_CRT
2020-09-03 15:51:28 +08:00
b80b84fda5
fix #1542 ; fix avx2 uint16_t including ( #1968 )
...
* fix #1542 ; fix avx2 uint16_t including
for #1542 , it is for compatibility for opencv 2.x, such as on ubuntu 16.04 apt installed opencv
2020-07-27 18:39:39 +08:00
3ef995ed1e
format code style and setup restyled.io ( #1840 )
2020-06-13 22:17:51 +08:00
f09ac067a5
update quantize doc ( #1831 )
2020-06-10 11:23:48 +08:00
ce2251db05
Improve ROIAlign (accelerate ROIAlign, support sampling ratio and aligned ROIAlign) ( #1820 )
...
* update roialign
* update tool for roialign
* fix ceil for roialign
* fix ios build
* fix
* fix
* make it compatibile with the old version
* fix ios build
* trigger CI
* fix test
* order
* build
* remove code
* merge roialign
* accelerate ROIAlign
* update note
* rename func
* roialign version
* trigger CI
* fix roialign
* use ref for pre-calc in roialign
* retrigger CI
* pre-alloc
2020-06-08 21:40:24 +08:00
ceef2470a5
Add logger.h ( #1753 )
2020-05-12 19:33:21 +08:00
8a84077429
ncnnoptimize lstm
2020-03-15 11:56:23 +08:00
風的彷徨