Commit Graph

283 Commits

Author SHA1 Message Date
nihui f74639deeb gemm block quantization infrastructure for llm decoder style (#6814)
* another mha oob read
2026-07-13 10:13:39 +08:00
nihui f6f734f44d vulkan int8 kernels (#6751)
* vulkan: add int8 kernels for core layers
Implement Vulkan int8 paths for convolution, convolutiondepthwise, innerproduct, and gemm with scalar fallback and integer dot product shader branches. Add goal.md and enable existing int8 unit tests for Vulkan coverage.
* enable int8 gpu benchmark
2026-06-17 17:39:49 +08:00
securekim cb137e07dc docs: add guide for AHB zero-copy input on Android (#6733)
* docs: add guide for AHB zero-copy input on Android

Address #5531 - the AHB import API (VkAndroidHardwareBufferImageAllocator,
ImportAndroidHardwareBufferPipeline) has no user-facing docs today and
existing users hit linker errors against the official prebuilt because
build-android.cmd targets -DANDROID_PLATFORM=android-21 which strips the
AHB symbols.

The new page covers build prerequisites (__ANDROID_API__ >= 26),
AImageReader_newWithUsage setup, the per-frame call sequence, per-AHB-
pointer pipeline caching (~5000x setup-cost reduction), and trouble-
shooting for #5531 / #5190.

* docs(ahb): add ex.input(VkMat) format-matching caveat + soften cache bound

Two findings from a follow-up spike on Adreno 830:

- ex.input(VkMat) does NOT auto-convert format. The previous draft's hint
  "// ex.input(\"data\", dst, cmd);" actually crashes if the network was
  loaded with use_fp16_storage / use_fp16_packed (common on Vulkan path) -
  NetPrivate::forward_layer recurses on itself looking for an implicit
  converter that doesn't exist, observed as a ~180-frame stack overflow at
  net.cpp:251. Add a dedicated caveat explaining the failure mode and the
  vkdev->convert_packing workaround, and weaken the snippet hint accordingly.

- AImageReader's AHB pool isn't strictly bounded by maxImages. Under steady
  state it stays at maxImages (3 -> 3 cache_n in 36 consecutive cycles), but
  surface/focus events can transiently push it higher (observed cache_n=7
  with maxImages=3 in one run). Soften the "<= maxImages entries" claim.

* docs(ahb): note cross-vendor validation (Adreno + Mali)

Same import path verified end-to-end on:
  - Adreno 830, Vulkan 1.3.284 (Galaxy S25 Ultra)
  - Mali-G925 Immortalis MC12, Vulkan 1.3.278 (Galaxy Tab S11)

Both report VK_ANDROID_external_memory_android_hardware_buffer spec
version 5 and surface AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420 AHBs from the
camera HAL. Only auxiliary usage bits differ between vendors and are
not consumed by this path (Adreno: 0x20103, Mali: 0x20133).
2026-05-25 16:12:29 +08:00
nihui 706c4d1964 support 4d mat in more operators (#6737)
Add 4d Mat handling across generic and optimized backends, extend operator tests, and document supported input and output Mat dimensions.
2026-05-22 15:31:01 +08:00
nihui 5f41567534 Remove outdated msa.h workaround instructions (#6734)
Removed workaround instructions for msa.h header for gcc version < 8.5.
2026-05-21 19:47:44 +08:00
Layla 3724d10fc8 tools:Move Embed/MHA/RNN/LSTM/GRU weight scale generation to ncnn2table (#6688)
Co-authored-by: nihui <nihuini@tencent.com>
2026-05-18 19:38:07 +08:00
nihui 20a654e05f refactor bf16 logic to work around ohos clang crash on aarch64 (#6725)
* Refactor BF16 conversion to use float32_to_bfloat16
* Fix image URL and update build instructions in README
* Remove NCNN_SIMPLEOMP option from workflow
* Update build instructions in how-to-build.md
2026-05-15 14:08:42 +08:00
nihui 8c18666821 feat(vulkan): persist pipeline cache (#6702)
Add single-file Vulkan pipeline cache save/load support with strict device, driver, shader, and payload validation. Cache compiled SPIR-V per shader hash alongside the driver pipeline cache blob, add focused cache tests, and document the current Vulkan pipeline cache architecture and usage.

Limit Vulkan layer test cache growth with a shared test-only pipeline cache window so repeated test_layer() calls can still exercise cache reuse.

Co-authored-by: futz12 <56149058+futz12@users.noreply.github.com>

Co-authored-by: CLV-Iclucia <89170890+CLV-Iclucia@users.noreply.github.com>
2026-05-09 09:37:06 +08:00
Benson Muite 020b8b0123 Indicate correct build dependencies for RHEL/Centos (#6692) 2026-04-22 15:11:05 +08:00
風的彷徨 c0f138e466 update docs for new convertmodel website (#6617) 2026-03-30 10:21:56 +08:00
nihui e5e09d5ee3 readonly mappedfile wrapper class, use_mapped_model_loading option (#6537)
* test mmap loading
2026-02-09 22:15:51 +08:00
nihui 526e5e65ef unroll 4x4 for gemm and sdpa vulkan, vectorize a and b loading, avoid bank conflict (#6524)
* unroll 4x4 for vulkan gemm, vectorize loading a and b
* unroll 4x4 for vulkan sdpa, vectorize loading a and b, avoid bank conflict
* Update glsl-extension.md
* Update glsl-extension.zh.md
* gemm vulkan subgroup optimization

* apply code-format changes

* enable it

---------

Co-authored-by: nihui <171016+nihui@users.noreply.github.com>
2026-01-28 13:55:23 +08:00
nihui fe7c0cd2ec dynamic vulkan loading on apple platforms, honor VK_DRIVER_FILES env (#6499) 2026-01-11 19:54:54 +08:00
Sophon 41187d42b0 simplevk: add qcvkarm64xum.dll from qualcomm windows driver (#6502) 2026-01-11 17:36:28 +08:00
nihui c7f2ab1ebb doc print VkMat content (#6290) 2026-01-06 16:47:10 +08:00
RoachZhao 01d9ece3e4 Update build instructions to include 'make install' (#6363)
Added 'make install' command to build instructions for AArch32 and AArch64 targets.
2026-01-06 16:45:09 +08:00
nihui 419db5e04d net load api wchar_t version (#6491) 2026-01-06 15:33:26 +08:00
nihui 6499fc0348 vulkan bf16 infrastructure (#6469)
* skip gpu bf16 winograd
* update doc
2026-01-04 19:34:50 +08:00
nihui 32414ba787 fix windows xp clang ci, unify xp workflow file (#6435) 2025-12-02 10:46:36 +08:00
nihui 82af4a113c cmake NCNN_WINXP option, do not define _WIN32_WINNT macro (#6430) 2025-11-28 20:24:25 +08:00
nihui 430b3124a7 fix ncnn2int8 top_blob int8 scales saving for dequant only style (#6425) 2025-11-26 17:31:57 +08:00
nihui 559a8e8cb0 rotaryembed layer (#6407)
* add deepseek_v3 attention test
* fuse non interleaved rotary embed
* fuse t5 layernorm without gamma
* test qwen2 attention
* sdpa pattern ++
2025-11-19 17:28:08 +08:00
nihui 11fb99795b sdpa kvcache (#6405) 2025-11-18 11:05:27 +08:00
nihui 641a3e5be8 ncnn sdpa layer (#6397)
* fuse gqa
2025-11-11 11:14:04 +08:00
nihui 63a7a8a590 optional layer support any packing (#6392)
* split gemm tests, enable gemm vulkan any packing
2025-11-07 14:23:53 +08:00
nihui 3ab4b58d1d optional layer support vulkan packing (#6389)
* pyncnn expose layer support vulkan packing
* doc layer support behavior
2025-11-05 16:27:23 +08:00
nihui adee6a0a53 multiheadattention kvcache (#6333)
* skip input layer forward
* unified kvcache layout
* mha kvcache test
2025-10-15 14:19:57 +08:00
nihui 414666a6d1 doc supported onnx operators (#6327) 2025-09-23 19:41:06 +08:00
nihui a6eda68f23 disable onnx2ncnn build (#6324)
Migrate most of onnx2ncnn functions to pnnx
2025-09-23 19:06:07 +08:00
nihui a514cf5e9c drop deprecated ex.set_num_threads and ex.set_vulkan_compute api (#6307) 2025-09-09 19:07:43 +08:00
nihui b5bc05e2e8 drop vulkan shader pack8 (#6270) 2025-08-22 11:05:59 +08:00
Willaaaaaaa c112480451 fix(doc): esp32 build (#6254) 2025-08-17 15:17:05 +08:00
nihui 9b91fe5153 implement flip layer and pnnx torch.flip conversion (#6233)
Co-authored-by: 佰阅 <43716063+Baiyuetribe@users.noreply.github.com>
2025-08-06 14:49:16 +08:00
Copilot 4644540ea4 Add Windows XP support merging PRs #6176 and #6177 (#6204)
Co-authored-by: Sugar-Baby <87747602+Sugar-Baby@users.noreply.github.com>
Co-authored-by: AtomAlpaca <66774326+AtomAlpaca@users.noreply.github.com>
2025-07-24 15:45:02 +08:00
GIBEREZ 44982d0d23 About the update to the GLSL documentation after the image functions are deprecated (#6173) 2025-07-12 14:22:30 +08:00
Christopher 260f493ada add cross toolchain cmake config of AK3918(AK) and SS928(hisi) (#6164) 2025-07-10 19:23:45 +08:00
chri321 4c72f52954 docs: update Chinese glsl-extension documentation (#6162)
- synchronize the latest English content to the Chinese documentation
- correct spelling errors in the English version of glsl-extention
- Fix spelling 'enable_validation_layer' in src/gpu.cpp
2025-07-10 14:21:34 +08:00
nihui bd0b111775 vulkan tight fp16p pack1 (#6127) 2025-06-16 23:09:35 +08:00
nihui abf0de4488 update ruapu to detect zfh zvfh xtheadvector (#5841)
* always prefer xtheadvector
* update ci toolchain
2025-06-14 14:54:04 +08:00
nihui 1f6f01ca20 fix apple glslang package, drop glslang-default-resource-limits (#6022) 2025-04-25 17:21:02 +08:00
nihui 80da741307 glsl define ncnn_glsl_version macro (#6003) 2025-04-18 18:49:33 +08:00
nihui ef0b0e631c interp output size expression (#5994) 2025-04-17 14:24:57 +08:00
nihui 39c055d7f2 crop axes starts ends expression (#5976)
* skip dynamic tensor index

* handle clone oom
2025-04-14 19:43:50 +08:00
nihui 84970eed4d vulkan validation layer enables NCNN_LOGE in shader source (#5963)
* NCNN_LOGE in glsl

* Update glsl-extension.md
2025-03-31 16:17:16 +08:00
Zhuo Zhang 23cdc817ca refactor qnx cmake toolchain files (#5961)
* Support multiple QNX SDP versions: 700, 710, 800, etc

* Support windows and linux hosts when find programs like C/C++ compilers

* Better hint message by recommend users invoking QNX SDP bundled script

* Precise to aarch64 in file naming

* Update how-to-build.md QNX section
2025-03-31 10:43:27 +08:00
nihui 092200a966 faq vulkan in docker (#5884) 2025-03-18 19:19:28 +08:00
shacon wang 701f53794d add read npy for ncnn2table (#5930)
* add read npy for ncnn2table

* optimize npy to mat to avoid non-contiguous memory data

* add create the calibration table from npy

* add catch exception
2025-03-18 16:50:36 +08:00
nihui 6396a732ef reshape shape expression, drop reshape permute, test reshape oom (#5918) 2025-03-10 15:31:34 +08:00
nihui 1e3fcb9dda paramdict value string type, natural array representation (#5915) 2025-02-20 19:04:16 +08:00
nihui 2535a6d5b7 update doc about vulkan-sdk (#5911) 2025-02-19 18:48:29 +08:00