Commit Graph
40 Commits
Author SHA1 Message Date
nihui 171b9d1bba use spdx license header, copyright Tencent (#6152) 2025-07-08 11:01:29 +08:00
nihui 8a2eab1114 set localsize as multiple of subgroup size (#2483)
* fix innerproduct gemm vulkan
2025-06-09 15:32:50 +08:00
nihui 0b9925cfef intergrate VK_EXT_subgroup_size_control features and properties (#5940) 2025-03-24 11:06:04 +08:00
邓实诚andHonestDeng a1e3ebf8e5 implement simplemath (#4905)
* complete abs, fmod and sin function in simplemath.h

* remove some unused variables in simplemath.cpp

* modify test-coverage.yml and add some functions to simplemath.cpp

* modify erf.cpp which included math.h

* include platform.h for NCNN_SIMPLEMATH definition

* move utility constants and functions in simplemath.h to simplemath.cpp

* guard simplemath functions with extern "C"

* add NCNN_EXPORT macro in simplemath.h

* include plateform.h and guard all declarations with NCNN_SIMPLEMATH

* clean unused code in test_unaryop.cpp

* guard #include <vector> with NCNN_SIMPLEMATH in benchncnn.cpp

* add 'static' to guard functions that not declarated in header file

* modify sin and cos with better implementation

---------

Co-authored-by: HonestDeng <HonestDeng@users.noreply.github.com>
2023-10-23 10:51:51 +08:00
nihui 54c0a13b9f build shared library (#2525)
* build shared lib and enable lto

* reserved for layer and option

* allocator pimpl

* datareader pimpl

* paramdict pimpl, disable copy assign for allocator and datareader

* modelbin pimpl

* net extractor pimpl

* gpu pimple

* disable copy assign vulkandevice, code format

* command pimpl, dummy image readonly

* pipeline pipelinecache pimpl, export platform class

* code format, export simple family

* update ci

* disable lto on android armv7, merge webassembly ci

* link libgcc, fix macos dylib version

* pipeline pimpl, gpu info pimpl

* destroy gpu info after vulkan device

* ignore msvc stl class warning

* fix ncnn_paramdict_get_float return type

* fix vktransfer upload fp16 without flatten, add command test
2021-01-01 22:55:10 +08:00
nihui 79efe33fdc cmake option for platform api uses (#2502)
* cmake option for platform api uses
* adroid gpu ci does not rely on glslangvalidator, add android termux ci
2020-12-25 11:24:56 +08:00
nihui b9296c259d bring up vulkan 1.1 (#2191)
* query subgroup features

* compile spirv 1.3

* drop offline spirv build

* do not build tests for android and ios, as they are never tested anyway

* code style
2020-10-11 22:53:38 +08:00
Leo 5afd318b86 Support remove libstdc++ denpendency (#2030)
* [build] add toolchain file w/o stdcxx dependency

* [build] link m and gcc lib explicitly

* [ncnn] complete simple stl impl

* [ncnn] adapt for ncnn simplestl

* [test] adapt for ncnn simplestl

* [ncnn] fix missing algorithm and list when simplestl disabled

* [ncnn] fix guard for operator new and delete

* [style] fix the code style

* [build] fix build failed on darwin and emscripten

* [ci] do not import cxx to avoid operator conflict

* [ncnn] add temporary partial_sort impl using bubble sort

heap sort should be used for better perf.

* [ncnn] add std greater and less function

* [ncnn] fix placement new operator overload

* [ncnn] add operator delete with size info

* [build] disable exception, rtti, example and tools when simplestl on

* [build] add toolchain for arm simplestl

* [build] add toolchain for aarch64 simplestl

* [ncnn] move initializer to constructor

* [ncnn] use deteiled type instead of auto

* [ncnn] use plain lib name in target_link_libraries
2020-08-28 09:20:56 +08:00
nihui 164273de61 online pipeline cache (#1792)
* online pipeline cache wip

* device-wide pipeline cache

* enable model-wide pipeline cache

* drop pre-created shader modules

* always use pipeline cache

* use implicit model-wide pipeline cache, code format

* code clean
2020-07-04 12:30:50 +08:00
nihui 3ef995ed1e format code style and setup restyled.io (#1840) 2020-06-13 22:17:51 +08:00
nihuini aeba24b371 enable implicit fp16a on arm mali variants, add bug tag for layout binding id alias 2020-06-01 17:50:24 +08:00
nihui 17c445480f runtime spir-v compilation with libglslang (#1779) 2020-05-22 10:34:35 +08:00
705dd36a31 simplestl is an alternative std vector string implementation (#1762)
* 去掉对stl的依赖

* 头文件名,push_back改正

* 去掉构造托管

* 好像是折腾

* data 的返回改为指针,非指针引用

* resize一处写错

* stdint

* 加入c_str

* 改文件名为小写

* NCNN_SIMPLESTL option

* simplestl default to OFF

* Update linux-x64-cpu-gcc.yml

* Update linux-x64-cpu-gcc.yml

* Update linux-x64-cpu-clang.yml

* drop functional header

* arm32 arm64 simplestl ci

* 修改一处内存泄漏, 去掉编译器警告

* resize时默认量的bug

Co-authored-by: nihuini <nihuini@tencent.com>
Co-authored-by: nihui <shuizhuyuanluo@126.com>
2020-05-15 10:42:44 +08:00
Naiyang Lin ceef2470a5 Add logger.h (#1753) 2020-05-12 19:33:21 +08:00
nihuini 6682cd1638 image fp16pa, mark some bugihfa todo 2020-05-12 16:20:23 +08:00
nihui 9a9a618229 image storage is mandatory, less options makes life easier 2020-05-08 00:09:35 +08:00
nihui 62da1228e1 adreno image shader + fp16 + fp16a (#1714)
* wip

* wip

* fix

* image and imageview can not be destroyed until command execution ends

* fast copy path for tightly packed data

* wip

* texture load works

* 1d 3d image

* record clone image, multiple commands share one image reference

* upload download image

* layer forward accept vkimagemat

* vkimagemat graph works

* staging vkimagemat for passing dynamic parameters, macro for fp32+image shader, padding image shader

* vkimagemat elemsize

* convolution test pass

* conv1x1s1 image shader

* fast staging image allocator from host memory, pooling image shader

* convolutiondepthwise image shader

* innerproduct image shader

* packing image shader

* crop deconvolution image shader

* resolve spirv binding types

* image fp16 and fp16a, cast image shader

* eltwise image shader

* wip

* absval image shader

* deconvolutiondepthwise image shader

* concat image shader, squeezenet works

* noop split image shader

* uniform precision hint

* layer support_image_storage

* wip

* vulkan device utility operator

* command is storage and packing option aware

* fallback to cpu on image allocation failed, mobilenetssd works

* flatten image shader, enable more test

* ci test

* check imgfp32 imgfp16 imgfp16a features

* fix ci test

* fix ci test

* upgrade swiftshader

* wip

* opt aggressive

* imgfp16p

* opt none

* convolution winograd image shader

* fix flush range, fast copy path for continous buffer

* minor fix

* fix innerproduct

* wip ...

* wip

* cast fix

* packing test

* wip

* image fp16p is fp16p

* wip

* silence

* more line info

* code clean

* softmax image shader
2020-04-30 17:08:30 +08:00
nihui 7365bb80a2 vkmat and command api breaks (#1689)
* vkmat and command api breaks

* always use compute queue for compute buffer transfer

* no barrier for readonly weight buffer

* record clone, drop queue_owner

* bring back layer forward

* fix validation errors

* lifecycle inside command makes life easier

* update doc

* record_import_android_hardware_buffer
2020-04-12 13:14:20 +08:00
nihuini 69b7683d0e adapt api changes 2020-04-01 19:14:25 +08:00
nihuini 1ea9de3bdf create shader pipeline by type index, resolve binding count and push constant count from spirv. since we don't create compound shader module for macos and ios compatibility, it is enough to use fixed main as the shader entry point 2020-04-01 18:35:00 +08:00
nihuini ee118e7d70 reconstruct import android hardwarebuffer api, wip 2020-04-01 15:54:02 +08:00
nihuini 4d6cf47db8 pixel type BGRA 2020-03-28 13:10:50 +08:00
nihui 82fa27967e priorbox shader fix 2020-02-24 19:49:00 +08:00
nihui bbaa4dcce2 compile fp16pa, optimize shader for size, enable implicit fp16 arithmetic for qcom855 and qcom855plus 2020-02-22 10:31:01 +08:00
nihui 0f7e7bca02 shader shape specialization constant and basic local group size partition (#1523)
* use Mat class for Shape description

* shape specialization constant in compute shader

* wip

* wip

* test forward_inplace, add binaryop unaryop sigmoid test

* fix arm unaryop test

* fix arm binaryop test

* make shape hint optional, cast int8 to fp32, add cast test

* wip

* follow the good and old local size setting for conv1x1

* the optimal local size rewrite

* fix build on msvc

* add permute shader for all packing layout, add permute test

* concat and slice patial shape constant, slice test

* fix slice test

* interp test

* add lrn test, test packing layout implicitly

* add eltwise test

* add normalize test

* add instancenorm test

* reorg shape constant

* simple local group size partition

* add shape constant param
2020-02-17 15:45:29 +08:00
nihuini a86c2f44c3 vkimagemat, vkimageallocator, convenient construct from android hardware buffer 2019-12-25 14:34:34 +08:00
nihuini a477aee0ba print graphics queue info, const++ 2019-12-16 15:43:27 +08:00
nihui 8a87f0267a workaround local workgroup size specialization constant bug for old arm mali vulkan driver, fix #1424 2019-12-14 12:56:22 +08:00
nihui 4a3c70f5b8 import android hardware buffer, fix #1329 2019-12-08 22:36:48 +08:00
nihuini e09607bc22 add option to upload model function, pipeline creation honors option use flags, setting allocator per extractor do not make much sense 2019-06-05 19:03:39 +08:00
nihuini 838c5df839 option api changes 2019-06-04 14:20:18 +08:00
nihui 25b9736f82 shader fp16 packed 2019-05-26 19:33:41 +08:00
nihui 3e003ffd98 fuse sigmoid 2019-05-11 11:06:10 +08:00
nihuini 8322a14964 set fixed local size 2019-05-10 11:28:46 +08:00
nihui c180e87502 add compile shader module function, create pipeline from custom shader spv data 2019-04-12 22:56:09 +08:00
nihui d753fe2589 upload fp16 weight, enable fp16 storage and arithmetic 2019-03-23 10:34:55 +08:00
ShuangLiu1992 c6a2d0417a add missing header for pipeline.cpp and fix compile error for emscripten (#861) 2019-03-22 23:58:16 +08:00
nihui 058bd65c88 fix fp16 shader creation 2019-03-22 22:21:26 +08:00
nihuini 1f4bdd91b5 uint32_t typed workgroup size 2019-03-11 18:28:24 +08:00
nihui 10b8ac68cc [WIP] vulkan compute (#618)
* vulkan infrastructure

* vkallocator and vkmat

* layer interface for vulkan compute

* wip...

* default vulkan device, command wrapper, upload model weight in load_model to simplify layer interface

* simplify command api, vkmat holds staging buffer, relu works

* initialize specialization constant, simplify command dispatch, fix staging buffer copy with different shape, convolution works

* init extension functions

* dynamic local size and group count

* group count=1 is invalid

* regard device max workgroup size limit

* fix relu oooops

* decouple command record and staging allocation

* create result blob

* add pooling shader

* buffer is faster than image :)

* fix pooling shader

* add innerproduct shader

* readonly writeonly decoration

* simplify buffer creation

* decouple command and layer, VK_KHR_descriptor_update_template extension makes descriptor binding update easy :D

* fix vulkan building issues in visual studio (#1)

* fix building issues on visual studio

* ignore benchmark

* cancel changes

* ... ...

* decouple paramdict and vulkandevice

* fix staging buffer destroy in model loading

* remove vkdev member in option

* add padding shader

* simplify vulkan layer creation, simplify convolution and pooling shader for no padding, less debug output

* add convolutiondepthwise and softmax shader

* specialization float type, add leakyrelu

* add dropout shader

* add batchnorm shader

* split vulkan forward

* add scale shader

* push constant type can be int or float

* set_optimal_local_size_xyz

* add eltwise shader

* concat vulkan forward

* fix convolution without bias

* add dummy shader for concat and split, more fix ...

* optional VK_KHR_descriptor_update_template and VK_KHR_push_descriptor

* check VK_KHR_push_descriptor for vkCmdPushDescriptorSetWithTemplateKHR

* binaryop and unaryop shader

* hide raw command buffer

* simple vkbenchncnn benchmark

* create device with transfer queue

* rename command to vkcompute, add vktransfer and layer upload_model interface

* external VkMat, copy and map wrt buffer offset

* command copy respect offset and size

* decouple weight upload and load, simplify upload weight api, use one big staging buffer for uploading weights

* fix build on android

* binding count can not vary :(

* barrier check state, fix sub-op destruction

* declare local_size_xyz constant, fix crash on radv

* fix local_size_xyz, second try

* more barrier and state fix

* fix softmax

* reconstruct buffer memory allocator, reuse blob buffer, less verbose output

* find unified memory type index

* weight staging buffer allocator and weight buffer allocator, respect descriptor buffer offset alignment

* use VK_KHR_descriptor_update_template for faster descriptor update if available, multithread pipeline creation

* find more useful vulkan extensions and enable them

* fix msvc build

* respect VK_KHR_dedicated_allocation for weight buffer allocation

* fix android build

* fix bias name conflicts with metal

* decouple pipeline and layer, building shader sources into shader module, dedicated create_pipeline api, simplify pipeline recording

* drop dummy shader, inplace softmax, multiple shader module works

* fix unique queue family index error

* flatten support vulkan

* mnasnet run

* find shader module by name, each entry point per shader module, fix attribute/id conflict on moltenvk

* some minor changes

* add some high level api

* use dedicated transfer queue to upload weight model

* prefer mappable buffer on unified memory

* global pooling and convolution fc, reuse staging buffer

* implement ring-buffer style blob allocator, add VkBufferMemory capacity

* use blob allocator for workspace blob, it works fine :)

* vulkan option off

* Update layer.cpp

* fix build with vulkan off

* less verbose output, fix crash on vulkan_compute off

* merge benchncnn tool

* allocator clear api, use new weight buffer allocator per net

* add default locked allocator

* mapped mat ptr api, persistent mapped memory works generally :)

* travis ci linux vulkan

* travis ci vulkan wip ...

* more gpu wip ...

* more gpu wip ...

* wip...

* wip...

* wip... ...

* wip... ios vulkan build...

* find glslangValidator on ios build

* use dynamic moltenvk library

* travis ci wip ...

* ios simulator does not support metal at all

* fix cpu only extractor

* optimize workgroup size, first try

* optimize workgroup size, second try

* conv1x1s1d1 vec4

* revert build system

* fix ncnn2mem build

* fix ncnn2mem build
2019-01-23 16:21:17 +08:00