Commit Graph
14 Commits
Author SHA1 Message Date
nihuini 7a8f68aca6 move vulkan code to subdir, new layer interface create_pipeline and destroy_pipeline for post-loading works 2019-05-08 18:36:25 +08:00
nihui 10b8ac68cc [WIP] vulkan compute (#618)
* vulkan infrastructure

* vkallocator and vkmat

* layer interface for vulkan compute

* wip...

* default vulkan device, command wrapper, upload model weight in load_model to simplify layer interface

* simplify command api, vkmat holds staging buffer, relu works

* initialize specialization constant, simplify command dispatch, fix staging buffer copy with different shape, convolution works

* init extension functions

* dynamic local size and group count

* group count=1 is invalid

* regard device max workgroup size limit

* fix relu oooops

* decouple command record and staging allocation

* create result blob

* add pooling shader

* buffer is faster than image :)

* fix pooling shader

* add innerproduct shader

* readonly writeonly decoration

* simplify buffer creation

* decouple command and layer, VK_KHR_descriptor_update_template extension makes descriptor binding update easy :D

* fix vulkan building issues in visual studio (#1)

* fix building issues on visual studio

* ignore benchmark

* cancel changes

* ... ...

* decouple paramdict and vulkandevice

* fix staging buffer destroy in model loading

* remove vkdev member in option

* add padding shader

* simplify vulkan layer creation, simplify convolution and pooling shader for no padding, less debug output

* add convolutiondepthwise and softmax shader

* specialization float type, add leakyrelu

* add dropout shader

* add batchnorm shader

* split vulkan forward

* add scale shader

* push constant type can be int or float

* set_optimal_local_size_xyz

* add eltwise shader

* concat vulkan forward

* fix convolution without bias

* add dummy shader for concat and split, more fix ...

* optional VK_KHR_descriptor_update_template and VK_KHR_push_descriptor

* check VK_KHR_push_descriptor for vkCmdPushDescriptorSetWithTemplateKHR

* binaryop and unaryop shader

* hide raw command buffer

* simple vkbenchncnn benchmark

* create device with transfer queue

* rename command to vkcompute, add vktransfer and layer upload_model interface

* external VkMat, copy and map wrt buffer offset

* command copy respect offset and size

* decouple weight upload and load, simplify upload weight api, use one big staging buffer for uploading weights

* fix build on android

* binding count can not vary :(

* barrier check state, fix sub-op destruction

* declare local_size_xyz constant, fix crash on radv

* fix local_size_xyz, second try

* more barrier and state fix

* fix softmax

* reconstruct buffer memory allocator, reuse blob buffer, less verbose output

* find unified memory type index

* weight staging buffer allocator and weight buffer allocator, respect descriptor buffer offset alignment

* use VK_KHR_descriptor_update_template for faster descriptor update if available, multithread pipeline creation

* find more useful vulkan extensions and enable them

* fix msvc build

* respect VK_KHR_dedicated_allocation for weight buffer allocation

* fix android build

* fix bias name conflicts with metal

* decouple pipeline and layer, building shader sources into shader module, dedicated create_pipeline api, simplify pipeline recording

* drop dummy shader, inplace softmax, multiple shader module works

* fix unique queue family index error

* flatten support vulkan

* mnasnet run

* find shader module by name, each entry point per shader module, fix attribute/id conflict on moltenvk

* some minor changes

* add some high level api

* use dedicated transfer queue to upload weight model

* prefer mappable buffer on unified memory

* global pooling and convolution fc, reuse staging buffer

* implement ring-buffer style blob allocator, add VkBufferMemory capacity

* use blob allocator for workspace blob, it works fine :)

* vulkan option off

* Update layer.cpp

* fix build with vulkan off

* less verbose output, fix crash on vulkan_compute off

* merge benchncnn tool

* allocator clear api, use new weight buffer allocator per net

* add default locked allocator

* mapped mat ptr api, persistent mapped memory works generally :)

* travis ci linux vulkan

* travis ci vulkan wip ...

* more gpu wip ...

* more gpu wip ...

* wip...

* wip...

* wip... ...

* wip... ios vulkan build...

* find glslangValidator on ios build

* use dynamic moltenvk library

* travis ci wip ...

* ios simulator does not support metal at all

* fix cpu only extractor

* optimize workgroup size, first try

* optimize workgroup size, second try

* conv1x1s1d1 vec4

* revert build system

* fix ncnn2mem build

* fix ncnn2mem build
2019-01-23 16:21:17 +08:00
nihuini 0239d2ef48 another internal_nconsumed_ptr reset case, fix #706 2018-12-18 14:15:58 +08:00
nihuini 4e68a29eff fix build on msvc, second try 2018-10-16 11:13:52 +08:00
nihuini 4a9e452cb5 fix build on msvc, first try 2018-10-15 20:06:37 +08:00
xinyu391 666ec2820e add Net::load_param_mem(), ParamDict::load_param_mem() load plain text paramters from memory buf. (#587) 2018-09-27 14:21:44 +08:00
nihui a169cec363 core int8 inference, quantize and dequantize, net using flag, caffe2ncnn reads int8 scale table 2018-07-29 16:19:01 +08:00
Yantao Xie 89c7aa26f8 support the scientific notation when parsing layer's paramters. 2018-04-23 15:08:37 +08:00
nihui 8ccae1d4fd prevent reuse of param array, fix #258 2018-02-04 10:39:09 +08:00
nihuini 67b80183dd fix param load using external memory 2018-01-08 15:28:45 +08:00
nihuini a84ba8fc0f element type storage support in Mat, move data member the first so that a pointer to Mat is a pointer to data, convenient index access for float vector 2018-01-05 15:49:45 +08:00
nihui 2a62a98e1a allow constructing paramdict and modelbin from userside 2018-01-01 18:17:34 +08:00
nihui 8e490d4b68 fix array parsing, first try 2017-10-16 15:23:56 +08:00
nihui 1e2265dd99 new param load api 2017-10-15 11:30:29 +08:00