Keep persistent KV caches on the blob allocator, broadcast single-channel attention masks across heads on CPU and Vulkan paths, and add long multi-token cache regressions.
* fix interp scale factor sampling and pnnx recomputed sizes
Use the original scale factor for linear and cubic interpolation instead
of deriving coordinates from the truncated output size. Preserve explicit
sizes, reference sizes, and size expressions across CPU and Vulkan paths.
Keep same-axis linear behavior, resample non-unit same-size bicubic input,
and clamp interpolation taps for short axes without out-of-bounds reads.
Lower recompute_scale_factor to runtime size expressions, retain the flag
when importing nn.Upsample, and avoid deleting non-unit same-size resizes.
Add independent ncnn reference checks and pnnx static/dynamic round-trip
tests covering fractional up/downscales, recomputed sizes, short axes,
align_corners, and nn.Upsample.
Validated CPU interpolation tests, ASan/UBSan, Vulkan llvmpipe, and pnnx
TorchScript interpolation tests against PyTorch 2.10.0 CPU. The original
bilinear/bicubic reproductions fail on the base and pass with this fix.
Refs: Tencent/ncnn#6836, Tencent/ncnn#6837, Tencent/ncnn#6844
---------
Co-authored-by: Acture <acturea@gmail.com>
Co-authored-by: xy200303 <3483421977@qq.com>
Co-authored-by: nihui <171016+nihui@users.noreply.github.com>
* RotaryEmbed: support full-width cos/sin cache for 2D / vision RoPE
The non-interleaved path read only the first embed_dim/2 values of the
cos/sin cache and reused them for both halves of each head vector. That is
correct for standard 1-D RoPE (cos = concat(freqs, freqs), identical halves),
but the pnnx fuse_convert_rotaryembed pass lowers the general expression
x*cos + rotate_half(x)*sin — with a full-width cos — into this op without
checking that the halves are equal. For 2-D / vision RoPE where the halves
differ (cos = concat(cos_h, cos_w), e.g. the Penguin-VL vision encoder) the
second half was rotated with the wrong frequencies, silently producing
wrong output (max|diff| ~3.4 vs PyTorch on the Penguin-VL vision encoder).
Read the second half's cos/sin for the second output half. Backward
compatible via a cache-width guard: when the cache is full width (== embed_dim)
the two halves are independent; when it is embed_dim/2 (the standard-rope form,
as the unit test supplies) the second half reuses the first, so existing
half-width callers stay bit-identical. Applied to every backend
(naive / arm / x86 / loongarch / mips / vulkan) and its bf16/fp16 paths.
Adds a full-width differing-halves test case; the previous test only exercised
half-width caches, which are trivially identical-halves.
After the fix the Penguin-VL vision encoder converts in-graph at
max|diff| = 6.2e-4 vs PyTorch (fp32).
Fixes#6833
* test: add a pnnx test that can actually fail for the full-width cache fix
tests/test_rotaryembed.cpp cannot detect this bug. ncnn's test_layer() compares
the optimized backends against the naive layer, and this PR changes the naive
layer too, so the test stays green whether the semantics are right or wrong —
rebuilding the PR's test against master's src/layer still exits 0.
tools/pnnx/tests/ncnn/ is the only harness in-tree that pins semantics to an
external reference (torch.allclose against PyTorch), so the check belongs there.
Every input is drawn with torch.rand in forward() parameter order, because the
generated test_ncnn_fuse_rotaryembed_ncnn.py takes no arguments: it reseeds with
manual_seed(0) and regenerates each input the same way. An input built any other
way would leave the two sides comparing different tensors, and the test would
fail unconditionally while appearing to test something.
Random caches are also the stronger choice here: the defect is that the layer
reads one cache row for both halves, so it only shows when the halves differ,
which random values guarantee and a hand-built cat(f, f) cache would hide. A
"standard RoPE stays correct" case is deliberately absent — the cache must stay a
pnnx.Input for the fusion pattern to match, and the generator always randomizes
inputs, so that case cannot be expressed in this harness.
Also parameterizes the interleaved path in tests/test_rotaryembed.cpp. On a
CPU-only build that is code-path coverage rather than a correctness check, for
the reason above; its value is on the Vulkan backend.
Not run here: building pnnx needs libtorch. Worth confirming under ctest that the
generated .param really contains RotaryEmbed layers — if the fusion does not
fire, the test passes without exercising anything.
* fix pnnx rotaryembed regression test layout and fusion checks
---------
Co-authored-by: nihui <nihuini@tencent.com>
* pnnx: preserve leading dimensions in spectrogram lowering
* pnnx: fix linear leading dimensions and spectrogram tests
* pnnx: fix padding on native batch axes
---------
Co-authored-by: nihui <nihuini@tencent.com>
* pnnx: store bool MemoryData attributes as raw int8
Keep bool constants as one byte per element and select MemoryData
load_type=3. Implement the reserved raw int8 ModelBin loading mode and
pad exported bool data to four bytes so subsequent weights stay aligned.
This preserves the byte representation used by the proposed logical ops.
Add regression coverage for scalar and padded bool exports in both fp16
modes, multidimensional MemoryData, raw int8 loading through reference
and copy readers, trailing fp32 weights, and missing padding. Document
the format.
Based on the bool attribute serialization fixes proposed in:
https://github.com/Tencent/ncnn/pull/6956https://github.com/Tencent/ncnn/pull/6866Fixes#6855
* tools: preserve byte MemoryData when optimizing models
Save MemoryData load_type so raw int8 constants retain their byte
representation after ModelWriter saves the model. Skip fusion paths that
interpret MemoryData as float32 when its elements are not four bytes.
This change does not add bool arithmetic support or automatic type
promotion. Byte bool storage is intended for future logical, comparison
and selection operators.
Validated ncnnoptimize save and reload for scalar and one-to-five-element
bool constants in fp32 and fp16 storage modes, preserving byte values
and element size in all 12 cases.
---------
Co-authored-by: ktz03 <2484593937@qq.com>
Co-authored-by: Giulio Corradi <53930027+giuliocorradi@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Add backend support for new UnaryOp operations across scalar, SIMD and Vulkan paths, including sign, sinh, cosh, asinh, acosh, atanh, expm1 and log1p.
Wire the new ops through onnx2ncnn and pnnx expression conversion, update generated Python emission, and add regression coverage for the new conversion paths.
Fix portability issues for expm1/log1p math declarations, NCNN_SIMPLEMATH builds, ONNX-enabled pnnx linking, and stabilize unary op tests across fp16, bf16 and torch exporter/version differences.
Co-authored-by: nihui <nihuini@tencent.com>
Normalize 4-element aten pad values from (left, right, top, bottom) to nn.Conv2d padding order (height, width) when fusing Conv2d modules.
Also add regression coverage for asymmetric ZeroPad2d + depthwise Conv2d to keep the output shape and ncnn pad params correct.
LI