Files
UA-.NETStandard/fuzzing/OneFuzz/README.md
Marc Schier f5ad124891 Additional fuzzing plus fixes for every defect the campaigns found (#4459)
# Description

Brings the stack's fuzz testing to parity with the fork
(`aio-opcua-sdk`), then fixes every defect the resulting campaigns
exposed.

The fork's fuzzing was used as the behavioural baseline: all **28
fork-local callbacks** and **13 OneFuzz registrations** are now
accounted for by an explicit mapping, not just by a larger target count.
The work extends the existing SharpFuzz/NUnit infrastructure rather than
introducing another framework.

### Coverage

- **84 continuous libFuzzer targets** and **81 AFL adapters** across
Encoders, Certificates, Network and PubSub.
- **PubSub area integrated** (Corpus/Tools/Tests siblings, solution
entries) — it previously had host source only and was absent from the
solution and replay infrastructure.
- Segmented binary adapters, binary-to-JSON and XML-to-JSON conversions,
standalone certificate loading, both chain-parser variants, CRL encoding
and canonical TBS round-trips.
- The 12 original fork message inputs are retained byte-identically
alongside richer deterministic generators; the three historical
crash/slow/timeout reproducers are unchanged (verified identical by
SHA-256).

### Production fixes found by fuzzing

These are real robustness and data-integrity bugs, each with a retained
regression input:

- `UadpBinaryReader.Advance` accepted attacker-controlled lengths on the
receive path. Declared payload sizes are now authoritative bounded
sections, with negative-count rejection and overflow-safe reassembly
arithmetic.
- PubSub JSON decoding leaked `InvalidOperationException` for invalid
UTF-8.
- `JsonEncoder.WriteExtensionObject` leaked
`System.Text.Json.JsonReaderException`.
- `JsonEncoder` silently emitted a lossy `{}` for an `ExtensionObject`
whose TypeId could not be represented in JSON.
- `JsonDecoder` wrapped a bodyless `ExtensionObject` envelope as a raw
JSON body instead of preserving `ExtensionObjectEncoding.None`.
- `Node.Encode` lost the null/empty collection distinction.
- Binary/XML Variant matrix dimensions are prevalidated rather than
relying on a caught `ArgumentException`.
- `X509CRL.ReadTime` pre-validates strict RFC 5280 whole-second Zulu
time forms.
- `Utils.ParseCertificateChainBlob` disposes correctly on partial-chain
failure.
- **`nodeid.dict` was unloadable by libFuzzer** (empty token, unescaped
quote). libFuzzer rejects the whole file on one bad line, so four parser
targets silently executed **zero** inputs. Fixed and now guarded by
dictionary validation.

### Infrastructure

- `fuzzing/fuzz-targets.json` manifest recording adapters, corpora,
dictionaries, oracles and fork-behaviour mappings, with
`validate-fuzz-manifest.ps1` and **27 negative controls**.
- Strict host/tools discovery and replay: `--list`, `--replay`, non-zero
exits on unknown targets, missing or empty corpora, and failed callbacks
(playback previously logged failures and returned success).
- A real process watchdog for the timeout/slow regressions — NUnit
`CancelAfter` cannot interrupt a synchronous fuzz callback.
- Runner scripts made non-destructive (unique work directories, no
generic `bin`/findings deletion) and corrected to the native driver's
single-`--target_arg` apphost contract.
- Opt-in `OneFuzz=true` class-library publication producing a
relocatable, **uninstrumented** drop, validated by exact callback
discovery, dependency closure, corpora and dictionaries.
- CI: fuzz replay matrix plus a public drop gate, triggered by corpus-,
dictionary- and script-only changes.

### Verification

| Gate | Result |
|---|---|
| Bounded campaigns, all 84 targets (1,000 inputs, seed 1) | 84/84,
84,000 executions, zero findings |
| OneFuzz drop | 84/84 published callbacks replayed, zero skips |
| Fuzz replay net10.0 / net8.0 / net9.0 | 9,470 passed each, 0 failed |
| Fuzz replay net48 | 8,996 passed, 0 failed (Network/PubSub are
modern-only) |
| `Opc.Ua.Types.Tests` / `Opc.Ua.Core.Tests` | 9,800 and 4,442 passed, 0
failed |
| Retained regression assets | 132 `crash-*` inputs replayed by every
target each run |

### Known gaps (deliberately not closed)

- **Fidelity is not enforced against the pipeline crash corpus.** The
differential binary-to-JSON targets assert both robustness (no unhandled
exception, expected of arbitrary bytes) and fidelity (a decoded value
re-encodes equivalently, a property of well formed values). The pipeline
supplies 22,293 blobs collected for the targets that existed before this
PR, none of which asserted fidelity; 522 of them violate it, and that
count did not move across two rounds of genuine fixes because each fix
only lets the mutator reach the next representability gap. Rather than
weaken the assertion, the two contracts are separated: fidelity failures
raise `EncodingFidelityException`; curated assets under `Assets/Repo`,
the curated `Testcases` corpus and every continuous libFuzzer campaign
stay strictly enforced, while externally supplied assets are gated on
robustness only and their fidelity findings are reported with counts and
bounded base64 reproducers.
- **Internal MSFT OneFuzz canary has not been run.** It needs an
owner-supplied ownership profile, pipeline authorisation and corpus
snapshot; none were available. The drop builds and validates locally and
records `serviceCanaryPerformed: false`. A build or upload is not a
successful worker run.
- **Legacy JSON wire forms are a partial mapping.** The fork accepts its
four original `TypeId`/`Body` messages; this stack rejects that retired
envelope (measured JSON-only control: fork `JsonEncoder` 630/1356 lines
vs 0/1007 here). Recorded as an explicit unresolved mapping rather than
an exemption. `LegacyNonReversible` is a RawData-derived profile and is
not counted as independent coverage.

## Related Issues

_No tracking issue was supplied for this work; happy to open one and
link it if maintainers prefer._

- Fixes #github-issue-number, ...

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

> Note on the checklist: the fuzz replay matrix, `Opc.Ua.Types.Tests`
and `Opc.Ua.Core.Tests` were run locally on **net48** and **net10.0**
(plus net8.0/net9.0) and all passed. CI and CodeQL results are not yet
available for this branch, so those boxes are left unchecked. The CLA
box is for the author to confirm.
2026-09-10 05:56:31 +02:00

161 lines
9.5 KiB
Markdown

# Managed OneFuzz publication
`OneFuzz=true` changes each existing fuzz host into a `net10.0` class library.
The host compiles its normal target sources, but neither linked `Program.cs` nor
`FuzzMethods.cs`, and has no direct SharpFuzz reference. Default local executable
publication is unchanged. The service drop is **not preinstrumented**; the internal
service owns instrumentation and its actual version must be recorded by the canary owner.
## Public build and relocation gate
From the repository root, invoke the script with PowerShell 7:
```powershell
.\fuzzing\Scripts\build-onefuzz.ps1 `
-Areas Encoders `
-OutputDirectory 'C:\temp\opcua-onefuzz-drop' `
-WorkDirectory 'C:\temp\opcua-onefuzz-build'
```
Omit `-Areas` to require every manifest area. Output and work directories must not
already exist. Paths inside the script are independent of the caller's working directory.
No directory is deleted or reused. All project outputs **and restore
assets**, including project references, use the SDK's isolated `--artifacts-path`;
passing only `OutputPath` or a shared `BaseIntermediateOutputPath` is insufficient.
Direct publication must supply `-p:OneFuzz=true -p:CustomTestTarget=net10.0`,
`-f net10.0`, and `--artifacts-path` too.
Also pass `-p:FuzzCoverage=true`, as the script does, to make generated protocol
serialization code visible to instrumentation. Ordinary builds keep their existing
coverage exclusions; no collector binary or compiled assembly is rewritten to remove them.
The script merges actual publish outputs, rejects conflicting files, preserves the
manifest-relative corpus/dictionary layout, relocates the payload, and invokes the
published callbacks from that relocated directory. It does not generate a partial
target list from whichever callbacks happened to build. A missing declared target,
unexpected exported span callback, missing required bucket, empty corpus, absent
dictionary, or incomplete dependency closure is a failure.
`-CorpusRoot` explicitly selects a different root for **all** selected corpus paths.
It must contain their complete repository-relative layout; there is no silent fallback
to checked-in inputs. Dictionaries still come from the repository. Without this override,
the publisher runs the selected manifest's declared generators in the isolated work directory
and requires every generated output bucket to be nonempty. The Certificates generator creates
positive certificate/chain/CRL fixtures and checks them through the real parser/oracle cores;
no new certificates or keys are added to source control. Only declared output buckets enter
the drop, and `build.json` records their actual SHA-256 inventories.
For example, a certificate corpus under an external root still lives at
`<CorpusRoot>\fuzzing\Opc.Ua.Certificates.Fuzz.Corpus\...`, not directly at that root.
When an external root is supplied, publication copies the declared inputs and records their
actual hashes without labeling them freshly generated or promising deterministic
cryptographic fixture bytes. This is the external-corpus alternative to per-target
profile overrides; ownership profiles remain ownership/routing metadata only.
The drop contains the selected snapshot of `fuzz-targets.json`, flat managed DLL
closure with native/runtime subdirectories as published, `.deps.json`,
`.runtimeconfig.json`, matching portable PDBs, corpora, dictionaries, `build.json`,
`validation.json`, `publication.json`, and `files.sha256`. A failed build does not write
`publication.json`. Work directories retain per-target completion receipts and logs.
The original manifest hash and source revision/dirty state are recorded. The checksum
inventory covers all final files except the inventory itself.
Revalidate a copied drop independently:
```powershell
.\fuzzing\Scripts\validate-onefuzz.ps1 `
-DropDirectory 'C:\temp\copied-onefuzz-drop'
```
The BCL-only validator may also be published separately and passed with
`-ValidatorPath`. It never resolves application dependencies from the checkout or a
NuGet cache. Each exact `public static void(ReadOnlySpan<byte>)` callback runs in its
own process under the published host's runtime configuration, with a hard watchdog.
The default 120-second budget covers a target's entire seed replay, including child
startup; override it explicitly with `-TimeoutSeconds`. A successful exit without a
complete, hash-matched input receipt is a failure. There are no skip switches.
## Explicit service configuration
Public builds need no ownership profile and emit **no `OneFuzzConfig.json`**.
For an internally owned drop, pass `-OwnershipProfile` pointing to a protected file
matching [ownership-profile.schema.json](ownership-profile.schema.json). Do not place
credentials in that file: it carries non-secret ownership/routing metadata only.
The profile must explicitly provide the intended `azurelinux3`/`net10.0` worker
architecture and service instrumentation, notification email, SDL work item,
complete ADO ownership/template and coverage-pipeline binding, and exactly one
target record per selected manifest target. Each record supplies its `ProjectName`,
`TargetName`, and provisioned, stack-owned `SeedCorpusContainer`. There are no
default emails, IDs, project names, or containers. The private profile is not copied
into the drop; only its hash and the required final service configuration are retained.
Including seeds in `JobDependencies` does not populate that cloud container. The
internal owner must provision/seed it through the authorized service mechanism;
publication never uploads to or overwrites an existing corpus.
The generator emits the reference integration's `ConfigVersion: 3` contract:
`Fuzzer.$type = libfuzzerDotNet`, exact `Dll` / `Class` / `Method`, `Skip: false`,
`MinAvailableMemoryMB: 100`, and `FuzzerTimeoutInSeconds: 120`. The memory field is
**not** an RSS cap. `JobDependencies` explicitly lists real closure/seed/dictionary
files rather than guessed package-name globs. Multiple dictionaries are merged
deterministically into one `-dict={setup_dir}/...` option without dropping tokens.
The generated configuration is checked against actual published callback discovery
before any complete drop is reported.
For an unfinalized published payload, the low-level generator is also available as:
```powershell
dotnet Opc.Ua.OneFuzz.Validator.dll configure `
--drop C:\temp\opcua-onefuzz-drop --profile C:\protected\onefuzz-profile.json
```
It refuses to replace an existing configuration or modify a completed drop carrying
`publication.json`, whose hashes and evidence would become stale. Use the build
script with `-OwnershipProfile` for a finalized service drop. The build script
validates the generated configuration before publishing completion evidence.
## Solution and CI integration
`UA.slnx` includes these support projects:
- `fuzzing\OneFuzz\Opc.Ua.OneFuzz.Validator\Opc.Ua.OneFuzz.Validator.csproj`
- `fuzzing\OneFuzz\Opc.Ua.OneFuzz.Validator.Tests\Opc.Ua.OneFuzz.Validator.Tests.csproj`
- `fuzzing\OneFuzz\Opc.Ua.OneFuzz.TestTarget\Opc.Ua.OneFuzz.TestTarget.csproj`
The last project is a **synthetic negative-control fixture, never a service target**.
The GitHub public drop gate runs the NUnit contracts on `net10.0` and publishes a drop with
the scripts above. Manifest, corpus, dictionary, host, shared harness, OneFuzz support and
publication-script changes trigger this gate. Zero failed/skipped cases are required;
the retained drop and local evidence do not constitute a service canary.
[.azurepipelines\onefuzz.yml](../../.azurepipelines/onefuzz.yml) is opt-in: no PR,
branch, or scheduled triggers. Its default builds/tests only. An authorized internal
owner can explicitly set `submit: true`, provide the ownership profile/corpora,
install and authorize `onefuzz-task@0`, and supply the task's secret variables.
Use the explicit `ownershipProfileSecureFile` parameter to download an authorized
Azure secure file without committing private ownership metadata. Alternatively,
provide `ownershipProfilePath` for a file made available by the owner's
`prepareSteps`. Those optional steps can prepare a complete external `corpusRoot`;
without that override, publication materializes declared fixture generators automatically.
The task uses `onefuzzOSes: azurelinux3` and `onefuzzDropDirectory`. Its Ubuntu
build agent is **not** evidence of the resolved Azure Linux worker image.
No script submits or uploads anything. Build/publication checks do not establish
the hidden service instrumenter's compatibility, provision cloud corpora, verify
worker architecture/image selection, or prove a successful internal canary.
Those remain service-owner prerequisites, with actual execution/coverage evidence
required by the approved parity plan.
## Separate local driver argument contract
The local SharpFuzz runner uses `Metalnem/libfuzzer-dotnet`, not the internal service
runner. In release `v2025.05.02.0904`, both its
[Unix parser](https://github.com/Metalnem/libfuzzer-dotnet/blob/v2025.05.02.0904/libfuzzer-dotnet.cc)
and [Windows parser](https://github.com/Metalnem/libfuzzer-dotnet/blob/v2025.05.02.0904/libfuzzer-dotnet-windows.cc)
retain only the **first nonempty `--target_arg=...`**. Repeating that flag does not
construct an argument list. Unix passes that value as a single `execlp` argument;
Windows concatenates that one value into the child command line.
The portable local adapter is a published executable apphost passed as
`--target_path=...`, with one `--target_arg=<callback>` value. Do not attempt to pass
`dotnet`, a DLL, and a callback using repeated `--target_arg` flags. This local
driver contract does not establish the internal OneFuzz instrumenter's behavior.