GithubStar/UA-.NETStandard
1
0
Fork 0
mirror of https://github.com/OPCFoundation/UA-.NETStandard.git synced 2026-10-17 05:11:13 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,803 Commits 101 Branches 105 Tags
master
Commit Graph

3 Commits

Author SHA1 Message Date
romanett 52f8520baa Detect resource and limit abuse in the encoder/decoder fuzz harness (#4666) 
## Why

The encoder/decoder fuzz targets only flagged crashes, unhandled
exceptions and
encoding-fidelity mismatches. They could not observe a decode that stays
within the
exception contract but consumes far more memory, CPU or stack than its
input justifies,
or that materialises values larger than the context's encoding limits.
None of the
resource-related findings of the codec audit would have been caught.

## What

Adds four shared oracles to the fuzz harness
(`fuzzing/Common/Fuzz/FuzzOracles*.cs`),
applied by every host (libFuzzer, AFL, `--replay`, Tools playback and
the NUnit replay):

- **Allocation** — wraps the Binary/JSON/XML decode cores, the
built-in-type targets, the
PubSub UADP/JSON decoders and the UADP reassembler; fails when a decode
allocates more
than `64x` its input plus `8 MB`, whether it returns or throws. Uses the
per-thread
  allocation counter (the AppDomain counter on .NET Framework).
- **Time** — a per-input budget of `1 s + 100 ms/KB`, re-run twice and
reported only if the
  fastest run is still over, so JIT/GC/CI noise does not trip it
  (`OPCUA_FUZZ_TIME_BUDGET_SCALE`, `0` disables).
- **Stack** — every input runs on a worker thread with a `256 KB` stack,
so recursion that
comes close to the limit of a normal thread is caught in-process as a
crash rather than
near 200 levels. `StackTestcases` inputs replay through every target in
a child process
  (`--fuzz-replay-all`), since such a crash cannot be caught.
- **Decoded limits** — walks a decoded value for strings, identifiers,
ByteStrings and
arrays beyond the context's
`MaxStringLength`/`MaxByteStringLength`/`MaxArrayLength`;
matrices are additionally boxed the legacy way so a value no consumer
can materialise is
reported. The fuzz context uses `4096/8192/4096` so small inputs reach
the limits.

Supporting changes:
- `ArrayOf<T>`/`MatrixOf<T>` are not `IEnumerable`, so the limit walker
unpacks them via
  `ToArray`/`ToArrayOf` (otherwise it silently traversed nothing).
- `Decimal` is registered in the fuzz context so the targets exercise
`Decimal.Decode`.
- **B1-2 bug fix**: `BinaryDecoder.ReadExtensionObject` dereferenced a
null element for an
XML body that is not an element; now rejected with `BadDecodingError`.
Found by replaying
the nightly crash corpus under the new exception contract; 5 corpus
inputs are added as
  gating reproducers.
- Reproducers for B1-1, T1-3, X1-1 and U1-3, built in process by
documented builders
(`ResourceReproducerTests`, `PubSubReproducerTests`) and checked in to
the corpus, where
every target replays them under the oracles. Each was verified by
reverting the matching
fix locally and confirming the guard fires (e.g. B1-1 then allocates ~21
MB from 481
  input bytes).
- `Fuzzing.md` / `CrashCorpus.md` document the oracles and the
private-input log policy.

## Testing

Green on net10.0 and net48:

| Suite | net10.0 | net48 |
|---|---|---|
| `Opc.Ua.Encoders.Fuzz.Tests` | 8130 | 8130 |
| `Opc.Ua.PubSub.Fuzz.Tests` | 149 | n/a (modern-only) |
| `Opc.Ua.Fuzzing.Tests` | 154 | 154 |
| `Opc.Ua.Types.Tests` (encoders) | 3698 | builds |

Also verified: `Network`/`Certificates` fuzz test suites, all fuzz hosts
and Tools
projects build, the fuzz manifest validator passes, `dotnet format
whitespace` is clean,
and the full ~22k-input nightly crash corpus replays with no oracle
false positives.

Stacked on `romanett/encoder-decoder-spec-security-83c9f5`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 19:16:23 +00:00
romanett d623007a3f Remove the Azure Pipelines CI and publishing definitions from master (#4575) 
# Description

Removes the Azure Pipelines CI and package-publishing definitions from
`master`. GitHub Actions already owns all CI and package publishing for
`master`, but pushes to `master` still queued the Azure definitions.
They duplicated the Actions work and posted failing checks, for example
[`Nuget Preview (Push Nuget Preview Pack Nugets
Debug)`](https://github.com/OPCFoundation/UA-.NETStandard/runs/109065914524)
on `19dc504c5`.

## Removed

- `azure-pipelines.yml`, `azure-pipelines-preview.yml`
- `.azurepipelines/` templates: `ci.yml`, `coverage.yml`, `preview.yml`,
`sample-ha-longhaul.yml`, `sln.yml`, `test.yml`, `test-aot.yml`
- Helpers only those templates used: `get-matrix.ps1`, `get-root.ps1`,
`evaluate-test-results.ps1`
- `TrxGateScriptTests`: it tested only `evaluate-test-results.ps1`. The
Actions equivalent, `.github/scripts/get-test-verdict.ps1`, keeps its
own `CiTestVerdictTests`.

## Kept

**OneFuzz:** `.azurepipelines/onefuzz.yml` stays. Submitting to the
internal OneFuzz service needs the Azure DevOps-only `onefuzz-task`,
which has no GitHub Actions equivalent. The pipeline has no push, PR or
schedule trigger, so it never runs on `master` pushes. The public drop
build keeps running in the GitHub `fuzz-public-drop` job.

**Scripts GitHub Actions still invokes:** these stay in
`.azurepipelines/`, so every workflow path and `release.yml`'s
candidate-bound checkout keep working. Only `generate-slnx.ps1` changes,
and only its comment.

| Script | Used by |
| --- | --- |
| `set-version.ps1` | buildandtest, nightly, nuget-publish,
docker-image, stress / stability / long-haul, `run-dotnet-tests` action
|
| `generate-slnx.ps1`, `signlist{Debug,Release}.txt`,
`validate-source-generator-packages.ps1` (+ `expected-packages.txt`) |
nuget-publish |
| `package-version-policy.ps1`, `validate-nuget-package-set.ps1`,
`validate-preview-package-ordering.ps1`,
`assert-published-packages-match.ps1` | nuget-publish, release |
| `check-coverage.ps1` | buildandtest, nightly |
| `validate-migration-plugin.ps1` | buildandtest |

I didn't rename the directory on purpose. `release.yml` checks out the
candidate commit and runs `./.azurepipelines/package-version-policy.ps1`
from it, so moving the scripts would break promotion of any candidate
built before the move.

## Docs and comments

- `UA.slnx` drops the deleted files.
- `codecov.yml`, `coverage-thresholds.json`, `targets.props`, the
Redundancy sample test comments, the `.github` scripts and workflows,
`CONTRIBUTING.md`, `docs/` and `fuzzing/` no longer describe Azure as
running on `master` (`/azp run`, the `azure` Codecov flag, "the Azure
gate", the Azure long-haul and AoT legs).
- In the Developer Guide, the "Triggering a pipeline run" and "Migration
status" sections are replaced by "Pull requests from outside
contributors" and "Azure Pipelines on other branches".
- The new section names `onefuzz.yml` as the one remaining Azure
definition.
- It keeps the warning that the `Release` ruleset's Azure required check
must be lifted before cutting a new `release/2.<minor>` branch.
- `master` is merged in, including the #4555 docs rewrite. Its new
`docs/RedundancySampleTests.md` needed the Azure references removed
again. `docs/README.md` already had none.

## Reviewer notes

- **Azure DevOps side (admin action, not in this PR):** the definitions
are configured in the Azure DevOps portal. With the YAML gone, `master`
pushes give the CI and preview definitions nothing to run. Disabling
definition 14's push trigger/schedule, definition 16's build-completion
trigger, and the stale definition 13 stops any residual "file not found"
runs. Leave the OneFuzz definition enabled.
- `master378`, the 1.x `release/*` lines and the frozen `release/2.0.0`
keep their own `azure-pipelines.yml`. This PR only touches `master`.

## Related Issues

- Follow-up to #4513 (GitHub Actions for CI and nightly).

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [ ] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

This PR removes a test and adds none. Locally, at `9b00da817`, the full
`Opc.Ua.Tools.Tests` suite passed on net10.0: 884 passed, 1 skipped. The
later commits (OneFuzz restore, merge with `master`) touch only YAML and
docs.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-29 07:44:36 +02:00
romanett e0c56a7c00 Replay the fuzz crash corpus from the fuzz-corpus orphan branch (#4539) 
# Description

Since the move to GitHub Actions, CI replays only the checked-in fuzz
corpus. The full crash corpus used to be the Azure DevOps secure file
`FuzzingArtifacts.zip`, which `.azurepipelines/test.yml` overlaid onto
`Opc.Ua.Encoders.Fuzz.Tests`. The nightly `private-corpus` job that was
meant to replace it depended on an Azure storage account, federated
credentials and a protected environment. None of these were ever
provisioned, so the job never ran.

**The corpus now lives on the orphan branch
[`fuzz-corpus`](https://github.com/OPCFoundation/UA-.NETStandard/tree/fuzz-corpus)**
(commit `843ba944a7b144a7ae6429151846d9e5ab28d29f`, already pushed):
- It contains 22,267 `crash-*`, 25 `slow-unit-*` and 1 `timeout-*` input
under `Opc.Ua.Encoders.Fuzz.Tests/Assets/`.
- The files are byte-identical to the secure file (SHA-256
`85198e46…fe1d61b`), recovered from the `drop` artifact of Azure build
18397.
- The branch shares no history with `master`, so clones of `master`
don't get 22k extra files in their checkout.

**Why the corpus is no longer treated as private:**
- The inputs date from April–June 2024.
- The `drop` artifacts could be downloaded anonymously.
- Every master build printed up to about 7k base64 reproducers into
anonymously readable Azure logs.
- Current `master` replays every input without a test failure.

New, unfixed crashes still go through
[SECURITY.md](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/SECURITY.md)
and are added to the branch only after the fix ships.

## Changes

- **`nightly.yml`:** `private-corpus` is replaced by `crash-corpus`. The
new job:
- checks out the commit pinned in `FUZZ_CORPUS_COMMIT` (no secrets,
environment or OIDC);
- verifies the SHA and copies each top-level folder over
`fuzzing/<folder>`;
- runs `Opc.Ua.Encoders.Fuzz.Tests` on the 12 non-macOS profiles through
the shared `run-dotnet-tests` action and uploads
`crash-corpus-results-*`.

The `run_private_corpus` input and the INCOMPLETE label are removed. The
job runs on every scheduled and manual run, and the nightly summary
fails if it was skipped.
- **`fuzzing/CrashCorpus.md` (new):** the Azure history, branch layout,
how CI replays the corpus, local replay, and how to add inputs and move
the pin. `Fuzzing.md` links to it.
- **`docs/DeveloperGuide.md`:** the private-corpus section and its
secrets table are replaced, and the migration to-do list is updated.
- **`nightly.yml` AOT job (fix for a failure found by the first nightly
run):** the Linux and macOS legs looked up the published binary by
`BaseName`/`Extension`. A NativeAOT executable has no extension on those
platforms, so PowerShell splits `Opc.Ua.Aot.Tests` into `Opc.Ua.Aot` +
`.Tests`, and every leg reported *No NativeAOT executable was produced*
even though publish succeeded ([run
36017168717](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/36017168717),
`aot-ubuntu-latest`). The lookup now matches the exact file name. A
failure now reports the publish errors, a missing publish directory, or
which files were produced instead.
- **`Opc.Ua.Redundancy.Samples.Tests` / `Opc.Ua.WotCon.Samples.Tests`
(fix for failures found by the first nightly run):** the test runner's
framework guard rejected both projects on the nightly net8.0, net9.0,
net472 and netstandard2.1 profiles. Redundancy built a net48 placeholder
for every target except net10.0, and WotCon was pinned to net10.0. Both
now build the requested target on net8.0/9.0/10.0 and use
`RestrictForLegacyTfm` to become explicit not-applicable shells on
net48, net472, netstandard2.0 and netstandard2.1. The redundancy tests
no longer hard-code `bin/<cfg>/net10.0` when locating the sample
binaries, and `SampleAppProcess` falls back to an `object` lock below
net9.0. The net48 placeholder remains for Azure's plain `dotnet test -f
net48` leg. Checked locally by building Redundancy for
`CustomTestTarget=net9.0` and WotCon for `netstandard2.1`.
- **`DefaultServerRedundancyHandler` (fix for a failure found by the
first nightly run):** `ResolveServerAsync` gave the resolver a token
linked to the caller's, but waited on the caller's token directly. On
cancellation the wait could finish, and its `using` could dispose the
linked source, before the linked source's own callback had run. That
unhooks it, so the in-flight peer lookup never saw the cancellation
(`PeerDiscoveryPropagatesCallerCancellationAsync`, netstandard2.0 build
on net48). The wait now uses the linked token. A timeout is still logged
as a discovery failure, as before. A companion test fix makes
`PeerTimeoutPreservesOtherEndpointsAndCoalescesPendingDiscoveryAsync(False)`
deterministic. The timed-out lookup drains on the thread pool, so the
test's retry could join it and then have its own time limit expire
before it started a fresh lookup. The test now waits for the second
resolver call before advancing the fake clock.
- **`Session` keep-alive logging (found while investigating the net48
nightly failures):** `OnKeepAliveError` and the keep-alive read failure
log used `Endpoint?.EndpointUrl`. `ClientBase.Endpoint` dereferences the
transport channel, so once the channel was detached it threw a
`NullReferenceException` on every keep-alive tick, and
`BadNoCommunication` never reached the `KeepAlive` subscribers. Both now
log the configured endpoint's URL. Three new `SessionKeepAliveTests`
cover a detached channel.
- **`ClientTestFramework` shared session (fix for 67
`Opc.Ua.Sessions.Tests` failures on the net48 nightly):** the
fixture-wide shared session used the 10 s default session timeout. A
test-host stall of about 11–18 s while other fixtures restarted their
servers let the server expire it, and every later test in the
`ClientTest("https")` and `ClientTest("opc.https")` fixtures failed with
`BadSessionIdInvalid`. The shared session now requests
`SharedSessionTimeout` = 120 s; per-test sessions keep 10 s.
- **`run-dotnet-tests.ps1`:** `-QuietOutput` help text only; it no
longer names the removed job.

The Azure secure file stays in place, because `master378` and the 1.x
release pipelines keep their own `test.yml` and still use it.

## Verification

- `nightly.yml` parses as valid YAML, and no job, output or summary
field still refers to the removed job.
- I ran the job's overlay script against a clone of `fuzz-corpus` at the
pinned commit. It overlaid all 22,293 files, left the tracked
`Assets/Repo` untouched, removed the checkout folder and wrote the
summary line.
- **Not yet run:** the workflow itself. It is
`schedule`/`workflow_dispatch` only, so it needs a manual `nightly.yml`
run on this branch. On master the corpus has so far only been replayed
on Windows (net48/net10.0 in Azure), so the Linux legs are new ground.

## Follow-ups (admin)

- Add a ruleset for `fuzz-corpus`: require pull requests and block force
pushes and deletion, so the pinned commit stays reachable.
- The `fuzz-private-corpus` environment, if it was ever created, can be
deleted.

## Related Issues

- No issue exists yet; this is part of the Azure DevOps → GitHub Actions
migration (#4513, #4524).

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [ ] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-09-25 07:49:00 +02:00