GithubStar/UA-.NETStandard
1
0
Fork 0
mirror of https://github.com/OPCFoundation/UA-.NETStandard.git synced 2026-10-20 16:52:07 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,739 Commits 108 Branches 105 Tags
fix/encoder-stack-depth-json-flake
Commit Graph

8 Commits

Author SHA1 Message Date
romanett 28f1401324 Server: cut request-path lock contention and allocation; add load harness (#4707) 
Profiling the reference server under load showed most request-path CPU
spent spinning on contended locks, plus several avoidable allocations
per request or per node. This PR removes them, switches the reference
server to Server GC, and adds the load harness used to measure it.

## Results

Paired A/B against `master` (3 alternating rounds, medians, 20 sessions,
workstation GC on both sides, 24-core machine):

| Scenario | Throughput | p50 | p90 | Server CPU / op | Server alloc /
op | Lock contention |
|---|---|---|---|---|---|---|
| Read, 1 value | **+45%** (94k → 137k req/s) | −20% | −46% | −37% |
−39% | −94% |
| Read, 100 values | **+40%** | −26% | −17% | −24% | −25% | +130% ¹ |
| Read, 100 values, SignAndEncrypt | **+48%** | −36% | −20% | −30% |
−26% | +106% ¹ |
| Browse (Server object) | **+33%** | −31% | −12% | −13% | −14% | −40% |
| Write, 100 values | +13% | −12% | −4% | −10% | −19% | −23% |
| Data-change subscriptions | ±0 ² | ≈ | ≈ | −14% | −17% | −29% |

¹ The faster server spends more time in the remaining per-node locks.
² Throughput is bounded by the writer rate in this scenario.

Separately, Server GC instead of workstation GC doubles Read-100
throughput (21k → 44k req/s) and cuts p90 from 2.1 ms to 0.6 ms, because
workstation GC paused the server for about half the wall time under
load. The working set grows from ~210 MB to ~430 MB.

## Changes

- **`ObjectPool<T>`**: lock-free slot array. `ConcurrentBag.Count` took
every per-thread lock on each `Return` of the pooled response value-task
source, once per request.
- **`FairRequestQueue`**:
- drops redundant gate entries: the fail-fast pre-checks and the stopped
check read volatile fields and are re-checked under the gate; the owner
count is folded into selection;
- coalesces wake pulses with a pending flag, so the bounded signal
channel's lock is only taken when no token is outstanding. Wake-count
semantics in the tests are unchanged.
- **`DefaultServerResourceIsolationProvider`**:
  - compares peer addresses without `GetAddressBytes()` allocations;
- builds the unsecured verification context only when a classifier is
configured.
- **TCP listener / channel**: caches the certificate DER copies per
channel, keyed on the certificate instance. `X509Certificate2.RawData`
returns a fresh copy on every access, and the listener read it for every
request.
- **`EndpointBase`**: resolves the request `ActivitySource` once instead
of `Assembly.GetCallingAssembly()` per request.
- **Node managers**:
- one `ServerSystemContext` copy per request (cached on the
`OperationContext`) for per-node callbacks: permission metadata, node
metadata, view membership, method lookup;
  - removes a discarded copy in `GetReferenceDescriptionAsync`;
- per-request operation caches are plain `Dictionary` instead of a
`ConcurrentDictionary` with 24 lock objects;
  - the validation cache is built in one pass;
  - fixes zero-capacity error lists.
- **`ResourceManager`**: translation lookups read an immutable snapshot
that is rebuilt after writes. Browse serialised on the global lock for
every display name.
- **`MonitoredItem`**: shares the default `DataChangeFilter` instead of
allocating one per sampled value.
- **Reference server**: `ServerGarbageCollection=true`; new "Garbage
collector mode" section in `docs/ServerScalability.md`.
- **`perf/`**: `ServerLoadHarness` (out-of-process server plus load
driver, server-side CPU/GC/alloc/contention deltas, optional
`dotnet-trace` capture), `TraceAnalyzer` (TraceEvent-based
CPU/allocation/contention aggregation), and paired A/B scripts. Not part
of `UA.slnx`.

### Behaviour notes for custom node managers
- The `IDictionary<NodeId, NodeState>` operation cache handed to
overrides is now a `Dictionary`. It was a `ConcurrentDictionary`, but
only one request flow uses it.
- Within one request, per-node callbacks of a node manager share one
`ServerSystemContext` copy.

### Later commits on this PR
- **Permission checks without `NodeMetadata`** (`9068684c3`): the
role-permission and access-restriction validators run on an internal
`PermissionMetadata` struct; `AsyncCustomNodeManager` fills it on the
stack from the same attributes and namespace defaults. Browse's
reference filter reads 3 attributes per target instead of 13. Read 100
values: 120.6 → 95.7 KB/request, ~21k → 26k req/s; Browse 67.6 → 51.8
KB, 26.7k → 29.8k req/s.
- **Read-only channel evidence** (`936b28d76`, `dc638ab9f`):
`SecureChannelContext.ClientChannelCertificate`,
`ServerChannelCertificate` and `ChannelThumbprint` are `ByteString`, so
the cached per-channel certificate buffer is shared without a copy and
cannot be altered by handlers or classifiers. **Public API change**, see
the new section in `docs/MigrationGuide.md`. The isolation provider's
revalidation snapshot still copies.
- **Session lookup** (`936b28d76`):
`SessionManager.TryGetSessionContext` takes the candidate snapshot
without the bindings lock; the validating check under the lock is
unchanged (2 instead of 4 global lock acquisitions per request).
- Unit tests for `ObjectPool`, the per-request system context cache, the
translation snapshot, `SecureChannelContext` null semantics and a
stopped `FairRequestQueue`; `perf/` follows the repository conventions.

## Testing
- `Opc.Ua.Server.Tests` (net10.0): 8386 passed before the `ByteString`
change. After it, 13 secure-session and isolation tests failed; both
causes are fixed in `dc638ab9f` and those tests (25 with their variants)
pass. A full rerun of the Server, Sessions, Core, Redundancy, LDS and
GDS suites on the final commit is in progress.
- Full solution builds for net10.0 and net48; `dotnet format whitespace`
clean.

### o6-automation opcua-benchmarks (sampling, deep pipelines)
Replicated both tests of the [o6-automation
benchmark](https://github.com/o6-automation/opcua-benchmarks) in the
harness (`--scenario counters`, and `read` with 10 clients × N
outstanding), paired against `origin/master`:

| Test | master | this PR |
|---|---|---|
| Subscription sampling, 100 Hz, every item ≥ 99 % of samples, 1 item |
FAIL (75–94 %) | PASS (100 %) |
| same, 1024 items | FAIL (64–73 %) | PASS (99.9 %) |
| same, capacity | — | between 4096 and 6144 items (~410–520k samples/s)
|
| Read, 10 clients × 100 outstanding, None | stalls (0–1.7k req/s) |
121–131k req/s |
| Read, 10 clients × 100 outstanding, SignAndEncrypt | stalls (0 req/s)
| 89–91k req/s |
| Read, 10 clients × 1000 outstanding | channels faulted
(`BadTcpNotEnoughResources`) | 0 errors, ~45–54k req/s |

- **Fixed-rate sampling** (`30b769d13`): the sampling loop waited one
interval after each sample, adding timer overshoot and sampling time to
every cycle; samples now have absolute deadlines.
- **Async response send** (`cdac5c5ca`): `SendResponse` blocked
thread-pool workers on the channel gate; with many requests in flight
the receive loop that was handed the gate waited for a pool thread and
the channel stalled. The TCP, Kestrel TCP and WebSocket listeners now
use `SendResponseAsync`.
- **Backpressure instead of channel fault** (`191cd2ba7`): server
channels stop reading while more than 100 responses are queued and
resume at 50, instead of faulting the channel. Client channels keep
reading. At 10,000 outstanding requests throughput is ~50k req/s with or
without the limit engaged, so the remaining gap to the 100-per-client
case is not the backpressure.

## Not changed
- `SentMessageQueue` deep-clones every `NotificationMessage`. Retained
messages return their pooled payloads on acknowledgement while the
response copy may still be queued for send, and a client can acknowledge
a sequence number before receiving it; the clone keeps a recycled
payload from reaching another session's response. Removing it needs a
send-completion hook from the transport.
- Generated responses allocate a default `ResponseHeader` that is
replaced (~2-3 % of per-request allocation). Fixing it means making
structure fields of every generated type lazily initialized, a
generator-wide change to default values, equality and encoding.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 13:00:49 +02:00
romanett 9b034f8291 Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700) 
# Description

Nullable reference types are now enabled once, in `common.props`, and
every project in the repository builds warning-clean with them.

**Configuration**
- `<Nullable>enable</Nullable>` lives in `common.props`. All per-project
`<Nullable>` settings are removed: 178 project files (enable,
annotations, and the `disable` in the three Aot test projects).
- 419 per-file `#nullable enable`/`restore` directives and the 4
`#nullable disable` directives are removed.
- Kept on purpose:
- The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type
Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`,
`PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the
project setting in generated code, so the directive is required there.
- `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against
the 1.5.378 packages and does not import the repository props.
- `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now
describe the central setting.

**Migration** (the 43 unset and 13 annotations-only projects, roughly
10k diagnostics)
- **Production code: annotated at the declarations.** Covers
`tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the
LDS sample.
- Members, parameters and returns are nullable where null actually
flows.
- Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`,
`IsMethodTypeDesign`).
- `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take
`params object?[]`. `Template.AddReplacement` takes `string?` (rendering
already skips null replacements).
- `IRegisteredServerStore.RegisterAsync` returns
`Task<RegistrationEntry?>`: it returns null when a registration is
removed.
- Dereferences backed by an invariant (for example, the validated model)
use `!`, so runtime behaviour is unchanged.
- **Behavioural changes in production code** (small, deliberate):
- `NodeSetToModelDesign` throws `InvalidOperationException` when the
file is not a NodeSet. Before, it threw a NullReferenceException on the
next line.
- `ModelDesignValidator` starts its lookup tables empty instead of null.
A redundant `if (m_nodes == null) return;` guard is gone.
- **Test code: minimal annotations.**
  - `?` on values that can be null, `!` where the test asserts non-null.
  - `= null!` for fixture members assigned in SetUp.
- `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the
`IEncodeable` constraint.
- A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`)
folded into the adjacent `IsNull`/`Count` checks.

**Notes for reviewers**
- The diff is large (~1,500 files) but almost entirely mechanical: `?`,
`!` and directive removal. The non-annotation changes are listed above.
- Known follow-up, not done here:
`IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay
non-nullable even though the validator returns null before it loads a
dictionary. Making them nullable would ripple through every generator
call site.

## Related Issues

- Fixes #4439

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

The "no new warnings" box covers net10.0 and net48: full non-incremental
builds of `UA.slnx`, `tools/SourceGeneration.slnx`,
`tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0
warnings and 0 errors under the repository's warnings-as-errors
settings. net9.0 and net8.0 also had no nullable diagnostics, but they
were not rebuilt after the final fixes. Tests have not been run locally
yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 08:48:13 +02:00
romanett 7212607f16 OPC UA Part 3 compliance: permissions, Write semantics, NodeManagement, NodeSet DataTypes, locales (#4675) 
# Description

Fixes the High and Medium findings of an OPC UA Part 3 (OPC 10000-3,
Address Space Model) compliance audit. Each finding was checked against
the full spec text (Parts 3, 4, 5, 6, 14 and 18) before it was fixed.
Every fix comes with regression tests that fail without it.

### Role permissions (Part 3 §8.55, Part 4 §5.12.2.2)
- The type-hierarchy exemption in
`MasterNodeManager.ValidateRolePermissions` /
`ValidateAccessRestrictions` now covers only Browse, Read and
ReadRolePermissions. Type nodes stay visible to generic clients, which
was the intent of #3705; every other permission bit is now enforced on
them.
- `ReceiveEvents` is enforced on the EventType node. The standard
NodeSet grants it on `AuditEventType` and its subtypes only to
SecurityAdmin, so only SecurityAdmin sessions now receive audit events.
- Call checks the `Call` permission and AccessRestrictions on the Object
as well as on the Method.
- When a Call's `methodId` is the Method declaration of the ObjectType,
the Object's Method with the same BrowseName is permission-checked and
invoked. The declaration is used only if the Object has no such Method.
This applies to `AsyncCustomNodeManager`, `CustomNodeManager2` and the
sample node manager.

### Write service (Part 3 §5.6.2, §8.40, §8.57; Part 4 §5.11.4)
- The synchronous `OnWriteValue` path (fluent `OnWrite`) checks DataType
and ValueRank first (`Bad_TypeMismatch`). After an IndexRange write it
merges the slice into the cached value instead of replacing it.
- StatusWrite / TimestampWrite are enforced (`Bad_WriteNotSupported`).
- Undefined enumeration values and invalid OptionSet writes return
`Bad_OutOfRange`. OptionSet writes merge only the valid bits into the
stored value.
- New `ReportPropertyValueChanged` / `ReportSemanticChange` let server
code raise SemanticChangeEvents for changes it makes itself.
- Required follow-up (Part 14 §6.2.11.1): if an external target rejects
a timestamp, the PubSub subscriber's TargetVariables writer retries
without it.

### NodeManagement and Call arguments (Part 3 §5.5.2, §5.7.3, §6.4.2;
Part 4 §5.8, §5.12.2)
- AddNodes creates the Mandatory InstanceDeclarations of the
TypeDefinition and rejects abstract or Interface types
(`Bad_TypeDefinitionInvalid`).
- AddReferences, DeleteReferences and DeleteNodes bump NodeVersion and
raise GeneralModelChangeEvents.
- Clients may omit trailing optional input arguments
(`HasOptionalInputArgumentDescription`).

### NodeSet DataTypes (Part 3 §5.8.3, §8.48; Part 6 Annex F.12)
- `UANodeSet.Import` completes structure definitions with the base
type's fields (base fields first). It creates definitions for structures
without fields and sets `DefaultEncodingId`.
`StructureDefinition.FirstExplicitFieldIndex` marks the subtype's own
fields.
- `DataTypeState` no longer mutates the stored definition on read.

### Locales (Part 3 §5.2.5, §8.5; Part 4 §5.4)
- Read, Browse and data change notifications return the translation that
best matches the session's LocaleIds, or all translations for `mul`. The
selection lives in the server dispatcher and is built on
`ResourceManager`: the new `TranslateValue` calls the overridable
`Translate`.
- Locale ids are matched case-insensitively, and fallback translations
carry their real locale.

### Client complex types (Part 6 §5.1.7)
- The Reflection.Emit builder encodes only `IsOptional` subtyped fields
as ExtensionObject. It also supports OptionSet subtypes: an unsupported
type is skipped instead of aborting the whole load.
- `TypeInfo.GetXmlName` uses the data type's XML name for dynamic
structures and OptionSets (the XML body element was `<Structure>`). Both
builders now produce identical XML.

## Behaviour changes
All are documented in `docs/MigrationGuide.md`.
- **Audit events:** only SecurityAdmin receives them. The CTT "Auditing
Connections" unit needs a SecurityAdmin user (e.g. sysadmin);
`AuditingOperationTests` was updated.
- **Write timestamps and status:** clients writing a SourceTimestamp or
a non-Good StatusCode get `Bad_WriteNotSupported` unless the Variable
has StatusWrite/TimestampWrite. No variable in `src/` sets these bits.
- **AddNodes:** it rejects abstract types (including `BaseVariableType`
and `BaseEventType`), and new instances now come with child nodes.
- **Call:** the Object needs the Call permission. The Object's own
Method runs even when the call names the type declaration.

## Not included
- Low-severity findings: missing validation on optional services and
author-side NodeSet rules.
- LocalizedText nested inside structures or Variant arrays, and in
HistoryRead or Call outputs.

## Testing
net10.0, Debug, one suite at a time on the merged branch. All suites
below pass:

| Suite | Passed |
|---|---|
| Types | 11430 |
| Core.Encoders | 2947 |
| Core.Schema | 179 |
| Core.Security | 558 |
| Client | 2761 |
| Server | 8210 |
| InformationModel | 1080 |
| Features | 145 |
| History | 536 |
| Di | 455 |
| Machinery | 167 |
| Robotics | 661 |
| Robotics.Intent | 22 |
| Pumps | 75 |
| Scales | 122 |
| Positioning | 69 |
| ISA95 | 144 |
| Vision | 484 |
| PubSub | 1490 |
| PubSub.Adapter | 151 |
| PubSub.Server | 177 |
| Redundancy.Server | 856 |
| Lds | 173 |
| Configuration | 241 |
| WotCon | 1496 |

- **Client.ComplexTypes:** one test wrote a SourceTimestamp to a
variable without TimestampWrite. It is fixed in this PR, and the suite
has not been re-run since.
- **Core:** 6068 passed, 1 failure in
`ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`. That telemetry
test passed when re-run alone, so it looks flaky.
- **Still running:** Sessions, Subscriptions (Classic, Durable,
standard) and Interop. Results will follow on this PR.
- `UA.slnx` builds with 0 errors for net10.0 and net48, with no new
analyzer warnings in changed files.

## Related Issues

- Part 3 compliance audit (no tracking issue)

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 12:43:59 +00:00
romanett a451053758 Fix the recurring and permanent CI failures on master (#4446) 
# Description

Almost every `master` build was reporting yellow or red, which meant the
pipeline signal had stopped being useful: a genuinely broken build
looked the same as a healthy one. I triaged every `master` build from
the last week across Azure DevOps (definition 14, 38 builds) and GitHub
Actions, clustered the errors by root cause, and fixed the nine that
were still live.

The headline finding is that the two loudest problems were not test bugs
at all. Codecov rejects the upload on **28 out of 28 builds**, and that
single warning is what turned every master build `partiallySucceeded`.
Separately, the weekly HA long-haul soak was mathematically incapable of
passing: the configured duration is per *scenario*, and the two
`[NonParallelizable]` soaks run back to back, so the floor is 2 x 60
minutes plus teardown against a 110-minute step budget.

A tenth cluster (the `Opc.Ua.Sessions.Tests` reverse-connect flake, 6
hits around Aug 31) was already fixed by #4366 and has not recurred, so
it needed no work.

## Clusters and root causes

| Cluster | Frequency | Root cause |
|---|---|---|
| codecov upload exits 1 | **28/28 builds** | Codecov returns
`Repository not found`; the resulting warning made every master build
yellow |
| `No space left on device` in coverage merge | nightly | Hosted Ubuntu
agent disk exhausted by the per-matrix coverage artifacts |
| Sample HA Long-Haul timeout | weekly | 2 soaks x 60 min against a 110
min budget, so it could never pass |
| `Artifact coverage-report already exists` | on stage re-run | Pipeline
artifact names must be unique per build |
| `NodeCache*` -> `address already in use` | macOS | `ServerFixture`
retried only `BadNoCommunication`; Kestrel https throws
`IOException`/`AddressInUseException` and escaped the retry, failing the
whole `OneTimeSetUp` |
| `MetricsAreEmittedForChannelLifetimeAsync` | macOS | Entry teardown
emits `close` *before* the `active(-1)` up/down counter, but the test
only waited for `close` |
| `OnPublishReceivedKeepAliveShouldDispatchKeepAliveAsync` | net472 |
The processor queues the ack *after* dispatching the notification the
test waits on |
| `ACarriedPartIsAuthoredAtTheGripperItRidesInAsync` | macOS | A single
starved sample (0.2125 m) tripped a 0.20 m absolute worst-case bound |
| `RobotIntentLiveChannelTests` fixture timeout | net472 | 30 s startup
budget too tight, and the failure text interpolated `lastException`
*before* the polling that assigns it, so it always read `Last error: .`
|

## Approach

**Pipeline.** Coverage reporting is best-effort by design, so the
codecov step now swallows the uploader's exit code and reports the
failure as plain log output rather than a warning that colours the run.
The coverage merge job reclaims the unused Android/GHC/Swift/CodeQL
toolchains before anything writes to the agent, and qualifies its
artifact name on a retry only, so the normal first attempt still
publishes `coverage-report`. The long-haul budgets were raised above the
real floor in both the workflow and the ADO pipeline, and the display
names now say `min/scenario` so the next reader does not repeat the same
miscalculation.

**Tests.** Each test fix targets the specific ordering or bound that was
wrong, rather than adding sleeps. The two metric/ack races now wait on
the measurement the assertions actually depend on. The OpenUSD test
gates the 95th percentile instead of the absolute worst case: the defect
it guards is systematic and moves *every* sample 0.25-0.49 m while the
robot drives, so it still fails that gate loudly, while one straddled
sample no longer can.

## Worth a careful look

- **The port-collision fix is the one real behavioural change.**
`ServerFixtureUtils.IsPortUnavailable` now classifies every transport's
form of the collision, including matching Kestrel's
`AddressInUseException` by type name so the test framework needs no
reference to the ASP.NET Core connection abstractions. It deliberately
does *not* match unrelated failures, which is covered by a negative
test, so a certificate error still surfaces immediately instead of being
retried 25 times on different ports.
- **I dropped an integration test I had written for that fix.** While
validating it I found the UA-TCP listener tolerates a partial bind on
Windows, so `StartAsync` succeeds on the same port and the test asserted
something untrue. I replaced it with deterministic unit tests of the
classifier instead of shipping a test that would itself become flaky.
- **The codecov fix is a mitigation, not a cure.** The `Repository not
found` response is account-side: the `CODECOV_TOKEN` is stale or the
repository is not activated. This change stops it from colouring the
build and logs the failure plainly, but uploads will not actually land
until someone refreshes the token. Flagging that explicitly so it is not
mistaken for a full fix.

## Related Issues

N/A - this addresses CI failures observed directly on `master` builds
rather than a filed issue.

## Checklist

_Put an `x` in the boxes that apply. You can complete these step by step
after opening the PR._

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [x] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

### Notes on the checklist

- Documentation: no doc updates were needed. The changes are CI
configuration and test-harness fixes with no public API surface, and the
rationale is captured in comments at each changed site.
- Local verification: all touched projects build clean on **net10.0**
and **net48**, and the targeted tests pass (Server 5/5, Client 46/46,
Features 31/31, Robotics.Intent 20/20, plus the Di and Core.Tests
cases). One pre-existing local `net48` PKI-store failure in
`Opc.Ua.Robotics.Intent.Tests` reproduces on unmodified `master` and is
unrelated to this change; CI's `net472` leg runs that project fine.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Marc Schier <marcschier@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com>
 2026-09-09 09:16:35 +00:00
Marc Schier aad8350bc1 Pluggable symmetric cryptography, an awaitable asymmetric path, registrable security policies, and PubSub crypto offboarding (#4206, #4207, #4208, #4210) (#4232) 
# Description

Closes the three crypto-offboarding follow-ups split out of #4192. Each
was recorded there as knowingly *not* done, with the reason and what
would justify revisiting it. Those conditions are met, and where one of
them turned out to be unachievable that is written down rather than
worked around.

The safety property throughout: **every seam is opt-in and null-checked,
and in the default software configuration the added code is observably
inert.** No interface dispatch on the per-message path, and the
asynchronous paths complete synchronously — so channel locking order,
buffer ownership and error sequencing are unchanged for a deployment
that configures nothing.

## #4207 — symmetric, key-derivation and RNG provider seams

`ISymmetricCryptoProvider`, `IKeyDerivationProvider` and
`ISecureRandomSource`, added on the terms #4207 set: only once a
concrete requirement arrived (a validated module that must perform
*every* operation, not just the asymmetric ones), behind a fast path,
with `SymmetricChannelCryptoBenchmarks` as the gate.

They are **facets discovered by type test**, not members of
`ICryptoProvider`, so providers written against the shipped interface
still compile. `PlatformCryptoProvider` implements all three, which
answers the "an interface nothing implements" objection recorded in the
issue.

`CryptoProviderFacets` returns `null` both when no registry is
configured *and* when resolution lands on the platform, because the
platform facets are the inline code. Resolution happens once, in
`CalculateSymmetricKeySizes`, and is held for the life of the channel.

`CryptoCompliance.GetUnservedOperationPurposes` closes the failure the
seam would otherwise introduce: a provider bound to `ChannelSymmetric`
without implementing the facet is silently replaced by the platform, so
a deployment could believe its validated module performed the
per-message cryptography while the platform did. Under `FipsOnly` that
now refuses to start.

## #4208 — the asymmetric path is awaitable

`RSA` and `ECDsa` are .NET's contracts and cannot be replaced without
making every ready-made hardware and cloud implementation unusable.
Instead an implementation may **also** declare `IAsyncRsaKey` /
`IAsyncEcdsaKey`, which the stack finds by type test.

Now asynchronous: the secure channel open and renew path, user identity
token signing and decryption, and session activation. A software key
declares neither facet, so those paths complete synchronously and
nothing about their ordering changes.

`ReadAsymmetricMessage` is refactored into shared helpers rather than
duplicated — adding an `await` was not worth two hundred cloned lines of
signature, padding and endpoint-selection logic. `AsymmetricMessage` /
`AsymmetricWriteResult` exist because an asynchronous method cannot have
`out` parameters.

The channel no longer serialises its state on a monitor. A monitor
cannot be held across an `await`, and a plain `SemaphoreSlim` was not an
option at the time: the code relied on re-entrancy in eight places
(`HandleIncomingMessage` holds the lock and calls `ForceChannelFault`,
which takes it again; `ForceReconnect` and `Shutdown` do the same).
`ChannelGate` replaces it at 32 sites and `DataLock` is `[Obsolete]`.
That reliance has since been removed as well — see [Removing the gate's
re-entrancy](#removing-the-gates-re-entrancy) below.

### Defects this surfaced, all fixed and regression-tested

**Detached work started inline.** Writes were started with `_ =
WriteBuffersAsync(...)`. An async method runs its prologue on the
caller's stack, so `LeaveInheritedContext` — which exists to stop
*detached* work inheriting the right to re-enter the gate — was
stripping the **caller's own** entitlement; when the send then completed
synchronously, the completion blocked on a gate that very thread was
holding. The client sent its OpenSecureChannel request and never
processed the response. Writes are now queued.

**A synchronous gate handle held across an `await`.** `Enter()` records
the acquiring thread so an inline completion callback can re-enter —
`ChannelAsyncOperation` invokes its callback both inline and detached.
That record is only sound while the holder is synchronously on that
thread; the token renewal path held it across an await, so a reused pool
thread was recognised as the holder.

**`SignAsync` rejected the policy that signs nothing.** It validated the
certificate before the algorithm, so `SecurityPolicies.None` — which
carries no certificate — threw where the synchronous `Sign` returns
`null`.

**`EnterAsync` lost the holder when contended** *(found in review)*. The
`AsyncLocal` recording ownership was written inside the awaiting method,
and .NET restores the caller's execution context when an async method
completes, so the write was discarded. The caller returned owning the
semaphore while `IsHeldByCurrentContext` reported `false` — and any
nested `Enter()` then blocked forever on a gate its own context held.
Reachable remotely: `HandleIncomingMessageAsync` → error path →
`ForceChannelFault`. The holder is now published from the caller's frame
before the wait, and only its depth is raised in the continuation.

**Queued writes lost their order** *(found in review)*. Sequence numbers
are assigned under the gate, but independently queued work items reach
the transport's FIFO send lock in arbitrary order, so a peer could
reject a chunk with `BadSequenceNumberInvalid`. Writes are now appended
to a chain: still off the caller's stack, but a write cannot start
before its predecessor finishes.

### Four more this surfaced in CI, on paths only the integration suites
reach

Queuing the writes to fix the deadlock above turned out to have moved
three implicit guarantees, and the gate had one more liveness bug. Each
was reproduced locally against a clean baseline before it was changed.

**A faulting channel's error message never reached the peer.** A
listener channel sends a UASC `Error` describing why it faulted and
closes the transport as its next statement. That worked only because the
write started on the caller's stack, so the bytes reached the socket
first; queued, the close discarded them and the client saw
`BadConnectionClosed` instead of the certificate status it was told to
expect. Nine `SecurityCertValidationTests` cases assert exactly that
status. The terminal error is written inline again, on a path that
touches the gate only to disclaim what it inherited and reports nothing
— a channel that is already faulted is owed nothing beyond its buffer.

**The certificate of a rejected sender leaked.** An asynchronous method
cannot have `out` parameters, so the asymmetric read returns a record
instead. That silently dropped an ownership guarantee: an `out`
parameter reaches the caller even when the method goes on to throw, so
the caller's `catch` disposed the sender certificate and reported it to
the audit. A return value does not — nine leaked certificates, one per
rejection case, which the leak detector fails the run on. The parsed
certificate is now handed to the caller before anything that can reject
the message, restoring both the disposal and the audit.

**The gate could strand its own permit.** It released only when the
holder's depth returned to zero, but that depth lives in an `AsyncLocal`
and is therefore shared with every context forked from the owner's — a
timer callback whose timer was created inside the region, for instance.
Such a context entering the gate raised the shared depth, so the owner's
own exit saw a non-zero depth and returned without releasing, and
nothing afterwards could: the nested exit is not the owner. Every later
entry on that channel blocked forever. `Opc.Ua.Features.Tests` hung
after its tests had finished, with a receive loop waiting on the gate
and the server unable to stop. The owner now releases when it leaves;
depth still gates re-entrancy, but liveness no longer depends on a count
another context can move.

**Write completion throttled the writer.** Reporting a completed write
sat inside the ordering chain, and the client channel implements that
callback by entering the gate — so every write queued behind a gate the
receive loop contends for on each incoming chunk. A session with fifty
subscriptions publishing continuously had its publish requests throttled
enough that the client kept topping them up past its own limit, which
`Subscriptions.Classic` asserts against. Completion is now reported off
the chain, which covers only what ordering actually requires.

### Still synchronous, and why

Certificate, certificate request and revocation list signing cannot be
made asynchronous at all: `X509SignatureGenerator.SignData` is invoked
by .NET's own builders. Service faults and the `Reconnect` override are
reached from synchronous call sites (`SendServiceFault` alone has eight)
and are left as they are behind scoped, commented suppressions. All
three are documented rather than left to be rediscovered.

## #4210 — PubSub

This one produced a finding rather than the API the issue asked for, and
the finding is the point:

> **With a standard Security Key Service the key necessarily exists in
process memory.** `GetSecurityKeys` (Part 14 §8.3.2) returns raw key
bytes over the wire, so the "key never leaves the device" property #4192
achieved for client and server **cannot** be achieved for PubSub through
the SKS pull profile. That is a property of the specification, not of
this stack.

A wrapped-key envelope would change what is on the wire and break
interoperability with third-party key services and publishers, so it was
rejected rather than built.

What is achievable is delivered: the per-message AES-CTR and HMAC route
through `ISymmetricCryptoProvider`, so a validated module performs them.
`IPubSubSecurityPolicy` is **not** modified — new members would break
every implementation of it; the provider is supplied through the policy
constructor. `PlatformSymmetricCryptoProvider` gained AES counter mode,
which the seam should have covered anyway since the algorithm enum names
it.

Key lifetime is also tightened: unpacking an SKS response copied key
material into intermediates that `ByteString.Create` then copied again,
leaving plain keys in the heap for the lifetime of the process. Those
are now zeroed.

### Two wiring gaps this depended on, both found in review

The registrations above were **inert**, so this section originally
claimed more than the code did:

- `CryptoProviderConfiguration` was applied nowhere outside tests, so
`AddCryptoProvider(configure)` never reached the registry in a real
host. The registry now applies every registered configuration when it is
first resolved — which also affects #4207's DI story.
- `PubSubSecurityWrapperResolver` selected
`PubSubAes256CtrPolicy.Instance`, the provider-less static singleton,
ignoring the provider-backed bundles registered for it. A `FipsOnly`
deployment therefore certified while the platform did the work. The
resolver now selects from the registered policies; the container and
`PubSubApplicationBuilder` both supply theirs.

A third, smaller one: `CryptoProviderBuilder.Use` published a
**policy-scoped** random source as the process-wide nonce source, so
`.For(purpose, policyUri).Use(hsm)` redirected nonce generation for
every other policy too. Only an unscoped binding does that now.

## #4206 — a provider can contribute a security policy

Added in review: the policy set was fixed at compile time, so a provider
could contribute key custody but not a *profile*. It is now table-driven
and open, and the blockers #4206 listed are gone.

```csharp
using IDisposable registration = SecurityPolicies.Register(
    new SecurityPolicyInfo("urn:vendor:SecurityPolicy#CustomProfile", "CustomProfile")
    {
        PlatformSupport = () => true,
        SupportedCertificateTypes = [ObjectTypeIds.RsaSha256ApplicationCertificateType],
        IsDefault = true
    });

services.AddOpcUa()
    .AddCryptoProvider(crypto => crypto
        .For(CryptoPurpose.ChannelSymmetric, "urn:vendor:SecurityPolicy#CustomProfile").Use(module))
    .AddSecurityPolicy(customPolicy);
```

- **The reflection is gone.** Four lookup tables were built by
reflecting over the fields of `SecurityPolicies` and
`SecurityPolicyInfo`. They are one explicit table now, which removes the
**last reflection in `Security/Constants`** — so this is a trimming and
Native AOT improvement as much as a feature.
- **`SecurityPolicyInfo` is constructible from another assembly** and
carries the metadata the helpers need, so `IsPlatformSupportedName`,
`GetDefaultUris`, `MapSecurityPolicyToCertificateTypes` and
`GetCurveFromCertificateTypeId` read that table instead of each being
hand-written. Adding a policy is one entry rather than five edits.
- **The dead `#if CURVE25519` conditional is deleted** from
`Security/Constants`.
- Registration is copy-on-write, so the read path stays lock-free.
Shadowing a built-in policy requires `replaceExisting: true` and is
reversible by disposing the registration.

`RegisterLightsUpCurvePoliciesFromOutsideCore` is the acceptance test
the issue asked for: it registers `ECC_curve25519` and `ECC_curve448`
from the test project and asserts they become discoverable through the
public API.

**Registering a policy makes it advertised and resolvable; it does not
supply the cryptography behind it.** For those two curve profiles the
in-tree key agreement is still behind a compile symbol no project
defines plus a BouncyCastle dependency, so a deployment lighting them up
provides the operations through a provider. That division is the point:
the policy set says *what* is offered, the provider says *who* performs
it.

## Removing the gate's re-entrancy

Folded in from #4242, which was merged into this branch.

`ChannelGate` was re-entrant, and that forced the design behind most of
the defects above. A holder that awaits releases its thread, so
ownership could not be tracked by thread identity — it had to live in an
`AsyncLocal<Holder?>`. A logical context is *inherited*, so anything
started while the gate was held inherited the right to re-enter and ran
inside the guarded region alongside whatever started it, unless it opted
out by hand via `LeaveInheritedContext()` at each of eight sites,
enforced by review alone. Five of the defects listed above trace to that
mechanism, including both CI hangs.

So the *requirement* was removed rather than the mechanism, and the
`AsyncLocal` then deleted itself. Every path that re-entered now calls a
lock-free `Core` method (`ForceChannelFaultCore`, `ShutdownCore`,
`ForceReconnectCore`, `CompleteConnectCore`), and three regions stopped
running foreign code under the lock: the `RequestReceived` dispatch is
deferred until the gate is released, `OnTransportError` no longer wraps
the `HandleSocketError` virtual, and the callback setters and
`EndpointDescription` publish a single reference with a volatile write
instead of locking — the last is load-bearing, because
`AsyncResultBase.OperationCompleted` invokes the reverse-hello callback
inline while the gate is held.

`ChannelGate` is now a `SemaphoreSlim(1,1)` behind a `Releaser`;
`Holder`, the thread-identity fallback, `LeaveInheritedContext`,
`IsHeldByCurrentContext` and all eight disclaimer calls are gone, with
the call shape at all 32 sites unchanged. `SaveIntermediateChunk`,
`GetSavedChunks` and `DoMessageLimitsExceeded` gained a `gateHeld`
argument, because the client reaches them both with and without the gate
and must pick the matching teardown — without it an oversized
`OpenSecureChannel` response deadlocked the receive loop against itself.

The remaining paths were not found by reading the code, since that is
what missed them originally. A temporary strict mode made every
re-entrant acquisition throw with its stack, the suite was run under it
until clean, and it was deleted with the re-entrancy it measured. It
caught two sites review had missed: the `EndpointDescription` **property
getter** taking the gate, and the `RequestReceived` dispatch to
application code happening inside the guarded region.

Two deliberate residuals are documented for reviewers: `Dispose` no
longer takes the gate (unavoidable — it is reached from `ChannelFaulted`
with the gate held, where the old re-entrant acquisition excluded
nothing anyway), and the deferred request dispatch builds
`SecureChannelContext` just after gate release rather than under it.

## Testing

New: the three seams and their platform implementations, facet discovery
and resolution precedence, `FipsOnly` refusing a provider that cannot
serve what it was bound to, AES counter mode (including a
non-block-aligned round trip and a check that the counter carries, since
one that did not would silently reuse key stream), `ChannelGate`
(exclusion, re-entrancy across an await, re-entrancy on a reused thread,
and the deadlocks above), the asynchronous channel paths, and the PubSub
policy provider.

Each review finding got a test that fails without its fix:

- `ContendedEntryStillRecordsTheHolderAsync` — established contention
deterministically, then re-entered; failed on the pre-fix gate. Removed
with the re-entrancy it covered; `ChannelGateTests` now asserts the
non-re-entrant contract instead.
- `SecureClientAndTcpListenerKeepChunkOrderUnderConcurrentRequestsAsync`
— 60 concurrent multi-chunk `SignAndEncrypt` requests over a real
loopback listener; failed 3 runs out of 3 with the previous dispatch,
passes 3 out of 3 with the chain.
-
`DependencyInjectionRoutesPubSubCryptoThroughTheRegisteredProviderAsync`
— builds a container, registers a counting provider, wraps a message and
asserts the provider actually performed the encryption and the
signature. This is the assertion the PubSub claim above now rests on.
- `PolicyScopedRandomBindingDoesNotEscapeItsPolicy`.

Two further assertions are the ones that would catch a silent defect: a
provider and the platform must produce **byte-identical** output — a
validated module that disagreed could not interoperate — and with a
software key the returned `ValueTask` must report
`IsCompletedSuccessfully`, which is what makes "ordering is unchanged"
checkable rather than merely asserted.

`SymmetricChannelCryptoBenchmarks` gains a provider-registered variant
beside the existing baseline, so the cost of the indirection is measured
rather than assumed.

Verified locally, every suite at or above its recorded baseline: `dotnet
build UA.slnx -c Release` **0 warnings, 0 errors on all six target
frameworks**; Core **4413** (net10.0); Core.Security **617**, with its
certificate-leak detector clean; PubSub **1350** plus PubSub.Diagnostics
**54** and PubSub.Udp **233**; Features **156**; Subscriptions.Classic
**32**; Server integration **4053**; Client integration **2123**;
MigrationAnalyzer **151** and MigrationAnalyzer.Core **13**. Zero
failures in each.

Core moved from 4388 to 4381 when the gate change landed — eight
re-entrancy tests were removed along with the behaviour they covered,
and one deadlock regression test added — and has since grown to 4413
with the registry isolation, provider-output and security-review tests.
`ChannelGateTests` was rewritten around the new contract — exclusion
across both entry modes, an uncontended asynchronous entry not
suspending, re-entry blocking by contract, and the property that used to
be the hazard: work started while the gate is held is excluded without
having to opt out.

Every CI failure this PR produced was reproduced locally, diagnosed
against a clean `origin/master` baseline in a second worktree, and fixed
at the source. Two hangs were found from `--blame-hang-timeout` dumps
read with `dotnet-dump`. One failure that looked like a candidate was
confirmed **not** to be: an intermittent `Opc.Ua.Aot.Tests` connect
failure, pre-existing on `origin/master` (5 of 8 baseline runs failed,
versus 3 of 14 here).

### The publish request overshoot

`Subscriptions.Classic.PublishRequestCountAsync` failed on the Windows
net10.0 leg with **207** outstanding publish requests against a cap of
`max(maxServerPublishRequest, subscriptions)` = 50. It failed in 2 of 6
pipeline builds on this branch, never on `master` in 13 sampled builds,
and never locally in 12 attempts including under deliberate CPU
contention.

The mechanism is a genuine race.
`ClassicSubscriptionEngine.QueueBeginPublish` was a **check-then-act**:
it read `GoodPublishRequestCount`, compared it against
`GetDesiredPublishRequestCount(false)`, and issued a publish if it was
below. It runs on every publish completion, so *n* completions that
overlap each read the same value, each conclude they are below the cap,
and each issue — overshooting by up to *n* per round. The read also
*lags*: a request only becomes visible to `GoodPublishRequestCount` once
`AsyncRequestStarted` has recorded it, which happens after the request
has already been sent.

That code is pre-existing and was untouched by this PR. What this PR
changed is how hard the race is hit: the server used to hold the channel
gate across request processing, which serialised every request on a
channel, so publish completions arrived one at a time. Dispatching
`RequestReceived` outside the gate — an intended improvement, since a
channel should not serialise on application request-processing time —
lets them complete concurrently.

Rather than change the publish pipeline blind, the race was first made
reproducible.
`ConcurrentPublishReEvaluationDoesNotExceedDesiredRequestCount` drives
64 barrier-released completions against a mocked context whose count
lags the send exactly the way the session's does. It issued **64
requests where 5 were desired** — the same defect the CI leg hit, now
deterministic and running in milliseconds.

The engine now tracks the requests it has in flight and reserves a slot
with a compare-exchange before sending, so the decision and the send are
one atomic step. The reservation is released when the request completes,
when the send is declined, or when it throws. The repro test then issues
at most the desired count.

`StartPublishing` is deliberately left **uncapped**. It is the recovery
valve that refills a pipeline whose requests are outstanding but no
longer expected to return, so gating it on the in-flight count would
remove the only path out of a stalled pipeline. An earlier revision of
this fix did gate it, and that is the one change here that could not be
justified on the evidence — it was reverted.

The assertion was **not** loosened, retried away, or marked flaky: it
caught a real throttling regression earlier in this PR and is worth
keeping sharp.

Validated: the repro test red before the fix and green after;
`Subscriptions.Classic` 32/32 **five** consecutive runs on net10.0 and
once on net48; `Opc.Ua.Client.Tests` 2124 on net10.0 across four
consecutive runs and 2127 on net48; `Opc.Ua.Subscriptions.Tests` 604
passed with this fix in place.

### The SSH.NET advisory that turned CI red repo-wide

Partway through this work every build job began failing with `NU1903:
Package 'SSH.NET' 2025.1.0 has a known high severity vulnerability`
([GHSA-q939-rpr3-3284](https://github.com/advisories/GHSA-q939-rpr3-3284),
`<= 2025.1.0` affected, first patched `2026.0.0`).

Nothing in the dependency graph changed — the advisory was published.
Build 17026 failed on it where build 17000, two hours earlier on an
identical graph, had passed every build job. SSH.NET is not referenced
anywhere in the repository; it arrives transitively through
`Testcontainers.Kafka`.

`master` landed the remediation independently while this was being
investigated (a central transitive pin to `2026.0.0`, which
`CentralPackageTransitivePinningEnabled` exists for). This branch
briefly carried its own identical pin, which made the merge commit hold
**two** `PackageVersion` entries for SSH.NET and fail restore with
`NU1506`. The duplicate is removed and `master` is merged, so the pin
now comes from one place only.

The audit was **not** suppressed: `NuGetAudit` stays enabled and
`NU1903` stays an error. Verified after the merge — `dotnet restore
UA.slnx` reports neither NU1903 nor NU1506, and `dotnet build UA.slnx -c
Release` is **0 warnings, 0 errors** with the audit live.

The remaining red check in build 17026 was not from this change either.
`aot-ubuntu-latest` is the intermittent `Opc.Ua.Aot.Tests` failure
already noted above — `ReconnectSessionAsync` timed out at 30s with 135
of 136 passing. It remains below the `origin/master` baseline rate (4 of
15 sampled here against 5 of 8 on the baseline), and the managed
reconnect paths are green throughout: `Opc.Ua.Client.Tests` 2124 across
four consecutive runs and `Opc.Ua.Subscriptions.Tests` 604.


## Documentation

`docs/CryptoProvider.md` gains sections on substituting the symmetric
primitives, using a key served over a network, and PubSub — including
what cannot be claimed and why, the scope rule for a random-source
binding, and how to supply a provider-backed policy outside the
container. `docs/MigrationGuide.md` covers the `[Obsolete]` channel
virtuals and `DataLock`, and the non-re-entrant gate's contract for
channel subclasses. `docs/WhatsNewIn2.0.md` is updated.
`plans/cryptooffboard.md` is deleted now that the work it planned has
landed.

## Note for reviewers

This touches the secure channel handshake, which is the most
security-critical code in the stack. Nine distinct concurrency and
lifetime defects were found and fixed in it during this work — three
during development, two in review, four from CI — so `ChannelGate`, the
write path and the certificate ownership across the new suspension
points deserve the closest reading.

Five of those nine traced to the gate's `AsyncLocal`, which is why the
re-entrancy was subsequently removed and the `AsyncLocal` deleted. The
wrapper/`Core` split and the two documented residuals in that section
are the parts most worth a second reading.

The #4206 work removes the last reflection from `Security/Constants` and
rewrites the policy lookup tables, so
`SecurityConfiguration.SupportedSecurityPolicies` and the endpoint set
it feeds are worth a second look: an earlier revision of it changed that
set and broke six server tests, which is now covered by
`SecurityConfigurationBuildsExpectedSupportedPolicySet`.

## Related Issues

- Fixes #4206
- Fixes #4207
- Fixes #4208
- Fixes #4210
- Follow-up #4250 — this PR makes the security policy set injectable,
but nothing in the stack resolves `ISecurityPolicyRegistry` from DI yet,
so every production path still lands on the `Default` fallback. #4250
tracks threading it through `Session` and `UaSCBinaryChannel`, which is
what makes the injection points already added here load-bearing.

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [x] I have addressed **all** PR feedback received.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com>
Copilot-Session: 7a68ca72-83e8-496c-b2f5-3df4c718320b
Copilot-Session: 3f5ab76a-c0ba-4f47-87ac-54d1f4182c6b
 2026-08-15 05:23:20 -07:00
Marc Schier e73e71184d [WotCon] Add OPC UA WoT Connectivity 1.1 with the Types, source generation, Server and xRegistry work it builds on (#4128) 
## Summary

Adds OPC UA WoT Connectivity 1.1 to the stack, together with the Types,
source generation, Server and xRegistry work it is built on. This branch
is the merge point for the fourteen-PR stack that replaced integration
PR #4093; every constituent PR was reviewed and approved separately and
is listed below.

414 files changed, +108,936 / -3,199. Roughly 46k added lines of product
code, 57k of tests, plus samples, tools and documentation.

## Types

- Complete `ExtensionObject` raw-body decoding for binary, XML and JSON
bodies by resolving the concrete type through the message context's
encodeable factory, and simplify the `Variant` structure helpers by
delegating to it (#4128).
- Add `IAtomicFileReplace` as an optional `IFileSystem` capability so
existing external implementations keep working, with atomic publish for
`LocalFileSystem` via `File.Replace`/`File.Move` and for
`VirtualFileSystem` by re-keying the in-memory entry (#4128).
- Add lossless conversion between WoT documents and NodeSet2
(`Opc.Ua.Wot.WotNodeSetConverter`). A byte-exact `uav:nodeSet` envelope
is preserved when requested, the structured `uav:nodes` projection is
used when the readable vocabulary is incomplete, and NodeSet2 is
otherwise synthesized from readable WoT terms. Unmapped JSON members
survive a round trip as pointer-addressed residue (#4131).

## Source generation

- Improve the NodeSet to ModelDesign conversion and the generated node
state (#4132).
- Generate OPC UA models directly from WoT Thing Description files
(#4134).

## Server

- Add live NodeManager shadow and immediate reload. A reload
materializes a new generation beside the active one and switches
atomically; the superseded generation is retired gracefully so existing
MonitoredItems keep being served until they drain, or immediately when
the caller asks for it. Includes the request admission and drain
machinery that lets an orderly shutdown wait for admitted requests
instead of tearing down underneath them (#4147).
- Make subscription transfer between sessions transactional, so a failed
transfer leaves neither session holding a partially moved subscription
(#4150).

## xRegistry

- Allow clients to use explicit registry roots rather than assuming a
well-known location (#4130).

## WoT Connectivity

- Add the WoT Connectivity 1.1 information model: a registry-first
revision layered on the abstract xRegistry base model, incorporating the
complete published OPC 10100-1 v1.02 surface as deprecated nodes in the
same namespace so existing 1.02 clients keep working (#4139).
- Add the protocol binding abstractions and planners that compile WoT
forms into executable plans (#4140).
- Add the HTTP, Modbus, OPC UA and MQTT binding executors (#4144).
- Add the registry and its stores, including document versioning,
validation, dependency resolution and refresh (#4146).
- Add the registry client (#4142).
- Materialize WoT Thing Descriptions into the server address space,
deriving types from Thing Models and instances from Thing Descriptions
(#4154).
- Restore the WoT asset registry bridge so the deprecated 1.02 asset
surface is backed by the registry (#4171).
- Add the WoT Connectivity samples and complete the documentation
(#4156), and restore the remaining coverage and docs (#4172).

## New projects

`src/Opc.Ua.WotCon.Bindings`, `src/Opc.Ua.WotCon.Bindings.Mqtt`,
`tests/Opc.Ua.WotCon.Bindings.Tests`,
`tests/Opc.Ua.WotCon.Samples.Tests`, and the `samples/WotCon`
AggregationServer, AggregationClient and FlatTagServer samples.

## Constituent pull requests

Merged into this branch top-down, each independently approved:

| PR | Title |
| --- | --- |
| #4128 | [Types] Complete ExtensionObject decoding and add an atomic
file replace capability |
| #4131 | [Types] Add lossless conversion between WoT documents and
NodeSet2 |
| #4132 | [SourceGeneration] Improve NodeSet to ModelDesign conversion
and generated node state |
| #4134 | [SourceGeneration] Generate OPC UA models from WoT Thing
Description files |
| #4147 | [Server] Add live NodeManager shadow and immediate reload |
| #4150 | [Server] Make subscription transfer between sessions
transactional |
| #4130 | [XRegistry] Allow clients to use explicit registry roots |
| #4139 | [WotCon] Add the WoT Connectivity 1.1 information model |
| #4140 | [WotCon] Add the WoT protocol binding abstractions and
planners |
| #4144 | [WotCon] Add the HTTP, Modbus, OPC UA and MQTT binding
executors |
| #4146 | [WotCon] Add the WoT Connectivity registry and its stores |
| #4142 | [WotCon] Add the WoT Connectivity registry client |
| #4154 | [WotCon] Materialize WoT Thing Descriptions into the server
address space |
| #4156 | [WotCon] Add the WoT Connectivity samples and complete the
documentation |
| #4171 | Restore WoT asset registry bridge |
| #4172 | Restore WoT coverage and docs |

Supersedes #4093.

## Follow-up

The WoT Connectivity and WoT Binding drafts have moved to 1.1-draft2
since this work was authored. Aligning with that revision - the removed
group vocabulary, the new projection/View construct, the remaining model
vocabulary terms and the two new portable-identity validation rules - is
tracked separately and will follow in its own pull request.

## Validation

Built for every target framework with zero warnings and zero errors, and
validated at the tip of the stack with `Opc.Ua.WotCon.Tests` at 999
passed / 0 failed and `Opc.Ua.Server.Tests` at 4024 passed / 0 failed.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
 2026-08-05 10:14:15 +02:00
Marc Schier fde50c5700 Enforce ActivateSession transfer and reconnect security (OPC 10000-4 §5.7.3.1) (#4050) 
# Description

This draft isolates the Session/ActivateSession, reconnect, and
distributed Session security work split from #4021.

### Failure

- `ActivateSession` could transfer a Session to a new SecureChannel
without consistently enforcing the original client certificate,
MessageSecurityMode, SecurityPolicy, and exact OPC `ClientUserId`
semantics.
- Missing or legacy distributed Session security state did not provide
enough information to validate transfer safely.
- Malformed or reused nonces, concurrent activation, and cancellation
could leave activation state inconsistent.
- Client reconnect could treat recoverable Session-loss/security
responses as fatal, reuse a Session after client-certificate rotation,
or report a managed channel ready before Session recreation completed.

### Fix

- Serialize activation, consume server nonces once, validate application
signatures and user tokens asynchronously, and enforce transfer
continuity against the original channel and authenticated identity.
- Persist versioned transfer-security state for distributed Sessions and
fail closed when certificate, identity, activation, or nonce state is
absent or invalid.
- Validate server signatures/nonces on the client, recreate Sessions for
certificate rotation and recoverable activation failures, and await
participant recreation before transitioning the managed channel to
`Ready`.
- Make the restart recovery integration test use bounded reconnect
cycles instead of a fixed startup delay.

### Specification

- OPC 10000-4 §5.7.2.2 — CreateSession signatures and nonces.
- OPC 10000-4 §5.7.3.1 — ActivateSession identity continuity and
SecureChannel transfer.
- OPC 10000-4 §6.1.8 — application nonce generation, validation, and
reuse requirements.

### Validation

- `git diff --check`
- Release `net10.0`: `Opc.Ua.Redundancy.Server.Tests` 567/567 plus the
targeted server Session, Subscription transfer, ClientUserId, client
reconnect and certificate rotation suites pass.
- Release `net48`: the same redundancy suite (567/567) and the targeted
Session/Subscription suites pass.
- Full `Opc.Ua.Server.Tests` run on `net10.0`: 3633/3634. The single
failure,
`ServerFluentApiHostingTests.ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`,
reproduces identically on the unmodified commit and is a local
certificate-store path issue rather than a regression.

These were targeted validation runs; no long-soak result is claimed.
 2026-07-28 12:07:02 +02:00
Marc Schier 7cec2d614d Move repository files to the consolidated layout 2026-07-16 06:25:54 +02:00