19 Commits

Include renames

Author	SHA1	Message	Date
4 people	aad8350bc1	Pluggable symmetric cryptography, an awaitable asymmetric path, registrable security policies, and PubSub crypto offboarding (#4206, #4207, #4208, #4210) (#4232) ... # Description Closes the three crypto-offboarding follow-ups split out of #4192. Each was recorded there as knowingly *not* done, with the reason and what would justify revisiting it. Those conditions are met, and where one of them turned out to be unachievable that is written down rather than worked around. The safety property throughout: **every seam is opt-in and null-checked, and in the default software configuration the added code is observably inert.** No interface dispatch on the per-message path, and the asynchronous paths complete synchronously — so channel locking order, buffer ownership and error sequencing are unchanged for a deployment that configures nothing. ## #4207 — symmetric, key-derivation and RNG provider seams `ISymmetricCryptoProvider`, `IKeyDerivationProvider` and `ISecureRandomSource`, added on the terms #4207 set: only once a concrete requirement arrived (a validated module that must perform *every* operation, not just the asymmetric ones), behind a fast path, with `SymmetricChannelCryptoBenchmarks` as the gate. They are **facets discovered by type test**, not members of `ICryptoProvider`, so providers written against the shipped interface still compile. `PlatformCryptoProvider` implements all three, which answers the "an interface nothing implements" objection recorded in the issue. `CryptoProviderFacets` returns `null` both when no registry is configured *and* when resolution lands on the platform, because the platform facets are the inline code. Resolution happens once, in `CalculateSymmetricKeySizes`, and is held for the life of the channel. `CryptoCompliance.GetUnservedOperationPurposes` closes the failure the seam would otherwise introduce: a provider bound to `ChannelSymmetric` without implementing the facet is silently replaced by the platform, so a deployment could believe its validated module performed the per-message cryptography while the platform did. Under `FipsOnly` that now refuses to start. ## #4208 — the asymmetric path is awaitable `RSA` and `ECDsa` are .NET's contracts and cannot be replaced without making every ready-made hardware and cloud implementation unusable. Instead an implementation may **also** declare `IAsyncRsaKey` / `IAsyncEcdsaKey`, which the stack finds by type test. Now asynchronous: the secure channel open and renew path, user identity token signing and decryption, and session activation. A software key declares neither facet, so those paths complete synchronously and nothing about their ordering changes. `ReadAsymmetricMessage` is refactored into shared helpers rather than duplicated — adding an `await` was not worth two hundred cloned lines of signature, padding and endpoint-selection logic. `AsymmetricMessage` / `AsymmetricWriteResult` exist because an asynchronous method cannot have `out` parameters. The channel no longer serialises its state on a monitor. A monitor cannot be held across an `await`, and a plain `SemaphoreSlim` was not an option at the time: the code relied on re-entrancy in eight places (`HandleIncomingMessage` holds the lock and calls `ForceChannelFault`, which takes it again; `ForceReconnect` and `Shutdown` do the same). `ChannelGate` replaces it at 32 sites and `DataLock` is `[Obsolete]`. That reliance has since been removed as well — see [Removing the gate's re-entrancy](#removing-the-gates-re-entrancy) below. ### Defects this surfaced, all fixed and regression-tested **Detached work started inline.** Writes were started with `_ = WriteBuffersAsync(...)`. An async method runs its prologue on the caller's stack, so `LeaveInheritedContext` — which exists to stop *detached* work inheriting the right to re-enter the gate — was stripping the **caller's own** entitlement; when the send then completed synchronously, the completion blocked on a gate that very thread was holding. The client sent its OpenSecureChannel request and never processed the response. Writes are now queued. **A synchronous gate handle held across an `await`.** `Enter()` records the acquiring thread so an inline completion callback can re-enter — `ChannelAsyncOperation` invokes its callback both inline and detached. That record is only sound while the holder is synchronously on that thread; the token renewal path held it across an await, so a reused pool thread was recognised as the holder. **`SignAsync` rejected the policy that signs nothing.** It validated the certificate before the algorithm, so `SecurityPolicies.None` — which carries no certificate — threw where the synchronous `Sign` returns `null`. **`EnterAsync` lost the holder when contended** *(found in review)*. The `AsyncLocal` recording ownership was written inside the awaiting method, and .NET restores the caller's execution context when an async method completes, so the write was discarded. The caller returned owning the semaphore while `IsHeldByCurrentContext` reported `false` — and any nested `Enter()` then blocked forever on a gate its own context held. Reachable remotely: `HandleIncomingMessageAsync` → error path → `ForceChannelFault`. The holder is now published from the caller's frame before the wait, and only its depth is raised in the continuation. **Queued writes lost their order** *(found in review)*. Sequence numbers are assigned under the gate, but independently queued work items reach the transport's FIFO send lock in arbitrary order, so a peer could reject a chunk with `BadSequenceNumberInvalid`. Writes are now appended to a chain: still off the caller's stack, but a write cannot start before its predecessor finishes. ### Four more this surfaced in CI, on paths only the integration suites reach Queuing the writes to fix the deadlock above turned out to have moved three implicit guarantees, and the gate had one more liveness bug. Each was reproduced locally against a clean baseline before it was changed. **A faulting channel's error message never reached the peer.** A listener channel sends a UASC `Error` describing why it faulted and closes the transport as its next statement. That worked only because the write started on the caller's stack, so the bytes reached the socket first; queued, the close discarded them and the client saw `BadConnectionClosed` instead of the certificate status it was told to expect. Nine `SecurityCertValidationTests` cases assert exactly that status. The terminal error is written inline again, on a path that touches the gate only to disclaim what it inherited and reports nothing — a channel that is already faulted is owed nothing beyond its buffer. **The certificate of a rejected sender leaked.** An asynchronous method cannot have `out` parameters, so the asymmetric read returns a record instead. That silently dropped an ownership guarantee: an `out` parameter reaches the caller even when the method goes on to throw, so the caller's `catch` disposed the sender certificate and reported it to the audit. A return value does not — nine leaked certificates, one per rejection case, which the leak detector fails the run on. The parsed certificate is now handed to the caller before anything that can reject the message, restoring both the disposal and the audit. **The gate could strand its own permit.** It released only when the holder's depth returned to zero, but that depth lives in an `AsyncLocal` and is therefore shared with every context forked from the owner's — a timer callback whose timer was created inside the region, for instance. Such a context entering the gate raised the shared depth, so the owner's own exit saw a non-zero depth and returned without releasing, and nothing afterwards could: the nested exit is not the owner. Every later entry on that channel blocked forever. `Opc.Ua.Features.Tests` hung after its tests had finished, with a receive loop waiting on the gate and the server unable to stop. The owner now releases when it leaves; depth still gates re-entrancy, but liveness no longer depends on a count another context can move. **Write completion throttled the writer.** Reporting a completed write sat inside the ordering chain, and the client channel implements that callback by entering the gate — so every write queued behind a gate the receive loop contends for on each incoming chunk. A session with fifty subscriptions publishing continuously had its publish requests throttled enough that the client kept topping them up past its own limit, which `Subscriptions.Classic` asserts against. Completion is now reported off the chain, which covers only what ordering actually requires. ### Still synchronous, and why Certificate, certificate request and revocation list signing cannot be made asynchronous at all: `X509SignatureGenerator.SignData` is invoked by .NET's own builders. Service faults and the `Reconnect` override are reached from synchronous call sites (`SendServiceFault` alone has eight) and are left as they are behind scoped, commented suppressions. All three are documented rather than left to be rediscovered. ## #4210 — PubSub This one produced a finding rather than the API the issue asked for, and the finding is the point: > **With a standard Security Key Service the key necessarily exists in process memory.** `GetSecurityKeys` (Part 14 §8.3.2) returns raw key bytes over the wire, so the "key never leaves the device" property #4192 achieved for client and server **cannot** be achieved for PubSub through the SKS pull profile. That is a property of the specification, not of this stack. A wrapped-key envelope would change what is on the wire and break interoperability with third-party key services and publishers, so it was rejected rather than built. What is achievable is delivered: the per-message AES-CTR and HMAC route through `ISymmetricCryptoProvider`, so a validated module performs them. `IPubSubSecurityPolicy` is **not** modified — new members would break every implementation of it; the provider is supplied through the policy constructor. `PlatformSymmetricCryptoProvider` gained AES counter mode, which the seam should have covered anyway since the algorithm enum names it. Key lifetime is also tightened: unpacking an SKS response copied key material into intermediates that `ByteString.Create` then copied again, leaving plain keys in the heap for the lifetime of the process. Those are now zeroed. ### Two wiring gaps this depended on, both found in review The registrations above were **inert**, so this section originally claimed more than the code did: - `CryptoProviderConfiguration` was applied nowhere outside tests, so `AddCryptoProvider(configure)` never reached the registry in a real host. The registry now applies every registered configuration when it is first resolved — which also affects #4207's DI story. - `PubSubSecurityWrapperResolver` selected `PubSubAes256CtrPolicy.Instance`, the provider-less static singleton, ignoring the provider-backed bundles registered for it. A `FipsOnly` deployment therefore certified while the platform did the work. The resolver now selects from the registered policies; the container and `PubSubApplicationBuilder` both supply theirs. A third, smaller one: `CryptoProviderBuilder.Use` published a **policy-scoped** random source as the process-wide nonce source, so `.For(purpose, policyUri).Use(hsm)` redirected nonce generation for every other policy too. Only an unscoped binding does that now. ## #4206 — a provider can contribute a security policy Added in review: the policy set was fixed at compile time, so a provider could contribute key custody but not a *profile*. It is now table-driven and open, and the blockers #4206 listed are gone. ```csharp using IDisposable registration = SecurityPolicies.Register( new SecurityPolicyInfo("urn:vendor:SecurityPolicy#CustomProfile", "CustomProfile") { PlatformSupport = () => true, SupportedCertificateTypes = [ObjectTypeIds.RsaSha256ApplicationCertificateType], IsDefault = true }); services.AddOpcUa() .AddCryptoProvider(crypto => crypto .For(CryptoPurpose.ChannelSymmetric, "urn:vendor:SecurityPolicy#CustomProfile").Use(module)) .AddSecurityPolicy(customPolicy); ``` - **The reflection is gone.** Four lookup tables were built by reflecting over the fields of `SecurityPolicies` and `SecurityPolicyInfo`. They are one explicit table now, which removes the **last reflection in `Security/Constants`** — so this is a trimming and Native AOT improvement as much as a feature. - **`SecurityPolicyInfo` is constructible from another assembly** and carries the metadata the helpers need, so `IsPlatformSupportedName`, `GetDefaultUris`, `MapSecurityPolicyToCertificateTypes` and `GetCurveFromCertificateTypeId` read that table instead of each being hand-written. Adding a policy is one entry rather than five edits. - **The dead `#if CURVE25519` conditional is deleted** from `Security/Constants`. - Registration is copy-on-write, so the read path stays lock-free. Shadowing a built-in policy requires `replaceExisting: true` and is reversible by disposing the registration. `RegisterLightsUpCurvePoliciesFromOutsideCore` is the acceptance test the issue asked for: it registers `ECC_curve25519` and `ECC_curve448` from the test project and asserts they become discoverable through the public API. **Registering a policy makes it advertised and resolvable; it does not supply the cryptography behind it.** For those two curve profiles the in-tree key agreement is still behind a compile symbol no project defines plus a BouncyCastle dependency, so a deployment lighting them up provides the operations through a provider. That division is the point: the policy set says *what* is offered, the provider says *who* performs it. ## Removing the gate's re-entrancy Folded in from #4242, which was merged into this branch. `ChannelGate` was re-entrant, and that forced the design behind most of the defects above. A holder that awaits releases its thread, so ownership could not be tracked by thread identity — it had to live in an `AsyncLocal<Holder?>`. A logical context is *inherited*, so anything started while the gate was held inherited the right to re-enter and ran inside the guarded region alongside whatever started it, unless it opted out by hand via `LeaveInheritedContext()` at each of eight sites, enforced by review alone. Five of the defects listed above trace to that mechanism, including both CI hangs. So the *requirement* was removed rather than the mechanism, and the `AsyncLocal` then deleted itself. Every path that re-entered now calls a lock-free `Core` method (`ForceChannelFaultCore`, `ShutdownCore`, `ForceReconnectCore`, `CompleteConnectCore`), and three regions stopped running foreign code under the lock: the `RequestReceived` dispatch is deferred until the gate is released, `OnTransportError` no longer wraps the `HandleSocketError` virtual, and the callback setters and `EndpointDescription` publish a single reference with a volatile write instead of locking — the last is load-bearing, because `AsyncResultBase.OperationCompleted` invokes the reverse-hello callback inline while the gate is held. `ChannelGate` is now a `SemaphoreSlim(1,1)` behind a `Releaser`; `Holder`, the thread-identity fallback, `LeaveInheritedContext`, `IsHeldByCurrentContext` and all eight disclaimer calls are gone, with the call shape at all 32 sites unchanged. `SaveIntermediateChunk`, `GetSavedChunks` and `DoMessageLimitsExceeded` gained a `gateHeld` argument, because the client reaches them both with and without the gate and must pick the matching teardown — without it an oversized `OpenSecureChannel` response deadlocked the receive loop against itself. The remaining paths were not found by reading the code, since that is what missed them originally. A temporary strict mode made every re-entrant acquisition throw with its stack, the suite was run under it until clean, and it was deleted with the re-entrancy it measured. It caught two sites review had missed: the `EndpointDescription` **property getter** taking the gate, and the `RequestReceived` dispatch to application code happening inside the guarded region. Two deliberate residuals are documented for reviewers: `Dispose` no longer takes the gate (unavoidable — it is reached from `ChannelFaulted` with the gate held, where the old re-entrant acquisition excluded nothing anyway), and the deferred request dispatch builds `SecureChannelContext` just after gate release rather than under it. ## Testing New: the three seams and their platform implementations, facet discovery and resolution precedence, `FipsOnly` refusing a provider that cannot serve what it was bound to, AES counter mode (including a non-block-aligned round trip and a check that the counter carries, since one that did not would silently reuse key stream), `ChannelGate` (exclusion, re-entrancy across an await, re-entrancy on a reused thread, and the deadlocks above), the asynchronous channel paths, and the PubSub policy provider. Each review finding got a test that fails without its fix: - `ContendedEntryStillRecordsTheHolderAsync` — established contention deterministically, then re-entered; failed on the pre-fix gate. Removed with the re-entrancy it covered; `ChannelGateTests` now asserts the non-re-entrant contract instead. - `SecureClientAndTcpListenerKeepChunkOrderUnderConcurrentRequestsAsync` — 60 concurrent multi-chunk `SignAndEncrypt` requests over a real loopback listener; failed 3 runs out of 3 with the previous dispatch, passes 3 out of 3 with the chain. - `DependencyInjectionRoutesPubSubCryptoThroughTheRegisteredProviderAsync` — builds a container, registers a counting provider, wraps a message and asserts the provider actually performed the encryption and the signature. This is the assertion the PubSub claim above now rests on. - `PolicyScopedRandomBindingDoesNotEscapeItsPolicy`. Two further assertions are the ones that would catch a silent defect: a provider and the platform must produce **byte-identical** output — a validated module that disagreed could not interoperate — and with a software key the returned `ValueTask` must report `IsCompletedSuccessfully`, which is what makes "ordering is unchanged" checkable rather than merely asserted. `SymmetricChannelCryptoBenchmarks` gains a provider-registered variant beside the existing baseline, so the cost of the indirection is measured rather than assumed. Verified locally, every suite at or above its recorded baseline: `dotnet build UA.slnx -c Release` **0 warnings, 0 errors on all six target frameworks**; Core **4413** (net10.0); Core.Security **617**, with its certificate-leak detector clean; PubSub **1350** plus PubSub.Diagnostics **54** and PubSub.Udp **233**; Features **156**; Subscriptions.Classic **32**; Server integration **4053**; Client integration **2123**; MigrationAnalyzer **151** and MigrationAnalyzer.Core **13**. Zero failures in each. Core moved from 4388 to 4381 when the gate change landed — eight re-entrancy tests were removed along with the behaviour they covered, and one deadlock regression test added — and has since grown to 4413 with the registry isolation, provider-output and security-review tests. `ChannelGateTests` was rewritten around the new contract — exclusion across both entry modes, an uncontended asynchronous entry not suspending, re-entry blocking by contract, and the property that used to be the hazard: work started while the gate is held is excluded without having to opt out. Every CI failure this PR produced was reproduced locally, diagnosed against a clean `origin/master` baseline in a second worktree, and fixed at the source. Two hangs were found from `--blame-hang-timeout` dumps read with `dotnet-dump`. One failure that looked like a candidate was confirmed **not** to be: an intermittent `Opc.Ua.Aot.Tests` connect failure, pre-existing on `origin/master` (5 of 8 baseline runs failed, versus 3 of 14 here). ### The publish request overshoot `Subscriptions.Classic.PublishRequestCountAsync` failed on the Windows net10.0 leg with **207** outstanding publish requests against a cap of `max(maxServerPublishRequest, subscriptions)` = 50. It failed in 2 of 6 pipeline builds on this branch, never on `master` in 13 sampled builds, and never locally in 12 attempts including under deliberate CPU contention. The mechanism is a genuine race. `ClassicSubscriptionEngine.QueueBeginPublish` was a **check-then-act**: it read `GoodPublishRequestCount`, compared it against `GetDesiredPublishRequestCount(false)`, and issued a publish if it was below. It runs on every publish completion, so *n* completions that overlap each read the same value, each conclude they are below the cap, and each issue — overshooting by up to *n* per round. The read also *lags*: a request only becomes visible to `GoodPublishRequestCount` once `AsyncRequestStarted` has recorded it, which happens after the request has already been sent. That code is pre-existing and was untouched by this PR. What this PR changed is how hard the race is hit: the server used to hold the channel gate across request processing, which serialised every request on a channel, so publish completions arrived one at a time. Dispatching `RequestReceived` outside the gate — an intended improvement, since a channel should not serialise on application request-processing time — lets them complete concurrently. Rather than change the publish pipeline blind, the race was first made reproducible. `ConcurrentPublishReEvaluationDoesNotExceedDesiredRequestCount` drives 64 barrier-released completions against a mocked context whose count lags the send exactly the way the session's does. It issued **64 requests where 5 were desired** — the same defect the CI leg hit, now deterministic and running in milliseconds. The engine now tracks the requests it has in flight and reserves a slot with a compare-exchange before sending, so the decision and the send are one atomic step. The reservation is released when the request completes, when the send is declined, or when it throws. The repro test then issues at most the desired count. `StartPublishing` is deliberately left **uncapped**. It is the recovery valve that refills a pipeline whose requests are outstanding but no longer expected to return, so gating it on the in-flight count would remove the only path out of a stalled pipeline. An earlier revision of this fix did gate it, and that is the one change here that could not be justified on the evidence — it was reverted. The assertion was **not** loosened, retried away, or marked flaky: it caught a real throttling regression earlier in this PR and is worth keeping sharp. Validated: the repro test red before the fix and green after; `Subscriptions.Classic` 32/32 **five** consecutive runs on net10.0 and once on net48; `Opc.Ua.Client.Tests` 2124 on net10.0 across four consecutive runs and 2127 on net48; `Opc.Ua.Subscriptions.Tests` 604 passed with this fix in place. ### The SSH.NET advisory that turned CI red repo-wide Partway through this work every build job began failing with `NU1903: Package 'SSH.NET' 2025.1.0 has a known high severity vulnerability` ([GHSA-q939-rpr3-3284](https://github.com/advisories/GHSA-q939-rpr3-3284), `<= 2025.1.0` affected, first patched `2026.0.0`). Nothing in the dependency graph changed — the advisory was published. Build 17026 failed on it where build 17000, two hours earlier on an identical graph, had passed every build job. SSH.NET is not referenced anywhere in the repository; it arrives transitively through `Testcontainers.Kafka`. `master` landed the remediation independently while this was being investigated (a central transitive pin to `2026.0.0`, which `CentralPackageTransitivePinningEnabled` exists for). This branch briefly carried its own identical pin, which made the merge commit hold **two** `PackageVersion` entries for SSH.NET and fail restore with `NU1506`. The duplicate is removed and `master` is merged, so the pin now comes from one place only. The audit was **not** suppressed: `NuGetAudit` stays enabled and `NU1903` stays an error. Verified after the merge — `dotnet restore UA.slnx` reports neither NU1903 nor NU1506, and `dotnet build UA.slnx -c Release` is **0 warnings, 0 errors** with the audit live. The remaining red check in build 17026 was not from this change either. `aot-ubuntu-latest` is the intermittent `Opc.Ua.Aot.Tests` failure already noted above — `ReconnectSessionAsync` timed out at 30s with 135 of 136 passing. It remains below the `origin/master` baseline rate (4 of 15 sampled here against 5 of 8 on the baseline), and the managed reconnect paths are green throughout: `Opc.Ua.Client.Tests` 2124 across four consecutive runs and `Opc.Ua.Subscriptions.Tests` 604. ## Documentation `docs/CryptoProvider.md` gains sections on substituting the symmetric primitives, using a key served over a network, and PubSub — including what cannot be claimed and why, the scope rule for a random-source binding, and how to supply a provider-backed policy outside the container. `docs/MigrationGuide.md` covers the `[Obsolete]` channel virtuals and `DataLock`, and the non-re-entrant gate's contract for channel subclasses. `docs/WhatsNewIn2.0.md` is updated. `plans/cryptooffboard.md` is deleted now that the work it planned has landed. ## Note for reviewers This touches the secure channel handshake, which is the most security-critical code in the stack. Nine distinct concurrency and lifetime defects were found and fixed in it during this work — three during development, two in review, four from CI — so `ChannelGate`, the write path and the certificate ownership across the new suspension points deserve the closest reading. Five of those nine traced to the gate's `AsyncLocal`, which is why the re-entrancy was subsequently removed and the `AsyncLocal` deleted. The wrapper/`Core` split and the two documented residuals in that section are the parts most worth a second reading. The #4206 work removes the last reflection from `Security/Constants` and rewrites the policy lookup tables, so `SecurityConfiguration.SupportedSecurityPolicies` and the endpoint set it feeds are worth a second look: an earlier revision of it changed that set and broke six server tests, which is now covered by `SecurityConfigurationBuildsExpectedSupportedPolicySet`. ## Related Issues - Fixes #4206 - Fixes #4207 - Fixes #4208 - Fixes #4210 - Follow-up #4250 — this PR makes the security policy set injectable, but nothing in the stack resolves `ISecurityPolicyRegistry` from DI yet, so every production path still lands on the `Default` fallback. #4250 tracks threading it through `Session` and `UaSCBinaryChannel`, which is what makes the injection points already added here load-bearing. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com> Copilot-Session: 7a68ca72-83e8-496c-b2f5-3df4c718320b Copilot-Session: 3f5ab76a-c0ba-4f47-87ac-54d1f4182c6b	2026-08-15 05:23:20 -07:00
Marc SchierandCopilot	e05a820c91	Removal of the exposed diagnostics and node locks (#4188) ... ## What this is Design analysis for `Opc.Ua.Server`, the two rounds of design exploration behind it, and the first two implementation chunks it recommends. Draft: the analyses are ready to read, but the *conclusions* deserve discussion before the larger phases proceed. Later chunks stack onto this PR as they land. | File | Kind | Lines | |---|---|---:| | `plans/NodeManagerAnalysis.md` | Evidence — the node-manager plugin surface | 488 | | `plans/ServerRuntimeAnalysis.md` | Evidence — the server runtime cluster | 576 | | `plans/ServerDesignExploration.md` | Design round 2 — delivery strategy | 1,394 | | `plans/ServerDesignExplorationAlt.md` | Design round 1 — four interface designs | 2,119 | | `src/Opc.Ua.Types/State/NodeState.cs` | **Code** — node self-guards attributes | | | `src/Opc.Ua.Types/State/BaseVariableState.cs` | **Code** — value accessors take the lock | | | `src/Opc.Ua.Server/NodeManager/AsyncCustomNodeManager.cs` | **Code** — 6 external locks removed | | | `tests/…/NodeStateAttributeConcurrencyTests.cs` | **Code** — concurrency safety net | 449 | `docs/` is untouched — it holds shipped feature documentation, and none of this is shipped API. ## The analyses Counted mechanically over `src`, `samples` and `tests`; **every figure is reproducible** from commands printed in each document. **Node manager surface.** `AsyncCustomNodeManager` declares **93 `virtual`/`abstract` members** (80 distinct names). Across the repository: **112 override sites, 25 distinct names ever overridden, 55 of 80 never**. Six carry nearly all real use. For scale, `ReferenceNodeManager` is 6,373 lines and overrides **8**. **Server runtime.** Seven findings, **two of which recommend no change**. Findings 1–5 are filed as #4183–#4187. The headline is an `object`-typed sync root published on three public interfaces and taken in **88 `lock` statements across 7 files**, including a shipped sample. **What is deliberately not flagged.** `IStandardServer` — 9 members over 5,575 lines, 16 of 37 virtuals used — is identified as the reference shape to preserve. `IApplicationInstance` and the already-`[Obsolete]` `Stop()` are examined and left alone. ## The design exploration **Round 1** explored four alternative interfaces in parallel under the current back-compat constraint. Its value is the convergence: four designers who never saw each other's work independently arrived at a 6–9 member `IServerContext` replacing the 57-member locator, removal of the `object` node-manager handle, elimination of the `Processed` protocol, and `IHistorianProvider` as the pattern to copy. **Round 2** re-runs it with back-compat lifted and asks how to deliver: full rewrite versus side-by-side fork. **Both option designers independently declined to recommend their own assignment.** Three structural facts were verified: 1. **The fork line already sits at `IAsyncNodeManager`** — old and new node managers can coexist with zero runtime changes, so a fork duplicates ~10% of the assembly, not all of it. 2. **`[OpcUaShim]` cannot shim an inheritance surface** — honest auto-fix ceiling for the authoring surface is **~10%**, against ~70% for the shipped analyzers. 3. **A second lock leak exists** — `NodeState` takes `lock (this)` under a `CA2002` suppression whose own comment documents the contract. Recommendation: **neither pure option**, but a staged path reaching the rewrite end state incrementally, gated on migrating the five framework-internal node managers that today make it impossible for the old base to ever freeze. ## Implementation: NodeState now guards its own attributes This removes the prose-only contract in which `NodeState` took `lock(this)` and every external caller was required to take `lock(source)` on the node instance. The code already carried a TODO describing exactly this fix and noting it required updating every external site. - The node self-guards rather than relying on callers. `BaseVariableState` gains a **fully private** `System.Threading.Lock` covering its value fields; `NodeState`'s other collections keep their existing private locks. - All **6** `lock(this)` sites and both `CA2002`/`RCS1059` suppressions are gone. - `Value`, `Timestamp` and `StatusCode` on `BaseVariableState` take the lock too — without that, a direct property write raced an attribute read once the external lock was gone. - **6 of 9** external sites removed. Five were already redundant (`GetReferences`, `GetChildren`, `ReferenceExists` and `SetAreEventsMonitored` guard themselves; `FindMethod` reaches its state via `GetChildren`); the sixth was an attribute read now covered by the node's own lock. - Attribute read/write deliberately take **no** node-wide lock: both dispatch to caller-supplied handlers (`OnReadValue` and siblings) that may take locks of their own, and holding a lock on the node across one gives the node no fixed place in the lock order. - **3 deliberately retained and annotated in code**: two `CreateBrowser` calls, which are not internally synchronized (tracked in #4216), and one in `CustomNodeManager`, whose class is slated for deletion so refactoring its lock would be wasted work. ### No lock is exposed anywhere, to anyone An earlier revision put the attribute lock on `NodeState` as `private protected`, which still published a lock-sharing contract on the inheritance surface. Review pushed back and it is gone: `BaseVariableState` was the only user, so the lock moved there as a plain `private readonly Lock`. There is no member for a derived type to call, in this assembly or any other. `BaseVariableValue` was the last public leak — it handed out the lock guarding its value through a `public Lock` property, so any caller could hold it across a call back into the stack. That property is removed: - Derived value classes (what the source generator emits for every structure variable) synchronize through a `protected EnterLock()` / `ExitLock()` pair. Callback helpers were the first choice, but the generated bodies pass `ref Variant` / `ref StatusCode` / `ref DateTimeUtc`, and `ref` arguments cannot be captured by a lambda. The lock *object* stays private either way. - `NodeStateTemplates` emits the new form. - `ServerInternalData`, the one cross-assembly caller, was reading the lock back off a value it had itself constructed with that lock. It now takes `m_diagnosticsLock` directly. `NodeBrowser.DataLock` remains `protected` and is tracked in #4215: a subclass extending `Next()` legitimately needs its enumeration state atomic with the base browser's, so the fix needs a base-owned acquisition rather than a deletion. ### Migration coverage for the removed lock members None of the removed lock members get an `[Obsolete]` shim, and the migration guide now says why: a shim would have to be `object`-typed to keep old call sites compiling, and on .NET 9+ `lock (o)` on a statically-`object` reference is `Monitor.Enter`, which does **not** interlock with `Lock.Enter()`. It would compile, run, and silently synchronize nothing — strictly worse than a compile error. Re-adding a member to a downstream-implemented interface is also a break in its own right on `net48` / `netstandard2.0`, where there are no default interface methods. Instead, every removed member has a named replacement on the migration path: | Rule | Covers | Replacement it names | |---|---|---| | `UA0024` | `IServerInternal` / `ISession` / `ISubscription` `DiagnosticsLock`, `DiagnosticsWriteLock` | `UpdateServerDiagnostics(...)` / `UpdateDiagnostics(...)`, `ReadServerDiagnostics` / `ReadDiagnostics` | | `UA0025` | `ILocalNode.DataLock`, `Node.DataLock` | a lock the caller owns; the node guards its own state | | `UA0026` | `BaseVariableValue.Lock` | a lock passed to the `BaseVariableValue` constructor | Each rule fires on sources still written against 1.5.378, where the compiler only says a member is missing. `docs/MigrationGuide.md` gains a section per removal with before/after. ### The tests earned their keep The four concurrency tests were landed **first**, against the unmodified code, because removing a synchronization contract cannot be verified by the compiler and the existing `NodeStateHandlerConcurrencyTests` covers a different concern (racing a handler assignment against a write, not reader/writer exclusion). They now run with **no external locking**, which is what demonstrates the node guards itself. They caught two real problems, and one of them was mine: - The `Value`/`StatusCode`/`Timestamp` setters bypassed the new lock — a genuine race. - One test asserted an invariant the design does not offer. Two property setters are two lock acquisitions, so that pair was **never** atomic; it only passed originally because the test held a lock across both. It now uses a single attribute write carrying a whole `DataValue`, which genuinely is atomic and is the path the server uses. A third issue was found and fixed before it shipped: the reader loops gated on `!writer.IsCompleted`, so a reader scheduled after the writer finished performed zero reads — failing two runs in three on `net48`. Converted to `do…while`; now five consecutive green runs. **Verified:** `Stack.State` 116 passed on `net10.0` *and* `net48`; the concurrency category green 5/5 consecutive on `net48`; `Opc.Ua.Types.Tests` 8,456 passed; the server node-manager suite 1,205 passed, including `NodeManagerLifecycleTests` and `AsyncCustomNodeManagerTests` which exercise the changed paths. ## Two corrections to my own published figures Both were found while starting implementation, and both are recorded on #4183. **`ForceDiagnosticsScan()` was overstated.** The analysis claimed the `DiagnosticsWriteLock` getter *"triggers a full address-space diagnostics rescan"*. It is in fact a single flag assignment (`DiagnosticsNodeManager.cs:700-703`) consumed by the read paths and the periodic timer. The defect stands — a side-effecting property getter with a racy write on a synchronisation path — but its cost was misstated. **The contract was 15 sites, not 19.** A naive search matched ten comment and `#pragma` lines that *mention* `lock(source)` while explaining it. The documents now carry a counting command that excludes comment matches. A third finding expanded scope the other way: the `object`-typed lock leak is **repo-wide**, with ten such members across `Opc.Ua.Core`, `Opc.Ua.Types` and `Opc.Ua.Server` — including `Node.DataLock => this` and `ApplicationConfiguration.PropertiesLock => m_properties`, the same defect class as `Subscription.DiagnosticsLock`. ## Caveats stated in the documents - **Nothing in the designs is implemented or benchmarked.** Every performance claim across all seven designs is a design intention, not a measurement. - The larger phases are a **major undertaking against a live plugin API**; the documents say so rather than letting the neatness of the proposals imply otherwise. - The recommended sequencing puts low-risk, self-contained steps first and notes they are **worth doing regardless** of which strategy is eventually chosen. - Round 2 notes the two option designers' cost estimates are **not independent of their assignments**. ## Review guidance The two analyses make factual claims checkable by running the included commands. The design explorations are exploratory and need not be agreed with to merge. The code change is small, test-backed, and independent of which delivery strategy is eventually chosen. Reasonable outcomes include merging as-is, merging only the analyses and the lock fix, or disagreeing with the recommendation while keeping the evidence. --- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e9c13dd0-a6e3-4ad2-8ad6-1c3fe6323041 Copilot-Session: d18f87e9-2484-44ce-98ce-463ba6cf5f63	2026-08-10 06:47:50 +02:00
Marc Schier	0612a43fe4	Pluggable cryptography and hardware-held private keys (#4192) ... Makes the stack's cryptographic operations replaceable - by another library, an offboard service, or hardware (TPM 2.0 / HSM / PKCS#11 token / cloud KMS) - with no performance cost in the default configuration, and extends this to certificates so a private key need never exist in process memory. Also fixes three pre-existing defects found while validating: STACKGEN001 erroring for every compilation the stack generator was merely loaded into, certificates failing to load from paths past MAX_PATH on Windows, and the MonitoredNode2 tests starving the thread pool. Fixes #4190	2026-08-09 12:40:26 +02:00
3 people	537b10c8b3	Fix History User Access Levels and add CTT coverage nodes (#4026) ... # Description Improve Reference Server coverage for OPC UA Compliance Test Tool scenarios and document the latest CTT findings. ## Reference Server coverage - Historize the configured AccessRights nodes and seed them with the same deterministic mixed-quality history as the other test nodes while preserving their access controls. - Add a historian node that intentionally does not support ServerTimestamp. - Add a SelectionListType String instance whose Selections property is String[] as required by Part 5, with localized descriptions and RestrictToList enforcement. - Add an EUR CurrencyUnit property and verify its structured value and binary encoding. - Add a node with explicit read/write base access bits plus NonatomicRead and NonatomicWrite AccessLevelEx flags. - Ensure the first ByteString array elements meet the CTT minimum-length prerequisite. ## Latest CTT findings - Confirm run-23 A&C residuals are CTT re-entrant Read and branch-unaware Retain checks rather than server regressions. - Confirm NumberOfTransitions includes Uncertain values as required by Part 13. - Document the CTT CurrencyUnit decoder dropping a correctly encoded Exponent. ## Related Issues - Follow-up to #3960 and #4002. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. Focused affected tests pass on .NET 10 and .NET Framework 4.8. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@hotmail.com>	2026-07-21 13:52:12 +02:00
4 people	84fa8e8906	CTT follow-up: monitored items, alarms, codecs, history, and model validation (#4002) ... # Description Follow-up to #3978 based on the latest OPC UA CTT aggregate, matrix, Historical Access, Attribute, and Node Management runs. ## Aggregate run 16 - Ignore synthetic `BadBoundNotFound` raw-bound markers when feeding aggregate calculators. - Add direct and live historian regression coverage. ## Multi-dimensional Variant matrices - Fix `Variant.Copy()` so rank-2 Variant matrices preserve their payload and dimensions. - Enforce valid Variant matrix dimensions consistently across Binary, JSON, XML stream, and XML parser codecs. - Encoders return `BadEncodingError`; decoders return `BadDecodingError` for malformed matrix dimensions. - Add cross-codec valid/null/empty/malformed tests and a batched ReferenceServer Read regression. - Correct the previous CTT issue attribution: the invalid `Scalar_Static_Arrays2D_Variant` server encoding poisoned the batched CTT Read. ## Node Management AddNodes - Return `BadNodeIdRejected` for unsupported requested-NodeId namespaces. - Validate hierarchical ReferenceType source/target NodeClass constraints. - Document run-18 confirmation that all remaining 15 AddNodes errors are CTT project/script configuration issues. ## Historical Access and Attribute Services run 18 - Reject whitespace in `NumericRange` syntax with `BadIndexRangeInvalid`. - Add HistoryRead tests for operation-level invalid syntax and per-DataValue `BadIndexRangeNoData`. - Document CTT defects: - `012.js` expects `BadIndexRangeNoData` at the operation level instead of on returned DataValues; - `Err-012.js` uses a non-historizing node for an access-denied test; - generic Attribute array helpers omit NodeId[] and StatusCode[] conversion paths. ## Base Information model validation run 19 - Run the CTT main session as `sysadmin` over Basic256Sha256 SignAndEncrypt so SecurityAdmin RolePermissions and EncryptionRequired restrictions are both satisfied without weakening the server model. - Preserve predefined well-known NodeIds while making generated dynamic type and optional-child factories allocate compatible per-instance NodeIds, remap internal references, and update references from the owning subtree. - Materialize dynamic RoleType properties through generated helpers and keep role-child string NodeIds disjoint from numeric RoleManager root IDs. - Add net48/net10 regressions for generator output, NodeId allocation/reference remapping, DI mandatory children, dynamic roles, and low-privilege versus SecurityAdmin Browse visibility. - Document CTT defects for permission-unaware mandatory-child validation, the UA 1.04 reference model used against a UA 1.05 server, stale ServerDiagnostics cache, and the `ConformanceUnits` scalar/array mismatch. ## Related Issues - Follow-up to #3960 and #3978. ## Checklist - [x] I have signed the CLA and read the CONTRIBUTING document. - [x] Tests prove the fixes and improve coverage. - [x] Required CTT documentation is updated. - [x] Focused builds/tests introduce no warnings. - [x] Affected tests pass on .NET Framework and .NET 10. - [ ] CI and CodeQL are green. - [ ] All PR feedback is addressed. ## Latest CTT full run and A&C run 20 - Fix Monitor Value Change V2 `036.js`: an ambiguous `DataValue(StatusCode)` call encoded `BadIndexRangeNoData` as a Good value. - Prove all 19 configured matrix monitored items notify when the selected value actually changes; document the independent `042.js` script defects. - Rebase dynamic `CertificateExpired` / `TrustListOutOfDate` alarm roots and descendants, restore standard type declarations and ModellingRules, and expose the cross-node-manager references. - Deliver method-triggered events to every authorized Session, fixing the two-Session Confirm scenario. - Preserve locale-only `LocalizedText` values consistently across Binary, JSON, and XML and align Acknowledge/Confirm comment behavior and tests. - Document the remaining proven CTT defects for TransactionDiagnostics, SemanticChange, History `013`, PolicyId scope, A&C, durable subscriptions, and Subscription Minimum events. ## Base synchronization Merged `master` through `007cfaf90` (`#4010`, repository layout consolidation) and retained the matrix regression tests at their new `tests/` locations. ## Latest validation - Focused merged-tree tests pass on net10 and net48. - `Opc.Ua.Server` and `Quickstarts.Servers` Release builds complete with 0 warnings and 0 errors on net10 and net48. - The existing `DisableEnableViaTypeAndInstanceMethodNoEventsDuringDisabledAsync` timing test took its documented inconclusive timeout path after the layout merge; no test failure was reported. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: romanett <romanett98@gmail.com>	2026-07-18 05:01:03 +02:00
Marc Schier	8538f531e7	Update paths for the consolidated repository layout	2026-07-16 06:26:05 +02:00
3 people	0da85b9d89	CTT conformance: access rights, X509, aggregates, and matrix diagnostics (#3978) ... # Description Investigates successive OPC UA CTT runs and fixes the proven server conformance defects while documenting proven CTT script/codec defects in `plans/ctt-issues.md`. ## Server fixes - Grant `WriteAttribute` on `Scalar_Static_Int32` so attributes advertised by `WriteMask` are writable. - Disable the client-certificate-keyed authentication lockout in the dedicated CTT configuration so negative token tests do not mask later results with `BadUserAccessDenied`. - Map an X509 user-token signature algorithm mismatch to token-level `BadIdentityTokenInvalid`. - Persist rejected X509 user certificates to `pki/rejectedUser` to simplify trusted-user provisioning. - Populate `HistoricalDataConfiguration.AggregateConfiguration` with the server's actual defaults. - Return per-node `BadInvalidArgument` for processed reads with equal StartTime/EndTime. - Feed reverse processed reads in request order and preserve exact non-Bad interpolation boundaries. - Correct PercentGood/PercentBad uncertain classification. - Correct WorstQuality/WorstQuality2 eligible bounds, StatusCode preservation, and `MultipleValues`. - Correct reverse Min/Max Raw-vs-Calculated semantics. - Fix `HistoryClient.GetConfigurationAsync` traversal and aggregate-configuration reads. ## Tests and CTT evidence - Added direct calculator and live historian Part 11/13 oracle tests covering forward/reverse intervals, equal-time validation, bounds, status aggregates, DurationInState, and Min/Max. - Added exact Part 6 Table 26 binary golden-vector coverage for a 2×3 Int32 Matrix Variant. - Added in-process whole-matrix read/write and multidimensional `NumericRange` coverage. - Updated `plans/ctt-issues.md` with exact evidence and recommended fixes for CTT aggregate, X509, multidimensional-array, A&C, and AliasName script defects. No CTT failure is suppressed or accepted without a spec-based classification. ## Related Issues - Relates to #3960. ## Checklist - [x] Tests proving the server fixes are included. - [x] CTT-side defects are documented with script locations and specification references. - [x] Targeted aggregate/routing/matrix suites pass on net48 and net10. - [x] History aggregate/client suites pass on net48 and net10. - [x] Modified lines are analyzer-clean. - [ ] CI and CodeQL complete. - [ ] Maintainer review complete. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Roman Ettlinger <romanett98@gmail.com>	2026-07-13 19:14:19 +02:00
Marc Schier	d7cb170b86	CTT compliance (#3960) + HA/Aggregate facets (#3875) + server-side pcap capture (#3962) ... ## Summary This PR bundles two independent bodies of work on the branch: 1. **CTT compliance & Historical Access** — improves OPC UA Compliance Test Tool (CTT) conformance of the Console Reference Server (issue #3960) and advertises the correct Historical Raw Data + Aggregate server facets (issue #3875). 2. **Server-side in-process packet capture (pcap)** — lets a hosted server capture its inbound client→server traffic in-process, with a documented non-DI wiring path. The CTT work was driven by an actual CTT run (`NewCTT2`, script v1.05.513) whose 4757 error results were parsed and clustered by conformance unit. Fixes are verified live via MCP against the running `--ctt` server and covered by unit/integration tests that are green on **net48 + net10**. ## Server-side in-process packet capture (pcap) Adds the ability to capture a hosted server's **inbound** (client→server) `opc.tcp` traffic in-process, symmetric to the existing client-channel capture in `Opc.Ua.Core.Diagnostics`. Previously `PcapBindings.Install(server.Server.TransportBindings)` only installed a client *channel* decorator — a server accepts connections through *listeners*, which never consult channel factories — so nothing was captured, and `InProcessServerCaptureSource` was a placeholder. Installing a binding also records nothing on its own until a `CaptureSessionManager` session is started (which is now documented). - **Core seam** (`Opc.Ua.Core`, opt-in, zero cost when unused): `TransportListenerSettings.AcceptedTransportDecorator` + `OnAcceptedChannel`, applied in `TcpListenerChannel.Attach(socket)` and the reverse-connect path; `TcpTransportListener` wires them per accepted channel. Default `null` → no behavior change for non-capturing servers. - **`PcapTransportListenerBinding`** — an `ITransportListenerFactory` **listener decorator** (the server-side mirror of the client `PcapTransportChannelBinding`) that injects the capture seam via `OpenAsync`, wraps each accepted channel's byte transport in `CapturingByteTransport`, and forwards channel-token key material to the active observer. - **`PcapBindings`** — `Install` now wires **both** client and server; new `InstallClient` / `InstallServer` helpers. `AddPcap` covers both sides too. `InProcessServerCaptureSource` is now functional. - **Non-DI usability** — new public `ClientChannelManager.DefaultChannelBindings` accessor so non-DI clients can install the binding into the registry the client actually uses, plus a new "Enable pcap capture without dependency injection" guide in `Docs/Diagnostics.md` (server / client / both recipes, install-before-start timing, the two-registries nuance) and `NugetREADME` updates. - **Tests** — Core settings-seam unit tests (net48 + net10), `PcapTransportListenerBinding` / `PcapBindings` unit tests, and a real server↔client integration test asserting a non-empty pcap of server-accepted frames. ## Fixes (CTT compliance / Historical Access) ### Historical Access — Read Raw (~1996 CTT errors) The new historian returned `Good_MoreData` when handing back a `ContinuationPoint`. Per OPC UA Part 11 §6.5.3.2 (and the CTT HA Read Raw tests 008/009, which assert `Good` **and** a `ContinuationPoint` together) the operation status must be **`Good`** — the non-empty ContinuationPoint alone signals more data. Fixed all four continuation sites (raw / modified / annotation / processed-event) in `HistorianDispatcher`, and additionally report **`Good_NoData`** for a raw/modified read of an interval that contains no data and requests no bounds (CTT HA Read Raw 007). Adds regression tests. ### Historical Access — HA Configuration (~282 CTT errors) Historizing reference-server nodes had no `HasHistoricalConfiguration` reference, so History Access aggregate clients (and the CTT) could not discover the archive/aggregate configuration. The reference server now attaches a populated `HistoricalDataConfigurationType` companion object ("HA Configuration") to every historizing node. This also fixes a latent bug in `HistoricalDataConfigurationInstaller`: a freshly created configuration instance (and its mandatory `Stepped` / `AggregateConfiguration` children) retained the type's shared NodeIds, so the reference targeted the type node and multiple nodes collided — the installer now assigns instance NodeIds to the whole subtree via the node manager's `INodeIdFactory`. Adds a regression test. ### Historical Access — Read Raw status codes Additional CTT HA Read Raw conformance fixes: - Insufficient details (<2 of startTime/endTime/numValuesPerNode) → `Bad_HistoryOperationInvalid` (Err-027). - IndexRange on scalar history → `Bad_IndexRangeNoData` at the operation level (Err-009). - Unknown/foreign ContinuationPoint (e.g. a Browse CP) → `Bad_ContinuationPointInvalid` (Err-014/Err-024). - No spurious ContinuationPoint on the final page of a one-value-at-a-time read (008/009). - `TimestampsToReturn.Neither` → `Bad_TimestampsToReturnInvalid` (Err-002). All validated against the actual CTT run (`NewCTT2` Run2, an HA-only re-run built with the base fixes) which confirms the 1999 "Good_MoreData vs Good" failures are gone. Covered by tests. ### Earlier commits in this PR Variant/BaseDataType array write `BadTypeMismatch`, DiagnosticInfos gating on `DiagnosticsMask`, multi-dimensional array values; strengthened Attribute Write conformance tests; Condition comment retention on null Acknowledge/Confirm; historized the full scalar type set + paired aggregate nodes; populated `ServerCapabilities.ConformanceUnits`; advertised `HistoricalRawData2022` + `AggregateHistorical2022` facets (#3875) with `Docs/Profiles.md` updates. ## Diagnosed (not server bugs — CTT connection configuration) ### Auditing Connections (~1684 CTT errors) All 1684 failures are identical ("Unable to Find Entry for ClientAuditEntryId"). Audit events are, per spec, delivered only to sessions holding the `SecurityAdmin` role: the server validates `PermissionType.ReceiveEvents` per event-monitored-item, and the integration test `AuditingOperationTests` connects as `sysadmin`/`demo` for exactly this reason. The CTT run collected zero audit events because it connected **anonymously**. Documented in `Docs/Diagnostics.md`: configure the CTT to authenticate as `sysadmin`/`demo` (mapped to `[SecurityAdmin, AuthenticatedUser]` in `ReferenceServer.cs`). ### Address Space WriteMask (~126 CTT errors) `revertToOriginalNodeAttributes` writes back all node attributes and gets `BadUserAccessDenied` (a permission error) rather than Good/BadNotWritable/BadAttributeIdInvalid — the same anonymous-connection cause; expected to clear when the CTT connects as `sysadmin`/`demo`. ## Still open Alarms & Conditions Retain/ConditionDisabled/InputNode (logic is spec-aligned on inspection; needs a CTT-faithful reproduction to separate a genuine bug from alarm-simulation timing), Aggregate function values (now unblocked by the HA Configuration fix), GDS self-registration, residual Attribute IndexRange/type-mismatch. CTT-script errors (`ArrayItems` / `ItemsHistoryCountExceeds` undefined) are not server-fixable. ## Testing `Opc.Ua.Server.Tests` (Historian) and `Opc.Ua.History.Tests` green on net48 + net10; historian and HA-configuration behaviour verified live against the running `--ctt` server. Server-side pcap capture: Core builds clean on net48 + net10, and the new unit + server↔client integration tests pass (the only failing `Opc.Ua.Core.Diagnostics.Tests` are 3 pre-existing, unrelated `OfflineSecureChannelSmokeTests`). The single most decisive remaining CTT step is a re-run authenticated as `sysadmin`/`demo`, which clears the permission-based clusters and isolates the true remaining server bugs.	2026-07-09 09:08:27 +02:00
Marc Schier	76bceb51e2	Env-var driven Pcap auto-capture + rename to Opc.Ua.Core.Diagnostics + Docs/Diagnostics.md (#3888) ... # Description This PR also renames the project itself from `Opc.Ua.Bindings.Pcap` (assembly / NuGet / folder / root namespace) to `Opc.Ua.Core.Diagnostics`. `Pcap`-prefixed type names (`PcapFileWriter`, `PcapOptions`, `IPcapAuditSink`, ...) and the `AddPcap*` extension methods are preserved &mdash; only the project shell broadens to `Core.Diagnostics`. The new NuGet id is `OPCFoundation.NetStandard.Opc.Ua.Core.Diagnostics`. Folders were moved with `git mv` so blame and log are preserved. This PR also renames the **C# namespace** from `Opc.Ua.Core.Diagnostics.*` to `Opc.Ua.Pcap.*` (commit `efc03691c`). The **assembly identity** stays `Opc.Ua.Core.Diagnostics` (assembly name, NuGet id `OPCFoundation.NetStandard.Opc.Ua.Core.Diagnostics`, folder, csproj filename, `InternalsVisibleTo`, .slnx project paths, sign-lists are all unchanged). Result: one `Opc.Ua.Core.Diagnostics.dll` assembly hosting types in the `Opc.Ua.Pcap.*` namespace — standard .NET pattern (cf. `System.Net.Http.dll` hosting `System.Web.*`). Future diagnostic features can add other `Opc.Ua.*` namespaces to the same assembly without another rename. 149 files changed (+411/-411 net); 422/422 tests pass on net10.0. This PR also merges `Docs/PacketCapture.md` and `Docs/Observability.md` into a new `Docs/Diagnostics.md`. `ITelemetryContext` leads the doc with expanded usage and extensibility guidance (custom contexts, OpenTelemetry exporter wiring, ActivitySource / Meter naming conventions, testing patterns). New sections cover OPC UA server audit events and the built-in `Server.ServerDiagnostics` info-model nodes. The 1.4 / 1.5 evolution narrative has been dropped; the still-relevant migration content has moved into `Docs/MigrationGuide.md`. All 8 inbound doc / slnx / XML-doc references are retargeted. This PR also adds a new public DI extension `AddPcapFromEnvironment` to `OPCFoundation.NetStandard.Opc.Ua.Core.Diagnostics` that activates packet capture and/or symmetric-key logging at host start based on two environment variables. The variables are evaluated **once** during the `AddXXX` call (matching the existing `OPCUA_PCAP_ENABLE_DIAGNOSTICS` opt-in semantics): | Variable | Effect when set | |---|---| | `OPCUA_PCAP_FILE` | Auto-start an in-process capture session on host start. Frames are written verbatim to the literal path; the parent directory becomes `PcapOptions.BaseFolder` so the existing `SessionFolder` path-traversal validation still runs. | | `OPCUA_KEYLOGFILE` | When set without `OPCUA_PCAP_FILE`: install a stand-alone keylog observer (SSLKEYLOGFILE-style) directly into `IChannelCaptureRegistry`. When set together with `OPCUA_PCAP_FILE`: keylog records go to this path instead of the conventional `*.uakeys.json` sibling of the pcap. | When neither variable is set the call behaves exactly like `AddPcap` &mdash; binding registered, no auto-start. When the auto-start path is active the host emits a `Warning`-level log line naming the env var(s) consumed and the resolved path(s); key material is never logged. ## Public surface The new public API is intentionally narrow: ```csharp namespace Opc.Ua.Core.Diagnostics.DependencyInjection { public static class PcapEnvironmentVariableNames { public const string OpcuaPcapFile = "OPCUA_PCAP_FILE"; public const string OpcuaKeyLogFile = "OPCUA_KEYLOGFILE"; } public static partial class PcapServiceCollectionExtensions { public static IServiceCollection AddPcapFromEnvironment(this IServiceCollection services); public static IServiceCollection AddPcapFromEnvironment(this IServiceCollection services, Action<PcapOptions> configure); } } ``` `StandaloneKeyLogObserver`, `PcapEnvironmentDefaults`, and `PcapEnvironmentAutoStartHostedService` are internal &mdash; the env-var names and DI methods are the only public surface, which keeps it AOT-clean. ## Behaviour-impacting code change `InProcessCaptureSourceBase` now honors `StartCaptureRequest.PcapFilePath` and `StartCaptureRequest.KeyLogFilePath` for in-process capture (previously these properties were respected only by the Replay source, despite their generic XML docs). Behaviour is **opt-in**: callers that pass `null` continue to get the conventional `capture.pcap` / `keys.uakeys.json` / `keys.uakeys.txt` filenames inside `SessionFolder`. Path-traversal validation in `CaptureSessionManager.StartAsync` is unchanged. ## Operator example ```yaml # Kubernetes manifest snippet env: - name: OPCUA_PCAP_FILE value: /var/log/opcua/cap.pcap - name: OPCUA_KEYLOGFILE value: /var/log/opcua/keys.uakeys.json ``` ```csharp // Program.cs HostApplicationBuilder builder = Host.CreateApplicationBuilder(); builder.Services.AddOpcUa().AddClient(_ => { }); builder.Services.AddPcapFromEnvironment(); await builder.Build().RunAsync(); ``` For SSLKEYLOGFILE-style keylog only (no pcap): ```bash export OPCUA_KEYLOGFILE=$HOME/opcua-keys.uakeys.json ``` ## Security model Anyone (process, container runtime, host operator, sidecar) who can set these env vars can divert OPC UA channel keys and traffic to attacker-controlled paths. `Docs/PacketCapture.md` gains a security note and an operator-checklist item that ties this to the existing security guidance: the destination path / directory must be operator-controlled and protected with file-system permissions in addition to the binding's own `0600`-on-Unix file mode.	2026-06-20 21:23:08 +02:00
Marc Schier	f358078b1c	[Client] Central channel manager: ref-counted shared channels, coalesced reconnect, IRetryBudget (#3288) (#3852) ... # Description Introduces a central `IClientChannelManager` that owns client-side transport channels with reference counting, sharing, coalesced reconnect, and asynchronous participant notification. Sessions, discovery clients, and registration clients targeting the same `ConfiguredEndpoint` (with the same reverse-connect identity) now share a single underlying `ITransportChannel`; reconnect is transparent to callers; and the previous `AttachChannel`/`DetachChannel` + `SessionReconnectHandler` patterns are obsoleted. ## What lands ### Core types (`Stack/Opc.Ua.Core/Stack/Client/`) - `ManagedChannelKey`, `ChannelState`, `ChannelStateChange`, `ParticipantReconnectResult` - `IReconnectParticipant`, `IManagedTransportChannel`, `IClientChannelManager` - `IChannelReconnectPolicy` + `ExponentialBackoffChannelReconnectPolicy` (default mirrors historical `SessionReconnectHandler`: 500 ms → 30 s exponential) - `IRetryBudget` + `RetryBudget` — shared retry deadline so the two-level retry no longer compounds multiplicatively - `ClientChannelManager` — single non-partial sealed class with SOLID-extracted internal types under `Stack/Opc.Ua.Core/Stack/Client/Channels/Internal/`: `ChannelEntry` (behind `IChannelEntryHost`), `ManagedTransportChannelLease`, `ClientChannel`, `ClientChannelManagerMetrics` (IMeter), `ClientChannelManagerCertRotation` (auto `ReconnectAllAsync` on `CertificateManager` rotation), `ClientChannelManagerDiagnostics` (ActivitySource + EventSource). Refcount, coalesced reconnect, and transparent faulted-entry recovery via `SwapFaultedEntryAsync` (swaps a lease's underlying entry on subsequent `ReconnectAsync` when the prior cycle Faulted/Closed, preserves participant, back-off via policy `GetDelay(SwapCount)`) ### Three-state lifecycle gate `Disconnected` → `TransportConnecting` / `TransportReconnecting` → `TransportConnectedSessionReactivating` → `Ready` (or `Faulted` / `Closed`). Only `Ready` releases the service-call gate; the manager bypasses the gate internally for participant `ActivateSession` traffic during reactivation via an `AsyncLocal` scope. ### Session integration (`Libraries/Opc.Ua.Client/Session/`) - New `Session.CreateAsync(IClientChannelManager, …)` overload - `Session` implements `IReconnectParticipant`; reactivation handler distinguishes `Reactivated` / `RequiresSessionRecreate` / `TransientFailure` / `FatalForParticipant` / `FatalForChannel` - `Session.ReconnectAsync(ct)` automatically delegates to the channel manager when wired - `Session.RecreateInPlaceAsync` swaps managed-channel leases for failover-to-different-endpoint via a new atomic `IClientChannelManager.GetAsync(endpoint, participantFactory, …)` overload; same-key recreates delegate to manager reconnect; explicit-channel/connection callers retain legacy behavior - KA failure routes through `IClientChannelManager.ReconnectAsync(channel)` ### ManagedSession integration - `WithChannelManager(...)` builder + ctor param; DI-resolved automatically - Two-level retry: channel-mgr handles transparent reconnect; outer `ConnectionStateMachine` + `IReconnectPolicy` only triggers on terminal channel `Faulted`. Outer-state churn suppressed during channel-mgr cycles via `_channelReconnectInProgress` flag wired through `IManagedTransportChannel.StateChanged` - New `ManagedSession.ChannelStateChanged` event surfaces transparent reconnects - `ConnectionStateChangedEventArgs.UnderlyingChannelState` populated when outer reconnect was triggered by a channel fault - Shared `IRetryBudget` propagated so the layers enforce a single max-total-time ### Discovery / Registration / GDS Channel-manager-aware `CreateAsync` overloads on `DiscoveryClient`, `RegistrationClient`, `ServerPushConfigurationClient`, `LocalDiscoveryServerClient`, `GlobalDiscoveryServerClient`. Each registers a minimal `IReconnectParticipant`; co-located sessions and discovery probes targeting the same endpoint share a channel. ### HTTPS transport (`Stack/Opc.Ua.Bindings.Https/`) - New `IOpcUaHttpClientFactory` abstraction - HTTPS / OPC-HTTPS channels migrated to `IHttpClientFactory` + `Microsoft.Extensions.Http.Resilience` (10.6.0; AOT-compatible on net10 via source generators; legacy direct-HTTP path retained for older TFMs / no-DI consumers) ### DI `AddOpcUa().AddClient(...)` registers `IClientChannelManager` and the named HTTPS `HttpClient` + standard resilience handler automatically. `ManagedSessionBuilder` resolves the manager from DI when present. ### Obsoletions (functional, `[Obsolete]` with guidance) - `IClientBase.AttachChannel` / `DetachChannel` - `SessionReconnectHandler` - `SessionExtensions.ReconnectAsync(connection, ct)` and `(channel, ct)` legacy overloads ### Tests - Core: `ClientChannelManagerManagedTests`, `ClientChannelManagerCertRotationTests`, `RetryBudgetTests`, `OpcUaHttpClientFactoryTests` — **56 / 56 pass** - Client: `ManagedSession*Tests` updated; legacy `SessionReconnectHandlerTests` + `SessionExtensionsTests` preserved with `#pragma warning disable CS0618` — **129 / 129 pass** - Sessions integration: `ChannelManagerSharing` / `TransparentReconnect` / `SessionLifecycle` / `CertRotation` — **7 pass, 1 skipped** (`ChannelManagerExhaustionEscalatesAndRecoversWhenServerReturns` un-`[Ignore]`d and implemented in this PR via `SwapFaultedEntryAsync`; remaining skip is end-to-end `RequiresSessionRecreate` plumbing — carry-forward) - Stress (consolidated `Tests/Opc.Ua.Stress.Tests/Subscriptions/` + `Channels/`): chaos / contract / soak / lifecycle / leak-accuracy / gap / participant-result-aggregation tests merged from the previously-separate `Opc.Ua.Channels.Stress.Tests` project; nightly run via `.github/workflows/stress-test.yml` Builds clean (0 warnings / 0 errors) across all six TFMs: `net472`, `net48`, `netstandard2.1`, `net8.0`, `net9.0`, `net10.0`. ### Docs - `Docs/Sessions.md` new § 4 documents the channel manager, three-state gating, two-level retry with shared `IRetryBudget`, HTTPS resilience layering - `Docs/MigrationGuide.md` migration recipes for the four obsoleted API groups - `Docs/DependencyInjection.md` channel-manager and HTTPS factory registration ## Carry-forward (separate PRs) - End-to-end `ParticipantReconnectResult.RequiresSessionRecreate` → `Session.RecreateAsync` lifecycle wiring (currently incomplete — flagged in the 1 remaining skipped integration test) ## Related Issues - Fixes #3288 - Builds on #3282 (PR that introduced the initial `ITransportChannelManager` skeleton) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. *(Channel-manager-specific tests + ManagedSession/client tests verified across all 6 TFMs; full UA.slnx run is pending CI.)* - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received.	2026-06-13 16:34:48 +02:00
Marc Schier	4e6c3303dd	Add V2 subscription triggering (SetTriggering) with N:M support and recreate replay (#3854) ... # Description Adds **triggered-item (SetTriggering) support to the V2 (modern) subscription engine** with a hybrid API, batched apply pipeline, and automatic replay on recreate/reconnect — closing the gap from #3834 and adding store/load round-trip for triggering relationships. ## Public API | Concern | Surface | |---|---| | Declarative | `MonitoredItemOptions.TriggeredByNames` (`IReadOnlyList<string>`) — set at item-add time | | Imperative | `ISubscription.SetTriggeringAsync(IMonitoredItem triggeringItem, IReadOnlyCollection<IMonitoredItem>? linksToAdd, IReadOnlyCollection<IMonitoredItem>? linksToRemove, CancellationToken ct)` returning `SetTriggeringResult` with per-link statuses | | Fluent helpers | name-based overloads `SetTriggeringAsync(this ISubscription, string triggeringItemName, params string[] triggeredItemNames)` (and `add`/`remove`/`ct` variant) | | N:M navigation | `IMonitoredItem.TriggeringItems` (items that trigger this) and `IMonitoredItem.TriggeredItems` (items I trigger), both on-demand projections | | Result type | `public sealed record SetTriggeringResult(IMonitoredItem TriggeringItem, IReadOnlyList<(IMonitoredItem Item, StatusCode Status)> AddResults, IReadOnlyList<(IMonitoredItem Item, StatusCode Status)> RemoveResults, StatusCode ServiceResult)` | No client handles surface on the public API. Validation in the imperative path is by **reference identity** under the manager lock — items from a different subscription with a matching name cannot sneak through, and a removed-then-re-added item with the same name does not silently re-target. ## Engine — batched apply A new manager-level operations queue + a batched apply step in `MonitoredItemManager.ApplyMonitoredItemChangesAsync`: 1. After Create / Modify / SetMonitoringMode, drain the operations queue. 2. Group by triggering item (reference identity). 3. Re-queue ops whose triggering item is not yet `Created` (bounded by `MaxTriggeringRetryCount`; permanently failed items propagate the create-failure status to the operation TCS). 4. Fold per-edge across operations targeting the same triggering item — last-intent wins; same-edge add+remove resolves to one net operation. 5. Pre-resolve: triggered items with `Created=false` resolve to `Bad_MonitoredItemIdInvalid` (adds) or `Good` no-op (removes, per Part 4 §5.13.1.6 server auto-cleanup). 6. Issue **one `SetTriggering` RPC per triggering item** carrying both `linksToAdd` and `linksToRemove` (the server processes remove before add per §5.13.5.2 — atomic swap semantics). 7. Map per-link results back to contributing operations; complete each TCS with its own `SetTriggeringResult` subset. Imperative writes mutate the per-item runtime `DesiredTriggeredByNames` synchronously so navigation (`TriggeringItems`/`TriggeredItems`) and `Snapshot()` reflect intent before the RPC fires. Per-link failures roll back the optimistic mutation so the client view matches server reality. ## State ownership `MonitoredItemOptions.TriggeredByNames` is the **initial declarative input**. The canonical runtime source of truth is a per-item `DesiredTriggeredByNames` field, initialized from options and mutated by both the imperative API and subsequent options changes. The field is preserved across `Reset` (recreate path), which is what enables automatic SetTriggering replay after recreate. ## Save / load (closes #3834) - `MonitoredItemStateSnapshot.TriggeredByNames` (`ArrayOf<string>`) replaces the old `TriggeringItemClientHandle` field (V2-only — not yet shipped, so a clean break; classic snapshot format unchanged). - `Snapshot()` reads from runtime `DesiredTriggeredByNames`; `ApplyLoadState` writes loaded names into both options and runtime state without leaking through the OnChange tracker. - **TransferSubscriptions** success path: only the local view is restored — trusting the spec (`§5.13.5`) that servers preserve triggering links across subscription transfer. No SetTriggering RPC is issued. - **Recreate-fallback** (transfer rejected), **in-session `RecreateAsync`**, and **Reconnect** paths: `Reset` preserves `DesiredTriggeredByNames` and enqueues an add-delta per preserved triggering name; the batched apply pass replays SetTriggering after items finish (re)creating. ## Error handling Per OPC UA Part 4 §5.13.5.3/§5.13.5.4 (Tables 73/74): | Condition | Engine response | |---|---| | Per-link `Bad_MonitoredItemIdInvalid` | Surfaced in `SetTriggeringResult.AddResults`/`RemoveResults`; optimistic `DesiredTriggeredByNames` mutation rolled back | | Service-level `Bad_SubscriptionIdInvalid` | Operations re-queued, not terminal — replays after the subscription state machine recreates | | Service-level `Bad_TooManyOperations` | Surfaced as `ServiceResultException` to all contributing TCSs (no auto-chunking in this version) | | Triggering item never reaches `Created` | After `MaxTriggeringRetryCount` re-queues, op completes with the triggering item's propagated error (no infinite loop) | | Communication errors | Routed through the existing retry policy with bounded retry count | ## Spec consultation Cross-checked against **OPC UA Part 4 v1.05.07 §5.13.5** (verified Table 73 / Table 74 directly via the OPC Foundation spec site) and §5.13.1.6 (deletion side-effects, monitoring-mode interactions). The only per-link spec-specific status code is `Bad_MonitoredItemIdInvalid`; `linksToRemove` is processed server-side before `linksToAdd` (driving the single-RPC-per-trigger batch shape). ## Tests - **9 new unit tests** in `MonitoredItemManagerTriggeringTests.cs` (single op, multi-op merge into one RPC, per-trig grouping, requeue-when-not-Created, per-link rollback, `Bad_SubscriptionIdInvalid` recovery, deletion races, last-intent same-edge conflict resolution). - **Updated** integration tests `SetTriggeringTrackingV2Async` and `SnapshotCapturesTriggeringForReplayAsync` to use the new API. - **2 new integration tests** in `SubscriptionTests.cs`: `DeclarativeTriggeredByNamesV2Async` (options-driven path end-to-end) and `NToMTriggeringV2Async` (one triggered item linked to two trigger items, per-link removal). - **AOT smoke test** in `SubscriptionAotTests.cs` for the `MonitoredItemStateSnapshot.TriggeredByNames` binary encoder/decoder round-trip. - All **344 V2 subscription/monitored-item unit tests** and **71 V2 subscription integration tests** pass locally (net10). ## Documentation - New: `Docs/Subscriptions.md` — *Subscriptions and Monitored Items Service Set*. Merges the previous `StreamingSubscription.md` and `SubscriptionTriggering.md` into a single guide; leads with a Classic-vs-V2 engine overview (recommending V2 for new code) and links to `Sessions.md` §4 for the full engine comparison. Covers the V2 callback API, declarative + imperative SetTriggering with N:M, save/load/restore behavior, error handling, and the `IStreamingSubscription` async-enumerable surface. - Updated: `Docs/README.md`, `Docs/AlarmsAndConditions.md`, `Docs/StateMachines.md`, `Docs/Profiles.md` — all point to the new merged doc. - Updated: `plans/26-v2-subscription-parity.md` — rows 49 (`SetTriggeringAsync`) and 111 (`TriggeringItems` plural / `TriggeredItems`). ## Related Issues - Fixes #3834 — `TransferSubscription does not automatically restore/retain links between trigger and monitored items`. The fix is two-pronged: (a) transfer preserves links server-side per spec and the client snapshot captures `TriggeredByNames` so the local view restores automatically; (b) when transfer fails and the engine falls back to recreate (or when `RecreateAsync` is invoked explicitly), `Reset` enqueues per-triggering-name add-deltas from the preserved runtime state and the batched apply pass replays `SetTriggering` after re-creation. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received.	2026-06-12 10:38:41 +02:00
Marc Schier	4756e7c245	DI (OPC 10000-100) with samples (Pump, SW Update), fluent API gap closures and model dependency implementation (#3808) ... This PR implements the **OPC UA Device Integration (DI, OPC 10000-100)** companion-spec library trio (`Opc.Ua.Di` / `Opc.Ua.Di.Server` / `Opc.Ua.Di.Client`) plus a worked example (`Applications/MinimalPumpServer`) demonstrating the OPC 40223 Pumps companion specification end-to-end with full simulation. It also closes 11 fluent-API gaps in `Opc.Ua.Server.Fluent`, three source-generator bugs, integrates the DI layer into the unified `services.AddOpcUa()` Microsoft.Extensions DI hosting pattern, and adds 250+ new tests. **Worked example — `Applications/MinimalPumpServer/`** - AOT-compatible console host using `Host.CreateApplicationBuilder` + `AddOpcUa()`. - `PumpNodeManager` inherits from `DiNodeManager`; multi-namespace (DI + Machinery + Pumps) via runtime NodeSet2 loading and `ModelLoaderBuilder`. - Demonstrates declarative second-pump creation via `.ConfigureDevicesFor<PumpNodeManager>(...)` and software-package seeding from the same configurator. - Full simulation: pressure, temperatures, power, flow, efficiency, level, cavitation/overheat fault injection, alarm setup, NAMUR-style supervision. **Documentation** - `Docs/Opc.Ua.Di/` — README, DeviceBuilder, Hosting, LockService, SoftwareUpdate, ClientHelpers (6 new developer guides). - `Docs/Opc.Ua.Di/ComplianceMatrix.md` — every DI declaration mapped to In-v1 / Generated / Deferred / Excluded. - `Docs/CompanionSpecLibraries.md` — packaging guide for new companion-spec libraries. - `Docs/FluentApiGaps.md` — historical record of G1–G11. - `Docs/SourceGeneratedNodeManagers.md` updated for richer node managers. - `Docs/plans/StateMachineBuilder.md` — deferred state-machine builder design.	2026-06-04 05:57:57 +02:00
Marc Schier	49305d9085	V2 subscription engine close gap for test and feature parity with classic engine (#3824) ... # Description V2 subscription engine feature parity round to make the classic engine eligible for deprecation. Splits the test surface so the classic and V2 engines can evolve independently, ports the gaps the classic-engine integration tests had been exercising, and lands the public V2 API additions needed to close them. - Splits `Tests/Opc.Ua.Subscriptions.Tests` into a classic (`Opc.Ua.Subscriptions.Classic.Tests`) and a V2 (`Opc.Ua.Subscriptions.Tests`) project to shrink CI time and let the two engines diverge cleanly. - Adds the V2 public-surface gaps required by the test ports (`Subscription.ServerId` on the class, `SetSubscriptionDurableAsync`, `SendInitialValuesOnTransfer`, unified `OnSubscriptionStateChangedAsync` handler callback, per-item `ConditionRefreshAsync`, fluent `LoadSubscriptionsAsync(Stream, …)` and `Subscription.Snapshot()` / `RestoreAsync()` for save/load + transfer). - Lands handler-centric design (no `PublishingStopped` property handlers maintain derived state); always-sequential per-subscription delivery documented + tested. - **Adds short and long haul stability tests for the V2 `ManagedSession` + subscription engine. Closes #3744** (see "Issue #3744 Haul tests" below). - **Migrates SubscriptionStateSnapshot / MonitoredItemStateSnapshot to `[DataType]` source-generator codecs**. - `SubscriptionDurableV2Tests.cs` 5 V2 ports of the classic durable test patterns. - `V2FollowUpCoverageTests.cs` handler state-change callback, fluent stream-based `LoadSubscriptionsAsync`, `SendInitialValuesOnTransfer`, snapshot edge cases (empty / `DataChangeFilter` round-trip / concurrent mutation). - `SubscriptionFailoverV2Tests.cs` forced channel-break with and without `WithTransferSubscriptionsOnRecreate`. - `MonitoredItemConditionRefreshLiveV2Tests.cs` — live `ConditionRefresh` against the reference server's event source; asserts `RefreshStart` / `RefreshEnd` events flow through the handler. ## Fixes - #3744 Haul tests (V2 ManagedSession + subscription engine) `Tests/Opc.Ua.Sessions.Tests/ManagedSessionStabilityTest.cs` adds the short/long haul coverage requested by [#3744](https://github.com/OPCFoundation/UA-.NETStandard/issues/3744) plus the bonus fault-injection variant: | Test | Duration | Category | Purpose | |---|---|---|---| | `ShortHaulManagedSessionV2Async` | 2 minutes | `ManagedSessionHaul` (`[Explicit]`) | Quick local sanity check of the V2 monotonicity contract. | | `LongHaulManagedSessionV2Async` | 90 minutes (configurable via `TEST_DURATION_MINUTES`) | `ManagedSessionHaul` (`[Explicit]`) | Long-running V2 stability + monotonicity. | | `LongHaulManagedSessionWithFaultInjectionV2Async` | 90 minutes (configurable) + fault every `FAULT_INJECTION_INTERVAL_SECONDS` (default 60s) | `ManagedSessionHaul` (`[Explicit]`) | **Bonus** — force-closes the subscriber transport channel periodically; verifies V2 manager transfers/recreates the subscription and continues monotonic delivery across reconnects. | **Pattern.** A writer `ManagedSession` increments a `uint32` counter and writes it into `Scalar_Static_Mass_UInt32_UInt32_00` every 250ms. A subscriber `ManagedSession` runs a V2 subscription with a single monitored item on the same node. `MonotonicCounterHandler` asserts each received sample is *strictly greater than* the previous one. V2 `ISubscriptionNotificationHandler` per-subscription ordering guarantee. Skipped values (sampling rate < write rate) are tolerated; duplicates and reorderings are violations.	2026-06-03 04:39:08 +02:00
Marc Schier	01d7162313	Part 9 Alarms & Conditions: complete server + client implementation (#3805) ... Complete implementation of **OPC UA Part 9 — Alarms & Conditions** for both server and client, **OPC UA Part 16 — State Machines** (generic extensible client + server API), and the supporting infrastructure (source-generated event records, source-generated ObjectType proxies, `IAsyncEnumerable` streaming subscriptions, address-space model change tracking, per-node `INodeCache` invalidation, unified DI integration). ### Server-side (Part 9 complete) - **`AlarmConditionState`** — every Part 9 method wired automatically in `OnAfterCreate`: `Silence`, `Suppress`/`Suppress2`, `Unsuppress`/`Unsuppress2`, `RemoveFromService`/`RemoveFromService2`, `PlaceInService`/`PlaceInService2`, `Reset`/`Reset2`, `GetGroupMemberships`. Each method generates the spec-mandated audit event and supports an optional veto-delegate hook. - **Latched alarms** — `ActiveState` reflects real process state; `LatchedState` persists until explicit `Reset` (with full precondition validation). - **Re-alarming** — `ProcessReAlarm`/`ResetReAlarmRepeatCount`/`IsReAlarmEnabled` helpers (opt-in, no hidden timer). - **Audible alarms** — `UpdateAudibleState` for `AudibleEnabled`/`AudibleSound`/`SilenceState` lifecycle. - **Suppression + groups** — `AlarmGroup`, `AlarmSuppressionEngine` (with first-in-group), `AlarmRateTracker` (for `AlarmMetricsType`). - **Spec-compliance fixes** — `SuppressedOrShelved` now includes `OutOfServiceState`; `GetRetainState` includes `LatchedState`; `EffectiveDisplayName` composition includes `OutOfServiceState` + `LatchedState`. ### Client-side Part 9 API - **`AlarmClient`** — typed Part 9 operations. Each method delegates to the matching source-generated `*TypeClient` proxy (`ConditionTypeClient`, `AcknowledgeableConditionTypeClient`, `AlarmConditionTypeClient`, `DialogConditionTypeClient`, `ShelvedStateMachineTypeClient`) — one source of truth for every method NodeId. - **Interface hierarchy** — `IConditionOperations` → `IAcknowledgeableConditionOperations` → `IAlarmOperations` + `IDialogConditionOperations`. - **Source-generated event records** — new `EventRecordGenerator` (alongside `ObjectTypeProxyGenerator`) emits `{Type}TypeRecord` per OPC UA event type: `ConditionTypeRecord` → `AcknowledgeableConditionTypeRecord` → `AlarmConditionTypeRecord` → `LimitAlarmTypeRecord`/`ExclusiveLimitAlarmTypeRecord`/… all the way to vendor subtypes. **Vendor alarm types get records automatically.** - **`AlarmEventDecoder`** — decodes `EventNotification.Fields` into the most-specific generated record. - **`AlarmEventFilterBuilder`** — fluent builder for Part 9 `EventFilter`s. - **`ISession.GetAlarmClient(telemetry)`** convenience extension. ### Part 16 state machines (generic, extensible) - **Client**: extension methods on the source-generated `StateMachineTypeClient` / `FiniteStateMachineTypeClient` — `GetCurrentStateAsync`, `ObserveFiniteTransitionsAsync` (`IAsyncEnumerable<FiniteStateSnapshot>`), `WaitForStateAsync`, `GetAvailableStatesAsync` / `GetAvailableTransitionsAsync`. **Vendor proxies inherit the API automatically** through the generated inheritance chain. - **Server**: `StateMachineBuilder` (fluent: `AddState` / `AddTransition` / `OnCause` / validation) + `FluentFiniteStateMachineState` — declare a Part 16 state machine without subclassing the abstract base. - **Alarms alignment**: `AlarmClient.GetShelvingStateAsync` / `ObserveShelvingTransitionsAsync` surface the API for every alarm's `ShelvingState` child. ### IAsyncEnumerable streaming subscription - **`IStreamingSubscription`** with `SubscribeDataChangesAsync`/`SubscribeEventsAsync` returning `IAsyncEnumerable<T>` (Channel-backed). - **`ManagedSession.DefaultStreaming`** — lazy shared subscription; disposed with the session. - **Helpers** — `TakeUntilAsync`, `WithTimeoutAsync`, `TakeAsync`, `BufferedAsync` for state-machine waits and bounded observation windows. - **`AlarmStreamExtensions`** — `SubscribeAlarmsAsync`/`SubscribeConditionsAsync`/`SubscribeDialogsAsync` returning the source-generated record types. ### Address-space model change tracking - **Server**: `RaiseGeneralModelChangeEvent` on both `CustomNodeManager` and `AsyncCustomNodeManager`; `ModelChangeAggregator` (in `Opc.Ua.Server.NodeManager`, used by every node manager) batches changes per service call; **auto-emit** from `CreateNode`/`DeleteNode` (opt-out via `ModelChangeEmissionEnabled = false`). - **Client**: `IModelChangeTracker` subscribes to `BaseModelChangeEventType` via the streaming subscription, decodes `ModelChangeStructureDataType` payloads, invalidates the `INodeCache` per-node. - **`INodeCache.InvalidateNode(NodeId)`** — new interface member (true per-node eviction in `NodeCache`). - **`ManagedSession.EnableModelChangeTrackingAsync()`** + `ManagedSessionOptions.ModelChangeTracking` + `ManagedSessionBuilder.WithModelChangeTracking()` fluent method. ### Dependency Injection integration - **`services.AddOpcUa().AddClient(...).AddAlarms()`** — registers `AlarmClientFactory` so DI-hosted apps can resolve a Part 9 client per session without `new`-ing one manually. - Cross-link added to the unified DI surface (`AddServer` / `AddClient` / `AddComplexTypes` / `AddGdsServer` / `AddLdsServer` / `AddWotConServer`). ### Quickstart server modernization - `Quickstarts.Servers/Alarms/AlarmNodeManager.cs` ported to `AsyncCustomNodeManager` and demonstrates the new `AlarmGroup` + `AlarmSuppressionEngine` end-to-end via a writable `MaintenanceMode` boolean that suppresses every member alarm. ### Documentation - New: `Docs/AlarmsAndConditions.md`, `Docs/StreamingSubscription.md`, `Docs/ModelChangeTracking.md`, `Docs/StateMachines.md` — full developer guides. - Updated: `Docs/DependencyInjection.md` (`AddAlarms()` row + section), `Docs/README.md` index, `Docs/MigrationGuide.md` (Part 9 / Part 16 / model change tracking sections covering behavior changes + breaking API surface). - Sample: `Applications/ConsoleReferenceClient/AlarmClientSample.cs`.	2026-05-29 23:24:53 +02:00
Marc Schier	889572472f	Convert DataValue from class to readonly struct for further GC relief (#3796) ... Convert `DataValue` from a mutable reference type (`class`) to a `readonly struct` to eliminate per-allocation GC pressure on every attribute read, subscription publish, and history sample. ### Key API changes (user-visible) | Before | After | Pattern | |---|---|---| | `class DataValue` | `readonly struct DataValue : INullable` | Foundation | | `Equals(DataValue?)` / `operator ==(DataValue?, DataValue?)` | `Equals(DataValue)` / `operator ==(DataValue, DataValue)` | Value semantics | | `IEncoder.WriteDataValue(string?, DataValue?)` | `IEncoder.WriteDataValue(string?, DataValue)` | Non-nullable wire API | | `IDecoder.ReadDataValue(...) -> DataValue?` | `IDecoder.ReadDataValue(...) -> DataValue` | Same | | `IDataChangeMonitoredItemQueue.PeekLast/Oldest()` | `TryPeekLastValue/Oldest(out DataValue)` | TryGet pattern | | `IAggregateCalculator.GetProcessedValue(bool)` | `TryGetProcessedValue(bool, out DataValue)` | Same | | `IUaPubSubDataStore.ReadPublishedDataItem(...)` | `TryReadPublishedDataItem(NodeId, uint, out DataValue)` | Same | | `Dictionary<uint, DataValue?>` (NodeCacheContext) | `Dictionary<uint, DataValue>` | Default + IsNull | | `IHistoryDataSource.FirstRaw/NextRaw` | `TryFirstRaw/TryNextRaw(out DataValue)` | TryGet pattern | | `NodeState.ReadAttribute(... DataValue)` | `NodeState.ReadAttribute(... ref DataValue)` | Sync mutation via ref | | `NodeState.ReadAttributeAsync(... DataValue)` | `NodeState.ReadAttributeAsync(...) -> ValueTask<(ServiceResult, DataValue)>` | Async return tuple | | `obj as DataValue` (heterogeneous Dictionary<string, object>) | `obj is DataValue dv ? dv : default(DataValue)` | Safe struct cast | | `Assert.That(dv, Is.Not.Null)` (tests) | `Assert.That(dv.IsNull, Is.False)` | Struct test pattern | | `[Obsolete] Value` setter, all property setters | Removed; use ctor or `With*()` | Immutability | | `[DataContract]` + `[DataMember]` | Removed; surrogate via `SerializableDataValue` + `DataValueSurrogateProvider` | Serialization | ### Wire format Wire format is **byte-identical** to the previous class-based implementation across binary / JSON / XML encoders. Default DataValue (`.IsNull == true`) is treated as the equivalent of the old null reference (writes JSON `null`, binary `byte 0`, XML omitted element). ### Why TryGet pattern (instead of leaving `DataValue?`) For methods that previously returned `DataValue?` to signal "no value available" (queue peek, aggregate result, data store lookup, history navigation), the conversion to `bool TryGet(out DataValue)` matches BCL conventions (`Dictionary.TryGetValue`, `Queue.TryPeek`, `Int32.TryParse`) and gives the caller an explicit "may be absent" affordance without `Nullable<DataValue>` overhead. ### Breaking changes summary - `DataValue` is now a `readonly struct` - boxing/unboxing semantics change for callers using `object`-typed APIs. - `IEncoder.WriteDataValue` / `IDecoder.ReadDataValue` / `ReadDataValueArray` interface signatures lose the `Nullable` wrapper. - `IDataChangeMonitoredItemQueue.PeekLastValue` / `PeekOldestValue` -> `TryPeekLastValue` / `TryPeekOldestValue` (out parameter). - `IAggregateCalculator.GetProcessedValue` -> `TryGetProcessedValue` (out parameter). - `IUaPubSubDataStore.ReadPublishedDataItem` -> `TryReadPublishedDataItem` (out parameter). - `NodeState.ReadAttribute` takes `ref DataValue`; `ReadAttributeAsync` returns `ValueTask<(ServiceResult, DataValue)>`. - `DataValue` property setters removed (use constructor + `With*` mutators).	2026-05-23 11:38:49 +02:00
Marc Schier	3e5f129c6f	Enable nullable in Stack, Libraries (9), and Applications projects (#3732)	2026-05-17 12:55:24 +02:00
Marc Schier	96f8964704	Fix erroneous debug.assert in SessionManager (#3759) ... * Erroneous debug.assert in SessionManager * Improve build pipelines	2026-05-13 18:05:39 +02:00
Marc Schier	d14cd10a86	Model dependency annotations using source generator + flaky test fixes (#3687) ... * Fix publish/dispose race that hangs Client tests Subscription.Dispose pre-disposed m_publishTimer and m_messageWorkerCts before calling ResetPublishTimerAndWorkerState, causing Cancel() to throw ObjectDisposedException and leaving the publish worker stuck. Removed the duplicate disposal so the reset method is the single owner of those resources, and guarded Cancel/Dispose with try/catch ObjectDisposedException for defense in depth. Session.OnPublishComplete could run after Session.Dispose disposed m_reconnectLock, throwing ObjectDisposedException from SemaphoreSlim.Wait() and re-arming via the throttle path indefinitely. Added an early Disposed guard, wrapped Wait/Release in try/catch ObjectDisposedException, and added a Disposed guard in QueueBeginPublish so the throttle cannot reschedule publishes after disposal. Fixes the macOS net10.0 hang seen in Azure pipeline build 13257 (TransferSubscriptionAsync, ReconnectWithSavedSessionSecretsAsync). * fix lockout test and use UserName token for SuccessfulAuthenticationClearsFailedAttempts test After c1e11b1ab made anonymous activations no-op for clearing the lockout counter, the test must use a real (non-anonymous) credential to demonstrate that successful authentication resets failed attempts. Use user1/password with PolicyId=1 (UserName_1 in the reference server config). * Fix DirectoryCertificateStore cache staleness across instances When two DirectoryCertificateStore instances point at the same directory (e.g. a CertificateValidator's internal store and a separate store opened by a caller), the directory LastWriteTimeUtc seen by one DirectoryInfo handle does not reliably reflect modifications made through another handle on Windows, even after Refresh(). The mtime-based cache check therefore returned stale entries indefinitely, causing VerifyRejectedCertsDoNotOverflowStoreAsync to flake when the test fixture's RejectedStore failed to observe deletes performed by the validator's internal store. Add a cheap on-disk file count cross-check in the cache-hit branch: if the count of cert files on disk does not match the cached entry count, force a reload. This avoids re-parsing every certificate when nothing has changed but reliably detects external modifications. * test: fix keep-alive mock to return Int32 ServerState The mock for the Server_ServerStatus_State read returned new Variant(0u) (UInt32). The keep-alive worker expects Int32 (ServerState enum is wire-encoded as Int32), so ClientBase.ValidateDataValue logged a BadTypeMismatch warning on every keep-alive throughout the test run. Real servers always emit Int32 for enum variables (BinaryEncoder downgrades the enum encoding byte to Int32), so the warning was purely a test-mock artifact. Switch the mock to new Variant((int)ServerState.Running) to eliminate the spurious warnings. * Source generator: emit and consume ModelDependencyAttribute. Emit one [assembly: ModelDependencyAttribute(...)] per model the generated assembly emits and per upstream model it transitively consumes (new ModelDependencyGenerator in Tools/Opc.Ua.SourceGeneration.Core). * Replace the unused PEReader-based ReferenceMetadataScanner with a Roslyn IAssemblySymbol.GetAttributes() scan (ReferencedModelDependencyScanner in Tools/Opc.Ua.SourceGeneration). Hooks the per-symbol incremental cache, works for both PortableExecutableReference and CompilationReference, AOT-safe. * Wire the scanner result through ModelSourceGenerator into ModelCompilation. The reduced URI-keyed map is passed to the public Generators.GenerateCode extension methods. * NodesetFileCollection.GetDesignFileListForModel now consults the referenced-model map, so transitive nodeset dependencies that are satisfied by a referenced assembly no longer raise the missing-dependency error. * Override resolution: when a local AdditionalFile and a referenced assembly both declare the same model URI under the same C# prefix the local generation is silently skipped (avoids CS0101); different prefixes still generate so downstream consumers can intentionally override types under a different namespace. * New diagnostics MODELGEN010 (override skipped) and MODELGEN012 (multiple referenced assemblies provide same model URI -> tie-break on highest version+publication date). * Bump asserted GeneratedSources counts in ModelGeneratorTests for the new *.ModelDependencies.g.cs files (one per generated model). * Fix unrelated CS1574 broken cref in NodeId.TryParseLongForm xmldoc that was blocking the Opc.Ua.Types build. * Document the new flow in Docs/ModelDependencies.md. * tests for ModelDependencyAttribute flow * Refactor recently added "parsing" constructors to ParseLongForm static method. Method delegates to Parse(IServiceMessageContext) NodeId, ExpandedNodeId and QualifiedName ParseLongForm overloads are now thin wrappers around Parse(context, options) using NodeIdParsingOptions with RequireResolvedUris=true and updateTables=false. Adds the FallbackToStringIdentifier opt-in to preserve the legacy lenient quirk for callers that need it (off by default for strict parity). Tests for the refactored long-form parsers were added to the existing per-type test fixtures. * Update nuget dependencies to latest	2026-04-22 10:15:12 +02:00
Marc Schier	09e44453a4	Generate type safe (method) proxies for ObjectTypes inside models (#3685) ... Add interop test agent The new stack-pass ObjectMethodProxyGenerator emits proxies that derive from Opc.Ua.ObjectTypeClient and reference Session.MessageContext.NamespaceUris. Neither type exists in the in-memory test compilation, which only references Opc.Ua.Types.dll. Moving ObjectTypeClient to Opc.Ua.Types would cascade ISessionClient, IClientBase, ITransportChannel and the generated service interfaces with it, so the type stays in Opc.Ua.Core and we add a minimal stub to OpcUaCoreStubs instead. Wire Microsoft.Extensions.Options 10.0.5 + DependencyInjection.Abstractions 10.0.5 into Directory.Packages.props and the Gds.Client.Common project to enable upcoming options-driven configuration work. Modernize Opc.Ua.Gds.Client.Common (DI, options, interfaces, async dispose)	2026-04-22 05:59:06 +02:00