mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-20 00:31:32 +08:00
fix/diagnostic-delete-nodeversion-master
312 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Cristian Pogacean
|
bef85adf9f |
Preserve NodeVersion eligibility when deleting runtime nodes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
romanett
|
40288fa7d8 |
Tests: weighted FairRequestQueue test asserts the median of three windows (#4763)
# Description `FairRequestQueueTests.WeightedServiceHoldsUnderConcurrentWorkersAsync` failed once on GitHub Actions macOS net10.0 (#4753, run 38349863236): light=124 heavy=276, ratio 2.23 against an expected 2.5-3.5. It passed on every other CI run and 30/30 locally. **Root cause.** `TrySelect` dequeues an owner's turn token from the lock-free ready queue. On the last token of a turn, `TakeTurnToken` CASes the turn and calls `PushTurn` for the next one. A worker preempted between those two steps leaves the owner with no queued token while it is stalled. The work-conserving dispatch serves the other owner in the meantime, and that share is never made up. Delay injection confirms this: sleeping 1 ms on every 7th turn publish collapses 1:3 to roughly 1:1 (light=199 heavy=201). Publishing a turn ahead of time would only cover stalls shorter than one round (microseconds), while real preemption lasts milliseconds. Making up the lost share (deficit or virtual time) would mean redesigning the scheduler. So this PR documents the bounded skew and makes the test robust instead. **Changes** - `FairRequestQueue` remarks: the weighting holds over time, not in every window, and a preempted worker can cost an owner its share for the length of the stall. - `WeightedServiceHoldsUnderConcurrentWorkersAsync` now measures 3 independent windows through `MeasureWeightedServiceAsync` and asserts that the **median** heavy/light ratio is in 2.5-3.5. The failure message lists all three windows. - Each window uses perOwner = 480 and measured = 560, with queue limits of `2 * perOwner`. That stays under the TestProvider's 1000-request cap and keeps both owners backlogged. - Each window still checks that both owners stay backlogged, then drains the queue and checks that everything is back to zero. The median tolerates one skewed window and still fails when the weighting is lost systematically. With stalls injected into one window only, the median is 3.0 and the test passes (e.g. `light=238 heavy=322; light=140 heavy=420; light=140 heavy=420`). With stalls injected into every window, it still fails. **Validation:** the `FairRequestQueueTests` fixture passes 93/93 on net10.0 Release. The weighted test passed 30/30 runs pinned to 2 cores (`start /affinity 3`). `dotnet format whitespace --verify-no-changes` is clean, and the build has 0 warnings. ## Related Issues - Flaky test seen on #4753 after #4750 (FairRequestQueue without a queue-wide lock) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
3e433eb6c4 |
XRegistry: registration lifecycle, identifier allocation, registrar roles and atomic Resource replace (#4756)
# Description Extracts the generic xRegistry registration lifecycle from the WoT work into a standalone, master-targeted change: - Canonical xRegistry identifier allocation (`XRegistryIdentifier`) with case-folded bounded collision handling and exact Version allocation/delete semantics. - Registrar roles / capabilities in the xRegistry NodeSet2 model (only the roles/capabilities facts). - Default-Version views and Resource Meta handling. - FileType read/write modes, session ownership/discard, cursor handling, and short reads until actual EOF on the client. - Atomic whole-document `ReplaceAsync` on the in-memory and file-system resource stores. - Generic projection engine/provisioning and projection contracts, label ownership/no-op behavior, Resource-vs-Version event targets. - Documentation in `docs/XRegistry.md` and XML docs for owned members. - Tests: Identifier, RegistrarRoles, Attribute, Projection, RegistrationEvent, FileMode, ResourceStore, Client, VersionAssignment, concurrent read. Intentionally excluded (handled in separate PRs): trusted Federation / Registry origin model and provider (G9); WoT lease/dispatcher, prepared manifest, 0.7 model/version bump (G12). Merged current `master` ( |
||
|
Marc Schier
|
c2b95e2085 |
XRegistry: trusted logical-Resource federation identity (#4755)
# Description Adds trusted **logical-Resource** federation identity to XRegistry - A federation proxy binds an origin registry (`RegistryOriginDataType`), application, portable `ResourceNodeId` and Xid, never an endpoint or content-digest identity. - `XRegistryFederationTarget`, `IXRegistryFederationProvider`, `XRegistryClient.Federation` (verify/follow/relocate; uses master's `ISessionBindingProvider.CreateBindingAsync`). - `XRegistryFederationNodeManager`, `XRegistryServerOptions`/DI: required target/provider verification and proxy admission. - Model hunks only: `RegistryOriginDataType` 63026 (+encodings 63591/63592), `OriginRegistry` 63593 + `ResourceType` HasProperty, `ExternalReference`/`ResourceUrl` description rewrites. - Tests (`XRegistryFederationNodeManagerTests`), package READMEs, `docs/XRegistry.md` Federation section (links `SessionBindings.md`). Merged current master ( |
||
|
romanett
|
3a31473cf4 |
Server: no server-wide or per-session locks in per-request session validation (#4753)
# Description
Every request passes through `SessionManager.TryGetSessionContext` at
admission and again at dispatch
(`FairRequestQueue.GetRevalidationStatus` →
`DefaultServerResourceIsolationProvider` →
`ISessionBindingProvider.TryGetSessionContext`), and then through
`Session.ValidateRequest`. Together they took:
1. the server-wide `m_bindingsLock`, to confirm the candidate binding
was still the committed one;
2. the per-session diagnostics lock in `Session.HasExpired`. It is
inlined, so CPU traces charge its spin to `TryGetSessionContext`;
3. the session lock in `Session.IsSecureChannelValid`;
4. the session lock around all of `Session.ValidateRequest`, plus the
diagnostics lock twice (contact refresh, diagnostic counters);
5. the server-wide `SessionManager` event lock, to raise
`SessionDiagnosticsChanged` for every request even when nothing
subscribes.
With the FairRequestQueue lock also removed (a local change without the
queue-wide lock, not yet a PR), these were the next waiting points:
14–16.5 % of on-CPU samples spun in `Lock.TryEnterSlow` under
`TryGetSessionContext`, and once that was fixed another 7–10 % under
`Session.ValidateRequest`.
## Changes
**SessionManager: confirm the binding without the bindings lock**
- `SessionActivationState` publishes a `DispatchBinding` (the committed
binding while no activation is committing, otherwise `null`) with
`Volatile.Write`. The `BindingContext` and `IsCommitting` setters
republish it, and every writer of those already runs under
`m_bindingsLock`.
- The lookup reads it once for the candidate and once to confirm. A
commit always creates a new `SessionBindingContext`, so an unchanged
reference means nothing replaced it. A token is mapped to a session
object at most once and is never re-added after removal, so finding the
token still mapped to the same session afterwards means both held at the
instant of the confirming read, which is the guarantee the lock gave.
- IsCommitting, transfer, close/claim, shutdown and the
certificate/security checks are unchanged.
**Session: read the channel id without the session lock**
- `SecureChannelId` is published with a volatile write in `Activate`,
and `IsSecureChannelValid` reads it without the session lock.
**Session: request validation and diagnostic counters without locks**
- Request counts are interlocked increments of per-session slots (total
and error per `RequestType`, plus unauthorized). They are applied to
`SessionDiagnostics` under the diagnostics lock whenever it is read
(`ReadDiagnostics`, `UpdateDiagnostics`, the diagnostics-node callback,
`ClientLastContactTime`), so readers see the same totals.
- The contact refresh is a compare-and-swap on the tick count, and
requests within the same millisecond do not write. An expiry observed by
`HasExpired` or by a refresh is recorded in the contact word
(`kExpiredFlag`), so a request that read the clock before the monitor
saw the timeout cannot refresh it afterwards. This keeps the documented
"a request cannot revive a session the monitor saw expire" guarantee
without the lock. The intermediate commit that made `HasExpired`
lock-free on its own had opened that window; this commit closes it.
- `ValidateRequest` takes no session lock: the channel id and
`Activated` are published with volatile writes.
- `RaiseSessionEvent` returns before taking the event lock when a
`DiagnosticsChanged` event has no subscriber.
Behaviour differences reviewers should know about:
- `ClientLastContactTime` is millisecond-granular, and two concurrent
refreshes can publish their wall-clock times in reverse order, which
leaves it up to about 1 ms older than the latest contact.
- After an expiry has been observed, an `Activate` no longer refreshes
the contact time. `SessionManager` already rejects activation of an
expired session before that point.
## Tests
- `LookupDuringActivationCommitReturnsNoBindingAsync`: a lookup inside
the commit window (after `Session.Activate`, before the binding commit)
returns no binding. A test session re-implements `ISession.Activate` to
run the hook. Before this test, publishing the binding regardless of
`IsCommitting` passed all 690 session tests.
- `LookupRacingTransfersAndCloseReturnsOnlyCommittedBindingsAsync`: 4
readers look up while 1000 transfers alternate between two channels,
followed by a close. Each result must be a committed binding of the
queried channel, no older than the last completed activation and never
decreasing, and nothing may come back after the close.
- `RequestRacingObservedExpiryDoesNotReviveTheSessionAsync`: a clock
hook lets the monitor observe the expiry between a request's clock read
and its refresh, and the request must fail with `Bad_SessionClosed`. It
fails if `HasExpired` does not record the expiry.
- `ConcurrentRequestsAreCountedExactlyAsync`: 8 threads × 2100
validations (Read/Browse, plus rejected Writes). The totals, per-service
counts, error counts and unauthorized count must be exact, and a
concurrent reader never sees the total go backwards.
- Local runs (net10.0): `Opc.Ua.Server.Tests` 8487 passed / 0 failed;
`Opc.Ua.Sessions.Tests` 851 passed / 0 failed / 379 skipped. net48
builds.
## Measurements
`perf/ServerLoadHarness`, Server GC, paired alternating A/B, 7 rounds.
Baseline is master
|
||
|
romanett
|
a14b23e7dd |
Server: cheaper symmetric crypto path, FairRequestQueue without a queue-wide lock (#4750)
# Server: cheaper symmetric crypto path, FairRequestQueue without a queue-wide lock Roadmap items E1 and 5b of the o6 benchmark gap analysis, one commit each. - **E1:** Windows one-shot `Aes.EncryptCbc`/`DecryptCbc` import the key (`BCryptImportKey`) on every message, although #4720 caches one `Aes` per channel token. The AES-GCM and ChaCha20-Poly1305 policies also created a new keyed instance per message. - **5b:** with `AsyncCustomNodeManager`, which is what the o6 benchmark's 2.0.1 worker uses, the `FairRequestQueue` gate became the largest waiting point. `TryEnqueue`, `TrySelect` and `TryDequeue` spun on one `Lock` for every request. No wire, configuration or public API changes. ## Results These are paired A/B runs against `origin/master`, five rounds per scenario. The two harness builds alternate per round, starting side swapped every round, on the same shared machine (Ryzen AI 9 HX 370, 12 cores / 24 threads, Windows 11). The tables show medians, with ranges in brackets where they matter. The server is configured like the o6 2.0.1 worker: `AsyncCustomNodeManager`, Server GC, queue sized clients × outstanding + 3. o6read uses the benchmark's native open62541 client workers; `read` uses the harness's .NET client. On this machine throughput moves between rounds by up to 2×, so CPU and bytes per request are the reliable signals. ### E1: symmetric crypto (measured on |
||
|
romanett
|
63adef0d10 |
Fix the Sessions LoadTest fixture and the server bottlenecks it exposed (#4749)
The `[Explicit]` Sessions `LoadTest` fixture has not started since #4541 (also on the 2.0.0 tag): it set `MaxChannelCount = MaxSessionCount = 10100`, and resource isolation refuses to start unless the channel cap covers the sessions plus the bootstrap and reconnect floors plus one. Every LoadTest case failed in `OneTimeSetUp`. Running the fixture again, and profiling it and the `perf/ServerLoadHarness`, found several bottlenecks that shipped in 2.0. ## Changes **Test** - `MaxChannelCount = MaxSessionCount + 3`, the same margin as the framework defaults (100 / 103). - Notifications are counted with per-node `Interlocked` counters. The shared `ConcurrentDictionary<NodeId, int>`, updated by 750 subscriptions, took 13 % of the process CPU in bucket-lock contention. - After the writer stops, and after the last event, the tests wait until the received count stops growing instead of a fixed delay, and log the drain time. **Server / client** - **Detached monitored item recovery** (#4094): every node a NodeManager added (`AddPredefinedNodeAsync`, recursively for children) scanned every monitored item of every subscription for detached items. `CreateSubscription` adds ~30 diagnostics nodes, so with 450k monitored items subscription creation was quadratic: 69 % of CPU, and `CreateSubscription` timed out. `MonitoredItem` now keeps an exact process-wide count of live detached or deleted items, and `MasterNodeManager` skips recovery while it is zero. All flag changes go through one helper under the item lock; a disposed item does not count, because it cannot be attached again. - **Client channel gate** (#4232): `UaSCUaBinaryClientChannel.SendRequestAsync` entered the gate with a blocking `Gate.Enter()` inside an async method, and `SemaphoreSlim.Wait` spun for 19 % of CPU with many requests in flight on one channel. It now awaits `Gate.EnterAsync(ct)`, like the server side. - **ResourceManager** - Status code and symbolic id translation looked up their maps under the manager lock and translated while holding it. Sessions that request diagnostics do this for every notification: 83 % of lock contention. The maps are copy-on-write and read without the lock. - Translation tables were `SortedDictionary<string, string>` with the default, culture-sensitive comparer: up to 1 µs per Browse display name, 22 % of Browse server CPU. They now use an ordinal `Dictionary`. - **Namespace default permissions**: every permission check read `DefaultAccessRestrictions` / `DefaultRolePermissions` / `DefaultUserRolePermissions` of the node's namespace through two server-wide locks and deep-copied the role permission array, per node. That was 99 % of lock contention for Read/Write of 100 values. The values are now read once per namespace per service request and remembered on the `OperationContext`. Contexts that are not a service request (monitored item and session contexts, which can live long) remember nothing, so a changed default applies to them at once. ## Results `perf/ServerLoadHarness`, paired A/B against `master` (3 alternating rounds, medians, 20 sessions, 24 cores). The harness numbers predate the recovery, client gate and status code translation commits, which those scenarios do not exercise: | Scenario | Throughput | Server CPU / op | Server alloc / op | Lock contention | |---|---|---|---|---| | Read, 100 values | +19 % | −28 % | −18 % | −96 % | | Read, 100 values, SignAndEncrypt | +27 % | −26 % | −18 % | −98 % | | Write, 100 values | noise ¹ | −19 % | −17 % | −88 % | | Browse (Server object) | +3 % | −16 % | −10 % | −2 % ² | | Read, 1 value | +12 % | −4 % | ±0 | ±0 | ¹ Throughput varied run to run on the shared machine; per-op CPU and allocation are the reliable signal. ² Browse contention is the Server node's browse lock in `NodeState.CreateBrowser` (all sessions browse the same node); not changed here. `LoadTest` (opc.tcp, Release, in-process client and server): | Case | master | this PR | |---|---|---| | `ServerReadLoadTestAsync` | setup failure | passes (50M value reads, 0 errors) | | `ServerManySessionsLoadTestAsync(500)` | setup failure | passes | | `ServerSubscribeLoadTest_Classic` | setup failure; on 2026-09-26 (before #4541): `CreateSubscription` `BadRequestTimeout` | 76–85 % received, **fails** (needs 99 %) | | `ServerSubscribeLoadTest_ManagedSessionV2` | setup failure; 2026-09-26: 1,885 notifications received | 35–45 M notifications, 77–85 % received, **fails** | | `ServerEventSubscribeLoadTestAsync` | setup failure; 2026-09-26: 34.6 % | passes: 100 % delivered, drained ~40 s after the last event | The subscribe cases now lose notifications rather than deliver them late: delivery stops ~2 s after the last write. Every monitored item has queue size 1 and the writer is not paced by delivery, so whatever exceeds the publish rate (~585k notifications/s here, SignAndEncrypt with the client in the same process) is overwritten. The event test lost nothing either: it waited a fixed 2 s after the last event and counted the backlog as lost. It now waits until delivery stops. Its throughput is bound by garbage collection: the event consumer of the Server node spends ~90 % of its wall time suspended for workstation GCs of the shared test process. With `DOTNET_gcServer=1` it generates 702 instead of ~210 events/s and drains in 7 s instead of ~40 s, and the subscribe cases reach 87 % (Classic) and 93 % (ManagedSessionV2) at 78–106 M notifications. ## Testing - `Opc.Ua.Server.Tests` 8468 passed / 5 skipped, `Opc.Ua.Client.Tests` 2788 passed, `Opc.Ua.Sessions.Tests` 851 passed / 379 skipped, `Opc.Ua.Core.Tests` Tcp/Channel 797 passed (net10.0). - New `OperationContextDefaultPermissionsCacheTests`. - `Opc.Ua.Server` builds for net48; `dotnet format whitespace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
7c965fd636 |
Capture metadata images and generation-bound session clients (#4752)
# Description - Add isolated NodeState reference images, TypeTable graph snapshots, EncodeableFactory registration forks/views, StringTable mutation versions/private copies, and ServiceMessageContext mapping-holder generation tracking. Reference images are lazy for empty nodes; selected-image clones own independent storage and publication ownership. - Add the complete generic NodeBrowser continuation-dependency capability. This does not implement server browse-history retention, NodeManager read images, or server batch/readiness admission. - Add optional client binding capture for SessionClient, native Session and ManagedSession, with managed-lease/native-channel generation composition. Bindings capture authenticated-session identity, mapping holders/versions and owned endpoint/policy/configuration observations. Replacement, reconnect, mapping ABA and disposal permanently invalidate stale bindings; disposing a binding does not dispose its owner. - Preserve cancellation and master pooled-response ownership: a rejected pooled response is released exactly once; an accepted response remains caller-owned. Keep native transport submission under its replacement gate and completion outside the gate, without routing ordinary unbound requests through binding machinery. - Rename the existing server lookup interface and all actual consumers with minimal hunks; add focused fixtures, generated FileTypeClient usage guidance, and package/transport documentation. **Breaking preview type identity:** the existing 2.0-preview server lookup `Opc.Ua.ISessionBindingProvider` becomes `Opc.Ua.IServerSessionBindingProvider`. Server lookup consumers must update their referenced type and rebuild. This is neither source-compatible nor binary-additive for that preview type. The new client capability is optional and adds no mandatory members to existing ISession/ISessionClient/ITransportChannel contracts. This preview-to-preview rename is documented in SessionBindings/Transports, not in the 1.5-to-2.0 MigrationGuide. **Ownership boundary:** no RequestLifetime changes, certificate-manager/channel-entry adoption, BaseVariableState changes, Method-result feature extraction, NodeManager batch/readiness or browse-history retention, session restoration, registry federation, WoT conversion, or WoT runtime/planners/events. DependencyInjection/ResourceIsolation/index changes are lookup/link closure only. SessionManager's PR hunk changes only its implemented lookup-interface name; newer master public diagnostic masking is not replaced. ## Related Issues _Reference all GitHub issues this PR addresses. If there is no issue yet, open one and link it here._ _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ - Part of #4511: user-approved combined groups 5+6 Wave A extraction. This PR does not close the umbrella or publish Wave B. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. ### Review qualification Review fixes are published at **`6b0eb1d5c32a439300fbcaff974b9af8b909161b`**, changing eight already-owned files. All five received review threads were replied to and resolved after verification. Factory selector installation and target-selection-plus-commit share the root publication gate, with consistent root→image release/capture ordering. TypeTable selector installation and final mutation routing share its existing gate. Native bound channel validity uses volatile observations; the two owned XML defects are corrected. | Requirement | Evidence | |---|---| | Selector installation must not hide TypeTable mutations | `SelectorInstalledDuringNodePreparationReceivesMutation`: old Add routing failed on both TFMs; corrected routing passes on both within **148/148** factory/TypeTable controls | | Selector installation must not race factory commit routing | Static common-root-gate/order proof plus baseline-GREEN `BuilderCapturedBeforeOwnershipUsesSelectedFactory`, `ViewOwnerReleasePreservesRegistrationsAndAllowsReuse`, `ViewOwnerReleaseCanRetryAfterPendingPublication`, `PublicationProtectsPendingAndRetiredFactoryImages` | | Lock-free validity must observe owner transitions | Static volatile-read reasoning plus native `BoundRequestCannotDispatchWhileNativeReplacementIsPendingAsync`, `BoundResponseRejectsInvalidationAfterNativeDispatchAsync`, `BoundClientCannotRetryAcrossFaultedManagedEntryReplacementAsync`: **12/12 each TFM**; Core binding/pooled ownership controls **17/17 each TFM** | | Correct owned XML documentation | “an unmanaged” and one multiline `ReferenceUpdate.Next` summary; scoped whitespace/INFO and build evidence below | Factory controls are **not** claimed as a deterministic factory RED, and public transport controls are **not** claimed as a deterministic weak-memory RED. No private-lock reflection, artificial production seam, or source-name assertion was added. The TypeTable RED uses the real `ILocalNode.NodeClass` callback during node preparation, and both expected RED receipts/TRX were retained. All accepted review runs use the unchanged admitted SDK wrapper, normal project references, sole mutex, one node/no shared compiler, finite resource guards and canonical short-C build outputs. Source hashes were frozen per command and matched the committed bytes afterward. SDK ownership was released **before** commit/push/review replies. No automatic merge was requested. Scoped whitespace passed across the four affected projects. Review INFO classification is **0 owned findings / 13 untouched baseline findings excluded** (whole-file formatter exits 1,1,0,0); no baseline RCS1140/RCS0055 expansion or suppression. The final normal net10 public Client/Core/Types trim/AOT-analyzer closure passed **0 warnings / 0 errors**. This is not a NativeAOT publish/run. <details> <summary>Accepted review command payloads and results</summary> These exact argument payloads were executed through the unchanged guarded SDK wrapper. `$proof` expands to the protected results-directory below. Successful explicit builds of the same source preceded the `--no-build` runs. ```powershell $proof = 'C:\Users\mschier\.copilot\session-state\a3b74048-c52f-4c87-8773-7a46c9ab0087\files\qualification-review-4752' # Exit 0: 148 passed, 0 failed dotnet test tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj -f net48 -p:CustomTestTarget=net48 -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --no-restore --verbosity minimal --filter "FullyQualifiedName~EncodeableFactorySnapshotTests|FullyQualifiedName~TypeTableTests" --logger "trx;LogFileName=review-types-green-net48.trx" --results-directory $proof # Exit 0: 148 passed, 0 failed dotnet test tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj -f net10.0 --no-restore --no-build --filter "FullyQualifiedName~EncodeableFactorySnapshotTests|FullyQualifiedName~TypeTableTests" --logger "trx;LogFileName=review-types-green-net10.trx" --results-directory $proof -p:CustomTestTarget=net10.0 -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --verbosity minimal # The following two per-TFM payloads each exited 0: # Core: 17/17 net48 and 17/17 net10.0; Client: 12/12 net48 and 12/12 net10.0 foreach ($tfm in @('net48', 'net10.0')) { dotnet test tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -f $tfm --no-restore --no-build --filter "FullyQualifiedName~SessionBindingResponseTests|FullyQualifiedName~SessionBindingContextTests" --logger "trx;LogFileName=review-actual-core-green-$tfm.trx" --results-directory $proof -p:CustomTestTarget=$tfm -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --verbosity minimal dotnet test tests\Opc.Ua.Client.Tests\Opc.Ua.Client.Tests.csproj -f $tfm --no-restore --no-build --filter "FullyQualifiedName~BoundRequestCannotDispatchWhileNativeReplacementIsPendingAsync|FullyQualifiedName~BoundResponseRejectsInvalidationAfterNativeDispatchAsync|FullyQualifiedName~BoundClientCannotRetryAcrossFaultedManagedEntryReplacementAsync" --logger "trx;LogFileName=review-actual-client-green-$tfm.trx" --results-directory $proof -p:CustomTestTarget=$tfm -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --verbosity minimal } # Exit 0: 0 warnings, 0 errors dotnet build src\Opc.Ua.Client\Opc.Ua.Client.csproj -f net10.0 -p:CustomTestTarget=net10.0 -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --no-restore --verbosity minimal ``` The six expanded GREEN commands, exact results/TRX hashes and two expected RED commands are retained in `qualification-review-4752\qualification-summary.json`, SHA256 `C5E71A4D8D37BAC8803F6DF38B6BCE2B70F3D0AFD5DEC8493FAE9DB440DCFB5A`. Committed/qualified byte identity is in `publication-identity.json`, SHA256 `CD9515FE96703F8B241F2B7B9B71F73EF0FF2626C2C8BCCE40307971A8F8E809`. Per-command raw exits, guards, hashes and reports are retained outside disposable build roots. For each record in the protected `format-inputs.json` (Types/Core/Client libraries and Types.Tests), the recorded Project/Include values expand these exact payloads: ```powershell # Workspace properties recorded with the formatter receipts: $env:CustomTestTarget = 'net10.0' $env:UseArtifactsOutput = 'true' $env:ArtifactsPath = 'C:\cp56-a3b7-v2' # All four exit 0 dotnet format whitespace $Project --no-restore --verify-no-changes --verbosity minimal --include $Include # Exits 1,1,0,0 due only to untouched baseline findings; owned=0, baseline=13 dotnet format $Project --no-restore --severity info --verify-no-changes --verbosity minimal --report "$proof\formats\review-info-$i" --include $Include ``` Initial fresh matrix evidence remains separately identified and unchanged: Types274/Core119/Client151/server137 net10; Types274/Core119/Client151/server135 net48, all passing. Qualification base remains `0dff94bd378e79b8cf1a47e1aaf340f9b4b40048`; the initial published head was `9fee4583`. All 16 initial exact command arrays/results/TRX hashes remain in protected `qualification-v2\qualification-summary.json`, SHA256 `88E10240C20049B59A3401BE007C6528A85DF2F3DCFAF9D11D908CEF2946DA4E`. Failures/incidents remain retained rather than relabeled as success: original disappeared-root historical counts are invalidated; the large-TRX controller DOM-parser failure is preserved separately from accepted streaming receipts; the review argument-array defect/default-D-output controls are preserved but excluded from canonical review qualification. Correctly separated argument rechecks produced the accepted short-C Core/Client receipts. The sole SDK wrapper was not changed. Full UA.slnx, measured coverage increase, CLA verification, NativeAOT publication and current-head/master CI/CodeQL remain unclaimed. Actual merge—not resolved review threads—unlocks Wave B. </details> |
||
|
Marc Schier
|
84d6387a4f |
Make request cancellation own exactly one terminal state (#4744)
# Description `RequestLifetime` now publishes the winning cancellation status and terminal ownership in a single atomic state, so concurrent client Cancel, Session close and timeout can never overwrite each other's status or double-own the terminal transition. External token registrations are owned by the lifetime and released on completion, and disposal is deferred until running cancellation callbacks have finished (no use-after-dispose of the CancellationTokenSource). Adds regression tests: 16384-iteration race tests asserting status/owner/callback outcomes with a finite 2 min watchdog (`RequestLifetimeTests`, `RequestManagerTests`) and `WaitAsync` cancellation tests (`TaskWaitAsyncTests`; the polyfill change is only 2 XML doc lines, no runtime change). Files: `RequestLifetime.cs`, `System.Threading.Tasks.cs` (docs), and the three test files. `StandardServer.cs` is intentionally untouched (already on master). ## Related Issues - None. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Validation actually run (net10.0 and net48, all passed): Types.Tests `TaskWaitAsync` 23/23; Core.Tests `RequestLifetime` 16/16; Server.Tests `RequestManager` 56/56; `dotnet format whitespace --verify-no-changes` on owned files. Full UA.slnx not run. |
||
|
Marc Schier
|
184f19a2b0 |
Add complete method invocation results and resolved diagnostics (#4747)
# Description `MethodInvocationResult` carries the operation result, declaration-ordered outputs and optional position-preserving input results. Asynchronous callbacks retain the actual receiver, cancellation token and resolved diagnostic strings. Fluent predefined, typed, instance, reference-child, resolved and virtual method builders reject conflicting handlers in either registration order and never overwrite materialized handlers. Direct configuration remains available. Shared Call-result assembly retains input diagnostics, including useful text on Good input positions, and requested operation diagnostics on Good results. Diagnostic encoding preserves nested Good-status text, status information bits and the outer-inclusive depth limit. Request admission clears client-supplied internal diagnostic permission flags before early returns or validation failures; admitted sessions grant the flag only after the existing administrator-role checks. Includes corresponding unit/integration regressions, complete XML API documentation, and method-result guidance consolidated with the fluent APIs in `docs/NodeManagers.md`. No WoT receiver/projection/remote-execution, event-readiness/native-admission, or bulk/history implementation is included. ## Related Issues Part of the user-approved two-wave extraction plan tracked by #4511. _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ - Related to #4511. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. |
||
|
romanett
|
8b360da7d2 |
Server publish path: no deep clone per Publish, pooled notifications that are actually reused, ready list instead of an O(N) scan (#4748)
Roadmap item A5 of the o6 benchmark gap analysis. Every Publish response deep-cloned its NotificationMessage in `SentMessageQueue` (PR #4707 kept the clone because a retained message returns its pooled payload when the client acknowledges it, which can happen before the response was sent). This PR replaces the clone with a send-completion lease, makes the activator pool that recycles the notifications O(1) and sizes it for the server's load, builds messages without intermediate copies (the notification array is rented), and lets monitored items tell their subscription when they have something to report so the publish timer no longer walks every item. ## Results Paired A/B against master `0dff94bd3` (includes #4742 and #4743; alternating builds, 3 rounds, medians; 20 sessions unless noted, workstation GC, 24-core machine shared with other sessions). Server = the reference server child process. | Scenario | Server alloc / op | Server GC pause | Server CPU / op | Write→notify p50 / p99 | Throughput | |---|---|---|---|---|---| | `sub`, 5 subs × 100 items, pub 100 ms | **−57 %** (373 → 159 B) | **−57 %** | −7 % ¹ | −1 % / −2 % | +1 % ² | | `sub`, 5 subs × 100 items, pub 10 ms | **−57 %** (375 → 163 B) | **−52 %** | +9 % ¹ | −22 % / +1 % | ±0 ² | | `sub`, pub 100 ms, SignAndEncrypt | **−57 %** (380 → 163 B) | **−53 %** | +4 % ¹ | −11 % / **−12 %** | ±0 ² | | `sub`, 2 sessions × 1 sub × 1000 items, 20 changing, pub 50 ms (9 rounds) | −2 % ³ | ±0 | −23 % ³ | ±0 / −2 % | ±0 | | `counters`, 4000 items, 10 ms sampling, pub 1 s (per notification) | −2 % | −6 % | −6 % | – | ±0 | | `o6counters`, 2048 items (per sample) | −3 % ⁴ (111 → 108 B) | −5 % | −7 % | – | ±0, all PASS | | `o6counters`, 4096 items (per sample) | −3 % ⁴ | +14 % ⁵ | −6 % | – | ±0, all PASS | ¹ CPU per notification varies by ±15 % between rounds on this machine; in an earlier 8-round batch against `071660354` the same scenarios measured −14 % / −1 %. Read it as "no regression". Client CPU per notification moved −18 %…0 %, within its spread. ² Bounded by the writer rate. ³ The server is nearly idle (0.05–0.12 cores, 360 notifications/s; its allocation is the writer and keep-alives). The first 3 rounds measured +31 % CPU, 6 more rounds −23 % (medians 969 → 742 ms per window): noise-bound, no regression. ⁴ With #4743's sampling permission cache a sample costs 111 B; one Publish per second carries 2048/4096 notifications. ⁵ The activator pools no longer allocate their 1023-slot arrays up front; the heap is a few MB smaller, which shifts the GC's generation budgets so that some low-allocation runs make every collection gen1 (26 of 26 instead of 2). Pause totals differ by milliseconds per 10 s window (37 → 42 ms here, 21 → 20 ms at 2048 items). Allocating the arrays eagerly restored master's gen1 counts in a separate test. In a single CPU trace of `sub` at pub 10 ms, recycling acknowledged payloads (pool `Return`) went from 16 % of server on-CPU time (master, the full pool scanned per return) to 5 %. ## Changes - **Send completion instead of a deep clone** (`NotificationMessageLease`, `LeasedPublishResponse`, `SentMessageQueue`): the retransmission queue and every Publish response that carries a message each own a reference. The transports already call `IPooledEncodeable.Reuse()` on a response once it is encoded or dropped; the Publish response is now a `LeasedPublishResponse` whose `Reuse()` releases its reference. The pooled payload (`NotificationMessage`, `DataChangeNotification`/`EventNotificationList`, `MonitoredItemNotification`/`EventFieldList`) is recycled when the last owner releases it, so an acknowledgement that overtakes the send no longer recycles a payload that is still being encoded, and the queue and the response share one message. A reference that is never released only leaves the payload to the GC; every owner releases at most once. Keep-alive messages are owned by their response alone. Republish, the durable snapshot and the retransmission mirror still copy. - **Transports**: the HTTPS (binary and JSON), WebSocket (UA-SC, JSON, OpenAPI) and Kestrel TCP listeners and the WebAPI endpoint dispatcher now release responses after encoding, as the TCP listener does, so leased payloads are recycled there too. A response retained for a reconnect is released when it is sent; a send that throws leaves the response to the GC. - **Activator pool** (`BoundedObjectPool`): rent and return scanned the 1023 slots linearly; with the pool full every returned notification scanned all of them before being dropped (16 % of server CPU at 94k notifications/s). It is now a `ConcurrentQueue` behind the lock-free first slot with an `Interlocked` bound (a locked stack spun under contention when transport threads return whole messages). The queue is created on first use, so activators of types that are never recycled no longer allocate a 1023-slot array each. The client decoder rents through the same pool. - **Pool bound** (`PooledEncodeableType<T>.EnsureMaximumRetained`, internal): the bound can be raised, never lowered. The subscriptions raise the `MonitoredItemNotification` and `EventFieldList` pools to twice the number of monitored items in the process (a notification is recycled about two publishing cycles after it was built), between the default 1024 and 65536. At the cap the retained notifications take about 5 MB; the bound stays raised after items are deleted. - **Message construction** (`Subscription.InnerPublish`/`ConstructMessage`, `MonitoredItem.Publish`): the per-Publish `Queue<>`/`List<>` objects are reused by the subscription; messages are built at their final size (no `List`→`ArrayOf` copy, no `AddItem` reallocation); the `MonitoredItemNotification[]` of a message is rented from `ArrayPool` and the message carries an exact-length slice of it; the lease returns the array (cleared) with the payload, and messages dropped on queue overflow return theirs at once; only arrays the subscription rented are tracked, so restored, mirrored and copied messages never reach the array pool; the `DiagnosticInfo` list is only allocated when a notification has one; timestamps are stripped on the struct before the pooled notification is filled. - **Ready list** (`IReadySignalingMonitoredItem`, `Subscription.PromoteReadyMonitoredItems`): `MonitoredItem` acquires a ready signal (compare-exchange, a full fence after its flags are set) when it queues a value or event or a resend is requested, and hands itself to the subscription's intake list. The publish timer checks only these candidates instead of every item under the subscription lock. An item stays a candidate while it has something pending that can become reportable without a new value (sampling deadline, aggregate interval, samples of a SAMPLING item, trigger flag); after releasing the signal the subscription checks again, so a value queued concurrently is never missed. Created, restored, modified, monitoring-mode and triggering changes mark candidates, as does an item that leaves the publish list with something pending. ### Behaviour notes (custom subscription or monitored-item extensions) - **Publish responses share the retained message.** `PublishResponse.NotificationMessage` returned by `SubscriptionManager.PublishAsync` is the retained message itself, not a copy, and its payload objects return to the activator pools after the response was sent and acknowledged. Code that keeps or modifies the message of a Publish response (for example in a `StandardServer.PublishAsync` override or a request interceptor) must copy it first (`CoreUtils.Clone`). `Republish` still returns a copy. - **Custom transports**: a transport that does not call `Reuse()` on responses keeps working; the payload is then collected by the GC instead of being pooled. - **Custom `IMonitoredItem` implementations** (not derived from `MonitoredItem`) cannot signal readiness: a subscription that contains one scans all its items on every publishing cycle, as before. `MonitoredItem` subclasses signal through the base class. - **`DataChangeNotification.MonitoredItems` of a published message is a slice of a pooled array**: read it through `ArrayOf<T>` (`Span`, indexer, `Count`); code that unwraps the underlying array would see a longer array. - **Order within a message**: notifications of different monitored items appear in the order the items became ready rather than in the internal list order. OPC UA does not define an order across items; the order of the values of one item is unchanged. ## Harness (`perf/`) - `--write-nodes K` for `sub`: the writer changes only the first K monitored nodes, for large subscriptions in which few items change. - `server-scenarios.txt`: publish-path scenarios (`--pub 10`, 1000 items per subscription with 20 changing). ## Testing - New `NotificationEncodingLifetimeRegressionTests` cases: an acknowledgement before send completion does not recycle the payload still being encoded (the send completion does); Republish after the send completed returns the original content and stays independent of the recycled original; a failed send that releases the response twice, plus a copy of the response, recycles the payload exactly once and only after the queue released it; a keep-alive is owned by its response; the rented notification array stays intact until both acknowledgement and send completion (either order), the message encodes only its slice, and queue overflow returns the arrays of dropped messages. The existing 30-case matrix (publish / queued / republish / mirrors × acknowledge / evict / clear × data / events) now asserts that a shared payload survives every queue mutation until its response is released. - New `ReadyListRegressionTests`: the same scenarios on a subscription that uses the ready list and on one that scans (it also has an item that cannot signal) must publish the same notifications, for triggering (sampling items, backlog of the triggering item), monitoring-mode changes (Sampling↔Reporting, Disabled, disabled before publish), disabled publishing with keep-alives and ResendData, a value queued while the item was in the publish list, and the backlog of an item that left Reporting while in the publish list. Mutation check: removing the item's signal fails 5 of 6, removing the subscription-side marks fails the backlog case. Plus: a published item without pending values is not tracked; creating 1200 items raises the notification pool bounds to at least 2400. - `PooledEncodeableTypeTests`: the raised bound is respected and never lowered. - net10.0 on the rebased branch: Server 8481 passed (5 skipped); Sessions 851 (379 skipped); Client 2788; Subscriptions.Classic 57; Subscriptions.Durable 36 (11 skipped); Redundancy.Server 884, Redundancy.Client 134, Redundancy.Samples 17; Stress Contract+Integration 45; Subscriptions 657 (6 skipped); Types pool tests 165. All 0 failed. - net48 build of `Opc.Ua.Server.Tests` and `Opc.Ua.Bindings.Https` and `Opc.Ua.Types.Tests` clean; `dotnet format whitespace` clean. ## Not changed (follow-ups) - Republish still copies the message (`ISubscriptionManager.Republish` returns a `NotificationMessage`, so the response cannot carry a lease without an API change). - `EventFieldList[]` per event message could be rented like the data change array. - Sampling remains the larger share of allocation and CPU in the `counters`/`o6counters` scenarios. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
39dd5632e3 |
Fix cache invalidation after accepted async indexed writes (#4746)
# Description After the full async write hook accepts an indexed update, merge the slice into the whole cached value when possible. If the cache is missing or stale and cannot absorb the slice, clear the cached value, set its status to `BadWaitingForInitialData`, and reset its timestamp to `DateTimeUtc.MinValue`. Preserve the handler's successful operation result, including `GoodClamped`, and report a value change. Never expose the replacement slice or stale data as the whole successfully updated cache. Synchronous fallback and simple-hook behavior are unchanged. The existing fixture remains intact, with its unusable-cache expectations updated for this intentional behavior correction. Fourteen additional cases cover successful merges, missing/stale cache invalidation, Good/GoodClamped operation results independently of cached status, handler rejection, type/rank validation, node/user access, and unsupported status/source/server timestamps. XML documentation explains the cache-versus-operation contract. Scoped analyzer cleanup uses asynchronous fixture helpers without removing the cancellation, lock-release, access, type, status, timestamp, or change-mask oracles. **Validation:** final head `aaf78b8df1bed46fb14033f3f3c9f52945469000` passed 32/32 cases on actual net10.0 and 32/32 on actual .NET Framework 4.8, with zero failures or skips. Both owned fixtures and the owned production file passed formatting/analyzer verification at info severity. The normal-reference .NET Standard 2.0 Types build passed with zero warnings/errors. The full `UA.slnx` suite and numerical coverage collection were not run locally; no full-suite or coverage-percentage claim is made. ## Related Issues Related to [#4511](https://github.com/OPCFoundation/UA-.NETStandard/pull/4511), the approved Wave A extraction plan. This standalone master PR does not close the tracking PR or imply completion of the remaining extraction groups. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
Marc Schier
|
4fd4535362 |
Fix managed-channel certificate ownership on early create failure (#4745)
# Description `CreateChannelAsync` now releases transferred certificate/chain handles when disposal or argument validation fails before transport creation. Managed entries use the existing inherited channel-creation method directly and share entry-retained, idempotently disposed handles with the transport, without adding another reference for non-adopting transports. The ownership contract is documented on the existing method; no internal redeclaration or forwarding implementation is needed. New and changed ownership APIs have multiline XML summaries and parameter documentation. Fourteen regression cases use generated certificates persisted in an isolated directory store and resolved through the real `CertificateManager.CertificateProvider`. They sign and verify SHA-256 digests to prove private-key usability through failed open/retry, publication/rotation, forward/reverse connection, and shutdown, and probe each certificate core's final retirement independently of global counters. Both adopting and non-adopting transports and optional chains are covered. Temporary stores are cleaned up. ## Related Issues Related to #4511: user-approved independent Wave A group 2 extraction. This PR does not close the overall tracking issue or include the other extraction groups. _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Tests were added and verified as described above; the coverage checklist remains unchecked because coverage was not measured. |
||
|
romanett
|
0dff94bd37 |
Server quick wins from the o6 benchmark gap analysis: admission sizing, sampling permission cache, routed Read dispatch (#4743)
The o6-automation benchmark gap analysis listed several server quick
wins that need no wire or public API change. This PR implements three of
them. The 2.0 server now passes the benchmark's 10 clients × 1000
outstanding Reads test with the benchmark's own queue sizing. Its 100 Hz
sampling capacity grows about 7–10× on the same machine, and a
single-value Read allocates 8 % less.
## Results
Paired A/B against `master` (
|
||
|
romanett
|
25ebf5a4d8 |
Server: remove the server-wide locks from the request pipeline, bounded transport buffer pool (#4742)
# Server: remove the server-wide locks from the request pipeline,
bounded transport buffer pool
Quick wins #5 and #6 of the o6 benchmark gap analysis. Under 10 channels
× 100 outstanding single-value Reads on 24 cores, about 59 % of the
server's on-CPU time spun in `Monitor`/`Lock` slow paths, and roughly 20
of the ~30 lock acquisitions per Read were on server-wide locks in the
request pipeline. Separately, on hosts with one or two processors every
response allocated a fresh 64 KiB chunk, because
`ArrayPool<byte>.Shared` runs out with about a thousand responses in
flight.
No wire or configuration changes. The only public API change is an
additive `RequestLifetime` constructor. The admission *sizing*
(`MaxQueuedRequestCount` → shared capacity and reserved floors) is
untouched; this PR only changes how the accounting is locked.
## Results
Paired A/B against `origin/master` (
|
||
|
Marc Schier
|
1bb4bdca1d |
Add Machine Vision Parts 1 and 2 (#4733)
# Description Implement the released [OPC 40100-1](https://reference.opcfoundation.org/specs/OPC-40100-1) and [OPC 40100-2](https://reference.opcfoundation.org/specs/OPC-40100-2) specifications with generated models, provider-backed runtimes, typed clients, MCP workflows, and DI/Machinery integration. - **Part 1 stays in `Opc.Ua.Vision`** and its client/server packages alongside the distinct Perception model. Provide control/acquisition state machines, jobs and synchronized steps, recipe/product/configuration/result management, file transfers, diagnostics, events, and conditions. - **Part 2 is in `Opc.Ua.MachineVision`, `.Client`, and `.Server`.** Provide asset categories, identification, image-handling aspects, health/maintenance data, topology, and transactional updates. The base Vision assemblies do not reference these optional packages; the Part 2 wire model does not depend on Part 1. - Preserve the release-pinned model bytes, wire namespaces and NodeIds. Perception types are in `Opc.Ua.Vision.Perception`; the AMCM generated namespace remains `Opc.Ua.Vision.MachineVision.AMCM`, and asset client/server extension namespaces remain `Opc.Ua.Vision.Client` and `Opc.Ua.Vision.Server`. - Use `AddOpcUaMachineVision` and `AddOpcUaMachineVisionAMCM` for model loading and factory registration. Each has node-collection, encodeable-factory-builder and node-state-factory-builder overloads. These short entry points delegate to the generated implementations; existing namespace-derived generated names remain available for compatibility. - Support multiple independent named systems/assets under one namespace-owning manager. Reuse injectable providers, file binding, bounded memory stores, optional transactional `IFileSystemProvider` persistence, state machines, telemetry and conformance infrastructure. - Keep one `docs/Vision.md` guide, titled `Machine Vision (OPC 40100-1 and OPC 40100-2)`, and one `Opc.Ua.Mcp.Vision` package. Include deterministic server/client samples. - Expose 17 Machine Vision MCP tools, including capabilities/status, bounded relationships and observation, complete small-file uploads/downloads, management, explicit control, conditions, and chunked transfers. Catalogue counts: 39 Vision-specific, 43 with connection tools, 81 Vision+Robotics, 155 Full. Every Machine Vision wire parameter has a schema description. **Compatibility and limits:** `AddVision()` remains Perception-only unless released parts are selected. Standalone Part 2 uses `AddVision().AddMachineVision()`; Machinery composition uses `AddVisionToMachinery()`. Existing tool defaults and command signatures are preserved. Observation owns and cleans up its subscription: 5 seconds/50 notifications by default, capped at 60 seconds/100 notifications. Complete-file tools are capped at 1 MiB; larger files use chunked transfers. Domain Error values remain distinct from UA service failures. **Versioning:** the seven Vision/MachineVision/MCP packages and their Debug variants follow the repository root version instead of being forced to preview. Vision.OpenUsd and its descendants remain preview-only. Development builds still inherit the root prerelease version. No release version, tag, release branch, publication policy, or feed promotion is changed. **Review actions:** all fourteen review threads have action/evidence replies and are resolved. Changes include the unified guide, Perception namespace, separate Part 2 packages and final naming, multi-system ownership explanation, interrupted-write cleanup, observer failure isolation, complete MCP parameter descriptions, and explicit package inventory. The preview-layout migration topic and the requested ReleaseProcess caveat were removed. The test Publish collector now retries recognized transient failures within one cancellation budget and propagates other exceptions. Commit |
||
|
romanett
|
0716603543 |
Fix concurrent session creation on a shared ClientChannelManager channel (#4739)
# Description Sessions created concurrently through one `ClientChannelManager` against the same endpoint (`Session.CreateAsync(IClientChannelManager, ...)`) failed with `BadSecureChannelClosed 'The managed transport is closed.'`, and the log showed `Background operation OpenChannel started by ClientChannelManager failed. Channel is Closed.` Creating the same sessions one after another worked. ## Root cause A caller that finds an existing shared entry gets a lease right away, even while the entry's creator is still opening the transport: 1. That session's `OpenAsync` called `lease.Entry.SnapshotClientCertificate()` before any transport was installed and threw *"The managed transport is closed."* 2. The failed session disposed its lease. The refcount dropped to 0 and the entry was torn down. 3. The creator's in-flight `OpenInitialAsync` then found the entry closing and threw *"Channel is Closed."* Every session attached to that entry failed. ## Changes - **`Session.OpenAsync`** waits until the shared entry has an installed transport (new `ChannelEntry.WaitForTransportAsync`) before it reads the certificate. The signal fires when the initial open installs a transport, or when a reconnect that started during the initial open does. It does not wait for `Ready`: `OpenCoreAsync` also runs for session recreate inside a reconnect cycle, and the cycle installs the transport before participants reactivate. - **`ClientChannelManager.GetCoreAsync`**: the creator holds no lease while it opens the entry, so other participants' releases can close the entry under the open. The creator now retries on a fresh entry instead of failing. When the initial open fails on its own, the entry records it (`InitialOpenFaulted`, set under the entry lock at failure time, because the later lifecycle state cannot tell the two cases apart). A faulted open, or one closed by manager disposal, still fails the creator. ## Tests - `SharedChannelConcurrentSessionTests.SessionAttachedWhileChannelOpensWaitsForTransportAsync` (Client.Tests): holds the creator's transport through gated channel bindings until a second session has attached to the opening entry, then checks that both sessions open and share the channel. Fails deterministically without the fix. - `ClientChannelManagerManagedTests.WaitForTransportWaitsForReconnectStartedDuringInitialOpenAsync` and `CreatorDoesNotRetryInitialOpenThatFaultedAsync` (Core.Tests): cover the reconnect-during-open wait and the no-retry-on-fault path. - `SharedChannelConcurrentSessionTests.ConcurrentlyCreatedSessionsShareOneChannelAsync` (Client.Tests): creates 20 sessions concurrently through one manager against the reference server. It checks that they share one channel with refcount 20 and that every session can read. Fails without the fix. - `ClientChannelManagerManagedTests.CreatorRetriesWhenLeaseReleasedDuringInitialOpenAsync` (Core.Tests): holds the initial open with a mock transport, then attaches and releases a second lease during it. It checks that the creator gets a lease on a fresh entry. Fails without the fix with "Channel is Closed." Verification: - **Local runs (net10.0, sequential):** Opc.Ua.Client.Tests 2778 passed; Opc.Ua.Sessions.Tests 851 passed, 379 skipped; Opc.Ua.Core.Tests 6241 passed, 88 skipped. - **net48:** Client.Tests and Core.Tests compile cleanly. - **ServerLoadHarness repro:** `--client shared --sessions 20 --connect-parallel 4/20` now connects every run. The same build without the fix still fails. Note: when the creator's open is cut short this way, the `OpenChannel` background failure is still logged at `fail` level even though the creator now recovers. ## Related Issues - None filed yet. Found while load-testing the shared-channel client with the ServerLoadHarness. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
f5062be349 |
Complete each pooled service response at most once (#4729) (#4736)
## Proposed changes Fixes the cross-talk part of #4729 on master. `ServiceResponsePooledValueTaskSource` comes from one pool shared by the whole process, and `ManualResetValueTaskSourceCore.SetResult` never checks the version token. A second completion of an `EndpointIncomingRequest` therefore either threw `InvalidOperationException` or, once the source had been read and recycled, answered whichever request rented it next. That request could belong to another server instance in the same process. master's request queue (Channel + `FairRequestQueue`) already completes each request exactly once. But `ServerBase.ScheduleIncomingRequest` and `ProcessRequestAsync` are virtual, so a single override that completes and then throws (the `ProcessAsync` catch completes again) was enough to corrupt unrelated requests. - Each rental of the pooled source now gets a lease number, which is never reused for a later rental of the same instance. - `EndpointIncomingRequest` captures the lease when it is constructed and completes through `TrySetResult(lease, response)`. That is a single CAS on the pending lease, so only the first completion of the current rental is accepted. Late or stale completions are discarded and can never reach the next owner. - `SetResult`/`SetException`, which had no callers outside this path, are replaced by `TrySetResult`. The master378 backport #4738 also fixes the double completions in the old `RequestQueue` itself (snapshot in `Dispose`, and enqueue followed by a throw). ## Tests - `ServiceResponsePooledValueTaskSourceTests`: a second completion of the same rental is ignored, and a late completion after the source is recycled does not complete the next rental. Both use a private pool, so the result is deterministic. - `EndpointIncomingRequestTests.ProcessAsync_ScheduleCompletesThenThrows_KeepsFirstCompletionAsync`: a scheduler that completes the request and then throws. Without the fix it fails with `InvalidOperationException`. - `Opc.Ua.Core.Tests` Stack.Server + Stack.Tcp (net10.0): 1871 passed, 0 failed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
2c431ad728 |
Client/transport: one recv per small chunk, ManagedSession lock fast path, write coalescing (#4737)
Follow-ups from the "Not changed" list of #4720 (client per-request allocation and CPU): three changes that the paired A/B shows a gain for, plus a write-up of the fourth (discarded `ResponseHeader`), which is not changed. ## Results Paired A/B, each change against the commit before it: 4 alternating rounds, medians, 20 sessions, 24-core machine shared with other sessions. Client = the harness process, server = the reference server child. Allocated bytes per operation are deterministic; throughput and latency carry the run-to-run noise noted below. | Change | Scenario | Throughput | p50 | p90 | Client CPU / op | Client alloc / op | Server CPU / op | |---|---|---|---|---|---|---|---| | Receive staging | Read, 1 value | +4% | **−9%** | **−13%** | ≈ | ±0 | ≈ | | | Read, 1 value, SignAndEncrypt | +6% | −6% | −11% | ≈ | ±0 | ≈ | | | Read, 100 values | +2% | −4% | ≈ | ≈ | ±0 | ≈ | | | Data changes | ≈ | ≈ | ≈ | ≈ | ±0 | ≈ | | ManagedSession lock | Read, 1 value, ManagedSession | ≈ | ≈ | ≈ | ≈ | **−125 B** (2632 → 2507) | ≈ | | | Same, inflight 8 | ≈ | ≈ | ≈ | ≈ | −133 B | ≈ | | Write coalescing | Read, 1 value, 20 sessions on one shared channel, inflight 4 | **+13%** | **−13%** | −3% | −6% | ±0 | −4% | | | Same, inflight 1 | +20% ¹ | −8% | ¹ | −9% | ±0 | −13% | | | Read, 1 value, inflight 8 (one channel per session) | +2% | −4% | −6% | ≈ | ±0 | ≈ | | | Read, 100 values, inflight 8 | +3% | −4% | −8% | ≈ | ±0 | ≈ | | | Data changes | ≈ | ≈ | ≈ | ≈ | ±0 | ≈ | ≈: within run-to-run noise (about ±5% for CPU per op here). ¹ Rounds ranged from −42% to +32%; the inflight-4 runs were better in all four rounds. ## Changes - **`TcpByteTransport` receive**: `ReceiveChunkAsync` read the 8-byte chunk header and then the body, two `recv` calls per chunk. It now waits for data with a small staging buffer (8 KB, allocated on the first receive, not from the `BufferManager` budget). One receive returns the header and, when the chunk is small and already there, the whole chunk; bytes of following chunks are kept for the next call. A larger chunk is completed with a receive into the rented buffer, as before. - The receive buffer is still rented only after the header has been validated (message type, size against the negotiated limit), so an idle peer holds none of the budget. - A receive cancelled while its bytes are in the staging buffer keeps them for the next call; on .NET Framework the staging receive takes no token, so cancellation is observed once data arrives. - A receive that fails while reading the body of a chunk larger than the staging buffer has consumed part of the chunk; later receives then fail with `BadTcpInternalError` rather than read body bytes as a header. Before this PR such a failure lost the header and body bytes silently. The channel cancels receives only when it closes. - Server listener channels use the same transport. - **`ManagedSession` service calls**: each call was an async wrapper (await the reader lock, await the inner call), one more state machine box per request. - When services are available and no writer holds or waits for the lock, the reader lock is taken synchronously (`AsyncReaderWriterLock.TryEnterReader`). It is released by a cached continuation on the inner call's task, registered before the caller's await, so the lock is free again when the caller resumes. - Otherwise the call waits for the lock as before, so reconnect and failover still block new calls and drain the ones in flight. - The remaining 88 B per call is the task's continuation list (two continuations). - **Write coalescing** (`UaSCUaBinaryChannel`): when more writes for the same transport are queued behind the one being sent, the writer takes up to 16 of them and sends their segments in one vectored send. - Each write is then reported on its own, in order, with its size. A failed send is reported for every write in it, and their buffers are returned. - Writes still never start on the caller's stack, and one writer sends one batch at a time, so chunk order is unchanged. - Only byte-stream transports opt in (`TcpByteTransport`, also through the server's admission lease, via the internal `IUaSCChunkCoalescing`). WebSocket and capture decorators keep one send per chunk. ### Behaviour notes - Each TCP connection that has received holds an 8 KB staging array (pinned on .NET 5+), outside the `BufferManager` budget. The previous transport held an 8-byte header array. - `ManagedSession` service calls: if the inner call completes while the release continuation is being registered, the reader lock is released on the thread pool shortly after, instead of before the caller resumes. - Concurrent `ReceiveChunkAsync` calls on one `TcpByteTransport` now throw `InvalidOperationException`. The channel has a single receive loop, and interleaved receives on a stream were never valid. ## Not changed: discarded `ResponseHeader` Every generated structure's `Initialize()` constructs its structure-typed fields: `new ResponseHeader()` (112 B), `new RequestHeader()` (104 B). The decoders always create the instance (`new T()` or the activator) and then assign every field in `Decode`, so the default is thrown away. That is 4.6% of the client's 2.4 KB per Read and about 3% on the server, which discards a `RequestHeader` per decoded request and a `ResponseHeader` per response it constructs. `PublishResponse` also discards a default `NotificationMessage`. Options considered: - **Lazy getter** (`get => m_x ??= new X()`): `IsEqual`/`GetHashCode` compare fields, so a never-read default (null) would no longer equal an explicit default. Two threads reading an unset header could get different instances. Pooled instances reset through `Initialize()` anyway. - **Decode-only constructor** called by the activator: only the plain `Class`/`DerivedClass` templates assign every field in `Decode`, not optional-field or union types. A derived type's constructor must chain to a base built by an older generator in another assembly, which lacks it (the same problem `ForeignDerivedPooledExtensionClass` works around). It also needs a new public marker type in `Opc.Ua.Types`. - **Decode into the existing instance**: needs a new `IDecoder` member in all decoders. It also changes behaviour when a caller decodes into an object whose header it still references. Each touches every generated type for about 100 B per message, so this is left as is. ## Harness - `--client shared`: classic sessions whose secure channels come from one `ClientChannelManager`. - Created sequentially (`--connect-parallel`, default 1). Concurrent `Session.CreateAsync` calls on one manager fail with `BadSecureChannelClosed`, a separate bug. - 20 sessions on one channel reach about 45–70k req/s, against about 140k on 20 channels. - `ab.sh`: `CANDIDATE=` selects the candidate build. ## Testing - New: `TcpByteTransportTests`: chunks sent together, a chunk split across sends (inside the next header), a chunk larger than the staging buffer, cancellation keeping staged bytes. `UaSCBinaryChannelWriteQueueTests`: coalesced in order (1 + 16 + 16 + 7 sends), framing transport not coalesced, a failed batch reported for every write and later writes still sent. `AsyncReaderWriterLockTests`: `TryEnterReader` against a writer, the reader held until the operation completes and released on failure and on a completed operation, the awaiter resuming after the release. - Before the rebase onto master, net10.0, run sequentially: Core Stack+Security 4689 passed, 1 failed (`ClientBaseTests.ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`, a meter-listener flake fixed on master by #4708; passes 3/3 in isolation); Client 2768; Sessions 851 (379 skipped); Server 8398; Subscriptions 643. - After the rebase: Core Stack+Security 4811 passed; Client 2781; Stress Contract+Integration 45 passed; all 0 failed. Sessions, Server and Subscriptions were not rerun on the new base. - net48 build of `UA.slnx`: clean (0 warnings, 0 errors); `dotnet format whitespace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
b25429bccc |
Client: cut per-request allocation and CPU in the channel and session; measure the client in the load harness (#4720)
Follows #4707 (replaces #4714, which closed when the #4707 branch was deleted). Profiling the client process under load showed per-request allocation spread over the channel write path, the response wait, per-chunk header encoders and an AES instance per message, plus a stack walk per service call. This PR removes them and extends the load harness to measure the client. ## Results Paired A/B against the #4707 head, the server work now on master (3 alternating rounds, medians, 20 sessions unless noted, workstation GC on both sides, 24-core machine). Client = the harness process, server = the reference server child. | Scenario | Throughput | p50 | p90 | Client CPU / op | Client alloc / op | Server alloc / op | |---|---|---|---|---|---|---| | Read, 1 value | **+21%** (141k → 170k req/s) | −9% | −11% | −13% | −46% (4.5 → 2.4 KB) | −20% | | Read, 1 value, ManagedSession | **+18%** | −9% | −12% | −18% | −44% | −20% | | Read, 1 value, SignAndEncrypt | **+28%** | −14% | −20% | −24% | −52% | −26% | | Read, 1 value, 200 sessions | **+29%** | −11% | −17% | −17% | −46% | −20% | | Read, 100 values | +5% | −2% | −3% | −6% | −44% (15.6 → 8.8 KB) | −2% | | Read, 100 values, SignAndEncrypt | +8% | −7% | −2% | −2% | −46% | −3% | | Write, 100 values | ±0 | ≈ | ≈ | −14% | **−63%** (10.8 → 4.0 KB) | −2% | | Browse (Server object) | **+12%** | −7% | −11% | −19% | −17% | −12% | | Data changes, classic fast callback | ±0 ¹ | ≈ | ≈ | −21% | −10% | −5% | | Data changes, classic item events | ±0 ¹ | ≈ | ≈ | ≈ ² | −7% | −5% | | Data changes, ManagedSession (V2) | ±0 ¹ | ≈ | ≈ | ±0 | −5% | −5% | | Data changes, SignAndEncrypt | ±0 ¹ | ≈ | ≈ | −22% | −14% | −7% | ¹ Throughput is bounded by the writer rate; latency is dominated by the 100 ms publishing interval. Per notification the client uses about 2 µs and 170–260 B. ² The client uses 0.2–0.4 cores in the subscription scenarios; the CPU per notification is within run-to-run noise there. Client GC pause time in the window drops 38–66% in the Read and Write scenarios (17% for Browse). About 70% of the remaining client CPU per small request is kernel time in loopback socket send/receive. ## Changes - **Activity source**: `Session`, `SessionClientBatched`, the classic `Subscription` and `ClassicSubscriptionEngine`, and the TCP/HTTPS client channels resolve their `ActivitySource` once. `TelemetryExtensions.StartActivity` called `Assembly.GetCallingAssembly()` (a stack walk) on every service call, ~8% of client CPU. - **Channel write queue** (`UaSCUaBinaryChannel`, client and server): an ordered queue drained by one thread-pool writer replaces the per-write `ContinueWith`/`Unwrap`/trailing continuation chain. The writer is queued to the issuing thread's local pool queue, like the task continuation was (queuing it globally cost ~15 µs of p50 in a first iteration). Successful writes are reported inline instead of through `Task.Run`; failed writes still go through the pool. The client channel enters its gate only for a failed write instead of on every write completion. - **`TcpByteTransport`**: no linked `CancellationTokenSource` when the caller's token cannot be cancelled (the channel always passes `None`); a single-chunk `BufferCollection` uses the `ReadOnlyMemory` send instead of the vectored `Task<int>` send; `PoolingAsyncValueTaskMethodBuilder` on .NET 6+ for the receive/send state machines; no closure per vectored send. - **`BinaryEncoder.GetNodeIdEncoding`**: `Convert.ToUInt32(uint, IFormatProvider)` bound to the `object` overload and `Convert.ToByte(enum, ...)` boxed the enum, two boxes per encoded NodeId. 44% of the client allocation of a 100-value Read; the server encodes NodeIds the same way. - **Symmetric chunk headers**: the 24 header bytes are written and read with `BinaryPrimitives` instead of a `MemoryStream` + `BinaryEncoder`/`BinaryWriter` (send) and `BinaryDecoder`/`BinaryReader` (receive) per chunk. A chunk too short for a field still gets the decoder's `BadDecodingError`. - **AES-CBC**: `ChannelToken` keeps one `Aes` per direction, used with the one-shot `EncryptCbc`/`DecryptCbc` on .NET 6+ (net48 reuses the instance and creates the transform). Previously every message created an `Aes`, imported the key (`BCryptImportKey`) and created a transform, ~15% of client CPU with SignAndEncrypt. The instance is taken out of the token while in use, so concurrent callers never share one. The public `CryptoUtils` overloads are unchanged. - **`ChannelAsyncOperation`**: `SendRequestAsync` waits on an internal task-free completion source instead of `EndAsync` (`TaskCompletionSource` + `Task` + `WaitAsync` promise + state machine); `EndAsync` registers cancellation instead of `WaitAsync(ct)`; the timeout timer uses a static callback; completing with `default(T)` shares a box. - **Session keep alive**: every good response called `Timer.Change` under the session lock, taking the timer queue lock. It now records the time; the timer callback defers itself to one interval after the last response, which is when the per-response reset made it fire. ### Behaviour notes - `TcpByteTransport.SendChunkAsync`/`ReceiveChunkAsync` return pooled `ValueTask`s on .NET 6+: like any `ValueTask` they must be awaited once. - A successful write's `HandleWriteComplete` now runs on the writer instead of a separate pool work item. Overrides outside the stack that block in it delay later writes on that channel. - The write itself no longer flows the caller's `ExecutionContext` (`Activity.Current` is not set inside the transport send). ## Harness (`perf/`) - `client-process:` and `client-per-op:` result lines; `--trace-side client` records the harness itself. - `--client managed` (ManagedSession, V2 engine) and `--callback item` (classic per-item events). - Classic fixture sessions no longer request diagnostics in every response (the server otherwise sends a `DiagnosticInfo` per notification). - `ab.sh`: `SCENARIOS`/`ROUNDS`/`OUT`, CRLF tolerant, defaults before the scenario (a `--sessions 200` scenario silently ran with 20 before); `client-scenarios.txt`; `ab-summary.ps1` reports client and server medians. - README notes two profile traps: `Monitor.Enter_Slowpath` in the sampled CPU profile without matching contention events (46 in 10 s here), and allocation ticks over-attributing small objects allocated after large arrays. ## Testing - New: `UaSCBinaryChannelWriteQueueTests` (order, one send at a time, never on the caller's stack, failures reported and later writes still sent, concurrent writers), `ChannelAsyncOperationWaitTests` (late/early completion, faults, cancellation, disposal, continuation off the completing thread, 2000-iteration completion/cancellation race, timeout), `CryptoUtilsAesCacheTests` (cached = fresh cipher text across IVs, round trips, concurrent users, token disposal), `TcpByteTransportTests` single/multi-buffer and cancellable sends, `SessionKeepAliveTests` keep-alive read exactly one interval after the last response. - After merging master: Core Stack 3571 passed, Client 2763 passed, 0 failed. - Before merging master, net10.0: Core Stack+Security 4680 passed; Client 2763; Sessions 849 (379 skipped); Server 8391; Subscriptions 643; all 0 failed. - Stress Contract+Integration: 5 failures, the same 5 on the unchanged #4707 head (`CertRotationContractTests`, `GateAndBypassTests`, `LeaseLifecycleTests`, `RetryBudgetEnforcementTests` reflection seam, `CertRotationLiveTests`), not caused by this PR. - net48 build of `UA.slnx` clean; `dotnet format whitespace` clean. ## Not changed (follow-ups) - Two `recv` calls per chunk (8-byte header, then body). Merging them needs read-ahead with carry-over, which the transport avoids so that idle peers do not hold receive buffers. - `ManagedSession` service wrappers add an async state machine and a reader lock per call (~200 B). - Generated responses allocate a discarded `ResponseHeader` (generator-wide, as noted in #4707). - Decoded `MonitoredItemNotification` objects and the V2 `DataValueChange[]` per message dominate notification allocation (`WithPoolNotifications` pools the latter). - Coalescing queued writes on one channel into one vectored send would cut syscalls under concurrent requests on a shared channel. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc <marcschier@hotmail.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
romanett
|
c7fdeb6de1 |
Fix stress Contract/Integration failures and run them in stress-test.yml (#4718)
## Description `Opc.Ua.Stress.Tests` with `Category=Contract|Category=Integration` had five failures on master. The README says these categories run on every PR, but no workflow ran them: `get-ci-matrix.ps1` excludes the Stress project, and `stress-test.yml` only ran `ChaosTCP`. The old `Channels.Stress` special case in `buildandtest.yml` never matched the project name, and #4513 removed it. So the tests rotted unseen. Two of the five failures were real product bugs. ### Product fixes - **Session recreate after a server restart failed forever (regression from #4503).** User-token PolicyIds are assigned by the server and are not stable across a restart. In the failing run the client had cached `Anonymous|20` and the restarted server offered `Anonymous|2`. The CreateSession check added in #4503 requires an exact match, including PolicyId, so every `RecreateInPlace` failed with `BadSecurityChecksFailed` ("different security policy for the selected user identity token"). The managed session then retried without end. - `Session.OpenCoreAsync` now adopts the server's policy when only the PolicyId differs. TokenType, SecurityPolicyUri, IssuedTokenType and IssuerEndpointUrl must still match, so a downgrade is still rejected. The token is restamped with the new id. - **A certificate rotation during a reconnect reactivated every session twice.** `ChannelEntry.ReconnectCycleAsync` only noticed the superseded client certificate *after* it had notified participants. It then replaced the transport and notified them again. The check now also runs right after the transport reconnects, before participants are notified. ### Tests updated to the #4516 contracts - `RetryBudgetEnforcementTests`: `Session.ReconnectManagedChannelAsync` gained an optional reverse-connection parameter, which the reflection call must supply. - `GateAndBypassTests`: the participant callback token is cancelled once `OnReconnectAsync` returns. The spawned worker now runs on the test token. - `LeaseLifecycleTests`: releasing the last lease cancels the recovery in progress, as documented. The reconnect caller now expects `OperationCanceledException`. ### CI - New `Contract and Integration` job in `stress-test.yml`, with the same gate as Chaos TCP: nightly, manual, and PRs labelled `stress`. It takes about 1 minute of test time. - The README now says where these categories actually run. ### Tests - New `SessionIdentityReviewTests.CreateAdoptsRenumberedUserTokenPolicyIdAsync`, covering both a renumbered id (accepted) and a changed token security policy (rejected). It fails without the fix. - Each of the 5 fixed tests passed 3/3 runs in isolation. The full `Contract|Integration` filter passed 45/45 twice in Debug and once in Release. - Related tests also pass: Core.Tests channel tests 587/587, Sessions.Tests channel tests 119 passed (63 skipped), Client.Tests channel/session/reconnect tests 512/512. - The net48 build of `UA.slnx` is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Cristi Pogacean
|
9344354aaa |
Fix empty LocalizedText binary field encoding (#4709)
# Description Omit empty `Locale` and `Text` fields from the OPC UA Binary DataEncoding for `LocalizedText`, as required by OPC 10000-6, Section 5.2.2.14, Table 24. Add wire-format tests for empty and non-empty fields, including translation-backed localized text. Validation: - `git diff --check` passed before commit. - `dotnet build tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj --framework net10.0 --no-restore --verbosity minimal -p:NuGetAudit=false` could not complete because access to `https://api.nuget.org/v3/index.json` failed with `NU1301` and a TLS handshake error. ## Related Issues No related issue. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Cristi Pogacean
|
2cdf0de97f |
Fix scoped certificate-store resolution in GDS push operations (#4726)
# Description Fix the remaining GDS push certificate paths that bypass instance-scoped certificate stores, while preserving OPC UA 1.05.07 trust and transaction requirements. - Use the configured resolver/registry for previous-key loading, slot resolution, application-store commit/rollback, issuer import/compensation, and rejected-store opening. - Validate application-certificate uploads through the configured manager and the group's registered trust scope, instead of a temporary manager that cannot see custom providers. - Isolate per-call `AcceptError` validation cores from normal cached trust decisions. Part 12 installation-time acceptance must neither reuse nor populate the connection-validation cache. - Compensate the application-certificate swap if opening the issuer store fails, not only if importing an issuer fails. - Add scoped-provider, cross-group trust, cache-isolation, and compensation regressions. Reuse the portable PEM writer and short temporary PKI paths so the tests work on .NET Framework too. - Use source-generated failure logging without private-key bytes, and document custom-store/pending-key requirements. The branch starts at master `7f14fb864fc93532f3dbab1e1b5e2ba9b5f605c5`. It excludes the unrelated LocalizedText commits, the local handoff, PLC changes, logs, keys, certificates, and build outputs. No public API, dependency, package version, global provider registration, or analyzer configuration is changed. ### Specification review The online Part 12 reference identifies version **1.05.07**. This is a focused review and regression validation of the changed behavior, **not a claim of complete stack conformance or CTT certification**. | Requirement | Review / evidence | | --- | --- | | [Part 12, 7.10.5](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.5): validate application certificates using the associated TrustList; ignore the supplied issuer list | Existing issuer-argument regressions retained. New flat-layout scoped-provider cases accept a CA installed in the selected trusted/issuer store and reject one installed only in another scope. | | [Part 4, 6.1.3](https://reference.opcfoundation.org/specs/OPC-10000-4/6.1.3) and Part 12, 7.10.5: suppressible installation errors must not bypass normal peer trust or non-suppressible failures | Missing-issuer, signature/revocation, explicit-trust, and validation-core tests; new two-way cache-isolation cases with `UseValidatedCertificates = true`; normal peer validation remains untrusted after issuer-only push acceptance. | | Part 12, 7.10.5: matching private key, encrypted channel and SecurityAdmin | Existing access/key/type tests retained; scoped own-store upload cases cover no uploaded key and PEM. PFX and regenerated-key cases remain in the selected suite. | | [Part 12, 7.10.2](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.2), [7.10.9](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.9), and [7.10.11](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.11): staging, ApplyChanges, cancellation, ownership and compensation | Push/transaction/rollback suites; new custom-provider compensation cases cover both issuer-import and store-open failure, retaining the old application certificate and pre-existing issuers. | | [Part 12, 7.10.7](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.7): do not delete endpoint-referenced certificates | Existing ApplyChanges endpoint-reference and delete/create replacement tests retained. | | [Part 12, 7.10.10](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.10): CSR key handling and persistence | Existing-key scoped CSR test, SDK pending-key/cancel cases, and supporting PLC encrypted round trips/restarts. Arbitrary custom backends still require a durable pending-key implementation; the separate PLC implementation is not included here. | ### Validation Windows, .NET SDK 10.0.401, Release, NUnit/VSTest. Build analyzers remained enforced: **no `WarningsNotAsErrors`, suppressions, or analyzer downgrades**. `NuGetAudit=false` was needed because the local nuget.org TLS connection failed; no repository audit setting was changed. | Selection | Result | | --- | --- | | SDK GDS push/trust/transaction/rollback, net10.0 | **192 passed, 0 failed** | | SDK certificate-manager/provider/explicit-trust/validation-core, net10.0 | **180 passed, 0 failed** | | SDK certificate-manager/provider/explicit-trust/validation-core, net48 | **180 passed, 0 failed** | | SDK GDS push/trust/transaction/rollback, net48 | **192 passed, 0 failed** | | Final portable-PEM/compensation tests, net10.0 | **5 passed, 0 failed** | | Supporting PLC GDS/pending-key/compatibility selection, net10.0 | **29 passed, 0 failed** against this SDK source | The PLC tests exercise encrypted UA calls, signed PEM/CSR updates, persistence, fresh connections and restarts across Directory, FlatDirectory and KubernetesSecret providers. The Kubernetes API is an in-memory double, **not a real Kubernetes deployment**. PLC tests and its durable pending-key implementation are local companion-repository changes, not SDK PR contents or independently reproducible CI evidence from this branch alone. Commands below use the same isolated artifact locations as the validation run, expressed relative to the user profile: ```powershell $files = Join-Path $env:USERPROFILE '.copilot\session-state\003a7438-0312-4c10-beed-cd95c0fe400e\files' $sdk = Join-Path $files 'sdk-build' $net48 = Join-Path $files 'solution-net48' $push = 'FullyQualifiedName~ConfigurationNodeManagerPushTests|FullyQualifiedName~ConfigurationNodeManagerPushValidationTests|FullyQualifiedName~PushComplianceRegressionTests|FullyQualifiedName~PushRollbackCancellationRegressionTests|FullyQualifiedName~PushConfigurationTransactionCoordinatorTests' $manager = 'FullyQualifiedName~CertificateManagerTests|FullyQualifiedName~CertificateManagerExplicitTrustTests|FullyQualifiedName~CertificateManagerProviderResolutionRegressionTests|FullyQualifiedName~CertificateValidationCoreTests' # Passed: 192/192. dotnet test .\tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -f net10.0 -m:1 --no-restore -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" -p:NuGetAudit=false --filter $push # Passed: 180/180. dotnet test .\tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -c Release -f net10.0 --no-build --no-restore -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" --filter $manager # Passed: 180/180. dotnet test .\tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -c Release -f net48 --no-build --no-restore -p:CustomTestTarget=net48 -p:UseArtifactsOutput=true "-p:ArtifactsPath=$net48" --filter $manager # Passed: 192/192 after the portable PEM writer and shorter temporary PKI paths. dotnet test .\tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -f net48 -m:1 --no-restore -p:CustomTestTarget=net48 -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$net48" -p:NuGetAudit=false --filter $push # Passed: 5/5 after final test portability changes. dotnet test .\tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -f net10.0 -m:1 --no-restore -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" -p:NuGetAudit=false --filter 'FullyQualifiedName~CertificatePushUsesScopedOwnStoreProviderAsync|FullyQualifiedName~ApplyCertificateSlotChangeSelfCompensatesWhenIssuerImportFailsAfterAppCertSwapAsync' # Passed: 29/29; the companion project and this SDK had been built together first. dotnet test C:\src\iot-edge-opc-plc\tests\opc-plc-tests.csproj -c Release --no-build --no-restore -p:UseLocalOpcUaStack=true -p:OpcUaSourceRoot=C:\src\UA-.NETStandard -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" --filter 'FullyQualifiedName~GdsPush|FullyQualifiedName~PlcPendingCertificateKeyStoreTests|FullyQualifiedName~PreviewCompatibilityTests' ``` The `--no-restore` / `--no-build` invocations require prior restoration/build. The initially empty artifact-directory `--no-restore` invocation produced no test execution and is **not** counted. Initial compile/test portability failures were fixed before the final results above. Full-solution attempts did **not** pass: ```powershell # Failed/incomplete: AMB server startup timed out in two tests; stopped after failures. dotnet test .\UA.slnx -c Release -f net48 -m:1 -p:CustomTestTarget=net48 -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$net48" -p:NuGetAudit=false # Failed: NU1301 restoring OpenUsd-related projects due to nuget.org TLS handshake failure. dotnet test .\UA.slnx -c Release -f net10.0 -m:1 -p:CustomTestTarget=net10.0 -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$(Join-Path $files 'solution-net10')" -p:NuGetAudit=false ``` A retry through an already configured local package source progressed further, but the OneFuzz nested publish did not inherit the audit override and caused **210 setup failures (NU1900)**; the broad run was stopped. These runs are not a clean full-suite result. AMB startup failures have not been baseline-qualified and are not claimed to be fixed by this PR. `git diff --check` passed. A changed-file whitespace check reported seven pre-existing diagnostics at lines 371-424 of `CertificateManagerTests.cs`, outside this change (new tests start at line 563); no unrelated formatting sweep was made. **Remaining gates:** successful full solution on net48/net10.0, quantitative coverage, Linux, required CI/CodeQL, and applicable CTT/interoperability qualification. Keep this PR draft until the required gates are resolved; the tests above do not establish 100% OPC UA compliance. ## Related Issues _Reference all GitHub issues this PR addresses. If there is no issue yet, open one and link it here._ _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ - Fixes #4725. - Follow-up to #4546, #4530 and #4531 for remaining certificate-push consumers. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Tests were added and pass as recorded, but coverage was not measured. CLA status is not asserted on behalf of the contributor. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
7f14fb864f |
Secure the UADP DataSetMetaData announcement on secured connections (#4712)
# Description `MetaDataPublisher` encoded the DataSetMetaData announcement itself and called `IPubSubTransport.SendAsync` directly. That bypassed the connection's security wrapper and its fail-closed check. A WriterGroup configured for `SignAndEncrypt` therefore announced its DataSet layout (field names, types, version) unsigned and unencrypted. A secured subscriber dropped the announcement (`Dropping unsecured inbound frame ... requiring SignAndEncrypt`, documented as an expected warning in docs/PubSub.md), and any listener could read it. OPC 10000-14 §6.2.5.2 applies the WriterGroup's SecurityMode to the NetworkMessages it publishes, and §7.2.4.6.3 carries the security of a discovery announcement in its SecurityHeader. Confirmed on the wire: the documented `udp-uadp` exchange sent secured DataSetMessages plus one DiscoveryResponse (announcement type 2, DataSetMetaData) with ExtendedFlags1 `0x81`, meaning the SecurityHeader bit was clear. Review feedback widened the scope to every UADP discovery message of a secured connection, and to the diagnostics of secured messages. Fix: - `MetaDataPublisher` builds the announcement and hands it to the connection (`PubSubConnection.SendMetaDataAnnouncementAsync`). - One encode-and-secure step (`PubSubConnection.EncodeOutboundAsync`) now serves data messages, discovery requests and responses, and the announcements that UDP and Ethernet send to their transport-specific discovery destination (ApplicationInformation, PublisherEndpoints, PubSubConnection). On a connection configured for message security, it applies the SecurityMode (`Sign`, or `SignAndEncrypt`) to UADP messages and refuses to send any message it cannot secure. - Before, a secured connection dropped its own discovery requests, so a secured subscriber could not ask for metadata, writer configuration, or endpoints. It also refused metadata and writer-configuration probe responses, and sent the transport-specific announcements in plaintext. - A discovery response or announcement that cannot be secured or sent is logged (`DiscoveryMessageNotSent`, Warning) and dropped. Securing announcements makes them depend on a current key; without this, a key provider with no key yet, for example before a Security Key Service delivers the first key, made `StartAsync` fail, and a probe response that could not be secured ended the receive loop. Requests sent through `RequestDiscoveryAsync` still throw to the caller. - Secured messages are counted in `SentNetworkMessages` and `SentDataSetMessages` like unsecured ones. Before, they bypassed the encoder that counts them, so secured data messages were never counted. - `UadpDiscoveryCoder` can encode a discovery message with the SecurityHeader bit set, and reports the security boundary after the PublisherId (and DataSetClassId), which is where `UadpDecoder.TryReadOuterPrefix` ends the prefix. `UadpEncoder.EncodeWithSecurityBoundary` accepts discovery messages. - The receive path is unchanged: it already unwraps secured discovery frames and registers authenticated metadata. - Not changed: the MQTT last-will status message is registered with the broker when the connection opens and published later, so it cannot carry a current SecurityHeader; it stays unsecured. - docs/PubSub.md no longer lists the unsecured-frame warning. It describes how the SecurityMode applies to announcements and discovery (`Sign` authenticates; `SignAndEncrypt` also encrypts) and the last-will exception. Tests: - `MetaDataPublisherTests.UadpPathSecuresTheAnnouncementOfASecuredWriterGroupAsync`: the announcement of a SignAndEncrypt WriterGroup carries a SecurityHeader, does not contain the DataSet name in cleartext, and unwraps and decodes with the subscriber's keys. Before the fix, the announcement had no SecurityHeader. - `MetaDataPublisherTests.SecuredNetworkMessagesAreCountedAsSentAsync`: before the fix, `SentNetworkMessages` read 0 on a secured connection. - `PubSubDiscoveryTests.SecuredSubscriberDiscoversSecuredPublisherWithoutPlaintextFramesAsync`: a secured subscriber and a secured publisher, connected through an in-memory transport that also sends announcements to a separate destination, discover metadata, writer configuration, and endpoints, and no frame lacks a SecurityHeader. Before the fix, the metadata request was dropped. With only the announcement change reverted, the PublisherEndpoints response was dropped. - `PubSubDiscoveryTests.StartupAnnouncementsWithoutACurrentKeyDoNotFailTheStartAsync` and `ReceiveLoopSurvivesADiscoveryResponseThatCannotBeSecuredAsync`: both failed before the fail-soft change. - `UadpDiscoveryTests.SecuredDiscoveryMessagesSplitWhereTheReceiverExpects`: for requests and responses, with and without a DataSetClassId, the secured encoding differs from the unsecured one only by the SecurityHeader flag, and its boundary equals the receiver's prefix length. - `UadpDiscoveryTests.EncodeWithSecurityBoundaryRejectsAMessageItCannotSecure`. ### Validation | Command | Result | | --- | --- | | `dotnet test tests/Opc.Ua.PubSub.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 1498 passed, 1 skipped (the UDP multicast loopback discovery test, which this host skips) | | `dotnet test tests/Opc.Ua.PubSub.Tests -f net48 -p:CustomTestTarget=net48` | 1498 passed, 1 skipped | | `dotnet test` on net10.0 for `Opc.Ua.PubSub.Udp.Tests`, `.Eth.Tests`, `.Mqtt.Tests`, `.Server.Tests`, `fuzzing/Opc.Ua.PubSub.Fuzz.Tests` | 234, 55, 157, 177, 149 passed | | `dotnet test` on net10.0 for `Opc.Ua.PubSub.Kafka.Tests`, `.Adapter.Tests`, `.Diagnostics.Tests`, `Opc.Ua.Redundancy.PubSub.Tests` (on |
||
|
Marc Schier
|
aeeef5b420 |
Send UDP multicast on the network interface the transport joins (#4711)
# Description `UdpDatagramTransport` joined multicast groups on the resolved network interface: the connection address's `NetworkInterface`, the `PreferredNetworkInterface` option, or the first operational interface. It never set `IP_MULTICAST_IF` / `IPV6_MULTICAST_IF`, though, so multicast data and discovery announcements left through the operating system's default multicast route. On a computer with several network interfaces, a publisher configured for one network sent on another, and subscribers on the configured network missed the messages (OPC 10000-14 §7.3.2.2, `NetworkAddressDataType.NetworkInterface`). `ConfigureSocket` now sets the multicast send interface to the resolved interface for IPv4 and IPv6 sockets. That includes the separate socket that sends discovery announcements for loopback-bound connections. A failure is logged at Debug level, like the other socket options. **Behavior change:** without a configured interface, multicast now leaves through the first operational non-loopback interface, which is the interface subscribers join on, instead of the default multicast route. `docs/PubSub.md` and the XML docs of `PreferredNetworkInterface` and `UdpNetworkInterfaceResolver` describe this and recommend naming the interface on multi-homed computers. Tests (`UdpDatagramTransportMulticastInterfaceTests`): - The IPv4 and IPv6 egress options equal the resolved interface. Before the fix they were `0.0.0.0` and `0`. - A publisher and subscriber pinned to the loopback interface exchange a payload. A raw control sender tells a host without loopback multicast apart from a publisher that sends elsewhere. The test fails with the bug on Linux; Windows also delivers local copies sent on other interfaces, so there it only confirms the exchange. ### Validation | Command | Result | | --- | --- | | `dotnet test tests/Opc.Ua.PubSub.Udp.Tests -f net10.0 -p:CustomTestTarget=net10.0 --filter "FullyQualifiedName~UdpDatagramTransportMulticastInterfaceTests"`, with the fix temporarily disabled | IPv4 and IPv6 option tests failed (`Expected: <192.168.178.101> But was: <0.0.0.0>`, `Expected: 4 But was: 0`) | | The same tests from the Windows-built binaries in a Linux container (`mcr.microsoft.com/dotnet/sdk:10.0`, `dotnet test Opc.Ua.PubSub.Udp.Tests.dll`), fix disabled, 5 runs | 3 of 3 failed in every run, including the loopback exchange (the control sender arrived, the publisher's multicast did not) | | Same, with the fix, 5 runs | 3 of 3 passed in every run | | `dotnet test tests/Opc.Ua.PubSub.Udp.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 237 passed (Windows, and the same binaries in the Linux container) | | `dotnet test tests/Opc.Ua.PubSub.Udp.Tests -f net48 -p:CustomTestTarget=net48` | 232 passed | | `dotnet test tests/Opc.Ua.PubSub.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 1490 passed, 1 skipped | | `dotnet build src/Opc.Ua.PubSub.Udp -c Release` (net10.0, net9.0, net8.0, net48) | 0 warnings | | The docs/PubSub.md first exchange (unicast loopback), run verbatim, with #4710 applied | 11 DataSets; the unsecured-frame warning appears once, as documented | ## Related Issues - No tracking issue. Found while verifying the PubSub first-exchange documentation (follow-up to #4555). ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
9600f0de9c |
Let servers start in globalization-invariant mode (#4716)
# Description
A server did not start in globalization-invariant mode
(`InvariantGlobalization`, or
`DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1`), which Native AOT and
container images often enable. When the server loaded its default status
texts, `ResourceManager` created `CultureInfo("en-US")`. In this mode
the runtime throws `CultureNotFoundException` for every culture except
the invariant one, so the server crashed in
`ResourceManager.LoadDefaultText`.
`ResourceManager` now:
- keeps the locale id of each translation table as a string, and falls
back to its language subtag to find another region of the same language;
- when the runtime cannot create the culture, parses a locale id that
matches the grammar of RFC 5646 §2.1 into the recommended casing of its
subtags. It still rejects a language without a region, and rejects a tag
that does not match the grammar, as before;
- caches the resolved locale ids, so loading the default texts creates
each culture once.
`docs/NativeAoT.md` no longer warns against invariant mode for servers.
The new tests (`ResourceManagerInvariantGlobalizationTests`) reproduce
both invariant-mode behaviors of the runtime: no cultures at all, and
cultures that carry invariant data. They also cover malformed tags, and
well-formed tags with extended language, variant, extension, and private
use subtags.
The branch includes a merge of master: #4707 moved translation lookups
to a lock-free snapshot of the tables, and the snapshot now copies the
locale-id-based tables.
### Validation
| Command | Result |
| --- | --- |
| `dotnet test tests/Opc.Ua.Server.Tests -f net10.0
-p:CustomTestTarget=net10.0 --filter
"FullyQualifiedName~ResourceManager"` | 100 passed, including the 39
tests of the new fixture. Before the fix, 10 of its first 22 tests
failed; 7 of the 17 grammar cases failed against the first version of
the parser |
| `dotnet test tests/Opc.Ua.Server.Tests -f net48
-p:CustomTestTarget=net48 --filter "FullyQualifiedName~ResourceManager"`
| 100 passed |
| `dotnet test tests/Opc.Ua.Server.Tests -f net10.0
-p:CustomTestTarget=net10.0`, on
|
||
|
romanett
|
36194f6a02 |
Avoid boxing and per-value copies on encoder, enum and server hot paths (#4715)
## Summary Allocation audit of the per-message / per-value paths (binary codec, built-in types, server request path, monitored items, client notification dispatch) looking for boxing and avoidable per-value copies. Measured with `GC.GetAllocatedBytesForCurrentThread` probe loops on net10.0 **and net48**, and with `dotnet-trace` allocation traces of the server load harness (#4707) for read/write/browse/subscription, server and client side — a value type in the "allocated types" list is a box. Wire output and error behaviour are unchanged. ### Fixed | Where | Cause | Frequency | Before → after | |---|---|---|---| | `Polyfills/System.IO.cs` (net48/netstandard2.0) | `BinaryWriter.Write(ReadOnlySpan<byte>)` polyfill copied with `ToArray()` — every string, array, ByteString, Guid write | per value | 32–64 B → 0; BrowseResponse×50 encode 15.3 → 10.9 KB | | `EnumHelper` (net48/netstandard2.0) | `EnumToInt32`/`Int32ToEnum`/`EnumToInt64` boxed outside `NET8_0_OR_GREATER` (every enum field, `Variant.From<TEnum>`, `EnumValue.From`) | per value | 24 B → 0 | | `BinaryDecoder.ReadString` + stream raw reads (net48) | temp `byte[]` per string, `BinaryReader.ReadBytes` | per value | BrowseResponse×50 decode 15.7 → 11.3 KB | | `WriteGuid` / `ReadGuid` (all) | `ToByteArray()` / `byte[16]` | per value (also Guid NodeIds) | 40 B → 0 each | | `VariantHelper.TryCastTo<TEnum>` (`DataValue.GetValue<TEnum>`), `EventRecordFieldReaders.GetEnum` | `Enum.ToObject` box | per call / per event field | 24 B → 0 | | `NodeManagerServiceDispatcher.GetManagerHandleAsync` | interface enumerator per validated node | per node | Write-100: 151.8 → 148.5 KB/request | ### Cold paths (third commit) - `Server.Auditing` write handler: `Convert.ToBoolean(object)` got a boxed `Variant` (not `IConvertible`) and threw `InvalidCastException` on any in-process write; now takes the boolean, `BadTypeMismatch` otherwise (new test). - `UpdateServerServiceLevel`: `Convert.ToByte(byte, IFormatProvider)` bound to the object overload. - Client publish-state / recovery-policy `HasFlag` (boxes on net48) → `(x & F) == F`. - `SequenceReader` big-endian float on net48/netstandard2.0 used `Convert.ToSingle(int)` (numeric conversion) instead of reinterpreting the bits. - JSON enum strings: one `Enum.GetName` + `string.Concat` instead of `IsDefined` + boxing `CoreUtils.Format`; same output (new test for the `EnumValue` overload). Left as is: `SamplingGroup` start-up `Convert.ToInt32(double, provider)` (rewritten by #4707), `SecureChannelRenewed` formatting (transport files in flight on the client perf branch), `Utils.IsEqual<T>` object path for structs (cold callers; changing it would mix `CompareTo` and `Equals` semantics), Machinery `HasFlag` (startup config). ### Not changed here - `BinaryEncoder.GetNodeIdEncoding` `Convert.ToUInt32(object)` / `Convert.ToByte(enum)` (72 B per NodeId encode, largest single item) and `ChannelAsyncOperation.Complete` boxing the `int` result are handled on the client performance branch. - By design: `Variant` / `NodeId` store `ArrayOf<T>`, Guid, ByteString identifiers boxed (`Variant.Copy` re-boxes RolePermissions during permission checks). ## Tests - New `BinaryCodecAllocationTests`: round trips in buffer and stream decoder mode, truncation errors, allocation per call (< 1 B/call over 20k calls; the counter can be off by an allocation quantum when other tests GC). - `Enum.ToObject` equivalence for `TryCastTo` and `EventRecordFieldReaders.GetEnum` across all underlying types. - net10.0 (third commit): Types 11465, affected Server 87, Client subscription 642 passed; net48 Types 11458 passed, `UA.slnx` 0 warnings. - net10.0: Types 11464, Core.Encoders 2947, Server 8386 passed; Core 6197 passed, 1 unrelated (`ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`, passes alone). - net48: Types 11457, Core.Encoders 2947, EventRecordFieldReaders 25 passed; `UA.slnx` builds with 0 warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
28f1401324 |
Server: cut request-path lock contention and allocation; add load harness (#4707)
Profiling the reference server under load showed most request-path CPU spent spinning on contended locks, plus several avoidable allocations per request or per node. This PR removes them, switches the reference server to Server GC, and adds the load harness used to measure it. ## Results Paired A/B against `master` (3 alternating rounds, medians, 20 sessions, workstation GC on both sides, 24-core machine): | Scenario | Throughput | p50 | p90 | Server CPU / op | Server alloc / op | Lock contention | |---|---|---|---|---|---|---| | Read, 1 value | **+45%** (94k → 137k req/s) | −20% | −46% | −37% | −39% | −94% | | Read, 100 values | **+40%** | −26% | −17% | −24% | −25% | +130% ¹ | | Read, 100 values, SignAndEncrypt | **+48%** | −36% | −20% | −30% | −26% | +106% ¹ | | Browse (Server object) | **+33%** | −31% | −12% | −13% | −14% | −40% | | Write, 100 values | +13% | −12% | −4% | −10% | −19% | −23% | | Data-change subscriptions | ±0 ² | ≈ | ≈ | −14% | −17% | −29% | ¹ The faster server spends more time in the remaining per-node locks. ² Throughput is bounded by the writer rate in this scenario. Separately, Server GC instead of workstation GC doubles Read-100 throughput (21k → 44k req/s) and cuts p90 from 2.1 ms to 0.6 ms, because workstation GC paused the server for about half the wall time under load. The working set grows from ~210 MB to ~430 MB. ## Changes - **`ObjectPool<T>`**: lock-free slot array. `ConcurrentBag.Count` took every per-thread lock on each `Return` of the pooled response value-task source, once per request. - **`FairRequestQueue`**: - drops redundant gate entries: the fail-fast pre-checks and the stopped check read volatile fields and are re-checked under the gate; the owner count is folded into selection; - coalesces wake pulses with a pending flag, so the bounded signal channel's lock is only taken when no token is outstanding. Wake-count semantics in the tests are unchanged. - **`DefaultServerResourceIsolationProvider`**: - compares peer addresses without `GetAddressBytes()` allocations; - builds the unsecured verification context only when a classifier is configured. - **TCP listener / channel**: caches the certificate DER copies per channel, keyed on the certificate instance. `X509Certificate2.RawData` returns a fresh copy on every access, and the listener read it for every request. - **`EndpointBase`**: resolves the request `ActivitySource` once instead of `Assembly.GetCallingAssembly()` per request. - **Node managers**: - one `ServerSystemContext` copy per request (cached on the `OperationContext`) for per-node callbacks: permission metadata, node metadata, view membership, method lookup; - removes a discarded copy in `GetReferenceDescriptionAsync`; - per-request operation caches are plain `Dictionary` instead of a `ConcurrentDictionary` with 24 lock objects; - the validation cache is built in one pass; - fixes zero-capacity error lists. - **`ResourceManager`**: translation lookups read an immutable snapshot that is rebuilt after writes. Browse serialised on the global lock for every display name. - **`MonitoredItem`**: shares the default `DataChangeFilter` instead of allocating one per sampled value. - **Reference server**: `ServerGarbageCollection=true`; new "Garbage collector mode" section in `docs/ServerScalability.md`. - **`perf/`**: `ServerLoadHarness` (out-of-process server plus load driver, server-side CPU/GC/alloc/contention deltas, optional `dotnet-trace` capture), `TraceAnalyzer` (TraceEvent-based CPU/allocation/contention aggregation), and paired A/B scripts. Not part of `UA.slnx`. ### Behaviour notes for custom node managers - The `IDictionary<NodeId, NodeState>` operation cache handed to overrides is now a `Dictionary`. It was a `ConcurrentDictionary`, but only one request flow uses it. - Within one request, per-node callbacks of a node manager share one `ServerSystemContext` copy. ### Later commits on this PR - **Permission checks without `NodeMetadata`** (`9068684c3`): the role-permission and access-restriction validators run on an internal `PermissionMetadata` struct; `AsyncCustomNodeManager` fills it on the stack from the same attributes and namespace defaults. Browse's reference filter reads 3 attributes per target instead of 13. Read 100 values: 120.6 → 95.7 KB/request, ~21k → 26k req/s; Browse 67.6 → 51.8 KB, 26.7k → 29.8k req/s. - **Read-only channel evidence** (`936b28d76`, `dc638ab9f`): `SecureChannelContext.ClientChannelCertificate`, `ServerChannelCertificate` and `ChannelThumbprint` are `ByteString`, so the cached per-channel certificate buffer is shared without a copy and cannot be altered by handlers or classifiers. **Public API change**, see the new section in `docs/MigrationGuide.md`. The isolation provider's revalidation snapshot still copies. - **Session lookup** (`936b28d76`): `SessionManager.TryGetSessionContext` takes the candidate snapshot without the bindings lock; the validating check under the lock is unchanged (2 instead of 4 global lock acquisitions per request). - Unit tests for `ObjectPool`, the per-request system context cache, the translation snapshot, `SecureChannelContext` null semantics and a stopped `FairRequestQueue`; `perf/` follows the repository conventions. ## Testing - `Opc.Ua.Server.Tests` (net10.0): 8386 passed before the `ByteString` change. After it, 13 secure-session and isolation tests failed; both causes are fixed in `dc638ab9f` and those tests (25 with their variants) pass. A full rerun of the Server, Sessions, Core, Redundancy, LDS and GDS suites on the final commit is in progress. - Full solution builds for net10.0 and net48; `dotnet format whitespace` clean. ### o6-automation opcua-benchmarks (sampling, deep pipelines) Replicated both tests of the [o6-automation benchmark](https://github.com/o6-automation/opcua-benchmarks) in the harness (`--scenario counters`, and `read` with 10 clients × N outstanding), paired against `origin/master`: | Test | master | this PR | |---|---|---| | Subscription sampling, 100 Hz, every item ≥ 99 % of samples, 1 item | FAIL (75–94 %) | PASS (100 %) | | same, 1024 items | FAIL (64–73 %) | PASS (99.9 %) | | same, capacity | — | between 4096 and 6144 items (~410–520k samples/s) | | Read, 10 clients × 100 outstanding, None | stalls (0–1.7k req/s) | 121–131k req/s | | Read, 10 clients × 100 outstanding, SignAndEncrypt | stalls (0 req/s) | 89–91k req/s | | Read, 10 clients × 1000 outstanding | channels faulted (`BadTcpNotEnoughResources`) | 0 errors, ~45–54k req/s | - **Fixed-rate sampling** (`30b769d13`): the sampling loop waited one interval after each sample, adding timer overshoot and sampling time to every cycle; samples now have absolute deadlines. - **Async response send** (`cdac5c5ca`): `SendResponse` blocked thread-pool workers on the channel gate; with many requests in flight the receive loop that was handed the gate waited for a pool thread and the channel stalled. The TCP, Kestrel TCP and WebSocket listeners now use `SendResponseAsync`. - **Backpressure instead of channel fault** (`191cd2ba7`): server channels stop reading while more than 100 responses are queued and resume at 50, instead of faulting the channel. Client channels keep reading. At 10,000 outstanding requests throughput is ~50k req/s with or without the limit engaged, so the remaining gap to the 100-per-client case is not the backpressure. ## Not changed - `SentMessageQueue` deep-clones every `NotificationMessage`. Retained messages return their pooled payloads on acknowledgement while the response copy may still be queued for send, and a client can acknowledge a sequence number before receiving it; the clone keeps a recycled payload from reaching another session's response. Removing it needs a send-completion hook from the transport. - Generated responses allocate a default `ResponseHeader` that is replaced (~2-3 % of per-request allocation). Fixing it means making structure fields of every generated type lazily initialized, a generator-wide change to default values, equality and encoding. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
7fe7794a19 |
Enable AES-GCM and ChaCha20-Poly1305 policies on net48 via BouncyCastle (stacked on #4703) (#4705)
> Follow-up to #4703 (now merged to master; this PR has been retargeted
to `master`).
Part of #4510. #4703 enabled the CBC ECC policies on net48 through a
BouncyCastle raw-ECDH polyfill. The AEAD policies were still unavailable
there because .NET Framework has no `AesGcm` or `ChaCha20Poly1305`. This
PR adds those ciphers the same way, so on net48 all of these are now
offered:
`ECC_{nistP256,nistP384,brainpoolP256r1,brainpoolP384r1}_AesGcm` /
`_ChaChaPoly`, `RSA_DH_AesGcm`, `RSA_DH_ChaChaPoly`.
## Changes
- **Polyfill**
`src/Opc.Ua.Security.Certificates/Org.BouncyCastle/AeadCiphers.cs`
(internal, `#if NETFRAMEWORK`, next to `EcdhRawAgreement`): `AesGcm` and
`ChaCha20Poly1305` types with the .NET 8 BCL surface. They wrap
BouncyCastle's `GcmBlockCipher(AesEngine)` and
`Org.BouncyCastle.Crypto.Modes.ChaCha20Poly1305`.
- A tag mismatch throws `CryptographicException`, never BC's
`InvalidCipherTextException`, so
`PlatformSymmetricCryptoProvider.DecryptAuthenticated` still returns
`false` and the channel still raises a protocol error.
- GCM releases plaintext blocks before it checks the tag, so plaintext
stays in a scratch buffer and is copied out only after the tag verifies.
On failure the destination is cleared, as the BCL does. Scratch buffers
and the key copy are zeroed.
- **Opc.Ua.Core**: aliases the polyfill types on `NETFRAMEWORK` and
removes the `NET8_0_OR_GREATER` gates in `SecurityPolicies`
(`SupportsAesGcmPolicy` / `SupportsChaCha20Poly1305Policy`),
`CryptoUtils` (`SymmetricEncryptAndSign` / `SymmetricDecryptAndVerify`,
AEAD helpers) and `PlatformSymmetricCryptoProvider`. The channel, the
provider facet and the encrypted ECC user tokens (`EncryptedSecret`,
through `CryptoUtils`) now run the same code on every target. .NET 8+
keeps the BCL ciphers.
- **FIPS** (requested by the #4703 session, after
|
||
|
romanett
|
2ba58a8cbf |
WoT: move GeneratesEvent off Object instances (OPC 10000-3 §7.15) (#4706)
Fixes #4669. OPC 10000-3 §7.15 says the SourceNode of `GeneratesEvent` "shall be an ObjectType, a VariableType or a Method". CTT `Base Info Core Structure 2 001.js` flags every Object that is the source of one. #4667 fixed this in the source generator. This PR fixes the three remaining WoT places. ### 1. WotCon asset registry - `AssetRegistry.BuildEventNode` / `RemoveEventTypes` no longer add or remove `GeneratesEvent` from the asset Object. Event delivery already relies on `EventNotifier = SubscribeToEvents` and root-notifier registration. - Tests now assert the reference is **absent** and the notifier bit is set. The re-apply test asserts that the old EventType node is gone. ### 2. WoT NodeSet converter (`src/Opc.Ua.Types/Wot`) - **TD → NodeSet:** a last pass (`RelocateInstanceGeneratesEvent`) moves forward `GeneratesEvent` references off an Object/Variable root: - An event the bound type already declares needs nothing more. Matching is by qualified BrowseName against the local context's declarations, the same rule as the declaration merge. - Any other event is declared by a synthesized non-abstract `<rootLocal>Type`. It derives from the bound type (or from `BaseObjectType` / `BaseDataVariableType` when the TD binds none) and becomes the instance's `HasTypeDefinition`. A Variable root gets a `UAVariableType` with the instance's DataType and ValueRank. - Thing Models are unchanged: the projected type keeps its `GeneratesEvent`. - **NodeSet → TD:** - An instance's event affordances are now collected from its type definition and that type's supertypes. Instance-level `GeneratesEvent` is still read for third-party NodeSets. - In a single readable document, a type that only carries one instance's events (only `HasSubtype` + `GeneratesEvent`, typing exactly one instance) is not chosen as root. It is folded back: its supertype becomes the type binding, and no link is written when that supertype is the default base type. The TD → NodeSet → TD round trip stays readable, with no `uav:nodes` fallback. - A document set gives the carrier its own Thing Model, and the instance TD binds to it. - **Archive check:** an event affordance is accepted when the root's type definition, or one of its supertypes, generates it. - `docs/WoTNodeSetConversion.md` documents the behaviour in a new "Event source" row. ### 3. Aggregation sample - `SamplePump.NodeSet2.xml`: each pump is typed by its own `PumpType` subtype, `ns=1;s=Pump1.Type` (`Pump_1Type`) and `Pump2.Type`, which declares its alarm EventTypes. - It is per pump because each pump owns its alarm EventTypes. `Pump2.Type` is cloned with the rest of Pump2. - The pumps keep `EventNotifier="1"` and no longer carry `GeneratesEvent`. - The checked-in documents are regenerated, adding `sample-pump-pump-1type.json` / `-2type.json`. The pump TDs still list the alarm event affordances. - The sample binding lookup ignores Thing Models, because the type's TM declares the same EventType as the pump TD. A declaration missing from the instance still fails as before. - Docs: `WoTConnectivity.md` (asset event affordances, §12.4.2) and `WotBindings.md`. The `GeneratesEvent` clause in the sample-limitation note was already stale, so I removed it. ### Runtime check `WotNativeOwnershipIndex` only uses `GeneratesEvent` in its alias table. Projected event routing takes the owner from `uav:componentOf` / `uav:id` and the notifier from Parent/`HasNotifier`. Nothing in `Opc.Ua.WotCon.Server` reads an instance-level `GeneratesEvent`. ### Tests (net10.0) - `Opc.Ua.Types.Tests`: 11446 passed, 1 skipped. New `WotInstanceEventSourceTests` covers object and variable roots, TMs, the TD with no events, the folded round trip, the legacy instance reference and the document set. Two bound-type cases were added to `WotDeclarationMergeBoundaryTests`. - `Opc.Ua.WotCon.Bindings.Tests`: 714 passed. - `Opc.Ua.WotCon.Tests`: 1496 passed, including the byte-exact sample regeneration tests. - net48 build of both test projects is clean, and `dotnet format whitespace` is clean. Not run: CTT `Base Info Core Structure 2 001.js` (p02-base-info) against a WotCon server. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
romanett
|
1da9fc0543 |
Redundancy.Server session mirroring: fix fast reconnect lifecycle (#4627) (#4704)
Fixes #4627 — the eight Redundancy.Server session mirroring defects (HotAndMirrored fast reconnect). ## Changes | ID | Fix | |----|-----| | RS-1 | The owning replica mirrors a liveness heartbeat (`LastContactAt`), written at most once per quarter of the session timeout while the client uses the session (`DistributedSessionOptions.LivenessInterval`, default 1 s). A restore expires the timeout after the later of the last activation and the last mirrored contact, plus the heartbeat's maximum lag. | | RS-2 | Entries record their owning replica (`OwnerId`). Only the owner mirrors and deletes the entry. A restore takes ownership with a conditional write. A stale copy on the previous replica is dropped without deleting the entry, both when its heartbeat sees the change and when the client fails back to it. `ISharedSessionStore` gains `TryReplaceAsync`, `TryRemoveAsync` and `EnumerateAsync`. Over a store without CAS (the CRDT gossip store) they fall back to compare-then-write. | | RS-3 | A restored session keeps the SessionId from `CreateSession`: `Session.RequestedId`, and `DiagnosticsNodeManager` honours a requested id when it is free. `DistributedSessionManager` mints GUID SessionIds that are unique across the replica set. Continuation points therefore stay under one owner id across repeated failovers. | | RS-4 | Sessions whose user token is encrypted with an ECC/RSA-DH EphemeralKey are flagged. Their restore is rejected up front (`EphemeralKeyUnavailable`), before anything is consumed or counted as a failed authentication. The documented option from the issue was chosen over mirroring ephemeral private keys. | | RS-5 | New `SessionManager.AdmitRestoredSessionAsync` hook: the mirrored nonce is consumed only after the restored activation has passed the signature, identity and continuity checks. If a restored session's first activation fails, the session is discarded without the virtual close, so the shared entry is untouched. Restored copies that are not admitted now also release their diagnostics node, and `CurrentSessionCount` is balanced for restored sessions. | | RS-6 | Expired entries are deleted on restore and by a periodic sweep (`SweepInterval`, default 5 min). The sweep also purges consumed-nonce markers that no live entry references and that are older than `NonceMarkerRetention` (default 1 h), via `ISingleUseNonceRegistry.PurgeAsync`. | | RS-7 | `SharedKeyValueSessionStore` and `DistributedSessionManagerFactory` now throw for an external store without an `IRecordProtector`, as the DI registration already does. Pass `NullRecordProtector.Instance` to opt out explicitly. | | RS-8 | Entries carry the original server certificate. A restored session accepts a client signature over either that certificate or its own one (`Session.OriginalServerCertificate`). | Entries are now written at `SecurityStateVersion` 5. Entries at versions 3 and 4 can still be restored, without an owner. ### API changes (Redundancy.Server) - New members on `ISharedSessionStore` and `ISingleUseNonceRegistry`. - An optional `TimeProvider` parameter on `SharedSingleUseNonceRegistry`. - New `DistributedSessionOptions` properties. - `SharedSessionEntry` v5 fields. - `SharedKeyValueSessionStore` and `DistributedSessionManagerFactory` throw for external stores without a protector. ## Tests - New unit tests for every finding: the restore/heartbeat/ownership/sweep/SessionId paths of `DistributedSessionManager`; store conditional writes, the CAS fallback and v5 round-trips; the registry purge; the factory/store guard; base `SessionRestoreTests` (admitted once, discarded on validation failure or rejected admission); and `SessionOriginalServerCertificateTests`. - New end-to-end test `SessionUsedPastItsTimeoutFailsOverWithItsSessionIdAsync`. The client uses the session for 2× its timeout, then fails over. The test checks that the SessionId is preserved and that the standby serves that SessionId and owns the entry. It also closes the stale copy and checks the entry survives. With the heartbeat disabled, the test fails as expected. - Local net10.0 runs: Redundancy.Server.Tests 883/883, Server.Tests 8347 passed / 5 skipped, Sessions.Tests 849 passed / 379 skipped (the usual baseline). The net48 build is clean. The Redundancy session tests pass on net48; the ECC-only cases are skipped there. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
d552634258 |
Enable ECC security policies on net48 via a BouncyCastle raw-ECDH polyfill (#4703)
Closes #4510. ## Summary OPC UA Part 6 feeds the **raw** ECDH shared secret into HKDF. .NET Framework only exposes `ECDiffieHellman.DeriveKeyMaterial`, which always hashes it, so the ECC SecureChannel and user-token policies were unavailable on `net48` (`SecurityPolicies.GetInfo` returned `null`). This adds a BouncyCastle polyfill for just that primitive: - **`Opc.Ua.Security.Certificates/Org.BouncyCastle/EcdhRawAgreement.cs`** (`net48` only): an internal `DeriveRawSecretAgreement(this ECDiffieHellman, ECDiffieHellmanPublicKey)` extension using `ECDHBasicAgreement`. It returns the X coordinate padded to the field size, exactly like the .NET 8 BCL method. Keys stay CNG keys; ECDSA signatures and HKDF stay on the platform — only the agreement step is managed code. The point is validated (on curve, not infinity) and mismatched curves are rejected. - Because the extension mirrors the .NET 8 signature, `Nonce.GenerateSecret` now has one code path on every target. - `SupportsRawEccSecretAgreement()` is removed; the CBC-based ECC policies (`ECC_nistP256`, `ECC_nistP384`, `ECC_brainpoolP256r1`, `ECC_brainpoolP384r1`) are offered on every target (still subject to OS curve support). - BouncyCastle was already a `net48` dependency of `Opc.Ua.Security.Certificates`; no new package. (Only `net48` is downlevel now — `netstandard2.1`/`net472` from the issue text are no longer targets.) **Not in scope:** the `_AesGcm` / `_ChaChaPoly` ECC variants (and PubSub DTLS) still need .NET 8 because .NET Framework has no AEAD ciphers — follow-up stacked on this branch. ## Docs `EccProfiles.md` (Known limitations), `migrate/2.0.x/certificates.md`, `ProductionChecklist.md`, `Profiles.md`, `WhatsNewIn2.0.md`. Calls out that on `net48` the agreement step does not run in a FIPS/validated module. ## Tests (local) | Suite | Result | |---|---| | `Opc.Ua.Core.Tests` full, net48 | 5956 passed, 0 failed | | `EccInteropTests` net48 vs 1.5.378 peer (net8.0, native raw ECDH) | 26 passed; 28 skipped (AEAD + curve25519/448) — all four CBC curves, both directions, UserName tokens, token renewal | | `Opc.Ua.Sessions.Tests` ECC filter, net48 | 52 passed, 0 failed | | Server `SessionCoverageTests`/`EphemeralNonce*`, Configuration builder tests, net48 | all pass | | Nonce/ECC/EncryptedSecret + session tests, net10.0 | all pass | New tests: symmetric agreement on all four curves; on net48 an oracle against CNG (`SHA256(raw) == DeriveKeyFromHash`); curve mismatch and off-curve rejection. Tests that asserted the old net48 limitation were updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
78e66f3473 |
Aggregates: TreatUncertainAsBad never applies to bounding values; DurationGood/Bad count Good/Bad regions only (#4702)
# Description Server-side aggregate fixes that follow three answers from the OPC Foundation on Mantis: - [11462](https://mantis.opcfoundation.org/view.php?id=11462) (~0025840): TreatUncertainAsBad does not apply to bounding values. - [11425](https://mantis.opcfoundation.org/view.php?id=11425) (~0025847): DurationGood counts only Good regions and DurationBad only Bad regions, and TreatUncertainAsBad does not apply. - [11426](https://mantis.opcfoundation.org/view.php?id=11426) (~0025852): Minimum, Maximum, the ActualTime variants and Range look at Uncertain values as if they are Good. The principle in 11425 and 11426 is that the aggregate definition wins over TreatUncertainAsBad. The CTT-side defects are filed as [11487](https://mantis.opcfoundation.org/view.php?id=11487). ### Bounding values (`AggregateCalculator`, `StartEndAggregateCalculator`) - Simple (§3.1.9) and Interpolated (§3.1.8) Bounding Values are formed from **non-Bad** raw values (`IsBoundCandidate`), whatever TreatUncertainAsBad is. - An Uncertain raw value before or after the timestamp makes the bound `UncertainDataSubNormal`. - A raw value exactly at the timestamp is the bound and keeps its own status. - Only a **Bad** value after a Simple bound reverts to stepped interpolation. - StartBound/EndBound keep an Uncertain bound instead of `Bad_NoData` (Tables 76/77). - WorstQuality2 now includes an Uncertain start bound, as §5.4.3.36 requires ("the server shall ignore the TreatUncertainAsBad for this aggregate"). ### How aggregates use the values (unchanged rule, two gaps closed) TreatUncertainAsBad still governs how an aggregate uses the values inside the interval, including an Uncertain bound. - `GetRegionsInValueSet` now also converts an Uncertain **first** region to Bad for Simple-bound aggregates. TimeAverage2/Total2 therefore omit it like any other Uncertain region (§5.4.3.7, Table 55 "Bound Uncertain"). - TimeAverage/Total (Interpolated bounds) still use an Uncertain bound and report Uncertain (Table 54). - TimeAverage2/Total2 return the PercentTime status `Bad` instead of `Bad_NoData` for an interval that has data but only omitted data (§5.4.3.2.1). - Minimum2/Maximum2 are unchanged: an Uncertain bound is a candidate only when TreatUncertainAsBad is false (§5.4.3.15). ### NumberOfTransitions (`CountAggregateCalculator`) §5.4.3.24 speaks only of "non-Bad" values, and Table 72 says "Bound Uncertain: Use as value". So: - Uncertain values are counted and can be the previous value, whatever TreatUncertainAsBad is. - The previous value is the nearest non-Bad value before the interval (the early bound). This reverses an earlier change that made NumberOfTransitions honour TreatUncertainAsBad. ### Minimum, Maximum, MinimumActualTime, MaximumActualTime, Range (`MinMaxAggregateCalculator`) Per the 11426 answer: - Every non-Bad value is a candidate, whatever TreatUncertainAsBad is. - An Uncertain extremum is returned with `UncertainDataSubNormal` plus Calculated. ActualTime keeps the timestamp of the first such value. - Otherwise the status is Good unless the interval contains Bad values (then `UncertainDataSubNormal`); it is `BadNoData` if all values are Bad. - Range is max − min over the non-Bad values. It is Uncertain if either extremum is Uncertain or Bad values exist. Before this change the server ignored an Uncertain value beyond the Good extremum and returned the Good one with an Uncertain status. Minimum2/Maximum2/Range2 are unchanged, because §5.4.3.15 ties them to TreatUncertainAsBad explicitly. ### DurationGood/DurationBad/PercentGood/PercentBad (`StatusAggregateCalculator`) - A region is Good only if its data point is Good, and Bad only if it is Bad. - Uncertain counts for neither (§5.4.3.31–34). - The first region still takes the status of the raw value at or before the start. ### Tests - New `Aggregates/BoundingValueTreatUncertainAsBadTests`: DeltaBounds, StartBound/EndBound, TimeAverage2/Total2, Interpolative, NumberOfTransitions (forward and reverse), DurationGood/Bad/PercentGood/Bad. - New Min/Max test for Bad-only and Bad-next-to-Good intervals. - Updated tests that encoded the old interpretation: - Interpolative with an Uncertain raw value at the start: 2 tests. - Duration/Percent Good/Bad: 2 tests. - NumberOfTransitions with TreatUncertainAsBad: 2 tests. - Minimum/Maximum/Range with an Uncertain extremum or only Uncertain values: 2 tests in Server.Tests, 1 in History.Tests. - `ProcessedReadUsesAdvertisedNodeAggregateDefaultsAsync`: it now probes the TreatUncertainAsBad default with Count, because PercentGood no longer depends on it. ### CTT verification CTT 1.05.513 with the patched scripts (c3674ff, which encode the 11462/11425/11426 answers), Aggregates part (p06): | Server | Errors | Warnings | Pass | NotImpl | | --- | --- | --- | --- | --- | | master |
||
|
romanett
|
9b034f8291 |
Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
cd4dbbe66e |
WebApi: apply the REST auth opt-ins to the opcua+openapi WebSocket upgrade (#4699)
# Description Follow-up to #4674 for the WebSocket side of the OpenAPI binding. - **Plain `opcua+openapi` upgrade uses the REST auth.** When `AddWebApiBasicAuth` / `AddWebApiBearerAuth` / `AddWebApiMutualTlsAuth` is configured, the upgrade runs the same default authorization policy as the REST route group (`RequireAuthorization()`), because the upgrade is dispatched by the listener after routing. Requests that don't satisfy it get the schemes' challenge (401 + `WWW-Authenticate`) or 403. Without an auth opt-in the behaviour is unchanged. - **Upstream identity on the channel.** For both `opcua+openapi` and `opcua+openapi+<accesstoken>`, the authenticated principal is mapped through `ISessionlessIdentityProvider` into `SecureChannelContext.UpstreamIdentity`, as the REST routes already do per request. - **Token lifetime.** A credential whose `exp` claim has already passed is refused at upgrade, also within the JwtBearer clock skew, and the socket is closed when the token expires. - **Refused upgrades complete their response.** Before, the admission lease aborted the connection when an upgrade was refused, so clients could see a connection reset instead of the 401/503. Refused upgrades on HTTP/1.x now carry `Connection: close` and Kestrel closes the connection after sending the response. The other WebSocket sub-protocols (`opcua+uacp`, `opcua+uajson`) and the HTTPS binary/JSON endpoint are unchanged: per Part 6 §7.4 they authenticate through the OPC UA session layer, not HTTP credentials. ## Tests New cases in `RealHttpsListenerIntegrationTests.Authentication.cs` against a real `HttpsTransportListener`: - the plain upgrade for Basic, Bearer and mTLS with no, wrong, wrong-scheme and valid credentials (status, challenge, and that nothing reaches the server callback); - `UpstreamIdentity` set for both `opcua+openapi` variants; - an expired token, in the `Authorization` header and in the sub-protocol; - the socket closing when the token expires. Locally on net10.0: `Opc.Ua.Bindings.Https.WebApi.Tests` 499/499, `Opc.Ua.Core.Tests` (Wss/OpenApi/Https) 243/243, `Opc.Ua.Sessions.Tests` (Wss/Https/WebApi) 53/53. `Opc.Ua.Bindings.Https` builds on all TFMs with 0 warnings. ## Related Issues - Follow-up to #4674 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [ ] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
b8f07843b2 |
Secure channel: never answer with an expired token; accept a just-expired one (+ labelled soak/stress CI) (#4698)
## Summary **Secure channel: a client could drop the channel when the server answered with a just-expired token.** Running the long-dormant ConnectionStability soak locally, the subscription channel shut down with `BadSecureChannelClosed`. The session then timed out and the subscription was abandoned. Root cause, against OPC 10000-4 §5.6.2.1: 1. **Server (rule 2: use the old token only "until the SecurityToken expires").** After a renew, the server keeps securing messages with the old token until the client sends something secured with the new one. That activation only happened on *receive*. A client that sends nothing, for example one whose Publish requests are all outstanding, kept getting responses secured with the old token after it expired. 2. **Client (rule 3: "accept Messages secured by an expired SecurityToken for up to 25 % of the token lifetime").** The client rejected any such response the moment the token expired, throwing `BadTcpSecureChannelUnknown`, and shut the channel down. From the failing run (token lifetime 10 s): the client renewed to token #11 at 09:59:01.686 and sent nothing on that channel afterwards. The server, still on #10, sent a Publish response at 09:59:04.389, 380 ms after #10 expired. The client rejected it and closed the channel. Nothing reconnected, the 10 s session timeout expired, and the server abandoned the subscription. Both halves date back to the 2016 initial commit; this is not a recent regression. It hits whenever the client sends nothing on the channel between renewing and the old token's expiry, so it is timing-dependent. ### Fix - `UaSCBinaryChannel.ActivateRenewedTokenIfDue()` activates a pending renewed token once the current one passes `TokenActivationPeriod` (95 %). This is the rule the receive path already applied. It is now shared, and `TcpServerChannel.SendResponse` calls it before securing a response. - `ReadSymmetricMessage` accepts responses (not requests) secured with an expired token for `TcpMessageLimits.TokenExpiryGracePeriod` (25 %) of its lifetime. ### Tests `TcpReconnectKeyAgreementTests` now runs the server channel and the peer on one shared `FakeTimeProvider`: - `ResponseSentAfterTheCurrentTokenExpiresUsesTheRenewedTokenAsync`: a response sent at 96 % of the old token's lifetime is secured with the renewed token. - `ResponseSecuredWithTheExpiredTokenIsAcceptedWithinTheGracePeriodAsync`: accepted at 120 % of the lifetime, rejected at 130 %. Both fail without the fix. All 40 tests in the fixture pass with it. ## CI: soak and stress runs on labelled PRs (and the soak that tested nothing) The bug stayed hidden because `stability-test.yml` still filtered `Opc.Ua.Client.Tests` after #3810 moved the test to `Opc.Ua.Sessions.Tests` (2026-05-28). Since then the nightly has run **zero tests** while reporting green: runs dropped from ~100 min to ~15 min. - `stability-test.yml` now targets `Opc.Ua.Sessions.Tests` and writes a TRX file. - New `.github/scripts/assert-tests-ran.ps1`, used by all three opt-in workflows, fails a run in which no test executed, or none passed. Contract-tested in `CiAssertTestsRanTests`. - Like `interop`, the opt-in workflows can now run on a labelled PR. They are not required checks. A later push cancels the run in progress, and adding an unrelated label neither starts nor cancels a run. | Label | Workflows | Run on a PR | | --- | --- | --- | | `stress` | Stress Tests (ChaosTCP) | Random seed, saved as `chaos-seed.txt` in the results artifact | | `soak` | Connection Stability Test, Sample HA Long-Haul Test | 30 min stability; 15 min per HA scenario | ## Test plan - [x] `TcpReconnectKeyAgreementTests`: 40/40 pass; the 2 new tests fail without the fix - [x] All `Opc.Ua.Core.Tests.Stack` tests (net10.0): 3548 passed, 0 failed - [x] `CiAssertTestsRanTests`: 4/4 - [x] `soak` + `stress` labelled run on this PR (before the fix): both stability tests, both HA long-haul scenarios and ChaosTCP passed on Linux (the label triggers and push-cancellation work) - [x] Local short-haul soak with the fix (Windows, 10 s tokens), 2 runs: both pass with full notification counts; run 2 hit the window once (`Token activated forced` on send) and kept the channel - [x] `soak` + `stress` labelled run with the fix: Connection Stability (2 tests), Sample HA Long-Haul and ChaosTCP all pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Klaus Landsdorf
|
fa55157ce4 |
Server: hold anonymous session-less requests to the channel security of CreateSession (#4689)
# Description `AddSessionlessInvocation(o => o.AllowAnonymous = true)` (#4659) lets a request without an Access Token run as an anonymous user. OPC 10000-4 §6.3.1 allows that only "if application authentication through the SecureChannel is sufficient". On HTTPS the channel gave no such authentication: on an endpoint with mutual TLS and a SignAndEncrypt policy, a session-less request without a TLS client certificate and without an Access Token was answered `Good`, over the REST binding and over the binary POST with `OPCUA-SecurityPolicy: Basic256Sha256`. `CreateSession` on the same endpoint answers `Bad_SecurityChecksFailed` without the certificate, and ActivateSession on a new channel does too (OPC 10000-6 §7.4.1: applications that use HTTPS and require application authentication check the application certificates during the CreateSession/ActivateSession handshake). This PR makes a session-less request without an Access Token meet what the endpoint demands of a Session. #4659 is merged, in no release tag and opt-in, so this is a change to a new feature; the default `AllowAnonymous = false` is not affected. ## Changes ### `SessionManager.ValidateSessionlessRequestAsync` (`src/Opc.Ua.Server/Session/SessionManager.cs`) With `AllowAnonymous`, a request without an Access Token now needs: | Check | Where it applies | Status otherwise | Same as | | --- | --- | --- | --- | | The client certificate of the channel (`SecureChannelContext.ClientChannelCertificate`) | endpoints with a security mode or policy other than None, and HTTPS endpoints when `HttpsMutualTls` is set | `Bad_SecurityChecksFailed` | CreateSession on the HTTPS listener; ActivateSession on a new channel | | An Anonymous user token policy on the endpoint (an endpoint without policies does not restrict) | every endpoint | `Bad_IdentityTokenRejected` | ActivateSession with an anonymous identity token | | An endpoint on the channel | every request | `Bad_SecurityModeInsufficient` | the same request with an Access Token | - The certificate is the one the transport has validated: the SecureChannel certificate of opc.tcp and opc.wss, the TLS client certificate of the HTTPS listener (REST, binary, JSON). So opc.tcp channels with security keep working as they did, and an HTTPS client that wants an anonymous session-less call presents its application certificate in the TLS handshake. - An HTTPS listener without `HttpsMutualTls` advertises SecurityMode None without an Anonymous user token policy (`HttpsServiceHost`); an anonymous session-less request there is now rejected as an anonymous Session is. - A request **with an Access Token** is unchanged: the token authenticates the caller and HTTPS keeps it confidential (HTTPS clients are not required to present a certificate, OPC 10000-6 §7.4.1). A `ValidateSessionLessRequest` handler still takes over completely. - Not changed: the transport credential authentication of the REST routes (#4674). `SessionlessInvocationOptions.AllowAnonymous` documents the rule. `docs/SessionlessInvocation.md` has a new section "Anonymous requests and the channel" with the table. ### Tests - `tests/Opc.Ua.Sessions.Tests/SessionlessChannelSecurityTests.cs` (new, net8.0+): a real reference server with an HTTPS listener, twice (mutual TLS on with a SignAndEncrypt-only endpoint; mutual TLS off), over REST and the binary POST, with and without a TLS client certificate: anonymous without a certificate, with a certificate, `AllowAnonymous = false`, a valid and a bogus Access Token, and CreateSession on the binary POST without the certificate for comparison. 44 cases. - `tests/Opc.Ua.Server.Tests/SessionlessInvocationTests.cs`: opc.tcp with and without a channel certificate (Sign, SignAndEncrypt, None), HTTPS with and without mutual TLS and with a secured endpoint, user token policies with and without Anonymous, no endpoint, an Access Token and a handler without a certificate. The existing tests that use `AllowAnonymous` set `HttpsMutualTls = false` or give the channel an endpoint. - Fail without the change: 7 of the 9 new `SessionlessInvocationTests` cases fail with the `SessionManager.cs` change removed (the opc.tcp with security, HTTPS with and without mutual TLS, user token policy and unknown endpoint tests); the 6 failing `SessionlessChannelSecurityTests` cases are below. ## Testing - **Reproduction on master (`c03c12838`)**: `SessionlessChannelSecurityTests` without the change: 6 of 44 cases fail, 38 pass. With mutual TLS on and no TLS client certificate the anonymous request is answered `Good` instead of `Bad_SecurityChecksFailed` (REST and binary). With mutual TLS off it is answered `Good` instead of `Bad_IdentityTokenRejected` (REST and binary, with and without a certificate). All 44 pass with the change; the Access Token cases pass both ways. - Results on macOS: `Opc.Ua.Server.Tests` full run on net10.0 8197 passed / 9 skipped / 0 failed; the session-less and SessionManager filters 139 passed / 0 failed on net10.0 and on net8.0. `Opc.Ua.Sessions.Tests` with the HTTPS/WebApi/Sessionless filters 125 passed / 0 failed on net10.0 and on net8.0, and with the Wss/OpenApi/ClientTest filters 320 passed / 241 skipped / 0 failed on net10.0. - Builds: `Opc.Ua.Server`, `Opc.Ua.Server.Tests` and `Opc.Ua.Sessions.Tests` for net48, net8.0, net9.0 and net10.0: 0 warnings, 0 errors. - `dotnet format` (whitespace, style, analyzers `--severity info`, one `--include` per changed file): no finding on a changed line (CA1849 at `SessionManager.cs` 235 is on master). - Coverage (`.azurepipelines/check-coverage.ps1` against master, session-less and SessionManager tests of `Opc.Ua.Server.Tests` on net10.0): patch 36/36 changed lines (100 %). The project floor is not meaningful for this subset. - CI and CodeQL on the PR branch: [CI run 37529750287](https://github.com/biancode/UA-.NETStandard/actions/runs/37529750287) green (158/158 jobs, incl. macOS and Windows net48), [CodeQL run 37529754883](https://github.com/biancode/UA-.NETStandard/actions/runs/37529754883) green, on `03a8e3d91`. ## Related Issues - Fixes #4688 - Relates to #4659 (session-less invocation), #4657 (the REST binding's session-less service set) and #4674 (REST authentication) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the branch CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
romanett
|
5572107ab8 |
Push Model compliance: UpdateCertificate issuer check, TrustList audit events, NoConfiguration state, DefaultHttpsGroup (#4684)
# Description
Fixes found by a manual run of the OPC 10000-12 *Push Model for Global
Certificate and TrustList Management* test plan (183 test cases,
TC0–TC182) against the Reference Server. After these changes no test
case fails; the remaining non-pass verdicts are test-plan expectations
that contradict Part 12/Part 4/Part 20 (documented in the run report).
### Server behaviour changes
- **UpdateCertificate requires the issuer in the TrustList**
(`ConfigurationNodeManager.PushValidation.cs`). The validator classifies
an incomplete chain as suppressible; OPC 10000-4 Table 100 does not ("An
error during the chain creation may not be suppressed") and §7.10.5
requires the issuers to already be in the group's TrustList. A
certificate of an unknown CA is now rejected with
`Bad_CertificateChainIncomplete`. Expiry / host-name errors stay ignored
as §7.10.5 requires.
- **TrustList audit events reach the Server Object.**
`TrustListUpdateRequested`/`TrustListUpdated` were reported on the
TrustList node (`EventNotifier=0`, not linked to the Server), so no
client could receive them. `TrustList.SetAuditEventServer` routes them
through `IAuditEventServer`; wired for the Push TrustLists and the GDS
CertificateGroup TrustLists. New `AuditEvents` overloads keep the
node-based ones.
- **Duplicate AddCertificate** no longer makes the following
ApplyChanges fail with a bare `Bad` (store refused the second copy); the
commit is idempotent and its rollback keeps the pre-existing
certificate.
- **CloseAndUpdate validates CRLs** (incl. CRL-only updates): every CRL
must be signed by a CA of the new TrustList (§7.8.2.5). Previously an
unknown/forged CRL passed CloseAndUpdate and only failed ApplyChanges.
- **X.509 v1 certificates are rejected** by
CloseAndUpdate/AddCertificate/UpdateCertificate (OPC 10000-6 §6.2.2; new
`X509Utils.IsX509Version3`).
- **Application setup state (Annex G.2):** new
`StandardServer.StartupServerState` (virtual); the DI host returns
`NoConfiguration` when `ServerConfigurationOptions.InApplicationSetup ==
true`. `IsRunning` and `NodeManagerLifecycle` treat `NoConfiguration` as
operational. The Reference Server `--provision` mode starts in
`NoConfiguration` and exposes `InApplicationSetup=TRUE`.
- **DefaultHttpsGroup:** the group is only created when Https stores are
configured — the Reference Server config had none, so its
`opc.https`/`opc.wss` endpoints used the RSA application certificate
with an empty TLS trust list and the Https certificate could not be
pushed. The sample config now declares
`HttpsIssuerCertificates`/`TrustedHttpsCertificates` and an
`HttpsCertificateType` certificate (new `CertificateTypeString` value
`Https`). An Https certificate is assigned to DefaultHttpsGroup only (no
longer also listed in DefaultApplicationGroup). Fixed
`AddSecurityConfigurationHttpsStore`, which built the trusted Https
store path from the store *type*.
- Minor: AddCertificate logs a warning when it adds a CA without a CRL
while `RejectUnknownRevocationStatus` is enabled; UpdateCertificate
refused for an insufficient SecureChannel raises
`CertificateUpdateRequestedAuditEvent` with Status=false.
### Kept on purpose
- PEM certificates are accepted (converted to DER).
- `Bad_NotWritable` when opening an already open TrustList (OPC 10000-20
§4.2.2) and for AddCertificate while a read handle is open (§7.8.2.6).
- No validation rules for DefaultUserTokenGroup / DefaultHttpsGroup
TrustLists (Part 12 defines them only for ApplicationCertificateType
groups).
### Notes for reviewers
- Existing tests in `ConfigurationNodeManagerPushTests` and
`PushCertificateRequestRegressionTests` signed pushed certificates with
CAs that were not in the TrustList; they now trust their CA first
(behaviour change above).
- Adding the Https certificate to
`Quickstarts.ReferenceServer.Config.xml` gives the Reference Server's
opc.https/opc.wss endpoints their own TLS certificate.
### Tests
-
`tests/Opc.Ua.Server.Tests/Configuration/PushComplianceRegressionTests.cs`
— duplicate add/rollback, PEM, v1/v3, CRL issuer/forged/CRL-only, audit
routing.
- `tests/Opc.Ua.Server.Tests/ApplicationSetupServerStateTests.cs` —
NoConfiguration startup state.
- `tests/Opc.Ua.Gds.Tests/PushComplianceTests.cs` — end-to-end over a
secure channel: TrustList transaction lifecycle, open-handle rules,
masked reads, duplicate/PEM add, unknown CRL issuer, unknown certificate
issuer, cross-Session CSR key, nonce rules, RemoveCertificate chain/CRL
handling. Mutation-checked: reverting the issuer, duplicate-add and CRL
fixes makes the corresponding tests fail.
- Additional cases in `PushCertificateRequestRegressionTests`,
`CertificateIdentifierTests`, `ApplicationConfigurationBuilderTests`.
Local runs (net10.0): Opc.Ua.Gds.Tests 1257 passed / 0 failed / 56
skipped; Opc.Ua.Server.Tests push/TrustList/audit/lifecycle subset 1209
passed; Core and Configuration test classes passed. net48 build of the
affected test projects succeeds; `dotnet format whitespace` clean.
## Related Issues
- None (findings from a manual Push Model compliance run).
## Checklist
- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [ ] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Copilot
|
c03c128384 |
Treat code analysis (CA) warnings as errors and fix all surfaced analyzer warnings (#4680)
# Description `TreatWarningsAsErrors` was already enabled, but `CodeAnalysisTreatWarningsAsErrors` was `false`, so Microsoft Code Analysis (`CAxxxx`) warnings were non-fatal. This PR sets it to `true` in `common.props` and fixes all 55 analyzer warnings the net10.0 build reported. **Production code** - `SamplingGroup.cs`: CA2213 is a false positive here because the detached `CancellationTokenSource` is disposed by `CancelSamplingLoop` once the loop completes. It is suppressed with a pragma, a justification and a TODO, following the existing pattern in `ServerBase.RequestQueue.cs`. **Tests** - CA1861: constant array arguments are moved to `static readonly` fields or replaced with `ArrayOf<T>` collection expressions. In the legacy interop server I used locals instead, because those arrays become node values that writes can change in place. - CA2025: tasks that finish before disposal (`Assert.ThrowsAsync`, or awaited before `using` scope exit) are suppressed with justification comments, following existing test patterns. - CA1812: classes created by the DI container are suppressed with a comment and a TODO. - CA2208: benchmark switch defaults now throw `InvalidOperationException`. - CA1845: `Substring(0, 8)` is replaced with `[..8]`. - CA1307: `string.Replace` is replaced with `Split`/`Join`, which also works on net48. - CA5394: `Random` is replaced with `RandomNumberGenerator`. **Docs** - `docs/DeveloperGuide.md` now says CA warnings also fail the build, and that any suppression needs a comment explaining why plus a TODO. **Validation** - I fixed the 55 warnings from a net10.0 build of `UA.slnx` (`CustomTestTarget=net10.0`). That build ran before the setting was turned on, and I have not rebuilt net10.0 since the fixes. - `CustomTestTarget=net48` build of `UA.slnx`: 0 warnings, 0 errors. - I also started a full build across all target frameworks with the setting on. Its log showed no errors or warnings while it ran, but I couldn't read its final result. - No tests have been run yet; CI should confirm the multi-TFM build and the tests. ## Related Issues - N/A ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: romanett <7413710+romanett@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
romanett
|
21e04bdba5 |
Make Bad_CertificateChainIncomplete unsuppressible; add UpdateCertificate renewal regression tests (#4683)
# Description Follow-up to the master378 fix in #4678 (CTT: application certificate renewal must use the configured trust stores). Master already ignores the supplied `issuerCertificates` for the default application group (OPC 10000-12 §7.10.5), but it had no regression tests for CA-signed renewals. While adding them I found a real gap: **The validator treated `Bad_CertificateChainIncomplete` as suppressible.** OPC 10000-4 Table 100 (Build Certificate Chain) says: "An error during the chain creation may not be suppressed." §6.1.3 adds: "Processing fails with Bad_SecurityChecksFailed if an element in the chain cannot be found." UpdateCertificate for the application group accepts every *suppressible* error (as §7.10.5 requires). As a result it accepted a renewal signed by a CA that is **not** in the TrustList and was only passed in the ignored issuer list, which is exactly what §7.10.5 rules out. ## Changes - `CertificateValidationCore`: remove `BadCertificateChainIncomplete` from the suppressible status codes. An accept callback (`AcceptError`) is no longer asked about it. - `TrustList`: remove the local workaround that filtered out this error; plain "accept suppressible errors" is now enough. - Docs (`Certificates.md`, `CertificateManager.md`): drop it from the suppressible list, and fix a sample that suppressed it. ## Tests - `ConfigurationNodeManagerPushTests`: - A CA-signed renewal with the CA installed succeeds with a missing, empty, unrelated or malformed issuer list, and the trust stores stay unchanged. - A renewal whose CA is only supplied as an issuer is rejected with `BadCertificateChainIncomplete`; stores and the current certificate are untouched. - End to end on an isolated server: a CA-signed renewal with no issuer list is committed through ApplyChanges. - `ConfigurationNodeManagerPushValidationTests`: the group TrustList validator accepts a CA from the trusted or issuer store and rejects an uninstalled CA. - `CertificateValidationCoreTests` / `CertificateValidatorTest`: - An incomplete chain is unsuppressible and never reaches the callback. - A leaf trusted directly **stays valid when the peer supplies its CA** (§6.1.3: chain certificates "may be provided with the application Certificate"). - One existing test attached its approver only after validating, so it never exercised suppression; it now validates with the approver attached. - Tests that only passed because of the bug (`PushCertificateRequestRegressionTests`, the regenerated-key helper and the rollback test) now install the signing CA in the TrustList first. The two new rejection tests fail on master and pass with this change. ## Reviewer notes / behavior change This was suppressible in 1.5.x as well. Application code whose accept handler approved a CA-signed peer certificate without its CA (CA neither installed nor sent by the peer) will now get a rejection. Without a handler such certificates were already rejected, and the stack's own handlers (`ApplicationInstance`, push validation) never accepted this error. Worth a release note. Local runs (net10.0), all passing: | Suite | Passed | Failed | |---|---|---| | Server.Tests | 8310 | 0 | | Core.Tests (security/transport filter) | 2039 | 0 | | Security.Certificates.Tests | 358 | 0 | | Core.Security.Tests | 558 | 0 | | Client.Tests | 2761 | 0 | | Gds.Tests (PushTest and TrustList tests) | 345 | 0 | The net48 builds of Server.Tests and Core.Tests also succeed. ## Related Issues - Related to #4678 (master378 counterpart) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Klaus Landsdorf
|
cc7908eaa1 |
WebApi: serve a generated OpenAPI document of the REST binding (opt-in) and map the session-less service set (#4657)
Reworked after the review: the OpenAPI document is generated from the code, not embedded; the route is opt-in; `StatusCode` is not accepted. The branch is a new history on top of master (the first version conflicted); master with #4674 is merged in. # Description The REST binding implements the routes of the OPC Foundation OpenAPI mapping (OPC 10000-6 §G.3), but it does not describe them: `GET /openapi.json` is answered with HTTP 405, and a client generator has nothing to read from the server. `docs/WebApi.md` listed a generated OpenAPI document as a deferred follow-up. The coded API is the source of truth, so this PR **generates** the document from the route table and from the structure definitions of the request and response types, and compares it with the document the model compiler publishes in tests. Nothing from `UA-Nodeset/OpenApi` ships in the package. ## Changes ### Service sets (`Opc.Ua.Core`) - `WebApiServiceSet` (`AllServices`, `Sessionless`) and `WebApiServiceRoutes.GetRoutes(serviceSet)`. `Sessionless` is the eight services of `opc.ua.openapi.sessionless.json` (OPC 10000-4 §6.3): Read, Write, HistoryRead, HistoryUpdate, Call, Browse, BrowseNext, TranslateBrowsePathsToNodeIds. ### Generator (`Opc.Ua.Core.Schema`, `Opc.Ua.Schema.OpenApi`) - `WebApiOpenApiGenerator.Generate(serviceSet, includeSchemas, serverUrl)` returns the OpenAPI 3.0 document as a `JsonObject`. It is built at runtime from `WebApiServiceRoutes` and the `StructureDefinition`/`EnumDefinition` of the message types (the same definitions the schema generation uses, resolved through `IDataTypeDefinitionResolver`). No reflection, so it is NativeAOT compatible; the AOT test project covers it. - **Two levels of detail**, because full schemas make big documents: | | Content | All 28 services | | --- | --- | --- | | default | paths, `operationId`s, every body a JSON object | about 12 KB | | `includeSchemas: true` | one component schema per message, structure and enumeration (100), for typed client generators | about 46 KB | The two published files are about 300 KB together. - The schemas describe the Compact JSON encoding, with the property names, types, formats and integer ranges of the published documents. DataValue is the one structure that differs: its status property is `Status`, as in Part 6 Table 42. - `OpenApiBuiltInSchemas` holds the schemas of the built-in types (StatusCode, LocalizedText, Variant, DataValue, DiagnosticInfo, ExtensionObject and the scalar types) as the encoder writes them. Abstract `Structure` fields are ExtensionObjects, abstract number fields are Variants. ### Serving the document (`Opc.Ua.Bindings.Https`) - `WebApiTransportOptions`: `ServiceSet`, `OpenApiDocumentPath` (default `null`: **off**) and `OpenApiIncludeSchemas` (default `false`). - `MapWebApiEndpoints(WebApiTransportOptions)` maps the routes of the service set and, when a path is set, the `GET` route of the document **in the same route group**. The route has no `AllowAnonymous` metadata. When an authentication scheme is registered, the contributor's `group.RequireAuthorization()` covers the document like `POST /read`; only `/findservers` and `/getendpoints` stay anonymous. The existing `MapWebApiEndpoints()` maps all routes as before. - The document is generated once per path base (listed as a relative `servers` URL; without a path base the list is omitted) by the `WebApiOpenApiGenerator` that `AddWebApiTransport` registers, or by a default one. - `Opc.Ua.Bindings.Https` references `Opc.Ua.Core.Schema` on net8.0 and later. ### Documentation - `docs/OpenApi.md` (new, linked from `docs/README.md`): serving the document, the two levels, generating clients, the JSON generated clients send, known issues, how the document is kept correct, troubleshooting. - `docs/WebApi.md`: new section "Service sets and the OpenAPI document"; the deferred follow-up is removed. `docs/SchemaGeneration.md` and both `NugetREADME.md` files describe the generator and the options. ## Comparison with the model compiler documents The two published documents are test resources of `Opc.Ua.Core.Schema.Tests` only (README next to them: source commit and update procedure). `WebApiOpenApiConformanceTests` compares the generated document with them for both service sets: - paths, `operationId`s, and the request and response schema of every operation; - the set of component schemas the operations reach: equal (100 for all services); - every enumeration (members and values); - for every structure: property names, types, formats, integer ranges, and inheritance. Everything both describe is equal. The test lists the differences and fails when a listed one disappears or a new one appears: | Difference | Generated | Published | Reason | | --- | --- | --- | --- | | DataValue status property | `Status` | `StatusCode` | Part 6 Table 42; UA-Nodeset#146 is open. The stack only knows `Status` (encoder and decoder). | | `description`, `default` keywords | not written | spec links, .NET defaults | documentation and defaults, not contract; Compact omits default values | `WebApiOpenApiWireFormatTests` validates what the binding really writes (the default request and response of all 28 routes, plus populated requests and responses with variants, status codes, references, identity tokens and notifications) against the generated component schemas. ## Tests - `Opc.Ua.Core.Schema.Tests` `OpenApi/`: `WebApiOpenApiGeneratorTests` (shape of the document, both levels, both service sets, servers list, sorting and determinism, concurrency, argument and resolver errors, every built-in type, arrays and ranks, enumerations, option sets, unions, base types, self-references, unknown definitions), `WebApiOpenApiConformanceTests`, `WebApiOpenApiWireFormatTests`, `OpenApiBuiltInSchemasTests`: 81 tests. - `Opc.Ua.Bindings.Https.WebApi.Tests`: `WebApiOpenApiDocumentEndpointTests` (TestServer with the startup contributor: off by default, configured path, both levels, service set maps and describes only the sessionless routes, path base, repeated requests, registered generator is used, **a request without credentials gets 401 when an authentication scheme is registered while one with credentials gets 200, the route carries `IAuthorizeData` and no `IAllowAnonymous`**, argument checks), options defaults and DI registration of the generator. - `Opc.Ua.Core.Tests`: `GetRoutes` for both sets and an undefined value. - `Opc.Ua.Aot.Tests`: the generator and a served document under the AOT fixture. - Results (macOS arm64): `Opc.Ua.Core.Schema.Tests` 260/260, `Opc.Ua.Bindings.Https.WebApi.Tests` 457/457, `Opc.Ua.Core.Tests` WebApi tests 110/110, each on net10.0 and net8.0 (Release). `Opc.Ua.Aot.Tests` (JIT run) SchemaAotTests 5/5, WebApiAotTests 8/8. - `Opc.Ua.Core`, `Opc.Ua.Core.Schema` and `Opc.Ua.Bindings.Https` and the three test projects build for net48, net8.0, net9.0 and net10.0 (Release) with 0 warnings and 0 errors (net48 builds only; the Windows legs of the fork CI run its tests). `dotnet format` whitespace, style and analyzers (`--severity info`) are clean on every changed file; the two `RCS1090` hits in `SchemaAotTests.cs` are on lines that master already has. - Patch coverage with `.azurepipelines/check-coverage.ps1` against master (`Opc.Ua.Core.Schema.Tests`, `Opc.Ua.Bindings.Https.WebApi.Tests` and the WebApi tests of `Opc.Ua.Core.Tests`, net10.0): 362/362 changed lines covered. - Fork CI (full matrix incl. macOS) and CodeQL on `f990af5db` (master with #4674 merged in): [CI run 37310622902](https://github.com/biancode/UA-.NETStandard/actions/runs/37310622902) green, [CodeQL run 37310626786](https://github.com/biancode/UA-.NETStandard/actions/runs/37310626786) green. The run on the earlier head `abda0e491` had one failure outside this PR: `McpServerProcessTests.HttpTransportAliasesExposeOnlyMcpRouteAsync("http")` timed out after 30 s on Windows net10.0; this PR does not touch `tools/`. ## Not in this PR - Authentication: the fix discussed in the review is @romanett's #4674. The document route uses whatever the REST routes use; this PR does not change authentication. - `StatusCode` tolerance in the decoder: dropped, the spec text is the reference. The first commit of the previous version is gone. - Samples. - Session-less invocation on the server side (OPC 10000-4 §6.3) is a separate draft PR. With `ServiceSet = Sessionless` this PR only limits the mapped routes and the document. ## Questions 1. `Opc.Ua.Bindings.Https` now references `Opc.Ua.Core.Schema` (net8.0 and later) because the generator lives next to the other schema generators and is trimmed away when unused. Fine, or should it move into `Opc.Ua.Core`? 2. Should `OpenApiIncludeSchemas` default to `false` as here, or should a host get typed clients out of the box? ## Related Issues - Fixes #4656 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
Klaus Landsdorf
|
b0d8ca0d3a |
Server: session-less service invocation (OPC 10000-4 §6.3) with AddSessionlessInvocation (#4659)
Marc answered the five design questions (2026-10-05). This update
implements the answers; see "Design decisions" at the end.
# Description
OPC 10000-4 §6.3 lets a client invoke a subset of the services without
CreateSession/ActivateSession. The request then carries either an Access
Token in `RequestHeader.authenticationToken` or no token at all. The
server today only offers the low-level
`ISessionManager.ValidateSessionLessRequest` event. Without a handler,
every request whose token names no session is answered with
`Bad_SessionIdInvalid`, including a request without a token. That is the
code a client of an expired session expects; for a session-less request,
§6.3.1 has a server that does not support the invocation answer
`Bad_ServiceUnsupported`.
This PR adds built-in session-less invocation behind an opt-in and keeps
the existing event as the way to take over completely.
## Changes
### `AddSessionlessInvocation()`
(`src/Opc.Ua.Server/Hosting/OpcUaServerBuilderExtensions.cs`)
```csharp
services.AddOpcUa()
.AddServer(options => { /* … */ })
.AddSessionlessInvocation(o => o.AllowAnonymous = false) // the default
.AddJwtIssuer(/* … */); // validates the Access Tokens
```
- Registers `SessionlessInvocationOptions`.
`DependencyInjectionStandardServer` applies them to the `SessionManager`
in use, including one created by a registered session manager factory,
unless that manager already has options.
- Without dependency injection: `SessionManager.SessionlessInvocation =
new SessionlessInvocationOptions()`.
- Without either, nothing changes apart from the
`Bad_ServiceUnsupported` answer below.
### `SessionlessInvocationOptions` (new, `src/Opc.Ua.Server/Session/`)
- `AcceptAccessTokens` (default `true`): an Access Token in
`RequestHeader.authenticationToken`, carried as a String NodeId, is
handed to the server's identity registry (`IServerIdentityRegistry`, the
one ActivateSession uses) as an issued JWT token (OPC 10000-6
§7.6.5.2.3). The resulting identity runs the request. §6.3.1 requires an
encrypted channel for the token, so it is only accepted over
SignAndEncrypt or HTTPS.
- `AllowAnonymous` (default `false`): a request without a token runs as
an anonymous user. Without it, such a request is rejected with
`Bad_IdentityTokenInvalid`.
- `MaxConcurrentRequests` (default 64) and
`MaxConcurrentRequestsPerChannel` (default 16): the budget for
session-less requests, see below. Zero does not limit; a negative value
is rejected.
### Budget for session-less requests (`SessionlessRequestBudget`, new,
internal)
#4541 bounds channels and incomplete messages. A session-less request
has no Session to be accounted for, but it occupies a worker and keeps
its channel active. It gets a budget of its own:
- **Two limits**, checked and taken together under one
`System.Threading.Lock`: concurrent requests on the server, and on one
channel. A request refused by the channel limit takes no server-wide
place. The channel is the SecureChannel on opc.tcp. The HTTPS bindings
use one SecureChannel id per listener, so their requests are grouped by
peer address (IPv4-mapped IPv6 addresses count as IPv4).
- **Answer**: `Bad_ServerTooBusy` ("The Server does not have the
resources to process the request at this time."), the code #4541's
request queue uses when it has no room. `Bad_TooManyOperations` means
too many operations in one request, so it does not fit.
- **Lifetime**: the place is taken before the identity is checked, which
also bounds the validations that run at the same time. The lease is
attached to the `OperationContext` and returned when the context is
disposed, which every service does with `using`, however the request
ends (completed, failed, cancelled). A rejected token, a handler error
or exception, and a cancelled validation return it at once. A lease
releases once, however often it is disposed. Channel entries are removed
with their last request.
- **Scope**: it applies to every session-less request on a manager that
has `SessionlessInvocation` set, including requests a
`ValidateSessionLessRequest` handler decides. A handler without
`AddSessionlessInvocation` runs without a limit, as before. Requests
with a Session token and services outside the §6.3.1 set are not
counted.
- The limits are read for each request, so a change of the options
applies to the next request.
### `SessionManager`
- **Service sets:** a request without a session is limited to the
service sets §6.3.1 allows: View (without
RegisterNodes/UnregisterNodes), Attribute, Method, NodeManagement and
Query. Every other service keeps answering `Bad_SessionIdInvalid`.
`IsSessionlessService(RequestType)` is public.
- **Request without a token:** with session-less invocation not enabled
and no `ValidateSessionLessRequest` handler, it is answered with
`Bad_ServiceUnsupported` (§6.3.1). A token that names no session stays
`Bad_SessionIdInvalid`.
- **Precedence:** a registered `ValidateSessionLessRequest` handler
still decides first, unchanged apart from the service-set limit.
`ValidateSessionlessRequestAsync` is `protected virtual` for servers
that need their own rules.
- **Roles:** the identity of a session-less request goes through the
role resolution ActivateSession applies to the effective identity of a
Session (the `ResolveRoles` part of `AddMandatoryRoles`: the
`MustChangePassword` restriction, `IRoleManager.ResolveGrantedRoles`, so
mapped Access Token claims and the built-in AuthenticatedUser role).
This includes an identity a `ValidateSessionLessRequest` handler sets.
There is no client application certificate without a Session, so no
TrustedApplication role and no application-based mapping.
- **Cancellation:** cancelling an Access Token validation propagates as
`OperationCanceledException` so the endpoint maps it to the status of
the request lifetime (previously it became `Bad_UnexpectedError`). The
budget place is still returned.
- **Log:** a rejected Access Token is logged with a dedicated
`[LoggerMessage]` (`ServerEventIds.SessionManager + 18`, the next free
id after #4632).
### Documentation
- `docs/SessionlessInvocation.md` (new, linked from `docs/README.md`):
which services, enabling it, caller identity, the answers without the
feature, the request limits, the handler, calling the server over
opc.tcp and the REST binding.
- `docs/ResourceIsolation.md`: a short section "Requests without a
Session" pointing to the budget.
- `docs/DependencyInjection.md`: `AddSessionlessInvocation` in the
identity extension tables.
## Tests
- `Opc.Ua.Server.Tests/SessionlessInvocationTests` (mocked server
internals):
- `SessionlessRequestIsUnsupportedUnlessEnabled`,
`AnUnknownSessionTokenStaysBadSessionIdInvalid`,
`ServicesOutsideTheSessionlessSetStillNeedASession` (7 services),
`TheServiceSetLimitAppliesBeforeAValidateSessionLessRequestHandler`,
`AValidateSessionLessRequestHandlerTakesPrecedenceAsync`;
-
`SessionlessRequestWithoutIdentityIsRejectedUnlessAnonymousIsAllowedAsync`,
`SessionlessAccessTokenIsValidatedByTheIdentityRegistryAsync`,
`AnAccessTokenNoAuthenticatorHandlesIsRejected`,
`SessionlessAccessTokenRequiresAnEncryptedChannel` (None, Sign),
`SessionlessAccessTokenIsAcceptedOverSignAndEncryptAsync`,
`AccessTokensCanBeDisabled`,
`IsSessionlessServiceFollowsTheServiceSetsOfPartFour` (21 cases),
`ValidateSessionlessRequestRejectsMissingArguments`;
- wiring: `AddSessionlessInvocationRegistersTheConfiguredOptions`,
`AddSessionlessInvocationRejectsANullBuilder`,
`TheHostedServerAppliesTheOptionsToItsSessionManager`,
`TheHostedServerAppliesTheOptionsToAFactoryCreatedSessionManager`,
`TheHostedServerKeepsOptionsTheSessionManagerAlreadyHas`.
- End to end against the reference server,
`Opc.Ua.Sessions.Tests/SessionlessServiceInvocationTests` (opc.tcp,
`SessionClient` on a channel without CreateSession):
- `ReadWithAnAccessTokenNeedsNoSessionAsync` (SignAndEncrypt),
`AnAccessTokenOnAnUnencryptedChannelIsRejectedAsync`,
`ARequestWithoutAnIdentityIsRejectedAsync`,
`SubscriptionsStillNeedASessionAsync`;
- one sample per service set: `ViewServicesRunWithoutASessionAsync`
(Browse, TranslateBrowsePathsToNodeIds),
`AttributeServicesRunWithoutASessionAsync` (Write, HistoryRead),
`MethodServicesRunWithoutASessionAsync` (Call),
`NodeManagementServicesRunWithoutASessionAsync` (AddNodes). They check
that the service runs and reports its operation result, not a missing
session or an internal error, since #4632 hardened only
Browse/BrowseNext against a missing session. Query is not sampled:
`StandardServer` does not implement QueryFirst/QueryNext.
- REST binding,
`HttpsWebApiIntegrationTests.SessionlessReadOverRestRunsWithAnAccessTokenAsync`
(Compact and Verbose): `POST /read` with the Access Token as String
NodeId (`"AuthenticationToken": "s=<token>"`), and the same request
without a token.
- **Budget**, `Opc.Ua.Server.Tests/SessionlessRequestBudgetTests` (22
tests):
- the budget itself: server limit and release, channel limit that takes
nothing from the server, server limit checked first, zero does not
limit, a lease releases once, HTTPS grouped by peer address (IPv4-mapped
form too), 200 concurrent callers against the limits (exactly the limit
granted, none above the channel limit, parallel release leaves zero and
no tracked channel), 16 callers acquiring and releasing in a loop
(high-water mark never above the limit);
- through `SessionManager.ValidateRequestAsync`: `Bad_ServerTooBusy`
over the server and the channel limit, disposing a context twice
releases once, places returned after a rejected Access Token, an
unencrypted channel, a missing identity, a cancelled validation (while
it waits, the place is taken), a handler error and a handler exception;
the budget also limits requests a handler decides, and a handler-only
server has none; requests outside the service set and requests a
disabled server refuses take no place; changed limits apply to the next
request and new options start a new budget;
- options: finite defaults, negative values rejected.
- **Budget end to end**,
`Opc.Ua.Sessions.Tests/SessionlessServiceInvocationTests.SessionlessRequestsRunInABudgetOfTheirOwnAsync`
(opc.tcp, requests held in the Access Token validation): the third
request over a limit of two gets `Bad_ServerTooBusy`; after release,
five sequential reads and five reads of an unknown node (operation
result bad) all pass; a channel limit of one refuses the second request
on that channel while another channel's request is admitted.
`ARequestWithoutATokenIsUnsupportedWhileTheFeatureIsOffAsync`:
`Bad_ServiceUnsupported` for a request without token on a server without
the feature, `Bad_SessionIdInvalid` for a token that names no Session.
- **Fail without the change**: the places-returned tests fail when the
budget does not release on failure (4 of 22 fail with the release in
`SessionManager` removed; the other budget tests use API this change
adds). `SessionlessRequestIsUnsupportedUnlessEnabled` and
`TheServiceSetLimitAppliesBeforeAValidateSessionLessRequestHandler` are
unchanged from the first version.
- **Review follow-up (`e9a35980b`)**:
`SessionlessAccessTokenReceivesTheRolesMappedFromItsClaimsAsync`,
`SessionlessAccessTokenWithoutTheMappedClaimIsNotGrantedTheRoleAsync`,
`AnAnonymousSessionlessRequestIsGrantedTheAnonymousRoleOnlyAsync`,
`TheIdentityOfAValidateSessionLessRequestHandlerReceivesTheMappedRolesAsync`,
`AValidateSessionLessRequestHandlerThatSetsNoIdentityKeepsNoIdentityAsync`;
end to end
`ARoleMappedNodeIsReadWithTheRolesMappedFromTheAccessTokenAsync` (the
ServerDiagnosticsSummary, readable for SecurityAdmin only, is denied for
a token whose claim is not mapped, readable once the rule maps it,
denied for another token) and
`ACancelledAccessTokenValidationAnswersTheTimeoutAndReturnsItsPlaceAsync`.
The anonymous and no-identity tests guard unchanged behavior. With the
`SessionManager.cs` change removed the three claim-mapping tests, the
end-to-end role test (`Bad_UserAccessDenied`), the end-to-end
cancellation test (`Bad_UnexpectedError` instead of `Bad_Timeout`) and
`ACancelledValidationReturnsItsPlaceAsync` fail. Lines of this commit
were not re-measured for patch coverage; the figure below is for
`496f84fce`.
- **`ClientTest`**: the three assertions that accepted
`Bad_ServiceUnsupported` too are back to master (`Bad_SessionIdInvalid`
only), so the PR no longer touches that file. Since #4646 a closed
`Session` refuses a request without authenticationToken itself; the
server code is not reached. The three tests pass with the single code
(12 cases on net10.0).
- Results on macOS: `Opc.Ua.Server.Tests` full run on net10.0 8076
passed / 9 skipped / 1 failed
(`ACancelledValidationReturnsItsPlaceAsync`, which asserted the old
`Bad_UnexpectedError` mapping; updated to the propagated cancellation
and passing); the session-less, SessionManager and role filters 368
passed / 0 failed on net10.0 and on net8.0, and the session-less
invocation and budget tests 73 passed on net10.0 after the last test was
added. `Opc.Ua.Sessions.Tests` with the
HTTPS/WebApi/WSS/OpenApi/Sessionless/ClientTest filter: 400 passed / 241
skipped / 0 failed on net10.0 and on net8.0 (the earlier host-name
failures were an environment issue of the test machine and are gone).
- Builds: `Opc.Ua.Server`, `Opc.Ua.Server.Tests` and
`Opc.Ua.Sessions.Tests` for net48, net8.0, net9.0 and net10.0 and
`Opc.Ua.Server` in Release net10.0: 0 errors, no new warnings (the
CA2213 in `SamplingGroup.cs` and the CA1861/CA2025 in other test files
are on master too).
- `dotnet format` (whitespace, style, analyzers `--severity info`, one
`--include` per changed file): no finding on a changed line. What
remains is on master lines (`SessionManager.cs` 552, 1374, 2205 and
`DependencyInjectionStandardServer.cs` 228) and the line-ending finding
of a macOS checkout.
- Coverage (`.azurepipelines/check-coverage.ps1` against master, only
the session-less tests of `Opc.Ua.Server.Tests` on net10.0): patch
105/116 changed lines (90.5 %). The 11 uncovered lines are Session-bound
lines of `ValidateRequestAsync` and the lockout logger that moved with
the change; the rest of the suite covers them. The project floor is not
meaningful for this subset.
- Fork CI and CodeQL: previous head `0fba9f21d` (master incl. #4674
merged): [CI run
37307824769](https://github.com/biancode/UA-.NETStandard/actions/runs/37307824769)
green (one net48 timing test, `KeepAliveHandlerV2Async`, failed in the
first attempt and passed on the job rerun; the PR does not touch
subscriptions), [CodeQL run
37307829138](https://github.com/biancode/UA-.NETStandard/actions/runs/37307829138)
green. Current head `e9a35980b`: [CI run
37364178979](https://github.com/biancode/UA-.NETStandard/actions/runs/37364178979),
[CodeQL run
37364182697](https://github.com/biancode/UA-.NETStandard/actions/runs/37364182697)
(results pending).
## Design decisions
Marc's answers (2026-10-05) and what the PR does:
1. **Built in** when `AddSessionlessInvocation` was used. Done.
2. **Service-set limit for all paths**: it applies before a
`ValidateSessionLessRequest` handler too. Done, unchanged.
3. **`Bad_ServiceUnsupported`** for a request without a token on a
server without the feature. Done, unchanged. A token that names no
Session stays `Bad_SessionIdInvalid`.
4. **Identity model**: an Access Token in the RequestHeader, validated
as an issued JWT token through the identity registry, anonymous only by
opt-in. Unchanged. The REST authentication is #4674, now in master.
5. **Separate budget** for session-less requests: added, see "Budget for
session-less requests". Open: no limit per identity yet (identities are
claims-defined, so the keys would be unbounded); say if you want one.
## Related Issues
- Fixes #4658
- Relates to #4657 (the REST binding's session-less service set), #4646
(a closed `Session` no longer sends requests without token), #4541
(resource isolation) and #4674 (REST authentication)
## Checklist
- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed. (Affected
suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered
by the fork CI's Windows net48 legs.)
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
---------
Co-authored-by: Marc Schier <marcschier@users.noreply.github.com>
|
||
|
romanett
|
7212607f16 |
OPC UA Part 3 compliance: permissions, Write semantics, NodeManagement, NodeSet DataTypes, locales (#4675)
# Description Fixes the High and Medium findings of an OPC UA Part 3 (OPC 10000-3, Address Space Model) compliance audit. Each finding was checked against the full spec text (Parts 3, 4, 5, 6, 14 and 18) before it was fixed. Every fix comes with regression tests that fail without it. ### Role permissions (Part 3 §8.55, Part 4 §5.12.2.2) - The type-hierarchy exemption in `MasterNodeManager.ValidateRolePermissions` / `ValidateAccessRestrictions` now covers only Browse, Read and ReadRolePermissions. Type nodes stay visible to generic clients, which was the intent of #3705; every other permission bit is now enforced on them. - `ReceiveEvents` is enforced on the EventType node. The standard NodeSet grants it on `AuditEventType` and its subtypes only to SecurityAdmin, so only SecurityAdmin sessions now receive audit events. - Call checks the `Call` permission and AccessRestrictions on the Object as well as on the Method. - When a Call's `methodId` is the Method declaration of the ObjectType, the Object's Method with the same BrowseName is permission-checked and invoked. The declaration is used only if the Object has no such Method. This applies to `AsyncCustomNodeManager`, `CustomNodeManager2` and the sample node manager. ### Write service (Part 3 §5.6.2, §8.40, §8.57; Part 4 §5.11.4) - The synchronous `OnWriteValue` path (fluent `OnWrite`) checks DataType and ValueRank first (`Bad_TypeMismatch`). After an IndexRange write it merges the slice into the cached value instead of replacing it. - StatusWrite / TimestampWrite are enforced (`Bad_WriteNotSupported`). - Undefined enumeration values and invalid OptionSet writes return `Bad_OutOfRange`. OptionSet writes merge only the valid bits into the stored value. - New `ReportPropertyValueChanged` / `ReportSemanticChange` let server code raise SemanticChangeEvents for changes it makes itself. - Required follow-up (Part 14 §6.2.11.1): if an external target rejects a timestamp, the PubSub subscriber's TargetVariables writer retries without it. ### NodeManagement and Call arguments (Part 3 §5.5.2, §5.7.3, §6.4.2; Part 4 §5.8, §5.12.2) - AddNodes creates the Mandatory InstanceDeclarations of the TypeDefinition and rejects abstract or Interface types (`Bad_TypeDefinitionInvalid`). - AddReferences, DeleteReferences and DeleteNodes bump NodeVersion and raise GeneralModelChangeEvents. - Clients may omit trailing optional input arguments (`HasOptionalInputArgumentDescription`). ### NodeSet DataTypes (Part 3 §5.8.3, §8.48; Part 6 Annex F.12) - `UANodeSet.Import` completes structure definitions with the base type's fields (base fields first). It creates definitions for structures without fields and sets `DefaultEncodingId`. `StructureDefinition.FirstExplicitFieldIndex` marks the subtype's own fields. - `DataTypeState` no longer mutates the stored definition on read. ### Locales (Part 3 §5.2.5, §8.5; Part 4 §5.4) - Read, Browse and data change notifications return the translation that best matches the session's LocaleIds, or all translations for `mul`. The selection lives in the server dispatcher and is built on `ResourceManager`: the new `TranslateValue` calls the overridable `Translate`. - Locale ids are matched case-insensitively, and fallback translations carry their real locale. ### Client complex types (Part 6 §5.1.7) - The Reflection.Emit builder encodes only `IsOptional` subtyped fields as ExtensionObject. It also supports OptionSet subtypes: an unsupported type is skipped instead of aborting the whole load. - `TypeInfo.GetXmlName` uses the data type's XML name for dynamic structures and OptionSets (the XML body element was `<Structure>`). Both builders now produce identical XML. ## Behaviour changes All are documented in `docs/MigrationGuide.md`. - **Audit events:** only SecurityAdmin receives them. The CTT "Auditing Connections" unit needs a SecurityAdmin user (e.g. sysadmin); `AuditingOperationTests` was updated. - **Write timestamps and status:** clients writing a SourceTimestamp or a non-Good StatusCode get `Bad_WriteNotSupported` unless the Variable has StatusWrite/TimestampWrite. No variable in `src/` sets these bits. - **AddNodes:** it rejects abstract types (including `BaseVariableType` and `BaseEventType`), and new instances now come with child nodes. - **Call:** the Object needs the Call permission. The Object's own Method runs even when the call names the type declaration. ## Not included - Low-severity findings: missing validation on optional services and author-side NodeSet rules. - LocalizedText nested inside structures or Variant arrays, and in HistoryRead or Call outputs. ## Testing net10.0, Debug, one suite at a time on the merged branch. All suites below pass: | Suite | Passed | |---|---| | Types | 11430 | | Core.Encoders | 2947 | | Core.Schema | 179 | | Core.Security | 558 | | Client | 2761 | | Server | 8210 | | InformationModel | 1080 | | Features | 145 | | History | 536 | | Di | 455 | | Machinery | 167 | | Robotics | 661 | | Robotics.Intent | 22 | | Pumps | 75 | | Scales | 122 | | Positioning | 69 | | ISA95 | 144 | | Vision | 484 | | PubSub | 1490 | | PubSub.Adapter | 151 | | PubSub.Server | 177 | | Redundancy.Server | 856 | | Lds | 173 | | Configuration | 241 | | WotCon | 1496 | - **Client.ComplexTypes:** one test wrote a SourceTimestamp to a variable without TimestampWrite. It is fixed in this PR, and the suite has not been re-run since. - **Core:** 6068 passed, 1 failure in `ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`. That telemetry test passed when re-run alone, so it looks flaky. - **Still running:** Sessions, Subscriptions (Classic, Durable, standard) and Interop. Results will follow on this PR. - `UA.slnx` builds with 0 errors for net10.0 and net48, with no new analyzer warnings in changed files. ## Related Issues - Part 3 compliance audit (no tracking issue) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
8bdc13ce27 |
WebApi: enforce AddWebApi*Auth on the HTTPS listener's own host (REST routes were served unauthenticated) (#4674)
# Description `AddWebApiBasicAuth` / `AddWebApiBearerAuth` / `AddWebApiMutualTlsAuth` registered their ASP.NET Core authentication schemes only on the **application** `IServiceCollection`. `HttpsTransportListener` serves the REST routes from its **own** Kestrel host with a separate DI container (own-host and shared-host paths), into which only `IHttpsListenerServiceContributor.ConfigureServices` registrations flow. `WebApiHttpsStartupContributor.Configure` checked for schemes in the listener container, found none, and therefore skipped `UseAuthentication()` / `UseAuthorization()` and `RequireAuthorization()`. Result: on a real listener the REST routes (`/read`, `/write`, `/call`, sessions, subscriptions, …) were served **without** the configured HTTP credential. The existing unit tests only checked the application container, so they passed. Present since the WebApi binding was introduced (#3880). ## Changes - **Record + replay.** Each `AddWebApi*Auth` opt-in still registers its scheme on the application container (so hosting `MapWebApiEndpoints()` in your own ASP.NET Core app keeps working), and also records it as an internal `WebApiListenerAuthRegistration`. `AddWebApiTransport()` now hands the application `IServiceProvider` to `WebApiHttpsStartupContributor`, whose `ConfigureServices` replays the recorded schemes, the policy scheme and the application's `ISessionlessIdentityProvider` into the listener container. Auth middleware and `RequireAuthorization()` are now installed on the pipeline that actually serves the routes. - Side effects fixed by the same change: `UseJwtClaimIdentityProvider()` / Bearer claims mapping now reaches the dispatcher, and the WSS `opcua+openapi+<accesstoken>` upgrade finds the Bearer scheme instead of always failing closed. - **No-credentials fallback scheme.** Previously the policy selector returned `null` for a request without a credential; ASP.NET Core's `PolicySchemeHandler` then throws on challenge (→ 500). The selector now forwards to a new internal `WebApiNoCredentialsAuthenticationHandler` that authenticates nobody and answers the challenge with **401** plus the `WWW-Authenticate` challenges of the registered Basic / Bearer schemes. - **Selector only picks registered schemes.** A `Bearer` header when only Basic is registered (or vice versa) used to forward to a non-existent scheme (→ 500); it now falls through to the no-credentials scheme (→ 401). ## Reviewer notes - Only options configured through the opt-in callbacks are replayed. Options added separately via `services.Configure<JwtBearerOptions>(scheme, …)` on the application container do not reach the listener container. - mTLS-only: a request without a client certificate gets 401 without a `WWW-Authenticate` header (there is no HTTP challenge for client certificates). - Discovery routes (`/findservers`, `/getendpoints`) keep their `AllowAnonymous` metadata. ## Tests New `RealHttpsListenerIntegrationTests.Authentication.cs` builds the services exactly as an operator would (`AddOpcUa().AddWebApiTransport().AddWebApi*Auth(...)`) and posts `/read` through a real `HttpsTransportListener` + Kestrel TLS endpoint: | Auth | Credential | Expected | |---|---|---| | Basic | none / wrong password / Bearer token | 401 + `WWW-Authenticate: Basic`, not reaching the server | | Basic | valid | 200 | | Bearer | none / `not.a.jwt` / JWT with wrong key / Basic | 401 + `WWW-Authenticate: Bearer`, not reaching the server | | Bearer | valid JWT | 200 | | mTLS | none / Basic | 401, not reaching the server | | Basic | `/findservers` without credential | 200 | With the replay disabled, all 9 negative cases fail with 200 (reproducing the bug). Locally on net10.0: `Opc.Ua.Bindings.Https.WebApi.Tests` 450/450, WebApi/OpenApi tests in `Opc.Ua.Sessions.Tests` 17/17; `Opc.Ua.Bindings.Https` builds for all target frameworks with no warnings. ## Related Issues - No tracking issue; reported externally. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Klaus Landsdorf
|
b25c1f6ad4 |
Asset Management Basics: the OPC 10000-110 companion SDK (#4662)
# Description Stacked on #4650. Until that is merged, this PR also shows its commits; the six commits of this PR start after `fix(server): keep conformance unit aggregation idempotent and ordered with registrations`. This adds OPC 10000-110 Asset Management Basics 1.01.1 (`http://opcfoundation.org/UA/AMB/`) as a companion SDK, in the pattern of the DI, Machinery, Pumps and Scales packages. The tracking issue #4649 describes the scope, the design and the open questions. In short: | Package | Content | |---|---| | `Opc.Ua.AMB` | the model generated by the source generator from the unmodified NodeSet, plus the contracts server and client share (`AssetFaultSeverity`, `AssetLocationKind`, `MaintenanceStateKind`, `DocumentationLinkProperties`) | | `Opc.Ua.AMB.Server` | a sidecar node manager next to the owner of the Device Integration address space (`AddOpcUaDi`, `AddMachinery`, `AddPumps` or `AddScales`), the asset registry `IAssetManagement` with a fluent builder, `AddAssetManagement()` hosting, memory and file system stores, the AMB conformance units and server facet | | `Opc.Ua.AMB.Client` | `AmbClient`: discovery, identification, health alarm and maintenance streams, documentation links, locations, structure and snapshots, plus `AddAssetManagementClient()` | **Sidecar.** An asset stays in the node manager that owns it. Everything AMB adds to it (`DeviceHealth`, health alarms, maintenance conditions, the DocumentationLinks AddIn, AssetId and the other properties) is created there. The AMB node manager serves the AMB model, the alias categories `Assets`, `AssetsByProductInstanceUri` and `AssetsByAssetId` below `0:Aliases`, the location hierarchies below `0:Locations` and the server-specific alarm and condition types. Owners register assets with the shared registry from their setup, in either startup order. **Implemented conformance units** (Table 54): Asset Identification, Configurable Asset Identification, Discovery by ProductInstanceUri and by AssetId, Health Status Base, Alarms, Root Causes and Alarm Categories, Version Information, Operation Counters (inspected only; AMB defines no counters), DocumentationLinks Base, Edit Base and Edit Advanced, Requirements, Capabilities, Classification, Current and Future Maintenance Activities, Local Time, the three location properties and both location object hierarchies, Sub-assets and Asset relations. The two history units (Health Tracking, Past Maintenance Activities) are not part of this PR. `docs/AMB.md` has the matrix with the rule the server applies for each unit and whether a test checks it over a session. **Dependency on #4650.** The health alarms and maintenance conditions are created through `CreateAlarm<TState>`, and the conformance units follow the registered assets through the recomputation in `ConformanceUnitsManager`. ## Changes Six commits, each building on its own: 1. **`feat(amb)`: model package** (`src/Opc.Ua.AMB/`, about 2.4k lines, of which the NodeSet and the two identifier tables are 1.6k). - The NodeSet is the publication from `UA-Nodeset/latest/AMB`, byte for byte (git blob `21f9475b646e9c5508171277ffe5b74edbeb0a58`). `.gitattributes` checks it out with LF endings, because a test pins its SHA-256. - The published identifier table is vendored unchanged; a derived table pins the symbol-to-NodeId mapping, and the generator fails the build when they disagree. `tools/nodesets/README.md` documents the one new kind of renamed symbol (top-level instances below `0:Aliases` and `0:Locations`). 2. **`feat(amb)`: server package** (`src/Opc.Ua.AMB.Server/`, about 10.7k lines). Node manager, registry and builders, discovery, health, maintenance (`MaintenanceEventStateMachine` with all transitions), documentation links, locations, version information, classification (IRDI dictionary entries below `Server/Dictionaries`) and structure, stores, conformance evaluation, hosting. Log messages use the EventId block 8300 to 8379, after Pumps (8100) and Scales (8200). 3. **`feat(amb)`: client package** (`src/Opc.Ua.AMB.Client/`, about 3.0k lines). Calls AddLink and RemoveLink through the generated `DocumentationLinksTypeClient`. 4. **`test(amb)`: test suite** (`tests/Opc.Ua.AMB.Tests/`, 163 tests, about 8.8k lines; commit 6 adds one more). 5. **`docs(amb)`: guide and samples**: `docs/AMB.md`, links from `README.md`, `docs/README.md`, `docs/Samples.md` and `docs/AliasNames.md`; `samples/AMB/AmbServer` (AMB next to `AddMachinery`: two DI devices and a Machinery press as assets, with a simulation) and `samples/AMB/AmbClient`. The sample server listens on 62553, which no other sample uses. 6. **`feat(amb)`: tell the links users added from the links of the manufacturer.** OPC 10000-110 gives a client no way to see which links `RemoveLink` accepts (§10.5.4: only those `AddLink` created). As §10.5.1 lets vendors add metadata Properties to the links, the server gives every link `AddLink` created, and every such link restored after a restart, the `Boolean` Property `UserLink` (`true`), qualified with its namespace of server-specific types, with a NodeId derived from the persisted link identifier; declared links carry none, and `RemoveLink` deletes it with the link. `DocumentationLinkRecord` gets the additive `IsUserLink`, which `ReadDocumentationLinksAsync` fills with one Browse over all links and one Read of the markers found (comparing the name only, as the namespace is configurable; `false` for a server that does not mark links). The name is shared as `DocumentationLinkProperties.UserLink`. `UA.slnx` and `.azurepipelines/expected-packages.txt` get the new projects and packages in the commit that adds them. All projects target `net48;net8.0;net9.0;net10.0` through the repository properties (`LibTargetFrameworks`, `TestsTargetFrameworks`, `AppTargetFrameworks`); `IsAotCompatible` and `PublishAot` are set from net10.0 on, as in the other companion projects. ## Tests - **`Opc.Ua.AMB.Tests`: 164 tests**, most of them against a hosted server over real client sessions: - the model: the SHA-256 of the NodeSet and the generator behaviours AMB is the first companion model to rely on; - each building block in-process and over a session: discovery through every category with NodeVersion and model change events, AssetId writes, health alarms and maintenance activities through all transitions with acknowledgement, AddLink and RemoveLink, locations and structure, the advertised conformance units and the facet; - a restart with the file system stores (links and AssetId survive); - the `UserLink` marker: present on an added link only, the same NodeId after a restart, removed with `RemoveLink`, and `IsUserLink` on the client in the snapshot, after `AddLink`, after a restart and after `RemoveLink`; - composition with `AddOpcUaDi` (both startup orders), `AddPumps` and `AddMachinery`; - `AmbClient` end to end against a hosted server, including a Machinery machine as an asset. - **Results (macOS, local):** 164 passed / 0 failed on net10.0 and on net8.0. .NET Framework is covered by the fork CI's Windows legs. - **Coverage** of the three AMB assemblies (coverlet with `tests/coverlet.runsettings.xml`, net10.0, generated `*.g.cs`, `*TypeClient` proxies and `*Log` classes excluded): lines 93.9 % (4890 of 5206), branches 84.0 % (1294 of 1539); model 100 %, client 94.5 %, server 93.4 %. - **Build:** the three packages, the tests and the two samples build without warnings for net48, net8.0, net9.0 and net10.0; `UA.slnx` builds in Release (its warnings are all in files this PR does not touch, as on master). `dotnet format` (whitespace, style, analyzers with `--severity info`) reports nothing on the 95 changed C# files. - **Samples:** `AmbServer` and `AmbClient` ran against each other on net10.0: the client discovered the three assets, printed their snapshots and the location hierarchies, added and removed a documentation link and streamed alarms and maintenance. - Fork CI (full matrix incl. macOS) and CodeQL: green on the same five package commits before the rebase onto the current #4650 head — [CI run 37144486107](https://github.com/biancode/UA-.NETStandard/actions/runs/37144486107), [CodeQL run 37144487873](https://github.com/biancode/UA-.NETStandard/actions/runs/37144487873). The run on the current head (rebased, plus the user-link commit; AMB tests 164/164 locally on net10.0) is in progress: [CI run 37203517457](https://github.com/biancode/UA-.NETStandard/actions/runs/37203517457). ## Notes for review - **Size.** About 26.8k lines, larger than each of the companion PRs #4577 to #4580. Without changes to the code it splits into the model, server and server tests, and the client, end-to-end tests, samples and docs on top; the issue asks which you prefer. - **Interfaces on types.** §9.4.1 and §12.1 ask for alarm and condition types that implement `IRootCauseIndicationType` and `IMaintenanceEventType`. The server creates subtypes of the DI alarm types in its own type namespace (`urn:opcua-netstandard:amb:types`); `UseServerDefinedAlarmTypes = false` uses `HasInterface` on the instance instead, and then does not report the units that need the type-level interfaces. - **Conformance unit names** follow Tables 54 to 56; the profile database has no AMB entries yet. The QualifiedNames are in namespace 0, as DI and Machinery publish theirs (question 5 in the issue). - **Generator gap.** Children a NodeSet gives to a top-level instance (here the `NodeVersion` properties of the alias categories) get no identifier constants. The fix changes the identifiers of other generated models, the reference server's among them, so it is not part of this PR; `AmbModelTests.NodeVersionHasNoIdentifierConstant` records it and fails once the generator emits the constants. - **net48.** The code that the SDK so far only built for .NET 8 and later uses the `NET5_0_OR_GREATER` pattern of `JsonEncoder` for `Enum.IsDefined` and API forms that .NET Framework has (`string.Format`, `Guid.ToString(format, provider)`); the tests use a small async file helper instead of `File.WriteAllTextAsync`. - **User links.** The `UserLink` Property is metadata of this server, not an AMB name: commit 6 explains why and how a client uses it. Changed lines of that commit covered: 69/70 (the one line left is the loop continuation for a link that has other Properties but no marker). - **Known limitations** are listed in the guide: the `DeviceHealth` SourceTimestamp across a restart, removing optional maintenance members once set, `PatchIdentifiers`, locations only at the deepest level, and the history units. ## Related Issues - Part of #4649 - Stacked on #4650 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (The AMB suite on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
Klaus Landsdorf
|
15f9a5ebcc |
Machinery client: enumerate each machine, component and process value once (#4672)
# Description `MachineryClient` browses the `Machines` folder, the `Components` add-in, the `MachineryBuildingBlocks` organizer, the energy resources and the process values with `HierarchicalReferences` and subtypes, and turned every returned `ReferenceDescription` into a `MachineEntry`. Browse returns one description per reference, not per node, and `HasEventSource` and its subtype `HasNotifier` are hierarchical references. A machine that the `Machines` folder organizes and that is also registered as a notifier of the folder - how a server exposes its event hierarchy below the `Server` object once alarms are attached - was therefore returned twice by `EnumerateMachinesAsync` and `DiscoverMachinesAsync`. The same held for a component that is an event source of its `Components` folder and for a process value that is an event source of `Monitoring`. The fix: - The shared browse helper reports each target NodeId once and keeps the first description. Its `BrowseName`, `DisplayName` and `TypeDefinition` belong to the node and are the same for every reference that leads to it. - All hierarchical references are still followed. Restricting the browse to `Organizes` would lose machines, components and process values a server references with another hierarchical reference type such as `HasComponent`, so an address space without duplicate references returns exactly what it returned before. - `EnumerateProcessValuesAsync` looks below the machinery item and below its `Monitoring`; a process value reachable from both is now reported once as well. The public API does not change. ## Changes - `src/Opc.Ua.Machinery.Client/MachineryClient.cs`: `BrowseChildrenAsync` skips a NodeId it has already returned; the remarks of `EnumerateMachinesAsync` say so. - `src/Opc.Ua.Machinery.Client/MachineryClient.ProcessValues.cs`: `EnumerateProcessValuesAsync` skips a process value below `Monitoring` it has already returned below the item. Every enumeration of the client goes through `BrowseChildrenAsync`: `EnumerateMachinesAsync`, `DiscoverMachinesAsync`, `EnumerateComponentsAsync`, `EnumerateBuildingBlocksAsync`, `EnumerateEnergyResourcesAsync`, `EnumerateMeteringPointsAsync`, `EnumerateSubMetersAsync`, `EnumerateProcessValuesAsync`, and the child walks of `ReadLifetimeCountersAsync`, `EnumerateEquipmentAsync`, `ReadMeteringPointAsync` and `ReadPublishedResultsAsync`. ## Tests - **New, end-to-end** (`MachineryEndToEndTests.NodesThatAreAlsoEventSourcesOfTheirParentAreEnumeratedOnceAsync`): the test press links the machine to `Machines` with `HasNotifier`, the `HydraulicUnit` component to `Components` and the oil-temperature process value to `Monitoring` with `HasEventSource`, each in addition to its organizing reference, and expects each node once from `EnumerateMachinesAsync`, `DiscoverMachinesAsync`, `EnumerateComponentsAsync` and `EnumerateProcessValuesAsync` over `opc.tcp`. - **New, unit** (`MachineryClientTests.ANodeReachedThroughTwoReferencesIsEnumeratedOnceAsync`): a browse result that carries the same machine under `Organizes` and `HasNotifier`. - **Fail without the fix:** with `src/Opc.Ua.Machinery.Client` from master, all four assertions of the end-to-end test fail - two machines, two NodeIds, two components and two process values instead of one each. - `Opc.Ua.Machinery.Tests` passed 167/167 on net10.0 and 167/167 on net8.0 (macOS). - `Opc.Ua.Machinery.Client` builds for net48, net8.0, net9.0 and net10.0 with 0 warnings. - `dotnet format` (whitespace, style, analyzers) on the changed files: nothing on the changed lines (the `ENDOFLINE` findings concern the whole of every file in a checkout with LF line endings). - Changed lines covered: 8/8 (100 %). - Fork CI (full matrix incl. macOS) and CodeQL: [CI run 37201766371](https://github.com/biancode/UA-.NETStandard/actions/runs/37201766371) and [CodeQL run 37201768280](https://github.com/biancode/UA-.NETStandard/actions/runs/37201768280), both green. ## Notes for review - No extra request: the de-duplication happens on the result of the browse the client already made. - Of the sibling companion clients, `PumpsClient.EnumeratePumpsAsync` and `ScalesClient.EnumerateScalesAsync` already de-duplicate across their two roots. `EnumeratePumpsUnderAsync`, `EnumerateScalesUnderAsync`, `EnumerateSystemScalesAsync` and the DI device discovery follow the same browse pattern without de-duplication; they are not changed here. ## Related Issues - Fixes #4670 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suite on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
Klaus Landsdorf
|
dbee7b9988 |
Robotics client: fill the engineering units and ranges of the axis values and the load mass, and the gear ratio (#4661)
# Description Follow-up to #4619. It was prepared stacked on #4619 and is now one commit on top of master, with #4619's review changes in place. `AxisSnapshot.Engineering`, `LoadSnapshot.MassEngineering` and `GearSnapshot.GearRatio` are part of the snapshot contracts and documented in `docs/Robotics.md` (the URML capability manifest takes the joint limits from them), but `RoboticsClient` never set them: every axis came back without units or limits, every load without the unit of its mass and every gear without its ratio. OPC 40010-1 declares `ActualPosition` (mandatory), `ActualSpeed` and `ActualAcceleration` (optional) below the axis's DI `ParameterSet`, and the load's `Mass`, as `AnalogUnitType` variables with a mandatory `EngineeringUnits` and an optional `EURange` property, and `GearRatio` as a mandatory `RationalNumber` variable of the gear. The client now reads them: - **Axis:** `ReadAxisAsync`, and with it `ReadSystemAsync`, resolves the `EngineeringUnits` and `EURange` of the three axis values with one TranslateBrowsePathsToNodeIds and reads them with one Read per axis (none when nothing resolves) into `PositionUnit`, `SpeedUnit`, `AccelerationUnit` and `Limits.Position/Speed/Acceleration`. - **Load:** `Mass` is resolved and read together with its `EngineeringUnits` and `EURange` (`MassEngineering`), replacing the separate resolve and read. - **Gear:** `GearRatio` is read in the same request as `Pitch`. - Loads and gears cost no extra request. - A property the server does not publish, whose read fails or whose value has a bad status stays `null`. The structures go through the existing `ToStructure<T>`, which checks `StatusCode.IsGood` as for `MotorTemperatureEngineering`. The rest of the snapshot is still returned. - Because `Mass` and `Pitch` are now part of that request, a failed `Mass` or `Pitch` read keeps the server's status in the `DataValue`, like the axis values and `MotorTemperature`. Before, `Session.ReadValueAsync` threw and failed the whole snapshot read. The public API does not change. ## Changes - `src/Opc.Ua.Robotics.Client/RoboticsClient.Topology.cs`: - new `ReadAxisEngineeringAsync`, called from `ReadAxisAsync`; - `ReadLoadAsync` and `ReadGearAsync` use the existing `ReadChildrenAsync`. - `docs/Robotics.md`: - describes the members and their null cases; - adds them to the values that carry no NodeId; - names `AxisSnapshot.Engineering` instead of the non-existent `AxisSnapshot.Limits` in the URML section. ## Tests - **New, unit** (`Opc.Ua.Robotics.Tests`, `RoboticsClientFacadeTests`): - `EngineeringValuesAndTheGearRatioComeFromTheVariablesTheyDescribe`: the values are checked against same-named decoys below the axis and a `GearRatio` in namespace 0. - `AbsentOrUnreadableEngineeringValuesAndGearRatiosStayNull`: a missing `EURange`, a load without engineering properties, a failing unit read and a failing ratio read. - `BadQualityEngineeringValuesAndGearRatiosReadAsNoStructure`: a range, a unit and a ratio with a bad status read as `null`. - `AFailedMassOrPitchReadKeepsTheStatusAndTheSnapshot`. - `AnAxisWithoutAParameterSetHasNoEngineeringValues`. - **New, end-to-end** (`RoboticsClientServerE2eTests.EngineeringValuesAndGearRatiosAreReadAsync`): - the test cell now publishes a unit and range for `ActualSpeed`, a unit without a range for `ActualAcceleration`, a range for the flange load's mass and a tool load with a mass only; - the test reads the units, ranges and gear ratios back through `ReadSystemAsync` and `ReadAxisAsync`. - **Fail without the fix:** with `src/Opc.Ua.Robotics.Client` from master, five of the six new tests fail. Four get `null` units, ranges or ratios. `AFailedMassOrPitchReadKeepsTheStatusAndTheSnapshot` fails with `ServiceResultException` `Bad_NodeIdUnknown`. `AnAxisWithoutAParameterSetHasNoEngineeringValues` passes there by design: it pins that an axis without a `ParameterSet` still reads without error. - `Opc.Ua.Robotics.Tests` passed 661/661 on net10.0 and 601/601 on net8.0 (macOS). - `Opc.Ua.Robotics.Client` builds for net48, net8.0, net9.0 and net10.0 with 0 warnings. - `dotnet format` (whitespace, style, analyzers) on the changed files: nothing on the changed lines beyond the `IDE0002` info that the existing `RoboticsSessionHarness.Ua(Opc.Ua.BrowseNames.…)` calls in the same test class also get. - Changed lines covered: 52/52 (100 %). - Fork CI (full matrix incl. macOS) and CodeQL: [CI run 37147764118](https://github.com/biancode/UA-.NETStandard/actions/runs/37147764118) and [CodeQL run 37147766527](https://github.com/biancode/UA-.NETStandard/actions/runs/37147766527). CodeQL is green; CI is green except `Opc.Ua.Interop.Tests` (`LegacyClientExtendedCheckAsync("ComplexTypes")` / `LoadAndDecodeLegacyComplexTypesAsync`), which is flaky on master since #4630 (details in #4617), and except one Windows run of `StartIsIdempotentAndDisposeStopsListenerAsync` in `Opc.Ua.Redundancy.Kubernetes.Tests` (HttpListener prefix already registered), which this change does not touch either. ## Notes for review - `ReadAxisAsync` costs one TranslateBrowsePathsToNodeIds and at most one Read more per axis. The other snapshots keep their request count. - `AxisSnapshot.Engineering`, `LoadSnapshot.MassEngineering` and `GearSnapshot.GearRatio` are read once and carry no NodeId, like `MotorTemperatureEngineering`. ## Related Issues - Fixes #4660 - Follows up #4619 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suite on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
Klaus Landsdorf
|
3690341c80 |
DI: wire the PrepareForUpdate and Installation methods per spec and advertise only the facets that are met (#4617)
# Description
**Software update methods** (`fix(di): wire PrepareForUpdate
Abort/Resume and Installation Resume`)
- New internal `SoftwareUpdateTransitions` (one per `SoftwareUpdate`)
checks the current state and moves it under one lock, so concurrent
calls cannot both pass a check.
- `Prepare` (§8.4.8.3) is accepted only from `Idle`. The application's
`OnPrepare` callback gets a token that `Abort` cancels; the token source
is disposed only after both the Prepare call and a racing Abort are done
with it, so a handler still running after Abort can use its token.
- `Abort` (§8.4.8.4) moves `Preparing` to `Idle` (`PreparingToIdle`),
cancels the preparation in flight and reports `Failed` ("Aborted")
through `OnPrepareStateChanged`; the aborted `Prepare` call returns
`Bad_InvalidState`. Outside `Preparing` it returns `Bad_InvalidState`.
- `Resume` (§8.4.8.5) no longer runs the Prepare callback. It moves
`PreparedForUpdate` through `Resuming` to `Idle` and is rejected with
`Bad_InvalidState` outside `PreparedForUpdate` or while the installation
is `Installing`.
- `Installation.Resume` (§8.4.9) moves `Error` to `Idle`
(`ErrorToIdle`),
`Bad_InvalidState` otherwise.
- The dispatcher knows the `Resuming` state and the `ResumingToIdle`
transition, so `CurrentState` / `LastTransition` show their names.
**Advertising** (`fix(di): advertise only the DI facets whose mandatory
units are met`)
- `DI Offline`, `DI SU UpdateStatus`, `DI SU FileSystem Loading`,
`DI SU Installation for File System` and `DI SU Installation for Cached
Loading` are no longer claimed (see the issue for why each is unmet).
- The facets are derived from the advertised units with the mandatory
units of the DI 1.05 profile database, so a facet cannot be listed
without them. Today this leaves Locking and Software Update Base;
DeviceIntegrationHost, FileSystem, Direct and Cached Loading come back
by themselves once the missing pieces are built.
- `docs/DeviceIntegration.md` documents both tables and the missing
pieces.
Tests: `SoftwareUpdateMethodTransitionTests` (9 tests) and
`DiNodeManagerConformanceTests` (5 tests) are new;
`DiConformanceFacetTests`
and `PumpAddressSpaceComplianceTests` expected the old claims and are
updated. All 9 transition tests and 8 of the 10 conformance tests fail
against master; the other two are the unchanged locking test and a guard
that every advertised facet carries its mandatory units (master
over-claims
the units too, so the guard alone holds there).
Behaviour change for users: a plain DI server no longer lists the
DeviceIntegrationHost facet, and Direct/Cached/Package software updates
list only Software Update Base.
Fork CI (full matrix incl. macOS) and CodeQL: green — [CI run
36779552603](https://github.com/biancode/UA-.NETStandard/actions/runs/36779552603),
[CodeQL run
36779556125](https://github.com/biancode/UA-.NETStandard/actions/runs/36779556125).
## Related Issues
- Fixes #4616
## Checklist
- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
---------
Co-authored-by: Marc Schier <marcschier@users.noreply.github.com>
|
||
|
Klaus Landsdorf
|
a3166fa560 |
WebApiClient: report HTTP failures as StatusCodes and accept gzip responses (#4648)
# Description `WebApiClient` threw the exceptions of `HttpClient` for every HTTP failure except 429/503. It now throws `ServiceResultException` with a matching StatusCode, as the transport channels already do. It can also ask for and read gzip compressed JSON (OPC 10000-6 §7.4.5). Two commits: the error mapping, then gzip. ## Changes ### HTTP failures (`src/Opc.Ua.Client/WebApi/WebApiClient.cs`, new internal `WebApiHttpErrors`) | HTTP / failure | StatusCode | |---|---| | 400 | `Bad_DecodingError` | | 401 | `Bad_IdentityTokenInvalid` when the client sent no credentials (bearer token or basic credentials), otherwise `Bad_UserAccessDenied` | | 403 | `Bad_UserAccessDenied` | | 404, 405, 501 | `Bad_ServiceUnsupported` | | 408, 504 | `Bad_Timeout` | | 413 | `Bad_RequestTooLarge` | | 415 | `Bad_DataEncodingUnsupported` | | 429, 503 | `Bad_ServerTooBusy` with the Retry-After hint (unchanged) | | 500 | `Bad_InternalError` | | any other status | `Bad_CommunicationError` | | elapsed `RequestTimeout` / `HttpClient.Timeout` | `Bad_RequestTimeout`, as in `HttpsTransportChannel` and `WebApiTransportChannel` | | failure below HTTP (connection, TLS handshake) | the code `HttpsTransportChannel.MapRequestFailure` assigns, e.g. `Bad_NotConnected` for a refused connection | - The message names the HTTP status, the reason phrase and the route. - For an HTTP status, the inner exception is an `HttpRequestException` naming it (with `StatusCode` set on .NET 5 and later). For a failure below HTTP, it is the original `HttpRequestException`; for a timeout, the `OperationCanceledException`. - A cancellation of the caller's token stays an `OperationCanceledException`. - `WebApiTransportChannel` sends through `WebApiClient`. For an HTTP error status it now reports the mapped code instead of `Bad_UnknownResponse`; its codes for timeouts and connection failures do not change. - `docs/WebApi.md`: new section "Client errors". ### gzip (`WebApiClientOptions.AcceptCompressedResponses`, `src/Opc.Ua.Core/Stack/Https/HttpResponseBodyReader.cs`) - **Requesting gzip:** `AcceptCompressedResponses` (default `false`) sends `Accept-Encoding: gzip`; on a shared `HttpClient` it goes on each request instead of the default headers. `WebApiTransportChannel` passes the option through. - **Reading gzip:** `HttpResponseBodyReader` inflates a body with `Content-Encoding: gzip`. - `MaxMessageSize` applies to the inflated size, so a small compressed body cannot expand past it. - A corrupt gzip body is `Bad_DecodingError`. - A content coding other than gzip or identity is rejected with `Bad_DecodingError` instead of being decoded as if it were plain. - **Stream ownership:** the reader releases only its inflating wrapper. The response stream stays with the `HttpContent` the caller disposes. - `Opc.Ua.Core` references `System.IO.Compression` on net48 explicitly (it compiled through the expanded .NET Standard facades before, but should not depend on them). - `Opc.Ua.Client` grants `InternalsVisibleTo` to `Opc.Ua.Bindings.Https.WebApi.Tests` for the internal shared-`HttpClient` constructor. - `docs/WebApi.md`: new section "Compressed responses". ## Tests - **New:** `WebApiClientErrorAndCompressionTests` in `Opc.Ua.Bindings.Https.WebApi.Tests` (stub `HttpMessageHandler`, no server): - error mapping: `HttpErrorStatusSurfacesAsServiceResultException` (14 cases, every row of the status table), `ThrottledRequestSurfacesAsServerTooBusyWithRetryAfter` (429, 503), `ConnectionFailureMapsLikeTheTransportChannel`, `TlsHandshakeFailureMapsLikeTheTransportChannel`, `ElapsedRequestTimeoutSurfacesAsBadRequestTimeout`, `CallerCancellationStaysOperationCanceled`; - gzip: `AcceptCompressedResponsesAdvertisesGzipAndDecompressesAsync`, `SharedClientCarriesAcceptEncodingPerRequestAsync`, `GzipResponseIsDecompressedEvenWhenNotAskedForAsync`, `GzipBodyIsBoundedByMaxMessageSizeAfterInflating`, `CorruptGzipIsADecodingError`, `UnsupportedContentEncodingIsRejected`, `IdentityContentEncodingIsReadAsIsAsync`, `ResponseStreamIsLeftToItsContentAsync` (plain and gzip). - **Updated** (they expected the old exceptions): - `WebApiClientTests.InvokeAsyncThrowsServiceResultExceptionOnNon2xxStatus` (was `InvokeAsyncThrowsHttpRequestExceptionOnNon2xxStatus`): `Bad_InternalError` with the `HttpRequestException` as inner exception; - `WebApiResponseQuotaRegressionTests.RequestTimeoutRemainsActiveWhileReadingTheOpenApiBodyAsync`: the elapsed timeout is `Bad_RequestTimeout` with the `OperationCanceledException` as inner exception. - **Fail without the fix:** with `WebApiClient.cs` and `HttpResponseBodyReader.cs` from master, 24 of the 29 new test cases fail, as do `InvokeAsyncThrowsServiceResultExceptionOnNon2xxStatus` and three of the six `RequestTimeoutRemainsActiveWhileReadingTheOpenApiBodyAsync` cases. The five that pass there guard behaviour that must not change: throttling (2), caller cancellation, identity coding, and a plain body's stream. With a reader that disposes the response stream, both `ResponseStreamIsLeftToItsContentAsync` cases fail. - `Opc.Ua.Bindings.Https.WebApi.Tests` 436/436 and `Opc.Ua.Client.Tests` 2733/2733 on net10.0 and net8.0 (macOS). - `Opc.Ua.Client` and `Opc.Ua.Core` build for net48, net8.0, net9.0 and net10.0 with 0 warnings; `dotnet format` (whitespace, style, analyzers) is clean on every changed file; changed lines covered 112/112 (100 %) with the two suites above (`.azurepipelines/check-coverage.ps1` against master). - Fork CI (full matrix incl. macOS) and CodeQL: green — [CI run 37128530025](https://github.com/biancode/UA-.NETStandard/actions/runs/37128530025) (154 jobs; one net48 job, `ReconnectAsyncSwapsFaultedLeaseEntryAsync` in `Opc.Ua.Client.Tests`, timed out on the first attempt and passed on rerun; this change does not touch reconnect), [CodeQL run 37128531971](https://github.com/biancode/UA-.NETStandard/actions/runs/37128531971). ## Notes for review - `HttpResponseBodyReader` is shared with the binary `HttpsTransportChannel`, so rejecting unknown content codings applies there too. Before, such a body was decoded as plain bytes and failed later with a less helpful error. - The failure-below-HTTP and timeout codes reuse what the transport channels report, so `WebApiTransportChannel`, which sends through `WebApiClient`, keeps its codes for those cases. Its own `HttpRequestException` path no longer sees exceptions from `WebApiClient`; I left it in place. ## Related Issues - Fixes #4647 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
romanett
|
52f8520baa |
Detect resource and limit abuse in the encoder/decoder fuzz harness (#4666)
## Why The encoder/decoder fuzz targets only flagged crashes, unhandled exceptions and encoding-fidelity mismatches. They could not observe a decode that stays within the exception contract but consumes far more memory, CPU or stack than its input justifies, or that materialises values larger than the context's encoding limits. None of the resource-related findings of the codec audit would have been caught. ## What Adds four shared oracles to the fuzz harness (`fuzzing/Common/Fuzz/FuzzOracles*.cs`), applied by every host (libFuzzer, AFL, `--replay`, Tools playback and the NUnit replay): - **Allocation** — wraps the Binary/JSON/XML decode cores, the built-in-type targets, the PubSub UADP/JSON decoders and the UADP reassembler; fails when a decode allocates more than `64x` its input plus `8 MB`, whether it returns or throws. Uses the per-thread allocation counter (the AppDomain counter on .NET Framework). - **Time** — a per-input budget of `1 s + 100 ms/KB`, re-run twice and reported only if the fastest run is still over, so JIT/GC/CI noise does not trip it (`OPCUA_FUZZ_TIME_BUDGET_SCALE`, `0` disables). - **Stack** — every input runs on a worker thread with a `256 KB` stack, so recursion that comes close to the limit of a normal thread is caught in-process as a crash rather than near 200 levels. `StackTestcases` inputs replay through every target in a child process (`--fuzz-replay-all`), since such a crash cannot be caught. - **Decoded limits** — walks a decoded value for strings, identifiers, ByteStrings and arrays beyond the context's `MaxStringLength`/`MaxByteStringLength`/`MaxArrayLength`; matrices are additionally boxed the legacy way so a value no consumer can materialise is reported. The fuzz context uses `4096/8192/4096` so small inputs reach the limits. Supporting changes: - `ArrayOf<T>`/`MatrixOf<T>` are not `IEnumerable`, so the limit walker unpacks them via `ToArray`/`ToArrayOf` (otherwise it silently traversed nothing). - `Decimal` is registered in the fuzz context so the targets exercise `Decimal.Decode`. - **B1-2 bug fix**: `BinaryDecoder.ReadExtensionObject` dereferenced a null element for an XML body that is not an element; now rejected with `BadDecodingError`. Found by replaying the nightly crash corpus under the new exception contract; 5 corpus inputs are added as gating reproducers. - Reproducers for B1-1, T1-3, X1-1 and U1-3, built in process by documented builders (`ResourceReproducerTests`, `PubSubReproducerTests`) and checked in to the corpus, where every target replays them under the oracles. Each was verified by reverting the matching fix locally and confirming the guard fires (e.g. B1-1 then allocates ~21 MB from 481 input bytes). - `Fuzzing.md` / `CrashCorpus.md` document the oracles and the private-input log policy. ## Testing Green on net10.0 and net48: | Suite | net10.0 | net48 | |---|---|---| | `Opc.Ua.Encoders.Fuzz.Tests` | 8130 | 8130 | | `Opc.Ua.PubSub.Fuzz.Tests` | 149 | n/a (modern-only) | | `Opc.Ua.Fuzzing.Tests` | 154 | 154 | | `Opc.Ua.Types.Tests` (encoders) | 3698 | builds | Also verified: `Network`/`Certificates` fuzz test suites, all fuzz hosts and Tools projects build, the fuzz manifest validator passes, `dotnet format whitespace` is clean, and the full ~22k-input nightly crash corpus replays with no oracle false positives. Stacked on `romanett/encoder-decoder-spec-security-83c9f5`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |