mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-20 08:41:51 +08:00
fix/diagnostic-delete-nodeversion-master
365 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Cristian Pogacean
|
bef85adf9f |
Preserve NodeVersion eligibility when deleting runtime nodes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
romanett
|
8c52cba0b2 |
CTT reference server: faster alarm cycle in --ctt mode, MinSubscriptionLifetime 1000 ms (#4766)
Shortens CTT runs against the reference server without changing what the CTT verifies. ## Changes - **Alarm simulation step in `--ctt` mode: 1000 → 500 ms.** `AlarmNodeManagerFactory.SimulationInterval` (default stays 1000 ms) is passed to `AlarmNodeManager`; `ConsoleReferenceServer --ctt` sets it via `Quickstarts.Servers.Utils.ConfigureCttModeNodeManagers()` to `Utils.CttAlarmSimulationInterval` (500 ms) before the node managers are created. A full alarm cycle (40 steps through all limits) takes 20 s instead of 40 s. Every alarm still passes through all states and limits; the alarm NodeIds carry the interval (`…ExclusiveLevelAlarmType.500.Alarm` in CTT mode). - **`Ctt.ReferenceServer.Config.xml`: `MinSubscriptionLifetime` 10000 → 1000 ms.** CTT scripts request short subscription lifetimes on purpose and then wait for them to expire (Subscription Basic 063/016, Durable 012, Minimum 02 004, Publish Min 05 Err-001, Diagnostics 024); the 10 s floor inflated those waits. Part 4 sets no minimum; the server still enforces lifetime ≥ 3 × keep-alive. ## Measured (patched CTT 1.05.513 scripts) | Part | Before | After | | --- | ---: | ---: | | A&C (p18), alarm step only + CTT Alarm Cycle Time 20 | 932 s | 576 s | | Subscriptions (p16), MinSubscriptionLifetime only | 679 s | 579 s | Combined full run with the matching CTT-side changes (ReadyToRun server build, publish interval 150 ms, SecureChannel min lifetime 1000 ms, A&C Refresh script waits): **72 → 54 min**, 0 regressions against the CTT baseline (the only error is the known CTT-binary defect GDS 019). The CTT CI repo derives its `Alarm Cycle Time` from `Utils.CttAlarmSimulationInterval`, so runs against branches without this change keep the old 30 s setting. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
40288fa7d8 |
Tests: weighted FairRequestQueue test asserts the median of three windows (#4763)
# Description `FairRequestQueueTests.WeightedServiceHoldsUnderConcurrentWorkersAsync` failed once on GitHub Actions macOS net10.0 (#4753, run 38349863236): light=124 heavy=276, ratio 2.23 against an expected 2.5-3.5. It passed on every other CI run and 30/30 locally. **Root cause.** `TrySelect` dequeues an owner's turn token from the lock-free ready queue. On the last token of a turn, `TakeTurnToken` CASes the turn and calls `PushTurn` for the next one. A worker preempted between those two steps leaves the owner with no queued token while it is stalled. The work-conserving dispatch serves the other owner in the meantime, and that share is never made up. Delay injection confirms this: sleeping 1 ms on every 7th turn publish collapses 1:3 to roughly 1:1 (light=199 heavy=201). Publishing a turn ahead of time would only cover stalls shorter than one round (microseconds), while real preemption lasts milliseconds. Making up the lost share (deficit or virtual time) would mean redesigning the scheduler. So this PR documents the bounded skew and makes the test robust instead. **Changes** - `FairRequestQueue` remarks: the weighting holds over time, not in every window, and a preempted worker can cost an owner its share for the length of the stall. - `WeightedServiceHoldsUnderConcurrentWorkersAsync` now measures 3 independent windows through `MeasureWeightedServiceAsync` and asserts that the **median** heavy/light ratio is in 2.5-3.5. The failure message lists all three windows. - Each window uses perOwner = 480 and measured = 560, with queue limits of `2 * perOwner`. That stays under the TestProvider's 1000-request cap and keeps both owners backlogged. - Each window still checks that both owners stay backlogged, then drains the queue and checks that everything is back to zero. The median tolerates one skewed window and still fails when the weighting is lost systematically. With stalls injected into one window only, the median is 3.0 and the test passes (e.g. `light=238 heavy=322; light=140 heavy=420; light=140 heavy=420`). With stalls injected into every window, it still fails. **Validation:** the `FairRequestQueueTests` fixture passes 93/93 on net10.0 Release. The weighted test passed 30/30 runs pinned to 2 cores (`start /affinity 3`). `dotnet format whitespace --verify-no-changes` is clean, and the build has 0 warnings. ## Related Issues - Flaky test seen on #4753 after #4750 (FairRequestQueue without a queue-wide lock) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
3e433eb6c4 |
XRegistry: registration lifecycle, identifier allocation, registrar roles and atomic Resource replace (#4756)
# Description Extracts the generic xRegistry registration lifecycle from the WoT work into a standalone, master-targeted change: - Canonical xRegistry identifier allocation (`XRegistryIdentifier`) with case-folded bounded collision handling and exact Version allocation/delete semantics. - Registrar roles / capabilities in the xRegistry NodeSet2 model (only the roles/capabilities facts). - Default-Version views and Resource Meta handling. - FileType read/write modes, session ownership/discard, cursor handling, and short reads until actual EOF on the client. - Atomic whole-document `ReplaceAsync` on the in-memory and file-system resource stores. - Generic projection engine/provisioning and projection contracts, label ownership/no-op behavior, Resource-vs-Version event targets. - Documentation in `docs/XRegistry.md` and XML docs for owned members. - Tests: Identifier, RegistrarRoles, Attribute, Projection, RegistrationEvent, FileMode, ResourceStore, Client, VersionAssignment, concurrent read. Intentionally excluded (handled in separate PRs): trusted Federation / Registry origin model and provider (G9); WoT lease/dispatcher, prepared manifest, 0.7 model/version bump (G12). Merged current `master` ( |
||
|
Marc Schier
|
c2b95e2085 |
XRegistry: trusted logical-Resource federation identity (#4755)
# Description Adds trusted **logical-Resource** federation identity to XRegistry - A federation proxy binds an origin registry (`RegistryOriginDataType`), application, portable `ResourceNodeId` and Xid, never an endpoint or content-digest identity. - `XRegistryFederationTarget`, `IXRegistryFederationProvider`, `XRegistryClient.Federation` (verify/follow/relocate; uses master's `ISessionBindingProvider.CreateBindingAsync`). - `XRegistryFederationNodeManager`, `XRegistryServerOptions`/DI: required target/provider verification and proxy admission. - Model hunks only: `RegistryOriginDataType` 63026 (+encodings 63591/63592), `OriginRegistry` 63593 + `ResourceType` HasProperty, `ExternalReference`/`ResourceUrl` description rewrites. - Tests (`XRegistryFederationNodeManagerTests`), package READMEs, `docs/XRegistry.md` Federation section (links `SessionBindings.md`). Merged current master ( |
||
|
romanett
|
3a31473cf4 |
Server: no server-wide or per-session locks in per-request session validation (#4753)
# Description
Every request passes through `SessionManager.TryGetSessionContext` at
admission and again at dispatch
(`FairRequestQueue.GetRevalidationStatus` →
`DefaultServerResourceIsolationProvider` →
`ISessionBindingProvider.TryGetSessionContext`), and then through
`Session.ValidateRequest`. Together they took:
1. the server-wide `m_bindingsLock`, to confirm the candidate binding
was still the committed one;
2. the per-session diagnostics lock in `Session.HasExpired`. It is
inlined, so CPU traces charge its spin to `TryGetSessionContext`;
3. the session lock in `Session.IsSecureChannelValid`;
4. the session lock around all of `Session.ValidateRequest`, plus the
diagnostics lock twice (contact refresh, diagnostic counters);
5. the server-wide `SessionManager` event lock, to raise
`SessionDiagnosticsChanged` for every request even when nothing
subscribes.
With the FairRequestQueue lock also removed (a local change without the
queue-wide lock, not yet a PR), these were the next waiting points:
14–16.5 % of on-CPU samples spun in `Lock.TryEnterSlow` under
`TryGetSessionContext`, and once that was fixed another 7–10 % under
`Session.ValidateRequest`.
## Changes
**SessionManager: confirm the binding without the bindings lock**
- `SessionActivationState` publishes a `DispatchBinding` (the committed
binding while no activation is committing, otherwise `null`) with
`Volatile.Write`. The `BindingContext` and `IsCommitting` setters
republish it, and every writer of those already runs under
`m_bindingsLock`.
- The lookup reads it once for the candidate and once to confirm. A
commit always creates a new `SessionBindingContext`, so an unchanged
reference means nothing replaced it. A token is mapped to a session
object at most once and is never re-added after removal, so finding the
token still mapped to the same session afterwards means both held at the
instant of the confirming read, which is the guarantee the lock gave.
- IsCommitting, transfer, close/claim, shutdown and the
certificate/security checks are unchanged.
**Session: read the channel id without the session lock**
- `SecureChannelId` is published with a volatile write in `Activate`,
and `IsSecureChannelValid` reads it without the session lock.
**Session: request validation and diagnostic counters without locks**
- Request counts are interlocked increments of per-session slots (total
and error per `RequestType`, plus unauthorized). They are applied to
`SessionDiagnostics` under the diagnostics lock whenever it is read
(`ReadDiagnostics`, `UpdateDiagnostics`, the diagnostics-node callback,
`ClientLastContactTime`), so readers see the same totals.
- The contact refresh is a compare-and-swap on the tick count, and
requests within the same millisecond do not write. An expiry observed by
`HasExpired` or by a refresh is recorded in the contact word
(`kExpiredFlag`), so a request that read the clock before the monitor
saw the timeout cannot refresh it afterwards. This keeps the documented
"a request cannot revive a session the monitor saw expire" guarantee
without the lock. The intermediate commit that made `HasExpired`
lock-free on its own had opened that window; this commit closes it.
- `ValidateRequest` takes no session lock: the channel id and
`Activated` are published with volatile writes.
- `RaiseSessionEvent` returns before taking the event lock when a
`DiagnosticsChanged` event has no subscriber.
Behaviour differences reviewers should know about:
- `ClientLastContactTime` is millisecond-granular, and two concurrent
refreshes can publish their wall-clock times in reverse order, which
leaves it up to about 1 ms older than the latest contact.
- After an expiry has been observed, an `Activate` no longer refreshes
the contact time. `SessionManager` already rejects activation of an
expired session before that point.
## Tests
- `LookupDuringActivationCommitReturnsNoBindingAsync`: a lookup inside
the commit window (after `Session.Activate`, before the binding commit)
returns no binding. A test session re-implements `ISession.Activate` to
run the hook. Before this test, publishing the binding regardless of
`IsCommitting` passed all 690 session tests.
- `LookupRacingTransfersAndCloseReturnsOnlyCommittedBindingsAsync`: 4
readers look up while 1000 transfers alternate between two channels,
followed by a close. Each result must be a committed binding of the
queried channel, no older than the last completed activation and never
decreasing, and nothing may come back after the close.
- `RequestRacingObservedExpiryDoesNotReviveTheSessionAsync`: a clock
hook lets the monitor observe the expiry between a request's clock read
and its refresh, and the request must fail with `Bad_SessionClosed`. It
fails if `HasExpired` does not record the expiry.
- `ConcurrentRequestsAreCountedExactlyAsync`: 8 threads × 2100
validations (Read/Browse, plus rejected Writes). The totals, per-service
counts, error counts and unauthorized count must be exact, and a
concurrent reader never sees the total go backwards.
- Local runs (net10.0): `Opc.Ua.Server.Tests` 8487 passed / 0 failed;
`Opc.Ua.Sessions.Tests` 851 passed / 0 failed / 379 skipped. net48
builds.
## Measurements
`perf/ServerLoadHarness`, Server GC, paired alternating A/B, 7 rounds.
Baseline is master
|
||
|
romanett
|
a14b23e7dd |
Server: cheaper symmetric crypto path, FairRequestQueue without a queue-wide lock (#4750)
# Server: cheaper symmetric crypto path, FairRequestQueue without a queue-wide lock Roadmap items E1 and 5b of the o6 benchmark gap analysis, one commit each. - **E1:** Windows one-shot `Aes.EncryptCbc`/`DecryptCbc` import the key (`BCryptImportKey`) on every message, although #4720 caches one `Aes` per channel token. The AES-GCM and ChaCha20-Poly1305 policies also created a new keyed instance per message. - **5b:** with `AsyncCustomNodeManager`, which is what the o6 benchmark's 2.0.1 worker uses, the `FairRequestQueue` gate became the largest waiting point. `TryEnqueue`, `TrySelect` and `TryDequeue` spun on one `Lock` for every request. No wire, configuration or public API changes. ## Results These are paired A/B runs against `origin/master`, five rounds per scenario. The two harness builds alternate per round, starting side swapped every round, on the same shared machine (Ryzen AI 9 HX 370, 12 cores / 24 threads, Windows 11). The tables show medians, with ranges in brackets where they matter. The server is configured like the o6 2.0.1 worker: `AsyncCustomNodeManager`, Server GC, queue sized clients × outstanding + 3. o6read uses the benchmark's native open62541 client workers; `read` uses the harness's .NET client. On this machine throughput moves between rounds by up to 2×, so CPU and bytes per request are the reliable signals. ### E1: symmetric crypto (measured on |
||
|
romanett
|
63adef0d10 |
Fix the Sessions LoadTest fixture and the server bottlenecks it exposed (#4749)
The `[Explicit]` Sessions `LoadTest` fixture has not started since #4541 (also on the 2.0.0 tag): it set `MaxChannelCount = MaxSessionCount = 10100`, and resource isolation refuses to start unless the channel cap covers the sessions plus the bootstrap and reconnect floors plus one. Every LoadTest case failed in `OneTimeSetUp`. Running the fixture again, and profiling it and the `perf/ServerLoadHarness`, found several bottlenecks that shipped in 2.0. ## Changes **Test** - `MaxChannelCount = MaxSessionCount + 3`, the same margin as the framework defaults (100 / 103). - Notifications are counted with per-node `Interlocked` counters. The shared `ConcurrentDictionary<NodeId, int>`, updated by 750 subscriptions, took 13 % of the process CPU in bucket-lock contention. - After the writer stops, and after the last event, the tests wait until the received count stops growing instead of a fixed delay, and log the drain time. **Server / client** - **Detached monitored item recovery** (#4094): every node a NodeManager added (`AddPredefinedNodeAsync`, recursively for children) scanned every monitored item of every subscription for detached items. `CreateSubscription` adds ~30 diagnostics nodes, so with 450k monitored items subscription creation was quadratic: 69 % of CPU, and `CreateSubscription` timed out. `MonitoredItem` now keeps an exact process-wide count of live detached or deleted items, and `MasterNodeManager` skips recovery while it is zero. All flag changes go through one helper under the item lock; a disposed item does not count, because it cannot be attached again. - **Client channel gate** (#4232): `UaSCUaBinaryClientChannel.SendRequestAsync` entered the gate with a blocking `Gate.Enter()` inside an async method, and `SemaphoreSlim.Wait` spun for 19 % of CPU with many requests in flight on one channel. It now awaits `Gate.EnterAsync(ct)`, like the server side. - **ResourceManager** - Status code and symbolic id translation looked up their maps under the manager lock and translated while holding it. Sessions that request diagnostics do this for every notification: 83 % of lock contention. The maps are copy-on-write and read without the lock. - Translation tables were `SortedDictionary<string, string>` with the default, culture-sensitive comparer: up to 1 µs per Browse display name, 22 % of Browse server CPU. They now use an ordinal `Dictionary`. - **Namespace default permissions**: every permission check read `DefaultAccessRestrictions` / `DefaultRolePermissions` / `DefaultUserRolePermissions` of the node's namespace through two server-wide locks and deep-copied the role permission array, per node. That was 99 % of lock contention for Read/Write of 100 values. The values are now read once per namespace per service request and remembered on the `OperationContext`. Contexts that are not a service request (monitored item and session contexts, which can live long) remember nothing, so a changed default applies to them at once. ## Results `perf/ServerLoadHarness`, paired A/B against `master` (3 alternating rounds, medians, 20 sessions, 24 cores). The harness numbers predate the recovery, client gate and status code translation commits, which those scenarios do not exercise: | Scenario | Throughput | Server CPU / op | Server alloc / op | Lock contention | |---|---|---|---|---| | Read, 100 values | +19 % | −28 % | −18 % | −96 % | | Read, 100 values, SignAndEncrypt | +27 % | −26 % | −18 % | −98 % | | Write, 100 values | noise ¹ | −19 % | −17 % | −88 % | | Browse (Server object) | +3 % | −16 % | −10 % | −2 % ² | | Read, 1 value | +12 % | −4 % | ±0 | ±0 | ¹ Throughput varied run to run on the shared machine; per-op CPU and allocation are the reliable signal. ² Browse contention is the Server node's browse lock in `NodeState.CreateBrowser` (all sessions browse the same node); not changed here. `LoadTest` (opc.tcp, Release, in-process client and server): | Case | master | this PR | |---|---|---| | `ServerReadLoadTestAsync` | setup failure | passes (50M value reads, 0 errors) | | `ServerManySessionsLoadTestAsync(500)` | setup failure | passes | | `ServerSubscribeLoadTest_Classic` | setup failure; on 2026-09-26 (before #4541): `CreateSubscription` `BadRequestTimeout` | 76–85 % received, **fails** (needs 99 %) | | `ServerSubscribeLoadTest_ManagedSessionV2` | setup failure; 2026-09-26: 1,885 notifications received | 35–45 M notifications, 77–85 % received, **fails** | | `ServerEventSubscribeLoadTestAsync` | setup failure; 2026-09-26: 34.6 % | passes: 100 % delivered, drained ~40 s after the last event | The subscribe cases now lose notifications rather than deliver them late: delivery stops ~2 s after the last write. Every monitored item has queue size 1 and the writer is not paced by delivery, so whatever exceeds the publish rate (~585k notifications/s here, SignAndEncrypt with the client in the same process) is overwritten. The event test lost nothing either: it waited a fixed 2 s after the last event and counted the backlog as lost. It now waits until delivery stops. Its throughput is bound by garbage collection: the event consumer of the Server node spends ~90 % of its wall time suspended for workstation GCs of the shared test process. With `DOTNET_gcServer=1` it generates 702 instead of ~210 events/s and drains in 7 s instead of ~40 s, and the subscribe cases reach 87 % (Classic) and 93 % (ManagedSessionV2) at 78–106 M notifications. ## Testing - `Opc.Ua.Server.Tests` 8468 passed / 5 skipped, `Opc.Ua.Client.Tests` 2788 passed, `Opc.Ua.Sessions.Tests` 851 passed / 379 skipped, `Opc.Ua.Core.Tests` Tcp/Channel 797 passed (net10.0). - New `OperationContextDefaultPermissionsCacheTests`. - `Opc.Ua.Server` builds for net48; `dotnet format whitespace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
7c965fd636 |
Capture metadata images and generation-bound session clients (#4752)
# Description - Add isolated NodeState reference images, TypeTable graph snapshots, EncodeableFactory registration forks/views, StringTable mutation versions/private copies, and ServiceMessageContext mapping-holder generation tracking. Reference images are lazy for empty nodes; selected-image clones own independent storage and publication ownership. - Add the complete generic NodeBrowser continuation-dependency capability. This does not implement server browse-history retention, NodeManager read images, or server batch/readiness admission. - Add optional client binding capture for SessionClient, native Session and ManagedSession, with managed-lease/native-channel generation composition. Bindings capture authenticated-session identity, mapping holders/versions and owned endpoint/policy/configuration observations. Replacement, reconnect, mapping ABA and disposal permanently invalidate stale bindings; disposing a binding does not dispose its owner. - Preserve cancellation and master pooled-response ownership: a rejected pooled response is released exactly once; an accepted response remains caller-owned. Keep native transport submission under its replacement gate and completion outside the gate, without routing ordinary unbound requests through binding machinery. - Rename the existing server lookup interface and all actual consumers with minimal hunks; add focused fixtures, generated FileTypeClient usage guidance, and package/transport documentation. **Breaking preview type identity:** the existing 2.0-preview server lookup `Opc.Ua.ISessionBindingProvider` becomes `Opc.Ua.IServerSessionBindingProvider`. Server lookup consumers must update their referenced type and rebuild. This is neither source-compatible nor binary-additive for that preview type. The new client capability is optional and adds no mandatory members to existing ISession/ISessionClient/ITransportChannel contracts. This preview-to-preview rename is documented in SessionBindings/Transports, not in the 1.5-to-2.0 MigrationGuide. **Ownership boundary:** no RequestLifetime changes, certificate-manager/channel-entry adoption, BaseVariableState changes, Method-result feature extraction, NodeManager batch/readiness or browse-history retention, session restoration, registry federation, WoT conversion, or WoT runtime/planners/events. DependencyInjection/ResourceIsolation/index changes are lookup/link closure only. SessionManager's PR hunk changes only its implemented lookup-interface name; newer master public diagnostic masking is not replaced. ## Related Issues _Reference all GitHub issues this PR addresses. If there is no issue yet, open one and link it here._ _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ - Part of #4511: user-approved combined groups 5+6 Wave A extraction. This PR does not close the umbrella or publish Wave B. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. ### Review qualification Review fixes are published at **`6b0eb1d5c32a439300fbcaff974b9af8b909161b`**, changing eight already-owned files. All five received review threads were replied to and resolved after verification. Factory selector installation and target-selection-plus-commit share the root publication gate, with consistent root→image release/capture ordering. TypeTable selector installation and final mutation routing share its existing gate. Native bound channel validity uses volatile observations; the two owned XML defects are corrected. | Requirement | Evidence | |---|---| | Selector installation must not hide TypeTable mutations | `SelectorInstalledDuringNodePreparationReceivesMutation`: old Add routing failed on both TFMs; corrected routing passes on both within **148/148** factory/TypeTable controls | | Selector installation must not race factory commit routing | Static common-root-gate/order proof plus baseline-GREEN `BuilderCapturedBeforeOwnershipUsesSelectedFactory`, `ViewOwnerReleasePreservesRegistrationsAndAllowsReuse`, `ViewOwnerReleaseCanRetryAfterPendingPublication`, `PublicationProtectsPendingAndRetiredFactoryImages` | | Lock-free validity must observe owner transitions | Static volatile-read reasoning plus native `BoundRequestCannotDispatchWhileNativeReplacementIsPendingAsync`, `BoundResponseRejectsInvalidationAfterNativeDispatchAsync`, `BoundClientCannotRetryAcrossFaultedManagedEntryReplacementAsync`: **12/12 each TFM**; Core binding/pooled ownership controls **17/17 each TFM** | | Correct owned XML documentation | “an unmanaged” and one multiline `ReferenceUpdate.Next` summary; scoped whitespace/INFO and build evidence below | Factory controls are **not** claimed as a deterministic factory RED, and public transport controls are **not** claimed as a deterministic weak-memory RED. No private-lock reflection, artificial production seam, or source-name assertion was added. The TypeTable RED uses the real `ILocalNode.NodeClass` callback during node preparation, and both expected RED receipts/TRX were retained. All accepted review runs use the unchanged admitted SDK wrapper, normal project references, sole mutex, one node/no shared compiler, finite resource guards and canonical short-C build outputs. Source hashes were frozen per command and matched the committed bytes afterward. SDK ownership was released **before** commit/push/review replies. No automatic merge was requested. Scoped whitespace passed across the four affected projects. Review INFO classification is **0 owned findings / 13 untouched baseline findings excluded** (whole-file formatter exits 1,1,0,0); no baseline RCS1140/RCS0055 expansion or suppression. The final normal net10 public Client/Core/Types trim/AOT-analyzer closure passed **0 warnings / 0 errors**. This is not a NativeAOT publish/run. <details> <summary>Accepted review command payloads and results</summary> These exact argument payloads were executed through the unchanged guarded SDK wrapper. `$proof` expands to the protected results-directory below. Successful explicit builds of the same source preceded the `--no-build` runs. ```powershell $proof = 'C:\Users\mschier\.copilot\session-state\a3b74048-c52f-4c87-8773-7a46c9ab0087\files\qualification-review-4752' # Exit 0: 148 passed, 0 failed dotnet test tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj -f net48 -p:CustomTestTarget=net48 -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --no-restore --verbosity minimal --filter "FullyQualifiedName~EncodeableFactorySnapshotTests|FullyQualifiedName~TypeTableTests" --logger "trx;LogFileName=review-types-green-net48.trx" --results-directory $proof # Exit 0: 148 passed, 0 failed dotnet test tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj -f net10.0 --no-restore --no-build --filter "FullyQualifiedName~EncodeableFactorySnapshotTests|FullyQualifiedName~TypeTableTests" --logger "trx;LogFileName=review-types-green-net10.trx" --results-directory $proof -p:CustomTestTarget=net10.0 -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --verbosity minimal # The following two per-TFM payloads each exited 0: # Core: 17/17 net48 and 17/17 net10.0; Client: 12/12 net48 and 12/12 net10.0 foreach ($tfm in @('net48', 'net10.0')) { dotnet test tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -f $tfm --no-restore --no-build --filter "FullyQualifiedName~SessionBindingResponseTests|FullyQualifiedName~SessionBindingContextTests" --logger "trx;LogFileName=review-actual-core-green-$tfm.trx" --results-directory $proof -p:CustomTestTarget=$tfm -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --verbosity minimal dotnet test tests\Opc.Ua.Client.Tests\Opc.Ua.Client.Tests.csproj -f $tfm --no-restore --no-build --filter "FullyQualifiedName~BoundRequestCannotDispatchWhileNativeReplacementIsPendingAsync|FullyQualifiedName~BoundResponseRejectsInvalidationAfterNativeDispatchAsync|FullyQualifiedName~BoundClientCannotRetryAcrossFaultedManagedEntryReplacementAsync" --logger "trx;LogFileName=review-actual-client-green-$tfm.trx" --results-directory $proof -p:CustomTestTarget=$tfm -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --verbosity minimal } # Exit 0: 0 warnings, 0 errors dotnet build src\Opc.Ua.Client\Opc.Ua.Client.csproj -f net10.0 -p:CustomTestTarget=net10.0 -p:UseArtifactsOutput=true -p:ArtifactsPath=C:\cp56-a3b7-v2 -nr:false --no-restore --verbosity minimal ``` The six expanded GREEN commands, exact results/TRX hashes and two expected RED commands are retained in `qualification-review-4752\qualification-summary.json`, SHA256 `C5E71A4D8D37BAC8803F6DF38B6BCE2B70F3D0AFD5DEC8493FAE9DB440DCFB5A`. Committed/qualified byte identity is in `publication-identity.json`, SHA256 `CD9515FE96703F8B241F2B7B9B71F73EF0FF2626C2C8BCCE40307971A8F8E809`. Per-command raw exits, guards, hashes and reports are retained outside disposable build roots. For each record in the protected `format-inputs.json` (Types/Core/Client libraries and Types.Tests), the recorded Project/Include values expand these exact payloads: ```powershell # Workspace properties recorded with the formatter receipts: $env:CustomTestTarget = 'net10.0' $env:UseArtifactsOutput = 'true' $env:ArtifactsPath = 'C:\cp56-a3b7-v2' # All four exit 0 dotnet format whitespace $Project --no-restore --verify-no-changes --verbosity minimal --include $Include # Exits 1,1,0,0 due only to untouched baseline findings; owned=0, baseline=13 dotnet format $Project --no-restore --severity info --verify-no-changes --verbosity minimal --report "$proof\formats\review-info-$i" --include $Include ``` Initial fresh matrix evidence remains separately identified and unchanged: Types274/Core119/Client151/server137 net10; Types274/Core119/Client151/server135 net48, all passing. Qualification base remains `0dff94bd378e79b8cf1a47e1aaf340f9b4b40048`; the initial published head was `9fee4583`. All 16 initial exact command arrays/results/TRX hashes remain in protected `qualification-v2\qualification-summary.json`, SHA256 `88E10240C20049B59A3401BE007C6528A85DF2F3DCFAF9D11D908CEF2946DA4E`. Failures/incidents remain retained rather than relabeled as success: original disappeared-root historical counts are invalidated; the large-TRX controller DOM-parser failure is preserved separately from accepted streaming receipts; the review argument-array defect/default-D-output controls are preserved but excluded from canonical review qualification. Correctly separated argument rechecks produced the accepted short-C Core/Client receipts. The sole SDK wrapper was not changed. Full UA.slnx, measured coverage increase, CLA verification, NativeAOT publication and current-head/master CI/CodeQL remain unclaimed. Actual merge—not resolved review threads—unlocks Wave B. </details> |
||
|
Marc Schier
|
84d6387a4f |
Make request cancellation own exactly one terminal state (#4744)
# Description `RequestLifetime` now publishes the winning cancellation status and terminal ownership in a single atomic state, so concurrent client Cancel, Session close and timeout can never overwrite each other's status or double-own the terminal transition. External token registrations are owned by the lifetime and released on completion, and disposal is deferred until running cancellation callbacks have finished (no use-after-dispose of the CancellationTokenSource). Adds regression tests: 16384-iteration race tests asserting status/owner/callback outcomes with a finite 2 min watchdog (`RequestLifetimeTests`, `RequestManagerTests`) and `WaitAsync` cancellation tests (`TaskWaitAsyncTests`; the polyfill change is only 2 XML doc lines, no runtime change). Files: `RequestLifetime.cs`, `System.Threading.Tasks.cs` (docs), and the three test files. `StandardServer.cs` is intentionally untouched (already on master). ## Related Issues - None. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Validation actually run (net10.0 and net48, all passed): Types.Tests `TaskWaitAsync` 23/23; Core.Tests `RequestLifetime` 16/16; Server.Tests `RequestManager` 56/56; `dotnet format whitespace --verify-no-changes` on owned files. Full UA.slnx not run. |
||
|
Marc Schier
|
184f19a2b0 |
Add complete method invocation results and resolved diagnostics (#4747)
# Description `MethodInvocationResult` carries the operation result, declaration-ordered outputs and optional position-preserving input results. Asynchronous callbacks retain the actual receiver, cancellation token and resolved diagnostic strings. Fluent predefined, typed, instance, reference-child, resolved and virtual method builders reject conflicting handlers in either registration order and never overwrite materialized handlers. Direct configuration remains available. Shared Call-result assembly retains input diagnostics, including useful text on Good input positions, and requested operation diagnostics on Good results. Diagnostic encoding preserves nested Good-status text, status information bits and the outer-inclusive depth limit. Request admission clears client-supplied internal diagnostic permission flags before early returns or validation failures; admitted sessions grant the flag only after the existing administrator-role checks. Includes corresponding unit/integration regressions, complete XML API documentation, and method-result guidance consolidated with the fluent APIs in `docs/NodeManagers.md`. No WoT receiver/projection/remote-execution, event-readiness/native-admission, or bulk/history implementation is included. ## Related Issues Part of the user-approved two-wave extraction plan tracked by #4511. _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ - Related to #4511. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. |
||
|
romanett
|
8b360da7d2 |
Server publish path: no deep clone per Publish, pooled notifications that are actually reused, ready list instead of an O(N) scan (#4748)
Roadmap item A5 of the o6 benchmark gap analysis. Every Publish response deep-cloned its NotificationMessage in `SentMessageQueue` (PR #4707 kept the clone because a retained message returns its pooled payload when the client acknowledges it, which can happen before the response was sent). This PR replaces the clone with a send-completion lease, makes the activator pool that recycles the notifications O(1) and sizes it for the server's load, builds messages without intermediate copies (the notification array is rented), and lets monitored items tell their subscription when they have something to report so the publish timer no longer walks every item. ## Results Paired A/B against master `0dff94bd3` (includes #4742 and #4743; alternating builds, 3 rounds, medians; 20 sessions unless noted, workstation GC, 24-core machine shared with other sessions). Server = the reference server child process. | Scenario | Server alloc / op | Server GC pause | Server CPU / op | Write→notify p50 / p99 | Throughput | |---|---|---|---|---|---| | `sub`, 5 subs × 100 items, pub 100 ms | **−57 %** (373 → 159 B) | **−57 %** | −7 % ¹ | −1 % / −2 % | +1 % ² | | `sub`, 5 subs × 100 items, pub 10 ms | **−57 %** (375 → 163 B) | **−52 %** | +9 % ¹ | −22 % / +1 % | ±0 ² | | `sub`, pub 100 ms, SignAndEncrypt | **−57 %** (380 → 163 B) | **−53 %** | +4 % ¹ | −11 % / **−12 %** | ±0 ² | | `sub`, 2 sessions × 1 sub × 1000 items, 20 changing, pub 50 ms (9 rounds) | −2 % ³ | ±0 | −23 % ³ | ±0 / −2 % | ±0 | | `counters`, 4000 items, 10 ms sampling, pub 1 s (per notification) | −2 % | −6 % | −6 % | – | ±0 | | `o6counters`, 2048 items (per sample) | −3 % ⁴ (111 → 108 B) | −5 % | −7 % | – | ±0, all PASS | | `o6counters`, 4096 items (per sample) | −3 % ⁴ | +14 % ⁵ | −6 % | – | ±0, all PASS | ¹ CPU per notification varies by ±15 % between rounds on this machine; in an earlier 8-round batch against `071660354` the same scenarios measured −14 % / −1 %. Read it as "no regression". Client CPU per notification moved −18 %…0 %, within its spread. ² Bounded by the writer rate. ³ The server is nearly idle (0.05–0.12 cores, 360 notifications/s; its allocation is the writer and keep-alives). The first 3 rounds measured +31 % CPU, 6 more rounds −23 % (medians 969 → 742 ms per window): noise-bound, no regression. ⁴ With #4743's sampling permission cache a sample costs 111 B; one Publish per second carries 2048/4096 notifications. ⁵ The activator pools no longer allocate their 1023-slot arrays up front; the heap is a few MB smaller, which shifts the GC's generation budgets so that some low-allocation runs make every collection gen1 (26 of 26 instead of 2). Pause totals differ by milliseconds per 10 s window (37 → 42 ms here, 21 → 20 ms at 2048 items). Allocating the arrays eagerly restored master's gen1 counts in a separate test. In a single CPU trace of `sub` at pub 10 ms, recycling acknowledged payloads (pool `Return`) went from 16 % of server on-CPU time (master, the full pool scanned per return) to 5 %. ## Changes - **Send completion instead of a deep clone** (`NotificationMessageLease`, `LeasedPublishResponse`, `SentMessageQueue`): the retransmission queue and every Publish response that carries a message each own a reference. The transports already call `IPooledEncodeable.Reuse()` on a response once it is encoded or dropped; the Publish response is now a `LeasedPublishResponse` whose `Reuse()` releases its reference. The pooled payload (`NotificationMessage`, `DataChangeNotification`/`EventNotificationList`, `MonitoredItemNotification`/`EventFieldList`) is recycled when the last owner releases it, so an acknowledgement that overtakes the send no longer recycles a payload that is still being encoded, and the queue and the response share one message. A reference that is never released only leaves the payload to the GC; every owner releases at most once. Keep-alive messages are owned by their response alone. Republish, the durable snapshot and the retransmission mirror still copy. - **Transports**: the HTTPS (binary and JSON), WebSocket (UA-SC, JSON, OpenAPI) and Kestrel TCP listeners and the WebAPI endpoint dispatcher now release responses after encoding, as the TCP listener does, so leased payloads are recycled there too. A response retained for a reconnect is released when it is sent; a send that throws leaves the response to the GC. - **Activator pool** (`BoundedObjectPool`): rent and return scanned the 1023 slots linearly; with the pool full every returned notification scanned all of them before being dropped (16 % of server CPU at 94k notifications/s). It is now a `ConcurrentQueue` behind the lock-free first slot with an `Interlocked` bound (a locked stack spun under contention when transport threads return whole messages). The queue is created on first use, so activators of types that are never recycled no longer allocate a 1023-slot array each. The client decoder rents through the same pool. - **Pool bound** (`PooledEncodeableType<T>.EnsureMaximumRetained`, internal): the bound can be raised, never lowered. The subscriptions raise the `MonitoredItemNotification` and `EventFieldList` pools to twice the number of monitored items in the process (a notification is recycled about two publishing cycles after it was built), between the default 1024 and 65536. At the cap the retained notifications take about 5 MB; the bound stays raised after items are deleted. - **Message construction** (`Subscription.InnerPublish`/`ConstructMessage`, `MonitoredItem.Publish`): the per-Publish `Queue<>`/`List<>` objects are reused by the subscription; messages are built at their final size (no `List`→`ArrayOf` copy, no `AddItem` reallocation); the `MonitoredItemNotification[]` of a message is rented from `ArrayPool` and the message carries an exact-length slice of it; the lease returns the array (cleared) with the payload, and messages dropped on queue overflow return theirs at once; only arrays the subscription rented are tracked, so restored, mirrored and copied messages never reach the array pool; the `DiagnosticInfo` list is only allocated when a notification has one; timestamps are stripped on the struct before the pooled notification is filled. - **Ready list** (`IReadySignalingMonitoredItem`, `Subscription.PromoteReadyMonitoredItems`): `MonitoredItem` acquires a ready signal (compare-exchange, a full fence after its flags are set) when it queues a value or event or a resend is requested, and hands itself to the subscription's intake list. The publish timer checks only these candidates instead of every item under the subscription lock. An item stays a candidate while it has something pending that can become reportable without a new value (sampling deadline, aggregate interval, samples of a SAMPLING item, trigger flag); after releasing the signal the subscription checks again, so a value queued concurrently is never missed. Created, restored, modified, monitoring-mode and triggering changes mark candidates, as does an item that leaves the publish list with something pending. ### Behaviour notes (custom subscription or monitored-item extensions) - **Publish responses share the retained message.** `PublishResponse.NotificationMessage` returned by `SubscriptionManager.PublishAsync` is the retained message itself, not a copy, and its payload objects return to the activator pools after the response was sent and acknowledged. Code that keeps or modifies the message of a Publish response (for example in a `StandardServer.PublishAsync` override or a request interceptor) must copy it first (`CoreUtils.Clone`). `Republish` still returns a copy. - **Custom transports**: a transport that does not call `Reuse()` on responses keeps working; the payload is then collected by the GC instead of being pooled. - **Custom `IMonitoredItem` implementations** (not derived from `MonitoredItem`) cannot signal readiness: a subscription that contains one scans all its items on every publishing cycle, as before. `MonitoredItem` subclasses signal through the base class. - **`DataChangeNotification.MonitoredItems` of a published message is a slice of a pooled array**: read it through `ArrayOf<T>` (`Span`, indexer, `Count`); code that unwraps the underlying array would see a longer array. - **Order within a message**: notifications of different monitored items appear in the order the items became ready rather than in the internal list order. OPC UA does not define an order across items; the order of the values of one item is unchanged. ## Harness (`perf/`) - `--write-nodes K` for `sub`: the writer changes only the first K monitored nodes, for large subscriptions in which few items change. - `server-scenarios.txt`: publish-path scenarios (`--pub 10`, 1000 items per subscription with 20 changing). ## Testing - New `NotificationEncodingLifetimeRegressionTests` cases: an acknowledgement before send completion does not recycle the payload still being encoded (the send completion does); Republish after the send completed returns the original content and stays independent of the recycled original; a failed send that releases the response twice, plus a copy of the response, recycles the payload exactly once and only after the queue released it; a keep-alive is owned by its response; the rented notification array stays intact until both acknowledgement and send completion (either order), the message encodes only its slice, and queue overflow returns the arrays of dropped messages. The existing 30-case matrix (publish / queued / republish / mirrors × acknowledge / evict / clear × data / events) now asserts that a shared payload survives every queue mutation until its response is released. - New `ReadyListRegressionTests`: the same scenarios on a subscription that uses the ready list and on one that scans (it also has an item that cannot signal) must publish the same notifications, for triggering (sampling items, backlog of the triggering item), monitoring-mode changes (Sampling↔Reporting, Disabled, disabled before publish), disabled publishing with keep-alives and ResendData, a value queued while the item was in the publish list, and the backlog of an item that left Reporting while in the publish list. Mutation check: removing the item's signal fails 5 of 6, removing the subscription-side marks fails the backlog case. Plus: a published item without pending values is not tracked; creating 1200 items raises the notification pool bounds to at least 2400. - `PooledEncodeableTypeTests`: the raised bound is respected and never lowered. - net10.0 on the rebased branch: Server 8481 passed (5 skipped); Sessions 851 (379 skipped); Client 2788; Subscriptions.Classic 57; Subscriptions.Durable 36 (11 skipped); Redundancy.Server 884, Redundancy.Client 134, Redundancy.Samples 17; Stress Contract+Integration 45; Subscriptions 657 (6 skipped); Types pool tests 165. All 0 failed. - net48 build of `Opc.Ua.Server.Tests` and `Opc.Ua.Bindings.Https` and `Opc.Ua.Types.Tests` clean; `dotnet format whitespace` clean. ## Not changed (follow-ups) - Republish still copies the message (`ISubscriptionManager.Republish` returns a `NotificationMessage`, so the response cannot carry a lease without an API change). - `EventFieldList[]` per event message could be rented like the data change array. - Sampling remains the larger share of allocation and CPU in the `counters`/`o6counters` scenarios. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
39dd5632e3 |
Fix cache invalidation after accepted async indexed writes (#4746)
# Description After the full async write hook accepts an indexed update, merge the slice into the whole cached value when possible. If the cache is missing or stale and cannot absorb the slice, clear the cached value, set its status to `BadWaitingForInitialData`, and reset its timestamp to `DateTimeUtc.MinValue`. Preserve the handler's successful operation result, including `GoodClamped`, and report a value change. Never expose the replacement slice or stale data as the whole successfully updated cache. Synchronous fallback and simple-hook behavior are unchanged. The existing fixture remains intact, with its unusable-cache expectations updated for this intentional behavior correction. Fourteen additional cases cover successful merges, missing/stale cache invalidation, Good/GoodClamped operation results independently of cached status, handler rejection, type/rank validation, node/user access, and unsupported status/source/server timestamps. XML documentation explains the cache-versus-operation contract. Scoped analyzer cleanup uses asynchronous fixture helpers without removing the cancellation, lock-release, access, type, status, timestamp, or change-mask oracles. **Validation:** final head `aaf78b8df1bed46fb14033f3f3c9f52945469000` passed 32/32 cases on actual net10.0 and 32/32 on actual .NET Framework 4.8, with zero failures or skips. Both owned fixtures and the owned production file passed formatting/analyzer verification at info severity. The normal-reference .NET Standard 2.0 Types build passed with zero warnings/errors. The full `UA.slnx` suite and numerical coverage collection were not run locally; no full-suite or coverage-percentage claim is made. ## Related Issues Related to [#4511](https://github.com/OPCFoundation/UA-.NETStandard/pull/4511), the approved Wave A extraction plan. This standalone master PR does not close the tracking PR or imply completion of the remaining extraction groups. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
Marc Schier
|
4fd4535362 |
Fix managed-channel certificate ownership on early create failure (#4745)
# Description `CreateChannelAsync` now releases transferred certificate/chain handles when disposal or argument validation fails before transport creation. Managed entries use the existing inherited channel-creation method directly and share entry-retained, idempotently disposed handles with the transport, without adding another reference for non-adopting transports. The ownership contract is documented on the existing method; no internal redeclaration or forwarding implementation is needed. New and changed ownership APIs have multiline XML summaries and parameter documentation. Fourteen regression cases use generated certificates persisted in an isolated directory store and resolved through the real `CertificateManager.CertificateProvider`. They sign and verify SHA-256 digests to prove private-key usability through failed open/retry, publication/rotation, forward/reverse connection, and shutdown, and probe each certificate core's final retirement independently of global counters. Both adopting and non-adopting transports and optional chains are covered. Temporary stores are cleaned up. ## Related Issues Related to #4511: user-approved independent Wave A group 2 extraction. This PR does not close the overall tracking issue or include the other extraction groups. _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Tests were added and verified as described above; the coverage checklist remains unchecked because coverage was not measured. |
||
|
romanett
|
0dff94bd37 |
Server quick wins from the o6 benchmark gap analysis: admission sizing, sampling permission cache, routed Read dispatch (#4743)
The o6-automation benchmark gap analysis listed several server quick
wins that need no wire or public API change. This PR implements three of
them. The 2.0 server now passes the benchmark's 10 clients × 1000
outstanding Reads test with the benchmark's own queue sizing. Its 100 Hz
sampling capacity grows about 7–10× on the same machine, and a
single-value Read allocates 8 % less.
## Results
Paired A/B against `master` (
|
||
|
romanett
|
25ebf5a4d8 |
Server: remove the server-wide locks from the request pipeline, bounded transport buffer pool (#4742)
# Server: remove the server-wide locks from the request pipeline,
bounded transport buffer pool
Quick wins #5 and #6 of the o6 benchmark gap analysis. Under 10 channels
× 100 outstanding single-value Reads on 24 cores, about 59 % of the
server's on-CPU time spun in `Monitor`/`Lock` slow paths, and roughly 20
of the ~30 lock acquisitions per Read were on server-wide locks in the
request pipeline. Separately, on hosts with one or two processors every
response allocated a fresh 64 KiB chunk, because
`ArrayPool<byte>.Shared` runs out with about a thousand responses in
flight.
No wire or configuration changes. The only public API change is an
additive `RequestLifetime` constructor. The admission *sizing*
(`MaxQueuedRequestCount` → shared capacity and reserved floors) is
untouched; this PR only changes how the accounting is locked.
## Results
Paired A/B against `origin/master` (
|
||
|
Marc Schier
|
1bb4bdca1d |
Add Machine Vision Parts 1 and 2 (#4733)
# Description Implement the released [OPC 40100-1](https://reference.opcfoundation.org/specs/OPC-40100-1) and [OPC 40100-2](https://reference.opcfoundation.org/specs/OPC-40100-2) specifications with generated models, provider-backed runtimes, typed clients, MCP workflows, and DI/Machinery integration. - **Part 1 stays in `Opc.Ua.Vision`** and its client/server packages alongside the distinct Perception model. Provide control/acquisition state machines, jobs and synchronized steps, recipe/product/configuration/result management, file transfers, diagnostics, events, and conditions. - **Part 2 is in `Opc.Ua.MachineVision`, `.Client`, and `.Server`.** Provide asset categories, identification, image-handling aspects, health/maintenance data, topology, and transactional updates. The base Vision assemblies do not reference these optional packages; the Part 2 wire model does not depend on Part 1. - Preserve the release-pinned model bytes, wire namespaces and NodeIds. Perception types are in `Opc.Ua.Vision.Perception`; the AMCM generated namespace remains `Opc.Ua.Vision.MachineVision.AMCM`, and asset client/server extension namespaces remain `Opc.Ua.Vision.Client` and `Opc.Ua.Vision.Server`. - Use `AddOpcUaMachineVision` and `AddOpcUaMachineVisionAMCM` for model loading and factory registration. Each has node-collection, encodeable-factory-builder and node-state-factory-builder overloads. These short entry points delegate to the generated implementations; existing namespace-derived generated names remain available for compatibility. - Support multiple independent named systems/assets under one namespace-owning manager. Reuse injectable providers, file binding, bounded memory stores, optional transactional `IFileSystemProvider` persistence, state machines, telemetry and conformance infrastructure. - Keep one `docs/Vision.md` guide, titled `Machine Vision (OPC 40100-1 and OPC 40100-2)`, and one `Opc.Ua.Mcp.Vision` package. Include deterministic server/client samples. - Expose 17 Machine Vision MCP tools, including capabilities/status, bounded relationships and observation, complete small-file uploads/downloads, management, explicit control, conditions, and chunked transfers. Catalogue counts: 39 Vision-specific, 43 with connection tools, 81 Vision+Robotics, 155 Full. Every Machine Vision wire parameter has a schema description. **Compatibility and limits:** `AddVision()` remains Perception-only unless released parts are selected. Standalone Part 2 uses `AddVision().AddMachineVision()`; Machinery composition uses `AddVisionToMachinery()`. Existing tool defaults and command signatures are preserved. Observation owns and cleans up its subscription: 5 seconds/50 notifications by default, capped at 60 seconds/100 notifications. Complete-file tools are capped at 1 MiB; larger files use chunked transfers. Domain Error values remain distinct from UA service failures. **Versioning:** the seven Vision/MachineVision/MCP packages and their Debug variants follow the repository root version instead of being forced to preview. Vision.OpenUsd and its descendants remain preview-only. Development builds still inherit the root prerelease version. No release version, tag, release branch, publication policy, or feed promotion is changed. **Review actions:** all fourteen review threads have action/evidence replies and are resolved. Changes include the unified guide, Perception namespace, separate Part 2 packages and final naming, multi-system ownership explanation, interrupted-write cleanup, observer failure isolation, complete MCP parameter descriptions, and explicit package inventory. The preview-layout migration topic and the requested ReleaseProcess caveat were removed. The test Publish collector now retries recognized transient failures within one cancellation budget and propagates other exceptions. Commit |
||
|
romanett
|
0716603543 |
Fix concurrent session creation on a shared ClientChannelManager channel (#4739)
# Description Sessions created concurrently through one `ClientChannelManager` against the same endpoint (`Session.CreateAsync(IClientChannelManager, ...)`) failed with `BadSecureChannelClosed 'The managed transport is closed.'`, and the log showed `Background operation OpenChannel started by ClientChannelManager failed. Channel is Closed.` Creating the same sessions one after another worked. ## Root cause A caller that finds an existing shared entry gets a lease right away, even while the entry's creator is still opening the transport: 1. That session's `OpenAsync` called `lease.Entry.SnapshotClientCertificate()` before any transport was installed and threw *"The managed transport is closed."* 2. The failed session disposed its lease. The refcount dropped to 0 and the entry was torn down. 3. The creator's in-flight `OpenInitialAsync` then found the entry closing and threw *"Channel is Closed."* Every session attached to that entry failed. ## Changes - **`Session.OpenAsync`** waits until the shared entry has an installed transport (new `ChannelEntry.WaitForTransportAsync`) before it reads the certificate. The signal fires when the initial open installs a transport, or when a reconnect that started during the initial open does. It does not wait for `Ready`: `OpenCoreAsync` also runs for session recreate inside a reconnect cycle, and the cycle installs the transport before participants reactivate. - **`ClientChannelManager.GetCoreAsync`**: the creator holds no lease while it opens the entry, so other participants' releases can close the entry under the open. The creator now retries on a fresh entry instead of failing. When the initial open fails on its own, the entry records it (`InitialOpenFaulted`, set under the entry lock at failure time, because the later lifecycle state cannot tell the two cases apart). A faulted open, or one closed by manager disposal, still fails the creator. ## Tests - `SharedChannelConcurrentSessionTests.SessionAttachedWhileChannelOpensWaitsForTransportAsync` (Client.Tests): holds the creator's transport through gated channel bindings until a second session has attached to the opening entry, then checks that both sessions open and share the channel. Fails deterministically without the fix. - `ClientChannelManagerManagedTests.WaitForTransportWaitsForReconnectStartedDuringInitialOpenAsync` and `CreatorDoesNotRetryInitialOpenThatFaultedAsync` (Core.Tests): cover the reconnect-during-open wait and the no-retry-on-fault path. - `SharedChannelConcurrentSessionTests.ConcurrentlyCreatedSessionsShareOneChannelAsync` (Client.Tests): creates 20 sessions concurrently through one manager against the reference server. It checks that they share one channel with refcount 20 and that every session can read. Fails without the fix. - `ClientChannelManagerManagedTests.CreatorRetriesWhenLeaseReleasedDuringInitialOpenAsync` (Core.Tests): holds the initial open with a mock transport, then attaches and releases a second lease during it. It checks that the creator gets a lease on a fresh entry. Fails without the fix with "Channel is Closed." Verification: - **Local runs (net10.0, sequential):** Opc.Ua.Client.Tests 2778 passed; Opc.Ua.Sessions.Tests 851 passed, 379 skipped; Opc.Ua.Core.Tests 6241 passed, 88 skipped. - **net48:** Client.Tests and Core.Tests compile cleanly. - **ServerLoadHarness repro:** `--client shared --sessions 20 --connect-parallel 4/20` now connects every run. The same build without the fix still fails. Note: when the creator's open is cut short this way, the `OpenChannel` background failure is still logged at `fail` level even though the creator now recovers. ## Related Issues - None filed yet. Found while load-testing the shared-channel client with the ServerLoadHarness. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
f5062be349 |
Complete each pooled service response at most once (#4729) (#4736)
## Proposed changes Fixes the cross-talk part of #4729 on master. `ServiceResponsePooledValueTaskSource` comes from one pool shared by the whole process, and `ManualResetValueTaskSourceCore.SetResult` never checks the version token. A second completion of an `EndpointIncomingRequest` therefore either threw `InvalidOperationException` or, once the source had been read and recycled, answered whichever request rented it next. That request could belong to another server instance in the same process. master's request queue (Channel + `FairRequestQueue`) already completes each request exactly once. But `ServerBase.ScheduleIncomingRequest` and `ProcessRequestAsync` are virtual, so a single override that completes and then throws (the `ProcessAsync` catch completes again) was enough to corrupt unrelated requests. - Each rental of the pooled source now gets a lease number, which is never reused for a later rental of the same instance. - `EndpointIncomingRequest` captures the lease when it is constructed and completes through `TrySetResult(lease, response)`. That is a single CAS on the pending lease, so only the first completion of the current rental is accepted. Late or stale completions are discarded and can never reach the next owner. - `SetResult`/`SetException`, which had no callers outside this path, are replaced by `TrySetResult`. The master378 backport #4738 also fixes the double completions in the old `RequestQueue` itself (snapshot in `Dispose`, and enqueue followed by a throw). ## Tests - `ServiceResponsePooledValueTaskSourceTests`: a second completion of the same rental is ignored, and a late completion after the source is recycled does not complete the next rental. Both use a private pool, so the result is deterministic. - `EndpointIncomingRequestTests.ProcessAsync_ScheduleCompletesThenThrows_KeepsFirstCompletionAsync`: a scheduler that completes the request and then throws. Without the fix it fails with `InvalidOperationException`. - `Opc.Ua.Core.Tests` Stack.Server + Stack.Tcp (net10.0): 1871 passed, 0 failed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
2c431ad728 |
Client/transport: one recv per small chunk, ManagedSession lock fast path, write coalescing (#4737)
Follow-ups from the "Not changed" list of #4720 (client per-request allocation and CPU): three changes that the paired A/B shows a gain for, plus a write-up of the fourth (discarded `ResponseHeader`), which is not changed. ## Results Paired A/B, each change against the commit before it: 4 alternating rounds, medians, 20 sessions, 24-core machine shared with other sessions. Client = the harness process, server = the reference server child. Allocated bytes per operation are deterministic; throughput and latency carry the run-to-run noise noted below. | Change | Scenario | Throughput | p50 | p90 | Client CPU / op | Client alloc / op | Server CPU / op | |---|---|---|---|---|---|---|---| | Receive staging | Read, 1 value | +4% | **−9%** | **−13%** | ≈ | ±0 | ≈ | | | Read, 1 value, SignAndEncrypt | +6% | −6% | −11% | ≈ | ±0 | ≈ | | | Read, 100 values | +2% | −4% | ≈ | ≈ | ±0 | ≈ | | | Data changes | ≈ | ≈ | ≈ | ≈ | ±0 | ≈ | | ManagedSession lock | Read, 1 value, ManagedSession | ≈ | ≈ | ≈ | ≈ | **−125 B** (2632 → 2507) | ≈ | | | Same, inflight 8 | ≈ | ≈ | ≈ | ≈ | −133 B | ≈ | | Write coalescing | Read, 1 value, 20 sessions on one shared channel, inflight 4 | **+13%** | **−13%** | −3% | −6% | ±0 | −4% | | | Same, inflight 1 | +20% ¹ | −8% | ¹ | −9% | ±0 | −13% | | | Read, 1 value, inflight 8 (one channel per session) | +2% | −4% | −6% | ≈ | ±0 | ≈ | | | Read, 100 values, inflight 8 | +3% | −4% | −8% | ≈ | ±0 | ≈ | | | Data changes | ≈ | ≈ | ≈ | ≈ | ±0 | ≈ | ≈: within run-to-run noise (about ±5% for CPU per op here). ¹ Rounds ranged from −42% to +32%; the inflight-4 runs were better in all four rounds. ## Changes - **`TcpByteTransport` receive**: `ReceiveChunkAsync` read the 8-byte chunk header and then the body, two `recv` calls per chunk. It now waits for data with a small staging buffer (8 KB, allocated on the first receive, not from the `BufferManager` budget). One receive returns the header and, when the chunk is small and already there, the whole chunk; bytes of following chunks are kept for the next call. A larger chunk is completed with a receive into the rented buffer, as before. - The receive buffer is still rented only after the header has been validated (message type, size against the negotiated limit), so an idle peer holds none of the budget. - A receive cancelled while its bytes are in the staging buffer keeps them for the next call; on .NET Framework the staging receive takes no token, so cancellation is observed once data arrives. - A receive that fails while reading the body of a chunk larger than the staging buffer has consumed part of the chunk; later receives then fail with `BadTcpInternalError` rather than read body bytes as a header. Before this PR such a failure lost the header and body bytes silently. The channel cancels receives only when it closes. - Server listener channels use the same transport. - **`ManagedSession` service calls**: each call was an async wrapper (await the reader lock, await the inner call), one more state machine box per request. - When services are available and no writer holds or waits for the lock, the reader lock is taken synchronously (`AsyncReaderWriterLock.TryEnterReader`). It is released by a cached continuation on the inner call's task, registered before the caller's await, so the lock is free again when the caller resumes. - Otherwise the call waits for the lock as before, so reconnect and failover still block new calls and drain the ones in flight. - The remaining 88 B per call is the task's continuation list (two continuations). - **Write coalescing** (`UaSCUaBinaryChannel`): when more writes for the same transport are queued behind the one being sent, the writer takes up to 16 of them and sends their segments in one vectored send. - Each write is then reported on its own, in order, with its size. A failed send is reported for every write in it, and their buffers are returned. - Writes still never start on the caller's stack, and one writer sends one batch at a time, so chunk order is unchanged. - Only byte-stream transports opt in (`TcpByteTransport`, also through the server's admission lease, via the internal `IUaSCChunkCoalescing`). WebSocket and capture decorators keep one send per chunk. ### Behaviour notes - Each TCP connection that has received holds an 8 KB staging array (pinned on .NET 5+), outside the `BufferManager` budget. The previous transport held an 8-byte header array. - `ManagedSession` service calls: if the inner call completes while the release continuation is being registered, the reader lock is released on the thread pool shortly after, instead of before the caller resumes. - Concurrent `ReceiveChunkAsync` calls on one `TcpByteTransport` now throw `InvalidOperationException`. The channel has a single receive loop, and interleaved receives on a stream were never valid. ## Not changed: discarded `ResponseHeader` Every generated structure's `Initialize()` constructs its structure-typed fields: `new ResponseHeader()` (112 B), `new RequestHeader()` (104 B). The decoders always create the instance (`new T()` or the activator) and then assign every field in `Decode`, so the default is thrown away. That is 4.6% of the client's 2.4 KB per Read and about 3% on the server, which discards a `RequestHeader` per decoded request and a `ResponseHeader` per response it constructs. `PublishResponse` also discards a default `NotificationMessage`. Options considered: - **Lazy getter** (`get => m_x ??= new X()`): `IsEqual`/`GetHashCode` compare fields, so a never-read default (null) would no longer equal an explicit default. Two threads reading an unset header could get different instances. Pooled instances reset through `Initialize()` anyway. - **Decode-only constructor** called by the activator: only the plain `Class`/`DerivedClass` templates assign every field in `Decode`, not optional-field or union types. A derived type's constructor must chain to a base built by an older generator in another assembly, which lacks it (the same problem `ForeignDerivedPooledExtensionClass` works around). It also needs a new public marker type in `Opc.Ua.Types`. - **Decode into the existing instance**: needs a new `IDecoder` member in all decoders. It also changes behaviour when a caller decodes into an object whose header it still references. Each touches every generated type for about 100 B per message, so this is left as is. ## Harness - `--client shared`: classic sessions whose secure channels come from one `ClientChannelManager`. - Created sequentially (`--connect-parallel`, default 1). Concurrent `Session.CreateAsync` calls on one manager fail with `BadSecureChannelClosed`, a separate bug. - 20 sessions on one channel reach about 45–70k req/s, against about 140k on 20 channels. - `ab.sh`: `CANDIDATE=` selects the candidate build. ## Testing - New: `TcpByteTransportTests`: chunks sent together, a chunk split across sends (inside the next header), a chunk larger than the staging buffer, cancellation keeping staged bytes. `UaSCBinaryChannelWriteQueueTests`: coalesced in order (1 + 16 + 16 + 7 sends), framing transport not coalesced, a failed batch reported for every write and later writes still sent. `AsyncReaderWriterLockTests`: `TryEnterReader` against a writer, the reader held until the operation completes and released on failure and on a completed operation, the awaiter resuming after the release. - Before the rebase onto master, net10.0, run sequentially: Core Stack+Security 4689 passed, 1 failed (`ClientBaseTests.ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`, a meter-listener flake fixed on master by #4708; passes 3/3 in isolation); Client 2768; Sessions 851 (379 skipped); Server 8398; Subscriptions 643. - After the rebase: Core Stack+Security 4811 passed; Client 2781; Stress Contract+Integration 45 passed; all 0 failed. Sessions, Server and Subscriptions were not rerun on the new base. - net48 build of `UA.slnx`: clean (0 warnings, 0 errors); `dotnet format whitespace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
5e71e2243c |
Test that a secured connection refuses its JSON metadata announcement (#4724)
# Description #4712 makes a connection configured for message security send its DataSetMetaData announcement through the connection's security step, so the announcement is never sent unsecured. Its tests cover the UADP announcement. This adds the JSON case. The UADP security wrapper cannot protect a JSON message, so a connection whose WriterGroup uses `SignAndEncrypt` must refuse the JSON metadata announcement rather than publish the DataSet layout in plaintext. `MetaDataPublisherTests.JsonAnnouncementOfASecuredWriterGroupIsRefusedAsync` checks that nothing is published on the metadata topic and that the refusal is counted in `EncryptionErrors`. `PubSubApplication.StartAsync` returns after the initial announcement was sent or refused, so the check needs no wait. The test covers the fail-closed branch of `PubSubConnection.EncodeOutboundAsync` and the refused-send return of `SendNetworkMessageAsync`, two of the three changed lines the patch-coverage report of #4712 listed as uncovered. Test-only change. Review: the test double `RecordingTransport` guards its recorded sends with a private `System.Threading.Lock` and exposes `SendCount` and `GetSendsSnapshot()`, so the tests no longer lock the list itself ( |
||
|
romanett
|
b25429bccc |
Client: cut per-request allocation and CPU in the channel and session; measure the client in the load harness (#4720)
Follows #4707 (replaces #4714, which closed when the #4707 branch was deleted). Profiling the client process under load showed per-request allocation spread over the channel write path, the response wait, per-chunk header encoders and an AES instance per message, plus a stack walk per service call. This PR removes them and extends the load harness to measure the client. ## Results Paired A/B against the #4707 head, the server work now on master (3 alternating rounds, medians, 20 sessions unless noted, workstation GC on both sides, 24-core machine). Client = the harness process, server = the reference server child. | Scenario | Throughput | p50 | p90 | Client CPU / op | Client alloc / op | Server alloc / op | |---|---|---|---|---|---|---| | Read, 1 value | **+21%** (141k → 170k req/s) | −9% | −11% | −13% | −46% (4.5 → 2.4 KB) | −20% | | Read, 1 value, ManagedSession | **+18%** | −9% | −12% | −18% | −44% | −20% | | Read, 1 value, SignAndEncrypt | **+28%** | −14% | −20% | −24% | −52% | −26% | | Read, 1 value, 200 sessions | **+29%** | −11% | −17% | −17% | −46% | −20% | | Read, 100 values | +5% | −2% | −3% | −6% | −44% (15.6 → 8.8 KB) | −2% | | Read, 100 values, SignAndEncrypt | +8% | −7% | −2% | −2% | −46% | −3% | | Write, 100 values | ±0 | ≈ | ≈ | −14% | **−63%** (10.8 → 4.0 KB) | −2% | | Browse (Server object) | **+12%** | −7% | −11% | −19% | −17% | −12% | | Data changes, classic fast callback | ±0 ¹ | ≈ | ≈ | −21% | −10% | −5% | | Data changes, classic item events | ±0 ¹ | ≈ | ≈ | ≈ ² | −7% | −5% | | Data changes, ManagedSession (V2) | ±0 ¹ | ≈ | ≈ | ±0 | −5% | −5% | | Data changes, SignAndEncrypt | ±0 ¹ | ≈ | ≈ | −22% | −14% | −7% | ¹ Throughput is bounded by the writer rate; latency is dominated by the 100 ms publishing interval. Per notification the client uses about 2 µs and 170–260 B. ² The client uses 0.2–0.4 cores in the subscription scenarios; the CPU per notification is within run-to-run noise there. Client GC pause time in the window drops 38–66% in the Read and Write scenarios (17% for Browse). About 70% of the remaining client CPU per small request is kernel time in loopback socket send/receive. ## Changes - **Activity source**: `Session`, `SessionClientBatched`, the classic `Subscription` and `ClassicSubscriptionEngine`, and the TCP/HTTPS client channels resolve their `ActivitySource` once. `TelemetryExtensions.StartActivity` called `Assembly.GetCallingAssembly()` (a stack walk) on every service call, ~8% of client CPU. - **Channel write queue** (`UaSCUaBinaryChannel`, client and server): an ordered queue drained by one thread-pool writer replaces the per-write `ContinueWith`/`Unwrap`/trailing continuation chain. The writer is queued to the issuing thread's local pool queue, like the task continuation was (queuing it globally cost ~15 µs of p50 in a first iteration). Successful writes are reported inline instead of through `Task.Run`; failed writes still go through the pool. The client channel enters its gate only for a failed write instead of on every write completion. - **`TcpByteTransport`**: no linked `CancellationTokenSource` when the caller's token cannot be cancelled (the channel always passes `None`); a single-chunk `BufferCollection` uses the `ReadOnlyMemory` send instead of the vectored `Task<int>` send; `PoolingAsyncValueTaskMethodBuilder` on .NET 6+ for the receive/send state machines; no closure per vectored send. - **`BinaryEncoder.GetNodeIdEncoding`**: `Convert.ToUInt32(uint, IFormatProvider)` bound to the `object` overload and `Convert.ToByte(enum, ...)` boxed the enum, two boxes per encoded NodeId. 44% of the client allocation of a 100-value Read; the server encodes NodeIds the same way. - **Symmetric chunk headers**: the 24 header bytes are written and read with `BinaryPrimitives` instead of a `MemoryStream` + `BinaryEncoder`/`BinaryWriter` (send) and `BinaryDecoder`/`BinaryReader` (receive) per chunk. A chunk too short for a field still gets the decoder's `BadDecodingError`. - **AES-CBC**: `ChannelToken` keeps one `Aes` per direction, used with the one-shot `EncryptCbc`/`DecryptCbc` on .NET 6+ (net48 reuses the instance and creates the transform). Previously every message created an `Aes`, imported the key (`BCryptImportKey`) and created a transform, ~15% of client CPU with SignAndEncrypt. The instance is taken out of the token while in use, so concurrent callers never share one. The public `CryptoUtils` overloads are unchanged. - **`ChannelAsyncOperation`**: `SendRequestAsync` waits on an internal task-free completion source instead of `EndAsync` (`TaskCompletionSource` + `Task` + `WaitAsync` promise + state machine); `EndAsync` registers cancellation instead of `WaitAsync(ct)`; the timeout timer uses a static callback; completing with `default(T)` shares a box. - **Session keep alive**: every good response called `Timer.Change` under the session lock, taking the timer queue lock. It now records the time; the timer callback defers itself to one interval after the last response, which is when the per-response reset made it fire. ### Behaviour notes - `TcpByteTransport.SendChunkAsync`/`ReceiveChunkAsync` return pooled `ValueTask`s on .NET 6+: like any `ValueTask` they must be awaited once. - A successful write's `HandleWriteComplete` now runs on the writer instead of a separate pool work item. Overrides outside the stack that block in it delay later writes on that channel. - The write itself no longer flows the caller's `ExecutionContext` (`Activity.Current` is not set inside the transport send). ## Harness (`perf/`) - `client-process:` and `client-per-op:` result lines; `--trace-side client` records the harness itself. - `--client managed` (ManagedSession, V2 engine) and `--callback item` (classic per-item events). - Classic fixture sessions no longer request diagnostics in every response (the server otherwise sends a `DiagnosticInfo` per notification). - `ab.sh`: `SCENARIOS`/`ROUNDS`/`OUT`, CRLF tolerant, defaults before the scenario (a `--sessions 200` scenario silently ran with 20 before); `client-scenarios.txt`; `ab-summary.ps1` reports client and server medians. - README notes two profile traps: `Monitor.Enter_Slowpath` in the sampled CPU profile without matching contention events (46 in 10 s here), and allocation ticks over-attributing small objects allocated after large arrays. ## Testing - New: `UaSCBinaryChannelWriteQueueTests` (order, one send at a time, never on the caller's stack, failures reported and later writes still sent, concurrent writers), `ChannelAsyncOperationWaitTests` (late/early completion, faults, cancellation, disposal, continuation off the completing thread, 2000-iteration completion/cancellation race, timeout), `CryptoUtilsAesCacheTests` (cached = fresh cipher text across IVs, round trips, concurrent users, token disposal), `TcpByteTransportTests` single/multi-buffer and cancellable sends, `SessionKeepAliveTests` keep-alive read exactly one interval after the last response. - After merging master: Core Stack 3571 passed, Client 2763 passed, 0 failed. - Before merging master, net10.0: Core Stack+Security 4680 passed; Client 2763; Sessions 849 (379 skipped); Server 8391; Subscriptions 643; all 0 failed. - Stress Contract+Integration: 5 failures, the same 5 on the unchanged #4707 head (`CertRotationContractTests`, `GateAndBypassTests`, `LeaseLifecycleTests`, `RetryBudgetEnforcementTests` reflection seam, `CertRotationLiveTests`), not caused by this PR. - net48 build of `UA.slnx` clean; `dotnet format whitespace` clean. ## Not changed (follow-ups) - Two `recv` calls per chunk (8-byte header, then body). Merging them needs read-ahead with carry-over, which the transport avoids so that idle peers do not hold receive buffers. - `ManagedSession` service wrappers add an async state machine and a reader lock per call (~200 B). - Generated responses allocate a discarded `ResponseHeader` (generator-wide, as noted in #4707). - Decoded `MonitoredItemNotification` objects and the V2 `DataValueChange[]` per message dominate notification allocation (`WithPoolNotifications` pools the latter). - Coalescing queued writes on one channel into one vectored send would cut syscalls under concurrent requests on a shared channel. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc <marcschier@hotmail.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
romanett
|
c7fdeb6de1 |
Fix stress Contract/Integration failures and run them in stress-test.yml (#4718)
## Description `Opc.Ua.Stress.Tests` with `Category=Contract|Category=Integration` had five failures on master. The README says these categories run on every PR, but no workflow ran them: `get-ci-matrix.ps1` excludes the Stress project, and `stress-test.yml` only ran `ChaosTCP`. The old `Channels.Stress` special case in `buildandtest.yml` never matched the project name, and #4513 removed it. So the tests rotted unseen. Two of the five failures were real product bugs. ### Product fixes - **Session recreate after a server restart failed forever (regression from #4503).** User-token PolicyIds are assigned by the server and are not stable across a restart. In the failing run the client had cached `Anonymous|20` and the restarted server offered `Anonymous|2`. The CreateSession check added in #4503 requires an exact match, including PolicyId, so every `RecreateInPlace` failed with `BadSecurityChecksFailed` ("different security policy for the selected user identity token"). The managed session then retried without end. - `Session.OpenCoreAsync` now adopts the server's policy when only the PolicyId differs. TokenType, SecurityPolicyUri, IssuedTokenType and IssuerEndpointUrl must still match, so a downgrade is still rejected. The token is restamped with the new id. - **A certificate rotation during a reconnect reactivated every session twice.** `ChannelEntry.ReconnectCycleAsync` only noticed the superseded client certificate *after* it had notified participants. It then replaced the transport and notified them again. The check now also runs right after the transport reconnects, before participants are notified. ### Tests updated to the #4516 contracts - `RetryBudgetEnforcementTests`: `Session.ReconnectManagedChannelAsync` gained an optional reverse-connection parameter, which the reflection call must supply. - `GateAndBypassTests`: the participant callback token is cancelled once `OnReconnectAsync` returns. The spawned worker now runs on the test token. - `LeaseLifecycleTests`: releasing the last lease cancels the recovery in progress, as documented. The reconnect caller now expects `OperationCanceledException`. ### CI - New `Contract and Integration` job in `stress-test.yml`, with the same gate as Chaos TCP: nightly, manual, and PRs labelled `stress`. It takes about 1 minute of test time. - The README now says where these categories actually run. ### Tests - New `SessionIdentityReviewTests.CreateAdoptsRenumberedUserTokenPolicyIdAsync`, covering both a renumbered id (accepted) and a changed token security policy (rejected). It fails without the fix. - Each of the 5 fixed tests passed 3/3 runs in isolation. The full `Contract|Integration` filter passed 45/45 twice in Debug and once in Release. - Related tests also pass: Core.Tests channel tests 587/587, Sessions.Tests channel tests 119 passed (63 skipped), Client.Tests channel/session/reconnect tests 512/512. - The net48 build of `UA.slnx` is clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Cristi Pogacean
|
9344354aaa |
Fix empty LocalizedText binary field encoding (#4709)
# Description Omit empty `Locale` and `Text` fields from the OPC UA Binary DataEncoding for `LocalizedText`, as required by OPC 10000-6, Section 5.2.2.14, Table 24. Add wire-format tests for empty and non-empty fields, including translation-backed localized text. Validation: - `git diff --check` passed before commit. - `dotnet build tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj --framework net10.0 --no-restore --verbosity minimal -p:NuGetAudit=false` could not complete because access to `https://api.nuget.org/v3/index.json` failed with `NU1301` and a TLS handshake error. ## Related Issues No related issue. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
9099aee13e |
Fail CI when a test fixture's setup or teardown fails (#4723)
# Description
`get-test-verdict.ps1` accepts a non-zero `dotnet test` exit whenever
the TRX counters show no failure, on the assumption that the host died
at exit after every teardown had run. A failed fixture setup or teardown
breaks that assumption. The assembly-level `[OneTimeTearDown]`
certificate leak check fails after its tests passed, no TRX counter
records it, and the host exits 1. In the last 10 master runs, CI
accepted exactly that as an "at-exit stall" for `Opc.Ua.Gds.Tests` and
`Opc.Ua.Server.Tests` on every leg (net48, and net10.0 on Windows,
Linux, and macOS). #4719 fixed the Server.Tests leak; #4721 fixes the
GDS leak.
The NUnit adapter reports such a failure as a TRX run message, `<RunInfo
outcome="Error"><Text>TearDown failed for test fixture
Opc.Ua.Gds.Tests.LeakDetectionSetup</Text></RunInfo>`, followed by a
second one with the failure message. For a setup failure it writes
`Setup failed for test fixture …`.
Changes:
- `Measure-TestResults` collects these messages. `Get-TestRunVerdict
-FixtureFailures` rejects a run that has any of them, whatever the exit
code, and names the fixture in the reason.
- A host that dies at exit after a clean run is still tolerated. The
only other tolerated case in the last 10 master runs,
`Opc.Ua.Client.Tests` on net48 in run 37835429177, logged "Test host
process crashed" and stays tolerated.
- `Measure-TestResults` and `Get-CounterValue` move from
`run-dotnet-tests.ps1` to the dot-sourced `get-test-verdict.ps1`, so
`CiTestVerdictTests` can feed them TRX files. The source check of the
counter set becomes a behavioral test.
- `docs/DeveloperGuide.md` describes the rule.
**Merge order:** after #4727 (#4719 and #4721 are on master). With #4721
merged, this PR's CI still rejects `Opc.Ua.Gds.Tests` on net48: eight
`PushTest` fixtures fail in `TearDown` and leak 88 certificates when
their setup is ignored, which master accepted as an "at-exit stall".
#4727 fixes them. On every other leg, `Opc.Ua.Gds.Tests` now passes
under the rule.
### Validation
| Command | Result |
| --- | --- |
| `dotnet test tests/Opc.Ua.Tools.Tests -f net10.0 --filter
FullyQualifiedName~CiTestVerdictTests` with the scripts from master | 6
new tests fail, 13 pass |
| Same with this change | 19 passed |
| `Measure-TestResults` and `Get-TestRunVerdict` on the TRX of a full
`Opc.Ua.Gds.Tests` run without #4721, exit code 1 | Rejected: `1 test
fixture setup or teardown failure(s): TearDown failed for test fixture
Opc.Ua.Gds.Tests.LeakDetectionSetup.` |
| Same on the TRX of a run with #4721, exit code 1 | Tolerated, as
before |
| CI of this PR before #4721 merged | `Opc.Ua.Gds.Tests` rejected on all
four legs for `TearDown failed for test fixture
Opc.Ua.Gds.Tests.LeakDetectionSetup`; every other project passed |
| CI after merging master with #4721 (
|
||
|
Cristi Pogacean
|
2cdf0de97f |
Fix scoped certificate-store resolution in GDS push operations (#4726)
# Description Fix the remaining GDS push certificate paths that bypass instance-scoped certificate stores, while preserving OPC UA 1.05.07 trust and transaction requirements. - Use the configured resolver/registry for previous-key loading, slot resolution, application-store commit/rollback, issuer import/compensation, and rejected-store opening. - Validate application-certificate uploads through the configured manager and the group's registered trust scope, instead of a temporary manager that cannot see custom providers. - Isolate per-call `AcceptError` validation cores from normal cached trust decisions. Part 12 installation-time acceptance must neither reuse nor populate the connection-validation cache. - Compensate the application-certificate swap if opening the issuer store fails, not only if importing an issuer fails. - Add scoped-provider, cross-group trust, cache-isolation, and compensation regressions. Reuse the portable PEM writer and short temporary PKI paths so the tests work on .NET Framework too. - Use source-generated failure logging without private-key bytes, and document custom-store/pending-key requirements. The branch starts at master `7f14fb864fc93532f3dbab1e1b5e2ba9b5f605c5`. It excludes the unrelated LocalizedText commits, the local handoff, PLC changes, logs, keys, certificates, and build outputs. No public API, dependency, package version, global provider registration, or analyzer configuration is changed. ### Specification review The online Part 12 reference identifies version **1.05.07**. This is a focused review and regression validation of the changed behavior, **not a claim of complete stack conformance or CTT certification**. | Requirement | Review / evidence | | --- | --- | | [Part 12, 7.10.5](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.5): validate application certificates using the associated TrustList; ignore the supplied issuer list | Existing issuer-argument regressions retained. New flat-layout scoped-provider cases accept a CA installed in the selected trusted/issuer store and reject one installed only in another scope. | | [Part 4, 6.1.3](https://reference.opcfoundation.org/specs/OPC-10000-4/6.1.3) and Part 12, 7.10.5: suppressible installation errors must not bypass normal peer trust or non-suppressible failures | Missing-issuer, signature/revocation, explicit-trust, and validation-core tests; new two-way cache-isolation cases with `UseValidatedCertificates = true`; normal peer validation remains untrusted after issuer-only push acceptance. | | Part 12, 7.10.5: matching private key, encrypted channel and SecurityAdmin | Existing access/key/type tests retained; scoped own-store upload cases cover no uploaded key and PEM. PFX and regenerated-key cases remain in the selected suite. | | [Part 12, 7.10.2](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.2), [7.10.9](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.9), and [7.10.11](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.11): staging, ApplyChanges, cancellation, ownership and compensation | Push/transaction/rollback suites; new custom-provider compensation cases cover both issuer-import and store-open failure, retaining the old application certificate and pre-existing issuers. | | [Part 12, 7.10.7](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.7): do not delete endpoint-referenced certificates | Existing ApplyChanges endpoint-reference and delete/create replacement tests retained. | | [Part 12, 7.10.10](https://reference.opcfoundation.org/specs/OPC-10000-12/7.10.10): CSR key handling and persistence | Existing-key scoped CSR test, SDK pending-key/cancel cases, and supporting PLC encrypted round trips/restarts. Arbitrary custom backends still require a durable pending-key implementation; the separate PLC implementation is not included here. | ### Validation Windows, .NET SDK 10.0.401, Release, NUnit/VSTest. Build analyzers remained enforced: **no `WarningsNotAsErrors`, suppressions, or analyzer downgrades**. `NuGetAudit=false` was needed because the local nuget.org TLS connection failed; no repository audit setting was changed. | Selection | Result | | --- | --- | | SDK GDS push/trust/transaction/rollback, net10.0 | **192 passed, 0 failed** | | SDK certificate-manager/provider/explicit-trust/validation-core, net10.0 | **180 passed, 0 failed** | | SDK certificate-manager/provider/explicit-trust/validation-core, net48 | **180 passed, 0 failed** | | SDK GDS push/trust/transaction/rollback, net48 | **192 passed, 0 failed** | | Final portable-PEM/compensation tests, net10.0 | **5 passed, 0 failed** | | Supporting PLC GDS/pending-key/compatibility selection, net10.0 | **29 passed, 0 failed** against this SDK source | The PLC tests exercise encrypted UA calls, signed PEM/CSR updates, persistence, fresh connections and restarts across Directory, FlatDirectory and KubernetesSecret providers. The Kubernetes API is an in-memory double, **not a real Kubernetes deployment**. PLC tests and its durable pending-key implementation are local companion-repository changes, not SDK PR contents or independently reproducible CI evidence from this branch alone. Commands below use the same isolated artifact locations as the validation run, expressed relative to the user profile: ```powershell $files = Join-Path $env:USERPROFILE '.copilot\session-state\003a7438-0312-4c10-beed-cd95c0fe400e\files' $sdk = Join-Path $files 'sdk-build' $net48 = Join-Path $files 'solution-net48' $push = 'FullyQualifiedName~ConfigurationNodeManagerPushTests|FullyQualifiedName~ConfigurationNodeManagerPushValidationTests|FullyQualifiedName~PushComplianceRegressionTests|FullyQualifiedName~PushRollbackCancellationRegressionTests|FullyQualifiedName~PushConfigurationTransactionCoordinatorTests' $manager = 'FullyQualifiedName~CertificateManagerTests|FullyQualifiedName~CertificateManagerExplicitTrustTests|FullyQualifiedName~CertificateManagerProviderResolutionRegressionTests|FullyQualifiedName~CertificateValidationCoreTests' # Passed: 192/192. dotnet test .\tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -f net10.0 -m:1 --no-restore -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" -p:NuGetAudit=false --filter $push # Passed: 180/180. dotnet test .\tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -c Release -f net10.0 --no-build --no-restore -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" --filter $manager # Passed: 180/180. dotnet test .\tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -c Release -f net48 --no-build --no-restore -p:CustomTestTarget=net48 -p:UseArtifactsOutput=true "-p:ArtifactsPath=$net48" --filter $manager # Passed: 192/192 after the portable PEM writer and shorter temporary PKI paths. dotnet test .\tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -f net48 -m:1 --no-restore -p:CustomTestTarget=net48 -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$net48" -p:NuGetAudit=false --filter $push # Passed: 5/5 after final test portability changes. dotnet test .\tests\Opc.Ua.Server.Tests\Opc.Ua.Server.Tests.csproj -c Release -f net10.0 -m:1 --no-restore -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" -p:NuGetAudit=false --filter 'FullyQualifiedName~CertificatePushUsesScopedOwnStoreProviderAsync|FullyQualifiedName~ApplyCertificateSlotChangeSelfCompensatesWhenIssuerImportFailsAfterAppCertSwapAsync' # Passed: 29/29; the companion project and this SDK had been built together first. dotnet test C:\src\iot-edge-opc-plc\tests\opc-plc-tests.csproj -c Release --no-build --no-restore -p:UseLocalOpcUaStack=true -p:OpcUaSourceRoot=C:\src\UA-.NETStandard -p:UseArtifactsOutput=true "-p:ArtifactsPath=$sdk" --filter 'FullyQualifiedName~GdsPush|FullyQualifiedName~PlcPendingCertificateKeyStoreTests|FullyQualifiedName~PreviewCompatibilityTests' ``` The `--no-restore` / `--no-build` invocations require prior restoration/build. The initially empty artifact-directory `--no-restore` invocation produced no test execution and is **not** counted. Initial compile/test portability failures were fixed before the final results above. Full-solution attempts did **not** pass: ```powershell # Failed/incomplete: AMB server startup timed out in two tests; stopped after failures. dotnet test .\UA.slnx -c Release -f net48 -m:1 -p:CustomTestTarget=net48 -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$net48" -p:NuGetAudit=false # Failed: NU1301 restoring OpenUsd-related projects due to nuget.org TLS handshake failure. dotnet test .\UA.slnx -c Release -f net10.0 -m:1 -p:CustomTestTarget=net10.0 -p:GeneratePackageOnBuild=false -p:UseArtifactsOutput=true "-p:ArtifactsPath=$(Join-Path $files 'solution-net10')" -p:NuGetAudit=false ``` A retry through an already configured local package source progressed further, but the OneFuzz nested publish did not inherit the audit override and caused **210 setup failures (NU1900)**; the broad run was stopped. These runs are not a clean full-suite result. AMB startup failures have not been baseline-qualified and are not claimed to be fixed by this PR. `git diff --check` passed. A changed-file whitespace check reported seven pre-existing diagnostics at lines 371-424 of `CertificateManagerTests.cs`, outside this change (new tests start at line 563); no unrelated formatting sweep was made. **Remaining gates:** successful full solution on net48/net10.0, quantitative coverage, Linux, required CI/CodeQL, and applicable CTT/interoperability qualification. Keep this PR draft until the required gates are resolved; the tests above do not establish 100% OPC UA compliance. ## Related Issues _Reference all GitHub issues this PR addresses. If there is no issue yet, open one and link it here._ _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ - Fixes #4725. - Follow-up to #4546, #4530 and #4531 for remaining certificate-push consumers. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Tests were added and pass as recorded, but coverage was not measured. CLA status is not asserted on behalf of the contributor. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
09ca425b3a |
Make the encoder stack-depth tests hold on a reused thread stack (#4722)
# Description `EncoderStackDepthTests.DecodeWithUnboundedNestingLimitFailsBeforeTheStackOverflows(Json, VariantArray)` and `(Json, DataValue)` fail intermittently on Linux CI, for example in run 37921301898, attempt 1, `test-linux-net10.0 (12/15)`. The 450-level JSON input decodes without tripping the stack check, so the expected `BadEncodingLimitsExceeded` is missing. A re-run passes. Cause: a thread that asks for a 256 KB stack can run on more. glibc reuses the cached stack of an exited thread when it is at most four times the requested size, and the fixture's own 1 MB threads leave such stacks behind. `RuntimeHelpers.EnsureSufficientExecutionStack` measures the stack the thread actually got, and 450 JSON levels fit a 1 MB stack. The product's stack check is correct; the test relied on the requested size. Deepest nesting that fits before the check fires (net10.0, Release): - On a 256 KB thread: about 100 JSON levels on Linux and 70 on Windows. - On a 1 MB thread, Linux: 548 levels for a JSON Variant array, and up to 2477 for a JSON EncodeableTree. Encoding a Binary EncodeableTree fits 1899 levels, so the 2000 levels of the other codecs had no margin either. Fix: - The unbounded-limit decode and encode tests run on a 1 MB and a 256 KB stack: the largest stack glibc can hand a 256 KB request, and the requested size. - Every codec nests 5000 levels, twice the deepest nesting that fits a 1 MB stack. - The JsonEncoder cannot nest deeper than 1000 JSON levels, so the JSON input is written by hand. `NestedJsonMatchesTheJsonEncoder` checks it against the encoder's output for every shape. ### Validation | Command | Result | | --- | --- | | Original test logic in the `mcr.microsoft.com/dotnet/sdk:10.0` container, each decode right after a 1 MB thread exits, 20 times per shape | Decode succeeded 20 of 20 times for VariantArray and DataValue, the CI failure; 0 of 20 without the 1 MB thread, and 0 of 20 on Windows either way | | `dotnet test tests/Opc.Ua.Types.Tests -f net10.0 --filter FullyQualifiedName~EncoderStackDepthTests`, Release and Debug on Windows, 2 runs each | 80 passed | | Same in the Linux container (Release), 3 runs | 80 passed | | Same with `-f net48` | 80 passed | | Linux, depth temporarily set back to 2000 | `EncodeWithUnboundedNestingLimitFailsBeforeTheStackOverflows(Binary,EncodeableTree)` fails on the 1 MB stack | | Linux, `EncodingLimits.EnsureSufficientStack` temporarily disabled | The test host crashes with a stack overflow, so the tests still catch a missing check | ## Related Issues - No tracking issue. Seen on the CI of #4710. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
0abe194b25 |
Dispose the certificate leaked by PushComplianceTests (#4721)
# Description `dotnet test tests/Opc.Ua.Gds.Tests` exits 1 even when every test passes. The assembly-level `LeakDetectionSetup` teardown reports `Certificate leak detected: 1 instance(s) created but not disposed`. CI accepts the exit code as an "at-exit stall" and only warns: the last 10 master runs show the warning for `Opc.Ua.Gds.Tests` on all four legs (net48, and net10.0 on Windows, Linux, and macOS). Cause: with `OPCUA_CERTIFICATE_LEAK_TRACKING=1`, the leak report's allocation stack names `PushComplianceTests.CreateSelfSigned()`, called inline in `TrustListTransactionLifecycleAsync`. The certificate passed to `AddCertificateAsync` was never disposed. Fix: the test creates the certificate with a `using` declaration, like every other certificate in the fixture. This is a test-only change. ### Validation | Command | Result | | --- | --- | | `dotnet test tests/Opc.Ua.Gds.Tests -f net10.0 --logger trx` with `OPCUA_CERTIFICATE_LEAK_TRACKING=1`, before the fix | 1258 passed, 0 failed, but exit 1: `TearDown failed for test fixture Opc.Ua.Gds.Tests.LeakDetectionSetup`, `Certificate leak detected: 1 instance(s) created but not disposed (created=70028, disposed=70027)`, allocated in `PushComplianceTests.CreateSelfSigned()` | | `dotnet test tests/Opc.Ua.Gds.Tests -f net10.0 --filter "FullyQualifiedName~PushComplianceTests.TrustListTransactionLifecycleAsync"`, before the fix | 1 passed, exit 1 with the same leak report | | Same, after the fix | 1 passed, exit 0 | | `dotnet test tests/Opc.Ua.Gds.Tests -f net10.0` with leak tracking, after the fix | 1258 passed, 0 failed, 56 skipped; exit 0 | Not run locally on net48: on this machine the fixture's `OneTimeSetUp` fails on net48 with `BadSecurityChecksFailed` (the server cannot verify the push client's OpenSecureChannel request) before any test body runs, and the `PushTest` fixture logs the same error. CI's net48 leg runs the fixture. ## Related Issues - No tracking issue. Found while surveying the CI "at-exit stall" tolerance; #4719 fixes the same class of leak in `Opc.Ua.Server.Tests`. A follow-up makes CI fail on such teardown failures. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
7f14fb864f |
Secure the UADP DataSetMetaData announcement on secured connections (#4712)
# Description `MetaDataPublisher` encoded the DataSetMetaData announcement itself and called `IPubSubTransport.SendAsync` directly. That bypassed the connection's security wrapper and its fail-closed check. A WriterGroup configured for `SignAndEncrypt` therefore announced its DataSet layout (field names, types, version) unsigned and unencrypted. A secured subscriber dropped the announcement (`Dropping unsecured inbound frame ... requiring SignAndEncrypt`, documented as an expected warning in docs/PubSub.md), and any listener could read it. OPC 10000-14 §6.2.5.2 applies the WriterGroup's SecurityMode to the NetworkMessages it publishes, and §7.2.4.6.3 carries the security of a discovery announcement in its SecurityHeader. Confirmed on the wire: the documented `udp-uadp` exchange sent secured DataSetMessages plus one DiscoveryResponse (announcement type 2, DataSetMetaData) with ExtendedFlags1 `0x81`, meaning the SecurityHeader bit was clear. Review feedback widened the scope to every UADP discovery message of a secured connection, and to the diagnostics of secured messages. Fix: - `MetaDataPublisher` builds the announcement and hands it to the connection (`PubSubConnection.SendMetaDataAnnouncementAsync`). - One encode-and-secure step (`PubSubConnection.EncodeOutboundAsync`) now serves data messages, discovery requests and responses, and the announcements that UDP and Ethernet send to their transport-specific discovery destination (ApplicationInformation, PublisherEndpoints, PubSubConnection). On a connection configured for message security, it applies the SecurityMode (`Sign`, or `SignAndEncrypt`) to UADP messages and refuses to send any message it cannot secure. - Before, a secured connection dropped its own discovery requests, so a secured subscriber could not ask for metadata, writer configuration, or endpoints. It also refused metadata and writer-configuration probe responses, and sent the transport-specific announcements in plaintext. - A discovery response or announcement that cannot be secured or sent is logged (`DiscoveryMessageNotSent`, Warning) and dropped. Securing announcements makes them depend on a current key; without this, a key provider with no key yet, for example before a Security Key Service delivers the first key, made `StartAsync` fail, and a probe response that could not be secured ended the receive loop. Requests sent through `RequestDiscoveryAsync` still throw to the caller. - Secured messages are counted in `SentNetworkMessages` and `SentDataSetMessages` like unsecured ones. Before, they bypassed the encoder that counts them, so secured data messages were never counted. - `UadpDiscoveryCoder` can encode a discovery message with the SecurityHeader bit set, and reports the security boundary after the PublisherId (and DataSetClassId), which is where `UadpDecoder.TryReadOuterPrefix` ends the prefix. `UadpEncoder.EncodeWithSecurityBoundary` accepts discovery messages. - The receive path is unchanged: it already unwraps secured discovery frames and registers authenticated metadata. - Not changed: the MQTT last-will status message is registered with the broker when the connection opens and published later, so it cannot carry a current SecurityHeader; it stays unsecured. - docs/PubSub.md no longer lists the unsecured-frame warning. It describes how the SecurityMode applies to announcements and discovery (`Sign` authenticates; `SignAndEncrypt` also encrypts) and the last-will exception. Tests: - `MetaDataPublisherTests.UadpPathSecuresTheAnnouncementOfASecuredWriterGroupAsync`: the announcement of a SignAndEncrypt WriterGroup carries a SecurityHeader, does not contain the DataSet name in cleartext, and unwraps and decodes with the subscriber's keys. Before the fix, the announcement had no SecurityHeader. - `MetaDataPublisherTests.SecuredNetworkMessagesAreCountedAsSentAsync`: before the fix, `SentNetworkMessages` read 0 on a secured connection. - `PubSubDiscoveryTests.SecuredSubscriberDiscoversSecuredPublisherWithoutPlaintextFramesAsync`: a secured subscriber and a secured publisher, connected through an in-memory transport that also sends announcements to a separate destination, discover metadata, writer configuration, and endpoints, and no frame lacks a SecurityHeader. Before the fix, the metadata request was dropped. With only the announcement change reverted, the PublisherEndpoints response was dropped. - `PubSubDiscoveryTests.StartupAnnouncementsWithoutACurrentKeyDoNotFailTheStartAsync` and `ReceiveLoopSurvivesADiscoveryResponseThatCannotBeSecuredAsync`: both failed before the fail-soft change. - `UadpDiscoveryTests.SecuredDiscoveryMessagesSplitWhereTheReceiverExpects`: for requests and responses, with and without a DataSetClassId, the secured encoding differs from the unsecured one only by the SecurityHeader flag, and its boundary equals the receiver's prefix length. - `UadpDiscoveryTests.EncodeWithSecurityBoundaryRejectsAMessageItCannotSecure`. ### Validation | Command | Result | | --- | --- | | `dotnet test tests/Opc.Ua.PubSub.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 1498 passed, 1 skipped (the UDP multicast loopback discovery test, which this host skips) | | `dotnet test tests/Opc.Ua.PubSub.Tests -f net48 -p:CustomTestTarget=net48` | 1498 passed, 1 skipped | | `dotnet test` on net10.0 for `Opc.Ua.PubSub.Udp.Tests`, `.Eth.Tests`, `.Mqtt.Tests`, `.Server.Tests`, `fuzzing/Opc.Ua.PubSub.Fuzz.Tests` | 234, 55, 157, 177, 149 passed | | `dotnet test` on net10.0 for `Opc.Ua.PubSub.Kafka.Tests`, `.Adapter.Tests`, `.Diagnostics.Tests`, `Opc.Ua.Redundancy.PubSub.Tests` (on |
||
|
Marc Schier
|
60dc5ab404 |
Reset the push coordinator after each PushComplianceRegressionTests test (#4719)
# Description `dotnet test tests/Opc.Ua.Server.Tests` exits 1 even when every test passes. The assembly-level `LeakDetectionSetup` teardown reports `Certificate leak detected: 3 instance(s) created but not disposed`. Cause: `CloseAndUpdateRejectsRetainedCrlOfRemovedCaAsync` and `CloseAndUpdateAcceptsVersion3CertificateWithoutExtensionsAsync` stage TrustList updates through `CloseAndUpdate` and finish without `ApplyChanges` or `CancelChanges`. `TearDown` disposed the TrustLists but not the staged operations, so the `PushConfigurationTransactionCoordinator` kept the staged and original `CertificateCollection`s undisposed. CI did not catch it: `.github/scripts/get-test-verdict.ps1` accepts a nonzero exit when the TRX counters show no failures, and reports it only as an "at-exit stall" warning. Fix: `TearDown` calls `m_coordinator.Reset()`, which discards the staged operations and disposes their collections. This is a test-only change; in production, a session's staged operations are applied, cancelled, or discarded when the session closes. ### Validation | Command | Result | | --- | --- | | `dotnet test tests/Opc.Ua.Server.Tests -f net10.0 -p:CustomTestTarget=net10.0 --filter "FullyQualifiedName~PushComplianceRegressionTests"`, before the fix | 15 passed, but exit 1: `TearDown failed for test fixture Opc.Ua.Server.Tests.LeakDetectionSetup`, `Certificate leak detected: 3 instance(s) created but not disposed (created=49, disposed=46)` | | Same, after the fix, 3 runs | 15 passed, exit 0, no leak report | | Same with `-f net48 -p:CustomTestTarget=net48` | 15 passed, exit 0 | | `dotnet test tests/Opc.Ua.Server.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 8386 passed, 0 failed, 5 skipped; exit 0 | ## Related Issues - No tracking issue. Found while explaining the exit code 1 of a full `Opc.Ua.Server.Tests` run for #4716. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
aeeef5b420 |
Send UDP multicast on the network interface the transport joins (#4711)
# Description `UdpDatagramTransport` joined multicast groups on the resolved network interface: the connection address's `NetworkInterface`, the `PreferredNetworkInterface` option, or the first operational interface. It never set `IP_MULTICAST_IF` / `IPV6_MULTICAST_IF`, though, so multicast data and discovery announcements left through the operating system's default multicast route. On a computer with several network interfaces, a publisher configured for one network sent on another, and subscribers on the configured network missed the messages (OPC 10000-14 §7.3.2.2, `NetworkAddressDataType.NetworkInterface`). `ConfigureSocket` now sets the multicast send interface to the resolved interface for IPv4 and IPv6 sockets. That includes the separate socket that sends discovery announcements for loopback-bound connections. A failure is logged at Debug level, like the other socket options. **Behavior change:** without a configured interface, multicast now leaves through the first operational non-loopback interface, which is the interface subscribers join on, instead of the default multicast route. `docs/PubSub.md` and the XML docs of `PreferredNetworkInterface` and `UdpNetworkInterfaceResolver` describe this and recommend naming the interface on multi-homed computers. Tests (`UdpDatagramTransportMulticastInterfaceTests`): - The IPv4 and IPv6 egress options equal the resolved interface. Before the fix they were `0.0.0.0` and `0`. - A publisher and subscriber pinned to the loopback interface exchange a payload. A raw control sender tells a host without loopback multicast apart from a publisher that sends elsewhere. The test fails with the bug on Linux; Windows also delivers local copies sent on other interfaces, so there it only confirms the exchange. ### Validation | Command | Result | | --- | --- | | `dotnet test tests/Opc.Ua.PubSub.Udp.Tests -f net10.0 -p:CustomTestTarget=net10.0 --filter "FullyQualifiedName~UdpDatagramTransportMulticastInterfaceTests"`, with the fix temporarily disabled | IPv4 and IPv6 option tests failed (`Expected: <192.168.178.101> But was: <0.0.0.0>`, `Expected: 4 But was: 0`) | | The same tests from the Windows-built binaries in a Linux container (`mcr.microsoft.com/dotnet/sdk:10.0`, `dotnet test Opc.Ua.PubSub.Udp.Tests.dll`), fix disabled, 5 runs | 3 of 3 failed in every run, including the loopback exchange (the control sender arrived, the publisher's multicast did not) | | Same, with the fix, 5 runs | 3 of 3 passed in every run | | `dotnet test tests/Opc.Ua.PubSub.Udp.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 237 passed (Windows, and the same binaries in the Linux container) | | `dotnet test tests/Opc.Ua.PubSub.Udp.Tests -f net48 -p:CustomTestTarget=net48` | 232 passed | | `dotnet test tests/Opc.Ua.PubSub.Tests -f net10.0 -p:CustomTestTarget=net10.0` | 1490 passed, 1 skipped | | `dotnet build src/Opc.Ua.PubSub.Udp -c Release` (net10.0, net9.0, net8.0, net48) | 0 warnings | | The docs/PubSub.md first exchange (unicast loopback), run verbatim, with #4710 applied | 11 DataSets; the unsecured-frame warning appears once, as documented | ## Related Issues - No tracking issue. Found while verifying the PubSub first-exchange documentation (follow-up to #4555). ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc Schier
|
9600f0de9c |
Let servers start in globalization-invariant mode (#4716)
# Description
A server did not start in globalization-invariant mode
(`InvariantGlobalization`, or
`DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1`), which Native AOT and
container images often enable. When the server loaded its default status
texts, `ResourceManager` created `CultureInfo("en-US")`. In this mode
the runtime throws `CultureNotFoundException` for every culture except
the invariant one, so the server crashed in
`ResourceManager.LoadDefaultText`.
`ResourceManager` now:
- keeps the locale id of each translation table as a string, and falls
back to its language subtag to find another region of the same language;
- when the runtime cannot create the culture, parses a locale id that
matches the grammar of RFC 5646 §2.1 into the recommended casing of its
subtags. It still rejects a language without a region, and rejects a tag
that does not match the grammar, as before;
- caches the resolved locale ids, so loading the default texts creates
each culture once.
`docs/NativeAoT.md` no longer warns against invariant mode for servers.
The new tests (`ResourceManagerInvariantGlobalizationTests`) reproduce
both invariant-mode behaviors of the runtime: no cultures at all, and
cultures that carry invariant data. They also cover malformed tags, and
well-formed tags with extended language, variant, extension, and private
use subtags.
The branch includes a merge of master: #4707 moved translation lookups
to a lock-free snapshot of the tables, and the snapshot now copies the
locale-id-based tables.
### Validation
| Command | Result |
| --- | --- |
| `dotnet test tests/Opc.Ua.Server.Tests -f net10.0
-p:CustomTestTarget=net10.0 --filter
"FullyQualifiedName~ResourceManager"` | 100 passed, including the 39
tests of the new fixture. Before the fix, 10 of its first 22 tests
failed; 7 of the 17 grammar cases failed against the first version of
the parser |
| `dotnet test tests/Opc.Ua.Server.Tests -f net48
-p:CustomTestTarget=net48 --filter "FullyQualifiedName~ResourceManager"`
| 100 passed |
| `dotnet test tests/Opc.Ua.Server.Tests -f net10.0
-p:CustomTestTarget=net10.0`, on
|
||
|
Marc Schier
|
e80d0ca9ac |
Use a 4-byte KeyNonce in the reference PubSub sample (#4710)
# Description
The first PubSub exchange in
[docs/PubSub.md](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/docs/PubSub.md#run-a-first-publisher-and-subscriber)
delivers no DataSets on master or in the 2.0.0 source. The subscriber
only logs `DataSetReader Reader 1 faulted on MessageReceiveTimeout`,
while the publisher logs `UADP security wrap failed; dropping message`
with `ArgumentException: AES-CTR nonce must be exactly 12 bytes` for
every NetworkMessage.
Cause: the reference PubSub sample's demo keys use a 12-byte KeyNonce.
Since #4573, the AES-CTR counter block follows OPC 10000-14 Table 157
(`KeyNonce[4] || MessageNonce[8] || BlockCounter[4]`). With the sample's
keys, the publisher builds a 20-byte nonce, and `AesCtrTransform`
rejects it. The 2.0.0 release (release/2.0 at
|
||
|
romanett
|
36194f6a02 |
Avoid boxing and per-value copies on encoder, enum and server hot paths (#4715)
## Summary Allocation audit of the per-message / per-value paths (binary codec, built-in types, server request path, monitored items, client notification dispatch) looking for boxing and avoidable per-value copies. Measured with `GC.GetAllocatedBytesForCurrentThread` probe loops on net10.0 **and net48**, and with `dotnet-trace` allocation traces of the server load harness (#4707) for read/write/browse/subscription, server and client side — a value type in the "allocated types" list is a box. Wire output and error behaviour are unchanged. ### Fixed | Where | Cause | Frequency | Before → after | |---|---|---|---| | `Polyfills/System.IO.cs` (net48/netstandard2.0) | `BinaryWriter.Write(ReadOnlySpan<byte>)` polyfill copied with `ToArray()` — every string, array, ByteString, Guid write | per value | 32–64 B → 0; BrowseResponse×50 encode 15.3 → 10.9 KB | | `EnumHelper` (net48/netstandard2.0) | `EnumToInt32`/`Int32ToEnum`/`EnumToInt64` boxed outside `NET8_0_OR_GREATER` (every enum field, `Variant.From<TEnum>`, `EnumValue.From`) | per value | 24 B → 0 | | `BinaryDecoder.ReadString` + stream raw reads (net48) | temp `byte[]` per string, `BinaryReader.ReadBytes` | per value | BrowseResponse×50 decode 15.7 → 11.3 KB | | `WriteGuid` / `ReadGuid` (all) | `ToByteArray()` / `byte[16]` | per value (also Guid NodeIds) | 40 B → 0 each | | `VariantHelper.TryCastTo<TEnum>` (`DataValue.GetValue<TEnum>`), `EventRecordFieldReaders.GetEnum` | `Enum.ToObject` box | per call / per event field | 24 B → 0 | | `NodeManagerServiceDispatcher.GetManagerHandleAsync` | interface enumerator per validated node | per node | Write-100: 151.8 → 148.5 KB/request | ### Cold paths (third commit) - `Server.Auditing` write handler: `Convert.ToBoolean(object)` got a boxed `Variant` (not `IConvertible`) and threw `InvalidCastException` on any in-process write; now takes the boolean, `BadTypeMismatch` otherwise (new test). - `UpdateServerServiceLevel`: `Convert.ToByte(byte, IFormatProvider)` bound to the object overload. - Client publish-state / recovery-policy `HasFlag` (boxes on net48) → `(x & F) == F`. - `SequenceReader` big-endian float on net48/netstandard2.0 used `Convert.ToSingle(int)` (numeric conversion) instead of reinterpreting the bits. - JSON enum strings: one `Enum.GetName` + `string.Concat` instead of `IsDefined` + boxing `CoreUtils.Format`; same output (new test for the `EnumValue` overload). Left as is: `SamplingGroup` start-up `Convert.ToInt32(double, provider)` (rewritten by #4707), `SecureChannelRenewed` formatting (transport files in flight on the client perf branch), `Utils.IsEqual<T>` object path for structs (cold callers; changing it would mix `CompareTo` and `Equals` semantics), Machinery `HasFlag` (startup config). ### Not changed here - `BinaryEncoder.GetNodeIdEncoding` `Convert.ToUInt32(object)` / `Convert.ToByte(enum)` (72 B per NodeId encode, largest single item) and `ChannelAsyncOperation.Complete` boxing the `int` result are handled on the client performance branch. - By design: `Variant` / `NodeId` store `ArrayOf<T>`, Guid, ByteString identifiers boxed (`Variant.Copy` re-boxes RolePermissions during permission checks). ## Tests - New `BinaryCodecAllocationTests`: round trips in buffer and stream decoder mode, truncation errors, allocation per call (< 1 B/call over 20k calls; the counter can be off by an allocation quantum when other tests GC). - `Enum.ToObject` equivalence for `TryCastTo` and `EventRecordFieldReaders.GetEnum` across all underlying types. - net10.0 (third commit): Types 11465, affected Server 87, Client subscription 642 passed; net48 Types 11458 passed, `UA.slnx` 0 warnings. - net10.0: Types 11464, Core.Encoders 2947, Server 8386 passed; Core 6197 passed, 1 unrelated (`ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`, passes alone). - net48: Types 11457, Core.Encoders 2947, EventRecordFieldReaders 25 passed; `UA.slnx` builds with 0 warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
28f1401324 |
Server: cut request-path lock contention and allocation; add load harness (#4707)
Profiling the reference server under load showed most request-path CPU spent spinning on contended locks, plus several avoidable allocations per request or per node. This PR removes them, switches the reference server to Server GC, and adds the load harness used to measure it. ## Results Paired A/B against `master` (3 alternating rounds, medians, 20 sessions, workstation GC on both sides, 24-core machine): | Scenario | Throughput | p50 | p90 | Server CPU / op | Server alloc / op | Lock contention | |---|---|---|---|---|---|---| | Read, 1 value | **+45%** (94k → 137k req/s) | −20% | −46% | −37% | −39% | −94% | | Read, 100 values | **+40%** | −26% | −17% | −24% | −25% | +130% ¹ | | Read, 100 values, SignAndEncrypt | **+48%** | −36% | −20% | −30% | −26% | +106% ¹ | | Browse (Server object) | **+33%** | −31% | −12% | −13% | −14% | −40% | | Write, 100 values | +13% | −12% | −4% | −10% | −19% | −23% | | Data-change subscriptions | ±0 ² | ≈ | ≈ | −14% | −17% | −29% | ¹ The faster server spends more time in the remaining per-node locks. ² Throughput is bounded by the writer rate in this scenario. Separately, Server GC instead of workstation GC doubles Read-100 throughput (21k → 44k req/s) and cuts p90 from 2.1 ms to 0.6 ms, because workstation GC paused the server for about half the wall time under load. The working set grows from ~210 MB to ~430 MB. ## Changes - **`ObjectPool<T>`**: lock-free slot array. `ConcurrentBag.Count` took every per-thread lock on each `Return` of the pooled response value-task source, once per request. - **`FairRequestQueue`**: - drops redundant gate entries: the fail-fast pre-checks and the stopped check read volatile fields and are re-checked under the gate; the owner count is folded into selection; - coalesces wake pulses with a pending flag, so the bounded signal channel's lock is only taken when no token is outstanding. Wake-count semantics in the tests are unchanged. - **`DefaultServerResourceIsolationProvider`**: - compares peer addresses without `GetAddressBytes()` allocations; - builds the unsecured verification context only when a classifier is configured. - **TCP listener / channel**: caches the certificate DER copies per channel, keyed on the certificate instance. `X509Certificate2.RawData` returns a fresh copy on every access, and the listener read it for every request. - **`EndpointBase`**: resolves the request `ActivitySource` once instead of `Assembly.GetCallingAssembly()` per request. - **Node managers**: - one `ServerSystemContext` copy per request (cached on the `OperationContext`) for per-node callbacks: permission metadata, node metadata, view membership, method lookup; - removes a discarded copy in `GetReferenceDescriptionAsync`; - per-request operation caches are plain `Dictionary` instead of a `ConcurrentDictionary` with 24 lock objects; - the validation cache is built in one pass; - fixes zero-capacity error lists. - **`ResourceManager`**: translation lookups read an immutable snapshot that is rebuilt after writes. Browse serialised on the global lock for every display name. - **`MonitoredItem`**: shares the default `DataChangeFilter` instead of allocating one per sampled value. - **Reference server**: `ServerGarbageCollection=true`; new "Garbage collector mode" section in `docs/ServerScalability.md`. - **`perf/`**: `ServerLoadHarness` (out-of-process server plus load driver, server-side CPU/GC/alloc/contention deltas, optional `dotnet-trace` capture), `TraceAnalyzer` (TraceEvent-based CPU/allocation/contention aggregation), and paired A/B scripts. Not part of `UA.slnx`. ### Behaviour notes for custom node managers - The `IDictionary<NodeId, NodeState>` operation cache handed to overrides is now a `Dictionary`. It was a `ConcurrentDictionary`, but only one request flow uses it. - Within one request, per-node callbacks of a node manager share one `ServerSystemContext` copy. ### Later commits on this PR - **Permission checks without `NodeMetadata`** (`9068684c3`): the role-permission and access-restriction validators run on an internal `PermissionMetadata` struct; `AsyncCustomNodeManager` fills it on the stack from the same attributes and namespace defaults. Browse's reference filter reads 3 attributes per target instead of 13. Read 100 values: 120.6 → 95.7 KB/request, ~21k → 26k req/s; Browse 67.6 → 51.8 KB, 26.7k → 29.8k req/s. - **Read-only channel evidence** (`936b28d76`, `dc638ab9f`): `SecureChannelContext.ClientChannelCertificate`, `ServerChannelCertificate` and `ChannelThumbprint` are `ByteString`, so the cached per-channel certificate buffer is shared without a copy and cannot be altered by handlers or classifiers. **Public API change**, see the new section in `docs/MigrationGuide.md`. The isolation provider's revalidation snapshot still copies. - **Session lookup** (`936b28d76`): `SessionManager.TryGetSessionContext` takes the candidate snapshot without the bindings lock; the validating check under the lock is unchanged (2 instead of 4 global lock acquisitions per request). - Unit tests for `ObjectPool`, the per-request system context cache, the translation snapshot, `SecureChannelContext` null semantics and a stopped `FairRequestQueue`; `perf/` follows the repository conventions. ## Testing - `Opc.Ua.Server.Tests` (net10.0): 8386 passed before the `ByteString` change. After it, 13 secure-session and isolation tests failed; both causes are fixed in `dc638ab9f` and those tests (25 with their variants) pass. A full rerun of the Server, Sessions, Core, Redundancy, LDS and GDS suites on the final commit is in progress. - Full solution builds for net10.0 and net48; `dotnet format whitespace` clean. ### o6-automation opcua-benchmarks (sampling, deep pipelines) Replicated both tests of the [o6-automation benchmark](https://github.com/o6-automation/opcua-benchmarks) in the harness (`--scenario counters`, and `read` with 10 clients × N outstanding), paired against `origin/master`: | Test | master | this PR | |---|---|---| | Subscription sampling, 100 Hz, every item ≥ 99 % of samples, 1 item | FAIL (75–94 %) | PASS (100 %) | | same, 1024 items | FAIL (64–73 %) | PASS (99.9 %) | | same, capacity | — | between 4096 and 6144 items (~410–520k samples/s) | | Read, 10 clients × 100 outstanding, None | stalls (0–1.7k req/s) | 121–131k req/s | | Read, 10 clients × 100 outstanding, SignAndEncrypt | stalls (0 req/s) | 89–91k req/s | | Read, 10 clients × 1000 outstanding | channels faulted (`BadTcpNotEnoughResources`) | 0 errors, ~45–54k req/s | - **Fixed-rate sampling** (`30b769d13`): the sampling loop waited one interval after each sample, adding timer overshoot and sampling time to every cycle; samples now have absolute deadlines. - **Async response send** (`cdac5c5ca`): `SendResponse` blocked thread-pool workers on the channel gate; with many requests in flight the receive loop that was handed the gate waited for a pool thread and the channel stalled. The TCP, Kestrel TCP and WebSocket listeners now use `SendResponseAsync`. - **Backpressure instead of channel fault** (`191cd2ba7`): server channels stop reading while more than 100 responses are queued and resume at 50, instead of faulting the channel. Client channels keep reading. At 10,000 outstanding requests throughput is ~50k req/s with or without the limit engaged, so the remaining gap to the 100-per-client case is not the backpressure. ## Not changed - `SentMessageQueue` deep-clones every `NotificationMessage`. Retained messages return their pooled payloads on acknowledgement while the response copy may still be queued for send, and a client can acknowledge a sequence number before receiving it; the clone keeps a recycled payload from reaching another session's response. Removing it needs a send-completion hook from the transport. - Generated responses allocate a default `ResponseHeader` that is replaced (~2-3 % of per-request allocation). Fixing it means making structure fields of every generated type lazily initialized, a generator-wide change to default values, equality and encoding. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
1fb1134974 |
Tests: scope MeterListeners to the meter under test (fix flaky ClientBaseTests metrics test) (#4708)
## Problem `ClientBaseTests.ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly` failed intermittently in full-suite runs: ``` Expected: "opc.ua.client.request.duration" But was: "opc.ua.channel.reconnect.duration" ``` `TelemetryContextBase.CreateMeter()` returns a new `Meter` per call, named after the calling assembly, so `ClientBase`'s meter and `ClientChannelManagerMetrics`' meter are both named `Opc.Ua.Core`. `TestMeterListener` enabled every instrument whose `Meter.Name` matched, so reconnect-duration measurements from `[Parallelizable]` channel manager fixtures were recorded and could precede the request duration. ## Fix (test-only) - `ClientBaseTests.TestMeterListener`: enable instruments only when `ReferenceEquals(instrument.Meter, meter)`, i.e. the meter instance the client under test created. Assertions unchanged. - `CryptoProviderModelTests.AuditorPublishesAProviderMetric`: same flaw — it enabled every `opc.ua.crypto*` instrument process-wide (fixture is `ParallelScope.All`), so another auditor's uncertified gauge could overwrite the observed count. Now scoped to the auditor's own meter (read via reflection, as `ClientBaseTests` already does for `ClientBase`). Other `MeterListener` users reviewed and left as-is: `NodeCacheTests` / `AdapterMetricsTests` are non-parallel fixtures; channel manager listeners only assert existence and filter by endpoint tag. ## Verification - `ClientBaseTests` + `CryptoProviderModelTests`: 4× consecutive, 25/25. - `FullyQualifiedName~Opc.Ua.Core.Tests.Stack` (net10.0): 2 clean passes, 3548 passed / 80 skipped. One attempt hung in unrelated `ClientChannelManagerManagedTests` async tests (`ReconnectAsyncNotifiesAttachedParticipantsAsync` / `ReleasingOneLeaseKeepsChannelAliveForOtherAsync`) and was rerun — a separate intermittent hang. - net48 build of `Opc.Ua.Core.Tests`: 0 warnings, 0 errors; `dotnet format whitespace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
7fe7794a19 |
Enable AES-GCM and ChaCha20-Poly1305 policies on net48 via BouncyCastle (stacked on #4703) (#4705)
> Follow-up to #4703 (now merged to master; this PR has been retargeted
to `master`).
Part of #4510. #4703 enabled the CBC ECC policies on net48 through a
BouncyCastle raw-ECDH polyfill. The AEAD policies were still unavailable
there because .NET Framework has no `AesGcm` or `ChaCha20Poly1305`. This
PR adds those ciphers the same way, so on net48 all of these are now
offered:
`ECC_{nistP256,nistP384,brainpoolP256r1,brainpoolP384r1}_AesGcm` /
`_ChaChaPoly`, `RSA_DH_AesGcm`, `RSA_DH_ChaChaPoly`.
## Changes
- **Polyfill**
`src/Opc.Ua.Security.Certificates/Org.BouncyCastle/AeadCiphers.cs`
(internal, `#if NETFRAMEWORK`, next to `EcdhRawAgreement`): `AesGcm` and
`ChaCha20Poly1305` types with the .NET 8 BCL surface. They wrap
BouncyCastle's `GcmBlockCipher(AesEngine)` and
`Org.BouncyCastle.Crypto.Modes.ChaCha20Poly1305`.
- A tag mismatch throws `CryptographicException`, never BC's
`InvalidCipherTextException`, so
`PlatformSymmetricCryptoProvider.DecryptAuthenticated` still returns
`false` and the channel still raises a protocol error.
- GCM releases plaintext blocks before it checks the tag, so plaintext
stays in a scratch buffer and is copied out only after the tag verifies.
On failure the destination is cleared, as the BCL does. Scratch buffers
and the key copy are zeroed.
- **Opc.Ua.Core**: aliases the polyfill types on `NETFRAMEWORK` and
removes the `NET8_0_OR_GREATER` gates in `SecurityPolicies`
(`SupportsAesGcmPolicy` / `SupportsChaCha20Poly1305Policy`),
`CryptoUtils` (`SymmetricEncryptAndSign` / `SymmetricDecryptAndVerify`,
AEAD helpers) and `PlatformSymmetricCryptoProvider`. The channel, the
provider facet and the encrypted ECC user tokens (`EncryptedSecret`,
through `CryptoUtils`) now run the same code on every target. .NET 8+
keeps the BCL ciphers.
- **FIPS** (requested by the #4703 session, after
|
||
|
romanett
|
2ba58a8cbf |
WoT: move GeneratesEvent off Object instances (OPC 10000-3 §7.15) (#4706)
Fixes #4669. OPC 10000-3 §7.15 says the SourceNode of `GeneratesEvent` "shall be an ObjectType, a VariableType or a Method". CTT `Base Info Core Structure 2 001.js` flags every Object that is the source of one. #4667 fixed this in the source generator. This PR fixes the three remaining WoT places. ### 1. WotCon asset registry - `AssetRegistry.BuildEventNode` / `RemoveEventTypes` no longer add or remove `GeneratesEvent` from the asset Object. Event delivery already relies on `EventNotifier = SubscribeToEvents` and root-notifier registration. - Tests now assert the reference is **absent** and the notifier bit is set. The re-apply test asserts that the old EventType node is gone. ### 2. WoT NodeSet converter (`src/Opc.Ua.Types/Wot`) - **TD → NodeSet:** a last pass (`RelocateInstanceGeneratesEvent`) moves forward `GeneratesEvent` references off an Object/Variable root: - An event the bound type already declares needs nothing more. Matching is by qualified BrowseName against the local context's declarations, the same rule as the declaration merge. - Any other event is declared by a synthesized non-abstract `<rootLocal>Type`. It derives from the bound type (or from `BaseObjectType` / `BaseDataVariableType` when the TD binds none) and becomes the instance's `HasTypeDefinition`. A Variable root gets a `UAVariableType` with the instance's DataType and ValueRank. - Thing Models are unchanged: the projected type keeps its `GeneratesEvent`. - **NodeSet → TD:** - An instance's event affordances are now collected from its type definition and that type's supertypes. Instance-level `GeneratesEvent` is still read for third-party NodeSets. - In a single readable document, a type that only carries one instance's events (only `HasSubtype` + `GeneratesEvent`, typing exactly one instance) is not chosen as root. It is folded back: its supertype becomes the type binding, and no link is written when that supertype is the default base type. The TD → NodeSet → TD round trip stays readable, with no `uav:nodes` fallback. - A document set gives the carrier its own Thing Model, and the instance TD binds to it. - **Archive check:** an event affordance is accepted when the root's type definition, or one of its supertypes, generates it. - `docs/WoTNodeSetConversion.md` documents the behaviour in a new "Event source" row. ### 3. Aggregation sample - `SamplePump.NodeSet2.xml`: each pump is typed by its own `PumpType` subtype, `ns=1;s=Pump1.Type` (`Pump_1Type`) and `Pump2.Type`, which declares its alarm EventTypes. - It is per pump because each pump owns its alarm EventTypes. `Pump2.Type` is cloned with the rest of Pump2. - The pumps keep `EventNotifier="1"` and no longer carry `GeneratesEvent`. - The checked-in documents are regenerated, adding `sample-pump-pump-1type.json` / `-2type.json`. The pump TDs still list the alarm event affordances. - The sample binding lookup ignores Thing Models, because the type's TM declares the same EventType as the pump TD. A declaration missing from the instance still fails as before. - Docs: `WoTConnectivity.md` (asset event affordances, §12.4.2) and `WotBindings.md`. The `GeneratesEvent` clause in the sample-limitation note was already stale, so I removed it. ### Runtime check `WotNativeOwnershipIndex` only uses `GeneratesEvent` in its alias table. Projected event routing takes the owner from `uav:componentOf` / `uav:id` and the notifier from Parent/`HasNotifier`. Nothing in `Opc.Ua.WotCon.Server` reads an instance-level `GeneratesEvent`. ### Tests (net10.0) - `Opc.Ua.Types.Tests`: 11446 passed, 1 skipped. New `WotInstanceEventSourceTests` covers object and variable roots, TMs, the TD with no events, the folded round trip, the legacy instance reference and the document set. Two bound-type cases were added to `WotDeclarationMergeBoundaryTests`. - `Opc.Ua.WotCon.Bindings.Tests`: 714 passed. - `Opc.Ua.WotCon.Tests`: 1496 passed, including the byte-exact sample regeneration tests. - net48 build of both test projects is clean, and `dotnet format whitespace` is clean. Not run: CTT `Base Info Core Structure 2 001.js` (p02-base-info) against a WotCon server. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
romanett
|
1da9fc0543 |
Redundancy.Server session mirroring: fix fast reconnect lifecycle (#4627) (#4704)
Fixes #4627 — the eight Redundancy.Server session mirroring defects (HotAndMirrored fast reconnect). ## Changes | ID | Fix | |----|-----| | RS-1 | The owning replica mirrors a liveness heartbeat (`LastContactAt`), written at most once per quarter of the session timeout while the client uses the session (`DistributedSessionOptions.LivenessInterval`, default 1 s). A restore expires the timeout after the later of the last activation and the last mirrored contact, plus the heartbeat's maximum lag. | | RS-2 | Entries record their owning replica (`OwnerId`). Only the owner mirrors and deletes the entry. A restore takes ownership with a conditional write. A stale copy on the previous replica is dropped without deleting the entry, both when its heartbeat sees the change and when the client fails back to it. `ISharedSessionStore` gains `TryReplaceAsync`, `TryRemoveAsync` and `EnumerateAsync`. Over a store without CAS (the CRDT gossip store) they fall back to compare-then-write. | | RS-3 | A restored session keeps the SessionId from `CreateSession`: `Session.RequestedId`, and `DiagnosticsNodeManager` honours a requested id when it is free. `DistributedSessionManager` mints GUID SessionIds that are unique across the replica set. Continuation points therefore stay under one owner id across repeated failovers. | | RS-4 | Sessions whose user token is encrypted with an ECC/RSA-DH EphemeralKey are flagged. Their restore is rejected up front (`EphemeralKeyUnavailable`), before anything is consumed or counted as a failed authentication. The documented option from the issue was chosen over mirroring ephemeral private keys. | | RS-5 | New `SessionManager.AdmitRestoredSessionAsync` hook: the mirrored nonce is consumed only after the restored activation has passed the signature, identity and continuity checks. If a restored session's first activation fails, the session is discarded without the virtual close, so the shared entry is untouched. Restored copies that are not admitted now also release their diagnostics node, and `CurrentSessionCount` is balanced for restored sessions. | | RS-6 | Expired entries are deleted on restore and by a periodic sweep (`SweepInterval`, default 5 min). The sweep also purges consumed-nonce markers that no live entry references and that are older than `NonceMarkerRetention` (default 1 h), via `ISingleUseNonceRegistry.PurgeAsync`. | | RS-7 | `SharedKeyValueSessionStore` and `DistributedSessionManagerFactory` now throw for an external store without an `IRecordProtector`, as the DI registration already does. Pass `NullRecordProtector.Instance` to opt out explicitly. | | RS-8 | Entries carry the original server certificate. A restored session accepts a client signature over either that certificate or its own one (`Session.OriginalServerCertificate`). | Entries are now written at `SecurityStateVersion` 5. Entries at versions 3 and 4 can still be restored, without an owner. ### API changes (Redundancy.Server) - New members on `ISharedSessionStore` and `ISingleUseNonceRegistry`. - An optional `TimeProvider` parameter on `SharedSingleUseNonceRegistry`. - New `DistributedSessionOptions` properties. - `SharedSessionEntry` v5 fields. - `SharedKeyValueSessionStore` and `DistributedSessionManagerFactory` throw for external stores without a protector. ## Tests - New unit tests for every finding: the restore/heartbeat/ownership/sweep/SessionId paths of `DistributedSessionManager`; store conditional writes, the CAS fallback and v5 round-trips; the registry purge; the factory/store guard; base `SessionRestoreTests` (admitted once, discarded on validation failure or rejected admission); and `SessionOriginalServerCertificateTests`. - New end-to-end test `SessionUsedPastItsTimeoutFailsOverWithItsSessionIdAsync`. The client uses the session for 2× its timeout, then fails over. The test checks that the SessionId is preserved and that the standby serves that SessionId and owns the entry. It also closes the stale copy and checks the entry survives. With the heartbeat disabled, the test fails as expected. - Local net10.0 runs: Redundancy.Server.Tests 883/883, Server.Tests 8347 passed / 5 skipped, Sessions.Tests 849 passed / 379 skipped (the usual baseline). The net48 build is clean. The Redundancy session tests pass on net48; the ECC-only cases are skipped there. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
d552634258 |
Enable ECC security policies on net48 via a BouncyCastle raw-ECDH polyfill (#4703)
Closes #4510. ## Summary OPC UA Part 6 feeds the **raw** ECDH shared secret into HKDF. .NET Framework only exposes `ECDiffieHellman.DeriveKeyMaterial`, which always hashes it, so the ECC SecureChannel and user-token policies were unavailable on `net48` (`SecurityPolicies.GetInfo` returned `null`). This adds a BouncyCastle polyfill for just that primitive: - **`Opc.Ua.Security.Certificates/Org.BouncyCastle/EcdhRawAgreement.cs`** (`net48` only): an internal `DeriveRawSecretAgreement(this ECDiffieHellman, ECDiffieHellmanPublicKey)` extension using `ECDHBasicAgreement`. It returns the X coordinate padded to the field size, exactly like the .NET 8 BCL method. Keys stay CNG keys; ECDSA signatures and HKDF stay on the platform — only the agreement step is managed code. The point is validated (on curve, not infinity) and mismatched curves are rejected. - Because the extension mirrors the .NET 8 signature, `Nonce.GenerateSecret` now has one code path on every target. - `SupportsRawEccSecretAgreement()` is removed; the CBC-based ECC policies (`ECC_nistP256`, `ECC_nistP384`, `ECC_brainpoolP256r1`, `ECC_brainpoolP384r1`) are offered on every target (still subject to OS curve support). - BouncyCastle was already a `net48` dependency of `Opc.Ua.Security.Certificates`; no new package. (Only `net48` is downlevel now — `netstandard2.1`/`net472` from the issue text are no longer targets.) **Not in scope:** the `_AesGcm` / `_ChaChaPoly` ECC variants (and PubSub DTLS) still need .NET 8 because .NET Framework has no AEAD ciphers — follow-up stacked on this branch. ## Docs `EccProfiles.md` (Known limitations), `migrate/2.0.x/certificates.md`, `ProductionChecklist.md`, `Profiles.md`, `WhatsNewIn2.0.md`. Calls out that on `net48` the agreement step does not run in a FIPS/validated module. ## Tests (local) | Suite | Result | |---|---| | `Opc.Ua.Core.Tests` full, net48 | 5956 passed, 0 failed | | `EccInteropTests` net48 vs 1.5.378 peer (net8.0, native raw ECDH) | 26 passed; 28 skipped (AEAD + curve25519/448) — all four CBC curves, both directions, UserName tokens, token renewal | | `Opc.Ua.Sessions.Tests` ECC filter, net48 | 52 passed, 0 failed | | Server `SessionCoverageTests`/`EphemeralNonce*`, Configuration builder tests, net48 | all pass | | Nonce/ECC/EncryptedSecret + session tests, net10.0 | all pass | New tests: symmetric agreement on all four curves; on net48 an oracle against CNG (`SHA256(raw) == DeriveKeyFromHash`); curve mismatch and off-curve rejection. Tests that asserted the old net48 limitation were updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
78e66f3473 |
Aggregates: TreatUncertainAsBad never applies to bounding values; DurationGood/Bad count Good/Bad regions only (#4702)
# Description Server-side aggregate fixes that follow three answers from the OPC Foundation on Mantis: - [11462](https://mantis.opcfoundation.org/view.php?id=11462) (~0025840): TreatUncertainAsBad does not apply to bounding values. - [11425](https://mantis.opcfoundation.org/view.php?id=11425) (~0025847): DurationGood counts only Good regions and DurationBad only Bad regions, and TreatUncertainAsBad does not apply. - [11426](https://mantis.opcfoundation.org/view.php?id=11426) (~0025852): Minimum, Maximum, the ActualTime variants and Range look at Uncertain values as if they are Good. The principle in 11425 and 11426 is that the aggregate definition wins over TreatUncertainAsBad. The CTT-side defects are filed as [11487](https://mantis.opcfoundation.org/view.php?id=11487). ### Bounding values (`AggregateCalculator`, `StartEndAggregateCalculator`) - Simple (§3.1.9) and Interpolated (§3.1.8) Bounding Values are formed from **non-Bad** raw values (`IsBoundCandidate`), whatever TreatUncertainAsBad is. - An Uncertain raw value before or after the timestamp makes the bound `UncertainDataSubNormal`. - A raw value exactly at the timestamp is the bound and keeps its own status. - Only a **Bad** value after a Simple bound reverts to stepped interpolation. - StartBound/EndBound keep an Uncertain bound instead of `Bad_NoData` (Tables 76/77). - WorstQuality2 now includes an Uncertain start bound, as §5.4.3.36 requires ("the server shall ignore the TreatUncertainAsBad for this aggregate"). ### How aggregates use the values (unchanged rule, two gaps closed) TreatUncertainAsBad still governs how an aggregate uses the values inside the interval, including an Uncertain bound. - `GetRegionsInValueSet` now also converts an Uncertain **first** region to Bad for Simple-bound aggregates. TimeAverage2/Total2 therefore omit it like any other Uncertain region (§5.4.3.7, Table 55 "Bound Uncertain"). - TimeAverage/Total (Interpolated bounds) still use an Uncertain bound and report Uncertain (Table 54). - TimeAverage2/Total2 return the PercentTime status `Bad` instead of `Bad_NoData` for an interval that has data but only omitted data (§5.4.3.2.1). - Minimum2/Maximum2 are unchanged: an Uncertain bound is a candidate only when TreatUncertainAsBad is false (§5.4.3.15). ### NumberOfTransitions (`CountAggregateCalculator`) §5.4.3.24 speaks only of "non-Bad" values, and Table 72 says "Bound Uncertain: Use as value". So: - Uncertain values are counted and can be the previous value, whatever TreatUncertainAsBad is. - The previous value is the nearest non-Bad value before the interval (the early bound). This reverses an earlier change that made NumberOfTransitions honour TreatUncertainAsBad. ### Minimum, Maximum, MinimumActualTime, MaximumActualTime, Range (`MinMaxAggregateCalculator`) Per the 11426 answer: - Every non-Bad value is a candidate, whatever TreatUncertainAsBad is. - An Uncertain extremum is returned with `UncertainDataSubNormal` plus Calculated. ActualTime keeps the timestamp of the first such value. - Otherwise the status is Good unless the interval contains Bad values (then `UncertainDataSubNormal`); it is `BadNoData` if all values are Bad. - Range is max − min over the non-Bad values. It is Uncertain if either extremum is Uncertain or Bad values exist. Before this change the server ignored an Uncertain value beyond the Good extremum and returned the Good one with an Uncertain status. Minimum2/Maximum2/Range2 are unchanged, because §5.4.3.15 ties them to TreatUncertainAsBad explicitly. ### DurationGood/DurationBad/PercentGood/PercentBad (`StatusAggregateCalculator`) - A region is Good only if its data point is Good, and Bad only if it is Bad. - Uncertain counts for neither (§5.4.3.31–34). - The first region still takes the status of the raw value at or before the start. ### Tests - New `Aggregates/BoundingValueTreatUncertainAsBadTests`: DeltaBounds, StartBound/EndBound, TimeAverage2/Total2, Interpolative, NumberOfTransitions (forward and reverse), DurationGood/Bad/PercentGood/Bad. - New Min/Max test for Bad-only and Bad-next-to-Good intervals. - Updated tests that encoded the old interpretation: - Interpolative with an Uncertain raw value at the start: 2 tests. - Duration/Percent Good/Bad: 2 tests. - NumberOfTransitions with TreatUncertainAsBad: 2 tests. - Minimum/Maximum/Range with an Uncertain extremum or only Uncertain values: 2 tests in Server.Tests, 1 in History.Tests. - `ProcessedReadUsesAdvertisedNodeAggregateDefaultsAsync`: it now probes the TreatUncertainAsBad default with Count, because PercentGood no longer depends on it. ### CTT verification CTT 1.05.513 with the patched scripts (c3674ff, which encode the 11462/11425/11426 answers), Aggregates part (p06): | Server | Errors | Warnings | Pass | NotImpl | | --- | --- | --- | --- | --- | | master |
||
|
romanett
|
9b034f8291 |
Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
cd4dbbe66e |
WebApi: apply the REST auth opt-ins to the opcua+openapi WebSocket upgrade (#4699)
# Description Follow-up to #4674 for the WebSocket side of the OpenAPI binding. - **Plain `opcua+openapi` upgrade uses the REST auth.** When `AddWebApiBasicAuth` / `AddWebApiBearerAuth` / `AddWebApiMutualTlsAuth` is configured, the upgrade runs the same default authorization policy as the REST route group (`RequireAuthorization()`), because the upgrade is dispatched by the listener after routing. Requests that don't satisfy it get the schemes' challenge (401 + `WWW-Authenticate`) or 403. Without an auth opt-in the behaviour is unchanged. - **Upstream identity on the channel.** For both `opcua+openapi` and `opcua+openapi+<accesstoken>`, the authenticated principal is mapped through `ISessionlessIdentityProvider` into `SecureChannelContext.UpstreamIdentity`, as the REST routes already do per request. - **Token lifetime.** A credential whose `exp` claim has already passed is refused at upgrade, also within the JwtBearer clock skew, and the socket is closed when the token expires. - **Refused upgrades complete their response.** Before, the admission lease aborted the connection when an upgrade was refused, so clients could see a connection reset instead of the 401/503. Refused upgrades on HTTP/1.x now carry `Connection: close` and Kestrel closes the connection after sending the response. The other WebSocket sub-protocols (`opcua+uacp`, `opcua+uajson`) and the HTTPS binary/JSON endpoint are unchanged: per Part 6 §7.4 they authenticate through the OPC UA session layer, not HTTP credentials. ## Tests New cases in `RealHttpsListenerIntegrationTests.Authentication.cs` against a real `HttpsTransportListener`: - the plain upgrade for Basic, Bearer and mTLS with no, wrong, wrong-scheme and valid credentials (status, challenge, and that nothing reaches the server callback); - `UpstreamIdentity` set for both `opcua+openapi` variants; - an expired token, in the `Authorization` header and in the sub-protocol; - the socket closing when the token expires. Locally on net10.0: `Opc.Ua.Bindings.Https.WebApi.Tests` 499/499, `Opc.Ua.Core.Tests` (Wss/OpenApi/Https) 243/243, `Opc.Ua.Sessions.Tests` (Wss/Https/WebApi) 53/53. `Opc.Ua.Bindings.Https` builds on all TFMs with 0 warnings. ## Related Issues - Follow-up to #4674 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [ ] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
b8f07843b2 |
Secure channel: never answer with an expired token; accept a just-expired one (+ labelled soak/stress CI) (#4698)
## Summary **Secure channel: a client could drop the channel when the server answered with a just-expired token.** Running the long-dormant ConnectionStability soak locally, the subscription channel shut down with `BadSecureChannelClosed`. The session then timed out and the subscription was abandoned. Root cause, against OPC 10000-4 §5.6.2.1: 1. **Server (rule 2: use the old token only "until the SecurityToken expires").** After a renew, the server keeps securing messages with the old token until the client sends something secured with the new one. That activation only happened on *receive*. A client that sends nothing, for example one whose Publish requests are all outstanding, kept getting responses secured with the old token after it expired. 2. **Client (rule 3: "accept Messages secured by an expired SecurityToken for up to 25 % of the token lifetime").** The client rejected any such response the moment the token expired, throwing `BadTcpSecureChannelUnknown`, and shut the channel down. From the failing run (token lifetime 10 s): the client renewed to token #11 at 09:59:01.686 and sent nothing on that channel afterwards. The server, still on #10, sent a Publish response at 09:59:04.389, 380 ms after #10 expired. The client rejected it and closed the channel. Nothing reconnected, the 10 s session timeout expired, and the server abandoned the subscription. Both halves date back to the 2016 initial commit; this is not a recent regression. It hits whenever the client sends nothing on the channel between renewing and the old token's expiry, so it is timing-dependent. ### Fix - `UaSCBinaryChannel.ActivateRenewedTokenIfDue()` activates a pending renewed token once the current one passes `TokenActivationPeriod` (95 %). This is the rule the receive path already applied. It is now shared, and `TcpServerChannel.SendResponse` calls it before securing a response. - `ReadSymmetricMessage` accepts responses (not requests) secured with an expired token for `TcpMessageLimits.TokenExpiryGracePeriod` (25 %) of its lifetime. ### Tests `TcpReconnectKeyAgreementTests` now runs the server channel and the peer on one shared `FakeTimeProvider`: - `ResponseSentAfterTheCurrentTokenExpiresUsesTheRenewedTokenAsync`: a response sent at 96 % of the old token's lifetime is secured with the renewed token. - `ResponseSecuredWithTheExpiredTokenIsAcceptedWithinTheGracePeriodAsync`: accepted at 120 % of the lifetime, rejected at 130 %. Both fail without the fix. All 40 tests in the fixture pass with it. ## CI: soak and stress runs on labelled PRs (and the soak that tested nothing) The bug stayed hidden because `stability-test.yml` still filtered `Opc.Ua.Client.Tests` after #3810 moved the test to `Opc.Ua.Sessions.Tests` (2026-05-28). Since then the nightly has run **zero tests** while reporting green: runs dropped from ~100 min to ~15 min. - `stability-test.yml` now targets `Opc.Ua.Sessions.Tests` and writes a TRX file. - New `.github/scripts/assert-tests-ran.ps1`, used by all three opt-in workflows, fails a run in which no test executed, or none passed. Contract-tested in `CiAssertTestsRanTests`. - Like `interop`, the opt-in workflows can now run on a labelled PR. They are not required checks. A later push cancels the run in progress, and adding an unrelated label neither starts nor cancels a run. | Label | Workflows | Run on a PR | | --- | --- | --- | | `stress` | Stress Tests (ChaosTCP) | Random seed, saved as `chaos-seed.txt` in the results artifact | | `soak` | Connection Stability Test, Sample HA Long-Haul Test | 30 min stability; 15 min per HA scenario | ## Test plan - [x] `TcpReconnectKeyAgreementTests`: 40/40 pass; the 2 new tests fail without the fix - [x] All `Opc.Ua.Core.Tests.Stack` tests (net10.0): 3548 passed, 0 failed - [x] `CiAssertTestsRanTests`: 4/4 - [x] `soak` + `stress` labelled run on this PR (before the fix): both stability tests, both HA long-haul scenarios and ChaosTCP passed on Linux (the label triggers and push-cancellation work) - [x] Local short-haul soak with the fix (Windows, 10 s tokens), 2 runs: both pass with full notification counts; run 2 hit the window once (`Token activated forced` on send) and kept the channel - [x] `soak` + `stress` labelled run with the fix: Connection Stability (2 tests), Sample HA Long-Haul and ChaosTCP all pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
e5b6d1352d |
Fix CI test flakes: WotCon end-to-end timeout and HTTPS listener port race (#4697)
Fixes two CI test failures. Both changes are test-only. ## WotCon: `UnavailableUpstreamEndpointFailsMappedReadAsync` timed out on Windows Debug This test failed the same way in two nightlies in a row (runs [37171493229](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/37171493229) and [36287479915](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/36287479915), `test-windows-net10.0-debug`, 4m06s). It threw `TaskCanceledException` instead of the expected `ServiceResultException`. The read did not hang. The cancellation came from the test's own 4-minute token, which is passed into `ReadAsync`. That budget covers the whole workflow: - Uploading and refreshing the 533 sample documents takes about 3m45s on a Windows Debug runner. - On Windows, each refused loopback connect to the unavailable upstream costs about 2s, because of SYN retries. The mapped read touches 9 SourceA nodes, and each one tries twice, which adds about 36s. Same commit on other legs: Windows Release passed at 2m35s, Linux Debug at 3m53s. This test and `BadTargetMappingFailsRefreshAsync` (3m52s on the same leg) now use the 8-minute budget the other full-workflow tests already have. The assertions are unchanged. ## HTTPS: `RealHttpsListenerIntegrationTests` failed with AddressInUse on macOS Run [37581622082](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/37581622082) failed in `test-macos-net10.0` with `Failed to bind to address https://[::]:49402: address already in use`. The failing tests were `AllBindingsForwardObservedPeerAddressAsync` (3 cases) and `AuthOptInIsEnforcedByRealHttpsListenerAsync`. `FindAvailableTcpPort()` probed `127.0.0.1:0`, closed the socket and handed the port to Kestrel, which then binds `[::]:port`. The port is in the ephemeral range, so another process could take it between the probe and the bind. Nothing retried. Changes: - The setup and the auth listener open the listener through a new `OpenListenerOnFreePortAsync` helper. On AddressInUse it disposes the listener and retries with a fresh listener and a new port, up to 5 attempts. The AddressInUse check uses the same matching as `ServerFixtureUtils.IsPortUnavailable`. - The port probe now uses `TcpListener.Create(0)`, a dual-mode IPv6 any socket, so it checks the same address Kestrel binds. ## Testing - `Opc.Ua.WotCon.Samples.Tests`, both affected tests: pass on net10.0 in Debug and Release. - `Opc.Ua.Bindings.Https.WebApi.Tests`: 483/483 pass on net10.0. The retry path was checked by forcing a taken port on the first probe. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
731600801a |
Release: stop retrying 404s in the feed gates; keep GitHub Release notes under the size limit (#4696)
# Description `Release to nuget.org` [run 37591540644](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/37591540644) was the stable 2.0.0 promotion. It spent more than an hour in **Verify already-published package identity**. It pushed every package to both feeds and created the `2.0.0` tag, then [failed](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/37591540644/job/112693955843) in **Tag published package candidate** before the GitHub Release existed. This PR fixes both problems. ## 1. Feed gates retried every 404 PowerShell 7's `Invoke-WebRequest -MaximumRetryCount` retries every status from 400 to 599, and that includes 404. `-SkipHttpErrorCheck` stops the error from being thrown but doesn't stop the retries. Both feed gates ask once per package whether an id/version is published, and 404 is the normal "not yet" answer. Each absent package therefore cost 3 retries × 5 s, about 15 s. A stable set is never pushed automatically, so for 2.0.0 every package, `.Debug` included, was absent on **both** feeds. - `package-version-policy.ps1`: adds `Invoke-FeedRequest`. It retries only network failures, 408, 429 and 5xx, and returns every other response straight away. After the last retry it returns the final response, or rethrows the network error. - `assert-published-packages-match.ps1` and `validate-preview-package-ordering.ps1` call it instead of `Invoke-WebRequest -MaximumRetryCount 3 -RetryIntervalSec 5`. The status handling in each caller is unchanged, so the gates still fail closed: anything other than 200 or 404 still blocks. - `PackageVersionPolicyTests.GetGitHubPackagesPublishedVersionPagesResultsAndFailsClosedAsync` now dot-sources the policy script so the extracted function can find `Invoke-FeedRequest`. The `Invoke-WebRequest` stub still intercepts the call. - New stub-based tests check that `Invoke-FeedRequest` makes one request for 200/400/403/404 and retries 408/429/5xx exactly `MaximumRetryCount` times. They also check that it stops once a retry is not transient, rethrows an exhausted network failure and passes `-OutFile` with `-PassThru`. | Response | Before | After | |---|---|---| | 404 | 4 requests, 15 s | 1 request, immediate (live nuget.org: 0.58 s) | | 403 | 4 requests | 1 request | | 503 / 429 | retried 3× | retried 3× (unchanged) | | connection refused | — | retried, then rethrown | ## 2. GitHub Release body too long ``` HTTP 422: Validation Failed (https://api.github.com/repos/OPCFoundation/UA-.NETStandard/releases) body is too long (maximum is 125000 characters) ``` `gh release create --generate-notes` diffs against the latest release when no previous tag is given. Here that was `1.5.378.182`, which is on another branch, so the generated notes for 2.0.0 came to 191,048 characters. The step now calls `POST /releases/generate-notes` with `previous_tag_name` from the new `Get-PreviousReleaseTag` helper in `package-version-policy.ps1`. It picks the greatest same-major semantic version strictly lower than the tag, ignoring non-semver tags like 1.5.378.x, so a servicing 2.0.1 published after 2.1.0 still diffs against 2.0.0. For 2.0.0 that is `2.0.0-preview.2`, and the notes come to 31,408 characters (measured against the real repo; that call has no side effects). If a body still exceeds 120,000 characters, it is cut at a line boundary and a compare link is appended. The tag and release checks are idempotent, and the pushes use `--skip-duplicate` and verify identity, so re-running the promotion finishes the release. **For run 37591540644 to pass:** the dispatch runs `release.yml` from `release/2.0`, so these changes have to reach `release/2.0` before the promotion is re-run. ## Verification - All 3 scripts parse with 0 errors. The modified workflow step also parses, and the YAML loads. - Live nuget.org: absent version → 404 in 0.58 s; package index → 200. httpbin 503 with `-RetryIntervalSec 1`: retried, 503 after 4.7 s. - `dotnet test tests/Opc.Ua.Tools.Tests`: 939 passed (1 skipped) on net10.0, plus net8/9/48. ## Related Issues - No issue; found investigating run 37591540644. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [ ] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Klaus Landsdorf
|
fa55157ce4 |
Server: hold anonymous session-less requests to the channel security of CreateSession (#4689)
# Description `AddSessionlessInvocation(o => o.AllowAnonymous = true)` (#4659) lets a request without an Access Token run as an anonymous user. OPC 10000-4 §6.3.1 allows that only "if application authentication through the SecureChannel is sufficient". On HTTPS the channel gave no such authentication: on an endpoint with mutual TLS and a SignAndEncrypt policy, a session-less request without a TLS client certificate and without an Access Token was answered `Good`, over the REST binding and over the binary POST with `OPCUA-SecurityPolicy: Basic256Sha256`. `CreateSession` on the same endpoint answers `Bad_SecurityChecksFailed` without the certificate, and ActivateSession on a new channel does too (OPC 10000-6 §7.4.1: applications that use HTTPS and require application authentication check the application certificates during the CreateSession/ActivateSession handshake). This PR makes a session-less request without an Access Token meet what the endpoint demands of a Session. #4659 is merged, in no release tag and opt-in, so this is a change to a new feature; the default `AllowAnonymous = false` is not affected. ## Changes ### `SessionManager.ValidateSessionlessRequestAsync` (`src/Opc.Ua.Server/Session/SessionManager.cs`) With `AllowAnonymous`, a request without an Access Token now needs: | Check | Where it applies | Status otherwise | Same as | | --- | --- | --- | --- | | The client certificate of the channel (`SecureChannelContext.ClientChannelCertificate`) | endpoints with a security mode or policy other than None, and HTTPS endpoints when `HttpsMutualTls` is set | `Bad_SecurityChecksFailed` | CreateSession on the HTTPS listener; ActivateSession on a new channel | | An Anonymous user token policy on the endpoint (an endpoint without policies does not restrict) | every endpoint | `Bad_IdentityTokenRejected` | ActivateSession with an anonymous identity token | | An endpoint on the channel | every request | `Bad_SecurityModeInsufficient` | the same request with an Access Token | - The certificate is the one the transport has validated: the SecureChannel certificate of opc.tcp and opc.wss, the TLS client certificate of the HTTPS listener (REST, binary, JSON). So opc.tcp channels with security keep working as they did, and an HTTPS client that wants an anonymous session-less call presents its application certificate in the TLS handshake. - An HTTPS listener without `HttpsMutualTls` advertises SecurityMode None without an Anonymous user token policy (`HttpsServiceHost`); an anonymous session-less request there is now rejected as an anonymous Session is. - A request **with an Access Token** is unchanged: the token authenticates the caller and HTTPS keeps it confidential (HTTPS clients are not required to present a certificate, OPC 10000-6 §7.4.1). A `ValidateSessionLessRequest` handler still takes over completely. - Not changed: the transport credential authentication of the REST routes (#4674). `SessionlessInvocationOptions.AllowAnonymous` documents the rule. `docs/SessionlessInvocation.md` has a new section "Anonymous requests and the channel" with the table. ### Tests - `tests/Opc.Ua.Sessions.Tests/SessionlessChannelSecurityTests.cs` (new, net8.0+): a real reference server with an HTTPS listener, twice (mutual TLS on with a SignAndEncrypt-only endpoint; mutual TLS off), over REST and the binary POST, with and without a TLS client certificate: anonymous without a certificate, with a certificate, `AllowAnonymous = false`, a valid and a bogus Access Token, and CreateSession on the binary POST without the certificate for comparison. 44 cases. - `tests/Opc.Ua.Server.Tests/SessionlessInvocationTests.cs`: opc.tcp with and without a channel certificate (Sign, SignAndEncrypt, None), HTTPS with and without mutual TLS and with a secured endpoint, user token policies with and without Anonymous, no endpoint, an Access Token and a handler without a certificate. The existing tests that use `AllowAnonymous` set `HttpsMutualTls = false` or give the channel an endpoint. - Fail without the change: 7 of the 9 new `SessionlessInvocationTests` cases fail with the `SessionManager.cs` change removed (the opc.tcp with security, HTTPS with and without mutual TLS, user token policy and unknown endpoint tests); the 6 failing `SessionlessChannelSecurityTests` cases are below. ## Testing - **Reproduction on master (`c03c12838`)**: `SessionlessChannelSecurityTests` without the change: 6 of 44 cases fail, 38 pass. With mutual TLS on and no TLS client certificate the anonymous request is answered `Good` instead of `Bad_SecurityChecksFailed` (REST and binary). With mutual TLS off it is answered `Good` instead of `Bad_IdentityTokenRejected` (REST and binary, with and without a certificate). All 44 pass with the change; the Access Token cases pass both ways. - Results on macOS: `Opc.Ua.Server.Tests` full run on net10.0 8197 passed / 9 skipped / 0 failed; the session-less and SessionManager filters 139 passed / 0 failed on net10.0 and on net8.0. `Opc.Ua.Sessions.Tests` with the HTTPS/WebApi/Sessionless filters 125 passed / 0 failed on net10.0 and on net8.0, and with the Wss/OpenApi/ClientTest filters 320 passed / 241 skipped / 0 failed on net10.0. - Builds: `Opc.Ua.Server`, `Opc.Ua.Server.Tests` and `Opc.Ua.Sessions.Tests` for net48, net8.0, net9.0 and net10.0: 0 warnings, 0 errors. - `dotnet format` (whitespace, style, analyzers `--severity info`, one `--include` per changed file): no finding on a changed line (CA1849 at `SessionManager.cs` 235 is on master). - Coverage (`.azurepipelines/check-coverage.ps1` against master, session-less and SessionManager tests of `Opc.Ua.Server.Tests` on net10.0): patch 36/36 changed lines (100 %). The project floor is not meaningful for this subset. - CI and CodeQL on the PR branch: [CI run 37529750287](https://github.com/biancode/UA-.NETStandard/actions/runs/37529750287) green (158/158 jobs, incl. macOS and Windows net48), [CodeQL run 37529754883](https://github.com/biancode/UA-.NETStandard/actions/runs/37529754883) green, on `03a8e3d91`. ## Related Issues - Fixes #4688 - Relates to #4659 (session-less invocation), #4657 (the REST binding's session-less service set) and #4674 (REST authentication) ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the branch CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
romanett
|
fbaabf5b28 |
Cut CI runner minutes: duration-packed test batches, CodeQL on Linux/daily, cancellable branch packing (#4694)
Reduces GitHub Actions runner minutes without removing any OS, target framework, test project or check. All figures come from master CI run 37520359929 (158 jobs, ~1,880 runner-minutes) and the last two weeks of run history. ## 1. Test batches packed by duration (`get-ci-matrix.ps1`) Previously the expander picked the **smallest** batch size that fit the 256-job limit, which meant 2 projects per job and 33 jobs per profile. Job logs show ~45% of the test matrix (674 of ~1,516 min) went to compiling. Each batch rebuilds the shared dependency graph: the first project's build took 3.8–4.9 min, the second only 0.4–0.9 min. - Projects are now packed in path order (which keeps `PubSub.*`, `Redundancy.*`, `WotCon.*` together) up to a 30-minute weight target, using `.github/ci-test-durations.json`. Each weight is the slowest PR profile's test minutes, rounded up, plus 1. - A batch never holds more projects than the job timeout can budget for: at most 7, given 20 + n×45 ≤ 350. - If the matrix would exceed the job limit, the target is raised instead. - **PR test matrix: 132 → 60 jobs** (15 per profile; macOS 33 → 15). Each batch is estimated at ≤36 min on the slowest runner, comparable to today's slowest jobs (37–43 min), so wall-clock should be about the same. Expected saving is roughly 300+ runner-min per run, plus less queueing for the scarce macOS runners. - The weekly full scope shrinks the same way (still 13 profiles). - The table only steers balance: a project missing from it gets `defaultMinutes`, so it is still scheduled. - `run-dotnet-tests.ps1` now records per-project `buildSeconds`/`testSeconds` (batch summary and job summary). The new `update-test-durations.ps1` refreshes the table from any run's `dotnet-results-*` artifacts. - `discover` output `batch_size` is renamed to `batch_target_minutes`. Nothing consumed it. New `CiMatrixScriptTests` cases: - each profile runs each of its projects exactly once; - only a lone project may exceed the target; - PR profiles average ≥3 projects per job; - the duration table names only existing projects. ## 2 + 3. CodeQL (`codeql-analysis.yml`) - **No longer runs on every push to master.** Instead, a daily schedule on master (was weekly) gives an analysis that always completes. Each analysis takes ~3 h, and with `cancel-in-progress: true` per branch, 49 of 76 push runs in two weeks were cancelled by the next merge before uploading anything. `master378`/`release/*`/`develop/*` keep their push trigger. PRs are still analyzed as before. - Only PR runs are cancelled when superseded. Push and scheduled runs always finish. - **Runs on `ubuntu-latest` instead of `windows-2025`.** Only net10.0 is built, so the extracted sources are identical. The traced Windows build alone took 70 min, against 12 min for the untraced Linux `UA.slnx` net10.0 leg. This PR touches a `.cs` file, so its own CodeQL run is the first measurement. ## 4. NuGet packing on feature branches (`nuget-publish.yml`) 298 of ~400 runs in two weeks were feature-branch pushes, each about 59 Windows-min across two signed builds. - A superseded feature-branch run is now cancelled. Protected lines and tags are never cancelled: they publish previews, and `release.yml` promotes a run by ID. Each manual (`workflow_dispatch`) run gets its own concurrency group, so a later push cannot cancel it. - A new `changes` job skips packing when a feature branch, as a whole, changes only documentation. It diffs against the merge base with master, not the previous push, so a docs-only push that cancels a code push cannot mask that code change. It uses the same `get-path-relevance.ps1` rule as CI. - Protected refs, tags, manual runs, and any branch whose merge base cannot be resolved always pack. Checked locally for docs-only branch (skip), code branch, protected ref, manual run and unknown commit (all pack). ## Verification - `CiMatrixScriptTests`, `CiPathRelevanceTests` and the new `CiTestDurationsScriptTests` (covering `update-test-durations.ps1`): all `*.Ci*` tests in `Opc.Ua.Tools.Tests` pass locally, 63/63. - This PR's own CI run shows the new matrix. Compare its total runner-minutes and longest test job against run 37520359929. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanett
|
f32259a1a7 |
GDS: pin SecurityAdmin-only delivery of ApplicationRegistrationChangedAuditEvents; document CTT defect C57 (#4693)
## Description CTT 1.05.513 (patched scripts) GDS Application Directory `011.js`, `018.js` and `028.js` fail on master ( |
||
|
romanett
|
7c2c6e7f8e |
Number commit-id-only prerelease previews 0 so they never outrank a release (#4692)
## Problem When a stable `version.json` (for example `2.0.0`) is built from a ref that isn't a public release branch, Nerdbank.GitVersioning makes the root version `2.0.0-g<commit>`. The "other prerelease label" case in `version.targets` turned that into `2.0.0-preview.g<commit>` for the preview-only package families. SemVer 2 ranks the alphanumeric `g…` identifier above every numeric one, so that version outranks every `2.0.0-preview.N` ever released, and NuGet versions are immutable. This happened on 2026-10-06 while master briefly carried `2.0.0`. Two such versions on GitHub Packages now block the stable 2.0.0 release on `release/2.0` (handled there in #4691). ## Change - `version.targets`: a new Case 2a maps a prerelease that is only a commit id to `<base>-preview.0.g<commit>`, keeping any build metadata last. Preview number 0 sorts below any committed `PreviewPackageBuildNumber`. - `package-version-policy.ps1`: `ConvertTo-PreviewPackageVersion` applies the same rule, so the package-set validator agrees with the build. - Tests: commit id only, commit id plus metadata, and `-gamma.1`, which is not a commit id. Checked with MSBuild against the real `version.targets`: `2.0.0-ge78c648295` maps to `2.0.0-preview.0.ge78c648295`. `2.0.0`, `2.0.0+g…`, `-rc.1` and `2.0.1-preview.N.g…` are unchanged. All 90 `PackageVersionPolicyTests` pass on net10.0. Same commit as the prevention part of #4691 (release/2.0). This PR doesn't include the release-only `PreviewPackageOrderingExemptions` change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |