15 Commits

Author	SHA1	Message	Date
romanett	414957c88a	Reduce ISubscription: internalize the publish pipeline, delete the dead members (#4349) ... # Description Reduces the server subscription API surface per the measurements from issue #4185. ## ISubscription: 42 → 30 members (#4185) A call-site census of the 11 "publish-pipeline protocol" members plus `Publish` (all 68 `src/` projects, samples, tools, fuzzing, tests) found **zero callers outside `Opc.Ua.Server`** and split the members three ways: - **Deleted (4):** `ItemReadyToPublish` and `ItemNotificationsAvailable` were dead — zero callers repo-wide and implementation bodies commented out since before 1.5.378 — the obsolete parameterless `SessionClosed()` had zero callers, and `TransferSessionAsync` (the one-shot direct transfer, which bypassed the reservation protecting a transfer against a concurrently closing source session) lost its last caller once the manager always uses the claim/prepare/commit protocol. - **Internalized (8):** `PublishTimerExpired`, `Acknowledge`, `PublishTimeout`, `SubscriptionTransferred`, `AvailableSequenceNumbersForRetransmission`, `QueueOverflowHandler`, `SessionClosed(ISession)` and `Publish` move to `internal interface ISubscriptionPublishPipeline : ISubscription`, implemented **explicitly** by `Subscription` (the `ISubscriptionMonitoredItemLifecycle` shape) so they are unreachable through the public concrete class too. `Publish` goes with them deliberately — it is the same hazard class (consumes notifications, advances sequence numbers) with the same single production caller. - **Kept public (1):** `ResendData` drops out of the issue's list — its only caller is the `Server_ResendData` UA-method handler in `DiagnosticsNodeManager`, resolving via `TryGetSubscription` like `GetMonitoredItems` does. It is service-facing, not pipeline. ## The pipeline contract is complete: SessionPublishQueue traffics only in it `SessionPublishQueue` becomes `internal sealed` (zero consumers outside the assembly; `SentMessageQueue` already was) and works **solely with `ISubscriptionPublishPipeline`** — no `ISubscription` and no concrete `Subscription` anywhere in the file. To get there, the internal transfer-claim protocol (`TryBeginTransfer`, `PrepareSessionTransferAsync`, `CompleteTransfer`, `AbortTransfer`, `TryRestoreSessionAfterFailedTransfer`) moved from `internal` methods on the concrete class onto the internal interface, implemented explicitly. That completes the contract, so `TransferSubscriptionsAsync` loses its two-tier shape: the ~6 `subscription as Subscription` null-guard branches and the degraded `TransferSessionAsync` fallback for non-`Subscription` implementations are gone — one code path, always the claim/prepare/commit/rollback protocol, and `ISubscription.TransferSessionAsync` is deleted with it. Subscription creation fails with `Bad_InternalError` when a `CreateSubscription` override returns a type not derived from `Subscription`, instead of publishing partially — such implementations were already second-class (transfer required the concrete type before this change). This is the headline behavior change for exotic extenders and is documented prominently in `docs/migrate/2.0.x/sessions-subscriptions.md`. The durable-subscription open question in the issue is answered **no**: `Subscription/Persistence/` never references `ISubscription` (the store deals in `IStoredSubscription` DTOs; restore constructs the concrete type via `Subscription.RestoreAsync` inside the manager), so the internalization cannot break a store implementer. ## #4186: measured, prototyped, and deliberately kept as-is Body inspection showed only 4 of the 10 `ISubscriptionManager` wrappers are pure lookup-and-delegate (`Republish`, `SetTriggering`, `ModifyMonitoredItemsAsync`, `SetMonitoringModeAsync` — one caller each); the other 6 carry logic with no `ISubscription` substitute. A collapse of the four was implemented and then reverted on this branch: it traded ten uniform id-addressed members for a mixed dispatch shape in `StandardServer` without reducing what a reader must understand. **Every subscription service keeps going through `ISubscriptionManager`**; the wrapper bodies now resolve against the pipeline-typed registry. The decision is recorded in `plans/ServerRuntimeAnalysis.md` (open question 5). ## Compatibility treatment Removals are outright, per the documented no-`[Obsolete]`-shim policy for these interfaces (`docs/MigrationGuide.md` — re-adding an interface member would break every implementer; net48/netstandard2.0 have no default interface methods). In its place: - Migration analyzer **UA0030** (pipeline members + `SessionPublishQueue`, distinguishing deleted no-ops from internalized protocol), with tests, following the UA0024 syntax-based pattern. The transfer-claim members were never public in 1.5.378, so they need no migration story. - `docs/MigrationGuide.md#ua0030` section with before/after code; the `docs/migrate/2.0.x/sessions-subscriptions.md` section that told implementers to *add* `SessionClosed(ISession)` is replaced with the derive-from-`Subscription` story; the 2.0.x README symptom table routes the rule. - All 11 removed/internalized members existed in 1.5.378 (verified against `master378`), so each is documented as a 2.0 break. ## Tests Converted, not deleted. `Mock<ISubscription>` setups of pipeline members become `Mock<ISubscriptionPublishPipeline>` — Moq proxies the internal interface via a new `InternalsVisibleTo("DynamicProxyGenAssembly2")` (keyless; the SDK appends `$(PublicKey)` under `SignAssembly`, and signed pipelines do not run tests, matching the existing `Opc.Ua.Redundancy.Kubernetes` precedent). A test-side extension shim forwards the original member names (pipeline + transfer protocol) so the direct call sites on concrete `Subscription` instances compile unchanged. The transfer-race tests' typed reflection probes into `m_abandonedSubscriptions` follow the field's new type. - `Opc.Ua.Server.Tests` (net10.0): 4795 tests, 0 failures - `Opc.Ua.Subscriptions.Tests`, `Opc.Ua.Subscriptions.Durable.Tests`, `Opc.Ua.Subscriptions.Classic.Tests` (net10.0): green - `Opc.Ua.MigrationAnalyzer.Tests`: green incl. the new UA0030 tests - net48 gate: solution build green; subscription-area fixtures green (the known pre-existing local hosting-test environment failures reproduce identically on unmodified master) ## Related Issues - Fixes #4185 - #4186 was measured and prototyped on this branch and deliberately **not** taken (see the section above); the decision is recorded in `plans/ServerRuntimeAnalysis.md`. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. _(net10 suites fully green locally; net48 build green and subscription-area fixtures green — the remaining net48 hosting-test failures on my machine reproduce identically on unmodified master.)_ - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-08-31 10:53:51 +00:00
Marc Schier	cef83550cb	OPC UA Vision: a vision-guided bin-picking demo an LLM agent can see and drive (#4235) ... # Description Implements **OPC UA — Vision** and uses it to build the demo the companion specifications were drafted for: an eye-in-hand camera watches a bin of parts, a language model *looks at the frame*, decides which part matches the instruction, and commands the robot through Robot Intent — with the OpenUSD viewer showing the arm move and the bin empty out. > **Draft, and stacked.** [#4195](https://github.com/OPCFoundation/UA-.NETStandard/pull/4195) has merged and master is merged in, so what is left on top of `master` is the Vision work plus the unmerged `marcschier/ai-model-management` branch, which the demo needs for the model/dataset/deployment provenance. Review the Vision commits; the AI Model Management work lands with its own PR. ## What is here **The specification, implemented.** `Opc.Ua.Vision` is generated from the spec's own `Opc.Ua.Vision.NodeSet2.xml`, vendored byte-identically (SHA-256 `79C592C6…`, verified against a fresh download and stored LF-preserved like the Robot Intent NodeSet). `Opc.Ua.Vision.Server` builds the §4.2 discovery object, sensors, the coordinate-frame tree, calibrations, media endpoints, inference pipelines and the §9 feedback methods, with rendering and inference behind injectable providers. `Opc.Ua.Vision.Client` is built on the generated ObjectType proxies and composes transforms across the frame tree, which is what turns a detection in camera coordinates into a pose a robot can act on. **Two perception paths behind one contract**, which is exactly what `InferenceLocationEnum` is for. `OnServer` is a deterministic detector derived from the stage's ground truth — no model, no network, no GPU, so CI can exercise the whole loop. `EdgeOffServer` is the agent: it sees the frame over MCP and calls `SubmitDetections`, and the Server publishes results it did not compute. A client reads `DetectionResultType` identically either way. **The agent's eyes.** `tools/Opc.Ua.Mcp.Vision` adds 22 `vision_*` tools, which with the four shared connection tools makes the `vision` profile 26; `vision_get_frame` returns the camera image as an MCP `ImageContentBlock`, so the model looks at pixels rather than reading a description of them. Tool profiles now compose — `--profile vision,robotics` yields 62 tools — so one agent can both see and act without dragging in the full 136-tool catalogue. **The cell.** `samples/Robotics/BinPickingCell` follows the spec's own Robotics-Vision Addendum: the `world → robot_base → flange → gripper_tcp` frame tree with `camera_eih` on the flange, the `HandEye` extrinsics and the `Intrinsics612x512` intrinsics — the addendum's 2448×2048 calibration scaled to the 4×4-binned grid the camera actually delivers. `samples/Robotics/BinPickingClient` hosts the composed MCP catalogue and optionally the viewer. **Documentation.** `docs/Vision.md` (~800 lines) covers the model, hosting, the build context, every topology builder, both perception paths, feedback, the MCP profile and the limitations; linked from `docs/README.md`, with the profile table in `docs/McpServer.md` and READMEs for both samples. ## Evidence, not assertions Each of these was measured on a running system rather than assumed: | Claim | Evidence | |---|---| | The camera sees the bin | 612×512 frame with the bin and all five parts in view — red `(212,47,47)`, green `(61,193,71)`, blue `(54,85,218)`, yellow `(215,196,58)`, orange `(209,119,55)` | | An agent can act on what it sees | Over MCP, `vision_get_frame` returns a 612×512 PNG that decodes from base64 to 1,254,051 bytes, the descriptor beside the detections says 612×512, and all five `BoundingBox2D` centres land **inside** the frame on the pixel colour their own detection claims | | The geometry is right | A detection composed `camera_eih → world` lands on the authored position, residual **0.0000 m** — on both the on-server and off-server paths | | The detector tracks the world | Placing a part drops the next scan from 5 detections to 4, with that part absent | | The robot obeys the agent | Scripted run took command authority and completed a pick and a place, both `Succeeded` | | Tool catalogues are honest | Counted from a running server: vision 26, robotics 40, `vision,robotics` 62, full 136, **zero duplicates** | | Every node is browsable by NodeId | Eight representative NodeIds — both folders, a pipeline, a frame, the sensor, its Calibrations folder, the HandEye calibration, the Media folder — all `Good` | That tool-count check matters because the profile tests collapse names into a `HashSet` and *cannot see* a duplicate registration — so the new tests count registrations, not names. ## Things that were wrong, and are not now Writing the tests found three defects of one kind — code that answered instead of refusing. A zero-norm quaternion was silently rewritten to identity, so a caller with an all-zero orientation got a confident wrong grasp pose. `SubmitCorrection` rewrote a null `ResultId` to empty and called the sink anyway. A bad `StageIdentifier` **killed the host process** inside native `UsdStage.Open`, via an exception .NET cannot catch. All three now refuse loudly and are pinned by tests. **Nodes that existed but could not be reached.** The node manager indexes the Vision root before configurators run, so everything the fluent builder grafted on afterwards — both folders, the pipelines with their Results and Feedback, the frames, the sensors with their optics, calibrations and media endpoints — was never added to the index. Browsing *forward from a parent* worked, because that walks `NodeState.Children` in memory; browsing any of those nodes *by its own NodeId* returned `BadNodeIdUnknown`, which is how an ordinary client and the MCP discovery tools navigate. Registration is now deferred and flushed after each configurator. The first fix covered only the DI path and left the documented non-DI fallback broken in exactly the same way, so `ConfigureVisionAsync` now exists and registers what it builds. That defect survived a green suite because every test browsed children. The new tests assert by NodeId, and were checked against the broken code — with the flush removed, they fail. **Every Method was uncallable.** The Server passed 426 unit tests while not one of its Methods could be invoked over a real session — `RunInference` returned `Bad_TooManyArguments`, and so did every other Method that takes arguments, which is all of them but `StartContinuous` and `Stop`. Three independent faults, each sufficient on its own: the Method nodes were never created (they are Optional children, and the dispatcher guards every attachment with a null check, so it silently attached nothing); nothing declared the arguments, so the stack concluded none were expected; and `MethodDeclarationId` pointed at a synthesised NodeId rather than the NodeSet declaration a client calls with, so the lookup missed the instance Method entirely. The builder now creates the Methods, the `Results` folder and the `Feedback` object when a provider or sink is configured, declares every signature, and names the right declaration. Unit tests could not have caught this: they invoke the handler delegates directly and never go through `MethodState.Call`. It took an end-to-end test over a real session. Relatedly, children created by the generated `CreateOrReplace` helpers carry no `ReferenceTypeId`, so nothing could reach them — fixed once in the build context rather than at the several dozen call sites. **And one where I was wrong.** Earlier in this branch I reversed my own agents' decision and made `SubmitDetections` accept an empty detection set, reasoning that "I looked and the bin is empty" is a correct observation and refusing it forces a correct agent to invent a detection. The reasoning is sound; it is also an argument *against* the specification rather than an implementation of it. §9.5 is normative and explicit — its StatusCode table lists `Bad_InvalidArgument` for "`Detections` empty; or `SubmitCorrection` supplies both or neither corrected array". Worse, **none of §9.5's argument rules were enforced anywhere** — the dispatcher forwarded every submission straight to the feedback sink, so conformance depended on whichever sink a host installed. They are Server obligations, so they now live in the dispatcher, ahead of the sink, proven by strict mocks that would throw if the sink were consulted. The usability problem is real and does not go away by conforming, so it is raised where it can be fixed: [opcua-drafts#70](https://github.com/marcschier/opcua-drafts/issues/70). ## Tracking the updated drafts The Vision, Robot Intent and AI Model Management NodeSets on `opcua-drafts` `main` have moved, partly because of feedback this branch raised. All three are re-vendored byte-identically — verified by comparing git blob hashes against the upstream files rather than by eye — and the implementations follow. **Two statements that were previously inexpressible now have a wire representation.** `SubmitDetections` takes `SceneIsEmpty` and `SubmitCorrection` takes `RetractAll`, so an agent can report "I examined this frame and there is nothing in it" — the terminating condition of the bin-picking task — and can retract a false positive by correcting a result down to nothing. Both are checked in both directions, because the flag is exactly what separates a deliberate empty observation from a lost payload: an empty array without the flag is refused, and the flag with an array attached is refused as two contradictory claims about one frame. Implemented in the dispatcher, the client, the MCP tools the agent drives, and the sample cell. Also absorbed: `LampType` and `LightingMode` became enums (§66), `ProfileName` became `DefaultProfileName` on the endpoint (distinct from the `ProfileName` a caller passes to `GetStreamEndpoint`, which keeps its name), AI `Invoke`/`InvokeAsync` gained `PayloadUri` under the same exactly-one rule, and AI `ListModels` gained a `ContinuationPoint` — because `MaxResults` alone puts everything past the bound permanently out of reach. **OpenUSD is on `0.9.0-alpha`**, verified on the running cell rather than by restore alone: D3D12 backend, 612×512 frame, the bin and all five parts in view and detected — the same result `0.8.0-alpha` produced. The one local workaround that survives is `VisionMethodArguments`, and its reason has changed rather than gone away: the builder materialises these Methods as Optional children through `CreateOrReplace`, which constructs the state object directly and never runs the generated factory that now carries the signatures. Checked rather than assumed — removing the declarations fails five of the eight `VisionMethodSurfaceTests`. ## Review feedback Three threads, all addressed in `0fe2803a3`: - **The AI Model Management work is now a library family, not a sample.** `src/Opc.Ua.AI` (model), `.Inference` (`IInferenceBackend` and backends), `.Server` (node manager) and `.Client` (discovery, reads, calls, artefact transfer), in the same shape as the Robotics and Vision families; the samples become `samples/AI/ModelManagementServer` and `ModelManagementClient`. The client library needed real work rather than relocation: its browsing surface was `private` inside the sample's scenario runner, so moving the file alone would have produced a library nobody could call. - **No cloud-vendor SDK is referenced anywhere.** Inference goes through `Microsoft.Extensions.AI` — an `IChatClient` is implemented by hosted services and on-device runtimes alike, which is exactly the property clause 8.1 asserts. `Azure.Identity` is gone: workload identity is not an Azure feature, every platform projects a token into a file and each SDK reads that file behind its own API, so the resolver reads it directly. The comment claiming `Microsoft.Extensions.AI` would force a `System.Text.Json` bump was stale — it pins 10.0.10, which this repository already centralises on. - **`BinPickingCell` and `BinPickingClient` moved into `samples/Robotics/`**, which is where they belonged: the cell was already linking the arm, its kinematics and both USD assets out of `IntentEnabledRobot`. ## Upstream Work that belonged elsewhere went elsewhere rather than being worked around locally: - **`openusd-dotnet` [#13](https://github.com/marcschier/openusd-dotnet/pull/13)** — merged and shipped in `0.8.0-alpha`. `SilkFrameCapture.Capture` rendered only the first capture of a session and silently returned a blank frame for every one after it. Also filed [#14](https://github.com/marcschier/openusd-dotnet/issues/14) (no camera-by-prim API) and [#15](https://github.com/marcschier/openusd-dotnet/issues/15) (skipped tests reported as passed — which fooled me for several runs). Both are fixed; this branch takes `0.8.0-alpha` and deletes two workarounds, including 143 lines of hand-rolled projection maths. - **`opcua-drafts` [#66](https://github.com/marcschier/opcua-drafts/issues/66)–[#69](https://github.com/marcschier/opcua-drafts/issues/69)** — a close read of the spec produced 22 candidate ambiguities; 12 were discarded as misreadings with clause citations and the remaining 10 filed. One is concrete: five members exist in the NodeId CSV that no clause documents. - **`opcua-drafts` [#70](https://github.com/marcschier/opcua-drafts/issues/70)** — §9.5 could not express an empty observation or a false-positive retraction. **Adopted**, with the explicit-flag resolution argued for: `SubmitDetections` gained `SceneIsEmpty`, `SubmitCorrection` gained `RetractAll`, and the corrected-array rule relaxed from *exactly one* to *at most one*. - **`opcua-drafts` [#71](https://github.com/marcschier/opcua-drafts/issues/71)** — the Vision NodeSet browse-named all 13 `InputArguments` / `OutputArguments` Properties in the Vision namespace rather than the base namespace, so a stack could not find them and every Method in the specification was uncallable. **Fixed**; this branch now tracks the corrected artefact. ## The defect underneath it all The end-to-end suite was removed earlier in this branch because nine of its fourteen tests failed on one symptom: a client could not open a pipeline's `Feedback` object or enumerate its `Results`, even though browsing the same parent listed both. Restoring it meant explaining that, and the explanation was not in Vision. `RelativePathElement` initialised **`IsInverse` to `true`**. A caller that sets a `ReferenceTypeId` and a `TargetName` — which reads as *"the child reached by this Reference"* — was therefore asking the Server for the **inverse** Reference, and got `Bad_NoMatch` with nothing in the request to suggest why. `ObjectTypeClient.ResolveChildNodeIdAsync` is such a caller, and it is what **every source-generated Optional-child accessor in the stack** is built on — the accessor whose own documentation cites `AlarmConditionTypeClient.GetShelvingStateAsync` as an example. Vision is merely where it surfaced, because Vision puts two Optional children on the path an off-Server agent must take. The sibling helper in `StateMachineTypeClientExtensions` already set `IsInverse = false` by hand, which is the shape of a trap rather than a default. The default is now forward, `ResolveChildNodeIdAsync` says so explicitly, and the existing test that had recorded `true` as *expected* now pins `false` with the reasoning. Of the forty `RelativePathElement` construction sites in the tree, that accessor was **the only one relying on the default at all** — every deliberate caller already set it. Two further defects were found underneath it, both real and both fixed: - The generated `CreateOrReplace` helpers construct a child state object directly and leave **`TypeDefinitionId`** unset as well as `ReferenceTypeId`. A child referenced by nothing cannot be browsed; one with no type definition is a malformed Object that a client filtering by type silently skips — which is exactly what `EnumerateResultsAsync` does. Normalisation moved to `VisionNodeManager`, so it also covers the case the builder cannot see: a result published **at runtime** by an inference provider, long after the address space was built. That path produced results a client could list but not read. - `Opc.Ua.AI` shipped without an `AssemblyInfo`, so the solution built with six `CA1014` warnings. I had initially "fixed" this in the Vision client with Browse-based fallbacks, on the strength of a probe that appeared to show `TranslateBrowsePathsToNodeIds` failing even for `Server → ServerStatus`. That probe was wrong: it used an unqualified `ReferenceTypeIds`, which inside a `Opc.Ua.Vision.*` namespace binds to *Vision's* `ReferenceTypeIds` and yields a nonsense NodeId. Chasing that to the end is what turned up the real cause. **The fallbacks are gone** — they would have hidden a stack-wide bug behind a Vision-shaped workaround, and one of them matched on browse name while ignoring the namespace index. `tests/Opc.Ua.Vision.Intent.Tests` is restored and back in `UA.slnx`, passing **14/14** on its own merit rather than through a client-side workaround. A registration test now pins the pipeline's `Feedback` and `Results` children, which nothing covered before. ## Running it, and what that found The demo's premise is that a language model **looks at the bin and reasons about what it sees**. Running it end to end showed it could not, for reasons no test covered, because every one of them lived between components. **The sample did not compile.** Two `CS8604` nullable errors with `TreatWarningsAsErrors`, so neither the cell server nor the demo could be started at all. Worth recording that my first explanation was wrong: CI *does* build this project - the Linux leg's `net10.0` pass compiles it, log-confirmed - so this is not a coverage gap in the workflow, and no workflow change was made. **`Pick` and `Place` never moved the arm.** They were a delay plus a gripper action, so the operations reported `Succeeded` while nothing travelled. They now solve IK once and interpolate in joint space. **And the parts never moved**: the arm swung but the bin stayed full, because nothing tied the held object to the world state. The connector now creates 11 monitored items where it created 6, and a scripted run delivers `SeqNo=151` across 10 messages where it delivered `SeqNo=3` across 1. **The frame was smuggled through a string field.** The media provider put a base64 *data URI* of the PNG into `VisionImageReferenceDataType.Uri` - a field meant to be a *reference*, while the bytes were already returned in the inline `ByteString`. Every response therefore shipped the image twice, and a 1,311,400-byte PNG became a 1,748,536-character string against a `MaxStringLength` of 65,535: **26.7x over**. That is why raising `MaxByteStringLength`, `MaxArrayLength` and `MaxMessageSize` never helped - none of them govern a `String`. Separately the MCP tool assigned raw bytes where `ImageContentBlock` requires base64, so the wire carried `"data":"�PNG..."` - every non-UTF-8 byte replaced by U+FFFD, destroying the image and inflating the response to 6.8 MB. **The camera was aimed at nothing.** The arm's home pose, the flange scan pose the Vision model declared and the joint angles in the USD stage were three independent claims about where the camera was, and they disagreed - the declared flange orientation pointed the view 86 degrees away from the projection camera. Solving them as one thing exposed two further constraints that are easy to miss: the camera has to sit off the tool axis or it photographs its own gripper, and the IK branch has to be elbow-back or a link parks under the camera and fills the frame with the arm's own upper arm. A third was a regression this branch introduced and `Opc.Ua.Robotics.Tests` caught - aiming a straight-down camera from a point on the base's own X-Z plane puts the wrist on the J4/J6 singularity, so every motion away from home fails to solve. A 15-degree camera roll, free because the camera still looks straight down, gets 25 degrees clear, which is the margin the poses before this branch had. **The frame and the detections were different images.** The sensor declared 2448x2048, the clip endpoint 1280x1024, and the renderer produced 640x512 - not even the same aspect ratio. The ground-truth detector projects through the declared intrinsics, so its boxes lived in 2448x2048 space while an agent was handed a 640x512 picture: *"pick the red cube you can see"* arrived with coordinates that pointed off the image. Everything now agrees at 612x512, an exact 4x4 bin of the native sensor with the intrinsics scaled to match; the native size stays in the model and serial number, where it describes the hardware rather than the image. **`LatestClip` was created and never written**, so it reported `Bad_NoDataAvailable` for the life of the Server and a consumer following the model - read the published frame, call the method only if there is none - never got a frame. The dispatcher now publishes a clip it has just encoded, for every Vision server rather than this sample alone. One genuine API gap turned up on the way: `OpcUaServerOptions` exposes `MaxByteStringLength`, `MaxArrayLength` and `MaxMessageSize` and **no `MaxStringLength`**, so a Server that legitimately needs a long string could not raise the limit through the hosting API at all. Added, with tests. It is not the fix for anything above - that was a design bug - but the missing knob was real. ## Not finished - **Two projects sit below the repository's 80% line coverage.** `Opc.Ua.Vision.OpenUsd` is at 71.1% — what is left only executes with a real OpenUSD stage and plugin tree, which is not on every build agent. `Opc.Ua.AI.Inference` is now at **86.7%**, up from 12.9%: the code this branch *wrote* is covered by 25 tests, and `RestChatCompletionsBackend` — previously the whole of the shortfall — is covered by 18 more against a stubbed `HttpMessageHandler`, exercising success, refusal, throttling, retry-after, timeout and cancellation without a network. Everything else clears the bar — `Opc.Ua.Vision.Server` 92.1%, `Opc.Ua.AI` 91.4%, `.Vision.Client` 88.1%, `Opc.Ua.Vision` 85.7%, `Opc.Ua.AI.Server` 80.1%, all measured with coverlet. The CI coverage gate reports itself as advisory and non-blocking. - **`SamplesCollected` is not counted here, and `docs/Vision.md` now says so.** Section 9.4 requires a Server to count a negative example (`SceneIsEmpty` / `RetractAll` carrying a `GroundTruthLabel`) exactly as it counts one carrying geometry. `VisionNodeManager` does not, and structurally cannot: `SamplesCollected` is a property of `LearningJobType`, which the *AI Model Management* companion defines, and Vision reaches the job through a `NodeId` value rather than a Reference **precisely so this model takes no dependency on the model that defines it**. What the Server does guarantee is that the negative example survives the hop intact — both flags are carried verbatim to `IVisionFeedbackSink` — so a host that binds a learning job has everything it needs to satisfy the clause on the counter it owns. Stated in the limitations rather than left implied. ## Related Issues Was stacked on #4195, now merged. No tracking issue yet for the Vision implementation itself — happy to open one if you would like the design recorded as an ADR before this goes further. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. — *Types 8562, Server 4072, Client 2123, InformationModel 1063, Robotics 540, Tools 466, Vision 442, AI 74 and the restored Vision+Intent end-to-end suite 14/14, all green on net10.0. Only `.OpenUsd` (71.1%) is below the 80% bar — see Not finished.* - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. — *`dotnet build UA.slnx -c Release`: 0 errors, 0 warnings.* - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. — *net48: Types 8555, Client 2126, Vision 385 green. Two net48 Server tests fail (`CreateUserManagementSeamBindsTheModelToTheServerAsync`, and `DurableDataValueQueueVerifyReferenceBatchingAsync` only under parallel load); both were confirmed **pre-existing** by re-running them with the `IsInverse` change reverted, which fails identically.* - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. — *The net48 build legs failed because the restored end-to-end project used `ValueTask.FromResult` / `string.Create`, which do not exist on .NET Framework; fixed in `fb57522da` and verified building on every TFM. `DisposeDrainsHeldConnectionCallbackBeforeListenerDisposal` failed once on the Ubuntu Client leg — a 13 ms timing-sensitive reverse-connect disposal test that passes locally 2123/2123 and touches no browse paths.* - [ ] I have addressed **all** PR feedback received. ## Typed Vision-guided Robotics MCP workflows The branch now also makes the demo workflow first-class in the Full MCP profile: - replaces stringified Robotics intent and mission inputs with typed request schemas and exact name-or-NodeId selectors; - adds bounded operation/mission paging, stable step-to-operation identity, retained mission discovery, and `robotics_wait_mission`; - makes `vision_run_inference` return bounded typed detection, inspection, or segmentation summaries in one call; - adds same-session `robotics_vision_pick` to select a detection and submit Pick or Pick/Place work without implicitly taking authority, retrying, cancelling, or waiting; and - starts Publish workers for classic subscriptions so an independently launched OpenUSD viewer receives live robot and part updates. The migration guide documents the intentionally breaking MCP request-schema changes. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1dab1302-19c5-4a9b-a50c-d97d389713aa Copilot-Session: a248a589-9a20-4372-868e-5d347e57001b Copilot-Session: 99d1ab42-f83a-41ab-bf3e-85bf81fadace Copilot-Session: bcbc8ac9-5abd-42a8-8bac-e10533790e2e	2026-08-29 19:34:15 +02:00
Marc Schier	aad8350bc1	Pluggable symmetric cryptography, an awaitable asymmetric path, registrable security policies, and PubSub crypto offboarding (#4206, #4207, #4208, #4210) (#4232) ... # Description Closes the three crypto-offboarding follow-ups split out of #4192. Each was recorded there as knowingly *not* done, with the reason and what would justify revisiting it. Those conditions are met, and where one of them turned out to be unachievable that is written down rather than worked around. The safety property throughout: **every seam is opt-in and null-checked, and in the default software configuration the added code is observably inert.** No interface dispatch on the per-message path, and the asynchronous paths complete synchronously — so channel locking order, buffer ownership and error sequencing are unchanged for a deployment that configures nothing. ## #4207 — symmetric, key-derivation and RNG provider seams `ISymmetricCryptoProvider`, `IKeyDerivationProvider` and `ISecureRandomSource`, added on the terms #4207 set: only once a concrete requirement arrived (a validated module that must perform *every* operation, not just the asymmetric ones), behind a fast path, with `SymmetricChannelCryptoBenchmarks` as the gate. They are **facets discovered by type test**, not members of `ICryptoProvider`, so providers written against the shipped interface still compile. `PlatformCryptoProvider` implements all three, which answers the "an interface nothing implements" objection recorded in the issue. `CryptoProviderFacets` returns `null` both when no registry is configured *and* when resolution lands on the platform, because the platform facets are the inline code. Resolution happens once, in `CalculateSymmetricKeySizes`, and is held for the life of the channel. `CryptoCompliance.GetUnservedOperationPurposes` closes the failure the seam would otherwise introduce: a provider bound to `ChannelSymmetric` without implementing the facet is silently replaced by the platform, so a deployment could believe its validated module performed the per-message cryptography while the platform did. Under `FipsOnly` that now refuses to start. ## #4208 — the asymmetric path is awaitable `RSA` and `ECDsa` are .NET's contracts and cannot be replaced without making every ready-made hardware and cloud implementation unusable. Instead an implementation may **also** declare `IAsyncRsaKey` / `IAsyncEcdsaKey`, which the stack finds by type test. Now asynchronous: the secure channel open and renew path, user identity token signing and decryption, and session activation. A software key declares neither facet, so those paths complete synchronously and nothing about their ordering changes. `ReadAsymmetricMessage` is refactored into shared helpers rather than duplicated — adding an `await` was not worth two hundred cloned lines of signature, padding and endpoint-selection logic. `AsymmetricMessage` / `AsymmetricWriteResult` exist because an asynchronous method cannot have `out` parameters. The channel no longer serialises its state on a monitor. A monitor cannot be held across an `await`, and a plain `SemaphoreSlim` was not an option at the time: the code relied on re-entrancy in eight places (`HandleIncomingMessage` holds the lock and calls `ForceChannelFault`, which takes it again; `ForceReconnect` and `Shutdown` do the same). `ChannelGate` replaces it at 32 sites and `DataLock` is `[Obsolete]`. That reliance has since been removed as well — see [Removing the gate's re-entrancy](#removing-the-gates-re-entrancy) below. ### Defects this surfaced, all fixed and regression-tested **Detached work started inline.** Writes were started with `_ = WriteBuffersAsync(...)`. An async method runs its prologue on the caller's stack, so `LeaveInheritedContext` — which exists to stop *detached* work inheriting the right to re-enter the gate — was stripping the **caller's own** entitlement; when the send then completed synchronously, the completion blocked on a gate that very thread was holding. The client sent its OpenSecureChannel request and never processed the response. Writes are now queued. **A synchronous gate handle held across an `await`.** `Enter()` records the acquiring thread so an inline completion callback can re-enter — `ChannelAsyncOperation` invokes its callback both inline and detached. That record is only sound while the holder is synchronously on that thread; the token renewal path held it across an await, so a reused pool thread was recognised as the holder. **`SignAsync` rejected the policy that signs nothing.** It validated the certificate before the algorithm, so `SecurityPolicies.None` — which carries no certificate — threw where the synchronous `Sign` returns `null`. **`EnterAsync` lost the holder when contended** *(found in review)*. The `AsyncLocal` recording ownership was written inside the awaiting method, and .NET restores the caller's execution context when an async method completes, so the write was discarded. The caller returned owning the semaphore while `IsHeldByCurrentContext` reported `false` — and any nested `Enter()` then blocked forever on a gate its own context held. Reachable remotely: `HandleIncomingMessageAsync` → error path → `ForceChannelFault`. The holder is now published from the caller's frame before the wait, and only its depth is raised in the continuation. **Queued writes lost their order** *(found in review)*. Sequence numbers are assigned under the gate, but independently queued work items reach the transport's FIFO send lock in arbitrary order, so a peer could reject a chunk with `BadSequenceNumberInvalid`. Writes are now appended to a chain: still off the caller's stack, but a write cannot start before its predecessor finishes. ### Four more this surfaced in CI, on paths only the integration suites reach Queuing the writes to fix the deadlock above turned out to have moved three implicit guarantees, and the gate had one more liveness bug. Each was reproduced locally against a clean baseline before it was changed. **A faulting channel's error message never reached the peer.** A listener channel sends a UASC `Error` describing why it faulted and closes the transport as its next statement. That worked only because the write started on the caller's stack, so the bytes reached the socket first; queued, the close discarded them and the client saw `BadConnectionClosed` instead of the certificate status it was told to expect. Nine `SecurityCertValidationTests` cases assert exactly that status. The terminal error is written inline again, on a path that touches the gate only to disclaim what it inherited and reports nothing — a channel that is already faulted is owed nothing beyond its buffer. **The certificate of a rejected sender leaked.** An asynchronous method cannot have `out` parameters, so the asymmetric read returns a record instead. That silently dropped an ownership guarantee: an `out` parameter reaches the caller even when the method goes on to throw, so the caller's `catch` disposed the sender certificate and reported it to the audit. A return value does not — nine leaked certificates, one per rejection case, which the leak detector fails the run on. The parsed certificate is now handed to the caller before anything that can reject the message, restoring both the disposal and the audit. **The gate could strand its own permit.** It released only when the holder's depth returned to zero, but that depth lives in an `AsyncLocal` and is therefore shared with every context forked from the owner's — a timer callback whose timer was created inside the region, for instance. Such a context entering the gate raised the shared depth, so the owner's own exit saw a non-zero depth and returned without releasing, and nothing afterwards could: the nested exit is not the owner. Every later entry on that channel blocked forever. `Opc.Ua.Features.Tests` hung after its tests had finished, with a receive loop waiting on the gate and the server unable to stop. The owner now releases when it leaves; depth still gates re-entrancy, but liveness no longer depends on a count another context can move. **Write completion throttled the writer.** Reporting a completed write sat inside the ordering chain, and the client channel implements that callback by entering the gate — so every write queued behind a gate the receive loop contends for on each incoming chunk. A session with fifty subscriptions publishing continuously had its publish requests throttled enough that the client kept topping them up past its own limit, which `Subscriptions.Classic` asserts against. Completion is now reported off the chain, which covers only what ordering actually requires. ### Still synchronous, and why Certificate, certificate request and revocation list signing cannot be made asynchronous at all: `X509SignatureGenerator.SignData` is invoked by .NET's own builders. Service faults and the `Reconnect` override are reached from synchronous call sites (`SendServiceFault` alone has eight) and are left as they are behind scoped, commented suppressions. All three are documented rather than left to be rediscovered. ## #4210 — PubSub This one produced a finding rather than the API the issue asked for, and the finding is the point: > **With a standard Security Key Service the key necessarily exists in process memory.** `GetSecurityKeys` (Part 14 §8.3.2) returns raw key bytes over the wire, so the "key never leaves the device" property #4192 achieved for client and server **cannot** be achieved for PubSub through the SKS pull profile. That is a property of the specification, not of this stack. A wrapped-key envelope would change what is on the wire and break interoperability with third-party key services and publishers, so it was rejected rather than built. What is achievable is delivered: the per-message AES-CTR and HMAC route through `ISymmetricCryptoProvider`, so a validated module performs them. `IPubSubSecurityPolicy` is **not** modified — new members would break every implementation of it; the provider is supplied through the policy constructor. `PlatformSymmetricCryptoProvider` gained AES counter mode, which the seam should have covered anyway since the algorithm enum names it. Key lifetime is also tightened: unpacking an SKS response copied key material into intermediates that `ByteString.Create` then copied again, leaving plain keys in the heap for the lifetime of the process. Those are now zeroed. ### Two wiring gaps this depended on, both found in review The registrations above were **inert**, so this section originally claimed more than the code did: - `CryptoProviderConfiguration` was applied nowhere outside tests, so `AddCryptoProvider(configure)` never reached the registry in a real host. The registry now applies every registered configuration when it is first resolved — which also affects #4207's DI story. - `PubSubSecurityWrapperResolver` selected `PubSubAes256CtrPolicy.Instance`, the provider-less static singleton, ignoring the provider-backed bundles registered for it. A `FipsOnly` deployment therefore certified while the platform did the work. The resolver now selects from the registered policies; the container and `PubSubApplicationBuilder` both supply theirs. A third, smaller one: `CryptoProviderBuilder.Use` published a **policy-scoped** random source as the process-wide nonce source, so `.For(purpose, policyUri).Use(hsm)` redirected nonce generation for every other policy too. Only an unscoped binding does that now. ## #4206 — a provider can contribute a security policy Added in review: the policy set was fixed at compile time, so a provider could contribute key custody but not a *profile*. It is now table-driven and open, and the blockers #4206 listed are gone. ```csharp using IDisposable registration = SecurityPolicies.Register( new SecurityPolicyInfo("urn:vendor:SecurityPolicy#CustomProfile", "CustomProfile") { PlatformSupport = () => true, SupportedCertificateTypes = [ObjectTypeIds.RsaSha256ApplicationCertificateType], IsDefault = true }); services.AddOpcUa() .AddCryptoProvider(crypto => crypto .For(CryptoPurpose.ChannelSymmetric, "urn:vendor:SecurityPolicy#CustomProfile").Use(module)) .AddSecurityPolicy(customPolicy); ``` - **The reflection is gone.** Four lookup tables were built by reflecting over the fields of `SecurityPolicies` and `SecurityPolicyInfo`. They are one explicit table now, which removes the **last reflection in `Security/Constants`** — so this is a trimming and Native AOT improvement as much as a feature. - **`SecurityPolicyInfo` is constructible from another assembly** and carries the metadata the helpers need, so `IsPlatformSupportedName`, `GetDefaultUris`, `MapSecurityPolicyToCertificateTypes` and `GetCurveFromCertificateTypeId` read that table instead of each being hand-written. Adding a policy is one entry rather than five edits. - **The dead `#if CURVE25519` conditional is deleted** from `Security/Constants`. - Registration is copy-on-write, so the read path stays lock-free. Shadowing a built-in policy requires `replaceExisting: true` and is reversible by disposing the registration. `RegisterLightsUpCurvePoliciesFromOutsideCore` is the acceptance test the issue asked for: it registers `ECC_curve25519` and `ECC_curve448` from the test project and asserts they become discoverable through the public API. **Registering a policy makes it advertised and resolvable; it does not supply the cryptography behind it.** For those two curve profiles the in-tree key agreement is still behind a compile symbol no project defines plus a BouncyCastle dependency, so a deployment lighting them up provides the operations through a provider. That division is the point: the policy set says *what* is offered, the provider says *who* performs it. ## Removing the gate's re-entrancy Folded in from #4242, which was merged into this branch. `ChannelGate` was re-entrant, and that forced the design behind most of the defects above. A holder that awaits releases its thread, so ownership could not be tracked by thread identity — it had to live in an `AsyncLocal<Holder?>`. A logical context is *inherited*, so anything started while the gate was held inherited the right to re-enter and ran inside the guarded region alongside whatever started it, unless it opted out by hand via `LeaveInheritedContext()` at each of eight sites, enforced by review alone. Five of the defects listed above trace to that mechanism, including both CI hangs. So the *requirement* was removed rather than the mechanism, and the `AsyncLocal` then deleted itself. Every path that re-entered now calls a lock-free `Core` method (`ForceChannelFaultCore`, `ShutdownCore`, `ForceReconnectCore`, `CompleteConnectCore`), and three regions stopped running foreign code under the lock: the `RequestReceived` dispatch is deferred until the gate is released, `OnTransportError` no longer wraps the `HandleSocketError` virtual, and the callback setters and `EndpointDescription` publish a single reference with a volatile write instead of locking — the last is load-bearing, because `AsyncResultBase.OperationCompleted` invokes the reverse-hello callback inline while the gate is held. `ChannelGate` is now a `SemaphoreSlim(1,1)` behind a `Releaser`; `Holder`, the thread-identity fallback, `LeaveInheritedContext`, `IsHeldByCurrentContext` and all eight disclaimer calls are gone, with the call shape at all 32 sites unchanged. `SaveIntermediateChunk`, `GetSavedChunks` and `DoMessageLimitsExceeded` gained a `gateHeld` argument, because the client reaches them both with and without the gate and must pick the matching teardown — without it an oversized `OpenSecureChannel` response deadlocked the receive loop against itself. The remaining paths were not found by reading the code, since that is what missed them originally. A temporary strict mode made every re-entrant acquisition throw with its stack, the suite was run under it until clean, and it was deleted with the re-entrancy it measured. It caught two sites review had missed: the `EndpointDescription` **property getter** taking the gate, and the `RequestReceived` dispatch to application code happening inside the guarded region. Two deliberate residuals are documented for reviewers: `Dispose` no longer takes the gate (unavoidable — it is reached from `ChannelFaulted` with the gate held, where the old re-entrant acquisition excluded nothing anyway), and the deferred request dispatch builds `SecureChannelContext` just after gate release rather than under it. ## Testing New: the three seams and their platform implementations, facet discovery and resolution precedence, `FipsOnly` refusing a provider that cannot serve what it was bound to, AES counter mode (including a non-block-aligned round trip and a check that the counter carries, since one that did not would silently reuse key stream), `ChannelGate` (exclusion, re-entrancy across an await, re-entrancy on a reused thread, and the deadlocks above), the asynchronous channel paths, and the PubSub policy provider. Each review finding got a test that fails without its fix: - `ContendedEntryStillRecordsTheHolderAsync` — established contention deterministically, then re-entered; failed on the pre-fix gate. Removed with the re-entrancy it covered; `ChannelGateTests` now asserts the non-re-entrant contract instead. - `SecureClientAndTcpListenerKeepChunkOrderUnderConcurrentRequestsAsync` — 60 concurrent multi-chunk `SignAndEncrypt` requests over a real loopback listener; failed 3 runs out of 3 with the previous dispatch, passes 3 out of 3 with the chain. - `DependencyInjectionRoutesPubSubCryptoThroughTheRegisteredProviderAsync` — builds a container, registers a counting provider, wraps a message and asserts the provider actually performed the encryption and the signature. This is the assertion the PubSub claim above now rests on. - `PolicyScopedRandomBindingDoesNotEscapeItsPolicy`. Two further assertions are the ones that would catch a silent defect: a provider and the platform must produce **byte-identical** output — a validated module that disagreed could not interoperate — and with a software key the returned `ValueTask` must report `IsCompletedSuccessfully`, which is what makes "ordering is unchanged" checkable rather than merely asserted. `SymmetricChannelCryptoBenchmarks` gains a provider-registered variant beside the existing baseline, so the cost of the indirection is measured rather than assumed. Verified locally, every suite at or above its recorded baseline: `dotnet build UA.slnx -c Release` **0 warnings, 0 errors on all six target frameworks**; Core **4413** (net10.0); Core.Security **617**, with its certificate-leak detector clean; PubSub **1350** plus PubSub.Diagnostics **54** and PubSub.Udp **233**; Features **156**; Subscriptions.Classic **32**; Server integration **4053**; Client integration **2123**; MigrationAnalyzer **151** and MigrationAnalyzer.Core **13**. Zero failures in each. Core moved from 4388 to 4381 when the gate change landed — eight re-entrancy tests were removed along with the behaviour they covered, and one deadlock regression test added — and has since grown to 4413 with the registry isolation, provider-output and security-review tests. `ChannelGateTests` was rewritten around the new contract — exclusion across both entry modes, an uncontended asynchronous entry not suspending, re-entry blocking by contract, and the property that used to be the hazard: work started while the gate is held is excluded without having to opt out. Every CI failure this PR produced was reproduced locally, diagnosed against a clean `origin/master` baseline in a second worktree, and fixed at the source. Two hangs were found from `--blame-hang-timeout` dumps read with `dotnet-dump`. One failure that looked like a candidate was confirmed **not** to be: an intermittent `Opc.Ua.Aot.Tests` connect failure, pre-existing on `origin/master` (5 of 8 baseline runs failed, versus 3 of 14 here). ### The publish request overshoot `Subscriptions.Classic.PublishRequestCountAsync` failed on the Windows net10.0 leg with **207** outstanding publish requests against a cap of `max(maxServerPublishRequest, subscriptions)` = 50. It failed in 2 of 6 pipeline builds on this branch, never on `master` in 13 sampled builds, and never locally in 12 attempts including under deliberate CPU contention. The mechanism is a genuine race. `ClassicSubscriptionEngine.QueueBeginPublish` was a **check-then-act**: it read `GoodPublishRequestCount`, compared it against `GetDesiredPublishRequestCount(false)`, and issued a publish if it was below. It runs on every publish completion, so *n* completions that overlap each read the same value, each conclude they are below the cap, and each issue — overshooting by up to *n* per round. The read also *lags*: a request only becomes visible to `GoodPublishRequestCount` once `AsyncRequestStarted` has recorded it, which happens after the request has already been sent. That code is pre-existing and was untouched by this PR. What this PR changed is how hard the race is hit: the server used to hold the channel gate across request processing, which serialised every request on a channel, so publish completions arrived one at a time. Dispatching `RequestReceived` outside the gate — an intended improvement, since a channel should not serialise on application request-processing time — lets them complete concurrently. Rather than change the publish pipeline blind, the race was first made reproducible. `ConcurrentPublishReEvaluationDoesNotExceedDesiredRequestCount` drives 64 barrier-released completions against a mocked context whose count lags the send exactly the way the session's does. It issued **64 requests where 5 were desired** — the same defect the CI leg hit, now deterministic and running in milliseconds. The engine now tracks the requests it has in flight and reserves a slot with a compare-exchange before sending, so the decision and the send are one atomic step. The reservation is released when the request completes, when the send is declined, or when it throws. The repro test then issues at most the desired count. `StartPublishing` is deliberately left **uncapped**. It is the recovery valve that refills a pipeline whose requests are outstanding but no longer expected to return, so gating it on the in-flight count would remove the only path out of a stalled pipeline. An earlier revision of this fix did gate it, and that is the one change here that could not be justified on the evidence — it was reverted. The assertion was **not** loosened, retried away, or marked flaky: it caught a real throttling regression earlier in this PR and is worth keeping sharp. Validated: the repro test red before the fix and green after; `Subscriptions.Classic` 32/32 **five** consecutive runs on net10.0 and once on net48; `Opc.Ua.Client.Tests` 2124 on net10.0 across four consecutive runs and 2127 on net48; `Opc.Ua.Subscriptions.Tests` 604 passed with this fix in place. ### The SSH.NET advisory that turned CI red repo-wide Partway through this work every build job began failing with `NU1903: Package 'SSH.NET' 2025.1.0 has a known high severity vulnerability` ([GHSA-q939-rpr3-3284](https://github.com/advisories/GHSA-q939-rpr3-3284), `<= 2025.1.0` affected, first patched `2026.0.0`). Nothing in the dependency graph changed — the advisory was published. Build 17026 failed on it where build 17000, two hours earlier on an identical graph, had passed every build job. SSH.NET is not referenced anywhere in the repository; it arrives transitively through `Testcontainers.Kafka`. `master` landed the remediation independently while this was being investigated (a central transitive pin to `2026.0.0`, which `CentralPackageTransitivePinningEnabled` exists for). This branch briefly carried its own identical pin, which made the merge commit hold **two** `PackageVersion` entries for SSH.NET and fail restore with `NU1506`. The duplicate is removed and `master` is merged, so the pin now comes from one place only. The audit was **not** suppressed: `NuGetAudit` stays enabled and `NU1903` stays an error. Verified after the merge — `dotnet restore UA.slnx` reports neither NU1903 nor NU1506, and `dotnet build UA.slnx -c Release` is **0 warnings, 0 errors** with the audit live. The remaining red check in build 17026 was not from this change either. `aot-ubuntu-latest` is the intermittent `Opc.Ua.Aot.Tests` failure already noted above — `ReconnectSessionAsync` timed out at 30s with 135 of 136 passing. It remains below the `origin/master` baseline rate (4 of 15 sampled here against 5 of 8 on the baseline), and the managed reconnect paths are green throughout: `Opc.Ua.Client.Tests` 2124 across four consecutive runs and `Opc.Ua.Subscriptions.Tests` 604. ## Documentation `docs/CryptoProvider.md` gains sections on substituting the symmetric primitives, using a key served over a network, and PubSub — including what cannot be claimed and why, the scope rule for a random-source binding, and how to supply a provider-backed policy outside the container. `docs/MigrationGuide.md` covers the `[Obsolete]` channel virtuals and `DataLock`, and the non-re-entrant gate's contract for channel subclasses. `docs/WhatsNewIn2.0.md` is updated. `plans/cryptooffboard.md` is deleted now that the work it planned has landed. ## Note for reviewers This touches the secure channel handshake, which is the most security-critical code in the stack. Nine distinct concurrency and lifetime defects were found and fixed in it during this work — three during development, two in review, four from CI — so `ChannelGate`, the write path and the certificate ownership across the new suspension points deserve the closest reading. Five of those nine traced to the gate's `AsyncLocal`, which is why the re-entrancy was subsequently removed and the `AsyncLocal` deleted. The wrapper/`Core` split and the two documented residuals in that section are the parts most worth a second reading. The #4206 work removes the last reflection from `Security/Constants` and rewrites the policy lookup tables, so `SecurityConfiguration.SupportedSecurityPolicies` and the endpoint set it feeds are worth a second look: an earlier revision of it changed that set and broke six server tests, which is now covered by `SecurityConfigurationBuildsExpectedSupportedPolicySet`. ## Related Issues - Fixes #4206 - Fixes #4207 - Fixes #4208 - Fixes #4210 - Follow-up #4250 — this PR makes the security policy set injectable, but nothing in the stack resolves `ISecurityPolicyRegistry` from DI yet, so every production path still lands on the `Default` fallback. #4250 tracks threading it through `Session` and `UaSCBinaryChannel`, which is what makes the injection points already added here load-bearing. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com> Copilot-Session: 7a68ca72-83e8-496c-b2f5-3df4c718320b Copilot-Session: 3f5ab76a-c0ba-4f47-87ac-54d1f4182c6b	2026-08-15 05:23:20 -07:00
Marc Schier	1cca744b5c	Retire the last exposed locks: node, browser, server diagnostics and application properties (#4222) ... Closes #4201. Closes #4202. Closes #4203. Closes #4215. Closes #4216. (#4215 and #4216 are @marcschier's originals from the #4188 review; #4202 and #4201 are my later duplicates of the same two findings. All four describe the two changes in section 1 and are closed together.) **This PR now carries what was #4221 as well.** The two were a stack; they have been unstacked and combined here, and #4221 is closed. Everything below the "Read the server diagnostics" section is the work that was previously reviewed as #4221. --- ## 1. The browse path guards itself, and the last node locks are gone `AsyncCustomNodeManager` kept two `lock (source)` sites around `CreateBrowser`, carrying a NOTE that browser creation was not internally synchronized. `NodeBrowser` still handed its lock to derived types through `protected Lock DataLock`. `NodeState.CreateBrowser` now holds a browse lock across `PopulateBrowser` and `OnPopulateBrowser`, so browser construction on a node is serialized — which is what the external `lock (source)` was actually providing, and what a handler that mutates the node during population (the historian's lazy config installer) depends on. Both `lock (source)` statements and the NOTE are gone. **What that lock does not do**, stated plainly in the XML docs and in `docs/NodeManagers.md` rather than over-promised: it is not an atomic snapshot across the node's children, notifiers and references. Writers take those collections' own locks, not the browse lock. The browser is a point-in-time copy and each collection is read consistently, but the combination is not a transaction. No `lock` takes a `NodeState` as its sync root any more. That was the last thing keeping `NodeState` from being documentably self-guarding, so the other eleven went with it: `lock (activeNode)`, `lock (target)`, `lock (source.Node)`, `lock (target.Node)`, `lock (notifier)`, `lock (handle.Node)` in `AsyncCustomNodeManager`, and `lock (node)` in `CustomNodeManager2`. Most were a check-then-act pair now served by `NodeState.AddReferenceIfMissing`, which does the check and the insert under the node's own lock. `NodeBrowser.DataLock` is removed. A browser is **single-consumer**: it belongs to whoever created it, and both server browse paths already serialize on the owning side. The exposed lock only added an inheritance-level locking contract that a derived browser could not reason about. ## 2. Read the server diagnostics under the lock every writer takes `OnUpdateDiagnostics` locked the diagnostics **payload** rather than the lock every writer takes through `UpdateServerDiagnostics`, so the snapshot a client reads from the `ServerDiagnostics` node was never excluded from a concurrent writer. Both halves matter, and the second is the one that is easy to miss: ```csharp lock (m_diagnosticsLock) // was: lock (ServerDiagnostics) { value = Variant.FromStructure(ServerDiagnostics, copy: true); // was: copy defaulted to false } ``` `copy: true` is what makes it a snapshot. `FromStructure(value, copy: false)` does `new ExtensionObject(value, copy)` — it wraps the **live** structure without copying it, so the caller would read the fields long after the lock was released and see them change underneath. Swapping the lock alone would have looked like a fix while still excluding nothing. ## 3. Analyzer numbering The browser rule is **UA0027**, not UA0025. This branch had independently taken UA0025 for `NodeBrowser.DataLock` while #4188 published UA0025 for `ILocalNode.DataLock` and UA0026 for `BaseVariableValue.Lock`. Master's are shipped, so they keep the numbers: | | UA0025 | UA0026 | UA0027 | |---|---|---|---| | master (#4188) | `ILocalNode.DataLock` | `BaseVariableValue.Lock` | — | | **this PR** | `ILocalNode.DataLock` | `BaseVariableValue.Lock` | **`NodeBrowser.DataLock`** | Both sides had written a different `UA0025Tests.cs`, which arrived as an add/add conflict. They are now split: `UA0025Tests.cs` holds master's `ILocalNode` tests, `UA0027Tests.cs` holds the `NodeBrowser` ones. UA0024's help text also drops its reference to `ReadServerDiagnostics`, which #4193 removed as unused, and names `ISession.ReadDiagnostics` / `ISubscription.ReadDiagnostics` instead — those do still exist. ## 4. Two more locks reachable from outside their owner **`ApplicationConfiguration.PropertiesLock` is removed.** It returned the properties dictionary itself, so `lock (configuration.PropertiesLock)` was `lock (configuration.Properties)` — the lock *was* the data it guarded, the same shape as #4203. It had exactly one production consumer (`OAuth2CredentialCollection.Load`, a get-or-add with `ParseExtension` called inside the lock), so it could not simply be deleted. `Properties` is now a `ConcurrentDictionary` — same public type, every operation atomic on its own — and the one multi-operation need gets a member of its own: ```csharp public T GetOrAddProperty<T>(string key, Func<T> valueFactory); ``` It deliberately does **not** invoke the factory under a lock, so `ParseExtension` now runs outside any critical section. Holding a lock across a caller-supplied callback is exactly the shape that produced the attribute/diagnostics deadlock earlier in this work. The two existing tests asserted only `PropertiesLock is not null`, so they tested nothing but its presence. They are replaced by tests of the contract: factory runs once for a missing key, not at all for a present one, 32 concurrent callers all observe the same instance, null arguments throw. `PropertiesLock` ships in 1.5.378, so this is a breaking removal — migration guide section plus analyzer **UA0028**. **`SharedDefinitionCache` no longer hands out its lock.** Three sites in `SharedKeyValueSubscriptionStore` took the nested cache's `Lock` to operate on the cache's own dictionary. Both are private now, behind `ReplaceAll` and `CloneWhere`. `SnapshotCommitLock` stays exposed on purpose: it is a `SemaphoreSlim` sequencing an async commit across `await` points, which `System.Threading.Lock` cannot span, so folding it in would change behaviour rather than hide a lock. ## Verification Merged `master` (`0edfe237a`, including #4234) with no conflicts. - `dotnet build UA.slnx` — 0 warnings, 0 errors - `Opc.Ua.Server.Tests` — 4074 passed - `Opc.Ua.Types.Tests` — 8501 passed - `Opc.Ua.MigrationAnalyzer.Tests` — 151 passed on all five TFMs (138 → 151, the new ones being UA0027 and UA0028) - `Opc.Ua.Core.Tests` — 4249 passed - `Opc.Ua.Redundancy.Server.Tests` — 573 passed ## Note for the reviewer @romanett — your approval was given when this PR's diff was the 7 diagnostics files, because it was measured against `marcschier/node-browse-locks`. Now that the two are combined and the base is `master`, the diff is 23 files, and the extra 16 are the node/browser lock work that was #4221. GitHub kept the approval across the retarget, but it did not cover that code, so please re-review rather than let it ride. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e9c13dd0-a6e3-4ad2-8ad6-1c3fe6323041 Copilot-Session: b3c0e146-5b9e-4d08-a0f6-db3f940484b1 Copilot-Session: d18f87e9-2484-44ce-98ce-463ba6cf5f63 Copilot-Session: 0ee22c1d-c319-4257-bf78-a5a077a1824a	2026-08-11 22:14:10 +02:00
Marc Schier	5bc8a9d705	Reduce ISession: type the continuation points, delete the obsolete validate overload (#4199) ... ## Stacked on #4197 Base is `marcschier/server-bind-phase`. The full stack is **#4188 → #4193 → #4197 → this**. GitHub retargets each base automatically as the stack merges. **Review the earlier three first.** **#4217 has been merged into this branch** (squash `53dd4b618`), so the last remaining item from the issue — `ISession.SessionDiagnostics` — is now part of this PR. Closes #4187. Refs #4203 — **does not close it.** #4217 swapped `OnUpdateDiagnostics` onto `m_diagnosticsLock`, but it still calls `Variant.FromStructure(ServerDiagnostics)` with the default `copy: false`, which wraps the *live* structure in the `ExtensionObject`. The caller reads its fields after the lock is released, so the lock currently excludes nothing. #4222 adds `copy: true` and is what actually closes #4203. ## The finding that changed the approach `ISession` has **exactly one production adapter**: `src/Opc.Ua.Server/Session/Session.cs`. The other `ISession` implementations in the tree are the unrelated **client** interface (`src/Opc.Ua.Client/Session/Session.cs`, `RedundantClientSession`) and a migration-analyzer test stub. One adapter means the seam is hypothetical; two means it is real. The issue proposed splitting `ISession` into five concern-interfaces — that would turn one hypothetical seam into five. The interface earns its keep for testability and because node managers receive it through `OperationContext`, but that argues for making it **smaller**, not for multiplying it. **So: reduce, don't split.** Reasoning posted to the issue as [comment 5226712936](https://github.com/OPCFoundation/UA-.NETStandard/issues/4187#issuecomment-5226712936) so the five-way framing does not get re-proposed later. ## Measurement 38 members, 368 references. External production consumers are **4 files using 5 members**: | Consumer | Members used | |---|---| | `Opc.Ua.Redundancy.Server/Sessions/DistributedSessionManager.cs` | `Id`, `InitializeAsync`, `Dispose` | | `Opc.Ua.Di.Server/Locking/DefaultLockService.cs` | `Id` | | `samples/ConsoleReferenceServer/UAServer.cs` | `Id`, `Identity`, `ReadDiagnostics` | | `samples/Quickstarts.Servers/.../DataChangeMonitoredItem.cs` | type reference only | Every production caller of the five continuation-point members is inside `Opc.Ua.Server` (`MasterNodeManager`, `NodeManagerLifecycle`, `HistorianDispatcher`). **`ISession` is 38 → 32 members.** ## Commits ### `882b2937` Type the history continuation point `ISession` leaked `object` twice, against the repository rule: ```csharp void SaveHistoryContinuationPoint(Guid id, object continuationPoint); object? RestoreHistoryContinuationPoint(ByteString continuationPoint); ``` The erasure was **not** hiding a genuinely opaque plugin token. `HistorianDispatcher` is the only production caller, it always passes a `HistorianContinuationState`, and it casts straight back to that type on restore. The `object` was a round trip through the type system that bought nothing. `HistorianContinuationState` is `internal`, so it cannot appear on a public interface. `IHistoryContinuationPoint` names the concept instead: it carries an `Id` and extends `IDisposable`. Two latent bugs fell out of typing it: - The session disposed only those points that **happened** to implement `IDisposable` and **silently leaked the rest**. Every point is now disposable by construction, and three `as IDisposable` probes are gone. - Carrying the `Id` on the point removes the separate `Guid` parameter, which could disagree with the payload it was filed under. The mock stores in the historian tests had the same latent inconsistency. ### `0d020b77` Delete the obsolete synchronous `ValidateBeforeActivate` **Zero production callers** — the nine references were all tests written for it. `[Obsolete]` since 1.5.378, which the repository rules allow removing in master. It is also the divergent-signature problem the runtime analysis flagged: the async form returns the validated handler and policy as a tuple, while the sync one took the same two values as trailing `out` parameters, so a caller had to learn two shapes for one operation. The overload could not verify a user token requiring decryption, so on a secure endpoint it failed closed and told the caller to use the async path anyway. The one test asserting that fail-closed behaviour goes with the method it described; the path it pointed at is already covered by `ActivationValidationDecodesBinaryEncodedUserIdentityToken`. The other eight tests are **converted, not deleted** — the channel, signature, security-policy and token-policy validation they cover is still asserted, now against the path the server actually uses. ### `821e8302` Route the continuation points behind one member Five `ISession` members were one-line pass-throughs to `SessionContinuationPoints`, a 419-line module that already owns the behaviour and already has its own `IContinuationPointStore` seam. The interface restated the module's surface without adding anything. ```csharp // was session.SaveContinuationPoint(cp); session.RestoreHistoryContinuationPoint(bytes); // now session.ContinuationPoints.SaveBrowse(cp); session.ContinuationPoints.RestoreHistory(bytes); ``` The holder is `internal sealed`, so `ISessionContinuationPoints` is what makes it addressable. **This is composition, not the service locator `IServerContext` refused to be.** A subsystem with an independent lifetime should be injected; continuation points belong to the session and cannot exist without it. `InvalidateContinuationPoints` called `RemoveBrowseForManager` and `RemoveHistoryForManager` in sequence, so the pair becomes `RemoveForManager` — the caller's intent is that a node manager is going away, not that two collections need visiting. ## Tests Three harnesses had hand-rolled a `Dictionary<Guid, object>` to stand in for the holder. They now use the **real** one, so the dispatcher and `BrowseNext` meet the eviction and disposal rules a live session actually applies. **One hazard worth recording for future interface work.** A Moq member that is never stubbed returns `null`. Where the old code called a method straight on `ISession`, that `null` was harmless. Reaching *through* a property means an unstubbed `ContinuationPoints` null-derefs instead. Three tests failed exactly that way; they are wired to a real holder, not restubbed. | Suite | net10.0 | net48 | |---|---|---| | `Opc.Ua.Server.Tests` | 4032 passed, 1 failed | builds clean | | Touched areas (continuation points, historian, master node manager, session) | 898 passed | — | The single failure is `ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`, a local certificate-store problem verified to reproduce unchanged at HEAD with these changes stashed. Not caused by this PR. `Opc.Ua.Redundancy.Server`, `Opc.Ua.Di.Server`, `Quickstarts.Servers` and `ConsoleReferenceServer` all build clean. ## Docs `docs/MigrationGuide.md` gains a "Removed members on ISession" section covering the `ValidateBeforeActivate` removal with the async replacement, and the `IHistoryContinuationPoint` change with before/after. ## The SessionDiagnostics removal (folded in from #4217) `ISession.SessionDiagnostics` handed out the whole mutable `SessionDiagnosticsDataType` — the structure the session's diagnostics lock protects — so a caller could read a field while the owner was writing it. Every server-side reader wanted exactly one value out of it, so the interface now exposes those two values (`SessionName`, `ClientApplicationUri`) directly and the structure stays behind the lock. That work landed here as #4217 rather than as a separate PR, and it also fixes #4203: `OnUpdateDiagnostics` now takes `m_diagnosticsLock` instead of locking the diagnostics payload it was handed. ## Reviewer questions - `ISessionContinuationPoints` exposes `MaxBrowse` as read-only while the concrete holder keeps a setter (the test framework adjusts it to exercise limits). Should the setter be on the interface, or is test-only mutation better left to the concrete type? - `RemoveForManager` collapses two removal methods. The concrete `RemoveBrowseForManager`/`RemoveHistoryForManager` remain public on the holder — worth making them private now that nothing outside calls them individually? --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e9c13dd0-a6e3-4ad2-8ad6-1c3fe6323041	2026-08-11 06:57:40 +02:00
Marc Schier	9375a2431b	Retire the IServerInternal bind mutators and freeze the bind phase (#4197) ... ## Stacked on #4193 Base is `marcschier/server-context`, which is itself based on #4188. **Review #4188 and #4193 first.** GitHub retargets each base automatically as the stack merges. Closes #4184. ## What this does `IServerInternal` published twelve `Set*` binding methods plus `CreateServerObjectAsync`, so any holder of the interface could rewire a running server. This takes them off the interface and makes a late bind fail loudly. ## The audit changed the shape of the work The plan this came from said "design a frozen bind phase". **One already exists.** `StandardServer` has a seventeen-method `Create*` factory seam, and `DependencyInjectionStandardServer` overrides seven of those to resolve from the container: ```csharp protected override IRoleManager CreateRoleManager(IServerInternal server, ApplicationConfiguration configuration) => m_services.GetService<IRoleManager>() ?? base.CreateRoleManager(server, configuration); ``` `StandardServer` then carries each factory result into the datastore: ```csharp ServerInternal.SetRoleManager(CreateRoleManager(m_serverInternal, configuration)); ``` So an integrator already had two supported routes — override the virtual, or register in DI. `Set*` on the interface was a **third route that bypassed the ordering**, and the only one that let a caller rebind a running server. The job was therefore to stop publishing plumbing, not to build a phase. ## Measurement Every call site of the twelve mutators plus `CreateServerObjectAsync`: | Area | Count | Where | |---|---:|---| | `Opc.Ua.Server` | **11** | `StandardServer.cs:3729–3844`, one contiguous block | | samples | **1** | `ReferenceServer.cs:219`, inside a `CreateMasterNodeManager` override | | tests | 20 | `ServerInternalDataTests` (18), `SubscriptionTests` (2) — **all on the concrete `ServerInternalData`** | `SetIdentityRegistry` had **zero** call sites anywhere. The earlier estimate for this work was 268 files. That number counted files *referencing* `IServerInternal`, which this change never touches: the interface survives and only the mutators move off it. ## Commits ### `e719a88a` Add the missing `CreateUserManagement` factory seam The one gap, and the reason this had to land first. User management was the only subsystem with **no factory seam** — the only way to populate `IServerInternal.UserManagement` was `SetUserManagement`, which is what `ReferenceServer` called from inside a `CreateMasterNodeManager` override. Removing the mutator without a replacement would have broken that scenario with no migration path. The DI path at `OpcUaServerBuilderExtensions.cs:1178` builds a `UserNamePasswordAuthenticator` from a registered `IUserManagement` — a different thing entirely that never reaches the Part 18 §5 model on the address space. Purely additive. `ReferenceServer` moves onto the override. The test covers the **override** seam rather than the DI seam deliberately: registering an `IUserManagement` in the container also switches on the username/password authenticator, so a DI-based test exercises two unrelated paths and fails for reasons unrelated to binding. ### `cbcfda47` Remove `SetIdentityRegistry` Zero call sites, despite a doc comment advertising it as the integrator seam. The supported route has always been `ServerIdentityRegistryExtensions.RegisterDefaultAuthenticators`, which *adds* authenticators to the default registry rather than replacing it. `IdentityRegistry` is read five times outside `Opc.Ua.Server`, so the default is what every consumer already uses. **The analyzer confirmed the reading**: with the setter gone, `RCS1170` fired on the property — it is provably never reassigned after construction, so it is now read-only. ### `ea0937a2` Take the bind mutators off `IServerInternal` The mutators stay `public` on `ServerInternalData`, which owns the state. Only the interface loses them. The eleven call sites switch from the `ServerInternal` property to the `m_serverInternal` field, already typed `ServerInternalData?` — the same one-token change `GetStatus` and `EnterState` took in #4193. **No test changed.** All twenty test call sites already used the concrete type, which is the clearest evidence these were never interface-level API. `CS1574` found six doc comments across `IRoleManager`, `RoleManager`, `IUserManagement`, `UserManagement`, `IServerInternal` and `SessionManager` that pointed integrators at the mutators. They now point at the factory seam. ### `f3c4b9d3` Refuse to bind after the server has started Each bind goes through a guard; `StandardServer` closes the phase once the session manager is in place. A late `Set*` throws `ServiceResultException(BadInvalidState)` **naming the operation** and pointing at the factory seam — otherwise the failure is indistinguishable from a configuration problem. Rebinding *during* startup stays legal, which is what let both audit hazards resolve with no behaviour change: - **`SetNodeManager` also binds three more managers** (`DiagnosticsNodeManager`, `ConfigurationNodeManager`, `CoreNodeManager`). Intentional — they are reached through the master node manager and have no independent lifetime — and now documented rather than silent. - **`SetSessionManager` unhooks the previous manager's handlers**, which reads as written-for-repeat-calls. Nothing rebinds it, but the freeze does not make the path dead either, since binding twice during startup is still allowed. Left as-is; removing it is a separate question. ## Tests Four new tests on the freeze: the refusal itself across four representative mutators, that the message names the operation, that rebinding stays legal until the phase closes, and that closing twice is harmless. Plus one hosted-server test for the `CreateUserManagement` seam. | Suite | net10.0 | net48 | |---|---|---| | `Opc.Ua.Server.Tests` | 4033 passed, 1 failed | 230 passed, 1 failed (filtered) | | `Opc.Ua.Redundancy.Server.Tests` | 573 passed | — | | `Opc.Ua.Redundancy.Kubernetes.Tests` | 97 passed | n/a (net8/net10 only) | The single failure is `ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`, a local certificate-store problem verified to reproduce unchanged at HEAD with these changes stashed. Not caused by this PR. ## Docs `docs/MigrationGuide.md` gains a table mapping every removed mutator to the `Create*` override and the DI registration that replace it, plus the `SetIdentityRegistry` removal with its rationale. ## Net effect across #4193 and this PR `IServerInternal` goes from ~460 lines to **~200**. It no longer publishes locks, dead members, or the ability to rewire a running server. ## Reviewer questions - `CompleteBindPhase()` is public on `ServerInternalData` but deliberately **not** on `IServerInternal` — only the server owning the startup sequence should decide when binding is over. Reasonable, or should it be `internal`? - `SetSessionManager`'s unhook path is now unreachable in practice but still legal during startup. Leave it, or remove it and assert single-bind for that one subsystem? --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e9c13dd0-a6e3-4ad2-8ad6-1c3fe6323041 Copilot-Session: b3c0e146-5b9e-4d08-a0f6-db3f940484b1 Copilot-Session: d18f87e9-2484-44ce-98ce-463ba6cf5f63	2026-08-11 00:27:45 +02:00
Marc Schier	4e0b82af5f	Add IServerContext and move server startup tasks onto it (#4193) ... ## Stacked on #4188 Base is `marcschier/server-seam-analysis`, not `master`. `IServerInternal.cs` and `ServerInternalData.cs` both diverge from master on that branch (chunk 4c removed the two `object` locks), so branching off master would conflict on both files *and* design `IServerContext` against a stale interface. GitHub retargets the base to `master` automatically when #4188 merges. **Review #4188 first.** Closes part of #4184. ## What this does `IServerInternal` is a service locator: 56 members, 30-odd of which hand out one of 20+ subsystems. A component that only needs to observe the server and act on it has to take the whole thing, and no signature tells a reader which parts a component actually touches. This adds `IServerContext` — the ambient view of a running server — and moves `IServerStartupTask` onto it. ```csharp public interface IServerContext { ServerState CurrentState { get; } ServerObjectState ServerObject { get; } IServiceMessageContext MessageContext { get; } ServerSystemContext DefaultSystemContext { get; } ServerSystemContext CreateSystemContext(ISession session); T? FindPredefinedNode<T>(NodeId nodeId) where T : NodeState; IEnumerable<T> FindNodeManagers<T>() where T : class; ValueTask ReportEventAsync(IFilterTarget e, CancellationToken ct = default); ValueTask CloseSessionAsync(OperationContext context, NodeId sessionId, bool deleteSubscriptions, CancellationToken ct = default); ValueTask DeleteSubscriptionAsync(uint subscriptionId, CancellationToken ct = default); void UpdateServerDiagnostics(Action<ServerDiagnosticsSummaryDataType> update); } ``` Nothing on it returns a subsystem. A component that needs one takes it as a constructor dependency, which states the requirement instead of hiding it behind an ambient handle. `IServerInternal : IServerContext`, so every existing handle already satisfies the new seam and consumers migrate one at a time. ## The member set was measured, not guessed Every member of `IServerInternal` was counted by read site across `src`, `samples` and `tests`, split by area. `Ext` is non-test production consumers, which is what decides whether something belongs on an ambient interface. | Member | InServer | OtherSrc | Sample | Test | **Ext** | On `ISystemContext`? | |---|---:|---:|---:|---:|---:|---| | `NamespaceUris` | 63 | 50 | 34 | 131 | **84** | yes | | `Telemetry` | 58 | 23 | 35 | 5 | **58** | yes | | `MessageContext` | 2 | 8 | 7 | 11 | **15** | no | | `TypeTree` | 42 | 3 | 11 | 10 | **14** | yes, as `TypeTable` | | `NodeManager` | 66 | 7 | 6 | 74 | **13** | no | | `DefaultSystemContext` | 21 | 4 | 1 | 7 | **5** | no | | `IdentityRegistry` | 1 | 4 | 1 | 1 | **5** | no | | `SubscriptionManager` | 23 | 0 | 0 | 17 | **0** | no | | `SessionManager` | 21 | 0 | 0 | 15 | **0** | no | | `RequestManager` | 10 | 0 | 0 | 23 | **0** | no | | `AggregateManager` | 12 | 0 | 0 | 2 | **0** | no | | `ModellingRulesManager`, `ConformanceUnitsManager`, `DiagnosticsEnabled` | 0 | 0 | 0 | 0 | **0** | no | Two things fall out. The highest-traffic members are already carried by `ISystemContext`, which `ServerSystemContext` copies off the server at construction — so most of those 278 `NamespaceUris` reads are callers reaching through the server for something they already hold. And the subsystems with zero external demand are exactly the ones that should be injected rather than published. The check that matters: the eight `IServerStartupTask` implementations in `Opc.Ua.Redundancy.Server` are the heaviest real consumer of the ambient handle, and between them they touch **eight distinct members**. That is the shape the interface is cut to. ## Commits ### `e4a912eb` Remove seven unused members from `IServerInternal` The deletion test, applied before anything is designed against the interface. | Removed | Evidence | |---|---| | `CloseSession(OperationContext, NodeId, bool)` | zero call sites; sync-over-async; already obsolete | | `ReadServerDiagnostics<TResult>` | zero production sites — **added one commit earlier in #4188 for symmetry and never called** | | `ModellingRulesManager` | property never read; `ServerInternalData` owns the lifetime | | `ConformanceUnitsManager` | same | | `DiagnosticsEnabled` | façade over `IDiagnosticsNodeManager.DiagnosticsEnabled`, which is what callers use | | `Status` | one site, itself inside an obsolete method | | `ServerDiagnostics` | only reachable through the diagnostics methods that replaced it | `EndpointAddresses` was kept — `Opc.Ua.Di.Tests` uses it to resolve an endpoint URL. Thin, but real. The only build fallout was three lines in one test assembly. `Opc.Ua.Redundancy.Server`, `Opc.Ua.Redundancy.Kubernetes`, `Opc.Ua.Gds.Server.Common`, `Opc.Ua.WotCon.Server`, `Opc.Ua.PubSub.Server`, `Opc.Ua.Di.Server`, `ConsoleReferenceServer` and `Quickstarts.Servers` all built untouched. That is the evidence the members were dead. ### `ffcc80b9` Add `IServerContext` Additive. `ServerInternalData` implements it; no caller changes. ### `e8c4e15f` Hand startup tasks `IServerContext` Nine implementations, `OpcUaServerHostedService`, and the test call sites. ### `1b4de97a` Drop the `IServerStartupTask` migration-guide section Review feedback. `IServerStartupTask` does not exist in `master378`, so it is a 2.0-only concept and nobody migrating from 1.5.378 can own an implementation of it. The **Removed members on `IServerInternal`** section stays - those members *are* in master378 - and is now self-contained. ### `2ab7d689` Merge `marcschier/server-seam-analysis` Picks up the base branch including its merge of master. Textually clean, no conflicts. ### `6a650f12` Migrate `RobotIntentHostStartupTask` to `IServerContext` The merge was textually clean but **semantically broken**: master's Robot Intent feature (#4165) added a tenth `IServerStartupTask` implementation written against the old `IServerInternal` signature, so it no longer satisfied the interface. Its body walked `server.NodeManager.AsyncNodeManagers` and type-tested each entry for `RobotIntentNodeManager` - exactly the pattern `FindNodeManagers<T>()` replaces - so it migrated the same way as the other nine. `FindNodeManagers` also searches the synchronous registry and de-dupes by reference, so a manager registered on both faces still starts its intent controller hosts exactly once. ## Three findings from migrating real consumers **`MessageContext` is not derivable.** The design had callers use `DefaultSystemContext.AsMessageContext()`, on the reasoning that a message context can be built from a system context. It can't: that conversion carries the namespace tables and the factory but **not** `MaxStringLength`, `MaxArrayLength` or `MaxByteStringLength`. Four replication paths in `Opc.Ua.Redundancy.Server` would have silently fallen back to default decoding limits — a component quietly accepting more than the server is configured to. It is a real member, and the interface documents why. **`new`-shadowed members split the mock surface.** `IServerInternal` first redeclared `DefaultSystemContext` with a narrower type and `CurrentState` with a setter. That compiles, but a shadowed member is a *second interface slot*: production code calling through `IServerContext` read a slot that every existing `Mock<IServerInternal>` leaves unstubbed. Seventeen tests went from passing to `NullReferenceException` **while still compiling**. The narrower typing bought no encapsulation anyway — the object really is a `ServerSystemContext` and any caller can cast — so both shadows are gone. `CurrentState` is read-only on the ambient interface and the one site that assigns it uses the concrete type, as `GetStatus` already does. **`IServiceProvider` is deliberately *not* on `OnServerStartedAsync`.** The plan sketched it. All nine implementations already receive their dependencies through their constructors from the builder extensions, so a container on the callback would have replaced one service locator with another. ## `FindNodeManagers<T>` Two tasks walked `server.NodeManager.NodeManagers` and type-tested each entry for `ILocalAddressSpaceSource`; two more reached for a specific manager to cast it to a capability. Asking for the capability says what the caller wants; asking for the registry says how to find it. ```csharp // before foreach (INodeManager nodeManager in server.NodeManager.NodeManagers) { if (nodeManager is not ILocalAddressSpaceSource source) { continue; } ... } // after foreach (ILocalAddressSpaceSource source in server.FindNodeManagers<ILocalAddressSpaceSource>()) { ... } ``` It searches both the asynchronous and synchronous registries: `ToSyncNodeManager()` returns `this` when a manager already implements `INodeManager`, so the two lists overlap and a capability may sit on either face. ## Tests Mock harnesses are **wired, not stubbed** — they return the node managers and contexts the tests already build, so the tests keep asserting against real behaviour rather than against a mock that returns whatever is convenient. Four new tests cover `CreateSystemContext` (identity/locale propagation, null guard), the explicit `DefaultSystemContext` identity, and `FindPredefinedNode` before the address space is bound. | Suite | net10.0 | net48 | |---|---|---| | `Opc.Ua.Redundancy.Server.Tests` | 573 passed | 573 passed | | `Opc.Ua.Redundancy.Kubernetes.Tests` | 97 passed | n/a (net8/net10 only) | | `Opc.Ua.Server.Tests` | 4028 passed, 1 failed | 225 passed, 1 failed (filtered) | The single failure is `ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`, a local certificate-store problem. Verified it reproduces unchanged at HEAD with these changes stashed — it is not caused by this PR. All twelve dependent assemblies and samples build clean on both TFMs, zero warnings. ### Re-verified after the base merge (`6a650f12`) `Opc.Ua.Server`, `Redundancy.Server`, `Redundancy.Kubernetes`, `Robotics.Server`, `WotCon.Server`, `Di.Server`, `PubSub.Server`, `Gds.Server.Common` and `XRegistry.Server` all build clean on net10.0. `Opc.Ua.Robotics.Tests` 491 passed, `Opc.Ua.Redundancy.Server.Tests` 573 passed, `Opc.Ua.Redundancy.Kubernetes.Tests` 97 passed. ### CI status: 119 passed, 3 failed - none caused by this PR | Failing job | Why it is not this PR | |---|---| | `SourceGeneration_Core_Tests_windows` (net48) | `GenerateAndCompileDesignFileAsync(..., CSharp11, ...)` fails with `CS9058: Feature 'Lock object' is not available in C# 11.0` because generated `Opc.Ua.NodeStates.g.cs` emits `System.Threading.Lock`. **Identical failure on base #4188 (build 16737)**, which contains none of these commits. This PR touches no source-generation file. | | `SourceGeneration_Core_Tests_windows` (net10.0) | Same root cause; also failing on base #4188. | | `PubSub_Eth_Tests_windows` (net48) | Agent DNS outage: `NuGetToolInstaller` died with `ERR:getaddrinfo ENOTFOUND dist.nuget.org`. The job aborted before restore/build, so **the test step never executed**. Infra flake; needs a re-run, not a code change. | Both `SourceGeneration.Core` failures belong to the base branch and should be fixed there rather than papered over here. ## Correction to the recorded plan Chunk 7 (the frozen bind phase) was recorded as the largest mechanical footprint remaining and the one most likely to stall. It isn't: **each of the 12 `Set*` mutators has exactly one call site**, in the `StandardServer` wiring, plus one sample. It is a ~13 call-site change. The 268-file figure that produced that assessment counted files *referencing* `IServerInternal`, which chunk 7 never has to touch, because the interface survives and only the mutators move off it. ## Docs `docs/MigrationGuide.md` gains one section: the removed `IServerInternal` members with their replacements, plus the note that `MessageContext`, `DefaultSystemContext`, `CurrentState`, `ServerObject`, `ReportEventAsync`, `CloseSessionAsync`, `DeleteSubscriptionAsync` and `UpdateServerDiagnostics` moved *down* to `IServerContext` and stay reachable through `IServerInternal` unchanged, so no call site has to move. An earlier revision also documented the `IServerStartupTask` signature change there. That was dropped after review (1b4de97a4): `IServerStartupTask` does not exist in `master378`, so it is a 2.0-only concept and nobody migrating from 1.5.378 can own an implementation of it. The rationale for the ambient view - including the `AsMessageContext()` decoding-limit trap - lives on the `IServerContext` and `IServerStartupTask` XML docs instead, which is where a 2.0-only concept belongs. ## Reviewer questions - `IServerContext` is 11 members, not the 9 the design targeted. `MessageContext` and `FindNodeManagers<T>` were added because migrating real consumers showed the alternatives were wrong (silent decoding-limit loss) or a pass-through (`NodeManager.NodeManagers` plus a type test at every call site). Reasonable? - `IServerInternal : IServerContext` keeps the fat interface alive on purpose — it is what makes this a signature change rather than a 268-file rewrite. It only shrinks once the DI migration lands. The 6a deletions are unconditional, so even a full stall leaves it seven members smaller. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e9c13dd0-a6e3-4ad2-8ad6-1c3fe6323041 Copilot-Session: b3c0e146-5b9e-4d08-a0f6-db3f940484b1 Copilot-Session: d18f87e9-2484-44ce-98ce-463ba6cf5f63	2026-08-10 21:14:09 +02:00
Marc Schier	e05a820c91	Removal of the exposed diagnostics and node locks (#4188) ... ## What this is Design analysis for `Opc.Ua.Server`, the two rounds of design exploration behind it, and the first two implementation chunks it recommends. Draft: the analyses are ready to read, but the *conclusions* deserve discussion before the larger phases proceed. Later chunks stack onto this PR as they land. | File | Kind | Lines | |---|---|---:| | `plans/NodeManagerAnalysis.md` | Evidence — the node-manager plugin surface | 488 | | `plans/ServerRuntimeAnalysis.md` | Evidence — the server runtime cluster | 576 | | `plans/ServerDesignExploration.md` | Design round 2 — delivery strategy | 1,394 | | `plans/ServerDesignExplorationAlt.md` | Design round 1 — four interface designs | 2,119 | | `src/Opc.Ua.Types/State/NodeState.cs` | **Code** — node self-guards attributes | | | `src/Opc.Ua.Types/State/BaseVariableState.cs` | **Code** — value accessors take the lock | | | `src/Opc.Ua.Server/NodeManager/AsyncCustomNodeManager.cs` | **Code** — 6 external locks removed | | | `tests/…/NodeStateAttributeConcurrencyTests.cs` | **Code** — concurrency safety net | 449 | `docs/` is untouched — it holds shipped feature documentation, and none of this is shipped API. ## The analyses Counted mechanically over `src`, `samples` and `tests`; **every figure is reproducible** from commands printed in each document. **Node manager surface.** `AsyncCustomNodeManager` declares **93 `virtual`/`abstract` members** (80 distinct names). Across the repository: **112 override sites, 25 distinct names ever overridden, 55 of 80 never**. Six carry nearly all real use. For scale, `ReferenceNodeManager` is 6,373 lines and overrides **8**. **Server runtime.** Seven findings, **two of which recommend no change**. Findings 1–5 are filed as #4183–#4187. The headline is an `object`-typed sync root published on three public interfaces and taken in **88 `lock` statements across 7 files**, including a shipped sample. **What is deliberately not flagged.** `IStandardServer` — 9 members over 5,575 lines, 16 of 37 virtuals used — is identified as the reference shape to preserve. `IApplicationInstance` and the already-`[Obsolete]` `Stop()` are examined and left alone. ## The design exploration **Round 1** explored four alternative interfaces in parallel under the current back-compat constraint. Its value is the convergence: four designers who never saw each other's work independently arrived at a 6–9 member `IServerContext` replacing the 57-member locator, removal of the `object` node-manager handle, elimination of the `Processed` protocol, and `IHistorianProvider` as the pattern to copy. **Round 2** re-runs it with back-compat lifted and asks how to deliver: full rewrite versus side-by-side fork. **Both option designers independently declined to recommend their own assignment.** Three structural facts were verified: 1. **The fork line already sits at `IAsyncNodeManager`** — old and new node managers can coexist with zero runtime changes, so a fork duplicates ~10% of the assembly, not all of it. 2. **`[OpcUaShim]` cannot shim an inheritance surface** — honest auto-fix ceiling for the authoring surface is **~10%**, against ~70% for the shipped analyzers. 3. **A second lock leak exists** — `NodeState` takes `lock (this)` under a `CA2002` suppression whose own comment documents the contract. Recommendation: **neither pure option**, but a staged path reaching the rewrite end state incrementally, gated on migrating the five framework-internal node managers that today make it impossible for the old base to ever freeze. ## Implementation: NodeState now guards its own attributes This removes the prose-only contract in which `NodeState` took `lock(this)` and every external caller was required to take `lock(source)` on the node instance. The code already carried a TODO describing exactly this fix and noting it required updating every external site. - The node self-guards rather than relying on callers. `BaseVariableState` gains a **fully private** `System.Threading.Lock` covering its value fields; `NodeState`'s other collections keep their existing private locks. - All **6** `lock(this)` sites and both `CA2002`/`RCS1059` suppressions are gone. - `Value`, `Timestamp` and `StatusCode` on `BaseVariableState` take the lock too — without that, a direct property write raced an attribute read once the external lock was gone. - **6 of 9** external sites removed. Five were already redundant (`GetReferences`, `GetChildren`, `ReferenceExists` and `SetAreEventsMonitored` guard themselves; `FindMethod` reaches its state via `GetChildren`); the sixth was an attribute read now covered by the node's own lock. - Attribute read/write deliberately take **no** node-wide lock: both dispatch to caller-supplied handlers (`OnReadValue` and siblings) that may take locks of their own, and holding a lock on the node across one gives the node no fixed place in the lock order. - **3 deliberately retained and annotated in code**: two `CreateBrowser` calls, which are not internally synchronized (tracked in #4216), and one in `CustomNodeManager`, whose class is slated for deletion so refactoring its lock would be wasted work. ### No lock is exposed anywhere, to anyone An earlier revision put the attribute lock on `NodeState` as `private protected`, which still published a lock-sharing contract on the inheritance surface. Review pushed back and it is gone: `BaseVariableState` was the only user, so the lock moved there as a plain `private readonly Lock`. There is no member for a derived type to call, in this assembly or any other. `BaseVariableValue` was the last public leak — it handed out the lock guarding its value through a `public Lock` property, so any caller could hold it across a call back into the stack. That property is removed: - Derived value classes (what the source generator emits for every structure variable) synchronize through a `protected EnterLock()` / `ExitLock()` pair. Callback helpers were the first choice, but the generated bodies pass `ref Variant` / `ref StatusCode` / `ref DateTimeUtc`, and `ref` arguments cannot be captured by a lambda. The lock *object* stays private either way. - `NodeStateTemplates` emits the new form. - `ServerInternalData`, the one cross-assembly caller, was reading the lock back off a value it had itself constructed with that lock. It now takes `m_diagnosticsLock` directly. `NodeBrowser.DataLock` remains `protected` and is tracked in #4215: a subclass extending `Next()` legitimately needs its enumeration state atomic with the base browser's, so the fix needs a base-owned acquisition rather than a deletion. ### Migration coverage for the removed lock members None of the removed lock members get an `[Obsolete]` shim, and the migration guide now says why: a shim would have to be `object`-typed to keep old call sites compiling, and on .NET 9+ `lock (o)` on a statically-`object` reference is `Monitor.Enter`, which does **not** interlock with `Lock.Enter()`. It would compile, run, and silently synchronize nothing — strictly worse than a compile error. Re-adding a member to a downstream-implemented interface is also a break in its own right on `net48` / `netstandard2.0`, where there are no default interface methods. Instead, every removed member has a named replacement on the migration path: | Rule | Covers | Replacement it names | |---|---|---| | `UA0024` | `IServerInternal` / `ISession` / `ISubscription` `DiagnosticsLock`, `DiagnosticsWriteLock` | `UpdateServerDiagnostics(...)` / `UpdateDiagnostics(...)`, `ReadServerDiagnostics` / `ReadDiagnostics` | | `UA0025` | `ILocalNode.DataLock`, `Node.DataLock` | a lock the caller owns; the node guards its own state | | `UA0026` | `BaseVariableValue.Lock` | a lock passed to the `BaseVariableValue` constructor | Each rule fires on sources still written against 1.5.378, where the compiler only says a member is missing. `docs/MigrationGuide.md` gains a section per removal with before/after. ### The tests earned their keep The four concurrency tests were landed **first**, against the unmodified code, because removing a synchronization contract cannot be verified by the compiler and the existing `NodeStateHandlerConcurrencyTests` covers a different concern (racing a handler assignment against a write, not reader/writer exclusion). They now run with **no external locking**, which is what demonstrates the node guards itself. They caught two real problems, and one of them was mine: - The `Value`/`StatusCode`/`Timestamp` setters bypassed the new lock — a genuine race. - One test asserted an invariant the design does not offer. Two property setters are two lock acquisitions, so that pair was **never** atomic; it only passed originally because the test held a lock across both. It now uses a single attribute write carrying a whole `DataValue`, which genuinely is atomic and is the path the server uses. A third issue was found and fixed before it shipped: the reader loops gated on `!writer.IsCompleted`, so a reader scheduled after the writer finished performed zero reads — failing two runs in three on `net48`. Converted to `do…while`; now five consecutive green runs. **Verified:** `Stack.State` 116 passed on `net10.0` *and* `net48`; the concurrency category green 5/5 consecutive on `net48`; `Opc.Ua.Types.Tests` 8,456 passed; the server node-manager suite 1,205 passed, including `NodeManagerLifecycleTests` and `AsyncCustomNodeManagerTests` which exercise the changed paths. ## Two corrections to my own published figures Both were found while starting implementation, and both are recorded on #4183. **`ForceDiagnosticsScan()` was overstated.** The analysis claimed the `DiagnosticsWriteLock` getter *"triggers a full address-space diagnostics rescan"*. It is in fact a single flag assignment (`DiagnosticsNodeManager.cs:700-703`) consumed by the read paths and the periodic timer. The defect stands — a side-effecting property getter with a racy write on a synchronisation path — but its cost was misstated. **The contract was 15 sites, not 19.** A naive search matched ten comment and `#pragma` lines that *mention* `lock(source)` while explaining it. The documents now carry a counting command that excludes comment matches. A third finding expanded scope the other way: the `object`-typed lock leak is **repo-wide**, with ten such members across `Opc.Ua.Core`, `Opc.Ua.Types` and `Opc.Ua.Server` — including `Node.DataLock => this` and `ApplicationConfiguration.PropertiesLock => m_properties`, the same defect class as `Subscription.DiagnosticsLock`. ## Caveats stated in the documents - **Nothing in the designs is implemented or benchmarked.** Every performance claim across all seven designs is a design intention, not a measurement. - The larger phases are a **major undertaking against a live plugin API**; the documents say so rather than letting the neatness of the proposals imply otherwise. - The recommended sequencing puts low-risk, self-contained steps first and notes they are **worth doing regardless** of which strategy is eventually chosen. - Round 2 notes the two option designers' cost estimates are **not independent of their assignments**. ## Review guidance The two analyses make factual claims checkable by running the included commands. The design explorations are exploratory and need not be agreed with to merge. The code change is small, test-backed, and independent of which delivery strategy is eventually chosen. Reasonable outcomes include merging as-is, merging only the analyses and the lock fix, or disagreeing with the recommendation while keeping the evidence. --- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e9c13dd0-a6e3-4ad2-8ad6-1c3fe6323041 Copilot-Session: d18f87e9-2484-44ce-98ce-463ba6cf5f63	2026-08-10 06:47:50 +02:00
Marc Schier	0612a43fe4	Pluggable cryptography and hardware-held private keys (#4192) ... Makes the stack's cryptographic operations replaceable - by another library, an offboard service, or hardware (TPM 2.0 / HSM / PKCS#11 token / cloud KMS) - with no performance cost in the default configuration, and extends this to certificates so a private key need never exist in process memory. Also fixes three pre-existing defects found while validating: STACKGEN001 erroring for every compilation the stack generator was merely loaded into, certificates failing to load from paths past MAX_PATH on Windows, and the MonitoredNode2 tests starving the thread pool. Fixes #4190	2026-08-09 12:40:26 +02:00
Marc Schier	fb80bfdc82	[WotCon] Track WoT Connectivity/Binding 1.1 and add projections (Section 12) (#4182) ... Follow-up to #4128, which merged the WoT Connectivity 1.1 work against **draft2** of the companion specifications. Both drafts have moved since, and this branch tracks them through to **draft3**, adds the projection feature they introduced, and closes the gaps a bidirectional re-audit found. The specifications are the private [`OPCF-Members/spec-drafts`](https://github.com/OPCF-Members/spec-drafts) drafts of *OPC UA — WoT Connectivity 1.1* and *OPC UA — WoT Binding 1.1*. They are still drafts, which is why this is opened as a draft PR — see [Status](#status) at the bottom. ## What changed in the specifications Two revisions landed upstream while #4128 was in flight. **draft2** replaced the `eventGroup` / `actionGroup` grouping constructs with a single **projection / view** concept, added fifteen Section 6 vocabulary terms, and made two identifier rules normative. **draft3** ([spec PR #8](https://github.com/OPCF-Members/spec-drafts/pull/8)) un-deprecated the incorporated OPC 10100-1 v1.02 surface, restructured the conformance clause into three profiles, and tightened two security obligations. The draft3 NodeSet is **structurally identical** to draft2 — I diffed all 286 nodes node-by-node. Every change is metadata: 96 `ReleaseStatus="Deprecated"` attributes removed, `PublicationDate` advanced, and `WOTC-ProjectionMaterialization` claimed by three Type Nodes that previously implemented it in name only. ## Projections (WoT Binding Section 12) The substantial addition. A projection document selects affordances from other Thing Descriptions or Thing Models and resolves to a document of its own; materialized, it becomes an OPC UA `View` that **creates no affordance Node** and only `Organizes` what its sources already materialized. | Piece | Where | |---|---| | Document model and validation | `Opc.Ua.Types/Wot/WotProjection.cs`, `WotProjectionModel.cs` | | Four-stage resolution engine | `Opc.Ua.Types/Wot/WotProjectionResolver.cs` | | Materialization plan | `Opc.Ua.WotCon.Server/Materialization/WotViewProjectionPlan.cs`, `WotProjectionViewBuilder.cs` | | Live address-space host | `WotProjectionViewNodeManager.cs`, `LifecycleWotViewProjectionHost.cs`, `IWotViewProjectionHost.cs` | | Node lookup seam | `IWotMaterializedNodeIndex.cs` | Groups reached by `ua:Organizes` become organizational Objects rather than nested Views, only the outermost materialization is a `View`, and a member whose source is served by another server is omitted and reported rather than recreated — the federation case the clause calls out. ### How this is validated The specification publishes twenty worked examples, two of which are a **golden pair**: a projection document and the resolved view it is defined to produce. `WotSpecExampleTests` embeds all twenty and runs the pair through the resolver, asserting against *the specification's own expected output* rather than against my reading of the prose. That single document exercises all three selection forms, the bulk naming rule, the security closure naming and the provenance term. This is the strongest evidence in the PR, and it is worth more than the unit tests around it: it is the same check the upstream `validate_local.py` performs. ## Other spec work - **Section 6 vocabulary** — fifteen missing model and platform terms with their validation rules (`WotNodeSetConverter.ModelVocabulary.cs`). - **Portable identifiers** — the two rules release 1.1 made normative, with `WotNodeSetConverterOptions.AllowNonPortableIdentifiers` as a documented downgrade-to-warning switch for documents authored against v1.00. Defaults to `false`. - **xRegistry §6.9 identifier construction** (`XRegistryIdentifier.cs`) — reverse-authority symbolic identifiers; all seven published worked examples are test cases. ## Gaps found by re-auditing against draft3 I audited in both directions — every normative requirement to code, and every `uav:` term in code back to a spec that defines it. Three findings, all fixed: 1. **`Wot-Con 1.02` accepted anything well-formed.** draft3 makes format validation part of the unit, because *WoT-Con Minimal* is one unit wide and materialization projects a *valid* document. The 1.02 upload path only deserialized into the `ThingDescription` POCO, which happily accepts JSON that is not a WoT document at all — every member of that type is optional, so `{}` deserialized and was materialized. It now parses via `WotDocument.Parse` and requires the document to identify itself by a non-empty `name` or `title`, returning `Bad_DecodingError` and materializing nothing otherwise. The rule lives in `ThingDescriptionFormatValidator` and is shared with `CreateAssetForEndpoint`, whose auto-generated Thing Descriptions §11 also treats as untrusted input. 2. **Two `WoTFile` operations were ungated.** draft3 §11 names `Write` and `CloseAndUpdate` alongside the five management methods; both reached the materializer without an access check. `EnforceManagementAccess` is now wired into `WotAssetFileManager`. 3. **Two invented `uav:` terms.** `uav:NodeSet2Preservation` was declared and never used — removed. `uav:eventFields` is read by `OpcUaBindingPlanner` and defined in *neither* spec, while a comment claimed it came from OPC 10101. It still works for compatibility but is now documented as a non-standard extension at both sites rather than misattributed. The SSRF prohibition is satisfied by construction: there is no HTTP client anywhere in these assemblies, and all resolution goes through the injected `IWotThingResolver`. ## Two ambiguities fixed in the specifications themselves The audit turned up two places where a conforming implementer had to invent an answer. Rather than pick silently, I took both upstream — they are pushed to [spec-drafts PR #8](https://github.com/OPCF-Members/spec-drafts/pull/8) (all checks green) and are now normative in *Binding* §12.6 and *Connectivity* §7.13: - **`ViewVersion`** had one sentence saying only *that* it changes. It now changes iff the resolved membership changes, is updated at commit time alongside the `NodeVersion` stamping, and is a deterministic function of the membership alone in a canonical order — so two servers that resolved the same membership agree, which a per-server counter would not across a redundant pair. Explicitly **not monotonic**: compare for inequality only. - **"Type-level view"** was used once in §12.2 and defined nowhere, while every other statement of materialization was universal. §12.6 now *defines* it rather than contradicting §12.2: the NodeClass is `View` in both cases. Both were already what this code did, so neither changed behaviour — but the order-insensitivity the clause added was **not** covered by any test, and `ViewVersionIsUnchangedWhenOnlyTheOrderOfTheMembershipChanges` now holds it. It is mutation-verified: removing the ordinal sort turns it red. `docs/WotBindings.md` no longer lists these as points "the specification leaves open". ## Validation | Suite | Result | |---|---| | `Opc.Ua.Types.Tests` | 8525 ✅ (8518 on net48) | | `Opc.Ua.WotCon.Tests` | 1057 ✅ (net10.0 and net48) | | `Opc.Ua.WotCon.Bindings.Tests` | 559 ✅ | | `Opc.Ua.XRegistry.Tests` | 196 ✅ | | `Opc.Ua.Server.Tests` | 4027 ✅ | | `Opc.Ua.SourceGeneration.Core.Tests` | 3771 ✅ | | `Opc.Ua.WotCon.Samples.Tests` | 9 ✅ | The full `UA.slnx` builds with **0 errors and 0 warnings attributable to this PR** on all 6 TFMs (`TreatWarningsAsErrors` is on). `Opc.Ua.Server.Tests` and the source-generation suites are listed because this PR now also touches `MasterNodeManager` and the WoT-Con NodeSet. ## Compatibility No public API is removed or broken. Everything added is new surface using `ArrayOf<T>` / `ByteString` per the repo conventions, wired for DI with a direct-construction fallback, and the one behavioural tightening (`Wot-Con 1.02` format validation) rejects input that was never valid to begin with. One pre-existing issue is **not** addressed here because it came in with #4128 and is out of scope: `WotAssetFileManager.OnCloseAndUpdate` uses `.GetAwaiter().GetResult()`, which the repo rules ban. Worth a follow-up. ## Landed since this PR was opened - **DI upgraded to 1.05.0.** `ConnectsTo` was declared a subtype of `HierarchicalReferences`, contradicting OPC 10000-100 §5.5 Table 48. That is an upstream error in every official DI NodeSet up to 1.04, corrected by the OPC Foundation in 1.05.0, so the fixture is replaced with the verbatim 1.05.0 file rather than hand-patched. A guard test pins the corrected form. This also closed a second gap: six DI types the WoT model was missing are 1.05 additions. - **The sample Pump has the type definition OPC 40223 declares for it.** `Pump1.Identification` carried DI's `FunctionalGroupType` instead of `PumpIdentificationType`. Every test, the sample documents and the README agreed with each other about it, which is why it survived. - **`WotPumpAddressSpaceComparisonTests`** compares this server against `PumpDeviceIntegrationServer`, which builds the same OPC 40223 Pump from generated NodeSets and hand-written wiring. An independent oracle rather than a restatement of the same assumptions - it is what found the defect above. - **Error containment in `MaterializeProjectionViewsAsync`.** An exception mid-loop discarded the applied View handles before they were published, leaving View Nodes in the address space that nothing could remove. - **`HasWoTProjection`** is now created; it was documented and plumbed but never wired. - **The default in-memory View host** no longer deletes the View it has just applied. - **An undeclared NodeSet alias now says so.** It previously surfaced as `IdentifierMissing`, naming neither the value nor the fact that an alias was expected. - **`ViewVersion` follows the algorithm the specification now states.** My earlier spec text was rejected on review, correctly: it named neither the function nor the canonical order it demanded, ignored that OPC 10000-3 §5.4 requires a value greater than zero, and its "when, and only when" is unsatisfiable for a 32-bit value. §12.6 now specifies the algorithm outright and this implementation follows it exactly, pinned by an independently computed test vector. Implementing it surfaced a further defect in the clause - joining members on U+000A is not injective, because a NodeId string identifier may itself contain U+000A - which is fixed in the specification by length-prefixing each member. - **An auto-generated Thing Description is treated as untrusted input.** §11 requires it; `CreateAssetForEndpoint` validated the endpoint but handed whatever the discovery provider returned straight to the materializer. The `Wot-Con 1.02` format rule now gates both paths from one place. - **Document bytes moved out of the registry snapshot** into a content-addressed store, so a commit no longer rewrites the whole corpus to edit one document. Writes stage and are promoted during the commit, which preserves both durability-first ordering and the fail-closed rule that a blob directory without a manifest means operator recovery. Restoring streaming digest verification along the way closed a real gap: content integrity had degenerated to a length comparison. - **External references now reach a NodeManager registered later.** A reference to a Node that does not exist yet is dropped rather than queued, and the dynamic-registration path had no second phase to apply them in, so registering a NodeSet before the one it points into lost the forward edge permanently. Review of that change then found a state leak in its rollback path, fixed with a red-to-green regression test. - **`WOTC-Legacy` is renamed `Wot-Con 1.02`** in the specification and here. Editorial: the NodeSet diff is `<Category>` values only. ## Status Open for review. Two things may still move: 1. **The specifications are unpublished drafts.** Anything here can change if the working group decides differently. The `ViewVersion` definition has since been settled - §12.6 specifies the algorithm and this follows it - but the same could happen to any other clause tracked here. 2. **`uav:eventFields`** is kept as a documented non-standard extension. That is a judgement call the WG may want to reverse, either by adopting the term or removing it. Two known items are recorded rather than fixed: - `Pump1` has no hierarchical parent, so it is reachable by NodeId but not by browsing down from `Objects`. The cross-NodeSet ordering defect this was originally attributed to is fixed above, and cross-manager references are now symmetric in both registration orders. The Pump itself is still blocked on a different cause found while verifying that fix: with the reference added, `DeviceSet` exists but the Pump has **zero** inverse references, so the edge is dropped at *import* - before any external-reference machinery runs. That is a separate defect in the materialization path, not the one that was fixed, and it is left recorded rather than guessed at. - `WotAssetFileManager.OnCloseAndUpdate` uses `.GetAwaiter().GetResult()`, which the repo rules ban. Pre-existing from #4128 and out of scope here. Review of the projection resolver and the materialization host is the most valuable thing to look at; the vocabulary and identifier commits are mechanical by comparison. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8	2026-08-08 09:08:23 +02:00
Marc Schier	416d619c41	Split the CI matrix across both systems, split the required test gate from the advisory coverage check, and give background work an owner (#4158) ... # Description Two changes to how CI is routed and how it reports. ## 1. Split the CI matrix back across both systems Flips both halves of the CI backend switch from `ado` to `actions`, so the workload is shared instead of concentrated on the managed pool. The switch is checked into source in **two places that must always be flipped together**: | File | Setting | Before | After | | --- | --- | --- | --- | | `azure-pipelines.yml` | `parameters.ciBuildBackend` | `ado` | `actions` | | `.github/workflows/buildandtest.yml` | `env.CI_BUILD_BACKEND` | `ado` | `actions` | | Work | GitHub Actions | Azure Pipelines (managed pool) | | --- | --- | --- | | All-TFM solution builds | `build-all-tfm-windows` + `-linux` | two-TFM PR gate (`net10.0`, `net48`) | | Test matrix | ubuntu, every test project | Windows `net48` + `net10.0` fast PR legs | | Native AoT | ubuntu | Schedule/Manual only | | macOS | always (no managed-pool macOS image) | — | ## 2. Split the required gate from the advisory coverage check The coverage gate job was appended to *every* test stage, so `Fast PR test` emitted three sibling checks whose display names collided — `Coverage Release (net10.0)` appeared twice, once for the Windows leg and once for macOS, with nothing to tell them apart: ```text ✓ Coverage Release (net48) 55s ● Coverage Release (net10.0) ○ Coverage Release (net10.0) ``` That job also fused two unrelated concerns: whether the tests passed (which must block a merge) and whether coverage met its thresholds (which should not — the `net10.0` leg had `enforceCoverage: true`, so a coverage dip blocked merges). Both concerns now get a stage of their own, mirrored in both CI systems: | Concern | Azure Pipelines | GitHub Actions | In the branch ruleset? | | --- | --- | --- | --- | | Every test passed | **`Tests passed`** stage | **`build-and-test summary`** job | **Yes — required** | | Coverage meets thresholds | **`Code coverage`** stage | **`code coverage`** job | **No — advisory** | The coverage check reports a clean failure when the thresholds are missed, so the miss is visible, but it never blocks the merge. Coverage is now merged **once** across every leg of the run instead of once per leg, which also removes the `enforceCoverage` / `publishCoverage` "exactly one invocation may set this" parameters. Neither check re-runs any tests — the previous design re-ran the whole suite in one job with the collector attached, which serialised a suite that is deliberately fanned out and always exceeded the stage timeout. ### Coverage checks now exist in GitHub Actions `build-and-test` already ran with `--collect:"XPlat Code Coverage"` and uploaded the fragments, but nothing ever merged or evaluated them. It now runs the same `check-coverage.ps1` against the same `coverage-thresholds.json` as Azure. Since `CI_BUILD_BACKEND` defaults to `actions`, that ubuntu matrix — which runs *every* test project — is the fullest coverage signal in the repository. ### Reporting the numbers `check-coverage.ps1` gained `-SummaryPath`, rendering a markdown report that both systems surface: - **GitHub Actions** — appended to the job summary and upserted as a single **sticky pull-request comment**, updated in place on each run. Threshold misses also appear as `::error::` annotations. - **Azure Pipelines** — attached to the build summary via `##vso[task.uploadsummary]`. The report is a pass/fail table of the project floor, patch coverage and baseline delta, with the uncovered changed lines listed by file in a collapsible block. Rendered example: > ## Code coverage > > ❌ **Coverage gate failed.** This check is advisory and does not block the merge. > > | Check | Result | Threshold | > | --- | --- | --- | > | ✅ Project line rate | **72.10%** (40 731/56 496 lines) | >= 70.00% | > | ❌ Patch coverage | **0.00%** (0/1 changed lines) | >= 75.00% | The comment is fork-safe: a `pull_request` from a fork gets a read-only token regardless of the `permissions:` block, so it degrades to the job summary alone. The sticky comment uses `actions/github-script` rather than a community action, keeping this workflow on first-party actions only. ## Notable correctness details These were found and fixed during review, and are the parts most worth a second pair of eyes: - **`dependencies.<stage>.result` resolves at *stage* scope only.** At job scope `dependencies` refers to other jobs of the same stage, so reading stage results from a jobs template would have silently returned empty and the gate would have passed everything. `gate.yml` and `coverage.yml` are therefore **stage** templates that derive their own `dependsOn` from the same stage list they evaluate, so the two can never drift apart. An unresolved result is a hard error. - **Both rollups run on `not(canceled())` / `always()`**, not on success. A check that reports **skipped** is treated by GitHub as **satisfied**, so a required check that skips when its dependency fails would wave a red build straight through. - **`get-matrix.ps1` used to emit an empty matrix on a discovery miss.** The downstream job is then skipped, a skipped job rolls up as a *succeeded* stage, and the gate would have approved a run that executed no tests. It now fails, with an `-AllowEmpty` opt-out. - **Zero coverage reports used to be a green advisory check**, which would hide a broken collector indefinitely. Both systems now distinguish "the tests never ran" (warning) from "the tests ran and published nothing" (error). - The gate reads stage-level results rather than enumerating matrix-generated job names, which change whenever a test project or agent is added. ## Test-host hang and the reconnect defect behind it — both fixed Flipping `CI_BUILD_BACKEND` to `actions` started exercising the ubuntu matrix, which `master` never runs, and it immediately surfaced two real defects. **1. The test host hung for ten minutes.** `Assert.ThrowsAsync` blocks the calling thread with no timeout, and these tests drive a *fake clock* from that same thread — so once the reconnect parked on a fake-clock timer, nothing could ever advance it. NUnit sat in `WaitForCompletion` until `--blame-hang-timeout` killed the host, aborting the run at 3902 of 4238 tests and taking every other test in the assembly down with it. A hang dump named the culprit exactly. Replaced with an awaited, bounded `AssertThrowsAsync` helper (the repo's own rules forbid sync over async). The run now completes in **~1m29s** with all tests executed. **2. A budget-aware reconnect never returned.** With the hang gone, two tests failed with `Expected ServiceResultException but got TimeoutException`. `dumpasync` on a hang dump gave the whole chain: ``` ReconnectAsyncWithBudgetShrinksDelayToFitRemainingAsync → AssertThrowsAsync → ClientChannelManager.ReconnectAsync → ClientChannelManager.ReconnectLeaseAsync → ClientChannelManager.SwapFaultedEntryAsync ← parked here ``` `master` (via #4128) added `IsTerminalReconnectRace` so a reconnect that loses a race against a concurrent close recovers on a freshly swapped entry. The predicate only asks whether the entry ended up `Closed`/`Faulted` with `BadSecureChannelClosed` — which is *also* exactly how a reconnect ends when the caller's retry budget or the reconnect policy is deliberately exhausted. So a budget-exhausted reconnect was misread as a race: the manager swapped the entry and started a **second, unbudgeted** cycle, waiting out the swap back-off first. `ReconnectAsync(channel, budget)` therefore could not return within its budget, and under a fake clock it never returned at all. The fix requires the budget to have room left before treating a terminal failure as a race: ```csharp return !ct.IsCancellationRequested && !budget.IsExhausted && sre.StatusCode == StatusCodes.BadSecureChannelClosed && entry.State is ChannelState.Closed or ChannelState.Faulted; ``` The parameterless `ReconnectAsync(channel, ct)` overload passes an unlimited budget, so genuine race recovery is unchanged. Verified locally: Core 4152, Client 2122 and Sessions 776 tests all pass, and the full GitHub Actions PR run is green across 56 jobs. Two smaller fixes came along the way, neither weakening an assertion: - `ObservableFakeTimeProvider` waiters made **relative** ("N more timers from now") rather than absolute ("the Nth timer of the run"). Absolute numbering let an earlier reconnect consume the slots, so a waiter completed before the awaited timer existed and `Advance()` fired nothing. - The raw worker thread in `NodeStateHandlerConcurrencyTests` was a **foreground** thread whose stop flag was read without a barrier and set only on the success path — a failed assertion leaked a spinning thread that pins the process at exit. Now background, `Volatile`, stopped in a `finally`, joined with a bound. ## Background work now has an owner The reconnect hang above was one symptom of a pattern the codebase had **26 times** across seven assemblies: a bare `_ = Task.Run(...)`. Each one handed work to the thread pool and forgot it — nothing observed the exception if it threw, nothing bounded how many ran at once, and disposal raced them, so a component could finish tearing itself down while work it started was still touching its fields. Two of those sites already carried a standing `// TODO: Await the task completion in shutdown`. `BackgroundTaskScope` (in `Opc.Ua.Types`) gives them an owner: schedule with `Run`, drain with `DisposeAsync`, or signal-only with `Dispose` where the owner's teardown is synchronous and awaiting would be sync over async. `Run` never blocks and never throws — which matters, because most of these call sites went to the thread pool precisely to escape a lock they were holding. 11 unit tests cover drain, cancellation, exception observation, the concurrency cap, post-shutdown rejection and idempotent disposal. Two of the fixes were more than re-wiring: - **`SessionManager` / `SubscriptionManager` loops were structurally unawaitable.** Both return `ValueTask`, so `Task.Factory.StartNew(() => ...)` handed back a `Task<ValueTask>` that completes at the first yield. Storing it would not have helped — awaiting it would only have awaited the *scheduling*. Both now go through `AsTask().Unwrap()`, matching the neighbouring `StartConditionRefreshWorker`. - **`ChannelAsyncOperation` was swallowing subscriber exceptions.** Its `doNotBlock` branch had no try/catch while the inline branch beside it did. It is per-operation with no owner to hang a drain on, so it gained the missing handler rather than a scope. The Kubernetes readiness endpoint keeps a purpose-built semaphore instead: an unauthenticated endpoint needs a hard cap on concurrent handlers, not just an owner. ### ⚠️ Breaking: `ISessionManager.Shutdown()` is removed Replaced by `ShutdownAsync(CancellationToken)`. There is no correct synchronous way to wait for the session monitor loop, so leaving the overload would only have preserved the race. This mirrors `ISubscriptionManager`, which already had `ShutdownAsync`. Migration notes are in `docs/MigrationGuide.md`; only one in-repo implementer existed. Also converted the injected `object partitionLock` shared by `LogicalSubscription` and `CompositeMonitoredItemCollection` to `System.Threading.Lock`, per the repo rule. ## Patch coverage now scales with the size of the patch A flat floor over changed lines punishes small changes for arithmetic: two uncovered lines in a four-line fix reads as 50 % and failed an 80/5 floor — which is exactly how this branch earned a red coverage check for a four-line predicate change. A check that fires on changes nobody considers undertested is one people learn to ignore. | Coverable changed lines | Floor | Below it | | --- | --- | --- | | 1 – 10 | 50 % | warning, still passes | | 11 – 100 | 60 % | warning, still passes | | more than 100 | 75 % | **failure** | Only changes past the last band can fail. Below it the author still gets a warning naming the uncovered lines. Bands live in `patch.bands` in `coverage-thresholds.json` and each carries its own `enforced` flag. Verified end-to-end against synthetic repositories at every boundary. ## Codecov is back, as reporting Dropped in #4122 when the in-pipeline gate replaced it. Keeping the gate was right, but dropping Codecov also lost the PR comment, the file-by-file diff view and the trend. It returns with both statuses `informational: true` — two gates with two sets of thresholds would eventually disagree, and `coverage-thresholds.json` stays the single source of truth. The upload is switchable and can never fail a build: `enableCodecov` (Azure) and `ENABLE_CODECOV` (Actions), both defaulting on, both skipped when `CODECOV_TOKEN` is absent as on fork PRs. Each system uploads the single merged report it already builds, under its own flag. The old `codecov.io/bash` uploader was sunset, so Azure uses the current CLI. ## Follow-up actions for maintainers 1. **Update the branch ruleset.** The per-leg `Coverage Release (…)` checks are gone. Azure reports checks as `<pipeline> (<stage> <job>)`, so the exact names are: | Check | Require? | | --- | --- | | `OPCFoundation.UA-.NETStandard (Tests passed Verify stage results)` | **yes** | | `build-and-test summary` | **yes** (already required) | | `OPCFoundation.UA-.NETStandard (Code coverage Merge and evaluate)` | no | | `code coverage` | no | 2. **Re-calibrate the thresholds.** `coverage-thresholds.json` is still seeded from the last Codecov reading (73.61 %). The two systems now report *different* numbers by design — Actions merges every project on ubuntu, Azure only the Windows fast-PR legs, and scheduled runs read higher still because the Debug / .NET 8 / .NET 9 / netstandard stages also contribute. Re-seed from the GitHub Actions figure. 3. **Note:** a failed advisory coverage stage still turns the whole Azure run red, which the README build badge reflects on `master`. That follows from wanting a visible red X on a threshold miss; say the word if you would rather it were a warning. 4. **The Azure agent pool is the slowest thing here.** Several runs on this PR sat queued for hours, and one build failed with `Insufficient system resources` while writing an XML doc file — an agent exhaustion error, not a code error. It passed on re-run. Worth raising against the `netstandard` Managed DevOps Pool capacity separately. ## Related Issues _No tracking issue; this is CI infrastructure work requested directly. It builds on the switch introduced in #4122._ ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - No product code changed, so there are no unit tests to add. The new logic was verified with purpose-built harnesses instead: - `check-coverage.ps1` run for real against a merged Cobertura report on all three paths — pass, threshold failure, and patch coverage with uncovered changed lines (a scratch commit adding an uncovered method was correctly resolved to `DataTypeException.cs:193`, reported 0 % patch coverage and exited 1). - The gate's accept/reject logic dry-run over `Succeeded` / `SucceededWithIssues` / `Skipped` / empty / `Failed` / `Canceled` — 6/6 correct. - `get-matrix.ps1` exit codes verified in clean processes: empty → 1, normal → 0, `-AllowEmpty` → 0. - All five pipeline/workflow files parse; both PowerShell scripts parse. - Action versions checked against the GitHub API and pinned to current majors. - [x] I have added all necessary documentation. - Rewrote the coverage section of `docs/DeveloperGuide.md` as *Required checks and coverage*, covering which checks belong in the ruleset and which must not, how coverage is measured, where the numbers appear, and why the two systems disagree. Corrected the stale "BLOCKS" wording in `coverage-thresholds.json`. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - CI YAML, two PowerShell scripts and documentation only; no compilation involved. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - Not applicable — no product code changed. `Opc.Ua.Core.Schema.Tests` was run repeatedly on net10.0 (119 passed) to generate the real Cobertura fragments used to validate the gate. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. - Feedback on the path-relevance diff (now a merge-base diff) and the target-framework numbering in `docs/DeveloperGuide.md` is fixed in `15b8e38df`. - A review pass over the gate/coverage split produced the four correctness fixes listed above, in `863eefc4d`. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 26aa55a2-5731-4a4c-b61c-df2f7a785eca	2026-08-07 06:56:05 +02:00
Marc Schier	724ae71f8a	Make NodeId assignment during a node copy explicit (#4157) ... Fixes #4137. ## The problem `NodeState.Initialize(ISystemContext, NodeState)` copies a node. Every child it materialises through `CreateChild` gets a per-instance NodeId from `ISystemContext.NodeIdFactory`, and is then initialised from its source on the very next statement - which overwrites that NodeId. The assignment is therefore *always* discarded: it only burns identifiers, and permanently leaks them for factories that track outstanding allocations. Booting the Robotics node manager left ten reservations behind that every later allocation skipped. #4123 fixed the symptom by hiding the factory for the duration of the copy, wrapping the context in one that reports `NodeIdFactory` as `null`. The review objection was fair: a context that lies about one property is hard to reason about, and it only works because that property is allowed to be null. ## What this does States the intent instead of faking the environment. NodeId assignment is now an argument of the two members that materialise children: ```csharp public virtual BaseInstanceState? CreateChild( ISystemContext context, QualifiedName browseName, bool assignInstanceNodeIds = true); protected virtual BaseInstanceState? FindChild( ISystemContext context, QualifiedName browseName, bool createOrReplace, BaseInstanceState? replacement, bool assignInstanceNodeIds = true); ``` A copy passes `assignInstanceNodeIds: false` and every node type - generated or hand-written - sees the real context. There is no capability property, no wrapper, and no second overload to keep in sync. `MethodState`, `BaseDataVariableState` and the source generator template each collapse to a single `FindChild` override that threads the argument into its `CreateOrReplace<Child>` helpers. ## Breaking change The four argument `FindChild` and the two argument `CreateChild` virtuals are gone. - **Call sites are unaffected.** The new parameter defaults to `true`, so `CreateChild(context, browseName)` and `FindChild(context, browseName, true, null)` keep compiling and keep the 1.5.378 behaviour. - **Overrides must add the parameter.** A node manager that overrides either member gets `CS0115` until it does. The compiler points at every site; `docs/migrate/2.0.x/node-states.md` shows the before/after. This is deliberate, per review: the stack ships all base types, and types generated from a model are regenerated for 2.0 anyway. ## Also fixed `MethodState.FindChild` returned `OutputArguments` when asked for `InputArguments` with `createOrReplace: false` - a pre-existing bug in the method being restructured. Covered by `FindChildReturnsTheRequestedArgumentsProperty`. ## Validation - Full `dotnet build UA.slnx`: **0 errors, 0 warnings**. - `Opc.Ua.Types.Tests`, `Opc.Ua.SourceGeneration.Core.Tests`, `Opc.Ua.SourceGeneration.Tests`, `Opc.Ua.Di.Tests`, `Opc.Ua.Robotics.Tests` - all passing. - Each new regression test was checked to fail without its corresponding fix. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Copilot-Session: 8cbb8cd0-f0cb-4ab0-bea2-6202fbf69485 Copilot-Session: f45109d9-59cb-4ffd-a4f7-16e1ea449ae4	2026-08-03 20:36:32 +00:00
Marc Schier	8a10139b8c	Fix OPC UA address-space compliance in the device integration stack (#4117) ... # Description A live OPC UA compliance audit of `samples/PumpDeviceIntegrationServer` against the **OPC 40223 Pumps** and **OPC 10000-100 (DI)** companion specifications found **13 address-space violations** plus a serialization bug in the MCP tool. Every fix lands at its **root cause in the shared stack** rather than as a workaround in the sample, so all servers benefit. The sample now exposes **N fully simulated pumps** (`--pumps N`) and advertises the DI conformance facets it actually satisfies. > This branch has been merged with `master` including **#4116** ("Add simulation to PumpDeviceIntegrationServer and Event modelling"), which independently reworked the same pump simulation and the same fluent alarm builders. See [Relationship to #4116](#relationship-to-4116) below for how the two were reconciled. ## Findings and fixes | # | Finding | Root cause | Fix | |---|---------|-----------|-----| | 1 | `OverTempAlarm` created by the fluent API did not exist in the address space | `AttachAlarm` attached the alarm with `AddChild` but never indexed it into the node manager | Register the node; the same gap existed in `AddObject`, `CreateInstance` and the state-machine creators | | 2 | Alarms were unsubscribable — `EventNotifier = 0` everywhere, no `HasNotifier` from `Server` | No event-source wiring at all | Set `HasCondition`, initialise `SourceNode`/`SourceName`/`ConditionName`/`InputNode`, promote `EventNotifier` up the ancestor chain, publish `HasNotifier` from the `Server` object | | 3 | Every instance NodeId was minted in the **DI standard namespace** (`ns=4;s=5001_Pump #1`) | `DiNodeManager.New` inherited the parent's namespace index | New `DiNodeManager.InstanceNamespaceIndex`; instances move to the server's application namespace | | 4 | `Server/Namespaces` described only 2 of 6 namespaces | The NodeSet2 importer drops nodes parented to namespace 0, so companion-spec metadata objects never reached the address space — and no route covered the server's own namespace | New `NamespaceMetadataPublisher` walks the `NamespaceArray` and fills version/publication date from `ModelDependencyAttribute` | | 5 | Vendor BrowseNames (`Diagnostics`, `LastError`, …) sat in the DI namespace | Builders defaulted to the parent's namespace | Default to the server namespace; **spec-defined BrowseNames are unchanged** | | 6 | `AccessLevel="5"` from the Pumps NodeSet was reported as `1` | The ModelDesign `AccessLevel` enum is not `[Flags]` and cannot represent `CurrentRead \| HistoryRead` | Carry the verbatim NodeSet2 bitmask and emit named `AccessLevels` constants | | 7 | `Historizing = true` while `HistoryRead` returned `BadHistoryOperationUnsupported` | Nothing reconciled the declared history surface with the wired historians | Startup clears the advertisement (including masking the attribute read callbacks) when no provider resolves; the sample wires a real historian | | 8 | Mandatory `ProductInstanceUri` unset | Sample omission | Populated on every pump | | 9 | Mandatory `TrueState`/`FalseState` empty on every supervision boolean | Absent in the official NodeSet; a server must still populate them | Populated on every pump | | 10 | Machinery `Machines` folder empty (OPC 40001-1 §9.2 says **shall**) | `TryAddToMachinesFolder` used `AddChild`, which *re-parents* the device away from `DeviceSet` | Add the `Organizes` reference instead | | 11 | BrowseName contained a space and `#` | Sample naming | `Pump_1` with the readable label in `DisplayName` | | 12 | `Maintenance` group materialised but empty | Sample omission | Populated | | 13 | Uninitialised values reported `Good` | No initial-value status | `BadWaitingForInitialData` until the first simulation tick | | — | `ServerProfileArray` advertised only `StandardUA2017` | `DiNodeManager` did not implement the existing `IConformanceContributor` | Contribute DI 1.05 server facets computed at runtime from what is actually wired, merged with `StandardUA2017` | | — | MCP tool serialised boolean `false` and numeric `0` as JSON `null` | `Variant.Null` is `default`, so `Variant.Equals` adopts the non-null type and compares against the default value | Dispatch on `BuiltInType` and use typed `TryGetValue` accessors instead of the prohibited `AsBoxedObject` | ## Relationship to #4116 #4116 landed on `master` while this branch was in flight and targeted overlapping ground. The merge is a **union of both intents**, not a fast-forward: - **`AlarmBuilderExtensions`** — kept both sides. #4116 contributes the parent-must-be-an-Object guard, `SetEnableState`, and the `HasEventSource` references; this branch contributes the `HasCondition` reference, condition source initialisation, **registration of the alarm with the node manager so it is browsable**, notifier promotion up the whole ancestor chain, and root-notifier registration. - **`SupervisionBuilderExtensions`** — took #4116's `SetAlarmActive`. It is more spec-correct than this branch's version: it honours `EnabledState`, resets Acked/Confirmed on activation, and computes `Retain` per OPC 10000-9 so an unacknowledged Condition stays retained after going inactive. This branch's null guard was kept so a malformed alarm cannot dereference a missing `EnabledState`. - **`PumpNodeManager.Configure.cs`** — took #4116's **push-based** simulation (`RegisterPumpSimulation`, `CreatePumpSimulation`, `IValueUpdater<T>`, `Initialize`/`Advance`/`Publish`, deterministic per-pump phase offsets) as the structure, then folded in this branch's compliance work: configurable pump count, Maintenance group, `TrueState`/`FalseState`, historian wiring, per-pump mandatory Identification values, and `BadWaitingForInitialData` until the first tick. - **README** — union, keeping #4116's address-space validation workflow section. Two test adjustments were needed, both because the merge legitimately changed behaviour the tests pinned: - `PumpHostedReferenceTests` now resolves pump NodeIds from the browse result instead of hard-coding the DI namespace, and waits for the first published value. `PumpCreatedAfterStartupJoinsTheLiveSimulationAsync` was **added** to restore #4116's coverage that a pump created after startup through `ConfigureDevicesFor` joins the live simulation. - `FluentAlarmRegistrationIntegrationTests` no longer asserts that `Retain` clears when an alarm goes inactive — that pinned this branch's weaker semantics, and #4116's Part 9 behaviour is correct. ## Breaking change Runtime-created DI device instance NodeIds move from the companion-spec namespace to the server's application namespace, e.g. `ns=<DI>;s=5001_Pump #1` becomes `ns=<application>;s=5001_Pump #1`. This invalidates NodeIds cached by clients from earlier 2.0 previews. It is **not** a model change — standard DI/Machinery/Pumps type NodeIds and spec-defined BrowseNames are untouched. Device integration hosting is new in 2.0, so there is nothing to migrate from 1.5.378 and this is deliberately *not* in `docs/MigrationGuide.md`. Anyone tracking 2.0 previews should rediscover devices by browsing `DeviceSet` rather than persisting NodeIds, and resolve namespace indexes from `NamespaceArray` per connection. ## Verification Compliance is pinned by regression tests that assert **through the server's service surface** (`BrowseAsync` / `ReadAsync` / `HistoryReadAsync`), not by inspecting `NodeState` objects in memory. That distinction mattered: three defects survived in-memory assertions and were only caught through the read path — most subtly, `Historizing` was cleared while `AccessLevel` still advertised `HistoryRead`, because attribute read callbacks re-applied the NodeSet bits. Post-merge on net10.0: | Suite | Result | |---|---| | `Opc.Ua.Di.Tests` | 326 passed | | `Opc.Ua.Server.Tests` | 3846 passed, 5 skipped | | `Opc.Ua.Tools.Tests` | 288 passed | | `Opc.Ua.History.Tests` | 506 passed, 27 skipped | | `Opc.Ua.SourceGeneration.Core.Tests` | 3722 passed, 8 skipped | The full solution was also built and tested on **net48** before the merge. Review feedback from `copilot-pull-request-reviewer` has been addressed in commit `badad31dc`: the shared `ServerObjectState` lock was removed (the forward `HasNotifier` edge is published through the owning node manager instead), MCP array serialization now preserves element types, and the pump option guards report the specific rejected option. The only failure is `ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`, a pre-existing certificate-store issue that **passes when run in isolation** and is unrelated to this change. ## Documentation Updated `docs/DeviceIntegration.md`, `docs/MigrationGuide.md`, `docs/HistoricalAccess.md`, `docs/DeveloperGuide.md`, `docs/McpServer.md`, `docs/README.md`, `samples/PumpDeviceIntegrationServer/README.md` and `tools/Opc.Ua.Mcp/README.md`, and consolidated the four node-manager documents into a single [`docs/NodeManagers.md`](docs/NodeManagers.md) — overview, built-in managers (including `MasterNodeManager`), core vs custom, registration, server address-space metadata, and source generation — with a TOC and all inbound links repointed. ## Related Issues No tracking issue exists for this work yet — it originated from an ad-hoc compliance audit of the pump sample. Given the size of the change and the runtime-breaking NodeId move, please open (or link) a tracking issue so the design is recorded as an ADR before merging. - Fixes # ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0d50c3fc-58b1-4c99-85a4-1e32c88666d9	2026-07-30 16:04:18 +02:00
Copilot	a03bca97cb	Remove TraceConfiguration and related APIs (#4092) ... # Description This change removes the remaining `TraceConfiguration`-driven apply/setter API path and aligns configuration startup with the 2.0 telemetry model (`ITelemetryContext`/`ILoggerFactory`). It also updates migration docs to make the removal explicit and actionable. - **Core API removals** - Deleted `TraceConfiguration.ApplySettings()` implementation file. - Removed runtime apply calls from configuration/server startup paths. - Removed fluent builder trace setters: - `SetOutputFilePath(...)` - `SetDeleteOnLoad(...)` - `SetTraceMasks(...)` - Simplified builder interface inheritance accordingly. - **Call-site and test fixture updates** - Removed usage of deleted trace fluent APIs in affected test infrastructure and fixtures. - Updated GDS log readers to tolerate missing `TraceConfiguration` output path safely. - **Migration guidance** - Updated migration docs to explicitly state removal of `TraceConfiguration` apply APIs and direct users to telemetry-based logging configuration. ```csharp // Removed API pattern app.Build(uri, product) .AsClient() .SetOutputFilePath("trace.log") .SetTraceMasks(Utils.TraceMasks.Error); // New direction // Configure logging via ITelemetryContext / ILoggerFactory at composition/startup. ``` ## Related Issues - OPCFoundation/UA-.NETStandard#4091 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. <!-- START COPILOT CODING AGENT SUFFIX --> - Fixes #4091 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>	2026-07-26 00:50:59 +02:00
Marc Schier	7cec2d614d	Move repository files to the consolidated layout	2026-07-16 06:25:54 +02:00