mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-27 04:59:55 +08:00
0bd7aaaad6bc1e116ddc8ad35fc93d04eec4bbb1
2843
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
0bd7aaaad6 |
AutoID: keep every reader reachable from DeviceSet, and let a reader be a component of a machine (#4853)
# Description Follow-up to the AutoID companion (#4825). Two changes to where the AutoID server places its readers: 1. **`IAutoIdReaderBuilder.WithParent` is made conformant with OPC 10000-100 4.9.** Until now it placed a reader below another node *instead of* `DeviceSet`, and the docs suggested "a component of a machine" as the use case. A machine in the Machinery `Machines` folder is not reachable from `DeviceSet`, so such a reader could not be located from `DeviceSet`, which DI 4.9 requires for every `ComponentType` instance (every AutoID device type derives from DI `DeviceType`). Now the reader is still placed below the parent (`HasComponent`, unchanged), and: - when `DeviceSet` already reaches the parent through hierarchical references (for example another reader, or any device it organizes), nothing else is added: the reader is a component of a composed device and DI 4.9 wants "only the root instance" referenced from `DeviceSet`; - otherwise `DeviceSet` also organizes the reader (`Organizes`, both directions), so it stays locatable. 2. **New opt-in `IAutoIdReaderBuilder.AsComponentOf(BaseObjectState machine)`**, available only when AutoID is composed with Machinery (`AddAutoIdToMachinery`, or a DI owner that loads the Machinery model). It makes the reader an identifiable component of the machine per OPC 40001-1 11: - the reader is placed below the machine's `Components` add-in (`HasComponent`, as `MachineComponentsType` declares for `<Component>`); - it gets the `2:Identification` add-in (`HasAddIn`) typed `MachineryComponentIdentificationType`, which holds no copies: it references the reader's own DI nameplate properties with `HasProperty` (`Manufacturer`, `SerialNumber`, and those of `Model`, `ManufacturerUri`, `ProductCode`, `HardwareRevision`, `SoftwareRevision`, `DeviceRevision`, `DeviceClass`, `ProductInstanceUri` that are set, plus `AssetId`/`ComponentName` when published; `DeviceManual` and `RevisionCounter` are left out as 40001-1 10.2 recommends); - the reader stays a device: the same `DeviceSet` rule as for `WithParent` applies with `Components` as the parent, so in the usual case (a machine in `Machines` that `DeviceSet` does not reach) `DeviceSet` organizes it directly. It is not added to `Machines`. - Checked before anything is registered, each failing with `Bad_ConfigurationError` and a message naming the way out: the server has no Machinery model (names `AddAutoIdToMachinery`), the machine is not a node of the AutoID owner, the machine has no `Components` add-in (names `WithComponents`), or the reader nameplate lacks `Manufacturer` and/or `SerialNumber` (mandatory in `MachineryItemIdentificationType`). `WithParent` and `AsComponentOf` exclude each other (`InvalidOperationException`). Default behaviour is unchanged: a reader built without `WithParent`/`AsComponentOf` is organized by `DeviceSet` and not added to `Machines`. Rollback on failure covers the new references: a failed build removes the `DeviceSet` reference, the component and its add-in, so the name can be used again. ## Specification basis | Statement | Source | | --- | --- | | "The DeviceSet Object is the starting point to locate Devices. It shall either directly or indirectly reference all instances of a subtype of ComponentType with a Hierarchical Reference. For complex Devices that are composed of various components that are also Devices, only the root instance shall be referenced from the DeviceSet Object. The components of such complex Devices shall be locatable by following Hierarchical References from the root instance." | OPC 10000-100 (DI) 1.05.0, 4.9 | | AutoID device types are "Subtype of DeviceType defined in OPC 10000-100". OPC 30010 does not mention `DeviceSet`, Machinery or `Machines`. | OPC 30010 (AutoID) 1.01, 6.1 Table 7 | | "Each Machine providing this building block provides the Components Object, which directly references all identifiable components of a Machine." / "Note: That does not preclude that a Machine is organizing its components in various other hierarchies as well." | OPC 40001-1 (Machinery) 1.04.2, 11.1 | | "The MachineComponentsType provides HasComponent References to all identifiable components of a Machine"; Table 27: `0:HasComponent Object <Component> 0:BaseObjectType OP`; Table 28: `<Component> 0:HasAddIn Object 2:Identification MachineryItemIdentificationType M`; "As all identifiable components shall contain the Identification Object". | OPC 40001-1 1.04.2, 11.2 | | `MachineryItemIdentificationType`: `2:Manufacturer` M, `2:SerialNumber` M, all other members O. | OPC 40001-1 1.04.2, 8.3 Table 16 | | `MachineryComponentIdentificationType` is the component subtype of `MachineryItemIdentificationType` (adds `2:DeviceRevision`); "It is not recommended to use the optional Properties 2:RevisionCounter and 2:DeviceManual"; a missing component `2:ProductInstanceUri` "shall be omitted". | OPC 40001-1 1.04.2, 10.2 | | "it is recommended to keep this approach and let several References lead to the same Node, rather than duplicating the Node" / Robotics example: identification Properties "are only referenced from the Identification functional group". | OPC 40001-1 1.04.2, 7.3 and Annex B.2 (informative) | | "It is expected that domain-specific companion specifications using this specification will define what assets of the domain in what usage are considered to be Machines or components of Machines." | OPC 40001-1 1.04.2, 11.1 (Note) | | Organizes "can be used to span multiple hierarchies"; more than one hierarchical path to a Node is allowed. | OPC 10000-3 1.05, 7.11, 7.5 | **SDK decisions** (not specified by OPC 30010, which predates Machinery): - A reader is never added to `Machines`: no specification classifies an AutoID device as a machine (40001-1 11.1 leaves that to the domain specifications, AutoID is silent). - A reader becomes a component of a machine only when the application asks for it (`AsComponentOf`). - `WithParent` keeps its parent and adds the `DeviceSet` reference rather than rejecting parents outside `DeviceSet` (see below). - The component `Identification` references the reader's nameplate instead of copying it. - AutoID does not add Machinery conformance units; the Machinery owner advertises them for what its builders created. ### Why `WithParent` adds a reference instead of rejecting DI 4.9 asks for reachability, not for a particular parent, and OPC 10000-3 allows several hierarchical paths to one node. Adding `Organizes` from `DeviceSet` satisfies the "shall" while keeping the parent the application asked for, and it is the same reference type `DeviceSet` already uses for every reader (and the DI node manager for every device). Rejecting such parents with `Bad_ConfigurationError` would break existing callers that hang a reader below a machine or folder. When `DeviceSet` already reaches the parent, no reference is added, so the "only the root instance" rule of 4.9 holds for composed devices. Reachability is decided once when the reader is built, by following the forward hierarchical references of the owner's nodes. ### Why `AsComponentOf` keeps the reader referenced from `DeviceSet` Machinery does not require machines to be in `DeviceSet` (the stock builders put them in `Machines` only), so a reader that is only below `Components` would again be unreachable from `DeviceSet`. The same reachability rule as for `WithParent` applies: if the machine is itself reachable from `DeviceSet` (for example a machine that is also a DI device organized by `DeviceSet`), the reader is reached through it and gets no direct reference, otherwise `DeviceSet` organizes it. ### Shape of `AsComponentOf` It takes the machine's `BaseObjectState`, like `WithParent(NodeState)` and `IMachineryBuildContext.BindMachine(TState, ...)`; with the stock builders that is `IMachineHandle<TState>.State`. It requires the machine's `Components` add-in to exist (`IMachineBuilder.WithComponents`, or a companion-declared one bound with `WithComponents(components, ...)`) instead of creating one, so the Machinery builders stay the only place that creates Machinery building blocks (`MachineryBuildingBlocks` organization, facets). The reader's NodeState parent is the `Components` object, so the DI node manager attaches it with `HasComponent` and the existing registration and rollback code applies unchanged. ## Behaviour change for existing callers - `WithParent(parent)` with a parent that `DeviceSet` does not reach (for example a machine in `Machines`, or an application folder): the reader is now *also* browsable from `DeviceSet` (one extra `Organizes` reference, both directions), and an `AutoIdClient.EnumerateReadersAsync()` (which browses `DeviceSet`) now finds it. Its parent, NodeId and `HasComponent` reference are unchanged. - `WithParent(parent)` with a parent `DeviceSet` reaches (for example another reader): unchanged. - Default placement: unchanged. - `IAutoIdReaderBuilder` gains a member. The interface is implemented only inside the package (instances come from `AutoIdRuntime.Reader`), and the package is new in 2.0. ## Changes - `src/Opc.Ua.AutoID.Server/AutoIdReaderPlacement.cs` (new, internal): `DeviceSet` lookup, the reachability check, the `Organizes` reference and its removal, `Components` resolution and validation, the nameplate check, and the `Identification` add-in. - `src/Opc.Ua.AutoID.Server/AutoIdRuntime.cs`: the builder's nameplate delegate runs once (validated before anything is registered, then copied into the DI builder); `AsComponentOf` resolves `Components` as the parent before the device is created; after registration the add-in is added and the `DeviceSet` reference kept; rollback removes the reference. - `src/Opc.Ua.AutoID.Server/Builders/IAutoIdReaderBuilder.cs`, `AutoIdReaderBuilder.cs`: `AsComponentOf`, mutual exclusion with `WithParent`, XML docs stating the exact behaviour. - `docs/AutoID.md`: new "Placement" subsection under "Hosting readers" (what DI 4.9 and Machinery 11 require, what is an SDK decision, a table per builder call, an example), the composition table rows for readers and `Machines`, the test fixture table. `src/Opc.Ua.AutoID.Server/NugetREADME.md`: a short paragraph. - Tests (below). ## Tests - `AutoIdReaderLifecycleTests` (stand-alone owner): `ReadersCanHangBelowAnExplicitParentAsync` now also asserts that `DeviceSet` does not organize a reader below another reader; `AReaderBelowAParentOutsideTheDeviceSetIsStillOrganizedByItAsync`; `AFailedCreationBelowAnOutsideParentLeavesNoDeviceSetReferenceAsync` (rollback); `AStandaloneReaderCannotBeAComponentOfAMachineAsync` (`Bad_ConfigurationError` naming `AddAutoIdToMachinery`); argument checks for `AsComponentOf(null)` and both call orders of `WithParent`/`AsComponentOf`. - `AutoIdCompositionTests` (Machinery owner, in process): `AReaderBecomesAComponentOfAMachineAndStaysADeviceAsync` (parent is `Components`, `HasComponent`, `DeviceSet` organizes it, `Machines` does not, `Identification` with `HasAddIn` and the component type, `HasProperty` to the reader's own `Manufacturer`/`SerialNumber`/`Model` nodes, no `DeviceManual`, owned subtree); `AComponentNeedsAMachineWithComponentsAndRegistersNothingOtherwiseAsync`; `AComponentNeedsTheMandatoryIdentificationAsync`; `AComponentNeedsAMachineOfTheOwnerAsync`; `AFailedComponentCreationIsRolledBackAsync`. - `AutoIdPlacementEndToEndTests` (new, hosted Machinery + AutoID over a real session): a default reader, a reader below that reader, a reader `AsComponentOf` a machine and a reader `WithParent` the machine. Browsing `DeviceSet` finds the three roots and not the nested reader, the outer reader reaches the nested one; `MachineryClient.DiscoverMachinesAsync` returns only the machine and `Machines` contains no reader; `MachineryClient.EnumerateComponentsAsync` lists only the component reader, `ReadIdentificationAsync` reads `Manufacturer`, `SerialNumber`, `ProductCode` through the add-in, which is `HasAddIn`-referenced and typed `MachineryComponentIdentificationType` and whose `HasProperty` targets are the reader's own nameplate nodes; `AutoIdClient.EnumerateReadersAsync` finds the three roots. - The existing default assertions (`TheReadersShareTheDeviceSetWithThePumpAndStayOutOfTheMachinesFolderAsync`, `TheMachineryAndPumpsClientsStillFindTheirInstancesAsync`, `ComposedReadersLiveBelowTheSharedDeviceSetAsync`) are unchanged. - Fail-without: with the runtime wiring reverted to master (builder surface kept so the tests compile), 10 of the new tests fail: all `AsComponentOf` cases, both `WithParent` cases outside `DeviceSet` and the three session tests that look for the component or the extra `DeviceSet` reference. Removing only the rollback of the `DeviceSet` reference makes both rollback tests fail. `NoReaderBecomesAMachineAsync` and the existing default tests pass with and without the change, as intended. Local results (macOS arm64, SDK 10.0.402, head `238ed9b69` on master `9bfc780e1`; the same three commits are now based on master `ac24123dd` as `c6d70909b`, no AutoID file changed in between): `Opc.Ua.AutoID.Tests` net10.0 318/318 passed; `Opc.Ua.AutoID.Server` all TFMs and the test project with `CustomTestTarget=net48` build with 0 warnings, 0 errors (net48 tests run on Windows CI; there is no mono host here). `dotnet format whitespace/style/analyzers --verify-no-changes` on the changed files: no new findings beyond the conventions of the surrounding test files. ## Commits 1. `fix(autoid): keep every reader reachable from DeviceSet` 2. `feat(autoid): make a reader a component of a machine` 3. `docs(autoid): document where readers are placed` The branch is based on OPCFoundation/master `ac24123dd` (#4852, the last master commit with a green build). Fork CI was not repeated on this head: earlier fork runs of this branch failed only in test projects outside this PR (the master build was broken by the NUnit 5 migration #4840 and fixed in #4852; master is currently broken again in `Opc.Ua.Core.Tests` by #4849). The upstream checks of this PR test it against the current master. ## Related Issues - Follow-up to #4825 (tracking issue #4824). - #4823 (open) edits prose in other paragraphs of `docs/AutoID.md` (table of contents, the DI-owner paragraph, providers). This PR adds a subsection and changes two rows of the composition table and one row of the test table; no hunk overlaps. The table of contents is deliberately not touched here because #4823 adds the subsection entries for "Hosting readers"; the "Placement" entry can be added there or after both are merged. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. (Local build of the changed projects: 0 warnings, 0 errors.) - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Locally the AutoID test project on net10.0; net48 is built but runs only on Windows CI.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. (Pending the upstream checks; see the CI note above.) - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
a51f08ad6f |
AutoID client: read a device's status without browsing (#4854)
# Description A follow-up to #4825. It adds `AutoIdClient.RefreshDeviceAsync(AutoIdDeviceSnapshot, CancellationToken)`, which reads the variables of a reader device that change at runtime again through the NodeIds a snapshot already carries. It makes one Read and no Browse. ## Why `ReadDeviceAsync` browses the children of the device on every call. OPC 30010 makes most device variables and methods optional, so the browse is how the client finds out what this device publishes. `AutoIdReadOptions.StatusOnly` only narrows the Read. This is the point the Copilot review raised on #4825 ([discussion](https://github.com/OPCFoundation/UA-.NETStandard/pull/4825#discussion_r4308032841)). |
||
|
|
2836da7ec8 |
Fix sessionless ResultTransfer ownership bypass and other issues. (#4834)
# Description Correct ownership, endpoint validation, policy propagation, subscription delivery, file handling and generated-model behavior across the stack, tools and samples. Add focused regression tests and update the corresponding guides and sample documentation. ### Previously included corrections - Require an active, matching session to generate or access transient ResultTransfer file handles. Cover both sessionless generation and reads against a handle owned by an active session, using live session owners in the Machinery tests. - Keep physical filesystem operations within their configured mount and apply the same confinement checks throughout file operations. - Validate LDS semaphore paths consistently during server registration and registration-store maintenance. ### Individual issues addressed | # | Component | Issue and correction | |---|---|---| | 1 | AI jobs and transfers | Jobs and transfers did not consistently retain their creator's identity and session. Apply ownership checks to descendants, file handles, subscriptions and execution, and recheck deployment permissions before dispatch. | | 2 | Password-management client | Password-bearing calls could use signing-only channels or follow a replacement connection. Capture an encrypted transport generation and keep dispatch bound to it; preserve ordinary non-password edits. | | 3 | GDS and certificate-push clients | Connections omitted hostname verification. Validate the original configured DNS name or IP address before session activation. | | 4 | Certificate push | Private-key uploads could use signing-only calls. Require an encrypted, generation-bound dispatch while preserving certificate-only updates and restoring the caller's identity afterward. | | 5 | MQTT client | The .NET Framework custom-trust callback could discard TLS hostname, certificate-availability and usage errors. Retain those checks while supporting configured issuer trust. | | 6 | DTLS handshake | Certificate trust was not tied to the configured peer address. Match the certificate's DNS or IP identity to the intended endpoint. | | 7 | Replicated address spaces | Serialized nodes lost per-node and descendant permission metadata. Preserve role permissions, user-role permissions and access restrictions, including updates to existing replica nodes. | | 8 | MCP PubSub imports | Imports without groups skipped checks on the effective broker destination. Check the actual endpoint before resolving credentials or activating the connection. | | 9 | Reference-server accounts | A disabled username account could authenticate again with its previous roles. Reject disabled accounts during fresh authentication. | | 10 | Console reference client | The sample disabled hostname checks when connecting to an otherwise trusted peer. Retain the original requested host and validate it before activation. | | 11 | Capture replay and export | Replay and raw export could accept non-capture files or include key material. Require complete PCAP/PCAPNG containers and exclude key sidecars and decryption-secret blocks. | | 12 | PubSub adapter fallback | The fallback client configuration automatically accepted peer certificates. Apply normal trust and hostname validation before activation or reads. | | 13 | Kafka transport | Explicit TLS enablement was checked but did not reach both final Confluent configurations. Use the same effective protocol for validation, producers and consumers. | | 14 | PubSub actions | Configuration rebuilds could widen a connection-local action handler's scope. Preserve its original connection association and reject mismatched dispatch. | | 15 | PubSub readers | Reader-level message-protection settings were accepted but ignored. Apply the effective group and reader settings before accepting and publishing received values. | | 16 | PubSub key groups | Different `SecurityGroupId` values could share one connection-wide key context. Reject mixed groups on a connection and require separate connections for distinct groups. | | 17 | MCP session registrations | Session-name replacement or reuse could retain another registration's credential bindings. Bind access to the registration, caller and authenticated endpoint, and preserve that identity when borrowing sessions. | | 18 | Viewer HTTP host | Requests reached the viewer's OPC UA session without consistent Host, Origin and port checks. Validate the request destination and browser origin before using the session, while retaining supported local MCP clients. | | 19 | Vision file operations | File operations did not consistently apply protected-file checks, and remote handles lacked caller and connection context. Apply the shared checks and use host-side opaque handles bound to caller, target and generation. | | 20 | DTLS record integrity | The integrity-only MAC used a truncated sequence number. Authenticate the full reconstructed sequence number and epoch so records remain distinct across rollover. | | 21 | AMB client sample | Connection setup omitted hostname validation. Preserve the originally requested host and verify the peer identity before activation. | | 22 | Robot sample writes | The R2 `SpeedOverride` callback could target R1 state. Bind each callback to its own robot and apply that robot's write policy. | | 23 | AI result properties | Optional result children inherited remotely writable defaults. Preserve their declaration restrictions and keep server-produced results read-only. | | 24 | WSS peer-trust updates | Existing channels did not consistently respond to peer-trust changes. Revalidate complete chains for forward and reverse WSS channels and close only those that no longer validate. | | 25 | GDS revocation checks | Caller-provided certificates could cause issuer retrieval from embedded URLs. Resolve validation from configured local trust, issuers and CRLs without fetching caller-provided locations. | | 26 | GMS observations | Published correction observations retained writable model defaults. Reject remote writes while preserving publisher-side updates. | | 27 | Production-dataset monitoring | Subscription admission and delivery could use the reporter's context instead of each subscriber's permissions. Evaluate the subscriber at admission and on every delivery, including cancellation and disposal paths. | | 28 | WoT MQTT destinations | Hostless MQTT forms could imply a localhost destination before loopback checks. Reject them before credential lookup or connection attempts and validate the effective endpoint. | | 29 | XRegistry opaque reads | Content-ID reads and notifications did not consistently honor encrypted-read requirements or the currently accepted transport. Recheck the bound endpoint and reject transport downgrades before rebinding, including when protected values are already queued. | | 30 | PubSub capture dissection | Key loading did not consistently check the configured capture-root boundary. Check each actual key path immediately before opening it. | | 31 | PubSub publication queues | Imported WriterGroup queues were not fully checked against publish-topic grants. Check the destination actually used by each group. | | 32 | MCP PubSub mutations | A caller-supplied mode could select subscribe grants for an operation that publishes. Derive the required direction and parent context from the actual mutation. | | 33 | Generated write masks | Model import and generation could widen an explicit `UserWriteMask`. Preserve it through import, copy, merge, inheritance and dependency metadata. | | 34 | Generated method flags | An explicit `UserExecutable=false` could be lost. Preserve its specified state and resulting method behavior through the same model paths. | | 35 | Generated role identifiers | Unresolved role identifiers could cause permission entries to be omitted. Preserve namespace-qualified role IDs and report a diagnostic when a policy cannot be represented. | | 36 | Aggregation-server defaults | The `localhost` default did not reliably constrain listener binding. Use numeric loopback for the default endpoint and command-line fallback. | | 37 | Certificate-validation cache | Cached results could skip current leaf or issuer validity checks. Recheck validity windows on cache hits and keep hostname validation independent of cached chain trust. | | 38 | Software-update folders | Revision names and marker contents containing dot segments could select paths outside the software root. Normalize and check every derived path against that root. | | 39 | USDA arrays | Array elements were not encoded for their string or asset context. Escape them as data and preserve valid composition and round-trip behavior. | | 40 | Adapter hot reload | A changed dataset definition with the same name could keep old fields, subscriptions or cached values active. Compare complete immutable definitions and retire old state before replacement. | | 41 | Adapter index ranges | Subscription publication could ignore `IndexRange` and return the complete value. Preserve the requested projection and its status and timestamps; reject unsupported range providers. | | 42 | PubSub metadata and targets | Metadata from one connection could change another connection's target mapping. Scope metadata to its connection and provenance, and resolve targets using actual typed field IDs rather than field names or unintended positional fallback. | | 43 | Scales product locks | Product locks were not applied across every remote mutation path. Apply them consistently to value writes, filling calculations and related recipe/controller operations while retaining permitted updates. | | 44 | Event permission caching | Cached event decisions could outlive identity, role or node-policy changes. Revalidate the current subscriber, event type and source permissions instead of reusing a stale decision. | | 45 | WoT Modbus destinations | Hostless Modbus forms could imply localhost before loopback checks. Reject them before connecting and validate the effective endpoint. | | 46 | MCP local PKI tools | Peer-trust mutations did not consistently require the caller's `CertificateAdministration` grant. Check the capability before trust and untrust operations. | | 47 | Robot-control HTTP bridge | The bridge lacked consistent Host, Origin and port validation and caller-specific session grants. Apply those checks before borrowing the OPC UA session and retain supported local clients. | | 48 | Subscription transfers | Transferred subscriptions could retain cached notifications after the destination's permissions changed. Require authorization continuity between source and destination identities and roles, invalidate stale retransmission data when continuity fails, and preserve retransmission behavior for compatible transfers. | ### Compatibility notes - Replication uses the versioned NSRP v1 policy envelope. Upgrade peers together and re-encode or reseed legacy topology from the authoritative model; missing permission metadata is not inferred from old records. - Integrity-only DTLS peers must be upgraded together for the full sequence/epoch MAC. AEAD behavior is unchanged. - Distinct PubSub key groups require separate connections. - Custom sessions that cannot capture the actual encrypted transport, and subscription providers that cannot honor a requested range, are rejected rather than silently changing the requested behavior. - Subscription transfers retain retransmission data only when the destination authorization remains compatible with the source snapshot; otherwise the old sequence numbers and cached data are invalidated. ## Related Issues No separate tracking issue is linked. The individual corrections are listed above. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [ ] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
994d7b34da |
Fix CI test duration regressions (#4861)
# Description Partition the long-running sample and WoT test suites into bounded, non-overlapping CI shards, and give all workloads the standard 45-minute test-execution budget. Account for setup/build overhead separately in the job timeout so builds do not consume test timeouts. Remove the former sample, integration, WoT, and server-specific timeout exceptions while retaining complete test scheduling. Focused matrix tests pass (25/25); full and PR matrix generation succeeds at 238 and 98 jobs, respectively. PowerShell syntax parsing and `git diff --check` pass. The full solution test suite was not run locally. ## Related Issues - Fixes #4858 ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [ ] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
6d2c7c0c67 |
WoT: retire a generation's event bindings before releasing their sources (#4860)
Fixes the intermittent failure of `WotEventModesLiveTests.NativeRetiringGenerationUsesCapturedActionsUntilDrainAsync` (introduced with #4757), which currently fails on master and on unrelated PRs, e.g. the master push run [39410642994](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/39410642994) and #4836's run [39410362062](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/39410362062/job/118588752785): ``` Expected: BadEventIdUnknown [0x809A0000] But was: BadSecurityChecksFailed [0x80130000] ``` ### Cause `WotProjectionBindingRuntime.DisposeAsync` disposed the event sources and channel slots (which disposes the captured `WotEventSource` binding) before it cleared its `WotProjectedEventBinding`s. During a shadow reload, the route registry resolves a Condition action's EventId across generations, so a call arriving in that window still resolved on the retiring binding and invoked through the disposed source: `WotCapturedConditionAction.InvokeAsync` → `Source.Validate()` → `BadSecurityChecksFailed`. ### Fix - The runtime clears its event bindings right after cancelling its lifetime, so no further action resolves on them, and waits until the action calls already resolved on them have ended before disposing the sources and slots. - `TryResolveOwnAction` / `ResolveAction` hand out a call handle that the Method handler holds until the invocation ends (`WotProjectedEventBinding.WhenActionCallsCompletedAsync`). All touched types are internal; no public API changes. ### Tests - New `WotRetainedConditionRefreshTests.RetiredBindingWaitsForAResolvedActionCallAndResolvesNoOtherAsync` (both identity modes): a retired binding keeps a resolved call until it is disposed and resolves no other call (`BadEventIdUnknown`). - `NativeRetiringGenerationUsesCapturedActionsUntilDrainAsync` locally (Windows, net10.0), 20 runs each: master 17 passed / 3 failed with `BadSecurityChecksFailed`; this branch 20 / 20 passed. - `WotRetainedConditionRefreshTests`, `WotPrivateEventCaptureTests`, `WotEventCapacityOwnershipTests` (123) and `WotEventModesLiveTests` (74) pass. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
b72eb44a31 |
Redundancy: bound the Raft log with the engine's snapshots (#4836)
# Description **Draft: builds on #4835 and waits for a RaftCs release** containing marcschier/raft-cs#9, marcschier/raft-cs#10 and marcschier/raft-cs#11, anticipated as 1.1.4. It contains the commit of #4835; review only the second commit. `RaftSharedKeyValueStore` can capture and restore its state (#4794), but `IRaftConsensus` carries no log indices, so the store cannot bound the RaftCs log, which grows without limit. This change uses the engine's snapshots: - `IRaftSnapshotConsensus` (`CommittedItems` with log indices and snapshots, `CompactAsync`) and `RaftCommittedItem`. `DefaultRaftConsensus` implements it over `RaftNode.CommittedItems`, `CreateSnapshotAsync` and `CompactAsync`. - `RaftSharedKeyValueStore` gains a constructor overload with a compaction interval and an optional logger (the existing constructors are unchanged) and `AppliedIndex`. Over an `IRaftSnapshotConsensus` it applies the indexed stream. A snapshot on that stream (a lagging replica installing the leader's snapshot, or the first item after a restart) replaces the state and publishes the differences to watchers. Every compaction interval it snapshots its state into the engine, which discards the log behind it. A compaction that fails is logged and retried at the next interval. - The engine snapshot is a format marker followed by the snapshot encoding of #4794, so it carries the command position: the commands after it keep the positions they have on every replica, and `CreateSnapshotAsync`/`RestoreSnapshotAsync` keep working over it. No compaction runs while a store restored with `FromFirstCommand` still skips the replayed commands its snapshot holds. - A snapshot in another format stops the applier. Pending and later proposals and watches then fail with its error instead of waiting. - `AppliedIndex` is published once the state holds an item, before watchers and the proposer learn of it. - Registration: `RedundancyConsistencyOptions.RaftCompactionInterval` (server), and overloads of `AddRaftClientSharedStore` and `AddRedundantClientSharedStore` that take the interval (client). Zero, the default, never compacts. The store's logger comes from the registered `ITelemetryContext`, else `ILoggerFactory`. - `HighAvailability.md` describes the compaction and warns that a replica must not be rolled back to a release without engine snapshots once the cluster compacted its log. No public member is removed or changed. ## Related Issues - #4794 (store snapshots), #4835 (RaftCs 1.1.4) - marcschier/raft-cs#9, marcschier/raft-cs#10, marcschier/raft-cs#11 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. (`RaftSharedKeyValueStoreCompactionTests`; registration tests in Server and Client.) - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Redundancy test projects on .NET 10 against a RaftCs build of the three pull requests: Server 935, Client 135, Kubernetes 103, PubSub 45, Samples 17, all pass; the Server and Client test projects build for net48.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. (CI cannot restore RaftCs 1.1.4 until it is released.) - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
2ba86e3a3b |
Server: queue the items of an event on several threads (#4859)
# Description
E2 of the event-heavy server load investigation (follows #4837 and
#4845).
### Problem
After #4845 (each event is evaluated once per distinct filter), the
event consumer of the Server node still ran on one core.
`MonitoredNode2` started the delivery of an event to every item and
awaited `Task.WhenAll`. Unless a permission check completed
asynchronously, every delivery ran synchronously on the consumer thread,
so one thread queued every item of every event. Each event also
allocated an `IEventMonitoredItem[]` snapshot
(`ConcurrentDictionary.Values`) and a `List<Task>`/`Task[]` sized to the
item count.
### Change (`MonitoredNode2`, multi-consumer nodes such as the Server
node)
- **Collect on the consumer.** Items are enumerated straight from
`EventMonitoredItems`. The audit filter and the #4845 group lookup run
there too. The results go into an `(item, key, evaluation)` struct
buffer that is reused for every batch and cleared afterwards, so the
per-event snapshot array and the per-item task list are gone.
- **Consumer keeps up (no notification waiting in the channel).** It
queues the event itself and does not wake pool threads. It splits only
what is left of an event after 1 ms (a very large event).
- **Consumer is behind (notifications waiting).** It also takes up to 3
more waiting events for the same items. It splits the batch at once into
ranges of at least 32 items, at most `min(ProcessorCount, 8)`, with one
range on the consumer thread. Each range queues the events of the batch
to its items one after the other. An event for different items (for
example an audit event that only reaches secure sessions) ends the batch
and is processed next; a fence or data change ends it too.
- **Ordering.** Every item still gets its events in order. A range waits
for its items' pending (asynchronous) permission checks before it queues
the next event of the batch, and the next batch is read only after all
ranges finished. A delivery that waits for a permission check still does
not hold up the other items.
The node-wide barrier per event now applies per batch: while one item
waits for its permission check, items in other ranges may already get
the next event of the same batch.
### Results
perf/ServerLoadHarness `events`: 50 sessions × 15 subscriptions,
publishing 100 ms, encrypt, `DOTNET_gcServer=1`. Paired A/B against
master
|
||
|
|
bd38870cf7 |
perf: pump the server child's stdout on a dedicated thread (#4855)
## Summary The 1-core client stall in the server load harness (`--client-cores 1`, write->notify p99 0.7-1.7 s) was caused by the harness, not the OPC UA client. `StartServerChild` started `PumpServerOutput` with `Task.Run`. The pump blocks in a synchronous `StreamReader.ReadLine` on the server's stdout for the whole run, so it held one thread-pool worker permanently. With `DOTNET_PROCESSOR_COUNT=1` the pool's minimum is one worker, so all other work items waited for the pool's starvation injection. That is why `DOTNET_ThreadPool_ForceMinWorkerThreads=8` and two processors both hid the stall. **Evidence:** 14 `dotnet-stack` samples of the constrained client during the measurement window. In every sample, the only blocked pool worker was in `ReadFile <- StreamReader.ReadLine <- Program.PumpServerOutput <- StartServerChild`, and the other workers were idle. No `Opc.Ua` frame was blocked. **Fix:** run the pump on a dedicated background thread. ## Numbers `run --scenario sub --sessions 25 --subs 20 --nodes 100 --pub 500 --write-interval 100 --duration 10 --warmup 6 --client-cores 1 --client-memory 512`, 5 alternating rounds (ms): | round | base write->notify p99 | fix write->notify p99 | base write p99 | fix write p99 | |---|---|---|---|---| | 1 | 659 | 61 | 619 | 12 | | 2 | 885 | 310 | 859 | 15 | | 3 | 306 | 303 | 52 | 23 | | 4 | 1717 | 455 | 408 | 81 | | 5 | 234 | 193 | 148 | 67 | The baseline stalled (> 600 ms) in 3 of 5 runs and the fix in 0 of 5 (max 492 ms). The remaining ~300 ms p99 comes from the 500 ms publishing interval. ## Test plan - [x] Harness Release build (net10.0) - [x] Alternating A/B runs above 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
50062cde1b |
PKCS#11 store: pair a certificate without a CKA_ID only with its own key (#4826)
# Description Follow-up to #4792. `Pkcs11CertificateStore.LoadPrivateKeyAsync` skipped a certificate whose CKA_ID cannot be read, but a certificate with an **empty** CKA_ID still reached `Pkcs11Token.FindPrivateKey`, which then selects a key by the configured object filters alone and, without any filter, returns the first private key of the token's key type. That key may belong to another certificate, so the store could return an identity whose signatures its certificate does not verify. A certificate without a CKA_ID is now paired with: - the key that carries its public key (the same lookup `AddAsync` uses to link a certificate it writes: RSA modulus and exponent, or EC point and curve), else - the key that the configured `ObjectLabel` / `ObjectId` selects, but only after that key signs a random challenge that the certificate's public key verifies (a shared label does not establish the pairing), else - no key. Before this change, a certificate without a CKA_ID on a token without object filters got the first key of its type. A certificate with a CKA_ID is handled as before. No public API changes. The package README describes the pairing. ## Related Issues Follow-up to #4792. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Opc.Ua.Security.Pkcs11.Tests on .NET 10, 112 passed / 10 SoftHSM skipped; the library builds for net48, net8.0, net9.0, net10.0.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
5f6faa60d7 |
Server: fewer allocations per single-value Read (isolation reservations, scope reuse, routing capture without restore) (#4856)
Removes per-request allocations on the server's single-value Read path
with small, internal changes. No wire-format change, no public API
change, no change of the execution model (the receive loop is
untouched). Exactly-once semantics of the isolation leases,
cancellation, timeouts, diagnostics and auditing are unchanged.
## Effect
Allocated bytes per single-value Read (perf harness, `native-read`, 10
clients x 100 in flight, `AsyncCustomNodeManager`, Server GC;
deterministic):
| | 1 core | 4 cores |
|---|---|---|
| master
|
||
|
|
65a67dd2f4 |
WoT runtime materialization, binding execution and native source emission (#4757)
# Description - WoT registry, prepared file store, immutable content, version leases, dependency-graph sharing (advanced graph rewrite), coordinator, materialization, views and atomic refresh. - Explicit caller-context propagation (`IWotRegistryService` / versioned registry / prepared host / publication capture) and the additive `IXRegistryProjectionContextualDelete` capability. - UA/HTTP/MQTT binding execution and compilation, plus the 1.02 FileType bridge and discovery. - WoT-specific native event, source-emission and readiness integration (`MasterNodeManager.SourceEmissions`, `EventManager` identity admission, external-value monitored sources, Fluent attach/detach) and event lease conservation. - All corresponding `WotCon.*` runtime tests, WotCon sample app code, WoT docs, CI feature budgets and Samples.Tests, plus the WotCon Design NodeSet/csv/csproj and ModelSourceParity tests. - Joint model metadata: XRegistry model 0.7.0 / 2026-09-12 (NamespaceVersion 63563, NamespacePublicationDate 63564), RequiredModel date 2025-01-08. - XML-doc work from the frozen WIP, applied to G12-owned declarations only. ## Related Issues - Fixes #4751. - Part of the WoT split tracked in #4511. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. (Tests included, not yet run against current master.) - [ ] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. (Not run.) - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Not run.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
0edc3b55a6 |
Server: allocate data change notifications instead of pooling them (#4830)
Stacked on #4829 (publish timer sweep): this PR targets its branch, so the diff shows only this change. Retarget to master once #4829 merges. MonitoredItem.Publish rented every MonitoredItemNotification from the process-wide activator pool (BoundedObjectPool, bound 2 x monitored items, capped at 65536) and NotificationMessageLease returned it once the message was sent and acknowledged. Under load the pool costs more CPU than it saves: - A notification lives until its message is acknowledged, so a whole publishing cycle of notifications is in flight at once. With the publish sweep finishing within a few ms (fixed cadence + parallel remainder) the in-flight burst is far above the cap and most Gets miss anyway (MonitoredItemNotification 6 % -> 45 % of server allocation). - Raising the cap to 1M made it worse: 5.5 vs 4.4 us per notification, lock contention 16k vs 5k per window, delivery 93 vs 95.5 %. - Every hit touches an instance that went cold in the ~100-200 ms since it was returned, Reuse touches it once more, and the shared queue and counters bounce between cores. A fresh allocation comes from the thread's hot allocation context. The notifications are now allocated in MonitoredItem.Publish and left to the GC; RecyclePayload no longer reuses them, and the pool bound is only raised for EventFieldList. NotificationMessage, DataChangeNotification, EventNotificationList and EventFieldList pooling and the rented MonitoredItemNotification[] arrays are unchanged. Paired A/B on top of the publish sweep fix (perf/ServerLoadHarness, encrypted, alternating rounds, medians; runs repeated while processes outside the harness used more than 1.5 cores): | shape (sessions x subs x items) | GC | n | us/notif | server cores | delivered | GC pause / 15 s | p99 write->notify | |---------------------------------|--------|---|------------|--------------|--------------|-----------------|-------------------| | 50 x 15 x 600, pub 100, w 150 | server | 2 | 4.4 -> 3.1 | 11.9 -> 8.2 | 89 -> 89 % | 0.8 -> 1.5 s | 278 -> 325 ms | | 10 x 15 x 600, pub 100, w 150 | server | 3 | 5.1 -> 3.3 | 2.9 -> 2.0 | 100 -> 100 % | 15 -> 88 ms | 137 -> 141 ms | | 1000 x 1 x 1, pub 1000, w 500 | server | 7 | 158 -> 190 | 0.16 -> 0.19 | 50 -> 50 % | 15 -> 28 ms | 480 -> 490 ms | | 2 x 1 x 100, pub 100, w 150 | server | 3 | 59 -> 47 | 0.08 -> 0.06 | 100 -> 100 % | 7 -> 6 ms | 103 -> 114 ms | | 50 x 15 x 600, pub 100, w 150 | ws | 3*| 4.5 -> 3.5 | 10.5 -> 7.0 | 86 -> 96 % | 5.1 -> 8.6 s | 447 -> 373 ms | | 10 x 15 x 600, pub 100, w 150 | ws | 5*| 5.4 -> 3.6 | 3.3 -> 1.9 | 100 -> 100 % | 0.45 -> 1.1 s | 144 -> 155 ms | (*) workstation GC rows combine the A/B with the switch-build runs of the same comparison. Excluded: six runs (both builds) in which the 50-session shape collapsed with the server mostly idle (up to 16.7 s GC pause in the 15 s window, 0-4 % delivered), the GC suspension stall pattern seen before on this machine under external CPU saturation. The 1000-session cores differ by 0.03 with a same-build spread of 0.13-0.25 at ~1000 notifications/s. Trade-off: CPU per notification drops 25-40 % wherever notifications flow, but short-lived notifications now die young: gen0 GCs per 15 s 2 -> 24 (server GC, 10 sessions), gen1 35 -> 154 (workstation GC, 10 sessions), so GC pause time rises about 2x, and p99 latency rises with workstation GC at moderate load (+8 % at 10 sessions). When pooling could pay off again is tracked in #4819. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
4ae2395f37 |
perf: client GC measurement, V2 client by default, container-like client constraints (#4844)
Extends the out-of-process load harness (`perf/`, not part of the product build) so it can measure client-side GC, run the V2 client by default, and emulate a container-sized client. No product code changes. ## Client GC measurement - New `client-gc:` result line from an in-process GC event listener (`GcListener.cs`, runtime GC keyword at informational level): GC mode, collections by condemned generation and trigger, pause and suspension p50/p99/max, bytes surviving gen0/gen1, pinned objects, gen2/LOH size, peak heap after a collection, committed bytes, heap limit, DATAS heap count and gen0 budget. `--gc-events 0` turns it off. - The latency recorder is now a fixed log-linear histogram (~1.6 % resolution). The old `ConcurrentBag<long>` kept ~25 MB of samples alive in the measured client's gen2 (client heap 29–31 MB vs 4 MB now). - **Fix:** GC, thread-pool and processor-count settings set for the harness (`DOTNET_GC*`, `COMPlus_GC*`, `DOTNET_ThreadPool_*`, `DOTNET_PROCESSOR_COUNT`) no longer leak into the server child. Before, `DOTNET_gcServer=1` switched the server too, although the README said it switched the client. `SERVER_`-prefixed variables still configure the server. - TraceAnalyzer `gc` decodes the condemn reasons and breaks each collection into phases (marking by root type with the bytes marked, plan/relocate/compact). ## V2 client by default - `--client` defaults to `managed` (`ManagedSession` + V2 subscription engine). The `events` and `counters` workloads gained V2 paths; `--pool-notifications 1` enables `WithPoolNotifications`. `client-scenarios.txt` lists V2 scenarios. `--client classic` and `shared` still work. ## Container-like client - `--client-cores N`: the harness runs itself again with `DOTNET_PROCESSOR_COUNT=N` on the last N processors and pins the server to the others. - `--client-memory MB`: the client's GC heap hard limit becomes 75 % of MB, the limit .NET derives from a container memory limit (native memory is not capped). ## What it found The report is published separately. Highlights: - Request path: a gen0 pause costs ~1 ms of fixed work (stack and card scanning, compaction), so pause time follows collection count. Read-1 allocates ~2 KB/request on the client, and its p99 is the GC pause (a 64 MB gen0 halves it). - V2 subscriptions and events never reach gen2 (classic `Subscription` with its message cache: 22 % of wall time paused for events). - Containers: each V2 monitored item costs 2.8–3.8 KB, mostly a per-item `ConcurrentDictionary` in `OptionsMonitor<T>` (one lock per processor) and a per-item `ConcurrentQueue`. 100k items do not fit in a 256 MB container. A 1-processor client stalls ~1 s unless the thread-pool minimum is raised. ## Testing - Harness builds in Release (`CustomTestTarget=net10.0`). I ran every changed scenario: read/write/browse/sub/events/counters on managed sessions, constrained runs with 1, 2 and 4 cores and 256 MB to 1 GB, and `TraceAnalyzer gc/alloc` on recorded traces. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com> |
||
|
|
ad8c6ce59f |
TCP server: close the transport only after the terminal error message was sent (#4850)
### Why The weekly CTT regression run reported **Security Certificate Validation `029.js`** as a regression: the OpenSecureChannel with a CA certificate got `BadConnectionClosed` instead of `BadCertificateUseNotAllowed`. It passed in all 10 earlier full runs. When a channel faults (here a rejected client certificate during OpenSecureChannel), `ForceChannelFaultCore` sends the ERR message with `WriteMessageInline` and then closes the transport on the same stack. The send is not awaited. If it cannot complete synchronously, `Close()` cancels it or disposes the socket underneath it: - it may wait for the transport's send lock, for example while the queued writer is still finishing the Acknowledge; - or the socket send may still be pending. The client then sees the connection drop without the reason. OPC 10000-6 §7.1.5 requires the server to send the Error message and then close the connection. ### Change - `UaSCBinaryChannel.WriteMessageInline` keeps the terminal send task. - The new `UaSCBinaryChannel.CloseTransport` closes the transport at once when no terminal send is pending. Otherwise it closes once that send completes, after 2 s at the latest (`TimeProvider`). - `TcpListenerChannel.ChannelFaulted`, `ChannelClosed` and `UaSCBinaryChannel.Dispose` use it in place of `Transport.Close()`. - Nothing else changes: the channel still faults and is removed from the listener synchronously; only the socket close waits. ### Tests - New in `TcpServerChannelDeterministicTests`, both of which fail without the fix: - `FaultClosesTheTransportOnlyAfterTheErrorMessageWasSentAsync`: the ERR send is held, the transport must stay open, then close after the send. - `FaultClosesTheTransportWhenTheErrorMessageNeverCompletesAsync`: the ERR send never completes, so the transport closes after 2 s (FakeTimeProvider). - `Opc.Ua.Core.Tests` Stack.Transport/Stack.Tcp (net10.0): 858/858. - Opc.Ua.Core builds for net48. _Includes the master build fix of #4852 (cherry-picked so CI can run); it drops out on rebase once #4852 is merged._ 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6a156dfa2e |
Client: fewer allocations per request (shared timeout timer, pooled state machines, fewer buffer collections) (#4849)
# Description Reduces client allocations per request. A client GC investigation found that a gen0 pause on the request path costs about 1.1 ms however much survives. That makes the number of collections, and therefore the bytes allocated per request, what matters. At the default gen0 budget, each KB removed from a request removes about one collection in eight. Read-1 client bytes/request (`perf/ServerLoadHarness run --scenario read --nodes 1 --sessions 20`, builds run alternately, 3–4 rounds each, medians). Bytes/op repeated within ±5 B between rounds, while req/s varied 3x from other load on the machine, so throughput is not reported. | Commit | Change | classic | managed | |---|---|---|---| | master |
||
|
|
170d9460a0 |
plans/ctt-issues.md: C63 observed failure and CTT CI workaround (#4851)
Adds detail to the C63 entry, which only had a one-line description: - **What was seen.** In the weekly CTT run of 2026-10-18, A & C Refresh2 `Test_002.js`, `Test_003.js` and `cleanup.js` failed with `BadSubscriptionIdInvalid`, and the unit stopped after `Test_004.js`. - **How often.** With a 250 ms publish interval, the stall still hits about 1 in 5 A&C runs on the GitHub-hosted runner, sometimes inside a unit. - **The CTT CI workaround.** Each unit checks at its start that the publish thread is still alive. A part whose first attempt has regressions together with the CTT's own stall warning runs once more, and the second attempt is the result. No server change. Documentation only. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ac24123dde |
Await NUnit 5 Assert.ThrowsAsync/CatchAsync in the #4759 tests; fixes the master build (#4852)
master has failed to build since #4840 (NUnit 5) and #4759 (batch lifecycle and continuation tests) both landed: - #4759 calls `Assert.ThrowsAsync`/`Assert.CatchAsync` without `await`, which was the blocking NUnit 4 form. - Under NUnit 5 those methods return a `Task`. This PR adds the missing `await`, the same form as the rest of the migrated tests: - `NodeManagerLifecycleTests.Batch.cs`: six calls (`CS4014`, `NUnit2059`). - `ProductionDatasetClientTests.cs`: one call (`CS4014`). - `HistoryReadContinuationDispatchTests.FaultedRequestKeepsAssignedClientSuppliedContinuationPoint`: this test is synchronous, so it compiled. It discarded the returned `Task`, which means the assertion was never checked. It is now `async Task ...Async` and awaits it. No assertion changes. Tests (net10.0): - `UA.slnx` builds. - `NodeManagerLifecycleTests`: 317/317. - `ProductionDatasetClientTests`: 5/5. - `HistoryReadContinuationDispatchTests`: 6/6. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
2997cb8a2c |
SessionBindingTests: let readers confirm the live session before racing transfers (#4848)
## Problem `BindingLivenessRacingTransfersAndCloseNeverConfirmsReplacedBindingAsync` failed on the net48 CI runner (test-windows-net48 11/16, run 39250649914, on docs-only PR #4841) with `confirmed = 0`. The safety check (`errors`) was empty. The reader tasks were not scheduled until all 1000 transfers and the close had finished, so the liveness check never saw a live binding. ## Fix (test only) Before the first transfer, the test now waits until every reader has confirmed the live binding at least once (bounded: 30 s, then a clear `Assert.Fail`). The race still covers transfers and close, but the positive assertion no longer depends on thread-pool scheduling. The safety assertion is unchanged. ## Verification I ran the test with a temporary `[Repeat(100)]` in Release builds: - net10.0 and net48: 100 repeats each, no load - net10.0 and net48: 3 × 100 repeats each with 2 busy loops per core 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a6cc7c4bc9 |
Server: evaluate an event once per distinct event filter (#4845)
## Summary
The event consumer of a notifier (`MonitoredNode2`) evaluated the where
clause and read every select clause of a reported event once **per event
monitored item**, although items with an equal filter and equal session
locales get the same result. With 750 event items on the Server object
this single consumer thread capped event throughput (follow-up to #4837,
item E1 of the event load investigation).
Now each reported event is evaluated **once per distinct filter**:
- The consumer groups the items of an event by a key of their
`EventFilter` (value equality, generated structural
`IsEqual`/`GetHashCode`) and their session's `PreferredLocales`. The key
is cached per item and recreated when the item's filter or the session's
locales change (ModifyMonitoredItems, locale change). A key remembers
the equal key it was matched against, so after the first event a group
compares by reference.
- A per-event `SharedEventEvaluation` holds the where-clause result and
the field values. The first item that needs them evaluates them.
Evaluation is thread-safe, so a later partitioned fan-out (E2) can queue
a group's items on several threads without evaluating again.
- **Per item, unchanged:** duplicate check, permission and
audit-visibility checks, filtered-retain bookkeeping, queue size and
overflow events, and an own `EventFieldList` with its `ClientHandle` and
`Handle`. A trailing filtered-retain event reads its own fields through
`FilteredRetainTarget`. An item whose filter or locales changed after
the event was grouped evaluates the event itself.
- **No public API or wire change, no reflection.** The evaluation is
offered to the virtual `MonitoredItem.QueueEvent` for the duration of
the call (a `[ThreadStatic]` scope keyed by item and target). Derived
items that override `QueueEvent` therefore still see every event, and
they use the shared result only when they pass the same event on to the
base class. Detecting overrides by reflection was rejected because it is
not NativeAOT-safe.
- The shared `Variant[]` is never written after it is created.
`EventQueueHandler.Publish` only touches the `StatusResult` objects
inside it, which were already shared through the event node,
`TransformQueuedEvents` builds new lists, and pooled `EventFieldList`
reuse only reassigns its fields.
## Measurements
`perf/ServerLoadHarness`, `events` with 50 sessions × 15 subscriptions
(750 items on the Server object), publish interval 100 ms, encrypt,
server GC, `TC_QuickJitForLoops=0`/`OSR=0`. Paired A/B in 2 alternating
rounds; each run was repeated if processes outside the harness used more
than 1.5 cores. The base was #4837's tip (
|
||
|
|
d7c2ed6caf |
Client: smaller per-monitored-item footprint (listener array, lazy pending-change queue, shared logger) (#4847)
# Description Every V2 monitored item (`Opc.Ua.Client.Subscriptions`) cost about 3.8 KB of managed heap on a 24-core machine (2.8 KB on 1 core). Most of that was per-item infrastructure whose size grows with the processor count. With 100k items, a client limited to 256 MB (container-like) ran out of memory during subscription setup. - **`OptionsMonitor<T>`**: `OnChange` listeners are kept in a `Listener[]` that is replaced as a whole on each add or remove (`Interlocked.CompareExchange`), instead of a `ConcurrentDictionary`. That dictionary's default concurrency level allocated one lock object per processor for every monitored item (~1.4 KB on 24 cores). Notification, dispose and double-dispose behave as before; listeners are now called in registration order. - **`MonitoredItem`**: pending changes are kept in a `Queue<Change>` guarded by the existing `m_pendingChangesLock` instead of a per-item `ConcurrentQueue<Change>` (~810 B, 32-slot segment). The queue is created on enqueue and dropped once the last change completes, so idle items hold none. `HasPendingChanges`, `TryGetPendingChange` and `CompleteChange` now take the lock. `TryRequeue` used to enqueue without the lock and now enqueues under it. - **`DefaultSubscription`** shares one `ILogger` across its monitored items instead of creating one per item. ## Measurements These numbers come from the `ServerLoadHarness` client GC line (harness changes are in #4844): `run --scenario sub --sessions 25 --nodes 100 --pub 500 --write-interval 100 --duration 10 --warmup 6`. All figures are client gen2 MB. | Config | Items | Before | After | |---|---|---|---| | 24 cores, `--subs 20` | 50k | 190.0 / 188.7 | 62.9 / 64.0 | | `--client-cores 2 --client-memory 256`, `--subs 20` | 50k | 142.6 | 54.6 | | `--client-cores 2 --client-memory 256`, `--subs 40` | 100k | out of memory | 122.6 (limit 192) | That saves about 2.6 KB per item on 24 cores and about 1.8 KB on 2 cores. The whole client heap is now about 1.3–1.4 KB per item. ## Related Issues - None ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. (`OptionsMonitorTests`: order, removal, duplicate callbacks, concurrent add/remove) - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. (`Opc.Ua.Client` Release build on all targets, including net48: 0 warnings) - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Only `Opc.Ua.Client.Tests` on net10.0 was run: 3231 passed, 0 failed, 2 skipped.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9bfc780e14 |
Update NuGet dependencies and migrate tests to NUnit 5 (#4840)
# Description Update 55 NuGet dependencies to the latest stable releases available on 2026-10-19, preserving documented compatibility pins and framework-specific version bands. Existing OpenUSD prereleases remain unchanged because no newer releases are available. Migrate existing tests to NUnit 5: await asynchronous assertions, propagate async through helpers, handle nullable predicates, and replace obsolete `Order` attributes with sequencing dependencies. Sequencing dependencies use `AllowFailure = true`; filtering a dependent test also runs prerequisites. Adjust fixture-level parallelism where NUnit dependency scheduling requires it. Keep Roslyn API/runtime pins intact; select newer Immutable/Metadata runtime packages only for test and profiling hosts requiring them. Update the approved SDK pin to 10.0.402 across local, CI, and container builds. Document NUnit 5 conventions. |
||
|
|
a07f4eb1ac |
Aggregates: Average, Count, Delta, StdDev, Variance use Good values only (Mantis 11495) (#4846)
### Why The OPC Foundation answered [Mantis 11495](https://mantis.opcfoundation.org/view.php?id=11495) on 2026-10-12 (~0025874), as it had in [7975](https://mantis.opcfoundation.org/view.php?id=7975) before: - The general rules of Part 13 §4.2.1.2 apply only where an aggregate definition says nothing else. - Count counts only Good values; Bad and Uncertain values are never counted, but they affect the status. Average (§5.4.3.3), Delta (§5.4.3.27) and StandardDeviation*/Variance* (§5.4.3.37–§5.4.3.40) also speak of Good values. With TreatUncertainAsBad = False the server treated Uncertain values as Good for these aggregates. That is CTT item U5 in `plans/ctt-issues.md`: about 34 test cases in Aggregate `003-02/03`, `004-02/03`. ### Change - `AggregateCalculator.IsGoodValue`: Good only, whatever TreatUncertainAsBad is. It is used by Average, Count, Delta and StandardDeviation/Variance. - `GetValueBasedStatusCode(..., goodValuesOnly: true)` is used by Average and Count: an Uncertain value counts as non-Good, so 2 Good + 1 Uncertain gives `UncertainDataSubNormal`, not Good. - Unchanged: - NumberOfTransitions: its definition compares non-Bad values; U6 is still open. - Time-based aggregates, bounds, and Minimum2/Maximum2 still follow TreatUncertainAsBad as before. ### Tests - Updated to the answer: `AverageExcludesUncertainValues` and `DirectAndLiveCountExcludesUncertainValuesAsync`. - New: `DirectAndLiveGoodValueAggregatesExcludeUncertainValuesAsync`, which covers Delta, VarianceSample and StandardDeviationPopulation, direct and live, with both TreatUncertainAsBad values. - `ProcessedReadUsesAdvertisedNodeAggregateDefaultsAsync` now checks the advertised TreatUncertainAsBad default through TimeAverage, because Count no longer depends on it. - Local runs (net10.0): - Opc.Ua.Server.Tests aggregates: 496/496; - Opc.Ua.History.Tests: 536 passed, 28 skipped; - client Historian tests: 118/118. - CTT p06 (Aggregates) against this build, with the patched scripts no longer correcting U5: 1264 pass, 0 errors, 0 warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
a10f9e252d |
Generic runtime continuation, batch lifecycle and readiness (#4759)
# Description This pull request extracts and integrates generic server runtime and continuation-management capabilities for OPC UA NodeManagers. - **NodeManager lifecycle and routing:** add batch registration and reload, staged publication and readiness notifications, captured routing snapshots and reservations, and safe retirement. Wire the lifecycle through host interfaces, async/sync adapters, `MasterNodeManager`, and `StandardServer` startup and shutdown. - **Browse and History continuations:** retain continuation ownership and manager dependencies across requests and reloads; add atomic claims, session caching, and cleanup when requests fail or generations retire. Add historian resume state, tokens, dependency capabilities, and shared continuation-store support. - **MonitoredItem lifecycle:** support detaching and reattaching items across node-manager generations, including event subscriptions shared across root notifiers and correct defaults for detached-item modification. - **Runtime NodeSet references:** track reference additions and deletions through supported API paths and preserve the tracked references during reload. - **Published views and materialization:** add immutable `TypeTable` and `EncodeableFactory` snapshots with publication-owner lifetime management, and provide a live node-materialization query for the existing WoT projection-view consumer. - **CI trace support and documentation:** add scripts and tests for staging NUnit internal traces, and document NodeManager lifecycle, session continuation ownership, and historical-access behavior. ## Related Issues - Extracts the generic runtime portion of #4511. - #4757 depends on this pull request; this pull request does not depend on #4757. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: perf <perf@local> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
|
e8c5d800a6 |
plans/ctt-issues.md: refresh Mantis states and OPC answers, condense (#4843)
Brings `plans/ctt-issues.md` up to date with Mantis as of 2026-10-19 and makes it shorter (427 → 347 lines). Documentation only. ### Mantis state - Resolved issues are marked **(fixed)**: 11097, 11106, 11125/11248, 11144/11269, 11257, 11259, 11262, 11263, 11268, 11347. - Abstracts were corrected after the Mantis discussions: C10, C11, C13, C23, C24, C29, C46, 15, 18 and 19. - New rows: - C15b ([11476](https://mantis.opcfoundation.org/view.php?id=11476), View validator); - [11475](https://mantis.opcfoundation.org/view.php?id=11475) on item 18; - duplicates [11466](https://mantis.opcfoundation.org/view.php?id=11466) (C1) and [11509](https://mantis.opcfoundation.org/view.php?id=11509) (item 14). - **C64:** Int32 bound truncation, now filed by another reporter as [11494](https://mantis.opcfoundation.org/view.php?id=11494) (spec clone [11499](https://mantis.opcfoundation.org/view.php?id=11499)). It replaces the unfiled "Int32 conversion" text. - **C21:** the OPC Foundation decided on [11411](https://mantis.opcfoundation.org/view.php?id=11411) to test negated Like lists only at the start or end of a string until Part 4 is clarified. ### OPC Foundation answers - **U4** is rewritten around the common rule from Paul Hunkar's answers: *the aggregate definition wins over TreatUncertainAsBad* ([11462](https://mantis.opcfoundation.org/view.php?id=11462), [11425](https://mantis.opcfoundation.org/view.php?id=11425), [11426](https://mantis.opcfoundation.org/view.php?id=11426), [11427](https://mantis.opcfoundation.org/view.php?id=11427), [11495](https://mantis.opcfoundation.org/view.php?id=11495)). - It records that 11427 ~0025875 and 11426 ~0025852 conflict for an Uncertain value below the minimum. - The 11462 follow-up (~0025849) is still unanswered. - **U5** is answered by 11495 ~0025874 (and the earlier 7975): Count (and by the same rule Average, Delta, StandardDeviation* and Variance*) excludes Uncertain values also with TreatUncertainAsBad = False. **The server currently includes them, so a server change is needed.** - **U6** (new, open): NumberOfTransitions with TreatUncertainAsBad = True. - A new **Server follow-ups** section lists what the answers require of the server: U5, a possible C48/C49 change, C38 diagnostics visibility (11445 ~0025750), and C64 depending on 11499. ### Not filed - Added C58–C63: CTT script defects fixed in the patched CTT scripts, including the C60 publish-thread stall. Also added C65 (the GetBadData forward read, related to [11505](https://mantis.opcfoundation.org/view.php?id=11505)) and C66 (non-numeric aggregates; not filed because the CTT's values were not recorded). - Removed the inline draft Mantis texts for C54/C55, and condensed the prose and the configuration notes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
42bfb709c5 |
Channel send order: tests for concurrent producers, reconnects and the writer stop (#4838)
Test-only. Adds tests for the guarantees of the channel writer introduced in #4782 ("seal in the writer"). The existing tests cover them for a few producers or one message at a time. No product code changes. They come from roadmap item R6 of the o6 benchmark gap analysis, a lock-free queue for the producers' responses. That queue no longer pays off once #4805 and #4833 are in: at steady state the send path costs about 0.5 µs per request, and removing its lock left CPU per request unchanged. So the queue is not proposed, but its tests check the current writer and pass on master unchanged. ## Tests All in `UaSCBinaryChannelSendOrderTests`: - **`ManyProducersWithNumberedMessagesAndTokenChangesKeepTheWireOrderAsync`** (repeated 3×): - at least 32 producer threads (twice the processor count) send responses of random size, some split into chunks; - meanwhile, one producer queues messages numbered under `SendOrder`, as OpenSecureChannel responses and faults are, and another activates 15 new tokens. - Checked: consecutive sequence numbers on the wire, contiguous chunks, every response exactly once, and no token used after its successor. - **`ResponsesQueuedWhileTheWriterStopsAreSentAsync`**: one producer queues each response the moment the previous one reaches the wire, which is when the writer finds its queue empty and stops. It runs 20,000 times, and every response must be sent. - The producer waits with `Task.Yield`, not by spinning. The writer is queued to the producer's local work queue, and a spinning producer would hold it. - It uses one producer on purpose: with several, the next producer's response restarts the writer and carries a stranded one along, which hides the bug. - **`AReconnectWhileProducersQueueSendsTheOpenResponseFirstAsync`** (repeated 3×): eight producers keep sending while the transport is replaced. The new transport is attached and the OpenSecureChannel response numbered under one hold of `SendOrder`, as a reconnect does. Checked: - the OpenSecureChannel response is the first message on the new transport, and the transport numbers on without a gap; - nothing is sent on both transports; - every response queued after the reconnect arrives; - every buffer is returned. - **`ASealFailureInTheMiddleOfABatchStopsTheNumberingAsync`**: six responses wait in one batch behind a held send. The first fails to secure after it took its sequence numbers. Checked: - no message after it is sent, including one queued after the encryption works again; - the channel reports one transport error; - every buffer is returned. Test helpers: - `OrderServerChannel`: `ReconnectForTest`, `ActivateTokenForTest`; - `RecordingTransport`: `SendCount`, `SentChunks()`; - the test class: `AssertContiguous`, for a transport that only carries part of the messages. ## Mutations Each test was run against a deliberately broken channel. Each break was temporary and is not in this PR: | Change to `UaSCUaBinaryChannel` | Fails | |---|---| | A producer reads the writer's running flag before it takes the write queue lock, so a writer that stops in between strands the response | `ResponsesQueuedWhileTheWriterStopsAreSentAsync` (3 of 3 runs) | | The writer takes unsealed responses ahead of producer-numbered writes | `ManyProducersWithNumberedMessagesAndTokenChangesKeepTheWireOrderAsync`, `RandomReceiveSendAndRenewalInterleavingKeepsTheWireOrderAsync` | | The writer numbers again after a seal failure left a gap | `ASealFailureInTheMiddleOfABatchStopsTheNumberingAsync`, `ASealFailureAfterNumberingStopsTheChannelAsync` | ## Testing - Local, net10.0: `UaSCBinaryChannelSendOrderTests` and `TcpReconnectKeyAgreementTests`, 61 passed. - net48 builds without warnings. `dotnet format whitespace --verify-no-changes` passes on the changed file. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
33d6d29e12 |
Server: wake fair queue workers through a LIFO signal, let a looking worker take new requests (#4833)
Request queue workers of the `FairRequestQueue` waited on a bounded `Channel<bool>(1)` for the coalesced wake token. With many cores, its monitor became the server's largest lock contention point, and each wake cost a channel operation, an allocated wait and a thread-pool hand-off. This PR replaces that signal and lets a worker that just finished a request keep looking for the next one for a few microseconds before it parks. The concept is unchanged: - requests are admitted into the fair queue; - `MinRequestThreadCount`/`MaxRequestThreadCount` workers dequeue and run them; - the receive loop is not touched; - weighted fairness, the per-owner and global limits, leases, admission and `BadServerTooBusy`, cancellation with an exactly-once lease release, stop and drain, and parked publish requests behave as before. There are no wire, public API or configuration changes. ## Results Paired A/B against `master` |
||
|
|
0f85a9f719 |
AutoID: the OPC UA for AutoID Devices companion SDK (OPC 30010) (#4825)
# Description This completes the AutoID contribution proposed in #4824: the model package for [OPC 30010 OPC UA for AutoID Devices](https://reference.opcfoundation.org/AutoID/v101/docs/) (1.01) plus the server, the client, an MCP profile and a sample pair, in one PR. It follows the newest companion pattern (GMS #4765, MachineTool, CranesHoists, MachineVision). ## Commits 1. `fix(sourcegen)`: a structure property whose DataType a subtype refines is redeclared with its own field (the NodeSet needs it; unchanged from the first slice). 2. `feat(autoid)` model package `Opc.Ua.AutoID` (unchanged from the first slice). Adds only `AutoIdReaderKind`. 3. `feat(autoid)` server package `Opc.Ua.AutoID.Server`. 4. `feat(autoid)` client package `Opc.Ua.AutoID.Client`. 5. `feat(autoid)` MCP profile `Opc.Ua.Mcp.AutoID`. 6. `feat(autoid)` samples `samples/AutoID`. 7. `build(autoid)` solution, package list and package validation tests. 8. `fix(autoid)` x3, `docs(autoid)` and `test(autoid)`: review fixes, net48 compatibility, documentation, the native AOT test. 9. `fix(autoid)` x3 and `feat(autoid-client)`: `GetLocation` location types, the `Client` of access events, the default scan cycle of the simulated readers, and NodeIds and methods in the device snapshot (details below). 10. `feat(autoid)`, `test(autoid)`, `docs(autoid)` (`afdd7858b`, `22bc3b26e`, `ed25fe4ae`, on top of the master merge `1c7948f65`): AutoID composes into an existing Device Integration owner like Pumps and Scales (review feedback; see "Composition with other DI-based companions"). 11. `1458af9e9` (marcschier): the `AutoIdReadOptions` remarks state that `StatusOnly` limits the values read and that each call still browses (Copilot review). 12. `fix(autoid)` `3ef2f1507`: `GetDeviceLocation` answers with the same status codes as `GetLocation`; `feat(autoid-mcp)` `f6405152f`: `autoid_read_device` shows the new snapshot NodeIds and methods (details below). ## Server: `Opc.Ua.AutoID.Server` - Reader devices (RFID, OCR, optical, optical verifier, RTLS) on the DI device model, created by a fluent builder (`AutoIdRuntime.Reader(name, provider)`) and hosted with `AddAutoId` / `ConfigureAutoId` on `IServiceCollection`. - Hardware is behind provider interfaces: `IAutoIdReaderProvider`, `IAutoIdScanProvider` (Scan), `IAutoIdAsyncScanProvider` (ScanStart/ScanStop), `IAutoIdRfidProvider` (ReadTag, WriteTag, WriteTagID, KillTag, LockTag, SetTagPassword, each opt-in by flag), `IAutoIdRtlsProvider`. Providers push scans, presence, log entries and access events through `IAutoIdReaderSink`. In-memory providers (an EPC Gen2 tag model with banks, passwords, locks and kill) are included for tests and samples. - Optional children (methods, `LastScanData`, `DeviceLocation`, `AntennaNames`, location methods) exist only when the provider supports them. Facet profiles and conformance units are advertised only for what is implemented (`IConformanceContributor`); the Full facet is withheld. - `GetLocation` (OPC 30010 6.6.3.2) returns the location in the requested `LocationType`. An unsupported type is `Bad_InvalidArgument`, an unknown object `Bad_NotFound`, and a supported type the object's location is not available in `Bad_InvalidState` (6.6.3.2: "not available or not allowed e.g. due to special configuration"); a provider result of another type never reaches the client. `InMemoryRtlsProvider` takes an optional location converter, for example from the configured local coordinate system (9.3.4) into WGS84. - `GetDeviceLocation` (6.1.3.7) uses the same mapping: an undefined location type is `Bad_InvalidArgument`, a device location that is not available in the requested type `Bad_InvalidState` (it answered `Bad_NotSupported` before). 6.1.3.7 lists only "Standard OPC UA status codes defined for the Call Service in OPC 10000-4"; there `Bad_NotSupported` means "The Method is not supported for the Object instance", which is wrong for a published method, and OPC 10000-4 5.12.2.4 allows reusing an existing status code with the same canonical meaning, so the method follows 6.6.3.2. - Access events (7.10, 7.11): `Client` is "the name of the client application which was the originator of the command", so it is the application name of the session's client description (else its application URI), not the user identity. The results of a `ScanStart` scan name the client that started it; scans the device reports on its own name the device. - The simulated readers default to one scan cycle per second (`CycleInterval`, configurable per provider; the length of a cycle is device specific in OPC 30010), so an unbounded `ScanStart` raises at most one scan event per second. - Behaviour: one scan at a time with a generation counter; ScanStart/ScanStop serialised; a scan started by a session stops when that session closes; the maximum scan duration, tag read length and result count are limited by `AutoIdServerOptions`; device status changes and events are raised outside locks; registration is rolled back on failure; readers, providers and the runtime are disposed in order and in-flight operations are drained. ## Composition with other DI-based companions A server has one Device Integration owner, and AutoID now uses the existing harmonized composition instead of requiring its own node manager: - `AutoIdMachineryModelProvider` adds the AutoID model to the Machinery owner's model-provider pipeline (dependency-ordered, `Order` 200) and keeps the DI nodes the owner loaded, so DI and `DeviceSet` exist once. - `AddAutoIdToMachinery` (after `AddMachinery`) registers it; `ConfigureAutoIdFor<TNodeManager>` creates readers on any `DiNodeManager` owner, successive delegates share one `AutoIdRuntime`; `owner.CreateAutoIdRuntime()` is the path without DI hosting. The owner disposes the runtime on shutdown. - Readers are registered in the owner below its `DeviceSet`, NodeIds in the owner's instance namespace (`AutoIdRuntime.InstanceNamespaceIndex`), not in the `Machines` folder; the AutoID conformance units and facets are published by the owner next to the DI and Machinery ones. - `AddAutoId` remains for the "AutoID is the only DI-based companion" case and is built from the same provider. Next to another DI owner it is still rejected; the message names the composition API. (Bypassing that check gives two `DeviceSet` copies; the server routes `DeviceSet` to the first owner and readers in the second are not browsable, checked locally.) - `Opc.Ua.AutoID.Server` now references `Opc.Ua.Machinery.Server` (a stable package), as Vision does; a stand-alone server still loads only DI and AutoID. ## Client: `Opc.Ua.AutoID.Client` Reader discovery (a failed browse propagates), typed calls for every published method, a device snapshot with documented `AutoIdReadOptions`, and an async event stream (`ObserveScan/Access/Presence/LogEntries`) on `IStreamingSubscription`. Cancellation is honoured throughout. `AddAutoIdClient` registers it for DI. Because most children of a device are optional (6.1, 6.5.3, 6.6.3), the snapshot also carries the NodeIds of `DeviceStatus`, `LastScanData`, `LastScanTimestamp` and `DeviceLocation` and the browse names of the published methods (`Methods`, `HasMethod`), taken from the same browse whatever the read options, so a client can monitor the device and knows what it may call without browsing again. ## MCP profile: `Opc.Ua.Mcp.AutoID` `McpToolProfile.AutoId` with nine tools: list readers, read device, scan, scan start/stop, observe scan events, read tag, get supported location types, get location. `autoid_read_device` also returns `deviceStatusId`, `lastScanDataId`, `lastScanTimestampId` and `deviceLocationId` (NodeId strings, null when the device does not publish the variable) and `methods` (the browse names of the published methods), built explicitly without reflection. Registered in the host builder, the profile switches of the other profiles, `Test-McpGrade.ps1`, `McpREADME.md` and `.mcp/server.json`. ## Samples `samples/AutoID/AutoIdServer` (a dock-door reader with a simulated pallet) and `AutoIdClient` with a README and `Test-AutoIdSamplePair.ps1`. The default port is not a common one; tests use port 0. ## Other changes - `Opc.Ua.Di.Server` gets `InternalsVisibleTo` for `Opc.Ua.AutoID.Server` (`RegisterRootNotifierAsync`). - `UA.slnx`, `.azurepipelines/expected-packages.txt`, `PackageVersionPolicyTests`, `NuGetPackageSetValidationTests`. - Docs: `docs/AutoID.md`, `docs/McpServer.md`, `docs/Profiles.md`, `docs/Samples.md`, `docs/DeveloperGuide.md`, `docs/README.md`, `docs/WhatsNewIn2.0.md`, `README.md`, NugetREADMEs. The "model package only" statements are removed. - All packages follow the root version (no preview override). ## Specification notes and questions for the maintainers - The model is 1.01, published 2020-06-18 (the NodeSet header). OPC 30010 6.1.3.16 prescribes the `AutoIdModelVersion` string `"1.00"`, which is what the NodeSet carries and what the server publishes. - **Question:** the NodeSet's optical verifier `Scan` method returns `WGS84Coordinate`, which looks like a model error. It is not published, and providers that implement `IAutoIdScanProvider` for a verifier are rejected. Is that the behaviour you want? - The stack answers a call to an unpublished optional method with `BadNotImplemented`. ## Design choices reviewers may want changed - `Opc.Ua.Di.Server` gets `InternalsVisibleTo` for `Opc.Ua.AutoID.Server`, so the server can call `RegisterRootNotifierAsync`. - `AutoIdReaderKind` (and `AutoIdReaderKinds.TypeIdOf`) lives in the model package `Opc.Ua.AutoID`, not in the server package, so the client and the MCP tools share it. - The MCP profile is read-only/safe: no tools that write, kill or lock tags. ## Tests `tests/Opc.Ua.AutoID.Tests` has 306 tests (116 of the model slice, 190 new): reader lifecycle, scan semantics, tag access handlers, limits, conformance, in-memory providers, hosting, and over-the-wire client/server tests with result codes, events, concurrency and dispose. `tests/Opc.Ua.Tools.Tests` has 13 AutoID MCP tests plus the updated profile-count tests. New behaviour tests fail without the code (for example a provider's own cancellation, a synchronous scan while a scan stop is in progress; for the last round: `GetLocation` of a supported type the object is not located in answered `Bad_NotFound`, access events named the user "Anonymous" and attributed `ScanStart` results to the device, and an unbounded `ScanStart` on a default simulated reader reported 20 times in 400 ms). Composition: `AutoIdCompositionTests` (stock Machinery owner, providers out of order, DI loaded once, instance ownership, rollback, cancellation, runtime disposal, an application-owned DI owner, a second DI owner rejected) and `AutoIdMachineryCompositionEndToEndTests` (Machinery + Pumps + AutoID in one hosted server, read back by the AutoID, Machinery and Pumps clients, including a scan and the published profiles and conformance units). Mutating the change (no lifetime registration, the DI instance namespace for browse names, the old ownership message) fails three of them. The sample pair script passes. `tests/Opc.Ua.Aot.Tests.Mcp` has an AutoID profile test (`AutoIdMcpAotTests`); the project was published with Native AOT on osx-arm64 and all 64 tests passed. Publishing it also found two `JsonArray.Add<T>` uses in the MCP tools (IL2026/IL3050), now fixed. ## Verification (local, macOS arm64, SDK pinned in `global.json`) The branch contains the master merges `fc556be7e`, `1c7948f65` and `af7feb886`; the three composition commits sit between the last two (the latest merge touches no AutoID, DI or Machinery file). `1458af9e9` (marcschier) and the two commits of the last round follow `af7feb886`. - **Last round (`3ef2f1507`, `f6405152f`).** Builds with `-m:2 -nodeReuse:false -p:UseSharedCompilation=false`: `Opc.Ua.AutoID.Tests` on all target frameworks and with `-p:CustomTestTarget=net48`, `Opc.Ua.Mcp.AutoID` on net8.0/net9.0/net10.0, `Opc.Ua.Tools.Tests` on net10.0: 0 warnings, 0 errors. `Opc.Ua.AutoID.Tests` 306/306 on net10.0; `Opc.Ua.Tools.Tests` AutoID tests 60/60, and the profile, catalogue, composition and package tests that reference AutoID (2241) pass. The changed tests fail without the code: `GetDeviceLocation` answered `Bad_NotSupported` for both cases, and `autoid_read_device` had no `deviceStatusId`. `tests/Opc.Ua.Aot.Tests.Mcp` published with Native AOT on osx-arm64 (no IL2026/IL3050 warning): 64/64 passed. `dotnet format` whitespace, style and analyzers `--verify-no-changes` on the changed files: no finding on a changed line (line endings aside, which the LF checkout on macOS reports for every file). - **Build (composition round).** `Opc.Ua.AutoID.Server` on all target frameworks, `Opc.Ua.AutoID.Tests` on net10.0 and with `-p:CustomTestTarget=net48`, and the AutoID server sample: 0 warnings, 0 errors. MCP tools and JSON unchanged, so the native AOT test was not republished. - **Build (earlier rounds).** The AutoID packages and `Opc.Ua.AutoID.Tests` on net10.0 and, with `-p:CustomTestTarget=net48`, on net48/net8.0/net9.0/net10.0; `Opc.Ua.Tools.Tests` and `Opc.Ua.SourceGeneration.Core.Tests` on net10.0 (`-m:2 -nodeReuse:false -p:UseSharedCompilation=false`): 0 warnings, 0 errors. - **Tests.** `Opc.Ua.AutoID.Tests` 306/306 on net10.0 (composition round; earlier round 288/288); `Opc.Ua.Tools.Tests` (AutoID, host builder, profile and package tests) 2438/2438 on net10.0; `Opc.Ua.SourceGeneration.Core.Tests` 4358 passed, 8 skipped, after the master merge touched `NodeStateGenerator.cs`. net48 was built, not run (no Mono on macOS); the fork CI runs it. The MCP tools and their JSON are unchanged in the last round, so the native AOT test was not republished. - **Coverage.** Composition round: the local coverlet collector recorded zero hits for every module in this environment (also for unchanged code), so no new local number; the new paths are exercised by the 18 composition tests listed above and the fork CI uploads coverage. Earlier round: cobertura from the AutoID tests: `Opc.Ua.AutoID.Server` 92.0 % lines / 86.8 % branches, `Opc.Ua.AutoID.Client` 89.9 % / 80 %; 110 of the 112 coverable changed lines of the last round are covered (the two missed lines skip a browse result without NodeId or name). The generated code of the model package is excluded as in the other model packages. - **Format.** `dotnet format` whitespace, style and analyzers (`--severity info`) `--verify-no-changes` on the changed files: clean. Fork CI (full matrix including macOS): [run 39162077465](https://github.com/biancode/UA-.NETStandard/actions/runs/39162077465) on `f6405152f`: green. (Previous round: run 39130154771 on `ed25fe4ae`, 90/90 jobs green; the upstream checks on the merge `af7feb886` green, 108 passed, 5 skipped.) CodeQL: [run 39162079826](https://github.com/biancode/UA-.NETStandard/actions/runs/39162079826) on `f6405152f`: green, no alert in a file this PR touches. (Previous round: run 39130156598 on `ed25fe4ae`, green.) ## Related Issues - Part of #4824 (tracking issue: AutoID companion SDK). ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (The solution builds on all TFMs; the AutoID test projects ran on net10.0 locally, net48 runs in the fork CI.) - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
|
484596c2a8 |
Server: cheaper per-item event delivery (O(1) dequeue, duplicate index, filter context reuse, no per-read timestamps) (#4837)
# Description Low-risk speedups in the server event delivery path. With many event monitored items on one notifier (750 on the Server object in the `events` harness scenario), a single `MonitoredNode2` consumer thread delivers every event to every item and is 100 % busy, which caps the server at ~770 events/s. This PR makes the per-item work cheaper. There are no public API or wire changes. Each commit stands alone: 1. **Server: dequeue events in O(1).** `EventMonitoredItemQueue.Dequeue` used `List.RemoveAt(0)`, which shifted every queued event. It now advances a head index, compacts the list once the dequeued slots are half of it, and clears it when it runs empty. Discard oldest/newest, the `SetQueueSize` truncation and the newest-1000 duplicate range are unchanged. The mirroring redundancy queue snapshots only the queued range. 2. **Server: reuse the event filter context of a monitored item.** `QueueEvent` created a `FilterContext` per event and item. The item now keeps it until the session replaces its `PreferredLocales`. `GetEventFields` writes into a `Variant[]` instead of a `List` that was then copied, and translates with the context's locales instead of looking up the session per LocalizedText field. 3. **Server: check event duplicates against an index of the newest entries.** `IsEventContainedInQueue` loaded the handle of up to 1000 queued entries per item and event. A reference-keyed count of the event instances in the newest 1000 entries, kept current by `Enqueue`/`Dequeue`/`SetQueueSize`, makes it one lookup. The checked range is the same, and wrapped filtered-retain handles count as their instance. If a derived queue changes `m_events` directly, the index is rebuilt. 4. **Core: evaluate an empty content filter without an evaluator.** `ContentFilter.Evaluate` created a `FilterEvaluator` and, through the telemetry context, a new logger for every evaluation, also for the empty where clause of most event filters (~16 % of the scenario's allocated bytes). An empty filter now passes directly, and the evaluator creates its logger only to log a failure. 5. **Core: give the fields of an initialized event a source timestamp.** `BaseEventState.Initialize` created its field variables without a timestamp, so `BaseVariableState.ReadValueAttribute` called `DateTimeUtc.Now` on every read of every field for every monitored item. The fields now carry the event time (`ReceiveTime` its own value). The read path is unchanged, so no `ReadChildAttribute`/`ReadValueAttribute` override is bypassed. ## Measurements `perf/ServerLoadHarness run --scenario events --sessions 50 --subs 15 --pub 100 --security encrypt --event-rate 0 --duration 15 --warmup 8`, with `DOTNET_gcServer=1`, `SERVER_DOTNET_gcServer=1` and `SERVER_DOTNET_TC_QuickJitForLoops=0`, `SERVER_DOTNET_TC_OnStackReplacement=0`. Each build is stacked on the previous one; values are the mean of 2 alternating rounds. Runs where processes outside the harness used more than 1.5 cores were repeated. | build | generated events/s | event→notify p50 | KB / generated event | ms CPU / generated event | |---|---|---|---|---| | master | 768 | 69.8 ms | 567 | 2.97 | | + O(1) dequeue | 791 | 68.4 ms | 566 | 2.78 | | + filter context reuse | 818 | 68.8 ms | 354 | 2.75 | | + duplicate index | 854 | 63.7 ms | 356 | 2.96 | | + empty where clause | 891 | 63.7 ms | 273 | 2.77 | | + event field timestamps | **1001** | **61.3 ms** | **272** | **2.66** | - **`--event-rate 500` (below capacity):** 3.34 → 3.03 ms CPU and 590 → 294 KB per generated event. Latency is unchanged (bound by the publishing interval). - **Default JIT, 40 s warm-up:** 804 → 1063 events/s, 568 → 268 KB, 2.69 → 2.37 ms. - **CPU traces:** the consumer thread's `ReadValueAttribute` self share drops from 24 % to 6 % of the samples and `QueueEvent` self from 14 % to 5 %. The thread is still the bottleneck, now in the generic read chain (`GetAttributeValue` → `ReadChildAttribute` → `FindChild`). The next step would be to evaluate each event once per distinct filter rather than per item, which is out of scope here. ## Notes for reviewers - `EventMonitoredItemQueue.m_events` keeps its protected `List<EventFieldList>` type, but dequeued slots now stay null at the front until compaction. The queued events are the last `ItemsInQueue` entries. Nothing in the repository reads the field outside the class except the mirroring queue, which uses the new internal `CopyQueuedEvents()`. - Event field variables created by `Initialize` now report the event time as `SourceTimestamp` instead of the time of the read. ## Related Issues - No issue. Found by the server load investigation of publish-, write- and event-heavy load. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (net10.0 only: Opc.Ua.Server.Tests 8990 passed, Opc.Ua.Core.Tests 6449 passed, Opc.Ua.Redundancy.Server.Tests 922 passed. In the explicit Sessions `ServerEventSubscribeLoadTest`, opc.tcp passes; https and opc.https fail with HTTP 503, the same as on master.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
37a2a831f3 |
Server: keep the publish timer on a fixed cadence and parallelize the sweep by work (#4829)
The publish timer loop ran a pass over all session publish queues and
then
waited a full PublishingResolution, so the cycle period was pass time +
resolution and a slow pass delayed every following cycle. Whether the
pass
ran in parallel depended only on the session count (256 or more).
Fixed cadence: the loop now waits only for what is left of the period
(RunAtFixedPeriodAsync on the server TimeProvider). A pass that overruns
the period is followed by the next pass at once and the cadence restarts
from there; missed periods are not caught up with extra passes.
Shutdown, cancellation and abandoned subscription handling are
unchanged.
Parallelism by work: a pass checks the queues on the sweep thread until
it
has used a quarter of the publishing resolution (25 ms at 100 ms), then
checks the remaining queues with Parallel.For (CheckPublishTimers). The
session count no longer decides.
Why the budget:
- The pass cost is real per-item work. In the LoadTest subscribe shape
(50 sessions x 15 subscriptions x 600 items, all 600 written every
150 ms, publish 100 ms) every pass promotes up to 450k items; the
sequential pass takes 85-125 ms (~0.28 us per promoted item: ready list,
IsReadyToReport, LinkedList move, trigger check, ready signal release).
Ready-list promotion already skips items without pending values and
subscriptions whose timer is not due, so there is no idle work left to
skip; only parallelism makes the pass fit the period.
- Fixed cadence alone lifts delivery from 64 % to 91 % (pass ~85 ms in a
100 ms period, no headroom); with the parallel remainder 97-98 %.
- Parallel sweeps cost CPU when the pass fits anyway: at 10 sessions the
sequential pass takes ~15 ms; a 2 ms budget made it parallel and raised
server CPU per notification from 5.0 to 8.9 us. With the 25 ms budget
it stays sequential.
- Idle servers stay sequential: 1000 sessions x 1 slow subscription pass
in ~2 ms sequential (was always parallel above 256 sessions); 2
sessions in 0.05 ms.
Diagnostic probes (one build with temporary switches, one run each,
50x15x600 shape):
| loop / sweep | delivered | us/notif | pass avg | cycle |
|-------------------------|-----------|----------|----------|--------|
| old loop, sequential | 63.9 % | 5.1 | ~125 ms | ~225 ms|
| fixed, sequential | 91.2 % | 5.1 | ~85 ms | ~110 ms|
| old loop, all parallel | 96.9 % | 3.9 | ~24 ms | ~132 ms|
| fixed, all parallel | 97.0 % | 4.1 | ~21 ms | 100 ms |
| fixed, 25 ms budget | 97.8 % | 4.4 | ~44 ms | 100 ms |
Paired A/B against master (perf/ServerLoadHarness, Server GC, encrypted,
alternating rounds, medians; runs repeated when processes outside the
harness used more than 1.5 cores):
| shape (sessions x subs x items) | n | delivered | us/notif | server
cores |
|-----------------------------------|---|--------------|-------------|--------------|
| 50 x 15 x 600, pub 100, write 150 | 3 | 67.7 -> 97.8 | 5.5 -> 4.4 |
11.3 -> 13.3 |
| 10 x 15 x 600, pub 100, write 150 | 3 | 100.4 -> 99.9| 5.5 -> 5.1 |
3.32 -> 3.11 |
| 1000 x 1 x 1, pub 1000, write 500 | 7 | 50 -> 50 (*) | 216 -> 201 |
0.21 -> 0.20 |
| 2 x 1 x 100, pub 100, write 150 | 3 | 100 -> 100 | 48.5 -> 46.1| 0.06
-> 0.06 |
(*) two writes per publishing interval with queue size 1, by design.
Side effect at 50 sessions: server allocation per notification rises
from ~54 to ~96 B. An allocation trace attributes it to
MonitoredItemNotification: the parallel remainder makes all
subscriptions publish within a few ms, the in-flight burst exceeds the
notification pool bound (kMaxPooledNotifications = 65536) and the pool
misses; the slow sequential sweep spread gets and returns. Gen0 GCs in
the 15 s window 3-5 -> 8, pause 20-50 -> ~60 ms; CPU per notification
still drops. Addressed by the stacked #4830, which allocates these
notifications instead of pooling them (tracking issue #4819).
### Follow-up commits
- **Late wake / exact deadline** (review): a wait that ends a whole
period late restarts the cadence from the late pass (no catch-up pass),
and a sweep exactly at a subscription's deadline counts as due.
- **Phase aliasing with equal periods** (
|
||
|
|
014f99a53a |
WebSocket transport: a sender queued across Close fails as closed (#4832)
## Problem `WebSocketByteTransportTests.CloseFailsQueuedSendInsteadOfOrphaningItAsync` fails intermittently. One example is test-linux-net10.0 (3/16) in run 39114639155 on #4831, an unrelated MachineTool change. The queued send failed with a raw `WebSocketException` ("…transitioned into an invalid state") instead of `ServiceResultException` with `BadConnectionClosed`. ## Cause This is a product race in `WebSocketByteTransportBase`. - `Close()` cancels the shared send token. That wakes both the in-flight send and the queued `SemaphoreSlim.WaitAsync`. - `SemaphoreSlim` removes a cancelled waiter on a later continuation. If the in-flight sender's `finally` releases the lock first, the release *grants* the queued waiter despite its cancelled token. - `EnterSendAsync` then returned normally, and the queued sender called `SendAsync` on the already aborted socket. - That socket error is not an `OperationCanceledException`, so it reached the caller untranslated. ## Fix After acquiring the lock, `EnterSendAsync` checks whether the transport closed. If it did, it gives the lock back and throws `BadConnectionClosed`. `Close()` sets the closed flag before it cancels, so every waiter woken by `Close()` sees it. ## Tests - The existing test body moved into a helper. New `CloseFailsQueuedSendEvenWhenTheLockIsGrantedAsync` repeats the interleaving 2000 times (about 140 ms). - Without the fix it fails 3/3 runs within about 100 ms, with the CI exception. With the fix it passes 3/3. - `Opc.Ua.Core.Tests` `~WebSocket` (net10.0): 39/39 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
0005b0d5e6 |
WMTP: callers wait for the freshness tick instead of BadServerTooBusy (#4831)
## Problem `WmtpMcpIntegrationTests.ReportsCopyExactWholeSelectionsWithoutConsumingEitherStore` failed intermittently (test-linux-net10.0 13/16, run 38897795750, on #4815, which only changes `perf/`). The `wmtp_count_records` call came back with `BadServerTooBusy`. ## Cause This is a product race. The UA request queue (MaxQueuedRequestCount / MaxRequestThreadCount) is not involved, and the test makes no parallel calls. - `WmtpOperationCoordinator` admits one operation per peripheral and refuses a second one with `BadServerTooBusy`. - `WmtpPeripheral.StartMaintenance` starts a 1 s age timer (`AgeTick` → `RefreshQualityAsync`) that takes the same gate. - The Tools.Tests fixture clock does not override `CreateTimer`, so the timer is a real system timer. A client call that arrived while a tick was running was refused. On a loaded runner each tick takes longer, so collisions become more likely. - On a production server, any WMTP client call could randomly hit this, and the client cannot tell it apart from a real concurrent operation. - It did not start with #4812 or #4808. The code came in with #4765. ## Fix - The active operation now records whether it is maintenance. - Other operations wait up to one second for a running freshness tick instead of being refused. A regular tick makes no equipment calls and takes milliseconds. - A stalled tick, for example one blocked in a state-changed sink, still refuses the caller with `BadServerTooBusy` after that second. This keeps `ControlledQualityMaintenanceStillExcludesOverlappingReportsAsync` valid. - A tick is still refused while any operation is active, and it retries on the next tick, as before. - Two client operations still conflict with `BadServerTooBusy`. - The callback marker is now a chain of the peripherals a call passed through. Re-entering a ticking peripheral through another peripheral's callback (A → B → A) is refused instead of waiting for itself (review feedback). ## Tests - `WmtpRuntimeTests.CallerArrivingDuringFreshnessTickWaitsInsteadOfBusyAsync` holds a tick open through a `StateChangedAsync` sink. Without the fix it fails with the CI error. - `WmtpRuntimeTests.IndirectReentryIntoTickingPeripheralIsRefusedInsteadOfDeadlockingAsync` covers the A → B → A case. With the innermost-only check it times out. - Local net10.0 runs: - `Opc.Ua.MachineTool.Tests` `~Wmtp`: 124/124 passed. - `Opc.Ua.Tools.Tests` `~WmtpMcp`: 176/176 passed. - `Opc.Ua.Machinery.Tests` `~CompanionSuiteWmtp`: 2/2 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
fc0687a2dc |
Source generator: a NodeSet2 Variable without a Value has a null Value (#4798)
# Description A Variable or VariableType in a NodeSet2 that has no Value element has a null Value (OPC 10000-6 F.5); the runtime NodeSet2 importer loads it that way. The source generator initialized such a node with the default value of its DataType instead ((int)0, (double)0, ...), so a server built from the generated code reported values the model never declared. The NodeSet2 importer now records that the node declares no Value (NodeSetValueIsNull), the flag follows the node into the merged instance hierarchy, and the node state generator emits Variant.Null for it when no value was merged in from elsewhere. ModelDesign inputs, where a missing DefaultValue implies the DataType default, are unchanged. The tests generate a DI-style NodeSet2 together with the DI NodeSet2 and check the generated values of a type's InstanceDeclaration, of an instance's children with and without a Value, and of a VariableType. ## Related Issues None. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
4a7dfe7411 |
Server: release a restored event item that its node manager rejects (#4828)
# Description Follow-up to #4795. When a node manager returns a Bad result from `SubscribeToEventsAsync` for an event monitored item while durable subscriptions are restored, `NodeManagerServiceDispatcher.RestoreMonitoredItemsAsync` removes the item from the event manager and from the result list, but nothing disposes it, so its queues stay alive. The item is now disposed there. A queue that was staged for the restore (`IRestorableMonitoredItemQueue.BeginRestore`) keeps its stored contents when it is disposed, as for any aborted restore, so a later restore still finds them. The data-change restore path already disposes an item it fails to restore, and the `Objects.Server` path keeps every item, so neither needs a change. A test in `SubscriptionRestoreFailureTests` restores an event item that its node manager rejects and checks that the item is unregistered and its queue disposed; it fails without the change. ## Related Issues Follow-up to #4795. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (The subscription restore tests of Opc.Ua.Server.Tests on .NET 10, 35 passed; Opc.Ua.Server.Tests builds for net48.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
|
8441351902 |
Client: let ManagedSession validate a rediscovered endpoint (#4791)
# Description When a reconnect is rejected with a status that requires the endpoint to be refreshed, ManagedSession reads the endpoint again from the server's discovery endpoint and ConfiguredEndpoint.Update replaces the endpoint in use with whatever discovery returned. The application had no say: a discovery answer that, for example, drops the security mode or presents a different server certificate was taken over as it was. ManagedSession.ValidateRediscoveredEndpoint (a RediscoveredEndpointValidator) receives the endpoint in use and the rediscovered one before the session is recreated against it. A Bad result keeps the endpoint in use and fails the reconnect attempt with that result, so the reconnect policy decides what happens next. Without a validator the behaviour is unchanged. ## Related Issues None. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
55bd944609 |
WoT: Types-side TD/TM-NodeSet conversion with consolidated corpus and minimal consumer compat (#4758)
# Description Adds the pure Types-side WoT Thing Description / Thing Model to NodeSet conversion: resolution, context, declarations, types, values, localization, URI/format handling, bounded hashing, plus supporting native XML codec, NodeSet alias/value helpers, RelativePathFormatter and a JsonElement equality polyfill. - Carrier-witness fix: a missing native event is accepted only when the owning preserved type declares it through a `GeneratesEvent` reference (directly or via its type/supertype chain) AND a bounded, validated selected EventType carrier identity equals the normalized authored native target. Native-present events get a full compare; an ambiguous native key and a missing map stay strict; a readable document is never archive authority. - Support-only dependencies: additive per-instance, non-serialized `WotDocument.IsSupportOnly(kind, name)` (ArgumentNullException guard). The resolver marks support-only after closure; explicit selection wins; the View builder excludes only true support-only members, so unit/actsOn dependencies stay in the JSON. Nested controls cover inner value-only projection under an outer value-only selection (unit kept, support-only) versus an outer explicit SelectAll (explicit, not support-only). - Documentation/XML comments and member-order only in commit 5c368d3a; no logic change. - Timeout repair on current head |
||
|
|
768b39e1be |
Server: surface failures to restore durable subscriptions (#4795)
# Description SubscriptionManager.RestoreSubscriptionsAsync logged and swallowed every restore failure: a subscription store that threw, a store that reported an unsuccessful restore, and a stored subscription that could not be recreated. The server then started without those subscriptions, the store was told the restore had completed (so it cleaned up what it held), and nothing but a log line showed that subscriptions had been lost. A failed restore now fails the restore, and with it the startup: - a store that throws fails with BadInternalError and the store's exception as the inner exception; - a store that reports Success = false fails with BadInternalError; - stored subscriptions that cannot be recreated are collected and reported as an AggregateException, before the store is told that the restore completed, so the store keeps what it holds. SubscriptionManager.ContinueOnRestoreFailure (default false) restores the previous behaviour for applications that prefer to start without the subscriptions. Cancellation is no longer swallowed unless that option is set. The Quickstarts durable subscription store reported Success = false when it had nothing stored; it now reports a successful restore of no subscriptions, and keeps reporting a failure when its file cannot be read. ## Related Issues None. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
7fb986bd0d |
Add recycling operator console and sample container images (#4810)
# Description Add a local browser-based operator console for the synthetic recycling line, using direct, encrypted OPC UA connections and the existing typed coordination interface. - Extract a sample-local, non-packable `Recycling.Client` library shared by the server and console. Preserve the model XML bytes, namespace/type identifiers, resource identity, wire format and journal schema; the server retains its .NET 8/9/10 targets. - Add a .NET 10 ASP.NET Core workstation host with bounded SSE state/trend delivery, native companion inspection, paged material/genealogy reads, and role-limited coordinator commands. Exact command envelopes are retained for explicit retries; uncertain outcomes are not automatically reissued. - Add the operator interface using local HTML/SVG, CSS and JavaScript modules: ten process groups, substep inspection, faults/constraints, persistent quality holds, trends, material tracing and simulator administration. Show raw quality/provenance and handle stale data, run changes, browser expiry and large identifiers explicitly. No browser-generated plant telemetry or frontend framework/npm build is introduced. - Keep application/user credentials in external certificate stores. Use strict X.509 trust, signed/encrypted UA sessions, local browser bootstrap sessions, same-origin/CSRF checks, and a conservative configured control ceiling. This is a local synthetic sample, not a physical emergency-stop or shared remote-user system. - Register `recyclingserver` and `recyclingclient` in the existing Images CI matrix, using source/prebuilt Dockerfile stages and the existing portable-IL/multiarchitecture packaging flow. Add an optional loopback-published Compose console service. No new schedules or NuGet publication entries are added. - Add managed, real-UA, Chromium, NativeAOT and image-contract qualification, with Microsoft.Playwright as a test-only centrally managed dependency, and update usage/deployment documentation. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
|
44a8cb277d |
plans/ctt-issues.md: record C60 (CTT publish thread stall in the A&C Limit/Level CUs) (#4820)
The CTT regression run on master ( |
||
|
|
849840be5d |
perf: publish/write/event load scenarios for the server load harness (#4817)
Builds on #4815 (merged); rebased onto master. Two commits, touching only `perf/`. Extends `perf/ServerLoadHarness` and `perf/TraceAnalyzer` to profile publish-, write- and event-heavy server load that mirrors the explicit Sessions `LoadTest`. Built for the publish/write load investigation: [findings report](https://claude.ai/artifact/BWuEEPcvvbTAbjY44mJFsa). ## New options | Option | Scenario | Meaning | |---|---|---| | `--scenario events` | | Event subscriptions on the Server object; a server-side `EventGenerator` (in `EventWorkload.cs`) reports events, driven by the server child's new `events <rate>` / `events stop` stdin commands. | | `--event-rate` | `events` | Events/s the server reports; 0 = as fast as one thread can. | | `--writers N` | `sub` | Dedicated writer sessions in addition to `--sessions` (0: the first subscriber session writes, as before). | | `--write-interval 0` | `sub` | Closed-loop writers instead of a timer. | | `--write-batch` | `sub` | Values per Write; writers rotate through the nodes in slices. | | `--node-type all` | `sub` | The LoadTest's Int16…UInt64 mass node set (up to 600 nodes). | | `--queue-size` | `sub`, `events` | Monitored item queue size (default 1 / 10000). | | `--publish-requests` | all classic | Minimum outstanding Publish requests per session. | Also: - `Workload.Secondary` / `OnServerReport` hooks and a `per-op2:` result line: server cost per written value (`sub`) or per generated event (`events`). - TraceAnalyzer `threads` mode: busy share of each thread with its main roles (finds serialized stages). - TraceAnalyzer `cpu <trace> <top> <focus>` additionally lists the focus frame's direct callees. - README rows and paragraphs for all of the above. Conflicts with #4815's rewrite (native workers, `suspend`/`samples` modes) were resolved so #4815's code and text stay and these additions are layered on top. ## Verification Builds: `CustomTestTarget=net10.0 dotnet build perf/ServerLoadHarness -c Release` and `dotnet build perf/TraceAnalyzer -c Release`: 0 warnings, 0 errors. Smoke tests (`--duration 5 --warmup 3`, `DOTNET_gcServer=1 SERVER_DOTNET_gcServer=1`), each run only once foreign CPU was below 1.5 cores, with foreign CPU measured during the run (avg 0.5–0.6 cores): | Run | Result | |---|---| | `events --sessions 5 --subs 3 --event-rate 2000` | generated/s=2009, events/s=30413, received/expected=100.9 %; `per-op2` 588.9 µs CPU / 31.5 KB per generated event | | `sub --sessions 5 --subs 3 --nodes 600 --node-type all --writers 4 --write-interval 0 --write-batch 100` | 131k values/s written, 77k notif/s, writeErrors=0; received/expected=3.9 % (queue size 1 with closed-loop writers: values are overwritten before publishing, as intended); `per-op2` 50.5 µs / 1680 B per written value | | `sub --sessions 2 --subs 1 --nodes 100 --queue-size 2 --publish-requests 4` | received/expected=98.0 %, writeErrors=0 | | same as the second run with `--trace` | 23 MB trace; `TraceAnalyzer threads` lists 58 threads with busy shares and roles; `cpu … 10 MonitoredItem.Publish` prints callers, first Opc.Ua frame and the new "direct callees of" section | #4815's own scenarios still parse: `counters` passes; `native-read` (with `--server-affinity`, `--thread-states`) and `native-counters` start the contract server and fail only at launching the (absent) native worker executable, as expected without the workers. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
7e25e1a3d7 |
Extract remaining shared companion, rendering and packaging changes (#4806)
# Description The shared AI deployment contracts and instance registration; DI software-upload outcomes, digest handling and simulator; bounded Positioning/Pumps reads; PubSub raw-data/matrix and security-provider handling; Vision read-quality contracts; xRegistry resource writes and instance projection; OpenUSD authoring, embedded viewport and retained-frame capture APIs; standalone sample CLI controls; and the generic Debug/RID package-policy and fork/Dependabot coverage-comment fixes. The Linux native prerequisite step is retained for **Vision tests only**. ## Related Issues - Related tracking issue: [#4461](https://github.com/OPCFoundation/UA-.NETStandard/issues/4461), without automatic closure. - Source work: [#4462](https://github.com/OPCFoundation/UA-.NETStandard/pull/4462), which remains intact for the coordinator's later integration. - Immediate stacked predecessor: [#4804](https://github.com/OPCFoundation/UA-.NETStandard/pull/4804). ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
4e77ba3f84 |
Redundancy: snapshot and restore for RaftSharedKeyValueStore (#4794)
# Description RaftSharedKeyValueStore applies every committed command but offers no way to capture its state. An application that wants to bound the Raft log cannot snapshot the store before compacting its consensus log, and after a restart it cannot restore the state without replaying the whole log. Add CreateSnapshotAsync, which commits a barrier through the consensus and encodes the state in a versioned format, and RestoreSnapshot, which replaces the state before the store starts applying the log. A malformed snapshot raises FormatException; restoring after start raises InvalidOperationException. Tests: RaftSharedKeyValueStoreSnapshotTests (round trip, log applied after restore, restore after start rejected, corrupt snapshot rejected). ## Related Issues None. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
2de64d7d93 |
Source generator: keep NodeSet2 references to nodes of referenced models (#4796)
# Description A reference in a NodeSet2 input whose target is a node of another model was dropped without a diagnostic when that model is supplied by a referenced assembly rather than by an AdditionalFiles input. The dependency metadata of a referenced assembly carries types only, so the importer could not resolve an instance such as DI's DeviceSet and skipped every reference to it: a device organized by DeviceSet (inverse Organizes) lost that reference, and so did any non-hierarchical reference to such a node. When the target of a reference is in another, non-zero namespace and no loaded design defines it, the importer now creates a stand-in that carries only the NodeId the generated AddReference call needs. The stand-in is not generated. References whose reference type is unknown are still skipped. Tests generate a DI-style NodeSet2 whose device is organized by DeviceSet against DI supplied as a referenced assembly and as an AdditionalFile, and compile the result against the stack. ## Related Issues None. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
b1c0cd20fd |
Fix classic browse-path subscription restoration (#4814)
# Description Persisted classic subscriptions retain their server/client handles but previously lost relative-path resolution metadata. Transfer could succeed, yet a later `ApplyChangesAsync` attempted to resolve an already-created item and failed. The SDK now owns this restoration lifecycle without exposing a public resolution-state setter. - Preserve the originally monitored target in internal snapshot state and restore both `ResolvedNodeId` and `Status.NodeId`. Transfer/reactivation use handles without re-browsing; fresh creation resolves paths again. - Version classic subscription/configuration files and read existing unversioned 2.0 files using frozen legacy wire layouts. Legacy missing targets remain unknown rather than being guessed. - Exclude existing items from creation-time resolution, reserve creation before asynchronous translation, and reject stale translation results. Failed deletion retains identity and pending retry state. - Support forward-only snapshot input using position tracking and bounded, per-ExtensionObject buffering in the binary decoder, including short reads. No whole-file buffering or reflection-based serialization is introduced. - Add unit/integration regressions and document the supported lifecycle and migration implications. **Compatibility:** Newly written files require the updated SDK. Existing 2.0 files remain readable. Assigning a different `StartNodeId` or `RelativePath` to a created/in-flight item now immediately throws `BadInvalidState`; unchanged assignments remain allowed. No V2 subscription-engine changes or 1.5 XML import are included. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
8e96408b2a |
Fuzz oracles: confirm net48 allocation findings by re-running the target (#4818)
## Problem `FuzzCrashAssets(AflfuzzBinaryDecoder)` failed intermittently on net48 (run 38897795750, test-windows-net48 1/16, on #4815, which changes only `perf/`): > 10562496 bytes allocated for 42 input bytes, budget 8391296 bytes On .NET Framework, `FuzzOracles` measures allocations with `AppDomain.MonitoringTotalAllocatedMemorySize`. That counter covers the whole AppDomain, so it also counts other threads in the test host. Measured on net48 Release for `crash-0bdcbfafc13981ae6f665ad52cabc1cbe21b0238`: | Scenario | Bytes | |---|---| | First decode in the process (one-time XML setup) | 1,083,232 | | Later decodes, same thread or a fresh thread | 16,440 | | Same decode with one concurrent thread allocating | 10.7 to 23.8 MB (over budget) | The decoder does not have a per-thread cost, and #4812 did not regress it. The CI TRX shows that tests ran one at a time. The failure came about 2 s into the run, right after a burst of about 100 fast test cases, which fits vstest serializing results on a background thread. ## Fix `FuzzOracles.RunTarget` re-runs a target when the allocation finding comes from a counter that is not per thread (`AllocationConfirmationRuns = 2`). It reports the finding only if every run confirms it. A real over-budget decode allocates the same amount every time, while a burst on another thread rarely lines up with all three runs. This is the same approach the time oracle already uses (`TimeConfirmationRuns`). Runtimes with a per-thread counter (net8+) behave as before. `FuzzMethods` playback goes through `RunTarget` too, so it also gets the confirmation runs. ## Tests - New tests in `FuzzOraclesTests`: - A finding that every run confirms is still reported, after 3 runs on net48 and 1 run on net8+. - A finding inflated by another thread is ignored (net48 only). This test fails when the confirmation runs are set to 0. - net48 Release: Opc.Ua.Fuzzing.Tests 156/156, Opc.Ua.Encoders.Fuzz.Tests 8130/8130. - net10.0: FuzzOraclesTests pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
8b56937fdf |
perf: drive the server with external native benchmark workers; TraceAnalyzer GC modes (#4815)
## What this adds Harness-only changes under `perf/` (not part of `UA.slnx`, `net10.0`). No product code changes. ### External native worker scenarios (`ServerLoadHarness`) Two new scenarios run the native open62541 workers of the [o6-automation/opcua-benchmarks](https://github.com/o6-automation/opcua-benchmarks) project against a server, so the .NET server and other OPC UA stacks are measured with the same client as that project's published comparison: | Scenario | What it measures | |---|---| | `native-read` | Fixed-load scalar Read throughput (`throughput/open62541/client.c --worker async`): `--clients` worker processes, each with `--inflight` single-value Reads of ns=1;i=1001..1100 in flight, starting together. Reports aggregate req/s, and server CPU and allocated bytes per request. | | `native-counters` | Subscription capacity (`subscription/native/client.c`): one subscription with `--nodes` UInt64 counters at `--sampling` 10 ms / `--pub` 1000 ms, queue size 1. Passes when every item gets 99 % of the samples its sampling interval calls for. | The result block has the same lines as the other scenarios, and the CPU time of the native workers goes in `client-per-op:`. The server is either the harness child in **contract mode** (`server --contract 1`) or any external executable (`--server-exe`, `--server-args` with `{port}`, `{items}`, `{sampling}` and `{pki}` placeholders), for example the benchmark's open62541 server or one of its .NET workers. By default the harness child is configured like the benchmark's .NET worker for UA-.NETStandard 2.0.1: | Setting | Default | Override | |---|---|---| | Node managers | `AsyncCustomNodeManager` | `--async-node-manager 0` (`CustomNodeManager2`) | | Counter increment | under a lock owned by the counters | `--counter-lock 0` (interlocked) | | Garbage collector | Server GC | `SERVER_DOTNET_gcServer=0` | | `MaxQueuedRequestCount` | clients × in flight + 3 (minimum 200) | `--queue` | Other options: `--native-client`, `--clients`, `--inflight`, `--iterations`, `--warmup-calls`, `--server-affinity` (processor mask; combine it with `SERVER_DOTNET_PROCESSOR_COUNT`, because the mask is applied after the runtime has sized itself), `--client-affinity`, `--server-priority`, `--thread-states 1` (samples Running/Ready server threads and appends a `sched:` line), `--security encrypt` (Basic256Sha256 SignAndEncrypt with generated DER identities) and `--verbose`. Also, the `MinRequestThreadCount` of the server child is now the smaller of 50 and `--threads`, so `--threads` below 50 takes effect. **Worker protocol.** The workers define a few names that are kept as string literals in named constants: the environment variables `O6_BENCHMARK_SECURITY_POLICY`, `O6_BENCHMARK_CERTIFICATE` and `O6_BENCHMARK_PRIVATE_KEY`, the `O6_BENCHMARK_READY` stdout token, the identity application URI / subject prefixes, and the server URI `urn:o6:benchmark:server`. The workers need that server URI because the contract puts its nodes in the namespace of the server URI and the workers address them as ns=1. Identifiers, file names, scenario and option names are neutral. ### TraceAnalyzer - `cpu` with a focus frame also reports the first `Opc.Ua` frame above it, past framework wrappers such as `Monitor.Enter` or `SemaphoreSlim`. - **GC-poll attribution:** the thread-time sampler suspends running threads at their next GC poll, so many on-CPU samples have `Thread.PollGCWorker` as their leaf. `cpu` used to drop them as waits (in one trace only 3.5k of 20k on-CPU samples were kept); it now attributes them to the caller. - `gc` mode (for `--trace-profile gc`): collections by generation, type and reason; the suspension time and the whole pause, each with count, total and percentiles. - `samples` mode: all samples by type (managed/external) and by leaf and first `Opc.Ua` frame, including the ones `cpu` leaves out. - `suspend <trace> [thresholdMs]` mode: every SuspendEE longer than the threshold, with the events recorded inside it by thread (an `AllocationTick` there means a thread that had not reached a safe point yet); event counts including the heap count DATAS chose and GC handle churn by kind. ## Building the workers The benchmark repository is AGPL-3.0, so nothing of it is vendored here. Clone it and build the workers separately. On Windows they build with `zig cc` and CMake against the open62541 commit the benchmark pins (OpenSSL backend, `UA_ENABLE_DEBUG_SANITIZER=OFF`, Release). The subscription workers also need `poll()` on stdin replaced by `PeekNamedPipe`, plus unbuffered control output. `perf/README.md` describes all of this. ## Commits 1. `perf: drive the server with external native benchmark workers` 2. `perf: configure the contract server like the benchmark's 2.0.1 .NET worker` 3. `perf: TraceAnalyzer gc and samples modes, samples taken at GC polls` 4. `perf: scheduling and GC-suspension diagnostics for the native worker runs` Each commit builds with 0 warnings (`CustomTestTarget=net10.0 dotnet build perf/ServerLoadHarness -c Release`, `dotnet build perf/TraceAnalyzer -c Release`), and `dotnet format whitespace` makes no changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c3701e5804 |
CI CTT: call the private repository through Actions REST endpoints only (#4816)
The first run after configuring the GitHub App ([run](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/38895888727)) minted its token correctly but failed on the dispatch: > unable to determine default branch for OPCFoundation/UA-.NETStandard-CTT: GraphQL: Resource not accessible by integration (repository.defaultBranchRef) `gh workflow run` (and other `gh` commands) make GraphQL calls that need Contents access, but the App only has Actions read/write. Dispatch, run lookup, polling and the summary download now use `curl` on the Actions REST API, so the App needs no more than Actions read/write. ## Test plan - [x] `bash -n` on the step script - [x] Run-list filter, run status and `ctt-public-summary` artifact download checked with `curl` against the private repository (earlier run `repeat-1`) - [ ] After merge: Actions → CTT regression → Run workflow on master 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
716858aa7d |
Source generator: option to emit exactly the nodes of a NodeSet2 (#4797)
# Description For a NodeSet2 input the generator built the predefined nodes from the merged instance hierarchy. A subtype therefore got a copy of every Mandatory InstanceDeclaration of its supertypes as a new child node, with a NodeId the NodeSet2 never assigned (the supertype's numeric identifier in the subtype's namespace, or a string identifier derived from the path when the supertype comes from a referenced assembly). A DeviceType subtype got eight such children from DI's DeviceType, so the address space a server loads from the generated code differs from the NodeSet2 it was generated from. ModelSourceGeneratorEmitNodeSetNodesOnly (GeneratorOptions .EmitNodeSetNodesOnly), off by default, makes the predefined nodes of a NodeSet2 input exactly the nodes the NodeSet2 declares: a child the generator derives from a supertype or type definition is not added to a predefined node. The typed state classes are unchanged and still reach inherited members through the state classes of their supertypes, and the CreateInstanceOf* factories still create every Mandatory child of the type. ModelDesign inputs are not affected. IModelDesign.IsSourceNodeSet tells the generator whether the target model was read from a NodeSet2. A test in Opc.Ua.Di.Tests generates a DI-style NodeSet2 with the option on, exports the generated predefined nodes as a NodeSet2 and compares it with the input, node for node and reference for reference. ## Related Issues None. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
87012a147f |
Machinery: add bounded inspection and result transfers (#4804)
# Description - Add opt-in bounded browsing and strict reads with original source, status, timestamp, and structured-value evidence. Preserve the existing tolerant constructors and convenience projections, and wire options through session extensions, factories, and hosting registration. - Add selected-result streaming downloads with byte/request limits and independent cleanup, using `TemporaryFileTransferClient.CopyToAsync` from the client/server predecessor. Retain the detailed transfer regressions in the Machinery test layer. - Expose genuinely null unpublished result slots, carry the two Machinery sample updates, and include the complete Machinery guide and package README changes. ## Related Issues - Related tracking issue: #4461; this extraction does not close it. - Frozen source PR: #4462; this extraction does not close or modify it. - Immediate predecessor: #4803 (`marcschier-extract-amb-changes`). The earlier dependencies are client/server #4802 and core/types #4801. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
8772f37275 |
Fix recycling scenario defaults and MCP interoperability (#4808)
# Description Fix five failures reproduced while exercising the recycling sample through a locally built OPC UA MCP server: - Support Certificate authentication through named, host-approved X.509 identity selectors and the existing certificate/identity providers. Tool arguments cannot supply private keys or arbitrary certificate-store paths. - Register custom enumeration dependencies from the validated, bounded type snapshot before loading requested structure roots. - Preserve structured fields and opaque encoded bodies in generic read/call results using the selected session's UA codecs. - Add bounded, explicitly typed UA JSON method arguments while retaining the existing string argument behavior and C# overloads. - Preserve omitted defaults in sparse recycling scenario JSON; explicit invalid values and unknown members still fail validation. All six shipped scenario files are covered by regressions. Update the MCP and recycling documentation. This PR contains one fix commit on a feature branch in **OPCFoundation/UA-.NETStandard**, based directly on upstream `master` at `95850b7e5`. No local registration, certificates, scenario journals or live-test artifacts are included. ### Validation **Final commit `d199feba0`: 216 affected MCP tests and 16 recycling CLI/scenario tests passed on .NET 10**, with no failures, fixture setup/teardown errors or build/analyzer warnings. The repository's TRX verdict helper accepted both reports. **Draft: the full-solution gate is not green.** The .NET 10 solution run encountered an `Opc.Ua.OneFuzz.Validator.Tests` fixture-publication failure outside the files changed by this PR: `MSB3030`, missing `Opc.Ua.OneFuzz.TestDependency.runtimeconfig.json`. It affected 210 tests; the run was stopped rather than expanding this PR into unrelated OneFuzz work. The full .NET Framework 4.8 solution suite was not run. Before the final rebase, the complete managed recycling suite passed **117**, and the CLI/scenario tests passed **16 each on .NET 8 and .NET 9**, with no failures or fixture teardown errors. The five explicit NativeAOT smoke cases were not selected. Actual live qualification used one coordinator and all ten independent process servers with strict certificate trust. Healthy, combined-fault and zero-utility runs each produced 24 complete packages with exact component conservation. Also exercised native companion discovery, subscriptions, authorization, pause/step/enable/disable, durable request replay, quality holds, reset and coordinator crash recovery. NativeAOT executables, container deployment and parity against a complete external producer contract were not qualified. <details> <summary>Exact validation commands and results</summary> Commands ran from the repository root with an isolated .NET SDK 10.0.401. `$root` denotes the private validation-artifact directory; no machine-specific path or key material is included here. Affected MCP matrix, final commit — **passed: 216; failed: 0; skipped: 0**: ```powershell dotnet test tests\Opc.Ua.Tools.Tests\Opc.Ua.Tools.Tests.csproj -c Release -f net10.0 -p:CustomTestTarget=net10.0 -p:CustomTargetFrameworks=net10.0 -p:UseSharedCompilation=false -m:1 -nr:false --no-restore --nologo -v:quiet --filter 'FullyQualifiedName~SchemaRemoteTests|FullyQualifiedName~OpcUaJsonHelperTests|FullyQualifiedName~MethodServiceToolsTests|FullyQualifiedName~ConvenienceToolsTests|FullyQualifiedName~AttributeServiceToolsTests|FullyQualifiedName~ViewServiceToolsTests|FullyQualifiedName~OpcUaSessionManagerTests|FullyQualifiedName~ConnectionToolsTests|FullyQualifiedName~McpHostBuilderTests|FullyQualifiedName~ConfigurationToolsTests' --logger 'trx;LogFileName=mcp-pr.trx' --results-directory "$root\pr-validation\focused" ``` .NET 10 CLI/scenario regressions, final commit — **passed: 16; failed: 0; skipped: 0**: ```powershell dotnet test tests\Opc.Ua.Recycling.Samples.Tests\Opc.Ua.Recycling.Samples.Tests.csproj -c Release -f net10.0 -p:CustomTestTarget=net10.0 -p:CustomTargetFrameworks=net10.0 -p:UseSharedCompilation=false -m:1 -nr:false --no-restore --nologo -v:quiet --filter 'FullyQualifiedName~RecyclingCommandLineTests' --logger 'trx;LogFileName=recycling-cli-pr.trx' --results-directory "$root\pr-validation\focused" ``` Complete managed recycling suite, before rebase — **passed: 117; failed: 0; skipped: 0** (five explicit NativeAOT cases not selected): ```powershell dotnet test tests\Opc.Ua.Recycling.Samples.Tests\Opc.Ua.Recycling.Samples.Tests.csproj -c Release -f net10.0 -p:CustomTestTarget=net10.0 -p:CustomTargetFrameworks=net10.0 -p:UseSharedCompilation=false -m:1 -nr:false --no-restore --nologo -v:quiet --logger 'trx;LogFileName=recycling-final.trx' --results-directory (Join-Path $root 'results') ``` .NET 8 CLI/scenario regressions, before rebase — **passed: 16; failed: 0; skipped: 0**: ```powershell dotnet test tests\Opc.Ua.Recycling.Samples.Tests\Opc.Ua.Recycling.Samples.Tests.csproj -c Release -f net8.0 -p:CustomTestTarget=net8.0 -p:CustomTargetFrameworks=net8.0 -p:UseSharedCompilation=false -m:1 -nr:false --nologo -v:quiet --filter 'FullyQualifiedName~RecyclingCommandLineTests' --logger 'trx;LogFileName=recycling-cli-net8.trx' --results-directory "$root\results" ``` .NET 9 CLI/scenario regressions, before rebase — **passed: 16; failed: 0; skipped: 0**: ```powershell dotnet test tests\Opc.Ua.Recycling.Samples.Tests\Opc.Ua.Recycling.Samples.Tests.csproj -c Release -f net9.0 -p:CustomTestTarget=net9.0 -p:CustomTargetFrameworks=net9.0 -p:UseSharedCompilation=false -m:1 -nr:false --nologo -v:quiet --filter 'FullyQualifiedName~RecyclingCommandLineTests' --logger 'trx;LogFileName=recycling-cli-net9.trx' --results-directory "$root\results" ``` Full .NET 10 solution — **not passed**: stopped after the OneFuzz fixture failure above; incomplete solution run, not a clean validation result: ```powershell dotnet test UA.slnx -c Release -p:CustomTestTarget=net10.0 -p:CustomTargetFrameworks=net10.0 -p:UseSharedCompilation=false -m:1 -nr:false --nologo -v:quiet --logger trx --results-directory "$root\pr-validation\net10" ``` </details> ## Related Issues Follow-up to [#4772](https://github.com/OPCFoundation/UA-.NETStandard/pull/4772), which introduced the recycling sample. No separate tracking issue is currently linked. These fixes preserve the existing architecture and address failures reproduced during live qualification. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Regression tests were added and passed as recorded above; aggregate coverage was not collected, so the combined coverage checklist item remains unchecked. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com> |
||
|
|
68f062eecc |
Server request path: one response header, single-node Read without collections, codec streams reused per thread (S1-S6, -38 % allocation per Read) (#4812)
## Description
Six small changes to the per-request path from the o6 benchmark gap
report (S1–S6), chosen from the allocation profile of the o6 read
benchmark (`AsyncCustomNodeManager`, Server GC, 10 clients × 100
single-value Reads in flight). Each change is its own commit. The
receive loop, its threading and its dispatch are not touched; there are
no wire, public API or generator changes.
### Results
Paired A/B runs, builds alternated per round, 5 rounds unless noted,
medians (min..max). Baseline: master
|