GithubStar/UA-.NETStandard
1
0
Fork 0
mirror of https://github.com/OPCFoundation/UA-.NETStandard.git synced 2026-10-21 00:56:28 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,739 Commits 110 Branches 105 Tags
fix/encoder-stack-depth-json-flake
Commit Graph

8 Commits

Author SHA1 Message Date
romanett 9b034f8291 Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700) 
# Description

Nullable reference types are now enabled once, in `common.props`, and
every project in the repository builds warning-clean with them.

**Configuration**
- `<Nullable>enable</Nullable>` lives in `common.props`. All per-project
`<Nullable>` settings are removed: 178 project files (enable,
annotations, and the `disable` in the three Aot test projects).
- 419 per-file `#nullable enable`/`restore` directives and the 4
`#nullable disable` directives are removed.
- Kept on purpose:
- The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type
Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`,
`PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the
project setting in generated code, so the directive is required there.
- `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against
the 1.5.378 packages and does not import the repository props.
- `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now
describe the central setting.

**Migration** (the 43 unset and 13 annotations-only projects, roughly
10k diagnostics)
- **Production code: annotated at the declarations.** Covers
`tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the
LDS sample.
- Members, parameters and returns are nullable where null actually
flows.
- Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`,
`IsMethodTypeDesign`).
- `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take
`params object?[]`. `Template.AddReplacement` takes `string?` (rendering
already skips null replacements).
- `IRegisteredServerStore.RegisterAsync` returns
`Task<RegistrationEntry?>`: it returns null when a registration is
removed.
- Dereferences backed by an invariant (for example, the validated model)
use `!`, so runtime behaviour is unchanged.
- **Behavioural changes in production code** (small, deliberate):
- `NodeSetToModelDesign` throws `InvalidOperationException` when the
file is not a NodeSet. Before, it threw a NullReferenceException on the
next line.
- `ModelDesignValidator` starts its lookup tables empty instead of null.
A redundant `if (m_nodes == null) return;` guard is gone.
- **Test code: minimal annotations.**
  - `?` on values that can be null, `!` where the test asserts non-null.
  - `= null!` for fixture members assigned in SetUp.
- `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the
`IEncodeable` constraint.
- A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`)
folded into the adjacent `IsNull`/`Count` checks.

**Notes for reviewers**
- The diff is large (~1,500 files) but almost entirely mechanical: `?`,
`!` and directive removal. The non-annotation changes are listed above.
- Known follow-up, not done here:
`IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay
non-nullable even though the validator returns null before it loads a
dictionary. Making them nullable would ripple through every generator
call site.

## Related Issues

- Fixes #4439

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

The "no new warnings" box covers net10.0 and net48: full non-incremental
builds of `UA.slnx`, `tools/SourceGeneration.slnx`,
`tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0
warnings and 0 errors under the repository's warnings-as-errors
settings. net9.0 and net8.0 also had no nullable diagnostics, but they
were not rebuilt after the final fixes. Tests have not been run locally
yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 08:48:13 +02:00
romanett 7212607f16 OPC UA Part 3 compliance: permissions, Write semantics, NodeManagement, NodeSet DataTypes, locales (#4675) 
# Description

Fixes the High and Medium findings of an OPC UA Part 3 (OPC 10000-3,
Address Space Model) compliance audit. Each finding was checked against
the full spec text (Parts 3, 4, 5, 6, 14 and 18) before it was fixed.
Every fix comes with regression tests that fail without it.

### Role permissions (Part 3 §8.55, Part 4 §5.12.2.2)
- The type-hierarchy exemption in
`MasterNodeManager.ValidateRolePermissions` /
`ValidateAccessRestrictions` now covers only Browse, Read and
ReadRolePermissions. Type nodes stay visible to generic clients, which
was the intent of #3705; every other permission bit is now enforced on
them.
- `ReceiveEvents` is enforced on the EventType node. The standard
NodeSet grants it on `AuditEventType` and its subtypes only to
SecurityAdmin, so only SecurityAdmin sessions now receive audit events.
- Call checks the `Call` permission and AccessRestrictions on the Object
as well as on the Method.
- When a Call's `methodId` is the Method declaration of the ObjectType,
the Object's Method with the same BrowseName is permission-checked and
invoked. The declaration is used only if the Object has no such Method.
This applies to `AsyncCustomNodeManager`, `CustomNodeManager2` and the
sample node manager.

### Write service (Part 3 §5.6.2, §8.40, §8.57; Part 4 §5.11.4)
- The synchronous `OnWriteValue` path (fluent `OnWrite`) checks DataType
and ValueRank first (`Bad_TypeMismatch`). After an IndexRange write it
merges the slice into the cached value instead of replacing it.
- StatusWrite / TimestampWrite are enforced (`Bad_WriteNotSupported`).
- Undefined enumeration values and invalid OptionSet writes return
`Bad_OutOfRange`. OptionSet writes merge only the valid bits into the
stored value.
- New `ReportPropertyValueChanged` / `ReportSemanticChange` let server
code raise SemanticChangeEvents for changes it makes itself.
- Required follow-up (Part 14 §6.2.11.1): if an external target rejects
a timestamp, the PubSub subscriber's TargetVariables writer retries
without it.

### NodeManagement and Call arguments (Part 3 §5.5.2, §5.7.3, §6.4.2;
Part 4 §5.8, §5.12.2)
- AddNodes creates the Mandatory InstanceDeclarations of the
TypeDefinition and rejects abstract or Interface types
(`Bad_TypeDefinitionInvalid`).
- AddReferences, DeleteReferences and DeleteNodes bump NodeVersion and
raise GeneralModelChangeEvents.
- Clients may omit trailing optional input arguments
(`HasOptionalInputArgumentDescription`).

### NodeSet DataTypes (Part 3 §5.8.3, §8.48; Part 6 Annex F.12)
- `UANodeSet.Import` completes structure definitions with the base
type's fields (base fields first). It creates definitions for structures
without fields and sets `DefaultEncodingId`.
`StructureDefinition.FirstExplicitFieldIndex` marks the subtype's own
fields.
- `DataTypeState` no longer mutates the stored definition on read.

### Locales (Part 3 §5.2.5, §8.5; Part 4 §5.4)
- Read, Browse and data change notifications return the translation that
best matches the session's LocaleIds, or all translations for `mul`. The
selection lives in the server dispatcher and is built on
`ResourceManager`: the new `TranslateValue` calls the overridable
`Translate`.
- Locale ids are matched case-insensitively, and fallback translations
carry their real locale.

### Client complex types (Part 6 §5.1.7)
- The Reflection.Emit builder encodes only `IsOptional` subtyped fields
as ExtensionObject. It also supports OptionSet subtypes: an unsupported
type is skipped instead of aborting the whole load.
- `TypeInfo.GetXmlName` uses the data type's XML name for dynamic
structures and OptionSets (the XML body element was `<Structure>`). Both
builders now produce identical XML.

## Behaviour changes
All are documented in `docs/MigrationGuide.md`.
- **Audit events:** only SecurityAdmin receives them. The CTT "Auditing
Connections" unit needs a SecurityAdmin user (e.g. sysadmin);
`AuditingOperationTests` was updated.
- **Write timestamps and status:** clients writing a SourceTimestamp or
a non-Good StatusCode get `Bad_WriteNotSupported` unless the Variable
has StatusWrite/TimestampWrite. No variable in `src/` sets these bits.
- **AddNodes:** it rejects abstract types (including `BaseVariableType`
and `BaseEventType`), and new instances now come with child nodes.
- **Call:** the Object needs the Call permission. The Object's own
Method runs even when the call names the type declaration.

## Not included
- Low-severity findings: missing validation on optional services and
author-side NodeSet rules.
- LocalizedText nested inside structures or Variant arrays, and in
HistoryRead or Call outputs.

## Testing
net10.0, Debug, one suite at a time on the merged branch. All suites
below pass:

| Suite | Passed |
|---|---|
| Types | 11430 |
| Core.Encoders | 2947 |
| Core.Schema | 179 |
| Core.Security | 558 |
| Client | 2761 |
| Server | 8210 |
| InformationModel | 1080 |
| Features | 145 |
| History | 536 |
| Di | 455 |
| Machinery | 167 |
| Robotics | 661 |
| Robotics.Intent | 22 |
| Pumps | 75 |
| Scales | 122 |
| Positioning | 69 |
| ISA95 | 144 |
| Vision | 484 |
| PubSub | 1490 |
| PubSub.Adapter | 151 |
| PubSub.Server | 177 |
| Redundancy.Server | 856 |
| Lds | 173 |
| Configuration | 241 |
| WotCon | 1496 |

- **Client.ComplexTypes:** one test wrote a SourceTimestamp to a
variable without TimestampWrite. It is fixed in this PR, and the suite
has not been re-run since.
- **Core:** 6068 passed, 1 failure in
`ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly`. That telemetry
test passed when re-run alone, so it looks flaky.
- **Still running:** Sessions, Subscriptions (Classic, Durable,
standard) and Interop. Results will follow on this PR.
- `UA.slnx` builds with 0 errors for net10.0 and net48, with no new
analyzer warnings in changed files.

## Related Issues

- Part 3 compliance audit (no tracking issue)

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-05 12:43:59 +00:00
Cristi Pogacean 996de3ba1b Preserve partial runtime type loads and complete imported metadata (#4547) 
## Summary
Keep successfully loaded runtime definitions available when some types
cannot be resolved, and complete derivable metadata for imported server
structures.

- In `ComplexTypeSystem.LoadAsync`, report incomplete loading with
`false` while still committing successfully staged definitions when
dictionary loading is disabled. `LoadNamespaceAsync` is unchanged.
- In the server address-space resolver, clone an incomplete
`StructureDefinition` and fill missing BaseDataType / DefaultEncodingId
from the type tree and Default Binary reference. Keep the imported
definition untouched; explicit metadata is not overwritten.
- Add regressions for partial enum registration and imported
structure/encoding registration, including source-definition
immutability.

Fixes #4532
Fixes #4533

## Validation
Windows Release, net10.0, base `b4136d8e9`:
- Both new regressions failed against unchanged production code: the
valid enum and imported structure were absent from their factories (1
failure in each selected test).
- With the fixes, `ComplexTypeSystemCoverageTests`: 19 passed;
`ServerComplexTypeSystemTests`: 16 passed. Both processes exited 0 with
no failures/skips.
- The server fixture was rebuilt and rerun after adding binary-encoding
and source-immutability assertions: 16 passed again.
- Changed-file whitespace verification and `git diff --check` passed.

Validation is change-scoped locally; the complete framework/platform
matrix is left to PR CI. Local restore overrides are command-line-only,
with no feed or repository-guidance changes.
 2026-09-25 12:29:11 +02:00
Marc Schier be61d598a9 Fix startup NodeManager lifecycle and imported method arguments (#4427) 
# Description

Make startup-composed application NodeManagers participate in the live
lifecycle and bind imported NodeSet2 Method argument properties to their
typed `MethodState` children.

This change:

- adopts application NodeManagers created before startup into
`INodeManagerLifecycle.Registrations` after server startup succeeds,
while keeping diagnostics, configuration, and core managers protected;
- retains per-manager startup external-reference ownership and exact
routing positions so reload, removal, rollback, overlapping namespaces,
synchronous adapters, and server restart preserve existing behavior;
counted immutable reference snapshots avoid quadratic startup matching;
- rejects duplicate factory output across active, unpublished, and
retired lifecycle generations without deleting or disposing the live
manager; callers can catch the public
`NodeManagerAlreadyRegisteredException`;
- binds valid namespace-zero `Argument[]` `PropertyType`
`InputArguments` and `OutputArguments` Variables to
`MethodState.InputArguments` and `MethodState.OutputArguments` from
either `HasProperty` direction, without requiring `ParentNodeId`,
including local namespace-URI reference targets;
- preserves authored metadata during NodeSet typed-child promotion
without changing the initialization behavior of unrelated generated
nodes;
- retains generic-import guards for custom and malformed signatures,
duplicate standard-signature validation, and upstream factory-driven
generated-slot replacement and reconciliation callbacks;
- updates lifecycle and Runtime NodeSet documentation and clarifies
`OnServerStartedAsync` timing for both `StartAsync` overloads, including
the unopened default host returned to the caller.

This is a focused follow-up to #4418. That PR preserves registrations
from multiple fluent builders, but does not change lifecycle ownership
or NodeSet child typing. The branch includes upstream `master` through
`da7412c42`, including #4396, #4432, #4437, #4438, and #4440.

The completed review fixes in `686d0561f` are included. The latest base
merge (`a29fc6ca8`) combines them with #4432's shared batch/fluent
importer and generated state factories. One iterative linker supports
available parent nodes, deferred parent side tables, application-owned
handles, and placeholder reconciliation while retaining reference-driven
Method argument binding and URI handling. Upstream cancellation, GDS
assembly renames, certificate leak scoping, and xRegistry changes are
preserved.

## Related Issues

- Fixes #4421
- Fixes #4422

## CI regression fix

`e1a4436f0` fixes the GDS failure reported on Ubuntu and macOS. Copying
permission metadata in general `NodeState` initialization applied
`SecurityAdmin`-only defaults to generated GDS custom certificate groups
and denied their existing GDS-admin Browse requests. Metadata
preservation is now confined to NodeSet typed-child promotion; imported
Method argument permissions remain intact, while unrelated generated
nodes retain their existing initialization behavior.

Local Release validation also exposed a 260-character certificate path
in the startup-failure test added by this PR. Its isolated PKI now uses
a short temporary path. No assertions, authorization checks, or CI gates
were weakened, and no pre-existing tests were changed.

## macOS ARM64 NativeAOT suspension

Per maintainer request, `a15971f32` suspends only the failing
`macos-latest` / `osx-arm64` NativeAOT matrix entry. Re-enabling it once
the compiler contains the runtime fix is tracked in #4443. Intel macOS,
Linux, and Windows NativeAOT coverage, managed macOS tests, and all
other CI gates remain enabled. This is an explicit coverage suspension,
not a claim that the underlying compiler/linker defect is fixed.

The macOS ARM64 NativeAOT job on `a29fc6ca8` fails during native linking
with
`ld: Assertion failed: (_addend == uniqueIndex && "too many large
addends")`.
Both attempts of [run
34191755950](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/34191755950)
hit the same assertion before the native test executable can run; the
build/test summary failure is downstream.

This matches the NativeAOT Mach-O relocation issue in
dotnet/runtime#119380, fixed by relocation-anchor emission in
dotnet/runtime#124721 and its .NET 10 backport dotnet/runtime#132171.
The failing `Microsoft.DotNet.ILCompiler` 10.0.11 package's nuspec
points to `dotnet/dotnet` commit
`e2f47b0110ed922f21a1522da67279133ce28f32` (July 23); its object writer
does not contain the anchor fix merged on August 12. Version 10.0.11 is
currently the latest stable package.

The base commit passed on the same runner image and SDK, so this is not
being dismissed as a generally broken runner or a test flake. The
precise PR-specific object-layout trigger has not been isolated. A
compiler containing the relocation fix is needed before restoring the
lane. No test code, assertions, coverage thresholds, or shared tooling
settings were changed.

## Validation

Latest base-merge validation on both `net10.0` and `net48`, Release:

- Import helpers and argument-binding regressions: 51 passed per
framework.
- Batch/fluent import, generated-state overlays, runtime/startup
lifecycle, cross-source references, routing, reference snapshots, and
node-authoring selection: 344 passed per framework.
- Merge-specific cases cover available Method parents with
forward/inverse and URI/index references while retaining an
application-owned `NodeId` handle.
- The unchanged upstream `NodeSetImportIntegrationTests.cs:210` CA1861
warning remains; no diagnostic suppression or unrelated cleanup was
added.

Release CI-fix validation on both `net10.0` and `net48`:

- `CustomCertificateGroupIntegrationTest`: 4 passed, including the
unchanged CI failure.
- `UANodeSetHelpersTests`: 8 passed, including imported security
metadata.
- `StartupRuntimeNodeSetLifecycleTests`: 5 passed.

Earlier Debug merge and review validation on both frameworks:

- Lifecycle, RuntimeNodeSet, fluent, and OperationContext selection
after merging `978e6d1af`: 638 passed per framework.
- Startup/transport-binding and RuntimeNodeSet integration selection
after merging `017856f4a`: 28 passed per framework.
- Public exception contract and duplicate-registration selection in
`fbf08428b`: 6 passed per framework.

Targeted test invocations rebuilt the affected projects successfully.
The full `UA.slnx` suite was not rerun locally.

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [x] I have addressed **all** PR feedback received.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2bb8a834-7695-4ae6-84cb-2471c76d1b12
 2026-09-08 11:11:53 +00:00
romanett 30ed2390a8 Import NodeSet2 documents into a fluent node manager (#4432) 
## What

`INodeManagerBuilder.Import(UANodeSet, INodeSetImportFactoryProvider?)`
lets a source-generated (or hand-written fluent) node manager overlay a
NodeSet2 document that is only known at runtime onto the model it was
generated from.

Every document imported during one `Configure` pass forms a single batch
that is linked **exactly once** after the pass. A node may therefore
declare a `ParentNodeId` that lives in another document of the batch, or
a node the manager already owns — which is what makes an overlay
possible. Imported nodes resolve through
`Node(...)`/`NodeFromTypeId(...)` immediately, so they can be wired in
the same pass; they are registered with the manager when
`CompleteConfigureAsync` runs.

### Public surface

- **+1 method**: `INodeManagerBuilder.Import(UANodeSet nodeSet,
INodeSetImportFactoryProvider? factoryProvider = null)`.
- **+3 contracts** in `Opc.Ua.Server.Nodes`:
`INodeSetImportFactoryProvider`, `INodeSetImportFactory`,
`NodeSetImportDiscriminator` (TypeDefinition / MethodDeclaration /
NodeId).
- In `Opc.Ua.Export`: a public `UANodeSet.Import` overload taking a
`NodeSetImportStateFactory`, and a public batch
`LinkParentChildRelationships(context, nodes,
NodeSetImportLinkOptions)`. These replace what would otherwise have been
an `InternalsVisibleTo("Opc.Ua.Server")` on `Opc.Ua.Types`.
- `NodeState.IsExplicitlyDefinedChild(context, child)` and `protected
DetachExplicitlyDefinedChild(child)`, used to tell a generated slot
apart from an ordinary child.

### Typed states, no reflection

The generator emits `{Model}NodeSetImportFactoryProvider` — one factory
per model type/declaration, each calling a concrete constructor for an
*empty* state — and makes the generated node manager implement
`INodeSetImportFactoryProvider` by delegating to it. So the builder
resolves the provider from the manager (`NodeManager as
INodeSetImportFactoryProvider`); `Import` also accepts one directly, and
the generated manager exposes `partial void
AddNodeSetImportFactories(List<INodeSetImportFactory>)` for dependency
models. An imported node of a generated type materializes as its
generated `*State` subclass with no type lookup, which keeps the path
NativeAOT-safe.

Because the factories create empty states, an imported instance carries
exactly the children its document declares.

### Reconciliation

When an imported child lands in a slot the generated parent declares
(same BrowseName), it replaces the generated placeholder: the displaced
node and every descendant the document does not carry over are removed
from the manager, and references to them are retargeted at the
replacement. Replacing a node the same `Configure` pass already wired is
rejected with `BadInvalidState` rather than silently dropping the
wiring, and sealing a builder with an unregistered import batch is
rejected too.

### Also

- `RuntimeNodeSetNodeManager` now uses the same importer: its documents
are one batch, so a node may declare a parent in another document of the
group regardless of parse order, and duplicate NodeIds across documents
are detected while parsing.
- Linking resolves an imported child's reference type from its own
inverse reference (previously always `HasComponent`/`HasProperty`).
- Method import keeps the document's `UserExecutable`; it previously
copied `Executable` into it. View import keeps `EventNotifier`.

## Tests

- `NodeSetImporterTests` (15) — discriminator selection, generic
fallbacks, field/value/reference fidelity against a generic import,
cross-document linking, typed replacement of inherited and argument
slots, deferred linking preserving an application `Handle`, duplicate
NodeIds, factory misuse.
- `NodeSetImportBuilderTests` (15) — staging and lookup before
registration, batch linking, attaching to an existing node, provider
resolution, placeholder replacement with reference retargeting,
configured-placeholder rejection, seal/completion guards.
- `NodeSetImportIntegrationTests` (5) — a generated `[NodeManager]`
overlaying its own model on a running server: typed states through the
generated provider, cross-document linking, placeholder replacement, and
browse through a session.
- `NodeManagerGeneratorTests` (+4) — the emitted provider and the
manager's delegation, including that no factory uses
`Activator.CreateInstance`/reflection.
- `NodeSetAotTests` (+1) — the public state-factory import under
NativeAOT.

Full `Opc.Ua.Server.Tests` (4946), `Opc.Ua.SourceGeneration.Core.Tests`
(3821), `Opc.Ua.SourceGeneration.Tests` (165) and `Opc.Ua.Types.Tests`
pass locally on net10.0.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-07 23:12:43 +00:00
Marc Schier e73e71184d [WotCon] Add OPC UA WoT Connectivity 1.1 with the Types, source generation, Server and xRegistry work it builds on (#4128) 
## Summary

Adds OPC UA WoT Connectivity 1.1 to the stack, together with the Types,
source generation, Server and xRegistry work it is built on. This branch
is the merge point for the fourteen-PR stack that replaced integration
PR #4093; every constituent PR was reviewed and approved separately and
is listed below.

414 files changed, +108,936 / -3,199. Roughly 46k added lines of product
code, 57k of tests, plus samples, tools and documentation.

## Types

- Complete `ExtensionObject` raw-body decoding for binary, XML and JSON
bodies by resolving the concrete type through the message context's
encodeable factory, and simplify the `Variant` structure helpers by
delegating to it (#4128).
- Add `IAtomicFileReplace` as an optional `IFileSystem` capability so
existing external implementations keep working, with atomic publish for
`LocalFileSystem` via `File.Replace`/`File.Move` and for
`VirtualFileSystem` by re-keying the in-memory entry (#4128).
- Add lossless conversion between WoT documents and NodeSet2
(`Opc.Ua.Wot.WotNodeSetConverter`). A byte-exact `uav:nodeSet` envelope
is preserved when requested, the structured `uav:nodes` projection is
used when the readable vocabulary is incomplete, and NodeSet2 is
otherwise synthesized from readable WoT terms. Unmapped JSON members
survive a round trip as pointer-addressed residue (#4131).

## Source generation

- Improve the NodeSet to ModelDesign conversion and the generated node
state (#4132).
- Generate OPC UA models directly from WoT Thing Description files
(#4134).

## Server

- Add live NodeManager shadow and immediate reload. A reload
materializes a new generation beside the active one and switches
atomically; the superseded generation is retired gracefully so existing
MonitoredItems keep being served until they drain, or immediately when
the caller asks for it. Includes the request admission and drain
machinery that lets an orderly shutdown wait for admitted requests
instead of tearing down underneath them (#4147).
- Make subscription transfer between sessions transactional, so a failed
transfer leaves neither session holding a partially moved subscription
(#4150).

## xRegistry

- Allow clients to use explicit registry roots rather than assuming a
well-known location (#4130).

## WoT Connectivity

- Add the WoT Connectivity 1.1 information model: a registry-first
revision layered on the abstract xRegistry base model, incorporating the
complete published OPC 10100-1 v1.02 surface as deprecated nodes in the
same namespace so existing 1.02 clients keep working (#4139).
- Add the protocol binding abstractions and planners that compile WoT
forms into executable plans (#4140).
- Add the HTTP, Modbus, OPC UA and MQTT binding executors (#4144).
- Add the registry and its stores, including document versioning,
validation, dependency resolution and refresh (#4146).
- Add the registry client (#4142).
- Materialize WoT Thing Descriptions into the server address space,
deriving types from Thing Models and instances from Thing Descriptions
(#4154).
- Restore the WoT asset registry bridge so the deprecated 1.02 asset
surface is backed by the registry (#4171).
- Add the WoT Connectivity samples and complete the documentation
(#4156), and restore the remaining coverage and docs (#4172).

## New projects

`src/Opc.Ua.WotCon.Bindings`, `src/Opc.Ua.WotCon.Bindings.Mqtt`,
`tests/Opc.Ua.WotCon.Bindings.Tests`,
`tests/Opc.Ua.WotCon.Samples.Tests`, and the `samples/WotCon`
AggregationServer, AggregationClient and FlatTagServer samples.

## Constituent pull requests

Merged into this branch top-down, each independently approved:

| PR | Title |
| --- | --- |
| #4128 | [Types] Complete ExtensionObject decoding and add an atomic
file replace capability |
| #4131 | [Types] Add lossless conversion between WoT documents and
NodeSet2 |
| #4132 | [SourceGeneration] Improve NodeSet to ModelDesign conversion
and generated node state |
| #4134 | [SourceGeneration] Generate OPC UA models from WoT Thing
Description files |
| #4147 | [Server] Add live NodeManager shadow and immediate reload |
| #4150 | [Server] Make subscription transfer between sessions
transactional |
| #4130 | [XRegistry] Allow clients to use explicit registry roots |
| #4139 | [WotCon] Add the WoT Connectivity 1.1 information model |
| #4140 | [WotCon] Add the WoT protocol binding abstractions and
planners |
| #4144 | [WotCon] Add the HTTP, Modbus, OPC UA and MQTT binding
executors |
| #4146 | [WotCon] Add the WoT Connectivity registry and its stores |
| #4142 | [WotCon] Add the WoT Connectivity registry client |
| #4154 | [WotCon] Materialize WoT Thing Descriptions into the server
address space |
| #4156 | [WotCon] Add the WoT Connectivity samples and complete the
documentation |
| #4171 | Restore WoT asset registry bridge |
| #4172 | Restore WoT coverage and docs |

Supersedes #4093.

## Follow-up

The WoT Connectivity and WoT Binding drafts have moved to 1.1-draft2
since this work was authored. Aligning with that revision - the removed
group vocabulary, the new projection/View construct, the remaining model
vocabulary terms and the two new portable-identity validation rules - is
tracked separately and will follow in its own pull request.

## Validation

Built for every target framework with zero warnings and zero errors, and
validated at the tip of the stack with `Opc.Ua.WotCon.Tests` at 999
passed / 0 failed and `Opc.Ua.Server.Tests` at 4024 passed / 0 failed.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
 2026-08-05 10:14:15 +02:00
Marc Schier 168b3de353 Replace the RequestManager AsyncLocal guard with an explicit operation context (#4163) 
# Description

`RequestManager` tracked whether the calling flow was serving a Client
request in a `private readonly AsyncLocal<bool> m_inServiceDispatch`. It
was set by a `StandardServer.ProcessRequestAsync` override and read by
`NodeManagerLifecycle`, so that a lifecycle operation (`AddAsync` /
`ReloadAsync` / `RemoveAsync`) started from inside a request fails fast
with `InvalidOperationException` instead of draining — and therefore
waiting for — its own request.

Two problems with that mechanism, raised in review of #4093:

1. An `AsyncLocal<T>` written inside an `async` method never flows back
to the caller that awaited it, so every scope had to chain to the
previous value by hand and restore it on dispose. Subtle, and easy to
get wrong when a new entry point is added.
2. Ambient state is invisible at the call site, so the lifetime of the
mark could not be reasoned about from the code that depended on it —
especially once work is handed to a background task.

`RequestManager` also had no business carrying ambient state on behalf
of a different subsystem; its job is tracking requests for the drain.

## What changed

The guard is now an **exact identity lookup** against the request
registry `RequestManager` already maintains, and the operation is
threaded **explicitly**:

* **`RequestManager`** — `internal bool
IsExecutingRequest(IOperationContext?)` performs a reference-identity
lookup against `m_requests` under the existing `m_requestsLock`.
`m_inServiceDispatch`, `EnterServiceDispatchScope()`, the nested
`ServiceDispatchScope` class and the flag-based `IsExecutingRequest`
property are gone.
* **`StandardServer`** — the `ProcessRequestAsync` override is removed;
it existed only to open the dispatch scope.
* **`INodeManagerLifecycle`** — every member (`AddAsync` ×2 factory
kinds, `ReloadAsync` ×2, `RemoveAsync`) now takes the caller's
`IOperationContext?` directly, ahead of `CancellationToken ct =
default`. No overloads and no parallel signature are kept: the interface
is new in 2.0 and outside the 1.5.378 compatibility rule, so call sites
state their context explicitly or pass `null`.
* **`HostedNodeManagerLifecycle`** and
**`RuntimeNodeSetLifecycleExtensions`** thread the argument through;
`AddRuntimeNodeSetAsync` / `ReloadRuntimeNodeSetAsync` remain single
methods.
* **`SystemContextOperationExtensions.GetOperationContext(this
ISystemContext)`** (new, `Opc.Ua.Core`) lets a NodeManager or Method
callback hand over the operation it already received without
downcasting. It switches on `SessionSystemContext` and `SystemContext`,
the two independent roots every context derives from. No new interface
is introduced: casting the context to `IOperationContext` would not do,
because a context implements that by delegating to the operation it was
created for, so the cast yields the context rather than the operation.
* **`NodeManagerLifecycle.EnsureNotRequestCallback`** no longer consults
`ServerState`. The previous `CurrentState == Running` predicate could
let a genuinely re-entrant call slip past while the server was shutting
down; the request registry answers exactly in every state.

A NodeManager or Method callback now reads:

```csharp
await m_lifecycle.ReloadAsync(m_registration, replacement, context.GetOperationContext(), ct);
```

and a control-plane caller (hosted service, DI consumer) passes `null`.

## Behaviour

Unchanged for the case the guard exists to catch: a lifecycle call made
on behalf of an executing request is still rejected up front with the
same `InvalidOperationException` and the same message.

Two deliberate differences:

* **Improvement** — an internal `OperationContext` that was never
enrolled as a Client request is now correctly allowed through. The
boolean flag could not distinguish it from a real request.
* **Accepted trade-off** — a caller inside a request that passes no
operation is no longer detected at entry and falls back to the
pre-existing bounded `RequestManager.RequestDrainTimeout`, which already
covered any request that bypassed the service pipeline. Automatic
detection for an uncooperative caller inherently requires ambient state,
which is exactly what this change removes. Documented in
`docs/NodeManagers.md`.

## Alternatives considered and rejected

| Alternative | Why rejected |
| --- | --- |
| Delete the guard, rely only on `RequestDrainTimeout` | Turns a clear
immediate error into a multi-minute hang. |
| Infer the caller from the `CancellationToken` it passes | Works only
for the exact request token, breaks on linked tokens; ambient magic of a
different kind. |
| In-flight callback counter on `MasterNodeManager` | False positives:
rejects legitimate control-plane calls whenever any request is
concurrently dispatching. |
| Defer the drain to a background task so re-entrancy cannot deadlock |
The drains are woven into transactional commit / rollback and binding
reconciliation; deferring them breaks the transactional guarantees. |
| `Activity.Current` / `ThreadLocal` | The same ambient magic the issue
asks to remove. |

## Testing

* `RequestManagerTests` — the three `EnterServiceDispatchScope` tests
are replaced by coverage for `IsExecutingRequest(context)`: null,
never-registered, only-while-scope-open, visible across an `await` and a
background `Task.Run`, and two concurrently executing requests not
confusing each other.
* `NodeManagerLifecycleTests` — rejection from an executing request for
Add / Reload / Remove without invoking the factory, acceptance for a
context that is not an executing request, acceptance once the request
has completed, and the full callback shape
(`ServerSystemContext.Copy(operation)` → `GetOperationContext()` →
rejected).
* `HostedNodeManagerLifecycleTests` — forwarding tests for the
caller-context argument on all five members.
*
`tests/Opc.Ua.Core.Tests/Stack/State/SystemContextOperationExtensionsTests.cs`
(new) — `GetOperationContext` across both context roots and the
null-argument guard.

## Related Issues

- Fixes #4149

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 14402d11-e540-4faa-a5cd-94881bd7ce85
 2026-08-03 20:31:51 +00:00
Marc Schier 275c3800b5 [Server] Add live NodeManager lifecycle support (#4094) 
# Description

Adds supported live add, reload, and remove lifecycle operations for
NodeManager instances on a running server.

The change introduces a DI-backed `INodeManagerLifecycle` service with
direct `StandardServer` access, generation-aware registration handles,
copy-on-write routing, transactional prepare/commit/rollback, runtime
NodeSet convenience APIs, append-only namespace updates, model and
semantic change notifications, additive-compatible complex type priming,
and complete teardown behavior.

Lifecycle publication is coordinated with requests, sessions,
subscriptions, monitored items, continuation points, and shutdown so
retired managers cannot remain reachable or be disposed while still in
use. Compatible monitored items transfer to replacement generations
without a transient bad status. Removed or incompatible Nodes publish
`BadNodeIdUnknown` as required by OPC UA Part 4 section 5.8.4.1, retain
their MonitoredItem identity, and automatically recover when a
compatible Node with the same NodeId returns.

Registration is documented end to end in the new
[docs/NodeManagerRegistration.md](docs/NodeManagerRegistration.md),
which covers compile-time, startup, and runtime registration together
with the guarantees for MonitoredItems, Browse continuation points,
namespace indexes, DataTypes, and change notifications.
## Review feedback addressed

Interface surface reduced, as requested:

- `INodeManagerMonitoredItemTracker` folded into
`ISubscriptionMonitoredItemLifecycle`.
- `INodeManagerMutationCoordinator` and the asynchronous MonitoredItem
recovery entry point folded into `IDynamicNodeManagerHost`.
- `ISessionClosingRegistry` folded into `IServerInternal`, then replaced
by `ISession.IsClosing`.
- `ISubscriptionDeletionRegistry` folded into `ISubscriptionManager`,
then removed entirely.
- `INodeManagerContinuationPointTracker` folded into `ISession`.
- The synchronous recovery entry point kept on its own
`ISyncNodeManagerMonitoredItemRecovery`, because `AddPredefinedNode`
cannot await; it still routes through the asynchronous
`INodeManagerMonitoredItemLifecycle` that `AsyncNodeManagerAdapter`
exposes.
- `IMonitoredItemLifecycle` renamed to `IDetachableMonitoredItem`.

Reach into the Subscription area reduced to the minimum the feature
needs:

- The lifecycle feature itself leaves `SubscriptionManager` almost
untouched: two guards that reject a request from a Session that is
closing, on `CreateSubscription` and on `TransferSubscriptions`, because
it would create state that is about to be torn down. The deletion claim
registry, the session closing gates, the closed-session bookkeeping and
the deletion restore paths introduced by earlier revisions are gone.
Transferring a subscription away from a session that is closing stays
allowed, since that is the recovery path the service exists for; the
race that exposed is fixed by serialising source-queue closure with the
transfer rather than by rejecting it.
- Queueing is disabled again for a MonitoredItem with `QueueSize <= 1`,
as before this PR. Without a queue the Client is served the last sampled
value, so the missing-Node notification simply becomes that value;
losing values is expected for a MonitoredItem without queuing.
- What remains of the notification is one `DataValue` plus one bool in
`DataChangeQueueHandler`, mirroring the existing overflow pair, which
stops it being the value discarded when a configured queue is full.
- `MonitoredItem` keeps its original publish path, its original queue
creation and restore paths, its original triggering behaviour, and no
per-epoch bookkeeping.
- `IStoredMonitoredItem` carries `IsDeleted` and `IsDetached`, so a
restored item reports the state it was stored with rather than one
inferred from the last value's StatusCode.

Other feedback:

- Documented `HostedNodeManagerLifecycle`, `IDynamicNodeManagerHost`,
`NodeManagerRoutingTable`, and the `RequestManager` validation and
request scopes.
- Renamed `ValidateMonitoredItemAsync` to `CanAttachMonitoredItemAsync`
and `RestoreMonitoredItemAsync` to `RecoverMonitoredItemAsync`, so the
lifecycle API no longer collides with the durable Subscription
`RestoreMonitoredItem` path used at server startup.
- Removed the `CustomNodeManager2` dependency from the synchronous
MonitoredItem recovery path.
- `DataChangeQueueHandler` is back to the two constructors 1.5.378
shipped, each taking an optional `TimeProvider`.
- Session closing is a one-way latch, and the Session is removed even
when teardown throws.
- Documented what a custom NodeManager has to implement to take part in
live lifecycle operations, and which NodeManager implements
`INodeManagerReloadParticipant` today.

## Fixes found during review

- The guard that stops a NodeManager lifecycle operation being started
from inside a service handler never fired. It was backed by an
`AsyncLocal` written inside `async ValidateRequestAsync`, which is
invisible to the caller that awaited it, so a lifecycle call from a
NodeManager callback waited for its own request and held the lifecycle
semaphore forever. The mark is now applied by the service dispatcher,
and the drain is bounded so no path can wedge the lifecycle permanently.
- `ValidateRequestAsync` is no longer virtual. An override that rejected
a request after calling `base` left it registered forever; subclasses
now use `OnRequestValidatedAsync`, whose failures complete the request.
- A MonitoredItem deleted and disposed while a reload was handing it to
the replacement NodeManager could be attached anyway. The item now
reserves itself for the hand-over and the transition takes it back if it
was disposed meanwhile.
- The Overflow bit is recorded and applied when the value is published,
as the surrounding design does, rather than written into the value at
enqueue time.
- MonitoredItem cleanup for a Subscription being deleted no longer
passes null entries to `DeleteMonitoredItemsAsync`.
- Browse continuation points of a retired NodeManager are persisted and
disposed outside the Session lock.

## Open review discussions

Three threads are open, replied to and awaiting reviewer decisions:
separating service-call dispatch from NodeManager handling in
`MasterNodeManager` (tracked as #4098), the request-scoping shape in
`StandardServer`, and how much of the async MonitoredItem service
implementation should stay in `Subscription`.

Issue #4102 records a specification ambiguity found while implementing
the missing-Node notification: Part 4 §5.8.4.1 makes it mandatory, but
the protected over-capacity queue entry §5.13.1.5 defines applies to
`EventQueueOverflowEventType` only, so the specification does not say
how it survives a full queue.

## Validation

- Full `Opc.Ua.Server.Tests` on net10.0: 3,768 passed, 5 skipped.
- Lifecycle, MonitoredItem, Subscription and continuation point suites:
185 passed on net10.0 and 75 on net48.
- Server builds complete with zero warnings or errors.
- Two failures are pre-existing and unrelated:
`DurableDataValueQueueVerifyReferenceBatchingAsync` (flaky
`BatchPersistor` file contention, passes in isolation on net10.0 and
net48) and
`ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`
(fails on a clean tree with these changes stashed).

## Related Issues

- Fixes #3993
- Follow-up tracked in #4098
 2026-07-28 16:06:28 +02:00