Files
Marc Schier 4633aa7a34 Harden HA address-space hydration and replica-consistent NodeIds (#4445)
# Description

Follow-up to #4390 for distributed address-space ownership,
authoritative
hydration, and replica-consistent wire NodeIds. Tracks #4444.

Identity implementation: `a78d59067` (Server policy/lifecycle/adapter
seam) and
`07faf5cfd` (replica identity module, runtime wiring, examples and
proofs).

## Replica identity

- `UseReplicaNodeIdentity` / `ReplicaNodeIdFactory` reserve fixed shared
namespace
indexes before node-manager construction and reuse
`DefaultNodeIdFactory`.
  ApplicationUri and built-in diagnostics/configuration remain local.
- Replicated startup requires an explicit identity contract. Namespace
layout,
  deterministic mode, replica-set identity and writer policy must agree.
Standard factory views, replacement and registration preserve the
policy.
- Active/passive writer-assigned identities are replicated unchanged.
Independent
creation requires stable keys or explicit IDs. Live IDs and retained
tombstones
are reserved before writer allocation; no network/storage call runs in
`New`.
- Protected store descriptors use strong routing, including customized
hybrid
prefixes. Unknown legacy state and empty eventual scans cannot authorize
  automatic adoption. Verified-new hybrid provisioning is explicit.
- Active/active frames compare the descriptor before CRDT merge.
Rejected frames
are not reported as applied; incompatible admission remains at
ServiceLevel 0
  through the standard service-level wiring.
- Shared metadata validation covers type definitions, data types,
references,
permissions, NodeId/ExpandedNodeId/QualifiedName values,
arrays/matrices,
  DataValues and encodeable structures/arguments, without reflection.
- Runtime preparation hydrates retained IDs before replacement routes
publish.
Rebinding keeps the existing store/election or gossip transport/map and
attaches
to the new node-manager sources. Normal node-manager mutations,
including
  AddNodes, reach the cached local-address-space adapter.
- WoT materialization retains the stable configured factory. Samples
expose
genuinely factory-created nodes and a client `--identity` failover
workflow.

## Preserved HA repairs

The published `d99995003` and `79da74f14` fixes remain in place:

- Shared linearizable sequence/lease coordination; no pretend CAS on
CRDT and no
  private counter substituted for shared coordination.
- Strict authoritative record decoding, protected pending reservations,
safe
  snapshot publication and required delta retention.
- Accepted topology state controls authoritative membership; stale
deletion
  cannot remove a newer incarnation.
- Independent subtree attachment tracking and correct disposal of the
final
  snapshot task.
- Values wait for their topology, and parent tombstone horizons prevent
older
  descendant values crossing incarnations.

A new deterministic regression also covers writer hydration preparation:
temporary reconciliation removals must remain inbound application, not
be
recaptured as new shared deletion records.

## Validation

### Review feedback fixes

`3a682155b` addresses the active-node notification, WoT namespace-index
bounds,
and retired-manager XML documentation feedback. The address-space
adapter now
notifies listeners with the registered active node, and both WoT
managers reject
missing/out-of-range namespace indexes before ownership is established.

Targeted feedback validation passes on net10.0 and net48: 8
address-space adapter
cases and 74 WoT cases per framework, plus 17
hydration/gossip/live-client
integration cases on net10.0. The wrong-instance notification and
missing
namespace regressions were reproduced before the fixes.

### Identity implementation evidence

| Selected scope | net10.0 | net48 |
| --- | --- | --- |
| HA repair, identity, metadata, peer admission and real-client
identity/reload fixtures | 215 passed | 215 passed |
| Factory, fluent authoring/import, NodeManagement and address-space
adapter fixtures | 213 passed | 213 passed |
| WoT node-manager fixtures | 68 passed | 68 passed |

- Real UA-TCP clients compare full NodeIds/reference targets across
independent
servers in all four deterministic modes, with different manager/creation
order
and local allocations. Saved IDs work after endpoint failover and
restart.
- Real AddNodes-created IDs remain readable through successive live
manager
replacements in both replication modes, including nodes created after a
reload.
- Two process-level cases start three sample replicas, kill the serving
process,
verify a different serving ServerArray URI, and reuse cached
factory-generated
  IDs for Read and CreateMonitoredItems without remapping or rebrowsing.
- The identity/hydration scenario runs from the actual published win-x64
NativeAOT
  executable in the existing historian companion host.
- The focused writer-recapture regression was observed failing without
its fix,
  then passing. The real reload scenario was additionally repeated while
  diagnosing that race.
- Changed-file formatting/style/analyzers were applied. Existing CA1508,
CA1823,
CA1861 and CA1850 warnings in untouched upstream files were not
suppressed.

The selected checks above do not replace full-solution CI, coverage
gates or
maintainer review, and do not claim that the entire solution or every
native
platform was run. #4444 remains the tracking issue.

## Contract and consistency limits

Fixed identity does not make eventual payload reads linearizable. Hybrid
views
remain non-destructive, use explicit tombstones, and do not infer
empty-store
bootstrap or partition completion from absence. Compacted snapshots
require
authoritative storage. Failed/uncertain reservations require
reconciliation.

There is no automatic namespace-slot allocation, persistent logical-key
mapping
database, live renumbering, legacy-store wipe/adoption, or new hash
algorithm.
Opaque shared structures require a registered codec so their
namespace-bearing
fields can be validated. Application callbacks are still
application-owned.

## Related issues

- Tracks #4444.
- Follow-up to #4390.
- Builds on the merged NodeId factory (#4433), configure/seal (#4442)
and
  lifecycle/import (#4427) integration at `aedd3f738`.

## Checklist

- [ ] I have signed the CLA and read CONTRIBUTING.
- [x] Added targeted regression and integration tests.
- [x] Updated directly related API, HA, migration and sample
documentation.
- [x] Ran scoped formatting and the targeted framework checks listed
above.
- [ ] Full UA.slnx matrix and coverage gates completed.
- [ ] All CI and maintainer review completed.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9dbd9527-d818-47a0-984c-fbb1472d3d6b
2026-09-09 13:26:45 +02:00

204 lines
9.9 KiB
C#

/* ========================================================================
* Copyright (c) 2005-2026 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System;
using System.Threading;
using System.Threading.Tasks;
using Opc.Ua;
using Opc.Ua.Client;
namespace RedundantClient
{
public static partial class Program
{
private static async Task RunIdentityFailoverScenarioAsync(ManagedSession session, CancellationToken ct)
{
NodeId folder = await BrowseIdentityChildAsync(session, ObjectIds.ObjectsFolder, "HighAvailability", ct)
.ConfigureAwait(false);
NodeId generated = await BrowseIdentityChildAsync(session, folder, "FactoryAssigned", ct)
.ConfigureAwait(false);
NodeId value = await BrowseIdentityChildAsync(session, generated, "Value", ct).ConfigureAwait(false);
NodeId target = await BrowseIdentityChildAsync(session, generated, "Target", ct).ConfigureAwait(false);
await ReadCachedIdentityAsync(session, value, target, ct).ConfigureAwait(false);
string initialServer = await ReadServingServerUriAsync(session, ct).ConfigureAwait(false);
var changedReplica = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);
int interrupted = 0;
void OnConnectionStateChanged(object? _, ConnectionStateChangedEventArgs args)
{
if (args.NewState is ConnectionState.Reconnecting or ConnectionState.Failover)
{
Interlocked.Exchange(ref interrupted, 1);
}
else if (args.NewState == ConnectionState.Connected && Volatile.Read(ref interrupted) != 0)
{
changedReplica.TrySetResult(true);
}
}
session.ConnectionStateChanged += OnConnectionStateChanged;
try
{
Console.WriteLine(
"IDENTITY: cached shared NodeIds Value={0}, Target={1}; remove the active server now.",
value, target);
await changedReplica.Task.WaitAsync(TimeSpan.FromSeconds(90), ct).ConfigureAwait(false);
if (await ReadServingServerUriAsync(session, ct).ConfigureAwait(false) == initialServer)
{
throw new InvalidOperationException("The identity scenario requires a different serving replica.");
}
await ReadCachedIdentityAsync(session, value, target, ct).ConfigureAwait(false);
await CreateCachedIdentityMonitoredItemAsync(session, value, ct).ConfigureAwait(false);
Console.WriteLine(
"IDENTITY HA OK: exact cached NodeIds read and monitored on the promoted replica, " +
"without remapping or rebrowsing.");
}
finally
{
session.ConnectionStateChanged -= OnConnectionStateChanged;
}
}
private static async Task<NodeId> BrowseIdentityChildAsync(
ManagedSession session,
NodeId parent,
string name,
CancellationToken ct)
{
BrowseResponse response = await session.BrowseAsync(null, null, 0,
[
new BrowseDescription
{
NodeId = parent,
BrowseDirection = BrowseDirection.Forward,
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
IncludeSubtypes = true,
ResultMask = (uint)BrowseResultMask.All
}
], ct).ConfigureAwait(false);
if (response.Results.Count != 1 || StatusCode.IsBad(response.Results[0].StatusCode))
{
throw new ServiceResultException(
StatusCodes.BadNodeIdUnknown,
$"Cannot browse shared node '{parent}'.");
}
ArrayOf<ReferenceDescription> references = response.Results[0].References;
for (int i = 0; i < references.Count; i++)
{
ReferenceDescription reference = references[i];
if (reference.BrowseName.Name == name &&
reference.NodeId.ServerIndex == 0 &&
string.IsNullOrEmpty(reference.NodeId.NamespaceUri))
{
return ExpandedNodeId.ToNodeId(reference.NodeId, session.NamespaceUris);
}
}
throw new ServiceResultException(StatusCodes.BadNodeIdUnknown, $"Shared child '{name}' was not returned.");
}
private static async Task<string> ReadServingServerUriAsync(ManagedSession session, CancellationToken ct)
{
ReadResponse response = await session.ReadAsync(null, 0, TimestampsToReturn.Neither,
[new ReadValueId { NodeId = VariableIds.Server_ServerArray, AttributeId = Attributes.Value }], ct)
.ConfigureAwait(false);
if (response.Results.Count != 1 ||
StatusCode.IsBad(response.Results[0].StatusCode) ||
!response.Results[0].WrappedValue.TryGetValue(out ArrayOf<string> serverUris) ||
serverUris.IsEmpty)
{
throw new ServiceResultException(StatusCodes.BadUnexpectedError,
"The serving replica did not return its application URI in ServerArray.");
}
return serverUris[0];
}
private static async Task ReadCachedIdentityAsync(
ManagedSession session,
NodeId value,
NodeId target,
CancellationToken ct)
{
if (session.NamespaceUris.GetString(value.NamespaceIndex) != kHighAvailabilityNamespaceUri ||
session.NamespaceUris.GetString(target.NamespaceIndex) != kHighAvailabilityNamespaceUri)
{
throw new InvalidOperationException("The serving replica changed a cached shared namespace index.");
}
ReadResponse response = await session.ReadAsync(null, 0, TimestampsToReturn.Both,
[
new ReadValueId { NodeId = value, AttributeId = Attributes.Value },
new ReadValueId { NodeId = target, AttributeId = Attributes.Value }
], ct).ConfigureAwait(false);
if (response.Results.Count != 2 ||
StatusCode.IsBad(response.Results[0].StatusCode) ||
StatusCode.IsBad(response.Results[1].StatusCode) ||
response.Results[0].WrappedValue != Variant.From(12345) ||
response.Results[1].WrappedValue != Variant.From(value))
{
throw new ServiceResultException(StatusCodes.BadUnexpectedError,
"Cached shared identities or their NodeId-valued references did not survive failover unchanged.");
}
}
private static async Task CreateCachedIdentityMonitoredItemAsync(
ManagedSession session,
NodeId value,
CancellationToken ct)
{
CreateSubscriptionResponse subscription = await session.CreateSubscriptionAsync(
null, 1000, 60, 10, 0, false, 0, ct).ConfigureAwait(false);
try
{
CreateMonitoredItemsResponse items = await session.CreateMonitoredItemsAsync(
null, subscription.SubscriptionId, TimestampsToReturn.Both,
[
new MonitoredItemCreateRequest
{
ItemToMonitor = new ReadValueId { NodeId = value, AttributeId = Attributes.Value },
MonitoringMode = MonitoringMode.Reporting,
RequestedParameters = new MonitoringParameters
{
ClientHandle = 1,
SamplingInterval = 1000,
QueueSize = 1,
DiscardOldest = true
}
}
], ct).ConfigureAwait(false);
if (items.Results.Count != 1 || StatusCode.IsBad(items.Results[0].StatusCode))
{
throw new ServiceResultException(StatusCodes.BadMonitoredItemIdInvalid,
"A monitored item could not use the exact cached shared NodeId after failover.");
}
}
finally
{
await session.DeleteSubscriptionsAsync(null, [subscription.SubscriptionId], ct).ConfigureAwait(false);
}
}
}
}