mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-21 17:16:16 +08:00
87f253bb47
# Description
Adds **unbounded monitored items per subscription** to the V2
`ManagedSession` subscription engine via automatic, transparent
partitioning. Top-level public API and runtime semantics for callers
that stay under the server's `MaxMonitoredItemsPerSubscription` cap are
unchanged; callers that exceed the cap continue to succeed instead of
failing with `BadTooManyMonitoredItems`.
## Approach (OPC UA Part 4 1.05.07 conformant)
- The public `ISubscription` is now a `LogicalSubscription` that owns
one **primary** and zero-or-more **secondary** `Subscription`
partitions, each a real server-side subscription.
- `CompositeMonitoredItemCollection` aggregates the per-partition
`MonitoredItemCollection` views so `Items`, indexer access, and
enumeration look unsharded to the caller.
- `PartitionPlacementPolicy` decides the target partition at
`TryAddMonitoredItem` time:
- **`StrictAffinity`** — keep items tagged with the same `Affinity`
co-located until the partition fills, then split with a one-time
warning.
- **`BestEffortAffinity`** (default) — prefer co-location but spill
silently when the partition is full.
- No affinity → fall back to the least-loaded partition (or the
primary).
- `PartitionForwardingHandler` proxies notifications from secondary
partitions to the caller's handler with the **primary partition's**
`ISubscription` identity, so handler code never observes the split.
- **Durable subscriptions (Part 4 §5.13.9):** when `SetAsDurableAsync`
runs before any items are added, the wrapper records the intent and
installs an `OnAfterCreateAsync` hook on every existing partition. Each
newly-minted secondary inherits the hook so `SetSubscriptionDurable` is
invoked between `CreateSubscription` and `CreateMonitoredItems` — the
only order the standard allows.
- **Snapshot/restore** preserves the partition layout so reconnect and
failover restore items into the same partition they were created in,
with affinity tags intact.
- Secondary partitions that go idle (no items, no pending
`CreateMonitoredItems`) are deleted to release server resources.
## Performance
Hot-path dispatch goes through the primary partition unchanged for the
unsharded case (single-partition fast path); the forwarding handler only
allocates on the multi-partition path. The performance smoke test in
`Tests/Opc.Ua.Stress.Tests` shows no regression.
## New configuration
```csharp
session.AddSubscription(handler, new SubscriptionOptions
{
MaxMonitoredItemsPerPartition = 5000, // optional cap per server-side subscription
PartitionPlacementPolicy = // optional placement policy
PartitionPlacementPolicy.StrictAffinity
});
sub.TryAddMonitoredItem(
name,
nodeId,
o => o with { Affinity = "machine-A" }, // optional co-location hint
out _);
```
Defaults preserve pre-change behaviour (`MaxMonitoredItemsPerPartition =
uint.MaxValue`, `BestEffortAffinity`).
## New types (Libraries/Opc.Ua.Client/Subscription)
| File | Purpose |
| --- | --- |
| `CompositeMonitoredItemCollection.cs` | Aggregate collection view |
| `ILogicalSubscription.cs` / `IPartitionedSubscription.cs` | Internal
contracts used by tests and infrastructure |
| `LogicalSubscription.cs` | Wrapper that owns the partitions |
| `PartitionForwardingHandler.cs` | Notification proxy |
| `PartitionPlacementPolicy.cs` | Affinity-aware item placement |
Modified: `ISubscriptionNotificationHandler`, `MonitoredItemManager`,
`MonitoredItemOptions` (adds `Affinity`), `Subscription`,
`SubscriptionManager`, `SubscriptionOptions` (adds
`MaxMonitoredItemsPerPartition`, `PartitionPlacementPolicy`),
`SubscriptionManagerSerializer`, `MonitoredItemStateSnapshot`,
`SubscriptionStateSnapshot`, `ManagedSessionExtensions`.
## Documentation
- New: `Docs/UnboundedSubscriptions.md` — feature overview, affinity
semantics, snapshot/durable interaction.
- Cross-links added in `Docs/Sessions.md`, `Docs/MigrationGuide.md`,
`Docs/README.md`.
## Validation
| Suite | Result |
| --- | --- |
| Unit tests for new types — `Opc.Ua.Client.Tests.Subscription.*`
(net10.0 + net48) | **39 / 39** |
| All `Opc.Ua.Client.Subscriptions.*` unit tests (net48) | **152 / 152**
|
| Integration — `Opc.Ua.Subscriptions.Tests.UnboundedSubscriptionTests`
(net10.0 + net48) | **3 / 3** (1 durable-only skipped) |
| Integration — `Opc.Ua.Subscriptions.Durable.Tests.*V2Async` (net10.0)
| **6 / 6** (incl. new `DurableLifetimeAppliesToAllPartitionsV2Async`) |
| Adjacent — `Opc.Ua.Sessions.Tests` (net10.0) | **690 / 690** |
| Adjacent — `Opc.Ua.Subscriptions.Classic.Tests` (net10.0) | **32 /
32** |
| Adjacent — `Opc.Ua.Features.Tests` (net10.0) | **128 / 128** |
| Adjacent — `Opc.Ua.Stress.Tests` (net10.0) | **159 / 159** |
| NativeAOT smoke — `Opc.Ua.Aot.Tests.SubscriptionAotTests` (win-x64) |
**5 / 5** (incl. new `UnboundedSubscriptionWithCapAotAsync`) |
## Compatibility
- **No public type removed**, **no obsoletions added**.
- Public surface on `ISubscription` / `ManagedSession` unchanged.
- New options non-breaking by default (`MaxMonitoredItemsPerPartition =
uint.MaxValue` preserves prior behaviour; `BestEffortAffinity` matches
the closest analogue to the prior single-partition semantics).
- Classic V1 subscription engine unchanged.
## Related Issues
- _No tracking issue filed — please reference the appropriate one if any
exists._
## Security review
Scoped against Microsoft Service Tree
``59eec07a-6f20-42b9-b41b-d20e0a6322da`` (Azure Industrial IoT) using a
combination of the security-agent MCP (code-integrity, secrets-usage,
tls, policy-governance, ai-chat) and the ``security-review`` SAST
sub-agent on the PR diff. Bar: SDL secure-coding for a client library +
OPC UA Part 2 / Part 4 §5.5 / §5.13 spec.
Fixed in commit ``c6997689``:
| # | Sev | Title | Fix |
| --- | --- | --- | --- |
| 1 | CRITICAL | Unbounded partition fan-out under hostile server
(CWE-400/770) | Added ``SubscriptionOptions.MaxPartitionCount`` (default
``32``), enforced in ``PartitionPlacementPolicy``, surfaced as
``PlacementDecision.RejectMaxPartitionCountReached`` |
| 2 | HIGH | Snapshot ``LogicalGroupId`` option-consistency not
validated (CWE-502) | ``ValidateAndSortGroup`` now asserts all snapshots
in a group agree on every subscription-wide option |
| 3 | MEDIUM | Restore-path secondaries bypassed
``PartitionForwardingHandler`` (CWE-362) |
``AttachSecondaryPartitionAsync`` now reuses the wrapper's forwarding
handler when present |
Findings explicitly assessed and dismissed (sub-agent confirmed):
* Affinity-index growth — uses ``StringComparer.Ordinal`` (no
CVE-2022-29117 class); growth is application-controlled, not
attacker-controlled.
* Idle-timer race vs ``DisposeAsync`` — race exists; worst case is a
swallowed ``ObjectDisposedException``. No attacker trigger.
* Cross-wrapper notification routing — each ``Add`` allocates its own
forwarding handler; ``BindLogical`` rejects re-binding; no structural
path for crosstalk.
* OPC UA Part 4 §5.13.6 (``SetTriggering`` cross-partition) — enforced
by ``LogicalSubscription.SetTriggeringAsync``.
* OPC UA Part 4 §5.13.9 (durable hook ordering) — installed uniformly on
every minted partition.
Validation: ``Opc.Ua.Client`` builds clean on net10.0; 43 V2 unit tests
+ 3 unbounded integration tests + 6 V2 durable integration tests all
pass.
## Checklist
- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
282 lines
12 KiB
C#
282 lines
12 KiB
C#
/* ========================================================================
|
|
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
|
|
*
|
|
* OPC Foundation MIT License 1.00
|
|
*
|
|
* Permission is hereby granted, free of charge, to any person
|
|
* obtaining a copy of this software and associated documentation
|
|
* files (the "Software"), to deal in the Software without
|
|
* restriction, including without limitation the rights to use,
|
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
* copies of the Software, and to permit persons to whom the
|
|
* Software is furnished to do so, subject to the following
|
|
* conditions:
|
|
*
|
|
* The above copyright notice and this permission notice shall be
|
|
* included in all copies or substantial portions of the Software.
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
|
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
|
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
|
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
|
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
|
* OTHER DEALINGS IN THE SOFTWARE.
|
|
*
|
|
* The complete license agreement can be found here:
|
|
* http://opcfoundation.org/License/MIT/1.00/
|
|
* ======================================================================*/
|
|
|
|
using System;
|
|
using System.Collections.Generic;
|
|
using System.Globalization;
|
|
using System.IO;
|
|
using System.Threading;
|
|
using System.Threading.Tasks;
|
|
|
|
namespace Opc.Ua.Client.Subscriptions
|
|
{
|
|
/// <summary>
|
|
/// Save / Load support for the V2 <see cref="SubscriptionManager"/>.
|
|
/// </summary>
|
|
/// <remarks>
|
|
/// <para>
|
|
/// The on-wire format is the OPC UA <see cref="BinaryEncoder"/> format.
|
|
/// Each subscription is captured as a
|
|
/// <see cref="SubscriptionStateSnapshot"/> — an
|
|
/// <see cref="IEncodeable"/> emitted by the
|
|
/// <see cref="DataTypeAttribute"/> source generator — and written
|
|
/// directly via <see cref="IEncoder.WriteEncodeable{T}(string, T)"/>.
|
|
/// The reader instantiates <see cref="SubscriptionStateSnapshot"/>
|
|
/// statically by type (no <see cref="ExpandedNodeId"/> lookup is
|
|
/// required because the wire schema is implicit in the call site).
|
|
/// Future schema evolution is handled by adding new optional fields
|
|
/// to the snapshot record — they are encoded only when set and
|
|
/// recognized via <see cref="IDecoder.HasField(string)"/> by the
|
|
/// reader.
|
|
/// </para>
|
|
/// <para>
|
|
/// The stream still preserves the source session's namespace and
|
|
/// server URI tables so the snapshot is portable across sessions
|
|
/// whose tables index URIs at different positions; the loader
|
|
/// remaps those indices into the target session's tables before
|
|
/// decoding each snapshot's <c>NodeId</c> fields.
|
|
/// </para>
|
|
/// </remarks>
|
|
internal static class SubscriptionManagerSerializer
|
|
{
|
|
#pragma warning disable RCS1229 // Use async/await when necessary - this path is synchronous; ValueTask wraps work for future async I/O
|
|
public static ValueTask SaveAsync(
|
|
SubscriptionManager manager,
|
|
Stream stream,
|
|
IServiceMessageContext messageContext,
|
|
IEnumerable<ISubscription>? subscriptions,
|
|
CancellationToken ct = default)
|
|
#pragma warning restore RCS1229
|
|
{
|
|
ct.ThrowIfCancellationRequested();
|
|
if (manager == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(manager));
|
|
}
|
|
if (stream == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(stream));
|
|
}
|
|
if (messageContext == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(messageContext));
|
|
}
|
|
|
|
// Capture a snapshot per selected subscription. Default =
|
|
// all subscriptions managed by this instance. The public
|
|
// surface is the LogicalSubscription wrapper, which may
|
|
// own multiple partitions — emit one snapshot per
|
|
// partition so a later LoadAsync can regroup via
|
|
// SubscriptionStateSnapshot.LogicalGroupId. The older
|
|
// concrete Subscription path is retained for tests /
|
|
// direct usage.
|
|
IEnumerable<ISubscription> selected =
|
|
subscriptions ?? manager.Items;
|
|
var snapshots = new List<SubscriptionStateSnapshot>();
|
|
foreach (ISubscription s in selected)
|
|
{
|
|
if (s is LogicalSubscription logical)
|
|
{
|
|
snapshots.AddRange(logical.SnapshotAllPartitions());
|
|
}
|
|
else if (s is Subscription concrete)
|
|
{
|
|
snapshots.Add(concrete.Snapshot());
|
|
}
|
|
}
|
|
|
|
using var encoder = new BinaryEncoder(stream, messageContext, true);
|
|
encoder.WriteStringArray(null, messageContext.NamespaceUris.ToArrayOf());
|
|
encoder.WriteStringArray(null, messageContext.ServerUris.ToArrayOf());
|
|
encoder.WriteInt32(null, snapshots.Count);
|
|
foreach (SubscriptionStateSnapshot snapshot in snapshots)
|
|
{
|
|
// Write the snapshot directly as an IEncodeable.
|
|
// Schema identity is statically encoded by the call site
|
|
// (we always read back a SubscriptionStateSnapshot); schema
|
|
// evolution is handled by adding new optional fields with
|
|
// CanOmitFields-aware encoding.
|
|
encoder.WriteEncodeable(null, snapshot);
|
|
}
|
|
return default;
|
|
}
|
|
|
|
public static async ValueTask<IReadOnlyList<ISubscription>> LoadAsync(
|
|
SubscriptionManager manager,
|
|
Stream stream,
|
|
IServiceMessageContext messageContext,
|
|
Func<string, ISubscriptionNotificationHandler> handlerFactory,
|
|
bool transferSubscriptions,
|
|
CancellationToken ct)
|
|
{
|
|
if (manager == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(manager));
|
|
}
|
|
if (stream == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(stream));
|
|
}
|
|
if (messageContext == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(messageContext));
|
|
}
|
|
if (handlerFactory == null)
|
|
{
|
|
throw new ArgumentNullException(nameof(handlerFactory));
|
|
}
|
|
|
|
using var decoder = new BinaryDecoder(stream, messageContext, true);
|
|
ArrayOf<string?> nsUris = decoder.ReadStringArray(null);
|
|
ArrayOf<string?> serverUris = decoder.ReadStringArray(null);
|
|
decoder.SetMappingTables(
|
|
nsUris.IsNull
|
|
? new NamespaceTable()
|
|
: new NamespaceTable(nsUris.Memory.ToArray()!),
|
|
serverUris.IsNull
|
|
? new StringTable()
|
|
: new StringTable(serverUris.Memory.ToArray()!));
|
|
|
|
int count = decoder.ReadInt32(null);
|
|
if (count <= 0)
|
|
{
|
|
return [];
|
|
}
|
|
|
|
// Read every snapshot off the wire first; we group by
|
|
// LogicalGroupId before restoring so multi-partition
|
|
// wrappers come back as one logical subscription.
|
|
var raw = new List<SubscriptionStateSnapshot>(count);
|
|
for (int i = 0; i < count; i++)
|
|
{
|
|
ct.ThrowIfCancellationRequested();
|
|
raw.Add(decoder.ReadEncodeable<SubscriptionStateSnapshot>(null));
|
|
}
|
|
|
|
var restored = new List<ISubscription>();
|
|
int handlerIndex = 0;
|
|
|
|
// 1) Standalone snapshots (LogicalGroupId == null) keep
|
|
// today's behavior — one wrapper per snapshot. This
|
|
// covers V1 snapshot files where LogicalGroupId did
|
|
// not exist on the wire (decodes as null).
|
|
// 2) Grouped snapshots are validated as a contiguous
|
|
// 0..N-1 PartitionIndex sequence with exactly one
|
|
// primary and the same LogicalGroupId. Any
|
|
// discrepancy throws BadDecodingError so a corrupted
|
|
// snapshot fails loudly rather than silently
|
|
// fragmenting state.
|
|
var groups = new Dictionary<string, List<SubscriptionStateSnapshot>>(
|
|
StringComparer.Ordinal);
|
|
var standalone = new List<SubscriptionStateSnapshot>();
|
|
foreach (SubscriptionStateSnapshot snap in raw)
|
|
{
|
|
if (string.IsNullOrEmpty(snap.LogicalGroupId))
|
|
{
|
|
standalone.Add(snap);
|
|
}
|
|
else
|
|
{
|
|
if (!groups.TryGetValue(snap.LogicalGroupId,
|
|
out List<SubscriptionStateSnapshot>? bucket))
|
|
{
|
|
bucket = [];
|
|
groups[snap.LogicalGroupId] = bucket;
|
|
}
|
|
bucket.Add(snap);
|
|
}
|
|
}
|
|
|
|
foreach (SubscriptionStateSnapshot snap in standalone)
|
|
{
|
|
ct.ThrowIfCancellationRequested();
|
|
string syntheticName = handlerIndex.ToString(
|
|
CultureInfo.InvariantCulture);
|
|
handlerIndex++;
|
|
ISubscription subscription = await manager.RestoreAsync(
|
|
handlerFactory(syntheticName),
|
|
snap,
|
|
transferSubscriptions,
|
|
ct).ConfigureAwait(false);
|
|
restored.Add(subscription);
|
|
}
|
|
foreach (KeyValuePair<string, List<SubscriptionStateSnapshot>> entry in groups)
|
|
{
|
|
ct.ThrowIfCancellationRequested();
|
|
List<SubscriptionStateSnapshot> ordered =
|
|
ValidateAndSortGroup(entry.Key, entry.Value);
|
|
string syntheticName = handlerIndex.ToString(
|
|
CultureInfo.InvariantCulture);
|
|
handlerIndex++;
|
|
ISubscription subscription = await manager.RestoreGroupAsync(
|
|
handlerFactory(syntheticName),
|
|
ordered,
|
|
transferSubscriptions,
|
|
ct).ConfigureAwait(false);
|
|
restored.Add(subscription);
|
|
}
|
|
return restored;
|
|
}
|
|
|
|
/// <summary>
|
|
/// Validate that <paramref name="bucket"/> forms a coherent
|
|
/// multi-partition snapshot group: every snapshot must carry
|
|
/// the same <paramref name="groupId"/>, the
|
|
/// <c>PartitionIndex</c> values must be a contiguous
|
|
/// <c>0..N-1</c> sequence with no duplicates and exactly one
|
|
/// primary (<c>PartitionIndex == 0</c>). Returns the
|
|
/// snapshots sorted by <c>PartitionIndex</c>.
|
|
/// </summary>
|
|
/// <exception cref="ServiceResultException">
|
|
/// <see cref="StatusCodes.BadDecodingError"/> when the group
|
|
/// is malformed.
|
|
/// </exception>
|
|
private static List<SubscriptionStateSnapshot> ValidateAndSortGroup(
|
|
string groupId,
|
|
List<SubscriptionStateSnapshot> bucket)
|
|
{
|
|
bucket.Sort(static (a, b) => a.PartitionIndex.CompareTo(b.PartitionIndex));
|
|
|
|
for (int i = 0; i < bucket.Count; i++)
|
|
{
|
|
if (bucket[i].PartitionIndex != i)
|
|
{
|
|
throw ServiceResultException.Create(
|
|
StatusCodes.BadDecodingError,
|
|
"Multi-partition snapshot group '{0}' has " +
|
|
"non-contiguous or duplicated PartitionIndex " +
|
|
"values (expected {1} at position {1}, got {2}).",
|
|
groupId, i, bucket[i].PartitionIndex);
|
|
}
|
|
}
|
|
return bucket;
|
|
}
|
|
}
|
|
}
|