Files
UA-.NETStandard/tests/Opc.Ua.Server.Tests/NodeManagerDispatchRoutingTests.cs
romanett 0dff94bd37 Server quick wins from the o6 benchmark gap analysis: admission sizing, sampling permission cache, routed Read dispatch (#4743)
The o6-automation benchmark gap analysis listed several server quick
wins that need no wire or public API change. This PR implements three of
them. The 2.0 server now passes the benchmark's 10 clients × 1000
outstanding Reads test with the benchmark's own queue sizing. Its 100 Hz
sampling capacity grows about 7–10× on the same machine, and a
single-value Read allocates 8 % less.

## Results

Paired A/B against `master` (071660354): alternating builds, 3 rounds (5
for Read), medians, 24-core machine shared with other sessions.
Allocated bytes and CPU per operation are the reliable signal here.
Throughput varied up to 3× between rounds of the same build, so it is
listed only where the result is pass/fail.

| Scenario | Server alloc / op | Server CPU / op | Result |
|---|---|---|---|
| o6 `sub_client`, 2048 counters, 10 ms sampling, 1 s publishing |
**2309 → 111 B / sample (−95 %)** | **3.75 → 1.14 µs / sample (−70 %)**
| PASS on both |
| Harness `counters`, 2048 items, 10 ms / 1 s | 232 → 11.2 KB /
notification (−95 %) | −74 % | GC pause 644 → 24 ms in 12 s |
| **o6 sampling capacity** (largest count with every item ≥ 99 % of
samples, doubling from 1024, then bisect) | | | **2176–2304 →
16384–22528 items** (2 searches each) |
| o6 `client`, 10 × 1000 outstanding Reads, `MaxQueuedRequestCount =
10000` | | | **master fails 3/3 (BadServerTooBusy), this PR passes 3/3**
(31–36k req/s) |
| o6 `client`, 10 × 100 outstanding Reads | 7823 → 7556 B (−3 %) | ±0 |
|
| Read, 1 value (20 sessions) | 6534 → 6020 B (**−8 %**) | −1 % | |
| Read, 100 values | −1 % | ±0 | |
| Write, 100 values | −1 % | −5 % | |

For reference, open62541 (single thread, same Windows machine and native
client) sustained 5632 items in the gap analysis.

## Changes

### 1. `MaxQueuedRequestCount` admits that many established requests
(Balanced / TrustedReservations)
- `ServerResourceIsolationOptions.CreateRuntimePlan` carved the
bootstrap, reconnect, control and trusted floors out of the configured
queue, so established sessions could queue only `N − 3`. The default
totals of the decoded-request admission stages (RequestQueue,
RequestQueueBytes, ParkedRequest) now add the floors on top; the shared
capacity equals the configured value.
- `FairRequestQueue` takes its physical bound from the provider's stage
capacity (internal `IResourceIsolationCapacitySource`), so the protected
entries stay reachable when the shared part is full.
- RequestExecution keeps its floors inside the worker limit, so an idle
worker remains for protected requests. Explicit `Stages[].Capacity`
overrides keep their meaning as absolute totals including the floors.

### 2. Sampling without a role permission check per sample
- Since #4503 every sampling cycle called `ValidateRolePermissionsAsync`
for every item. For a `CustomNodeManager2` node that built full node
metadata (13-attribute read, namespace defaults via sync-over-async,
three nested node manager operations) only to return Good: about 80 % of
the 2.3 KB per sample.
- Each sampling loop now keeps per-item state between cycles:
ReadValueIds, result buffers and the last permission verdict. A verdict
is reused while the item's Session, effective identity and node handle
are unchanged and no permission change was signalled:
- `NodeState` bumps an internal version counter when `RolePermissions`
or `UserRolePermissions` of any node change (setter or Write);
- `IConfigurationNodeManager.DefaultPermissionsChanged` (namespace
defaults);
- a new `EffectiveIdentity` instance (ActivateSession with another user,
or roles refreshed after `RoleConfigurationChanged`).
- Independently, every item is revalidated once per second, spread over
the cycles, which bounds the delay for permissions that a node manager
computes without a signal (`OnReadRolePermissions`, custom
`ValidateRolePermissionsAsync`).
- The loop reuses its item snapshot until items are added or removed,
reads all items of one owner with one context instead of a LINQ
`GroupBy` (items moved to another session by TransferSubscriptions still
use the per-owner path), marks disabled items processed instead of
rebuilding the list, and checks for a detached loop without taking the
group lock.
- `Opc.Ua.Types` grants `InternalsVisibleTo` to `Opc.Ua.Server` for the
version counter.

### 3. Read and Write call only the node managers routed for the request
- `NodeManagerServiceDispatcher.ReadAsync`/`WriteAsync` called every
node manager. Every `AsyncCustomNodeManager` (configuration,
diagnostics, core, …) then copied the system context and allocated a
dictionary and a list, even with none of the requested nodes in its
namespaces.
- Validation now collects the namespace routes of the valid operations
and the dispatcher skips node managers outside them. Every manager of a
requested namespace is still called in dispatch order, so overlapping
managers keep their precedence. A namespace without a route still calls
every node manager, which keeps `BadNodeIdUnknown` and custom routing
unchanged.
- Not done: reusing the handle resolved during validation inside
`ReadAsync` needs a new node manager overload (public API), so nodes are
still resolved twice. HistoryRead and Call keep the old dispatch: Call
routes by object and method, and HistoryRead releases continuation
points per manager.

## Behaviour notes
- **`MaxQueuedRequestCount`** keeps its 1.5.x meaning again: established
sessions can have that many decoded requests waiting. Balanced and
TrustedReservations size the RequestQueue stage at
`MaxQueuedRequestCount + floors` (for example 10,003 for 10,000). A
deployment that relied on the smaller queue lowers
`MaxQueuedRequestCount` or sets the RequestQueue `Capacity`.
`docs/MigrationGuide.md`, `docs/ResourceIsolation.md` and
`docs/ServerScalability.md` are updated.
- **Permission changes for sampled items** still apply within one or two
sampling cycles when they come through
`NodeState.RolePermissions`/`UserRolePermissions`, namespace default
permissions, or a new effective identity. Permissions that a node
manager computes without changing those (for example
`OnReadRolePermissions` or a custom `ValidateRolePermissionsAsync`)
apply within one second instead of on the next sample.
`docs/NodeManagers.md` describes it.
- **Custom node managers** that process Reads or Writes of nodes in
namespaces they did not register receive those calls only while some
valid node of the request is in an unrouted namespace. Such a manager
registers its namespaces with `RegisterNamespaceManager` (documented in
`docs/NodeManagers.md`).

## Testing
- New unit tests:
- `RuntimeResourceIsolationTests`: plan sizing per mode, explicit
capacities, and a fair queue that admits exactly `MaxQueuedRequestCount`
requests from four owners, rejects the next and still admits bootstrap
and reconnect.
- `SamplingGroupPermissionCacheTests`: verdict reuse, revocation through
node RolePermissions in the same cycle, namespace default change,
identity change, periodic revalidation, disabled items. Each change is
injected inside the Read of a known cycle, so the tests do not depend on
timing.
- `NodeManagerDispatchRoutingTests`: non-owning managers are skipped,
several owners keep result order, an unrouted namespace still calls
everyone, two managers of one namespace keep their precedence, Write.
- net10.0: `Opc.Ua.Server.Tests` 8462 passed,
`Opc.Ua.Subscriptions.Tests` 643 passed, `Opc.Ua.Sessions.Tests` 851
passed / 379 skipped, `Opc.Ua.Core.Tests` (Stack.Server) 1858 passed,
`Opc.Ua.Stress.Tests` Contract|Integration 45 passed; 0 failures.
- net48 build of `Opc.Ua.Server.Tests` clean; `dotnet format whitespace`
clean.
- Measured with `perf/ServerLoadHarness` and the o6 scenarios (`o6read`,
`o6counters`) from the gap-analysis harness, which is not part of this
PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 08:52:09 +00:00

273 lines
12 KiB
C#

/* ========================================================================
* Copyright (c) 2005-2026 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Moq;
using NUnit.Framework;
using Opc.Ua.Server.TestFramework;
namespace Opc.Ua.Server.Tests
{
/// <summary>
/// Verifies that Read and Write are dispatched only to the node managers that serve
/// the namespaces of the requested nodes, with unchanged results.
/// </summary>
[TestFixture]
[Category("MasterNodeManager")]
[Parallelizable]
public sealed class NodeManagerDispatchRoutingTests
{
private const string kNamespaceA = "http://test.org/UA/RoutingA/";
private const string kNamespaceB = "http://test.org/UA/RoutingB/";
[Test]
public async Task ReadCallsOnlyTheNodeManagersOfTheRequestedNamespaceAsync()
{
await RunAsync(async (sut, a, b, nsA, nsB) =>
{
(ArrayOf<DataValue> values, _) = await sut.ReadAsync(
Context(), 0, TimestampsToReturn.Neither,
[Value(new NodeId(1, nsA)), Value(new NodeId(2, nsA))]).ConfigureAwait(false);
Assert.That(values[0].WrappedValue, Is.EqualTo(new Variant("A")));
Assert.That(values[1].WrappedValue, Is.EqualTo(new Variant("A")));
Assert.That(a.ReadCalls, Is.EqualTo(1));
Assert.That(b.ReadCalls, Is.Zero, "a node manager without a requested namespace is skipped");
}).ConfigureAwait(false);
}
[Test]
public async Task ReadCallsEveryOwningNodeManagerOnceAndKeepsResultOrderAsync()
{
await RunAsync(async (sut, a, b, nsA, nsB) =>
{
(ArrayOf<DataValue> values, _) = await sut.ReadAsync(
Context(), 0, TimestampsToReturn.Neither,
[
Value(new NodeId(1, nsB)),
Value(new NodeId(1, nsA)),
Value(new NodeId(2, nsB))
]).ConfigureAwait(false);
Assert.That(values[0].WrappedValue, Is.EqualTo(new Variant("B")));
Assert.That(values[1].WrappedValue, Is.EqualTo(new Variant("A")));
Assert.That(values[2].WrappedValue, Is.EqualTo(new Variant("B")));
Assert.That(a.ReadCalls, Is.EqualTo(1));
Assert.That(b.ReadCalls, Is.EqualTo(1));
}).ConfigureAwait(false);
}
[Test]
public async Task ReadOfAnUnroutedNamespaceStillCallsEveryNodeManagerAsync()
{
await RunAsync(async (sut, a, b, nsA, nsB) =>
{
(ArrayOf<DataValue> values, _) = await sut.ReadAsync(
Context(), 0, TimestampsToReturn.Neither,
[Value(new NodeId(1, nsA)), Value(new NodeId(1, 250))]).ConfigureAwait(false);
Assert.That(values[0].WrappedValue, Is.EqualTo(new Variant("A")));
Assert.That(values[1].StatusCode, Is.EqualTo(StatusCodes.BadNodeIdUnknown));
Assert.That(a.ReadCalls, Is.EqualTo(1));
Assert.That(b.ReadCalls, Is.EqualTo(1), "an unrouted node keeps the call to every node manager");
}).ConfigureAwait(false);
}
[Test]
public async Task ReadOfOnlyInvalidItemsCallsNoNodeManagerAsync()
{
await RunAsync(async (sut, a, b, nsA, nsB) =>
{
(ArrayOf<DataValue> values, _) = await sut.ReadAsync(
Context(), 0, TimestampsToReturn.Neither,
[new ReadValueId { NodeId = new NodeId(1, nsA), AttributeId = 999 }]).ConfigureAwait(false);
Assert.That(values[0].StatusCode, Is.EqualTo(StatusCodes.BadAttributeIdInvalid));
Assert.That(a.ReadCalls, Is.Zero);
Assert.That(b.ReadCalls, Is.Zero);
}).ConfigureAwait(false);
}
[Test]
public async Task SharedNamespaceKeepsDispatchOrderBetweenItsNodeManagersAsync()
{
// the first node manager of a namespace in dispatch order processes the node; the
// second one is still called and sees the node as processed, as before.
var second = new FakeNodeManager("A2", kNamespaceA);
await RunAsync(async (sut, a, b, nsA, nsB) =>
{
Assert.That(sut.NamespaceManagers[nsA], Has.Count.EqualTo(2));
(ArrayOf<DataValue> values, _) = await sut.ReadAsync(
Context(), 0, TimestampsToReturn.Neither,
[Value(new NodeId(1, nsA))]).ConfigureAwait(false);
Assert.That(values[0].WrappedValue, Is.EqualTo(new Variant("A")));
Assert.That(a.ReadCalls, Is.EqualTo(1));
Assert.That(second.ReadCalls, Is.EqualTo(1));
Assert.That(b.ReadCalls, Is.Zero);
}, second).ConfigureAwait(false);
}
[Test]
public async Task WriteCallsOnlyTheNodeManagersOfTheRequestedNamespaceAsync()
{
await RunAsync(async (sut, a, b, nsA, nsB) =>
{
(ArrayOf<StatusCode> results, _) = await sut.WriteAsync(
Context(),
[
new WriteValue
{
NodeId = new NodeId(1, nsB),
AttributeId = Attributes.Value,
Value = new DataValue(new Variant(1))
}
]).ConfigureAwait(false);
Assert.That(results[0], Is.EqualTo(StatusCodes.Good));
Assert.That(b.WriteCalls, Is.EqualTo(1));
Assert.That(a.WriteCalls, Is.Zero);
}).ConfigureAwait(false);
}
private delegate Task Scenario(
MasterNodeManager sut,
FakeNodeManager a,
FakeNodeManager b,
ushort nsA,
ushort nsB);
private static async Task RunAsync(Scenario scenario, FakeNodeManager? sameNamespaceAsA = null)
{
var fixture = new ServerFixture<StandardServer>(t => new StandardServer(t));
try
{
StandardServer server = await fixture.StartAsync().ConfigureAwait(false);
var a = new FakeNodeManager("A", kNamespaceA);
var b = new FakeNodeManager("B", kNamespaceB);
IAsyncNodeManager[] managers = sameNamespaceAsA == null
? [a.Object, b.Object]
: [a.Object, sameNamespaceAsA.Object, b.Object];
using var sut = new MasterNodeManager(
server.CurrentInstance, fixture.Config, null, managers);
NamespaceTable namespaces = server.CurrentInstance.NamespaceUris;
a.NamespaceIndex = (ushort)namespaces.GetIndex(kNamespaceA);
b.NamespaceIndex = (ushort)namespaces.GetIndex(kNamespaceB);
if (sameNamespaceAsA != null)
{
sameNamespaceAsA.NamespaceIndex = a.NamespaceIndex;
}
await scenario(sut, a, b, a.NamespaceIndex, b.NamespaceIndex).ConfigureAwait(false);
}
finally
{
await fixture.StopAsync().ConfigureAwait(false);
}
}
private static OperationContext Context()
{
return new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
}
private static ReadValueId Value(NodeId nodeId)
{
return new ReadValueId { NodeId = nodeId, AttributeId = Attributes.Value };
}
/// <summary>
/// A node manager that owns every node of its namespace and counts its calls.
/// </summary>
private sealed class FakeNodeManager
{
public FakeNodeManager(string tag, string namespaceUri)
{
var mock = new Mock<IAsyncNodeManager>();
mock.SetupGet(m => m.NamespaceUris).Returns([namespaceUri]);
mock.Setup(m => m.ReadAsync(
It.IsAny<OperationContext>(),
It.IsAny<double>(),
It.IsAny<ArrayOf<ReadValueId>>(),
It.IsAny<IList<DataValue>>(),
It.IsAny<IList<ServiceResult>>(),
It.IsAny<CancellationToken>()))
.Returns((OperationContext context, double maxAge, ArrayOf<ReadValueId> nodesToRead,
IList<DataValue> values, IList<ServiceResult> errors, CancellationToken ct) =>
{
Interlocked.Increment(ref m_readCalls);
for (int ii = 0; ii < nodesToRead.Count; ii++)
{
if (!nodesToRead[ii].Processed && nodesToRead[ii].NodeId.NamespaceIndex == NamespaceIndex)
{
nodesToRead[ii].Processed = true;
values[ii] = new DataValue(new Variant(tag));
errors[ii] = ServiceResult.Good;
}
}
return default;
});
mock.Setup(m => m.WriteAsync(
It.IsAny<OperationContext>(),
It.IsAny<ArrayOf<WriteValue>>(),
It.IsAny<IList<ServiceResult>>(),
It.IsAny<CancellationToken>()))
.Returns((OperationContext context, ArrayOf<WriteValue> nodesToWrite,
IList<ServiceResult> errors, CancellationToken ct) =>
{
Interlocked.Increment(ref m_writeCalls);
for (int ii = 0; ii < nodesToWrite.Count; ii++)
{
if (!nodesToWrite[ii].Processed && nodesToWrite[ii].NodeId.NamespaceIndex == NamespaceIndex)
{
nodesToWrite[ii].Processed = true;
errors[ii] = ServiceResult.Good;
}
}
return default;
});
Object = mock.Object;
}
public IAsyncNodeManager Object { get; }
public int ReadCalls => Volatile.Read(ref m_readCalls);
public int WriteCalls => Volatile.Read(ref m_writeCalls);
public ushort NamespaceIndex { get; set; }
private int m_readCalls;
private int m_writeCalls;
}
}
}