Files
UA-.NETStandard/tests/Opc.Ua.Server.Tests/AsyncCustomNodeManagerTests.SystemContext.cs
romanett 6cfb8cf196 Server: remove per-request context copies and lock hot spots from the request path (−16 % allocation per Read) (#4767)
## Description

Five small changes to the per-request server path, found in the CPU and
allocation profiles of the o6 read benchmark (`AsyncCustomNodeManager`,
Server GC, 10 clients × 100 Reads in flight) on master 3a31473cf. Each
change is its own commit and can be reviewed and reverted on its own.
The channel gate (about 40 % of waits) is not touched; that is A1.

### Results

Paired A/B runs against master (0e40e1229), alternating builds per
round, 5 rounds, medians (min..max). The machine is shared with other
sessions: allocated bytes per request are deterministic; CPU per request
is best read from the runs pinned to 4 cores.

| Scenario | master | this PR | Change |
|---|---|---|---|
| o6read 10 × 100, server pinned to 4 cores: CPU / request | 39.1 µs
(35.9..39.8) | 36.9 µs (35.5..37.6) | **−5.6 %** |
| same: allocated / request | 6,198 B | 5,224 B | **−15.7 %** |
| same: req/s | 98.7k (95.4..108.5k) | 104.6k (97.6..105.9k) | +5.9 % |
| o6read 10 × 100, 24 cores: req/s | 183.7k (173.3..198.0k) | 189.8k
(178.9..197.5k) | +3.4 % |
| same: allocated / request | 6,500 B | 5,578 B | −14.2 % |
| o6read 10 × 100 Basic256Sha256, 24 cores: req/s | 159.8k
(149.4..178.7k) | 164.2k (156.7..165.3k) | +2.7 % |
| same: allocated / request | 6,590 B | 5,694 B | −13.6 % |
| `read --nodes 1`, 20 sessions: req/s | 155.3k (141.6..157.6k) | 163.8k
(163.4..166.5k) | +5.5 % |
| same: allocated / request | 5,548 B | 4,689 B | −15.5 % |
| `read --nodes 100`: allocated / request | 74.4 KB | 70.1 KB | −5.8 % |
| `browse`: allocated / request | 39.8 KB | 35.0 KB | −12.2 % |
| `browse`: lock contentions / 1000 requests | 474 | 367 | −23 % |

At 24 cores CPU per request stays within noise, because the time these
changes free now waits at the channel gate (see below).

Per item, pinned to 4 cores (5 rounds; CPU deltas of one item are within
run-to-run noise, allocation deltas are not):

| Item | CPU / request | Allocated / request |
|---|---|---|
| Q1 system context | +1.8 % | −1.7 % |
| Q2 request table | −4.3 % | −1.4 % |
| Q3 request timeouts | −2.8 % | −2.2 % |
| Q4 namespace metadata | −3.5 % | ±0 |
| Q5 read-path allocations | −2.4 % | −10.9 % |

Sampled thread time at 24 cores (cpu trace, first `Opc.Ua` frame above
the wait or copy):

| Frame | master | this PR |
|---|---|---|
| `ServerSystemContext.Copy` (incl. `MemberwiseClone`) | 12.4 % | 0.1 %
|
| `RequestManager` (`ConcurrentDictionary` inside
`EnterValidationScope`, scope `Dispose`, `RequestReceived`,
`CompleteRequest`) | 7.1 % | 0 |
| `RequestLifetime` (CTS registration spin,
ctor/`RegisterExternal`/`DisposeResources`) | 4.8 % | 0.2 % |
| `NamespaceMetadataRegistry.GetAsync` lock | 4.0 % | 0 |
| `RuntimeHelpers.InitializeArray` (`ReadValidationAttributes`) | 2.2 %
| 0 |
| Channel gate (`ChannelGate+AwaitEntryAsync`, inclusive) | 31 % | 46 %
|

## Changes

**Q1: one system context copy per request and node manager.**
`AsyncCustomNodeManager`'s service methods (Read, Write, Browse,
TranslateBrowsePaths, Call, HistoryRead/Update, monitored item and event
services) use the per-request copy from
`OperationContext.GetSystemContext` instead of copying the node
manager's context again. The per-request cache keeps the first node
manager's copy without an extra object and up to eight more, so a
request that touches several node managers no longer evicts the other
one's copy. `ServerSystemContext` copies itself with a copy constructor
instead of `MemberwiseClone` (which allocates through a runtime call)
unless it is a derived type; `SessionSystemContext` gains a protected
copy constructor. Copies per Read went from about three to one.

**Q2: lock-free request registration in `RequestManager`.** #4742 took
the server-wide lock off the request path, but the two
`ConcurrentDictionary`s for executing requests and open validation
scopes stayed the main waiting point after the channel gate (four writes
per request from every worker). `RequestTable<T>` places an entry with a
compare-and-swap into a free slot near a scrambled hash of its key, and
the entry remembers its slot so completion is one more compare-and-swap;
only when every probed slot is taken does an entry go to an overflow set
under a lock. Drains, Cancel, Session close, the timeout sweep and
`Dispose` read the slots one after the other, which keeps the existing
ordering arguments (registration fence vs. Session/Cancel marks, drain
count published before the snapshot). Sharding the dictionaries over
per-processor locks was tried first and only moved about 5 % of waits
into the shard locks.

**Q3: no timer or shared-token registrations per request.**
`RequestLifetime`'s timeout constructor, used by the endpoint for every
request, no longer creates a `Timer` and no longer registers with the
server's request-queue token and the transport token, which every worker
registered with and so contended on. The deadline and the two tokens are
watched by `RequestDeadlineMonitor`: one sweep every 20 ms over
intrusive lists sharded by processor, running only while requests are
monitored. The `params` constructor keeps its synchronous registrations.

**Q4: namespace metadata lookups without a lock.**
`NamespaceMetadataRegistry` reads its caches (found and missing, by URI
and now also by index) without its lock; only filling and invalidating
take it. The generation moves before the caches are cleared, so a lookup
racing an invalidation never reports a stale miss.

**Q5: fewer per-call allocations on the read path.**
`ReadValidationAttributes` fills its attribute-id array element by
element instead of with an array initializer
(`RuntimeHelpers.InitializeArray` per call); single-node Read and Write
requests skip the per-request attribute cache and validate with
`permissionsOnly` (the same three attributes, read once); `ReadAsync`
allocates the list of nodes to validate only when needed. (Review: a
shared static attribute array, a per-thread `Variant[]` scratch array
and a per-thread Read node cache were dropped again because they crossed
virtual calls; the measurements below were taken with them, so Q5's
allocation saving is a little smaller than shown.)

## Behaviour notes

- **Request timeouts (Q3):** a `TimeoutHint` now fires up to 20 ms late
(the sweep period), and cancelling the server's or the transport's token
reaches the request's `CancellationToken` up to 20 ms later instead of
synchronously. A token that is already cancelled cancels the request at
once, and `TryCancel` checks the tokens first, so a token cancelled
before an explicit Cancel still owns the terminal state. The
exactly-once terminal state of #4744 is unchanged: the sweep claims it
through the same compare-and-swap.
- **System contexts (Q1):** a custom node manager that changed fields of
the context it got in a service call now changes the copy shared by that
request's other calls into the same node manager; it never reaches other
requests or the node manager's own `SystemContext`.
- **Request ids (Q2):** `RequestManager.RequestReceived` no longer
throws for a different `OperationContext` with a request id that is
still registered. Requests are tracked by their context; ids come from a
server-wide counter.

## Testing

- New: `ServerSystemContextCopyTests` (every field copied, derived types
kept), `AsyncCustomNodeManagerTests` (one copy per node manager and
request, two managers in one request, not shared across requests;
single- vs multi-node Read verdicts identical incl. a node read twice),
`RequestTableTests` (idempotent add, exactly-once remove, overflow,
concurrent churn while enumerating; drain racing completion on many
threads; Session close and Cancel by handle exactly once while others
register), `RequestDeadlineMonitorTests` (timeout precision with a fake
clock, token propagation, ownership vs. explicit Cancel, no leaked
entries, sweep/cancel/complete race), `NamespaceMetadataRegistryTests`
(cached miss by index invalidated, lookups racing `Invalidate`,
permission changes visible through the cached node).
- Run locally (net10.0): Opc.Ua.Types.Tests, Opc.Ua.Core.Tests (Stack),
Opc.Ua.Server.Tests, Opc.Ua.Sessions.Tests, Opc.Ua.Subscriptions.Tests,
Opc.Ua.Redundancy.Server.Tests, Opc.Ua.Stress.Tests Contract +
Integration. net48 builds.
- Measured with `perf/ServerLoadHarness` scenarios `o6read` (o6 harness,
not part of this PR), `read`, `browse`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:25:31 +02:00

184 lines
8.4 KiB
C#

/* ========================================================================
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
#pragma warning disable CA2000
using System.Collections.Generic;
using System.Threading.Tasks;
using NUnit.Framework;
namespace Opc.Ua.Server.Tests
{
public partial class AsyncCustomNodeManagerTests
{
/// <summary>
/// Verifies that the service calls of one request share one system context per node
/// manager: two node managers in one request each get a context made from their own
/// template, the copies are reused for the rest of the request, and a different
/// request gets new copies.
/// </summary>
[Test]
public async Task ReadSharesOneSystemContextPerNodeManagerAndRequestAsync()
{
if (m_managerType == AsyncCustomNodeManagerType.CustomNodeManager2ViaAdapter)
{
Assert.Ignore("CustomNodeManager2 copies its own system context per call.");
}
using ITestNodeManager first = CreateManager();
using ITestNodeManager second = CreateManager();
var firstContexts = new List<ISystemContext>();
var secondContexts = new List<ISystemContext>();
NodeId firstId = await AddCapturingVariableAsync(first, "First", firstContexts).ConfigureAwait(false);
NodeId secondId = await AddCapturingVariableAsync(second, "Second", secondContexts).ConfigureAwait(false);
var request = new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
// Interleave the two node managers within one request twice.
for (int round = 0; round < 2; round++)
{
await ReadValueAsync(first, request, firstId).ConfigureAwait(false);
await ReadValueAsync(second, request, secondId).ConfigureAwait(false);
}
Assert.That(firstContexts, Has.Count.EqualTo(2));
Assert.That(secondContexts, Has.Count.EqualTo(2));
Assert.That(firstContexts[1], Is.SameAs(firstContexts[0]), "first manager's copy evicted");
Assert.That(secondContexts[1], Is.SameAs(secondContexts[0]), "second manager's copy evicted");
Assert.That(firstContexts[0], Is.Not.SameAs(secondContexts[0]));
Assert.That(firstContexts[0], Is.Not.SameAs(first.SystemContext), "the template must not be handed out");
var firstContext = (ServerSystemContext)firstContexts[0];
var secondContext = (ServerSystemContext)secondContexts[0];
Assert.That(firstContext.NodeIdFactory, Is.SameAs(first.SystemContext.NodeIdFactory));
Assert.That(secondContext.NodeIdFactory, Is.SameAs(second.SystemContext.NodeIdFactory));
Assert.That(firstContext.OperationContext, Is.SameAs(request));
Assert.That(secondContext.OperationContext, Is.SameAs(request));
// A different request (or session) never sees the copy of another one.
var nextRequest = new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
await ReadValueAsync(first, nextRequest, firstId).ConfigureAwait(false);
Assert.That(firstContexts, Has.Count.EqualTo(3));
Assert.That(firstContexts[2], Is.Not.SameAs(firstContexts[0]));
Assert.That(((ServerSystemContext)firstContexts[2]).OperationContext, Is.SameAs(nextRequest));
Assert.That(first.SystemContext.OperationContext, Is.Null, "the template must stay unbound");
}
/// <summary>
/// Verifies that the per-request cache keeps a copy for every node manager a request
/// touches up to its bound, and still returns a correct copy past it.
/// </summary>
[Test]
public void OperationSystemContextCacheKeepsOneCopyPerTemplate()
{
var request = new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
var templates = new List<ServerSystemContext>();
for (int ii = 0; ii < 12; ii++)
{
templates.Add(new ServerSystemContext(m_mockServer.Object) { SystemHandle = ii });
}
// Eight node managers in one request are all kept.
var copies = new List<ServerSystemContext>();
for (int ii = 0; ii < 8; ii++)
{
copies.Add(request.GetSystemContext(templates[ii]));
}
for (int ii = 0; ii < 8; ii++)
{
Assert.That(request.GetSystemContext(templates[ii]), Is.SameAs(copies[ii]));
}
// Past the bound a template still gets a correct copy bound to the request.
for (int ii = 0; ii < templates.Count; ii++)
{
ServerSystemContext again = request.GetSystemContext(templates[ii]);
Assert.That(again.SystemHandle, Is.EqualTo(ii));
Assert.That(again.OperationContext, Is.SameAs(request));
Assert.That(again, Is.Not.SameAs(templates[ii]));
}
}
private async Task<NodeId> AddCapturingVariableAsync(
ITestNodeManager manager,
string name,
List<ISystemContext> contexts)
{
ServerSystemContext context = manager.SystemContext;
ushort nsIdx = manager.NamespaceIndexes[0];
var variable = new BaseDataVariableState(null);
variable.CreateAsPredefinedNode(context);
variable.NodeId = new NodeId(name, nsIdx);
variable.BrowseName = new QualifiedName(name, nsIdx);
variable.DataType = DataTypeIds.Int32;
variable.ValueRank = ValueRanks.Scalar;
variable.AccessLevel = AccessLevels.CurrentRead;
variable.UserAccessLevel = AccessLevels.CurrentRead;
variable.Value = 1;
variable.OnReadValue = (
ISystemContext readContext,
NodeState node,
NumericRange indexRange,
QualifiedName dataEncoding,
ref Variant value,
ref StatusCode statusCode,
ref DateTimeUtc timestamp) =>
{
lock (contexts)
{
contexts.Add(readContext);
}
return ServiceResult.Good;
};
await manager.AddNodeAsync(context, default, variable).ConfigureAwait(false);
return variable.NodeId;
}
private static async Task ReadValueAsync(
ITestNodeManager manager,
OperationContext request,
NodeId nodeId)
{
var nodesToRead = new List<ReadValueId>
{
new() { NodeId = nodeId, AttributeId = Attributes.Value }
};
var values = new List<DataValue> { default };
var errors = new List<ServiceResult> { null! };
await manager.ReadAsync(request, 0, nodesToRead, values, errors).ConfigureAwait(false);
Assert.That(ServiceResult.IsGood(errors[0]), Is.True, errors[0]?.ToString());
}
}
}