mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-18 22:55:08 +08:00
6cfb8cf196
## Description Five small changes to the per-request server path, found in the CPU and allocation profiles of the o6 read benchmark (`AsyncCustomNodeManager`, Server GC, 10 clients × 100 Reads in flight) on master3a31473cf. Each change is its own commit and can be reviewed and reverted on its own. The channel gate (about 40 % of waits) is not touched; that is A1. ### Results Paired A/B runs against master (0e40e1229), alternating builds per round, 5 rounds, medians (min..max). The machine is shared with other sessions: allocated bytes per request are deterministic; CPU per request is best read from the runs pinned to 4 cores. | Scenario | master | this PR | Change | |---|---|---|---| | o6read 10 × 100, server pinned to 4 cores: CPU / request | 39.1 µs (35.9..39.8) | 36.9 µs (35.5..37.6) | **−5.6 %** | | same: allocated / request | 6,198 B | 5,224 B | **−15.7 %** | | same: req/s | 98.7k (95.4..108.5k) | 104.6k (97.6..105.9k) | +5.9 % | | o6read 10 × 100, 24 cores: req/s | 183.7k (173.3..198.0k) | 189.8k (178.9..197.5k) | +3.4 % | | same: allocated / request | 6,500 B | 5,578 B | −14.2 % | | o6read 10 × 100 Basic256Sha256, 24 cores: req/s | 159.8k (149.4..178.7k) | 164.2k (156.7..165.3k) | +2.7 % | | same: allocated / request | 6,590 B | 5,694 B | −13.6 % | | `read --nodes 1`, 20 sessions: req/s | 155.3k (141.6..157.6k) | 163.8k (163.4..166.5k) | +5.5 % | | same: allocated / request | 5,548 B | 4,689 B | −15.5 % | | `read --nodes 100`: allocated / request | 74.4 KB | 70.1 KB | −5.8 % | | `browse`: allocated / request | 39.8 KB | 35.0 KB | −12.2 % | | `browse`: lock contentions / 1000 requests | 474 | 367 | −23 % | At 24 cores CPU per request stays within noise, because the time these changes free now waits at the channel gate (see below). Per item, pinned to 4 cores (5 rounds; CPU deltas of one item are within run-to-run noise, allocation deltas are not): | Item | CPU / request | Allocated / request | |---|---|---| | Q1 system context | +1.8 % | −1.7 % | | Q2 request table | −4.3 % | −1.4 % | | Q3 request timeouts | −2.8 % | −2.2 % | | Q4 namespace metadata | −3.5 % | ±0 | | Q5 read-path allocations | −2.4 % | −10.9 % | Sampled thread time at 24 cores (cpu trace, first `Opc.Ua` frame above the wait or copy): | Frame | master | this PR | |---|---|---| | `ServerSystemContext.Copy` (incl. `MemberwiseClone`) | 12.4 % | 0.1 % | | `RequestManager` (`ConcurrentDictionary` inside `EnterValidationScope`, scope `Dispose`, `RequestReceived`, `CompleteRequest`) | 7.1 % | 0 | | `RequestLifetime` (CTS registration spin, ctor/`RegisterExternal`/`DisposeResources`) | 4.8 % | 0.2 % | | `NamespaceMetadataRegistry.GetAsync` lock | 4.0 % | 0 | | `RuntimeHelpers.InitializeArray` (`ReadValidationAttributes`) | 2.2 % | 0 | | Channel gate (`ChannelGate+AwaitEntryAsync`, inclusive) | 31 % | 46 % | ## Changes **Q1: one system context copy per request and node manager.** `AsyncCustomNodeManager`'s service methods (Read, Write, Browse, TranslateBrowsePaths, Call, HistoryRead/Update, monitored item and event services) use the per-request copy from `OperationContext.GetSystemContext` instead of copying the node manager's context again. The per-request cache keeps the first node manager's copy without an extra object and up to eight more, so a request that touches several node managers no longer evicts the other one's copy. `ServerSystemContext` copies itself with a copy constructor instead of `MemberwiseClone` (which allocates through a runtime call) unless it is a derived type; `SessionSystemContext` gains a protected copy constructor. Copies per Read went from about three to one. **Q2: lock-free request registration in `RequestManager`.** #4742 took the server-wide lock off the request path, but the two `ConcurrentDictionary`s for executing requests and open validation scopes stayed the main waiting point after the channel gate (four writes per request from every worker). `RequestTable<T>` places an entry with a compare-and-swap into a free slot near a scrambled hash of its key, and the entry remembers its slot so completion is one more compare-and-swap; only when every probed slot is taken does an entry go to an overflow set under a lock. Drains, Cancel, Session close, the timeout sweep and `Dispose` read the slots one after the other, which keeps the existing ordering arguments (registration fence vs. Session/Cancel marks, drain count published before the snapshot). Sharding the dictionaries over per-processor locks was tried first and only moved about 5 % of waits into the shard locks. **Q3: no timer or shared-token registrations per request.** `RequestLifetime`'s timeout constructor, used by the endpoint for every request, no longer creates a `Timer` and no longer registers with the server's request-queue token and the transport token, which every worker registered with and so contended on. The deadline and the two tokens are watched by `RequestDeadlineMonitor`: one sweep every 20 ms over intrusive lists sharded by processor, running only while requests are monitored. The `params` constructor keeps its synchronous registrations. **Q4: namespace metadata lookups without a lock.** `NamespaceMetadataRegistry` reads its caches (found and missing, by URI and now also by index) without its lock; only filling and invalidating take it. The generation moves before the caches are cleared, so a lookup racing an invalidation never reports a stale miss. **Q5: fewer per-call allocations on the read path.** `ReadValidationAttributes` fills its attribute-id array element by element instead of with an array initializer (`RuntimeHelpers.InitializeArray` per call); single-node Read and Write requests skip the per-request attribute cache and validate with `permissionsOnly` (the same three attributes, read once); `ReadAsync` allocates the list of nodes to validate only when needed. (Review: a shared static attribute array, a per-thread `Variant[]` scratch array and a per-thread Read node cache were dropped again because they crossed virtual calls; the measurements below were taken with them, so Q5's allocation saving is a little smaller than shown.) ## Behaviour notes - **Request timeouts (Q3):** a `TimeoutHint` now fires up to 20 ms late (the sweep period), and cancelling the server's or the transport's token reaches the request's `CancellationToken` up to 20 ms later instead of synchronously. A token that is already cancelled cancels the request at once, and `TryCancel` checks the tokens first, so a token cancelled before an explicit Cancel still owns the terminal state. The exactly-once terminal state of #4744 is unchanged: the sweep claims it through the same compare-and-swap. - **System contexts (Q1):** a custom node manager that changed fields of the context it got in a service call now changes the copy shared by that request's other calls into the same node manager; it never reaches other requests or the node manager's own `SystemContext`. - **Request ids (Q2):** `RequestManager.RequestReceived` no longer throws for a different `OperationContext` with a request id that is still registered. Requests are tracked by their context; ids come from a server-wide counter. ## Testing - New: `ServerSystemContextCopyTests` (every field copied, derived types kept), `AsyncCustomNodeManagerTests` (one copy per node manager and request, two managers in one request, not shared across requests; single- vs multi-node Read verdicts identical incl. a node read twice), `RequestTableTests` (idempotent add, exactly-once remove, overflow, concurrent churn while enumerating; drain racing completion on many threads; Session close and Cancel by handle exactly once while others register), `RequestDeadlineMonitorTests` (timeout precision with a fake clock, token propagation, ownership vs. explicit Cancel, no leaked entries, sweep/cancel/complete race), `NamespaceMetadataRegistryTests` (cached miss by index invalidated, lookups racing `Invalidate`, permission changes visible through the cached node). - Run locally (net10.0): Opc.Ua.Types.Tests, Opc.Ua.Core.Tests (Stack), Opc.Ua.Server.Tests, Opc.Ua.Sessions.Tests, Opc.Ua.Subscriptions.Tests, Opc.Ua.Redundancy.Server.Tests, Opc.Ua.Stress.Tests Contract + Integration. net48 builds. - Measured with `perf/ServerLoadHarness` scenarios `o6read` (o6 harness, not part of this PR), `read`, `browse`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
184 lines
8.4 KiB
C#
184 lines
8.4 KiB
C#
/* ========================================================================
|
|
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
|
|
*
|
|
* OPC Foundation MIT License 1.00
|
|
*
|
|
* Permission is hereby granted, free of charge, to any person
|
|
* obtaining a copy of this software and associated documentation
|
|
* files (the "Software"), to deal in the Software without
|
|
* restriction, including without limitation the rights to use,
|
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
* copies of the Software, and to permit persons to whom the
|
|
* Software is furnished to do so, subject to the following
|
|
* conditions:
|
|
*
|
|
* The above copyright notice and this permission notice shall be
|
|
* included in all copies or substantial portions of the Software.
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
|
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
|
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
|
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
|
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
|
* OTHER DEALINGS IN THE SOFTWARE.
|
|
*
|
|
* The complete license agreement can be found here:
|
|
* http://opcfoundation.org/License/MIT/1.00/
|
|
* ======================================================================*/
|
|
|
|
#pragma warning disable CA2000
|
|
using System.Collections.Generic;
|
|
using System.Threading.Tasks;
|
|
using NUnit.Framework;
|
|
|
|
namespace Opc.Ua.Server.Tests
|
|
{
|
|
public partial class AsyncCustomNodeManagerTests
|
|
{
|
|
/// <summary>
|
|
/// Verifies that the service calls of one request share one system context per node
|
|
/// manager: two node managers in one request each get a context made from their own
|
|
/// template, the copies are reused for the rest of the request, and a different
|
|
/// request gets new copies.
|
|
/// </summary>
|
|
[Test]
|
|
public async Task ReadSharesOneSystemContextPerNodeManagerAndRequestAsync()
|
|
{
|
|
if (m_managerType == AsyncCustomNodeManagerType.CustomNodeManager2ViaAdapter)
|
|
{
|
|
Assert.Ignore("CustomNodeManager2 copies its own system context per call.");
|
|
}
|
|
|
|
using ITestNodeManager first = CreateManager();
|
|
using ITestNodeManager second = CreateManager();
|
|
var firstContexts = new List<ISystemContext>();
|
|
var secondContexts = new List<ISystemContext>();
|
|
NodeId firstId = await AddCapturingVariableAsync(first, "First", firstContexts).ConfigureAwait(false);
|
|
NodeId secondId = await AddCapturingVariableAsync(second, "Second", secondContexts).ConfigureAwait(false);
|
|
|
|
var request = new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
|
|
|
|
// Interleave the two node managers within one request twice.
|
|
for (int round = 0; round < 2; round++)
|
|
{
|
|
await ReadValueAsync(first, request, firstId).ConfigureAwait(false);
|
|
await ReadValueAsync(second, request, secondId).ConfigureAwait(false);
|
|
}
|
|
|
|
Assert.That(firstContexts, Has.Count.EqualTo(2));
|
|
Assert.That(secondContexts, Has.Count.EqualTo(2));
|
|
Assert.That(firstContexts[1], Is.SameAs(firstContexts[0]), "first manager's copy evicted");
|
|
Assert.That(secondContexts[1], Is.SameAs(secondContexts[0]), "second manager's copy evicted");
|
|
Assert.That(firstContexts[0], Is.Not.SameAs(secondContexts[0]));
|
|
Assert.That(firstContexts[0], Is.Not.SameAs(first.SystemContext), "the template must not be handed out");
|
|
|
|
var firstContext = (ServerSystemContext)firstContexts[0];
|
|
var secondContext = (ServerSystemContext)secondContexts[0];
|
|
Assert.That(firstContext.NodeIdFactory, Is.SameAs(first.SystemContext.NodeIdFactory));
|
|
Assert.That(secondContext.NodeIdFactory, Is.SameAs(second.SystemContext.NodeIdFactory));
|
|
Assert.That(firstContext.OperationContext, Is.SameAs(request));
|
|
Assert.That(secondContext.OperationContext, Is.SameAs(request));
|
|
|
|
// A different request (or session) never sees the copy of another one.
|
|
var nextRequest = new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
|
|
await ReadValueAsync(first, nextRequest, firstId).ConfigureAwait(false);
|
|
|
|
Assert.That(firstContexts, Has.Count.EqualTo(3));
|
|
Assert.That(firstContexts[2], Is.Not.SameAs(firstContexts[0]));
|
|
Assert.That(((ServerSystemContext)firstContexts[2]).OperationContext, Is.SameAs(nextRequest));
|
|
Assert.That(first.SystemContext.OperationContext, Is.Null, "the template must stay unbound");
|
|
}
|
|
|
|
/// <summary>
|
|
/// Verifies that the per-request cache keeps a copy for every node manager a request
|
|
/// touches up to its bound, and still returns a correct copy past it.
|
|
/// </summary>
|
|
[Test]
|
|
public void OperationSystemContextCacheKeepsOneCopyPerTemplate()
|
|
{
|
|
var request = new OperationContext(new RequestHeader(), null, RequestType.Read, RequestLifetime.None);
|
|
var templates = new List<ServerSystemContext>();
|
|
for (int ii = 0; ii < 12; ii++)
|
|
{
|
|
templates.Add(new ServerSystemContext(m_mockServer.Object) { SystemHandle = ii });
|
|
}
|
|
|
|
// Eight node managers in one request are all kept.
|
|
var copies = new List<ServerSystemContext>();
|
|
for (int ii = 0; ii < 8; ii++)
|
|
{
|
|
copies.Add(request.GetSystemContext(templates[ii]));
|
|
}
|
|
|
|
for (int ii = 0; ii < 8; ii++)
|
|
{
|
|
Assert.That(request.GetSystemContext(templates[ii]), Is.SameAs(copies[ii]));
|
|
}
|
|
|
|
// Past the bound a template still gets a correct copy bound to the request.
|
|
for (int ii = 0; ii < templates.Count; ii++)
|
|
{
|
|
ServerSystemContext again = request.GetSystemContext(templates[ii]);
|
|
Assert.That(again.SystemHandle, Is.EqualTo(ii));
|
|
Assert.That(again.OperationContext, Is.SameAs(request));
|
|
Assert.That(again, Is.Not.SameAs(templates[ii]));
|
|
}
|
|
}
|
|
|
|
private async Task<NodeId> AddCapturingVariableAsync(
|
|
ITestNodeManager manager,
|
|
string name,
|
|
List<ISystemContext> contexts)
|
|
{
|
|
ServerSystemContext context = manager.SystemContext;
|
|
ushort nsIdx = manager.NamespaceIndexes[0];
|
|
var variable = new BaseDataVariableState(null);
|
|
variable.CreateAsPredefinedNode(context);
|
|
variable.NodeId = new NodeId(name, nsIdx);
|
|
variable.BrowseName = new QualifiedName(name, nsIdx);
|
|
variable.DataType = DataTypeIds.Int32;
|
|
variable.ValueRank = ValueRanks.Scalar;
|
|
variable.AccessLevel = AccessLevels.CurrentRead;
|
|
variable.UserAccessLevel = AccessLevels.CurrentRead;
|
|
variable.Value = 1;
|
|
variable.OnReadValue = (
|
|
ISystemContext readContext,
|
|
NodeState node,
|
|
NumericRange indexRange,
|
|
QualifiedName dataEncoding,
|
|
ref Variant value,
|
|
ref StatusCode statusCode,
|
|
ref DateTimeUtc timestamp) =>
|
|
{
|
|
lock (contexts)
|
|
{
|
|
contexts.Add(readContext);
|
|
}
|
|
|
|
return ServiceResult.Good;
|
|
};
|
|
|
|
await manager.AddNodeAsync(context, default, variable).ConfigureAwait(false);
|
|
return variable.NodeId;
|
|
}
|
|
|
|
private static async Task ReadValueAsync(
|
|
ITestNodeManager manager,
|
|
OperationContext request,
|
|
NodeId nodeId)
|
|
{
|
|
var nodesToRead = new List<ReadValueId>
|
|
{
|
|
new() { NodeId = nodeId, AttributeId = Attributes.Value }
|
|
};
|
|
var values = new List<DataValue> { default };
|
|
var errors = new List<ServiceResult> { null! };
|
|
|
|
await manager.ReadAsync(request, 0, nodesToRead, values, errors).ConfigureAwait(false);
|
|
|
|
Assert.That(ServiceResult.IsGood(errors[0]), Is.True, errors[0]?.ToString());
|
|
}
|
|
}
|
|
}
|