mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-17 05:11:13 +08:00
6cfb8cf196
## Description Five small changes to the per-request server path, found in the CPU and allocation profiles of the o6 read benchmark (`AsyncCustomNodeManager`, Server GC, 10 clients × 100 Reads in flight) on master3a31473cf. Each change is its own commit and can be reviewed and reverted on its own. The channel gate (about 40 % of waits) is not touched; that is A1. ### Results Paired A/B runs against master (0e40e1229), alternating builds per round, 5 rounds, medians (min..max). The machine is shared with other sessions: allocated bytes per request are deterministic; CPU per request is best read from the runs pinned to 4 cores. | Scenario | master | this PR | Change | |---|---|---|---| | o6read 10 × 100, server pinned to 4 cores: CPU / request | 39.1 µs (35.9..39.8) | 36.9 µs (35.5..37.6) | **−5.6 %** | | same: allocated / request | 6,198 B | 5,224 B | **−15.7 %** | | same: req/s | 98.7k (95.4..108.5k) | 104.6k (97.6..105.9k) | +5.9 % | | o6read 10 × 100, 24 cores: req/s | 183.7k (173.3..198.0k) | 189.8k (178.9..197.5k) | +3.4 % | | same: allocated / request | 6,500 B | 5,578 B | −14.2 % | | o6read 10 × 100 Basic256Sha256, 24 cores: req/s | 159.8k (149.4..178.7k) | 164.2k (156.7..165.3k) | +2.7 % | | same: allocated / request | 6,590 B | 5,694 B | −13.6 % | | `read --nodes 1`, 20 sessions: req/s | 155.3k (141.6..157.6k) | 163.8k (163.4..166.5k) | +5.5 % | | same: allocated / request | 5,548 B | 4,689 B | −15.5 % | | `read --nodes 100`: allocated / request | 74.4 KB | 70.1 KB | −5.8 % | | `browse`: allocated / request | 39.8 KB | 35.0 KB | −12.2 % | | `browse`: lock contentions / 1000 requests | 474 | 367 | −23 % | At 24 cores CPU per request stays within noise, because the time these changes free now waits at the channel gate (see below). Per item, pinned to 4 cores (5 rounds; CPU deltas of one item are within run-to-run noise, allocation deltas are not): | Item | CPU / request | Allocated / request | |---|---|---| | Q1 system context | +1.8 % | −1.7 % | | Q2 request table | −4.3 % | −1.4 % | | Q3 request timeouts | −2.8 % | −2.2 % | | Q4 namespace metadata | −3.5 % | ±0 | | Q5 read-path allocations | −2.4 % | −10.9 % | Sampled thread time at 24 cores (cpu trace, first `Opc.Ua` frame above the wait or copy): | Frame | master | this PR | |---|---|---| | `ServerSystemContext.Copy` (incl. `MemberwiseClone`) | 12.4 % | 0.1 % | | `RequestManager` (`ConcurrentDictionary` inside `EnterValidationScope`, scope `Dispose`, `RequestReceived`, `CompleteRequest`) | 7.1 % | 0 | | `RequestLifetime` (CTS registration spin, ctor/`RegisterExternal`/`DisposeResources`) | 4.8 % | 0.2 % | | `NamespaceMetadataRegistry.GetAsync` lock | 4.0 % | 0 | | `RuntimeHelpers.InitializeArray` (`ReadValidationAttributes`) | 2.2 % | 0 | | Channel gate (`ChannelGate+AwaitEntryAsync`, inclusive) | 31 % | 46 % | ## Changes **Q1: one system context copy per request and node manager.** `AsyncCustomNodeManager`'s service methods (Read, Write, Browse, TranslateBrowsePaths, Call, HistoryRead/Update, monitored item and event services) use the per-request copy from `OperationContext.GetSystemContext` instead of copying the node manager's context again. The per-request cache keeps the first node manager's copy without an extra object and up to eight more, so a request that touches several node managers no longer evicts the other one's copy. `ServerSystemContext` copies itself with a copy constructor instead of `MemberwiseClone` (which allocates through a runtime call) unless it is a derived type; `SessionSystemContext` gains a protected copy constructor. Copies per Read went from about three to one. **Q2: lock-free request registration in `RequestManager`.** #4742 took the server-wide lock off the request path, but the two `ConcurrentDictionary`s for executing requests and open validation scopes stayed the main waiting point after the channel gate (four writes per request from every worker). `RequestTable<T>` places an entry with a compare-and-swap into a free slot near a scrambled hash of its key, and the entry remembers its slot so completion is one more compare-and-swap; only when every probed slot is taken does an entry go to an overflow set under a lock. Drains, Cancel, Session close, the timeout sweep and `Dispose` read the slots one after the other, which keeps the existing ordering arguments (registration fence vs. Session/Cancel marks, drain count published before the snapshot). Sharding the dictionaries over per-processor locks was tried first and only moved about 5 % of waits into the shard locks. **Q3: no timer or shared-token registrations per request.** `RequestLifetime`'s timeout constructor, used by the endpoint for every request, no longer creates a `Timer` and no longer registers with the server's request-queue token and the transport token, which every worker registered with and so contended on. The deadline and the two tokens are watched by `RequestDeadlineMonitor`: one sweep every 20 ms over intrusive lists sharded by processor, running only while requests are monitored. The `params` constructor keeps its synchronous registrations. **Q4: namespace metadata lookups without a lock.** `NamespaceMetadataRegistry` reads its caches (found and missing, by URI and now also by index) without its lock; only filling and invalidating take it. The generation moves before the caches are cleared, so a lookup racing an invalidation never reports a stale miss. **Q5: fewer per-call allocations on the read path.** `ReadValidationAttributes` fills its attribute-id array element by element instead of with an array initializer (`RuntimeHelpers.InitializeArray` per call); single-node Read and Write requests skip the per-request attribute cache and validate with `permissionsOnly` (the same three attributes, read once); `ReadAsync` allocates the list of nodes to validate only when needed. (Review: a shared static attribute array, a per-thread `Variant[]` scratch array and a per-thread Read node cache were dropped again because they crossed virtual calls; the measurements below were taken with them, so Q5's allocation saving is a little smaller than shown.) ## Behaviour notes - **Request timeouts (Q3):** a `TimeoutHint` now fires up to 20 ms late (the sweep period), and cancelling the server's or the transport's token reaches the request's `CancellationToken` up to 20 ms later instead of synchronously. A token that is already cancelled cancels the request at once, and `TryCancel` checks the tokens first, so a token cancelled before an explicit Cancel still owns the terminal state. The exactly-once terminal state of #4744 is unchanged: the sweep claims it through the same compare-and-swap. - **System contexts (Q1):** a custom node manager that changed fields of the context it got in a service call now changes the copy shared by that request's other calls into the same node manager; it never reaches other requests or the node manager's own `SystemContext`. - **Request ids (Q2):** `RequestManager.RequestReceived` no longer throws for a different `OperationContext` with a request id that is still registered. Requests are tracked by their context; ids come from a server-wide counter. ## Testing - New: `ServerSystemContextCopyTests` (every field copied, derived types kept), `AsyncCustomNodeManagerTests` (one copy per node manager and request, two managers in one request, not shared across requests; single- vs multi-node Read verdicts identical incl. a node read twice), `RequestTableTests` (idempotent add, exactly-once remove, overflow, concurrent churn while enumerating; drain racing completion on many threads; Session close and Cancel by handle exactly once while others register), `RequestDeadlineMonitorTests` (timeout precision with a fake clock, token propagation, ownership vs. explicit Cancel, no leaked entries, sweep/cancel/complete race), `NamespaceMetadataRegistryTests` (cached miss by index invalidated, lookups racing `Invalidate`, permission changes visible through the cached node). - Run locally (net10.0): Opc.Ua.Types.Tests, Opc.Ua.Core.Tests (Stack), Opc.Ua.Server.Tests, Opc.Ua.Sessions.Tests, Opc.Ua.Subscriptions.Tests, Opc.Ua.Redundancy.Server.Tests, Opc.Ua.Stress.Tests Contract + Integration. net48 builds. - Measured with `perf/ServerLoadHarness` scenarios `o6read` (o6 harness, not part of this PR), `read`, `browse`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
148 lines
7.3 KiB
C#
148 lines
7.3 KiB
C#
/* ========================================================================
|
|
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
|
|
*
|
|
* OPC Foundation MIT License 1.00
|
|
*
|
|
* Permission is hereby granted, free of charge, to any person
|
|
* obtaining a copy of this software and associated documentation
|
|
* files (the "Software"), to deal in the Software without
|
|
* restriction, including without limitation the rights to use,
|
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
* copies of the Software, and to permit persons to whom the
|
|
* Software is furnished to do so, subject to the following
|
|
* conditions:
|
|
*
|
|
* The above copyright notice and this permission notice shall be
|
|
* included in all copies or substantial portions of the Software.
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
|
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
|
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
|
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
|
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
|
* OTHER DEALINGS IN THE SOFTWARE.
|
|
*
|
|
* The complete license agreement can be found here:
|
|
* http://opcfoundation.org/License/MIT/1.00/
|
|
* ======================================================================*/
|
|
|
|
#pragma warning disable CA2000
|
|
using System.Threading.Tasks;
|
|
using Moq;
|
|
using NUnit.Framework;
|
|
using Opc.Ua.Tests;
|
|
|
|
namespace Opc.Ua.Server.Tests
|
|
{
|
|
public partial class AsyncCustomNodeManagerTests
|
|
{
|
|
private static readonly NodeId s_readerRoleId = new("ReadPermissionReader", 0);
|
|
|
|
/// <summary>
|
|
/// Verifies that a Read through the service dispatcher gives the same per-node
|
|
/// permission verdicts and values whether a node is read alone (no per-request
|
|
/// attribute cache) or together with other nodes (cached attributes), including a
|
|
/// node read twice in one request.
|
|
/// </summary>
|
|
[Test]
|
|
public async Task ReadPermissionVerdictsAreTheSameForSingleAndMultiNodeRequestsAsync()
|
|
{
|
|
m_mockServer.Setup(server => server.Telemetry).Returns(NUnitTelemetryContext.Create());
|
|
using var resources = new ResourceManager(m_configuration);
|
|
m_mockServer.Setup(server => server.ResourceManager).Returns(resources);
|
|
ITestNodeManager manager = CreateManager();
|
|
var core = new Mock<ICoreNodeManager>();
|
|
core.Setup(value => value.SyncNodeManager).Returns(Mock.Of<INodeManager>());
|
|
var factory = new Mock<IMainNodeManagerFactory>();
|
|
factory.Setup(value => value.CreateConfigurationNodeManager())
|
|
.Returns(Mock.Of<IConfigurationNodeManager>());
|
|
factory.Setup(value => value.CreateCoreNodeManager(It.IsAny<ushort>()))
|
|
.Returns(core.Object);
|
|
m_mockServer.Setup(server => server.MainNodeManagerFactory).Returns(factory.Object);
|
|
await using var dispatcher = new MasterNodeManager(m_mockServer.Object, m_configuration, null, manager);
|
|
m_mockServer.Setup(server => server.NodeManager).Returns(dispatcher);
|
|
|
|
NodeId allowed = await AddPermissionVariableAsync(manager, "ReadAllowed", 7, PermissionType.Browse | PermissionType.Read)
|
|
.ConfigureAwait(false);
|
|
NodeId denied = await AddPermissionVariableAsync(manager, "ReadDenied", 8, PermissionType.Browse)
|
|
.ConfigureAwait(false);
|
|
|
|
var identity = new Mock<IUserIdentity>();
|
|
identity.Setup(i => i.GrantedRoleIds).Returns([s_readerRoleId]);
|
|
var session = new Mock<ISession>();
|
|
session.Setup(s => s.Id).Returns(new NodeId("ReadPermissionSession", 0));
|
|
session.Setup(s => s.EffectiveIdentity).Returns(identity.Object);
|
|
session.Setup(s => s.Identity).Returns(identity.Object);
|
|
session.Setup(s => s.PreferredLocales).Returns([]);
|
|
|
|
DataValue singleAllowed = await ReadOneAsync(dispatcher, session.Object, allowed).ConfigureAwait(false);
|
|
DataValue singleDenied = await ReadOneAsync(dispatcher, session.Object, denied).ConfigureAwait(false);
|
|
|
|
using var context = new OperationContext(session.Object, DiagnosticsMasks.None);
|
|
(ArrayOf<DataValue> many, _) = await dispatcher.ReadAsync(
|
|
context,
|
|
0,
|
|
TimestampsToReturn.Neither,
|
|
[
|
|
new ReadValueId { NodeId = allowed, AttributeId = Attributes.Value },
|
|
new ReadValueId { NodeId = denied, AttributeId = Attributes.Value },
|
|
new ReadValueId { NodeId = allowed, AttributeId = Attributes.Value }
|
|
]).ConfigureAwait(false);
|
|
|
|
Assert.Multiple(() =>
|
|
{
|
|
Assert.That(singleAllowed.StatusCode, Is.EqualTo(StatusCodes.Good));
|
|
Assert.That((int)singleAllowed.WrappedValue, Is.EqualTo(7));
|
|
Assert.That(singleDenied.StatusCode, Is.EqualTo(StatusCodes.BadUserAccessDenied));
|
|
Assert.That(many.Count, Is.EqualTo(3));
|
|
Assert.That(many[0].StatusCode, Is.EqualTo(singleAllowed.StatusCode));
|
|
Assert.That(many[0].WrappedValue, Is.EqualTo(singleAllowed.WrappedValue));
|
|
Assert.That(many[1].StatusCode, Is.EqualTo(singleDenied.StatusCode));
|
|
Assert.That(many[2].StatusCode, Is.EqualTo(singleAllowed.StatusCode));
|
|
Assert.That(many[2].WrappedValue, Is.EqualTo(singleAllowed.WrappedValue));
|
|
});
|
|
}
|
|
|
|
private static async Task<NodeId> AddPermissionVariableAsync(
|
|
ITestNodeManager manager,
|
|
string name,
|
|
int value,
|
|
PermissionType readerPermissions)
|
|
{
|
|
ServerSystemContext context = manager.SystemContext;
|
|
ushort nsIdx = manager.NamespaceIndexes[0];
|
|
var variable = new BaseDataVariableState(null);
|
|
variable.CreateAsPredefinedNode(context);
|
|
variable.NodeId = new NodeId(name, nsIdx);
|
|
variable.BrowseName = new QualifiedName(name, nsIdx);
|
|
variable.DataType = DataTypeIds.Int32;
|
|
variable.ValueRank = ValueRanks.Scalar;
|
|
variable.AccessLevel = AccessLevels.CurrentRead;
|
|
variable.UserAccessLevel = AccessLevels.CurrentRead;
|
|
variable.Value = value;
|
|
variable.RolePermissions =
|
|
[
|
|
new RolePermissionType { RoleId = s_readerRoleId, Permissions = (uint)readerPermissions }
|
|
];
|
|
await manager.AddNodeAsync(context, default, variable).ConfigureAwait(false);
|
|
return variable.NodeId;
|
|
}
|
|
|
|
private static async Task<DataValue> ReadOneAsync(
|
|
MasterNodeManager dispatcher,
|
|
ISession session,
|
|
NodeId nodeId)
|
|
{
|
|
using var context = new OperationContext(session, DiagnosticsMasks.None);
|
|
(ArrayOf<DataValue> values, _) = await dispatcher.ReadAsync(
|
|
context,
|
|
0,
|
|
TimestampsToReturn.Neither,
|
|
[new ReadValueId { NodeId = nodeId, AttributeId = Attributes.Value }]).ConfigureAwait(false);
|
|
Assert.That(values.Count, Is.EqualTo(1));
|
|
return values[0];
|
|
}
|
|
}
|
|
}
|