mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-18 22:55:08 +08:00
8b56937fdf
## What this adds Harness-only changes under `perf/` (not part of `UA.slnx`, `net10.0`). No product code changes. ### External native worker scenarios (`ServerLoadHarness`) Two new scenarios run the native open62541 workers of the [o6-automation/opcua-benchmarks](https://github.com/o6-automation/opcua-benchmarks) project against a server, so the .NET server and other OPC UA stacks are measured with the same client as that project's published comparison: | Scenario | What it measures | |---|---| | `native-read` | Fixed-load scalar Read throughput (`throughput/open62541/client.c --worker async`): `--clients` worker processes, each with `--inflight` single-value Reads of ns=1;i=1001..1100 in flight, starting together. Reports aggregate req/s, and server CPU and allocated bytes per request. | | `native-counters` | Subscription capacity (`subscription/native/client.c`): one subscription with `--nodes` UInt64 counters at `--sampling` 10 ms / `--pub` 1000 ms, queue size 1. Passes when every item gets 99 % of the samples its sampling interval calls for. | The result block has the same lines as the other scenarios, and the CPU time of the native workers goes in `client-per-op:`. The server is either the harness child in **contract mode** (`server --contract 1`) or any external executable (`--server-exe`, `--server-args` with `{port}`, `{items}`, `{sampling}` and `{pki}` placeholders), for example the benchmark's open62541 server or one of its .NET workers. By default the harness child is configured like the benchmark's .NET worker for UA-.NETStandard 2.0.1: | Setting | Default | Override | |---|---|---| | Node managers | `AsyncCustomNodeManager` | `--async-node-manager 0` (`CustomNodeManager2`) | | Counter increment | under a lock owned by the counters | `--counter-lock 0` (interlocked) | | Garbage collector | Server GC | `SERVER_DOTNET_gcServer=0` | | `MaxQueuedRequestCount` | clients × in flight + 3 (minimum 200) | `--queue` | Other options: `--native-client`, `--clients`, `--inflight`, `--iterations`, `--warmup-calls`, `--server-affinity` (processor mask; combine it with `SERVER_DOTNET_PROCESSOR_COUNT`, because the mask is applied after the runtime has sized itself), `--client-affinity`, `--server-priority`, `--thread-states 1` (samples Running/Ready server threads and appends a `sched:` line), `--security encrypt` (Basic256Sha256 SignAndEncrypt with generated DER identities) and `--verbose`. Also, the `MinRequestThreadCount` of the server child is now the smaller of 50 and `--threads`, so `--threads` below 50 takes effect. **Worker protocol.** The workers define a few names that are kept as string literals in named constants: the environment variables `O6_BENCHMARK_SECURITY_POLICY`, `O6_BENCHMARK_CERTIFICATE` and `O6_BENCHMARK_PRIVATE_KEY`, the `O6_BENCHMARK_READY` stdout token, the identity application URI / subject prefixes, and the server URI `urn:o6:benchmark:server`. The workers need that server URI because the contract puts its nodes in the namespace of the server URI and the workers address them as ns=1. Identifiers, file names, scenario and option names are neutral. ### TraceAnalyzer - `cpu` with a focus frame also reports the first `Opc.Ua` frame above it, past framework wrappers such as `Monitor.Enter` or `SemaphoreSlim`. - **GC-poll attribution:** the thread-time sampler suspends running threads at their next GC poll, so many on-CPU samples have `Thread.PollGCWorker` as their leaf. `cpu` used to drop them as waits (in one trace only 3.5k of 20k on-CPU samples were kept); it now attributes them to the caller. - `gc` mode (for `--trace-profile gc`): collections by generation, type and reason; the suspension time and the whole pause, each with count, total and percentiles. - `samples` mode: all samples by type (managed/external) and by leaf and first `Opc.Ua` frame, including the ones `cpu` leaves out. - `suspend <trace> [thresholdMs]` mode: every SuspendEE longer than the threshold, with the events recorded inside it by thread (an `AllocationTick` there means a thread that had not reached a safe point yet); event counts including the heap count DATAS chose and GC handle churn by kind. ## Building the workers The benchmark repository is AGPL-3.0, so nothing of it is vendored here. Clone it and build the workers separately. On Windows they build with `zig cc` and CMake against the open62541 commit the benchmark pins (OpenSSL backend, `UA_ENABLE_DEBUG_SANITIZER=OFF`, Release). The subscription workers also need `poll()` on stdin replaced by `PeekNamedPipe`, plus unbuffered control output. `perf/README.md` describes all of this. ## Commits 1. `perf: drive the server with external native benchmark workers` 2. `perf: configure the contract server like the benchmark's 2.0.1 .NET worker` 3. `perf: TraceAnalyzer gc and samples modes, samples taken at GC polls` 4. `perf: scheduling and GC-suspension diagnostics for the native worker runs` Each commit builds with 0 warnings (`CustomTestTarget=net10.0 dotnet build perf/ServerLoadHarness -c Release`, `dotnet build perf/TraceAnalyzer -c Release`), and `dotnet format whitespace` makes no changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
254 lines
10 KiB
C#
254 lines
10 KiB
C#
/* ========================================================================
|
|
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
|
|
*
|
|
* OPC Foundation MIT License 1.00
|
|
*
|
|
* Permission is hereby granted, free of charge, to any person
|
|
* obtaining a copy of this software and associated documentation
|
|
* files (the "Software"), to deal in the Software without
|
|
* restriction, including without limitation the rights to use,
|
|
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
* copies of the Software, and to permit persons to whom the
|
|
* Software is furnished to do so, subject to the following
|
|
* conditions:
|
|
*
|
|
* The above copyright notice and this permission notice shall be
|
|
* included in all copies or substantial portions of the Software.
|
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
|
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
|
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
|
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
|
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
|
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
|
* OTHER DEALINGS IN THE SOFTWARE.
|
|
*
|
|
* The complete license agreement can be found here:
|
|
* http://opcfoundation.org/License/MIT/1.00/
|
|
* ======================================================================*/
|
|
|
|
|
|
using System;
|
|
using System.Collections.Generic;
|
|
using System.Diagnostics;
|
|
using System.Globalization;
|
|
using System.Threading;
|
|
using System.Threading.Tasks;
|
|
using Opc.Ua.Server;
|
|
|
|
namespace Opc.Ua.Perf.ServerLoadHarness
|
|
{
|
|
/// <summary>
|
|
/// The counters of the subscription capacity workload of the o6-automation opcua-benchmarks
|
|
/// (UInt64 counter per monitored item, advanced by its read callback, so only sampling drives
|
|
/// them), shared by the node manager variants.
|
|
/// </summary>
|
|
internal sealed class CounterValues
|
|
{
|
|
public CounterValues(int count, bool lockedCallback)
|
|
{
|
|
m_lockedCallback = lockedCallback;
|
|
m_values = new ulong[count];
|
|
m_marked = new ulong[count];
|
|
}
|
|
|
|
/// <summary>
|
|
/// Creates the counter variables ns=namespaceIndex;i=1..count below the Objects folder.
|
|
/// </summary>
|
|
public IEnumerable<BaseDataVariableState> CreateNodes(ushort namespaceIndex)
|
|
{
|
|
for (int i = 0; i < m_values.Length; i++)
|
|
{
|
|
int index = i;
|
|
var node = new BaseDataVariableState(null)
|
|
{
|
|
NodeId = new NodeId((uint)i + 1, namespaceIndex),
|
|
BrowseName = new QualifiedName("Counter" + i.ToString(CultureInfo.InvariantCulture), namespaceIndex),
|
|
DisplayName = new LocalizedText("Counter" + i.ToString(CultureInfo.InvariantCulture)),
|
|
TypeDefinitionId = VariableTypeIds.BaseDataVariableType,
|
|
ReferenceTypeId = ReferenceTypeIds.Organizes,
|
|
DataType = DataTypeIds.UInt64,
|
|
ValueRank = ValueRanks.Scalar,
|
|
MinimumSamplingInterval = 1,
|
|
AccessLevel = AccessLevels.CurrentRead,
|
|
UserAccessLevel = AccessLevels.CurrentRead,
|
|
Value = new Variant((ulong)0),
|
|
StatusCode = StatusCodes.Good,
|
|
Timestamp = DateTimeUtc.Now
|
|
};
|
|
node.OnReadValue = (ISystemContext context, NodeState state, NumericRange range,
|
|
QualifiedName encoding, ref Variant value, ref StatusCode status, ref DateTimeUtc timestamp) =>
|
|
{
|
|
value = new Variant(Increment(index));
|
|
status = StatusCodes.Good;
|
|
timestamp = DateTimeUtc.Now;
|
|
return ServiceResult.Good;
|
|
};
|
|
node.AddReference(ReferenceTypeIds.Organizes, true, ObjectIds.ObjectsFolder);
|
|
yield return node;
|
|
}
|
|
}
|
|
|
|
/// <summary>
|
|
/// Starts a measurement window.
|
|
/// </summary>
|
|
public void Mark()
|
|
{
|
|
for (int i = 0; i < m_values.Length; i++)
|
|
{
|
|
m_marked[i] = Interlocked.Read(ref m_values[i]);
|
|
}
|
|
m_markTimestamp = Stopwatch.GetTimestamp();
|
|
}
|
|
|
|
/// <summary>
|
|
/// Reports the sampling progress of the window: completed read callbacks per item
|
|
/// against the number the sampling interval calls for.
|
|
/// </summary>
|
|
public string Report(double samplingIntervalMs)
|
|
{
|
|
double elapsedMs = Stopwatch.GetElapsedTime(m_markTimestamp).TotalMilliseconds;
|
|
double expected = elapsedMs / samplingIntervalMs;
|
|
double min = double.MaxValue;
|
|
double sum = 0;
|
|
int belowTarget = 0;
|
|
for (int i = 0; i < m_values.Length; i++)
|
|
{
|
|
double progress = (Interlocked.Read(ref m_values[i]) - m_marked[i]) / expected;
|
|
min = Math.Min(min, progress);
|
|
sum += progress;
|
|
if (progress < 0.99)
|
|
{
|
|
belowTarget++;
|
|
}
|
|
}
|
|
return FormattableString.Invariant(
|
|
$"counters={m_values.Length} samplesExpected={expected:F0} progressMin={min:P2} ") +
|
|
FormattableString.Invariant(
|
|
$"progressMean={sum / m_values.Length:P2} itemsBelow99={belowTarget}");
|
|
}
|
|
|
|
private ulong Increment(int index)
|
|
{
|
|
if (m_lockedCallback)
|
|
{
|
|
// the o6-automation .NET worker increments under a lock it owns
|
|
lock (m_lock)
|
|
{
|
|
return ++m_values[index];
|
|
}
|
|
}
|
|
return Interlocked.Increment(ref m_values[index]);
|
|
}
|
|
|
|
private readonly bool m_lockedCallback;
|
|
private readonly Lock m_lock = new();
|
|
private readonly ulong[] m_values;
|
|
private readonly ulong[] m_marked;
|
|
private long m_markTimestamp = Stopwatch.GetTimestamp();
|
|
}
|
|
|
|
/// <summary>
|
|
/// The counters on <see cref="CustomNodeManager2"/> with sampling groups.
|
|
/// </summary>
|
|
internal sealed class CounterNodeManager : CustomNodeManager2
|
|
{
|
|
public const string Namespace = "urn:opcfoundation.org:perf:counters";
|
|
|
|
public CounterNodeManager(
|
|
IServerInternal server,
|
|
ApplicationConfiguration configuration,
|
|
CounterValues counters,
|
|
string namespaceUri)
|
|
: base(server, configuration, useSamplingGroups: true, namespaceUri)
|
|
{
|
|
m_counters = counters;
|
|
}
|
|
|
|
public override void CreateAddressSpace(IDictionary<NodeId, IList<IReference>> externalReferences)
|
|
{
|
|
if (!externalReferences.TryGetValue(ObjectIds.ObjectsFolder, out IList<IReference>? references))
|
|
{
|
|
externalReferences[ObjectIds.ObjectsFolder] = references = new List<IReference>();
|
|
}
|
|
foreach (BaseDataVariableState node in m_counters.CreateNodes(NamespaceIndex))
|
|
{
|
|
references.Add(new NodeStateReference(ReferenceTypeIds.Organizes, false, node.NodeId));
|
|
AddPredefinedNode(SystemContext, node);
|
|
}
|
|
}
|
|
|
|
private readonly CounterValues m_counters;
|
|
}
|
|
|
|
/// <summary>
|
|
/// The counters on <see cref="AsyncCustomNodeManager"/> with sampling groups, as the
|
|
/// o6-automation .NET worker for UA-.NETStandard 2.0 builds them.
|
|
/// </summary>
|
|
internal sealed class AsyncCounterNodeManager : AsyncCustomNodeManager
|
|
{
|
|
public AsyncCounterNodeManager(
|
|
IServerInternal server,
|
|
ApplicationConfiguration configuration,
|
|
CounterValues counters,
|
|
string namespaceUri)
|
|
: base(server, configuration, useSamplingGroups: true, namespaceUri)
|
|
{
|
|
m_counters = counters;
|
|
}
|
|
|
|
public override async ValueTask CreateAddressSpaceAsync(
|
|
IDictionary<NodeId, IList<IReference>> externalReferences,
|
|
CancellationToken cancellationToken = default)
|
|
{
|
|
await base.CreateAddressSpaceAsync(externalReferences, cancellationToken).ConfigureAwait(false);
|
|
if (!externalReferences.TryGetValue(ObjectIds.ObjectsFolder, out IList<IReference>? references))
|
|
{
|
|
externalReferences[ObjectIds.ObjectsFolder] = references = new List<IReference>();
|
|
}
|
|
foreach (BaseDataVariableState node in m_counters.CreateNodes(NamespaceIndex))
|
|
{
|
|
references.Add(new NodeStateReference(ReferenceTypeIds.Organizes, false, node.NodeId));
|
|
await AddPredefinedNodeAsync(SystemContext, node, cancellationToken).ConfigureAwait(false);
|
|
}
|
|
}
|
|
|
|
private readonly CounterValues m_counters;
|
|
}
|
|
|
|
/// <summary>
|
|
/// A server exposing only the counter node manager in addition to the standard nodes.
|
|
/// </summary>
|
|
internal sealed class CounterServer : StandardServer
|
|
{
|
|
public CounterServer(
|
|
ITelemetryContext telemetry,
|
|
int count,
|
|
string namespaceUri = CounterNodeManager.Namespace,
|
|
bool lockedCallback = false,
|
|
bool asyncNodeManager = false)
|
|
: base(telemetry)
|
|
{
|
|
Counters = new CounterValues(count, lockedCallback);
|
|
m_namespaceUri = namespaceUri;
|
|
m_asyncNodeManager = asyncNodeManager;
|
|
}
|
|
|
|
public CounterValues Counters { get; }
|
|
|
|
protected override IMasterNodeManager CreateMasterNodeManager(
|
|
IServerInternal server,
|
|
ApplicationConfiguration configuration)
|
|
{
|
|
return m_asyncNodeManager
|
|
? new MasterNodeManager(
|
|
server, configuration, null, new AsyncCounterNodeManager(server, configuration, Counters, m_namespaceUri))
|
|
: new MasterNodeManager(
|
|
server, configuration, null, new CounterNodeManager(server, configuration, Counters, m_namespaceUri));
|
|
}
|
|
|
|
private readonly string m_namespaceUri;
|
|
private readonly bool m_asyncNodeManager;
|
|
}
|
|
}
|