Files
romanett 8b56937fdf perf: drive the server with external native benchmark workers; TraceAnalyzer GC modes (#4815)
## What this adds

Harness-only changes under `perf/` (not part of `UA.slnx`, `net10.0`).
No product code changes.

### External native worker scenarios (`ServerLoadHarness`)

Two new scenarios run the native open62541 workers of the

[o6-automation/opcua-benchmarks](https://github.com/o6-automation/opcua-benchmarks)
project
against a server, so the .NET server and other OPC UA stacks are
measured with the same client
as that project's published comparison:

| Scenario | What it measures |
|---|---|
| `native-read` | Fixed-load scalar Read throughput
(`throughput/open62541/client.c --worker async`): `--clients` worker
processes, each with `--inflight` single-value Reads of
ns=1;i=1001..1100 in flight, starting together. Reports aggregate req/s,
and server CPU and allocated bytes per request. |
| `native-counters` | Subscription capacity
(`subscription/native/client.c`): one subscription with `--nodes` UInt64
counters at `--sampling` 10 ms / `--pub` 1000 ms, queue size 1. Passes
when every item gets 99 % of the samples its sampling interval calls
for. |

The result block has the same lines as the other scenarios, and the CPU
time of the native
workers goes in `client-per-op:`.

The server is either the harness child in **contract mode** (`server
--contract 1`) or any
external executable (`--server-exe`, `--server-args` with `{port}`,
`{items}`, `{sampling}` and
`{pki}` placeholders), for example the benchmark's open62541 server or
one of its .NET workers.

By default the harness child is configured like the benchmark's .NET
worker for
UA-.NETStandard 2.0.1:

| Setting | Default | Override |
|---|---|---|
| Node managers | `AsyncCustomNodeManager` | `--async-node-manager 0`
(`CustomNodeManager2`) |
| Counter increment | under a lock owned by the counters |
`--counter-lock 0` (interlocked) |
| Garbage collector | Server GC | `SERVER_DOTNET_gcServer=0` |
| `MaxQueuedRequestCount` | clients × in flight + 3 (minimum 200) |
`--queue` |

Other options: `--native-client`, `--clients`, `--inflight`,
`--iterations`, `--warmup-calls`,
`--server-affinity` (processor mask; combine it with
`SERVER_DOTNET_PROCESSOR_COUNT`, because the
mask is applied after the runtime has sized itself),
`--client-affinity`, `--server-priority`,
`--thread-states 1` (samples Running/Ready server threads and appends a
`sched:` line), `--security encrypt`
(Basic256Sha256 SignAndEncrypt with generated DER identities) and
`--verbose`. Also, the
`MinRequestThreadCount` of the server child is now the smaller of 50 and
`--threads`, so
`--threads` below 50 takes effect.

**Worker protocol.** The workers define a few names that are kept as
string literals in named
constants: the environment variables `O6_BENCHMARK_SECURITY_POLICY`,
`O6_BENCHMARK_CERTIFICATE`
and `O6_BENCHMARK_PRIVATE_KEY`, the `O6_BENCHMARK_READY` stdout token,
the identity
application URI / subject prefixes, and the server URI
`urn:o6:benchmark:server`. The workers
need that server URI because the contract puts its nodes in the
namespace of the server URI and
the workers address them as ns=1. Identifiers, file names, scenario and
option names are neutral.

### TraceAnalyzer

- `cpu` with a focus frame also reports the first `Opc.Ua` frame above
it, past framework
  wrappers such as `Monitor.Enter` or `SemaphoreSlim`.
- **GC-poll attribution:** the thread-time sampler suspends running
threads at their next GC
poll, so many on-CPU samples have `Thread.PollGCWorker` as their leaf.
`cpu` used to drop
them as waits (in one trace only 3.5k of 20k on-CPU samples were kept);
it now attributes them
  to the caller.
- `gc` mode (for `--trace-profile gc`): collections by generation, type
and reason; the
suspension time and the whole pause, each with count, total and
percentiles.
- `samples` mode: all samples by type (managed/external) and by leaf and
first `Opc.Ua` frame,
  including the ones `cpu` leaves out.
- `suspend <trace> [thresholdMs]` mode: every SuspendEE longer than the
threshold, with the
events recorded inside it by thread (an `AllocationTick` there means a
thread that had not
reached a safe point yet); event counts including the heap count DATAS
chose and GC handle
  churn by kind.

## Building the workers

The benchmark repository is AGPL-3.0, so nothing of it is vendored here.
Clone it and build the
workers separately. On Windows they build with `zig cc` and CMake
against the open62541 commit
the benchmark pins (OpenSSL backend, `UA_ENABLE_DEBUG_SANITIZER=OFF`,
Release). The
subscription workers also need `poll()` on stdin replaced by
`PeekNamedPipe`, plus unbuffered
control output. `perf/README.md` describes all of this.

## Commits

1. `perf: drive the server with external native benchmark workers`
2. `perf: configure the contract server like the benchmark's 2.0.1 .NET
worker`
3. `perf: TraceAnalyzer gc and samples modes, samples taken at GC polls`
4. `perf: scheduling and GC-suspension diagnostics for the native worker
runs`

Each commit builds with 0 warnings
(`CustomTestTarget=net10.0 dotnet build perf/ServerLoadHarness -c
Release`,
`dotnet build perf/TraceAnalyzer -c Release`), and `dotnet format
whitespace` makes no changes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 14:35:16 +00:00

254 lines
10 KiB
C#

/* ========================================================================
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.Threading;
using System.Threading.Tasks;
using Opc.Ua.Server;
namespace Opc.Ua.Perf.ServerLoadHarness
{
/// <summary>
/// The counters of the subscription capacity workload of the o6-automation opcua-benchmarks
/// (UInt64 counter per monitored item, advanced by its read callback, so only sampling drives
/// them), shared by the node manager variants.
/// </summary>
internal sealed class CounterValues
{
public CounterValues(int count, bool lockedCallback)
{
m_lockedCallback = lockedCallback;
m_values = new ulong[count];
m_marked = new ulong[count];
}
/// <summary>
/// Creates the counter variables ns=namespaceIndex;i=1..count below the Objects folder.
/// </summary>
public IEnumerable<BaseDataVariableState> CreateNodes(ushort namespaceIndex)
{
for (int i = 0; i < m_values.Length; i++)
{
int index = i;
var node = new BaseDataVariableState(null)
{
NodeId = new NodeId((uint)i + 1, namespaceIndex),
BrowseName = new QualifiedName("Counter" + i.ToString(CultureInfo.InvariantCulture), namespaceIndex),
DisplayName = new LocalizedText("Counter" + i.ToString(CultureInfo.InvariantCulture)),
TypeDefinitionId = VariableTypeIds.BaseDataVariableType,
ReferenceTypeId = ReferenceTypeIds.Organizes,
DataType = DataTypeIds.UInt64,
ValueRank = ValueRanks.Scalar,
MinimumSamplingInterval = 1,
AccessLevel = AccessLevels.CurrentRead,
UserAccessLevel = AccessLevels.CurrentRead,
Value = new Variant((ulong)0),
StatusCode = StatusCodes.Good,
Timestamp = DateTimeUtc.Now
};
node.OnReadValue = (ISystemContext context, NodeState state, NumericRange range,
QualifiedName encoding, ref Variant value, ref StatusCode status, ref DateTimeUtc timestamp) =>
{
value = new Variant(Increment(index));
status = StatusCodes.Good;
timestamp = DateTimeUtc.Now;
return ServiceResult.Good;
};
node.AddReference(ReferenceTypeIds.Organizes, true, ObjectIds.ObjectsFolder);
yield return node;
}
}
/// <summary>
/// Starts a measurement window.
/// </summary>
public void Mark()
{
for (int i = 0; i < m_values.Length; i++)
{
m_marked[i] = Interlocked.Read(ref m_values[i]);
}
m_markTimestamp = Stopwatch.GetTimestamp();
}
/// <summary>
/// Reports the sampling progress of the window: completed read callbacks per item
/// against the number the sampling interval calls for.
/// </summary>
public string Report(double samplingIntervalMs)
{
double elapsedMs = Stopwatch.GetElapsedTime(m_markTimestamp).TotalMilliseconds;
double expected = elapsedMs / samplingIntervalMs;
double min = double.MaxValue;
double sum = 0;
int belowTarget = 0;
for (int i = 0; i < m_values.Length; i++)
{
double progress = (Interlocked.Read(ref m_values[i]) - m_marked[i]) / expected;
min = Math.Min(min, progress);
sum += progress;
if (progress < 0.99)
{
belowTarget++;
}
}
return FormattableString.Invariant(
$"counters={m_values.Length} samplesExpected={expected:F0} progressMin={min:P2} ") +
FormattableString.Invariant(
$"progressMean={sum / m_values.Length:P2} itemsBelow99={belowTarget}");
}
private ulong Increment(int index)
{
if (m_lockedCallback)
{
// the o6-automation .NET worker increments under a lock it owns
lock (m_lock)
{
return ++m_values[index];
}
}
return Interlocked.Increment(ref m_values[index]);
}
private readonly bool m_lockedCallback;
private readonly Lock m_lock = new();
private readonly ulong[] m_values;
private readonly ulong[] m_marked;
private long m_markTimestamp = Stopwatch.GetTimestamp();
}
/// <summary>
/// The counters on <see cref="CustomNodeManager2"/> with sampling groups.
/// </summary>
internal sealed class CounterNodeManager : CustomNodeManager2
{
public const string Namespace = "urn:opcfoundation.org:perf:counters";
public CounterNodeManager(
IServerInternal server,
ApplicationConfiguration configuration,
CounterValues counters,
string namespaceUri)
: base(server, configuration, useSamplingGroups: true, namespaceUri)
{
m_counters = counters;
}
public override void CreateAddressSpace(IDictionary<NodeId, IList<IReference>> externalReferences)
{
if (!externalReferences.TryGetValue(ObjectIds.ObjectsFolder, out IList<IReference>? references))
{
externalReferences[ObjectIds.ObjectsFolder] = references = new List<IReference>();
}
foreach (BaseDataVariableState node in m_counters.CreateNodes(NamespaceIndex))
{
references.Add(new NodeStateReference(ReferenceTypeIds.Organizes, false, node.NodeId));
AddPredefinedNode(SystemContext, node);
}
}
private readonly CounterValues m_counters;
}
/// <summary>
/// The counters on <see cref="AsyncCustomNodeManager"/> with sampling groups, as the
/// o6-automation .NET worker for UA-.NETStandard 2.0 builds them.
/// </summary>
internal sealed class AsyncCounterNodeManager : AsyncCustomNodeManager
{
public AsyncCounterNodeManager(
IServerInternal server,
ApplicationConfiguration configuration,
CounterValues counters,
string namespaceUri)
: base(server, configuration, useSamplingGroups: true, namespaceUri)
{
m_counters = counters;
}
public override async ValueTask CreateAddressSpaceAsync(
IDictionary<NodeId, IList<IReference>> externalReferences,
CancellationToken cancellationToken = default)
{
await base.CreateAddressSpaceAsync(externalReferences, cancellationToken).ConfigureAwait(false);
if (!externalReferences.TryGetValue(ObjectIds.ObjectsFolder, out IList<IReference>? references))
{
externalReferences[ObjectIds.ObjectsFolder] = references = new List<IReference>();
}
foreach (BaseDataVariableState node in m_counters.CreateNodes(NamespaceIndex))
{
references.Add(new NodeStateReference(ReferenceTypeIds.Organizes, false, node.NodeId));
await AddPredefinedNodeAsync(SystemContext, node, cancellationToken).ConfigureAwait(false);
}
}
private readonly CounterValues m_counters;
}
/// <summary>
/// A server exposing only the counter node manager in addition to the standard nodes.
/// </summary>
internal sealed class CounterServer : StandardServer
{
public CounterServer(
ITelemetryContext telemetry,
int count,
string namespaceUri = CounterNodeManager.Namespace,
bool lockedCallback = false,
bool asyncNodeManager = false)
: base(telemetry)
{
Counters = new CounterValues(count, lockedCallback);
m_namespaceUri = namespaceUri;
m_asyncNodeManager = asyncNodeManager;
}
public CounterValues Counters { get; }
protected override IMasterNodeManager CreateMasterNodeManager(
IServerInternal server,
ApplicationConfiguration configuration)
{
return m_asyncNodeManager
? new MasterNodeManager(
server, configuration, null, new AsyncCounterNodeManager(server, configuration, Counters, m_namespaceUri))
: new MasterNodeManager(
server, configuration, null, new CounterNodeManager(server, configuration, Counters, m_namespaceUri));
}
private readonly string m_namespaceUri;
private readonly bool m_asyncNodeManager;
}
}