Files
UA-.NETStandard/tests/Opc.Ua.Sessions.Tests/SecurityPolicyBenchmarks.cs
romanett 9b034f8291 Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description

Nullable reference types are now enabled once, in `common.props`, and
every project in the repository builds warning-clean with them.

**Configuration**
- `<Nullable>enable</Nullable>` lives in `common.props`. All per-project
`<Nullable>` settings are removed: 178 project files (enable,
annotations, and the `disable` in the three Aot test projects).
- 419 per-file `#nullable enable`/`restore` directives and the 4
`#nullable disable` directives are removed.
- Kept on purpose:
- The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type
Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`,
`PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the
project setting in generated code, so the directive is required there.
- `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against
the 1.5.378 packages and does not import the repository props.
- `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now
describe the central setting.

**Migration** (the 43 unset and 13 annotations-only projects, roughly
10k diagnostics)
- **Production code: annotated at the declarations.** Covers
`tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the
LDS sample.
- Members, parameters and returns are nullable where null actually
flows.
- Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`,
`IsMethodTypeDesign`).
- `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take
`params object?[]`. `Template.AddReplacement` takes `string?` (rendering
already skips null replacements).
- `IRegisteredServerStore.RegisterAsync` returns
`Task<RegistrationEntry?>`: it returns null when a registration is
removed.
- Dereferences backed by an invariant (for example, the validated model)
use `!`, so runtime behaviour is unchanged.
- **Behavioural changes in production code** (small, deliberate):
- `NodeSetToModelDesign` throws `InvalidOperationException` when the
file is not a NodeSet. Before, it threw a NullReferenceException on the
next line.
- `ModelDesignValidator` starts its lookup tables empty instead of null.
A redundant `if (m_nodes == null) return;` guard is gone.
- **Test code: minimal annotations.**
  - `?` on values that can be null, `!` where the test asserts non-null.
  - `= null!` for fixture members assigned in SetUp.
- `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the
`IEncodeable` constraint.
- A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`)
folded into the adjacent `IsNull`/`Count` checks.

**Notes for reviewers**
- The diff is large (~1,500 files) but almost entirely mechanical: `?`,
`!` and directive removal. The non-annotation changes are listed above.
- Known follow-up, not done here:
`IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay
non-nullable even though the validator returns null before it loads a
dictionary. Making them nullable would ripple through every generator
call site.

## Related Issues

- Fixes #4439

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

The "no new warnings" box covers net10.0 and net48: full non-incremental
builds of `UA.slnx`, `tools/SourceGeneration.slnx`,
`tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0
warnings and 0 errors under the repository's warnings-as-errors
settings. net9.0 and net8.0 also had no nullable diagnostics, but they
were not rebuilt after the final fixes. Tests have not been run locally
yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:48:13 +02:00

1080 lines
39 KiB
C#
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
/* ========================================================================
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using NUnit.Framework;
using Opc.Ua.Client.TestFramework;
using ISession = Opc.Ua.Client.ISession;
namespace Opc.Ua.Sessions.Tests
{
/// <summary>
/// Custom config for Security Policy Benchmarks that increases build timeout.
/// </summary>
public class SecurityPolicyBenchmarkConfig : ManualConfig
{
public SecurityPolicyBenchmarkConfig()
{
// Increase build timeout to 10 minutes for large dependency builds
WithOptions(ConfigOptions.DisableOptimizationsValidator);
WithBuildTimeout(TimeSpan.FromMinutes(10));
}
}
/// <summary>
/// <para>
/// Benchmarks for measuring CPU, Memory, Latency, and Message Throughput across all Security Policies.
/// These benchmarks help detect performance regressions when changes are made to security-related code.
/// </para>
/// <para>Total: 171 benchmarks (19 methods × 9 security policies - None is excluded)</para>
/// <para>
/// USAGE:
/// cd tests/Opc.Ua.Client.Tests
/// </para>
/// <para>
/// Run all 171 benchmarks (takes ~30+ minutes):
/// dotnet run -c Release -f net10.0 -- --filter '*SecurityPolicyBenchmarks*' --job short
/// </para>
/// <para>
/// Run specific benchmark method across all security policies:
/// dotnet run -c Release -f net10.0 -- --filter '*ReadSmallMessageAsync*' --job short
/// dotnet run -c Release -f net10.0 -- --filter '*WriteSmallMessageAsync*' --job short
/// dotnet run -c Release -f net10.0 -- --filter '*BrowseAsync*' --job short
/// </para>
/// <para>
/// Run specific benchmark with specific policy:
/// To run only one policy, temporarily modify BenchPolicies() in ClientTestFramework.cs
/// to return only the desired policy, then run:
/// dotnet run -c Release -f net10.0 -- --filter '*ReadSmallMessageAsync*' --job short
/// </para>
/// <para>
/// Run as NUnit tests (faster, no separate build):
/// dotnet test -c Release -f net10.0 --filter 'FullyQualifiedName~SecurityPolicyBenchmarks'
/// dotnet test -c Release -f net10.0 --filter 'FullyQualifiedName~SecurityPolicyBenchmarks.ReadSmallMessageAsync'
/// </para>
/// <para>
/// View results:
/// cat BenchmarkDotNet.Artifacts/results/*.md
/// cat BenchmarkDotNet.Artifacts/results/*.csv
/// </para>
/// <para>
/// Available benchmark methods:
/// Latency benchmarks:
/// - ReadSmallMessageAsync, ReadMediumMessageAsync, ReadLargeMessageAsync
/// - WriteSmallMessageAsync
/// - BrowseAsync, BrowseMultipleNodesAsync
/// - CallMethodAsync
/// - CreateCloseSessionAsync, SessionLifecycleWithReadAsync
/// - DiscoverEndpointsAsync
/// - MixedWorkloadAsync
/// </para>
/// <para>
/// Throughput benchmarks (operations/second):
/// - ReadSmallMessageBurstAsync, ReadMediumMessageBurstAsync, ReadLargeMessageBurstAsync
/// - WriteSmallMessageBurstAsync
/// - ReadThroughputAsync, WriteThroughputAsync
/// - BrowseThroughputAsync, CallThroughputAsync
/// </para>
/// <para>
/// Available security policies (9 total, None is excluded):
/// - Basic128Rsa15, Basic256, Basic256Sha256
/// - Aes128_Sha256_RsaOaep, Aes256_Sha256_RsaPss
/// - ECC_nistP256, ECC_nistP384, ECC_brainpoolP256r1, ECC_brainpoolP384r1
/// </para>
/// <para>
/// THROUGHPUT CALCULATION:
/// Throughput benchmarks execute 100 operations and measure total time.
/// Calculate ops/sec using: Throughput = 100 / (Mean time in seconds)
/// </para>
/// <para>
/// Example from results:
/// Method: 'Read 100 ops (for throughput)'
/// SecurityPolicy: Basic128Rsa15
/// Mean: 140.8 ms = 0.1408 seconds
/// Throughput = 100 / 0.1408 = 710 ops/sec
/// </para>
/// </summary>
[TestFixture]
[Explicit]
[Category("Client")]
[Category("SecurityPolicyBenchmark")]
[SetCulture("en-us")]
[SetUICulture("en-us")]
[Config(typeof(SecurityPolicyBenchmarkConfig))]
[MemoryDiagnoser]
[DisassemblyDiagnoser]
public class SecurityPolicyBenchmarks : ClientTestFramework
{
private const int kMessageCount = 100;
private const int kSmallMessageNodeCount = 10;
private const int kMediumMessageNodeCount = 50;
private const int kLargeMessageNodeCount = 200;
private IList<NodeId> m_smallTestSet;
private IList<NodeId> m_mediumTestSet;
private IList<NodeId> m_largeTestSet;
private ArrayOf<ReadValueId> m_smallReadValueIds;
private ArrayOf<ReadValueId> m_mediumReadValueIds;
private ArrayOf<ReadValueId> m_largeReadValueIds;
public SecurityPolicyBenchmarks()
: base(Utils.UriSchemeOpcTcp)
{
SingleSession = false;
}
/// <summary>
/// Override to exclude None policy from benchmarks to avoid CI test failures.
/// Uses the base policy list so target-specific filtering is preserved.
/// </summary>
private static new IEnumerable<string> BenchPolicies()
{
foreach (string policyUri in Policies)
{
if (policyUri != SecurityPolicies.None)
{
yield return policyUri;
}
}
}
/// <summary>
/// Set up a Server and a Client instance.
/// </summary>
[OneTimeSetUp]
public new async Task OneTimeSetUpAsync()
{
SupportsExternalServerUrl = true;
await base.OneTimeSetUpAsync().ConfigureAwait(false);
await EnsureCachedEndpointsAsync().ConfigureAwait(false);
await PrepareTestDataAsync().ConfigureAwait(false);
}
/// <summary>
/// Tear down the Server and the Client.
/// </summary>
[OneTimeTearDown]
public new Task OneTimeTearDownAsync()
{
return base.OneTimeTearDownAsync();
}
/// <summary>
/// Test setup.
/// </summary>
[SetUp]
public new async Task SetUpAsync()
{
await base.SetUpAsync().ConfigureAwait(false);
await PrepareTestDataAsync().ConfigureAwait(false);
}
/// <summary>
/// Test teardown.
/// </summary>
[TearDown]
public new Task TearDownAsync()
{
return base.TearDownAsync();
}
/// <summary>
/// Global Setup for benchmarks.
/// </summary>
[GlobalSetup]
public override void GlobalSetup()
{
base.GlobalSetup();
EnsureCachedEndpointsAsync().GetAwaiter().GetResult();
PrepareTestDataAsync().GetAwaiter().GetResult();
}
/// <summary>
/// Global cleanup for benchmarks.
/// </summary>
[GlobalCleanup]
public override void GlobalCleanup()
{
base.GlobalCleanup();
}
/// <summary>
/// Prepare test data sets for benchmarking.
/// </summary>
private async Task PrepareTestDataAsync()
{
if (Session == null)
{
return;
}
// Get available test nodes
IList<NodeId> allTestNodes = GetTestSetStatic(Session.NamespaceUris);
// Ensure we have enough nodes for testing
if (allTestNodes.Count < kLargeMessageNodeCount)
{
// If not enough static nodes, add simulation nodes
IList<NodeId> simNodes = GetTestSetSimulation(Session.NamespaceUris);
allTestNodes = [.. allTestNodes.Concat(simNodes).Take(kLargeMessageNodeCount)];
}
// Create test sets of different sizes
m_smallTestSet = [.. allTestNodes.Take(kSmallMessageNodeCount)];
m_mediumTestSet = [.. allTestNodes.Take(kMediumMessageNodeCount)];
m_largeTestSet = [.. allTestNodes.Take(kLargeMessageNodeCount)];
// Prepare ReadValueId collections
m_smallReadValueIds = [.. m_smallTestSet.Select(nodeId => new ReadValueId
{
NodeId = nodeId,
AttributeId = Attributes.Value
})];
m_mediumReadValueIds = [.. m_mediumTestSet.Select(nodeId => new ReadValueId
{
NodeId = nodeId,
AttributeId = Attributes.Value
})];
m_largeReadValueIds = [.. m_largeTestSet.Select(nodeId => new ReadValueId
{
NodeId = nodeId,
AttributeId = Attributes.Value
})];
// Verify we can read the nodes
await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_smallReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
}
/// <summary>
/// Ensures endpoint discovery is completed once during setup, outside measured session lifecycle iterations.
/// </summary>
private async Task EnsureCachedEndpointsAsync()
{
if (Endpoints.IsNull)
{
Endpoints = await ClientFixture.GetEndpointsAsync(ServerUrl).ConfigureAwait(false);
}
}
/// <summary>
/// Benchmark: Read small message (10 nodes) - measures baseline performance.
/// Tests CPU and memory overhead for small messages with different security policies.
/// </summary>
[Test]
[Order(100)]
[Benchmark(Baseline = true, Description = "Read 10 nodes")]
public async Task ReadSmallMessageAsync()
{
ReadResponse response = await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_smallReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
Assert.That(response.Results.Count, Is.EqualTo(m_smallReadValueIds.Count));
}
/// <summary>
/// Benchmark: Read small messages repeatedly - measures sustained throughput.
/// </summary>
[Test]
[Order(101)]
[Benchmark(Description = "Read 10 nodes x100 iterations")]
public async Task ReadSmallMessageBurstAsync()
{
for (int i = 0; i < kMessageCount; i++)
{
await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_smallReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
}
}
/// <summary>
/// Benchmark: Read medium message (50 nodes) - measures typical workload performance.
/// </summary>
[Test]
[Order(200)]
[Benchmark(Description = "Read 50 nodes")]
public async Task ReadMediumMessageAsync()
{
ReadResponse response = await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_mediumReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
Assert.That(response.Results.Count, Is.EqualTo(m_mediumReadValueIds.Count));
}
/// <summary>
/// Benchmark: Read medium messages repeatedly - measures sustained medium-load throughput.
/// </summary>
[Test]
[Order(201)]
[Benchmark(Description = "Read 50 nodes x100 iterations")]
public async Task ReadMediumMessageBurstAsync()
{
for (int i = 0; i < kMessageCount; i++)
{
await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_mediumReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
}
}
/// <summary>
/// Benchmark: Read large message (200 nodes) - measures high-load performance.
/// Tests encryption/decryption overhead with larger payloads.
/// </summary>
[Test]
[Order(300)]
[Benchmark(Description = "Read 200 nodes")]
public async Task ReadLargeMessageAsync()
{
ReadResponse response = await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_largeReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
Assert.That(response.Results.Count, Is.EqualTo(m_largeReadValueIds.Count));
}
/// <summary>
/// Benchmark: Read large messages repeatedly - measures sustained high-load throughput.
/// </summary>
[Test]
[Order(301)]
[Benchmark(Description = "Read 200 nodes x100 iterations")]
public async Task ReadLargeMessageBurstAsync()
{
for (int i = 0; i < kMessageCount; i++)
{
await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_largeReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
}
}
/// <summary>
/// Benchmark: Write small message (10 nodes) - measures write performance.
/// Tests CPU and memory overhead for write operations with different security policies.
/// </summary>
[Test]
[Order(400)]
[Benchmark(Description = "Write 10 nodes")]
public async Task WriteSmallMessageAsync()
{
var writeValues =
m_smallTestSet.Select(nodeId => new WriteValue
{
NodeId = nodeId,
AttributeId = Attributes.Value,
Value = new DataValue(new Variant(UnsecureRandom.Shared.Next()))
}).ToArrayOf();
WriteResponse response = await Session.WriteAsync(
null,
writeValues,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
}
/// <summary>
/// Benchmark: Write small messages repeatedly - measures sustained write throughput.
/// </summary>
[Test]
[Order(401)]
[Benchmark(Description = "Write 10 nodes x100 iterations")]
public async Task WriteSmallMessageBurstAsync()
{
for (int i = 0; i < kMessageCount; i++)
{
var writeValues =
m_smallTestSet.Select(nodeId => new WriteValue
{
NodeId = nodeId,
AttributeId = Attributes.Value,
Value = new DataValue(new Variant(i))
}).ToArrayOf();
await Session.WriteAsync(
null,
writeValues,
CancellationToken.None
).ConfigureAwait(false);
}
}
/// <summary>
/// Benchmark: Browse operation - measures browse performance.
/// Tests how security policy affects browse operations and reference enumeration.
/// </summary>
[Test]
[Order(500)]
[Benchmark(Description = "Browse Objects folder")]
public async Task BrowseAsync()
{
ArrayOf<BrowseDescription> nodesToBrowse =
[
new BrowseDescription
{
NodeId = ObjectIds.ObjectsFolder,
BrowseDirection = BrowseDirection.Forward,
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
IncludeSubtypes = true,
NodeClassMask = (uint)NodeClass.Object | (uint)NodeClass.Variable,
ResultMask = (uint)BrowseResultMask.All
}
];
BrowseResponse response = await Session.BrowseAsync(
null,
null,
0,
nodesToBrowse,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
Assert.That(response.Results.IsEmpty, Is.True);
}
/// <summary>
/// Benchmark: Browse multiple nodes - measures browse throughput.
/// </summary>
[Test]
[Order(501)]
[Benchmark(Description = "Browse 10 nodes")]
public async Task BrowseMultipleNodesAsync()
{
//await Task.Delay(5000);
var nodesToBrowse =
m_smallTestSet.Select(nodeId => new BrowseDescription
{
NodeId = nodeId,
BrowseDirection = BrowseDirection.Forward,
ReferenceTypeId = ReferenceTypeIds.References,
IncludeSubtypes = true,
NodeClassMask = 0,
ResultMask = (uint)BrowseResultMask.All
}).ToArrayOf();
BrowseResponse response = await Session.BrowseAsync(
null,
null,
0,
nodesToBrowse,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
}
/// <summary>
/// Benchmark: Call GetMonitoredItems method - measures method call performance.
/// Tests how security policy affects method call operations.
/// </summary>
[Test]
[Order(600)]
[Benchmark(Description = "Call GetMonitoredItems method")]
public async Task CallMethodAsync()
{
ArrayOf<Variant> inputArguments =
[
new Variant(0u) // subscriptionId
];
ArrayOf<CallMethodRequest> requests =
[
new CallMethodRequest
{
ObjectId = ObjectIds.Server,
MethodId = MethodIds.Server_GetMonitoredItems,
InputArguments = inputArguments
}
];
CallResponse response = await Session.CallAsync(
null,
requests,
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
Assert.That(response.Results.IsNull, Is.False);
Assert.That(response.Results.Count, Is.EqualTo(1));
}
/// <summary>
/// Benchmark: Create and close session - measures session establishment overhead.
/// This is critical for understanding the cost of security policy negotiation.
/// </summary>
[Test]
[Order(700)]
[Benchmark(Description = "Create and close session")]
public async Task CreateCloseSessionAsync()
{
ISession session = await ClientFixture.ConnectAsync(
ServerUrl,
SecurityPolicy,
Endpoints
).ConfigureAwait(false);
Assert.That(session, Is.Not.Null);
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
session.Dispose();
}
/// <summary>
/// Benchmark: Create, use, and close session - measures full session lifecycle.
/// </summary>
[Test]
[Order(701)]
[Benchmark(Description = "Session lifecycle with read")]
public async Task SessionLifecycleWithReadAsync()
{
ISession session = await ClientFixture.ConnectAsync(
ServerUrl,
SecurityPolicy,
Endpoints
).ConfigureAwait(false);
Assert.That(session, Is.Not.Null);
// Perform a read operation
await session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_smallReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
session.Dispose();
}
/// <summary>
/// Benchmark: Discover endpoints - tracks endpoint discovery cost independently from session establishment.
/// </summary>
[Test]
[Order(702)]
[Benchmark(Description = "Discover endpoints")]
public async Task DiscoverEndpointsAsync()
{
ArrayOf<EndpointDescription> endpoints = await ClientFixture.GetEndpointsAsync(ServerUrl).ConfigureAwait(false);
Assert.That(endpoints.IsNull, Is.False);
}
/// <summary>
/// Benchmark: Mixed operations - measures realistic workload performance.
/// Combines read, write, browse, and call operations to simulate real applications.
/// </summary>
[Test]
[Order(800)]
[Benchmark(Description = "Mixed workload (read+write+browse+call)")]
public async Task MixedWorkloadAsync()
{
// Read
await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_smallReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
// Write
var writeValues =
m_smallTestSet.Take(5).Select(nodeId => new WriteValue
{
NodeId = nodeId,
AttributeId = Attributes.Value,
Value = new DataValue(new Variant(UnsecureRandom.Shared.Next()))
}).ToArrayOf();
await Session.WriteAsync(
null,
writeValues,
CancellationToken.None
).ConfigureAwait(false);
// Browse
ArrayOf<BrowseDescription> nodesToBrowse =
[
new BrowseDescription
{
NodeId = ObjectIds.ObjectsFolder,
BrowseDirection = BrowseDirection.Forward,
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
IncludeSubtypes = true,
NodeClassMask = (uint)NodeClass.Object,
ResultMask = (uint)BrowseResultMask.All
}
];
await Session.BrowseAsync(
null,
null,
0,
nodesToBrowse,
CancellationToken.None
).ConfigureAwait(false);
// Call
ArrayOf<Variant> inputArguments = [new Variant(0u)];
ArrayOf<CallMethodRequest> requests =
[
new CallMethodRequest
{
ObjectId = ObjectIds.Server,
MethodId = MethodIds.Server_GetMonitoredItems,
InputArguments = inputArguments
}
];
await Session.CallAsync(
null,
requests,
CancellationToken.None
).ConfigureAwait(false);
}
/// <summary>
/// Benchmark: Read throughput - measures read operations per second.
/// Executes 100 read operations and calculates ops/sec from elapsed time.
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
/// </summary>
[Test]
[Order(850)]
[Benchmark(Description = "Read 100 ops (for throughput)")]
public async Task ReadThroughputAsync()
{
const int operationCount = 100;
for (int i = 0; i < operationCount; i++)
{
await Session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
m_smallReadValueIds,
CancellationToken.None
).ConfigureAwait(false);
}
// Throughput = 100 operations / Mean time in seconds
// Example: Mean=1000ms → Throughput = 100/1.0 = 100 ops/sec
}
/// <summary>
/// Benchmark: Write throughput - measures write operations per second.
/// Executes 100 write operations and calculates ops/sec from elapsed time.
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
/// </summary>
[Test]
[Order(851)]
[Benchmark(Description = "Write 100 ops (for throughput)")]
public async Task WriteThroughputAsync()
{
const int operationCount = 100;
for (int i = 0; i < operationCount; i++)
{
var writeValues =
m_smallTestSet.Select(nodeId => new WriteValue
{
NodeId = nodeId,
AttributeId = Attributes.Value,
Value = new DataValue(new Variant(i))
}).ToArrayOf();
await Session.WriteAsync(
null,
writeValues,
CancellationToken.None
).ConfigureAwait(false);
}
// Throughput = 100 operations / Mean time in seconds
}
/// <summary>
/// Benchmark: Browse throughput - measures browse operations per second.
/// Executes 100 browse operations and calculates ops/sec from elapsed time.
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
/// </summary>
[Test]
[Order(852)]
[Benchmark(Description = "Browse 100 ops (for throughput)")]
public async Task BrowseThroughputAsync()
{
ArrayOf<BrowseDescription> nodesToBrowse =
[
new BrowseDescription
{
NodeId = ObjectIds.ObjectsFolder,
BrowseDirection = BrowseDirection.Forward,
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
IncludeSubtypes = true,
NodeClassMask = (uint)NodeClass.Object | (uint)NodeClass.Variable,
ResultMask = (uint)BrowseResultMask.All
}
];
const int operationCount = 100;
for (int i = 0; i < operationCount; i++)
{
await Session.BrowseAsync(
null,
null,
0,
nodesToBrowse,
CancellationToken.None
).ConfigureAwait(false);
}
// Throughput = 100 operations / Mean time in seconds
}
/// <summary>
/// Benchmark: Call throughput - measures method call operations per second.
/// Executes 100 call operations and calculates ops/sec from elapsed time.
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
/// </summary>
[Test]
[Order(853)]
[Benchmark(Description = "Call 100 ops (for throughput)")]
public async Task CallThroughputAsync()
{
ArrayOf<Variant> inputArguments = [new Variant(0u)];
ArrayOf<CallMethodRequest> requests =
[
new CallMethodRequest
{
ObjectId = ObjectIds.Server,
MethodId = MethodIds.Server_GetMonitoredItems,
InputArguments = inputArguments
}
];
const int operationCount = 100;
for (int i = 0; i < operationCount; i++)
{
await Session.CallAsync(
null,
requests,
CancellationToken.None
).ConfigureAwait(false);
}
// Throughput = 100 operations / Mean time in seconds
}
/// <summary>
/// Test all available security policies to ensure benchmarks work with each.
/// This is not a benchmark but validates that all security policies can be tested.
/// </summary>
[Test]
[Order(900)]
[Category("SecurityPolicyValidation")]
public async Task TestAllSecurityPoliciesAsync()
{
// Use the same policy list as benchmarks (excludes None)
var policies = BenchPolicies().ToList();
var results = new Dictionary<string, bool>();
TestContext.Out.WriteLine($"Testing {policies.Count} security policies:");
foreach (string policyUri in policies)
{
string displayName = SecurityPolicies.Default.GetDisplayName(policyUri)!;
TestContext.Out.WriteLine($"\nTesting policy: {displayName} ({policyUri})");
try
{
ISession session = await ClientFixture.ConnectAsync(
ServerUrl,
policyUri
).ConfigureAwait(false);
Assert.That(session, Is.Not.Null, $"Failed to create session with {displayName}");
// Perform a basic read to verify the connection works
ReadResponse response = await session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
[
new ReadValueId
{
NodeId = VariableIds.Server_ServerStatus_CurrentTime,
AttributeId = Attributes.Value
}
],
CancellationToken.None
).ConfigureAwait(false);
Assert.That(response, Is.Not.Null);
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
session.Dispose();
results[displayName!] = true;
TestContext.Out.WriteLine($"✓ {displayName}: SUCCESS");
}
catch (Exception ex)
{
results[displayName!] = false;
TestContext.Out.WriteLine($"✗ {displayName}: FAILED - {ex.Message}");
}
}
// Summary
TestContext.Out.WriteLine("\n=== Summary ===");
int successful = results.Count(r => r.Value);
int failed = results.Count(r => !r.Value);
TestContext.Out.WriteLine($"Successful: {successful}/{results.Count}");
TestContext.Out.WriteLine($"Failed: {failed}/{results.Count}");
if (failed > 0)
{
TestContext.Out.WriteLine("\nFailed policies:");
foreach (KeyValuePair<string, bool> kvp in results.Where(r => !r.Value))
{
TestContext.Out.WriteLine($" - {kvp.Key}");
}
}
// Assert at least some policies work
Assert.That(successful, Is.GreaterThan(0), "No security policies were successful");
}
}
/// <summary>
/// <para>
/// Session-establishment benchmarks restricted to the RSA security-policy suites that are common with
/// OPC UA .NET Standard 1.5.378.
/// </para>
/// <para>
/// The common-with-1.5.378 set is Basic128Rsa15, Basic256, Basic256Sha256,
/// Aes128_Sha256_RsaOaep, and Aes256_Sha256_RsaPss. These were the RSA suites available in
/// 1.5.378, so this fixture excludes SecurityPolicies.None and the 2.0-only ECC/RSA-DH AES-GCM and
/// ChaCha20-Poly1305 suites to provide an apples-to-apples session-establishment geomean.
/// </para>
/// <para>
/// Run with:
/// dotnet run -c Release -f net10.0 -- --filter '*SecurityPolicySessionCommonWithV15378Benchmarks*'
/// </para>
/// </summary>
[TestFixture]
[Explicit]
[Category("Client")]
[Category("SecurityPolicyBenchmark")]
[Category("SecurityPolicyBenchmarkCommonWithV15378")]
[SetCulture("en-us")]
[SetUICulture("en-us")]
[Config(typeof(SecurityPolicyBenchmarkConfig))]
[MemoryDiagnoser]
public class SecurityPolicySessionCommonWithV15378Benchmarks
{
private ClientTestFramework m_framework;
private ArrayOf<EndpointDescription> m_endpoints;
/// <summary>
/// Security-policy URIs that exist in both OPC UA .NET Standard 1.5.378 and 2.0.
/// </summary>
public static IEnumerable<string> BenchPoliciesCommonWithV15378()
{
yield return SecurityPolicies.Basic128Rsa15;
yield return SecurityPolicies.Basic256;
yield return SecurityPolicies.Basic256Sha256;
yield return SecurityPolicies.Aes128_Sha256_RsaOaep;
yield return SecurityPolicies.Aes256_Sha256_RsaPss;
}
/// <summary>
/// The security policy used by the common-with-1.5.378 session-establishment benchmarks.
/// </summary>
[ParamsSource(nameof(BenchPoliciesCommonWithV15378))]
public string SecurityPolicy = SecurityPolicies.Basic256Sha256;
/// <summary>
/// Set up a Server and a Client instance.
/// </summary>
[OneTimeSetUp]
[GlobalSetup]
public async Task SetUpAsync()
{
if (m_framework != null)
{
return;
}
m_framework = new ClientTestFramework(Utils.UriSchemeOpcTcp)
{
SingleSession = false,
SupportsExternalServerUrl = true
};
await m_framework.OneTimeSetUpAsync().ConfigureAwait(false);
m_endpoints = await m_framework.ClientFixture
.GetEndpointsAsync(m_framework.ServerUrl)
.ConfigureAwait(false);
}
/// <summary>
/// Tear down the Server and the Client.
/// </summary>
[OneTimeTearDown]
[GlobalCleanup]
public async Task TearDownAsync()
{
if (m_framework == null)
{
return;
}
await m_framework.OneTimeTearDownAsync().ConfigureAwait(false);
m_framework = null!;
m_endpoints = default;
}
/// <summary>
/// Benchmark: Create and close session for policies common with OPC UA .NET Standard 1.5.378.
/// </summary>
[Test]
[Order(700)]
[Benchmark(Description = "Create and close session (common with 1.5.378)")]
public async Task CreateCloseSessionAsync()
{
ISession session = await m_framework.ClientFixture.ConnectAsync(
m_framework.ServerUrl,
SecurityPolicy,
m_endpoints
).ConfigureAwait(false);
Assert.That(session, Is.Not.Null);
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
session.Dispose();
}
/// <summary>
/// Benchmark: Create, read, and close session for policies common with OPC UA .NET Standard 1.5.378.
/// </summary>
[Test]
[Order(701)]
[Benchmark(Description = "Session lifecycle with read (common with 1.5.378)")]
public async Task SessionLifecycleWithReadAsync()
{
ISession session = await m_framework.ClientFixture.ConnectAsync(
m_framework.ServerUrl,
SecurityPolicy,
m_endpoints
).ConfigureAwait(false);
Assert.That(session, Is.Not.Null);
await session.ReadAsync(
null,
0,
TimestampsToReturn.Both,
[
new ReadValueId
{
NodeId = VariableIds.Server_ServerStatus_CurrentTime,
AttributeId = Attributes.Value
}
],
CancellationToken.None
).ConfigureAwait(false);
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
session.Dispose();
}
/// <summary>
/// Benchmark: Discover endpoints for policies common with OPC UA .NET Standard 1.5.378.
/// </summary>
[Test]
[Order(702)]
[Benchmark(Description = "Discover endpoints (common with 1.5.378)")]
public async Task DiscoverEndpointsAsync()
{
ArrayOf<EndpointDescription> endpoints = await m_framework.ClientFixture
.GetEndpointsAsync(m_framework.ServerUrl)
.ConfigureAwait(false);
Assert.That(endpoints.IsNull, Is.False);
}
}
}