mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-22 01:26:20 +08:00
9b034f8291
# Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1080 lines
39 KiB
C#
1080 lines
39 KiB
C#
/* ========================================================================
|
||
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
|
||
*
|
||
* OPC Foundation MIT License 1.00
|
||
*
|
||
* Permission is hereby granted, free of charge, to any person
|
||
* obtaining a copy of this software and associated documentation
|
||
* files (the "Software"), to deal in the Software without
|
||
* restriction, including without limitation the rights to use,
|
||
* copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||
* copies of the Software, and to permit persons to whom the
|
||
* Software is furnished to do so, subject to the following
|
||
* conditions:
|
||
*
|
||
* The above copyright notice and this permission notice shall be
|
||
* included in all copies or substantial portions of the Software.
|
||
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||
* OTHER DEALINGS IN THE SOFTWARE.
|
||
*
|
||
* The complete license agreement can be found here:
|
||
* http://opcfoundation.org/License/MIT/1.00/
|
||
* ======================================================================*/
|
||
|
||
using System;
|
||
using System.Collections.Generic;
|
||
using System.Linq;
|
||
using System.Threading;
|
||
using System.Threading.Tasks;
|
||
using BenchmarkDotNet.Attributes;
|
||
using BenchmarkDotNet.Configs;
|
||
using NUnit.Framework;
|
||
using Opc.Ua.Client.TestFramework;
|
||
using ISession = Opc.Ua.Client.ISession;
|
||
|
||
namespace Opc.Ua.Sessions.Tests
|
||
{
|
||
/// <summary>
|
||
/// Custom config for Security Policy Benchmarks that increases build timeout.
|
||
/// </summary>
|
||
public class SecurityPolicyBenchmarkConfig : ManualConfig
|
||
{
|
||
public SecurityPolicyBenchmarkConfig()
|
||
{
|
||
// Increase build timeout to 10 minutes for large dependency builds
|
||
WithOptions(ConfigOptions.DisableOptimizationsValidator);
|
||
WithBuildTimeout(TimeSpan.FromMinutes(10));
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// <para>
|
||
/// Benchmarks for measuring CPU, Memory, Latency, and Message Throughput across all Security Policies.
|
||
/// These benchmarks help detect performance regressions when changes are made to security-related code.
|
||
/// </para>
|
||
/// <para>Total: 171 benchmarks (19 methods × 9 security policies - None is excluded)</para>
|
||
/// <para>
|
||
/// USAGE:
|
||
/// cd tests/Opc.Ua.Client.Tests
|
||
/// </para>
|
||
/// <para>
|
||
/// Run all 171 benchmarks (takes ~30+ minutes):
|
||
/// dotnet run -c Release -f net10.0 -- --filter '*SecurityPolicyBenchmarks*' --job short
|
||
/// </para>
|
||
/// <para>
|
||
/// Run specific benchmark method across all security policies:
|
||
/// dotnet run -c Release -f net10.0 -- --filter '*ReadSmallMessageAsync*' --job short
|
||
/// dotnet run -c Release -f net10.0 -- --filter '*WriteSmallMessageAsync*' --job short
|
||
/// dotnet run -c Release -f net10.0 -- --filter '*BrowseAsync*' --job short
|
||
/// </para>
|
||
/// <para>
|
||
/// Run specific benchmark with specific policy:
|
||
/// To run only one policy, temporarily modify BenchPolicies() in ClientTestFramework.cs
|
||
/// to return only the desired policy, then run:
|
||
/// dotnet run -c Release -f net10.0 -- --filter '*ReadSmallMessageAsync*' --job short
|
||
/// </para>
|
||
/// <para>
|
||
/// Run as NUnit tests (faster, no separate build):
|
||
/// dotnet test -c Release -f net10.0 --filter 'FullyQualifiedName~SecurityPolicyBenchmarks'
|
||
/// dotnet test -c Release -f net10.0 --filter 'FullyQualifiedName~SecurityPolicyBenchmarks.ReadSmallMessageAsync'
|
||
/// </para>
|
||
/// <para>
|
||
/// View results:
|
||
/// cat BenchmarkDotNet.Artifacts/results/*.md
|
||
/// cat BenchmarkDotNet.Artifacts/results/*.csv
|
||
/// </para>
|
||
/// <para>
|
||
/// Available benchmark methods:
|
||
/// Latency benchmarks:
|
||
/// - ReadSmallMessageAsync, ReadMediumMessageAsync, ReadLargeMessageAsync
|
||
/// - WriteSmallMessageAsync
|
||
/// - BrowseAsync, BrowseMultipleNodesAsync
|
||
/// - CallMethodAsync
|
||
/// - CreateCloseSessionAsync, SessionLifecycleWithReadAsync
|
||
/// - DiscoverEndpointsAsync
|
||
/// - MixedWorkloadAsync
|
||
/// </para>
|
||
/// <para>
|
||
/// Throughput benchmarks (operations/second):
|
||
/// - ReadSmallMessageBurstAsync, ReadMediumMessageBurstAsync, ReadLargeMessageBurstAsync
|
||
/// - WriteSmallMessageBurstAsync
|
||
/// - ReadThroughputAsync, WriteThroughputAsync
|
||
/// - BrowseThroughputAsync, CallThroughputAsync
|
||
/// </para>
|
||
/// <para>
|
||
/// Available security policies (9 total, None is excluded):
|
||
/// - Basic128Rsa15, Basic256, Basic256Sha256
|
||
/// - Aes128_Sha256_RsaOaep, Aes256_Sha256_RsaPss
|
||
/// - ECC_nistP256, ECC_nistP384, ECC_brainpoolP256r1, ECC_brainpoolP384r1
|
||
/// </para>
|
||
/// <para>
|
||
/// THROUGHPUT CALCULATION:
|
||
/// Throughput benchmarks execute 100 operations and measure total time.
|
||
/// Calculate ops/sec using: Throughput = 100 / (Mean time in seconds)
|
||
/// </para>
|
||
/// <para>
|
||
/// Example from results:
|
||
/// Method: 'Read 100 ops (for throughput)'
|
||
/// SecurityPolicy: Basic128Rsa15
|
||
/// Mean: 140.8 ms = 0.1408 seconds
|
||
/// Throughput = 100 / 0.1408 = 710 ops/sec
|
||
/// </para>
|
||
/// </summary>
|
||
[TestFixture]
|
||
[Explicit]
|
||
[Category("Client")]
|
||
[Category("SecurityPolicyBenchmark")]
|
||
[SetCulture("en-us")]
|
||
[SetUICulture("en-us")]
|
||
[Config(typeof(SecurityPolicyBenchmarkConfig))]
|
||
[MemoryDiagnoser]
|
||
[DisassemblyDiagnoser]
|
||
public class SecurityPolicyBenchmarks : ClientTestFramework
|
||
{
|
||
private const int kMessageCount = 100;
|
||
private const int kSmallMessageNodeCount = 10;
|
||
private const int kMediumMessageNodeCount = 50;
|
||
private const int kLargeMessageNodeCount = 200;
|
||
|
||
private IList<NodeId> m_smallTestSet;
|
||
private IList<NodeId> m_mediumTestSet;
|
||
private IList<NodeId> m_largeTestSet;
|
||
private ArrayOf<ReadValueId> m_smallReadValueIds;
|
||
private ArrayOf<ReadValueId> m_mediumReadValueIds;
|
||
private ArrayOf<ReadValueId> m_largeReadValueIds;
|
||
|
||
public SecurityPolicyBenchmarks()
|
||
: base(Utils.UriSchemeOpcTcp)
|
||
{
|
||
SingleSession = false;
|
||
}
|
||
|
||
/// <summary>
|
||
/// Override to exclude None policy from benchmarks to avoid CI test failures.
|
||
/// Uses the base policy list so target-specific filtering is preserved.
|
||
/// </summary>
|
||
private static new IEnumerable<string> BenchPolicies()
|
||
{
|
||
foreach (string policyUri in Policies)
|
||
{
|
||
if (policyUri != SecurityPolicies.None)
|
||
{
|
||
yield return policyUri;
|
||
}
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// Set up a Server and a Client instance.
|
||
/// </summary>
|
||
[OneTimeSetUp]
|
||
public new async Task OneTimeSetUpAsync()
|
||
{
|
||
SupportsExternalServerUrl = true;
|
||
await base.OneTimeSetUpAsync().ConfigureAwait(false);
|
||
await EnsureCachedEndpointsAsync().ConfigureAwait(false);
|
||
await PrepareTestDataAsync().ConfigureAwait(false);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Tear down the Server and the Client.
|
||
/// </summary>
|
||
[OneTimeTearDown]
|
||
public new Task OneTimeTearDownAsync()
|
||
{
|
||
return base.OneTimeTearDownAsync();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Test setup.
|
||
/// </summary>
|
||
[SetUp]
|
||
public new async Task SetUpAsync()
|
||
{
|
||
await base.SetUpAsync().ConfigureAwait(false);
|
||
await PrepareTestDataAsync().ConfigureAwait(false);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Test teardown.
|
||
/// </summary>
|
||
[TearDown]
|
||
public new Task TearDownAsync()
|
||
{
|
||
return base.TearDownAsync();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Global Setup for benchmarks.
|
||
/// </summary>
|
||
[GlobalSetup]
|
||
public override void GlobalSetup()
|
||
{
|
||
base.GlobalSetup();
|
||
EnsureCachedEndpointsAsync().GetAwaiter().GetResult();
|
||
PrepareTestDataAsync().GetAwaiter().GetResult();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Global cleanup for benchmarks.
|
||
/// </summary>
|
||
[GlobalCleanup]
|
||
public override void GlobalCleanup()
|
||
{
|
||
base.GlobalCleanup();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Prepare test data sets for benchmarking.
|
||
/// </summary>
|
||
private async Task PrepareTestDataAsync()
|
||
{
|
||
if (Session == null)
|
||
{
|
||
return;
|
||
}
|
||
|
||
// Get available test nodes
|
||
IList<NodeId> allTestNodes = GetTestSetStatic(Session.NamespaceUris);
|
||
|
||
// Ensure we have enough nodes for testing
|
||
if (allTestNodes.Count < kLargeMessageNodeCount)
|
||
{
|
||
// If not enough static nodes, add simulation nodes
|
||
IList<NodeId> simNodes = GetTestSetSimulation(Session.NamespaceUris);
|
||
allTestNodes = [.. allTestNodes.Concat(simNodes).Take(kLargeMessageNodeCount)];
|
||
}
|
||
|
||
// Create test sets of different sizes
|
||
m_smallTestSet = [.. allTestNodes.Take(kSmallMessageNodeCount)];
|
||
m_mediumTestSet = [.. allTestNodes.Take(kMediumMessageNodeCount)];
|
||
m_largeTestSet = [.. allTestNodes.Take(kLargeMessageNodeCount)];
|
||
|
||
// Prepare ReadValueId collections
|
||
m_smallReadValueIds = [.. m_smallTestSet.Select(nodeId => new ReadValueId
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value
|
||
})];
|
||
|
||
m_mediumReadValueIds = [.. m_mediumTestSet.Select(nodeId => new ReadValueId
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value
|
||
})];
|
||
|
||
m_largeReadValueIds = [.. m_largeTestSet.Select(nodeId => new ReadValueId
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value
|
||
})];
|
||
|
||
// Verify we can read the nodes
|
||
await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_smallReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Ensures endpoint discovery is completed once during setup, outside measured session lifecycle iterations.
|
||
/// </summary>
|
||
private async Task EnsureCachedEndpointsAsync()
|
||
{
|
||
if (Endpoints.IsNull)
|
||
{
|
||
Endpoints = await ClientFixture.GetEndpointsAsync(ServerUrl).ConfigureAwait(false);
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read small message (10 nodes) - measures baseline performance.
|
||
/// Tests CPU and memory overhead for small messages with different security policies.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(100)]
|
||
[Benchmark(Baseline = true, Description = "Read 10 nodes")]
|
||
public async Task ReadSmallMessageAsync()
|
||
{
|
||
ReadResponse response = await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_smallReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
Assert.That(response.Results.Count, Is.EqualTo(m_smallReadValueIds.Count));
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read small messages repeatedly - measures sustained throughput.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(101)]
|
||
[Benchmark(Description = "Read 10 nodes x100 iterations")]
|
||
public async Task ReadSmallMessageBurstAsync()
|
||
{
|
||
for (int i = 0; i < kMessageCount; i++)
|
||
{
|
||
await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_smallReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read medium message (50 nodes) - measures typical workload performance.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(200)]
|
||
[Benchmark(Description = "Read 50 nodes")]
|
||
public async Task ReadMediumMessageAsync()
|
||
{
|
||
ReadResponse response = await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_mediumReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
Assert.That(response.Results.Count, Is.EqualTo(m_mediumReadValueIds.Count));
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read medium messages repeatedly - measures sustained medium-load throughput.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(201)]
|
||
[Benchmark(Description = "Read 50 nodes x100 iterations")]
|
||
public async Task ReadMediumMessageBurstAsync()
|
||
{
|
||
for (int i = 0; i < kMessageCount; i++)
|
||
{
|
||
await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_mediumReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read large message (200 nodes) - measures high-load performance.
|
||
/// Tests encryption/decryption overhead with larger payloads.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(300)]
|
||
[Benchmark(Description = "Read 200 nodes")]
|
||
public async Task ReadLargeMessageAsync()
|
||
{
|
||
ReadResponse response = await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_largeReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
Assert.That(response.Results.Count, Is.EqualTo(m_largeReadValueIds.Count));
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read large messages repeatedly - measures sustained high-load throughput.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(301)]
|
||
[Benchmark(Description = "Read 200 nodes x100 iterations")]
|
||
public async Task ReadLargeMessageBurstAsync()
|
||
{
|
||
for (int i = 0; i < kMessageCount; i++)
|
||
{
|
||
await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_largeReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Write small message (10 nodes) - measures write performance.
|
||
/// Tests CPU and memory overhead for write operations with different security policies.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(400)]
|
||
[Benchmark(Description = "Write 10 nodes")]
|
||
public async Task WriteSmallMessageAsync()
|
||
{
|
||
var writeValues =
|
||
m_smallTestSet.Select(nodeId => new WriteValue
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value,
|
||
Value = new DataValue(new Variant(UnsecureRandom.Shared.Next()))
|
||
}).ToArrayOf();
|
||
|
||
WriteResponse response = await Session.WriteAsync(
|
||
null,
|
||
writeValues,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Write small messages repeatedly - measures sustained write throughput.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(401)]
|
||
[Benchmark(Description = "Write 10 nodes x100 iterations")]
|
||
public async Task WriteSmallMessageBurstAsync()
|
||
{
|
||
for (int i = 0; i < kMessageCount; i++)
|
||
{
|
||
var writeValues =
|
||
m_smallTestSet.Select(nodeId => new WriteValue
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value,
|
||
Value = new DataValue(new Variant(i))
|
||
}).ToArrayOf();
|
||
|
||
await Session.WriteAsync(
|
||
null,
|
||
writeValues,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Browse operation - measures browse performance.
|
||
/// Tests how security policy affects browse operations and reference enumeration.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(500)]
|
||
[Benchmark(Description = "Browse Objects folder")]
|
||
public async Task BrowseAsync()
|
||
{
|
||
ArrayOf<BrowseDescription> nodesToBrowse =
|
||
[
|
||
new BrowseDescription
|
||
{
|
||
NodeId = ObjectIds.ObjectsFolder,
|
||
BrowseDirection = BrowseDirection.Forward,
|
||
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
|
||
IncludeSubtypes = true,
|
||
NodeClassMask = (uint)NodeClass.Object | (uint)NodeClass.Variable,
|
||
ResultMask = (uint)BrowseResultMask.All
|
||
}
|
||
];
|
||
|
||
BrowseResponse response = await Session.BrowseAsync(
|
||
null,
|
||
null,
|
||
0,
|
||
nodesToBrowse,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
Assert.That(response.Results.IsEmpty, Is.True);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Browse multiple nodes - measures browse throughput.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(501)]
|
||
[Benchmark(Description = "Browse 10 nodes")]
|
||
public async Task BrowseMultipleNodesAsync()
|
||
{
|
||
//await Task.Delay(5000);
|
||
var nodesToBrowse =
|
||
m_smallTestSet.Select(nodeId => new BrowseDescription
|
||
{
|
||
NodeId = nodeId,
|
||
BrowseDirection = BrowseDirection.Forward,
|
||
ReferenceTypeId = ReferenceTypeIds.References,
|
||
IncludeSubtypes = true,
|
||
NodeClassMask = 0,
|
||
ResultMask = (uint)BrowseResultMask.All
|
||
}).ToArrayOf();
|
||
|
||
BrowseResponse response = await Session.BrowseAsync(
|
||
null,
|
||
null,
|
||
0,
|
||
nodesToBrowse,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Call GetMonitoredItems method - measures method call performance.
|
||
/// Tests how security policy affects method call operations.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(600)]
|
||
[Benchmark(Description = "Call GetMonitoredItems method")]
|
||
public async Task CallMethodAsync()
|
||
{
|
||
ArrayOf<Variant> inputArguments =
|
||
[
|
||
new Variant(0u) // subscriptionId
|
||
];
|
||
|
||
ArrayOf<CallMethodRequest> requests =
|
||
[
|
||
new CallMethodRequest
|
||
{
|
||
ObjectId = ObjectIds.Server,
|
||
MethodId = MethodIds.Server_GetMonitoredItems,
|
||
InputArguments = inputArguments
|
||
}
|
||
];
|
||
|
||
CallResponse response = await Session.CallAsync(
|
||
null,
|
||
requests,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
Assert.That(response.Results.IsNull, Is.False);
|
||
Assert.That(response.Results.Count, Is.EqualTo(1));
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Create and close session - measures session establishment overhead.
|
||
/// This is critical for understanding the cost of security policy negotiation.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(700)]
|
||
[Benchmark(Description = "Create and close session")]
|
||
public async Task CreateCloseSessionAsync()
|
||
{
|
||
ISession session = await ClientFixture.ConnectAsync(
|
||
ServerUrl,
|
||
SecurityPolicy,
|
||
Endpoints
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(session, Is.Not.Null);
|
||
|
||
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
|
||
session.Dispose();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Create, use, and close session - measures full session lifecycle.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(701)]
|
||
[Benchmark(Description = "Session lifecycle with read")]
|
||
public async Task SessionLifecycleWithReadAsync()
|
||
{
|
||
ISession session = await ClientFixture.ConnectAsync(
|
||
ServerUrl,
|
||
SecurityPolicy,
|
||
Endpoints
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(session, Is.Not.Null);
|
||
|
||
// Perform a read operation
|
||
await session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_smallReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
|
||
session.Dispose();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Discover endpoints - tracks endpoint discovery cost independently from session establishment.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(702)]
|
||
[Benchmark(Description = "Discover endpoints")]
|
||
public async Task DiscoverEndpointsAsync()
|
||
{
|
||
ArrayOf<EndpointDescription> endpoints = await ClientFixture.GetEndpointsAsync(ServerUrl).ConfigureAwait(false);
|
||
|
||
Assert.That(endpoints.IsNull, Is.False);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Mixed operations - measures realistic workload performance.
|
||
/// Combines read, write, browse, and call operations to simulate real applications.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(800)]
|
||
[Benchmark(Description = "Mixed workload (read+write+browse+call)")]
|
||
public async Task MixedWorkloadAsync()
|
||
{
|
||
// Read
|
||
await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_smallReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
// Write
|
||
var writeValues =
|
||
m_smallTestSet.Take(5).Select(nodeId => new WriteValue
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value,
|
||
Value = new DataValue(new Variant(UnsecureRandom.Shared.Next()))
|
||
}).ToArrayOf();
|
||
|
||
await Session.WriteAsync(
|
||
null,
|
||
writeValues,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
// Browse
|
||
ArrayOf<BrowseDescription> nodesToBrowse =
|
||
[
|
||
new BrowseDescription
|
||
{
|
||
NodeId = ObjectIds.ObjectsFolder,
|
||
BrowseDirection = BrowseDirection.Forward,
|
||
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
|
||
IncludeSubtypes = true,
|
||
NodeClassMask = (uint)NodeClass.Object,
|
||
ResultMask = (uint)BrowseResultMask.All
|
||
}
|
||
];
|
||
|
||
await Session.BrowseAsync(
|
||
null,
|
||
null,
|
||
0,
|
||
nodesToBrowse,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
// Call
|
||
ArrayOf<Variant> inputArguments = [new Variant(0u)];
|
||
ArrayOf<CallMethodRequest> requests =
|
||
[
|
||
new CallMethodRequest
|
||
{
|
||
ObjectId = ObjectIds.Server,
|
||
MethodId = MethodIds.Server_GetMonitoredItems,
|
||
InputArguments = inputArguments
|
||
}
|
||
];
|
||
|
||
await Session.CallAsync(
|
||
null,
|
||
requests,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Read throughput - measures read operations per second.
|
||
/// Executes 100 read operations and calculates ops/sec from elapsed time.
|
||
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
|
||
/// </summary>
|
||
[Test]
|
||
[Order(850)]
|
||
[Benchmark(Description = "Read 100 ops (for throughput)")]
|
||
public async Task ReadThroughputAsync()
|
||
{
|
||
const int operationCount = 100;
|
||
for (int i = 0; i < operationCount; i++)
|
||
{
|
||
await Session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
m_smallReadValueIds,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
// Throughput = 100 operations / Mean time in seconds
|
||
// Example: Mean=1000ms → Throughput = 100/1.0 = 100 ops/sec
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Write throughput - measures write operations per second.
|
||
/// Executes 100 write operations and calculates ops/sec from elapsed time.
|
||
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
|
||
/// </summary>
|
||
[Test]
|
||
[Order(851)]
|
||
[Benchmark(Description = "Write 100 ops (for throughput)")]
|
||
public async Task WriteThroughputAsync()
|
||
{
|
||
const int operationCount = 100;
|
||
for (int i = 0; i < operationCount; i++)
|
||
{
|
||
var writeValues =
|
||
m_smallTestSet.Select(nodeId => new WriteValue
|
||
{
|
||
NodeId = nodeId,
|
||
AttributeId = Attributes.Value,
|
||
Value = new DataValue(new Variant(i))
|
||
}).ToArrayOf();
|
||
|
||
await Session.WriteAsync(
|
||
null,
|
||
writeValues,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
// Throughput = 100 operations / Mean time in seconds
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Browse throughput - measures browse operations per second.
|
||
/// Executes 100 browse operations and calculates ops/sec from elapsed time.
|
||
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
|
||
/// </summary>
|
||
[Test]
|
||
[Order(852)]
|
||
[Benchmark(Description = "Browse 100 ops (for throughput)")]
|
||
public async Task BrowseThroughputAsync()
|
||
{
|
||
ArrayOf<BrowseDescription> nodesToBrowse =
|
||
[
|
||
new BrowseDescription
|
||
{
|
||
NodeId = ObjectIds.ObjectsFolder,
|
||
BrowseDirection = BrowseDirection.Forward,
|
||
ReferenceTypeId = ReferenceTypeIds.HierarchicalReferences,
|
||
IncludeSubtypes = true,
|
||
NodeClassMask = (uint)NodeClass.Object | (uint)NodeClass.Variable,
|
||
ResultMask = (uint)BrowseResultMask.All
|
||
}
|
||
];
|
||
|
||
const int operationCount = 100;
|
||
for (int i = 0; i < operationCount; i++)
|
||
{
|
||
await Session.BrowseAsync(
|
||
null,
|
||
null,
|
||
0,
|
||
nodesToBrowse,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
// Throughput = 100 operations / Mean time in seconds
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Call throughput - measures method call operations per second.
|
||
/// Executes 100 call operations and calculates ops/sec from elapsed time.
|
||
/// BenchmarkDotNet will show the total time; ops/sec = 100 / (Mean in seconds).
|
||
/// </summary>
|
||
[Test]
|
||
[Order(853)]
|
||
[Benchmark(Description = "Call 100 ops (for throughput)")]
|
||
public async Task CallThroughputAsync()
|
||
{
|
||
ArrayOf<Variant> inputArguments = [new Variant(0u)];
|
||
ArrayOf<CallMethodRequest> requests =
|
||
[
|
||
new CallMethodRequest
|
||
{
|
||
ObjectId = ObjectIds.Server,
|
||
MethodId = MethodIds.Server_GetMonitoredItems,
|
||
InputArguments = inputArguments
|
||
}
|
||
];
|
||
|
||
const int operationCount = 100;
|
||
for (int i = 0; i < operationCount; i++)
|
||
{
|
||
await Session.CallAsync(
|
||
null,
|
||
requests,
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
}
|
||
// Throughput = 100 operations / Mean time in seconds
|
||
}
|
||
|
||
/// <summary>
|
||
/// Test all available security policies to ensure benchmarks work with each.
|
||
/// This is not a benchmark but validates that all security policies can be tested.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(900)]
|
||
[Category("SecurityPolicyValidation")]
|
||
public async Task TestAllSecurityPoliciesAsync()
|
||
{
|
||
// Use the same policy list as benchmarks (excludes None)
|
||
var policies = BenchPolicies().ToList();
|
||
var results = new Dictionary<string, bool>();
|
||
|
||
TestContext.Out.WriteLine($"Testing {policies.Count} security policies:");
|
||
|
||
foreach (string policyUri in policies)
|
||
{
|
||
string displayName = SecurityPolicies.Default.GetDisplayName(policyUri)!;
|
||
TestContext.Out.WriteLine($"\nTesting policy: {displayName} ({policyUri})");
|
||
|
||
try
|
||
{
|
||
ISession session = await ClientFixture.ConnectAsync(
|
||
ServerUrl,
|
||
policyUri
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(session, Is.Not.Null, $"Failed to create session with {displayName}");
|
||
|
||
// Perform a basic read to verify the connection works
|
||
ReadResponse response = await session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
[
|
||
new ReadValueId
|
||
{
|
||
NodeId = VariableIds.Server_ServerStatus_CurrentTime,
|
||
AttributeId = Attributes.Value
|
||
}
|
||
],
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(response, Is.Not.Null);
|
||
|
||
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
|
||
session.Dispose();
|
||
|
||
results[displayName!] = true;
|
||
TestContext.Out.WriteLine($"✓ {displayName}: SUCCESS");
|
||
}
|
||
catch (Exception ex)
|
||
{
|
||
results[displayName!] = false;
|
||
TestContext.Out.WriteLine($"✗ {displayName}: FAILED - {ex.Message}");
|
||
}
|
||
}
|
||
|
||
// Summary
|
||
TestContext.Out.WriteLine("\n=== Summary ===");
|
||
int successful = results.Count(r => r.Value);
|
||
int failed = results.Count(r => !r.Value);
|
||
TestContext.Out.WriteLine($"Successful: {successful}/{results.Count}");
|
||
TestContext.Out.WriteLine($"Failed: {failed}/{results.Count}");
|
||
|
||
if (failed > 0)
|
||
{
|
||
TestContext.Out.WriteLine("\nFailed policies:");
|
||
foreach (KeyValuePair<string, bool> kvp in results.Where(r => !r.Value))
|
||
{
|
||
TestContext.Out.WriteLine($" - {kvp.Key}");
|
||
}
|
||
}
|
||
|
||
// Assert at least some policies work
|
||
Assert.That(successful, Is.GreaterThan(0), "No security policies were successful");
|
||
}
|
||
}
|
||
|
||
/// <summary>
|
||
/// <para>
|
||
/// Session-establishment benchmarks restricted to the RSA security-policy suites that are common with
|
||
/// OPC UA .NET Standard 1.5.378.
|
||
/// </para>
|
||
/// <para>
|
||
/// The common-with-1.5.378 set is Basic128Rsa15, Basic256, Basic256Sha256,
|
||
/// Aes128_Sha256_RsaOaep, and Aes256_Sha256_RsaPss. These were the RSA suites available in
|
||
/// 1.5.378, so this fixture excludes SecurityPolicies.None and the 2.0-only ECC/RSA-DH AES-GCM and
|
||
/// ChaCha20-Poly1305 suites to provide an apples-to-apples session-establishment geomean.
|
||
/// </para>
|
||
/// <para>
|
||
/// Run with:
|
||
/// dotnet run -c Release -f net10.0 -- --filter '*SecurityPolicySessionCommonWithV15378Benchmarks*'
|
||
/// </para>
|
||
/// </summary>
|
||
[TestFixture]
|
||
[Explicit]
|
||
[Category("Client")]
|
||
[Category("SecurityPolicyBenchmark")]
|
||
[Category("SecurityPolicyBenchmarkCommonWithV15378")]
|
||
[SetCulture("en-us")]
|
||
[SetUICulture("en-us")]
|
||
[Config(typeof(SecurityPolicyBenchmarkConfig))]
|
||
[MemoryDiagnoser]
|
||
public class SecurityPolicySessionCommonWithV15378Benchmarks
|
||
{
|
||
private ClientTestFramework m_framework;
|
||
private ArrayOf<EndpointDescription> m_endpoints;
|
||
|
||
/// <summary>
|
||
/// Security-policy URIs that exist in both OPC UA .NET Standard 1.5.378 and 2.0.
|
||
/// </summary>
|
||
public static IEnumerable<string> BenchPoliciesCommonWithV15378()
|
||
{
|
||
yield return SecurityPolicies.Basic128Rsa15;
|
||
yield return SecurityPolicies.Basic256;
|
||
yield return SecurityPolicies.Basic256Sha256;
|
||
yield return SecurityPolicies.Aes128_Sha256_RsaOaep;
|
||
yield return SecurityPolicies.Aes256_Sha256_RsaPss;
|
||
}
|
||
|
||
/// <summary>
|
||
/// The security policy used by the common-with-1.5.378 session-establishment benchmarks.
|
||
/// </summary>
|
||
[ParamsSource(nameof(BenchPoliciesCommonWithV15378))]
|
||
public string SecurityPolicy = SecurityPolicies.Basic256Sha256;
|
||
|
||
/// <summary>
|
||
/// Set up a Server and a Client instance.
|
||
/// </summary>
|
||
[OneTimeSetUp]
|
||
[GlobalSetup]
|
||
public async Task SetUpAsync()
|
||
{
|
||
if (m_framework != null)
|
||
{
|
||
return;
|
||
}
|
||
|
||
m_framework = new ClientTestFramework(Utils.UriSchemeOpcTcp)
|
||
{
|
||
SingleSession = false,
|
||
SupportsExternalServerUrl = true
|
||
};
|
||
|
||
await m_framework.OneTimeSetUpAsync().ConfigureAwait(false);
|
||
m_endpoints = await m_framework.ClientFixture
|
||
.GetEndpointsAsync(m_framework.ServerUrl)
|
||
.ConfigureAwait(false);
|
||
}
|
||
|
||
/// <summary>
|
||
/// Tear down the Server and the Client.
|
||
/// </summary>
|
||
[OneTimeTearDown]
|
||
[GlobalCleanup]
|
||
public async Task TearDownAsync()
|
||
{
|
||
if (m_framework == null)
|
||
{
|
||
return;
|
||
}
|
||
|
||
await m_framework.OneTimeTearDownAsync().ConfigureAwait(false);
|
||
m_framework = null!;
|
||
m_endpoints = default;
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Create and close session for policies common with OPC UA .NET Standard 1.5.378.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(700)]
|
||
[Benchmark(Description = "Create and close session (common with 1.5.378)")]
|
||
public async Task CreateCloseSessionAsync()
|
||
{
|
||
ISession session = await m_framework.ClientFixture.ConnectAsync(
|
||
m_framework.ServerUrl,
|
||
SecurityPolicy,
|
||
m_endpoints
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(session, Is.Not.Null);
|
||
|
||
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
|
||
session.Dispose();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Create, read, and close session for policies common with OPC UA .NET Standard 1.5.378.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(701)]
|
||
[Benchmark(Description = "Session lifecycle with read (common with 1.5.378)")]
|
||
public async Task SessionLifecycleWithReadAsync()
|
||
{
|
||
ISession session = await m_framework.ClientFixture.ConnectAsync(
|
||
m_framework.ServerUrl,
|
||
SecurityPolicy,
|
||
m_endpoints
|
||
).ConfigureAwait(false);
|
||
|
||
Assert.That(session, Is.Not.Null);
|
||
|
||
await session.ReadAsync(
|
||
null,
|
||
0,
|
||
TimestampsToReturn.Both,
|
||
[
|
||
new ReadValueId
|
||
{
|
||
NodeId = VariableIds.Server_ServerStatus_CurrentTime,
|
||
AttributeId = Attributes.Value
|
||
}
|
||
],
|
||
CancellationToken.None
|
||
).ConfigureAwait(false);
|
||
|
||
await session.CloseAsync(CancellationToken.None).ConfigureAwait(false);
|
||
session.Dispose();
|
||
}
|
||
|
||
/// <summary>
|
||
/// Benchmark: Discover endpoints for policies common with OPC UA .NET Standard 1.5.378.
|
||
/// </summary>
|
||
[Test]
|
||
[Order(702)]
|
||
[Benchmark(Description = "Discover endpoints (common with 1.5.378)")]
|
||
public async Task DiscoverEndpointsAsync()
|
||
{
|
||
ArrayOf<EndpointDescription> endpoints = await m_framework.ClientFixture
|
||
.GetEndpointsAsync(m_framework.ServerUrl)
|
||
.ConfigureAwait(false);
|
||
|
||
Assert.That(endpoints.IsNull, Is.False);
|
||
}
|
||
}
|
||
}
|