Fix concurrent channel reconnect certificate leaks (#4356)

# Description

Fix the intermittent one-certificate leak reported by the macOS Sessions
suite.

The leak had two concurrent ownership paths:

1. `ChannelEntry.OpenInitialAsync` could complete after a reconnect
cycle had already installed a
newer transport. The initial-open path overwrote `m_underlying`,
orphaning the reconnect
   transport and its certificate handles.
2. `UaSCUaBinaryTransportChannel.ReconnectAsync` did not dispose a newly
created inner channel when
`ConnectAsync` failed. Initial-open cleanup could also be skipped when
it reused the caller's
   already-cancelled token.

The fix keeps the transport installed first, closes and disposes the
losing candidate, disposes
failed reconnect candidates, and performs failed initial-open cleanup
independently of caller
cancellation.

This also adds opt-in Release allocation tracking for outstanding
`Certificate` handles. Set
`OPCUA_CERTIFICATE_LEAK_TRACKING=1` to include allocation stacks and
NUnit suite attribution in leak
failures. The contributor workflow is documented in
`docs/DeveloperGuide.md`.

## Tests and verification

- Added a deterministic `ChannelEntry` race test that orchestrates
concurrent initial open and
  reconnect, then verifies the losing transport is closed and disposed.
- Added deterministic UA-binary transport tests for failed reconnect
certificate cleanup and retry
  after a cancelled initial open.
- Pre-fix macOS diagnostic run reproduced 3 leaks in 15 full Sessions
runs:

https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/33329288604
- Post-fix targeted macOS run passed 50/50 exact reproducer executions:

https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/33334875967
- Post-fix full macOS run passed 15/15 complete Sessions suites:

https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/33335718790
- Full local net10 Sessions suite passed: 775 passed, 379 skipped, 1,154
total.
- Focused ownership regressions passed on net10 and net48.
- `Opc.Ua.Core` built warning-free across all supported target
frameworks.
- Final branch Build/Test, CodeQL, and NuGet workflows passed.

The temporary reproduction workflow and trigger tags were removed after
verification.

## Related Issues

- Fixes #4338

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [x] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Marc Schier
2026-08-31 15:04:50 +02:00
committed by GitHub
parent 57db405446
commit 8d3eccd41c
15 changed files with 1073 additions and 174 deletions
+15 -1
View File
@@ -68,6 +68,21 @@ Conventions and requirements:
- **Coverage.** Coverage is measured with **Coverlet** and must not regress; every non-application, non-test project should stay at or above **80 %**. Two gates enforce this in CI — see [Continuous integration](#continuous-integration).
- **Integration tests.** Client/server and pub/sub features need integration tests as well as unit tests. A feature library's integration tests normally live with its unit tests in `<Component>.Tests`, for example `Opc.Ua.Robotics.Tests`, and every test project name ends in `.Tests`. Split integration tests into a separate project only when they run long, destabilise the unit tests, or the suite needs further division. Keep them deterministic: allocate a free port per fixture rather than hard-coding one, wait on the actual signal instead of using `Thread.Sleep` as a synchronisation primitive, and dispose every session, subscription and server in teardown including on failure. A flaky integration test is worse than none.
- **Test output.** Do not write per-test diagnostics to `TestContext.Out` (or the console) unconditionally. The NUnit adapter forwards every captured line to the test runner as its own message over the socket it shares with the test host, so output that is harmless in one test becomes a bottleneck when a data-driven fixture repeats it thousands of times — it inflates the published results artifact, slows the run, and can wedge that socket until the CI job times out with no output at all (issue #4213). Buffer the dump and emit it only when the test does not pass; `EncoderCommon.TestOutput` in [`tests/Opc.Ua.Core.TestFramework/EncoderCommon.cs`](../tests/Opc.Ua.Core.TestFramework/EncoderCommon.cs) does exactly that and is the pattern to copy.
- **Certificate leak diagnostics.** Set `OPCUA_CERTIFICATE_LEAK_TRACKING=1` before starting a
test process to capture allocation stacks for `Certificate` handles that are not explicitly
disposed. The assembly-level leak detector includes live and unreachable outstanding handles,
their allocation stacks, and the NUnit fixture where available. Tracking is enabled by default
in Debug builds and opt-in in Release because stack capture adds per-certificate overhead. The
equivalent AppContext switch is `Opc.Ua.Security.Certificates.CertificateLeakTracking`; set it
before the first `Certificate` is created. For example, in PowerShell:
```powershell
$env:OPCUA_CERTIFICATE_LEAK_TRACKING = '1'
dotnet test `
tests\Opc.Ua.Sessions.Tests\Opc.Ua.Sessions.Tests.csproj `
-f net10.0 `
-p:CustomTestTarget=net10.0
```
- **Before a pull request** the `UA.slnx` suite must pass on at least **.NET Framework 4.8** and **.NET 10.0**.
- **Testing a specific target framework.** The libraries multi-target, but the test executables run on one framework at a time. To run the suite against a non-default framework, set `CustomTestTarget` (supported values: `netstandard2.0`, `netstandard2.1`, `net472`, `net48`, `net8.0`, `net9.0`, `net10.0`). The batch file [`tests/customtest.bat`](../tests/customtest.bat) cleans, restores, and runs the tests for a chosen target; in Visual Studio, uncomment and set the `CustomTestTarget` property in [`targets.props`](../targets.props). A clean build for the target is recommended when switching.
- **CI matrix.** The pull-request gate runs the test suite on **net48** and **net10.0**, and compiles the solution for *every* supported target framework; the remaining test matrices (Debug, .NET 9/8, .NET Framework 4.7.2, netstandard) run in scheduled or manual CI. Fix all failing, flaky, and CodeQL findings in the pipelines. See [Continuous integration](#continuous-integration).
@@ -418,4 +433,3 @@ Omit `-BaseRef` to check only the project floor, and `-SummaryPath` to skip the
- [Diagnostics](Diagnostics.md) — telemetry context, logging runtime, metrics, audit events, server diagnostics nodes, and packet capture.
- [Dependency Injection](DependencyInjection.md), [Certificates](Certificates.md) / [Certificate Manager](CertificateManager.md), [NativeAOT](NativeAoT.md), [Migration Guide](MigrationGuide.md), [What's New in 2.0](WhatsNewIn2.0.md).
- [Fuzz testing](../fuzzing/Fuzzing.md).
@@ -140,25 +140,50 @@ namespace Opc.Ua
clientCertificate, clientCertificateChain, ct)
.ConfigureAwait(false);
bool entryClosed;
bool channelInstalled;
bool recordActiveMetric;
lock (m_lock)
{
entryClosed = m_state is ChannelState.Closed or ChannelState.Faulted;
if (!entryClosed)
channelInstalled = !entryClosed && m_underlying == null;
if (channelInstalled)
{
m_underlying = channel;
m_clientCertificateVersion = clientCertificateVersion;
}
recordActiveMetric = !entryClosed && !m_activeMetricRecorded;
if (recordActiveMetric)
{
m_activeMetricRecorded = true;
}
}
if (entryClosed)
{
await CloseTransportBestEffortAsync(channel).ConfigureAwait(false);
OwnerManager.OnEntryClosed(this, ChannelCloseReason.Faulted);
throw ServiceResultException.Create(
StatusCodes.BadSecureChannelClosed,
"Channel is {0}.",
State);
}
OwnerManager.RecordChannelActiveChanged(this, 1);
if (recordActiveMetric)
{
OwnerManager.RecordChannelActiveChanged(this, 1);
}
if (!channelInstalled)
{
// A reconnect cycle can start while this initial open awaits
// network I/O. If that cycle installs a newer transport first,
// keep it and dispose this losing candidate instead of
// overwriting the only reference to the reconnect transport.
await CloseTransportBestEffortAsync(channel).ConfigureAwait(false);
OwnerManager.OnEntryClosed(this, ChannelCloseReason.Faulted);
return;
}
TransitionTo(ChannelState.Ready, error: null, attempt: 0);
SignalReady();
}
@@ -1376,7 +1401,8 @@ namespace Opc.Ua
public bool FatalForChannel;
}
private const string kReconnectOutcomeSuccess = "success"; private const string kReconnectOutcomeTransientFailure = "transient-failure";
private const string kReconnectOutcomeSuccess = "success";
private const string kReconnectOutcomeTransientFailure = "transient-failure";
private const string kReconnectOutcomeFatalChannel = "fatal-channel";
private const string kReconnectOutcomePolicyExhausted = "policy-exhausted";
private readonly Lock m_lock = new();
@@ -210,6 +210,7 @@ namespace Opc.Ua.Bindings
throw BadNotConnected();
}
UaSCUaBinaryClientChannel? previousChannel = null;
UaSCUaBinaryClientChannel? newChannel = null;
await m_connecting.WaitAsync(ct).ConfigureAwait(false);
try
{
@@ -223,7 +224,7 @@ namespace Opc.Ua.Bindings
// in the finally block after the new channel is connected.
previousChannel = m_channel;
m_channel = null;
UaSCUaBinaryClientChannel newChannel = CreateChannel(m_telemetry, connection);
newChannel = CreateChannel(m_telemetry, connection);
// Connect the new channel
await newChannel.ConnectAsync(
@@ -232,12 +233,18 @@ namespace Opc.Ua.Bindings
ct).ConfigureAwait(false);
m_channel = newChannel;
newChannel = null;
m_logger.UaSCTransportLog1(m_url);
}
finally
{
m_connecting.Release(); // Give access again to the new channel
// A failed connect never transfers the candidate to m_channel.
// Dispose it here so the certificate AddRef handles created by
// CreateChannel are always released.
newChannel?.Dispose();
// close previous channel.
if (previousChannel != null)
{
@@ -424,7 +431,10 @@ namespace Opc.Ua.Bindings
}
catch
{
await m_connecting.WaitAsync(ct).ConfigureAwait(false);
// Cleanup must not depend on the token that caused the connect
// attempt to fail; otherwise cancellation skips disposal of the
// channel's certificate handles and settings.
await m_connecting.WaitAsync(CancellationToken.None).ConfigureAwait(false);
try
{
Debug.Assert(m_channel == channel,
@@ -15,6 +15,7 @@
<InternalsVisibleTo Include="Opc.Ua.Security.Certificates.Tests" />
<InternalsVisibleTo Include="Opc.Ua.Core" />
<InternalsVisibleTo Include="Opc.Ua.Core.Tests" />
<InternalsVisibleTo Include="Opc.Ua.Test.Common" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="System.Diagnostics.DiagnosticSource" />
@@ -28,17 +28,14 @@
* ======================================================================*/
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Globalization;
using System.IO;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;
using System.Globalization;
using System.Threading;
using System.Collections.Concurrent;
using System.Text;
#if DEBUG
using System.Collections.Generic;
#endif
using System.Threading;
namespace Opc.Ua.Security.Certificates
{
@@ -64,9 +61,7 @@ namespace Opc.Ua.Security.Certificates
m_core = new CertificateCore(
X509CertificateLoader.LoadCertificate(rawData));
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
#if NET6_0_OR_GREATER
@@ -79,9 +74,7 @@ namespace Opc.Ua.Security.Certificates
m_core = new CertificateCore(
X509CertificateLoader.LoadCertificate(rawData));
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
#endif
@@ -104,9 +97,7 @@ namespace Opc.Ua.Security.Certificates
m_core = new CertificateCore(
X509CertificateLoader.LoadCertificate(File.ReadAllBytes(fileName)));
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
/// <summary>
@@ -125,9 +116,7 @@ namespace Opc.Ua.Security.Certificates
m_core = new CertificateCore(X509CertificateLoader.LoadPkcs12(
rawData, password, keyStorageFlags));
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
/// <summary>
@@ -151,9 +140,7 @@ namespace Opc.Ua.Security.Certificates
m_core = new CertificateCore(X509CertificateLoader.LoadPkcs12(
File.ReadAllBytes(fileName), password, keyStorageFlags));
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
/// <summary>
@@ -168,9 +155,7 @@ namespace Opc.Ua.Security.Certificates
m_core = new CertificateCore(certificate ??
throw new ArgumentNullException(nameof(certificate)));
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
/// <summary>
@@ -197,9 +182,7 @@ namespace Opc.Ua.Security.Certificates
detachedPrivateKey,
ownsDetachedPrivateKey);
Interlocked.Increment(ref s_instancesCreated);
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
/// <summary>
@@ -212,9 +195,7 @@ namespace Opc.Ua.Security.Certificates
private Certificate(CertificateCore core)
{
m_core = core;
#if DEBUG
Track();
#endif
InitializeLeakTracking();
}
/// <summary>
@@ -424,12 +405,11 @@ namespace Opc.Ua.Security.Certificates
m_core.Release();
// Only this handle has been finalised; suppress its finalizer
// (the DEBUG leak reporter). Other handles over the same core
// remain finalizable until they too are disposed.
#pragma warning disable CA1816 // Dispose methods should call SuppressFinalize
GC.SuppressFinalize(this);
#pragma warning restore CA1816 // Dispose methods should call SuppressFinalize
if (m_allocationInfo != null)
{
s_allocationTracker.TryRemove(m_allocationInfo.AllocationId, out _);
m_allocationInfo = null;
}
}
/// <summary>
@@ -752,115 +732,211 @@ namespace Opc.Ua.Security.Certificates
return new Certificate(m_core);
}
#if DEBUG
/// <summary>
/// Track the allocation
/// Whether per-certificate allocation tracking is enabled for this process.
/// </summary>
private void Track()
internal static bool LeakTrackingEnabled => s_leakTrackingEnabled;
/// <summary>
/// Gets or sets the diagnostic scope captured by newly allocated handles.
/// </summary>
internal static string? LeakTrackingScope
{
// Cache the allocation info on the instance so the finalizer
// can report it even after the static tracker has lost the
// weak reference.
m_allocationInfo = new CertificateAllocationInfo(
this,
new System.Diagnostics.StackTrace(true).ToString(),
X509.Thumbprint);
s_allocationTracker.Add(m_allocationInfo);
get => s_leakTrackingScope.Value;
set => s_leakTrackingScope.Value = value;
}
/// <summary>
/// Detects leaked certificates: a handle that was finalized without
/// being disposed (its reference to the shared core was never
/// released). Only compiled in DEBUG builds.
/// Enables allocation tracking for this handle when requested.
/// </summary>
#pragma warning disable CA1063 // Implement IDisposable Correctly
~Certificate()
#pragma warning restore CA1063 // Implement IDisposable Correctly
private void InitializeLeakTracking()
{
if (m_disposed == 0 && m_allocationInfo != null)
if (s_leakTrackingEnabled)
{
s_finalizedWithLeakedRef.Add(m_allocationInfo);
Track();
}
}
private CertificateAllocationInfo? m_allocationInfo;
/// <summary>
/// Tracks the allocation until this handle is disposed.
/// </summary>
private void Track()
{
long allocationId = Interlocked.Increment(ref s_nextAllocationId);
m_allocationInfo = new CertificateAllocationInfo(
allocationId,
this,
new System.Diagnostics.StackTrace(fNeedFileInfo: false).ToString(),
s_leakTrackingScope.Value);
s_allocationTracker[allocationId] = m_allocationInfo;
}
/// <summary>
/// Captures allocation context for leak-detection diagnostics.
/// </summary>
internal sealed class CertificateAllocationInfo
{
public WeakReference<Certificate> Reference { get; }
public string StackTrace { get; }
public string? Thumbprint { get; }
public DateTime CreatedAt { get; }
/// <summary>
/// Creates allocation diagnostics for a certificate handle.
/// </summary>
/// <param name="allocationId">Unique allocation identifier.</param>
/// <param name="certificate">The allocated certificate handle.</param>
/// <param name="stackTrace">Allocation stack trace.</param>
/// <param name="fixtureName">Optional test fixture attribution.</param>
public CertificateAllocationInfo(
long allocationId,
Certificate certificate,
string stackTrace,
string? thumbprint)
string? fixtureName)
{
AllocationId = allocationId;
Reference = new WeakReference<Certificate>(certificate);
StackTrace = stackTrace;
Thumbprint = thumbprint;
FixtureName = fixtureName;
CreatedAt = DateTime.UtcNow;
}
/// <summary>
/// Gets the unique allocation identifier.
/// </summary>
public long AllocationId { get; }
/// <summary>
/// Gets a weak reference to the allocated certificate handle.
/// </summary>
public WeakReference<Certificate> Reference { get; }
/// <summary>
/// Gets the allocation stack trace.
/// </summary>
public string StackTrace { get; }
/// <summary>
/// Gets the optional test fixture attribution.
/// </summary>
public string? FixtureName { get; }
/// <summary>
/// Gets the UTC allocation time.
/// </summary>
public DateTime CreatedAt { get; }
}
/// <summary>
/// Use a list of weak references for live-leak diagnostics.
/// ConditionalWeakTable doesn't expose enumeration on .NET
/// Framework, and we want the per-instance list anyway.
/// </summary>
private static readonly ConcurrentBag<CertificateAllocationInfo> s_allocationTracker = [];
/// <summary>
/// Set of allocation infos for Certificates that were finalised
/// while still holding a positive refcount (a real leak —
/// someone called AddRef without a matching Dispose). Cached so
/// the finalizer can record it before the instance dies.
/// </summary>
private static readonly ConcurrentBag<CertificateAllocationInfo> s_finalizedWithLeakedRef = [];
/// <summary>
/// Dumps allocation info for live <see cref="Certificate"/>
/// instances that are still reachable. Useful in tests to
/// surface the call site that created a leaking certificate.
/// </summary>
public static IEnumerable<(string Thumbprint, int RefCount, DateTime CreatedAt, string StackTrace)> EnumerateLiveCertificates()
internal static IEnumerable<(
string Thumbprint,
int RefCount,
DateTime CreatedAt,
string StackTrace,
string? FixtureName)>
EnumerateLiveCertificates()
{
foreach (CertificateAllocationInfo info in s_allocationTracker)
foreach (CertificateAllocationInfo info in s_allocationTracker.Values)
{
if (info.Reference.TryGetTarget(out Certificate? cert))
if (info.Reference.TryGetTarget(out Certificate? cert) &&
Volatile.Read(ref cert.m_disposed) == 0 &&
cert.m_core.RefCount > 0)
{
yield return (
info.Thumbprint ?? "(no thumbprint)",
GetThumbprintForDiagnostics(cert),
cert.m_core.RefCount,
info.CreatedAt,
info.StackTrace);
info.StackTrace,
info.FixtureName);
}
}
}
/// <summary>
/// Dumps allocation info for <see cref="Certificate"/> instances
/// that were finalized while still holding a positive refcount
/// (i.e., AddRef without matching Dispose).
/// Reads the thumbprint without allowing a concurrent disposal to abort a leak dump.
/// </summary>
public static IEnumerable<(string Thumbprint, DateTime CreatedAt, string StackTrace)> EnumerateFinalizedLeakedCertificates()
private static string GetThumbprintForDiagnostics(Certificate certificate)
{
foreach (CertificateAllocationInfo info in s_finalizedWithLeakedRef)
try
{
yield return (
info.Thumbprint ?? "(no thumbprint)",
info.CreatedAt,
info.StackTrace);
return certificate.Thumbprint ?? "(no thumbprint)";
}
catch (CryptographicException)
{
return "(unavailable after disposal)";
}
}
#endif
private static long s_instancesCreated;
private static long s_instancesDisposed;
/// <summary>
/// Dumps allocation info for undisposed <see cref="Certificate"/>
/// handles that are no longer reachable.
/// </summary>
internal static IEnumerable<(
DateTime CreatedAt,
string StackTrace,
string? FixtureName)>
EnumerateUnreachableUndisposedCertificates()
{
foreach (CertificateAllocationInfo info in s_allocationTracker.Values)
{
if (!info.Reference.TryGetTarget(out _))
{
yield return (
info.CreatedAt,
info.StackTrace,
info.FixtureName);
}
}
}
/// <summary>
/// Resolves whether allocation tracking is enabled.
/// </summary>
private static bool ResolveLeakTrackingEnabled()
{
bool defaultValue;
#if DEBUG
defaultValue = true;
#else
defaultValue = false;
#endif
bool switchFound = AppContext.TryGetSwitch(
c_leakTrackingSwitchName,
out bool switchValue);
return ResolveLeakTrackingEnabled(
switchFound,
switchValue,
Environment.GetEnvironmentVariable(c_leakTrackingEnvironmentVariable),
defaultValue);
}
/// <summary>
/// Resolves the allocation-tracking setting from its inputs.
/// </summary>
internal static bool ResolveLeakTrackingEnabled(
bool switchFound,
bool switchValue,
string? environmentValue,
bool defaultValue)
{
if (switchFound)
{
return switchValue;
}
if (string.Equals(environmentValue, "1", StringComparison.Ordinal) ||
string.Equals(environmentValue, bool.TrueString, StringComparison.OrdinalIgnoreCase))
{
return true;
}
if (string.Equals(environmentValue, "0", StringComparison.Ordinal) ||
string.Equals(environmentValue, bool.FalseString, StringComparison.OrdinalIgnoreCase))
{
return false;
}
return defaultValue;
}
/// <summary>
/// Total number of <see cref="Certificate"/> instances created
@@ -888,22 +964,37 @@ namespace Opc.Ua.Security.Certificates
{
Interlocked.Exchange(ref s_instancesCreated, 0);
Interlocked.Exchange(ref s_instancesDisposed, 0);
s_allocationTracker.Clear();
}
#if DEBUG
/// <summary>
/// Test-only hook used by the leak-detector self-tests to account
/// for a certificate that is deliberately abandoned (never disposed)
/// in order to exercise the finalizer-based leak tracking. Balances
/// in order to exercise unreachable-handle tracking. Balances
/// the global leak counters so the intentional leak does not trip
/// the assembly-level leak assertion. DEBUG-only and visible to
/// friend test assemblies via <c>InternalsVisibleTo</c>.
/// the assembly-level leak assertion and removes unreachable test
/// allocations from the tracker. Visible to friend test assemblies
/// via <c>InternalsVisibleTo</c>.
/// </summary>
internal static void AccountForDeliberatelyLeakedInstanceForTest()
/// <param name="allocationStackMarker">
/// Method name that uniquely identifies the deliberate test allocation.
/// </param>
internal static void AccountForDeliberatelyLeakedInstanceForTest(
string allocationStackMarker)
{
Interlocked.Increment(ref s_instancesDisposed);
foreach (KeyValuePair<long, CertificateAllocationInfo> entry in s_allocationTracker)
{
if (!entry.Value.Reference.TryGetTarget(out _) &&
entry.Value.StackTrace.Contains(
allocationStackMarker,
StringComparison.Ordinal))
{
s_allocationTracker.TryRemove(entry.Key, out _);
return;
}
}
}
#endif
/// <summary>
/// The shared, reference-counted state for a logical certificate. One
@@ -985,10 +1076,28 @@ namespace Opc.Ua.Security.Certificates
/// </summary>
private readonly CertificateCore m_core;
private CertificateAllocationInfo? m_allocationInfo;
/// <summary>
/// 0 while this handle is live, 1 once this handle has been disposed.
/// Makes Dispose idempotent per handle (SA-CERT-01).
/// </summary>
private int m_disposed;
private const string c_leakTrackingSwitchName =
"Opc.Ua.Security.Certificates.CertificateLeakTracking";
private const string c_leakTrackingEnvironmentVariable =
"OPCUA_CERTIFICATE_LEAK_TRACKING";
/// <summary>
/// Outstanding tracked allocations, removed when their owning handle is disposed.
/// </summary>
private static readonly ConcurrentDictionary<long, CertificateAllocationInfo>
s_allocationTracker = new();
private static readonly AsyncLocal<string?> s_leakTrackingScope = new();
private static readonly bool s_leakTrackingEnabled = ResolveLeakTrackingEnabled();
private static long s_instancesCreated;
private static long s_instancesDisposed;
private static long s_nextAllocationId;
}
}
+24 -21
View File
@@ -82,29 +82,32 @@ namespace Opc.Ua.Core.Tests
System.IO.File.AppendAllText(path,
$" net={net,4} created={kv.Value.created,4} disposed={kv.Value.disposed,4} {kv.Key}\n");
}
#if DEBUG
// In Debug builds, also dump the allocation stack of any
// live (reachable) certificate that still has a positive
// refcount. These are the actual leaks.
System.IO.File.AppendAllText(path, "\nLIVE LEAKED CERTIFICATES (DEBUG):\n");
foreach ((string thumbprint, int refCount, DateTime createdAt, string stackTrace) in
Certificate.EnumerateLiveCertificates())
if (Certificate.LeakTrackingEnabled)
{
System.IO.File.AppendAllText(path,
$"\n Thumbprint={thumbprint}, RefCount={refCount}, " +
$"CreatedAt={createdAt:O}\n StackTrace:\n{stackTrace}\n");
System.IO.File.AppendAllText(path, "\nLIVE LEAKED CERTIFICATES:\n");
foreach ((
string thumbprint,
int refCount,
DateTime createdAt,
string stackTrace,
string fixtureName) in
Certificate.EnumerateLiveCertificates())
{
System.IO.File.AppendAllText(path,
$"\n Thumbprint={thumbprint}, RefCount={refCount}, " +
$"CreatedAt={createdAt:O}, Fixture={fixtureName ?? "(unattributed)"}\n" +
$" StackTrace:\n{stackTrace}\n");
}
System.IO.File.AppendAllText(path, "\nUNREACHABLE UNDISPOSED CERTIFICATES:\n");
foreach ((DateTime createdAt, string stackTrace, string fixtureName) in
Certificate.EnumerateUnreachableUndisposedCertificates())
{
System.IO.File.AppendAllText(path,
$"\n CreatedAt={createdAt:O}, " +
$"Fixture={fixtureName ?? "(unattributed)"}\n" +
$" StackTrace:\n{stackTrace}\n");
}
}
// Certificates whose finaliser ran while their refcount was
// still > 0 (AddRef without matching Dispose).
System.IO.File.AppendAllText(path, "\nFINALIZED-WITH-LEAKED-REF CERTIFICATES (DEBUG):\n");
foreach ((string thumbprint, DateTime createdAt, string stackTrace) in
Certificate.EnumerateFinalizedLeakedCertificates())
{
System.IO.File.AppendAllText(path,
$"\n Thumbprint={thumbprint}, " +
$"CreatedAt={createdAt:O}\n StackTrace:\n{stackTrace}\n");
}
#endif
}
catch
{
@@ -489,6 +489,103 @@ namespace Opc.Ua.Core.Tests.Stack.Client
}
}
[Test]
public async Task ConcurrentInitialOpenAndReconnectDisposesLosingTransportAsync()
{
var initialOpenStarted = new TaskCompletionSource<bool>(
TaskCreationOptions.RunContinuationsAsynchronously);
var releaseInitialOpen = new TaskCompletionSource<bool>(
TaskCreationOptions.RunContinuationsAsynchronously);
var reconnectOpenStarted = new TaskCompletionSource<bool>(
TaskCreationOptions.RunContinuationsAsynchronously);
var initialChannel = new Mock<IChannel>();
var reconnectChannel = new Mock<IChannel>();
initialChannel
.Setup(channel => channel.OpenAsync(
It.IsAny<Uri>(),
It.IsAny<TransportChannelSettings>(),
It.IsAny<CancellationToken>()))
.Callback(() => initialOpenStarted.TrySetResult(true))
.Returns(() => new ValueTask(releaseInitialOpen.Task));
reconnectChannel
.Setup(channel => channel.OpenAsync(
It.IsAny<Uri>(),
It.IsAny<TransportChannelSettings>(),
It.IsAny<CancellationToken>()))
.Callback(() => reconnectOpenStarted.TrySetResult(true))
.Returns(new ValueTask());
initialChannel
.Setup(channel => channel.CloseAsync(It.IsAny<CancellationToken>()))
.Returns(new ValueTask());
reconnectChannel
.Setup(channel => channel.CloseAsync(It.IsAny<CancellationToken>()))
.Returns(new ValueTask());
int createCount = 0;
var bindings = new Mock<ITransportChannelBindings>();
bindings
.Setup(factory => factory.Create(
It.IsAny<string>(),
It.IsAny<ITelemetryContext>()))
.Returns(() => Interlocked.Increment(ref createCount) == 1
? initialChannel.Object
: reconnectChannel.Object);
ITelemetryContext telemetry = NUnitTelemetryContext.Create();
var configuration = new ApplicationConfiguration(telemetry);
await using var manager = new ClientChannelManager(
configuration,
telemetry,
bindings.Object);
ConfiguredEndpoint endpoint = GetNoneSecurityEndpoint(
new EndpointConfiguration { OperationTimeout = 6000 });
var entry = new ChannelEntry(
(IChannelEntryHost)manager,
ManagedChannelKey.FromEndpoint(endpoint),
endpoint,
reverseConnection: null);
ManagedTransportChannelLease? lease = null;
try
{
Task initialOpen = entry.OpenInitialAsync(
clientCertificate: null,
clientCertificateChain: null,
clientCertificateVersion: 0,
default);
await initialOpenStarted.Task.WaitAsync(s_completionTimeout)
.ConfigureAwait(false);
lease = entry.AcquireLease(new TestParticipant("p1", endpoint));
Task<bool> reconnect = entry.RequestReconnectAsync(default);
await reconnectOpenStarted.Task.WaitAsync(s_completionTimeout)
.ConfigureAwait(false);
Assert.That(
await reconnect.WaitAsync(s_completionTimeout).ConfigureAwait(false),
Is.True);
releaseInitialOpen.TrySetResult(true);
await initialOpen.WaitAsync(s_completionTimeout).ConfigureAwait(false);
Assert.That(entry.Underlying, Is.SameAs(reconnectChannel.Object));
initialChannel.Verify(
channel => channel.CloseAsync(It.IsAny<CancellationToken>()),
Times.Once);
initialChannel.Verify(channel => channel.Dispose(), Times.Once);
reconnectChannel.Verify(channel => channel.Dispose(), Times.Never);
}
finally
{
releaseInitialOpen.TrySetResult(true);
if (lease != null)
{
await lease.CloseAsync(default).ConfigureAwait(false);
}
await entry.DisposeAsync().ConfigureAwait(false);
}
}
[Test]
public async Task FatalForChannelTransitionsToFaultedStateAsync()
{
@@ -1195,6 +1292,7 @@ namespace Opc.Ua.Core.Tests.Stack.Client
/// Awaits <paramref name="task"/> and returns the exception it faulted
/// with, failing the test if it succeeded or threw something else.
/// </summary>
/// <typeparam name="TException">The expected exception type.</typeparam>
/// <remarks>
/// Deliberately not <c>Assert.ThrowsAsync</c>. That blocks the calling
/// thread until the task completes - sync over async - so on a
@@ -51,11 +51,11 @@ namespace Opc.Ua.Core.Tests.Stack.Transport
[NonParallelizable]
public sealed class UaSCBinaryLoopbackTests
{
private static readonly ICertificateFactory s_certificateFactory = DefaultCertificateFactory.Instance;
private static readonly ICertificateFactory s_certificateFactory = DefaultCertificateFactory.Instance;
[Test]
[CancelAfter(15000)]
public async Task ClientAndTcpListenerExchangeRequestAndCloseAsync()
[Test]
[CancelAfter(15000)]
public async Task ClientAndTcpListenerExchangeRequestAndCloseAsync()
{
ITelemetryContext telemetry = NUnitTelemetryContext.Create();
Uri endpointUrl = new($"opc.tcp://127.0.0.1:{GetFreeTcpPort()}");
@@ -340,6 +340,180 @@ namespace Opc.Ua.Core.Tests.Stack.Transport
await listener.CloseAsync(CancellationToken.None).ConfigureAwait(false);
}
[Test]
[CancelAfter(15000)]
public async Task FailedReconnectDisposesCandidateCertificateHandlesAsync()
{
long createdBefore = Certificate.InstancesCreated;
long disposedBefore = Certificate.InstancesDisposed;
ITelemetryContext telemetry = NUnitTelemetryContext.Create();
Certificate serverCertificate = s_certificateFactory
.CreateCertificate("CN=reconnect-server")
.CreateForRSA();
Uri endpointUrl = new($"opc.tcp://127.0.0.1:{GetFreeTcpPort()}");
EndpointDescription endpoint = CreateEndpoint(endpointUrl);
endpoint.ServerCertificate = serverCertificate.RawData.ToByteString();
EndpointConfiguration configuration = EndpointConfiguration.Create();
configuration.OperationTimeout = 5000;
var certificateRegistry = new Mock<ICertificateRegistry>();
certificateRegistry
.Setup(r => r.AcquireApplicationCertificateBySecurityPolicy(It.IsAny<string>()))
.Returns((CertificateEntry?)null);
await using var listener = new TcpTransportListener(telemetry);
var channel = new UaSCUaBinaryTransportChannel(
new TcpByteTransportFactory(telemetry),
telemetry)
{
OperationTimeout = 5000
};
try
{
await listener.OpenAsync(
endpointUrl,
new TransportListenerSettings
{
Descriptions = new List<EndpointDescription> { endpoint },
Configuration = configuration,
ServerCertificates = certificateRegistry.Object,
NamespaceUris = new NamespaceTable(),
Factory = EncodeableFactory.Create(),
MaxChannelCount = 10
},
new EchoCallback(),
CancellationToken.None).ConfigureAwait(false);
await channel.OpenAsync(
endpointUrl,
new TransportChannelSettings
{
Description = endpoint,
Configuration = configuration,
ServerCertificate = serverCertificate,
NamespaceUris = new NamespaceTable(),
Factory = EncodeableFactory.Create()
},
CancellationToken.None).ConfigureAwait(false);
await listener.CloseAsync(CancellationToken.None).ConfigureAwait(false);
Exception reconnectError = await CaptureExceptionAsync(
() => channel.ReconnectAsync(null, CancellationToken.None).AsTask())
.ConfigureAwait(false);
Assert.That(reconnectError, Is.InstanceOf<SocketException>());
}
finally
{
channel.Dispose();
serverCertificate.Dispose();
}
Assert.That(
Certificate.InstancesCreated - createdBefore,
Is.EqualTo(Certificate.InstancesDisposed - disposedBefore),
"A failed reconnect must release every AddRef handle on the server certificate.");
}
[Test]
[CancelAfter(15000)]
public async Task CancelledInitialOpenCanRetryWithFreshTransportAsync()
{
var connectStarted = new TaskCompletionSource<bool>(
TaskCreationOptions.RunContinuationsAsynchronously);
var firstTransport = new Mock<IUaSCByteTransport>();
firstTransport.SetupGet(transport => transport.Implementation).Returns("Test");
firstTransport
.Setup(transport => transport.ConnectAsync(
It.IsAny<Uri>(),
It.IsAny<CancellationToken>()))
.Returns((Uri _, CancellationToken ct) => new ValueTask(
WaitForCancellationAsync(connectStarted, ct)));
var expected = new InvalidOperationException("second transport reached");
var secondTransport = new Mock<IUaSCByteTransport>();
secondTransport.SetupGet(transport => transport.Implementation).Returns("Test");
secondTransport
.Setup(transport => transport.ConnectAsync(
It.IsAny<Uri>(),
It.IsAny<CancellationToken>()))
.Returns(new ValueTask(Task.FromException(expected)));
var factory = new Mock<IUaSCByteTransportFactory>();
factory.SetupGet(value => value.Implementation).Returns("Test");
factory
.SetupSequence(value => value.Create(
It.IsAny<BufferManager>(),
It.IsAny<int>(),
It.IsAny<ITelemetryContext>()))
.Returns(firstTransport.Object)
.Returns(secondTransport.Object);
ITelemetryContext telemetry = NUnitTelemetryContext.Create();
Uri endpointUrl = new("opc.tcp://127.0.0.1:4840");
EndpointDescription endpoint = CreateEndpoint(endpointUrl);
EndpointConfiguration configuration = EndpointConfiguration.Create();
configuration.OperationTimeout = 5000;
using var channel = new UaSCUaBinaryTransportChannel(factory.Object, telemetry)
{
OperationTimeout = 5000
};
using var cts = new CancellationTokenSource();
Task firstOpen = channel.OpenAsync(
endpointUrl,
CreateChannelSettings(endpoint, configuration),
cts.Token).AsTask();
await connectStarted.Task.ConfigureAwait(false);
cts.Cancel();
Exception cancellation = await CaptureExceptionAsync(() => firstOpen)
.ConfigureAwait(false);
Exception retry = await CaptureExceptionAsync(
() => channel.OpenAsync(
endpointUrl,
CreateChannelSettings(endpoint, configuration),
CancellationToken.None).AsTask()).ConfigureAwait(false);
Assert.That(cancellation, Is.InstanceOf<OperationCanceledException>());
Assert.That(retry, Is.SameAs(expected));
firstTransport.Verify(transport => transport.Close(), Times.AtLeastOnce);
secondTransport.Verify(transport => transport.Close(), Times.AtLeastOnce);
}
private static TransportChannelSettings CreateChannelSettings(
EndpointDescription endpoint,
EndpointConfiguration configuration)
{
return new TransportChannelSettings
{
Description = endpoint,
Configuration = configuration,
NamespaceUris = new NamespaceTable(),
Factory = EncodeableFactory.Create()
};
}
private static async Task<Exception> CaptureExceptionAsync(Func<Task> action)
{
try
{
await action().ConfigureAwait(false);
}
catch (Exception ex)
{
return ex;
}
throw new AssertionException("Expected the operation to fail.");
}
private static async Task WaitForCancellationAsync(
TaskCompletionSource<bool> connectStarted,
CancellationToken ct)
{
connectStarted.TrySetResult(true);
await Task.Delay(Timeout.Infinite, ct).ConfigureAwait(false);
}
private static EndpointDescription CreateEndpoint(Uri endpointUrl)
{
return new EndpointDescription
@@ -0,0 +1,119 @@
/* ========================================================================
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System.Text;
using NUnit.Framework;
using Opc.Ua.Tests;
namespace Opc.Ua.Security.Certificates.Tests
{
/// <summary>
/// Tests for per-fixture certificate leak attribution.
/// </summary>
[TestFixture]
[NonParallelizable]
[CertificateLeakAttribution]
public class CertificateLeakAttributionTests
{
[SetUp]
public void SetUp()
{
CertificateLeakAttribution.Reset();
}
[TearDown]
public void TearDown()
{
CertificateLeakAttribution.Reset();
}
[Test]
public void BuildSummaryIncludesUnbalancedFixtureActivity()
{
CertificateLeakAttribution.Record(
"Opc.Ua.Tests.LeakingFixture",
created: 3,
disposed: 2);
CertificateLeakAttribution.Record(
"Opc.Ua.Tests.BalancedFixture",
created: 4,
disposed: 4);
string summary = CertificateLeakAttribution.BuildSummary();
Assert.That(summary, Does.Contain("Opc.Ua.Tests.LeakingFixture"));
Assert.That(summary, Does.Contain("net=1, created=3, disposed=2"));
Assert.That(summary, Does.Not.Contain("Opc.Ua.Tests.BalancedFixture"));
}
[Test]
public void BuildSummaryAggregatesRepeatedFixtureActivity()
{
CertificateLeakAttribution.Record(
"Opc.Ua.Tests.RepeatedFixture",
created: 2,
disposed: 1);
CertificateLeakAttribution.Record(
"Opc.Ua.Tests.RepeatedFixture",
created: 5,
disposed: 3);
string summary = CertificateLeakAttribution.BuildSummary();
Assert.That(summary, Does.Contain("net=3, created=7, disposed=4"));
}
[Test]
public void LeakDumpReflectsTrackingMode()
{
using Certificate cert = CertificateBuilder.Create("CN=LeakDumpTest")
.SetRSAKeySize(2048)
.CreateForRSA();
var message = new StringBuilder();
LeakDetectionHelpers.AppendLeakDumps(message);
if (Certificate.LeakTrackingEnabled)
{
Assert.That(
message.ToString(),
Does.Contain(nameof(CertificateLeakAttributionTests)));
Assert.That(
message.ToString(),
Does.Contain(nameof(LeakDumpReflectsTrackingMode)));
}
else
{
Assert.That(
message.ToString(),
Does.Contain("OPCUA_CERTIFICATE_LEAK_TRACKING=1"));
}
}
}
}
@@ -964,27 +964,132 @@ namespace Opc.Ua.Security.Certificates.Tests
"handle Dispose must not affect the counters.");
}
#if DEBUG
[Test]
public void VerifyLeakDetectionTracksAllocation()
public void LeakTrackingConfigurationUsesAppContextBeforeEnvironment()
{
WeakReference weakRef = CreateLeakedCertificateRef("CN=LeakTest");
bool enabled = Certificate.ResolveLeakTrackingEnabled(
switchFound: true,
switchValue: false,
environmentValue: "true",
defaultValue: true);
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
Assert.That(enabled, Is.False);
}
Assert.That(weakRef.IsAlive, Is.False);
[TestCase("1", true)]
[TestCase("true", true)]
[TestCase("TRUE", true)]
[TestCase("0", false)]
[TestCase("false", false)]
[TestCase("FALSE", false)]
public void LeakTrackingConfigurationParsesEnvironmentValue(
string environmentValue,
bool expected)
{
bool enabled = Certificate.ResolveLeakTrackingEnabled(
switchFound: false,
switchValue: false,
environmentValue,
defaultValue: !expected);
// The certificate above is deliberately abandoned without Dispose
// to exercise the finalizer-based leak tracking. Account for it so
// the intentional leak does not trip the assembly-level leak
// assertion (the inner X509Certificate2 is reclaimed by its own
// finalizer).
Certificate.AccountForDeliberatelyLeakedInstanceForTest();
Assert.That(enabled, Is.EqualTo(expected));
}
[Test]
public void LeakTrackingConfigurationUsesDefaultForUnknownEnvironmentValue()
{
bool enabled = Certificate.ResolveLeakTrackingEnabled(
switchFound: false,
switchValue: false,
environmentValue: "unknown",
defaultValue: true);
Assert.That(enabled, Is.True);
}
[Test]
public void LeakTrackingMatchesExplicitEnvironmentSetting()
{
string setting = Environment.GetEnvironmentVariable(
"OPCUA_CERTIFICATE_LEAK_TRACKING");
if (setting is not ("0" or "1"))
{
Assert.Ignore("No explicit certificate leak tracking setting was provided.");
}
Assert.That(Certificate.LeakTrackingEnabled, Is.EqualTo(setting == "1"));
}
[Test]
[NonParallelizable]
public void LeakTrackingReportsLiveCertificateOnlyWhenEnabled()
{
const string fixtureName = "Opc.Ua.Tests.LiveTrackingFixture";
string previousScope = Certificate.LeakTrackingScope;
Certificate.LeakTrackingScope = fixtureName;
try
{
using Certificate cert = CertificateBuilder.Create("CN=LiveTrackingTest")
.SetRSAKeySize(2048)
.CreateForRSA();
var matches = Certificate.EnumerateLiveCertificates()
.Where(entry => entry.Thumbprint == cert.Thumbprint)
.ToArray();
Assert.That(
matches,
Has.Length.EqualTo(Certificate.LeakTrackingEnabled ? 1 : 0));
if (Certificate.LeakTrackingEnabled)
{
Assert.That(matches[0].FixtureName, Is.EqualTo(fixtureName));
}
}
finally
{
Certificate.LeakTrackingScope = previousScope;
}
}
[Test]
[NonParallelizable]
public void VerifyLeakDetectionTracksAllocation()
{
if (!Certificate.LeakTrackingEnabled)
{
Assert.Ignore("Certificate allocation tracking is disabled.");
}
WeakReference weakRef = CreateLeakedCertificateRef("CN=LeakTest");
try
{
GC.Collect();
GC.WaitForPendingFinalizers();
GC.Collect();
Assert.That(weakRef.IsAlive, Is.False);
Assert.That(
Certificate.EnumerateUnreachableUndisposedCertificates().Any(
entry => entry.StackTrace.Contains(
nameof(CreateLeakedCertificateRef),
StringComparison.Ordinal)),
Is.True);
}
finally
{
// The certificate above is deliberately abandoned without Dispose
// to exercise the finalizer-based leak tracking. Account for it so
// the intentional leak does not trip the assembly-level leak
// assertion (the inner X509Certificate2 is reclaimed by its own
// finalizer).
Certificate.AccountForDeliberatelyLeakedInstanceForTest(
nameof(CreateLeakedCertificateRef));
}
}
[Test]
[NonParallelizable]
public void VerifyNoLeakWhenProperlyDisposed()
{
WeakReference weakRef = CreateDisposedCertificateRef(
@@ -998,6 +1103,7 @@ namespace Opc.Ua.Security.Certificates.Tests
}
[Test]
[NonParallelizable]
public void VerifyAddRefDisposeNoLeak()
{
WeakReference weakRef = CreateAddRefDisposedCertificateRef(
@@ -1047,7 +1153,5 @@ namespace Opc.Ua.Security.Certificates.Tests
cert.Dispose();
return new WeakReference(cert);
}
#endif
}
}
@@ -340,7 +340,8 @@ namespace Opc.Ua.Sessions.Tests
{
if (serverStopped)
{
ReferenceServer = await ServerFixture.StartAsync(PkiRoot, serverPort).ConfigureAwait(false);
ReferenceServer = await ServerFixture.StartAsync(PkiRoot, serverPort)
.ConfigureAwait(false);
ReferenceServer.TokenValidator = TokenValidator;
}
@@ -28,5 +28,7 @@
* ======================================================================*/
using System;
using Opc.Ua.Tests;
[assembly: CLSCompliant(false)]
[assembly: CertificateLeakAttribution]
@@ -0,0 +1,177 @@
/* ========================================================================
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System;
using System.Collections.Concurrent;
using System.Linq;
using System.Text;
using System.Threading;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using Opc.Ua.Security.Certificates;
namespace Opc.Ua.Tests
{
/// <summary>
/// Records certificate counter deltas for each test fixture.
/// </summary>
[AttributeUsage(AttributeTargets.Assembly | AttributeTargets.Class | AttributeTargets.Method)]
public sealed class CertificateLeakAttributionAttribute : Attribute, ITestAction
{
/// <inheritdoc/>
public ActionTargets Targets => ActionTargets.Suite;
/// <inheritdoc/>
public void BeforeTest(ITest test)
{
s_baseline.Value = new Baseline(
Certificate.InstancesCreated,
Certificate.InstancesDisposed,
test.FullName,
s_baseline.Value);
Certificate.LeakTrackingScope = test.FullName;
}
/// <inheritdoc/>
public void AfterTest(ITest test)
{
if (s_baseline.Value is not Baseline baseline)
{
return;
}
s_baseline.Value = baseline.Parent;
Certificate.LeakTrackingScope = baseline.Parent?.FixtureName;
CertificateLeakAttribution.Record(
baseline.FixtureName,
Certificate.InstancesCreated - baseline.Created,
Certificate.InstancesDisposed - baseline.Disposed);
}
private sealed class Baseline
{
public Baseline(
long created,
long disposed,
string fixtureName,
Baseline parent)
{
Created = created;
Disposed = disposed;
FixtureName = fixtureName;
Parent = parent;
}
public long Created { get; }
public long Disposed { get; }
public string FixtureName { get; }
public Baseline Parent { get; }
}
private static readonly AsyncLocal<Baseline> s_baseline = new();
}
/// <summary>
/// Aggregates per-fixture certificate activity for leak diagnostics.
/// </summary>
public static class CertificateLeakAttribution
{
/// <summary>
/// Builds the per-fixture activity summary appended to a leak failure.
/// </summary>
internal static string BuildSummary()
{
var entries = s_fixtureActivity
.Select(entry => new
{
entry.Key,
entry.Value.Created,
entry.Value.Disposed,
Net = entry.Value.Created - entry.Value.Disposed
})
.Where(entry => entry.Net != 0)
.OrderByDescending(entry => entry.Net)
.ThenBy(entry => entry.Key, StringComparer.Ordinal)
.ToArray();
if (entries.Length == 0)
{
return string.Empty;
}
var summary = new StringBuilder();
summary.AppendLine(
"Overlapping per-fixture counter windows (diagnostic hints only):");
foreach (var entry in entries)
{
summary.AppendFormat(
System.Globalization.CultureInfo.InvariantCulture,
" net={0}, created={1}, disposed={2}: {3}",
entry.Net,
entry.Created,
entry.Disposed,
entry.Key);
summary.AppendLine();
}
return summary.ToString();
}
/// <summary>
/// Clears recorded fixture activity.
/// </summary>
internal static void Reset()
{
s_fixtureActivity.Clear();
}
/// <summary>
/// Records one fixture's certificate counter deltas.
/// </summary>
internal static void Record(string fixtureName, long created, long disposed)
{
if (created == 0 && disposed == 0)
{
return;
}
s_fixtureActivity.AddOrUpdate(
fixtureName,
(created, disposed),
(_, previous) => (
previous.Created + created,
previous.Disposed + disposed));
}
private static readonly ConcurrentDictionary<string, (long Created, long Disposed)>
s_fixtureActivity = new();
}
}
@@ -201,11 +201,11 @@ namespace Opc.Ua.Tests
/// hang). A genuine leak stays positive across the poll and still fails.
/// </para>
/// <para>
/// In <c>DEBUG</c> builds the failure message additionally includes
/// the allocation stack traces of every live (reachable) certificate
/// with a positive refcount and of every certificate whose finalizer
/// ran while still referenced, so the leak source is visible directly
/// in the CI test output.
/// When certificate allocation tracking is enabled, the failure message
/// additionally includes allocation stack traces for live certificates
/// with a positive refcount and unreachable handles that were not
/// disposed, so the leak source is visible directly in the CI test
/// output.
/// </para>
/// </remarks>
/// <param name="detail">Optional caller-supplied breakdown (for
@@ -231,7 +231,14 @@ namespace Opc.Ua.Tests
.Append(detail);
}
AppendDebugLeakDumps(message);
string fixtureSummary = CertificateLeakAttribution.BuildSummary();
if (!string.IsNullOrEmpty(fixtureSummary))
{
message.AppendLine()
.Append(fixtureSummary);
}
AppendLeakDumps(message);
Assert.Fail(message.ToString());
}
@@ -295,32 +302,83 @@ namespace Opc.Ua.Tests
}
/// <summary>
/// Appends the DEBUG-only allocation stack traces of leaked
/// certificates to <paramref name="message"/>. No-op in release
/// builds (the per-instance tracking is compiled out).
/// Appends allocation stack traces of leaked certificates to
/// <paramref name="message"/> when allocation tracking is enabled.
/// </summary>
private static void AppendDebugLeakDumps(StringBuilder message)
internal static void AppendLeakDumps(StringBuilder message)
{
#if DEBUG
message.AppendLine()
.AppendLine("LIVE LEAKED CERTIFICATES (DEBUG):");
foreach ((string thumbprint, int refCount, DateTime createdAt, string stackTrace) in
Certificate.EnumerateLiveCertificates())
if (!Certificate.LeakTrackingEnabled)
{
message.AppendLine(FormattableString.Invariant(
$" Thumbprint={thumbprint}, RefCount={refCount}, CreatedAt={createdAt:O}"))
.AppendLine(FormattableString.Invariant($" StackTrace:\n{stackTrace}"));
message.AppendLine()
.AppendLine(
"Certificate allocation tracking was disabled. Set " +
"OPCUA_CERTIFICATE_LEAK_TRACKING=1 before starting the test host " +
"to include allocation stacks.");
return;
}
message.AppendLine("FINALIZED-WITH-LEAKED-REF CERTIFICATES (DEBUG):");
foreach ((string thumbprint, DateTime createdAt, string stackTrace) in
Certificate.EnumerateFinalizedLeakedCertificates())
message.AppendLine()
.AppendLine("LIVE LEAKED CERTIFICATES:");
int liveCount = 0;
foreach ((
string thumbprint,
int refCount,
DateTime createdAt,
string stackTrace,
string fixtureName) in
Certificate.EnumerateLiveCertificates())
{
message.AppendLine(FormattableString.Invariant(
$" Thumbprint={thumbprint}, CreatedAt={createdAt:O}"))
.AppendLine(FormattableString.Invariant($" StackTrace:\n{stackTrace}"));
liveCount++;
if (liveCount <= c_maxLeakDumpEntries)
{
message.AppendFormat(
CultureInfo.InvariantCulture,
" Thumbprint={0}, RefCount={1}, CreatedAt={2:O}, Fixture={3}",
thumbprint,
refCount,
createdAt,
fixtureName ?? "(unattributed)");
message.AppendLine()
.AppendLine(FormattableString.Invariant($" StackTrace:\n{stackTrace}"));
}
}
#endif
AppendOmittedCount(message, liveCount);
message.AppendLine("UNREACHABLE UNDISPOSED CERTIFICATES:");
int unreachableCount = 0;
foreach ((DateTime createdAt, string stackTrace, string fixtureName) in
Certificate.EnumerateUnreachableUndisposedCertificates())
{
unreachableCount++;
if (unreachableCount <= c_maxLeakDumpEntries)
{
message.AppendFormat(
CultureInfo.InvariantCulture,
" CreatedAt={0:O}, Fixture={1}",
createdAt,
fixtureName ?? "(unattributed)");
message.AppendLine()
.AppendLine(FormattableString.Invariant($" StackTrace:\n{stackTrace}"));
}
}
AppendOmittedCount(message, unreachableCount);
}
/// <summary>
/// Appends the number of leak entries omitted by the diagnostic cap.
/// </summary>
private static void AppendOmittedCount(StringBuilder message, int count)
{
if (count > c_maxLeakDumpEntries)
{
message.AppendFormat(
CultureInfo.InvariantCulture,
" ... {0} additional allocation(s) omitted.",
count - c_maxLeakDumpEntries);
message.AppendLine();
}
}
private const int c_maxLeakDumpEntries = 50;
}
}
@@ -13,4 +13,7 @@
<ItemGroup>
<ProjectReference Include="..\..\src\Opc.Ua.Core\Opc.Ua.Core.csproj" />
</ItemGroup>
<ItemGroup>
<InternalsVisibleTo Include="Opc.Ua.Security.Certificates.Tests" />
</ItemGroup>
</Project>