Files
UA-.NETStandard/docs/CryptoProvider.md
Marc Schier aad8350bc1 Pluggable symmetric cryptography, an awaitable asymmetric path, registrable security policies, and PubSub crypto offboarding (#4206, #4207, #4208, #4210) (#4232)
# Description

Closes the three crypto-offboarding follow-ups split out of #4192. Each
was recorded there as knowingly *not* done, with the reason and what
would justify revisiting it. Those conditions are met, and where one of
them turned out to be unachievable that is written down rather than
worked around.

The safety property throughout: **every seam is opt-in and null-checked,
and in the default software configuration the added code is observably
inert.** No interface dispatch on the per-message path, and the
asynchronous paths complete synchronously — so channel locking order,
buffer ownership and error sequencing are unchanged for a deployment
that configures nothing.

## #4207 — symmetric, key-derivation and RNG provider seams

`ISymmetricCryptoProvider`, `IKeyDerivationProvider` and
`ISecureRandomSource`, added on the terms #4207 set: only once a
concrete requirement arrived (a validated module that must perform
*every* operation, not just the asymmetric ones), behind a fast path,
with `SymmetricChannelCryptoBenchmarks` as the gate.

They are **facets discovered by type test**, not members of
`ICryptoProvider`, so providers written against the shipped interface
still compile. `PlatformCryptoProvider` implements all three, which
answers the "an interface nothing implements" objection recorded in the
issue.

`CryptoProviderFacets` returns `null` both when no registry is
configured *and* when resolution lands on the platform, because the
platform facets are the inline code. Resolution happens once, in
`CalculateSymmetricKeySizes`, and is held for the life of the channel.

`CryptoCompliance.GetUnservedOperationPurposes` closes the failure the
seam would otherwise introduce: a provider bound to `ChannelSymmetric`
without implementing the facet is silently replaced by the platform, so
a deployment could believe its validated module performed the
per-message cryptography while the platform did. Under `FipsOnly` that
now refuses to start.

## #4208 — the asymmetric path is awaitable

`RSA` and `ECDsa` are .NET's contracts and cannot be replaced without
making every ready-made hardware and cloud implementation unusable.
Instead an implementation may **also** declare `IAsyncRsaKey` /
`IAsyncEcdsaKey`, which the stack finds by type test.

Now asynchronous: the secure channel open and renew path, user identity
token signing and decryption, and session activation. A software key
declares neither facet, so those paths complete synchronously and
nothing about their ordering changes.

`ReadAsymmetricMessage` is refactored into shared helpers rather than
duplicated — adding an `await` was not worth two hundred cloned lines of
signature, padding and endpoint-selection logic. `AsymmetricMessage` /
`AsymmetricWriteResult` exist because an asynchronous method cannot have
`out` parameters.

The channel no longer serialises its state on a monitor. A monitor
cannot be held across an `await`, and a plain `SemaphoreSlim` was not an
option at the time: the code relied on re-entrancy in eight places
(`HandleIncomingMessage` holds the lock and calls `ForceChannelFault`,
which takes it again; `ForceReconnect` and `Shutdown` do the same).
`ChannelGate` replaces it at 32 sites and `DataLock` is `[Obsolete]`.
That reliance has since been removed as well — see [Removing the gate's
re-entrancy](#removing-the-gates-re-entrancy) below.

### Defects this surfaced, all fixed and regression-tested

**Detached work started inline.** Writes were started with `_ =
WriteBuffersAsync(...)`. An async method runs its prologue on the
caller's stack, so `LeaveInheritedContext` — which exists to stop
*detached* work inheriting the right to re-enter the gate — was
stripping the **caller's own** entitlement; when the send then completed
synchronously, the completion blocked on a gate that very thread was
holding. The client sent its OpenSecureChannel request and never
processed the response. Writes are now queued.

**A synchronous gate handle held across an `await`.** `Enter()` records
the acquiring thread so an inline completion callback can re-enter —
`ChannelAsyncOperation` invokes its callback both inline and detached.
That record is only sound while the holder is synchronously on that
thread; the token renewal path held it across an await, so a reused pool
thread was recognised as the holder.

**`SignAsync` rejected the policy that signs nothing.** It validated the
certificate before the algorithm, so `SecurityPolicies.None` — which
carries no certificate — threw where the synchronous `Sign` returns
`null`.

**`EnterAsync` lost the holder when contended** *(found in review)*. The
`AsyncLocal` recording ownership was written inside the awaiting method,
and .NET restores the caller's execution context when an async method
completes, so the write was discarded. The caller returned owning the
semaphore while `IsHeldByCurrentContext` reported `false` — and any
nested `Enter()` then blocked forever on a gate its own context held.
Reachable remotely: `HandleIncomingMessageAsync` → error path →
`ForceChannelFault`. The holder is now published from the caller's frame
before the wait, and only its depth is raised in the continuation.

**Queued writes lost their order** *(found in review)*. Sequence numbers
are assigned under the gate, but independently queued work items reach
the transport's FIFO send lock in arbitrary order, so a peer could
reject a chunk with `BadSequenceNumberInvalid`. Writes are now appended
to a chain: still off the caller's stack, but a write cannot start
before its predecessor finishes.

### Four more this surfaced in CI, on paths only the integration suites
reach

Queuing the writes to fix the deadlock above turned out to have moved
three implicit guarantees, and the gate had one more liveness bug. Each
was reproduced locally against a clean baseline before it was changed.

**A faulting channel's error message never reached the peer.** A
listener channel sends a UASC `Error` describing why it faulted and
closes the transport as its next statement. That worked only because the
write started on the caller's stack, so the bytes reached the socket
first; queued, the close discarded them and the client saw
`BadConnectionClosed` instead of the certificate status it was told to
expect. Nine `SecurityCertValidationTests` cases assert exactly that
status. The terminal error is written inline again, on a path that
touches the gate only to disclaim what it inherited and reports nothing
— a channel that is already faulted is owed nothing beyond its buffer.

**The certificate of a rejected sender leaked.** An asynchronous method
cannot have `out` parameters, so the asymmetric read returns a record
instead. That silently dropped an ownership guarantee: an `out`
parameter reaches the caller even when the method goes on to throw, so
the caller's `catch` disposed the sender certificate and reported it to
the audit. A return value does not — nine leaked certificates, one per
rejection case, which the leak detector fails the run on. The parsed
certificate is now handed to the caller before anything that can reject
the message, restoring both the disposal and the audit.

**The gate could strand its own permit.** It released only when the
holder's depth returned to zero, but that depth lives in an `AsyncLocal`
and is therefore shared with every context forked from the owner's — a
timer callback whose timer was created inside the region, for instance.
Such a context entering the gate raised the shared depth, so the owner's
own exit saw a non-zero depth and returned without releasing, and
nothing afterwards could: the nested exit is not the owner. Every later
entry on that channel blocked forever. `Opc.Ua.Features.Tests` hung
after its tests had finished, with a receive loop waiting on the gate
and the server unable to stop. The owner now releases when it leaves;
depth still gates re-entrancy, but liveness no longer depends on a count
another context can move.

**Write completion throttled the writer.** Reporting a completed write
sat inside the ordering chain, and the client channel implements that
callback by entering the gate — so every write queued behind a gate the
receive loop contends for on each incoming chunk. A session with fifty
subscriptions publishing continuously had its publish requests throttled
enough that the client kept topping them up past its own limit, which
`Subscriptions.Classic` asserts against. Completion is now reported off
the chain, which covers only what ordering actually requires.

### Still synchronous, and why

Certificate, certificate request and revocation list signing cannot be
made asynchronous at all: `X509SignatureGenerator.SignData` is invoked
by .NET's own builders. Service faults and the `Reconnect` override are
reached from synchronous call sites (`SendServiceFault` alone has eight)
and are left as they are behind scoped, commented suppressions. All
three are documented rather than left to be rediscovered.

## #4210 — PubSub

This one produced a finding rather than the API the issue asked for, and
the finding is the point:

> **With a standard Security Key Service the key necessarily exists in
process memory.** `GetSecurityKeys` (Part 14 §8.3.2) returns raw key
bytes over the wire, so the "key never leaves the device" property #4192
achieved for client and server **cannot** be achieved for PubSub through
the SKS pull profile. That is a property of the specification, not of
this stack.

A wrapped-key envelope would change what is on the wire and break
interoperability with third-party key services and publishers, so it was
rejected rather than built.

What is achievable is delivered: the per-message AES-CTR and HMAC route
through `ISymmetricCryptoProvider`, so a validated module performs them.
`IPubSubSecurityPolicy` is **not** modified — new members would break
every implementation of it; the provider is supplied through the policy
constructor. `PlatformSymmetricCryptoProvider` gained AES counter mode,
which the seam should have covered anyway since the algorithm enum names
it.

Key lifetime is also tightened: unpacking an SKS response copied key
material into intermediates that `ByteString.Create` then copied again,
leaving plain keys in the heap for the lifetime of the process. Those
are now zeroed.

### Two wiring gaps this depended on, both found in review

The registrations above were **inert**, so this section originally
claimed more than the code did:

- `CryptoProviderConfiguration` was applied nowhere outside tests, so
`AddCryptoProvider(configure)` never reached the registry in a real
host. The registry now applies every registered configuration when it is
first resolved — which also affects #4207's DI story.
- `PubSubSecurityWrapperResolver` selected
`PubSubAes256CtrPolicy.Instance`, the provider-less static singleton,
ignoring the provider-backed bundles registered for it. A `FipsOnly`
deployment therefore certified while the platform did the work. The
resolver now selects from the registered policies; the container and
`PubSubApplicationBuilder` both supply theirs.

A third, smaller one: `CryptoProviderBuilder.Use` published a
**policy-scoped** random source as the process-wide nonce source, so
`.For(purpose, policyUri).Use(hsm)` redirected nonce generation for
every other policy too. Only an unscoped binding does that now.

## #4206 — a provider can contribute a security policy

Added in review: the policy set was fixed at compile time, so a provider
could contribute key custody but not a *profile*. It is now table-driven
and open, and the blockers #4206 listed are gone.

```csharp
using IDisposable registration = SecurityPolicies.Register(
    new SecurityPolicyInfo("urn:vendor:SecurityPolicy#CustomProfile", "CustomProfile")
    {
        PlatformSupport = () => true,
        SupportedCertificateTypes = [ObjectTypeIds.RsaSha256ApplicationCertificateType],
        IsDefault = true
    });

services.AddOpcUa()
    .AddCryptoProvider(crypto => crypto
        .For(CryptoPurpose.ChannelSymmetric, "urn:vendor:SecurityPolicy#CustomProfile").Use(module))
    .AddSecurityPolicy(customPolicy);
```

- **The reflection is gone.** Four lookup tables were built by
reflecting over the fields of `SecurityPolicies` and
`SecurityPolicyInfo`. They are one explicit table now, which removes the
**last reflection in `Security/Constants`** — so this is a trimming and
Native AOT improvement as much as a feature.
- **`SecurityPolicyInfo` is constructible from another assembly** and
carries the metadata the helpers need, so `IsPlatformSupportedName`,
`GetDefaultUris`, `MapSecurityPolicyToCertificateTypes` and
`GetCurveFromCertificateTypeId` read that table instead of each being
hand-written. Adding a policy is one entry rather than five edits.
- **The dead `#if CURVE25519` conditional is deleted** from
`Security/Constants`.
- Registration is copy-on-write, so the read path stays lock-free.
Shadowing a built-in policy requires `replaceExisting: true` and is
reversible by disposing the registration.

`RegisterLightsUpCurvePoliciesFromOutsideCore` is the acceptance test
the issue asked for: it registers `ECC_curve25519` and `ECC_curve448`
from the test project and asserts they become discoverable through the
public API.

**Registering a policy makes it advertised and resolvable; it does not
supply the cryptography behind it.** For those two curve profiles the
in-tree key agreement is still behind a compile symbol no project
defines plus a BouncyCastle dependency, so a deployment lighting them up
provides the operations through a provider. That division is the point:
the policy set says *what* is offered, the provider says *who* performs
it.

## Removing the gate's re-entrancy

Folded in from #4242, which was merged into this branch.

`ChannelGate` was re-entrant, and that forced the design behind most of
the defects above. A holder that awaits releases its thread, so
ownership could not be tracked by thread identity — it had to live in an
`AsyncLocal<Holder?>`. A logical context is *inherited*, so anything
started while the gate was held inherited the right to re-enter and ran
inside the guarded region alongside whatever started it, unless it opted
out by hand via `LeaveInheritedContext()` at each of eight sites,
enforced by review alone. Five of the defects listed above trace to that
mechanism, including both CI hangs.

So the *requirement* was removed rather than the mechanism, and the
`AsyncLocal` then deleted itself. Every path that re-entered now calls a
lock-free `Core` method (`ForceChannelFaultCore`, `ShutdownCore`,
`ForceReconnectCore`, `CompleteConnectCore`), and three regions stopped
running foreign code under the lock: the `RequestReceived` dispatch is
deferred until the gate is released, `OnTransportError` no longer wraps
the `HandleSocketError` virtual, and the callback setters and
`EndpointDescription` publish a single reference with a volatile write
instead of locking — the last is load-bearing, because
`AsyncResultBase.OperationCompleted` invokes the reverse-hello callback
inline while the gate is held.

`ChannelGate` is now a `SemaphoreSlim(1,1)` behind a `Releaser`;
`Holder`, the thread-identity fallback, `LeaveInheritedContext`,
`IsHeldByCurrentContext` and all eight disclaimer calls are gone, with
the call shape at all 32 sites unchanged. `SaveIntermediateChunk`,
`GetSavedChunks` and `DoMessageLimitsExceeded` gained a `gateHeld`
argument, because the client reaches them both with and without the gate
and must pick the matching teardown — without it an oversized
`OpenSecureChannel` response deadlocked the receive loop against itself.

The remaining paths were not found by reading the code, since that is
what missed them originally. A temporary strict mode made every
re-entrant acquisition throw with its stack, the suite was run under it
until clean, and it was deleted with the re-entrancy it measured. It
caught two sites review had missed: the `EndpointDescription` **property
getter** taking the gate, and the `RequestReceived` dispatch to
application code happening inside the guarded region.

Two deliberate residuals are documented for reviewers: `Dispose` no
longer takes the gate (unavoidable — it is reached from `ChannelFaulted`
with the gate held, where the old re-entrant acquisition excluded
nothing anyway), and the deferred request dispatch builds
`SecureChannelContext` just after gate release rather than under it.

## Testing

New: the three seams and their platform implementations, facet discovery
and resolution precedence, `FipsOnly` refusing a provider that cannot
serve what it was bound to, AES counter mode (including a
non-block-aligned round trip and a check that the counter carries, since
one that did not would silently reuse key stream), `ChannelGate`
(exclusion, re-entrancy across an await, re-entrancy on a reused thread,
and the deadlocks above), the asynchronous channel paths, and the PubSub
policy provider.

Each review finding got a test that fails without its fix:

- `ContendedEntryStillRecordsTheHolderAsync` — established contention
deterministically, then re-entered; failed on the pre-fix gate. Removed
with the re-entrancy it covered; `ChannelGateTests` now asserts the
non-re-entrant contract instead.
- `SecureClientAndTcpListenerKeepChunkOrderUnderConcurrentRequestsAsync`
— 60 concurrent multi-chunk `SignAndEncrypt` requests over a real
loopback listener; failed 3 runs out of 3 with the previous dispatch,
passes 3 out of 3 with the chain.
-
`DependencyInjectionRoutesPubSubCryptoThroughTheRegisteredProviderAsync`
— builds a container, registers a counting provider, wraps a message and
asserts the provider actually performed the encryption and the
signature. This is the assertion the PubSub claim above now rests on.
- `PolicyScopedRandomBindingDoesNotEscapeItsPolicy`.

Two further assertions are the ones that would catch a silent defect: a
provider and the platform must produce **byte-identical** output — a
validated module that disagreed could not interoperate — and with a
software key the returned `ValueTask` must report
`IsCompletedSuccessfully`, which is what makes "ordering is unchanged"
checkable rather than merely asserted.

`SymmetricChannelCryptoBenchmarks` gains a provider-registered variant
beside the existing baseline, so the cost of the indirection is measured
rather than assumed.

Verified locally, every suite at or above its recorded baseline: `dotnet
build UA.slnx -c Release` **0 warnings, 0 errors on all six target
frameworks**; Core **4413** (net10.0); Core.Security **617**, with its
certificate-leak detector clean; PubSub **1350** plus PubSub.Diagnostics
**54** and PubSub.Udp **233**; Features **156**; Subscriptions.Classic
**32**; Server integration **4053**; Client integration **2123**;
MigrationAnalyzer **151** and MigrationAnalyzer.Core **13**. Zero
failures in each.

Core moved from 4388 to 4381 when the gate change landed — eight
re-entrancy tests were removed along with the behaviour they covered,
and one deadlock regression test added — and has since grown to 4413
with the registry isolation, provider-output and security-review tests.
`ChannelGateTests` was rewritten around the new contract — exclusion
across both entry modes, an uncontended asynchronous entry not
suspending, re-entry blocking by contract, and the property that used to
be the hazard: work started while the gate is held is excluded without
having to opt out.

Every CI failure this PR produced was reproduced locally, diagnosed
against a clean `origin/master` baseline in a second worktree, and fixed
at the source. Two hangs were found from `--blame-hang-timeout` dumps
read with `dotnet-dump`. One failure that looked like a candidate was
confirmed **not** to be: an intermittent `Opc.Ua.Aot.Tests` connect
failure, pre-existing on `origin/master` (5 of 8 baseline runs failed,
versus 3 of 14 here).

### The publish request overshoot

`Subscriptions.Classic.PublishRequestCountAsync` failed on the Windows
net10.0 leg with **207** outstanding publish requests against a cap of
`max(maxServerPublishRequest, subscriptions)` = 50. It failed in 2 of 6
pipeline builds on this branch, never on `master` in 13 sampled builds,
and never locally in 12 attempts including under deliberate CPU
contention.

The mechanism is a genuine race.
`ClassicSubscriptionEngine.QueueBeginPublish` was a **check-then-act**:
it read `GoodPublishRequestCount`, compared it against
`GetDesiredPublishRequestCount(false)`, and issued a publish if it was
below. It runs on every publish completion, so *n* completions that
overlap each read the same value, each conclude they are below the cap,
and each issue — overshooting by up to *n* per round. The read also
*lags*: a request only becomes visible to `GoodPublishRequestCount` once
`AsyncRequestStarted` has recorded it, which happens after the request
has already been sent.

That code is pre-existing and was untouched by this PR. What this PR
changed is how hard the race is hit: the server used to hold the channel
gate across request processing, which serialised every request on a
channel, so publish completions arrived one at a time. Dispatching
`RequestReceived` outside the gate — an intended improvement, since a
channel should not serialise on application request-processing time —
lets them complete concurrently.

Rather than change the publish pipeline blind, the race was first made
reproducible.
`ConcurrentPublishReEvaluationDoesNotExceedDesiredRequestCount` drives
64 barrier-released completions against a mocked context whose count
lags the send exactly the way the session's does. It issued **64
requests where 5 were desired** — the same defect the CI leg hit, now
deterministic and running in milliseconds.

The engine now tracks the requests it has in flight and reserves a slot
with a compare-exchange before sending, so the decision and the send are
one atomic step. The reservation is released when the request completes,
when the send is declined, or when it throws. The repro test then issues
at most the desired count.

`StartPublishing` is deliberately left **uncapped**. It is the recovery
valve that refills a pipeline whose requests are outstanding but no
longer expected to return, so gating it on the in-flight count would
remove the only path out of a stalled pipeline. An earlier revision of
this fix did gate it, and that is the one change here that could not be
justified on the evidence — it was reverted.

The assertion was **not** loosened, retried away, or marked flaky: it
caught a real throttling regression earlier in this PR and is worth
keeping sharp.

Validated: the repro test red before the fix and green after;
`Subscriptions.Classic` 32/32 **five** consecutive runs on net10.0 and
once on net48; `Opc.Ua.Client.Tests` 2124 on net10.0 across four
consecutive runs and 2127 on net48; `Opc.Ua.Subscriptions.Tests` 604
passed with this fix in place.

### The SSH.NET advisory that turned CI red repo-wide

Partway through this work every build job began failing with `NU1903:
Package 'SSH.NET' 2025.1.0 has a known high severity vulnerability`
([GHSA-q939-rpr3-3284](https://github.com/advisories/GHSA-q939-rpr3-3284),
`<= 2025.1.0` affected, first patched `2026.0.0`).

Nothing in the dependency graph changed — the advisory was published.
Build 17026 failed on it where build 17000, two hours earlier on an
identical graph, had passed every build job. SSH.NET is not referenced
anywhere in the repository; it arrives transitively through
`Testcontainers.Kafka`.

`master` landed the remediation independently while this was being
investigated (a central transitive pin to `2026.0.0`, which
`CentralPackageTransitivePinningEnabled` exists for). This branch
briefly carried its own identical pin, which made the merge commit hold
**two** `PackageVersion` entries for SSH.NET and fail restore with
`NU1506`. The duplicate is removed and `master` is merged, so the pin
now comes from one place only.

The audit was **not** suppressed: `NuGetAudit` stays enabled and
`NU1903` stays an error. Verified after the merge — `dotnet restore
UA.slnx` reports neither NU1903 nor NU1506, and `dotnet build UA.slnx -c
Release` is **0 warnings, 0 errors** with the audit live.

The remaining red check in build 17026 was not from this change either.
`aot-ubuntu-latest` is the intermittent `Opc.Ua.Aot.Tests` failure
already noted above — `ReconnectSessionAsync` timed out at 30s with 135
of 136 passing. It remains below the `origin/master` baseline rate (4 of
15 sampled here against 5 of 8 on the baseline), and the managed
reconnect paths are green throughout: `Opc.Ua.Client.Tests` 2124 across
four consecutive runs and `Opc.Ua.Subscriptions.Tests` 604.


## Documentation

`docs/CryptoProvider.md` gains sections on substituting the symmetric
primitives, using a key served over a network, and PubSub — including
what cannot be claimed and why, the scope rule for a random-source
binding, and how to supply a provider-backed policy outside the
container. `docs/MigrationGuide.md` covers the `[Obsolete]` channel
virtuals and `DataLock`, and the non-re-entrant gate's contract for
channel subclasses. `docs/WhatsNewIn2.0.md` is updated.
`plans/cryptooffboard.md` is deleted now that the work it planned has
landed.

## Note for reviewers

This touches the secure channel handshake, which is the most
security-critical code in the stack. Nine distinct concurrency and
lifetime defects were found and fixed in it during this work — three
during development, two in review, four from CI — so `ChannelGate`, the
write path and the certificate ownership across the new suspension
points deserve the closest reading.

Five of those nine traced to the gate's `AsyncLocal`, which is why the
re-entrancy was subsequently removed and the `AsyncLocal` deleted. The
wrapper/`Core` split and the two documented residuals in that section
are the parts most worth a second reading.

The #4206 work removes the last reflection from `Security/Constants` and
rewrites the policy lookup tables, so
`SecurityConfiguration.SupportedSecurityPolicies` and the endpoint set
it feeds are worth a second look: an earlier revision of it changed that
set and broke six server tests, which is now covered by
`SecurityConfigurationBuildsExpectedSupportedPolicySet`.

## Related Issues

- Fixes #4206
- Fixes #4207
- Fixes #4208
- Fixes #4210
- Follow-up #4250 — this PR makes the security policy set injectable,
but nothing in the stack resolves `ISecurityPolicyRegistry` from DI yet,
so every production path still lands on the `Default` fallback. #4250
tracks threading it through `Session` and `UaSCBinaryChannel`, which is
what makes the injection points already added here load-bearing.

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [x] I have addressed **all** PR feedback received.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com>
Copilot-Session: 7a68ca72-83e8-496c-b2f5-3df4c718320b
Copilot-Session: 3f5ab76a-c0ba-4f47-87ac-54d1f4182c6b
2026-08-15 05:23:20 -07:00

28 KiB

Crypto provider

The stack performs its cryptography with the .NET platform by default. This document describes how to replace that: with another library, a remote service, or hardware such as a TPM, an HSM, a PKCS#11 token or a cloud key service — and how to keep private keys inside those devices so they never exist in process memory.

Nothing here changes behaviour until you configure it. A deployment that configures nothing behaves exactly as it did before the provider model existed.

Contents

The short version

System.Security.Cryptography.RSA and ECDsa are abstract classes, and the stack routes every private-key operation through them. Implementations backed by hardware and cloud services already exist — RSACng over a TPM key storage provider, Pkcs11Interop, Azure's RSAKeyVault. The stack therefore does not define its own signing interface, because doing so would make all of those unusable.

What the provider model adds is the part the platform does not express:

Concern Type
Which operations a provider can serve ICryptoProvider, CryptoCapability
Which provider serves which operation ICryptoProviderRegistry, CryptoPurpose
The symmetric, derivation and random operations themselves ISymmetricCryptoProvider, IKeyDerivationProvider, ISecureRandomSource
What may be said about the module behind it CryptoValidationStatus
How strictly that is enforced CryptoCompliancePolicy
Where a new key comes from IKeyPairGenerator
Where a certificate and its key live ICertificateStoreProvider, ICertificateStore

Holding a private key in a device

A device-held key is represented by a detached private key: the Certificate holds the key alongside the X509Certificate2 rather than inside it.

// The device generated the key and will not give it up. Only a handle is available.
RSA deviceKey = OpenKeyInDevice("app-instance");

using Certificate publicOnly = Certificate.FromRawData(certificateDer);
Certificate certificate = publicOnly.CopyWithDetachedPrivateKey(deviceKey);

// HasPrivateKey is true, GetRSAPrivateKey() returns the device key, and the
// stack signs and decrypts with it exactly as it would with a software key.

Using a PKCS#11 token

The optional OPCFoundation.NetStandard.Opc.Ua.Security.Pkcs11 package supplies a certificate store backed by a hardware token, smart card or HSM. It is never referenced by Opc.Ua.Core, so applications that do not use one are unaffected.

services.AddOpcUa()
    .AddPkcs11CertificateStore(new Pkcs11TokenOptions
    {
        ModulePath = "/usr/lib/softhsm/libsofthsm2.so",
        TokenLabel = "opcua",
        PinProvider = () => secretStore.Read("token-pin")
    })
    .AddPkcs11CryptoProvider(CryptoPurpose.ApplicationInstanceKey);

Stores are then addressed with an RFC 7512 URI, so an existing configuration moves to a token by changing only the store path:

pkcs11:token=opcua;object=server?module-path=/usr/lib/softhsm/libsofthsm2.so

The store binds the token key with CopyWithDetachedPrivateKey for the reason below. Signing supports PKCS#1 v1.5 and PSS, decryption supports OAEP and PKCS#1 v1.5, and ECDSA is supported on the curves the token implements. Revocation lists are not held on a token, and objects are provisioned with the vendor's tools rather than through the store.

On Linux the module is loaded through DllImport("libdl"), and glibc 2.34 folded libdl into libc. Distributions that ship only the libdl.so.2 ABI stub need libc6-dev (or an equivalent libdl.so symlink) or the first call fails with DllNotFoundException. That is a property of the interop library rather than of this package.

Why not CopyWithPrivateKey?

X509Certificate2.CopyWithPrivateKey cannot be used for this. On Windows the certificate layer has fast paths only for RSACng and RSACryptoServiceProvider; for any other implementation it falls back to exporting the private parameters, which a non-extractable key refuses by definition:

Approach Result on Windows
CopyWithPrivateKey(custom RSA) CryptographicException
CopyWithPrivateKey(RSACng, ExportPolicy=None) works
CopyWithDetachedPrivateKey(...) works, on every platform

CertificateRequest.CreateSelfSigned is affected for the same reason, since it calls CopyWithPrivateKey internally.

A CNG-backed key may use either route. Everything else — PKCS#11, cloud key services, bespoke implementations — must use CopyWithDetachedPrivateKey.

Lifetime

The detached key is shared by every handle created with AddRef() and is disposed once, with the last handle, unless you pass ownsPrivateKey: false. Each call to GetRSAPrivateKey() returns an independent non-owning view, so the usual using around the returned object is safe and does not destroy the device key.

Exporting such a certificate as PKCS#12 throws rather than silently producing a file without a key.

Selecting a provider per purpose and per policy

Real deployments mix sources. Selection is therefore a resolution over three discriminators — purpose, security policy and certificate type — with the most specific registration winning:

(purpose, policy) → purpose → registered default → platform
services.AddOpcUa()
    .AddCryptoProvider(crypto => crypto
        .For(CryptoPurpose.ApplicationInstanceKey).Use(tpmProvider)
        .For(CryptoPurpose.KeyAgreement).Use(tpmProvider)
        .For(CryptoPurpose.UserIdentityKey).Use(keyVaultProvider)
        .For(CryptoPurpose.CertificateIssuance, SecurityPolicies.ECC_nistP384).Use(hsmProvider));

Purposes:

Purpose Used for
ApplicationInstanceKey Signing and decryption while a secure channel is opened
UserIdentityKey Proving possession of a user certificate when a session is activated
KeyAgreement Ephemeral key agreement for the elliptic curve policies
CertificateIssuance Signing certificates, certificate requests and revocation lists
ChannelSymmetric Per-message symmetric encryption and signing
KeyDerivation Deriving the channel and session key material from a shared secret
RandomNumberGeneration Nonces and other random material

CryptoPurpose is a value type with well-known instances, not an enum, so you can define your own.

A provider bound to a purpose it does not declare is skipped, and resolution falls through to the next candidate. A configuration mistake therefore fails close to its cause instead of deep inside a handshake.

Registration is explicit. There is no assembly scanning: it would make the effective security configuration depend on what happens to be loaded, and it is incompatible with trimming and ahead-of-time compilation.

Where resolution happens

Resolve when something is bound — a channel opening, a certificate loading, a key being generated — and hold the result. The registry is not meant to be consulted per message. ChannelQuotas.CryptoProviders carries the registry to a channel, which resolves once and keeps the result for its lifetime, in the same way it caches the security policy on its token.

Generating a key inside a device

IKeyPairGenerator decides where the key behind a new application instance certificate comes from:

var application = new ApplicationInstance { KeyPairGenerator = new TpmKeyPairGenerator() };

The builder arrives with the subject, subject alternative names and lifetime already set. An implementation only chooses where the key comes from and how the certificate is signed.

An implementation backed by hardware cannot call the parameterless CreateForRSA() or CreateForECDsa(), because those generate a key in software. It must supply the public key generated in the device with SetRSAPublicKey / SetECDsaPublicKey, and sign with an X509SignatureGenerator that calls back into the device.

For the Part 12 push flow the equivalent seam is IPushCertificateKeyGenerator, which is already registered through dependency injection and can be replaced the same way.

Certificate renewal with a device held key

Only one of the two Part 12 flows works with a key that cannot leave its device:

Flow Compatible?
CreateSigningRequest(RegeneratePrivateKey = true) then UpdateCertificate with no private key ✅ the key is generated in the device and only a certificate request leaves it
UpdateCertificate with a private key supplied ❌ raw key material is pushed into the server

Between the two calls the regenerated key has to be staged. HardwarePendingCertificateKeyStore does this for device-held keys: there is nothing to export, because the device already is the durable store, so only the association between the pending certificate and its scope is kept. A software key is declined, so the caller falls back to a store that knows how to protect exportable material.

Validation status, compliance and audit

Every provider declares what may be said about the module behind it:

Level Meaning
FipsValidated The provider names a validation certificate for its module
FipsCapablePlatform It defers to platform cryptography, which is validated when the OS is configured for it
Uncertified It carries no validation — a third-party library or a bespoke implementation
Unknown It declined to say. Treated as uncertified, and always reported

CryptoCompliancePolicy decides how strictly this is enforced:

Policy Effect
Permissive (default) Existing behaviour. Nothing is filtered and nothing is warned about
WarnOnUncertified Every provider carrying no validation is reported
FipsOnly Providers carrying no validation are refused and the server does not start
using var auditor = new CryptoProviderAuditor(registry, telemetry, CryptoCompliancePolicy.FipsOnly);
auditor.Report();              // writes the effective configuration to the log
auditor.ThrowIfNotCompliant(); // refuses to continue if anything is uncertified

Metrics are published regardless of policy, because they are pull-based and cost nothing when nobody reads them:

Instrument Meaning
opc.ua.crypto.providers One measurement per provider, tagged with name and validation level
opc.ua.crypto.providers.uncertified How many providers in use carry no validation

What can and cannot be claimed about FIPS

.NET holds no cryptographic module validation certificate of its own. It calls through to the module the operating system supplies — CNG on Windows, OpenSSL on Linux, CoreCrypto on macOS. Whether that module is running in a validated mode is a property of how the machine is configured, not of this stack.

What can honestly be said: with FipsOnly, the stack performs no cryptography outside the platform modules, and those modules are validated when the operating system is configured for it.

What cannot be said: that the stack itself is validated.

These algorithms are not FIPS-approved and are enabled by default:

Algorithm Note
ChaCha20-Poly1305 Not a NIST-approved algorithm
Brainpool P-256r1 / P-384r1 Not in SP 800-186
Curve25519 / X25519 Not an approved curve
SHA-1 and P-SHA1 (Basic128Rsa15, Basic256) Deprecated for new signatures by SP 800-131A

net472 and net48 additionally use the BouncyCastle.Cryptography package for elliptic curve certificate building. That package is not validated — the validated Bouncy Castle product is a separate, commercially licensed distribution — so those target frameworks cannot make a FIPS claim at all.

What FipsOnly actually enforces at startup

CryptoProviderAuditor.ThrowIfNotCompliant() refuses to start when a provider bound to an operation would not perform it. Carrying the facet is necessary but not sufficient: Supports(algorithm) is consulted again at the point of use, and a provider that answers false there is bypassed in favour of the platform for that algorithm alone. So the audit checks the algorithms of every policy the application actually offers, and names what would fall through:

The compliance policy requires validated cryptography to perform every operation, but the
provider resolved for these cannot perform them and the platform would be used instead:
ChannelSymmetric (ChaCha20Poly1305) for ...#ECC_nistP256_ChaChaPoly, KeyDerivation (HkdfSha256) for ...

A module that covers AES-CBC but not ChaCha20-Poly1305 is therefore caught before it can run — where a facet-only check would have reported it fully compliant while the platform performed every message on any policy negotiating the algorithm it lacks. Nothing is reported when the platform provider is the one resolved: the platform performing the platform's work is not a shortfall.

Narrow the offered policy set, or extend the module, rather than lowering the compliance policy.

Output a provider did not produce is refused

IKeyDerivationProvider.DeriveKey and ISecureRandomSource.GetBytes return void, so a module that no-ops, fills part of the buffer, or swallows an internal failure is indistinguishable from one that succeeded — and the buffer it was handed becomes channel signing keys, encryption keys, initialization vectors and nonces. The platform paths these replace cannot fail this way: Utils.PSHA returns the array it built and RandomNumberGenerator throws.

That matters most for exactly the deployments this seam exists for. A network- or hardware-served module can be transiently offline, and because both ends of a channel usually run the same image, both would derive the same dead key material and the handshake would complete — traffic flowing with no confidentiality and forgeable integrity, invisible to the operator and to the peer.

So the buffer is stamped before the call and checked after it. Output left untouched or zeroed is rejected with BadSecurityChecksFailed rather than used. This cannot prove the output is good — no caller-side check can — it fails closed on output that is provably unusable. Implementations must fill the whole span or throw.

Substituting the symmetric primitives

The asymmetric operations are pluggable because RSA and ECDsa are already the right abstraction. The symmetric ones are not: the platform offers nothing that covers the block cipher, the authenticated cipher and the message authentication code together, so three interfaces declare those operations directly.

Facet Covers
ISymmetricCryptoProvider AES-CBC, AES-GCM, ChaCha20-Poly1305 and the HMAC signatures
IKeyDerivationProvider P_SHA1, P_SHA256, HKDF-SHA256, HKDF-SHA384
ISecureRandomSource Nonces and other random material

A provider opts in by implementing one alongside ICryptoProvider and declaring the matching purpose. They are separate interfaces rather than members of ICryptoProvider, so a provider written before they existed still compiles, and a provider that can serve only some of them says so.

public sealed class ValidatedModule : ICryptoProvider, ISymmetricCryptoProvider
{
    public string Name => "AcmeFIPS";

    public CryptoValidationStatus Validation => new(
        CryptoValidationLevel.FipsValidated, "Acme Cryptographic Module", "CMVP #1234");

    public ArrayOf<CryptoCapability> Capabilities { get; } =
        new(new[] { new CryptoCapability(CryptoPurpose.ChannelSymmetric) });

    // ... the operations
}

services.AddOpcUa()
    .AddCryptoProvider(crypto => crypto
        .For(CryptoPurpose.ChannelSymmetric).Use(module)
        .For(CryptoPurpose.KeyDerivation).Use(module)
        .For(CryptoPurpose.RandomNumberGeneration).Use(module));

The consumer this exists for is a validated software module that must perform every operation, not hardware offload. A device round trip per message would destroy throughput, and nothing here makes that viable.

RandomNumberGeneration is the one purpose with process-wide reach: nonces are created from many places that have no container in scope, so an unscoped binding also becomes the process's nonce source. A binding made for a single security policy — .For(purpose, policyUri).Use(module) — deliberately does not, because it would otherwise redirect nonce generation for every other policy as well.

What it costs when you do not use it

Nothing, by construction. CryptoProviderFacets returns null both when no registry is configured and when resolution lands on the platform provider, because the platform facets perform exactly the code the channel would otherwise run inline. A null facet tells the channel to take its existing path, so the per-message code has no interface dispatch at all unless a provider was registered.

Resolution happens once, in CalculateSymmetricKeySizes, and the result is held for the life of the channel. SymmetricChannelCryptoBenchmarks measures both paths — EncryptSignThenDecryptVerify is the baseline and EncryptSignThenDecryptVerifyThroughProvider is the same work through the seam — so the cost of the indirection is measured rather than assumed.

A provider that cannot do what it was bound to

Binding a provider to ChannelSymmetric without implementing ISymmetricCryptoProvider would otherwise be silent: resolution falls through to the platform and the channel keeps working, while a deployment believes its validated module performed the per-message cryptography.

CryptoCompliance.GetUnservedOperationPurposes reports exactly that case, and under FipsOnly CryptoProviderAuditor.ThrowIfNotCompliant() refuses to start rather than run on cryptography the operator did not ask for.

Using a key served over a network

RSA and ECDsa are synchronous contracts, and they are .NET's rather than this stack's, so a key backed by a cloud key service occupies a thread for the whole of every call. Rather than replace those contracts — which would make every ready-made hardware and cloud implementation unusable — an implementation may also declare an asynchronous path:

public sealed class KmsRsa : RSA, IAsyncRsaKey
{
    // The synchronous members are still implemented, and are what the paths
    // that are not yet asynchronous will use.

    public async ValueTask<byte[]> SignHashAsync(
        ReadOnlyMemory<byte> hash,
        HashAlgorithmName hashAlgorithm,
        RSASignaturePadding padding,
        CancellationToken ct = default)
        => await m_kms.SignAsync(hash, hashAlgorithm, padding, ct);

    public async ValueTask<byte[]> DecryptAsync(
        ReadOnlyMemory<byte> data,
        RSAEncryptionPadding padding,
        CancellationToken ct = default)
        => await m_kms.DecryptAsync(data, padding, ct);
}

The stack finds the facet by type test, so a key that does not implement it is unaffected and the asynchronous paths complete synchronously for it — ordering, and everything that depends on it, is unchanged.

Path Asynchronous?
Secure channel open and renew ✅
User identity token signing and decryption, session activation ✅
Service faults, and the synchronous reconnect handoff ❌ both are reached from synchronous call sites
Certificate, certificate request and revocation list signing ❌ by construction — X509SignatureGenerator.SignData is called by .NET internals

PubSub

The per-message cryptography a publisher and subscriber apply — AES-CTR and HMAC-SHA-256, per Part 14 §7.2.4.4.3.1 — routes through ISymmetricCryptoProvider when one is registered, so a validated module performs it:

services.AddOpcUa()
    .AddCryptoProvider(crypto => crypto
        .For(CryptoPurpose.ChannelSymmetric).Use(module))
    .AddPubSub(...);

The policies resolve the provider once, when they are constructed, and hold it; nothing consults a registry per message. A provider that does not declare the algorithms a policy needs is ignored rather than used, so a configuration mistake does not stop publishing. IPubSubSecurityPolicy is unchanged — a provider is supplied through the constructor, so implementations of that interface outside this stack are unaffected.

The wrapper resolver selects the bundle it wraps with from the registered policies rather than from a static default, which is what makes the registration above take effect. Outside the container, PubSubApplicationBuilder does the same with the policies it holds, and WithSecurityPolicySelector overrides the choice per connection:

var module = new ValidatedModule();

new PubSubApplicationBuilder(telemetry)
    .WithSecurityPolicySelector(
        (_, _) => new PubSubAes256CtrPolicy(module))
    // ...
    .Build();

What device custody can and cannot mean here

With a standard Security Key Service the key necessarily exists in process memory. GetSecurityKeys (Part 14 §8.3.2) returns raw key bytes over the wire, so the property the client and server side achieve — the key never leaves the device — cannot be achieved for PubSub through the SKS pull profile. That is a property of the specification, not of this stack.

Introducing a wrapped-key envelope would change what is on the wire and break interoperability with third-party key services and publishers, so it is deliberately not done.

What is achievable, and is supported:

  • The operations can be performed by a validated module, as above.
  • The key material can come from somewhere other than the SKS. IPubSubSecurityKeyProvider is the seam; an implementation may derive per-token keys from a long-lived secret that stays in a device, so only the derived material is in memory.
  • Its lifetime is bounded. PubSubSecurityKey zeroizes on disposal, the key ring disposes keys as it retires them, and the intermediate copies made while unpacking an SKS response are cleared rather than left in the heap.

Contributing a security policy

The policy set is no longer fixed at compile time. A provider that implements a profile this stack does not ship — a national or vendor profile, or one added by a later specification — can make it visible at runtime by constructing a SecurityPolicyInfo and registering it:

using IDisposable registration = SecurityPolicies.Register(
    new SecurityPolicyInfo("urn:vendor:SecurityPolicy#CustomProfile", "CustomProfile")
    {
        PlatformSupport = () => true,
        SupportedCertificateTypes = [ObjectTypeIds.RsaSha256ApplicationCertificateType],
        IsDefault = true
    });

or through the builder, alongside the provider that performs its cryptography:

services.AddOpcUa()
    .AddCryptoProvider(crypto => crypto
        .For(CryptoPurpose.ChannelSymmetric, "urn:vendor:SecurityPolicy#CustomProfile").Use(module))
    .AddSecurityPolicy(customPolicy);

A registered policy is discoverable through the same API as a built-in one — GetInfo, GetUri, GetDisplayName, GetDisplayNames and the default-URI helpers — because those are now driven from one table rather than from reflection over the constants. Registering a URI or name that already exists throws unless replaceExisting: true is passed, which makes shadowing a built-in policy deliberate and reversible: disposing the returned registration restores what was there before.

Resolving the policy set

Those lookups are members of ISecurityPolicyRegistry, which is the object that owns the policy set. Resolve it to work against the policies this application registered:

public sealed class MyEndpointPicker(ISecurityPolicyRegistry policies)
{
    public string[] Offered => policies.GetDisplayNames();
}

AddSecurityPolicy registers the registry for you; call AddSecurityPolicyRegistry() when you want to resolve one without contributing a policy of your own.

The identity token handlers take one too, so a token's security policy URI resolves against the policies that application offers rather than the process-wide fallback:

var handler = new UserNameIdentityTokenHandler(username, password, policies);

Passing nothing keeps the previous behaviour and uses SecurityPolicies.Default.

The registry a container builds is its own. A policy registered by one application is not visible to another hosted in the same process, and it is not visible to SecurityPolicies.Default. That fallback carries exactly the built-in set and is what the paths that run before any container exists — configuration loading, most obviously — resolve their policies through:

string? uri = SecurityPolicies.Default.GetUri("Basic256Sha256");

Construct a registry directly when you want an isolated set, which is also what makes the policy set testable without touching process-wide state:

using var policies = new SecurityPolicies(telemetry);
using IDisposable registration = policies.Register(customPolicy);

The registry creates its logger from the ITelemetryContext it is given, which is why Encrypt, Decrypt and the signature helpers take no ILogger argument.

PlatformSupport is what decides whether the policy is offered on the machine it is running on, so a policy whose algorithms are unavailable is filtered out the same way the built-in ones are. The ECC_curve25519 and ECC_curve448 profiles are the worked example: they ship in the tree but are not advertised, and a consumer can light them up from outside without rebuilding the stack — which is what RegisterLightsUpCurvePoliciesFromOutsideCore asserts.

Registering a policy makes it advertised and resolvable; it does not by itself supply the cryptography behind it. For the two curve profiles above, the in-tree key agreement is still behind a compile-time symbol that no project defines and a BouncyCastle dependency, so a deployment that lights them up supplies the operations through a provider, exactly as the sections above describe. That is the intended division: the policy set says what is offered, the provider says who performs it.

Removing the reflection that used to build these tables also removed the last reflection in Opc.Ua.Core.Security.Constants, which is why this is also a trimming and Native AOT improvement.

Limitations

  • HTTPS with a device-held key does not work on Windows or macOS. SChannel and the macOS Security framework require keys registered with a platform key storage provider. It works on Linux, where the TLS layer dispatches through the managed key. UA-TCP is unaffected on every platform.
  • The per-message symmetric path is not offloaded to hardware. Session keys are symmetric and derived per channel token; a device round trip per message would destroy throughput. Hardware is used only for the operations that happen when a channel opens or a session is activated. Substituting a software implementation is supported — see above.
  • Certificate issuance cannot be made asynchronous at all. X509SignatureGenerator.SignData is called by .NET's own CertificateRequest and CRL builders, so signing a certificate, a certificate request or a revocation list with a remote key occupies a thread by construction. Service faults and the synchronous reconnect handoff are in the same position, because both are reached from call sites that cannot become asynchronous without a contract break.