mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-25 11:50:03 +08:00
fix/encoder-stack-depth-json-flake
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 romanettandClaude Opus 5.5
|
1fb1134974 |
Tests: scope MeterListeners to the meter under test (fix flaky ClientBaseTests metrics test) (#4708)
## Problem `ClientBaseTests.ActivityTraceFlags_Metrics_ShouldRecordMetricsOnly` failed intermittently in full-suite runs: ``` Expected: "opc.ua.client.request.duration" But was: "opc.ua.channel.reconnect.duration" ``` `TelemetryContextBase.CreateMeter()` returns a new `Meter` per call, named after the calling assembly, so `ClientBase`'s meter and `ClientChannelManagerMetrics`' meter are both named `Opc.Ua.Core`. `TestMeterListener` enabled every instrument whose `Meter.Name` matched, so reconnect-duration measurements from `[Parallelizable]` channel manager fixtures were recorded and could precede the request duration. ## Fix (test-only) - `ClientBaseTests.TestMeterListener`: enable instruments only when `ReferenceEquals(instrument.Meter, meter)`, i.e. the meter instance the client under test created. Assertions unchanged. - `CryptoProviderModelTests.AuditorPublishesAProviderMetric`: same flaw — it enabled every `opc.ua.crypto*` instrument process-wide (fixture is `ParallelScope.All`), so another auditor's uncertified gauge could overwrite the observed count. Now scoped to the auditor's own meter (read via reflection, as `ClientBaseTests` already does for `ClientBase`). Other `MeterListener` users reviewed and left as-is: `NodeCacheTests` / `AdapterMetricsTests` are non-parallel fixtures; channel manager listeners only assert existence and filter by endpoint tag. ## Verification - `ClientBaseTests` + `CryptoProviderModelTests`: 4× consecutive, 25/25. - `FullyQualifiedName~Opc.Ua.Core.Tests.Stack` (net10.0): 2 clean passes, 3548 passed / 80 skipped. One attempt hung in unrelated `ClientChannelManagerManagedTests` async tests (`ReconnectAsyncNotifiesAttachedParticipantsAsync` / `ReleasingOneLeaseKeepsChannelAliveForOtherAsync`) and was rerun — a separate intermittent hang. - net48 build of `Opc.Ua.Core.Tests`: 0 warnings, 0 errors; `dotnet format whitespace` clean. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
7fe7794a19 |
Enable AES-GCM and ChaCha20-Poly1305 policies on net48 via BouncyCastle (stacked on #4703) (#4705)
> Follow-up to #4703 (now merged to master; this PR has been retargeted
to `master`).
Part of #4510. #4703 enabled the CBC ECC policies on net48 through a
BouncyCastle raw-ECDH polyfill. The AEAD policies were still unavailable
there because .NET Framework has no `AesGcm` or `ChaCha20Poly1305`. This
PR adds those ciphers the same way, so on net48 all of these are now
offered:
`ECC_{nistP256,nistP384,brainpoolP256r1,brainpoolP384r1}_AesGcm` /
`_ChaChaPoly`, `RSA_DH_AesGcm`, `RSA_DH_ChaChaPoly`.
## Changes
- **Polyfill**
`src/Opc.Ua.Security.Certificates/Org.BouncyCastle/AeadCiphers.cs`
(internal, `#if NETFRAMEWORK`, next to `EcdhRawAgreement`): `AesGcm` and
`ChaCha20Poly1305` types with the .NET 8 BCL surface. They wrap
BouncyCastle's `GcmBlockCipher(AesEngine)` and
`Org.BouncyCastle.Crypto.Modes.ChaCha20Poly1305`.
- A tag mismatch throws `CryptographicException`, never BC's
`InvalidCipherTextException`, so
`PlatformSymmetricCryptoProvider.DecryptAuthenticated` still returns
`false` and the channel still raises a protocol error.
- GCM releases plaintext blocks before it checks the tag, so plaintext
stays in a scratch buffer and is copied out only after the tag verifies.
On failure the destination is cleared, as the BCL does. Scratch buffers
and the key copy are zeroed.
- **Opc.Ua.Core**: aliases the polyfill types on `NETFRAMEWORK` and
removes the `NET8_0_OR_GREATER` gates in `SecurityPolicies`
(`SupportsAesGcmPolicy` / `SupportsChaCha20Poly1305Policy`),
`CryptoUtils` (`SymmetricEncryptAndSign` / `SymmetricDecryptAndVerify`,
AEAD helpers) and `PlatformSymmetricCryptoProvider`. The channel, the
provider facet and the encrypted ECC user tokens (`EncryptedSecret`,
through `CryptoUtils`) now run the same code on every target. .NET 8+
keeps the BCL ciphers.
- **FIPS** (requested by the #4703 session, after
|
||
|
romanettandClaude Opus 5.5
|
d552634258 |
Enable ECC security policies on net48 via a BouncyCastle raw-ECDH polyfill (#4703)
Closes #4510. ## Summary OPC UA Part 6 feeds the **raw** ECDH shared secret into HKDF. .NET Framework only exposes `ECDiffieHellman.DeriveKeyMaterial`, which always hashes it, so the ECC SecureChannel and user-token policies were unavailable on `net48` (`SecurityPolicies.GetInfo` returned `null`). This adds a BouncyCastle polyfill for just that primitive: - **`Opc.Ua.Security.Certificates/Org.BouncyCastle/EcdhRawAgreement.cs`** (`net48` only): an internal `DeriveRawSecretAgreement(this ECDiffieHellman, ECDiffieHellmanPublicKey)` extension using `ECDHBasicAgreement`. It returns the X coordinate padded to the field size, exactly like the .NET 8 BCL method. Keys stay CNG keys; ECDSA signatures and HKDF stay on the platform — only the agreement step is managed code. The point is validated (on curve, not infinity) and mismatched curves are rejected. - Because the extension mirrors the .NET 8 signature, `Nonce.GenerateSecret` now has one code path on every target. - `SupportsRawEccSecretAgreement()` is removed; the CBC-based ECC policies (`ECC_nistP256`, `ECC_nistP384`, `ECC_brainpoolP256r1`, `ECC_brainpoolP384r1`) are offered on every target (still subject to OS curve support). - BouncyCastle was already a `net48` dependency of `Opc.Ua.Security.Certificates`; no new package. (Only `net48` is downlevel now — `netstandard2.1`/`net472` from the issue text are no longer targets.) **Not in scope:** the `_AesGcm` / `_ChaChaPoly` ECC variants (and PubSub DTLS) still need .NET 8 because .NET Framework has no AEAD ciphers — follow-up stacked on this branch. ## Docs `EccProfiles.md` (Known limitations), `migrate/2.0.x/certificates.md`, `ProductionChecklist.md`, `Profiles.md`, `WhatsNewIn2.0.md`. Calls out that on `net48` the agreement step does not run in a FIPS/validated module. ## Tests (local) | Suite | Result | |---|---| | `Opc.Ua.Core.Tests` full, net48 | 5956 passed, 0 failed | | `EccInteropTests` net48 vs 1.5.378 peer (net8.0, native raw ECDH) | 26 passed; 28 skipped (AEAD + curve25519/448) — all four CBC curves, both directions, UserName tokens, token renewal | | `Opc.Ua.Sessions.Tests` ECC filter, net48 | 52 passed, 0 failed | | Server `SessionCoverageTests`/`EphemeralNonce*`, Configuration builder tests, net48 | all pass | | Nonce/ECC/EncryptedSecret + session tests, net10.0 | all pass | New tests: symmetric agreement on all four curves; on net48 an oracle against CNG (`SHA256(raw) == DeriveKeyFromHash`); curve mismatch and off-curve rejection. Tests that asserted the old net48 limitation were updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
9b034f8291 |
Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
ce88955f70 |
Harden encoders/decoders against untrusted input and align them with Part 6 / Part 14 (#4599)
# Description
Hardens the encoders and decoders against untrusted input and aligns
them with OPC 10000-6 (1.05) and OPC 10000-14. This comes from an audit
of every codec reachable from a peer:
- Binary, JSON and XML decoders and encoders;
- built-in and complex types;
- UA-TCP/UASC framing;
- PubSub UADP and PubSub JSON.
The audit found 97 issues: 1 critical, 7 high, 39 medium and 50 low.
They are all fixed here, one commit per finding (IDs B1-, J1-, X1-, T1-,
S1-, U1-, P1-, E1-, E2- in the commit subjects). Every fix was then
checked against the spec a second time, and the deviations that re-check
found were fixed as well (`*-spec` commits).
## Security (decoders reachable from unauthenticated peers)
- **Allocation from length prefixes (B1-1, B1-5, U1-1, U1-3).**
- Array, string and byte-string lengths are checked against the bytes
left in the message before anything is allocated.
- Arrays read element by element grow as elements are decoded, so nested
length prefixes can no longer multiply memory.
- UADP discovery counts and chunk reassembly buffers are bounded the
same way.
- **Stack exhaustion.** Nesting is checked for remaining stack
(`RuntimeHelpers.EnsureSufficientExecutionStack` mapped to
`BadEncodingLimitsExceeded`), and the large Variant read/write methods
were split into scalar, array and matrix methods. Measured on 1 MB and
256 KB threads, no codec overflows at the default limits any more.
- **XML element depth (X1-1).** XmlElement values and raw
ExtensionObject bodies are copied node by node with a depth limit,
instead of with the recursive `InnerXml`, `ImportNode`, `InnerText` and
`DeepEquals` calls.
- **Limit bypasses:**
- JSON NodeId, QualifiedName and opaque strings now respect
MaxStringLength and MaxByteStringLength (J1-1).
- Matrix rank is limited to 32.
- Decimal size is bounded (T1-3).
- XML base64 is checked before it is decoded.
- XML arrays are checked before each element is read.
- **CPU and algorithmic cost:**
- UADP reassembly uses an O(1) overlap check with a per-message chunk
cap (U1-2).
- PubSub JSON matches fields to metadata through a name index (P1-2).
- LocalizedText "mul" text is parsed lazily (T1-5).
- Unknown SecurityTokenIds trigger a throttled background SKS refresh
instead of an inline call per datagram (U1-9).
- **PubSub metadata poisoning (P1-1).** Inbound metadata is registered
only for identities a configured reader expects, is capped, and an
unauthenticated message can no longer pin a version.
- **Exception contract.** Decoders fail with `ServiceResultException`
(BadDecodingError / BadEncodingLimitsExceeded) instead of letting
ArgumentException, InvalidOperationException, TypeLoadException and
similar escape.
- **UASC (S1-*):**
- OPN and CLO must be single final chunks.
- The certificate of an unsigned OPN is ignored.
- The padding and ExtraPadding bytes are validated.
- A renewed token is activated only after a chunk has been verified with
it.
- A policy change on renewal is rejected before the channel certificate
is touched.
- Secured chunk length arithmetic is bounded.
## Wire-visible behaviour changes
All follow the spec exactly; there is no reader for the old forms unless
noted.
- **UA Binary:**
- Variant type ids 26-31 decode as ByteString.
- Picoseconds without their timestamp are ignored, and values of 10000
and above are clamped to 9999.
- A Null-type Variant with the array flag consumes its length and
dimensions.
- **UA JSON:**
- Messages are ExtensionObjects with the body inline; there is no
`UaBody` envelope any more (5.4.9 / 5.4.2.16).
- The Variant `Value` is omitted exactly for NULLs of nullable types
(Table 41).
- RawData keeps nested UaType/UaTypeId and the StatusCode Symbol.
- Duplicate member names are rejected.
- `{}` decodes as a null ExtensionObject.
- Numbers and identifiers are parsed strictly and culture-independently.
- QualifiedName follows 5.4.2.14.
- Opaque bodies of known types carry the DataType id.
- Enumerations without a literal are written as the number.
- **UA XML:**
- The earliest DateTime is written as `0001-01-01T00:00:00Z`.
- Null typed scalars and null array elements are written with `xsi:nil`.
- An ExtensionObject without a body has no `Body` element.
- Strings XML cannot represent are rejected.
- Only the xs:dateTime and xs:date forms are accepted; a value without a
zone is still read as UTC, because published model files use it.
- Only the Part 6 Guid form is accepted.
- Booleans are case-sensitive in XmlDecoder.
- **UA-TCP:**
- OPNC/OPNA/CLOC/CLOA fault the channel. Older releases that split a
long-chain OPN into chunks are rejected, so this needs a release note.
- The sender truncates the certificate chain to fit a single chunk.
- The received certificate chain is sized from the chunk size.
- Hello buffer sizes down to 1024 bytes are honoured; below that, an ERR
with Bad_TcpInternalError is sent.
- A 4096-byte Error reason is accepted, and the SecurityPolicyUri limit
is 255 bytes.
- **PubSub UADP:**
- Chunk NetworkMessages use the Part 14 7.2.4.4.4 layout, and each chunk
is secured on its own with its own sequence number.
- The Sizes array is part of the secured payload.
- Padded RawData keeps its length prefix, and null values are written as
-1.
- DataSetMessages with the valid bit clear are skipped.
- Null discovery arrays are accepted.
- **PubSub JSON:**
- Variants use `UaType`/`Value`, with the 7.2.5.4.2 top-level collapse.
The deprecated `Type`/`Body` form is still read.
- Status is written as a `{Code,Symbol}` object; a numeric Status is
still accepted.
- Discovery messages follow Tables 188-192, with GUID MessageIds.
- Enumeration fields are written as `Name_Value`.
- Fields appended with a MinorVersion bump are accepted.
## Known deviations and limitations (documented, not changed)
- A known type whose ExtensionObject XML body fails to decode fails the
message, like a failing binary body. 5.2.2.15 only says the decoder
*should* continue, and continuing reopens the log-flooding path.
- Reserved Variant type ids 26-31 are passed to the application as
ByteString without the raw type id.
- JSON UaTypeId of an opaque body of an *unknown* type stays the
encoding id, because no type system is available to map it.
- PubSub JSON encoder:
- Structure fields without a subtype are still wrapped with a UaType,
and concrete multi-dimensional or abstract-rank fields still carry a
UaType (Table 186).
- Namespace-0 enumerations without EnumDataTypes are written as a
numeric string.
- Chunked transcoded frames have no writer group, so their sequence
numbers continue from the source number.
- Pre-existing and out of scope: the HTTPS JSON binding uses
Content-Type `application/opcua+uajson`, while 1.05 7.4.5 requires
`application/json`.
## Tests
- Every fix has a regression test.
- Tests that pinned non-conformant behaviour were updated, and their
commit messages say so.
- Fuzz seeds were regenerated where the wire changed.
Local results:
net10.0, 27 suites green (Types 11354, Core.Encoders 2933, Encoders.Fuzz
8112, Core 5955 + 1 known intermittent ActivityTraceFlags metrics test
that passes alone, Client 2732, Server 7802, Sessions 790, GDS 1234,
Subscriptions 641, SourceGeneration.Core 4266, PubSub 1490 and all
PubSub transport suites). Full UA.slnx build 0 warnings / 0 errors on
net10.0 and net48.
## Related Issues
- Follow-up: #4598 (possible throughput regression of per-element binary
array reads).
## Checklist
- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed. (net10.0
suites run; net48 build only)
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
20c2d77d57 |
Fix Core/Types audit findings and conform them to the OPC UA spec (stacked on #4569) (#4573)
> **Stacked on #4569.** The base branch is `romanett/practical-murdock-fd9c56`, so this diff shows only the commits of this PR. Retarget to `master` after #4569 merges. ## Summary Bug audit of `Opc.Ua.Core`, `Opc.Ua.Core.Types`, `Opc.Ua.Types` and `Opc.Ua.Core.Schema`, with the findings fixed and the fixes grounded against the OPC UA specification (reference.opcfoundation.org). 133 commits in four groups: 1. **Audit fixes** (`Fix <ID>: ...`): 116 findings from a read-only audit, each with a regression test. IDs follow the audit scopes: T1-T6 `Opc.Ua.Types`, C1-C4 `Opc.Ua.Core`, S1 `Opc.Ua.Core.Types` + `Opc.Ua.Core.Schema`. 2. **Review fixes** (`Fix review <ID>: ...`): issues a code review of group 1 found, plus lower-severity leftovers. 3. **Spec fixes** (`Spec: #<n> ...`): the 18 places where an earlier fix only partly matched the spec text. 4. Test and analyzer follow-ups. Where `master` (#4516, #4522, #4540, #4564) or #4569 already fixed the same issue, their version was kept and this PR only adds what they didn't cover. ## Behaviour and user-facing changes ### Wire format and interoperability (breaking) - **PubSub AES-CTR (Aes128-CTR / Aes256-CTR)** now follows Part 14 Tables 156/157: - The counter block is `KeyNonce[4] | MessageNonce[8] | BlockCounter[4]`, and the block counter starts at 1. - The SecurityHeader nonce is 8 bytes: `Random[4] | UInt32 SequenceNumber`, with the sequence number starting at 1. - SKS key data is split with a 4-byte KeyNonce, giving 52 / 68 bytes. - **Not interoperable with earlier releases of this stack**, but matches the spec. There is a test against the independent RFC 3686 vector. - **User token signature** (Part 4 6.1.8): - The signed data is now built from the user token policy's SecurityPolicy on both client and server. - It also supports the SecurityMode None layout `ServerNonce | HASH(ServerCertificate) | ClientNonce`. - New overload `SecurityPolicyInfo.GetUserTokenSignatureData(..., MessageSecurityMode)`. - **Secure channel:** - The ECC AEAD `LastSequenceNumber` is correct across the 32-bit sequence wrap (Part 6 6.8.1). - The incoming chunk counts against MaxChunkCount/MaxMessageSize. - The sender no longer lets a message through that is 24 bytes over MaxMessageSize. - The OPN response signature is verified without overrunning the buffer (RSA_DH with differing key sizes). ### Sessions (server) - **CreateSession nonce:** - The client nonce is kept on None channels, because the None-mode user token signature covers it. - A nonce over 128 bytes returns `Bad_NonceInvalid` on every channel (Part 4 5.7.2.3). - The duplicate-nonce check runs only on secured channels. - A non-empty nonce under 32 bytes is still accepted on None, for compatibility. - **ActivateSession:** an embedded `EccEncryptedSecret` signing certificate is validated with the server's peer certificate validator (Part 6 6.8.3). Byte equality with the client certificate is only the fallback when no validator is available. New opt-in interface `ICertificateValidatorProvider`. - **FindServers / GetEndpoints:** `GatewayServerUri` and `DiscoveryProfileUri` are no longer swapped. ### Write, Read and Call (NodeState / node managers) - **UserWriteMask:** - Enforced for non-value attributes and for VariableType Value writes (`ValueForVariableType`), returning `Bad_UserAccessDenied`. - An unconfigured UserWriteMask (0, no handler) is not enforced, and reads back as the WriteMask so Read and Write agree (Part 3 8.60). - A throwing or Bad `OnReadUserWriteMask` handler fails only that item. - **Index-range writes:** - Slices are merged into the cached value under the attribute lock, also for `OnWriteValueAsync` handlers. - A data-change notification is still produced when the cache cannot absorb the slice. - The slice must match the unclipped range, otherwise `Bad_IndexRangeDataMismatch` (new status code in `Opc.Ua.Types`). - A range past the target returns `Bad_IndexRangeNoData`. - A handler without index-range support returns `Bad_WriteNotSupported` (Part 4 7.27, 5.11.4.4). - **Index-range reads on matrices** return partial results clipped to the source, instead of allocating the requested size (a Read with `"0:40000,0:40000"` no longer allocates about 1.6e9 elements). - **DataTypeDefinition writes:** - A value that isn't a decoded DataTypeDefinition, or whose kind doesn't match the DataType category, returns `Bad_TypeMismatch`. - Null is rejected for concrete Structure/Enumeration/OptionSet types. - WriteMask is checked first (Part 3 5.8.3). - **AnalogItem:** a null write skips the InstrumentRange check again. - **Method call:** the async fallback passes the ObjectId to the method handlers. - **Redundancy:** `DistributedValueParticipation` merges index-range writes and applies the index range on reads. - **Change masks:** `ClearChangeMasks` takes the bits atomically, so concurrent changes are no longer lost. - **`AccessLevelEx`:** preserved by Clone/Export. ### Alarms & Conditions (Part 9) - **Acknowledge / Confirm / AddComment:** - Accept any EventId the condition or its branches reported while that state is still pending (5.7.3). - An unknown id returns `Bad_EventIdUnknown`. - An already handled state returns `Bad_ConditionBranchAlreadyAcked` / `...AlreadyConfirmed`. - AddComment on a branch no longer changes the trunk, and the branch table is re-keyed when a branch's EventId changes. - **Shelving:** - OneShotShelve without MaxTimeShelved no longer overflows the timer. - A timed shelve is measured on the monotonic clock, so wall-clock steps don't delay the unshelve. - The state is named `OneShotShelved`. - **Audit events:** - They report failure for rejected method calls. - `SelectedResponse` is a UInt32. - A rejected negative Respond leaves SelectedResponse unset. - **`ContentFilterElement.Validate`:** an unknown FilterOperator is reported per element as `Bad_FilterOperatorInvalid`, instead of failing the whole service call. ### ContentFilter evaluation (Part 4 7.7.3, on top of #4564) - **Bitwise operators:** - The result takes the size of the larger operand. - Boolean, numeric String and one-element array operands are converted to integers implicitly. - Types without an implicit integer conversion give NULL. - **Strings:** ordering is ordinal, no longer culture dependent. - **Cast:** restricted to the Table 121 conversions, and QualifiedName to LocalizedText is added. The general `Variant.ConvertTo` keeps its wider set. - **Null and empty operands:** - `Like` with a NULL operand is NULL. - Empty arrays of the same DataType compare equal. - Null-string operands are NULL everywhere. ### Encoders and built-in types - **XML:** - Strings keep leading, trailing and whitespace-only content (Part 6 5.3.1.5). NodeSet import opts back into treating whitespace-only strings as empty with the new `XmlDecoder.TreatWhitespaceOnlyStringsAsEmpty`. - Layout whitespace around `<Identifier>` values is ignored, and trailing control characters are stripped from `s=` ids. - `XmlElement` ignores DTDs without expanding entities. - **JSON:** - A DateTime without an offset is read as UTC. - An unmappable `nsu=` NodeId and an unknown `svu=` ExpandedNodeId decode to a namespace-0 String NodeId (5.4.2.10 / 5.4.2.11). - `svr=` is rejected in a NodeId field. - Bad matrix dimensions return `BadDecodingError`. - The non-strict body stack stays balanced after an error. - The nesting limits match between encoder and decoder. - **Binary:** - `MaxByteStringLength` no longer applies to strings. - ExtensionObject bodies count toward the nesting limit on seekable streams (and in the XML encoder). - **Parsing:** - The NodeId parser requires `i=`/`s=`/`g=`/`b=`. - `ExpandedNodeId.Parse` rejects an empty `nsu=`/`svu=`. - NumericRange rejects a trailing `,` with `Bad_IndexRangeInvalid`. - RelativePath escapes `#`. - **Status codes:** `StatusCode.WithSubCode`/`WithCodeBits` keep the severity bits and look up the symbolic id for the new code. - **Variant / VariantHelper:** - `TryCastTo` returns false on a type mismatch. So `DataValue.GetValue<T>(default)` returns the default, and `GetValueOrDefault` / `CastTo` throw `BadTypeMismatch` as documented. - `System.DateTime` casts are supported. - Concrete encodeable arrays (e.g. `Argument[]`) no longer throw. - `ConvertTo` keeps one-element and empty arrays as arrays and checks the conversion is legal for empty arrays. - `CastFrom(object[])`, `CastFrom(Enum[])` and the obsolete `Variant(object)` constructors keep the array shape (previously a one-element array collapsed to a scalar and an empty one to Null). - `TypeInfo.Unknown` / `Variant.Null.TypeInfo` report `ValueRanks.Any`. - **Equality and hashing contracts fixed:** - Types: Matrix/MatrixOf, typed-null Variants, NodeId (the namespace index is now hashed), ExpandedNodeId, LocalizedText, DiagnosticInfo, StructureDefinition/EnumDefinition, CertificateIdentifier, UserIdentity and DateTimeUtc. - `MatrixOf.Dimensions` returns a copy. - Decimal equality and hashing now run in bounded time. - **Localized text:** a LocalizedText with locale `"mul"` and no usable translations, or null client locale ids, no longer throw. ### NodeSet, address space and utilities - **NodeSet export/import:** - Server index 0 is the local server on import and export, so local ids are no longer written as `svr=65535`. - `StructureDefinition.BaseDataType` is filled from the supertype on import. - **`NodeTable.Import`:** replaces duplicates, keeping the references neighbouring nodes hold. `TypeTable.Add` detaches a re-parented type from its old supertype, and `Node.Clone` gets its own ReferenceTable. - **`StringTable` / `NamespaceTable`:** capped at UInt16 indexes (`BadEncodingLimitsExceeded`) for Append and Update, instead of silently wrapping. - **Small fixes:** - `UnsecureRandom.Shared` is seeded randomly. - The peer's RetryAfterMs is clamped to one day. - The `CancelAsync` and `string.GetHashCode(StringComparison)` polyfills match BCL behaviour. - `CoreUtils.IsEqual` compares strings ordinally. ### Schema generation and ComplexTypeSystem - **JSON schema:** describes the 1.05 layout with the compact/verbose null rules, and now validates real `JsonEncoder` output. - **XSD:** - `EncodingMask` (`xs:unsignedLong`) only for StructureWithOptionalFields. - Guid and StatusCode are complex types. - Standard types are in the `Types.xsd` namespace. - **Structure-backed OptionSets** are described as `{Value, ValidBits}`. New `UaTypeDescription.IsStructureOptionSet` property and constructor overload. - **ComplexTypeSystem:** - A malformed type in a server dictionary is skipped per type instead of discarding the whole dictionary. - Cyclic supertype walks are bounded. - Declared definitions are no longer changed. - A dictionary validator is kept only if the dictionary parsed. ### Certificates and trust lists (Part 4 6.1.3, Part 12) - **Revocation:** - Checked for issuers supplied in the peer's chain and for inline-trusted issuers, against the CRLs in the trusted and issuer stores. - A store without CRL support no longer hides "revocation unknown". - `RejectUnknownRevocationStatus` is honoured for inline-trusted issuers unless the entry sets `SuppressRevocationStatusUnknown`. This changes the strict case of `ExplicitIssuerChainIsHonoredWithoutStorePathAsync`. - **Basic constraints / path length** violations fail with `Bad_CertificateInvalid`. - **`TreatAsInvalid`:** - Now yields `Bad_CertificateUntrusted`, which can't be suppressed or auto-accepted. - It is honoured on individual trust-list entries even when the certificate is also in a store. - **`EccEncryptedSecret`:** enforces the result of validating the sender certificate. - **`TrustListTransaction`:** - Routes each CRL to the store holding its issuer (TrustedCrls/IssuerCrls). - Validates staged CRLs, including the store's CRL support, before changing anything. - Applies certificates before CRLs. - **Other:** - One failing `CertificateChanges` observer no longer stops the others. - `InMemory:` stores share their contents per path, and `InMemoryStoreProvider` is `IDisposable`. ### WoT (`Opc.Ua.Types/Wot`) - **Projection security:** projection security schemes whose qualified names collide are refused. New diagnostic `WotDiagnosticCode.ProjectionSecurityConflict = 6063`. - **Input handling:** - Invalid Unicode (lone surrogates, bad UTF-8) is rejected as `JsonException`. - External-schema results are keyed per affordance. - A duplicate security-floor member is rejected. - `nsu=` ids are percent-encoded (Part 6 5.1.12). - The configured JSON depth is honoured on re-parse. - An oversized document is reported before it is parsed back. - **Performance and limits:** - Subtype-graph validation, `componentOf` linking and the namespace table are now linear. - Namespaces are capped at UInt16. ### New or changed public API - `ICertificateValidatorProvider` (server) - `SecurityPolicyInfo.GetUserTokenSignatureData(..., MessageSecurityMode)` overload - `XmlDecoder.TreatWhitespaceOnlyStringsAsEmpty` - `UaTypeDescription.IsStructureOptionSet` plus a constructor overload - `WotDiagnosticCode.ProjectionSecurityConflict` - `StatusCodes.BadIndexRangeDataMismatch` in the `Opc.Ua.Types` list - `InMemoryStoreProvider : IDisposable` ## Not changed / open - **Deferred:** - S1-14 (BSD OptionSet size needs the base type in `UaTypeDescription`, and BSD is deprecated). - C2-7 (restoring the own-certificate trust exemption would accept certificates the spec says to reject; removing the dead code is a separate decision). - A1-4 (config values keep surrounding whitespace, as the pre-April-2026 loader did). - H-2 (JsonDecoder has no real nesting counter). - A2-3 (the StringTable cap also applies to diagnostics tables; default `MaxArrayLength` already bounds them). - **Needs a product decision:** - The general `ConvertToString` writes a QualifiedName as `ns:Name`, while Table 121 drops the namespace. - Short (< 32 byte) client nonces are accepted on None channels. ## Verification - Every commit added a regression test that fails without its fix, where practical. - Before the rebases, with all audit and review fixes (before the `Spec: #…` commits): 0 build warnings and 16 test suites green on net10.0. The suites: Types, Core, Core.Encoders, Core.Security, Core.Schema, Client, Client.ComplexTypes, Server, Sessions, Gds, History, Features, Redundancy.Server, PubSub, WotCon and WotCon.Bindings. Core hit its known intermittent end-of-run hang once, and reruns were clean. - The `Spec: #…` fixes were verified per group on their own suites. Those include PubSub.Udp, Redundancy.PubSub, PubSub.Server, Configuration and SourceGeneration.Core. - **Not yet built or tested on this base.** It was rebased onto master (resolving overlaps with #4522, #4540 and #4564) and then onto #4569. This PR stays a draft until the full run on this base is green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
aad8350bc1 |
Pluggable symmetric cryptography, an awaitable asymmetric path, registrable security policies, and PubSub crypto offboarding (#4206, #4207, #4208, #4210) (#4232)
# Description Closes the three crypto-offboarding follow-ups split out of #4192. Each was recorded there as knowingly *not* done, with the reason and what would justify revisiting it. Those conditions are met, and where one of them turned out to be unachievable that is written down rather than worked around. The safety property throughout: **every seam is opt-in and null-checked, and in the default software configuration the added code is observably inert.** No interface dispatch on the per-message path, and the asynchronous paths complete synchronously — so channel locking order, buffer ownership and error sequencing are unchanged for a deployment that configures nothing. ## #4207 — symmetric, key-derivation and RNG provider seams `ISymmetricCryptoProvider`, `IKeyDerivationProvider` and `ISecureRandomSource`, added on the terms #4207 set: only once a concrete requirement arrived (a validated module that must perform *every* operation, not just the asymmetric ones), behind a fast path, with `SymmetricChannelCryptoBenchmarks` as the gate. They are **facets discovered by type test**, not members of `ICryptoProvider`, so providers written against the shipped interface still compile. `PlatformCryptoProvider` implements all three, which answers the "an interface nothing implements" objection recorded in the issue. `CryptoProviderFacets` returns `null` both when no registry is configured *and* when resolution lands on the platform, because the platform facets are the inline code. Resolution happens once, in `CalculateSymmetricKeySizes`, and is held for the life of the channel. `CryptoCompliance.GetUnservedOperationPurposes` closes the failure the seam would otherwise introduce: a provider bound to `ChannelSymmetric` without implementing the facet is silently replaced by the platform, so a deployment could believe its validated module performed the per-message cryptography while the platform did. Under `FipsOnly` that now refuses to start. ## #4208 — the asymmetric path is awaitable `RSA` and `ECDsa` are .NET's contracts and cannot be replaced without making every ready-made hardware and cloud implementation unusable. Instead an implementation may **also** declare `IAsyncRsaKey` / `IAsyncEcdsaKey`, which the stack finds by type test. Now asynchronous: the secure channel open and renew path, user identity token signing and decryption, and session activation. A software key declares neither facet, so those paths complete synchronously and nothing about their ordering changes. `ReadAsymmetricMessage` is refactored into shared helpers rather than duplicated — adding an `await` was not worth two hundred cloned lines of signature, padding and endpoint-selection logic. `AsymmetricMessage` / `AsymmetricWriteResult` exist because an asynchronous method cannot have `out` parameters. The channel no longer serialises its state on a monitor. A monitor cannot be held across an `await`, and a plain `SemaphoreSlim` was not an option at the time: the code relied on re-entrancy in eight places (`HandleIncomingMessage` holds the lock and calls `ForceChannelFault`, which takes it again; `ForceReconnect` and `Shutdown` do the same). `ChannelGate` replaces it at 32 sites and `DataLock` is `[Obsolete]`. That reliance has since been removed as well — see [Removing the gate's re-entrancy](#removing-the-gates-re-entrancy) below. ### Defects this surfaced, all fixed and regression-tested **Detached work started inline.** Writes were started with `_ = WriteBuffersAsync(...)`. An async method runs its prologue on the caller's stack, so `LeaveInheritedContext` — which exists to stop *detached* work inheriting the right to re-enter the gate — was stripping the **caller's own** entitlement; when the send then completed synchronously, the completion blocked on a gate that very thread was holding. The client sent its OpenSecureChannel request and never processed the response. Writes are now queued. **A synchronous gate handle held across an `await`.** `Enter()` records the acquiring thread so an inline completion callback can re-enter — `ChannelAsyncOperation` invokes its callback both inline and detached. That record is only sound while the holder is synchronously on that thread; the token renewal path held it across an await, so a reused pool thread was recognised as the holder. **`SignAsync` rejected the policy that signs nothing.** It validated the certificate before the algorithm, so `SecurityPolicies.None` — which carries no certificate — threw where the synchronous `Sign` returns `null`. **`EnterAsync` lost the holder when contended** *(found in review)*. The `AsyncLocal` recording ownership was written inside the awaiting method, and .NET restores the caller's execution context when an async method completes, so the write was discarded. The caller returned owning the semaphore while `IsHeldByCurrentContext` reported `false` — and any nested `Enter()` then blocked forever on a gate its own context held. Reachable remotely: `HandleIncomingMessageAsync` → error path → `ForceChannelFault`. The holder is now published from the caller's frame before the wait, and only its depth is raised in the continuation. **Queued writes lost their order** *(found in review)*. Sequence numbers are assigned under the gate, but independently queued work items reach the transport's FIFO send lock in arbitrary order, so a peer could reject a chunk with `BadSequenceNumberInvalid`. Writes are now appended to a chain: still off the caller's stack, but a write cannot start before its predecessor finishes. ### Four more this surfaced in CI, on paths only the integration suites reach Queuing the writes to fix the deadlock above turned out to have moved three implicit guarantees, and the gate had one more liveness bug. Each was reproduced locally against a clean baseline before it was changed. **A faulting channel's error message never reached the peer.** A listener channel sends a UASC `Error` describing why it faulted and closes the transport as its next statement. That worked only because the write started on the caller's stack, so the bytes reached the socket first; queued, the close discarded them and the client saw `BadConnectionClosed` instead of the certificate status it was told to expect. Nine `SecurityCertValidationTests` cases assert exactly that status. The terminal error is written inline again, on a path that touches the gate only to disclaim what it inherited and reports nothing — a channel that is already faulted is owed nothing beyond its buffer. **The certificate of a rejected sender leaked.** An asynchronous method cannot have `out` parameters, so the asymmetric read returns a record instead. That silently dropped an ownership guarantee: an `out` parameter reaches the caller even when the method goes on to throw, so the caller's `catch` disposed the sender certificate and reported it to the audit. A return value does not — nine leaked certificates, one per rejection case, which the leak detector fails the run on. The parsed certificate is now handed to the caller before anything that can reject the message, restoring both the disposal and the audit. **The gate could strand its own permit.** It released only when the holder's depth returned to zero, but that depth lives in an `AsyncLocal` and is therefore shared with every context forked from the owner's — a timer callback whose timer was created inside the region, for instance. Such a context entering the gate raised the shared depth, so the owner's own exit saw a non-zero depth and returned without releasing, and nothing afterwards could: the nested exit is not the owner. Every later entry on that channel blocked forever. `Opc.Ua.Features.Tests` hung after its tests had finished, with a receive loop waiting on the gate and the server unable to stop. The owner now releases when it leaves; depth still gates re-entrancy, but liveness no longer depends on a count another context can move. **Write completion throttled the writer.** Reporting a completed write sat inside the ordering chain, and the client channel implements that callback by entering the gate — so every write queued behind a gate the receive loop contends for on each incoming chunk. A session with fifty subscriptions publishing continuously had its publish requests throttled enough that the client kept topping them up past its own limit, which `Subscriptions.Classic` asserts against. Completion is now reported off the chain, which covers only what ordering actually requires. ### Still synchronous, and why Certificate, certificate request and revocation list signing cannot be made asynchronous at all: `X509SignatureGenerator.SignData` is invoked by .NET's own builders. Service faults and the `Reconnect` override are reached from synchronous call sites (`SendServiceFault` alone has eight) and are left as they are behind scoped, commented suppressions. All three are documented rather than left to be rediscovered. ## #4210 — PubSub This one produced a finding rather than the API the issue asked for, and the finding is the point: > **With a standard Security Key Service the key necessarily exists in process memory.** `GetSecurityKeys` (Part 14 §8.3.2) returns raw key bytes over the wire, so the "key never leaves the device" property #4192 achieved for client and server **cannot** be achieved for PubSub through the SKS pull profile. That is a property of the specification, not of this stack. A wrapped-key envelope would change what is on the wire and break interoperability with third-party key services and publishers, so it was rejected rather than built. What is achievable is delivered: the per-message AES-CTR and HMAC route through `ISymmetricCryptoProvider`, so a validated module performs them. `IPubSubSecurityPolicy` is **not** modified — new members would break every implementation of it; the provider is supplied through the policy constructor. `PlatformSymmetricCryptoProvider` gained AES counter mode, which the seam should have covered anyway since the algorithm enum names it. Key lifetime is also tightened: unpacking an SKS response copied key material into intermediates that `ByteString.Create` then copied again, leaving plain keys in the heap for the lifetime of the process. Those are now zeroed. ### Two wiring gaps this depended on, both found in review The registrations above were **inert**, so this section originally claimed more than the code did: - `CryptoProviderConfiguration` was applied nowhere outside tests, so `AddCryptoProvider(configure)` never reached the registry in a real host. The registry now applies every registered configuration when it is first resolved — which also affects #4207's DI story. - `PubSubSecurityWrapperResolver` selected `PubSubAes256CtrPolicy.Instance`, the provider-less static singleton, ignoring the provider-backed bundles registered for it. A `FipsOnly` deployment therefore certified while the platform did the work. The resolver now selects from the registered policies; the container and `PubSubApplicationBuilder` both supply theirs. A third, smaller one: `CryptoProviderBuilder.Use` published a **policy-scoped** random source as the process-wide nonce source, so `.For(purpose, policyUri).Use(hsm)` redirected nonce generation for every other policy too. Only an unscoped binding does that now. ## #4206 — a provider can contribute a security policy Added in review: the policy set was fixed at compile time, so a provider could contribute key custody but not a *profile*. It is now table-driven and open, and the blockers #4206 listed are gone. ```csharp using IDisposable registration = SecurityPolicies.Register( new SecurityPolicyInfo("urn:vendor:SecurityPolicy#CustomProfile", "CustomProfile") { PlatformSupport = () => true, SupportedCertificateTypes = [ObjectTypeIds.RsaSha256ApplicationCertificateType], IsDefault = true }); services.AddOpcUa() .AddCryptoProvider(crypto => crypto .For(CryptoPurpose.ChannelSymmetric, "urn:vendor:SecurityPolicy#CustomProfile").Use(module)) .AddSecurityPolicy(customPolicy); ``` - **The reflection is gone.** Four lookup tables were built by reflecting over the fields of `SecurityPolicies` and `SecurityPolicyInfo`. They are one explicit table now, which removes the **last reflection in `Security/Constants`** — so this is a trimming and Native AOT improvement as much as a feature. - **`SecurityPolicyInfo` is constructible from another assembly** and carries the metadata the helpers need, so `IsPlatformSupportedName`, `GetDefaultUris`, `MapSecurityPolicyToCertificateTypes` and `GetCurveFromCertificateTypeId` read that table instead of each being hand-written. Adding a policy is one entry rather than five edits. - **The dead `#if CURVE25519` conditional is deleted** from `Security/Constants`. - Registration is copy-on-write, so the read path stays lock-free. Shadowing a built-in policy requires `replaceExisting: true` and is reversible by disposing the registration. `RegisterLightsUpCurvePoliciesFromOutsideCore` is the acceptance test the issue asked for: it registers `ECC_curve25519` and `ECC_curve448` from the test project and asserts they become discoverable through the public API. **Registering a policy makes it advertised and resolvable; it does not supply the cryptography behind it.** For those two curve profiles the in-tree key agreement is still behind a compile symbol no project defines plus a BouncyCastle dependency, so a deployment lighting them up provides the operations through a provider. That division is the point: the policy set says *what* is offered, the provider says *who* performs it. ## Removing the gate's re-entrancy Folded in from #4242, which was merged into this branch. `ChannelGate` was re-entrant, and that forced the design behind most of the defects above. A holder that awaits releases its thread, so ownership could not be tracked by thread identity — it had to live in an `AsyncLocal<Holder?>`. A logical context is *inherited*, so anything started while the gate was held inherited the right to re-enter and ran inside the guarded region alongside whatever started it, unless it opted out by hand via `LeaveInheritedContext()` at each of eight sites, enforced by review alone. Five of the defects listed above trace to that mechanism, including both CI hangs. So the *requirement* was removed rather than the mechanism, and the `AsyncLocal` then deleted itself. Every path that re-entered now calls a lock-free `Core` method (`ForceChannelFaultCore`, `ShutdownCore`, `ForceReconnectCore`, `CompleteConnectCore`), and three regions stopped running foreign code under the lock: the `RequestReceived` dispatch is deferred until the gate is released, `OnTransportError` no longer wraps the `HandleSocketError` virtual, and the callback setters and `EndpointDescription` publish a single reference with a volatile write instead of locking — the last is load-bearing, because `AsyncResultBase.OperationCompleted` invokes the reverse-hello callback inline while the gate is held. `ChannelGate` is now a `SemaphoreSlim(1,1)` behind a `Releaser`; `Holder`, the thread-identity fallback, `LeaveInheritedContext`, `IsHeldByCurrentContext` and all eight disclaimer calls are gone, with the call shape at all 32 sites unchanged. `SaveIntermediateChunk`, `GetSavedChunks` and `DoMessageLimitsExceeded` gained a `gateHeld` argument, because the client reaches them both with and without the gate and must pick the matching teardown — without it an oversized `OpenSecureChannel` response deadlocked the receive loop against itself. The remaining paths were not found by reading the code, since that is what missed them originally. A temporary strict mode made every re-entrant acquisition throw with its stack, the suite was run under it until clean, and it was deleted with the re-entrancy it measured. It caught two sites review had missed: the `EndpointDescription` **property getter** taking the gate, and the `RequestReceived` dispatch to application code happening inside the guarded region. Two deliberate residuals are documented for reviewers: `Dispose` no longer takes the gate (unavoidable — it is reached from `ChannelFaulted` with the gate held, where the old re-entrant acquisition excluded nothing anyway), and the deferred request dispatch builds `SecureChannelContext` just after gate release rather than under it. ## Testing New: the three seams and their platform implementations, facet discovery and resolution precedence, `FipsOnly` refusing a provider that cannot serve what it was bound to, AES counter mode (including a non-block-aligned round trip and a check that the counter carries, since one that did not would silently reuse key stream), `ChannelGate` (exclusion, re-entrancy across an await, re-entrancy on a reused thread, and the deadlocks above), the asynchronous channel paths, and the PubSub policy provider. Each review finding got a test that fails without its fix: - `ContendedEntryStillRecordsTheHolderAsync` — established contention deterministically, then re-entered; failed on the pre-fix gate. Removed with the re-entrancy it covered; `ChannelGateTests` now asserts the non-re-entrant contract instead. - `SecureClientAndTcpListenerKeepChunkOrderUnderConcurrentRequestsAsync` — 60 concurrent multi-chunk `SignAndEncrypt` requests over a real loopback listener; failed 3 runs out of 3 with the previous dispatch, passes 3 out of 3 with the chain. - `DependencyInjectionRoutesPubSubCryptoThroughTheRegisteredProviderAsync` — builds a container, registers a counting provider, wraps a message and asserts the provider actually performed the encryption and the signature. This is the assertion the PubSub claim above now rests on. - `PolicyScopedRandomBindingDoesNotEscapeItsPolicy`. Two further assertions are the ones that would catch a silent defect: a provider and the platform must produce **byte-identical** output — a validated module that disagreed could not interoperate — and with a software key the returned `ValueTask` must report `IsCompletedSuccessfully`, which is what makes "ordering is unchanged" checkable rather than merely asserted. `SymmetricChannelCryptoBenchmarks` gains a provider-registered variant beside the existing baseline, so the cost of the indirection is measured rather than assumed. Verified locally, every suite at or above its recorded baseline: `dotnet build UA.slnx -c Release` **0 warnings, 0 errors on all six target frameworks**; Core **4413** (net10.0); Core.Security **617**, with its certificate-leak detector clean; PubSub **1350** plus PubSub.Diagnostics **54** and PubSub.Udp **233**; Features **156**; Subscriptions.Classic **32**; Server integration **4053**; Client integration **2123**; MigrationAnalyzer **151** and MigrationAnalyzer.Core **13**. Zero failures in each. Core moved from 4388 to 4381 when the gate change landed — eight re-entrancy tests were removed along with the behaviour they covered, and one deadlock regression test added — and has since grown to 4413 with the registry isolation, provider-output and security-review tests. `ChannelGateTests` was rewritten around the new contract — exclusion across both entry modes, an uncontended asynchronous entry not suspending, re-entry blocking by contract, and the property that used to be the hazard: work started while the gate is held is excluded without having to opt out. Every CI failure this PR produced was reproduced locally, diagnosed against a clean `origin/master` baseline in a second worktree, and fixed at the source. Two hangs were found from `--blame-hang-timeout` dumps read with `dotnet-dump`. One failure that looked like a candidate was confirmed **not** to be: an intermittent `Opc.Ua.Aot.Tests` connect failure, pre-existing on `origin/master` (5 of 8 baseline runs failed, versus 3 of 14 here). ### The publish request overshoot `Subscriptions.Classic.PublishRequestCountAsync` failed on the Windows net10.0 leg with **207** outstanding publish requests against a cap of `max(maxServerPublishRequest, subscriptions)` = 50. It failed in 2 of 6 pipeline builds on this branch, never on `master` in 13 sampled builds, and never locally in 12 attempts including under deliberate CPU contention. The mechanism is a genuine race. `ClassicSubscriptionEngine.QueueBeginPublish` was a **check-then-act**: it read `GoodPublishRequestCount`, compared it against `GetDesiredPublishRequestCount(false)`, and issued a publish if it was below. It runs on every publish completion, so *n* completions that overlap each read the same value, each conclude they are below the cap, and each issue — overshooting by up to *n* per round. The read also *lags*: a request only becomes visible to `GoodPublishRequestCount` once `AsyncRequestStarted` has recorded it, which happens after the request has already been sent. That code is pre-existing and was untouched by this PR. What this PR changed is how hard the race is hit: the server used to hold the channel gate across request processing, which serialised every request on a channel, so publish completions arrived one at a time. Dispatching `RequestReceived` outside the gate — an intended improvement, since a channel should not serialise on application request-processing time — lets them complete concurrently. Rather than change the publish pipeline blind, the race was first made reproducible. `ConcurrentPublishReEvaluationDoesNotExceedDesiredRequestCount` drives 64 barrier-released completions against a mocked context whose count lags the send exactly the way the session's does. It issued **64 requests where 5 were desired** — the same defect the CI leg hit, now deterministic and running in milliseconds. The engine now tracks the requests it has in flight and reserves a slot with a compare-exchange before sending, so the decision and the send are one atomic step. The reservation is released when the request completes, when the send is declined, or when it throws. The repro test then issues at most the desired count. `StartPublishing` is deliberately left **uncapped**. It is the recovery valve that refills a pipeline whose requests are outstanding but no longer expected to return, so gating it on the in-flight count would remove the only path out of a stalled pipeline. An earlier revision of this fix did gate it, and that is the one change here that could not be justified on the evidence — it was reverted. The assertion was **not** loosened, retried away, or marked flaky: it caught a real throttling regression earlier in this PR and is worth keeping sharp. Validated: the repro test red before the fix and green after; `Subscriptions.Classic` 32/32 **five** consecutive runs on net10.0 and once on net48; `Opc.Ua.Client.Tests` 2124 on net10.0 across four consecutive runs and 2127 on net48; `Opc.Ua.Subscriptions.Tests` 604 passed with this fix in place. ### The SSH.NET advisory that turned CI red repo-wide Partway through this work every build job began failing with `NU1903: Package 'SSH.NET' 2025.1.0 has a known high severity vulnerability` ([GHSA-q939-rpr3-3284](https://github.com/advisories/GHSA-q939-rpr3-3284), `<= 2025.1.0` affected, first patched `2026.0.0`). Nothing in the dependency graph changed — the advisory was published. Build 17026 failed on it where build 17000, two hours earlier on an identical graph, had passed every build job. SSH.NET is not referenced anywhere in the repository; it arrives transitively through `Testcontainers.Kafka`. `master` landed the remediation independently while this was being investigated (a central transitive pin to `2026.0.0`, which `CentralPackageTransitivePinningEnabled` exists for). This branch briefly carried its own identical pin, which made the merge commit hold **two** `PackageVersion` entries for SSH.NET and fail restore with `NU1506`. The duplicate is removed and `master` is merged, so the pin now comes from one place only. The audit was **not** suppressed: `NuGetAudit` stays enabled and `NU1903` stays an error. Verified after the merge — `dotnet restore UA.slnx` reports neither NU1903 nor NU1506, and `dotnet build UA.slnx -c Release` is **0 warnings, 0 errors** with the audit live. The remaining red check in build 17026 was not from this change either. `aot-ubuntu-latest` is the intermittent `Opc.Ua.Aot.Tests` failure already noted above — `ReconnectSessionAsync` timed out at 30s with 135 of 136 passing. It remains below the `origin/master` baseline rate (4 of 15 sampled here against 5 of 8 on the baseline), and the managed reconnect paths are green throughout: `Opc.Ua.Client.Tests` 2124 across four consecutive runs and `Opc.Ua.Subscriptions.Tests` 604. ## Documentation `docs/CryptoProvider.md` gains sections on substituting the symmetric primitives, using a key served over a network, and PubSub — including what cannot be claimed and why, the scope rule for a random-source binding, and how to supply a provider-backed policy outside the container. `docs/MigrationGuide.md` covers the `[Obsolete]` channel virtuals and `DataLock`, and the non-re-entrant gate's contract for channel subclasses. `docs/WhatsNewIn2.0.md` is updated. `plans/cryptooffboard.md` is deleted now that the work it planned has landed. ## Note for reviewers This touches the secure channel handshake, which is the most security-critical code in the stack. Nine distinct concurrency and lifetime defects were found and fixed in it during this work — three during development, two in review, four from CI — so `ChannelGate`, the write path and the certificate ownership across the new suspension points deserve the closest reading. Five of those nine traced to the gate's `AsyncLocal`, which is why the re-entrancy was subsequently removed and the `AsyncLocal` deleted. The wrapper/`Core` split and the two documented residuals in that section are the parts most worth a second reading. The #4206 work removes the last reflection from `Security/Constants` and rewrites the policy lookup tables, so `SecurityConfiguration.SupportedSecurityPolicies` and the endpoint set it feeds are worth a second look: an earlier revision of it changed that set and broke six server tests, which is now covered by `SecurityConfigurationBuildsExpectedSupportedPolicySet`. ## Related Issues - Fixes #4206 - Fixes #4207 - Fixes #4208 - Fixes #4210 - Follow-up #4250 — this PR makes the security policy set injectable, but nothing in the stack resolves `ISecurityPolicyRegistry` from DI yet, so every production path still lands on the `Default` fallback. #4250 tracks threading it through `Session` and `UaSCBinaryChannel`, which is what makes the injection points already added here load-bearing. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com> Copilot-Session: 7a68ca72-83e8-496c-b2f5-3df4c718320b Copilot-Session: 3f5ab76a-c0ba-4f47-87ac-54d1f4182c6b |
||
|
Marc Schier
|
0612a43fe4 |
Pluggable cryptography and hardware-held private keys (#4192)
Makes the stack's cryptographic operations replaceable - by another library, an offboard service, or hardware (TPM 2.0 / HSM / PKCS#11 token / cloud KMS) - with no performance cost in the default configuration, and extends this to certificates so a private key need never exist in process memory. Also fixes three pre-existing defects found while validating: STACKGEN001 erroring for every compilation the stack generator was merely loaded into, certificates failing to load from paths past MAX_PATH on Windows, and the MonitoredNode2 tests starving the thread pool. Fixes #4190 |