4 Commits

Author	SHA1	Message	Date
romanett	f6b27025a6	Classify sensitive log parameters with Microsoft.Extensions.Compliance data classification (#4864) ... ## Proposed changes Fixes #4857. The stack's log messages wrote network addresses (endpoint, discovery and reverse connect URLs, host names, IP addresses, ports), application and user identities, file paths and exception texts as plain parameters. Applications had no reliable way to keep them out of general-purpose logs. This PR replaces the stack's own redaction mechanism with the standard .NET [data classification and redaction](https://learn.microsoft.com/dotnet/core/extensions/data-redaction) packages. Each sensitive `[LoggerMessage]` parameter is classified. The application decides per classification what reaches its logs: erase it, replace it with an HMAC token so entries stay correlatable, or pass it through. ### Taxonomy `Opc.Ua.Redaction.OpcUaDataClassifications` defines the `OpcUa` taxonomy and one attribute per classification: | Classification | Used for | Parameters | |---|---|---| | `[NetworkAddress]` | URLs, host and domain names, IP/MAC addresses, end points, ports | 120 | | `[ApplicationIdentity]` | certificates, subject, issuer, thumbprint, ApplicationUri | 103 | | `[FilePath]` | file, directory and certificate store paths | 67 | | `[SensitiveText]` | exception or remote-supplied text passed as a parameter | 55 | | `[UserIdentity]` | user names, token subjects | 7 | Correlation ids (`ChannelId`, `SessionId`, ...), status codes and NodeIds stay unclassified. ### Pipeline and defaults - Opc.Ua.Core now references `Microsoft.Extensions.Telemetry(.Abstractions)` 10.11.0, so every project above Core uses the `Microsoft.Gen.Logging` generator. Opc.Ua.Client already did, through Http.Resilience. The packages ship net462 and netstandard2.0 builds, so net48 is supported. - **A plain `ILoggerFactory` logs `(null)` in place of classified values.** Classified values are only written by the redaction-aware logger. - The new `ILoggingBuilder.EnableOpcUaRedaction()` turns that logger on. It is `EnableRedaction()` with the discriminator off; otherwise every unredacted value gets `:ParamName` appended. `AddOpcUa().AddLogging(...)` and `DefaultTelemetry.Create(...)` call it, as do the samples, so they log the same information as before until redactors are registered. - `services.AddOpcUa()` alone does not change the host's logging. Generic-host apps call `builder.Logging.EnableOpcUaRedaction()`. This is documented as a behavior-breaking change in `docs/migrate/2.0.x/telemetry.md`. ### Obsolete API `Redact`, `RedactionWrapper<T>`, `IRedactionStrategy`, `RedactionStrategies` and `SimpleRedactionStrategy` are `[Obsolete]`. They still work, but the stack no longer uses them. ### Not classified - The `Exception` passed to a log call. Applications can drop exception messages with `LoggerEnrichmentOptions.IncludeExceptionMessage`. - Exception and `ServiceResult` message texts. - `ServiceResult`/`StatusCode` parameters and logged data values. - The few log calls whose level is decided at run time. - `Opc.Ua.Vision.OpenUsd`, which references only Opc.Ua.Types and cannot see the attributes. ### Generator behavior differences These are visible to log consumers and to tests: - `IFormattable` values such as `NodeId` are stored in the structured state as invariant strings, not as objects. - The generated log state is a thread-local object that is cleared after the call. - Classified values are pooled redactor objects, valid only during the `Log` call. Tests that inspected log state after the fact now capture it during the call. The shared `RecordingLoggerProvider` snapshots classified values. ## Related Issues - Fixes #4857 ## Types of changes - [ ] Bugfix (non-breaking change which fixes an issue) - [x] Enhancement (non-breaking change which adds functionality) - [ ] Test enhancement (non-breaking change to increase test coverage) - [x] Breaking change (fix or feature that would cause existing functionality to not work as expected; plain logger factories drop classified values) - [x] Documentation Update (if none of the other choices apply) ## Checklist - [x] I have read the CONTRIBUTING doc. - [x] I have signed the CLA. - [x] I ran tests locally with my changes, all passed. - [x] I fixed all failing tests in the CI pipelines. - [x] I fixed all introduced issues with CodeQL and LGTM. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added necessary documentation (if appropriate). - [ ] Any dependent changes have been merged and published in downstream modules. ## Further comments New tests: - `LogClassificationCoverageTests` (Tools.Tests) scans all 104 stack assemblies and fails if an address, identity or path parameter of a `[LoggerMessage]` method is unclassified. - `DataClassificationLoggingTests` (Core.Tests) pins erasing and HMAC redaction, pass-through, the plain-factory drop, and both stack defaults. Local results on net10.0: | Suite | Passed | Failed | |---|---|---| | Core.Tests | 6,489 | 0 | | Client.Tests | 3,231 | 0 | | Tools.Tests | 7,210 | 0 | | PubSub.Tests | 1,536 | 0 | | Core.Diagnostics.Tests | 571 | 0 | | Configuration.Tests | 241 | 0 | - **Server.Tests:** the full run predates the final test fixes and had 12 failures. All 12 were in the fixtures fixed afterwards, which now pass: RoleStateBinding (45/45) and the FileDirectory and compatibility-logging fixtures (74/74 in total). The full Core.Tests and Server.Tests runs were not repeated after those fixes. - **Builds:** `UA.slnx` builds with 0 errors for net10.0 and net48, and the whitespace check passes. - **NativeAOT:** not confirmed locally, because the machine has no C++ linker. A trimmed publish of `Opc.Ua.Aot.Tests` reported no trim or AOT warnings from the stack, the new packages or the generated logging code; its only errors come from the TUnit test framework (`FSharp.Core` not found). The `aot-test` CI job has to confirm the native build. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com>	2026-10-21 09:47:07 +00:00
Marc Schier	87012a147f	Machinery: add bounded inspection and result transfers (#4804) ... # Description - Add opt-in bounded browsing and strict reads with original source, status, timestamp, and structured-value evidence. Preserve the existing tolerant constructors and convenience projections, and wire options through session extensions, factories, and hosting registration. - Add selected-result streaming downloads with byte/request limits and independent cleanup, using `TemporaryFileTransferClient.CopyToAsync` from the client/server predecessor. Retain the detailed transfer regressions in the Machinery test layer. - Expose genuinely null unpublished result slots, carry the two Machinery sample updates, and include the complete Machinery guide and package README changes. ## Related Issues - Related tracking issue: #4461; this extraction does not close it. - Frozen source PR: #4462; this extraction does not close or modify it. - Immediate predecessor: #4803 (`marcschier-extract-amb-changes`). The earlier dependencies are client/server #4802 and core/types #4801. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received.	2026-10-16 12:24:35 +02:00
romanett	9b034f8291	Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700) ... # Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>	2026-10-08 08:48:13 +02:00
Klaus Landsdorf	01d6b5d26c	Machinery: the OPC 40001 companion SDK with Jobs, Result Transfer, Process Values and Energy (#4578) ... > Tracking issue #4576. #4577 is merged, so this branch is rebased onto > `master` and contains only the Machinery commits. ## Summary Second of four stacked PRs. Adds the OPC UA for Machinery series as model packages plus a server and a client SDK, following the companion pattern of DI and Robotics, and the two models it depends on that the repository does not carry yet. | Package | Content | |---|---| | `Opc.Ua.IA` | OPC 10000-200 IA 1.01.4, **moved out of `Opc.Ua.Robotics`** (see below) | | `Opc.Ua.ECM` | OPC 34100 Energy Consumption Management 1.0.1 | | `Opc.Ua.PADIM` | OPC 30081 PA-DIM 1.02.0 with the IRDI dictionary | | `Opc.Ua.Machinery` | OPC 40001-1 Basic Building Blocks 1.04.1 | | `Opc.Ua.Machinery.ProcessValues` | OPC 40001-2 Process Values 1.00.0 | | `Opc.Ua.Machinery.Jobs` | OPC 40001-3 Job Management 1.0.1 | | `Opc.Ua.Machinery.Energy` | OPC 40001-4 Energy 1.00 | | `Opc.Ua.Machinery.Result` | OPC 40001-101 Result Transfer 1.01.0 | | `Opc.Ua.Machinery.Server` | `AddMachinery()` / `ConfigureMachinery(...)`, fluent machine builder, server-driven state machines, process values, job management over the shared ISA-95 V2 binder, energy, result management incl. `GenerateFileForRead` | | `Opc.Ua.Machinery.Client` | machine discovery, identification, state observation, process values, job and result access | OPC 40001-101 `ResultTransferType` redeclares `GenerateFileForRead` with its own arguments; the generator fix for such overrides, merged with #4577, is what makes it appear on the `ResultTransfer` instances. All NodeSets are byte-identical to `UA-Nodeset/latest`. Each model carries a pinned `Model/*.NodeIds.csv` the generator validates, next to the unmodified upstream table; `tools/nodesets/derive-identifier-table.py` derives the former from the latter. ## Changes to existing code - **IA moves out of Robotics.** Machinery needs IA as well (OPC 40001-1 types `Stacklight` with `BasicStacklightType`), and two assemblies generating the same model under the same prefix collide at any consumer referencing both. The NodeSet moves verbatim, so no generated type changes; the prefix stays `Opc.Ua.IA`. Source-compatible, but IA types now live in `Opc.Ua.IA.dll` rather than `Opc.Ua.Robotics.dll` — binary-breaking for consumers of the Robotics package. - **`samples/OpenUsd/GeneratorServer`** drops its hand-reduced Machinery NodeSet copy and the script that produced it (a workaround for the generator defect fixed by #4577) and references `Opc.Ua.Machinery` instead. - `README.md` and `docs/README.md` list the new models; the README also corrects the Industrial Automation entry (Part 200, not OPC 40001-1). ## Review follow-ups Separate commits after the docs/samples commit, so the reviewed state stays comparable. Each fix comes with a test that fails without it. - Copilot review: a transition is committed only while the state machine is still in its source state; optional Jobs/Result namespaces the server does not load are reported as absent, as documented; resources of registered machines (result transfer, job-management change pumps) are released with the node manager (`DisposeMachineryResourcesAsync` for a custom DI manager); `NodeId`/`LocalizedText` use their null sentinels instead of `Nullable<T>`. - Own review with the rubric in `.github/copilot-instructions.md`: a failed browse call propagates from the client instead of reading as "no children"; the predefined result metadata compares `CreationTime` against the `DateTimeUtc` sentinels (the `DateTime` conversion shifted `DateTime.MaxValue` by the host's UTC offset); process values below `Monitoring` are found; a second metering point named `Main` is rejected; the zero-point event is reported when the handler throws; smaller state-sharing and documentation corrections. ## Testing - `tests/Opc.Ua.Machinery.Tests`: 162 tests on net8 and net10 and with the netstandard2.1 libraries; the integration tests connect a real client session over `opc.tcp` to a hosted server. - With IA extracted: Robotics 636, OpenUsd 921, DI 433 and SourceGeneration 263 tests pass. - `UA.slnx` builds for net472, net48, netstandard2.0/2.1 and net8/9/10 with SDK 10.0.401 and no warnings. - Changed-line coverage of this layer (`check-coverage.ps1`): 91.68 % (6059 of 6609 changed lines, against `master`). - Fork CI, full PR matrix incl. macOS, on the current master: green, all 184 jobs ([run 36582685653](https://github.com/biancode/UA-.NETStandard/actions/runs/36582685653)). CodeQL: green ([run 36582689589](https://github.com/biancode/UA-.NETStandard/actions/runs/36582689589)). - Beyond the in-repo tests the implementation has been validated end to end against a separate, non-public OPC UA client and server. ## Reviewing Commits are ordered so each builds: IA extraction → identifier-table tool → ECM/PA-DIM → Machinery models → server → client → tests → docs/samples → GeneratorServer sample → review follow-ups. The model commits are large only because of the NodeSets and identifier tables; the hand-written code starts at the server commit. --------- Co-authored-by: Marc Schier <marcschier@users.noreply.github.com>	2026-09-29 19:59:34 +02:00