mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-28 14:09:56 +08:00
master
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f6b27025a6 |
Classify sensitive log parameters with Microsoft.Extensions.Compliance data classification (#4864)
## Proposed changes Fixes #4857. The stack's log messages wrote network addresses (endpoint, discovery and reverse connect URLs, host names, IP addresses, ports), application and user identities, file paths and exception texts as plain parameters. Applications had no reliable way to keep them out of general-purpose logs. This PR replaces the stack's own redaction mechanism with the standard .NET [data classification and redaction](https://learn.microsoft.com/dotnet/core/extensions/data-redaction) packages. Each sensitive `[LoggerMessage]` parameter is classified. The application decides per classification what reaches its logs: erase it, replace it with an HMAC token so entries stay correlatable, or pass it through. ### Taxonomy `Opc.Ua.Redaction.OpcUaDataClassifications` defines the `OpcUa` taxonomy and one attribute per classification: | Classification | Used for | Parameters | |---|---|---| | `[NetworkAddress]` | URLs, host and domain names, IP/MAC addresses, end points, ports | 120 | | `[ApplicationIdentity]` | certificates, subject, issuer, thumbprint, ApplicationUri | 103 | | `[FilePath]` | file, directory and certificate store paths | 67 | | `[SensitiveText]` | exception or remote-supplied text passed as a parameter | 55 | | `[UserIdentity]` | user names, token subjects | 7 | Correlation ids (`ChannelId`, `SessionId`, ...), status codes and NodeIds stay unclassified. ### Pipeline and defaults - Opc.Ua.Core now references `Microsoft.Extensions.Telemetry(.Abstractions)` 10.11.0, so every project above Core uses the `Microsoft.Gen.Logging` generator. Opc.Ua.Client already did, through Http.Resilience. The packages ship net462 and netstandard2.0 builds, so net48 is supported. - **A plain `ILoggerFactory` logs `(null)` in place of classified values.** Classified values are only written by the redaction-aware logger. - The new `ILoggingBuilder.EnableOpcUaRedaction()` turns that logger on. It is `EnableRedaction()` with the discriminator off; otherwise every unredacted value gets `:ParamName` appended. `AddOpcUa().AddLogging(...)` and `DefaultTelemetry.Create(...)` call it, as do the samples, so they log the same information as before until redactors are registered. - `services.AddOpcUa()` alone does not change the host's logging. Generic-host apps call `builder.Logging.EnableOpcUaRedaction()`. This is documented as a behavior-breaking change in `docs/migrate/2.0.x/telemetry.md`. ### Obsolete API `Redact`, `RedactionWrapper<T>`, `IRedactionStrategy`, `RedactionStrategies` and `SimpleRedactionStrategy` are `[Obsolete]`. They still work, but the stack no longer uses them. ### Not classified - The `Exception` passed to a log call. Applications can drop exception messages with `LoggerEnrichmentOptions.IncludeExceptionMessage`. - Exception and `ServiceResult` message texts. - `ServiceResult`/`StatusCode` parameters and logged data values. - The few log calls whose level is decided at run time. - `Opc.Ua.Vision.OpenUsd`, which references only Opc.Ua.Types and cannot see the attributes. ### Generator behavior differences These are visible to log consumers and to tests: - `IFormattable` values such as `NodeId` are stored in the structured state as invariant strings, not as objects. - The generated log state is a thread-local object that is cleared after the call. - Classified values are pooled redactor objects, valid only during the `Log` call. Tests that inspected log state after the fact now capture it during the call. The shared `RecordingLoggerProvider` snapshots classified values. ## Related Issues - Fixes #4857 ## Types of changes - [ ] Bugfix (non-breaking change which fixes an issue) - [x] Enhancement (non-breaking change which adds functionality) - [ ] Test enhancement (non-breaking change to increase test coverage) - [x] Breaking change (fix or feature that would cause existing functionality to not work as expected; plain logger factories drop classified values) - [x] Documentation Update (if none of the other choices apply) ## Checklist - [x] I have read the CONTRIBUTING doc. - [x] I have signed the CLA. - [x] I ran tests locally with my changes, all passed. - [x] I fixed all failing tests in the CI pipelines. - [x] I fixed all introduced issues with CodeQL and LGTM. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added necessary documentation (if appropriate). - [ ] Any dependent changes have been merged and published in downstream modules. ## Further comments New tests: - `LogClassificationCoverageTests` (Tools.Tests) scans all 104 stack assemblies and fails if an address, identity or path parameter of a `[LoggerMessage]` method is unclassified. - `DataClassificationLoggingTests` (Core.Tests) pins erasing and HMAC redaction, pass-through, the plain-factory drop, and both stack defaults. Local results on net10.0: | Suite | Passed | Failed | |---|---|---| | Core.Tests | 6,489 | 0 | | Client.Tests | 3,231 | 0 | | Tools.Tests | 7,210 | 0 | | PubSub.Tests | 1,536 | 0 | | Core.Diagnostics.Tests | 571 | 0 | | Configuration.Tests | 241 | 0 | - **Server.Tests:** the full run predates the final test fixes and had 12 failures. All 12 were in the fixtures fixed afterwards, which now pass: RoleStateBinding (45/45) and the FileDirectory and compatibility-logging fixtures (74/74 in total). The full Core.Tests and Server.Tests runs were not repeated after those fixes. - **Builds:** `UA.slnx` builds with 0 errors for net10.0 and net48, and the whitespace check passes. - **NativeAOT:** not confirmed locally, because the machine has no C++ linker. A trimmed publish of `Opc.Ua.Aot.Tests` reported no trim or AOT warnings from the stack, the new packages or the generated logging code; its only errors come from the TUnit test framework (`FSharp.Core` not found). The `aot-test` CI job has to confirm the native build. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> |
||
|
|
5899354441 |
Clarify documentation, migration guidance, and SDK selection (#4823)
# Description Remove stock phrasing and correct misleading or inconsistent technical documentation. The initial two passes covered 367 Markdown files, followed by an interactive review of 169 editorial flags. The U5 interpretation originally deferred in that review has since received the upstream Mantis answer recorded by #4843; remaining server/spec follow-ups stay explicit in the CTT log. - Correct package responsibilities, migration steps, API names, ownership, protocol descriptions and links. Keep bundled migration guides synchronized. Preserve historical conclusions as historical records with separate re-check notes. - Provide at least two rendered TOC levels throughout `docs/`. Convert appropriate architecture figures to Mermaid without changing code examples, transcripts or historical quotations. - Address the October 19 feedback with concrete setup/migration instructions, corrected certificate paths and PubSub names, shorter repeated caveats, consistent XML comments, and a full review of the Concepts guide. Retain #4839's API corrections. - Integrate #4841's runtime/performance guidance and review incoming/recent prose. Preserve dated measurements rather than claiming new benchmark runs; clarify allocation ratios, startup-only processor settings, JIT warm-up and the benchmark commit's scope. Missing raw measurement artifacts and repeat ranges remain tracked by #4842. - Merge master through `e8c5d800a` in `32c6394b7`. Resolve the CTT log conflict using the updated Mantis states and OPC answers, retaining historical framing and re-checks for C28/C32/C43. - Address the latest naming/index feedback in `adb5bf693`: rename the guides to `docs/WotCon.md` and `docs/WotNodeSetConversion.md`; move the provider and DataType-declaration contracts into the conversion guide and remove `docs/WoT.md`. Rename `samples/DI` to `samples/Di` and `samples/Isa95` to `samples/ISA95`. Add content-index READMEs to Di, FluentApi, ISA95, Lds and PubSub; update solution entries, project references, Docker/CI paths, test fixture paths and documentation links with exact casing. **WoT guide scope.** This branch retains the detailed conversion guide that #4758 had replaced with an anchor stub. Conversion limits, verified document sets, preservation rules and provider ownership now live together in `WotNodeSetConversion.md`; binding/runtime guidance remains in `WotBindings.md` and `WotCon.md`. The design-rationale/test-ledger prose was shortened, the oversized argument-table cell moved into its section, and existing anchors preserved. **SDK policy is a separate, explicit user-directed change.** `global.json` sets a stable minimum of `10.0.400`, `rollForward: latestMajor`, and `allowPrerelease: false`. Newer major SDKs are permitted; CI's explicit SDK pins are unchanged. README, GettingStarted, DeveloperGuide and Recycling instructions reflect the possible analyzer differences. The image-contract regression fix accepts compatible stable SDK images instead of requiring exact equality with the floor. The latest four rename/index threads are resolved after publication. Three older threads remain open: universal obsolete-removal timing, unconditional nonce-strength wording, and a blanket Vision NativeAOT claim. Existing implementation/evidence constraints are explained in their inline replies. The two-level TOCs remain as requested. ## Validation - `git diff --check` and staged diff checks: passed. The untouched upstream AutoID NodeSet retains its provenance bytes, including two existing license-header trailing spaces. - `.\.azurepipelines\validate-migration-plugin.ps1`: passed; 15 bundled migration documents and manifests synchronized. - Documentation/path validation: **101** documents retain two rendered TOC levels (**2,228** entries), with idempotent formatting. **52** new relative links and their anchors passed exact-case checks. No tracked references to the old guide/sample paths remain. Both relocated provider-contract sections are preserved in the destination. - `dotnet sln UA.slnx list`: passed; shows the renamed Di/ISA95 projects and the existing FluentApi/Lds/PubSub projects. Updated solution and pump ProjectReference paths also passed exact-case existence checks. - Latest focused commands, run from the repository root: ```powershell # Passed: 61 channel send-order and reconnect tests. dotnet test tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -p:CustomTestTarget=net10.0 --filter 'FullyQualifiedName~UaSCBinaryChannelSendOrderTests|FullyQualifiedName~TcpReconnectKeyAgreementTests' --verbosity minimal # Passed: 5 pump datasheet tests; builds the renamed pump sample. dotnet test tests\Opc.Ua.Di.Tests\Opc.Ua.Di.Tests.csproj -p:CustomTestTarget=net10.0 --filter 'FullyQualifiedName~PumpDatasheetConformanceTests' --verbosity minimal # Passed: 4 repository NodeSet path/round-trip tests, including the moved pump models. dotnet test tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj -p:CustomTestTarget=net10.0 --filter 'FullyQualifiedName~RepositorySourceSurvivesSerializedLinkedPartitionsAsync' --verbosity minimal # Passed: build, zero warnings/errors. dotnet build samples\ISA95\MinimalIsa95Server\MinimalIsa95Server.csproj -p:CustomTestTarget=net10.0 -f net10.0 --verbosity minimal # Passed: dependent sample-test project build, zero warnings/errors. dotnet build tests\Opc.Ua.WotCon.Samples.Tests\Opc.Ua.WotCon.Samples.Tests.csproj -p:CustomTestTarget=net10.0 -f net10.0 --verbosity minimal ``` Earlier focused integration checks passed 2,620 tests across AutoID, event queues/scheduling, source generation, MCP/package profiles and WoT. Both reviewed Mermaid diagrams rendered locally without network access. These checks are not a full solution run. The full solution has **not** been rerun for this review. Earlier `dotnet test UA.slnx -p:CustomTestTarget=net10.0 --verbosity quiet` did not pass: three AOT projects rejected VSTest and 20 Vision tests failed during graphics initialization with `OutOfMemoryException`. Remaining long-running tests were stopped at the user's request. A focused Vision rerun passed 558 tests and reproduced those 20 failures. The earlier `dotnet test UA.slnx -p:CustomTestTarget=net48 --verbosity quiet` was blocked by the former SDK pin and was not retried after the policy change. The user approved proceeding with those limitations documented; current remote CI is not claimed green. ## Related Issues - Follow-up #4842 tracks the requested 2.0.2+ benchmark/tuning and diagnostic-tool documentation refresh. #4841 supplies documentation and earlier results, not all newly requested measurements. - #4758 explains the conversion guide's earlier consolidation; the current provider contracts are now consolidated into `WotNodeSetConversion.md`. - #4839 supplies the reviewed API corrections; #4843 supplies the updated CTT issue record. - The original prose cleanup and SDK policy were user-requested without a source issue. No new runtime architecture is introduced by these documentation edits. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Roman Ettlinger <romanett98@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7e25e1a3d7 |
Extract remaining shared companion, rendering and packaging changes (#4806)
# Description The shared AI deployment contracts and instance registration; DI software-upload outcomes, digest handling and simulator; bounded Positioning/Pumps reads; PubSub raw-data/matrix and security-provider handling; Vision read-quality contracts; xRegistry resource writes and instance projection; OpenUSD authoring, embedded viewport and retained-frame capture APIs; standalone sample CLI controls; and the generic Debug/RID package-policy and fork/Dependabot coverage-comment fixes. The Linux native prerequisite step is retained for **Vision tests only**. ## Related Issues - Related tracking issue: [#4461](https://github.com/OPCFoundation/UA-.NETStandard/issues/4461), without automatic closure. - Source work: [#4462](https://github.com/OPCFoundation/UA-.NETStandard/pull/4462), which remains intact for the coordinator's later integration. - Immediate stacked predecessor: [#4804](https://github.com/OPCFoundation/UA-.NETStandard/pull/4804). ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. |
||
|
|
9b034f8291 |
Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
dadd609ec0 |
Scales: the OPC 40200 companion SDK with the OPC 30050 PackML model (#4580)
> Tracking issue #4576. #4577, #4578 and #4579 are merged, so this branch is > rebased onto `master` and contains only the Scales and PackML commits. ## Summary Last of the four PRs of #4576. Adds OPC 40200 Weighing Technology 2.00 as a model package plus a server and a client SDK, and the OPC 30050 PackML model it composes. | Package | Content | |---|---| | `Opc.Ua.PackML` | OPC 30050 PackML 1.01, plus `PackMLStateMachineController`, which drives a base state machine together with its nested machine and execute state machines (activation and deactivation of the sub-state machines, the PackML methods with an `Executable` attribute that follows the state, acting states left automatically or by the equipment) | | `Opc.Ua.Scales` | OPC 40200 2.00 (`http://opcfoundation.org/UA/Scales/V2/`), generated from the unmodified UA-Nodeset publication, plus the contracts client and server share | | `Opc.Ua.Scales.Server` | `AddScales()` / `ConfigureScales(...)`; all fourteen scale kinds and scale systems below `DeviceSet` and `Machines`; a runtime that does the weighing arithmetic (ranges, rounding to d or e, zero band, overload/underload, zero, tare, preset tare, registration), production presets with DI locking, statistics, recipes and recipe files, Annex C events and alarms over a `HasNotifier` hierarchy, and the methods of every scale type | | `Opc.Ua.Scales.Client` | discovery, weight reads and streaming, zero/tare/register, products, PackML state and commands, type-specific methods, events and alarms, snapshots | `docs/Scales.md` includes a conformance matrix against the 21 OPC 40200 server facets with test evidence, the places where the specification and the NodeSet disagree, and known limitations. ## Review follow-ups Separate commits after the docs/samples commit, each with a test that fails without it where the defect can be observed. - Copilot review: `CurrentProductsChanged` fires only when the set of current products changes; the new public collection APIs (`ScalesServerOptions.AdditionalNamespaceUris`, `ScaleNamespaceIndices.SearchOrder`, `ScalesModel.AllKinds`) are `ArrayOf<T>`. - Own review with the rubric in `.github/copilot-instructions.md`: - PackML raises `StateChanged` after its lock is released (queued, in order), reads `StartParameters` under the lock, and reports the handler's status when an acting state cannot complete. - Scales: concurrent first creations share one runtime (the lock service refused a second attach); two creations of the same name cannot both publish; a removed product id stays reserved until its node is gone; the zone, target-count and vehicle handlers change products under the preset lock; the product factory and `configure` delegate run outside it; the recipe upload reads `ActiveRecipe` under the lock; resetting the statistics also resets the tolerance percentage. - The client lets a failed browse call propagate instead of reading it as "no children". ## Testing - `tests/Opc.Ua.Scales.Tests`: 115 tests on net8 and net10 and with the netstandard2.1 libraries, including the PackML runtime; the client/server tests connect a real session over `opc.tcp` to a hosted `ScalesNodeManager`. - `UA.slnx` builds for net472, net48, netstandard2.0/2.1 and net8/9/10 with SDK 10.0.401 and no new warnings. - Changed-line coverage of this layer (`check-coverage.ps1`): 94.59 % (4585 of 4847 changed lines, against `master`). - Fork CI, full PR matrix incl. macOS: in progress, [run 36676067489](https://github.com/biancode/UA-.NETStandard/actions/runs/36676067489); CodeQL: in progress, [run 36676070427](https://github.com/biancode/UA-.NETStandard/actions/runs/36676070427). - On the current `master` (with Pumps merged), `UA.slnx` builds for every target framework and the Scales and Pumps tests pass. - Beyond the in-repo tests the implementation has been validated end to end against a separate, non-public OPC UA client and server. ## Reviewing Commits: PackML → Scales model → server → client → tests → docs and samples → review follow-ups. |