5 Commits
Author SHA1 Message Date
f6b27025a6 Classify sensitive log parameters with Microsoft.Extensions.Compliance data classification (#4864)
## Proposed changes

Fixes #4857. The stack's log messages wrote network addresses (endpoint,
discovery and reverse connect URLs, host names, IP addresses, ports),
application and user identities, file paths and exception texts as plain
parameters. Applications had no reliable way to keep them out of
general-purpose logs.

This PR replaces the stack's own redaction mechanism with the standard
.NET [data classification and
redaction](https://learn.microsoft.com/dotnet/core/extensions/data-redaction)
packages. Each sensitive `[LoggerMessage]` parameter is classified. The
application decides per classification what reaches its logs: erase it,
replace it with an HMAC token so entries stay correlatable, or pass it
through.

### Taxonomy
`Opc.Ua.Redaction.OpcUaDataClassifications` defines the `OpcUa` taxonomy
and one attribute per classification:

| Classification | Used for | Parameters |
|---|---|---|
| `[NetworkAddress]` | URLs, host and domain names, IP/MAC addresses,
end points, ports | 120 |
| `[ApplicationIdentity]` | certificates, subject, issuer, thumbprint,
ApplicationUri | 103 |
| `[FilePath]` | file, directory and certificate store paths | 67 |
| `[SensitiveText]` | exception or remote-supplied text passed as a
parameter | 55 |
| `[UserIdentity]` | user names, token subjects | 7 |

Correlation ids (`ChannelId`, `SessionId`, ...), status codes and
NodeIds stay unclassified.

### Pipeline and defaults
- Opc.Ua.Core now references
`Microsoft.Extensions.Telemetry(.Abstractions)` 10.11.0, so every
project above Core uses the `Microsoft.Gen.Logging` generator.
Opc.Ua.Client already did, through Http.Resilience. The packages ship
net462 and netstandard2.0 builds, so net48 is supported.
- **A plain `ILoggerFactory` logs `(null)` in place of classified
values.** Classified values are only written by the redaction-aware
logger.
- The new `ILoggingBuilder.EnableOpcUaRedaction()` turns that logger on.
It is `EnableRedaction()` with the discriminator off; otherwise every
unredacted value gets `:ParamName` appended.
`AddOpcUa().AddLogging(...)` and `DefaultTelemetry.Create(...)` call it,
as do the samples, so they log the same information as before until
redactors are registered.
- `services.AddOpcUa()` alone does not change the host's logging.
Generic-host apps call `builder.Logging.EnableOpcUaRedaction()`. This is
documented as a behavior-breaking change in
`docs/migrate/2.0.x/telemetry.md`.

### Obsolete API
`Redact`, `RedactionWrapper<T>`, `IRedactionStrategy`,
`RedactionStrategies` and `SimpleRedactionStrategy` are `[Obsolete]`.
They still work, but the stack no longer uses them.

### Not classified
- The `Exception` passed to a log call. Applications can drop exception
messages with `LoggerEnrichmentOptions.IncludeExceptionMessage`.
- Exception and `ServiceResult` message texts.
- `ServiceResult`/`StatusCode` parameters and logged data values.
- The few log calls whose level is decided at run time.
- `Opc.Ua.Vision.OpenUsd`, which references only Opc.Ua.Types and cannot
see the attributes.

### Generator behavior differences
These are visible to log consumers and to tests:
- `IFormattable` values such as `NodeId` are stored in the structured
state as invariant strings, not as objects.
- The generated log state is a thread-local object that is cleared after
the call.
- Classified values are pooled redactor objects, valid only during the
`Log` call.

Tests that inspected log state after the fact now capture it during the
call. The shared `RecordingLoggerProvider` snapshots classified values.

## Related Issues
- Fixes #4857

## Types of changes
- [ ] Bugfix (non-breaking change which fixes an issue)
- [x] Enhancement (non-breaking change which adds functionality)
- [ ] Test enhancement (non-breaking change to increase test coverage)
- [x] Breaking change (fix or feature that would cause existing
functionality to not work as expected; plain logger factories drop
classified values)
- [x] Documentation Update (if none of the other choices apply)

## Checklist
- [x] I have read the CONTRIBUTING doc.
- [x] I have signed the CLA.
- [x] I ran tests locally with my changes, all passed.
- [x] I fixed all failing tests in the CI pipelines.
- [x] I fixed all introduced issues with CodeQL and LGTM.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added necessary documentation (if appropriate).
- [ ] Any dependent changes have been merged and published in downstream
modules.

## Further comments
New tests:
- `LogClassificationCoverageTests` (Tools.Tests) scans all 104 stack
assemblies and fails if an address, identity or path parameter of a
`[LoggerMessage]` method is unclassified.
- `DataClassificationLoggingTests` (Core.Tests) pins erasing and HMAC
redaction, pass-through, the plain-factory drop, and both stack
defaults.

Local results on net10.0:

| Suite | Passed | Failed |
|---|---|---|
| Core.Tests | 6,489 | 0 |
| Client.Tests | 3,231 | 0 |
| Tools.Tests | 7,210 | 0 |
| PubSub.Tests | 1,536 | 0 |
| Core.Diagnostics.Tests | 571 | 0 |
| Configuration.Tests | 241 | 0 |

- **Server.Tests:** the full run predates the final test fixes and had
12 failures. All 12 were in the fixtures fixed afterwards, which now
pass: RoleStateBinding (45/45) and the FileDirectory and
compatibility-logging fixtures (74/74 in total). The full Core.Tests and
Server.Tests runs were not repeated after those fixes.
- **Builds:** `UA.slnx` builds with 0 errors for net10.0 and net48, and
the whitespace check passes.
- **NativeAOT:** not confirmed locally, because the machine has no C++
linker. A trimmed publish of `Opc.Ua.Aot.Tests` reported no trim or AOT
warnings from the stack, the new packages or the generated logging code;
its only errors come from the TUnit test framework (`FSharp.Core` not
found). The `aot-test` CI job has to confirm the native build.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Marc Schier <marcschier@users.noreply.github.com>
2026-10-21 09:47:07 +00:00
5899354441 Clarify documentation, migration guidance, and SDK selection (#4823)
# Description

Remove stock phrasing and correct misleading or inconsistent technical
documentation. The initial two passes covered 367 Markdown files,
followed by an interactive review of 169 editorial flags. The U5
interpretation originally deferred in that review has since received the
upstream Mantis answer recorded by #4843; remaining server/spec
follow-ups stay explicit in the CTT log.

- Correct package responsibilities, migration steps, API names,
ownership, protocol descriptions and links. Keep bundled migration
guides synchronized. Preserve historical conclusions as historical
records with separate re-check notes.
- Provide at least two rendered TOC levels throughout `docs/`. Convert
appropriate architecture figures to Mermaid without changing code
examples, transcripts or historical quotations.
- Address the October 19 feedback with concrete setup/migration
instructions, corrected certificate paths and PubSub names, shorter
repeated caveats, consistent XML comments, and a full review of the
Concepts guide. Retain #4839's API corrections.
- Integrate #4841's runtime/performance guidance and review
incoming/recent prose. Preserve dated measurements rather than claiming
new benchmark runs; clarify allocation ratios, startup-only processor
settings, JIT warm-up and the benchmark commit's scope. Missing raw
measurement artifacts and repeat ranges remain tracked by #4842.
- Merge master through `e8c5d800a` in `32c6394b7`. Resolve the CTT log
conflict using the updated Mantis states and OPC answers, retaining
historical framing and re-checks for C28/C32/C43.
- Address the latest naming/index feedback in `adb5bf693`: rename the
guides to `docs/WotCon.md` and `docs/WotNodeSetConversion.md`; move the
provider and DataType-declaration contracts into the conversion guide
and remove `docs/WoT.md`. Rename `samples/DI` to `samples/Di` and
`samples/Isa95` to `samples/ISA95`. Add content-index READMEs to Di,
FluentApi, ISA95, Lds and PubSub; update solution entries, project
references, Docker/CI paths, test fixture paths and documentation links
with exact casing.

**WoT guide scope.** This branch retains the detailed conversion guide
that #4758 had replaced with an anchor stub. Conversion limits, verified
document sets, preservation rules and provider ownership now live
together in `WotNodeSetConversion.md`; binding/runtime guidance remains
in `WotBindings.md` and `WotCon.md`. The design-rationale/test-ledger
prose was shortened, the oversized argument-table cell moved into its
section, and existing anchors preserved.

**SDK policy is a separate, explicit user-directed change.**
`global.json` sets a stable minimum of `10.0.400`, `rollForward:
latestMajor`, and `allowPrerelease: false`. Newer major SDKs are
permitted; CI's explicit SDK pins are unchanged. README, GettingStarted,
DeveloperGuide and Recycling instructions reflect the possible analyzer
differences. The image-contract regression fix accepts compatible stable
SDK images instead of requiring exact equality with the floor.

The latest four rename/index threads are resolved after publication.
Three older threads remain open: universal obsolete-removal timing,
unconditional nonce-strength wording, and a blanket Vision NativeAOT
claim. Existing implementation/evidence constraints are explained in
their inline replies. The two-level TOCs remain as requested.

## Validation

- `git diff --check` and staged diff checks: passed. The untouched
upstream AutoID NodeSet retains its provenance bytes, including two
existing license-header trailing spaces.
- `.\.azurepipelines\validate-migration-plugin.ps1`: passed; 15 bundled
migration documents and manifests synchronized.
- Documentation/path validation: **101** documents retain two rendered
TOC levels (**2,228** entries), with idempotent formatting. **52** new
relative links and their anchors passed exact-case checks. No tracked
references to the old guide/sample paths remain. Both relocated
provider-contract sections are preserved in the destination.
- `dotnet sln UA.slnx list`: passed; shows the renamed Di/ISA95 projects
and the existing FluentApi/Lds/PubSub projects. Updated solution and
pump ProjectReference paths also passed exact-case existence checks.
- Latest focused commands, run from the repository root:

```powershell
# Passed: 61 channel send-order and reconnect tests.
dotnet test tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj -p:CustomTestTarget=net10.0 --filter 'FullyQualifiedName~UaSCBinaryChannelSendOrderTests|FullyQualifiedName~TcpReconnectKeyAgreementTests' --verbosity minimal

# Passed: 5 pump datasheet tests; builds the renamed pump sample.
dotnet test tests\Opc.Ua.Di.Tests\Opc.Ua.Di.Tests.csproj -p:CustomTestTarget=net10.0 --filter 'FullyQualifiedName~PumpDatasheetConformanceTests' --verbosity minimal

# Passed: 4 repository NodeSet path/round-trip tests, including the moved pump models.
dotnet test tests\Opc.Ua.Types.Tests\Opc.Ua.Types.Tests.csproj -p:CustomTestTarget=net10.0 --filter 'FullyQualifiedName~RepositorySourceSurvivesSerializedLinkedPartitionsAsync' --verbosity minimal

# Passed: build, zero warnings/errors.
dotnet build samples\ISA95\MinimalIsa95Server\MinimalIsa95Server.csproj -p:CustomTestTarget=net10.0 -f net10.0 --verbosity minimal

# Passed: dependent sample-test project build, zero warnings/errors.
dotnet build tests\Opc.Ua.WotCon.Samples.Tests\Opc.Ua.WotCon.Samples.Tests.csproj -p:CustomTestTarget=net10.0 -f net10.0 --verbosity minimal
```

Earlier focused integration checks passed 2,620 tests across AutoID,
event queues/scheduling, source generation, MCP/package profiles and
WoT. Both reviewed Mermaid diagrams rendered locally without network
access. These checks are not a full solution run.

The full solution has **not** been rerun for this review. Earlier
`dotnet test UA.slnx -p:CustomTestTarget=net10.0 --verbosity quiet` did
not pass: three AOT projects rejected VSTest and 20 Vision tests failed
during graphics initialization with `OutOfMemoryException`. Remaining
long-running tests were stopped at the user's request. A focused Vision
rerun passed 558 tests and reproduced those 20 failures. The earlier
`dotnet test UA.slnx -p:CustomTestTarget=net48 --verbosity quiet` was
blocked by the former SDK pin and was not retried after the policy
change. The user approved proceeding with those limitations documented;
current remote CI is not claimed green.

## Related Issues

- Follow-up #4842 tracks the requested 2.0.2+ benchmark/tuning and
diagnostic-tool documentation refresh. #4841 supplies documentation and
earlier results, not all newly requested measurements.
- #4758 explains the conversion guide's earlier consolidation; the
current provider contracts are now consolidated into
`WotNodeSetConversion.md`.
- #4839 supplies the reviewed API corrections; #4843 supplies the
updated CTT issue record.
- The original prose cleanup and SDK policy were user-requested without
a source issue. No new runtime architecture is introduced by these
documentation edits.

## Checklist

_Put an `x` in the boxes that apply. You can complete these step by step
after opening the PR._

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [ ] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

---------

Co-authored-by: Roman Ettlinger <romanett98@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 10:10:55 +02:00
Marc Schier 7e25e1a3d7 Extract remaining shared companion, rendering and packaging changes (#4806)
# Description

The shared AI deployment contracts and instance registration; DI
software-upload outcomes, digest handling and simulator; bounded
Positioning/Pumps reads; PubSub raw-data/matrix and security-provider
handling; Vision read-quality contracts; xRegistry resource writes and
instance projection; OpenUSD authoring, embedded viewport and
retained-frame capture APIs; standalone sample CLI controls; and the
generic Debug/RID package-policy and fork/Dependabot coverage-comment
fixes. The Linux native prerequisite step is retained for **Vision tests
only**.

## Related Issues

- Related tracking issue:
[#4461](https://github.com/OPCFoundation/UA-.NETStandard/issues/4461),
without automatic closure.
- Source work:
[#4462](https://github.com/OPCFoundation/UA-.NETStandard/pull/4462),
which remains intact for the coordinator's later integration.
- Immediate stacked predecessor:
[#4804](https://github.com/OPCFoundation/UA-.NETStandard/pull/4804).

## Checklist

_Put an `x` in the boxes that apply. You can complete these step by step
after opening the PR._

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [ ] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
2026-10-17 07:52:02 +02:00
romanettandClaude Opus 5.5 9b034f8291 Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description

Nullable reference types are now enabled once, in `common.props`, and
every project in the repository builds warning-clean with them.

**Configuration**
- `<Nullable>enable</Nullable>` lives in `common.props`. All per-project
`<Nullable>` settings are removed: 178 project files (enable,
annotations, and the `disable` in the three Aot test projects).
- 419 per-file `#nullable enable`/`restore` directives and the 4
`#nullable disable` directives are removed.
- Kept on purpose:
- The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type
Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`,
`PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the
project setting in generated code, so the directive is required there.
- `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against
the 1.5.378 packages and does not import the repository props.
- `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now
describe the central setting.

**Migration** (the 43 unset and 13 annotations-only projects, roughly
10k diagnostics)
- **Production code: annotated at the declarations.** Covers
`tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the
LDS sample.
- Members, parameters and returns are nullable where null actually
flows.
- Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`,
`IsMethodTypeDesign`).
- `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take
`params object?[]`. `Template.AddReplacement` takes `string?` (rendering
already skips null replacements).
- `IRegisteredServerStore.RegisterAsync` returns
`Task<RegistrationEntry?>`: it returns null when a registration is
removed.
- Dereferences backed by an invariant (for example, the validated model)
use `!`, so runtime behaviour is unchanged.
- **Behavioural changes in production code** (small, deliberate):
- `NodeSetToModelDesign` throws `InvalidOperationException` when the
file is not a NodeSet. Before, it threw a NullReferenceException on the
next line.
- `ModelDesignValidator` starts its lookup tables empty instead of null.
A redundant `if (m_nodes == null) return;` guard is gone.
- **Test code: minimal annotations.**
  - `?` on values that can be null, `!` where the test asserts non-null.
  - `= null!` for fixture members assigned in SetUp.
- `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the
`IEncodeable` constraint.
- A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`)
folded into the adjacent `IsNull`/`Count` checks.

**Notes for reviewers**
- The diff is large (~1,500 files) but almost entirely mechanical: `?`,
`!` and directive removal. The non-annotation changes are listed above.
- Known follow-up, not done here:
`IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay
non-nullable even though the validator returns null before it loads a
dictionary. Making them nullable would ripple through every generator
call site.

## Related Issues

- Fixes #4439

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

The "no new warnings" box covers net10.0 and net48: full non-incremental
builds of `UA.slnx`, `tools/SourceGeneration.slnx`,
`tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0
warnings and 0 errors under the repository's warnings-as-errors
settings. net9.0 and net8.0 also had no nullable diagnostics, but they
were not rebuilt after the final fixes. Tests have not been run locally
yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 08:48:13 +02:00
Klaus Landsdorf dadd609ec0 Scales: the OPC 40200 companion SDK with the OPC 30050 PackML model (#4580)
> Tracking issue #4576. #4577, #4578 and #4579 are merged, so this
branch is
> rebased onto `master` and contains only the Scales and PackML commits.

## Summary

Last of the four PRs of #4576. Adds OPC 40200 Weighing Technology 2.00
as a
model package plus a server and a client SDK, and the OPC 30050 PackML
model
it composes.

| Package | Content |
|---|---|
| `Opc.Ua.PackML` | OPC 30050 PackML 1.01, plus
`PackMLStateMachineController`, which drives a base state machine
together with its nested machine and execute state machines (activation
and deactivation of the sub-state machines, the PackML methods with an
`Executable` attribute that follows the state, acting states left
automatically or by the equipment) |
| `Opc.Ua.Scales` | OPC 40200 2.00
(`http://opcfoundation.org/UA/Scales/V2/`), generated from the
unmodified UA-Nodeset publication, plus the contracts client and server
share |
| `Opc.Ua.Scales.Server` | `AddScales()` / `ConfigureScales(...)`; all
fourteen scale kinds and scale systems below `DeviceSet` and `Machines`;
a runtime that does the weighing arithmetic (ranges, rounding to d or e,
zero band, overload/underload, zero, tare, preset tare, registration),
production presets with DI locking, statistics, recipes and recipe
files, Annex C events and alarms over a `HasNotifier` hierarchy, and the
methods of every scale type |
| `Opc.Ua.Scales.Client` | discovery, weight reads and streaming,
zero/tare/register, products, PackML state and commands, type-specific
methods, events and alarms, snapshots |

`docs/Scales.md` includes a conformance matrix against the 21 OPC 40200
server
facets with test evidence, the places where the specification and the
NodeSet disagree, and known limitations.

## Review follow-ups

Separate commits after the docs/samples commit, each with a test that
fails without it where the defect can be observed.

- Copilot review: `CurrentProductsChanged` fires only when the set of
  current products changes; the new public collection APIs
  (`ScalesServerOptions.AdditionalNamespaceUris`,
  `ScaleNamespaceIndices.SearchOrder`, `ScalesModel.AllKinds`) are
  `ArrayOf<T>`.
- Own review with the rubric in `.github/copilot-instructions.md`:
  - PackML raises `StateChanged` after its lock is released (queued, in
    order), reads `StartParameters` under the lock, and reports the
    handler's status when an acting state cannot complete.
- Scales: concurrent first creations share one runtime (the lock service
    refused a second attach); two creations of the same name cannot both
publish; a removed product id stays reserved until its node is gone; the
    zone, target-count and vehicle handlers change products under the
preset lock; the product factory and `configure` delegate run outside
    it; the recipe upload reads `ActiveRecipe` under the lock; resetting
    the statistics also resets the tolerance percentage.
- The client lets a failed browse call propagate instead of reading it
as
    "no children".

## Testing

- `tests/Opc.Ua.Scales.Tests`: 115 tests on net8 and net10 and with the
netstandard2.1 libraries, including the PackML runtime; the
client/server
  tests connect a real session over `opc.tcp` to a hosted
  `ScalesNodeManager`.
- `UA.slnx` builds for net472, net48, netstandard2.0/2.1 and net8/9/10
with
  SDK 10.0.401 and no new warnings.
- Changed-line coverage of this layer (`check-coverage.ps1`): 94.59 %
(4585 of 4847 changed lines, against `master`).
- Fork CI, full PR matrix incl. macOS: in progress, [run
36676067489](https://github.com/biancode/UA-.NETStandard/actions/runs/36676067489);
CodeQL: in progress, [run
36676070427](https://github.com/biancode/UA-.NETStandard/actions/runs/36676070427).
- On the current `master` (with Pumps merged), `UA.slnx` builds for
every target
  framework and the Scales and Pumps tests pass.
- Beyond the in-repo tests the implementation has been validated end to
end
  against a separate, non-public OPC UA client and server.

## Reviewing

Commits: PackML → Scales model → server → client → tests → docs and
samples
→ review follow-ups.
2026-09-30 10:54:48 +02:00