mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-26 12:40:03 +08:00
fix/encoder-stack-depth-json-flake
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 romanettandClaude Opus 5.5
|
9b034f8291 |
Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700)
# Description Nullable reference types are now enabled once, in `common.props`, and every project in the repository builds warning-clean with them. **Configuration** - `<Nullable>enable</Nullable>` lives in `common.props`. All per-project `<Nullable>` settings are removed: 178 project files (enable, annotations, and the `disable` in the three Aot test projects). - 419 per-file `#nullable enable`/`restore` directives and the 4 `#nullable disable` directives are removed. - Kept on purpose: - The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`, `PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the project setting in generated code, so the directive is required there. - `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against the 1.5.378 packages and does not import the repository props. - `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now describe the central setting. **Migration** (the 43 unset and 13 annotations-only projects, roughly 10k diagnostics) - **Production code: annotated at the declarations.** Covers `tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the LDS sample. - Members, parameters and returns are nullable where null actually flows. - Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`, `IsMethodTypeDesign`). - `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take `params object?[]`. `Template.AddReplacement` takes `string?` (rendering already skips null replacements). - `IRegisteredServerStore.RegisterAsync` returns `Task<RegistrationEntry?>`: it returns null when a registration is removed. - Dereferences backed by an invariant (for example, the validated model) use `!`, so runtime behaviour is unchanged. - **Behavioural changes in production code** (small, deliberate): - `NodeSetToModelDesign` throws `InvalidOperationException` when the file is not a NodeSet. Before, it threw a NullReferenceException on the next line. - `ModelDesignValidator` starts its lookup tables empty instead of null. A redundant `if (m_nodes == null) return;` guard is gone. - **Test code: minimal annotations.** - `?` on values that can be null, `!` where the test asserts non-null. - `= null!` for fixture members assigned in SetUp. - `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the `IEncodeable` constraint. - A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`) folded into the adjacent `IsNull`/`Count` checks. **Notes for reviewers** - The diff is large (~1,500 files) but almost entirely mechanical: `?`, `!` and directive removal. The non-annotation changes are listed above. - Known follow-up, not done here: `IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay non-nullable even though the validator returns null before it loads a dictionary. Making them nullable would ripple through every generator call site. ## Related Issues - Fixes #4439 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. The "no new warnings" box covers net10.0 and net48: full non-incremental builds of `UA.slnx`, `tools/SourceGeneration.slnx`, `tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0 warnings and 0 errors under the repository's warnings-as-errors settings. net9.0 and net8.0 also had no nullable diagnostics, but they were not rebuilt after the final fixes. Tests have not been run locally yet. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
b8f07843b2 |
Secure channel: never answer with an expired token; accept a just-expired one (+ labelled soak/stress CI) (#4698)
## Summary **Secure channel: a client could drop the channel when the server answered with a just-expired token.** Running the long-dormant ConnectionStability soak locally, the subscription channel shut down with `BadSecureChannelClosed`. The session then timed out and the subscription was abandoned. Root cause, against OPC 10000-4 §5.6.2.1: 1. **Server (rule 2: use the old token only "until the SecurityToken expires").** After a renew, the server keeps securing messages with the old token until the client sends something secured with the new one. That activation only happened on *receive*. A client that sends nothing, for example one whose Publish requests are all outstanding, kept getting responses secured with the old token after it expired. 2. **Client (rule 3: "accept Messages secured by an expired SecurityToken for up to 25 % of the token lifetime").** The client rejected any such response the moment the token expired, throwing `BadTcpSecureChannelUnknown`, and shut the channel down. From the failing run (token lifetime 10 s): the client renewed to token #11 at 09:59:01.686 and sent nothing on that channel afterwards. The server, still on #10, sent a Publish response at 09:59:04.389, 380 ms after #10 expired. The client rejected it and closed the channel. Nothing reconnected, the 10 s session timeout expired, and the server abandoned the subscription. Both halves date back to the 2016 initial commit; this is not a recent regression. It hits whenever the client sends nothing on the channel between renewing and the old token's expiry, so it is timing-dependent. ### Fix - `UaSCBinaryChannel.ActivateRenewedTokenIfDue()` activates a pending renewed token once the current one passes `TokenActivationPeriod` (95 %). This is the rule the receive path already applied. It is now shared, and `TcpServerChannel.SendResponse` calls it before securing a response. - `ReadSymmetricMessage` accepts responses (not requests) secured with an expired token for `TcpMessageLimits.TokenExpiryGracePeriod` (25 %) of its lifetime. ### Tests `TcpReconnectKeyAgreementTests` now runs the server channel and the peer on one shared `FakeTimeProvider`: - `ResponseSentAfterTheCurrentTokenExpiresUsesTheRenewedTokenAsync`: a response sent at 96 % of the old token's lifetime is secured with the renewed token. - `ResponseSecuredWithTheExpiredTokenIsAcceptedWithinTheGracePeriodAsync`: accepted at 120 % of the lifetime, rejected at 130 %. Both fail without the fix. All 40 tests in the fixture pass with it. ## CI: soak and stress runs on labelled PRs (and the soak that tested nothing) The bug stayed hidden because `stability-test.yml` still filtered `Opc.Ua.Client.Tests` after #3810 moved the test to `Opc.Ua.Sessions.Tests` (2026-05-28). Since then the nightly has run **zero tests** while reporting green: runs dropped from ~100 min to ~15 min. - `stability-test.yml` now targets `Opc.Ua.Sessions.Tests` and writes a TRX file. - New `.github/scripts/assert-tests-ran.ps1`, used by all three opt-in workflows, fails a run in which no test executed, or none passed. Contract-tested in `CiAssertTestsRanTests`. - Like `interop`, the opt-in workflows can now run on a labelled PR. They are not required checks. A later push cancels the run in progress, and adding an unrelated label neither starts nor cancels a run. | Label | Workflows | Run on a PR | | --- | --- | --- | | `stress` | Stress Tests (ChaosTCP) | Random seed, saved as `chaos-seed.txt` in the results artifact | | `soak` | Connection Stability Test, Sample HA Long-Haul Test | 30 min stability; 15 min per HA scenario | ## Test plan - [x] `TcpReconnectKeyAgreementTests`: 40/40 pass; the 2 new tests fail without the fix - [x] All `Opc.Ua.Core.Tests.Stack` tests (net10.0): 3548 passed, 0 failed - [x] `CiAssertTestsRanTests`: 4/4 - [x] `soak` + `stress` labelled run on this PR (before the fix): both stability tests, both HA long-haul scenarios and ChaosTCP passed on Linux (the label triggers and push-cancellation work) - [x] Local short-haul soak with the fix (Windows, 10 s tokens), 2 runs: both pass with full notification counts; run 2 hit the window once (`Token activated forced` on send) and kept the channel - [x] `soak` + `stress` labelled run with the fix: Connection Stability (2 tests), Sample HA Long-Haul and ChaosTCP all pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
731600801a |
Release: stop retrying 404s in the feed gates; keep GitHub Release notes under the size limit (#4696)
# Description `Release to nuget.org` [run 37591540644](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/37591540644) was the stable 2.0.0 promotion. It spent more than an hour in **Verify already-published package identity**. It pushed every package to both feeds and created the `2.0.0` tag, then [failed](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/37591540644/job/112693955843) in **Tag published package candidate** before the GitHub Release existed. This PR fixes both problems. ## 1. Feed gates retried every 404 PowerShell 7's `Invoke-WebRequest -MaximumRetryCount` retries every status from 400 to 599, and that includes 404. `-SkipHttpErrorCheck` stops the error from being thrown but doesn't stop the retries. Both feed gates ask once per package whether an id/version is published, and 404 is the normal "not yet" answer. Each absent package therefore cost 3 retries × 5 s, about 15 s. A stable set is never pushed automatically, so for 2.0.0 every package, `.Debug` included, was absent on **both** feeds. - `package-version-policy.ps1`: adds `Invoke-FeedRequest`. It retries only network failures, 408, 429 and 5xx, and returns every other response straight away. After the last retry it returns the final response, or rethrows the network error. - `assert-published-packages-match.ps1` and `validate-preview-package-ordering.ps1` call it instead of `Invoke-WebRequest -MaximumRetryCount 3 -RetryIntervalSec 5`. The status handling in each caller is unchanged, so the gates still fail closed: anything other than 200 or 404 still blocks. - `PackageVersionPolicyTests.GetGitHubPackagesPublishedVersionPagesResultsAndFailsClosedAsync` now dot-sources the policy script so the extracted function can find `Invoke-FeedRequest`. The `Invoke-WebRequest` stub still intercepts the call. - New stub-based tests check that `Invoke-FeedRequest` makes one request for 200/400/403/404 and retries 408/429/5xx exactly `MaximumRetryCount` times. They also check that it stops once a retry is not transient, rethrows an exhausted network failure and passes `-OutFile` with `-PassThru`. | Response | Before | After | |---|---|---| | 404 | 4 requests, 15 s | 1 request, immediate (live nuget.org: 0.58 s) | | 403 | 4 requests | 1 request | | 503 / 429 | retried 3× | retried 3× (unchanged) | | connection refused | — | retried, then rethrown | ## 2. GitHub Release body too long ``` HTTP 422: Validation Failed (https://api.github.com/repos/OPCFoundation/UA-.NETStandard/releases) body is too long (maximum is 125000 characters) ``` `gh release create --generate-notes` diffs against the latest release when no previous tag is given. Here that was `1.5.378.182`, which is on another branch, so the generated notes for 2.0.0 came to 191,048 characters. The step now calls `POST /releases/generate-notes` with `previous_tag_name` from the new `Get-PreviousReleaseTag` helper in `package-version-policy.ps1`. It picks the greatest same-major semantic version strictly lower than the tag, ignoring non-semver tags like 1.5.378.x, so a servicing 2.0.1 published after 2.1.0 still diffs against 2.0.0. For 2.0.0 that is `2.0.0-preview.2`, and the notes come to 31,408 characters (measured against the real repo; that call has no side effects). If a body still exceeds 120,000 characters, it is cut at a line boundary and a compare link is appended. The tag and release checks are idempotent, and the pushes use `--skip-duplicate` and verify identity, so re-running the promotion finishes the release. **For run 37591540644 to pass:** the dispatch runs `release.yml` from `release/2.0`, so these changes have to reach `release/2.0` before the promotion is re-run. ## Verification - All 3 scripts parse with 0 errors. The modified workflow step also parses, and the YAML loads. - Live nuget.org: absent version → 404 in 0.58 s; package index → 200. httpbin 503 with `-RetryIntervalSec 1`: retried, 503 after 4.7 s. - `dotnet test tests/Opc.Ua.Tools.Tests`: 939 passed (1 skipped) on net10.0, plus net8/9/48. ## Related Issues - No issue; found investigating run 37591540644. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [ ] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
fbaabf5b28 |
Cut CI runner minutes: duration-packed test batches, CodeQL on Linux/daily, cancellable branch packing (#4694)
Reduces GitHub Actions runner minutes without removing any OS, target framework, test project or check. All figures come from master CI run 37520359929 (158 jobs, ~1,880 runner-minutes) and the last two weeks of run history. ## 1. Test batches packed by duration (`get-ci-matrix.ps1`) Previously the expander picked the **smallest** batch size that fit the 256-job limit, which meant 2 projects per job and 33 jobs per profile. Job logs show ~45% of the test matrix (674 of ~1,516 min) went to compiling. Each batch rebuilds the shared dependency graph: the first project's build took 3.8–4.9 min, the second only 0.4–0.9 min. - Projects are now packed in path order (which keeps `PubSub.*`, `Redundancy.*`, `WotCon.*` together) up to a 30-minute weight target, using `.github/ci-test-durations.json`. Each weight is the slowest PR profile's test minutes, rounded up, plus 1. - A batch never holds more projects than the job timeout can budget for: at most 7, given 20 + n×45 ≤ 350. - If the matrix would exceed the job limit, the target is raised instead. - **PR test matrix: 132 → 60 jobs** (15 per profile; macOS 33 → 15). Each batch is estimated at ≤36 min on the slowest runner, comparable to today's slowest jobs (37–43 min), so wall-clock should be about the same. Expected saving is roughly 300+ runner-min per run, plus less queueing for the scarce macOS runners. - The weekly full scope shrinks the same way (still 13 profiles). - The table only steers balance: a project missing from it gets `defaultMinutes`, so it is still scheduled. - `run-dotnet-tests.ps1` now records per-project `buildSeconds`/`testSeconds` (batch summary and job summary). The new `update-test-durations.ps1` refreshes the table from any run's `dotnet-results-*` artifacts. - `discover` output `batch_size` is renamed to `batch_target_minutes`. Nothing consumed it. New `CiMatrixScriptTests` cases: - each profile runs each of its projects exactly once; - only a lone project may exceed the target; - PR profiles average ≥3 projects per job; - the duration table names only existing projects. ## 2 + 3. CodeQL (`codeql-analysis.yml`) - **No longer runs on every push to master.** Instead, a daily schedule on master (was weekly) gives an analysis that always completes. Each analysis takes ~3 h, and with `cancel-in-progress: true` per branch, 49 of 76 push runs in two weeks were cancelled by the next merge before uploading anything. `master378`/`release/*`/`develop/*` keep their push trigger. PRs are still analyzed as before. - Only PR runs are cancelled when superseded. Push and scheduled runs always finish. - **Runs on `ubuntu-latest` instead of `windows-2025`.** Only net10.0 is built, so the extracted sources are identical. The traced Windows build alone took 70 min, against 12 min for the untraced Linux `UA.slnx` net10.0 leg. This PR touches a `.cs` file, so its own CodeQL run is the first measurement. ## 4. NuGet packing on feature branches (`nuget-publish.yml`) 298 of ~400 runs in two weeks were feature-branch pushes, each about 59 Windows-min across two signed builds. - A superseded feature-branch run is now cancelled. Protected lines and tags are never cancelled: they publish previews, and `release.yml` promotes a run by ID. Each manual (`workflow_dispatch`) run gets its own concurrency group, so a later push cannot cancel it. - A new `changes` job skips packing when a feature branch, as a whole, changes only documentation. It diffs against the merge base with master, not the previous push, so a docs-only push that cancels a code push cannot mask that code change. It uses the same `get-path-relevance.ps1` rule as CI. - Protected refs, tags, manual runs, and any branch whose merge base cannot be resolved always pack. Checked locally for docs-only branch (skip), code branch, protected ref, manual run and unknown commit (all pack). ## Verification - `CiMatrixScriptTests`, `CiPathRelevanceTests` and the new `CiTestDurationsScriptTests` (covering `update-test-durations.ps1`): all `*.Ci*` tests in `Opc.Ua.Tools.Tests` pass locally, 63/63. - This PR's own CI run shows the new matrix. Compare its total runner-minutes and longest test job against run 37520359929. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
7c2c6e7f8e |
Number commit-id-only prerelease previews 0 so they never outrank a release (#4692)
## Problem When a stable `version.json` (for example `2.0.0`) is built from a ref that isn't a public release branch, Nerdbank.GitVersioning makes the root version `2.0.0-g<commit>`. The "other prerelease label" case in `version.targets` turned that into `2.0.0-preview.g<commit>` for the preview-only package families. SemVer 2 ranks the alphanumeric `g…` identifier above every numeric one, so that version outranks every `2.0.0-preview.N` ever released, and NuGet versions are immutable. This happened on 2026-10-06 while master briefly carried `2.0.0`. Two such versions on GitHub Packages now block the stable 2.0.0 release on `release/2.0` (handled there in #4691). ## Change - `version.targets`: a new Case 2a maps a prerelease that is only a commit id to `<base>-preview.0.g<commit>`, keeping any build metadata last. Preview number 0 sorts below any committed `PreviewPackageBuildNumber`. - `package-version-policy.ps1`: `ConvertTo-PreviewPackageVersion` applies the same rule, so the package-set validator agrees with the build. - Tests: commit id only, commit id plus metadata, and `-gamma.1`, which is not a commit id. Checked with MSBuild against the real `version.targets`: `2.0.0-ge78c648295` maps to `2.0.0-preview.0.ge78c648295`. `2.0.0`, `2.0.0+g…`, `-rc.1` and `2.0.1-preview.N.g…` are unchanged. All 90 `PackageVersionPolicyTests` pass on net10.0. Same commit as the prevention part of #4691 (release/2.0). This PR doesn't include the release-only `PreviewPackageOrderingExemptions` change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
fa06c66dbb |
Stop versioning DI and ISA95 packages as preview (#4686)
## Summary Removes the **DI** (`Opc.Ua.Di*`) and **ISA95** (`Opc.Ua.ISA95*`) package families from the preview-only set, so they ship with the stable root version (e.g. `2.0.1` instead of `2.0.1-preview.N`). - `version.targets`: drop the two `_IsPreviewPackage` conditions. - `.azurepipelines/package-version-policy.ps1`: drop `ISA95|Di` from `Test-PreviewPackageId`. It must match `version.targets`, because the package-set validator fails if the two disagree. - `PackageVersionPolicyTests`: ISA95/Di IDs are now expected to be non-preview, they are removed from the classifier-agreement token list, and the "already-preview base version" case now uses Robotics. - Docs (`DeveloperGuide.md`, `ReleaseProcess.md`, `docs/README.md`): the preview-family list no longer includes DI or ISA95, and the "preview package" label is removed from the DI, Software Update (part of the DI packages), and ISA-95 entries. ## Test `dotnet test tests/Opc.Ua.Tools.Tests --filter PackageVersionPolicy` (net10.0): 87/87 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6b8b2eb79b |
Tag promoted NuGet package releases (#4605)
# Description
Promoted NuGet packages lacked a corresponding repository tag. The
promotion workflow now records each published candidate with an
immutable version tag and matching GitHub Release.
### Promotion tagging
- Creates a tag from the signed candidate manifest after both feeds
accept the package set.
- Creates a GitHub Release; preview channels are marked prerelease.
- Validates existing tags/releases on recovery and rejects mismatched
targets.
```text
2.0.0-preview.5 ->
|
||
|
romanettandClaude Opus 5.5
|
a430239fde |
Allow release.yml to promote release-line previews to nuget.org (#4574)
# Description Adds a way to publish a numbered release-line preview (e.g. `2.0.0-preview.6`) to nuget.org. Since #4513, `nuget-publish.yml` pushes previews only to GitHub Packages, and `release.yml` (the only nuget.org path) refused every non-stable candidate. That left no way to continue the `2.0.0-preview.N` series on nuget.org (last published there: `2.0.0-preview.5`). ## Changes - **`release.yml`** gains a `channel` input (`stable` | `preview`, default `stable`). - `stable` behaves exactly as before. - `preview` accepts only a public release-line preview, `M.m.p-preview.N` with no commit id, from the matching canonical `release/M.m` branch. `master` / PR builds (`-preview.N.g<sha>`) are rejected. - The candidate manifest's recorded `channel` must equal the dispatched one, so a stable promotion can never publish a preview by accident, or the reverse. - **`package-version-policy.ps1`**: new `Test-PromotablePreviewPackageVersion`, and an `-AllowPreview` switch on `Test-CanonicalReleaseBranchForPackageVersion`. Callers without the switch (the stable gate in `nuget-publish.yml`) behave exactly as before. - **`docs/ReleaseProcess.md`**: new "Preview release" procedure, including how to pick the preview number with `versionHeightOffset`. - **Tests**: predicate cases, the `-AllowPreview` switch, and a pin that `release.yml` defaults to `stable` and relaxes the gate only for `channel=preview`. ## GitHub Packages preview feed is unchanged - `nuget-publish.yml` is not touched. `master` keeps auto-publishing `2.0.0-preview.<height>.g<sha>` to GitHub Packages, and release-branch builds keep pushing their own `<base>-preview.N` there. - The commit-id suffix means `master` versions can never collide with a bare `2.0.0-preview.N`. - For a preview, `release.yml`'s GitHub Packages push is a verified no-op. The candidate's own `nuget-publish.yml` run already pushed those bytes, and the existing identity preflight and post-push check prove they match. ## Getting `2.0.0-preview.6` after merge 1. Cut `release/2.0` from `master` after this merges. `release.yml` executes the policy script from the candidate commit, so that commit must contain this change. 2. On `release/2.0`, commit a `versionHeightOffset` so the build lands on `.6`. Verified locally with nbgv 3.10.94: at the current `master` tip (height 24), one commit setting `-19` yields `2.0.0-preview.6` and the next yields `.7`. Recompute for the actual cut point with `nbgv get-version -v NuGetPackageVersion`. 3. Dispatch `release.yml --ref release/2.0 -f release_run_id=<run> -f channel=preview`, with a dry run first. `2.0.0-preview.6` is currently free: nuget.org has `.2`–`.5`, and the retired `release/2.0.0` pushed up to `.5` to GitHub Packages. ## Related Issues - Follow-up to #4513. Supersedes the `master` → `release/2.0.0` merge attempts #4566 / #4570; `release/2.0.0` is retired and cannot publish. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. Locally: `PackageVersionPolicyTests` + `NuGetPackageSetValidationTests` on net10.0, 96/96 passed. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
romanettandClaude Opus 5.5
|
d623007a3f |
Remove the Azure Pipelines CI and publishing definitions from master (#4575)
# Description Removes the Azure Pipelines CI and package-publishing definitions from `master`. GitHub Actions already owns all CI and package publishing for `master`, but pushes to `master` still queued the Azure definitions. They duplicated the Actions work and posted failing checks, for example [`Nuget Preview (Push Nuget Preview Pack Nugets Debug)`](https://github.com/OPCFoundation/UA-.NETStandard/runs/109065914524) on `19dc504c5`. ## Removed - `azure-pipelines.yml`, `azure-pipelines-preview.yml` - `.azurepipelines/` templates: `ci.yml`, `coverage.yml`, `preview.yml`, `sample-ha-longhaul.yml`, `sln.yml`, `test.yml`, `test-aot.yml` - Helpers only those templates used: `get-matrix.ps1`, `get-root.ps1`, `evaluate-test-results.ps1` - `TrxGateScriptTests`: it tested only `evaluate-test-results.ps1`. The Actions equivalent, `.github/scripts/get-test-verdict.ps1`, keeps its own `CiTestVerdictTests`. ## Kept **OneFuzz:** `.azurepipelines/onefuzz.yml` stays. Submitting to the internal OneFuzz service needs the Azure DevOps-only `onefuzz-task`, which has no GitHub Actions equivalent. The pipeline has no push, PR or schedule trigger, so it never runs on `master` pushes. The public drop build keeps running in the GitHub `fuzz-public-drop` job. **Scripts GitHub Actions still invokes:** these stay in `.azurepipelines/`, so every workflow path and `release.yml`'s candidate-bound checkout keep working. Only `generate-slnx.ps1` changes, and only its comment. | Script | Used by | | --- | --- | | `set-version.ps1` | buildandtest, nightly, nuget-publish, docker-image, stress / stability / long-haul, `run-dotnet-tests` action | | `generate-slnx.ps1`, `signlist{Debug,Release}.txt`, `validate-source-generator-packages.ps1` (+ `expected-packages.txt`) | nuget-publish | | `package-version-policy.ps1`, `validate-nuget-package-set.ps1`, `validate-preview-package-ordering.ps1`, `assert-published-packages-match.ps1` | nuget-publish, release | | `check-coverage.ps1` | buildandtest, nightly | | `validate-migration-plugin.ps1` | buildandtest | I didn't rename the directory on purpose. `release.yml` checks out the candidate commit and runs `./.azurepipelines/package-version-policy.ps1` from it, so moving the scripts would break promotion of any candidate built before the move. ## Docs and comments - `UA.slnx` drops the deleted files. - `codecov.yml`, `coverage-thresholds.json`, `targets.props`, the Redundancy sample test comments, the `.github` scripts and workflows, `CONTRIBUTING.md`, `docs/` and `fuzzing/` no longer describe Azure as running on `master` (`/azp run`, the `azure` Codecov flag, "the Azure gate", the Azure long-haul and AoT legs). - In the Developer Guide, the "Triggering a pipeline run" and "Migration status" sections are replaced by "Pull requests from outside contributors" and "Azure Pipelines on other branches". - The new section names `onefuzz.yml` as the one remaining Azure definition. - It keeps the warning that the `Release` ruleset's Azure required check must be lifted before cutting a new `release/2.<minor>` branch. - `master` is merged in, including the #4555 docs rewrite. Its new `docs/RedundancySampleTests.md` needed the Azure references removed again. `docs/README.md` already had none. ## Reviewer notes - **Azure DevOps side (admin action, not in this PR):** the definitions are configured in the Azure DevOps portal. With the YAML gone, `master` pushes give the CI and preview definitions nothing to run. Disabling definition 14's push trigger/schedule, definition 16's build-completion trigger, and the stale definition 13 stops any residual "file not found" runs. Leave the OneFuzz definition enabled. - `master378`, the 1.x `release/*` lines and the frozen `release/2.0.0` keep their own `azure-pipelines.yml`. This PR only touches `master`. ## Related Issues - Follow-up to #4513 (GitHub Actions for CI and nightly). ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [ ] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. This PR removes a test and adds none. Locally, at `9b00da817`, the full `Opc.Ua.Tools.Tests` suite passed on net10.0: 884 passed, 1 skipped. The later commits (OneFuzz restore, merge with `master`) touch only YAML and docs. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Marc Schier
|
23be306c65 |
Update NuGet dependencies (#4536)
## Summary - update NuGet dependencies to their latest compatible stable releases while retaining documented compatibility, native, security, and mocking pins - align the nbgv CLI with the central `Nerdbank.GitVersioning` version (3.10.94) and fail explicitly on tool installation/update errors - update the synchronized OpenUSD family to `0.14.0-alpha`; adapt the Vision blank-frame fixture to the new `RenderDiagnosticsState` constructor argument - retain `Microsoft.CodeAnalysis` API packages at the existing 5.0/4.14 host bands: Roslyn 5.9 requires Immutable/Metadata 10.0.1+, while the retained older compiler-host closure is intentionally pinned to 9.0.0 ## Dependency updates - `Dekaf` 1.17.0 → 1.21.1 - legacy ASP.NET Core 2.3 dependencies → 2.3.13; net8/net9 authentication and WebApi test packages → 8.0.31/9.0.20 - `Microsoft.CodeAnalysis.Analyzers` 4.14.0 → 5.9.0 - `TUnit` 1.68.0 → 1.69.0 - `Avalonia` / `Avalonia.Desktop` 12.1.2 → 12.1.3 - schema-test `System.Collections.Immutable` overrides 10.0.11 → 10.0.12 - `AzureSignTool` 3.0.0 → 7.0.1 and `NuGetKeyVaultSignTool` 3.0.45 → 3.2.3 - eight direct OpenUSD packages 0.12.0-alpha → 0.14.0-alpha ## Validation - `dotnet restore UA.slnx --disable-parallel` - clean `UA.slnx` Release build with `CustomTestTarget=net10.0` - `UA.slnx` Release builds with `net48` and `netstandard2.1` - OpenUSD and Vision test suites on net8/net9/net10: 882/882/921 and 484 passed + 1 environment skip on each target - full `Opc.Ua.Tools.Tests` net10.0: 963 passed, 1 skipped - source-generator package validation against a locally packed release package set - updated signing tools installed in an isolated directory; existing `sign` CLI options verified without credentials - nbgv 3.10.94 bootstrap exercised in simulated GitHub CI; disposable committed fixture verified canonical and case-invalid release refs with a four-component version ## Limits No protected signing/publication, Linux/macOS native rendering, or full stress/nightly suites were run locally. |
||
|
Marc SchierandCopilot Autofix powered by AI
|
e9313ef957 |
Only use GitHub action for CI and nightly, establish release-branch-only package publication (#4513)
## Description Two related changes to how the 2.0 line is built and shipped. ### 1. Release-branch-only package publication - `master` produces development previews only. - Stable packages can only be built from a canonical `release/<major>.<minor>` branch and are promoted manually from the exact signed candidate artifact. - Core packages use explicit `major.minor.patch` versions (`2.0.0`, `2.0.1`, then `2.1.0`); git height never changes a stable package patch. - XRegistry, WoT Connectivity, Vision, Robotics, Redundancy, Positioning, OpenUSD, ISA95, AI, DI, the relevant MCP extensions, and OpenUSD connector tools remain `-preview.N` while following the core numeric version. - nuget.org receives only non-`.Debug` package IDs; GitHub Packages retains the complete signed Release/Debug candidate. - `docs/ReleaseProcess.md` is a reproducible runbook for developers and agents, with each step stating its completion evidence. Promotion is source-bound and byte-checked: `release.yml` checks out nothing until the candidate SHA is known, and `assert-published-packages-match.ps1` proves that any version already on a feed holds *this* candidate's content before `--skip-duplicate` is allowed to skip it. Canonical release-branch naming is matched case-sensitively with no leading zeroes, and `version.json`'s `publicReleaseRefSpec`, both policy helpers and the copy `release.yml` inlines are pinned to one expression by tests. ### 2. PR CI moved to GitHub-hosted runners - `.github/workflows/buildandtest.yml` (**CI**) now owns pull-request validation outright, covering the Windows Debug/Release net48 + net10.0 solution builds and fast tests that only Azure Pipelines ran, alongside the existing Linux/macOS, all-target and NativeAOT legs. `build-and-test summary` remains the stable required check name. - `.github/workflows/nightly.yml` is a new **manual-only** (`workflow_dispatch`) full-scope workflow covering the Azure nightly workload: all solutions x seven target profiles x Debug/Release, the net472/netstandard profiles, Windows + Linux NativeAOT, and the long-running and durable tiers. - Shared deterministic discovery/batching lives in `.github/scripts/get-ci-matrix.ps1` with a local composite action, so both entry points expand the same workload manifest instead of duplicating a large embedded script. - `azure-pipelines.yml` is set to `pr: none`. - The standalone pump sample image workflow is folded into the consolidated **Images CI** workflow, which now builds all ten images and publishes an `images summary` rollup suitable as a required check. ## Related Issues No related issue provided. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. ## Validation - `Opc.Ua.Tools.Tests` full `net10.0` suite: **959 passed, 1 skipped, 0 failed**. - Version/package-policy regression subset (`PackageVersionPolicyTests`): **60 passed, 0 failed**. - GitHub Actions **CI** run 35719807620: **149/149 jobs green** on GitHub-hosted runners. - Canonical release-branch rule verified against the pinned nbgv 3.7.115 by building throwaway branches: `release/2.0`, `release/0.9`, `release/10.42` -> `PublicRelease=True`; `release/02.0`, `release/2.00`, `Release/2.0` -> `False`. - Package identity gate verified against a live nuget.org package: served packages carry a repository `.signature.p7s`, so equality is decided on content excluding that part. Re-signing and zip reordering produce an identical digest; a one-byte content change does not. - Synthetic Release-only, Debug-only, combined, preview, duplicate, orphaned-symbol, mismatched-root and accidental-stable-family package sets exercise the shared validator. - Stable-release rehearsal in an isolated `release/2.0` clone: `Opc.Ua.Di` packed as `2.0.0-preview.6` depending on stable Core `2.0.0`; `Opc.Ua.Di.Client` likewise against stable Client `2.0.0`. - All workflow YAML, `version.json`, the package-policy PowerShell scripts and the internal release-runbook anchors parse. ## Known validation limits - Full `UA.slnx` `net48` and `net10.0` local validation remains blocked by pre-existing `NETSDK1005` and dependency-version failures in unmodified fuzz/sample projects; the corresponding checklist items intentionally stay open. - The TRX-driven test verdict added for the intermittent at-exit host stall is proven by `CiTestVerdictTests`; the stall itself has not recurred since. - `nightly.yml` has not yet been run against a full-scope Azure run for tuple-level parity — see below. ## Operational follow-up This remains a preparation PR targeting `master`. It does **not** authorize a release. Release sequence: 1. Merge this PR. 2. Create canonical `release/2.0`, advance `master` to `2.1.0-preview.{height}`, and open a separate `2.0.0` version PR against `release/2.0`. 3. Follow [`docs/ReleaseProcess.md`](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/docs/ReleaseProcess.md) for candidate dry-run, environment approval, promotion, tag/release finalization, and next-patch preview setup. CI migration steps still needing administrator access are tracked in [`docs/DeveloperGuide.md#migration-status`](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/docs/DeveloperGuide.md#migration-status): proving `nightly.yml` parity on a trusted SHA with the private fuzz corpus provisioned, turning off definition 14's service-side PR override, retiring the Azure full-scope schedules and push trigger, and retiring the duplicate `azure-pipelines-preview.yml` publisher. > **Before cutting `release/2.0`:** the `Release` ruleset requires the Azure context for *every* `refs/heads/release/*`, but a new 2.x branch inherits `pr: none`, so that context would never report and would block every PR into the new line. Exclude the new ref from that ruleset, or add a 2.x ruleset requiring `build-and-test summary`, as part of creating the branch. --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> |
||
|
Marc SchierandCopilot
|
138086863b |
Fix audit findings and review regressions across the OPC UA stack (#4503)
## Scope Audit fixes and review follow-ups across the OPC UA Core, Client, Server, transport, and redundancy components. This includes implemented Tier 1, Tier 2, and Tier 3 work; it does not claim that every item in the 244-finding audit backlog is complete. The review batch fixes mandatory full-key record protection and owned plaintext buffers; transactional pending-certificate claims; shared-channel nonce and certificate ownership; discovery and token-policy validation; reconnect/callback ordering without changing unbounded buffering defaults; server permissions, filters, monitored-item lifetime, and alarm topology; historian paging, retention, and annotations; atomic user/password metadata; and canonical file identities with bounded alias and role refresh workers. ## Review-batch verification at `ab686ccae` The following results come from fresh test builds using `dotnet test`, `--no-restore`, `-m:1`, and an explicit `CustomTestTarget`. They replace the earlier description's library-build/`--no-build` totals, which did not establish the state of the merged test sources. | Scope | Target | Passed | Skipped | |---|---|---:|---:| | Core review regressions | net10.0 | 827 | 0 | | Client and lifecycle regressions | net10.0 | 630 | 0 | | Server review regressions | net10.0 | 1970 | 0 | | Redundancy Server / Client / PubSub | net10.0 | 276 / 10 / 11 | 0 | | Sessions / file-and-alias / streaming / HTTPS-WSS integration | net10.0 | 8 / 21 / 30 / 20 | 0 | | Core review regressions | net48 | 786 | 39 | | Client recovery and policy regressions | net48 | 54 | 0 | | Server ownership, aliases, queues, and user-state regressions | net48 | 137 | 0 | Every listed run exited successfully. The 39 net48 skips are existing platform/capability cases. New key-agreement tests assert unavailable-policy behavior on unsupported targets rather than adding skips or pretending those targets performed an unsupported handshake. Additional exact node/permission/filter/alarm checks passed 13 Core and 46 Server cases. The PR-scoped formatter verification covers 26 projects and 389 non-generated C# files. A Roslyn syntax audit covers 53 changed interfaces and 263 members, with no missing parameter/type-parameter/return contracts and a 120-column limit for their declarations and documentation. No full-solution, all-target-framework, or NativeAOT matrix is claimed from these local runs; the new CI run must establish its own result. ## CI follow-up at `e1bc3ac3b` The follow-up fixes the failures reported by Azure build 18400. TLS certificates retain configured DNS names, including mixed-case URI schemes, without relaxing hostname validation. Reverse-WSS fixtures use the DNS identity in the client certificate. Reconnect completion is published only after cycle cleanup, so a subsequent reconnect cannot join an old completed cycle and retain old certificates. Certificate-alarm roots now support event subscriptions. The pump sample has one continuous notifier path, with no duplicate shortcut. Historian and aggregate fixtures share the provider's retention clock. Alarm collection lets the server expire idle Publish requests rather than canceling only the client wait. The corrected server, WoT, and managed-session fixtures exercise valid requests and initialized lifecycle contracts. Downlevel crypto tests preserve real credential and key-ownership assertions without enabling unsupported raw-ECDH policies or adding skips. Fresh local evidence after integration includes: | Scope | Target | Passed | Skipped | |---|---|---:|---:| | Client lifecycle / channel ownership / builder fixtures | net10.0 and net48, each | 96 / 103 / 123 | 0 | | Combined server lifecycle, events, aggregates, crypto, and contracts | net48 | 358 | 0 | | Full lifecycle plus certificate-alarm and aggregate fixtures | net10.0 | 116 | 0 | | Final server-contract and history-routing delta | net10.0 and net48, each | 84 | 0 | | Subscription / XRegistry / WoT event regressions | net10.0 | 7 / 2 / 3 | 0 | | Full Sessions project | net10.0 | 780 | 379 | | Full ComplexTypes project / downlevel resolver fixtures | net10.0 / net48 | 2854 / 15 | 0 | | Full WoT project / downlevel structured-write fixtures | net10.0 / net48 | 1496 / 27 | 0 | | TLS domain-name regressions | net10.0 and net48, each | 21 | 0 | All listed runs exited 0. The Sessions skips come from existing endpoint and policy capability guards; they are not counted as passes. No skip was added. Worker validation also covered full DI (426 passes), full History (540 passes, 21 existing skips), 38 exact downlevel event/history failures, and 60 repeated certificate-rotation race cases. The final crypto fixtures passed 73 cases on each target with the unchanged leak detector; a temporary premature-disposal mutation made all three borrower tests fail and was then removed. All 29 C# files in this follow-up pass included-file formatting at info severity across 12 projects. The complete PR interface audit now covers 54 interfaces and 271 members with zero missing contracts or line-width findings. No full-solution or NativeAOT matrix is claimed. New-head CI has started and remains authoritative for the remote result. ## Test-result gate follow-up at `4b6e1203a` Azure build 18401 reported the net48 Client job as successful despite a test-host crash after only 264 passing tests. Its old evaluator counted per-test failures but ignored the failed run verdict. This was a partial run, not a benign exit after the suite had completed. The shared result evaluator now rejects aborted, errored, incomplete, malformed, or inconsistent TRX reports. Each file must describe a completed run, so a passing file cannot mask another file's failure. It retains support for completed runs followed by a benign nonzero host exit, existing skipped-result conventions, and no-TRX MTP jobs. All 78 regression cases pass after integration using Windows PowerShell 5.1. Real completed Sessions and Client reports also pass. The process-test fixture follows the existing NET10_0-only tools-test pattern; its net48 build is not counted as a test run. Formatting passes for the added test file. The Client host crash itself has not reproduced locally. The isolated fixture passed 59 tests, and two fresh full fast-CI selections each completed with 2,392 passes and three existing skips, including a run with crash collection enabled. The CI merge tree is identical to the tested source tree. No speculative runtime change was made, and the original partial CI run is not claimed as a full-suite pass. Azure build 18402 has now completed the full net48 Client run: 2,392 passed, three existing console skips, and a successful test-host exit. The corrected evaluator independently validated outcome=Completed. Across the matrix, 119 test-project jobs succeeded; Sessions net10.0 was canceled when the remote provider deprovisioned its runner, so the overall Azure pipeline was canceled. That cancellation is not claimed as a code failure or a successful run. CodeQL and three Docker image jobs were still running at the latest check. The migration discovery failure remains separate. ## Documentation follow-up at `57ed02bea` The environment blocker has been removed by the maintainer. The three bundled migration references now exactly match the canonical documents after deterministic link rewriting; all 15 bundled documents pass `validate-migration-plugin.ps1`. The canonical migration prose was not rewritten. `FileSystemClient.md` now documents that extended Windows roots preserve literal trailing-dot names, completing the filesystem review follow-up. The filesystem thread has been replied to and resolved after publication. The review pass now has 108 of 110 threads resolved. The two explanation-only threads remain open as requested; no unfinished change request is being represented as resolved. ## Cross-platform follow-up at `0d9d936a8` With migration discovery restored, the Linux and macOS test matrix exposed two stale durable-subscription expectations. The expected value 83,442 encoded the old 32-bit overflow when converting 3,600 hours to milliseconds. The existing overflow-safe server correctly returns 14,400,000 lifetime intervals at a 900 ms publishing interval. Only those two expected constants changed; production behavior, negotiated-hour checks, and all other assertions remain unchanged. The exact local lifetime selection reproduced both failures before the correction. The full durable suite now passes 36 cases with 11 existing skips on each of net10.0 and net48. All four lifetime cases executed and passed on both targets, and included-file info-severity formatting passes. ## Session shutdown follow-up at `8daf7f091` A late net48 Sessions failure exposed a shutdown race. Canceling an in-flight keepalive read can produce a transport `ServiceResultException`. The retiring worker treated its own cancellation as proof that the peer was unavailable, so `CloseAsync` or `DisposeAsync` could skip the server-side `CloseSession` request. The worker now ignores that service error only when its own token is canceled; genuine peer errors still update liveness. The full integration fixture reproduced the original error-code mismatch. Four deterministic mock-transport cases then produced three failures and one passing genuine-error control before the fix. They verify successful close, returned server error, asynchronous disposal, and unchanged behavior for a real keepalive failure. Fresh related runs pass 75 client and 101 integration cases on each of net48 and net10.0, with no skips. Ten repetitions of the formerly flaky full net48 fixture pass all 1,010 cases. All three changed files pass info-severity formatting, and temporary diagnostic instrumentation was removed. ## Atomic listener snapshot follow-up at `82dc22b5d` Ubuntu Core CI exposed an unsafe formatter rewrite in TCP listener disposal: `ConcurrentDictionary.ToArray()` had become a collection spread. A concurrent channel removal between the spread's count and copy operations could leave a default entry, causing a null dereference during disposal. The listener now uses the dictionary's atomic `Values` snapshot and disposes those channels outside the listener lock. Deterministic regressions close zero, one, or all channels at the copy boundary. Before the fix, the one/all-removal cases reproduced the exact null dereference; the unchanged-count control passed. All 23 admission/listener cases now pass on net10.0 and net48, and the full fast Core net10.0 suite passes 5,267 cases with 88 existing platform skips. Both changed files pass info-severity formatting. A bounded audit of the formatter's other snapshot rewrites found no further verified unsafe concurrent-registry copies. ## Shutdown-test observation follow-up at `58403ee8b` The net48 shutdown-order test raced its own observation: it read `m_publishWorkerTask` after starting shutdown, which can legitimately drain the worker and clear that field immediately. The test now captures the worker before shutdown starts and confirms it is initially running. It still requires the worker to finish while shutdown remains blocked by the held manager semaphore. Production behavior is unchanged. A temporary scheduling perturbation reproduced the exact null-field failure. The corrected capture passed 100 perturbed iterations; both the temporary repeat and yield were then removed. The final 126 subscription/lifecycle cases pass on each of net48 and net10.0, and formatting passes. A complete local Sessions net10.0 run also passed 780 cases with 379 existing capability skips in 7m20s after the earlier keepalive correction. ## Retirement and alias-startup follow-up at `1a048ff39` The shadow-generation test now observes its documented asynchronous cleanup through the existing bounded helper. Removing the replacement does not join a background drain that already claimed the retired generation. The test still checks live-registration removal immediately and requires the old node to disappear; no timeout or production lifecycle behavior changed. All 81 lifecycle cases pass on each of net10.0 and net48. Initial alias-node materialization now retries one timed-out, read-only snapshot query before creating nodes for that root. The retry is logged through source-generated logging. Each query retains the original shared 100 ms deadline; repeated timeouts, other errors, and cancellation still abort startup. Client searches and live refresh do not gain this immediate retry. Fault-injection tests cover success after one timeout, a persistent timeout, a different store failure, and cancellation. All 187 related alias/hosting cases and the original certificate-rotation integration case pass on each target; the positive test records exactly one retry warning. The alias documentation now accurately describes the shared request budget. All changed C# files pass formatter checks, and the complete 54-interface/271-member documentation audit passes. Fresh CI is running on `1a048ff39`; the previous completed matrix exposed only the two failures corrected in this batch and their dependent summaries. ## Sample endpoint allocation follow-up at `d08a94925` The strong-cluster test harness allocated OPC UA and Raft endpoints in separate released-port batches. Uniqueness was enforced only inside each batch, so the same OS-assigned port could appear in both protocol groups. The harness now requests one distinct batch for both groups and splits it before launching any replicas. Deterministic one- and three-replica allocation cases reproduced overlapping groups with the old plan and pass with the corrected plan. The complete 17-case short-haul sample suite passes without skips; all failover, historian-continuation, and service assertions remain unchanged. Both changed files pass info-severity formatting. This prevents intra-cluster self-collisions but does not claim to reserve released ports against unrelated external processes. ## Code-review follow-up at `085fd9636` A review of the complete branch diff found five defects this PR introduced. All are now fixed, each in its own commit. `RelatedTo`'s recursive-to-iterative rewrite defaulted the optional `includeTypeDefinitionSubtypes` and `includeReferenceTypeSubtypes` operands to `false`. Part 4 §7.7.4 specifies both default to `true`, so a filter that omitted them silently stopped matching subtypes. Dynamically created key-credential nodes derived their namespace from the containing folder, which is the standard `i=18155` node in namespace 0 — the namespace reserved for the OPC UA standard address space; they now resolve the server-owned namespace and fail with `BadInternalError` rather than guessing index 1. The retained audience-less `CreateTokenData` overload produced tokens that verification always rejects, so it is now `[Obsolete]` and throws instead. A failed reference to a remote parent left the new node registered with a one-sided inverse reference, and is now rolled back. An existing `.pfx` is restricted before it is rewritten, rather than after its new key bytes are on disk. One further reported finding was rejected after investigation. `JwksIssuerKeyResolver` deliberately does not dispose retired key snapshots: `RetiredKeysAreCollectibleWhileResolverRemainsAliveAsync` proves the resolver retains no retired snapshot, and `ReturnedKeysSurviveRepeatedRefreshAndResolverDisposalAsync` requires already-returned keys to keep verifying across refreshes and resolver disposal. Adding disposal would have broken that tested contract, so the exploratory change was reverted. Validated on net10.0 and net48: 256 Core and 633 Server cases pass, with one pre-existing net48 skip. The build reports zero compiler and analyzer diagnostics, all eight included-file info-format checks pass, the contract audit covers 58 interfaces and 310 members with no findings, and the 15 bundled migration documents validate. All 122 review threads are resolved; the key-credential namespace thread was reopened to retract an earlier incorrect answer and then resolved against the real fix. ## Review-body follow-up at `da0d59cfb` The re-review at `8daf7f091 + 82dc22b5d` listed four items in its summary that had no inline thread of their own, so they had not been picked up. All four are now fixed. Both event-dispatch permission gates had been tightened from not-Bad to `IsGood`, which silently dropped events whose permission verdict is Uncertain; the data-change path in the same file still treated Uncertain as deliverable, so the two had diverged. `EventSourceRegistry` only cleared `ConsecutiveFailures` when a source was deactivated, so a source that had backed off to the 30-second cap kept that delay for the rest of the process even after starting successfully. `UserNameIdentityTokenHandler` built a temporary issuer chain whose `Add` calls take independent certificate references, and never released them. The fourth was the receiver-side sequence rollover, which the reviewer had correctly re-flagged as still open despite its thread being resolved. `m_sequenceRollover` was set permanently, so the second legal wrap on a long-lived channel was rejected and faulted the channel along with the transport-reconnect OPN. OPC 10000-6 does not cap a SecureChannel at one wrap. The flag is now cleared once an accepted number has traversed the normal range, which keeps rejecting a replayed low number that never left the rollover window. Validated on net10.0 and net48: 636 / 593 Core cases and 135 / 134 Server cases pass, including the new `UaSCBinaryChannelSequenceRolloverTests` and the Uncertain-permission delivery cases. The build reports zero compiler and analyzer diagnostics and the changed files pass included-file info-format checks on both targets. The previous head `085fd9636` was fully green: 279 checks passed and Azure build 18414 succeeded with 137 of 137 jobs, after the platform automatically retried a job whose NuGet tool download had hit a transient TLS error. ## Reported issues #4508 and #4509 at `6ad1ecf16` **#4508 (V2 subscription recreate deadlocks against a faulted transport) was not yet fixed and is now closed by `bac260a06`.** Both drain sites the report identifies were still unbounded. A publish worker that started its request before the quiesce parks inside the service call on the channel ready gate while holding an active publish request; the recreate that would make the channel ready is the very operation awaiting the drain, so the wait never ends. The drain now cancels in-flight attempts through a per-attempt token linked to the worker token. An aborted worker rolls its acknowledgements back, releases the request and returns to the paused park instead of terminating, so it resumes when the caller leaves the quiesce. Publish ingress stays quiesced for the caller's operation, so `DropPendingForSubscription` keeps the guarantee the report asks to preserve and no blind timeout is added. The new regression parks a worker on a cancellable gate that never completes on its own: it reproduces the hang for the full 30-second budget before the change and passes after it. **#4509 (identity provider fails the initial connect) was already fixed for its primary symptom.** `HandleConnectAsync` materializes the provider's identity through `CreateIdentitySelectionContext` before the first `CreateAsync`, so Anonymous is no longer substituted against a UserName-only endpoint. `6ad1ecf16` adds the two remaining details from the report: the enabled-policy set now includes the negotiated channel policy, so a client cannot reject the policy it already accepted as `NotEnabledByClient`; and when discovery has not populated the endpoint's token policies the connect keeps the configured identity so the attempt fails at the transport layer for the reconnect policy to retry, instead of failing at the identity layer. Validated on net10.0 and net48: 477 / 473 client session and subscription cases pass, the build is free of compiler and analyzer diagnostics, and the changed files pass included-file info-format checks on both targets. The full `Opc.Ua.Sessions.Tests` suite also passes locally on net10.0 with 782 passed and 379 pre-existing capability skips. GitHub checks are green on this head: Build and Test .NET 10.0, Docker Sample Images CI, and NuGet Publish all succeeded. Azure build 18417 completed 134 of its 135 jobs successfully; its net10.0 Sessions job was cancelled by the hosted-agent provider ("Received request to deprovision"), the same infrastructure flake that hit build 18409 earlier on a head predating these changes. That job is not claimed as passing, and the local full-suite run above covers the same project. The previous head `da0d59cfb` completed Azure build 18415 with every job succeeded. ## Review follow-up at `a08f0c12b` Eleven new review threads arrived, nearly all documentation. Before acting on each "this is new in 2.0" remark I checked the API against `master378` rather than assuming, because a genuine 1.5 API still needs its migration entry. `ManagedSession`'s server-redundancy refresh had two hard-coded two-second bounds. The new `ServerRedundancyOptions` exposes `RefreshTimeout` for the metadata read and `PeerDiscoveryTimeout` for each peer lookup, defaults both to the previous two seconds, validates that a bound is positive or `Timeout.InfiniteTimeSpan`, and flows through `ManagedSessionOptions`, `ManagedSession.CreateAsync`, the fluent builder, and `DefaultServerRedundancyHandler`. `AsyncCustomNodeManager` and `KeyCredentialBridgeAuthenticator` are both absent from `master378`, so their obsolete members had no compatibility constraint and are removed rather than carried: `AsyncCustomNodeManager.ReadInitialValue`, the audience-less `CreateTokenData` that only threw, and the audience-less `CreateProof` that produced a proof no server accepts. `CustomNodeManager2.ReadInitialValue` and the `MonitoredNode2` migration section are genuine 1.5 APIs and are untouched. The documentation now states master's behaviour instead of the delta from 1.5.378, with each moved item already covered under `docs/migrate/2.0.x/`. `IHistorianAnnotationProvider` is described by its annotation-time key rather than as "legacy" — it is new in 2.0, so the label implied a compatibility constraint that does not exist. The KeyCredential bridge payload's `"version"` field is explained as a vendor-extension format version with no OPC UA counterpart, and its version-1 upgrade narrative is gone. `CertificateManager.md`'s migration walkthrough is replaced by the resolver API it targets. `NodeManagers.md`, `Sessions.md`, `AsyncServerSupport.md`, `Certificates.md`, and `Profiles.md` no longer announce features relative to 1.5. `Benchmarks.md`, `WhatsNewIn2.0.md`, `DeveloperGuide.md`, and `docs/migrate/` keep their 1.5.378 references, which are their subject. CI then exposed a defect this batch introduced: five fixtures resolve `ManagedSession`'s private constructor by its exact positional parameter types, so the added parameter compiled cleanly but made `GetConstructor` return `null` at run time. `7df1d5c40` updates every site; `Redundancy.Client` passes 126 of 126 and the affected client fixtures 112 of 112. The BouncyCastle question is answered and tracked in #4510: `Opc.Ua.Core` already project-references `Opc.Ua.Security.Certificates`, which already carries the conditional BouncyCastle reference, so the raw-ECDH polyfill belongs there rather than as a new Core dependency. The `IUserDatabase` metadata question is addressed to another maintainer and is deliberately left open. ## Reported issue #4509 follow-up at `5d4d13d63` The reporter verified `6ad1ecf16` and found one case still failing, correctly. The empty-token-policies branch kept the configured identity so a server that is down surfaces a transport failure rather than an identity error. A provider-only configuration has no configured identity, so it kept `null`: `OpenCoreAsync` turned that into Anonymous and `OpenValidateIdentity` rejected it with `BadIdentityTokenInvalid`, silently ignoring the configured provider. The branch now requires a configured identity to defer to. Without one the provider is consulted, so its own `BadIdentityTokenRejected` names the offered and rejected policy set instead. Two regression tests cover both configurations; the first fails against the previous behaviour by reaching the session factory with a `null` identity, which is the exact step that produced the reported error. That branch had shipped without a test, which is why the case was missed. ## Review and CI status at `5d4d13d63` **All actionable review threads are resolved: 132 of 133.** The single open thread is a question the reviewer addressed to another maintainer about moving user metadata into `IUserDatabase`; it is left open for them rather than resolved silently. Every resolved thread carries a reply naming its fix commit. **Latest-head CI is green on `5d4d13d63`: 279 checks passed, zero failed, zero pending**, with one conditional signed-package publishing job skipped. Local validation for this batch: the full `Opc.Ua.Client.Tests` suite passes 2,430 of 2,430 on net10.0, `Opc.Ua.Redundancy.Client.Tests` 126 of 126, and `Opc.Ua.Server.Tests` 590 of 590 for the affected selections. Both net10.0 and net48 build clean for `Opc.Ua.Client`, `Opc.Ua.Server`, and the changed test projects, and all changed files pass included-file info-severity formatting on both targets. The PR remains open; no merge was performed. As stated at the top, this does not claim that every item in the 244-finding audit backlog is complete. ## Latest CI follow-up at `b6f2dfa43` Azure build 18442 exposed a net48 compile regression in the Kafka adapter: `RuntimeHelpers.GetUninitializedObject` is unavailable on .NET Framework. Commit `2cf1e0eed` now uses normal `ProducerMessage` construction on `NETFRAMEWORK` while retaining reflection-based construction for modern targets that can load a different Dekaf runtime asset. The exact net48 CI selection passes 92 of 92 tests, and the netstandard2.1 adapter guard selection passes 11 of 11. After merging the latest `master`, GitHub Actions run 35643592139 is green. The initial Ubuntu Core.Schema job built successfully and passed 160 of 160 tests, then hit an external 403 while finalizing its test-result artifact; its failed-job rerun completed successfully. Azure build 18451 was canceled when its remote provider deprovisioned the net10.0 Sessions runner (`Received request to deprovision: The request was cancelled by the remote provider`). Azure attached no test task log or source error, and the canceled parent left that child check stale as in-progress. No source workaround, test skip, or check relaxation was made for either infrastructure failure. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 68f26810-22b8-476f-abcd-0a8c82d6e208 |
||
|
Marc SchierandCopilot
|
90fc3cca2f |
Complete WoT Binding 1.1 alignment and conformance coverage (#4411)
# Description Completes the UA-.NETStandard implementation of the final OPC UA WoT Binding 1.1 semantics and aligns the WoT Connectivity 1.1 model with xRegistry 0.4.0. The change: - retires `uav:isEvent` and `uav:severity` from current Binding output while preserving unknown members through the generic converter's ordinary residue mechanism; - removes `uav:severity` from the asset Thing Description parser and materializer, so document metadata cannot set occurrence severity; - implements logical EventType definitions, ordered resolution providers, explicit/default selection precedence, materialized member paths, and the built-in `BaseEventType` fallback; - adds deterministic portable identity and browse-path anchoring; - populates bound type declarations instead of creating duplicate siblings, including inherited and closed-content validation; - resolves and validates `uav:externalSchema` without overriding authoritative DataType definitions; - hardens DataType dimensions, native projections, preservation envelopes, URI transmission, projection limits, and delete-policy handling; - synchronizes the xRegistry and WoT Connectivity source-generation inputs; - pins all 27 specification examples, all 97 implementation-owned normative requirements, and all 12 cross-language vector groups to the final specification revision; - adds `samples/WotCon/run-aggregation-demo.ps1` for a verified one-command aggregation client/server round trip and replaces the README topology sketch with Mermaid; - normalizes redirected PowerShell diagnostic presentation in the statement-digest test harness so its semantic assertions are stable on Windows, Linux, and macOS. WoT Connectivity remains model version `1.1`; draft iterations are identified by release labels such as `1.1-draft5`. **Specification dependency:** the evidence assets pin `OPCF-Members/spec-drafts` `main` commit `25db507121b503fd31c8816fbfae67a6822e813b`. The corresponding specification follow-up is `OPCF-Members/spec-drafts#35`. ## Validation - Latest GitHub/Azure CI: 266/266 checks passed. - .NET 10 feedback-focused tests: - WoT Connectivity event/model/traceability: 72 passed - binding capability/traceability: 8 passed - coverage-gate tests: 25 passed - Complete .NET 10 Tools project: 752 passed, 1 intentionally skipped. - .NET Framework 4.8: - WoT Connectivity event/model/traceability: 72 passed - WotCon bindings build: 0 warnings, 0 errors - NativeAOT build: 0 warnings, 0 errors. - WoT aggregation demo: 16 documents uploaded and all 10 mapped values read with Good status; exact processes and isolated PKI stores cleaned up. - Specification traceability: 402 normative statements mapped; all 97 implementation-owned statements have passing test evidence with zero gaps. - Shared conformance vectors: 12/12 groups execute. - Final CI coverage: - project line coverage: 86.83% (floor 70%) - project branch coverage: 76.71% (floor 60%) - patch coverage: 100.00% (2,591/2,591 changed lines) - coverage gate passed ## Related Issues Follow-up to #4375 and alignment with `OPCF-Members/spec-drafts#31` and `OPCF-Members/spec-drafts#35`. No separate tracking issue was opened for this follow-up. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce new build or analyzer warnings in the changed paths. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed all PR feedback currently received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6a9f41b1-72f0-4c74-bd7f-26fa531e5738 |
||
|
Marc SchierandCopilot
|
c84c168585 |
Merge OpenUSD scene packages (#4384)
# Description Merge the draft OpenUSD Scene Materialization projects into the existing OpenUSD core and server packages: - move the scene companion model, document/value model, and `.usda` conversion APIs into `Opc.Ua.OpenUsd`; - move discovery, materialization, export, and Part 1 interop into `Opc.Ua.OpenUsd.Server`; - rename the CLR APIs to `Opc.Ua.OpenUsd.Scene`, `Opc.Ua.OpenUsd.Scene.Conversion`, and `Opc.Ua.OpenUsd.Server.Scene`; - remove the `Opc.Ua.OpenUsdScene` and `Opc.Ua.OpenUsdScene.Server` projects and NuGet package IDs; - update the connector, the consolidated OpenUSD test project, package READMEs, and developer docs. This is an intentional source- and binary-breaking cleanup for the draft Scene packages. The OPC UA Scene model URI, version, NodeIds, and wire behavior remain unchanged. ## Related Issues - Fixes #4383 ## Validation - `UA.slnx` Release builds completed for `CustomTestTarget=net10.0` and `CustomTestTarget=net48`. - OpenUSD tests passed on net10.0 (915) and net48 (879), excluding the existing host-culture assertion; the isolated assertion passed under invariant globalization. - The generated Release pack contains the expected 75 packages, emits neither retired Scene package, and passes `.azurepipelines\validate-source-generator-packages.ps1`. - The two surviving packages expose the new nested namespaces and contain no retired package dependencies. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
Marc SchierandCopilot
|
cef83550cb |
OPC UA Vision: a vision-guided bin-picking demo an LLM agent can see and drive (#4235)
# Description Implements **OPC UA — Vision** and uses it to build the demo the companion specifications were drafted for: an eye-in-hand camera watches a bin of parts, a language model *looks at the frame*, decides which part matches the instruction, and commands the robot through Robot Intent — with the OpenUSD viewer showing the arm move and the bin empty out. > **Draft, and stacked.** [#4195](https://github.com/OPCFoundation/UA-.NETStandard/pull/4195) has merged and master is merged in, so what is left on top of `master` is the Vision work plus the unmerged `marcschier/ai-model-management` branch, which the demo needs for the model/dataset/deployment provenance. Review the Vision commits; the AI Model Management work lands with its own PR. ## What is here **The specification, implemented.** `Opc.Ua.Vision` is generated from the spec's own `Opc.Ua.Vision.NodeSet2.xml`, vendored byte-identically (SHA-256 `79C592C6…`, verified against a fresh download and stored LF-preserved like the Robot Intent NodeSet). `Opc.Ua.Vision.Server` builds the §4.2 discovery object, sensors, the coordinate-frame tree, calibrations, media endpoints, inference pipelines and the §9 feedback methods, with rendering and inference behind injectable providers. `Opc.Ua.Vision.Client` is built on the generated ObjectType proxies and composes transforms across the frame tree, which is what turns a detection in camera coordinates into a pose a robot can act on. **Two perception paths behind one contract**, which is exactly what `InferenceLocationEnum` is for. `OnServer` is a deterministic detector derived from the stage's ground truth — no model, no network, no GPU, so CI can exercise the whole loop. `EdgeOffServer` is the agent: it sees the frame over MCP and calls `SubmitDetections`, and the Server publishes results it did not compute. A client reads `DetectionResultType` identically either way. **The agent's eyes.** `tools/Opc.Ua.Mcp.Vision` adds 22 `vision_*` tools, which with the four shared connection tools makes the `vision` profile 26; `vision_get_frame` returns the camera image as an MCP `ImageContentBlock`, so the model looks at pixels rather than reading a description of them. Tool profiles now compose — `--profile vision,robotics` yields 62 tools — so one agent can both see and act without dragging in the full 136-tool catalogue. **The cell.** `samples/Robotics/BinPickingCell` follows the spec's own Robotics-Vision Addendum: the `world → robot_base → flange → gripper_tcp` frame tree with `camera_eih` on the flange, the `HandEye` extrinsics and the `Intrinsics612x512` intrinsics — the addendum's 2448×2048 calibration scaled to the 4×4-binned grid the camera actually delivers. `samples/Robotics/BinPickingClient` hosts the composed MCP catalogue and optionally the viewer. **Documentation.** `docs/Vision.md` (~800 lines) covers the model, hosting, the build context, every topology builder, both perception paths, feedback, the MCP profile and the limitations; linked from `docs/README.md`, with the profile table in `docs/McpServer.md` and READMEs for both samples. ## Evidence, not assertions Each of these was measured on a running system rather than assumed: | Claim | Evidence | |---|---| | The camera sees the bin | 612×512 frame with the bin and all five parts in view — red `(212,47,47)`, green `(61,193,71)`, blue `(54,85,218)`, yellow `(215,196,58)`, orange `(209,119,55)` | | An agent can act on what it sees | Over MCP, `vision_get_frame` returns a 612×512 PNG that decodes from base64 to 1,254,051 bytes, the descriptor beside the detections says 612×512, and all five `BoundingBox2D` centres land **inside** the frame on the pixel colour their own detection claims | | The geometry is right | A detection composed `camera_eih → world` lands on the authored position, residual **0.0000 m** — on both the on-server and off-server paths | | The detector tracks the world | Placing a part drops the next scan from 5 detections to 4, with that part absent | | The robot obeys the agent | Scripted run took command authority and completed a pick and a place, both `Succeeded` | | Tool catalogues are honest | Counted from a running server: vision 26, robotics 40, `vision,robotics` 62, full 136, **zero duplicates** | | Every node is browsable by NodeId | Eight representative NodeIds — both folders, a pipeline, a frame, the sensor, its Calibrations folder, the HandEye calibration, the Media folder — all `Good` | That tool-count check matters because the profile tests collapse names into a `HashSet` and *cannot see* a duplicate registration — so the new tests count registrations, not names. ## Things that were wrong, and are not now Writing the tests found three defects of one kind — code that answered instead of refusing. A zero-norm quaternion was silently rewritten to identity, so a caller with an all-zero orientation got a confident wrong grasp pose. `SubmitCorrection` rewrote a null `ResultId` to empty and called the sink anyway. A bad `StageIdentifier` **killed the host process** inside native `UsdStage.Open`, via an exception .NET cannot catch. All three now refuse loudly and are pinned by tests. **Nodes that existed but could not be reached.** The node manager indexes the Vision root before configurators run, so everything the fluent builder grafted on afterwards — both folders, the pipelines with their Results and Feedback, the frames, the sensors with their optics, calibrations and media endpoints — was never added to the index. Browsing *forward from a parent* worked, because that walks `NodeState.Children` in memory; browsing any of those nodes *by its own NodeId* returned `BadNodeIdUnknown`, which is how an ordinary client and the MCP discovery tools navigate. Registration is now deferred and flushed after each configurator. The first fix covered only the DI path and left the documented non-DI fallback broken in exactly the same way, so `ConfigureVisionAsync` now exists and registers what it builds. That defect survived a green suite because every test browsed children. The new tests assert by NodeId, and were checked against the broken code — with the flush removed, they fail. **Every Method was uncallable.** The Server passed 426 unit tests while not one of its Methods could be invoked over a real session — `RunInference` returned `Bad_TooManyArguments`, and so did every other Method that takes arguments, which is all of them but `StartContinuous` and `Stop`. Three independent faults, each sufficient on its own: the Method nodes were never created (they are Optional children, and the dispatcher guards every attachment with a null check, so it silently attached nothing); nothing declared the arguments, so the stack concluded none were expected; and `MethodDeclarationId` pointed at a synthesised NodeId rather than the NodeSet declaration a client calls with, so the lookup missed the instance Method entirely. The builder now creates the Methods, the `Results` folder and the `Feedback` object when a provider or sink is configured, declares every signature, and names the right declaration. Unit tests could not have caught this: they invoke the handler delegates directly and never go through `MethodState.Call`. It took an end-to-end test over a real session. Relatedly, children created by the generated `CreateOrReplace` helpers carry no `ReferenceTypeId`, so nothing could reach them — fixed once in the build context rather than at the several dozen call sites. **And one where I was wrong.** Earlier in this branch I reversed my own agents' decision and made `SubmitDetections` accept an empty detection set, reasoning that "I looked and the bin is empty" is a correct observation and refusing it forces a correct agent to invent a detection. The reasoning is sound; it is also an argument *against* the specification rather than an implementation of it. §9.5 is normative and explicit — its StatusCode table lists `Bad_InvalidArgument` for "`Detections` empty; or `SubmitCorrection` supplies both or neither corrected array". Worse, **none of §9.5's argument rules were enforced anywhere** — the dispatcher forwarded every submission straight to the feedback sink, so conformance depended on whichever sink a host installed. They are Server obligations, so they now live in the dispatcher, ahead of the sink, proven by strict mocks that would throw if the sink were consulted. The usability problem is real and does not go away by conforming, so it is raised where it can be fixed: [opcua-drafts#70](https://github.com/marcschier/opcua-drafts/issues/70). ## Tracking the updated drafts The Vision, Robot Intent and AI Model Management NodeSets on `opcua-drafts` `main` have moved, partly because of feedback this branch raised. All three are re-vendored byte-identically — verified by comparing git blob hashes against the upstream files rather than by eye — and the implementations follow. **Two statements that were previously inexpressible now have a wire representation.** `SubmitDetections` takes `SceneIsEmpty` and `SubmitCorrection` takes `RetractAll`, so an agent can report "I examined this frame and there is nothing in it" — the terminating condition of the bin-picking task — and can retract a false positive by correcting a result down to nothing. Both are checked in both directions, because the flag is exactly what separates a deliberate empty observation from a lost payload: an empty array without the flag is refused, and the flag with an array attached is refused as two contradictory claims about one frame. Implemented in the dispatcher, the client, the MCP tools the agent drives, and the sample cell. Also absorbed: `LampType` and `LightingMode` became enums (§66), `ProfileName` became `DefaultProfileName` on the endpoint (distinct from the `ProfileName` a caller passes to `GetStreamEndpoint`, which keeps its name), AI `Invoke`/`InvokeAsync` gained `PayloadUri` under the same exactly-one rule, and AI `ListModels` gained a `ContinuationPoint` — because `MaxResults` alone puts everything past the bound permanently out of reach. **OpenUSD is on `0.9.0-alpha`**, verified on the running cell rather than by restore alone: D3D12 backend, 612×512 frame, the bin and all five parts in view and detected — the same result `0.8.0-alpha` produced. The one local workaround that survives is `VisionMethodArguments`, and its reason has changed rather than gone away: the builder materialises these Methods as Optional children through `CreateOrReplace`, which constructs the state object directly and never runs the generated factory that now carries the signatures. Checked rather than assumed — removing the declarations fails five of the eight `VisionMethodSurfaceTests`. ## Review feedback Three threads, all addressed in `0fe2803a3`: - **The AI Model Management work is now a library family, not a sample.** `src/Opc.Ua.AI` (model), `.Inference` (`IInferenceBackend` and backends), `.Server` (node manager) and `.Client` (discovery, reads, calls, artefact transfer), in the same shape as the Robotics and Vision families; the samples become `samples/AI/ModelManagementServer` and `ModelManagementClient`. The client library needed real work rather than relocation: its browsing surface was `private` inside the sample's scenario runner, so moving the file alone would have produced a library nobody could call. - **No cloud-vendor SDK is referenced anywhere.** Inference goes through `Microsoft.Extensions.AI` — an `IChatClient` is implemented by hosted services and on-device runtimes alike, which is exactly the property clause 8.1 asserts. `Azure.Identity` is gone: workload identity is not an Azure feature, every platform projects a token into a file and each SDK reads that file behind its own API, so the resolver reads it directly. The comment claiming `Microsoft.Extensions.AI` would force a `System.Text.Json` bump was stale — it pins 10.0.10, which this repository already centralises on. - **`BinPickingCell` and `BinPickingClient` moved into `samples/Robotics/`**, which is where they belonged: the cell was already linking the arm, its kinematics and both USD assets out of `IntentEnabledRobot`. ## Upstream Work that belonged elsewhere went elsewhere rather than being worked around locally: - **`openusd-dotnet` [#13](https://github.com/marcschier/openusd-dotnet/pull/13)** — merged and shipped in `0.8.0-alpha`. `SilkFrameCapture.Capture` rendered only the first capture of a session and silently returned a blank frame for every one after it. Also filed [#14](https://github.com/marcschier/openusd-dotnet/issues/14) (no camera-by-prim API) and [#15](https://github.com/marcschier/openusd-dotnet/issues/15) (skipped tests reported as passed — which fooled me for several runs). Both are fixed; this branch takes `0.8.0-alpha` and deletes two workarounds, including 143 lines of hand-rolled projection maths. - **`opcua-drafts` [#66](https://github.com/marcschier/opcua-drafts/issues/66)–[#69](https://github.com/marcschier/opcua-drafts/issues/69)** — a close read of the spec produced 22 candidate ambiguities; 12 were discarded as misreadings with clause citations and the remaining 10 filed. One is concrete: five members exist in the NodeId CSV that no clause documents. - **`opcua-drafts` [#70](https://github.com/marcschier/opcua-drafts/issues/70)** — §9.5 could not express an empty observation or a false-positive retraction. **Adopted**, with the explicit-flag resolution argued for: `SubmitDetections` gained `SceneIsEmpty`, `SubmitCorrection` gained `RetractAll`, and the corrected-array rule relaxed from *exactly one* to *at most one*. - **`opcua-drafts` [#71](https://github.com/marcschier/opcua-drafts/issues/71)** — the Vision NodeSet browse-named all 13 `InputArguments` / `OutputArguments` Properties in the Vision namespace rather than the base namespace, so a stack could not find them and every Method in the specification was uncallable. **Fixed**; this branch now tracks the corrected artefact. ## The defect underneath it all The end-to-end suite was removed earlier in this branch because nine of its fourteen tests failed on one symptom: a client could not open a pipeline's `Feedback` object or enumerate its `Results`, even though browsing the same parent listed both. Restoring it meant explaining that, and the explanation was not in Vision. `RelativePathElement` initialised **`IsInverse` to `true`**. A caller that sets a `ReferenceTypeId` and a `TargetName` — which reads as *"the child reached by this Reference"* — was therefore asking the Server for the **inverse** Reference, and got `Bad_NoMatch` with nothing in the request to suggest why. `ObjectTypeClient.ResolveChildNodeIdAsync` is such a caller, and it is what **every source-generated Optional-child accessor in the stack** is built on — the accessor whose own documentation cites `AlarmConditionTypeClient.GetShelvingStateAsync` as an example. Vision is merely where it surfaced, because Vision puts two Optional children on the path an off-Server agent must take. The sibling helper in `StateMachineTypeClientExtensions` already set `IsInverse = false` by hand, which is the shape of a trap rather than a default. The default is now forward, `ResolveChildNodeIdAsync` says so explicitly, and the existing test that had recorded `true` as *expected* now pins `false` with the reasoning. Of the forty `RelativePathElement` construction sites in the tree, that accessor was **the only one relying on the default at all** — every deliberate caller already set it. Two further defects were found underneath it, both real and both fixed: - The generated `CreateOrReplace` helpers construct a child state object directly and leave **`TypeDefinitionId`** unset as well as `ReferenceTypeId`. A child referenced by nothing cannot be browsed; one with no type definition is a malformed Object that a client filtering by type silently skips — which is exactly what `EnumerateResultsAsync` does. Normalisation moved to `VisionNodeManager`, so it also covers the case the builder cannot see: a result published **at runtime** by an inference provider, long after the address space was built. That path produced results a client could list but not read. - `Opc.Ua.AI` shipped without an `AssemblyInfo`, so the solution built with six `CA1014` warnings. I had initially "fixed" this in the Vision client with Browse-based fallbacks, on the strength of a probe that appeared to show `TranslateBrowsePathsToNodeIds` failing even for `Server → ServerStatus`. That probe was wrong: it used an unqualified `ReferenceTypeIds`, which inside a `Opc.Ua.Vision.*` namespace binds to *Vision's* `ReferenceTypeIds` and yields a nonsense NodeId. Chasing that to the end is what turned up the real cause. **The fallbacks are gone** — they would have hidden a stack-wide bug behind a Vision-shaped workaround, and one of them matched on browse name while ignoring the namespace index. `tests/Opc.Ua.Vision.Intent.Tests` is restored and back in `UA.slnx`, passing **14/14** on its own merit rather than through a client-side workaround. A registration test now pins the pipeline's `Feedback` and `Results` children, which nothing covered before. ## Running it, and what that found The demo's premise is that a language model **looks at the bin and reasons about what it sees**. Running it end to end showed it could not, for reasons no test covered, because every one of them lived between components. **The sample did not compile.** Two `CS8604` nullable errors with `TreatWarningsAsErrors`, so neither the cell server nor the demo could be started at all. Worth recording that my first explanation was wrong: CI *does* build this project - the Linux leg's `net10.0` pass compiles it, log-confirmed - so this is not a coverage gap in the workflow, and no workflow change was made. **`Pick` and `Place` never moved the arm.** They were a delay plus a gripper action, so the operations reported `Succeeded` while nothing travelled. They now solve IK once and interpolate in joint space. **And the parts never moved**: the arm swung but the bin stayed full, because nothing tied the held object to the world state. The connector now creates 11 monitored items where it created 6, and a scripted run delivers `SeqNo=151` across 10 messages where it delivered `SeqNo=3` across 1. **The frame was smuggled through a string field.** The media provider put a base64 *data URI* of the PNG into `VisionImageReferenceDataType.Uri` - a field meant to be a *reference*, while the bytes were already returned in the inline `ByteString`. Every response therefore shipped the image twice, and a 1,311,400-byte PNG became a 1,748,536-character string against a `MaxStringLength` of 65,535: **26.7x over**. That is why raising `MaxByteStringLength`, `MaxArrayLength` and `MaxMessageSize` never helped - none of them govern a `String`. Separately the MCP tool assigned raw bytes where `ImageContentBlock` requires base64, so the wire carried `"data":"�PNG..."` - every non-UTF-8 byte replaced by U+FFFD, destroying the image and inflating the response to 6.8 MB. **The camera was aimed at nothing.** The arm's home pose, the flange scan pose the Vision model declared and the joint angles in the USD stage were three independent claims about where the camera was, and they disagreed - the declared flange orientation pointed the view 86 degrees away from the projection camera. Solving them as one thing exposed two further constraints that are easy to miss: the camera has to sit off the tool axis or it photographs its own gripper, and the IK branch has to be elbow-back or a link parks under the camera and fills the frame with the arm's own upper arm. A third was a regression this branch introduced and `Opc.Ua.Robotics.Tests` caught - aiming a straight-down camera from a point on the base's own X-Z plane puts the wrist on the J4/J6 singularity, so every motion away from home fails to solve. A 15-degree camera roll, free because the camera still looks straight down, gets 25 degrees clear, which is the margin the poses before this branch had. **The frame and the detections were different images.** The sensor declared 2448x2048, the clip endpoint 1280x1024, and the renderer produced 640x512 - not even the same aspect ratio. The ground-truth detector projects through the declared intrinsics, so its boxes lived in 2448x2048 space while an agent was handed a 640x512 picture: *"pick the red cube you can see"* arrived with coordinates that pointed off the image. Everything now agrees at 612x512, an exact 4x4 bin of the native sensor with the intrinsics scaled to match; the native size stays in the model and serial number, where it describes the hardware rather than the image. **`LatestClip` was created and never written**, so it reported `Bad_NoDataAvailable` for the life of the Server and a consumer following the model - read the published frame, call the method only if there is none - never got a frame. The dispatcher now publishes a clip it has just encoded, for every Vision server rather than this sample alone. One genuine API gap turned up on the way: `OpcUaServerOptions` exposes `MaxByteStringLength`, `MaxArrayLength` and `MaxMessageSize` and **no `MaxStringLength`**, so a Server that legitimately needs a long string could not raise the limit through the hosting API at all. Added, with tests. It is not the fix for anything above - that was a design bug - but the missing knob was real. ## Not finished - **Two projects sit below the repository's 80% line coverage.** `Opc.Ua.Vision.OpenUsd` is at 71.1% — what is left only executes with a real OpenUSD stage and plugin tree, which is not on every build agent. `Opc.Ua.AI.Inference` is now at **86.7%**, up from 12.9%: the code this branch *wrote* is covered by 25 tests, and `RestChatCompletionsBackend` — previously the whole of the shortfall — is covered by 18 more against a stubbed `HttpMessageHandler`, exercising success, refusal, throttling, retry-after, timeout and cancellation without a network. Everything else clears the bar — `Opc.Ua.Vision.Server` 92.1%, `Opc.Ua.AI` 91.4%, `.Vision.Client` 88.1%, `Opc.Ua.Vision` 85.7%, `Opc.Ua.AI.Server` 80.1%, all measured with coverlet. The CI coverage gate reports itself as advisory and non-blocking. - **`SamplesCollected` is not counted here, and `docs/Vision.md` now says so.** Section 9.4 requires a Server to count a negative example (`SceneIsEmpty` / `RetractAll` carrying a `GroundTruthLabel`) exactly as it counts one carrying geometry. `VisionNodeManager` does not, and structurally cannot: `SamplesCollected` is a property of `LearningJobType`, which the *AI Model Management* companion defines, and Vision reaches the job through a `NodeId` value rather than a Reference **precisely so this model takes no dependency on the model that defines it**. What the Server does guarantee is that the negative example survives the hop intact — both flags are carried verbatim to `IVisionFeedbackSink` — so a host that binds a learning job has everything it needs to satisfy the clause on the counter it owns. Stated in the limitations rather than left implied. ## Related Issues Was stacked on #4195, now merged. No tracking issue yet for the Vision implementation itself — happy to open one if you would like the design recorded as an ADR before this goes further. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. — *Types 8562, Server 4072, Client 2123, InformationModel 1063, Robotics 540, Tools 466, Vision 442, AI 74 and the restored Vision+Intent end-to-end suite 14/14, all green on net10.0. Only `.OpenUsd` (71.1%) is below the 80% bar — see Not finished.* - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. — *`dotnet build UA.slnx -c Release`: 0 errors, 0 warnings.* - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. — *net48: Types 8555, Client 2126, Vision 385 green. Two net48 Server tests fail (`CreateUserManagementSeamBindsTheModelToTheServerAsync`, and `DurableDataValueQueueVerifyReferenceBatchingAsync` only under parallel load); both were confirmed **pre-existing** by re-running them with the `IsInverse` change reverted, which fails identically.* - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. — *The net48 build legs failed because the restored end-to-end project used `ValueTask.FromResult` / `string.Create`, which do not exist on .NET Framework; fixed in `fb57522da` and verified building on every TFM. `DisposeDrainsHeldConnectionCallbackBeforeListenerDisposal` failed once on the Ubuntu Client leg — a 13 ms timing-sensitive reverse-connect disposal test that passes locally 2123/2123 and touches no browse paths.* - [ ] I have addressed **all** PR feedback received. ## Typed Vision-guided Robotics MCP workflows The branch now also makes the demo workflow first-class in the Full MCP profile: - replaces stringified Robotics intent and mission inputs with typed request schemas and exact name-or-NodeId selectors; - adds bounded operation/mission paging, stable step-to-operation identity, retained mission discovery, and `robotics_wait_mission`; - makes `vision_run_inference` return bounded typed detection, inspection, or segmentation summaries in one call; - adds same-session `robotics_vision_pick` to select a detection and submit Pick or Pick/Place work without implicitly taking authority, retrying, cancelling, or waiting; and - starts Publish workers for classic subscriptions so an independently launched OpenUSD viewer receives live robot and part updates. The migration guide documents the intentionally breaking MCP request-schema changes. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1dab1302-19c5-4a9b-a50c-d97d389713aa Copilot-Session: a248a589-9a20-4372-868e-5d347e57001b Copilot-Session: 99d1ab42-f83a-41ab-bf3e-85bf81fadace Copilot-Session: bcbc8ac9-5abd-42a8-8bac-e10533790e2e |
||
|
Marc SchierandCopilot
|
64d2834b2f |
Confine capture artifact paths to the configured base folder (#4247)
## What `CaptureSessionManager.StartAsync` validates the caller-supplied `SessionFolder` against the configured capture base folder, then copies `PcapFilePath` and `KeyLogFilePath` from the same request through **without validating them**. Both reach the filesystem: | Property | Sink | Effect | | --- | --- | --- | | `PcapFilePath` (write) | `PcapFileWriter.OpenCurrentFile` — `FileMode.Create`, after `EnsureParentDirectoryExists` | truncates and overwrites the named file, creating parent directories | | `PcapFilePath` (read) | `ReplayCaptureSource` → `PcapFormatter.FormatAsync` → `File.ReadAllBytesAsync` | returns the named file's bytes to the caller | | `KeyLogFilePath` | `OpenOrCreate` + seek to end | appends key material to the named file | `InProcessCaptureSource.ResolveArtifactPath` returns a rooted path unchanged and `Path.Combine`s a relative one without normalizing, so `..` traversal is not contained either. Where those values arrive from outside the process — the MCP `start_capture` tool passes a `StartCaptureRequest` straight through — a caller can name any path the process can open, in either direction. ## Why this looks like an oversight The containment already exists and is applied consistently everywhere else in this package. `PacketDecodeTools.DecodePcapWithKeysAsync` and `PacketReplayTools.ReplayPcapAsync` both run caller paths through `McpCapturePath.ResolveAndValidate(path, allowedRoot)` before use. `start_capture` is the one entry point that skips it. ## Change Adds `ValidateAndResolveArtifactPath` beside the existing `ValidateAndResolveSessionFolder` and applies it to both artifact properties. Relative paths are resolved against the validated session folder, preserving the `StartCaptureRequest` contract, and the result is then confined to the configured capture base folder. Containment comparisons are case-insensitive on Windows and case-sensitive on non-Windows platforms. Null and empty paths pass through untouched so sources keep deriving their default names inside the session folder. ## Tests Five added to `SessionFolderValidationTests`: - rejects a rooted `PcapFilePath` outside the base folder - rejects `..` traversal in `PcapFilePath` - rejects a rooted `KeyLogFilePath` outside the base folder - accepts artifact paths inside the base folder and resolves relative paths beneath the session folder - rejects a case-variant sibling path on Linux Each was verified to **fail without the change**. MCP capture, decode, and replay fixtures now place valid session artifacts beneath the configured capture root while preserving explicit outside-root rejection coverage. `SessionFolderValidationTests` (11/11) and the complete `Opc.Ua.Tools.Tests` suite (430/430) pass locally. ## Note This has security relevance for hosts that expose the MCP diagnostics tools, so it was also reported to `securityteam@opcfoundation.org` per `SECURITY.md`. Happy to close this and handle it privately instead if you would prefer. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b8ab3763-b9bd-46ae-b0ef-045db26e5782 |
||
|
Marc Schier
|
80db326567 |
Fix MCP diagnostics tool contracts and capture failures (#4241)
# Description Fix the MCP packet-diagnostics tool contract and the capture failures reported in #4238. - Accept and emit canonical hyphenated capture source and format names while preserving CLR-name and integer compatibility. - Restore typed enum constraints in generated MCP schemas and surface actionable diagnostics errors to MCP clients. - Stop `list_interfaces` from reading `LinkType` on unopened SharpPcap devices. - Close failed SharpPcap handles before retrying NIC opens and report permission failures immediately. - Normalize capture roots, remove duplicated path segments, and use a lazy secure per-user or atomic temporary fallback. - Align MCP/diagnostics documentation and interop-agent guidance. ## Related Issues - Fixes #4238 ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Copilot-Session: 641a4356-aa8d-45e6-9077-eeb50b9b7e0a |
||
|
|
466c5cb982 |
Robot intent follow-ups: multi-controller executors, spec fixes, and an MCP tool library (#4195)
Follow-ups to #4165, plus the work that followed from re-checking this branch against the specification. Everything started as a defect found by review or by probing the running code; the specification is unchanged except where noted. The TLS certificate lifetime fix (#4196) and the OpenUSD work (#4198) were both merged into this branch. #4196 has since landed on master, so it no longer appears in this diff. #4198 was closed in favour of this PR and its content is here. The MCP trim/AOT work (#4228), the `DataGenerator` null `NodeId` fix (#4227) and the NodeSet external-parent import fix (#4229) have since been merged in as well. All three are closed in favour of this PR and their content is here, described under *Making the MCP libraries trim and AOT clean*, *A generator that returned the value you opted out of* and *A NodeSet parent that lived outside the import batch* below. ## A two-arm cell could not be built `IntentExecution` carried no controller identity, and there was effectively one `IIntentExecutor` per server. An executor handed an execution therefore could not tell **which arm** it was for, so a cell with two `IntentControllerType` instances — an ordinary configuration — was not expressible through the builder or DI. Both halves are addressed: - `IntentExecution` now carries `ControllerId` (`NodeId`, the OPC UA identity) and `ControllerName`, so one shared executor can discriminate; - DI gained per-controller registration — `AddRobotIntentExecutor<TExecutor>(controllerBrowseName)` and an instance overload — so two genuinely different executors can coexist. Resolution order is `WithExecutor` → per-controller DI registration → the single DI executor → a build-time throw. There is still **no silent fallback** to a rejecting executor; that was a Critical defect fixed in #4165 and it stays fixed. `IIntentExecutor` is unchanged, so this is purely additive. ## Declarations that were quietly untrue - **`Accepts<T>` silently discarded a redeclaration.** Declaring an intent type twice — say to narrow `cancelSupported` on the second call — kept the first, wider declaration and said nothing. That is the same class of defect as the capability-honesty problems #4165 fixed (`PauseSupported` declaring a pause the host could not perform; `Aborting` degrading silently to `Buffered`). Identical redeclarations remain idempotent; conflicting ones now throw `BadInvalidArgument`. - **`RI-Interop-40010` assumed Annex B item 3 rather than checking it.** It now requires exact set equality between the OPC 40010 `TaskProgramName` values and the Robot Intent `ProgramType.ProgramId` values. Both sides are readable from the address space without commanding the robot, which is what makes the requirement structural under §12.1. ## Two things the specification requires that this did not do Re-checking the implementation against the merged draft revision turned up two genuine violations. Both are fixed, and the NodeSet is untouched by either. - **Every graphed mission reported `Failed` on success.** `MissionRules.SelectTransition` returned `null` for two different situations — a graph *terminus*, which is the normal success path, and a branch point whose conditions all evaluated false — and the caller collapsed both into `Failed`/`NoTransition`. A three-way result (`Terminus` / `Selected` / `NoConditionMatched`) separates them, so a mission that runs to its end now completes. The failure reason surfaces through the inherited Part 10 result container rather than a new member. - **Facet URIs were never published.** An upstream `may` became a `shall` in the revision this branch adopts. `RobotIntentConformanceUris.Facets` held 31 URIs with **zero** production references — the set existed but nothing emitted it. The published set is now derived from `RobotIntentFacetCalculator`, the same authority that already computed `SupportedFacets`, rather than from a second hand-maintained list that could drift; a reflection test pins the two together. ## An easy client mistake `RequestAuthorityAsync` returns a lease the caller must remember to inspect. Forgetting makes every later submission fail with `ControlNotOwned` for reasons that are invisible at the call site. `RequireAuthorityAsync` throws instead, and names the current owner. `RequestAuthorityAsync` is untouched — inspecting and branching is legitimate. ## Letting an agent drive the robot New `tools/Opc.Ua.Mcp.Robotics` gives a language model the intent API as MCP tools: 36 across discovery, monitoring, direct control and missions, behind a `Robotics` tool profile, composable into any MCP host exactly as the existing OPC UA tool packages are. The `IntentViewerClient` sample hosts them behind `--mcp` (stdio by default, HTTP when `--view`, since MCP stdio and the in-process viewer would otherwise share stdout), and the sample gained a pallet scene so an agent can pick and stack parts and watch the result in the OpenUSD viewer. All the scene logic is server-side — no viewer or connector changes. Four gaps in the client API had to be filled first, since the tools are a thin layer over it and could not have been written honestly without them: `ReadStateAsync`, `ListOperationsAsync`/`ListMissionsAsync`, controller-level `CancelIntent`/`CancelAll`/`Pause`/`Resume`/`Retry`/`ReleaseControl`, and `WaitForCompletionAsync`. The tools refuse rather than paper over: a refusal from the server is returned with its `IntentFailureEnum` and never retried, and command authority is never acquired as a side effect of a tool call. **One defect this surfaced:** `--profile robotics` shipped 36 tools that could not do anything. Every robotics tool resolves a named OPC UA session and only the connection tools can open one, so with no `Connect` in the catalogue the whole profile failed. A test had pinned the defect in place, asserting the profile did *not* contain `Connect` on the line below the `diagnostics` assertion requiring it. Each package now declares its own requirement in its registration extension rather than the host special-casing it, so an application embedding `Opc.Ua.Mcp.Robotics` directly gets a usable catalogue without reproducing the host's pairing. `Full` still takes the connection tools from the core package, so they are never registered twice. The replacement test states the invariant — a profile exposing session-scoped tools must expose the tools that open a session — rather than the instance. ## Making the MCP libraries trim and AOT clean *Merged from #4228.* `Opc.Ua.Mcp.Core` is now `IsAotCompatible` on `net10.0`, and the reflection the analysers then reported is gone. Turning the analysers on is the point: until now they were not running on these libraries at all — including on the tool library this branch adds. - **`OpcUaJsonHelper.Serialize`** went through `JsonSerializer` (IL2026 / IL3050). The conversion helpers in that class already reduce every OPC UA value to a closed set — `null`, the primitive scalars, `string`, `Dictionary<string, object?>` and lists thereof — so the JSON is now written directly with a `Utf8JsonWriter`. Output is unchanged, and that is asserted rather than assumed: `OpcUaJsonHelperTests.SerializeMatchesTheReflectionBasedSerializer` compares the new writer byte for byte against `JsonSerializer` for every shape the helpers emit, including all numeric widths, the custom-formatted scalars, nested sequences and strings that force the encoder to escape. One deliberate contract change: arbitrary object graphs are no longer accepted. Serializing them needs reflection, so `Serialize` now throws `NotSupportedException` naming the type instead of silently stringifying it. No call site in the repository passes anything outside the closed set. - **`McpSchemaFilters`** used `JsonSerializer.SerializeToElement` on a `JsonObject`; it now parses the node's own text, which needs no reflection. `Opc.Ua.Mcp.Diagnostics` is deliberately **not** marked. It depends on SharpPcap and on the reflective service-call dissection in `Opc.Ua.Core.Diagnostics`, neither of which is trim safe — which is why that package does not opt in either. Two improvements are kept regardless: `PacketDecodeTools` serializes `DecodedServiceCall` through a source-generated `JsonSerializerContext`, and `PacketCaptureTools.ListInterfaces` now carries the `RequiresUnreferencedCode` / `RequiresDynamicCode` annotations that `NetworkInterfaceEnumerator.ListLocalInterfaces` already declares, so the constraint is visible to callers instead of implicit. Nothing is suppressed. Marking an assembly `IsAotCompatible` only proves its own IL is clean; it says nothing about whether the library still functions once reflection is gone. So `Opc.Ua.Aot.Tests` references `Opc.Ua.Mcp.Core` and exercises it with JSON reflection disabled, stating what is and is not true today: - tool **results** serialize without reflection — this would fail outright if the `Utf8JsonWriter` change regressed; - tool **registration** still requires reflection, because the MCP SDK builds each tool's input schema by asking `JsonSchemaExporter` for a `JsonTypeInfo` of every parameter type. Without a source-generated context covering the tool signatures — and with the injected `OpcUaSessionManager` being described rather than treated as a service — that throws `NotSupportedException`. The second test asserts the throw. It documents the remaining gap and turns red the moment someone closes it, rather than leaving it to be found in a published binary. The package README says the same thing in prose. **Two pre-existing issues this surfaced, both outside this PR:** pulling `Opc.Ua.Mcp.Diagnostics` into an AOT publish reports `IL2046` / `IL3051` on `NicCaptureSource.StartAsync`, which is annotated `RequiresUnreferencedCode` / `RequiresDynamicCode` while the `ICaptureSource.StartAsync` it implements is not, and `IL2075` on `ServiceCallReassembler.GetCollectionCount` / `GetPropertyString`, which call `Type.GetProperty` on the result of `GetType()`. Happy to file these separately if useful. ## A generator that returned the value you opted out of *Merged from #4227. Fixes #4226.* `GetRandomNodeId(useBoundaryValues: false)` could still return `NodeId.Null` — one of the boundary values the same method offers when boundary values *are* requested. A caller that opted out should not receive one, and `GetRandomExpandedNodeId` builds on it and inherited the same behaviour. The `IdType.Opaque` branch is the practical path: `GetRandomByteString()` draws its length with `NextInt32(MaxStringLength)`, which is inclusive of zero, and a zero-length opaque identifier in namespace 0 satisfies `NodeId.IsNull`. With three namespaces and `MaxStringLength` at 10 that is roughly **one call in 130**. A numeric identifier of `0` in namespace 0 is null too, at around one in four billion. The draw is now repeated while the result is null and boundary values were not requested. Guarding the result rather than the individual branches covers every identifier type, including the numeric case, rather than only the one that happens to be reachable today. The `useBoundaryValues: true` path is untouched, so boundary values remain available when they are asked for. Four tests in `tests/Opc.Ua.Core.Tests/Types/DataGeneratorTests.cs` loop 20,000 draws, because at one call in 130 a single draw proves nothing. Against the **unmodified** generator they fail on draw 93 and draw 1346. Two of the four exist to keep the fix honest rather than to test the bug: `GetRandomNodeIdStillProducesEveryIdentifierType` catches a "fix" that simply stopped emitting opaque identifiers, which would satisfy the null assertions while silently narrowing the generator, and `GetRandomNodeIdWithBoundaryValuesStillYieldsThem` confirms the leak was closed rather than the feature removed. **How it was found:** an intermittent downstream test failure. A test server initialising a static `NodeId`-valued variable via `TestDataSystem.ReadValue` occasionally stored `NodeId.Null`; the encoder correctly writes that as JSON `null`, and a consumer asserting a value was present failed about one run in five — never reproducibly in isolation. It only surfaced under load because `UnsecureRandom.Shared` is a fixed-seed, process-global generator shared between `DataGenerator` and OPC UA transport jitter, so the number of draws preceding address-space initialisation depends on what else is running in the process. ## A NodeSet parent that lived outside the import batch *Merged from #4229.* `UANodeSetHelpers.LinkParentChildRelationships` builds its lookup table from **only the nodes in the current import batch**, links a child when the parent is found — and then cleared `instance.Handle` **unconditionally**, which is exactly where `Import` had stashed the authored `ParentNodeId`. A node whose parent lives in another NodeSet, or is owned by another NodeManager, therefore lost that parent **silently** at import, and nothing downstream could recover it. `RuntimeNodeSetNodeManager.CreateAddressSpaceAsync` already has a step that wires references to external node managers; the parent link never reached it. Such a parent is now recorded rather than discarded, and exposed through `UANodeSet.TryGetUnresolvedParentNodeId`. A parent that resolved normally inside the batch is **not** recorded, so a caller does not re-wire references that already exist. The record lives in a `ConditionalWeakTable`, so it does not keep an imported node alive and does not widen `NodeState` — whose `Handle` is a general-purpose slot a caller may legitimately use for its own purposes, and so is not a safe place to leave the value. `RuntimeNodeSetNodeManager` consumes it to report the one case that cannot be repaired automatically. In NodeSet2 the `ParentNodeId` attribute is a *hint*; the authoritative edge is the entry under `<References>`, which the reverse-reference pass already turns into an external reference. A node that names an out-of-manager parent but carries **no such Reference** materializes with no path from that parent, and no ReferenceType can be inferred to synthesise one — so it is reported as an authoring defect rather than guessed at or silently accepted. This is deliberately **not** a fix for a specific address-space symptom. It removes a silent loss of authored information and gives callers the means to act on it. Whether any particular orphaned-node report is caused by this path, or by namespace-index translation elsewhere, still needs separate diagnosis. **Changed while merging.** As authored in #4229 the accessor was `GetUnresolvedParentNodeId`, returning `NodeId?`. `NodeId` is a readonly struct implementing `INullable`, so that was a `System.Nullable<NodeId>` wrapped around a type that already carries its own null representation — which this repository excludes in favour of an `IsNull` check, and it asks for `TryGet` over casting on struct types. The tests correspondingly asserted `Is.Null` against a struct, which the testing rules also exclude. It is now `bool TryGetUnresolvedParentNodeId(NodeState, out NodeId)`, yielding `NodeId.Null` when there is no unresolved parent. The API is unshipped — it arrives with this branch — so nothing depended on the previous shape. Both tests were re-verified by mutation after the rewrite: disabling the record in `LinkParentChildRelationships` still fails `ImportKeepsAParentThatIsNotInTheSameBatch`. ## Dependencies - **OpenUSD to `0.7.0-alpha`.** All 11 issues we filed upstream are fixed, so the workarounds they justified are gone and the parts of the pump and generator samples that were blocked on them are implemented. - **ModelContextProtocol to `2.1.0`** across all five MCP tool libraries and the executable. No code changes were needed — the surface we use is unchanged in 2.x — but central package management required a cascade of pins, each commented where it sits. Both transports were verified against a running server rather than assumed. MCP 2.x is stateless (no `initialize`, no `Mcp-Session-Id`); an OPC UA session is application state addressed by a caller-supplied name, which fits that model but does assume process affinity, and that assumption is now written down rather than implied. ## Infrastructure The `code coverage` gate was measuring nothing on this branch and reporting success. It is fixed and now enforced, which immediately exposed real gaps — most usefully `RoboticsIntentJson`, which parses agent-supplied JSON (untrusted input from a language model) and had no tests at all. It is now at full line coverage in CI. Patch coverage is **90.20%** against a 75% floor. The `netstandard2.1` build leg was also failing. The MCP projects build as empty shells under a legacy `CustomTestTarget`, and the viewer sample referenced them on the target framework alone, so it tried to resolve a net10.0 shell from net8.0. The sample now derives MCP support from whether those projects are actually being built. ## From review A review pass over the branch found two real defects, both fixed. `RoboticsIntentJson` parses JSON written by a language model, so every malformed shape it can be handed is reachable input; it indexed pose arrays without checking their length and enumerated properties without checking their kind, so a short or wrongly-typed array escaped as `IndexOutOfRangeException` or `InvalidOperationException` rather than the argument error the tool descriptions promise the agent. Separately, `IntentOperationHandle` treated a zero `QueuePosition` as "not observed" and discarded it, so a handle kept reporting the last queued position after the server had said the operation left the queue. The pose validation immediately caught a latent test defect: the pallet scenario interpolated doubles into JSON with the current culture, so under a comma-decimal culture `[0.12, 0.16, 0.28]` became a six-element array. It passed only because that submission is expected to be refused for want of authority, so the wrong coordinates were never used. A third review finding — that a default-constructed `RobotIntentControllerState` hands out a null `NodeId` — was checked and rejected: `NodeId`, `QualifiedName` and `LocalizedText` are readonly structs here, so `default` is the null value and `IsNull` is `true`. The tests asserting that invariant are kept. A second review pass over the merged branch found two more, both fixed and both about an error surfacing as the wrong kind of failure rather than about a wrong result: - **Malformed agent JSON did not arrive as an argument error.** `RoboticsIntentJson` turns every *validation* failure into an `ArgumentException` — a wrong root kind, a missing property, a short pose array — but three `JsonDocument.Parse` calls and the NodeId parse were unguarded. A syntax error escaped as `JsonException` and an unparseable NodeId as `ServiceResultException`, which are the two shapes a language model is most likely to produce. That is the same defect class as the pose/kind checking above: a caller separating bad input from a genuine fault had to know to catch three unrelated exception types. Parsing now goes through one helper that reports a syntax error as an `ArgumentException` naming the argument, keeping the `JsonException` as the inner exception so the position is not lost, and the NodeId goes through `NodeId.TryParse` rather than `Parse`, collapsing its two failure modes into one. `MalformedIntentJsonIsRejected` had asserted `InstanceOf<JsonException>`, pinning the behaviour rather than the intended contract; it now asserts `ArgumentException` like every neighbouring test, and three tests cover the newly guarded paths. - **`ConditionalWeakTable.Add` throws on a duplicate key.** Not reachable today, but `TryGetUnresolvedParentNodeId` documents `Handle` as a general-purpose slot a caller may use for its own purposes — and a caller doing exactly that before re-importing into the same collection would take a hard throw from inside the import pipeline. Removing first makes the most recent import win. `AddOrUpdate` would say this more directly but does not exist on `netstandard2.0`. ## Deliberately not fixed here `FastenIntentDataType.Joint` is refused when the Server exposes no OPC 40450/40451 model, and a client cannot discover that before calling. That is **not fixable in this repository**: §5.4.2 makes `Joint` the defining mechanism while §11.3 says the parameters otherwise stand alone, and `IntentCapabilityDataType` declares capabilities at intent-type granularity with no member-level mechanism anywhere in the model. It is a specification gap, raised as S4 on [marcschier/opcua-drafts#60](https://github.com/marcschier/opcua-drafts/pull/60) rather than worked around with a bespoke mechanism the spec does not define. ## Validation `dotnet build UA.slnx -c Release` on the merged branch — **0 errors, 0 warnings**, with the AOT analysers now running over `Opc.Ua.Mcp.Core`. | Suite (net10.0) | Tests | |---|---| | `Opc.Ua.Core.Tests` | 4247 (86 skipped) | | `Opc.Ua.Robotics.Tests` | 540 | | `Opc.Ua.OpenUsd.Tests` | 909 | | `Opc.Ua.Tools.Tests` | 419 | | `Opc.Ua.Server.Tests` — `RuntimeNodeSet` | 94 | | `Opc.Ua.Robotics.Intent.Tests` | 22 | | `Opc.Ua.Aot.Tests` — `McpAotTests` | 3 | 0 failures. `Opc.Ua.Tools.Tests` rose from 409 to 419 with the serializer-equivalence tests from #4228 and the malformed-JSON tests from the second review pass, and `Opc.Ua.Core.Tests` covers the `DataGenerator` tests from #4227 and the import tests from #4229. `UANodeSetHelpers` also passes on **net48** (5/5). The MCP agent-usability grader scores Core 100 / Full 100. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Copilot-Session: a248a589-9a20-4372-868e-5d347e57001b Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8 Copilot-Session: a0718e8a-a158-45c5-929d-e1d6c0982431 |
||
|
Marc SchierandCopilot
|
fb80bfdc82 |
[WotCon] Track WoT Connectivity/Binding 1.1 and add projections (Section 12) (#4182)
Follow-up to #4128, which merged the WoT Connectivity 1.1 work against **draft2** of the companion specifications. Both drafts have moved since, and this branch tracks them through to **draft3**, adds the projection feature they introduced, and closes the gaps a bidirectional re-audit found. The specifications are the private [`OPCF-Members/spec-drafts`](https://github.com/OPCF-Members/spec-drafts) drafts of *OPC UA — WoT Connectivity 1.1* and *OPC UA — WoT Binding 1.1*. They are still drafts, which is why this is opened as a draft PR — see [Status](#status) at the bottom. ## What changed in the specifications Two revisions landed upstream while #4128 was in flight. **draft2** replaced the `eventGroup` / `actionGroup` grouping constructs with a single **projection / view** concept, added fifteen Section 6 vocabulary terms, and made two identifier rules normative. **draft3** ([spec PR #8](https://github.com/OPCF-Members/spec-drafts/pull/8)) un-deprecated the incorporated OPC 10100-1 v1.02 surface, restructured the conformance clause into three profiles, and tightened two security obligations. The draft3 NodeSet is **structurally identical** to draft2 — I diffed all 286 nodes node-by-node. Every change is metadata: 96 `ReleaseStatus="Deprecated"` attributes removed, `PublicationDate` advanced, and `WOTC-ProjectionMaterialization` claimed by three Type Nodes that previously implemented it in name only. ## Projections (WoT Binding Section 12) The substantial addition. A projection document selects affordances from other Thing Descriptions or Thing Models and resolves to a document of its own; materialized, it becomes an OPC UA `View` that **creates no affordance Node** and only `Organizes` what its sources already materialized. | Piece | Where | |---|---| | Document model and validation | `Opc.Ua.Types/Wot/WotProjection.cs`, `WotProjectionModel.cs` | | Four-stage resolution engine | `Opc.Ua.Types/Wot/WotProjectionResolver.cs` | | Materialization plan | `Opc.Ua.WotCon.Server/Materialization/WotViewProjectionPlan.cs`, `WotProjectionViewBuilder.cs` | | Live address-space host | `WotProjectionViewNodeManager.cs`, `LifecycleWotViewProjectionHost.cs`, `IWotViewProjectionHost.cs` | | Node lookup seam | `IWotMaterializedNodeIndex.cs` | Groups reached by `ua:Organizes` become organizational Objects rather than nested Views, only the outermost materialization is a `View`, and a member whose source is served by another server is omitted and reported rather than recreated — the federation case the clause calls out. ### How this is validated The specification publishes twenty worked examples, two of which are a **golden pair**: a projection document and the resolved view it is defined to produce. `WotSpecExampleTests` embeds all twenty and runs the pair through the resolver, asserting against *the specification's own expected output* rather than against my reading of the prose. That single document exercises all three selection forms, the bulk naming rule, the security closure naming and the provenance term. This is the strongest evidence in the PR, and it is worth more than the unit tests around it: it is the same check the upstream `validate_local.py` performs. ## Other spec work - **Section 6 vocabulary** — fifteen missing model and platform terms with their validation rules (`WotNodeSetConverter.ModelVocabulary.cs`). - **Portable identifiers** — the two rules release 1.1 made normative, with `WotNodeSetConverterOptions.AllowNonPortableIdentifiers` as a documented downgrade-to-warning switch for documents authored against v1.00. Defaults to `false`. - **xRegistry §6.9 identifier construction** (`XRegistryIdentifier.cs`) — reverse-authority symbolic identifiers; all seven published worked examples are test cases. ## Gaps found by re-auditing against draft3 I audited in both directions — every normative requirement to code, and every `uav:` term in code back to a spec that defines it. Three findings, all fixed: 1. **`Wot-Con 1.02` accepted anything well-formed.** draft3 makes format validation part of the unit, because *WoT-Con Minimal* is one unit wide and materialization projects a *valid* document. The 1.02 upload path only deserialized into the `ThingDescription` POCO, which happily accepts JSON that is not a WoT document at all — every member of that type is optional, so `{}` deserialized and was materialized. It now parses via `WotDocument.Parse` and requires the document to identify itself by a non-empty `name` or `title`, returning `Bad_DecodingError` and materializing nothing otherwise. The rule lives in `ThingDescriptionFormatValidator` and is shared with `CreateAssetForEndpoint`, whose auto-generated Thing Descriptions §11 also treats as untrusted input. 2. **Two `WoTFile` operations were ungated.** draft3 §11 names `Write` and `CloseAndUpdate` alongside the five management methods; both reached the materializer without an access check. `EnforceManagementAccess` is now wired into `WotAssetFileManager`. 3. **Two invented `uav:` terms.** `uav:NodeSet2Preservation` was declared and never used — removed. `uav:eventFields` is read by `OpcUaBindingPlanner` and defined in *neither* spec, while a comment claimed it came from OPC 10101. It still works for compatibility but is now documented as a non-standard extension at both sites rather than misattributed. The SSRF prohibition is satisfied by construction: there is no HTTP client anywhere in these assemblies, and all resolution goes through the injected `IWotThingResolver`. ## Two ambiguities fixed in the specifications themselves The audit turned up two places where a conforming implementer had to invent an answer. Rather than pick silently, I took both upstream — they are pushed to [spec-drafts PR #8](https://github.com/OPCF-Members/spec-drafts/pull/8) (all checks green) and are now normative in *Binding* §12.6 and *Connectivity* §7.13: - **`ViewVersion`** had one sentence saying only *that* it changes. It now changes iff the resolved membership changes, is updated at commit time alongside the `NodeVersion` stamping, and is a deterministic function of the membership alone in a canonical order — so two servers that resolved the same membership agree, which a per-server counter would not across a redundant pair. Explicitly **not monotonic**: compare for inequality only. - **"Type-level view"** was used once in §12.2 and defined nowhere, while every other statement of materialization was universal. §12.6 now *defines* it rather than contradicting §12.2: the NodeClass is `View` in both cases. Both were already what this code did, so neither changed behaviour — but the order-insensitivity the clause added was **not** covered by any test, and `ViewVersionIsUnchangedWhenOnlyTheOrderOfTheMembershipChanges` now holds it. It is mutation-verified: removing the ordinal sort turns it red. `docs/WotBindings.md` no longer lists these as points "the specification leaves open". ## Validation | Suite | Result | |---|---| | `Opc.Ua.Types.Tests` | 8525 ✅ (8518 on net48) | | `Opc.Ua.WotCon.Tests` | 1057 ✅ (net10.0 and net48) | | `Opc.Ua.WotCon.Bindings.Tests` | 559 ✅ | | `Opc.Ua.XRegistry.Tests` | 196 ✅ | | `Opc.Ua.Server.Tests` | 4027 ✅ | | `Opc.Ua.SourceGeneration.Core.Tests` | 3771 ✅ | | `Opc.Ua.WotCon.Samples.Tests` | 9 ✅ | The full `UA.slnx` builds with **0 errors and 0 warnings attributable to this PR** on all 6 TFMs (`TreatWarningsAsErrors` is on). `Opc.Ua.Server.Tests` and the source-generation suites are listed because this PR now also touches `MasterNodeManager` and the WoT-Con NodeSet. ## Compatibility No public API is removed or broken. Everything added is new surface using `ArrayOf<T>` / `ByteString` per the repo conventions, wired for DI with a direct-construction fallback, and the one behavioural tightening (`Wot-Con 1.02` format validation) rejects input that was never valid to begin with. One pre-existing issue is **not** addressed here because it came in with #4128 and is out of scope: `WotAssetFileManager.OnCloseAndUpdate` uses `.GetAwaiter().GetResult()`, which the repo rules ban. Worth a follow-up. ## Landed since this PR was opened - **DI upgraded to 1.05.0.** `ConnectsTo` was declared a subtype of `HierarchicalReferences`, contradicting OPC 10000-100 §5.5 Table 48. That is an upstream error in every official DI NodeSet up to 1.04, corrected by the OPC Foundation in 1.05.0, so the fixture is replaced with the verbatim 1.05.0 file rather than hand-patched. A guard test pins the corrected form. This also closed a second gap: six DI types the WoT model was missing are 1.05 additions. - **The sample Pump has the type definition OPC 40223 declares for it.** `Pump1.Identification` carried DI's `FunctionalGroupType` instead of `PumpIdentificationType`. Every test, the sample documents and the README agreed with each other about it, which is why it survived. - **`WotPumpAddressSpaceComparisonTests`** compares this server against `PumpDeviceIntegrationServer`, which builds the same OPC 40223 Pump from generated NodeSets and hand-written wiring. An independent oracle rather than a restatement of the same assumptions - it is what found the defect above. - **Error containment in `MaterializeProjectionViewsAsync`.** An exception mid-loop discarded the applied View handles before they were published, leaving View Nodes in the address space that nothing could remove. - **`HasWoTProjection`** is now created; it was documented and plumbed but never wired. - **The default in-memory View host** no longer deletes the View it has just applied. - **An undeclared NodeSet alias now says so.** It previously surfaced as `IdentifierMissing`, naming neither the value nor the fact that an alias was expected. - **`ViewVersion` follows the algorithm the specification now states.** My earlier spec text was rejected on review, correctly: it named neither the function nor the canonical order it demanded, ignored that OPC 10000-3 §5.4 requires a value greater than zero, and its "when, and only when" is unsatisfiable for a 32-bit value. §12.6 now specifies the algorithm outright and this implementation follows it exactly, pinned by an independently computed test vector. Implementing it surfaced a further defect in the clause - joining members on U+000A is not injective, because a NodeId string identifier may itself contain U+000A - which is fixed in the specification by length-prefixing each member. - **An auto-generated Thing Description is treated as untrusted input.** §11 requires it; `CreateAssetForEndpoint` validated the endpoint but handed whatever the discovery provider returned straight to the materializer. The `Wot-Con 1.02` format rule now gates both paths from one place. - **Document bytes moved out of the registry snapshot** into a content-addressed store, so a commit no longer rewrites the whole corpus to edit one document. Writes stage and are promoted during the commit, which preserves both durability-first ordering and the fail-closed rule that a blob directory without a manifest means operator recovery. Restoring streaming digest verification along the way closed a real gap: content integrity had degenerated to a length comparison. - **External references now reach a NodeManager registered later.** A reference to a Node that does not exist yet is dropped rather than queued, and the dynamic-registration path had no second phase to apply them in, so registering a NodeSet before the one it points into lost the forward edge permanently. Review of that change then found a state leak in its rollback path, fixed with a red-to-green regression test. - **`WOTC-Legacy` is renamed `Wot-Con 1.02`** in the specification and here. Editorial: the NodeSet diff is `<Category>` values only. ## Status Open for review. Two things may still move: 1. **The specifications are unpublished drafts.** Anything here can change if the working group decides differently. The `ViewVersion` definition has since been settled - §12.6 specifies the algorithm and this follows it - but the same could happen to any other clause tracked here. 2. **`uav:eventFields`** is kept as a documented non-standard extension. That is a judgement call the WG may want to reverse, either by adopting the term or removing it. Two known items are recorded rather than fixed: - `Pump1` has no hierarchical parent, so it is reachable by NodeId but not by browsing down from `Objects`. The cross-NodeSet ordering defect this was originally attributed to is fixed above, and cross-manager references are now symmetric in both registration orders. The Pump itself is still blocked on a different cause found while verifying that fix: with the reference added, `DeviceSet` exists but the Pump has **zero** inverse references, so the edge is dropped at *import* - before any external-reference machinery runs. That is a separate defect in the materialization path, not the one that was fixed, and it is left recorded rather than guessed at. - `WotAssetFileManager.OnCloseAndUpdate` uses `.GetAwaiter().GetResult()`, which the repo rules ban. Pre-existing from #4128 and out of scope here. Review of the projection resolver and the materialization host is the most valuable thing to look at; the vocabulary and identifier commits are mechanical by comparison. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8 |
||
|
Marc SchierandCopilot
|
8a10139b8c |
Fix OPC UA address-space compliance in the device integration stack (#4117)
# Description
A live OPC UA compliance audit of `samples/PumpDeviceIntegrationServer`
against the **OPC 40223 Pumps** and **OPC 10000-100 (DI)** companion
specifications found **13 address-space violations** plus a
serialization bug in the MCP tool. Every fix lands at its **root cause
in the shared stack** rather than as a workaround in the sample, so all
servers benefit.
The sample now exposes **N fully simulated pumps** (`--pumps N`) and
advertises the DI conformance facets it actually satisfies.
> This branch has been merged with `master` including **#4116** ("Add
simulation to PumpDeviceIntegrationServer and Event modelling"), which
independently reworked the same pump simulation and the same fluent
alarm builders. See [Relationship to #4116](#relationship-to-4116) below
for how the two were reconciled.
## Findings and fixes
| # | Finding | Root cause | Fix |
|---|---------|-----------|-----|
| 1 | `OverTempAlarm` created by the fluent API did not exist in the
address space | `AttachAlarm` attached the alarm with `AddChild` but
never indexed it into the node manager | Register the node; the same gap
existed in `AddObject`, `CreateInstance` and the state-machine creators
|
| 2 | Alarms were unsubscribable — `EventNotifier = 0` everywhere, no
`HasNotifier` from `Server` | No event-source wiring at all | Set
`HasCondition`, initialise
`SourceNode`/`SourceName`/`ConditionName`/`InputNode`, promote
`EventNotifier` up the ancestor chain, publish `HasNotifier` from the
`Server` object |
| 3 | Every instance NodeId was minted in the **DI standard namespace**
(`ns=4;s=5001_Pump #1`) | `DiNodeManager.New` inherited the parent's
namespace index | New `DiNodeManager.InstanceNamespaceIndex`; instances
move to the server's application namespace |
| 4 | `Server/Namespaces` described only 2 of 6 namespaces | The
NodeSet2 importer drops nodes parented to namespace 0, so companion-spec
metadata objects never reached the address space — and no route covered
the server's own namespace | New `NamespaceMetadataPublisher` walks the
`NamespaceArray` and fills version/publication date from
`ModelDependencyAttribute` |
| 5 | Vendor BrowseNames (`Diagnostics`, `LastError`, …) sat in the DI
namespace | Builders defaulted to the parent's namespace | Default to
the server namespace; **spec-defined BrowseNames are unchanged** |
| 6 | `AccessLevel="5"` from the Pumps NodeSet was reported as `1` | The
ModelDesign `AccessLevel` enum is not `[Flags]` and cannot represent
`CurrentRead \| HistoryRead` | Carry the verbatim NodeSet2 bitmask and
emit named `AccessLevels` constants |
| 7 | `Historizing = true` while `HistoryRead` returned
`BadHistoryOperationUnsupported` | Nothing reconciled the declared
history surface with the wired historians | Startup clears the
advertisement (including masking the attribute read callbacks) when no
provider resolves; the sample wires a real historian |
| 8 | Mandatory `ProductInstanceUri` unset | Sample omission | Populated
on every pump |
| 9 | Mandatory `TrueState`/`FalseState` empty on every supervision
boolean | Absent in the official NodeSet; a server must still populate
them | Populated on every pump |
| 10 | Machinery `Machines` folder empty (OPC 40001-1 §9.2 says
**shall**) | `TryAddToMachinesFolder` used `AddChild`, which
*re-parents* the device away from `DeviceSet` | Add the `Organizes`
reference instead |
| 11 | BrowseName contained a space and `#` | Sample naming | `Pump_1`
with the readable label in `DisplayName` |
| 12 | `Maintenance` group materialised but empty | Sample omission |
Populated |
| 13 | Uninitialised values reported `Good` | No initial-value status |
`BadWaitingForInitialData` until the first simulation tick |
| — | `ServerProfileArray` advertised only `StandardUA2017` |
`DiNodeManager` did not implement the existing `IConformanceContributor`
| Contribute DI 1.05 server facets computed at runtime from what is
actually wired, merged with `StandardUA2017` |
| — | MCP tool serialised boolean `false` and numeric `0` as JSON `null`
| `Variant.Null` is `default`, so `Variant.Equals` adopts the non-null
type and compares against the default value | Dispatch on `BuiltInType`
and use typed `TryGetValue` accessors instead of the prohibited
`AsBoxedObject` |
## Relationship to #4116
#4116 landed on `master` while this branch was in flight and targeted
overlapping ground. The merge is a **union of both intents**, not a
fast-forward:
- **`AlarmBuilderExtensions`** — kept both sides. #4116 contributes the
parent-must-be-an-Object guard, `SetEnableState`, and the
`HasEventSource` references; this branch contributes the `HasCondition`
reference, condition source initialisation, **registration of the alarm
with the node manager so it is browsable**, notifier promotion up the
whole ancestor chain, and root-notifier registration.
- **`SupervisionBuilderExtensions`** — took #4116's `SetAlarmActive`. It
is more spec-correct than this branch's version: it honours
`EnabledState`, resets Acked/Confirmed on activation, and computes
`Retain` per OPC 10000-9 so an unacknowledged Condition stays retained
after going inactive. This branch's null guard was kept so a malformed
alarm cannot dereference a missing `EnabledState`.
- **`PumpNodeManager.Configure.cs`** — took #4116's **push-based**
simulation (`RegisterPumpSimulation`, `CreatePumpSimulation`,
`IValueUpdater<T>`, `Initialize`/`Advance`/`Publish`, deterministic
per-pump phase offsets) as the structure, then folded in this branch's
compliance work: configurable pump count, Maintenance group,
`TrueState`/`FalseState`, historian wiring, per-pump mandatory
Identification values, and `BadWaitingForInitialData` until the first
tick.
- **README** — union, keeping #4116's address-space validation workflow
section.
Two test adjustments were needed, both because the merge legitimately
changed behaviour the tests pinned:
- `PumpHostedReferenceTests` now resolves pump NodeIds from the browse
result instead of hard-coding the DI namespace, and waits for the first
published value. `PumpCreatedAfterStartupJoinsTheLiveSimulationAsync`
was **added** to restore #4116's coverage that a pump created after
startup through `ConfigureDevicesFor` joins the live simulation.
- `FluentAlarmRegistrationIntegrationTests` no longer asserts that
`Retain` clears when an alarm goes inactive — that pinned this branch's
weaker semantics, and #4116's Part 9 behaviour is correct.
## Breaking change
Runtime-created DI device instance NodeIds move from the companion-spec
namespace to the server's application namespace, e.g.
`ns=<DI>;s=5001_Pump #1` becomes `ns=<application>;s=5001_Pump #1`. This
invalidates NodeIds cached by clients from earlier 2.0 previews. It is
**not** a model change — standard DI/Machinery/Pumps type NodeIds and
spec-defined BrowseNames are untouched.
Device integration hosting is new in 2.0, so there is nothing to migrate
from 1.5.378 and this is deliberately *not* in `docs/MigrationGuide.md`.
Anyone tracking 2.0 previews should rediscover devices by browsing
`DeviceSet` rather than persisting NodeIds, and resolve namespace
indexes from `NamespaceArray` per connection.
## Verification
Compliance is pinned by regression tests that assert **through the
server's service surface** (`BrowseAsync` / `ReadAsync` /
`HistoryReadAsync`), not by inspecting `NodeState` objects in memory.
That distinction mattered: three defects survived in-memory assertions
and were only caught through the read path — most subtly, `Historizing`
was cleared while `AccessLevel` still advertised `HistoryRead`, because
attribute read callbacks re-applied the NodeSet bits.
Post-merge on net10.0:
| Suite | Result |
|---|---|
| `Opc.Ua.Di.Tests` | 326 passed |
| `Opc.Ua.Server.Tests` | 3846 passed, 5 skipped |
| `Opc.Ua.Tools.Tests` | 288 passed |
| `Opc.Ua.History.Tests` | 506 passed, 27 skipped |
| `Opc.Ua.SourceGeneration.Core.Tests` | 3722 passed, 8 skipped |
The full solution was also built and tested on **net48** before the
merge.
Review feedback from `copilot-pull-request-reviewer` has been addressed
in commit `badad31dc`: the shared `ServerObjectState` lock was removed
(the forward `HasNotifier` edge is published through the owning node
manager instead), MCP array serialization now preserves element types,
and the pump option guards report the specific rejected option. The only
failure is
`ConfigureApplicationBuildsSharedClientAndServerConfigurationAsync`, a
pre-existing certificate-store issue that **passes when run in
isolation** and is unrelated to this change.
## Documentation
Updated `docs/DeviceIntegration.md`, `docs/MigrationGuide.md`,
`docs/HistoricalAccess.md`, `docs/DeveloperGuide.md`,
`docs/McpServer.md`, `docs/README.md`,
`samples/PumpDeviceIntegrationServer/README.md` and
`tools/Opc.Ua.Mcp/README.md`, and consolidated the four node-manager
documents into a single [`docs/NodeManagers.md`](docs/NodeManagers.md) —
overview, built-in managers (including `MasterNodeManager`), core vs
custom, registration, server address-space metadata, and source
generation — with a TOC and all inbound links repointed.
## Related Issues
No tracking issue exists for this work yet — it originated from an
ad-hoc compliance audit of the pump sample. Given the size of the change
and the runtime-breaking NodeId move, please open (or link) a tracking
issue so the design is recorded as an ADR before merging.
- Fixes #
## Checklist
_Put an `x` in the boxes that apply. You can complete these step by step
after opening the PR._
- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0d50c3fc-58b1-4c99-85a4-1e32c88666d9
|
||
|
|
16dd14f420 |
Improve MCP server coverage and agent usability (#4080)
## Summary - Expand deterministic MCP coverage across host startup, sessions, configuration, NodeSet export, PKI, packet diagnostics, and PubSub. - Fix Streamable HTTP startup by supplying the OPC UA client application identity. - Add client-visible required-argument errors and normalize generated tool schemas with explicit `required` arrays. - Introduce bounded `core`, `services`, `administration`, `pubsub`, and `diagnostics` tool profiles while retaining `full` as the current-major default. - Split configuration updates into focused transport, client, and security tools while retaining `SetConfiguration` in the `full` compatibility profile. - Improve tool descriptions/examples and add pinned, deterministic `mcpgrade` static/probe CI. ## Compatibility and behavior - Existing stdio clients continue to receive the `full` catalog by default. - Streamable HTTP is exposed at `/mcp`; the previous root route is intentionally removed. - `--transport http` is preferred; `--transport sse` remains a deprecated current-major alias. - Diagnostics decode/replay/key tools remain behind the existing explicit security gate. - N002 is disabled only for documented OPC UA standard and compatibility-sensitive name pairs. ## Validation - Release analyzer build: 0 warnings and 0 errors. - MCP tests: 276/276 passed with focused Coverlet collection. - MCP coverage: 82.73% lines, 78.96% branches. - MCP patch coverage: 94.79% lines (291/307 changed coverable lines). - `core`: 15 tools, mcpgrade A/100, zero findings. - `full`: 78 tools, mcpgrade A/100; only documented T001, T004, and S007 findings remain. - Full stdio probe: zero C003/C004 findings. Fixes #4079 --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@hotmail.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Marc Schier <marcschier@users.noreply.github.com> Copilot-Session: bb615463-338e-4bbb-9779-5bf6780b8b47 |
||
|
Marc SchierandCopilot
|
3c4661d2bf |
Protect certificate and identity secrets in MCP and durable samples (OPC 10000-4 security) (#4043)
## Failure MCP auto-accept mutated the shared application certificate manager and accepted every certificate error, so one permissive connection could affect concurrent or later strict sessions. The durable-subscription sample also serialized complete user identity tokens, including user-name passwords and issued bearer tokens. ## Fix - Isolate auto-accept certificate managers/configurations per connection and keep their lifetime with the managed session. - Auto-accept only `BadCertificateUntrusted`; retain strict validation for all other certificate errors and shared sessions. - Version the durable store and reject the previous unsafe format. - Remove user-name passwords before persistence and reject issued-token subscriptions rather than storing bearer credentials. - Add focused tools tests plus solution, project, and durable-subscription docs. ## Reference - OPC 10000-4 Session identity-token and application-certificate security. - Split from #4021. ## Tests - `Opc.Ua.Tools.Tests` Release: 7 passed on net10.0; 4 passed on net48. - `Quickstarts.Servers` Release builds passed on net10.0 and net48 with zero warnings/errors. - `Opc.Ua.Mcp` Release net10.0 build passed with zero warnings/errors. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |