# Description Implement the draft **OPC UA — Robot Intent** companion model end to end: task-level motion verbs, a Part 10 lifecycle, a server hosting surface, a client, two samples and a live integration suite. It replaces the opt-in convention API merged in #4127 with an implementation generated from a companion information model. ## Why OPC 40010-1 *Robotics* describes robot **topology** in detail and defines **no motion verbs at all** — its entire actuation surface is `Start`, `Stop` and loading a named program. A conformant client can discover everything about a robot's construction and cannot ask it to move anywhere. #4127 filled that gap with ten verbs in an application-owned namespace, resolved by BrowseName, and the file said what it was: *"opt-in, explicitly non-normative industrial operation conventions"*. That contribution established a **vocabulary**. It established no **lifecycle**, and the lifecycle is the harder half. A motion takes seconds; a pick takes a minute. OPC 10000-4 §5.12.2 discards a method result when the Session ends *"independent of the task actually performed at the Server"* — so a synchronous motion method does not merely time out, it **loses the outcome of work that has already physically happened**. OPC 10000-10 §4.1 gives the OPC Foundation's own resolution: a Method performs a calculation, a **Program** runs a batch process or a machine tool part program. So `SubmitIntent` returns as soon as the intent is *admitted*, and what it returns is a NodeId — an `IntentOperationState` (a Part 10 program instance) the client subscribes to for progress and reads for the result. ## What changed **Model and shared code** — `Opc.Ua.RobotIntent.NodeSet2.xml` is source-generated in `Opc.Ua.Robotics`, so the enums, the polymorphic intent structures, `IntentOperationState : ProgramStateMachineState`, the method states and the typed clients derive from the model rather than being hand-written. Its only `RequiredModel` is the base UA namespace, so a server can adopt it without OPC 40010 or DI. `PoseMath` implements the specification's Annex C conversion between its unit quaternion and the core `ThreeDFrame` — including the `asin` clamp that stops a pole orientation becoming a domain error, and the non-negative-`w` representative that makes two servers agree on four numbers. `FrameTree` composes transforms along the frame tree. **Verbs are a DataType hierarchy, not one Method each.** A single submission and a mission step are then the same shape, and a new intent is a subtype rather than a new method — which is what `AddOperation<TRequest,TResponse>` was working around. **Server** — `IntentControllerHost` owns admission in the specification's order, the queue with PLCopen buffer modes, blending, cancellation *with the server's right to refuse*, missions with a committed base and a revisable horizon plus an IEC 61131-3 step graph, safety-aware refusals, real-time channel brokerage, and the capability declaration. All thirteen Methods are wired; command authority and channel leases are released when the holding Session closes; every client-supplied NodeId is validated against the controller being commanded (clause 11.3). `RobotIntentNodeManager`, the fluent `IIntentControllerBuilder` and `AddRobotIntent`/`ConfigureRobotIntent` make it reachable, and `RobotIntentFacetCalculator` computes which of the specification's facets an instance actually satisfies. **Client** — `RobotIntentClient` discovers `Server/RobotIntent/Controllers` and reads what a robot accepts; `IntentOperationHandle` subscribes and completes on a terminal state, recovering the result after a reconnect because clause 6.7 promises it survives; `CommandAuthorityLease` and `RealTimeChannelLease` manage the two exclusive resources; fluent builders cover every intent and mission. **Samples** — all robotics samples move under `samples/Robotics/`. `IntentEnabledRobot` is one UR5e-style offset-wrist arm on a bench, declared completely enough to show what a conformant server publishes, with a simulated safety source so the clause 10.4 refusals are demonstrable. `IntentViewerClient` turns a **click on a target prim in the OpenUSD viewport** into an intent and watches the arm execute it; it also runs headless, which is how CI exercises it. **Intents execute serially**, which satisfies every `BlockingMode` constraint by construction: the specification forbids beginning a `Single` or `Hard` intent while another executes and merely *permits* `None` and `Soft` to overlap. The convention API (`RoboticsOperationConventions`, `RoboticsOperationsBuilders`, `RoboticsOperationsClient`) is removed. ### On safety — what this deliberately does *not* claim The interface is **non-safety-rated**, and that is a property of the technology rather than a scoping choice. OPC 10000-15 carries cyclic safety data from a SafetyProvider to a SafetyConsumer; the consumer's `RequestSPDU` holds an identifier, a monitoring number and one octet of explicitly **non-safety** flags, so a caller has no channel through which to supply safety-rated arguments. Every safety fieldbus (PROFIsafe, CIP Safety, FSoE, openSAFETY) expresses a safety command as a **continuously asserted cyclic signal**, because the integrity argument rests on the fail-safe state that follows when assertion *stops* — and a Method call has no defined behaviour when it stops being called. What the host does instead is **observe and refuse**: `UpdateSafetyState` is how the application reports what the safety system is enforcing, and admission then refuses on the same values a client can read, so a refusal is explainable from the address space rather than from Server-internal state. It may never *instruct* the safety system, and no Method here commands a safe motion function, changes an operational mode or clears a stop. ## Defects this found in the specification, fixed upstream Implementing a specification is the only way to find out whether it can be implemented. Seven defects went back to [marcschier/opcua-drafts#47](https://github.com/marcschier/opcua-drafts/pull/47) rather than being worked around here: - **Refusal was unobservable.** Clause 6.2 orders six distinct refusals and clause 5.8 calls the failure set "small and diagnosable on purpose" — but `SubmitIntent` returned only `IntentId` and `Operation`, so a client could see *that* something was refused and never *which*. It now returns `Accepted`, `Failure` and `Message`, and clause 6.2 states that a refusal returns `Good` rather than substituting a Bad `StatusCode`. - **The Part 10 promotion was illegal.** `IntentOperationType.ProgramDiagnostic` was declared as a Property of `PropertyType`; OPC 10000-10 declares it a Variable of `ProgramDiagnostic2Type` reached by `HasComponent`. The declaration added a *second* member beside the inherited one instead of promoting it — which is precisely why an earlier version of this branch had deleted it. Correcting the TypeDefinition made it generate cleanly with **no change to the curated Core**. - **Clause 9 said nothing about the Method surface.** A server could declare `MissionsSupported` true and omit `SubmitMission` entirely, because the mission and channel Methods are Optional. This implementation did exactly that until an integration test caught it. A fourth honesty rule and a table now fix which Methods each declaration implies. - **Conformance was not machine-readable, and two implementations disagreed.** Clause 12 defined conformance in terms of facets and gave a Server nowhere to publish which ones it satisfied, so every client had to re-derive Table 12.2 from the address space. This repository proved why that fails: the server''s `RobotIntentFacetCalculator` and the client''s `RobotIntentRules.DeriveFacets` each implemented clause 12.2 independently and **disagreed on essentially every row** — the client''s `Blending` was `BlendingSupported` alone where the server also requires all four blending buffer modes, its `QueuedIntents` was `MaxQueueDepth > 0` where the server also requires `Buffered`. A client and server from the same release answered "does this controller conform?" differently, and both were reading the specification correctly, because several rows are behavioural and no amount of browsing settles them. `IntentCapabilitiesType` now carries **`SupportedFacets`**, exactly as OPC 10000-5 does with `ServerCapabilitiesType.ServerProfileArray`, RI-Base requires it, and clause 12.2 separates structural requirements a Server shall meet before listing a facet from attested ones governed by clause 9. - `WaitIntentDataType.Signal` was unbounded where clause 11.3 has to check it; §11.3 gained a table fixing the expected type of every NodeId-valued member; §5.7.0 gave `Ready`/`ActiveIntent`/`ActiveMission` normative meaning; §6.9 bounded `RequestedLease`; §6.5 and §6.4 now say what a server that cannot differentiate `StopMode` must do, and which stop a superseded intent gets. ## Bugs the tests and reviews caught Each of these would have been silent in the field: - **The execution engine was never wired into the public API.** `IntentControllerBuilder.RegisterAsync` built the address space and never started an `IntentControllerHost`, so through the documented `AddRobotIntent` → `ConfigureRobotIntent` path **every Method returned `BadNotImplemented`** and the registered `IIntentExecutor` was never resolved. A browsable address space is not a working robot interface, and only a live integration test showed the difference. - **NodeId scoping failed open on an empty index.** A controller advertising `CallProgramIntentDataType` with no published `ProgramType` accepted an *arbitrary* NodeId as `CallProgram.Program` — and `CallProgram` runs code the server holds. - **Blending completed the predecessor at submission time** and abandoned its motion, telling the client it succeeded and publishing an all-zero pose at the origin as a genuine `AchievedPose`. - **Mission base immutability compared only identifiers**, so a client could keep the ids and swap the *motion* of an already-committed step. - **`FinalResultData/Result` was never registered** with the node manager: Browse showed it, Read returned `BadNodeIdUnknown` — so clause 6.7's promise to a Part 10 client did not hold. - `DisposeAsync` disposed primitives out from under in-flight executor code; re-using a terminated `IntentId` destroyed the retained result and duplicated a NodeId; neither side registered the Robot Intent encodeables, so `Pose3DDataType` did not decode; host startup was hooked to a task that ran in only one hosting shape. - The result was published **after** the state went terminal; capabilities were resolved against a stale namespace table; an empty `ContentFilter` arrives as an empty element array rather than a null filter, so testing only for null made every unconditional mission transition silently untaken — a mission would run its first step and stop. ### Found by a later test-quality audit, after the above was written An audit of the test suite itself found five tests whose **names** asserted a normative requirement while their **code** asserted something weaker or nothing at all. The worst **fabricated the answer it checked**: a helper caught a decode failure and invented "5 supported intents, 6 axes, all facets true", so six capability-honesty assertions across four tests could pass while reading nothing from the server. The fallback is gone and the suite passes with honest reads. Chasing the facet gaps it reported led to the clause 12 defect above. Two further defects followed from completing that work: - **`RI-Interop-40010` was never computed**, and the code documented the omission as impossible to fix "because the calculator receives only a Robot Intent controller, not the linked OPC 40010 model". That was wrong: Annex B's structural requirement is an **inverse** `HasIntentController` reference, reachable from `IntentControllerState` — and the integration suite was already browsing exactly that. A server implementing Annex B could not claim Annex B. - **The published claim could go stale.** `SupportedFacets` was a snapshot taken during `RegisterAsync`, so a reference attached afterwards left it wrong. It is now bound through `BindRead` and recomputed on read, so the claim tracks the address space by construction rather than by discipline — which matters for anyone bolting Robot Intent onto an existing OPC 40010 node manager, where the attach ordering is often not theirs to choose. Two tests that **could not fail** were also replaced: one asserted `AccessLevel` equals `CurrentRead` to pin a read-only marking, but `BaseVariableState` initialises every variable to exactly that, so deleting the marking left it green; it is now a write expecting `BadNotWritable`, including a real OPC UA `Write` service call over a live session. Finally, a build break that a plain build hides: `RoboticsAotTests` used `out string?` in a project declaring `<Nullable>disable</Nullable>`. Building the project **without** `-f` suppresses `CS8632` through `NoWarn=nullable` and succeeds; building it **with** `-f net10.0` resolves `NoWarn` differently, `CS8632` fires, and `TreatWarningsAsErrors` makes it an error. CI builds per-TFM, so this would have failed there while passing every local build to that point. ## Testing and coverage | | | |---|---| | `dotnet build UA.slnx -t:Rebuild` | **0 errors, 2 warnings** — both pre-existing `CA1861` in `tests/Opc.Ua.Tools.Tests`, a file this PR does not touch | | **Whole `UA.slnx` test suite, net10.0** | **50 of 52 assemblies fully green** (measured at `40d9cf213`, before the `master` merge; the other two are explained below) | | `Opc.Ua.Robotics.Tests` | **467** net10.0, **447** net48 | | `Opc.Ua.Robotics.Intent.Tests` (live client <-> server) | **20** on net10.0 and net48, run repeatedly | | `Opc.Ua.OpenUsd.Tests` | **904** after merging `master` (664 before; the rest arrive with #4160 / #4162 / #4180) | | `Opc.Ua.Aot.Tests` | **121/121** | | `Opc.Ua.Di.Tests` (regression from the sample move) | **364/365**, one pre-existing skip | | **Changed-line coverage** | **90.7 %**, no file below 80 % — comfortably above the enforced band, which after #4158 applies only to patches larger than 100 changed lines | The two non-green assemblies are **not** caused by this branch. Rather than assert that, I ran the same selections at the merge-base `ae241b662` on the same machine: - `ServiceCallReassemblerTests.PrivateRequestSummaryFormatsKnownRequestKinds` expects `12.5` and gets `12,5` — a culture-sensitive assertion failing on a European locale. Fails 4/4 on **both** commits. - `CertRotationLiveTests.L2Cert2RotateCertificateDuringServerRestartRecoversSharedChannelAsync` fails on a teardown count. Fails 4/4 on **both** commits. - A third failure seen once, AOT `FindServersAsync` with `BadSecureChannelClosed`, did not reproduce: 4/4 on retest and 121/121 in the final run. Environmental. Two things are worth calling out because they would otherwise be invisible: - `tools/**` is **not** in the `coverage-thresholds.json` ignore list, and no test project referenced the OpenUSD tools assemblies — so their changed lines counted as uncovered and the real patch figure was 75 %, exactly the blocking threshold. Rather than widening the ignore list, the pure decisions in the viewport host (pointer-to-physical-pixel conversion, `RenderPickRequest` construction, stale-retry, backend discovery, pick-mode selection) were separated from the Avalonia/native shell and tested headless against a fake picking backend. - `dotnet build UA.slnx` **hides warnings on unchanged projects**; only `-t:Rebuild` surfaces them. That is how a `CA2263` introduced by a net48 portability fix survived several "zero warning" builds. The multi-target requirement earned its keep repeatedly: it caught a default interface implementation, `ArgumentNullException.ThrowIfNull`, `string.Create` with an interpolated handler, `ValueTask.FromResult`/`CompletedTask`, `Task.WaitAsync`, generic `Enum.GetValues` and generic `Enum.IsDefined`, and a non-generic `TaskCompletionSource` — none of which the net10.0 build complained about. A net10.0-**only** hang in the simulated executor and two load-sensitive flaky tests were also found and fixed at the cause. ### Merged with `master` `master` (through `e73e71184`) is merged in as `08205d81d`. Four conflicts, all where the two branches touched the same robot and OpenUSD surfaces, and none resolved by dropping a side: - #4162 added six files to `samples/MinimalRobotServer/` while this branch moved that directory to `samples/Robotics/MinimalRobotServer/`. They are kept at the moved path, and `UA.slnx` takes the union of both sides' new projects. - #4160 replaced `object` with a `UsdValue` union and rewrote `Flatten`; this branch had reformatted the same method and fixed a path in its doc comment. Git produced a hybrid that did not compile. The resolution keeps `master`'s `UsdValue` arm and `Flatten(UsdValue)` overload in this branch's block-bodied form, with the path fix retained. - `master` inserted `FindStageCamera` directly before two methods this branch had relocated, so the merge duplicated `WriteStageUsda` and `GetPrivateStateRoot`. Only `FindStageCamera` is genuinely new — `master`'s diff shows the other two as unmodified context — so the duplicates are dropped and this branch's `internal` versions kept, including the `GetPrivateStateRoot(string?)` overload the connector tests use to stay out of the real user profile. - `master` moved the OpenUSD stack to `0.4.0-alpha`; this branch pins `OpenUsd.Rendering` in the test project so the renderer-pick tests can reach the picking backend. Left at `0.1.0-alpha` that pin is below what the viewer now resolves transitively, which is `CS1705`. The pin follows `master`. CI was green on `baaf35e9` with **190 checks passing and none failing**, across every build leg (net472 / net48 / netstandard2.0 / netstandard2.1 / net8.0 / net9.0 / net10.0, Debug and Release) and the full test matrix on Windows and Linux. Since then `master` has been merged twice more and two CI-stability fixes landed (see below), so the current head is re-running. Locally, `dotnet build UA.slnx -t:Rebuild` gives **0 errors** and the 2 pre-existing `CA1861` warnings; Robotics **467** net10.0 / **447** net48, Robot Intent **20** on both, OpenUSD **904**, AOT **121/121**, source generation **3771**. ### Review round Five reviewers over disjoint areas found defects worth calling out, all fixed in `3ffec2c1`: - **A refused mission step was reported `Succeeded`.** The advance path returned the same `false` for "steps exhausted" and "step refused", so a protective stop between steps told a supervisory system the mission had completed while the robot stood halted with work unexecuted. - **Three attacker-controlled NodeIds reached the executor unvalidated** — `Pick.Tool`, `Place.Tool`, `ToolChange.DockStation`. `DockStation` drives a physical tool-exchange motion to a location the server never authorised. The replacement test enumerates NodeId members by reflection so the next one added cannot be missed. - **A lock-order inversion that loses a stop.** `CanCancel` and the mission condition evaluator ran under the host lock while the executor's own thread re-entered it via `ReportBlendBegin`. - **A non-terminal executor outcome killed the pump** from outside the `try/catch` that exists because the executor is untrusted — `Ready` stayed true and submissions kept being accepted while nothing moved. - **`Pause` published `Suspended` while the robot kept moving.** Now queue-only and honest about it; the specification gained the matching rule ([opcua-drafts `357a558`](https://github.com/marcschier/opcua-drafts/pull/47)). - **`RI-RealTimeChannel` and `RI-Safety` were false claims** — granted on an empty folder with no methods, and on the mere presence of a Mandatory node. - **The Part 10 promotions never promoted.** Namespace-1 BrowseNames declared second members beside the inherited ones; the conformance test could not see it because its lookup stripped the prefix. Fixed upstream and regenerated. - **Renderer picking never worked.** The reflective probe bound a renderer that is not a picking backend and throws off its owner thread. It now binds only `IRenderPickingBackend`, falls back immediately and loudly, and the docs say plainly that renderer picking is unavailable with this package version. - **The client lost command authority the instant it gained it** — monitored items report the current value first, and the lease treated every notification as a loss. ### CI stability fixes for master-side failures The CI split in #4158 turned on legs that had never run, and two of the failures they exposed were real bugs on `master` rather than anything this branch introduced. Both are fixed here because they were blocking every branch: - **ECC key generation was broken on macOS.** `ExportExplicitParameters` throws `PlatformNotSupportedException` there — Apple's SecurityTransforms supports named curves only — and it was called for exactly one value, the curve order `n`. The four curves OPC 10000-12 admits have published orders, so those are carried directly, with `CurveOrderTableMatchesThePlatform` checking each against the platform's own exported order wherever export works, so the constants cannot silently drift. This accounts for the two ECC failures in `test-macOS-latest-Server`. - **The certificate-leak detector raced asynchronous disposal.** It polled a fixed budget and reported whatever it saw, which cannot distinguish a genuine leak (count steady) from a suite still draining fire-and-forget disposals (count falling) — on a loaded agent that produced `Certificate leak detected: 2 instance(s)` with every test passing. It now waits for the count to **settle**, so a real leak is reported sooner than before while a slow drain gets as long as it keeps progressing. Shared by every suite that asserts no certificate leaks. The remaining macOS failures on `master` (`Gds`, `Tools`, `Redundancy.Samples`, `aot-macos`) have different root causes, are not reproducible without a macOS host, and are **not** addressed here. ### Merged with `master` again `master` through `416d619c4` (#4158, the CI matrix split and graduated coverage bands) is merged in as `08ae4528`. One conflict, in `docs/DeveloperGuide.md`, where both sides had deliberately rewritten the coverage-gate section: - `master`'s rewrite is kept in full — graduated `patch.bands`, Codecov being informational-only, and where the numbers surface per CI system. - This branch's paragraph about the `ignore` globs is kept too, re-sited ahead of those subsections. I re-checked it against the merged `coverage-thresholds.json` rather than assuming it still held: the list is still `tests/**`, `samples/**`, `obj`, `bin`, `*.g.cs`, with **`tools/**` absent** — so the warning that anything under `tools/` is measured like product code, and that an assembly no test project references contributes changed lines counted as *uncovered*, is still accurate and worth keeping. - `master`'s "reproduce a coverage failure locally" line supersedes this branch's near-identical one, so only `master`'s survives. Post-merge: `dotnet build UA.slnx -t:Rebuild` **0 errors**, 2 pre-existing `CA1861`; Robotics **467** net10.0 / **447** net48; Robot Intent **20** on both. ### Getting CI green The pipeline had never actually run on this branch until now (it was waiting on the fork-PR approval gate), so the first run surfaced four real problems, each fixed at source: - **Seven build legs failed** because `IntentEnabledRobot` pins `net10.0` for the OpenUSD payload but never opted into `RestrictForLegacyTfm`. On the legacy legs the libraries build as net472/net48, where `Opc.Ua.Types` exposes the `System.Threading.Lock` polyfill publicly, and that collides with the sample's own net10.0 BCL (`CS0433`). `Opc.Ua.OpenUsd.Connector.Viewer` was already using the opt-out for the same reason. - **A Linux-only test failure** turned out to be a real defect, not a platform quirk: `SetPose` published the *commanded* interpolated pose beside joint frames derived from the *solved* joints, so the two disagreed by the inverse-kinematics residual — about 8 mm, constant after any real motion. That is the lying-digital-twin defect one level deeper than the per-step IK fix reached. `SetPose` now derives both from the joint angles. - **Three source-generator tests** asserted the old emitted shape. The clause 6.2 Null-Variant fix was implemented in `ObjectTypeProxyGenerator` so every generated proxy benefits, which changed that shape; the tests are updated and strengthened to pin the `IsNull` branch as well. - **A CI-only timeout** in the simulated arm drain: the executor runs on a manual clock so the tests do not depend on machine speed, but the drain guarded itself with a wall-clock deadline, which put that dependence back. CI runs with coverage instrumentation, which is enough to take a per-step IK probe from ~0.6 s to over 5 s. The guard is now a simulated step budget. One failure was **not** from this PR and did not recur: a Roslyn `AccessViolationException` compiling `Opc.Ua.ISA95` on one leg, while the Debug leg of the same target passed on identical code and the Release build succeeds locally. ## Related Issues - Relates to #3827 — the request that prompted the robotics intent work. - Builds on #4127. The companion specification is drafted in the open at [marcschier/opcua-drafts#47](https://github.com/marcschier/opcua-drafts/pull/47) (`metaverse-specs/robot-intent/`), with the prior art and the reasoning behind each decision in the research document beside it. **Nothing in it is normative or endorsed by the OPC Foundation, and its NodeIds and namespace URI are provisional** — which is the main reason this is a draft. ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [x] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. On the test checkbox, precisely: the whole `UA.slnx` suite now runs on net10.0 with **50 of 52 assemblies fully green**, and `Opc.Ua.Robotics.Tests` passes on **all five** target frameworks including net472, net8.0 and net9.0 which had never previously been exercised. The two remaining failures are reproduced identically at the merge-base and are detailed above, so I have ticked the box; if you would rather it stayed unticked until those two are green, say so and I will revert it. ## Points I would most like feedback on 1. **The namespace and NodeIds are provisional.** They come from a draft specification, not from the OPC Foundation. If this lands before the model is registered, the identifiers will move. 2. **The model lives in `Opc.Ua.Robotics`.** It is standalone on the base UA namespace and takes no dependency on OPC 40010, so a separate `Opc.Ua.RobotIntent` package would arguably be cleaner. Keeping it here was a deliberate call — this is where the robot-facing API already is — but it is the easiest thing to change now and the hardest later. 3. **Removing the convention API from #4127 is a breaking change** for anyone who adopted it. It was explicitly marked non-normative, so I have assumed that is acceptable — please say if a deprecation period is wanted instead. 4. **Serial execution** is a deliberate simplification. Concurrent `None`/`Soft` intents would be an optimisation, not a correction, but it is worth agreeing that reading. 5. **`samples/Robotics/` reorganisation** moves `MinimalRobotServer`. Happy to split that into its own commit if it makes review easier. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 1dab1302-19c5-4a9b-a50c-d97d389713aa Copilot-Session: a248a589-9a20-4372-868e-5d347e57001b
20 KiB
OPC UA — OpenUSD
This guide covers the client- and server-side libraries that bridge an OPC UA address space to an OpenUSD stage. For the OPC 40010 Robotics companion SDK used to label and drive robot-cell twins, see Robotics.
The work comes in two parts, covered here in turn. Part 1 — the binding model — keeps the USD scene outside OPC UA and binds process values to it. Part 2 — scene materialization — puts the scene inside the address space, so the composed prim tree becomes the node hierarchy and browsing the server is browsing the scene. The two interoperate — a Part 1 binding may target a Part 2 attribute Variable — but neither requires the other.
| Part 1 — bindings | Part 2 — scene materialization | |
|---|---|---|
| USD scene lives | outside OPC UA | inside OPC UA |
| OPC UA carries | which prim/attribute a value maps to | the prims and attributes themselves |
| Consumer | a connector that writes an external stage | a client that browses/subscribes, or exports .usda |
Both companion models are draft (experimental) models. The type NodeIds and the
Server/OpenUSD/Representationsregistry described here are subject to change until the companion specifications are ratified.
Libraries
| Package | Role |
|---|---|
OPCFoundation.NetStandard.Opc.Ua.OpenUsd |
The draft OpenUSD-binding companion model (source-generated NodeStates). |
OPCFoundation.NetStandard.Opc.Ua.OpenUsd.Client |
The generic, domain-agnostic OpenUsdConnector, the IUsdSink abstraction, and the file/mock sinks. |
OPCFoundation.NetStandard.Opc.Ua.OpenUsd.Server |
Server-side authoring helpers (UsdAssetDelivery, representation authoring). |
OPCFoundation.NetStandard.Opc.Ua.OpenUsd.Connector |
A ready-to-run console connector tool built on the client library. |
OPCFoundation.NetStandard.Opc.Ua.OpenUsd.Connector.Viewer |
Optional viewport for the connector's --view option. Renders the composed stage and exposes it as an IUsdSink. |
OPCFoundation.NetStandard.Opc.Ua.OpenUsdScene |
Part 2: the source-generated companion model, the scene document model, the .usda reader/writer, and the value-type map. |
OPCFoundation.NetStandard.Opc.Ua.OpenUsdScene.Server |
Part 2: materializer, exporter, discovery, and Part 1 interop. |
The connector
OpenUsdConnector is a client: it discovers a server's OpenUsdRepresentation instances through the Part 1
Server/OpenUSD/Representations registry, subscribes to the bound source Variables, applies the declared conversion,
and writes the target USD attributes into an IUsdSink. It is domain-agnostic — it knows only the OpenUSD binding
model, never "pump" or "robot".
Progressive API
The simplest usage needs only a connected session and a sink:
using Opc.Ua.OpenUsd.Client;
// session is a connected ISession (for example a ManagedSession).
var sink = new UsdFileSink("live.usda");
await using var connector = new OpenUsdConnector(session, sink);
await connector.StartAsync(cancellationToken); // discover + subscribe + compose
// ... the sink now receives live updates ...
await connector.StopAsync(cancellationToken); // stop streaming
OpenUsdConnector implements IAsyncDisposable; await using (or an explicit DisposeAsync) stops streaming and
closes any connector-owned remote sessions opened for cross-server federation. The caller-provided primary session is
never closed by the connector.
Advanced behaviour is configured through OpenUsdConnectorOptions:
var options = new OpenUsdConnectorOptions
{
EnableCommands = true, // opt in to UsdToUaCommand actuation (fail-closed by default)
RemoteSessionFactory = (endpointUrl, ct) => OpenRemoteSessionAsync(endpointUrl, ct), // §5.14 cross-server federation
MaxAssetBytes = 32 * 1024 * 1024, // per-asset read cap
MaxTotalAssetBytes = 128L * 1024 * 1024, // per-fetch read cap
};
await using var connector = new OpenUsdConnector(session, sink, options, telemetry);
Values cross the boundary as Variant
IUsdSink never exposes object. A scalar attribute is a double, a colour is a three-element float array
(ArrayOf<float>), and a token/visibility value is a string — all carried as Variant:
public interface IUsdSink
{
void SetAttribute(string primPath, string propertyName, Variant value);
void SetTimeSample(string primPath, string propertyName, DateTime time, Variant value);
void ComposePrim(string primPath, OpenUsdCompositionArc arc, string? assetReference, bool active);
IDisposable BeginBatch();
}
Two sinks ship in the box:
UsdFileSinkauthors a text USD override layer (live.usda). It validates every prim-path segment and property name as a USD identifier, escapes token values, and rejects unsafe asset references, so a hostile or malformed name from the server cannot corrupt or inject into the layer.MockUsdSinkis an in-memory, thread-safe sink used by tests and diagnostics.
BeginBatch() lets a file-backed sink defer flushes; history replay uses it to author many time samples with a single
file write.
History replay and commands
ReplayHistoryAsync(startTime, endTime, ct)replays Part 11 history for everyUaHistoryToUsdbinding, following continuation points and authoring the returned values as USD time samples. Sources that do not historize degrade to zero samples without throwing.IssueCommandAsync(value, ct)actuates the opt-inUsdToUaCommandbinding. It is fail-closed: it throws unless the connector was constructed withEnableCommands = true.
Integrity
When a stage advertises a RootLayerDigest, the connector verifies it (constant-time) before authoring any opinions,
and refuses to compose on a mismatch. Served asset closures (FetchServedAssetsAsync) are streamed through the Part 5
FileType, digest-verified fail-closed, and cached under sanitized relative paths (path-traversal is defended).
Dependency injection
Both a standalone IServiceCollection extension and the fluent IOpcUaClientBuilder extension register the singleton
OpenUsdConnectorFactory; the direct constructors remain available as a non-DI fallback.
// Standalone.
services.AddOpenUsdConnector(o => o.EnableCommands = true);
// Fluent, chained onto the client builder.
services.AddOpcUa()
.AddClient(configuration)
.AddOpenUsdConnector(o => o.MaxAssetBytes = 32 * 1024 * 1024);
Resolve the factory and create a connector per connected session:
public sealed class TwinWorker(OpenUsdConnectorFactory connectors)
{
public async Task RunAsync(ISession session, IUsdSink sink, CancellationToken ct)
{
await using OpenUsdConnector connector = connectors.Create(session, sink);
await connector.StartAsync(ct);
// ...
}
}
Observability is threaded through ITelemetryContext (resolved from DI), which the factory passes to each connector.
Server-side authoring
context.CreateRepresentation(owner, stage, primPath, openUsdNs) attaches an OpenUsdRepresentation to a domain
Object and points it at a stage and prim path. The representation is an AddIn, so it is mounted with
HasAddIn — not plain HasComponent. The distinction is easy to get wrong because HasAddIn is a subtype of
HasComponent: an AddIn mounted with the wrong reference type still browses, still aggregates, and still works
end to end, so only a conformance checker notices. Prefer this helper over hand-rolling the mount; the sample
servers assert the reference type in their E2E suites.
UsdAssetDelivery.AttachStageAssets(context, stage, openUsdNs, assets) serves artist-authored USD layers through the
address space as read-only Part 5 files with SHA-256 digests, so a generic connector can fetch, verify, cache, and
render the twin with no external asset resolver. assets is an ArrayOf<ServedAsset> and the method returns the
created OpenUsdAssetState nodes as an ArrayOf<OpenUsdAssetState>.
Samples
MinimalRobotServer— a self-contained server exposing an OPC 40010 MotionDeviceSystem with two independently mobile robots. OPC 10000-210 RSL frames drive livedouble3translation/rotation, and OPC 10000-211 GPOS locations drive geospatial metadata; a generic connector renders the cell live. See Positioning.- The Robotics samples also include a Robot Intent flow where an OpenUSD viewport prim pick becomes a robot command
through
UsdViewOptions.PrimPicked. See Robot Intent and the Robotics samples. PumpDeviceIntegrationServer— a DI pump line bound to OpenUSD, including component composition and served-asset delivery.GeneratorServer— the Generators companion specification with a datasheet-driven simulation and one independent twin per configured generating set.SiteCompositionServer— a supervisory server that owns no devices and composes the machines of the pump and generator servers into a single scene through cross-server components. Render it with the connector's--federateoption. See Samples.
The connector tool
Opc.Ua.OpenUsd.Connector is a console application that connects to any server implementing the draft binding,
discovers Server/OpenUSD/Representations, subscribes, optionally fetches the served asset closure, and authors a live
live.usda override layer. It is the end-to-end reference for the client library.
dotnet run --project tools/Opc.Ua.OpenUsd.Connector -- \
--server opc.tcp://localhost:62830/MinimalRobotServer \
--fetch-assets ./stage --insecure --seconds 30
| Option | Meaning |
|---|---|
--server <url> |
Endpoint to connect to. |
--out <live.usda> |
Override layer to author. Defaults to live.usda in the working directory. |
--seconds N |
Stop after N seconds instead of waiting for Ctrl+C. |
--fetch-assets <dir> |
Download the served USD layer closure (§5.15) and compose a self-contained stage.usda. |
--insecure |
Demo only: unsecured endpoint and blanket certificate acceptance. |
--enable-commands |
Opt in to actuating UsdToUaCommand bindings (fail-closed by default). |
--command-value <double> |
Setpoint written once at start when commands are enabled. |
--view |
Render the composed stage and stream the same live values into it. |
--renderer <Auto|Storm|D3D12|Vulkan> |
Renderer preference for --view. |
--stage <stage.usda> |
Render an existing local stage instead of a fetched one. |
--plugins <dir> |
Directory holding the staged USD plugin tree, when it is not beside the connector. |
--pick-command [<prim path>] |
With --view, print picked target prim paths. |
Rendering the twin live
--view opens a viewport on the composed stage and fans every subscribed value into both the override layer and
the stage being rendered, so the twin animates in one process:
dotnet run --project tools/Opc.Ua.OpenUsd.Connector -- \
--server opc.tcp://localhost:62830/MinimalRobotServer \
--insecure --view
Without --fetch-assets or --stage, --view fetches the asset closure into a temporary directory, because a
renderer needs geometry it can resolve. Closing the window stops the session; --seconds closes it automatically.
The renderer itself lives in a separate, optional assembly, Opc.Ua.OpenUsd.Connector.Viewer
(OPCFoundation.NetStandard.Opc.Ua.OpenUsd.Connector.Viewer), which the connector loads on demand from its own
directory. That keeps the connector package free of a UI framework and a native OpenUSD payload, and lets it keep
targeting .NET Framework. When the assembly is absent, --view explains how to install it and every other option
keeps working.
Internally the viewport supplies a sink that authors into the scheduler-owned stage the renderer already owns — the
connector never opens the stage a second time. CompositeUsdSink fans values out to that sink and to UsdFileSink,
so the on-disk artefact and the picture never diverge.
The connector client API also exposes UsdViewOptions.PrimPicked, a callback that receives picked USD prim paths.
Renderer-backed pointer picking works: the OpenUSD viewer owns input handling, DPI scaling, physical-pixel conversion
and stale-revision retry, and reports hits through the host callback. Auto uses the renderer first and falls back to
the command-prim watcher only when renderer picking is unavailable, Renderer requires renderer-backed picks, and
CommandPrim uses the fallback directly. Misses do not submit intents. For the fallback, set a targetPrim
relationship, string attribute, or token attribute on UsdViewOptions.CommandPrimPath (default /World/IntentCommand)
and the callback fires when that target changes.
The gaps this used to work around were tracked in marcschier/openusd-dotnet issues #1, #5, #8, #9, #10 and #11, all
of which are fixed in 0.5.0-alpha.
The viewport requires .NET 10 and the OpenUSD packages (
OpenUsd,OpenUsd.Viewer,OpenUsd.Runtime.Imaging), which are published on nuget.org, so a plain restore is enough. The RID-agnostic runtime metapackages resolve the correct native payload per RID;win-x64,linux-x64andosx-arm64are all supported. Publish the connector and the viewport into the same directory, substituting your own RID:dotnet publish tools/Opc.Ua.OpenUsd.Connector -c Release -f net10.0 -r win-x64 --self-contained false -o out dotnet publish tools/Opc.Ua.OpenUsd.Connector.Viewer -c Release -r win-x64 --self-contained false -o outPublishing both into the same directory is what puts the optional assembly, its dependencies, and the native plugin tree where the connector looks for them.
Viewport colour, materials and cameras
Current OpenUSD packages support the live viewport features the connector samples rely on:
primvars:displayColoris authored through the managedcolor3f[]API, so DisplayColor bindings animate in the viewport instead of only appearing in the override layer.- Bound
UsdPreviewSurfacematerial networks are shaded, so material colour inputs can be used for scalar shader colour targets without requiring a duplicate displayColor fallback. - Authored stage cameras are opened automatically, so samples can frame their intended operator view.
Visibility bindings remain a good fit for binary state such as run lamps, alarm beacons and fault halos. Use them when an on/off condition should be unmistakable; they are no longer needed as a workaround for colour or material limitations.
Part 2 — scene materialization
Where the binding model above keeps the USD scene outside OPC UA, scene materialization puts the composed prim tree inside the address space: browsing the server is browsing the scene, and an ordinary Subscription on an attribute Variable is a live feed of that part of it.
Materializing a stage
UsdStage stage = UsdaReader.ReadFile("Plant.usda");
FolderState stages = context.EnsureStagesFolder(serverObject, sceneNamespaceIndex);
UsdMaterializationResult result = context.MaterializeUsdStage(stages, stage, sceneNamespaceIndex);
MaterializeUsdStage walks the composed prim tree depth first and creates, for each prim, a
HasComponent child typed by its USD typeName; for each attribute a UsdAttributeType
Variable; and for each relationship a UsdRelationshipType Object with ordered Targets and
TargetPaths.
The result carries two indexes — PrimsByPath and AttributesByPath — so a caller can bind live
data to a materialized attribute without re-browsing.
Nothing is ever dropped
The specification is emphatic that an importer must not discard what it does not understand (§8.4), because an exporter has to be able to reproduce it. So:
- an unknown typed schema stays a
UsdPrimTypecarrying itsTypeNametoken; - an unknown API schema degrades to a generic
UsdApiSchemaTypeAddIn carrying itsSchemaName; - an unknown value type is carried opaquely as
BaseDataType, with the exactSdfValueTypeNamepreserved inUsdTypeName.
Types carry USD roles
USD's color3f, normal3f, point3f and vector3f all decompose to three floats and differ
only by role. Rather than flatten that away, the model gives each its own DataType that
subtypes the built-in — the same idiom the OPC UA standard uses for Duration : Double or
UtcTime : DateTime.
Because the role type subtypes a built-in, the value bytes are unchanged: a generic client reads
a Float[3] exactly as before, while a renderer or material editor can tell a colour from a point
straight from the type system instead of parsing a string. UsdValueTypeMap implements the
mapping, and the attribute's UsdTypeName keeps it reversible even where several USD types share
one OPC UA DataType.
Live attributes
A materialized attribute is an ordinary Variable, so USD's static/time-sampled duality maps onto the OPC UA value surface in two modes a Server may mix per attribute:
- Mode A — live. The Value is server-maintained and time-varying; a Subscription delivers
changes and, where retained,
HistoryReadexposes the timeline — the OPC UA counterpart of USD time samples. RotatingxformOps and process-driven attributes use this mode. - Mode B — static. The Value is the authored default.
Driving Mode A is a Server-side responsibility. An external Part 1 connector authors into a USD sink and cannot write an in-server Variable; a Part 1 binding declares the mapping, and the Server (or a server-hosted connector) applies it.
Georeferencing
Core OpenUSD has no geodetic schema, so a georeferenced stage is expressed today through vendor extension schemas (Cesium for Omniverse, NVIDIA's geospatial schema). Those materialize through the ordinary vendor-extension mechanism — a georeference prim type as an ObjectType subtype, an anchor API schema as an AddIn.
To give a client something portable, the model also defines UsdGeoreferenceApiType (a
stage-level origin: latitude, longitude, height, EPSG code, tangent plane) and
UsdGlobeAnchorApiType (a per-prim geodetic position). When the materializer recognises a Cesium
schema it dual-authors the portable one with the same values, so a generic client reads the
anchor from one well-known type while a vendor-aware client still reads the native schema.
A partial anchor is never published: a wrong position is worse than no position.
Conversion and the round-trip contract
UsdaReader and UsdaWriter convert between .usda and the scene document model, and
UsdSceneExporter reads a materialized address space back out, so the full round trip is
.usda → UsdStage → address space → UsdStage → .usda
The contract (§7.4) is composed-scene lossless: the exported stage is prim-for-prim,
attribute-for-attribute (name, SdfValueTypeName, resolved value and array shape),
relationship-for-relationship (ordered targets), metadata-, variant-selection-, kind- and
specifier-equivalent to the input's composed result, and the recorded composition arc list is
preserved.
It is deliberately not authoring-layer lossless: the input's per-layer opinion stack, sublayer
structure and value clips are summarised as provenance under Composition/ and
RootLayerIdentifier rather than reproduced layer by layer. UsdSceneSignature computes the
equivalence used to check the contract.
Discovery and Part 1 interop
EnsureStagesFolder returns the folder stages are organized under: Part 1's
Server/OpenUSD/Stages when the bindings model is also implemented — so one connector discovers
both external-stage bindings and in-server materialized stages — or a standalone
Server/OpenUSDScene/Stages otherwise.
A Part 1 live binding may target a Part 2 attribute Variable. Part 1 ≥ 0.3.0 carries the target two ways, and a Server should author both so that path-resolving and NodeId-resolving connectors agree:
NodeId target = result.ResolveBindingTargetNodeId(
"/Plant/Pumps/Pump_1/Pump/Impeller", "xformOp:rotateZ");
Conformance units
UsdMaterializationOptions maps one-to-one onto the specification's conformance units, each
independent and additive. Scene Structure is the baseline; the rest are opt-in:
composition provenance, typed schemas, applied schemas, georeferencing, live attributes,
conversion, and Part 1 interop.