mirror of https://github.com/OPCFoundation/UA-.NETStandard.git synced 2026-10-17 05:11:13 +08:00

Files

T

Marc Schier 466c5cb982 Robot intent follow-ups: multi-controller executors, spec fixes, and an MCP tool library (#4195 )

Follow-ups to #4165, plus the work that followed from re-checking this
branch against the specification. Everything started as a defect found
by review or by probing the running code; the specification is unchanged
except where noted.

The TLS certificate lifetime fix (#4196) and the OpenUSD work (#4198)
were both merged into this branch. #4196 has since landed on master, so
it no longer appears in this diff. #4198 was closed in favour of this PR
and its content is here.

The MCP trim/AOT work (#4228), the `DataGenerator` null `NodeId` fix
(#4227) and the NodeSet external-parent import fix (#4229) have since
been merged in as well. All three are closed in favour of this PR and
their content is here, described under *Making the MCP libraries trim
and AOT clean*, *A generator that returned the value you opted out of*
and *A NodeSet parent that lived outside the import batch* below.

## A two-arm cell could not be built

`IntentExecution` carried no controller identity, and there was
effectively one `IIntentExecutor` per server. An executor handed an
execution therefore could not tell **which arm** it was for, so a cell
with two `IntentControllerType` instances — an ordinary configuration —
was not expressible through the builder or DI.

Both halves are addressed:

- `IntentExecution` now carries `ControllerId` (`NodeId`, the OPC UA
identity) and `ControllerName`, so one shared executor can discriminate;
- DI gained per-controller registration —
`AddRobotIntentExecutor<TExecutor>(controllerBrowseName)` and an
instance overload — so two genuinely different executors can coexist.

Resolution order is `WithExecutor` → per-controller DI registration →
the single DI executor → a build-time throw. There is still **no silent
fallback** to a rejecting executor; that was a Critical defect fixed in
#4165 and it stays fixed. `IIntentExecutor` is unchanged, so this is
purely additive.

## Declarations that were quietly untrue

- **`Accepts<T>` silently discarded a redeclaration.** Declaring an
intent type twice — say to narrow `cancelSupported` on the second call —
kept the first, wider declaration and said nothing. That is the same
class of defect as the capability-honesty problems #4165 fixed
(`PauseSupported` declaring a pause the host could not perform;
`Aborting` degrading silently to `Buffered`). Identical redeclarations
remain idempotent; conflicting ones now throw `BadInvalidArgument`.
- **`RI-Interop-40010` assumed Annex B item 3 rather than checking it.**
It now requires exact set equality between the OPC 40010
`TaskProgramName` values and the Robot Intent `ProgramType.ProgramId`
values. Both sides are readable from the address space without
commanding the robot, which is what makes the requirement structural
under §12.1.

## Two things the specification requires that this did not do

Re-checking the implementation against the merged draft revision turned
up two genuine violations. Both are fixed, and the NodeSet is untouched
by either.

- **Every graphed mission reported `Failed` on success.**
`MissionRules.SelectTransition` returned `null` for two different
situations — a graph *terminus*, which is the normal success path, and a
branch point whose conditions all evaluated false — and the caller
collapsed both into `Failed`/`NoTransition`. A three-way result
(`Terminus` / `Selected` / `NoConditionMatched`) separates them, so a
mission that runs to its end now completes. The failure reason surfaces
through the inherited Part 10 result container rather than a new member.
- **Facet URIs were never published.** An upstream `may` became a
`shall` in the revision this branch adopts.
`RobotIntentConformanceUris.Facets` held 31 URIs with **zero**
production references — the set existed but nothing emitted it. The
published set is now derived from `RobotIntentFacetCalculator`, the same
authority that already computed `SupportedFacets`, rather than from a
second hand-maintained list that could drift; a reflection test pins the
two together.

## An easy client mistake

`RequestAuthorityAsync` returns a lease the caller must remember to
inspect. Forgetting makes every later submission fail with
`ControlNotOwned` for reasons that are invisible at the call site.
`RequireAuthorityAsync` throws instead, and names the current owner.
`RequestAuthorityAsync` is untouched — inspecting and branching is
legitimate.

## Letting an agent drive the robot

New `tools/Opc.Ua.Mcp.Robotics` gives a language model the intent API as
MCP tools: 36 across discovery, monitoring, direct control and missions,
behind a `Robotics` tool profile, composable into any MCP host exactly
as the existing OPC UA tool packages are. The `IntentViewerClient`
sample hosts them behind `--mcp` (stdio by default, HTTP when `--view`,
since MCP stdio and the in-process viewer would otherwise share stdout),
and the sample gained a pallet scene so an agent can pick and stack
parts and watch the result in the OpenUSD viewer. All the scene logic is
server-side — no viewer or connector changes.

Four gaps in the client API had to be filled first, since the tools are
a thin layer over it and could not have been written honestly without
them: `ReadStateAsync`, `ListOperationsAsync`/`ListMissionsAsync`,
controller-level
`CancelIntent`/`CancelAll`/`Pause`/`Resume`/`Retry`/`ReleaseControl`,
and `WaitForCompletionAsync`.

The tools refuse rather than paper over: a refusal from the server is
returned with its `IntentFailureEnum` and never retried, and command
authority is never acquired as a side effect of a tool call.

**One defect this surfaced:** `--profile robotics` shipped 36 tools that
could not do anything. Every robotics tool resolves a named OPC UA
session and only the connection tools can open one, so with no `Connect`
in the catalogue the whole profile failed. A test had pinned the defect
in place, asserting the profile did *not* contain `Connect` on the line
below the `diagnostics` assertion requiring it. Each package now
declares its own requirement in its registration extension rather than
the host special-casing it, so an application embedding
`Opc.Ua.Mcp.Robotics` directly gets a usable catalogue without
reproducing the host's pairing. `Full` still takes the connection tools
from the core package, so they are never registered twice. The
replacement test states the invariant — a profile exposing
session-scoped tools must expose the tools that open a session — rather
than the instance.

## Making the MCP libraries trim and AOT clean

*Merged from #4228.*

`Opc.Ua.Mcp.Core` is now `IsAotCompatible` on `net10.0`, and the
reflection the analysers then reported is gone. Turning the analysers on
is the point: until now they were not running on these libraries at all
— including on the tool library this branch adds.

- **`OpcUaJsonHelper.Serialize`** went through `JsonSerializer` (IL2026
/ IL3050). The conversion helpers in that class already reduce every OPC
UA value to a closed set — `null`, the primitive scalars, `string`,
`Dictionary<string, object?>` and lists thereof — so the JSON is now
written directly with a `Utf8JsonWriter`.

Output is unchanged, and that is asserted rather than assumed:
`OpcUaJsonHelperTests.SerializeMatchesTheReflectionBasedSerializer`
compares the new writer byte for byte against `JsonSerializer` for every
shape the helpers emit, including all numeric widths, the
custom-formatted scalars, nested sequences and strings that force the
encoder to escape.

One deliberate contract change: arbitrary object graphs are no longer
accepted. Serializing them needs reflection, so `Serialize` now throws
`NotSupportedException` naming the type instead of silently stringifying
it. No call site in the repository passes anything outside the closed
set.

- **`McpSchemaFilters`** used `JsonSerializer.SerializeToElement` on a
`JsonObject`; it now parses the node's own text, which needs no
reflection.

`Opc.Ua.Mcp.Diagnostics` is deliberately **not** marked. It depends on
SharpPcap and on the reflective service-call dissection in
`Opc.Ua.Core.Diagnostics`, neither of which is trim safe — which is why
that package does not opt in either. Two improvements are kept
regardless: `PacketDecodeTools` serializes `DecodedServiceCall` through
a source-generated `JsonSerializerContext`, and
`PacketCaptureTools.ListInterfaces` now carries the
`RequiresUnreferencedCode` / `RequiresDynamicCode` annotations that
`NetworkInterfaceEnumerator.ListLocalInterfaces` already declares, so
the constraint is visible to callers instead of implicit. Nothing is
suppressed.

Marking an assembly `IsAotCompatible` only proves its own IL is clean;
it says nothing about whether the library still functions once
reflection is gone. So `Opc.Ua.Aot.Tests` references `Opc.Ua.Mcp.Core`
and exercises it with JSON reflection disabled, stating what is and is
not true today:

- tool **results** serialize without reflection — this would fail
outright if the `Utf8JsonWriter` change regressed;
- tool **registration** still requires reflection, because the MCP SDK
builds each tool's input schema by asking `JsonSchemaExporter` for a
`JsonTypeInfo` of every parameter type. Without a source-generated
context covering the tool signatures — and with the injected
`OpcUaSessionManager` being described rather than treated as a service —
that throws `NotSupportedException`.

The second test asserts the throw. It documents the remaining gap and
turns red the moment someone closes it, rather than leaving it to be
found in a published binary. The package README says the same thing in
prose.

**Two pre-existing issues this surfaced, both outside this PR:** pulling
`Opc.Ua.Mcp.Diagnostics` into an AOT publish reports `IL2046` / `IL3051`
on `NicCaptureSource.StartAsync`, which is annotated
`RequiresUnreferencedCode` / `RequiresDynamicCode` while the
`ICaptureSource.StartAsync` it implements is not, and `IL2075` on
`ServiceCallReassembler.GetCollectionCount` / `GetPropertyString`, which
call `Type.GetProperty` on the result of `GetType()`. Happy to file
these separately if useful.

## A generator that returned the value you opted out of

*Merged from #4227. Fixes #4226.*

`GetRandomNodeId(useBoundaryValues: false)` could still return
`NodeId.Null` — one of the boundary values the same method offers when
boundary values *are* requested. A caller that opted out should not
receive one, and `GetRandomExpandedNodeId` builds on it and inherited
the same behaviour.

The `IdType.Opaque` branch is the practical path:
`GetRandomByteString()` draws its length with
`NextInt32(MaxStringLength)`, which is inclusive of zero, and a
zero-length opaque identifier in namespace 0 satisfies `NodeId.IsNull`.
With three namespaces and `MaxStringLength` at 10 that is roughly **one
call in 130**. A numeric identifier of `0` in namespace 0 is null too,
at around one in four billion.

The draw is now repeated while the result is null and boundary values
were not requested. Guarding the result rather than the individual
branches covers every identifier type, including the numeric case,
rather than only the one that happens to be reachable today. The
`useBoundaryValues: true` path is untouched, so boundary values remain
available when they are asked for.

Four tests in `tests/Opc.Ua.Core.Tests/Types/DataGeneratorTests.cs` loop
20,000 draws, because at one call in 130 a single draw proves nothing.
Against the **unmodified** generator they fail on draw 93 and draw 1346.
Two of the four exist to keep the fix honest rather than to test the
bug: `GetRandomNodeIdStillProducesEveryIdentifierType` catches a "fix"
that simply stopped emitting opaque identifiers, which would satisfy the
null assertions while silently narrowing the generator, and
`GetRandomNodeIdWithBoundaryValuesStillYieldsThem` confirms the leak was
closed rather than the feature removed.

**How it was found:** an intermittent downstream test failure. A test
server initialising a static `NodeId`-valued variable via
`TestDataSystem.ReadValue` occasionally stored `NodeId.Null`; the
encoder correctly writes that as JSON `null`, and a consumer asserting a
value was present failed about one run in five — never reproducibly in
isolation. It only surfaced under load because `UnsecureRandom.Shared`
is a fixed-seed, process-global generator shared between `DataGenerator`
and OPC UA transport jitter, so the number of draws preceding
address-space initialisation depends on what else is running in the
process.

## A NodeSet parent that lived outside the import batch

*Merged from #4229.*

`UANodeSetHelpers.LinkParentChildRelationships` builds its lookup table
from **only the nodes in the current import batch**, links a child when
the parent is found — and then cleared `instance.Handle`
**unconditionally**, which is exactly where `Import` had stashed the
authored `ParentNodeId`. A node whose parent lives in another NodeSet,
or is owned by another NodeManager, therefore lost that parent
**silently** at import, and nothing downstream could recover it.
`RuntimeNodeSetNodeManager.CreateAddressSpaceAsync` already has a step
that wires references to external node managers; the parent link never
reached it.

Such a parent is now recorded rather than discarded, and exposed through
`UANodeSet.TryGetUnresolvedParentNodeId`. A parent that resolved
normally inside the batch is **not** recorded, so a caller does not
re-wire references that already exist. The record lives in a
`ConditionalWeakTable`, so it does not keep an imported node alive and
does not widen `NodeState` — whose `Handle` is a general-purpose slot a
caller may legitimately use for its own purposes, and so is not a safe
place to leave the value.

`RuntimeNodeSetNodeManager` consumes it to report the one case that
cannot be repaired automatically. In NodeSet2 the `ParentNodeId`
attribute is a *hint*; the authoritative edge is the entry under
`<References>`, which the reverse-reference pass already turns into an
external reference. A node that names an out-of-manager parent but
carries **no such Reference** materializes with no path from that
parent, and no ReferenceType can be inferred to synthesise one — so it
is reported as an authoring defect rather than guessed at or silently
accepted.

This is deliberately **not** a fix for a specific address-space symptom.
It removes a silent loss of authored information and gives callers the
means to act on it. Whether any particular orphaned-node report is
caused by this path, or by namespace-index translation elsewhere, still
needs separate diagnosis.

**Changed while merging.** As authored in #4229 the accessor was
`GetUnresolvedParentNodeId`, returning `NodeId?`. `NodeId` is a readonly
struct implementing `INullable`, so that was a `System.Nullable<NodeId>`
wrapped around a type that already carries its own null representation —
which this repository excludes in favour of an `IsNull` check, and it
asks for `TryGet` over casting on struct types. The tests
correspondingly asserted `Is.Null` against a struct, which the testing
rules also exclude. It is now `bool
TryGetUnresolvedParentNodeId(NodeState, out NodeId)`, yielding
`NodeId.Null` when there is no unresolved parent. The API is unshipped —
it arrives with this branch — so nothing depended on the previous shape.
Both tests were re-verified by mutation after the rewrite: disabling the
record in `LinkParentChildRelationships` still fails
`ImportKeepsAParentThatIsNotInTheSameBatch`.

## Dependencies

- **OpenUSD to `0.7.0-alpha`.** All 11 issues we filed upstream are
fixed, so the workarounds they justified are gone and the parts of the
pump and generator samples that were blocked on them are implemented.
- **ModelContextProtocol to `2.1.0`** across all five MCP tool libraries
and the executable. No code changes were needed — the surface we use is
unchanged in 2.x — but central package management required a cascade of
pins, each commented where it sits. Both transports were verified
against a running server rather than assumed. MCP 2.x is stateless (no
`initialize`, no `Mcp-Session-Id`); an OPC UA session is application
state addressed by a caller-supplied name, which fits that model but
does assume process affinity, and that assumption is now written down
rather than implied.

## Infrastructure

The `code coverage` gate was measuring nothing on this branch and
reporting success. It is fixed and now enforced, which immediately
exposed real gaps — most usefully `RoboticsIntentJson`, which parses
agent-supplied JSON (untrusted input from a language model) and had no
tests at all. It is now at full line coverage in CI. Patch coverage is
**90.20%** against a 75% floor.

The `netstandard2.1` build leg was also failing. The MCP projects build
as empty shells under a legacy `CustomTestTarget`, and the viewer sample
referenced them on the target framework alone, so it tried to resolve a
net10.0 shell from net8.0. The sample now derives MCP support from
whether those projects are actually being built.

## From review

A review pass over the branch found two real defects, both fixed.
`RoboticsIntentJson` parses JSON written by a language model, so every
malformed shape it can be handed is reachable input; it indexed pose
arrays without checking their length and enumerated properties without
checking their kind, so a short or wrongly-typed array escaped as
`IndexOutOfRangeException` or `InvalidOperationException` rather than
the argument error the tool descriptions promise the agent. Separately,
`IntentOperationHandle` treated a zero `QueuePosition` as "not observed"
and discarded it, so a handle kept reporting the last queued position
after the server had said the operation left the queue.

The pose validation immediately caught a latent test defect: the pallet
scenario interpolated doubles into JSON with the current culture, so
under a comma-decimal culture `[0.12, 0.16, 0.28]` became a six-element
array. It passed only because that submission is expected to be refused
for want of authority, so the wrong coordinates were never used.

A third review finding — that a default-constructed
`RobotIntentControllerState` hands out a null `NodeId` — was checked and
rejected: `NodeId`, `QualifiedName` and `LocalizedText` are readonly
structs here, so `default` is the null value and `IsNull` is `true`. The
tests asserting that invariant are kept.

A second review pass over the merged branch found two more, both fixed
and both about an error surfacing as the wrong kind of failure rather
than about a wrong result:

- **Malformed agent JSON did not arrive as an argument error.**
`RoboticsIntentJson` turns every *validation* failure into an
`ArgumentException` — a wrong root kind, a missing property, a short
pose array — but three `JsonDocument.Parse` calls and the NodeId parse
were unguarded. A syntax error escaped as `JsonException` and an
unparseable NodeId as `ServiceResultException`, which are the two shapes
a language model is most likely to produce. That is the same defect
class as the pose/kind checking above: a caller separating bad input
from a genuine fault had to know to catch three unrelated exception
types. Parsing now goes through one helper that reports a syntax error
as an `ArgumentException` naming the argument, keeping the
`JsonException` as the inner exception so the position is not lost, and
the NodeId goes through `NodeId.TryParse` rather than `Parse`,
collapsing its two failure modes into one.
`MalformedIntentJsonIsRejected` had asserted
`InstanceOf<JsonException>`, pinning the behaviour rather than the
intended contract; it now asserts `ArgumentException` like every
neighbouring test, and three tests cover the newly guarded paths.
- **`ConditionalWeakTable.Add` throws on a duplicate key.** Not
reachable today, but `TryGetUnresolvedParentNodeId` documents `Handle`
as a general-purpose slot a caller may use for its own purposes — and a
caller doing exactly that before re-importing into the same collection
would take a hard throw from inside the import pipeline. Removing first
makes the most recent import win. `AddOrUpdate` would say this more
directly but does not exist on `netstandard2.0`.

## Deliberately not fixed here

`FastenIntentDataType.Joint` is refused when the Server exposes no OPC
40450/40451 model, and a client cannot discover that before calling.
That is **not fixable in this repository**: §5.4.2 makes `Joint` the
defining mechanism while §11.3 says the parameters otherwise stand
alone, and `IntentCapabilityDataType` declares capabilities at
intent-type granularity with no member-level mechanism anywhere in the
model. It is a specification gap, raised as S4 on
[marcschier/opcua-drafts#60](https://github.com/marcschier/opcua-drafts/pull/60)
rather than worked around with a bespoke mechanism the spec does not
define.

## Validation

`dotnet build UA.slnx -c Release` on the merged branch — **0 errors, 0
warnings**, with the AOT analysers now running over `Opc.Ua.Mcp.Core`.

| Suite (net10.0) | Tests |
|---|---|
| `Opc.Ua.Core.Tests` | 4247 (86 skipped) |
| `Opc.Ua.Robotics.Tests` | 540 |
| `Opc.Ua.OpenUsd.Tests` | 909 |
| `Opc.Ua.Tools.Tests` | 419 |
| `Opc.Ua.Server.Tests` — `RuntimeNodeSet` | 94 |
| `Opc.Ua.Robotics.Intent.Tests` | 22 |
| `Opc.Ua.Aot.Tests` — `McpAotTests` | 3 |

0 failures. `Opc.Ua.Tools.Tests` rose from 409 to 419 with the
serializer-equivalence tests from #4228 and the malformed-JSON tests
from the second review pass, and `Opc.Ua.Core.Tests` covers the
`DataGenerator` tests from #4227 and the import tests from #4229.
`UANodeSetHelpers` also passes on **net48** (5/5). The MCP
agent-usability grader scores Core 100 / Full 100.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot-Session: a248a589-9a20-4372-868e-5d347e57001b
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
Copilot-Session: a0718e8a-a158-45c5-929d-e1d6c0982431

2026-08-11 13:03:36 +02:00

8.1 KiB

Raw Permalink Blame History

Robotics samples

Three runnable samples covering the two robotics companion models this stack implements: OPC 40010 Robotics, which describes a robot, and the draft OPC UA — Robot Intent, which commands one.

Sample	What it is	What it shows
`MinimalRobotServer`	A server for a two-robot cell	OPC 40010 topology, RSL/GPOS positioning, and an OpenUSD representation of the whole cell
`IntentEnabledRobot`	A server for one collaborative arm	The Robot Intent command surface end to end, plus an OpenUSD representation you can drive
`IntentViewerClient`	A client with a 3-D viewport	Clicking a target in the OpenUSD viewport and watching the arm execute the intent

All three are non-production demonstration code. --insecure in the command lines below means an unsecured endpoint with blanket certificate acceptance; it exists so the samples start without a PKI dance and must never be used anywhere real.

`MinimalRobotServer`

A self-contained server exposing a robot cell: two KUKA-style six-axis arms on mobile platforms, their axes, power trains and safety states modelled per OPC 40010, their positions modelled per OPC 10000-210 RSL and OPC 10000-211 GPOS, and the whole thing published as an OpenUSD representation whose assets the server serves over the Part 5 FileType.

dotnet run --project samples/Robotics/MinimalRobotServer -- --host localhost --port 62830

Then render it live with the connector tool:

dotnet run --project tools/Opc.Ua.OpenUsd.Connector -- `
    --server opc.tcp://localhost:62830/MinimalRobotServer `
    --fetch-assets ./stage --insecure --view

See MinimalRobotServer/README.md for the full walkthrough, the address-space layout and the USD binding contract.

`IntentEnabledRobot`

One stationary UR5e-style collaborative arm with an offset wrist — deliberately a different kinematic family from the cell sample's arms — on a workbench with four target locations.

Where MinimalRobotServer shows you what a robot is, this shows you how to ask it to do something: the server declares an IntentControllerType with its capabilities, frames, tool centre point, locations, axes, outputs, kinematic description and safety state, and executes submitted intents against a simulated motion kernel with real kinematics. Every intent is tracked on a Part 10 program instance, so a motion that takes a minute outlives the Call that started it.

dotnet run --project samples/Robotics/IntentEnabledRobot -- --host localhost --port 62840

`IntentViewerClient`

The reason the intent surface exists, made visible. The client connects, discovers Server/RobotIntent/Controllers, reads the controller's capabilities, opens the OpenUSD viewport on the served stage, and streams live joint and tool-centre-point values into the picture. Click one of the four target pucks on the bench and the client submits a linear-move intent to that location, then tracks the resulting IntentOperation to a terminal state while the arm animates on screen.

Start the server first, then:

dotnet run --project samples/Robotics/IntentViewerClient -- `
    --server opc.tcp://localhost:62840/IntentEnabledRobot `
    --insecure --view

It also runs headless, which is how it is exercised in CI and how it works on a machine without the native OpenUSD payload:

dotnet run --project samples/Robotics/IntentViewerClient -- `
    --server opc.tcp://localhost:62840/IntentEnabledRobot --insecure

In headless mode the four targets are offered as a console menu instead of as prims to click, and everything downstream of the pick — submission, tracking, cancellation, command authority — is identical.

Agent-driven pallet stacking

IntentViewerClient --mcp is the agent entry point for the intent sample. Start IntentEnabledRobot, then run the viewer with MCP over stdio for a headless agent or over HTTP when the OpenUSD viewport is open. The agent and the human viewer share the same OPC UA SampleSession, so MCP calls, operation monitoring, payload outputs and OpenUSD live bindings all describe the same robot.

A well-behaved agent starts with discovery, not motion:

robotics_list_controllers
robotics_read_controller to inspect SupportedIntents, SupportedFacets, frames, tools, locations and outputs
robotics_read_state to verify Ready, mode, safety and current command owner
robotics_request_control before any command

The pallet demonstration deliberately proves refusal handling before the successful plan. Another session holds command authority, the agent submits a move, receives Accepted=false with Failure=ControlNotOwned, reads the enum and message, releases or waits for the owner, requests authority itself, and only then replans. It must not retry the refused submission blindly.

The stacking plan uses the published model seam. Parts begin under /World/Payloads/BinParts, the held part is driven by HeldPartPosition and HeldPartVisible, and completed slots are driven by PayloadSlotNNFilled. The viewer and connector do not contain pallet logic; they just render the server-published bindings. The agent can pick from Bin, move the held part to row/layer coordinates near Fixture, place it, then submit a mission that repeats the sequence for the next slot.

Agents must not assume:

command authority is implicit;
a supported tool means the controller supports every intent kind;
SupportedIntents, SupportedFacets, safety state or queue depth can be skipped;
a refusal should be retried without changing the plan;
a submitted operation is done until robotics_wait_operation or controller state says so.

How a click becomes an intent

The viewport is rendered by the optional Opc.Ua.OpenUsd.Connector.Viewer assembly. A pick becomes an intent the same way whichever path resolves it: the viewer raises UsdViewOptions.PrimPicked, and the client maps that prim path to the LocationType node the server published under the controller's Locations folder, builds an intent naming that location, and submits it.

UsdViewOptions.PickMode selects how the pick is resolved:

Mode	Behaviour
`Auto` (default)	Renderer-backed pointer picking first, falling back to the command prim only when renderer picking is unavailable
`Renderer`	Renderer-backed pointer picking only
`CommandPrim`	Watch `UsdViewOptions.CommandPrimPath` (default `/World/IntentCommand`) and raise the callback when its `targetPrim` changes

Renderer-backed picking works: the OpenUSD viewer owns input handling, DPI scaling, physical-pixel conversion and stale-revision retry, and reports hits through the host callback. Misses do not submit intents. The gaps this used to work around were filed as marcschier/openusd-dotnet issues #1, #5, #8, #9, #10 and #11, and all are fixed in the package version this sample ships against. The command-prim path remains supported, and is what makes picking work headlessly — which is also how an MCP agent drives the same robot.

Prerequisites

.NET SDK 10.0.
The --view option needs the optional viewer assembly and its native OpenUSD payload, which is supported on win-x64, linux-x64, and osx-arm64. A RID-less build or publish on a supported host uses that host's payload; publish with an explicit RID for another platform. Everything else, including both servers and the headless client, runs on every platform the stack supports.

8.1 KiB Raw Permalink Blame History

Robotics samples

MinimalRobotServer

IntentEnabledRobot

IntentViewerClient