mirror of https://github.com/OPCFoundation/UA-.NETStandard.git synced 2026-10-18 22:55:08 +08:00

Files

T

Marc Schier cef83550cb OPC UA Vision: a vision-guided bin-picking demo an LLM agent can see and drive (#4235 )

# Description

Implements **OPC UA — Vision** and uses it to build the demo the
companion specifications were drafted for: an eye-in-hand camera watches
a bin of parts, a language model *looks at the frame*, decides which
part matches the instruction, and commands the robot through Robot
Intent — with the OpenUSD viewer showing the arm move and the bin empty
out.

> **Draft, and stacked.**
[#4195](https://github.com/OPCFoundation/UA-.NETStandard/pull/4195) has
merged and master is merged in, so what is left on top of `master` is
the Vision work plus the unmerged `marcschier/ai-model-management`
branch, which the demo needs for the model/dataset/deployment
provenance. Review the Vision commits; the AI Model Management work
lands with its own PR.

## What is here

**The specification, implemented.** `Opc.Ua.Vision` is generated from
the spec's own `Opc.Ua.Vision.NodeSet2.xml`, vendored byte-identically
(SHA-256 `79C592C6…`, verified against a fresh download and stored
LF-preserved like the Robot Intent NodeSet). `Opc.Ua.Vision.Server`
builds the §4.2 discovery object, sensors, the coordinate-frame tree,
calibrations, media endpoints, inference pipelines and the §9 feedback
methods, with rendering and inference behind injectable providers.
`Opc.Ua.Vision.Client` is built on the generated ObjectType proxies and
composes transforms across the frame tree, which is what turns a
detection in camera coordinates into a pose a robot can act on.

**Two perception paths behind one contract**, which is exactly what
`InferenceLocationEnum` is for. `OnServer` is a deterministic detector
derived from the stage's ground truth — no model, no network, no GPU, so
CI can exercise the whole loop. `EdgeOffServer` is the agent: it sees
the frame over MCP and calls `SubmitDetections`, and the Server
publishes results it did not compute. A client reads
`DetectionResultType` identically either way.

**The agent's eyes.** `tools/Opc.Ua.Mcp.Vision` adds 22 `vision_*`
tools, which with the four shared connection tools makes the `vision`
profile 26; `vision_get_frame` returns the camera image as an MCP
`ImageContentBlock`, so the model looks at pixels rather than reading a
description of them. Tool profiles now compose — `--profile
vision,robotics` yields 62 tools — so one agent can both see and act
without dragging in the full 136-tool catalogue.

**The cell.** `samples/Robotics/BinPickingCell` follows the spec's own
Robotics-Vision Addendum: the `world → robot_base → flange →
gripper_tcp` frame tree with `camera_eih` on the flange, the `HandEye`
extrinsics and the `Intrinsics612x512` intrinsics — the addendum's
2448×2048 calibration scaled to the 4×4-binned grid the camera actually
delivers. `samples/Robotics/BinPickingClient` hosts the composed MCP
catalogue and optionally the viewer.

**Documentation.** `docs/Vision.md` (~800 lines) covers the model,
hosting, the build context, every topology builder, both perception
paths, feedback, the MCP profile and the limitations; linked from
`docs/README.md`, with the profile table in `docs/McpServer.md` and
READMEs for both samples.

## Evidence, not assertions

Each of these was measured on a running system rather than assumed:

| Claim | Evidence |
|---|---|
| The camera sees the bin | 612×512 frame with the bin and all five
parts in view — red `(212,47,47)`, green `(61,193,71)`, blue
`(54,85,218)`, yellow `(215,196,58)`, orange `(209,119,55)` |
| An agent can act on what it sees | Over MCP, `vision_get_frame`
returns a 612×512 PNG that decodes from base64 to 1,254,051 bytes, the
descriptor beside the detections says 612×512, and all five
`BoundingBox2D` centres land **inside** the frame on the pixel colour
their own detection claims |
| The geometry is right | A detection composed `camera_eih → world`
lands on the authored position, residual **0.0000 m** — on both the
on-server and off-server paths |
| The detector tracks the world | Placing a part drops the next scan
from 5 detections to 4, with that part absent |
| The robot obeys the agent | Scripted run took command authority and
completed a pick and a place, both `Succeeded` |
| Tool catalogues are honest | Counted from a running server: vision 26,
robotics 40, `vision,robotics` 62, full 136, **zero duplicates** |
| Every node is browsable by NodeId | Eight representative NodeIds —
both folders, a pipeline, a frame, the sensor, its Calibrations folder,
the HandEye calibration, the Media folder — all `Good` |

That tool-count check matters because the profile tests collapse names
into a `HashSet` and *cannot see* a duplicate registration — so the new
tests count registrations, not names.

## Things that were wrong, and are not now

Writing the tests found three defects of one kind — code that answered
instead of refusing. A zero-norm quaternion was silently rewritten to
identity, so a caller with an all-zero orientation got a confident wrong
grasp pose. `SubmitCorrection` rewrote a null `ResultId` to empty and
called the sink anyway. A bad `StageIdentifier` **killed the host
process** inside native `UsdStage.Open`, via an exception .NET cannot
catch. All three now refuse loudly and are pinned by tests.

**Nodes that existed but could not be reached.** The node manager
indexes the Vision root before configurators run, so everything the
fluent builder grafted on afterwards — both folders, the pipelines with
their Results and Feedback, the frames, the sensors with their optics,
calibrations and media endpoints — was never added to the index.
Browsing *forward from a parent* worked, because that walks
`NodeState.Children` in memory; browsing any of those nodes *by its own
NodeId* returned `BadNodeIdUnknown`, which is how an ordinary client and
the MCP discovery tools navigate. Registration is now deferred and
flushed after each configurator. The first fix covered only the DI path
and left the documented non-DI fallback broken in exactly the same way,
so `ConfigureVisionAsync` now exists and registers what it builds.

That defect survived a green suite because every test browsed children.
The new tests assert by NodeId, and were checked against the broken code
— with the flush removed, they fail.

**Every Method was uncallable.** The Server passed 426 unit tests while
not one of its Methods could be invoked over a real session —
`RunInference` returned `Bad_TooManyArguments`, and so did every other
Method that takes arguments, which is all of them but `StartContinuous`
and `Stop`. Three independent faults, each sufficient on its own: the
Method nodes were never created (they are Optional children, and the
dispatcher guards every attachment with a null check, so it silently
attached nothing); nothing declared the arguments, so the stack
concluded none were expected; and `MethodDeclarationId` pointed at a
synthesised NodeId rather than the NodeSet declaration a client calls
with, so the lookup missed the instance Method entirely. The builder now
creates the Methods, the `Results` folder and the `Feedback` object when
a provider or sink is configured, declares every signature, and names
the right declaration.

Unit tests could not have caught this: they invoke the handler delegates
directly and never go through `MethodState.Call`. It took an end-to-end
test over a real session. Relatedly, children created by the generated
`CreateOrReplace` helpers carry no `ReferenceTypeId`, so nothing could
reach them — fixed once in the build context rather than at the several
dozen call sites.
**And one where I was wrong.** Earlier in this branch I reversed my own
agents' decision and made `SubmitDetections` accept an empty detection
set, reasoning that "I looked and the bin is empty" is a correct
observation and refusing it forces a correct agent to invent a
detection. The reasoning is sound; it is also an argument *against* the
specification rather than an implementation of it. §9.5 is normative and
explicit — its StatusCode table lists `Bad_InvalidArgument` for
"`Detections` empty; or `SubmitCorrection` supplies both or neither
corrected array".

Worse, **none of §9.5's argument rules were enforced anywhere** — the
dispatcher forwarded every submission straight to the feedback sink, so
conformance depended on whichever sink a host installed. They are Server
obligations, so they now live in the dispatcher, ahead of the sink,
proven by strict mocks that would throw if the sink were consulted. The
usability problem is real and does not go away by conforming, so it is
raised where it can be fixed:
[opcua-drafts#70](https://github.com/marcschier/opcua-drafts/issues/70).

## Tracking the updated drafts

The Vision, Robot Intent and AI Model Management NodeSets on
`opcua-drafts` `main` have moved, partly because of feedback this branch
raised. All three are re-vendored byte-identically — verified by
comparing git blob hashes against the upstream files rather than by eye
— and the implementations follow.

**Two statements that were previously inexpressible now have a wire
representation.** `SubmitDetections` takes `SceneIsEmpty` and
`SubmitCorrection` takes `RetractAll`, so an agent can report "I
examined this frame and there is nothing in it" — the terminating
condition of the bin-picking task — and can retract a false positive by
correcting a result down to nothing. Both are checked in both
directions, because the flag is exactly what separates a deliberate
empty observation from a lost payload: an empty array without the flag
is refused, and the flag with an array attached is refused as two
contradictory claims about one frame. Implemented in the dispatcher, the
client, the MCP tools the agent drives, and the sample cell.

Also absorbed: `LampType` and `LightingMode` became enums (§66),
`ProfileName` became `DefaultProfileName` on the endpoint (distinct from
the `ProfileName` a caller passes to `GetStreamEndpoint`, which keeps
its name), AI `Invoke`/`InvokeAsync` gained `PayloadUri` under the same
exactly-one rule, and AI `ListModels` gained a `ContinuationPoint` —
because `MaxResults` alone puts everything past the bound permanently
out of reach.

**OpenUSD is on `0.9.0-alpha`**, verified on the running cell rather
than by restore alone: D3D12 backend, 612×512 frame, the bin and all
five parts in view and detected — the same result `0.8.0-alpha`
produced.

The one local workaround that survives is `VisionMethodArguments`, and
its reason has changed rather than gone away: the builder materialises
these Methods as Optional children through `CreateOrReplace`, which
constructs the state object directly and never runs the generated
factory that now carries the signatures. Checked rather than assumed —
removing the declarations fails five of the eight
`VisionMethodSurfaceTests`.

## Review feedback

Three threads, all addressed in `0fe2803a3`:

- **The AI Model Management work is now a library family, not a
sample.** `src/Opc.Ua.AI` (model), `.Inference` (`IInferenceBackend` and
backends), `.Server` (node manager) and `.Client` (discovery, reads,
calls, artefact transfer), in the same shape as the Robotics and Vision
families; the samples become `samples/AI/ModelManagementServer` and
`ModelManagementClient`. The client library needed real work rather than
relocation: its browsing surface was `private` inside the sample's
scenario runner, so moving the file alone would have produced a library
nobody could call.
- **No cloud-vendor SDK is referenced anywhere.** Inference goes through
`Microsoft.Extensions.AI` — an `IChatClient` is implemented by hosted
services and on-device runtimes alike, which is exactly the property
clause 8.1 asserts. `Azure.Identity` is gone: workload identity is not
an Azure feature, every platform projects a token into a file and each
SDK reads that file behind its own API, so the resolver reads it
directly. The comment claiming `Microsoft.Extensions.AI` would force a
`System.Text.Json` bump was stale — it pins 10.0.10, which this
repository already centralises on.
- **`BinPickingCell` and `BinPickingClient` moved into
`samples/Robotics/`**, which is where they belonged: the cell was
already linking the arm, its kinematics and both USD assets out of
`IntentEnabledRobot`.

## Upstream

Work that belonged elsewhere went elsewhere rather than being worked
around locally:

- **`openusd-dotnet`
[#13](https://github.com/marcschier/openusd-dotnet/pull/13)** — merged
and shipped in `0.8.0-alpha`. `SilkFrameCapture.Capture` rendered only
the first capture of a session and silently returned a blank frame for
every one after it. Also filed
[#14](https://github.com/marcschier/openusd-dotnet/issues/14) (no
camera-by-prim API) and
[#15](https://github.com/marcschier/openusd-dotnet/issues/15) (skipped
tests reported as passed — which fooled me for several runs). Both are
fixed; this branch takes `0.8.0-alpha` and deletes two workarounds,
including 143 lines of hand-rolled projection maths.
- **`opcua-drafts`
[#66](https://github.com/marcschier/opcua-drafts/issues/66)–[#69](https://github.com/marcschier/opcua-drafts/issues/69)**
— a close read of the spec produced 22 candidate ambiguities; 12 were
discarded as misreadings with clause citations and the remaining 10
filed. One is concrete: five members exist in the NodeId CSV that no
clause documents.
- **`opcua-drafts`
[#70](https://github.com/marcschier/opcua-drafts/issues/70)** — §9.5
could not express an empty observation or a false-positive retraction.
**Adopted**, with the explicit-flag resolution argued for:
`SubmitDetections` gained `SceneIsEmpty`, `SubmitCorrection` gained
`RetractAll`, and the corrected-array rule relaxed from *exactly one* to
*at most one*.
- **`opcua-drafts`
[#71](https://github.com/marcschier/opcua-drafts/issues/71)** — the
Vision NodeSet browse-named all 13 `InputArguments` / `OutputArguments`
Properties in the Vision namespace rather than the base namespace, so a
stack could not find them and every Method in the specification was
uncallable. **Fixed**; this branch now tracks the corrected artefact.

## The defect underneath it all

The end-to-end suite was removed earlier in this branch because nine of
its
fourteen tests failed on one symptom: a client could not open a
pipeline's
`Feedback` object or enumerate its `Results`, even though browsing the
same
parent listed both. Restoring it meant explaining that, and the
explanation
was not in Vision.

`RelativePathElement` initialised **`IsInverse` to `true`**. A caller
that
sets a `ReferenceTypeId` and a `TargetName` — which reads as *"the child
reached by this Reference"* — was therefore asking the Server for the
**inverse** Reference, and got `Bad_NoMatch` with nothing in the request
to
suggest why.

`ObjectTypeClient.ResolveChildNodeIdAsync` is such a caller, and it is
what
**every source-generated Optional-child accessor in the stack** is built
on
— the accessor whose own documentation cites
`AlarmConditionTypeClient.GetShelvingStateAsync`
as an example. Vision is merely where it surfaced, because Vision puts
two
Optional children on the path an off-Server agent must take. The sibling
helper in `StateMachineTypeClientExtensions` already set `IsInverse =
false`
by hand, which is the shape of a trap rather than a default.

The default is now forward, `ResolveChildNodeIdAsync` says so
explicitly,
and the existing test that had recorded `true` as *expected* now pins
`false` with the reasoning. Of the forty `RelativePathElement`
construction
sites in the tree, that accessor was **the only one relying on the
default at
all** — every deliberate caller already set it.

Two further defects were found underneath it, both real and both fixed:

- The generated `CreateOrReplace` helpers construct a child state object
directly and leave **`TypeDefinitionId`** unset as well as
`ReferenceTypeId`.
A child referenced by nothing cannot be browsed; one with no type
definition
is a malformed Object that a client filtering by type silently skips —
which
  is exactly what `EnumerateResultsAsync` does. Normalisation moved to
`VisionNodeManager`, so it also covers the case the builder cannot see:
a
result published **at runtime** by an inference provider, long after the
address space was built. That path produced results a client could list
but
  not read.
- `Opc.Ua.AI` shipped without an `AssemblyInfo`, so the solution built
with
  six `CA1014` warnings.

I had initially "fixed" this in the Vision client with Browse-based
fallbacks,
on the strength of a probe that appeared to show
`TranslateBrowsePathsToNodeIds`
failing even for `Server → ServerStatus`. That probe was wrong: it used
an
unqualified `ReferenceTypeIds`, which inside a `Opc.Ua.Vision.*`
namespace
binds to *Vision's* `ReferenceTypeIds` and yields a nonsense NodeId.
Chasing
that to the end is what turned up the real cause. **The fallbacks are
gone** —
they would have hidden a stack-wide bug behind a Vision-shaped
workaround, and
one of them matched on browse name while ignoring the namespace index.

`tests/Opc.Ua.Vision.Intent.Tests` is restored and back in `UA.slnx`,
passing
**14/14** on its own merit rather than through a client-side workaround.
A
registration test now pins the pipeline's `Feedback` and `Results`
children,
which nothing covered before.

## Running it, and what that found

The demo's premise is that a language model **looks at the bin and
reasons
about what it sees**. Running it end to end showed it could not, for
reasons
no test covered, because every one of them lived between components.

**The sample did not compile.** Two `CS8604` nullable errors with
`TreatWarningsAsErrors`, so neither the cell server nor the demo could
be
started at all. Worth recording that my first explanation was wrong: CI
*does*
build this project - the Linux leg's `net10.0` pass compiles it,
log-confirmed -
so this is not a coverage gap in the workflow, and no workflow change
was made.

**`Pick` and `Place` never moved the arm.** They were a delay plus a
gripper
action, so the operations reported `Succeeded` while nothing travelled.
They now
solve IK once and interpolate in joint space. **And the parts never
moved**: the
arm swung but the bin stayed full, because nothing tied the held object
to the
world state. The connector now creates 11 monitored items where it
created 6,
and a scripted run delivers `SeqNo=151` across 10 messages where it
delivered
`SeqNo=3` across 1.

**The frame was smuggled through a string field.** The media provider
put a
base64 *data URI* of the PNG into `VisionImageReferenceDataType.Uri` - a
field
meant to be a *reference*, while the bytes were already returned in the
inline
`ByteString`. Every response therefore shipped the image twice, and a
1,311,400-byte PNG became a 1,748,536-character string against a
`MaxStringLength` of 65,535: **26.7x over**. That is why raising
`MaxByteStringLength`, `MaxArrayLength` and `MaxMessageSize` never
helped - none
of them govern a `String`. Separately the MCP tool assigned raw bytes
where
`ImageContentBlock` requires base64, so the wire carried
`"data":"�PNG..."` -
every non-UTF-8 byte replaced by U+FFFD, destroying the image and
inflating the
response to 6.8 MB.

**The camera was aimed at nothing.** The arm's home pose, the flange
scan pose
the Vision model declared and the joint angles in the USD stage were
three
independent claims about where the camera was, and they disagreed - the
declared
flange orientation pointed the view 86 degrees away from the projection
camera.
Solving them as one thing exposed two further constraints that are easy
to miss:
the camera has to sit off the tool axis or it photographs its own
gripper, and
the IK branch has to be elbow-back or a link parks under the camera and
fills the
frame with the arm's own upper arm. A third was a regression this branch
introduced and `Opc.Ua.Robotics.Tests` caught - aiming a straight-down
camera
from a point on the base's own X-Z plane puts the wrist on the J4/J6
singularity, so every motion away from home fails to solve. A 15-degree
camera
roll, free because the camera still looks straight down, gets 25 degrees
clear,
which is the margin the poses before this branch had.

**The frame and the detections were different images.** The sensor
declared
2448x2048, the clip endpoint 1280x1024, and the renderer produced
640x512 - not
even the same aspect ratio. The ground-truth detector projects through
the
declared intrinsics, so its boxes lived in 2448x2048 space while an
agent was
handed a 640x512 picture: *"pick the red cube you can see"* arrived with
coordinates that pointed off the image. Everything now agrees at
612x512, an
exact 4x4 bin of the native sensor with the intrinsics scaled to match;
the
native size stays in the model and serial number, where it describes the
hardware rather than the image.

**`LatestClip` was created and never written**, so it reported
`Bad_NoDataAvailable` for the life of the Server and a consumer
following the
model - read the published frame, call the method only if there is none
- never
got a frame. The dispatcher now publishes a clip it has just encoded,
for every
Vision server rather than this sample alone.

One genuine API gap turned up on the way: `OpcUaServerOptions` exposes
`MaxByteStringLength`, `MaxArrayLength` and `MaxMessageSize` and **no
`MaxStringLength`**, so a Server that legitimately needs a long string
could not
raise the limit through the hosting API at all. Added, with tests. It is
not the
fix for anything above - that was a design bug - but the missing knob
was real.

## Not finished

- **Two projects sit below the repository's 80% line coverage.**
`Opc.Ua.Vision.OpenUsd` is at 71.1% — what is left only executes with a
real OpenUSD stage and plugin tree, which is not on every build agent.
`Opc.Ua.AI.Inference` is now at **86.7%**, up from 12.9%: the code this
branch *wrote* is covered by 25 tests, and `RestChatCompletionsBackend`
— previously the whole of the shortfall — is covered by 18 more against
a stubbed `HttpMessageHandler`, exercising success, refusal, throttling,
retry-after, timeout and cancellation without a network. Everything else
clears the bar — `Opc.Ua.Vision.Server` 92.1%, `Opc.Ua.AI` 91.4%,
`.Vision.Client` 88.1%, `Opc.Ua.Vision` 85.7%, `Opc.Ua.AI.Server` 80.1%,
all measured with coverlet. The CI coverage gate reports itself as
advisory and non-blocking.
- **`SamplesCollected` is not counted here, and `docs/Vision.md` now
says so.** Section 9.4 requires a Server to count a negative example
(`SceneIsEmpty` / `RetractAll` carrying a `GroundTruthLabel`) exactly as
it counts one carrying geometry. `VisionNodeManager` does not, and
structurally cannot: `SamplesCollected` is a property of
`LearningJobType`, which the *AI Model Management* companion defines,
and Vision reaches the job through a `NodeId` value rather than a
Reference **precisely so this model takes no dependency on the model
that defines it**. What the Server does guarantee is that the negative
example survives the hop intact — both flags are carried verbatim to
`IVisionFeedbackSink` — so a host that binds a learning job has
everything it needs to satisfy the clause on the counter it owns. Stated
in the limitations rather than left implied.

## Related Issues

Was stacked on #4195, now merged. No tracking issue yet for the Vision
implementation itself — happy to open one if you would like the design
recorded as an ADR before this goes further.

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage. — *Types 8562, Server 4072,
Client 2123, InformationModel 1063, Robotics 540, Tools 466, Vision 442,
AI 74 and the restored Vision+Intent end-to-end suite 14/14, all green
on net10.0. Only `.OpenUsd` (71.1%) is below the 80% bar — see Not
finished.*
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings. — *`dotnet build UA.slnx -c Release`: 0 errors, 0
warnings.*
- [x] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed. — *net48:
Types 8555, Client 2126, Vision 385 green. Two net48 Server tests fail
(`CreateUserManagementSeamBindsTheModelToTheServerAsync`, and
`DurableDataValueQueueVerifyReferenceBatchingAsync` only under parallel
load); both were confirmed **pre-existing** by re-running them with the
`IsInverse` change reverted, which fails identically.*
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings. — *The net48 build legs failed because the
restored end-to-end project used `ValueTask.FromResult` /
`string.Create`, which do not exist on .NET Framework; fixed in
`fb57522da` and verified building on every TFM.
`DisposeDrainsHeldConnectionCallbackBeforeListenerDisposal` failed once
on the Ubuntu Client leg — a 13 ms timing-sensitive reverse-connect
disposal test that passes locally 2123/2123 and touches no browse
paths.*
- [ ] I have addressed **all** PR feedback received.
## Typed Vision-guided Robotics MCP workflows

The branch now also makes the demo workflow first-class in the Full MCP
profile:

- replaces stringified Robotics intent and mission inputs with typed
request schemas and exact name-or-NodeId selectors;
- adds bounded operation/mission paging, stable step-to-operation
identity, retained mission discovery, and `robotics_wait_mission`;
- makes `vision_run_inference` return bounded typed detection,
inspection, or segmentation summaries in one call;
- adds same-session `robotics_vision_pick` to select a detection and
submit Pick or Pick/Place work without implicitly taking authority,
retrying, cancelling, or waiting; and
- starts Publish workers for classic subscriptions so an independently
launched OpenUSD viewer receives live robot and part updates.

The migration guide documents the intentionally breaking MCP
request-schema changes.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1dab1302-19c5-4a9b-a50c-d97d389713aa
Copilot-Session: a248a589-9a20-4372-868e-5d347e57001b
Copilot-Session: 99d1ab42-f83a-41ab-bf3e-85bf81fadace
Copilot-Session: bcbc8ac9-5abd-42a8-8bac-e10533790e2e

2026-08-29 19:34:15 +02:00

23 KiB

Raw Permalink Blame History

OPC UA .NET Standard stack documentation

Here is a list of available documentation for different topics:

New contributor? Start with the Developer Guide — prerequisites, building, testing, coding standards, and how-to recipes (including how to add logging).

OPC UA Profiles and Facets - Overview of supported OPC UA profiles, facets, security policies, and transport protocols.
Transport Profiles - Developer guide for the wire transports: opc.tcp / HTTPS (binary + JSON + REST) / WSS (binary + JSON), including server hosting and client connect examples.
REST Binding (OpenAPI Mapping) - OPC UA Part 6 §G.3 OpenAPI mapping: ASP.NET Core MVC controllers for every spec service, Compact / Verbose encoding negotiation, four pluggable auth modes (Anonymous / Bearer JWT / HTTP Basic / Mutual TLS), and the symmetric IWebApiClient.
What's New in 2.0 - Developer-facing tour of the 1.5.378 → 2.0 changes, grouped by theme and layer, with links to deeper feature docs.
Migration Guide - How to migrate from a previous version.
Sessions, Reconnection, and Subscription Engines - Architectural overview of Session, ManagedSession, SessionReconnectHandler, and the classic / V2 subscription engines, including guidance on which to use.
About .NET platform support, NuGet packages and versioning.
About continuous integration — which pipeline runs what, how to start a validation build on a pull request with /azp run, and the coverage gates a change has to satisfy.
How X.509 Certificates are used in the certificate stores.
CertificateManager - Centralized certificate lifecycle management, server-side push certificate rotation, and the OPC UA Part 12 PushManagement transaction model (ApplyChanges/CancelChanges, staged TrustList/Certificate updates, DeleteCertificate, pending-key persistence).
Crypto provider - Replacing the stack's cryptography with another library, a remote service or hardware (TPM, HSM, PKCS#11, cloud key service), keeping private keys inside the device, selecting providers per purpose and security policy, and auditing the use of uncertified cryptography.
Using the Reverse Connect for the UA-TCP and WSS transports.
Support for the TransferSubscriptions service set.
Diagnostics — logging, telemetry, server audit events, server diagnostics nodes, and packet capture.
Performance Benchmarks — BenchmarkDotNet methodology, the 2.0 (master) vs 1.5.378 (master378) comparison, root-cause analysis of the encoder/decoder/session regressions and their real-world impact, the subscription-notification (pooled encodeable) micro-benchmarks, server session scalability (the 500-session capability, sizing and bottlenecks), and planned future work.
Server Session Scalability — why a single node tops out at ~2000 concurrent sessions, the establishment vs steady-state boundaries (socket backlog, the BadTcpInternalError retry-storm amplifier, the O(N²) diagnostics rescan, CreateSession crypto-under-lock, the RSA CPU wall, and the held-Publish worker-accounting coupling) with code references, and a prioritized admission-control / rate-limiting roadmap for moving beyond it.
Rate Limiting and Admission Control — the server's deterministic, configurable connection- and session-establishment limiters (on by default, System.Threading.RateLimiting-based, DI-pluggable), the BadServerTooBusy signalling, the diagnostics-independent server retry-after carriers (ResponseHeader.additionalHeader, HTTP Retry-After, UA-TCP ERR, and load-based Server.ServiceLevel), and the client's server-signal-aware adaptive reconnect backoff (IReconnectPolicy.TryGetNextDelay).
Support for WellKnownRoles & RoleBasedUserManagement.
Pluggable Identity Providers — interfaces (IClientIdentityProvider, IUserTokenAuthenticator, IAccessTokenProvider, ITokenIssuer, IIdentityClaims) plus the OPC 10000-6 §6.5.2.2 IssuerEndpointUrl JSON parser for OAuth2 / OIDC / Entra / JWT flows.
Support for ECC Certificates.
Working with ComplexTypes - Custom structures and enumerations.
Client-based NodeSet Export - Export server address space to NodeSet2 XML.
Source generated [DataTypes] - How to annotate POCO classes and let the source generator generate the IEncodeable implementation.
Runtime Schema Generation - Produce XSD, OPC Binary (BSD) and JSON Schema (Part 6 Annex C, compact + verbose) for generated encodeable types and dynamically added complex types via the injectable ISchemaProvider; schemas are built as object models in code (trimmable, NativeAOT compatible).
NodeManagers - Overview of the server node-manager architecture, built-in managers (master, core, diagnostics/configuration), CoreNodeManager vs CustomNodeManager2 guidance, registration and runtime lifecycle rules, namespace metadata and historical-access reconciliation, and source-generated AsyncCustomNodeManager authoring with the fluent INodeManagerBuilder API. Includes NativeAOT single-file server guidance and samples such as MinimalBoilerServer and PumpDeviceIntegrationServer.
Runtime NodeSets - Load one or more NodeSet2 XML documents into the server address space at startup without source generation. Covers file and stream sources, dependency ordering, parent-child browse-path resolution, fluent Configure callbacks, default namespace inference, and the default complex-type loading path. Use when the XML content changes independently of the server binary or for rapid prototyping.
WoT / NodeSet conversion - WotNodeSetConverter readable mapping, preservation projections, and the WoT-to-NodeSet default/failure table.
WoT Connectivity - OPC 10100-1 asset connectivity, the WoT Connectivity 1.1 registry/client, dependency-closure materialization, and runtime NodeSet projection.
WoT protocol bindings - The bindings that ship today (planner/executor architecture, bundled and separate packages, operation coverage, target mapping, lazy channels, generation lifetime) and the contributor guide for adding your own, with a complete memory binding, registration, diagnostics, tests, packaging, TFM, trimming, and NativeAOT guidance.
Asset Administration Shell - OPC 30270 / AAS V2 ingestion-only metamodel support and the AAS V3 draft (I4AAS) model, JSON/XML/AASX serialization, metamodel server/client, xRegistry-based AAS registry, updateable materialization, package integrity, federation, DPP/WoT bridge, and conformance matrices.
WoT aggregation sample - Two flat OPC UA sources aggregated into a runtime-loaded DI/Machinery/Pumps Pump model, including commands, Refresh, monitoring, replacement, troubleshooting, and NativeAOT publishing.
Device Integration (DI) developer guide - End-to-end documentation for the Opc.Ua.Di* library trio: fluent IDeviceBuilder, device sub-type extensions (AddSoftware, AddBlock, AddConfigurableObject, AddLifetimeIndication, WithSupportInfo), hosting integration (AddOpcUaDi / ConfigureDevicesFor), lock service, software-update package store, and client helpers (DiLockClient, DiTopologyClient, SoftwareUpdateClient). Includes a section enumerating supported OPC 10000-100 features against the spec.
OpenUSD — bridge an OPC UA address space to an OpenUSD stage, in two parts. Part 1 — bindings: the generic domain-agnostic OpenUsdConnector (discovers Server/OpenUSD/Representations, subscribes, composes, verifies stage/asset digests, replays history), the Variant-based IUsdSink with UsdFileSink / MockUsdSink, the fluent/DI AddOpenUsdConnector extensions + OpenUsdConnectorFactory / OpenUsdConnectorOptions, server-side UsdAssetDelivery, and the optional --view viewport. Part 2 — scene materialization: materializes a composed USD stage inside the address space so the prim tree is the node hierarchy — the Opc.Ua.OpenUsdScene companion model plus scene document model, .usda reader/writer and the §6.2 UsdValueTypeMap (USD roles as DataTypes subtyping their built-in), and Opc.Ua.OpenUsdScene.Server with MaterializeUsdStage / ExportUsdStage, unknown-type fallbacks, Mode-A live attributes, portable Cesium georeference dual-authoring, discovery and Part 1 binding-target resolution. (Both companion models are currently drafts.)
Robotics developer guide — the Opc.Ua.Robotics / Opc.Ua.Robotics.Server / Opc.Ua.Robotics.Client trio for OPC 40010 Robotics 1.02 over OPC 40001-1 IA and OPC 10000-100 DI: source-generated models, AddRobotics / AddRoboticsModel / ConfigureRobotics(For) hosting, IRoboticsModelProvider / IRoboticsConfigurator / IRoboticsBuildContext, validated fluent topology builders (AddMotionDeviceSystemAsync down to axes, power trains, motors, gears, drives, safety states, and task controls), semantic references, ArrayOf<T> snapshot contracts, and RoboticsClient discovery. It also covers Robot Intent, the draft task-level motion verbs OPC 40010 leaves undefined, including the MCP surface for LLM agents.
Vision developer guide — the Opc.Ua.Vision / Opc.Ua.Vision.Server / Opc.Ua.Vision.Client / Opc.Ua.Vision.OpenUsd package family for the draft OPC UA — Vision companion specification: source-generated Vision model, AddVision / ConfigureVision hosting with IVisionMediaProvider / IVisionInferenceProvider / IVisionFeedbackSink, the two perception paths behind one contract (OnServer deterministic detector vs EdgeOffServer agent submissions), fluent topology builders for frames, sensors, calibrations, media endpoints and inference pipelines, VisionClient discovery + VisionFrameGraph §5.12 pose composition + VisionResultReader streaming detections + VisionFeedbackClient for off-server VLM agents, the §6.4 media-gating states, the NoRenderingBackend degrade path, facet derivation, and the composed vision MCP profile with the BinPickingCell / BinPickingClient example.
AI Model Management developer guide — the Opc.Ua.AI / Opc.Ua.AI.Inference / Opc.Ua.AI.Server / Opc.Ua.AI.Client package family for the draft OPC UA — AI Model Management and Inference companion specification over xRegistry: source-generated catalogues, datasets, deployments, inference endpoints and learning jobs, the IInferenceBackend contract with Microsoft.Extensions.AI IChatClient and OpenAI-compatible REST backends, AINodeManagerFactory hosting via AddNodeManager<AINodeManagerFactory>, Invoke routing, standard file-transfer artefact streaming, credential resolvers that keep secret material out of the address space, and the ModelManagementServer / ModelManagementClient example.
Relative Spatial Location and Global Positioning — source-generated OPC 10000-210 RSL and OPC 10000-211 GPOS models, standalone/composed server hosting, provider contracts, high-level clients, frame-chain resolution, WGS84/ENU conversion, and ground-control-point fitting.
Generators (generating sets) — the draft Generators companion specification realised end to end by GeneratorServer: a datasheet-driven simulation in which load fraction is the only independent variable, DI + Machinery integration, and one independent OpenUSD twin per configured set. Includes SiteCompositionServer, a supervisory server that owns no devices and composes the pump and generator servers into a single scene through cross-server components.
ISA-95 developer guide - End-to-end documentation for the Opc.Ua.ISA95* library trio: the OPC-10030 Common Model and OPC-10031-4 Job Control V1/V2, the two transparently-documented normative NodeSet repairs, AddIsa95Server/AddIsa95Client hosting, the typed common-model builder, the shared Job Control state engine and its Uncertain/Annex-B ReturnStatus result model, the provider-backed GeoSpatialLocationType seam and planned Part 210/211 RSL/GPOS integration, and a conformance matrix distinguishing static NodeSet structure from runtime-tested behavior.
Alias Names - Full server + client support for the OPC UA Part 17 alias-name model (AliasNameType, AliasNameCategoryType, FindAlias, FindAliasVerbose, AddAliasesToCategory, DeleteAliasesFromCategory, LastChange).
Alarms and Conditions - Full server + client support for OPC UA Part 9. Server-side state types for latched/silenced/out-of-service alarms, alarm groups and suppression engine, alarm rate metrics. Client-side AlarmClient, typed alarm event records, fluent AlarmEventFilterBuilder, IAsyncEnumerable alarm streaming via AlarmStreamExtensions.
Historical Access (Part 11) - Server provider model (IHistorianProvider family) and InMemoryHistorianProvider, plus the client HistoryClient (session.Historian()) for raw/modified/at-time/processed reads, annotations, and updates.
Aggregates (Part 13) - All 37 standard Part 13 v1.05.07 aggregate functions over historical data: server AggregateManager / calculators, native push-down vs framework fallback, AnnotationCount via the annotation provider, AggregateConfiguration defaults, and the client ReadProcessedAsync helper.
Subscriptions and Monitored Items Service Set - V2 subscription engine API. Covers ISubscriptionManager for long-lived callback-based subscriptions, the declarative+imperative SetTriggering API with N:M support and automatic replay on recreate/reconnect, and IStreamingSubscription (IAsyncEnumerable-based) for state-machine waits and short-lived monitoring (ManagedSession.DefaultStreaming, TakeUntilAsync / WithTimeoutAsync helpers).
Unbounded Monitored Items - V2 logical-subscription wrapper that transparently splits monitored items across multiple server-side partitions when the per-subscription cap is exceeded (IPartitionedSubscription, MonitoredItemOptions.Affinity, reactive Bad_TooManyMonitoredItems fallback, secondary-partition idle-delete).
State Machines - Generic, extensible Part 16 state-machine API. Client side: streaming + read helpers on the source-generated *TypeClient proxies (GetCurrentFiniteStateAsync, ObserveFiniteTransitionsAsync, WaitForStateAsync). Server side: unified fluent StateMachineBuilder with two complementary modes — definition (Create(...) + AddState / AddTransition / OnCause for ad-hoc machines via FluentFiniteStateMachineState) and lifecycle (For(...) / INodeBuilder.AsStateMachine() + OnEnterState / WithCause / WithTimedTransition to attach behavior to stack-shipped or generator-emitted FSMs). Vendor state machines inherit both ends of the API automatically.
Model Change Tracking - Client-side address-space change tracking with per-node INodeCache invalidation; server-side ModelChangeAggregator and auto-emitted GeneralModelChangeEvent from CustomNodeManager.CreateNode/DeleteNode.
NodeManagement Service Set - Server-side AddNodes / DeleteNodes / AddReferences / DeleteReferences, including the INodeManagementAsyncNodeManager opt-in pattern and per-NodeManager AllowNodeManagement gate.
High Availability and OPC UA Redundancy - OPC 10000-4 §6.6 mapping for server, client, and network redundancy; RedundancySupport, ServiceLevel, manual failover, transparent/non-transparent ManagedSession modes, HotAndMirrored/Transparent state mirroring, distributed address-space/session/subscription stores with CRDT (eventual) or Raft (strong) consistency, shared certificate/trust-list stores, snapshot+delta hydration, and the optional GetEndpoints load-direction seam.
- Kubernetes High Availability Deployment - Consolidated Kubernetes guide for the Opc.Ua.Redundancy.Kubernetes package: Lease leader election, EndpointSlice peer discovery, ServiceLevel-driven readiness, StatefulSet/Deployment and Service manifests, RBAC, probes, time sync, secrets, and GDS/NTRS registration.
- Redundant Sample Integration Tests - Process-level integration tests that launch the RedundantServer / RedundantClient / RedundantPubSub sample apps and assert on their failover, reconnect, and data-loss-visibility behavior; short-haul variants run on every PR, long-haul soak variants run via dedicated manual/scheduled GitHub Actions and Azure DevOps jobs (SAMPLE_HA_DURATION_MINUTES).
Dependency Injection - The unified services.AddOpcUa() / IOpcUaBuilder surface for hosting OPC UA components in Microsoft.Extensions.DependencyInjection / the .NET Generic Host (servers as IHostedService, options via Action<T> or IConfiguration, AOT-friendly).
AuthorizationService - Modern Part 12 StartRequestToken / FinishRequestToken, ITokenIssuer, and GDS token issuance.
Fuzz testing - SharpFuzz + afl-fuzz + libFuzzer integration. Three areas: Encoders (Binary/JSON/XML decoders, built-in type readers, parser entry points), Certificates (X509CRL, X509 extension parsers, PEMReader, Pkcs10CertificationRequest, ASN.1 helpers), and Network (UA-SC framing via Opc.Ua.Core.Diagnostics + internal TcpMessageParsers seam on Opc.Ua.Core). The fuzz-tester custom agent drives the whole toolchain autonomously: it detects OS-available engines, runs them in parallel, fixes novel findings per repo guidelines, adds the failing input as a regression asset, and pushes one commit per fix until the user says stop.
KeyCredentialService - Pull, Push, and experimental bridge guidance for Part 12 KeyCredential flows.
xRegistry (abstract registry base model) - Generic, registry-agnostic content-addressed resource registry: IResourceContentIdProvider identity, the Opaque-NodeId fast path (resolve a resource in one Read), the CreateResource/Write/Close registration lifecycle with auto-bootstrap, federated resource proxies, and the resource-exhaustion bounds. Shipped as Opc.Ua.XRegistry, Opc.Ua.XRegistry.Client, and Opc.Ua.XRegistry.Server.
PubSub (Part 14) - Publisher/subscriber support library: architecture, fluent builder, transports (UDP / MQTT 3.1.1 + 5.0 / Kafka / Ethernet Layer 2), encodings (UADP / JSON), security, and server-side address space.
- PubSub High Availability - Distributed Part 14 §9.1.6 active/standby publishers and subscribers with Cold/Warm/Hot modes, leader election or fenced leases, shared runtime checkpoints, protected SKS key storage, consistency guidance, and Kubernetes deployment notes.
- Migration sub-doc - 1.5.378 → 2.0 breaking API, transport, JSON, and field-encoding changes, plus the compatibility matrix.
- Ethernet transport - Layer 2 PubSub (opc.eth://, EtherType 0xB62C, 802.1Q VLAN) with native AF_PACKET / BPF, SharpPcap, and in-memory backends.
- Kafka transport - Apache Kafka broker transport (kafka://, kafkas://) for UADP and JSON PubSub profiles with SASL/TLS and NativeAOT support on net10.0.
- PubSub Transcoding - In-process subscriber-to-publisher transcoding with UADP/JSON cross-encoding, field/value/metadata transforms, identifier remap, receive hooks, egress, and managed UADP re-securing.
- External server adapter - Bind PubSub publishers, subscribers, and Action responders to an external OPC UA server through ManagedSession.
- Dependency Injection extensions - AddPubSub, AddPubSubPublisher, AddPubSubSubscriber, AddPubSubSecurityKeyServiceClient/Server, AddPubSubAddressSpace.
- Profiles - Datagram-v2, SKS pull / push, AES-128/256-CTR security facets.
- PubSub Diagnostics - packet capture, dissection and replay of UDP / MQTT PubSub traffic, including decryption of encrypted UADP messages.

Reference Client documentation for configuration of the console reference client using parameters.
Reference Server documentation for running against CTT.
ConsoleReferencePubSubClient documentation for the PubSub reference sample (publisher / subscriber / external-server adapter modes).
Provisioning Mode for secure certificate provisioning and initial server configuration.
Using the Container support of the Reference Server in Visual Studio 2026 and for local testing.

Starting with version 1.5.375.XX the Windows Forms reference client & reference server were moved to the OPC UA .NET Standard Samples repository.

Global Discovery Server (GDS)

GDS Developer Guide — Application registration, certificate management (pull & push models), roles and authorization, provider implementation, end-to-end examples, and a per-requirement OPC UA Part 12 conformance matrix that links every implemented ServerConfiguration/PushManagement, TrustList, certificate-alarm, KeyCredentialService and AuthorizationService requirement to its source and automated tests (with applicable profiles/conformance units and honest complete/partial/optional/unsupported marks).
KeyCredentialService — Credential issuance for non-OPC UA services (MQTT, REST), IKeyCredentialRequestStore provider guide, ISecretStore integration.
AuthorizationService — OAuth2-style access token issuance, IAccessTokenProvider implementation guide.
Role-Based Security — Part 18 roles and claim-based identity-mapping rules.
Identity Providers — server and client identity-provider architecture.
Dependency Injection — dependency injection hosting and identity registration extensions.

23 KiB Raw Permalink Blame History

OPC UA .NET Standard stack documentation

UA Core stack related

Reference application related

Global Discovery Server (GDS)

23 KiB

Raw Permalink Blame History