# Machine Vision (OPC 40100-1 and OPC 40100-2) The Vision package family provides Machine Vision control and management (Part 1), asset management and condition monitoring (Part 2), and the Perception model for sensors, inference, media, and coordinate frames. Applications can use the models independently or combine them on one OPC UA server. Part 1 and Perception contracts are in `Opc.Ua.Vision`; Part 2 contracts are in `Opc.Ua.MachineVision`. Each has client and server packages. The generated Perception types live in `Opc.Ua.Vision.Perception` and use the wire namespace `http://opcfoundation.org/UA/Vision/`. The model, client, server, Machinery, and MCP packages follow the root release version. Development builds inherit the root preview version. The optional `Opc.Ua.Vision.OpenUsd` renderer is preview-only. Package version eligibility and a model's specification status are separate: the Perception model is provisional. ## Contents - [Models and provenance](#models-and-provenance) - [Opt-in hosting](#opt-in-hosting) - [Control and acquisition providers](#control-and-acquisition-providers) - [Perception model](#perception-model) - [Packages](#packages) - [Perception hosting](#minimal-hosted-server) - [Perception client interfaces](#using-the-client-libraries) - [Bin-picking sample](#sample-bin-picking) - [Visual inspection](#visual-inspection-a-cross-companion-cell) ## Models and provenance | Model | CLR namespace | Wire namespace | Loader | | --- | --- | --- | --- | | Perception model | `Opc.Ua.Vision.Perception` | `http://opcfoundation.org/UA/Vision/` | `AddOpcUaVisionPerception` | | OPC 40100-1, specification 1.0 | `Opc.Ua.Vision.MachineVision` | `http://opcfoundation.org/UA/MachineVision` | `AddOpcUaMachineVision` | | OPC 40100-2, specification 1.00.0 | `Opc.Ua.Vision.MachineVision.AMCM` | `http://opcfoundation.org/UA/MachineVision/AMCM/` | `AddOpcUaMachineVisionAMCM` | Namespace URIs are exact identifiers: Part 1 has **no trailing slash**. Part 2 has one. Part 1 depends on UA core only. Part 2 depends on UA core, Device Integration, and Machinery, **not Part 1**. The Part 2 package references the DI and Machinery model producers; selecting one released model does not load the other. Licensed, release-pinned NodeSets and identifier tables are under [`src/Opc.Ua.Vision/Model`](../src/Opc.Ua.Vision/Model) for Part 1 and [`src/Opc.Ua.MachineVision/Model`](../src/Opc.Ua.MachineVision/Model) for Part 2. Repository-compatible identifier guards are derived with `tools/nodesets/derive-identifier-table.py`. | Part | Upstream release commit | NodeSet version and publication | SHA-256 | | --- | --- | --- | --- | | 1 | [`ba1b53eedd87da6ee9c79ef76e85f9f1d7bff0d0`](https://github.com/OPCFoundation/UA-Nodeset/tree/ba1b53eedd87da6ee9c79ef76e85f9f1d7bff0d0/MachineVision) | `1.0.0`, `2019-07-11T10:18:27Z` | `81B77250E5DE8D4FCCEDA7E05FD32BF199299CAAC32D15DF8F1FE25ED907C053` | | 2 | [`5cbfb6506684ffba82610ea50a95927389f46b8e`](https://github.com/OPCFoundation/UA-Nodeset/tree/5cbfb6506684ffba82610ea50a95927389f46b8e/MachineVision/AMCM) | `1.00.0`, `2024-05-17T00:00:00Z` | `C70ED87915AB9A9F72F471C1B852A16EF7596921C0937A93FE2C3F73A6C0EC1F` | The document publication date for Part 1 is 2019-08-01, distinct from its NodeSet date. Part 2's required floors are DI 1.04.0 and Machinery 1.03.0; the implementation reuses the repository's DI 1.05.0 and Machinery 1.04.1 models and their dependency closure. Generated states, method signatures, structures, proxies, and server fluent accessors come from those model producers rather than hand-maintained copies. ## Opt-in hosting `VisionServerOptions.MachineVisionParts` defaults to `None`. Select `Control`, `Assets`, or `All` independently. Part 2 builders require `Opc.Ua.MachineVision.Server` and the `AddMachineVision()` registration for a standalone manager. This example assumes an application implementation of `IMachineVisionOperationsProvider` has been registered with DI: ```csharp using Microsoft.Extensions.DependencyInjection; using Opc.Ua; using Opc.Ua.Machinery.Server.Builders; using Opc.Ua.Vision; using Opc.Ua.Vision.Server; builder.Services.AddSingleton(); builder.Services.AddOpcUa() .AddServer(options => options.EndpointUrls.Add("opc.tcp://localhost:62550/MachineVision")) .AddVision(options => options.MachineVisionParts = MachineVisionParts.All) .AddMachineVision() .ConfigureVision(async (context, ct) => { MachineVisionSystemHandle system = await context.AddMachineVisionSystemAsync( "Inspection", context.GetRequiredService(), cancellationToken: ct); MachineVisionAssetBuilder asset = context.AddMachineVisionAsset("InspectionAsset") .WithIdentification(new MachineryIdentificationData { Manufacturer = new LocalizedText("Example manufacturer"), SerialNumber = "SYSTEM-001", ProductInstanceUri = "urn:example:vision:system-001" }) .RepresentsSameSystem(system.State); asset.AddItem(MachineVisionItemKind.ImageSensor, "Camera") .WithIdentification(new MachineryIdentificationData { Manufacturer = new LocalizedText("Example manufacturer"), SerialNumber = "CAMERA-001" }) .WithHealth(health => health.Set("Temperature", Variant.From(22.5))); await asset.BuildAsync(ct); }); ``` For a Machinery host, use **one owner** for DI and Machinery: ```csharp builder.Services.AddOpcUa() .AddServer(options => options.EndpointUrls.Add("opc.tcp://localhost:62550/MachineVision")) .AddMachinery(options => options.Parts = MachineryParts.BuildingBlocks) .AddVisionToMachinery() // Selects both released parts by default. .ConfigureVisionFor(async (context, ct) => { await context.AddMachineVisionSystemAsync( "Inspection", context.GetRequiredService(), cancellationToken: ct); }); ``` Adding a second standalone asset manager to a DI/Machinery-owned server throws an `InvalidOperationException` during hosting registration: both managers would claim the same model namespaces and shared roots. This restriction concerns **node-manager ownership**, not the number of vision systems. ### Multiple vision systems Create each system under the same owner with a unique browse name and its own operations provider. For example, `LineOneOperations` and `LineTwoOperations` implement `IMachineVisionOperationsProvider` for separate devices: ```csharp builder.Services.AddSingleton(); builder.Services.AddSingleton(); builder.Services.AddOpcUa() .AddServer(options => options.EndpointUrls.Add("opc.tcp://localhost:62550/InspectionLines")) .AddMachinery(options => options.Parts = MachineryParts.BuildingBlocks) .AddVisionToMachinery() .ConfigureVisionFor(async (context, ct) => { MachineVisionSystemHandle lineOne = await context.AddMachineVisionSystemAsync( "LineOne", context.GetRequiredService(), cancellationToken: ct); MachineVisionSystemHandle lineTwo = await context.AddMachineVisionSystemAsync( "LineTwo", context.GetRequiredService(), cancellationToken: ct); await context.AddMachineVisionAsset("LineOneAsset") .WithIdentification(new MachineryIdentificationData { Manufacturer = new LocalizedText("Example manufacturer"), SerialNumber = "LINE-001", ProductInstanceUri = "urn:example:vision:line-one" }) .RepresentsSameSystem(lineOne.State) .BuildAsync(ct); await context.AddMachineVisionAsset("LineTwoAsset") .WithIdentification(new MachineryIdentificationData { Manufacturer = new LocalizedText("Example manufacturer"), SerialNumber = "LINE-002", ProductInstanceUri = "urn:example:vision:line-two" }) .RepresentsSameSystem(lineTwo.State) .BuildAsync(ct); }); ``` Each handle owns its state machine, job state, documents, conditions, and transfers. The owner supplies their shared lifetime and address-space registration. Duplicate system/asset browse names are rejected with `BadBrowseNameDuplicated`. Applications can deliberately share a provider or store, but must choose document identities and hardware scheduling appropriate to that sharing. `ConfigureVisionOnMachinery` supports an application-owned DI manager that loads the required models through its Machinery model-provider pipeline. Such a custom manager follows Machinery's `DisposeMachineryResourcesAsync` contract. The stock manager handles disposal itself. Direct construction is also supported: call `MachineVisionSystemHandle.CreateAsync` or construct `MachineVisionAssetBuilder` with an `IVisionBuildContext`. A directly owned runtime is `IAsyncDisposable`; hosted contexts register its lifetime with their owner. For direct Part 2 setup, include `VisionMachineryStandaloneModelProvider` alongside `VisionModelProvider` in the `VisionNodeManager` provider array. Factories validate model availability and instance names. Application instance namespaces must remain distinct from the three model namespaces. ## Control and acquisition providers The runtime owns OPC UA state, method validation, document metadata, transfer handles, events, and conditions. `IMachineVisionOperationsProvider` owns hardware behavior and opaque payload interpretation: | Provider member | Responsibility | | --- | --- | | `ControlAsync` | Apply Halt, Reset, mode selection, Stop, Abort, and simulation changes before the state transition is committed. | | `PrepareRecipeAsync` | Prepare or unprepare committed content; completion means the operation has finished. | | `ActivateConfigurationAsync` | Apply a committed configuration before it becomes active. | | `ValidateDocumentAsync` | Accept or reject uploaded bytes before the store commit. | | `StartJobAsync` | Validate and accept the proposed job context. | | `ExecuteAsync` | Yield acquisition notifications and optional partial/final results; ending the stream completes the job. | | `ExecuteStepAsync` | Perform one externally synchronized step and report whether another step is required. | All callbacks are asynchronous. Commands for one system are serialized; its execution stream can run concurrently with control calls. Providers must honor cancellation and must not re-enter a serialized command from inside that command. Stop, Abort, Halt, Reset, and runtime disposal cancel the stream. Results from superseded executions are not published into a new job. `MachineVisionOperationResult` keeps **service status and domain error separate**. Domain zero means success; negative values are application errors; positive values are reserved by the specification. A successful UA Call response can therefore still contain a failed domain result. Generated proxies preserve that result; workflow helpers use `MachineVisionSystemClient.ThrowIfDomainError`. A new system starts in `Preoperational`. `SelectModeAutomatic` enters `Initialized`; provider readiness or a prepared recipe permits `Ready`. `InitiallyReady` and `SetReadyAsync` support providers that do not expose recipe management. Reset clears provider readiness and prepared-recipe flags; it does not erase stored recipe content. No call implicitly selects automatic mode, prepares a recipe, or activates a configuration. `MachineVisionSystemOptions.Features` selects optional families; its default for a newly created native system is `All`. Control remains available with `None`. The families are Recipes, Configurations, Results, Folders, FileTransfer, Steps, Simulation, Safety, and Diagnostics. Namespace selection and per-system functionality selection are separate decisions. ### Step models, events, and conditions The seven parent state numbers are 1 Preoperational, 2 Halted, 3 Error, 5 Initialized, 6 Ready, 7 SingleExecution, and 8 ContinuousExecution. `SetStepSequenceAsync(number, true)` enables a sequence for that parent. `Sync` executes a waiting step. `SynchronizeStepAsync` provides the equivalent application-side trigger using `WaitToStepAuto`. Both use the same provider, index, and state/event implementation. An initialized sequence must finish before automatic readiness; an execution sequence must finish before its acquisition stream begins. An in-progress sequence or running job cannot be reconfigured. Use `ReportErrorAsync` for application errors. Acknowledgement and confirmation are independent of resolving the physical error: an active error stays retained after acknowledgement/confirmation. Resolving the final blocking error returns an errored system to Preoperational. Persistent errors also require acknowledgement and confirmation after their physical cause is remedied. ConfirmAll applies the same recovery decision as individual Confirm calls, including physically resolved conditions awaiting confirmation. Reset may leave Error for Preoperational but cannot re-enable production while a blocking condition remains unresolved. `ReportWarningAsync` provides the same acknowledgement/optional-confirmation lifecycle without interrupting production. Diagnostic, information, warning, error, and persistent-error events retain their distinct severity bands; a method diagnostic identifies the failed method as its SourceNode without exposing argument payloads. State transitions, readiness, job starts, acquisitions, results, recipe preparation, step entry/exit, errors, and safety changes use the model's event types. Failed bound methods, including argument validation rejected before the handler, produce a warning diagnostic. Argument payloads are not copied into diagnostic text. Diagnostic information obeys the selected level; `ReportInformationAsync` and `SetSystemStateAsync` provide application-side reporting. The shared `MethodState.OnCallValidationFailedAsync` callback observes asynchronous validation refusals without changing their service result or per-argument errors. It runs only when an observer is installed and is not invoked by synchronous Call. Observer failures are logged without replacing the validation result; requested caller cancellation propagates. `ReadyEventType` signals the end of a single or continuous execution, including Stop and Abort, and carries that job's identifier and the automatic state machine's SourceNode. Initial readiness is a state transition, not a completed-job event. Safety reporting is **not a safety controller or safety certification**. It must not replace an independently certified safety circuit. ## Recipes, configurations, results, and files All management methods are available through the generated sub-clients and the server runtime: recipe registration/preparation/removal, product links and preparation, configuration registration/activation/removal, result retrieval/components/filtering, and metadata-handle release. Optional folders expose the same store-backed records. Explicit internal recipe identifiers take precedence over external identifiers. Version and hash restrictions are preserved; result states remain extensible Int32 values. Preparation and activation record durable intent before invoking the hardware provider. A storage refusal therefore does not change the hardware's accepted state. Provider refusals restore the previous store record before returning; published active/readiness values change only after success. If that rollback also fails, production is blocked and the failure reports both causes. Repair the store and recreate the runtime rather than treating Reset as a persistence repair. On restart, saved intent is reapplied through the provider before it becomes available to clients. Registering metadata and transferring content are separate operations. AddRecipe and AddConfiguration report whether transfer is required. A temporary writer is already open when GenerateFileForWrite returns. CloseAndCommit validates and publishes content; ordinary Close discards an uncommitted temporary writer. Result transfers are read-only. The direct recipe FileType uses commit-on-close. These are **OPC 40100-1 protocols**, not replacements with the different OPC 40001-101 result protocol. File bindings reuse the stack's session-bound file-handle implementation, with independent reader positions, one writer, bounded files, and owned cleanup. Closing or expiring a session releases its handles without committing an abandoned writer; other sessions' readers remain open. Removing a recipe disposes its file binding and pending write before removing the published nodes, releasing the retained content and streams. The default per-system limits are 64 metadata handles, a five-minute metadata retention bound, eight temporary transfers, 16 MiB per transferred file, and a two-minute transfer inactivity timeout. Client metadata timeouts are retention hints, not request deadlines. Pagination accepts zero MaxResults for all remaining results; a nonzero MaxResults requires a StartIndex that is a multiple of that page size. ### Storage `IMachineVisionStore` is injectable per system. The default `InMemoryMachineVisionStore` clones metadata/content and limits storage to 1,024 documents and 64 MiB of opaque bytes. Capacity exhaustion is explicit; records are not silently evicted. `FileSystemMachineVisionStore` uses `IFileSystemProvider`, not ambient file paths. It stores namespace-aware binary records, hashes logical identifiers into safe filenames, stages replacement data, and recovers an interrupted replacement from its previous version. It restores prepared recipes and the active configuration through the provider when a runtime is rebuilt. The caller owns an explicitly supplied persistent store and its provider. Recovery removes incomplete `.pending` records before reading or replacing a document; enumeration also removes orphaned first writes. Cleanup runs under the store's operation gate and supports the earlier `.pending-` staging format. Incomplete writes are never returned as committed documents. The reference persistent store requires an **exclusive root on one host**. It is not a distributed database, consensus implementation, or multi-writer HA store. An application needing those guarantees supplies its own `IMachineVisionStore` rather than sharing this root across servers. ## Part 2 assets and live monitoring `MachineVisionAssetBuilder` covers all 25 component categories: computing devices, displays, physical interfaces, image sensors, frame grabbers, lenses, lens controllers, optical filters, other optical equipment, way encoders, trigger sensors, lamps, lighting controllers, pattern generators, calibration targets, acquisition backgrounds, surrounding environment, housings, motion devices, power supplies, climate controllers, licenses, software components, network devices, and cables. Every item has IVisionInfo identification. Unused category folders are omitted; a materialized category must satisfy its MandatoryPlaceholder. Manufacturer and serial number are required; system assets also require an absolute ProductInstanceUri. `WithComponents` exposes the same items through Machinery's Components add-in. Assets are discoverable through Machinery's Machines folder. `WithMaintenance`, `WithHealth`, `Child`, `Set`, and `Value` reach declared, typed optional members, including warranty, service/calibration dates, firmware, maintenance records, counters, remaining lifetime, temperature and engineering units, DI DeviceHealth, DeviceHealthAlarms, and the Part 2 SEMI E10 state structure. Undeclared names or incompatible Variant types are rejected instead of creating arbitrary vendor properties. ```csharp await assetHandle.UpdateAsync(asset => asset .WithMaintenance(maintenance => maintenance .Set("CalibrationNeeded", Variant.From(true))) .WithHealth(health => health.Set("Temperature", Variant.From(24.0))), ct); ``` Updates register newly materialized optional members and notify normal OPC UA data subscriptions. The callback edits an isolated snapshot: an invalid value or a thrown callback leaves the published values and children unchanged. Existing node identities and types cannot be replaced by a telemetry update. Retain the asset handle, not a snapshot builder captured inside an update callback. Hosted contexts own disposal; directly owned asset handles must be disposed asynchronously. Application history and maintenance-alarm providers use the stack's [Historical Access](HistoricalAccess.md) and alarm/DI integration, not a second vision-specific historian. Populate only measurements and maintenance facts your provider actually knows. `AddImageHandlingAspect` creates receiver, transmitter, or transceiver aspects. `HasOpticalPathTo` and `TransmitsDataTo` describe topology. Existing generated/vendor components can be included without replacing their TypeDefinition. Already parented or registered components are linked rather than reparented or deleted with a new asset; explicit fluent value configuration on those components is live. Part 2 defines no control methods. ## Shared identity, not duplicate execution Part 1 VisionSystemType and Part 2 VisionSystemAssetType remain distinct objects. `RepresentsSameSystem` adds the required `RepresentsSameEntityAs` relationship when they describe the same physical system. Use the same reference for deliberately associated draft sensors/components; matching names alone do not establish equivalence. The [server sample](../samples/Vision/MachineVisionServer) routes native commands and the draft inference pipeline through **one deterministic engine**. One acquisition publishes both result representations with the same result identity. It does not run an inspection twice to fill two models. Part 1's SEMI E10 enum/description and Part 2's Id/Priority structures are different wire types. Map application facts deliberately; do not cast between them or equate their numeric values. ## Typed clients Use the `AddVisionClient()` DI factory or `session.Vision(telemetry)`: ```csharp VisionClient vision = session.Vision(telemetry); MachineVisionClient native = vision.MachineVision(); await foreach (VisionNodeEntry entry in native.EnumerateSystemsAsync(cancellationToken: ct)) { MachineVisionSystemClient system = native.System(entry.NodeId); VisionStateMachineTypeClient control = await system.ControlAsync(ct); FiniteStateSnapshot state = await control.GetCurrentFiniteStateAsync(ct); } ``` Control and asset capability checks are independent; released-model discovery does not require the Perception namespace. Discovery recognizes subtypes and deduplicates graph visits. Asset snapshots retain browse-name namespaces, NodeIds, statuses, timestamps, and an explicit IsComplete flag when a depth/value limit is reached. The Control, Automatic, Recipes, Configurations, Results, and Safety sub-clients are the generated proxies, including optional child accessors and all declared method arguments. Use their state-machine and streaming-subscription extensions for observation. `UploadRecipeAsync`, `UploadConfigurationAsync`, and `DownloadResultAsync` reuse the standard temporary-file client. Uploads wait for a returned asynchronous completion state machine and surface its Error state; cancellation bounds the wait. Downloads release metadata handles and preserve both errors if transfer and release fail. The [client sample](../samples/Vision/MachineVisionClient) demonstrates secure discovery, upload, preparation, execution, state waiting, results, the draft/native bridge, and assets. ## Implementation and conformance map This map identifies implementation surfaces; it is **not a certification claim**. The specification's server/client facets and Part 2 VSA bundles are selected by actual application functionality, not merely by loading a NodeSet. No unverified OPC Foundation profile URI is invented or added to ServerProfileArray. The draft VIS-* facets remain the draft's separate reporting mechanism. The handles implement `IConformanceContributor`. Their specification-named units use the released model's namespace index to avoid collisions with generic names such as Automatic Mode. Standalone Vision and composed Machinery hosts aggregate these units through their server-capabilities pipeline. Part 2 units are derived from published instances, optional add-ins, and optical/data references; empty or rolled-back assets contribute nothing. Provider-dependent processing-time claims are not inferred, and the model's cable category does not acquire invented cable-specific units absent from Part 2 table 74. | Requirement family | Implementation | Representative regression evidence | | --- | --- | --- | | Released model identity and defaults | `MachineVisionModels`, model inputs, `VisionServerOptions` | `MachineVisionModelTests` | | Control, job identity, results | `MachineVisionSystemHandle`, state tables, operations provider | `AutomaticModeRunsOneJobPublishesResultAndReturnsReadyAsync` | | Initialization and synchronization | Steps partial and all seven slots | `InitializedStepSequenceDelaysReadyUntilSyncAsync`, `ApplicationSynchronizationUsesAutomaticStepTransitionAsync` | | Reset and selected optional surfaces | Control/options | `ResetClearsProviderReadinessAsync`, `DisabledDiagnosticsCannotMaterializeSystemStateAsync` | | Error payload, acknowledgement, confirmation | Events partial and core condition retention provider | `ReportedErrorPreservesApplicationDetailsAsync`, `AcknowledgementUsesReportedEventIdAndRetainsActiveErrorAsync`, `PersistentErrorPublishesCallableConfirmationAsync` | | Failed-call diagnostics | Bound methods and MethodState validation observer | `MissingControlArgumentsPublishFailureDiagnosticAsync` | | Recipe preparation and file commit/abort | Management, Documents, Transfers, `IFileTypeBinder` | `RecipeTransferCommitsContentThenPreparationEnablesReadyAsync`, `ClosingAnUncommittedRecipeTransferDiscardsItsBytesAsync` | | Result version filters | Management and generated result proxy | `ResultFilterHonorsExternalRecipeVersionAsync` | | Session-owned readers/writer | Shared FileType binding | `SharedFileBindingIsolatesReadersAndCommitsOneWriterAsync`, `SharedFileBindingReleasesClosedSessionHandlesAsync` | | Recipe file retirement | Documents and shared FileType binding | `RemovingRecipesReleasesTheirPublishedFileBindingsAsync`, `RemovingRecipeCancelsPendingFileCommitWithoutDeadlockingAsync` | | Bounded and persistent storage | Memory and file-system stores | `MemoryStoreClonesMetadataAndRefusesCapacityOverflowWithoutEvictionAsync`, `FileSystemStoreSurvivesRestartAndRecoversAnInterruptedReplacementAsync` | | Part 2 categories, identity, and health | Asset and declared-member builders | `AssetBuilderPublishesEveryComponentCategoryAndLiveHealthAsync` | | Component sharing and root identity | Reference-based adoption and root-name reservation | `AssetsCanShareAComponentWithoutReparentingOrReplacingItAsync`, `TwoStagedAssetsCannotPublishTheSameRootNameAsync` | | Standalone/Machinery ownership, both models, and TCP workflows | Hosting adapters and typed clients | `CompositionSupportsBothPartsAndSessionFileCleanupOverTcpAsync` | The source generator also has regressions for recursive optional declarations, a ResultState child/type name collision, and narrowed proxy method signatures. Those fixes preserve the upstream models rather than modifying their declarations to compile. ## References - [OPC 40100-1](https://reference.opcfoundation.org/specs/OPC-40100-1) - [OPC 40100-2](https://reference.opcfoundation.org/specs/OPC-40100-2) - [RepresentsSameEntityAs](https://reference.opcfoundation.org/specs/OPC-10000-23/4.8) - [Machinery hosting](Machinery.md), [model dependencies](ModelDependencies.md), [Perception model](#perception-model) ## Perception model The Perception model describes sensor capture, inference pipelines, feedback, results, media, and coordinate frames. Its CLR namespace is `Opc.Ua.Vision.Perception`; its providers and fluent hosting use the Vision client/server modules. It can share a server and hardware providers with the released Machine Vision models. > **Draft specification.** The Perception wire namespace > `http://opcfoundation.org/UA/Vision/` and its NodeIds are provisional. > Do not confuse its result/pose types and VIS-* facets with OPC 40100. ## Packages | Package | What it gives you | Depends on | |---|---|---| | `OPCFoundation.NetStandard.Opc.Ua.Vision` | Source-generated Perception and OPC 40100-1 models, node states, typed proxies, and distinct model loaders | `Opc.Ua.Core` | | `OPCFoundation.NetStandard.Opc.Ua.Vision.Server` | `VisionNodeManager`, `IVisionBuildContext`, Perception and Part 1 providers and builders, facet derivation, standalone or Machinery-composed hosting | `Opc.Ua.Vision`, `Opc.Ua.Server`, `Opc.Ua.Machinery.Server` | | `OPCFoundation.NetStandard.Opc.Ua.Vision.Client` | `VisionClient` discovery, `VisionSensorClient`, `VisionPipelineClient`, `VisionResultReader`, `VisionMediaClient`, `VisionFrameGraph`, `VisionFeedbackClient`, `session.Vision(...)` extension, `AddVisionClient()` DI | `Opc.Ua.Vision`, `Opc.Ua.Client` | | `OPCFoundation.NetStandard.Opc.Ua.MachineVision` | OPC 40100-2 asset models and generated proxies in `Opc.Ua.Vision.MachineVision.AMCM` | `Opc.Ua.Core`, `Opc.Ua.Di`, `Opc.Ua.Machinery` | | `OPCFoundation.NetStandard.Opc.Ua.MachineVision.Server` | Asset builders, information add-ins, conformance, runtime updates, and Part 2 hosting registration | `Opc.Ua.Vision.Server`, `Opc.Ua.MachineVision`, `Opc.Ua.Machinery.Server` | | `OPCFoundation.NetStandard.Opc.Ua.MachineVision.Client` | Part 2 asset discovery, snapshots, and relationship extensions in `Opc.Ua.Vision.Client` | `Opc.Ua.Vision.Client`, `Opc.Ua.MachineVision` | | `OPCFoundation.NetStandard.Opc.Ua.Vision.OpenUsd` | `ISceneCameraCaptureProvider` implementation that renders a `UsdGeomCamera` offscreen and reports `NoRenderingBackend` gracefully when no graphics device is available | `Opc.Ua.Types`, native OpenUSD renderer payload (optional per-RID) | | `OPCFoundation.NetStandard.Opc.Ua.Mcp.Vision` | Agent-facing tools | `Opc.Ua.Mcp.Core`, `Opc.Ua.Vision.Client`, `Opc.Ua.MachineVision.Client` | ```mermaid graph TD Model["Opc.Ua.Vision
source-generated model"] Server["Opc.Ua.Vision.Server
VisionNodeManager + builders"] Client["Opc.Ua.Vision.Client
typed discovery and readers"] Assets["Opc.Ua.MachineVision
Part 2 model"] AssetServer["Opc.Ua.MachineVision.Server"] AssetClient["Opc.Ua.MachineVision.Client"] OpenUsd["Opc.Ua.Vision.OpenUsd
scene-camera capture provider"] Mcp["Opc.Ua.Mcp.Vision
agent tools"] Media["IVisionMediaProvider"] Inference["IVisionInferenceProvider"] Feedback["IVisionFeedbackSink"] Model --> Server Model --> Client Client --> Mcp Assets --> AssetServer Assets --> AssetClient Server --> AssetServer Client --> AssetClient AssetClient --> Mcp OpenUsd -.->|captures frames for| Media Media -.->|plugs into| Server Inference -.->|plugs into| Server Feedback -.->|plugs into| Server ``` `Opc.Ua.Vision.OpenUsd` deliberately sits outside that dependency chain: it depends on `Opc.Ua.Types` alone and knows nothing about the Vision server. It offers a camera, and a host writes the `IVisionMediaProvider` that hands the resulting frames to a pipeline — which is what [`BinPickingCell`](../samples/Robotics/BinPickingCell) does. A host with a real camera writes the same interface over its own SDK and never references OpenUSD at all. The runtime address space is rooted under the standard Server object. Configurators add frames, sensors and pipelines through the fluent builder; providers plug into the sensor or pipeline nodes that the builder creates. ```mermaid flowchart TD ServerObj["Server"] --> VisionRoot["Vision
VisionRootType"] VisionRoot --> Sensors["Sensors"] VisionRoot --> Frames["Frames"] VisionRoot --> Pipelines["Pipelines"] Frames --> Frame["CoordinateFrameType
FrameId + Transform"] Sensors --> Sensor["VisionSensorType
ImageSensorType / Depth3DSensorType"] Sensor --> Calibrations["Calibrations"] Calibrations --> Intrinsic["IntrinsicCalibrationType"] Calibrations --> Extrinsic["ExtrinsicCalibrationType"] Sensor --> Media["Media
VisionMediaManagementType"] Media --> Streams["StreamEndpoints"] Media --> Clips["ClipEndpoints"] Sensor -.->|MountedOn| Frame Sensor -.->|HasScenePrim| Scene["OpenUSD scene prim"] Pipelines --> Pipeline["InferencePipelineType"] Pipeline --> Results["Results"] Pipeline --> Feedback["Feedback
VisionFeedbackType"] Pipeline -.->|Sensor| Sensor Pipeline -.->|Deployment| Deployment["AI deployment NodeId"] ``` The libraries multi-target `net8.0;net9.0;net10.0` and `netstandard2.0` where applicable; the MCP tool package multi-targets `net8.0;net9.0;net10.0`. ## Two perception paths behind one contract Every pipeline advertises exactly one of two inference locations, and a client reads a `DetectionResultType` identically regardless of which is in force: - **`InferenceLocation = OnServer`** — the Server holds an `IVisionInferenceProvider` and computes results locally. This is the deterministic path: it needs no model, no network and no GPU, and it is the default for CI and offline validation. `RunInference`, `StartContinuous` and `Stop` all delegate to the provider; the Server publishes the resulting `DetectionResultType` / `InspectionResultType` / `SegmentationResultType` under the pipeline's `Results` folder and advertises `VIS-Inference-OnServer`. - **`InferenceLocation = EdgeOffServer`** — the pipeline exposes a `VisionFeedbackType` object bound to an `IVisionFeedbackSink`; an off-Server agent (a vision-language model over MCP, an edge inference service, another Server) is expected to look at the current frame and call `SubmitDetections` / `SubmitInspectionResult` / `SubmitCorrection` / `SubmitImageReference`. The Server publishes those results into the address space unchanged, and advertises `VIS-Inference-OffServer`. Choose `OnServer` when a deterministic algorithm answers the question (vision-guided screwdriver alignment against a known fiducial; presence-or- absence in a clean scene) or when reproducibility on CI matters. Choose `EdgeOffServer` when a language model or a heavier out-of-process model is what actually sees the world — the Server publishes what it did not compute, and safety validation still applies (§9 refusals, class-label / box / pose / confidence checks; see [Feedback validation](#feedback-validation)). The two paths are exclusive per pipeline by design: mixing a running `OnServer` provider with a `SubmitDetections` sink would let a computed and a submitted result publish on the same pipeline out of any known order. ```mermaid flowchart LR Client["Client"] Server["Vision Server"] Provider["IVisionInferenceProvider"] Edge["Off-server agent"] Sink["IVisionFeedbackSink"] Results["Pipeline Results folder"] Client -->|RunInference| Server Server -->|OnServer delegates| Provider Provider -->|result id + result node| Results Edge -->|reads frame| Server Edge -->|SubmitDetections / SubmitInspectionResult| Server Server -->|EdgeOffServer delegates| Sink Sink -->|published result| Results Client -->|reads same result types| Results ``` ## Minimal hosted server The example below hosts a single simulated eye-in-hand camera, one inference pipeline, and a two-frame tree (`world` → `flange`). It is the smallest useful shape; the [`BinPickingCell`](../samples/Robotics/BinPickingCell) sample is the full end-to-end version. ```csharp using Microsoft.Extensions.Hosting; using Opc.Ua; using Opc.Ua.Server; using Opc.Ua.Vision; using Opc.Ua.Vision.Perception; using Opc.Ua.Vision.Server; using Opc.Ua.Vision.Server.Builders; HostApplicationBuilder builder = Host.CreateApplicationBuilder(args); builder.Services.AddSingleton(); builder.Services.AddSingleton(); builder.Services .AddOpcUa() .AddServer(options => { options.ApplicationName = "VisionServer"; options.ApplicationUri = "urn:localhost:OPCFoundation:VisionServer"; options.AutoAcceptUntrustedCertificates = true; options.EndpointUrls.Add("opc.tcp://localhost:62855/VisionServer"); }) .AddVision(options => { options.InstanceNamespaceUri = "urn:example:vision:instances"; }) .AddVisionMediaProvider(sensorBrowseName: "Camera01") .AddVisionInferenceProvider( pipelineBrowseName: "Detector", onServer: true) .ConfigureVision((context, ct) => { IVisionNodeBuilder nodes = context.Nodes; // Nodes is the fluent address-space builder: everything it creates // becomes real OPC UA nodes under Server/Vision that any client can // browse. The full build context is described further down. nodes.AddFrame("World", frame => frame .WithFrameId("world") .WithRole(VisionFrameRoleEnum.World)); nodes.AddFrame("Flange", frame => frame .WithFrameId("flange") .WithRole(VisionFrameRoleEnum.MechanicalInterface) .WithParent("world")); nodes.AddImageSensor("Camera01", sensor => sensor .WithSensorId("cam-01") .WithModality(VisionSensorModalityEnum.Area2D) .WithRealityKind(VisionRealityKindEnum.Physical) .WithFrameId("flange") .WithResolution(1920u, 1080u) .WithPixelFormat("Mono8") .AddClipEndpoint("Clips", ep => ep .WithEndpointId("clip-01") .WithEndpointUri("opcua-inline://visionserver/clips") .WithClipFormat(VisionClipFormatEnum.Png) .WithResolution(1920u, 1080u) .WithInlineDelivery(enabled: true, maxInlineClipSize: 8_388_608u))); // The pipeline is bound to its inference provider by // AddVisionInferenceProvider("Detector", ...) // above; the ConfigureVision delegate just creates the pipeline // node. A real cell resolves the sensor node the pipeline is // reading from off Server/Vision/Sensors — see the BinPickingCell // sample for the walk. nodes.AddPipeline("Detector", pipe => pipe .WithPipelineId("pipe-01") .WithSensor(NodeId.Null)); return ValueTask.CompletedTask; }); using IHost app = builder.Build(); await app.RunAsync().ConfigureAwait(false); ``` `AddVision` never modifies the Server's existing NodeManagers — it adds a standalone `VisionNodeManager` under the well-known `Server/Vision` object (§4.2). It also composes with `AddRobotIntent` and `AddRobotics`; both sample cells run all three side by side without any coupling in code. The `AddVisionMediaProvider` / `AddVisionInferenceProvider` / `AddVisionFeedbackSink` extensions resolve the provider from the DI container at build time and bind it to the sensor or pipeline whose BrowseName is passed. The equivalent `UseMediaProvider(provider)` / `UseInferenceProvider(provider, onServer)` / `UseFeedbackSink(sink)` methods on the fluent builders let a configurator bind a provider it holds directly. ## Hosting API The extension methods on `IOpcUaServerBuilder` that make up the Vision hosting surface: | Method | Purpose | |---|---| | `AddVision(Action?)` | Registers the standalone `VisionNodeManager` and its factory; accepts an optional options delegate | | `AddVisionMediaProvider(string sensorBrowseName)` | Registers a media provider type — resolved from DI — for the sensor with the given BrowseName | | `AddVisionMediaProvider(string sensorBrowseName, IVisionMediaProvider provider)` | Registers a media provider instance for the sensor with the given BrowseName | | `AddVisionInferenceProvider(string pipelineBrowseName, bool onServer)` | Registers an inference provider type — resolved from DI — for the pipeline; `onServer` controls the advertised `VIS-Inference-OnServer` / `VIS-Inference-OffServer` facet (§8.2) | | `AddVisionInferenceProvider(string pipelineBrowseName, bool onServer, IVisionInferenceProvider provider)` | Registers an inference provider instance | | `AddVisionFeedbackSink(string pipelineBrowseName)` | Registers a feedback sink type — resolved from DI — for the pipeline's `Feedback` object | | `AddVisionFeedbackSink(string pipelineBrowseName, IVisionFeedbackSink sink)` | Registers a feedback sink instance | | `ConfigureVision(Func)` | Async configurator, run on server start against the standalone `VisionNodeManager` | | `ConfigureVision(Action)` | Sync configurator overload | | `ConfigureVisionFor(...)` | Configurator targeting `VisionNodeManager` or the stock `MachineryNodeManager` | | `AddMachineVision()` | Registers Part 2 models/builders with a standalone Vision owner | | `AddVisionToMachinery(...)` | Loads the selected Vision models into a Machinery owner; selects both released parts by default | | `ConfigureVisionOnMachinery(...)` | Configures an application-owned DI/Machinery manager that loads the Vision models | The Robotics guide's `ConfigureFor<...>` pattern also applies here — any of these hosting extensions can be called from a class-based configurator that reads `IServiceProvider`, keeps its own logger, and does not put a lambda in `Program.cs`. ### `VisionServerOptions` | Property | Purpose | |---|---| | `InstanceNamespaceUri` | The application-owned namespace URI used for the instances the configurator materialises. Must be distinct from the OPC UA base namespace and from `http://opcfoundation.org/UA/Vision/`. Defaults to `urn:opcfoundation:UA:Vision:Instances`. | | `SpecificationVersion` | The value the Server reports on `Vision.SpecificationVersion`. Defaults to `"0.1.0"`. | | `AdditionalFacets` | The facets the Server declares beyond those the facet calculator derives structurally — the escape hatch for facets whose requirements are behavioural (an interop facet that the host meets by contract). | | `MachineVisionParts` | Opt-in `Control`, `Assets`, or `All` for the released OPC 40100 models. Defaults to `None`, selecting the Perception model without released-model instances. | ## Build context `ConfigureVision(...)` receives an `IVisionBuildContext`. Here `Nodes` is the fluent address-space builder rooted at the well-known `Server/Vision` object; it creates the frame, sensor, calibration, media-endpoint and pipeline nodes that ordinary OPC UA clients browse and read. | Member | Purpose | |---|---| | `Nodes` | The fluent `IVisionNodeBuilder` rooted at the well-known `Server/Vision` object | | `Manager` | The active `AsyncCustomNodeManager`, for the rare case a configurator must fall back to raw node authoring | | `Context` | The active `ISystemContext` | | `Root` | The `VisionRootState` (§4.2) | | `InstanceNamespaceIndex` | The namespace index of `VisionServerOptions.InstanceNamespaceUri` | | `VisionNamespaceIndex` | The namespace index of `http://opcfoundation.org/UA/Vision/` | | `CancellationToken` | The startup cancellation token | | `GetRequiredService()` | Application-scoped DI resolution | Everything a Vision cell needs (frames, sensors, pipelines, calibrations, media endpoints) is authored through `Nodes`. The low-level members are present for interop with hand-written NodeManagers and for the vendor extension patterns the Robotics guide describes. ### Without DI A `VisionNodeManager` created by hand exposes the same fluent surface through `ConfigureVisionAsync`: ```csharp await manager.ConfigureVisionAsync(context => { context.Nodes.AddFrame("World", f => f .WithFrameId("world") .WithRole(VisionFrameRoleEnum.World)); }); ``` Prefer it over `CreateVisionBuildContext()`. The node manager indexes the Vision root when it creates the address space, so anything the builder grafts on afterwards has to be registered as well before it can be browsed or read **by its own NodeId** — which is how an ordinary client and the MCP discovery tools navigate. `ConfigureVisionAsync` does that registration when the delegate returns; a context obtained from `CreateVisionBuildContext()` never does, so nodes built through it stay reachable only by browsing forward from their parent. ## Topology builders ### Frames ```csharp nodes.AddFrame("World", f => f .WithFrameId("world") .WithRole(VisionFrameRoleEnum.World)); nodes.AddFrame("RobotBase", f => f .WithFrameId("robot_base") .WithRole(VisionFrameRoleEnum.Base) .WithParent("world") .WithTransform(new VisionPose3DDataType { FrameId = "world", Position = new[] { 0.0, 0.0, 0.829 }.ToArrayOf(), Orientation = new[] { 0.0, 0.0, 0.0, 1.0 }.ToArrayOf(), Covariance = ArrayOf.Empty, })); ``` The `Transform.FrameId` names the parent frame per the §5.12 frame- precedence rule. When the parent is added later in the configurator, `WithParent("world")` resolves at finalise time. Passing a `NodeId` overload (`WithParent(NodeId)`) skips the deferred resolution when the caller already has one. Roles: `World`, `Base`, `MechanicalInterface`, `Tool`, `Object`, `Station`, `Camera`, `Custom`. `Camera` is the non-ISO addition Vision introduces for the sensor's own frame. ### Sensors `IVisionNodeBuilder` exposes three sensor entry points: - `AddImageSensor(browseName, configure)` for `ImageSensorType`; - `AddDepth3DSensor(browseName, configure)` for `Depth3DSensorType`; - `AddSensor(browseName, configure)` for the abstract `VisionSensorType` when a vendor subtype is materialised through a `IVisionModelProvider`. All sensor builders share the members on `IVisionSensorBuilder`: identity (`WithSensorId`, `WithManufacturer`, `WithModel`, `WithSerialNumber`, `WithDeviceUri`), the frame binding (`WithFrameId`, `MountedOn`, `HasScenePrim`), the reality kind (`WithRealityKind`), the modality (`WithModality`), and the nested builders — `WithOptics(...)`, `WithIllumination(...)`, `AddIntrinsicCalibration(...)`, `AddExtrinsicCalibration(...)`, `AddStreamEndpoint(...)`, `AddClipEndpoint(...)`, `UseMediaProvider(...)`. The model declares no value for the mandatory properties of sensors, frames, calibrations, media endpoints and pipelines. The builders publish one for the following when the configuration leaves them unset: the builder's reality kind (`Physical`), modality (`Area2D`) and frame role (`Other`); `Inactive` for endpoint and pipeline `State`, `None` for `Authentication`, `false` for `SecureTransport`; for calibrations an unknown `PerformedAt` (the null `DateTime`) and `Valid` `true`; and the null `NodeId` for the `SourceFrame`, `TargetFrame`, `Sensor` and `Deployment` references. `WithFrameId(frameId)` also adds a `MountedOn` reference to the `CoordinateFrameType` instance with the matching `FrameId` when one has been registered under `Vision/Frames`. This is the recommended way to attach a sensor to a frame; `MountedOn(NodeId)` and `HasScenePrim(NodeId)` are the fallbacks for mounts that are not vision frames. ### Pipelines A pipeline is the Vision object that connects one sensor, an optional AI deployment reference, and the result/feedback methods for one perception task such as detection or inspection. Clients discover pipelines first, then run or observe the task through that pipeline. ```csharp nodes.AddPipeline("Detector", pipe => pipe .WithPipelineId("pipe-01") .WithSensor(cameraNodeId) .WithDeployment(deploymentNodeId) .ProducedBy(controllerNodeId) .UseInferenceProvider(inferenceProvider, onServer: true) .UseFeedbackSink(feedbackSink)); ``` `WithSensor(NodeId)` points the pipeline at the sensor that supplies the frames. `WithDeployment(NodeId)` then records which model deployment, if any, is responsible for the task. The specification deliberately keeps `Deployment` typed as `NodeId` so a Server never has to depend on the AI Model Management companion; a host that implements that companion can point at its deployment node, and a host that does not can leave the value null. `ProducedBy(NodeId)` adds the `ProducedBy` semantic reference to a controller or process instance. `UseFeedbackSink(sink)` is optional for `OnServer` pipelines. `OnServer` pipelines without a feedback sink still expose a `Feedback` object whose `Submit*` methods return `Bad_NotSupported` — a client cannot publish detections into a pipeline where nothing consumes them. `EdgeOffServer` pipelines almost always want both a provider (whose `RunInference` explains the mode with `Bad_NotSupported`) and a sink that receives the agent's submissions. ### Providers The provider abstractions live in `Opc.Ua.Vision.Server`: - **`IVisionMediaProvider`** — supplies media without putting pixels on OPC UA. `GetStreamAsync` returns a leased URI; `GetClipAsync` returns a `VisionImageReferenceDataType` and, when the caller asked for it and the encoded bytes fit the effective inline limit, an inline `ByteString`. Providers implement the by-reference default (§6.4) — Servers keep pixel bytes off the OPC UA wire. - **`IVisionInferenceProvider`** — binds a pipeline to whatever actually computes results. The Server publishes the result nodes and applies the spec's method conventions regardless of whether the provider runs a deterministic detector, a GPU inference engine, an in-process simulation or refuses everything with `Bad_NotSupported` (the `EdgeOffServer` case). - **`IVisionFeedbackSink`** — receives `SubmitDetections`, `SubmitInspectionResult`, `SubmitCorrection` and `SubmitImageReference`. Off-Server agents publish through this path and the Server records what it did not compute. Every provider is registered as a DI singleton and constructed with the same lifetime as the server host — the framework never re-creates them per call. ## §5.12 conventions Vision inherits and adds a small number of numerical conventions that are silently wrong if misread. Every client, provider and configurator in this repository respects them, and public API points that carry pose or image geometry document them explicitly: - **Quaternion order is `(x, y, z, w)`.** Every `Orientation` array in `VisionPose3DDataType` is a unit quaternion ordered `(x, y, z, w)`. The frame graph checks `‖q‖ = 1` within tolerance `1e-6` and refuses a zero-norm quaternion with `Bad_InvalidArgument`. - **Positions are metres.** Every `Position` array is a 3-vector in metres. `MinDepth`, `MaxDepth`, `Baseline`, `WorkingDistance` and every distance-typed member are metres. - **The principal point is corner-datum.** `VisionIntrinsicsDataType.Cx` and `Cy` are measured from the top-left corner of the image (pixel centre `(0.5, 0.5)`). A client bridging to a library that uses centre-datum coordinates subtracts `0.5` from `Cx` and `Cy`. - **An empty covariance array is the sentinel for "not reported".** A pose that reports no covariance uses `Covariance = ArrayOf.Empty` — not a 6×6 zero matrix, which would misrepresent the pose as having been measured with perfect certainty. The `VisionFrameGraph` composes transforms strictly per these rules: right-handed frames, `(x, y, z, w)`-ordered quaternions, no substitutions. ## §6.4 media gating Vision separates the by-reference default path (a `VisionImageReference` descriptor with URI, timestamp and digest) from the optional inline delivery of encoded still image bytes. §6.4 fixes what a Server returns in each state, and `VisionMediaClient` classifies the raw `StatusCode` into a `VisionInlineClipState` enum so a caller can branch cleanly: | State | `StatusCode` | Meaning | |---|---|---| | `Available` | `Good` | The encoded image fits the inline limit and is returned in `VisionInlineClipReading.Bytes` | | `NotYetAvailable` | `Bad_NoDataAvailable` | The Server has not published a clip yet — §6.4 rule 5 requires this before the first acquisition | | `InlineDisabled` | `Bad_NotSupported` | `InlineDeliveryEnabled = false` on the endpoint; §6.4 rule 5 requires this exact code | | `Overflow` | `Bad_EncodingLimitsExceeded` | The last acquisition exceeded the effective inline size limit; §6.4 rule 3 requires no truncation | | `Faulted` | Other | The endpoint reported a different error | `GetClip` is always the safe path — it returns the by-reference descriptor, and returns the inline bytes as well when the caller passes `requestInline: true` and the still fits. `LatestClip` and `LatestClipMetadata` are the "read the latest" variant of the same contract. Crucially, `LatestClipMetadata` remains readable even when `LatestClip` reports `Bad_NotSupported` — the metadata carries the URI, timestamp, digest and pixel format the caller needs to walk the still out of band, and reporting `Bad_NotSupported` on the metadata read would be wrong. ```mermaid stateDiagram-v2 [*] --> NotYetAvailable: no acquisition yet NotYetAvailable --> Available: clip captured and fits Available --> Overflow: next clip exceeds limit Overflow --> Available: later clip fits Available --> InlineDisabled: inline delivery disabled InlineDisabled --> Available: inline delivery enabled Available --> Faulted: provider error Overflow --> Faulted: provider error Faulted --> Available: provider recovers ``` ## Rendering without pixels `Opc.Ua.Vision.OpenUsd` renders a `UsdGeomCamera` from a USD stage to an encoded still, and is the reference `ISceneCameraCaptureProvider` implementation the sample cell registers with `services.AddOpenUsdSceneCameraCaptureProvider()`. When rendering is not possible it reports the unavailable backend and leaves the address space walkable instead of throwing from browse/read paths. It supports the following behaviour explicitly: - On a machine with the native OpenUSD renderer payload present and a usable graphics device, it renders normally and returns encoded PNG bytes. Other requested formats return `InvalidRequest`. - When every graphics backend probe fails, `Backend.IsAvailable` is false, `Backend.UnavailableReason` explains the failure, and capture returns `SceneCameraCaptureStatus.NoRenderingBackend`. The sensor exists in the address space and supports browsing; only the pixel bytes are absent. Initialize the bundled OpenUSD resolver and imaging registries before opening stages. In particular, RID-less test hosts can load native libraries from `runtimes//native` while the manifests are at the application output root. Register those trusted, application-bundled manifests explicitly; do not use a server-supplied plugin path: ```csharp string core = Path.Combine(AppContext.BaseDirectory, "usd", "plugInfo.json"); string imaging = Path.Combine(AppContext.BaseDirectory, "plugin", "usd", "plugInfo.json"); if (!File.Exists(core) || !File.Exists(imaging)) { throw new DirectoryNotFoundException("The bundled OpenUSD runtime manifests are missing."); } _ = global::OpenUsd.UsdPluginRegistry.Register(core); _ = global::OpenUsd.UsdPluginRegistry.Register(imaging); ``` The `OpenUsdNativeCaptureTests` integration fixture uses the same initialization contract to exercise default and named cameras, repeated captures, scene replacement, and recovery after a missing camera. Registry initialization does not repair missing native libraries or nonportable native-library search paths. Linux capture hosts also need the system graphics libraries, including `libOpenGL.so.0` and the Vulkan loader. The shared CI action provisions these prerequisites for the Vision test shards. RID-less Linux hosts must also expose the application output directory and its `runtimes/linux-x64/native` directory through `LD_LIBRARY_PATH` **before** starting the process. .NET's native probing paths do not resolve the bundled OpenUSD libraries' transitive ELF dependencies, such as `libAlembic.so.1.8`. The shared CI action configures both paths for Linux Vision batches and preserves inherited search paths. This host configuration does not establish native package portability or rendering qualification. The intent is that a client can rely on the address space always being walkable, even when the process has no way to produce pixels. The `--demo` client path in the sample skips its compose step gracefully when the frame is unavailable rather than falsely reporting a rendering bug. ## Facets supported `VisionServerOptions.AdditionalFacets` is additive on top of the facets the address-space calculator derives structurally. **Supported** means the stock builder and calculator can claim the facet from the materialised address space. **Partial** means the model surface exists, but the host must attest the facet through `AdditionalFacets` because the calculator cannot verify the behaviour or provider-owned result nodes. **Not supported** means the stock server does not currently claim that facet. | Facet | Support | Structural requirement or limitation | |---|---|---| | `VIS-Base` | Supported | A registered sensor contributes the base Vision server shape. | | `VIS-Sensor-Params` | Supported | A sensor includes manufacturer, model or serial-number parameters. | | `VIS-Optics` | Supported | A sensor has an `Optics` child. | | `VIS-Media-Rtsp` | Supported | A stream endpoint uses `VisionStreamProtocolEnum.Rtsp`. | | `VIS-Media-Jpeg` | Supported | A clip endpoint uses `VisionClipFormatEnum.Jpeg`. | | `VIS-Media-Inline` | Supported | A clip endpoint has `InlineDeliveryEnabled = true`. | | `VIS-Media-DataChannel` | Not supported | There is no stock data-channel endpoint builder or calculator rule. | | `VIS-Endpoint-Config` | Supported | At least one stream endpoint is materialised. | | `VIS-Calibration` | Supported | An intrinsic or extrinsic calibration is materialised. | | `VIS-Result-Detection` | Partial | Providers and sinks can publish detection results, but result ownership is provider-side and not structurally derived. | | `VIS-Result-Inspection` | Partial | `SubmitInspectionResult` exists, but the calculator does not infer inspection-result publication. | | `VIS-Result-Segmentation` | Not supported | The stock conformance URI list and feedback surface do not claim a segmentation-result facet. | | `VIS-Feedback` | Supported | A pipeline has a `Feedback` object bound to a sink. | | `VIS-Inference-OnServer` | Supported | A pipeline was registered with `onServer: true`. | | `VIS-Inference-OffServer` | Supported | A pipeline was registered with `onServer: false`. | | `VIS-Simulation` | Supported | A sensor has `RealityKind = Simulated` or `Hybrid`. | | `VIS-Learning` | Partial | Learning jobs are modelled by reference; the host owns the job and any `SamplesCollected` accounting. | | `VIS-Interop-Scene` | Supported | A sensor carries a `HasScenePrim` reference. | | `VIS-Interop-40100` | Opt-in, application-specific | The released models are implemented separately; a host explicitly binds shared behavior and attests its draft interop facet through `AdditionalFacets`. See [Machine Vision](Vision.md). | | `VIS-Interop-RobotIntent` | Partial | The host must attest cross-model behaviour through `AdditionalFacets`. | The Server publishes the composed set on `Server.ServerCapabilities.ServerProfileArray`. ## Using the client libraries ### Registration The client hosting extensions register a `VisionClientFactory` and the factory function downstream services request: ```csharp using Microsoft.Extensions.DependencyInjection; using Opc.Ua.Client; using Opc.Ua.Vision.Client; builder.Services .AddOpcUa() .AddClient(options => { /* endpoint and application options */ }) .AddVisionClient(); ``` `AddVisionClient()` requires `AddClient(...)` to have been called first so the shared `ManagedSession` factory is available. Without DI, a `VisionClient` can be opened directly from any connected `ISession`: ```csharp using Opc.Ua; using Opc.Ua.Client; using Opc.Ua.Vision.Client; VisionClient vision = session.Vision(telemetry); if (!vision.IsVisionNamespaceAvailable) { // The Server does not implement the Vision companion. return; } ``` Sensor, calibration, pipeline and result snapshots reject Bad and Uncertain quality on members that were returned by a read; an unreadable member is not replaced by a successful default. Present members with an incompatible data type also fail. In particular, absent or wrongly typed `Detections` and `Characteristics` do not become empty observations. An explicitly empty detection array is a valid empty result, and absent optional members remain absent. Media status-classification reads use their separate inline-delivery status semantics described in [media gating](#64-media-gating). ### Discovery Generated instance folders contain their configured endpoint instances, not references back to uninstantiated `MandatoryPlaceholder` or `OptionalPlaceholder` declarations. The declarations and their references remain in the type model. ```csharp await foreach (VisionNodeEntry sensor in vision.EnumerateSensorsAsync(ct)) { Console.WriteLine($"{sensor.BrowseName} ({sensor.TypeDefinition})"); } await foreach (VisionNodeEntry pipeline in vision.EnumeratePipelinesAsync(ct)) { Console.WriteLine($"{pipeline.BrowseName} ({pipeline.TypeDefinition})"); } ``` Both `EnumerateSensorsAsync` and `EnumeratePipelinesAsync` are subtype aware: a Server that specialises `ImageSensorType` with a vendor subtype, or `DetectionResultType` with a domain result subtype, is enumerated as an instance of the closest declared Vision base type. The lower-level `DiscoverSensorsAsync` / `DiscoverPipelinesAsync` / `DiscoverFramesAsync` return `ArrayOf` for callers that already know how to render the picker themselves. ### Reading a detection ```csharp using Opc.Ua.Vision; using Opc.Ua.Vision.Perception; using Opc.Ua.Vision.Client; NodeId pipelineNodeId = /* discovered above */; VisionPipelineClient pipe = vision.Pipeline(pipelineNodeId); VisionPipelineSnapshot snapshot = await pipe.ReadAsync(ct); string runId = await pipe.RunInferenceAsync(cancellationToken: ct); await foreach (VisionNodeEntry result in pipe.EnumerateResultsAsync(ct)) { VisionDetectionResultSnapshot detection = await vision .Result(result.NodeId) .ReadDetectionAsync(ct); Console.WriteLine( $"result={detection.ResultId} frame={detection.FrameId} " + $"count={detection.Detections.Count}"); for (int ii = 0; ii < detection.Detections.Count; ii++) { VisionDetectionDataType d = detection.Detections[ii]; Console.WriteLine($" [{ii}] {d.ClassLabel} conf={d.Confidence:0.###}"); } } ``` `VisionPipelineClient.RunInferenceAsync(default, ct)` lets the Server acquire the frame "now"; passing a non-default `DateTimeUtc` requests a specific acquisition timestamp — the Server may honour it or refuse per §8. ### Composing a pose The Vision-side calibrations and frame names match the Robotics-side frame ids by convention — a client can walk from the vision-side pose to the robot-side world frame without any translation table: ```csharp VisionFrameGraph frames = vision.Frames(); NodeId cameraFrameId = /* from EnumerateFramesAsync, matching FrameId "camera_eih" */; NodeId worldFrameId = /* likewise, matching "world" */; VisionDetectionResultSnapshot detection = await vision.Result(resultNodeId).ReadDetectionAsync(ct); for (int ii = 0; ii < detection.Detections.Count; ii++) { VisionDetectionDataType d = detection.Detections[ii]; if (!d.HasPose) { continue; } // Compose the detection's pose (expressed in the camera frame) into // the world frame; the frame graph walks the parent chain and // multiplies transforms per §5.12. VisionPose3DDataType inWorld = await frames.ComposeAsync( d.Pose, cameraFrameId, worldFrameId, ct); Console.WriteLine( $"{d.ClassLabel}: pos=[{inWorld.Position[0]:0.###}, " + $"{inWorld.Position[1]:0.###}, {inWorld.Position[2]:0.###}]"); } ``` `ComposeAsync` walks up to 32 frames from each side, throws `Bad_NoMatch` when the two frames share no common ancestor, and refuses a non-unit quaternion within tolerance `1e-6` — none of which are substituted with an identity transform, because a silent substitution would make an incorrect pose look correct. ### Submitting feedback ```csharp VisionFeedbackClient? feedback = await pipe.OpenFeedbackAsync(ct); if (feedback is null) { // The pipeline does not expose a Feedback object — nothing to submit into. return; } ArrayOf detections = new[] { new VisionDetectionDataType { ClassLabel = "RedCube", Confidence = 0.94, HasBoundingBox2D = true, BoundingBox2D = new VisionBoundingBox2DDataType { CenterX = 812.0, CenterY = 604.0, Width = 96.0, Height = 96.0, }, HasPose = true, Pose = new VisionPose3DDataType { FrameId = "camera_eih", Position = new[] { 0.031, -0.017, 0.412 }.ToArrayOf(), Orientation = new[] { 0.0, 0.0, 0.0, 1.0 }.ToArrayOf(), Covariance = ArrayOf.Empty, }, }, }.ToArrayOf(); await feedback.SubmitDetectionsAsync( VisionFeedbackPurposeEnum.Reconciliation, detections, frameReference: null, inlineImage: ByteString.Empty, cancellationToken: ct); ``` The `Purpose` values `Overlay`, `Reconciliation`, `GroundTruthLabel` and `Trigger` are all defined; a Server refuses the ones it does not permit with `Bad_NotSupported`. `SubmitCorrection` accepts *at most one* of the two `corrected*` arrays non-empty — passing both is an argument error (§9.5). `SubmitInspectionResult` requires at least one characteristic. > **Reporting an empty scene, and retracting a false positive.** > `SubmitDetections` takes a `SceneIsEmpty` flag and `SubmitCorrection` > a `RetractAll` flag. They exist because an empty observation is a real > one: "I examined this frame and there is nothing in it" is the > terminating condition of a bin-picking task and a valid negative > training label, and a false positive is corrected by asserting that > nothing replaces it. Neither statement can be made by submitting an > array, because both *are* the empty array. > The pairing is checked in both directions. An empty `Detections` > without `SceneIsEmpty` is refused — the flag is what distinguishes a > deliberate observation from a lost payload — and `SceneIsEmpty` with > detections attached is refused because it asserts two contradictory > things about one frame. `RetractAll` behaves the same way against the > corrected arrays. §9.4 requires a Server to count a negative example > in `SamplesCollected` exactly as it counts one carrying geometry — > that counter belongs to the learning job, so see > [Limitations](#limitations) for where this implementation draws the > line and what it hands the host instead. ### Streaming detections `VisionResultReader.ObserveDetectionsAsync(IStreamingSubscription, ct)` publishes each `DetectionResultType` change as it arrives — either a new result or a mutation of an existing one. Use it when you need to react to detections rather than poll: ```csharp using Opc.Ua.Client; // A ManagedSession exposes a shared default IStreamingSubscription // that a caller can hand to any Observe*Async method. IStreamingSubscription streaming = session.DefaultStreaming; VisionResultReader reader = vision.Result(resultNodeId); await foreach (VisionDetectionResultSnapshot snapshot in reader.ObserveDetectionsAsync(streaming, ct)) { Console.WriteLine($"result {snapshot.ResultId}: {snapshot.Detections.Count} detections"); } ``` `ObserveInspectionAsync` and `ObserveSegmentationAsync` do the same for inspection and segmentation results. ## Sample: bin-picking [`samples/Robotics/BinPickingCell`](../samples/Robotics/BinPickingCell) is the reference from the *OPC UA Robotics-Vision Addendum*: a branch-stable four-axis palletizer with a parallel gripper, an eye-in-hand camera parented to the flange, a bin of five parts, a fixture, and the frame tree `world → robot_base → flange → gripper_tcp` with `camera_eih` on the flange. It hosts `Robot Intent`, the Vision companion and the `OpenUsdScene` companion side by side, and either the on-server deterministic detector (`--inferenceLocation OnServer`, the default) or the off-server agent path (`--inferenceLocation EdgeOffServer`). The selected detection identifies the workpiece and preserves its pose as provenance. The Pick carries the selected class plus the source Location and tool; the cell's executor resolves that workpiece at its observed world pose rather than substituting the Location centre. [`samples/Robotics/BinPickingClient`](../samples/Robotics/BinPickingClient) is the paired client: `--demo` runs the whole loop without an agent, `--mcp` exposes the composed MCP catalogue for a language-model agent, and `--view` opens the in-process OpenUSD viewport so a human sees the same scene the agent sees. ### Scene lighting The scene is lit by a single `DomeLight` with `intensity = 1000`. Do **not** reintroduce a `DistantLight` or any other bright directional or point light for a Vision demo: at any intensity that shows geometry, a `DistantLight` blows every surface to pure white regardless of `displayColor`, and any agent looking at the frame sees a uniform white blur. Under a `DomeLight` the five sample parts measure `red (220, 37, 37)`, `green (37, 208, 49)`, `blue (37, 73, 233)` — distinct enough for a vision-language model to reason about ("pick the red cube"). The `Cell.usda` header records this contract explicitly. Anyone authoring their own scene from scratch needs to know this or their agent will see nothing they can act on. ### Feedback validation When the cell runs in `EdgeOffServer` mode the agent sends detections through `SubmitDetections`. The sample's feedback sink refuses malformed submissions with `Bad_InvalidArgument` and a message the agent can act on: - **Unknown class label** — refused with the exact list of parts that do exist. `Detection 0 class 'PurplePyramid' is not a part in this cell. Known classes: RedCube, GreenCylinder, BlueSphere, YellowSlab, OrangeBrick.` - **Confidence outside `[0, 1]`** — refused with the observed value. - **Bounding box outside the image** — refused with the box coordinates and the image dimensions the box was measured against. - **Zero-norm quaternion or pose with fewer than three position components** — refused with the detection index. An empty detection set is accepted only when `SceneIsEmpty` is set; setting that flag with a nonempty set is rejected. For corrections, `RetractAll` allows both corrected arrays to be empty. Otherwise exactly one corrected array must be nonempty, and `RetractAll` must not accompany corrected entries. These flags let an agent report an empty scene or retract a false positive without inventing a detection. See [Submitting feedback](#submitting-feedback) for the payload contract and [Limitations](#limitations) for the host's responsibility for learning counters. ## Limitations - **The Vision specification is a draft.** The namespace URI and every NodeId are provisional; every ObjectType and BrowseName can change when the working group publishes. - **No vendor drivers ship.** The reference `IVisionMediaProvider` covered here renders an OpenUSD stage offscreen. There is no GigE Vision, USB3 Vision, GenICam or vendor-native driver in the box; a host implementing `IVisionMediaProvider` for a real camera is the supported extension point. - **Custom composition requires an existing owner.** `ConfigureVisionFor` supports the stock Vision and Machinery managers. Application-owned DI managers use `ConfigureVisionOnMachinery` and Machinery's existing model/lifetime integration instead of registering a second DI-owning node manager. - **Learning jobs are modelled but not driven, and `SamplesCollected` is not counted here.** `InferencePipelineType` carries a `LearningJob` optional child and the facet calculator publishes `VIS-Learning` when one is bound, but the standalone `VisionNodeManager` does not itself run training — a host provides a learning-job provider through the extension pattern. This has one named conformance consequence: §9.4 requires a Server to count a negative example (`SceneIsEmpty` or `RetractAll` carrying a `GroundTruthLabel`) in `SamplesCollected` exactly as it counts one carrying geometry, and `VisionNodeManager` does not do that counting. It cannot: `SamplesCollected` is a property of `LearningJobType`, which the *AI Model Management* companion defines, and Vision reaches it through a `NodeId` value rather than a Reference precisely so this model takes no dependency on the model that defines the job. What the Server does guarantee is that the negative example survives the hop intact — `SceneIsEmpty` and `RetractAll` are carried verbatim on `VisionSubmitDetectionsRequest` / `VisionSubmitCorrectionRequest`, so a host that binds a learning job has everything it needs to satisfy §9.4 on the counter it owns. - **AOT compatibility of the OpenUSD capture provider depends on the native renderer payload.** The managed layer is AOT-friendly; the native payload ships per-RID and its presence at runtime is what distinguishes a rendering capture from a `NoRenderingBackend` one. ## Visual inspection: a cross-companion cell Inspection is the part of the specification that composes with the most other models, so it is documented here in full rather than in isolation. The [Vision samples](../samples/Vision) build a cell where a camera photographs a machined bracket, a model measures it, deterministic code judges those measurements against a recipe, the verdict drives an ISA-95 job order, and anything the machine cannot decide escalates to an operator whose answer is captured as ground truth and counted as a learning sample. The sample READMEs cover how to run it; what follows is why it is built this way. ### The model never decides The central safety property is that model output is evidence, not authority. A model may be involved in producing measured characteristics and a confidence, but deterministic code applies the recipe tolerances and computes the verdict. That matters because a plant that let a language model schedule production from what it thought it saw in a photograph would have an image-shaped path straight into job control. In this sample, image content can influence the measured values, but it cannot become a free-form job-control instruction. The sample also routes inference through the AI companion's deployment `Invoke` method instead of letting the agent call a model privately in its own process. That keeps the deployment node, model version, and usage accounting in the address space and in the provenance trail. A private model call would make the most important part of the loop invisible. ### Address-space composition `VisualInspectionCell` composes four companion areas in one server process: - Vision publishes `BracketFixtureCamera`, `FixtureImages`, `BracketInspectionPipeline`, inspection results, and feedback. - AI Model Management publishes the primary deployment `visual-inspection-primary`, its model metadata, and the learning job whose `SamplesCollected` value is incremented by host code. - ISA-95 Job Control V2 publishes the fixed job-order catalogue and the V2 Methods the agent calls. - Alarms & Conditions publishes the `OperatorDispositionDialog` condition. `BracketInspectionPipeline` points at the AI deployment through `Deployment` and at the learning job through `LearningJob`. The Vision companion deliberately uses `NodeId` values for those bindings; it does not take a compile-time dependency on the AI companion. This sample is the host that binds both models and can therefore satisfy the learning-job counter semantics that standalone Vision cannot. ### Recipe and verdict rule The inspected part is a machined bracket with three dimensional characteristics in millimetres: | Characteristic | Nominal | Tolerance | |---|---:|---:| | `BoreDiameter` | 12.00 | ± 0.20 | | `SlotWidth` | 8.00 | ± 0.15 | | `EdgeOffset` | 20.00 | ± 0.25 | For each characteristic, the rule builds an interval from the measured value and physical uncertainty: ```text measurement interval = actual ± uncertainty tolerance interval = [nominal - lowerTol, nominal + upperTol] ``` Then it classifies the characteristic: - wholly inside the tolerance interval -> `Ok` - wholly outside the tolerance interval -> `NotOk` - straddling either tolerance limit -> `NotDecidable` The part verdict is the worst characteristic verdict, with `NotOk` worse than `NotDecidable`, and `NotDecidable` worse than `Ok`. ```mermaid flowchart TD Measurement["Measured characteristic
actual and uncertainty"] --> Interval["Build actual +/- uncertainty"] Interval --> Compare{"Compare with tolerance interval"} Compare -->|"wholly inside"| Ok["Ok"] Compare -->|"wholly outside"| NotOk["NotOk"] Compare -->|"straddles a limit"| NotDecidable["NotDecidable"] Ok --> Worst["Part takes worst characteristic verdict"] NotOk --> Worst NotDecidable --> Worst ``` ### Why uncertainty is physical The fixture images are 800 x 600 pixels at 10 px/mm. A feature edge can only land on a pixel boundary, so a dimensional measurement carries one-pixel quantisation uncertainty: 0.10 mm. That is the camera's pixel pitch. It is what makes `VisionCharacteristicDataType.Uncertainty` meaningful, and it is why `NotDecidable` arises naturally instead of being contrived. The three fixtures exercise all branches: | Fixture | Decisive characteristic | Interval | Verdict | |---|---|---|---| | `bracket-ok.png` | `BoreDiameter = 12.00` | `[11.90, 12.10]` is wholly inside `[11.80, 12.20]` | `Ok` | | `bracket-not-ok.png` | `BoreDiameter = 12.60` | `[12.50, 12.70]` is wholly outside the bore tolerance | `NotOk` | | `bracket-ambiguous.png` | `SlotWidth = 8.10` | `[8.00, 8.20]` straddles the 8.15 upper limit | `NotDecidable` | The ambiguous fixture is intentionally mundane: the intended 8.15 mm slot is 81.5 pixels at 10 px/mm and cannot be drawn exactly. The raster image therefore measures as 8.10 mm, and one pixel of uncertainty crosses the tolerance limit. That is precisely the case the Vision `NotDecidable` value exists for. ### Inspection loop The agent drives the process from outside the server. It discovers the Vision pipeline, opens the media endpoint, follows the pipeline's `Deployment` to the AI companion, discovers ISA-95 V2 endpoints, and finds the operator dialog. ```mermaid flowchart TD Discover["Discover pipeline, media, deployment, jobs, dialog"] --> Capture["Get fixture PNG"] Capture --> Measure["Measure bracket geometry"] Measure --> Invoke["Call AI deployment Invoke"] Invoke --> Judge["Apply recipe rule"] Judge --> Submit["Submit inspection result to Vision Feedback"] Submit --> Verdict{"Verdict"} Verdict -->|"Ok"| CloseOk["Start, stop, and clear inspection job"] CloseOk --> Next["StoreAndStart inspection order"] Verdict -->|"NotOk"| CloseBad["Start, stop, and clear inspection job"] CloseBad --> Reject["StoreAndStart rework/reject order"] Verdict -->|"NotDecidable"| Hold["Hold for operator"] ``` The important separation is that quality outcome and job execution state are separate facts. A defective part does not mean the inspection job failed. | Verdict | Inspection job | Next job | |---|---|---| | `Ok` | complete, close | schedule next inspection | | `NotOk` | complete, close | schedule rework/reject order | | `NotDecidable` | hold | none until the operator answers | Scheduling selects an order from a fixed allowlisted catalogue and calls V2 `StoreAndStart`. `InspectionJobControlProvider` accepts only `VIS-INSP-BRACKET-001` and `VIS-REWORK-REJECT-001`; the agent never invents a job payload. ### Escalation and ground truth `NotDecidable` activates the human path. The design dispositions are `AcceptAsOk`, `AcceptAsNotOk`, `Reinspect`, and `Stop`. The implementation maps those dispositions onto the dialog response and a bounded timeout: it holds or stops, but it does not auto-approve and does not block forever. ```mermaid sequenceDiagram participant Agent as VisualInspectionAgent participant Vision as Vision Feedback participant Dialog as Operator Dialog participant Operator as Human Operator participant AI as AI Learning Job Agent->>Vision: SubmitInspectionResult NotDecidable Agent->>Dialog: Wait for disposition Dialog->>Operator: Request Accept, Reinspect, Reject, or Stop Operator-->>Dialog: Disposition Dialog-->>Agent: Response or timeout alt Accept as ground truth Agent->>Vision: SubmitCorrection GroundTruthLabel Vision->>AI: RecordLearningSampleAsync AI-->>Vision: Idempotent count result else Reinspect Agent->>Vision: No correction Agent->>Agent: Schedule inspection order else Stop or timeout Agent->>Agent: Hold or stop without approval end ``` The operator answer becomes a Vision §9 ground-truth correction. The cell's feedback sink calls `AiNodeManager.RecordLearningSampleAsync` and uses a stable sample id, so a retry does not count the same label twice. A negative example is still a learning sample: it counts exactly once even when it carries no geometry. This closes a limitation called out in the [Vision developer guide](Vision.md#limitations): `SamplesCollected` belongs to the AI companion's `LearningJobType`, while Vision only names the learning job by `NodeId`. A standalone Vision node manager cannot increment a counter owned by another companion. A host binding Vision and AI Model Management together can, and this cell is that host. See also the [AI Model Management developer guide](AI.md) for the deployment and learning-job model. ### Modes `VisualInspectionAgent` supports three modes: - `scripted` — deterministic analyser, scripted operator policy, and a finite `--cycles N`. This is the unattended path. - `live-ai` — a real model path. It requires `--ai-endpoint`; if no endpoint is configured, the agent exits before creating any job and never silently falls back to the simulated analyser. - `human` — a real dialog subscriber path with a bounded `--operator-timeout`. The no-silent-fallback rule is part of the sample's safety story. A sample that quietly degrades can look green while proving neither model connectivity nor the provenance path it claims to demonstrate. ### What is deliberately not implemented The sample does not implement retraining or model promotion. It records learning samples honestly and increments the AI learning-job count, but it does not fake an MLOps workflow. A simulated retraining integration that appeared to work would mislead readers about the one part of the specification a sample cannot honestly demonstrate. The [AI Model Management sample](../samples/AI/README.md) takes the same line. ## See also - [Robotics developer guide](Robotics.md) — the sibling companion implementation Vision composes with; the `BinPickingCell` sample is the cross-companion example. - [OpenUSD guide](OpenUsd.md) — the OpenUSD connector and scene materialisation used to bind the Vision sample cell to a live USD stage. - [AI Model Management developer guide](AI.md) — the companion that owns the model, dataset and deployment a Vision pipeline's `Deployment` and `LearningJob` point at, and the counter §9.4 asks a learning job to keep. - [Dependency Injection](DependencyInjection.md) — the `AddOpcUa()` builder surface that hosts `AddVision`, `AddVisionClient`, and every other component.