Files
Marc Schier e73e71184d [WotCon] Add OPC UA WoT Connectivity 1.1 with the Types, source generation, Server and xRegistry work it builds on (#4128)
## Summary

Adds OPC UA WoT Connectivity 1.1 to the stack, together with the Types,
source generation, Server and xRegistry work it is built on. This branch
is the merge point for the fourteen-PR stack that replaced integration
PR #4093; every constituent PR was reviewed and approved separately and
is listed below.

414 files changed, +108,936 / -3,199. Roughly 46k added lines of product
code, 57k of tests, plus samples, tools and documentation.

## Types

- Complete `ExtensionObject` raw-body decoding for binary, XML and JSON
bodies by resolving the concrete type through the message context's
encodeable factory, and simplify the `Variant` structure helpers by
delegating to it (#4128).
- Add `IAtomicFileReplace` as an optional `IFileSystem` capability so
existing external implementations keep working, with atomic publish for
`LocalFileSystem` via `File.Replace`/`File.Move` and for
`VirtualFileSystem` by re-keying the in-memory entry (#4128).
- Add lossless conversion between WoT documents and NodeSet2
(`Opc.Ua.Wot.WotNodeSetConverter`). A byte-exact `uav:nodeSet` envelope
is preserved when requested, the structured `uav:nodes` projection is
used when the readable vocabulary is incomplete, and NodeSet2 is
otherwise synthesized from readable WoT terms. Unmapped JSON members
survive a round trip as pointer-addressed residue (#4131).

## Source generation

- Improve the NodeSet to ModelDesign conversion and the generated node
state (#4132).
- Generate OPC UA models directly from WoT Thing Description files
(#4134).

## Server

- Add live NodeManager shadow and immediate reload. A reload
materializes a new generation beside the active one and switches
atomically; the superseded generation is retired gracefully so existing
MonitoredItems keep being served until they drain, or immediately when
the caller asks for it. Includes the request admission and drain
machinery that lets an orderly shutdown wait for admitted requests
instead of tearing down underneath them (#4147).
- Make subscription transfer between sessions transactional, so a failed
transfer leaves neither session holding a partially moved subscription
(#4150).

## xRegistry

- Allow clients to use explicit registry roots rather than assuming a
well-known location (#4130).

## WoT Connectivity

- Add the WoT Connectivity 1.1 information model: a registry-first
revision layered on the abstract xRegistry base model, incorporating the
complete published OPC 10100-1 v1.02 surface as deprecated nodes in the
same namespace so existing 1.02 clients keep working (#4139).
- Add the protocol binding abstractions and planners that compile WoT
forms into executable plans (#4140).
- Add the HTTP, Modbus, OPC UA and MQTT binding executors (#4144).
- Add the registry and its stores, including document versioning,
validation, dependency resolution and refresh (#4146).
- Add the registry client (#4142).
- Materialize WoT Thing Descriptions into the server address space,
deriving types from Thing Models and instances from Thing Descriptions
(#4154).
- Restore the WoT asset registry bridge so the deprecated 1.02 asset
surface is backed by the registry (#4171).
- Add the WoT Connectivity samples and complete the documentation
(#4156), and restore the remaining coverage and docs (#4172).

## New projects

`src/Opc.Ua.WotCon.Bindings`, `src/Opc.Ua.WotCon.Bindings.Mqtt`,
`tests/Opc.Ua.WotCon.Bindings.Tests`,
`tests/Opc.Ua.WotCon.Samples.Tests`, and the `samples/WotCon`
AggregationServer, AggregationClient and FlatTagServer samples.

## Constituent pull requests

Merged into this branch top-down, each independently approved:

| PR | Title |
| --- | --- |
| #4128 | [Types] Complete ExtensionObject decoding and add an atomic
file replace capability |
| #4131 | [Types] Add lossless conversion between WoT documents and
NodeSet2 |
| #4132 | [SourceGeneration] Improve NodeSet to ModelDesign conversion
and generated node state |
| #4134 | [SourceGeneration] Generate OPC UA models from WoT Thing
Description files |
| #4147 | [Server] Add live NodeManager shadow and immediate reload |
| #4150 | [Server] Make subscription transfer between sessions
transactional |
| #4130 | [XRegistry] Allow clients to use explicit registry roots |
| #4139 | [WotCon] Add the WoT Connectivity 1.1 information model |
| #4140 | [WotCon] Add the WoT protocol binding abstractions and
planners |
| #4144 | [WotCon] Add the HTTP, Modbus, OPC UA and MQTT binding
executors |
| #4146 | [WotCon] Add the WoT Connectivity registry and its stores |
| #4142 | [WotCon] Add the WoT Connectivity registry client |
| #4154 | [WotCon] Materialize WoT Thing Descriptions into the server
address space |
| #4156 | [WotCon] Add the WoT Connectivity samples and complete the
documentation |
| #4171 | Restore WoT asset registry bridge |
| #4172 | Restore WoT coverage and docs |

Supersedes #4093.

## Follow-up

The WoT Connectivity and WoT Binding drafts have moved to 1.1-draft2
since this work was authored. Aligning with that revision - the removed
group vocabulary, the new projection/View construct, the remaining model
vocabulary terms and the two new portable-identity validation rules - is
tracked separately and will follow in its own pull request.

## Validation

Built for every target framework with zero warnings and zero errors, and
validated at the tip of the stack with `Opc.Ua.WotCon.Tests` at 999
passed / 0 failed and `Opc.Ua.Server.Tests` at 4024 passed / 0 failed.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9e6a5abf-3299-4cd1-9855-010fedbf0ad8
2026-08-05 10:14:15 +02:00

422 lines
17 KiB
C#

/* ========================================================================
* Copyright (c) 2005-2026 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System;
using System.Collections.Generic;
using System.Collections.Immutable;
using System.IO;
using System.Text;
using System.Text.Json;
using System.Threading;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.Text;
using Opc.Ua.Export;
using Opc.Ua.Wot;
namespace Opc.Ua.SourceGeneration
{
/// <summary>
/// The result of attempting to convert one WoT AdditionalFile input into
/// an in-memory NodeSet2 additional text. Conversion never throws: parse,
/// bounds, missing preservation/native mapping, dependency/resolver and
/// general conversion problems are all captured as <see cref="Diagnostic"/>s
/// instead, so a single malformed or unsupported WoT input can never fail
/// the whole generator run.
/// </summary>
internal sealed class WotConversionOutcome
{
/// <summary>
/// The path of the original WoT AdditionalFile.
/// </summary>
public string SourcePath { get; }
/// <summary>
/// The synthesized in-memory NodeSet2 additional text, or
/// <c>null</c> when conversion did not produce a usable result.
/// </summary>
public AdditionalText NodeSetText { get; }
/// <summary>
/// The AdditionalFiles options (Prefix, Name, ModelUri, Version,
/// Ignore) captured from the original WoT input, to be preserved
/// after wrapping it as a NodeSet2 file.
/// </summary>
public NodesetFileOptions Options { get; }
/// <summary>
/// The diagnostics produced while attempting the conversion, in the
/// order they occurred. Does not include virtual-path collision
/// diagnostics, which require the full set of inputs to detect.
/// </summary>
public ImmutableArray<Diagnostic> Diagnostics { get; }
public WotConversionOutcome(
string sourcePath,
AdditionalText nodeSetText,
NodesetFileOptions options,
ImmutableArray<Diagnostic> diagnostics)
{
SourcePath = sourcePath;
NodeSetText = nodeSetText;
Options = options;
Diagnostics = diagnostics;
}
}
/// <summary>
/// Presents a WoT model input to the existing generator as an in-memory
/// NodeSet2 file, using the completed <see cref="Wot"/> converter
/// entirely in memory (no file or network I/O beyond the supplied
/// <see cref="AdditionalText"/> content).
/// </summary>
internal sealed class WotNodeSetAdditionalText : AdditionalText
{
private WotNodeSetAdditionalText(string path, SourceText text)
{
Path = path;
m_text = text;
}
/// <inheritdoc/>
public override string Path { get; }
/// <inheritdoc/>
public override SourceText GetText(CancellationToken cancellationToken = default)
{
return m_text;
}
/// <summary>
/// Attempts to convert a WoT AdditionalFile into an in-memory
/// NodeSet2 additional text. Never throws (other than on
/// cancellation): every failure mode is reported as a diagnostic on
/// the returned <see cref="WotConversionOutcome"/> instead, so a
/// malformed or unsupported document can never crash the generator.
/// </summary>
/// <param name="source">The original WoT AdditionalFile.</param>
/// <param name="options">The AdditionalFiles options to preserve.</param>
/// <param name="cancellationToken">A cancellation token.</param>
public static WotConversionOutcome Convert(
AdditionalText source,
NodesetFileOptions options,
CancellationToken cancellationToken)
{
string sourcePath = source.Path;
ImmutableArray<Diagnostic>.Builder diagnostics = ImmutableArray.CreateBuilder<Diagnostic>();
SourceText sourceText;
try
{
sourceText = source.GetText(cancellationToken);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
diagnostics.Add(CreateParseDiagnostic(sourcePath, null, ex.Message));
return Failed(sourcePath, options, diagnostics);
}
if (sourceText is null)
{
diagnostics.Add(CreateParseDiagnostic(
sourcePath, null, "The WoT model source text could not be read."));
return Failed(sourcePath, options, diagnostics);
}
byte[] utf8Json;
try
{
utf8Json = GetUtf8Bytes(sourceText, cancellationToken);
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
diagnostics.Add(CreateParseDiagnostic(sourcePath, sourceText, ex.Message));
return Failed(sourcePath, options, diagnostics);
}
WotDocument document;
try
{
document = WotDocument.Parse(utf8Json);
}
catch (JsonException ex)
{
diagnostics.Add(CreateParseDiagnostic(sourcePath, sourceText, ex));
return Failed(sourcePath, options, diagnostics);
}
catch (FormatException ex)
{
// Thrown when the document exceeds a configured size bound.
diagnostics.Add(CreateParseDiagnostic(sourcePath, sourceText, ex.Message));
return Failed(sourcePath, options, diagnostics);
}
UANodeSet nodeSet;
try
{
using (document)
{
// No external resolver: source generation performs no
// file or network I/O beyond the supplied AdditionalText
// content, so referenced TD/TM documents are left
// unresolved (reported as a WotDiagnosticCode.
// UnresolvedReference warning by the converter).
WotConversionResult<UANodeSet> result = WotNodeSetConverter.ToNodeSetResult(
document,
options: null);
AppendConversionDiagnostics(diagnostics, sourcePath, result.Diagnostics);
// Exclude any result that produced an error diagnostic even
// when a (partial or inconsistent) NodeSet value was still
// produced. This decision is independent of whether the
// MODELGEN031 error diagnostic is later reported or suppressed,
// so a suppressed conversion error can never emit a model.
nodeSet = result.Success ? result.Value : null;
}
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
// Defence in depth: the converter is documented to report
// diagnostics rather than throw. Guard against any residual
// exception so a single malformed input can never take down
// the whole generator run.
diagnostics.Add(CreateConversionExceptionDiagnostic(sourcePath, ex));
return Failed(sourcePath, options, diagnostics);
}
if (nodeSet is null)
{
// An error diagnostic explaining why was already appended.
return Failed(sourcePath, options, diagnostics);
}
string xml;
try
{
using var stream = new MemoryStream();
nodeSet.Write(stream);
xml = Encoding.UTF8.GetString(stream.ToArray());
}
catch (Exception ex) when (ex is not OperationCanceledException)
{
diagnostics.Add(CreateConversionExceptionDiagnostic(sourcePath, ex));
return Failed(sourcePath, options, diagnostics);
}
if (xml.Length > 0 && xml[0] == '\uFEFF')
{
xml = xml[1..];
}
var nodeSetText = new WotNodeSetAdditionalText(
GetNodeSetPath(sourcePath),
SourceText.From(xml, Encoding.UTF8));
return new WotConversionOutcome(
sourcePath, nodeSetText, options, diagnostics.ToImmutable());
}
private static WotConversionOutcome Failed(
string sourcePath,
NodesetFileOptions options,
ImmutableArray<Diagnostic>.Builder diagnostics)
{
return new WotConversionOutcome(sourcePath, null, options, diagnostics.ToImmutable());
}
private static void AppendConversionDiagnostics(
ImmutableArray<Diagnostic>.Builder builder,
string sourcePath,
IReadOnlyList<WotDiagnostic> wotDiagnostics)
{
if (wotDiagnostics.Count == 0)
{
return;
}
Location location = CreateFileLocation(sourcePath);
for (int ii = 0; ii < wotDiagnostics.Count; ii++)
{
WotDiagnostic diagnostic = wotDiagnostics[ii];
DiagnosticDescriptor descriptor = diagnostic.Severity switch
{
WotDiagnosticSeverity.Error => SourceGenerator.WotConversionError,
WotDiagnosticSeverity.Warning => SourceGenerator.WotConversionWarning,
_ => SourceGenerator.WotConversionInfo
};
builder.Add(Diagnostic.Create(descriptor, location, sourcePath, diagnostic.ToString()));
}
}
private static Diagnostic CreateParseDiagnostic(
string sourcePath, SourceText sourceText, JsonException ex)
{
Location location = CreateLocation(sourcePath, sourceText, ex.LineNumber, ex.BytePositionInLine);
return Diagnostic.Create(SourceGenerator.WotParseError, location, sourcePath, ex.Message);
}
private static Diagnostic CreateParseDiagnostic(
string sourcePath, SourceText sourceText, string message)
{
Location location = CreateLocation(sourcePath, sourceText, null, null);
return Diagnostic.Create(SourceGenerator.WotParseError, location, sourcePath, message);
}
private static Diagnostic CreateConversionExceptionDiagnostic(string sourcePath, Exception ex)
{
Location location = CreateFileLocation(sourcePath);
return Diagnostic.Create(
SourceGenerator.WotConversionError,
location,
sourcePath,
$"{ex.GetType().Name}: {ex.Message}");
}
private static byte[] GetUtf8Bytes(SourceText sourceText, CancellationToken cancellationToken)
{
using var stream = new MemoryStream();
using (var writer = new StreamWriter(stream, new UTF8Encoding(false), 4096, leaveOpen: true))
{
sourceText.Write(writer, cancellationToken);
}
return stream.ToArray();
}
/// <summary>
/// Creates a location anchored at the start of the given file. Used
/// when no more precise position is available.
/// </summary>
internal static Location CreateFileLocation(string path)
{
return Location.Create(path, default, default);
}
private static Location CreateLocation(
string path,
SourceText text,
long? lineNumber,
long? bytePositionInLine)
{
if (text is null || text.Lines.Count == 0)
{
return CreateFileLocation(path);
}
int line = 0;
int character = 0;
if (lineNumber >= 0 && lineNumber.Value < text.Lines.Count)
{
line = (int)lineNumber.Value;
if (bytePositionInLine >= 0)
{
character = GetCharacterIndexFromUtf8ByteOffset(text, line, bytePositionInLine.Value);
}
}
var position = new LinePosition(line, character);
int offset = text.Lines[line].Start + character;
var span = new TextSpan(offset, 0);
return Location.Create(path, span, new LinePositionSpan(position, position));
}
private static int GetCharacterIndexFromUtf8ByteOffset(SourceText text, int line, long byteOffset)
{
if (byteOffset <= 0)
{
return 0;
}
TextLine textLine = text.Lines[line];
int lineLength = textLine.Span.Length;
long bytesSeen = 0;
int character = 0;
while (character < lineLength)
{
char current = text[textLine.Start + character];
int characterCount = 1;
int byteCount;
if (char.IsHighSurrogate(current) &&
character + 1 < lineLength &&
char.IsLowSurrogate(text[textLine.Start + character + 1]))
{
characterCount = 2;
byteCount = 4;
}
else if (current <= '\u007F')
{
byteCount = 1;
}
else if (current <= '\u07FF')
{
byteCount = 2;
}
else
{
byteCount = 3;
}
long nextBytes = bytesSeen + byteCount;
if (byteOffset < nextBytes)
{
return character;
}
character += characterCount;
bytesSeen = nextBytes;
if (byteOffset == bytesSeen)
{
return character;
}
}
return lineLength;
}
private static string GetNodeSetPath(string sourcePath)
{
string directory = System.IO.Path.GetDirectoryName(sourcePath) ?? string.Empty;
string name = System.IO.Path.GetFileName(sourcePath);
// The canonical suffixes are checked before the bare ".jsonld"
// fallback used by opted-in plain JSON-LD inputs: none of the
// canonical suffixes is itself a suffix of another, but every one
// of them ends with ".jsonld" or ".json" as plain text, so the
// untyped fallback must come last.
foreach (string suffix in Extensions.WotFileExtensions)
{
if (name.EndsWith(suffix, StringComparison.OrdinalIgnoreCase))
{
name = name[..^suffix.Length];
return System.IO.Path.Combine(directory, name + ".NodeSet2.xml");
}
}
const string plainJsonLd = ".jsonld";
if (name.EndsWith(plainJsonLd, StringComparison.OrdinalIgnoreCase))
{
name = name[..^plainJsonLd.Length];
}
return System.IO.Path.Combine(directory, name + ".NodeSet2.xml");
}
private readonly SourceText m_text;
}
}