mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-18 14:43:08 +08:00
cc7908eaa1
Reworked after the review: the OpenAPI document is generated from the code, not embedded; the route is opt-in; `StatusCode` is not accepted. The branch is a new history on top of master (the first version conflicted); master with #4674 is merged in. # Description The REST binding implements the routes of the OPC Foundation OpenAPI mapping (OPC 10000-6 §G.3), but it does not describe them: `GET /openapi.json` is answered with HTTP 405, and a client generator has nothing to read from the server. `docs/WebApi.md` listed a generated OpenAPI document as a deferred follow-up. The coded API is the source of truth, so this PR **generates** the document from the route table and from the structure definitions of the request and response types, and compares it with the document the model compiler publishes in tests. Nothing from `UA-Nodeset/OpenApi` ships in the package. ## Changes ### Service sets (`Opc.Ua.Core`) - `WebApiServiceSet` (`AllServices`, `Sessionless`) and `WebApiServiceRoutes.GetRoutes(serviceSet)`. `Sessionless` is the eight services of `opc.ua.openapi.sessionless.json` (OPC 10000-4 §6.3): Read, Write, HistoryRead, HistoryUpdate, Call, Browse, BrowseNext, TranslateBrowsePathsToNodeIds. ### Generator (`Opc.Ua.Core.Schema`, `Opc.Ua.Schema.OpenApi`) - `WebApiOpenApiGenerator.Generate(serviceSet, includeSchemas, serverUrl)` returns the OpenAPI 3.0 document as a `JsonObject`. It is built at runtime from `WebApiServiceRoutes` and the `StructureDefinition`/`EnumDefinition` of the message types (the same definitions the schema generation uses, resolved through `IDataTypeDefinitionResolver`). No reflection, so it is NativeAOT compatible; the AOT test project covers it. - **Two levels of detail**, because full schemas make big documents: | | Content | All 28 services | | --- | --- | --- | | default | paths, `operationId`s, every body a JSON object | about 12 KB | | `includeSchemas: true` | one component schema per message, structure and enumeration (100), for typed client generators | about 46 KB | The two published files are about 300 KB together. - The schemas describe the Compact JSON encoding, with the property names, types, formats and integer ranges of the published documents. DataValue is the one structure that differs: its status property is `Status`, as in Part 6 Table 42. - `OpenApiBuiltInSchemas` holds the schemas of the built-in types (StatusCode, LocalizedText, Variant, DataValue, DiagnosticInfo, ExtensionObject and the scalar types) as the encoder writes them. Abstract `Structure` fields are ExtensionObjects, abstract number fields are Variants. ### Serving the document (`Opc.Ua.Bindings.Https`) - `WebApiTransportOptions`: `ServiceSet`, `OpenApiDocumentPath` (default `null`: **off**) and `OpenApiIncludeSchemas` (default `false`). - `MapWebApiEndpoints(WebApiTransportOptions)` maps the routes of the service set and, when a path is set, the `GET` route of the document **in the same route group**. The route has no `AllowAnonymous` metadata. When an authentication scheme is registered, the contributor's `group.RequireAuthorization()` covers the document like `POST /read`; only `/findservers` and `/getendpoints` stay anonymous. The existing `MapWebApiEndpoints()` maps all routes as before. - The document is generated once per path base (listed as a relative `servers` URL; without a path base the list is omitted) by the `WebApiOpenApiGenerator` that `AddWebApiTransport` registers, or by a default one. - `Opc.Ua.Bindings.Https` references `Opc.Ua.Core.Schema` on net8.0 and later. ### Documentation - `docs/OpenApi.md` (new, linked from `docs/README.md`): serving the document, the two levels, generating clients, the JSON generated clients send, known issues, how the document is kept correct, troubleshooting. - `docs/WebApi.md`: new section "Service sets and the OpenAPI document"; the deferred follow-up is removed. `docs/SchemaGeneration.md` and both `NugetREADME.md` files describe the generator and the options. ## Comparison with the model compiler documents The two published documents are test resources of `Opc.Ua.Core.Schema.Tests` only (README next to them: source commit and update procedure). `WebApiOpenApiConformanceTests` compares the generated document with them for both service sets: - paths, `operationId`s, and the request and response schema of every operation; - the set of component schemas the operations reach: equal (100 for all services); - every enumeration (members and values); - for every structure: property names, types, formats, integer ranges, and inheritance. Everything both describe is equal. The test lists the differences and fails when a listed one disappears or a new one appears: | Difference | Generated | Published | Reason | | --- | --- | --- | --- | | DataValue status property | `Status` | `StatusCode` | Part 6 Table 42; UA-Nodeset#146 is open. The stack only knows `Status` (encoder and decoder). | | `description`, `default` keywords | not written | spec links, .NET defaults | documentation and defaults, not contract; Compact omits default values | `WebApiOpenApiWireFormatTests` validates what the binding really writes (the default request and response of all 28 routes, plus populated requests and responses with variants, status codes, references, identity tokens and notifications) against the generated component schemas. ## Tests - `Opc.Ua.Core.Schema.Tests` `OpenApi/`: `WebApiOpenApiGeneratorTests` (shape of the document, both levels, both service sets, servers list, sorting and determinism, concurrency, argument and resolver errors, every built-in type, arrays and ranks, enumerations, option sets, unions, base types, self-references, unknown definitions), `WebApiOpenApiConformanceTests`, `WebApiOpenApiWireFormatTests`, `OpenApiBuiltInSchemasTests`: 81 tests. - `Opc.Ua.Bindings.Https.WebApi.Tests`: `WebApiOpenApiDocumentEndpointTests` (TestServer with the startup contributor: off by default, configured path, both levels, service set maps and describes only the sessionless routes, path base, repeated requests, registered generator is used, **a request without credentials gets 401 when an authentication scheme is registered while one with credentials gets 200, the route carries `IAuthorizeData` and no `IAllowAnonymous`**, argument checks), options defaults and DI registration of the generator. - `Opc.Ua.Core.Tests`: `GetRoutes` for both sets and an undefined value. - `Opc.Ua.Aot.Tests`: the generator and a served document under the AOT fixture. - Results (macOS arm64): `Opc.Ua.Core.Schema.Tests` 260/260, `Opc.Ua.Bindings.Https.WebApi.Tests` 457/457, `Opc.Ua.Core.Tests` WebApi tests 110/110, each on net10.0 and net8.0 (Release). `Opc.Ua.Aot.Tests` (JIT run) SchemaAotTests 5/5, WebApiAotTests 8/8. - `Opc.Ua.Core`, `Opc.Ua.Core.Schema` and `Opc.Ua.Bindings.Https` and the three test projects build for net48, net8.0, net9.0 and net10.0 (Release) with 0 warnings and 0 errors (net48 builds only; the Windows legs of the fork CI run its tests). `dotnet format` whitespace, style and analyzers (`--severity info`) are clean on every changed file; the two `RCS1090` hits in `SchemaAotTests.cs` are on lines that master already has. - Patch coverage with `.azurepipelines/check-coverage.ps1` against master (`Opc.Ua.Core.Schema.Tests`, `Opc.Ua.Bindings.Https.WebApi.Tests` and the WebApi tests of `Opc.Ua.Core.Tests`, net10.0): 362/362 changed lines covered. - Fork CI (full matrix incl. macOS) and CodeQL on `f990af5db` (master with #4674 merged in): [CI run 37310622902](https://github.com/biancode/UA-.NETStandard/actions/runs/37310622902) green, [CodeQL run 37310626786](https://github.com/biancode/UA-.NETStandard/actions/runs/37310626786) green. The run on the earlier head `abda0e491` had one failure outside this PR: `McpServerProcessTests.HttpTransportAliasesExposeOnlyMcpRouteAsync("http")` timed out after 30 s on Windows net10.0; this PR does not touch `tools/`. ## Not in this PR - Authentication: the fix discussed in the review is @romanett's #4674. The document route uses whatever the REST routes use; this PR does not change authentication. - `StatusCode` tolerance in the decoder: dropped, the spec text is the reference. The first commit of the previous version is gone. - Samples. - Session-less invocation on the server side (OPC 10000-4 §6.3) is a separate draft PR. With `ServiceSet = Sessionless` this PR only limits the mapped routes and the document. ## Questions 1. `Opc.Ua.Bindings.Https` now references `Opc.Ua.Core.Schema` (net8.0 and later) because the generator lives next to the other schema generators and is trimmed away when unused. Fine, or should it move into `Opc.Ua.Core`? 2. Should `OpenApiIncludeSchemas` default to `false` as here, or should a host get typed clients out of the box? ## Related Issues - Fixes #4656 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received.
233 lines
13 KiB
Markdown
233 lines
13 KiB
Markdown
# Runtime Schema Generation
|
|
|
|
The `Opc.Ua.Core.Schema` library generates schemas for OPC UA data types at runtime. It works for the encodeable types emitted by the source generators and for complex types that are added dynamically by the complex-type client. Schemas are produced in every supported encoding:
|
|
|
|
- **XSD** for the XML encoding.
|
|
- **BSD** (OPC Binary, Part 6) for the binary encoding.
|
|
- **JSON Schema** (Part 6 Annex C, draft 2020-12) for the JSON encoding, in both the **compact** (reversible, BrowseName-keyed) and **verbose** flavors.
|
|
|
|
Schemas are built as strongly-typed object models in code — there are no embedded schema strings — so unused generation paths are trimmed away and the whole library is NativeAOT compatible. The XSD object model is the in-box `System.Xml.Schema.XmlSchema`. BSD uses
|
|
the existing `Opc.Ua.Schema.Binary.TypeDictionary`, and JSON uses
|
|
`System.Text.Json.Nodes.JsonObject`.
|
|
|
|
## Contents
|
|
|
|
- [Concepts](#concepts)
|
|
- [Registration](#registration)
|
|
- [Registering data types](#registering-data-types)
|
|
- [Namespace identity](#namespace-identity)
|
|
- [Generating a schema](#generating-a-schema)
|
|
- [Working with the object model](#working-with-the-object-model)
|
|
- [JSON encoding notes](#json-encoding-notes-part-6)
|
|
- [PubSub schemas](#pubsub-schemas)
|
|
- [OpenAPI document of the REST binding](#openapi-document-of-the-rest-binding)
|
|
- [Trimming and NativeAOT](#trimming-and-nativeaot)
|
|
|
|
## Concepts
|
|
|
|
Generation is driven by a type's runtime structure definition (`StructureDefinition` or `EnumDefinition`), which already captures every field, data type, value rank and optionality. The pieces fit together as follows:
|
|
|
|
- `ISchemaProvider` is the entry point. It produces an `IUaSchema` for a requested `UaSchemaFormat` and `UaSchemaScope`.
|
|
- `IUaSchema` is the generated document. It exposes the strongly-typed object model (for example `JsonSchemaDocument.Root`, `XmlSchemaDocument.Schema`, `BinarySchemaDocument.Dictionary`) and can serialize itself with `WriteTo(Stream)`, `WriteTo(TextWriter)` or `ToSchemaString()`.
|
|
- `IDataTypeDefinitionResolver` maps a data type id to its `UaTypeDescription` (the type id, browse name and definition). The default implementation, `DataTypeDefinitionRegistry`, is an in-memory registry that callers populate from browsed or otherwise available definitions. The resolver is also used to follow field references and to enumerate the types of a namespace.
|
|
|
|
`UaSchemaFormat` selects the encoding (`Xsd`, `Bsd`, `JsonCompact`, `JsonVerbose`). `UaSchemaScope` selects the document granularity: `Type` produces a document for a single type and the closure of the types it depends on; `Namespace` produces a dictionary document for all types in a namespace.
|
|
|
|
## Registration
|
|
|
|
The services are registered through the standard OPC UA dependency-injection surface:
|
|
|
|
```csharp
|
|
IServiceProvider services = new ServiceCollection()
|
|
.AddOpcUa()
|
|
.AddSchemaGeneration()
|
|
.Services
|
|
.BuildServiceProvider();
|
|
|
|
ISchemaProvider provider = services.GetRequiredService<ISchemaProvider>();
|
|
```
|
|
|
|
The provider can also be constructed directly when dependency injection is not used:
|
|
|
|
```csharp
|
|
var registry = new DataTypeDefinitionRegistry();
|
|
registry.Add(new UaTypeDescription(typeId, browseName, structureDefinition, namespaceUri));
|
|
|
|
ISchemaProvider provider = new DefaultSchemaProvider(
|
|
registry,
|
|
[
|
|
new JsonSchemaGenerator(),
|
|
new XsdSchemaGenerator(),
|
|
new BsdSchemaGenerator()
|
|
]);
|
|
```
|
|
|
|
## Registering data types
|
|
|
|
Schema generation needs the runtime definition of a type. A type's `StructureDefinition` / `EnumDefinition` is registered with the resolver from whichever source has it:
|
|
|
|
- **Server / browsed types** — a `DataTypeNode` obtained from a server (or the client node cache) carries its definition in `DataTypeNode.DataTypeDefinition`. Register it directly:
|
|
|
|
```csharp
|
|
var registry = serviceProvider.GetRequiredService<DataTypeDefinitionRegistry>();
|
|
registry.TryAddDataType(dataTypeNode, session.NamespaceUris);
|
|
```
|
|
|
|
- **Source-generated types** — the generated model registration extension adds
|
|
public structure and enumeration activators to an `IEncodeableFactory`.
|
|
These activators implement `IDataTypeDefinitionSource`; the generated data
|
|
value does not need to implement that interface. Use
|
|
`EncodeableFactoryDefinitionSource` to resolve their definitions without
|
|
reflection or manual registration:
|
|
|
|
```csharp
|
|
IEncodeableFactory factory = EncodeableFactory.Create();
|
|
factory.Builder
|
|
.AddOpcUa() // use the generated Add<Namespace>() extension for your model
|
|
.Commit();
|
|
|
|
IDataTypeDefinitionResolver resolver =
|
|
new EncodeableFactoryDefinitionSource(factory, namespaceUris);
|
|
|
|
ISchemaProvider provider = new DefaultSchemaProvider(
|
|
resolver,
|
|
[
|
|
new JsonSchemaGenerator(),
|
|
new XsdSchemaGenerator(),
|
|
new BsdSchemaGenerator()
|
|
]);
|
|
```
|
|
|
|
- **Dynamic complex types** — complex types built by the complex-type client carry a `StructureDefinition` (via `IStructureTypeInfo` / the structure-definition attribute) that can likewise be wrapped in a `UaTypeDescription` and registered.
|
|
|
|
- **Types already in the encodeable factory** — any structured or enumeration type known to an `IEncodeableFactory` (source-generated types and runtime complex-type stand-ins) exposes its definition through `IDataTypeDefinitionSource`, so `EncodeableFactoryDefinitionSource` resolves it directly from the factory without a separate registration. Built-in primitive DataTypes (and simple types that derive directly from them) carry no `DataTypeDefinition` and are handled by the built-in type system rather than this resolver. Use the factory source on its own, or combine it with a manual `DataTypeDefinitionRegistry` (for schema-only types that have no encodeable) via `CompositeDataTypeDefinitionResolver`:
|
|
|
|
```csharp
|
|
IDataTypeDefinitionResolver resolver = new CompositeDataTypeDefinitionResolver(
|
|
[new EncodeableFactoryDefinitionSource(factory, namespaceUris), registry]);
|
|
```
|
|
|
|
When `AddComplexTypeSystem()` is enabled, an OPC UA server wires this up
|
|
automatically (see [ComplexTypes.md](ComplexTypes.md)). The primed
|
|
encodeable factory becomes the schema resolver.
|
|
|
|
Once registered, fields that reference other registered types are resolved automatically and included in the generated document.
|
|
|
|
### Namespace identity
|
|
|
|
`BrowseName.NamespaceIndex` is not a mapping for the data type's NodeId namespace; the
|
|
two namespaces may legitimately differ. To support both local and URI-based lookup in
|
|
`DataTypeDefinitionRegistry`, register an index-form TypeId together with its namespace
|
|
URI, as `TryAddDataType(node, namespaceTable)` does:
|
|
|
|
```csharp
|
|
registry.Add(new UaTypeDescription(
|
|
new ExpandedNodeId(dataTypeNode.NodeId),
|
|
dataTypeNode.BrowseName,
|
|
definition,
|
|
session.NamespaceUris.GetString(dataTypeNode.NodeId.NamespaceIndex)));
|
|
```
|
|
|
|
A custom URI-only TypeId is resolvable by URI but does not create an inferred local
|
|
alias. The standard OPC UA namespace URI is the exception because its local index is
|
|
always zero. Registry entries must share one local namespace-table context.
|
|
|
|
URI lookup preserves identifier kind as well as value. Numeric zero, an empty GUID,
|
|
an empty string, and an empty ByteString are distinct identifiers in a nonzero namespace;
|
|
they must not be conflated with namespace-zero null NodeIds.
|
|
|
|
## Generating a schema
|
|
|
|
Once a type's definition is registered with the resolver, a schema can be produced from its type id:
|
|
|
|
```csharp
|
|
if (provider.TryGetSchema(typeId, UaSchemaFormat.JsonCompact, UaSchemaScope.Type, out IUaSchema? schema))
|
|
{
|
|
string json = schema.ToSchemaString();
|
|
}
|
|
```
|
|
|
|
When the starting point is a namespace-index `NodeId` obtained from a server, convert it
|
|
with the session namespace table. A registry populated with
|
|
`TryAddDataType(dataTypeNode, session.NamespaceUris)` accepts both this URI form and the
|
|
local index-form `ExpandedNodeId`:
|
|
|
|
```csharp
|
|
ExpandedNodeId typeId = NodeId.ToExpandedNodeId(dataTypeNode.NodeId, session.NamespaceUris);
|
|
provider.TryGetSchema(typeId, UaSchemaFormat.JsonCompact, UaSchemaScope.Type, out IUaSchema? schema);
|
|
```
|
|
|
|
The convenience extension methods read more naturally and make a type "expose" its schema:
|
|
|
|
```csharp
|
|
IUaSchema xsd = provider.GetXmlSchema(type);
|
|
IUaSchema bsd = provider.GetBinarySchema(type);
|
|
IUaSchema jsonCompact = provider.GetJsonSchema(type);
|
|
IUaSchema jsonVerbose = provider.GetJsonSchema(type, verbose: true);
|
|
|
|
// Resolve by type id and produce JSON in one call.
|
|
provider.TryGetJsonSchema(typeId, out IUaSchema? schema);
|
|
```
|
|
|
|
## Working with the object model
|
|
|
|
Because the schema is an object model, callers can inspect or post-process it before serializing. For JSON:
|
|
|
|
```csharp
|
|
var document = (JsonSchemaDocument)provider.GetJsonSchema(type);
|
|
JsonObject root = document.Root; // the draft 2020-12 schema
|
|
document.WriteTo(stream); // UTF-8, indented
|
|
```
|
|
|
|
## JSON encoding notes (Part 6)
|
|
|
|
The JSON schemas follow the Part 6 JSON encoding faithfully, matching what the stack's `JsonEncoder` produces:
|
|
|
|
- `Int64` and `UInt64` are encoded as JSON strings (to avoid precision loss), so they are typed as `string`.
|
|
- `Float`/`Double` accept the special string values `Infinity`, `-Infinity` and `NaN`, so they are typed as `["number", "string"]`.
|
|
- `ByteString` is a base64 `string`; `DateTime` is a `date-time` string; `Guid` is a `uuid` string.
|
|
- The standard structured built-ins (`NodeId`, `Variant`, `ExtensionObject`, `DataValue`, ...) are described once per document in the `$defs` section and referenced.
|
|
- Compact enums are integers (with the allowed values listed via `oneOf`); verbose enums are the `Name_Value` strings.
|
|
|
|
## PubSub schemas
|
|
|
|
The `Opc.Ua.PubSub.Schema` library generates JSON Schemas for the PubSub JSON message formats. It is registered with `services.AddOpcUa().AddPubSubSchema()` and exposes `IPubSubSchemaProvider`:
|
|
|
|
- `CreateDataSetSchema(metaData, fieldContentMask, verbose)` — the per-DataSet payload object, one property per `FieldMetaData`. The field value shape follows the same Part 6 JSON rules as the core library, and `DataSetFieldContentMask` controls whether each field is the raw value or a `DataValue` object (with the mask-selected `StatusCode` / `SourceTimestamp` / ... members).
|
|
- `CreateDataSetMessageSchema(metaData, messageContentMask, fieldContentMask, verbose)` — a single DataSetMessage whose header fields are gated by `JsonDataSetMessageContentMask` and whose `Payload` is the DataSet schema above.
|
|
- `CreateNetworkMessageSchema(metaData, networkContentMask, messageContentMask, fieldContentMask, verbose)` — the `ua-data` NetworkMessage envelope, gated by `JsonNetworkMessageContentMask`; `Messages` is an array of DataSetMessage schemas, or a single object when `SingleDataSetMessage` is set.
|
|
- `CreateMetaDataMessageSchema(metaData, verbose)` — the `ua-metadata` message.
|
|
|
|
The provider reuses the core `ISchemaProvider` to resolve complex (structured/enum) field data types, embedding them into the document `$defs` section.
|
|
|
|
## OpenAPI document of the REST binding
|
|
|
|
`WebApiOpenApiGenerator` generates the OpenAPI 3.0 document of the REST binding
|
|
(Part 6 G.3, see [REST binding](WebApi.md)) from `WebApiServiceRoutes` and the
|
|
structure definitions of the request and response types. It is not an
|
|
`IUaSchemaGenerator`: a document describes a set of routes rather than one data
|
|
type.
|
|
|
|
```csharp
|
|
var generator = new WebApiOpenApiGenerator();
|
|
JsonObject document = generator.Generate(
|
|
WebApiServiceSet.Sessionless, // or AllServices
|
|
includeSchemas: true, // false: a JSON object as every body
|
|
serverUrl: "/opcua/"); // null: no servers list
|
|
```
|
|
|
|
Without `includeSchemas` the document holds the paths and operations only
|
|
(about 12 KB for all 28 services). With it, every request, response and the
|
|
structures and enumerations they contain become component schemas (about 46 KB),
|
|
which is what OpenAPI client generators need. The schemas describe the Compact
|
|
JSON encoding with the property names and types the published OPC Foundation
|
|
documents use; [OPC UA over OpenAPI](OpenApi.md) lists the differences and the
|
|
tests that keep the document correct. Pass an `IDataTypeDefinitionResolver` to
|
|
the constructor to resolve the types from another source than the generated
|
|
types of the OPC UA namespace. The generator is registered by
|
|
`AddWebApiTransport()`, which serves the document.
|
|
|
|
## Trimming and NativeAOT
|
|
|
|
The library opts into `IsAotCompatible` and avoids reflection-based serialization. XSD is written with `System.Xml.Schema.XmlSchema`, BSD with a direct `System.Xml.XmlWriter`, and JSON with `System.Text.Json.Nodes` / `Utf8JsonWriter`. Schema generation is a configuration-time activity, not a hot path; documents are built lazily and can be cached by the caller. Because the generation logic lives in its own assembly, it is trimmed away entirely when an application does not generate schemas.
|