# Description Extract the reusable core/types layer from #4462 without including UaLens or the other planned extraction layers. - Preserve null versus empty in `ByteString.Copy`, and provide older-framework path, random-byte, and SHA-256 convenience APIs. - Encode abstract numeric structure fields (`Number`, `Integer`, and `UInteger`) as Variants in JSON, XSD, and BSD schemas, with regression coverage and schema documentation. - Add opt-in complete-file validation for finalized PCAP replay while retaining the existing tolerant reader default. - Keep placeholder references on generated type templates, not runtime instances, while preserving ordinary external references. **Initial extraction:** commit `8881e096fb5e4eaa36f3c13f8caa57733373bd19` preserves the assigned changes from source `390c4a7a51b71da40caec2d22ef74518bbe670ce`, based on pinned `master` commit `5b03e9afa492cdb36049ee52ae844f630f89d8db`: 20 exact full-file extractions and only the generic replay paragraph in `docs/Diagnostics.md`. The UaLens capture-window paragraph remains with the source PR. No UaLens projects, UI, native-rendering paths, dependencies, or workflows are included. **Review follow-up:** `814c0f2dc6056ea5eb898611565b3e96cfa65104` fixes direct-placeholder references that bypassed filtering through `TargetPath` and emitted an undefined `forInstance` guard in instance-only factories. Filtering now follows target resolution. Compilation/reference-shape regressions cover ObjectType/VariableType, both placeholder rules, and direct-child/external targets; `docs/NodeManagers.md` documents the behavior. The review thread was resolved after publication and verification. **CI follow-up:** `93ec3928bd5a9cb2ea86110085e79131db17d443` addresses both causes found in [CI run 38700770996](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/38700770996): - `MODELGEN012` treated an identical payload-less transitive dependency declaration as a competing model producer after the SDK Programs model appeared in newer `master`. Matching closure rows no longer emit that diagnostic. Producer ordering is unchanged; actual producers, differing declarations, and malformed payload handling retain their existing diagnostics. New regression cases exercise both reference orders, and `docs/ModelDependencies.md` records the distinction. - The queue-fairness test sampled OS scheduling and failed on macOS. This reuses the exact one-file correction from `84d2c34d3902bfe700a87f6c30708b435500878f` in #4765: hold the existing heavy-owner publication hook, prove concurrent work-conserving progress, then assert exact recovered 1:3 service, all 960 unique requests, and complete queue/token/owner/lease cleanup. No production scheduler or fairness bound was changed. The PR now changes 26 paths. Its pinned extraction ancestry is preserved: no rebase, force push, or merge of newer `master` was performed. This remains the bottom of native stack 4807, ordered 4801 -> 4802 -> 4803 -> 4804 -> 4806. ### Current CI-repair validation Windows, existing .NET SDK **10.0.401**, **Release**, with analyzers enabled. CI tested merge `9a2d252793da12aeb99ed379dfc9b03e89c912b7` (master `d40843c23a5fe8116ffe380cf937d88759f5218b` plus PR head `814c0f2dc6056ea5eb898611565b3e96cfa65104`), which includes newer tests absent from the pinned extraction branch. That exact merge was reproduced in a disposable detached worktree, then the proposed patch was applied. Git patch-id verified that the tested patch and committed branch patch were identical (`b7ed3ba2b61d1862825dd33d93c50c64d3a759ea`). Before the fix, all four failing client-accessor cases reproduced `MODELGEN012`, and both new matching-reexport cases failed. After the fix: | Context / scope | Framework | Passed | Failed | Skipped | | --- | --- | ---: | ---: | ---: | | Exact CI merge + patch: client-accessor and dependency fixtures | net10.0 | 32 | 0 | 0 | | Same fixtures with CI coverage instrumentation | net10.0 | 32 | 0 | 0 | | Exact CI merge + patch: entire SourceGeneration.Tests project | net10.0 | 295 | 0 | 0 | | Exact CI merge + patch: entire SourceGeneration.Tests project | net48 | 295 | 0 | 0 | | Preserved PR branch: dependency and placeholder fixtures | net10.0 | 30 | 0 | 0 | | Preserved PR branch: dependency and placeholder fixtures | net48 | 30 | 0 | 0 | | Preserved PR branch: entire FairRequestQueueTests fixture, with coverage | net10.0 | 93 | 0 | 0 | | Preserved PR branch: entire FairRequestQueueTests fixture | net48 | 93 | 0 | 0 | | Exact CI merge + patch: entire FairRequestQueueTests fixture, with coverage | net10.0 | 93 | 0 | 0 | All nine commands exited **0**. The repository TRX verdict helpers verified **993 passing test executions, 0 failures, and 0 fixture setup/teardown failures** across these overlapping runs. No build/analyzer warnings were reported. Normal restore resolved missing assets/target entries in the new checkout and when switching TFMs; no dependency declaration or SDK pin was changed. Exact successful test argument sets follow. Variables abbreviate the executable, project paths, and results directory only; output was also captured to separate local logs. ```powershell $env:DOTNET_ROOT = 'D:\git\UA-.NETStandard_master\TestResults\ualens-528-deep-repro\sdk' $env:PATH = "$env:DOTNET_ROOT;$env:PATH" $dotnet = "$env:DOTNET_ROOT\dotnet.exe" $generator = 'tests\Opc.Ua.SourceGeneration.Tests\Opc.Ua.SourceGeneration.Tests.csproj' $coreTests = 'tests\Opc.Ua.Core.Tests\Opc.Ua.Core.Tests.csproj' $results = 'C:\Users\mschier\.copilot\session-state\8f497e89-7fc2-4f48-880b-4803d97c1c12\files\ci-validation' # In the disposable exact-CI-merge checkout, with the identical proposed patch applied: Set-Location 'D:\Temp\ua-ci4801-8f497e89' # Passed: 32, Failed: 0, Skipped: 0 & $dotnet test $generator --no-restore --configuration Release --framework net10.0 -p:CustomTestTarget=net10.0 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true --filter 'FullyQualifiedName~ClientAccessorGeneratorTests|FullyQualifiedName~ModelDependencyScannerTests' --logger 'trx;LogFileName=ci-metadata-green-net10.trx' --results-directory $results --verbosity minimal # Passed: 295, Failed: 0, Skipped: 0; reuses the analyzer-enabled build above & $dotnet test $generator --no-build --no-restore --configuration Release --framework net10.0 -p:CustomTestTarget=net10.0 --logger 'trx;LogFileName=ci-sourcegeneration-merge-net10.trx' --results-directory $results --verbosity quiet # Passed: 295, Failed: 0, Skipped: 0 & $dotnet test $generator --configuration Release --framework net48 -p:CustomTestTarget=net48 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true --logger 'trx;LogFileName=ci-sourcegeneration-merge-net48.trx' --results-directory $results --verbosity minimal # Passed: 93, Failed: 0, Skipped: 0 & $dotnet test $coreTests --configuration Release --framework net10.0 -p:CustomTestTarget=net10.0 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true -p:CollectCoverage=true --collect 'XPlat Code Coverage' --settings 'tests\coverlet.runsettings.xml' --filter 'FullyQualifiedName~FairRequestQueueTests' --logger 'trx;LogFileName=ci-fairqueue-merge-net10.trx' --results-directory $results --verbosity minimal # Passed: 32, Failed: 0, Skipped: 0 & $dotnet test $generator --configuration Release --framework net10.0 -p:CustomTestTarget=net10.0 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true -p:CollectCoverage=true --collect 'XPlat Code Coverage' --settings 'tests\coverlet.runsettings.xml' --filter 'FullyQualifiedName~ClientAccessorGeneratorTests|FullyQualifiedName~ModelDependencyScannerTests' --logger 'trx;LogFileName=ci-metadata-coverage-net10.trx' --results-directory $results --verbosity minimal # In the preserved PR branch worktree: Set-Location 'D:\git\copilot-worktrees\UA-wot-d3-final\marcschier-congenial-umbrella' # Passed: 30, Failed: 0, Skipped: 0 & $dotnet test $generator --configuration Release --framework net10.0 -p:CustomTestTarget=net10.0 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true --filter 'FullyQualifiedName~ModelDependencyScannerTests|FullyQualifiedName~PlaceholderReferenceGeneratorTests' --logger 'trx;LogFileName=ci-sourcegeneration-branch-net10.trx' --results-directory $results --verbosity minimal # Passed: 93, Failed: 0, Skipped: 0 & $dotnet test $coreTests --no-restore --configuration Release --framework net10.0 -p:CustomTestTarget=net10.0 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true -p:CollectCoverage=true --collect 'XPlat Code Coverage' --settings 'tests\coverlet.runsettings.xml' --filter 'FullyQualifiedName~FairRequestQueueTests' --logger 'trx;LogFileName=ci-fairqueue-branch-net10.trx' --results-directory $results --verbosity minimal # Passed: 93, Failed: 0, Skipped: 0 & $dotnet test $coreTests --configuration Release --framework net48 -p:CustomTestTarget=net48 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true --filter 'FullyQualifiedName~FairRequestQueueTests' --logger 'trx;LogFileName=ci-fairqueue-branch-net48.trx' --results-directory $results --verbosity minimal # Passed: 30, Failed: 0, Skipped: 0 & $dotnet test $generator --configuration Release --framework net48 -p:CustomTestTarget=net48 -p:RunAnalyzers=true -p:RunAnalyzersDuringBuild=true --filter 'FullyQualifiedName~ModelDependencyScannerTests|FullyQualifiedName~PlaceholderReferenceGeneratorTests' --logger 'trx;LogFileName=ci-sourcegeneration-branch-net48.trx' --results-directory $results --verbosity minimal ``` `git diff --cached --check` passed. The queue-test blob was verified byte-for-byte against the existing published correction. The upstream branch was verified as `93ec3928bd5a9cb2ea86110085e79131db17d443` after push. **Remote CI is pending, not claimed green:** [new CI run 38712127138](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/38712127138). Cross-platform confirmation, CodeQL, coverage gates, and the complete `UA.slnx` suites on this new head are not established by these local checks. The source PR's checks are not claimed as evidence for this branch. ### Earlier validation The original extraction had 1,407 passing scoped test executions across net10.0/net48. The placeholder-review follow-up had six passing focused cases and 269 passing tests in the full source-generator project on each framework, with analyzer-enabled builds. Those are historical results for their respective commits, not fresh full-suite evidence for the CI-repair head. Earlier command details remain in this PR's edit history. ## Related Issues - Related to #4461 (existing tracking issue; not auto-closing). - Extracted from #4462 (source PR; remains open and is not auto-closing). - Reuses the bounded queue-test correction from #4765; does not incorporate that PR's other changes. _If this is a relatively large or complex change, a design must have been discussed in the related tracking issue and signed off (which becomes the Architectural Decision Record (ADR))._ This began as an exact extraction of existing behavior. The subsequent commits address the reported placeholder-generation defect and observed CI failures; they are not a new design or a claim of design sign-off. ## Checklist _Put an `x` in the boxes that apply. You can complete these step by step after opening the PR._ - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received.
14 KiB
Runtime Schema Generation
The Opc.Ua.Core.Schema library generates schemas for OPC UA data types at runtime. It works for the encodeable types emitted by the source generators and for complex types that are added dynamically by the complex-type client. Schemas are produced in every supported encoding:
- XSD for the XML encoding.
- BSD (OPC Binary, Part 6) for the binary encoding.
- JSON Schema (draft 2020-12) for the Part 6 JSON encoding, in both the compact and verbose flavors.
Schemas are built as strongly-typed object models in code — there are no embedded schema strings — so unused generation paths are trimmed away and the whole library is NativeAOT compatible. The XSD object model is the in-box System.Xml.Schema.XmlSchema. BSD uses
the existing Opc.Ua.Schema.Binary.TypeDictionary, and JSON uses
System.Text.Json.Nodes.JsonObject.
Contents
- Concepts
- Registration
- Registering data types
- Generating a schema
- Working with the object model
- JSON encoding notes
- PubSub schemas
- OpenAPI document of the REST binding
- Trimming and NativeAOT
Concepts
Generation is driven by a type's runtime structure definition (StructureDefinition or EnumDefinition), which already captures every field, data type, value rank and optionality. The pieces fit together as follows:
ISchemaProvideris the entry point. It produces anIUaSchemafor a requestedUaSchemaFormatandUaSchemaScope.IUaSchemais the generated document. It exposes the strongly-typed object model (for exampleJsonSchemaDocument.Root,XmlSchemaDocument.Schema,BinarySchemaDocument.Dictionary) and can serialize itself withWriteTo(Stream),WriteTo(TextWriter)orToSchemaString().IDataTypeDefinitionResolvermaps a data type id to itsUaTypeDescription(the type id, browse name and definition). The default implementation,DataTypeDefinitionRegistry, is an in-memory registry that callers populate from browsed or otherwise available definitions. The resolver is also used to follow field references and to enumerate the types of a namespace.
UaSchemaFormat selects the encoding (Xsd, Bsd, JsonCompact, JsonVerbose). UaSchemaScope selects the document granularity: Type produces a document for a single type and the closure of the types it depends on; Namespace produces a dictionary document for all types in a namespace.
Registration
The services are registered through the standard OPC UA dependency-injection surface:
IServiceProvider services = new ServiceCollection()
.AddOpcUa()
.AddSchemaGeneration()
.Services
.BuildServiceProvider();
ISchemaProvider provider = services.GetRequiredService<ISchemaProvider>();
The provider can also be constructed directly when dependency injection is not used:
var registry = new DataTypeDefinitionRegistry();
registry.Add(new UaTypeDescription(typeId, browseName, structureDefinition, namespaceUri));
ISchemaProvider provider = new DefaultSchemaProvider(
registry,
[
new JsonSchemaGenerator(),
new XsdSchemaGenerator(),
new BsdSchemaGenerator()
]);
Registering data types
Schema generation needs the runtime definition of a type. A type's StructureDefinition / EnumDefinition is registered with the resolver from whichever source has it:
- Server / browsed types — a
DataTypeNodeobtained from a server (or the client node cache) carries its definition inDataTypeNode.DataTypeDefinition. Register it directly:
var registry = serviceProvider.GetRequiredService<DataTypeDefinitionRegistry>();
registry.TryAddDataType(dataTypeNode, session.NamespaceUris);
- Source-generated types — the generated model registration extension adds
public structure and enumeration activators to an
IEncodeableFactory. These activators implementIDataTypeDefinitionSource; the generated data value does not need to implement that interface. UseEncodeableFactoryDefinitionSourceto resolve their definitions without reflection or manual registration:
IEncodeableFactory factory = EncodeableFactory.Create();
factory.Builder
.AddOpcUa() // use the generated Add<Namespace>() extension for your model
.Commit();
IDataTypeDefinitionResolver resolver =
new EncodeableFactoryDefinitionSource(factory, namespaceUris);
ISchemaProvider provider = new DefaultSchemaProvider(
resolver,
[
new JsonSchemaGenerator(),
new XsdSchemaGenerator(),
new BsdSchemaGenerator()
]);
-
Dynamic complex types — complex types built by the complex-type client carry a
StructureDefinition(viaIStructureTypeInfo/ the structure-definition attribute) that can likewise be wrapped in aUaTypeDescriptionand registered. -
Types already in the encodeable factory — any structured or enumeration type known to an
IEncodeableFactory(source-generated types and runtime complex-type stand-ins) exposes its definition throughIDataTypeDefinitionSource, soEncodeableFactoryDefinitionSourceresolves it directly from the factory without a separate registration. Built-in primitive DataTypes (and simple types that derive directly from them) carry noDataTypeDefinitionand are handled by the built-in type system rather than this resolver. Use the factory source on its own, or combine it with a manualDataTypeDefinitionRegistry(for schema-only types that have no encodeable) viaCompositeDataTypeDefinitionResolver:
IDataTypeDefinitionResolver resolver = new CompositeDataTypeDefinitionResolver(
[new EncodeableFactoryDefinitionSource(factory, namespaceUris), registry]);
When AddComplexTypeSystem() is enabled, an OPC UA server wires this up
automatically (see ComplexTypes.md). The primed
encodeable factory becomes the schema resolver.
Once registered, fields that reference other registered types are resolved automatically and included in the generated document.
Standard field encodings
Number (i=26), Integer (i=27) and UInteger (i=28) are abstract simple
types, not structures requiring a DataTypeDefinition. When used as structure
fields, they use Variant encoding as specified by
OPC 10000-6, 5.1.6.
Their scalar and array field schemas therefore reference Variant rather than an
unconstrained value or an ExtensionObject.
SchemaTypeInfo.GetFieldEncodingType exposes this classification to callers that
walk schema dependencies. It also preserves the standard primitive/alias mappings;
custom-namespace identifiers such as ns=1;i=26 still require their own definitions.
It does not change the general-purpose TypeInfo classification.
Namespace identity
BrowseName.NamespaceIndex is not a mapping for the data type's NodeId namespace; the
two namespaces may legitimately differ. To support both local and URI-based lookup in
DataTypeDefinitionRegistry, register an index-form TypeId together with its namespace
URI, as TryAddDataType(node, namespaceTable) does:
registry.Add(new UaTypeDescription(
new ExpandedNodeId(dataTypeNode.NodeId),
dataTypeNode.BrowseName,
definition,
session.NamespaceUris.GetString(dataTypeNode.NodeId.NamespaceIndex)));
A custom URI-only TypeId is resolvable by URI but does not create an inferred local alias. The standard OPC UA namespace URI is the exception because its local index is always zero. Registry entries must share one local namespace-table context.
URI lookup preserves identifier kind as well as value. Numeric zero, an empty GUID, an empty string, and an empty ByteString are distinct identifiers in a nonzero namespace; they must not be conflated with namespace-zero null NodeIds.
Generating a schema
Once a type's definition is registered with the resolver, a schema can be produced from its type id:
if (provider.TryGetSchema(typeId, UaSchemaFormat.JsonCompact, UaSchemaScope.Type, out IUaSchema? schema))
{
string json = schema.ToSchemaString();
}
When the starting point is a namespace-index NodeId obtained from a server, convert it
with the session namespace table. A registry populated with
TryAddDataType(dataTypeNode, session.NamespaceUris) accepts both this URI form and the
local index-form ExpandedNodeId:
ExpandedNodeId typeId = NodeId.ToExpandedNodeId(dataTypeNode.NodeId, session.NamespaceUris);
provider.TryGetSchema(typeId, UaSchemaFormat.JsonCompact, UaSchemaScope.Type, out IUaSchema? schema);
The convenience extension methods read more naturally and make a type "expose" its schema:
IUaSchema xsd = provider.GetXmlSchema(type);
IUaSchema bsd = provider.GetBinarySchema(type);
IUaSchema jsonCompact = provider.GetJsonSchema(type);
IUaSchema jsonVerbose = provider.GetJsonSchema(type, verbose: true);
// Resolve by type id and produce JSON in one call.
provider.TryGetJsonSchema(typeId, out IUaSchema? schema);
Working with the object model
Because the schema is an object model, callers can inspect or post-process it before serializing. For JSON:
var document = (JsonSchemaDocument)provider.GetJsonSchema(type);
JsonObject root = document.Root; // the draft 2020-12 schema
document.WriteTo(stream); // UTF-8, indented
JSON encoding notes (Part 6)
The JSON schemas follow the Part 6 JSON encoding faithfully, matching what the stack's JsonEncoder produces:
Int64andUInt64are encoded as JSON strings (to avoid precision loss), so they are typed asstring.Float/Doubleaccept the special string valuesInfinity,-InfinityandNaN, so they are typed as["number", "string"].ByteStringis a base64string;DateTimeis adate-timestring;Guidis auuidstring.- The standard structured built-ins (
NodeId,Variant,ExtensionObject,DataValue, ...) are described once per document in the$defssection and referenced. - Variant fields, including abstract numeric structure fields, use the
UaTypeandValueenvelope with optionalDimensions. Null array entries are allowed, and numeric defaults such as zero retain an explicitValuein both Compact and Verbose encoding.Valueis omitted only for the NULL value of a nullable type (Part 6, 5.4.2.17, Table 41), while retainingUaType. The deprecatedType/Bodyenvelope is not the Compact/Verbose format. - Compact enums are integers (with the allowed values listed via
oneOf); verbose enums are theName_Valuestrings.
PubSub schemas
The Opc.Ua.PubSub.Schema library generates JSON Schemas for the PubSub JSON message formats. It is registered with services.AddOpcUa().AddPubSubSchema() and exposes IPubSubSchemaProvider:
CreateDataSetSchema(metaData, fieldContentMask, verbose)— the per-DataSet payload object, one property perFieldMetaData. The field value shape follows the same Part 6 JSON rules as the core library, andDataSetFieldContentMaskcontrols whether each field is the raw value or aDataValueobject (with the mask-selectedStatusCode/SourceTimestamp/ ... members).CreateDataSetMessageSchema(metaData, messageContentMask, fieldContentMask, verbose)— a single DataSetMessage whose header fields are gated byJsonDataSetMessageContentMaskand whosePayloadis the DataSet schema above.CreateNetworkMessageSchema(metaData, networkContentMask, messageContentMask, fieldContentMask, verbose)— theua-dataNetworkMessage envelope, gated byJsonNetworkMessageContentMask;Messagesis an array of DataSetMessage schemas, or a single object whenSingleDataSetMessageis set.CreateMetaDataMessageSchema(metaData, verbose)— theua-metadatamessage.
The provider reuses the core ISchemaProvider to resolve complex (structured/enum) field data types, embedding them into the document $defs section.
OpenAPI document of the REST binding
WebApiOpenApiGenerator generates the OpenAPI 3.0 document of the REST binding
(Part 6 G.3, see REST binding) from WebApiServiceRoutes and the
structure definitions of the request and response types. It is not an
IUaSchemaGenerator: a document describes a set of routes rather than one data
type.
var generator = new WebApiOpenApiGenerator();
JsonObject document = generator.Generate(
WebApiServiceSet.Sessionless, // or AllServices
includeSchemas: true, // false: a JSON object as every body
serverUrl: "/opcua/"); // null: no servers list
Without includeSchemas the document holds the paths and operations only
(about 12 KB for all 28 services). With it, every request, response and the
structures and enumerations they contain become component schemas (about 46 KB),
which is what OpenAPI client generators need. The schemas describe the Compact
JSON encoding with the property names and types the published OPC Foundation
documents use; OPC UA over OpenAPI lists the differences and the
tests that keep the document correct. Pass an IDataTypeDefinitionResolver to
the constructor to resolve the types from another source than the generated
types of the OPC UA namespace. The generator is registered by
AddWebApiTransport(), which serves the document.
Trimming and NativeAOT
The library opts into IsAotCompatible and avoids reflection-based serialization. XSD is written with System.Xml.Schema.XmlSchema, BSD with a direct System.Xml.XmlWriter, and JSON with System.Text.Json.Nodes / Utf8JsonWriter. Schema generation is a configuration-time activity, not a hot path; documents are built lazily and can be cached by the caller. Because the generation logic lives in its own assembly, it is trimmed away entirely when an application does not generate schemas.