4 Commits

Author	SHA1	Message	Date
Klaus Landsdorf	cc7908eaa1	WebApi: serve a generated OpenAPI document of the REST binding (opt-in) and map the session-less service set (#4657) ... Reworked after the review: the OpenAPI document is generated from the code, not embedded; the route is opt-in; `StatusCode` is not accepted. The branch is a new history on top of master (the first version conflicted); master with #4674 is merged in. # Description The REST binding implements the routes of the OPC Foundation OpenAPI mapping (OPC 10000-6 §G.3), but it does not describe them: `GET /openapi.json` is answered with HTTP 405, and a client generator has nothing to read from the server. `docs/WebApi.md` listed a generated OpenAPI document as a deferred follow-up. The coded API is the source of truth, so this PR **generates** the document from the route table and from the structure definitions of the request and response types, and compares it with the document the model compiler publishes in tests. Nothing from `UA-Nodeset/OpenApi` ships in the package. ## Changes ### Service sets (`Opc.Ua.Core`) - `WebApiServiceSet` (`AllServices`, `Sessionless`) and `WebApiServiceRoutes.GetRoutes(serviceSet)`. `Sessionless` is the eight services of `opc.ua.openapi.sessionless.json` (OPC 10000-4 §6.3): Read, Write, HistoryRead, HistoryUpdate, Call, Browse, BrowseNext, TranslateBrowsePathsToNodeIds. ### Generator (`Opc.Ua.Core.Schema`, `Opc.Ua.Schema.OpenApi`) - `WebApiOpenApiGenerator.Generate(serviceSet, includeSchemas, serverUrl)` returns the OpenAPI 3.0 document as a `JsonObject`. It is built at runtime from `WebApiServiceRoutes` and the `StructureDefinition`/`EnumDefinition` of the message types (the same definitions the schema generation uses, resolved through `IDataTypeDefinitionResolver`). No reflection, so it is NativeAOT compatible; the AOT test project covers it. - **Two levels of detail**, because full schemas make big documents: | | Content | All 28 services | | --- | --- | --- | | default | paths, `operationId`s, every body a JSON object | about 12 KB | | `includeSchemas: true` | one component schema per message, structure and enumeration (100), for typed client generators | about 46 KB | The two published files are about 300 KB together. - The schemas describe the Compact JSON encoding, with the property names, types, formats and integer ranges of the published documents. DataValue is the one structure that differs: its status property is `Status`, as in Part 6 Table 42. - `OpenApiBuiltInSchemas` holds the schemas of the built-in types (StatusCode, LocalizedText, Variant, DataValue, DiagnosticInfo, ExtensionObject and the scalar types) as the encoder writes them. Abstract `Structure` fields are ExtensionObjects, abstract number fields are Variants. ### Serving the document (`Opc.Ua.Bindings.Https`) - `WebApiTransportOptions`: `ServiceSet`, `OpenApiDocumentPath` (default `null`: **off**) and `OpenApiIncludeSchemas` (default `false`). - `MapWebApiEndpoints(WebApiTransportOptions)` maps the routes of the service set and, when a path is set, the `GET` route of the document **in the same route group**. The route has no `AllowAnonymous` metadata. When an authentication scheme is registered, the contributor's `group.RequireAuthorization()` covers the document like `POST /read`; only `/findservers` and `/getendpoints` stay anonymous. The existing `MapWebApiEndpoints()` maps all routes as before. - The document is generated once per path base (listed as a relative `servers` URL; without a path base the list is omitted) by the `WebApiOpenApiGenerator` that `AddWebApiTransport` registers, or by a default one. - `Opc.Ua.Bindings.Https` references `Opc.Ua.Core.Schema` on net8.0 and later. ### Documentation - `docs/OpenApi.md` (new, linked from `docs/README.md`): serving the document, the two levels, generating clients, the JSON generated clients send, known issues, how the document is kept correct, troubleshooting. - `docs/WebApi.md`: new section "Service sets and the OpenAPI document"; the deferred follow-up is removed. `docs/SchemaGeneration.md` and both `NugetREADME.md` files describe the generator and the options. ## Comparison with the model compiler documents The two published documents are test resources of `Opc.Ua.Core.Schema.Tests` only (README next to them: source commit and update procedure). `WebApiOpenApiConformanceTests` compares the generated document with them for both service sets: - paths, `operationId`s, and the request and response schema of every operation; - the set of component schemas the operations reach: equal (100 for all services); - every enumeration (members and values); - for every structure: property names, types, formats, integer ranges, and inheritance. Everything both describe is equal. The test lists the differences and fails when a listed one disappears or a new one appears: | Difference | Generated | Published | Reason | | --- | --- | --- | --- | | DataValue status property | `Status` | `StatusCode` | Part 6 Table 42; UA-Nodeset#146 is open. The stack only knows `Status` (encoder and decoder). | | `description`, `default` keywords | not written | spec links, .NET defaults | documentation and defaults, not contract; Compact omits default values | `WebApiOpenApiWireFormatTests` validates what the binding really writes (the default request and response of all 28 routes, plus populated requests and responses with variants, status codes, references, identity tokens and notifications) against the generated component schemas. ## Tests - `Opc.Ua.Core.Schema.Tests` `OpenApi/`: `WebApiOpenApiGeneratorTests` (shape of the document, both levels, both service sets, servers list, sorting and determinism, concurrency, argument and resolver errors, every built-in type, arrays and ranks, enumerations, option sets, unions, base types, self-references, unknown definitions), `WebApiOpenApiConformanceTests`, `WebApiOpenApiWireFormatTests`, `OpenApiBuiltInSchemasTests`: 81 tests. - `Opc.Ua.Bindings.Https.WebApi.Tests`: `WebApiOpenApiDocumentEndpointTests` (TestServer with the startup contributor: off by default, configured path, both levels, service set maps and describes only the sessionless routes, path base, repeated requests, registered generator is used, **a request without credentials gets 401 when an authentication scheme is registered while one with credentials gets 200, the route carries `IAuthorizeData` and no `IAllowAnonymous`**, argument checks), options defaults and DI registration of the generator. - `Opc.Ua.Core.Tests`: `GetRoutes` for both sets and an undefined value. - `Opc.Ua.Aot.Tests`: the generator and a served document under the AOT fixture. - Results (macOS arm64): `Opc.Ua.Core.Schema.Tests` 260/260, `Opc.Ua.Bindings.Https.WebApi.Tests` 457/457, `Opc.Ua.Core.Tests` WebApi tests 110/110, each on net10.0 and net8.0 (Release). `Opc.Ua.Aot.Tests` (JIT run) SchemaAotTests 5/5, WebApiAotTests 8/8. - `Opc.Ua.Core`, `Opc.Ua.Core.Schema` and `Opc.Ua.Bindings.Https` and the three test projects build for net48, net8.0, net9.0 and net10.0 (Release) with 0 warnings and 0 errors (net48 builds only; the Windows legs of the fork CI run its tests). `dotnet format` whitespace, style and analyzers (`--severity info`) are clean on every changed file; the two `RCS1090` hits in `SchemaAotTests.cs` are on lines that master already has. - Patch coverage with `.azurepipelines/check-coverage.ps1` against master (`Opc.Ua.Core.Schema.Tests`, `Opc.Ua.Bindings.Https.WebApi.Tests` and the WebApi tests of `Opc.Ua.Core.Tests`, net10.0): 362/362 changed lines covered. - Fork CI (full matrix incl. macOS) and CodeQL on `f990af5db` (master with #4674 merged in): [CI run 37310622902](https://github.com/biancode/UA-.NETStandard/actions/runs/37310622902) green, [CodeQL run 37310626786](https://github.com/biancode/UA-.NETStandard/actions/runs/37310626786) green. The run on the earlier head `abda0e491` had one failure outside this PR: `McpServerProcessTests.HttpTransportAliasesExposeOnlyMcpRouteAsync("http")` timed out after 30 s on Windows net10.0; this PR does not touch `tools/`. ## Not in this PR - Authentication: the fix discussed in the review is @romanett's #4674. The document route uses whatever the REST routes use; this PR does not change authentication. - `StatusCode` tolerance in the decoder: dropped, the spec text is the reference. The first commit of the previous version is gone. - Samples. - Session-less invocation on the server side (OPC 10000-4 §6.3) is a separate draft PR. With `ServiceSet = Sessionless` this PR only limits the mapped routes and the document. ## Questions 1. `Opc.Ua.Bindings.Https` now references `Opc.Ua.Core.Schema` (net8.0 and later) because the generator lives next to the other schema generators and is trimmed away when unused. Fine, or should it move into `Opc.Ua.Core`? 2. Should `OpenApiIncludeSchemas` default to `false` as here, or should a host get typed clients out of the box? ## Related Issues - Fixes #4656 ## Checklist - [x] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. (Affected suites on net10.0 and net8.0 locally on macOS; .NET Framework is covered by the fork CI's Windows net48 legs.) - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received.	2026-10-06 09:43:28 +02:00
Marc Schier	c4e0721a29	Add a documentation learning path and fix issues found while verifying it (#4555) ... ## Summary ### Documentation - Stage the documentation index as a learning path: a first application, independent client, server, and PubSub paths, then data and services, companion models, production, advanced integration, and draft or experimental capabilities. - Add Getting started (repository samples or NuGet packages), OPC UA concepts, Your first information model, and a production readiness checklist. - Add Native AOT application publishing, a first PubSub publisher and subscriber exchange, and the UDP transport options. - Add or expand contents navigation throughout `docs/`, and keep contributor documentation separate. - Correct outdated SDK examples and capability guidance, consolidate repeated explanations into canonical guides, and repair local links and heading structure. - Rewrite What's New in 2.0 as a concise feature overview with short linked details; keep migration instructions in the Migration Guide. - Expand the feature catalogue with specification IDs, GDS, LDS, the OPC 10000-21 onboarding registrar, ISA-95, xRegistry, OpenUSD, and sample-local models; distinguish draft status, explain crypto pluggability, and bring performance guidance forward. - Address review feedback on wording, active monitored items during shadow reload, and the `Samples.md` filename and references. - Regenerate the `opcua-v20-migration` plugin's bundled copy of the migration guides so it matches the edited `docs/migrate/2.0.x` guides. - Merge the latest `master`, preserving upstream resource-isolation guidance within the task-based documentation index. - Retain deferred Lingity findings where appropriate; this PR does not claim that every document meets the Lingity score thresholds. ### Fixes found while verifying the guides Each fix has regression tests that fail without it. - **Client:** V2 subscriptions compared their lifetime with the operation timeout instead of the session timeout, and logged a spurious warning. `ISubscriptionEngineContext`, which is new in 2.0, gains a `SessionTimeout` property; implementations must add it. - **PubSub:** a subscriber now registers the `DataSetMetaData` configured on its readers (OPC 10000-14 6.2.9.4), and the UADP decoder decodes DataSetMessages that carry no ConfigurationVersion. Before, a secured subscriber of the reference sample received no DataSets. - **Source generator:** a ModelDesign method that declares its arguments inline, without a method type, produced code that did not compile; it now uses `MethodState`. The generated code of existing models is unchanged. - **MinimalClient sample:** waits for its first data-change notification before it disconnects. ### Known issues, not addressed here - UDP multicast: the publisher does not send on the interface that the subscriber joins, and `MulticastLoopback` defaults to false. The PubSub guide uses unicast for the exchange on one computer. - The UDP metadata announcement is sent unsecured on secured writer groups; a secured subscriber drops it with a warning. - A server does not start in globalization-invariant mode, because `ResourceManager` creates `CultureInfo("en-US")`. The Native AOT guide cautions against the mode. ## Validation - `git diff --check` and `./.azurepipelines/validate-migration-plugin.ps1` (15 bundled migration docs validated, plugin manifests synchronized). - Local Markdown links, anchors, file-link casing, and contents lists checked across all 87 documents under `docs/`. The index links every top-level guide and every target of the previous index. - Both Getting started routes, Your first information model, the PubSub exchange, and the Native AOT publish of the Getting started client and server were run as documented. The NuGet route and the tutorial use the public `2.0.0-preview.5` packages. - The changed libraries build for all target frameworks in Release without warnings. - Tests on .NET 10: Opc.Ua.SourceGeneration.Tests 199 passed; Opc.Ua.SourceGeneration.Core.Tests 3,977 passed, 8 skipped; Opc.Ua.Server.Tests `Fluent` category 621 passed; Opc.Ua.Client.Tests subscription-engine tests 70 passed; Opc.Ua.PubSub.Tests 1,374 passed, 2 skipped; PubSub fuzz replay 145 passed. The new and affected tests also pass on .NET Framework 4.8. - Earlier merge validation: 174 resource-isolation tests passed on .NET 10. - The full solution suite was not run locally. Review and validation artifacts are maintained outside the repository. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-09-28 19:19:39 +00:00
Marc Schier	62658d1a44	Fix runtime schema resolution and direct construction (#4426) ... # Description Fix runtime schema resolution and provide a supported DI-free construction path for generated data types. - Resolve registered data types by local/index-form and namespace-URI identifiers without using BrowseName as a namespace mapping. - Preserve identifier kind in URI registry keys, so numeric zero and empty GUID/string/ByteString identifiers remain distinct in nonzero namespaces, including during replacement. - Reconstruct typed identifiers during factory URI-to-index fallback while retaining exact lookup precedence and unknown-namespace/remote-server guards. - Make the built-in JSON, XSD, and BSD generators public, stabilize `EncodeableFactoryDefinitionSource`, and document generated activator/factory usage. - Keep the stable-API assertion effective across the netstandard2.1/net8.0 polyfill boundary by inspecting attribute metadata without referencing the ambiguous CLR type. - Merge the latest `master` without dropping upstream changes. ## Related Issues - Fixes #4423 - Fixes #4424 ## Validation - `Opc.Ua.Core.Schema.Tests`: 157/157 on `net10.0` - `Opc.Ua.Core.Schema.Tests`: 157/157 on `net48` - `Opc.Ua.Core.Schema.Tests`: 157/157 with `CustomTestTarget=netstandard2.1` and its `net8.0` consumer - Managed `SchemaAotTests`: 4/4 - NativeAOT Release publish and filtered `SchemaAotTests`: 4/4 - All three review reproductions pass against rebuilt assemblies. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [x] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [ ] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [ ] I have addressed **all** PR feedback received. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: b17fd166-fdef-4660-a9d5-97c0e69f1bd5	2026-09-06 19:17:49 +02:00
Marc Schier	7cec2d614d	Move repository files to the consolidated layout	2026-07-16 06:25:54 +02:00