GithubStar/UA-.NETStandard
1
0
Fork 0
mirror of https://github.com/OPCFoundation/UA-.NETStandard.git synced 2026-10-29 15:11:53 +08:00
Code Issues Packages Projects Releases Wiki Activity
2,875 Commits 77 Branches 105 Tags
master
Commit Graph
4 Commits
Author SHA1 Message Date
romanettandClaude Opus 5.5 78e66f3473 Aggregates: TreatUncertainAsBad never applies to bounding values; DurationGood/Bad count Good/Bad regions only (#4702) 
# Description

Server-side aggregate fixes that follow three answers from the OPC
Foundation on Mantis:
- [11462](https://mantis.opcfoundation.org/view.php?id=11462)
(~0025840): TreatUncertainAsBad does not apply to bounding values.
- [11425](https://mantis.opcfoundation.org/view.php?id=11425)
(~0025847): DurationGood counts only Good regions and DurationBad only
Bad regions, and TreatUncertainAsBad does not apply.
- [11426](https://mantis.opcfoundation.org/view.php?id=11426)
(~0025852): Minimum, Maximum, the ActualTime variants and Range look at
Uncertain values as if they are Good.

The principle in 11425 and 11426 is that the aggregate definition wins
over TreatUncertainAsBad. The CTT-side defects are filed as
[11487](https://mantis.opcfoundation.org/view.php?id=11487).

### Bounding values (`AggregateCalculator`,
`StartEndAggregateCalculator`)
- Simple (§3.1.9) and Interpolated (§3.1.8) Bounding Values are formed
from **non-Bad** raw values (`IsBoundCandidate`), whatever
TreatUncertainAsBad is.
- An Uncertain raw value before or after the timestamp makes the bound
`UncertainDataSubNormal`.
- A raw value exactly at the timestamp is the bound and keeps its own
status.
- Only a **Bad** value after a Simple bound reverts to stepped
interpolation.
- StartBound/EndBound keep an Uncertain bound instead of `Bad_NoData`
(Tables 76/77).
- WorstQuality2 now includes an Uncertain start bound, as §5.4.3.36
requires ("the server shall ignore the TreatUncertainAsBad for this
aggregate").

### How aggregates use the values (unchanged rule, two gaps closed)
TreatUncertainAsBad still governs how an aggregate uses the values
inside the interval, including an Uncertain bound.
- `GetRegionsInValueSet` now also converts an Uncertain **first** region
to Bad for Simple-bound aggregates. TimeAverage2/Total2 therefore omit
it like any other Uncertain region (§5.4.3.7, Table 55 "Bound
Uncertain").
- TimeAverage/Total (Interpolated bounds) still use an Uncertain bound
and report Uncertain (Table 54).
- TimeAverage2/Total2 return the PercentTime status `Bad` instead of
`Bad_NoData` for an interval that has data but only omitted data
(§5.4.3.2.1).
- Minimum2/Maximum2 are unchanged: an Uncertain bound is a candidate
only when TreatUncertainAsBad is false (§5.4.3.15).

### NumberOfTransitions (`CountAggregateCalculator`)
§5.4.3.24 speaks only of "non-Bad" values, and Table 72 says "Bound
Uncertain: Use as value". So:
- Uncertain values are counted and can be the previous value, whatever
TreatUncertainAsBad is.
- The previous value is the nearest non-Bad value before the interval
(the early bound).

This reverses an earlier change that made NumberOfTransitions honour
TreatUncertainAsBad.

### Minimum, Maximum, MinimumActualTime, MaximumActualTime, Range
(`MinMaxAggregateCalculator`)
Per the 11426 answer:
- Every non-Bad value is a candidate, whatever TreatUncertainAsBad is.
- An Uncertain extremum is returned with `UncertainDataSubNormal` plus
Calculated. ActualTime keeps the timestamp of the first such value.
- Otherwise the status is Good unless the interval contains Bad values
(then `UncertainDataSubNormal`); it is `BadNoData` if all values are
Bad.
- Range is max − min over the non-Bad values. It is Uncertain if either
extremum is Uncertain or Bad values exist.

Before this change the server ignored an Uncertain value beyond the Good
extremum and returned the Good one with an Uncertain status.
Minimum2/Maximum2/Range2 are unchanged, because §5.4.3.15 ties them to
TreatUncertainAsBad explicitly.

### DurationGood/DurationBad/PercentGood/PercentBad
(`StatusAggregateCalculator`)
- A region is Good only if its data point is Good, and Bad only if it is
Bad.
- Uncertain counts for neither (§5.4.3.31–34).
- The first region still takes the status of the raw value at or before
the start.

### Tests
- New `Aggregates/BoundingValueTreatUncertainAsBadTests`: DeltaBounds,
StartBound/EndBound, TimeAverage2/Total2, Interpolative,
NumberOfTransitions (forward and reverse),
DurationGood/Bad/PercentGood/Bad.
- New Min/Max test for Bad-only and Bad-next-to-Good intervals.
- Updated tests that encoded the old interpretation:
  - Interpolative with an Uncertain raw value at the start: 2 tests.
  - Duration/Percent Good/Bad: 2 tests.
  - NumberOfTransitions with TreatUncertainAsBad: 2 tests.
- Minimum/Maximum/Range with an Uncertain extremum or only Uncertain
values: 2 tests in Server.Tests, 1 in History.Tests.
- `ProcessedReadUsesAdvertisedNodeAggregateDefaultsAsync`: it now probes
the TreatUncertainAsBad default with Count, because PercentGood no
longer depends on it.

### CTT verification
CTT 1.05.513 with the patched scripts (c3674ff, which encode the
11462/11425/11426 answers), Aggregates part (p06):

| Server | Errors | Warnings | Pass | NotImpl |
| --- | --- | --- | --- | --- |
| master fa06c66db | 185 | 1 | 1078 | 3 |
| this branch (854769a0c) | **0** | 2 | 1262 | 3 |

The 185 errors on master break down as:
- Bounds: DeltaBounds 9, StartBound 12, EndBound 12, WorstQuality2 11,
TimeAverage2 5, Total2 5, Interpolative 1.
- Duration*/Percent*: DurationBad 18, PercentBad 18, DurationGood 4,
PercentGood 4.
- Minimum/Maximum/MinimumActualTime/MaximumActualTime/Range: 70.
- NumberOfTransitions: 16.

The 2 warnings on this branch are ActivateSession response-delay
warnings in `initialize.js`, caused by machine load.

Alarms & Conditions (p18) failed intermittently on PR builds: in the
Refresh2, Shelving and Exclusive/Non-Exclusive Level CUs, subscriptions
were lost or filtered events were missing. The same clean build of
854769a0c failed once (3 errors) and passed once (0 errors). The PR with
`src/Opc.Ua.Server/Aggregates` reverted passed, and master passes too.
These failures are A&C flakiness on a heavily loaded shared machine and
are not caused by this PR, whose code runs only in HistoryRead and
aggregate-filter MonitoredItems.

Full 23-part CTT run against this branch (854769a0c, patched scripts
c3674ff/c341918): **1 error in total**, GDS Application Directory
`019.js`, a known CTT native-stack defect (Mantis 11409) that also fails
on master. Aggregates (p06) and Alarms & Conditions (p18) have 0 errors,
and no test case went from Pass to Error compared with master.

### Reviewer notes
- §3.1.7 says *"In all cases the TreatUncertainAsBad (see 4.2.1.2) flag
is used to determine whether Uncertain values are Bad or non-Bad."* This
branch follows the answer to 11462, which states that the flag does not
apply to bounding values. This is documented in `plans/ctt-issues.md`
(U4).
- TimeAverage2/Total2 still apply TreatUncertainAsBad to an Uncertain
start bound when they use it: Table 55 says "Bound Uncertain: Treated as
any other Uncertain value", and the definition omits non-Good data. This
includes a start bound that is Uncertain only because the raw value
after it is Bad.
- Not in scope: `CastToOriginalType` uses `Convert.ToInt32(double)`
(banker's rounding), whereas Part 4 §7.7.3 says to add 0.5 and truncate.
The two differ only at .5 and for negative values.

## Related Issues

- Mantis 11462, 11425 (answers implemented), 11487 (CTT side)

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
(Aggregate/Historian tests of Opc.Ua.Server.Tests and
Opc.Ua.History.Tests on net10.0; net48 build only)
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 21:49:13 +02:00
romanettandClaude Opus 5.5 9b034f8291 Enable nullable reference types centrally; migrate all unset and annotations-only projects (#4700) 
# Description

Nullable reference types are now enabled once, in `common.props`, and
every project in the repository builds warning-clean with them.

**Configuration**
- `<Nullable>enable</Nullable>` lives in `common.props`. All per-project
`<Nullable>` settings are removed: 178 project files (enable,
annotations, and the `disable` in the three Aot test projects).
- 419 per-file `#nullable enable`/`restore` directives and the 4
`#nullable disable` directives are removed.
- Kept on purpose:
- The 5 `<auto-generated>` schema files (`UANodeSet.cs`, `UA Type
Dictionary.cs`, `OPCBinarySchema.cs`, `SecuredApplication.cs`,
`PInvokeHelper.cs`) keep `#nullable enable`. The compiler ignores the
project setting in generated code, so the directive is required there.
- `tests/Opc.Ua.Interop.LegacyPeer` keeps `disable`. It builds against
the 1.5.378 packages and does not import the repository props.
- `docs/DeveloperGuide.md` and `.github/copilot-instructions.md` now
describe the central setting.

**Migration** (the 43 unset and 13 annotations-only projects, roughly
10k diagnostics)
- **Production code: annotated at the declarations.** Covers
`tools/Opc.Ua.SourceGeneration.Core`, `src/Opc.Ua.Lds.Server` and the
LDS sample.
- Members, parameters and returns are nullable where null actually
flows.
- Null-check helpers carry `[NotNullWhen]` (`IsNull`, `IsValidXmlName`,
`IsMethodTypeDesign`).
- `CoreUtils.Format`, `Utils.Format`, `ITemplateWriter.WriteLine` take
`params object?[]`. `Template.AddReplacement` takes `string?` (rendering
already skips null replacements).
- `IRegisteredServerStore.RegisterAsync` returns
`Task<RegistrationEntry?>`: it returns null when a registration is
removed.
- Dereferences backed by an invariant (for example, the validated model)
use `!`, so runtime behaviour is unchanged.
- **Behavioural changes in production code** (small, deliberate):
- `NodeSetToModelDesign` throws `InvalidOperationException` when the
file is not a NodeSet. Before, it threw a NullReferenceException on the
next line.
- `ModelDesignValidator` starts its lookup tables empty instead of null.
A redundant `if (m_nodes == null) return;` guard is gone.
- **Test code: minimal annotations.**
  - `?` on values that can be null, `!` where the test asserts non-null.
  - `= null!` for fixture members assigned in SetUp.
- `TryGetStructure<T>(out T? x)` where the inferred `T?` broke the
`IEncodeable` constraint.
- A few redundant `== null` checks on structs (`NodeId`, `ArrayOf<T>`)
folded into the adjacent `IsNull`/`Count` checks.

**Notes for reviewers**
- The diff is large (~1,500 files) but almost entirely mechanical: `?`,
`!` and directive removal. The non-annotation changes are listed above.
- Known follow-up, not done here:
`IModelDesign.Namespaces/TargetNamespace/NamespaceUris` stay
non-nullable even though the validator returns null before it loads a
dictionary. Making them nullable would ripple through every generator
call site.

## Related Issues

- Fixes #4439

## Checklist

- [ ] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [ ] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

The "no new warnings" box covers net10.0 and net48: full non-incremental
builds of `UA.slnx`, `tools/SourceGeneration.slnx`,
`tools/MigrationAnalyzer.slnx` and the Aot test projects finished with 0
warnings and 0 errors under the repository's warnings-as-errors
settings. net9.0 and net8.0 also had no nullable diagnostics, but they
were not rebuilt after the final fixes. Tests have not been run locally
yet.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-08 08:48:13 +02:00
romanettandClaude Opus 5.5 4c1c3fd244 Aggregates: fix remaining CTT aggregate server defects and seed Bad data for the CTT (#4655) 
Fixes the remaining CTT Aggregate failures of the reference server that
are server defects or missing test data, using the patched CTT 1.05.513
scripts (all known Mantis defects fixed) as the oracle.

## Server fixes (`src/Opc.Ua.Server`)

- **Real-valued bounds for TimeAverage/TimeAverage2/Total/Total2.**
`SlopedInterpolate` converted every interpolated bound to the raw data
type, so on Int32 nodes these Double aggregates used rounded bounds
(ramp 0..23.8: 11.9034 instead of 11.9). The conversion (Part 4 §7.7.3)
is now only applied for aggregates that return the raw data type
(Interpolative, Start/EndBound, DeltaBounds, Minimum2/Maximum2/Range2,
ActualTime2); `AverageAggregateCalculator` keeps the real bound (Part 13
§3.1.8, Table 50).
- **Non-numeric values are interpolated stepped.** Boolean, String and
other non-numeric values cannot be interpolated with a sloped line (Part
13 §5.4.2.3, Table 50). The calculator now uses stepped interpolation
for them whatever the Stepped property says: a String EndBound between
"23" and "24" is "23" (was "23.8"), a Boolean bound between false and
true is false, and Boolean DurationInState no longer counts the region
before the first value as true.
- **TreatUncertainAsBad = false in Average and Count.** Both used only
strictly Good values for the result but counted Uncertain values as Good
for the status. They now include Uncertain values when
TreatUncertainAsBad is false (Part 13 §4.2.1.2), like StandardDeviation
and NumberOfTransitions already did.
- **Interpolative honors TreatUncertainAsBad for a raw value at the
interval start**, as the calculator already did for the raw values
around a bound.
- **NumberOfTransitions**: a Bad result carries no aggregate bits, like
Count.
- **Processed reads get the bounding values they need**
(`HistorianDispatcher`). The raw read only returned the raw value at or
beyond each end of the request. When that value is Bad (or Uncertain
with TreatUncertainAsBad), the interpolated bound needs the nearest Good
value further out, so an interval starting on a Bad value returned
`BadNoData` and the next interval lost its first region. The dispatcher
now reads outwards to the nearest Good values (reusing the ReadAtTime
bound search), only when the end value is not Good.

## Reference server test data and CTT template

- Every scalar history node also gets 400 values at a fixed date
(`2026-01-01T00:00:00Z`) with a block of five Bad values, laid out for
the CTT's Bad-data search (`HAAggregateHelper.GetBadData`).
StartOfArchive is the fixed date. The values relative to the server
start are unchanged.
- `samples/UAReferenceServer.ctt.xml`: all ten
`.../Aggregates/StartOfBadData*` settings point at the block
(`2026-01-01T00:25:01.234Z`), and `RequestedNodeId` is enabled with
namespace 2 (needs the C34 fix).
- `plans/ctt-issues.md` / `plans/ctt-testing.md`: updated configuration
notes, the oracle differences, U4, and new U5 (TreatUncertainAsBad =
false). Also documents a CTT defect: the Bad-data search's "reverse"
read sends EndTime = 0001-01-01 (null), which a server reads forward.

## CTT results (p06 Aggregates, 1267 test cases)

| | Error cases |
| --- | --- |
| Before (patched scripts, origin/master) | 253 |
| This PR, patched scripts | 125 |
| This PR, patched scripts + proposed Int32 oracle corrections for
TimeAverage/Total(2) and Range2 | 51 |

The 125 include 78 TimeAverage/TimeAverage2/Total/Total2 and 4 Range2
test cases on Int32 nodes where the server now returns the Part 13 value
but the patched oracle still truncates the bounds. The remaining 51 are
spec questions: U4 (DeltaBounds, 9; TimeAverage2/Total2 with an
Uncertain value after the start bound, 8) and U5 (TreatUncertainAsBad =
false, 34). p07 Historical Access: no change (0 errors).

## Tests

- New `AggregateBoundDataTypeTests` (real bounds on Int32, Interpolative
still rounds, Boolean/String stepped bounds, Boolean DurationInState,
Average with TreatUncertainAsBad).
- `AggregateCttRegressionTests`: direct and live (dispatcher) interval
starting on a Bad value, Interpolative with TreatUncertainAsBad,
NumberOfTransitions Bad result; the Count test now expects Uncertain
values to be counted with TreatUncertainAsBad = false.
- `AggregateCalculatorCoverageTests`: an Uncertain raw value at the
start with TreatUncertainAsBad is interpolated over.
- `Opc.Ua.Server.Tests` and `Opc.Ua.History.Tests` pass on net10.0;
net48 builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 19:04:17 +02:00
Marc Schier 7cec2d614d Move repository files to the consolidated layout 2026-07-16 06:25:54 +02:00