Files
Marc Schier b17007bd55 Stop the encoder tests flooding the VSTest channel with per-test output (#4219)
# Description

The `Core.Encoders` test job of run
[`31249610490`](https://github.com/OPCFoundation/UA-.NETStandard/actions/runs/31249610490)
printed nothing for 57 minutes and was killed by the 60-minute job
timeout. Issue #4213 recorded that as "hung at or before the first test"
— that reading was wrong, and the evidence to disprove it was already in
the job's own artifact.

The blame collector *had* written a hang dump and it *was* uploaded,
inside the ordinary `dotnet-results-Core.Encoders-*` artifact (11 MB of
it was the dump), where nobody looked for it. Opening it names the cause
precisely.

## What the dump shows

522 threads (408 background) in a process that finishes in 25 seconds
when healthy. Every test had already run; NUnit was in `OneTimeTearDown`
waiting on its event pump:

- the **event pump thread** is blocked in
`NUnitEventListener.TestFinished` →
`TestRunEventsHandler.HandleLogMessage` →
`LengthPrefixCommunicationChannel.Send` → `Socket.Send`, on a socket
that never drained. That send holds the channel's lock;
- `TestRunCache.SendResults` is blocked on that channel lock, and it
holds the cache lock;
- `TestRunCache`'s **one-second timer** then fired repeatedly into that
held cache lock, and the thread pool injected a fresh worker for each —
**399 threads** parked in `Monitor.Enter_Slowpath` behind a single lock;
- the NUnit run thread sits in `EventPump.Stop()` → `Thread.Join()`, so
the run can never complete.

## What put the socket under that much pressure

Volume of test output. Every encoder case dumps its input, its encoded
payload and its decoded result, the NUnit adapter forwards each captured
line to the runner as an individual message over that socket, and the
suite has 2929 cases. This is not marginal:

| | Before | After |
| --- | --- | --- |
| `Core.Encoders` TRX, green local run | **496 MB** | **4 MB** |
| `Core.Encoders` published CI artifact | **156 MB** | ~3 MB |
| Next-largest test job's artifact, for scale | 15 MB | 15 MB |
| `Client.ComplexTypes` TRX | 74 MB | 40 MB |

So the suite was shipping half a gigabyte of text through a socket, one
message per line, on every run and on every OS — and paying for it in
artifact storage even when nothing went wrong.

## The change

`EncoderCommon` gains a buffered `TestOutput` writer. The dumps are
collected per test and handed to `TestContext.Out` in `TearDown` **only
when the test did not pass** — which is the only time anyone reads them.
Nothing is lost: a failing test still reports its full
input/encoded/decoded dump under *Standard Output Messages*, verified by
injecting a deliberate failure. Explicit opt-ins
(`PrettifyAndValidateJson(..., outputFormatted: true)`) and the static
helpers keep writing immediately.

The same base class is shared by the `Client.ComplexTypes` fixtures, so
those are moved over too rather than left with half-buffered,
half-immediate output.

Two supporting changes:

- **`.github/workflows/buildandtest.yml`** — `timeout-minutes: 30` on
the `Test` step. The blame collector did its job here (dump at +50 min,
once the run went quiet), but the run still had not unwound 7 minutes
later when the *job* timeout cancelled everything, so the step needs a
bound below the job's. `Upload test results` already runs `if:
always()`, so the dump is still published.
- **`docs/DeveloperGuide.md`** — a "Test output" convention under
*Running tests*, so the next data-driven fixture does not re-introduce
this.

## Notes for reviewers

- `[Parallelizable]` on these fixtures is `ParallelScope.Self`, i.e.
fixture-level; tests inside a fixture stay sequential, so the
per-fixture buffer is as safe as the existing per-fixture
`RandomSource`/`DataGenerator` state that is already reset in `SetUp`.
- I separately confirmed that `--blame-hang-timeout` *does* fire on a
hang before the first test (minimal NUnit repro, dump at exactly the
configured timeout), which is what rules out the original "hung during
startup/discovery" theory independently of the dump.
- Local verification was scoped to the two affected suites rather than
the whole solution: `Opc.Ua.Core.Encoders.Tests` 2929/2929 on
**net10.0** *and* **net48**, `Opc.Ua.Client.ComplexTypes.Tests`
2773/2773 on net10.0, all building with 0 warnings. The full `UA.slnx`
run is left to CI.

## Related Issues

- Fixes #4213

## Checklist

- [x] I have signed the
[CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf)
and read the
[CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md)
doc.
- [x] I have added tests that prove my fix is effective or that my
feature works and increased code coverage.
- [x] I have added all necessary documentation.
- [x] I have verified that my changes do not introduce (new) build or
analyzer warnings.
- [ ] I ran **all** tests locally using the **UA.slnx** solution against
at least .net **framework** and .net **10**, and all passed.
- [ ] I fixed **all** failing and flaky tests in the CI pipelines and
**all** CodeQL warnings.
- [ ] I have addressed **all** PR feedback received.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: da92151a-3da2-4caf-a39c-5556e56f0506
2026-08-09 21:49:05 +02:00

375 lines
14 KiB
C#

/* ========================================================================
* Copyright (c) 2005-2025 The OPC Foundation, Inc. All rights reserved.
*
* OPC Foundation MIT License 1.00
*
* Permission is hereby granted, free of charge, to any person
* obtaining a copy of this software and associated documentation
* files (the "Software"), to deal in the Software without
* restriction, including without limitation the rights to use,
* copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the
* Software is furnished to do so, subject to the following
* conditions:
*
* The above copyright notice and this permission notice shall be
* included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
* OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
* HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
* WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
* FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
* OTHER DEALINGS IN THE SOFTWARE.
*
* The complete license agreement can be found here:
* http://opcfoundation.org/License/MIT/1.00/
* ======================================================================*/
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Threading;
using System.Xml;
using NUnit.Framework;
using Opc.Ua.Core.TestFramework;
using Opc.Ua.Test;
using ComplexStructure = Opc.Ua.Encoders.Structure;
namespace Opc.Ua.Client.ComplexTypes.Tests
{
/// <summary>
/// Namespace constants for the default complex type tests.
/// </summary>
public static class Namespaces
{
/// <summary>
/// The URI for the OpcUa namespace.
/// </summary>
public const string OpcUa = "http://opcfoundation.org/UA/";
/// <summary>
/// The URI for the OpcUaXsd namespace.
/// </summary>
public const string OpcUaXsd = "http://opcfoundation.org/UA/2008/02/Types.xsd";
/// <summary>
/// The URI for the test encoder namespace.
/// </summary>
public const string OpcUaEncoderTests = "http://opcfoundation.org/UA/DefaultComplexTypeTests/";
}
/// <summary>
/// Common base class for default complex type tests.
/// Uses DefaultComplexTypeFactory/Builder instead of Reflection.Emit.
/// </summary>
public abstract class DefaultComplexTypesCommon : EncoderCommon
{
protected DefaultComplexTypeFactory m_factory;
protected IComplexTypeBuilder m_complexTypeBuilder;
protected int m_nodeIdCount;
[OneTimeSetUp]
protected new void OneTimeSetUp()
{
m_nodeIdCount = 0;
m_factory = new DefaultComplexTypeFactory();
m_complexTypeBuilder = m_factory.Create(
Namespaces.OpcUaEncoderTests,
3,
"DefaultTests");
}
[OneTimeTearDown]
protected new void OneTimeTearDown()
{
}
[SetUp]
protected new void SetUp()
{
}
[TearDown]
protected new void TearDown()
{
}
/// <summary>
/// Parameter class for structure field tests.
/// </summary>
public class StructureFieldParameter : IFormattable
{
public StructureFieldParameter(StructureField structureField)
{
Name = structureField.Name;
BuiltInType = TypeInfo.GetBuiltInType(structureField.DataType);
}
public string Name { get; set; }
public BuiltInType BuiltInType { get; set; }
public string ToString(string format, IFormatProvider formatProvider)
{
return Name;
}
}
[DatapointSource]
public StructureType[] StructureTypes =
#if NET8_0_OR_GREATER && !NET_STANDARD_TESTS
Enum.GetValues<StructureType>();
#else
(StructureType[])Enum.GetValues(typeof(StructureType));
#endif
[DatapointSource]
public StructureFieldParameter[] StructureField =
[
.. GetAllBuiltInTypesFields().Select(s => new StructureFieldParameter(s))
];
internal IEncodeableType BuildComplexTypeWithAllBuiltInTypes(
StructureType structureType,
string testFunc)
{
return BuildComplexTypeWithAllBuiltInTypes(null, structureType, testFunc, out _);
}
/// <summary>
/// Builds a complex type with all BuiltInTypes as properties
/// using the DefaultComplexTypeBuilder.
/// </summary>
private IEncodeableType BuildComplexTypeWithAllBuiltInTypes(
IServiceMessageContext context,
StructureType structureType,
string testFunc,
out ExpandedNodeId nodeId)
{
uint typeId = (uint)Interlocked.Add(ref m_nodeIdCount, 100);
var complexTypeStructure = new StructureDefinition
{
BaseDataType = structureType == StructureType.Union
? DataTypeIds.Union
: DataTypeIds.Structure,
DefaultEncodingId = default,
Fields = GetAllBuiltInTypesFields(),
StructureType = structureType
};
IComplexTypeFieldBuilder fieldBuilder = m_complexTypeBuilder.AddStructuredType(
QualifiedName.From(structureType + "." + testFunc),
complexTypeStructure);
nodeId = new ExpandedNodeId(typeId++, m_complexTypeBuilder.TargetNamespace);
var binaryEncodingId = new ExpandedNodeId(
typeId++,
m_complexTypeBuilder.TargetNamespace);
var xmlEncodingId = new ExpandedNodeId(typeId++, m_complexTypeBuilder.TargetNamespace);
fieldBuilder.AddTypeIdAttribute(nodeId, binaryEncodingId, xmlEncodingId);
int i = 1;
foreach (StructureField field in complexTypeStructure.Fields)
{
IType fieldType = TypeInfo.GetSystemType(field.DataType, null);
field.IsOptional = structureType == StructureType.StructureWithOptionalFields;
fieldBuilder.AddField(field, fieldType, i++, false);
}
IEncodeableType complexType = fieldBuilder.CreateType();
if (context != null)
{
context.Factory.Builder.AddEncodeableType(nodeId, complexType).Commit();
context.Factory.Builder.AddEncodeableType(binaryEncodingId, complexType).Commit();
context.Factory.Builder.AddEncodeableType(xmlEncodingId, complexType).Commit();
}
return complexType;
}
/// <summary>
/// Return a collection of fields with BuiltInTypes.
/// </summary>
private static List<StructureField> GetAllBuiltInTypesFields()
{
var collection = new List<StructureField>();
foreach (BuiltInType builtInType in BuiltInTypes)
{
if (builtInType
is BuiltInType.Null
or BuiltInType.Variant
or BuiltInType.DataValue
or BuiltInType.ExtensionObject
or >= BuiltInType.Number)
{
continue;
}
collection.Add(
new StructureField
{
Name = builtInType.ToString(),
DataType = new NodeId((uint)builtInType),
ArrayDimensions = default,
Description = LocalizedText.From($"A BuiltInType.{builtInType} property."),
IsOptional = false,
MaxStringLength = 0,
ValueRank = -1
});
}
return collection;
}
/// <summary>
/// Create complex types for all structure types and register them.
/// </summary>
internal void CreateComplexTypes(
IServiceMessageContext context,
Dictionary<StructureType, (ExpandedNodeId, IEncodeableType)> dict,
string nameExtension)
{
foreach (StructureType structureType in StructureTypes)
{
IEncodeableType encodeableType = BuildComplexTypeWithAllBuiltInTypes(
context,
structureType,
nameof(CreateComplexTypes) + nameExtension,
out ExpandedNodeId nodeId);
dict[structureType] = (nodeId, encodeableType);
}
}
/// <summary>
/// Helper to fill a Structure-based type with default or random values.
/// </summary>
internal void FillStructWithValues(
ComplexStructure structType,
bool randomValues,
NamespaceTable namespaceUris)
{
int index = 0;
foreach (IStructureField property in structType.GetFields())
{
BuiltInType builtInType = property.TypeInfo.BuiltInType;
Variant newObj = randomValues
? DataGenerator.GetRandomVariant(builtInType, !property.TypeInfo.IsScalar)
: Variant.CreateDefault(TypeInfo.Create(builtInType, property.TypeInfo.ValueRank));
if (newObj.IsNull)
{
switch (builtInType)
{
case BuiltInType.XmlElement:
var doc = new XmlDocument();
newObj = Variant.From(XmlElement.From(doc.CreateElement("name")));
break;
case BuiltInType.ByteString:
newObj = Variant.From(ByteString.From([1, 2, 3]));
break;
case BuiltInType.String:
newObj = Variant.From("This is a test");
break;
case BuiltInType.ExtensionObject:
newObj = Variant.From(ExtensionObject.Null);
break;
default:
newObj = Variant.CreateDefault(property.TypeInfo);
break;
}
}
structType[property.Name] = newObj;
Assert.That(newObj, Is.EqualTo(structType[property.Name]));
Assert.That(newObj, Is.EqualTo(structType[index]));
index++;
}
}
/// <summary>
/// Encode and decode a default complex type, verify the result.
/// </summary>
protected void EncodeDecodeComplexType(
IServiceMessageContext encoderContext,
MemoryStreamType memoryStreamType,
EncodingType encoderType,
JsonEncodingType jsonEncodingType,
bool useXmlParser,
StructureType structureType,
ExpandedNodeId nodeId,
IEncodeable data)
{
string encodeInfo = $"Encoder: {encoderType} Type:{structureType}";
TestOutput.WriteLine(encodeInfo);
TestOutput.WriteLine(data);
ExtensionObject expected = CreateExtensionObject(structureType, nodeId, data);
Assert.That(expected.IsNull, Is.False, "Expected DataValue is Null, " + encodeInfo);
TestOutput.WriteLine("Expected:");
TestOutput.WriteLine(expected);
byte[] buffer;
using (MemoryStream encoderStream = CreateEncoderMemoryStream(memoryStreamType))
{
using (
IEncoder encoder = CreateEncoder(
encoderType,
encoderContext,
encoderStream,
typeof(DataValue),
jsonEncodingType))
{
encoder.WriteExtensionObject("ExtensionObject", expected);
}
buffer = encoderStream.ToArray();
}
switch (encoderType)
{
case EncodingType.Binary:
TestOutput.WriteLine(PrettifyAndValidateBinary(buffer));
break;
case EncodingType.Json:
TestOutput.WriteLine(PrettifyAndValidateJson(buffer));
break;
case EncodingType.Xml:
TestOutput.WriteLine(PrettifyAndValidateXml(buffer));
break;
}
using var decoderStream = new MemoryStream(buffer);
using IDecoder decoder = CreateDecoder(
encoderType,
useXmlParser,
encoderContext,
decoderStream,
typeof(DataValue));
ExtensionObject result = decoder.ReadExtensionObject("ExtensionObject");
TestOutput.WriteLine("Result:");
TestOutput.WriteLine(result);
Assert.That(result.IsNull, Is.False, "Resulting DataValue is Null, " + encodeInfo);
Assert.That(result.Encoding, Is.EqualTo(expected.Encoding), encodeInfo);
Assert.That(result, Is.EqualTo(expected),
$"Failed to compare expected and result.\r\n{encodeInfo}.\r\n{expected}\r\n!=\r\n{result}.");
}
/// <summary>
/// Create an ExtensionObject for a complex type.
/// </summary>
/// <exception cref="ArgumentException"></exception>
protected ExtensionObject CreateExtensionObject(
StructureType structureType,
ExpandedNodeId nodeId,
object data)
{
switch (data)
{
case ByteString b:
return new ExtensionObject(nodeId, b);
case XmlElement x:
return new ExtensionObject(nodeId, x);
case IEncodeable e:
return new ExtensionObject(nodeId, e, true);
default:
throw new ArgumentException(
$"Unsupported data type {data.GetType()}" +
$" for structure type {structureType}.");
}
}
}
}