Handle a case where a misconfigured server returns a null value for an attribute when a value type is expected. To avoid the NullReferenceException that stops the whole ReadNodes operation, flag the Node with the service result and or return a default value if required.
For convenience, the DirectoryStore class creates the default folders when the DirectoryStore is opened. (since #2720)
However this may lead into issues mapping folders to the store when started in docker.
Fix: remove the code which creates default folders and ensure a null directory does not throw a NullReferenceException.
Add support for Compact and Verbose JSON encodings.
Other bug fixes:
- Defaults for non reversible forces the namespace Uri in the NodeId encoding and all default values are encoded. This behavior is according to spec. Old behavior can be changed reestablished with properties on creation of the JSON encoders.
- Compact and Verbose are preview for the next spec release. Main difference is to always encode NodeIds and ExpandedNodeIds as string with NamespaceUri. Same for QualifiedName. Both are considered reversible encodings.
- JSON decoder accepts any of the new and existing JSOn encodings no matter if the NodeId are encoded as string or JSON object.
Co-authored-by: Randy Armstrong <randy.armstrong@opcfoundation.org>
- OpenStore behavior changed to return null if the storepath or storetype are invalid or empty, which is a change to the previous release. Hence some support functions should check the return value of OpenStore.
- Fix: Check for null in LoadPrivateKeyEx
- in addition: try fix the new flaky test issue
- fix#2637: an exception when the https server starts and a X509Store certificate has a non exportable private key, it is not needed to create a copy for the TLS transport. Ignore the error and start the cert with the original cert.
- restructure which platforms are supported by https and complextypes, to avoid dependency collisions. Remove netcoreapp3.1 (eol), support netstandard2.0/2.1 instead.
---------
Co-authored-by: Hoelterhoff, Noah <noah.hoelterhoff@sap.com>
- During a reconnect fault the channels remain open and are quickly exhausted. Immediately close channels in Opening state similar to channels in Connecting state instead of waiting for timeout cleanup.
- The channel concurrentDictionary sometimes failed to remove a channel. Removing ContainsKey and retry with TryAdd fixes the problem.
- cert validator semaphore WaitAsync timeout was not correctly handled, also move all semaphores out of try/finally block to follow best practice
- ParseCertBlob shows up in the hot path. API is changed to use ReadOnlyMemory<byte>.
- Enum.ToString() was flagged in perf analyzer, changed logger call
- Add a light weight AddRejected certificate store method to `ICertificateStore` to ensure rejected cert history remains small and doesn't have a lot of overhead.
- Number of history certificates is configurable using a new `MaxRejectedCertificates` entry.
- Rejected certs are only saved to directory cert store, others do ignore the attempt
- Fix the caching of `CertificateValidator` and flag some `ÒpenStore` methods deprecated which do not support the caching. Fixes across the board to support the methodology. Basically `store.Close` keeps the cache, `store.Dispose` clears the cache, so recommendation is to use `var store = certStoreIdentifier.OpenStore(); try { ... }-- finally { store.Close }`
Server: Fix a deadlock in the binary channel cleanup (regression from secure channel cleanup)
Server: Use ConcurrentDictionary for sessions and channels to reduce locking overhead
Server: Replace legacy implementation of NodeIdDictionary with standard dictionary, as with IEqualityComparer the look up is much faster than specialized SortedDictionary for each NodeId Type.
Co-authored-by: Luis Cantero <luis@cantero.org>
Add the reconnect exception text to the log message, otherwise the exception is filtered out.
Turn off the trace logging in tests, as it causes timeout issues on some ci VMs
reduce callbacks if client reconnect and call AddItems
Fix BadNothingToDo when Loading dictionaries
Revert benchmark.net to v13 due to weird build dependency issues with 14.x
Fix the hello message, if the send and receive buffer size is different.
The error was undetected as long as send and receive buffer size were the same size.
Ensure the right values are used for client and server and add some more sanity checks on buffer sizes.
Added a new method ManagedBrowse to the UA client, which takes care of calling browse next after calling browse, when necessary. It also handles the status codes BadNoContinuationPoint and BadInvalidContinuationPoint.
Added support for the new method in ISession, TraceableSession, Session and SessionAsync.
Use the new methods in the NodeCache and other API that use browse (FetchRefrences etc).
Added unit tests which verify correct behavior of the new methods.
- let all Xml encoders and decoders follow the dispose pattern, similar to all the other encoders.
- Add fuzz targets for Xml and binary indempotent fuzzer.
- add a ReadOnlySpan<byte> signature for WriteByteString
- add ArrayPool based IBufferWriter<byte> to support encoding of large byte string with buffers
- add Azure DevOps ci pipeline tests with known artifacts that caused fuzzing errors, to catch regressions.
- Indempotent fuzzing revealed issue in DiagnosticInfo
- Matrix may have been used to decode one dimensional array if encoder used the `ArrayDimensions` field, which doesn't seem forbidden. Matrix caused cast exceptionwhen encoding the one dim atrray wrapped in a Matrix object.
- JsonEncoder throws InvalidCastException if NodeId Guid is encoded as Uuid.
- Strings with trailing `0` need to strip off all, or indempotent encoding fails.
- change macOS 11 ci builds to newer ones, macOS 11 is deprecated
- fix Opc.Ua.Client EventSource message numbering
- bump nerdgit to 3.6.139
- include .NET Standard 2.0 builds back in Nuget packages for client and server (complex type support requires netcoreapp2.1 which is end of life)
- include .NET Framework 4.7.2 as lowest supported .NET Framework platform
- move https client channel back to core library, as it requires only netstandard2.0 and has no other dependencies.
- Opc.Ua.Https is now only needed for a https server with kestrel.
- fix security warning for system.private.uri
- upload Quickstarts.Servers to the preview feed to allow use in the samples repo and avoid duplicate codebase
This PR extends the existing X509 CertificateStore building on top of the .NET X509Store Class to support Certificate Revocation Lists (CRLs) This is important to make the X509Store viable for use as Trust and Issuer Store.
Considerations:
- only Works on Windows, on all other OS the "SupportsCRLs"-Property of the Store still returns false
- builds on top of the Win32 API
- uses source generated P-Invoke calls to call into Win32 API (https://github.com/microsoft/CsWin32)
- makes use of Extension Methods of .NET X509Store Class to make the CRL Enumerate / Add / Delete accessible for the Store Implementation
- guards ever call with OS Support calls and throws PlatformNotSupportedException
- Behaviour of the X509CertificateStore for Linux/MacOS is unchanged
- Added unit tests for the Certificate Store testing all relvant implemented crl functions
- Extended GDS Tests for Integration Testing by utilizing Client & Server making second run on windows with only X509 Stores
This fix makes the client select the most secure UserIdentity Token if a server offers more than one.
Before the client just selected the first UserIdentity Token offered by the server.
Also calculate the security level by the client, as the unsecure response of a server could be spoofed.
- Fix the UserIdentity constructor to load the private key of a provided certificateIdenifier.
- make UserIdentity constructor throw a ServiceResultException when a Certificate with no private key is specified.
- extend console reference client with two new parameters:
-uc Thumbprint of a user certifiate located in the TrustedUserCertificatesStore
-ucp Password of the private key of the user certificate
- add documentation for console reference client
Fix that if a socket is closed, new service calls are queued and a new channel is established without reconnect / activate.
Add some logic which ensures that a BeginConnect operation is only started if a valid service call is queued first.
The activate is prepared by the session reconnect handler which may take some time to respond. Until then, all other service fail immediately with BadNotConnected.
To detect socket failures earlier, the PublishRequest and KeepAlive send are also hooked up to the KeepAliveError to trigger a reconnect. To disable the keep alive timing calcualtion, an additional variable tracks the statuscode of the last error to set KeepAliveStopped to true.
* Avoid user configuration errors by setting unaligned max message size values, fix defaults.
* Set the MaxMessageSize defaults to multiple of MinBufferSize to avoid rounding errors in other UA implementations.
- Add a solution to provide fuzzer support for afl-fuzz and libfuzzer.
- Add fuzz targets for Binary, Json and Xml encoder and decoder fuzzing. For afl-fuzz and libfuzzer.
- Fix issues in encoders and decoders when unexpected exceptions are returned. The expectation is to return only ServiceResultException with BadDecodingError, BadEncodingError or BadEncodingLimitsExceeded.
- Add a tool to create Testcases.
- Add a tool to playback crashes and timeouts.
- Solution is extensible with common files to discover new fuzz targets by reflection. New fuzz projects can share many files with the first project.
- Breaking change: Fix how BinaryDecoder is used: Previously, calling close on the BinaryDecoder implicitly disposed the stream. Change behavior so that `using` pattern can be applied across the board.
* Added MaxChannelCount and detect and cleanup "stale" channels based on ChannelLifetime
* Add Environment.TickCount as time reference
* Set a socket backlog value to avoid socket open overflow
* Added a ChannelKeepAlive to signal when a session is still active
* Add options to console client to leak channels and run connect loop forever
Co-authored-by: Martin Regen <mregen@microsoft.com>
- Implement `int Read(Span<byte> buffer)` and `void Write(ReadOnlySpan<byte> buffer)` in ArraySegmentStream
- for ArraySegmentStream streaming lower the CPU load for .NET8 OPC UA binary encoding/decoding by 50%
- Implement dispose to return buffers that were not transferred.
- Implement a GetSequence in ArraySegmentStream
Added redaction (disabled by default)
Added redaction strategies for username, password, endpoint, and exception message.
Updated logging to use redaction where it was critical
In #2510 a message type check was added to catch non UA service connections earlier.
But the Abort message type was missing in the check, so an Abort caused a disconnect.
However, in such a case the transport can just continue with the next message.
Fix allowed Roles of Method Unregister Application allow Application Self Admin Privilge:
https://reference.opcfoundation.org/GDS/v105/docs/6.6.8
Add Parameter NoPrivateKeys to OpenStoreMethod of CertificateStoreIdentifier to allow GDS client to update the private key of a store which is not its own store.
NodeState classes are not thread safe. In many places event handlers are handled in a manner that can cause NullReferenceException in a case of a bad timing. In addition, in some places non-thread-safe collections are iterated which could cause issues if the collections are changed at the same time.
This PR addresses the aforementioned issues by using local variables for event handlers and then making null check and calling them. When applicable null-conditional operators are used. Non-thread-safe collection iteration is guarded by locking. The issues were reproduces with unit tests part of this PR and also the fix was verified by running the same unit tests.
During high load there were a few portions in the send/receive pipeline identified which may cause lock contention and thread starvation due to locks:
- Remove lock for the codepath when a incoming message is processed.
- Add a concurrent dictionary to allow for the processing of OperationCompleted and to add new requests without locking
- Allow to process normal service requests without locking
Due to the change it is possible to send KeepAlive even when a incoming message is received.
-Implement the support for ApplicationSelfAdmin privilege
-implement the workflow in a test to pull the trust list after sucessfully receiving a gds signed certificate.
-this authorization is described here in the OPC Spec: https://reference.opcfoundation.org/GDS/v105/docs/7.6
- only republish available sequence numbers
- store republish result to handle unsuccessful message republish in main loop
- republish in parallel instead of sequential to reduce response times / blocking in the publish response processing loop
EscapeString is a function which parses every string for special characters, but often times it is already known that there are no escape characters to process. Improve WriteSimpleField to only call JsonEncoder.EscapeString if necessary.
-- note: fixed a few csproj files
Co-authored-by: Bhupendra Naphade <bhnaphad@microsoft.com>
Update the function to improve performance.
Fix test code warnings (CA1305) related to ToString() method
Based on the results of the benchmark runs, `EscapeStringLegacyPlus` was picked for all platforms that do not support `StreamWriter.Write(ReadOnlySpan<char>)`, e.g. .NET Framework. There is a slight improvement for the cases which require a 4 digit hex encoding.
For platforms that support Span end to end `EscapeStringSpanCharsInline` was picked. The functions tries to optimize how often the underlying StreamWriter is called by combining calls to Write into to slices. It was also noted that up to three character strings, it performs better to do single character writes than calling with a small string.
romanett