- set `KeepAliveTimeout` default back to 5s, was unintentionally set to 50s in 368 release
- fix a bug that duplicate subscriptions were created after reconnect when transfer succeeds.
- add a property to enable subscription transfer in reconnect, set `Session.TransferSubscriptionsOnReconnect=true` to enable, IOP issues may not allow to use it with some servers.
- add a `SessionReconnectHandler(true)` constructor to support reconnect abort, if the connection recovers before reconnect timer called. If the new constructor is used, its a breaking change: Needs check of `Session` property in Reconnect handler callback, if aborted `Session==null`.
- Reduce warning log output if server timestamp > client time.
- Add handling when subscriptions can not be transferred, e.g. when false user credentials or no security prevent transfer.
- Improve the reconnect timer handling by using `OperationLimit` in reconnect.
- Add sample for `SessionReconnectHandler` in console client.
- Seperate session sample code from `UAClient`.
- Tested on popular demo servers of various SDK vendors.
- Fix Winforms ref client
- Certificates should compare X500DistinguishedName with binary ASN.1 compare
- Other compares should follow ordinal and not resort DN entries
- Fix all related issues to pass tests, e.g. GDS certificates accepted with reversed Subject, cert validation
- Add defensive checks in cert validation
Co-authored-by: mrsuciu <Mircea-Adrian.Suciu@softing.com>
The OPC Foundation Technical Control Board on March 30th decided on the following changes with regard to the license information in the published Nuget packages:
• Replace the “Redistributables” license for OPC UA Nuget packages with the Dual License.
• Change the dual license in the source files so that “members” are replaced with “corporate members”. (see #1763)
Effectively,
- Nuget packages from libraries provided as samples which only build from source code which is covered by a MIT license, are the directly tagged with the MIT license.
- Nuget packages which build from source code where the headers imply the dual license applies, are tagged with the Dual License 'LICENSE.txt' found in the repository root.
* Fixed session recreate TransferSubscriptions bug.
1. When session is recreated and there is no subscription is needed to transfer, it will throw Exception - BadNothingToDo. After adding check the subscriptions count, it won't try to subscribe non-existing subscriptions.
2. When session is recreated and there are subscriptions are needed to transer, it transferred wrong subscriptions. It should transfer old subscriptions in old session, but current code transfer new subscriptions in new session to new session - which will always throw Exception - "A subscription can not be transferred due to missing transfer Id"
* Log update and TransferSubscriptions update.
fixes#1733 and #1734
- Doc, docs, docs updates for new 368 support
- fix a missing null pointer check when ConditionRefresh is run
- Ref server: move NodeManager factory API to StandardServer, decoupling the NodeManager implementation from Ref server
- Ref server: add shadow mode flag, to allow to map a docker configuration in a mapped folder.
- Ref server: sample batch files how to run Github CR and local docker images
- Improve some misleading 'delete application certificate' log messages.
- Fixed issue: After transfer, available sequence numbers were not properly ack/republished.
- Fixed issue: After transfer, race condition could delete subscription before transfer finished.
- Fixed issue: After reload of a subscription, duplicate client handles may have been used.
- Add tests which emulate the disconnection of a client/server, then transfers
- Add subscription variable to select if missing publish requests are just acknoledged (default) or republished after transfer completes. Set `Subscription.RepublishAfterTransfer` to `true`.
- Improve test framework to allow for testing against external servers using a .runsettings file
- improve publish request logging, by moving calls into `CoreClientUtils.EventLog`
- disable available sequence numbers check, it causes false negatives when many subscriptions are active
- Provide the CTT configuration used for 1.4.9.398: Added tests with arrays except Variant
- Match the Reference server config for CTT run (Console Server was used, others tbd)
- add a few cases in ref server node manager to enable skipped tests
- preset random generators to create deterministic node data
CTT fixes:
- Trigger: fix a regression introduced with the TransferSubscription support, Trigger stopped working, revert change to remove LinkedList in MonitoredItems
- return BadDataEncodingUnsupported creating MonitoredItems
- ensure that MonitoredItem node data reported as required by CTT after initial reading, transfer and SetMonitoring
note: not yet 100% passy, a few uncritical failing tests are still under investigation
Co-authored-by: mrsuciu <Mircea-Adrian.Suciu@softing.com>
- Implement client and server side of Transfersubscription service calls.
- New Transcription service has been tested with CTT 1.04.9.398
- Unit tests for native service calls and client SDK
- Client implements multiple ways to transfer sessions:
- Client saves subscriptions to xml stream, restarts, loads subscriptions from stream and transfers on server.
- Client creates a new sessions and transfers into new session, while the old session is still active.
- Client closes old session with `DeleteSubscriptionsOnClose = false`, creates new session and transfers subscriptions to new session.
- `SessionReconnectHandler` tries to transfer subscription before recreating the subscriptions.
- breaking change: INodeManager got a new TransferMonitoredItems method
Co-authored-by: Alexander Koepke <alexander.koepke@microsoft.com>
Co-authored-by: Peter Bryntesson <pbrynt@microsoft.com>
* remove the SetDurableSubscription method, to skip the test cases
* DeleteSubscription returns BadSubscriptionIdInvalid
* MethodState return also BadTooManyArguments
* return AttributeIdInvalid for null attributes (fix UA Expert display)
* fix ToString of Variant array of ByteString
* fix Variant decoding 1-dim array of bytestring
- integrate 'LoadPrivateKey' in the interface to allow to implement cert stores which require a passcode to load/store certificates
Change of behavior:
- Introduce a 'NoPrivateKeys' flag to reduce the risk of leaking a private key to a unsecure store.
- CertificateStoreIdentifier opens 'NoPrivateKey' stores
- CertificateTrustList opens 'NoPrivateKey' stores
- CertificateIdentifier opens store with private keys for application or GDS authority certificates
- CertificateTrustList reuses a store if it is not disposed. A user can just close the store to keep the list of certificates in memory.
- DirectoryStore `Load` does not read private keys anymore, only file names for delete. Reading all private keys caused false errors, e.g. here: #1670
Co-authored-by: mheege-abb <marcus.heege@de.abb.com>
Co-authored-by: mheege-abb <85437567+mheege-abb@users.noreply.github.com>
- The missing fix in https://github.com/OPCFoundation/UA-.NETStandard/commit/6747b471920ce165b59411d37c227e08d49dd047 for the bouncy castle certificate builder which was used in .NET Core 2.1 (eol) and .NET 4.6.2, unnoticed in the tests because windows only uses the keyId to validate the chain.
- AuthorityKeyIdentifier in the SubCA contains the SubjectName of the Issuer instead of the IssuerName. Also an application certificate that is signed by a SubCA would contain the false information.
- The false information has no effect on Windows and macOS, however on linux OpenSSL tests all fields and a chain cannot be fully validated.
* Removed assigning a default value to any variable value in the AddReverseReferences method.
Set the default value of the StatusCode on BaseVariableState to BadWaitingForInitialData.
* Allow initalization from variable states with status code BadWaitingForInitialData
The current SDK supports only Windows and Directory certificate stores. For security reasons, it can be necessary to use custom certificate stores.
* Add support for custom certificate stores
* Certificate Store Type UnitTests
* Start cleanup of ICertificateStore interface in the CRL section (Co-authored-by: Martin Regen <mregen@microsoft.com>)
- Adapted getIssuer and getIssuers so that all findings get aggregated for later processing in the validation methods.
- Ensured that other methods calling getIssuer(s) will experience unchanged behavior
- Adapted unit tests
- AuthorityKeyIdentifier in the SubCA contains the SubjectName of the Issuer instead of the IssuerName. Also an application certificate that is signed by a SubCA would contain the false information.
- The false information has no effect on Windows and macOS, however on linux OpenSSL tests all fields and a chain cannot be fully validated.
- For CTT tests a few error codes were return false codes because ServiceResult replaces the error code with the InnerException error.
- Do not allow to ignore PartialChain validation error, it may have different root cause then the error returned by GetIssuer.
- For EventSource and legacy logger refactor logging to support the built in LogValueFormatter, which provides the format strings suitable for `string.Format`.
- **Breaking change**: For `TraceEvent` this is a breaking change, because there is no implementation yet to support to provide the reformatted format string. With this implementation if a logging framework is used which supports the semantic logging, there is no issue. With frameworks which do not understand semantic logging strings(e.g. log4net) it is a breaking change, string.format will throw exceptions --> move to ILogger hook using `SetLogger`.
- **Warning**: When Microsoft.Extensions.Logger based loggers are used, format strings must follow this convention:
i) All variable entries must follow order and count up, e.g. "{0} -- {1}" etc., no mix of "{2}--{0}"
ii) in the same way the args must follow this strict ordering.
- Format strings will be changed by future PR to include the semantic values.
The Opc.Ua.Client.ComplexTypes package does not support recursive structs, i.e. a struct with a field that has the same data type as the struct.
An example of a recursive struct data type may look as follows:
PersonStructDataType
- Name <String>
- Age <Int32>
- Children <PersonStructDataType[ ]> (ValueRank == 1)
* smaller API changes for .368 and minor cleanup / uncritical bug fixes
* refactor GDS server lib to get rid of UA configuration dependency
* app config builder enhancements to simplify someuse cases
-- add a few default parameter to simplify fluent API
-- Rejected folder doesn't default to pki root
-- if the leaf pki folder is already provided, the default leaf folder is stil added (e.g. becomes own/own)
* certificate validator match were to return true even if no keyId/serial number was checked
* Directory store delete cert add retry if cert is still in use.
* enable reverse connect tests on macOS
* fixes CVE-2020-1045 (High) detected in microsoft.aspnetcore.http.2.1.1.nupkg mregen/UA-.NetStandardLibrary#114
- vs2017 now only supports .NET 4.6.2 builds, .NET Core 2.1 is end of life.
- Roslynator was not properly excluded in VS2017, change the condition to <Choose>
- fixes#1630
- fix a few lgtm warnings
- fix a few license headers (everything outside of \Stack is MIT)
- fix flaky Hiresclock test.
- support for Microsoft.Extensions.Logger.ILogger in Opc.Ua.Core as the new logging interface
- backward compatible with config and TraceEvents
- new API Utils.SetLogger allows to set ILogger
- updated Console samples to demonstrate ILogger support with Serilog
- support for EventSource logging
- see Docs/Logging.md for more info
- fix the complex types of the test nodeset (requires https://github.com/OPCFoundation/UA-ModelCompiler/pull/96)
- improve some client / server tests
- add tests for complex types datatype definition integrity on ref server, validate bsd and xml dictionaries in the test server (open issue: https://mantis.opcfoundation.org/view.php?id=7393, xml validation always failing)
- improve server fixture, to allow for custom config changes
- fix an issue that caused kestrel not start on windows/.NET Core 3.1 and above, when a new certificate was issued, thereby ignoring all https client tests.
This can cause considerable slowdown when connecting to multiple MQTT
destinations or when waiting for pubsub application to start, and is
unecessary.
Instead, the DisconnectHandler fires on connect fail as well, so this
implementation works.
-fixes for complex types:
- a structure which contains an enum from namespace 0 can not be created. fixes#1561.
- a structure with multidimensional array may lead to a endless loop in `ComplexTypeSystem.Load`
- a malformed namespace Uri which contains a space character prevents creation of type system.
- select endpoint doesn't pick the right scheme if the default endpoint must be picked
-prep for net472 or greater and net5.0 or greater build, only build #ifdef
- do not silent recreate a certificate if a matching cert subject is available, enforce manual deletion or replacement
- allow the application cert to be used when expired or not yet valid
- warn in trace if an app cert is loaded without loading the private key
fixes#1162 , fixes#1102
Refactored WriterGroupPublishState out of IPubSubConnection file
JSON Discovery messages for DataSetMetaData
UADP Discovery messages for DataSetMetaData (Request + response)
Refactored the UaPublisher, extracted IntervalRunner and reused it in UdpDiscoveryPublisher & UdpDiscoverySubscriber
Proposal: automatic setting of ConfigurationVersion based on the rules in 6.2.2.1.5 (part 14)in new class Opc.Ua.PubSub.Configuration.ConfigurationVersionUtils
New events added to UaPubSubApplication
Implemented support for UADP Data Key Frame DataSetMessages
Unit tests were updated to cover the new functionality
- DefaultEncodingId is always set to binary encoding id. Helper added to StructureDefinition for other use cases (e.g. PubSub)
- the default encoding id is derived for the encodeable factory, since the typetree does not contain sufficent information to find the proper binary encoding id
- add a few helper functions in core for an upcoming PR that was used for testing with custom types, will simplify to add more nodemanagers to servers with custom types
Pertaining to the discussion started in Issue #957, this Pull Request seeks to add capabilities for Subscription to sequentially invoke its MonitoredItem and FastDataChange callbacks to maintain order properly. The key goal is to avoid out-of-order monitored items. To this effect, the Sequential Publishing capability is added to Subscription
Resolves#957
### SequentialPublishing
A toggle which forces Subscription to raise callbacks only in a sequential order by the SequenceNumber of the incoming message. It limits the number of tasks that can process publish responses to 1, and enforces a strict +1 Sequence Number requirement before releasing a message.
#### Backwards compatibility
The new property has been added as DataMember at the end to maintain backwards compatibility with the DataContract, where such may be used. The relevant copy constructor has also been updated.
The feature is disabled by default, so the existing behavior is the default. Users must explicitly set the property on the Subscription object to enable these new features.
#### Leveraging existing "time machine" in SaveMessageInCache
There is already a mechanism for re-ordering messages that entered in the wrong order. This change makes use of that existing "time machine" which sorts messages into chronological order (by the sequence number) to only pull off messages with a proper +1 sequence number each time.
#### KeepAlive messages advancing sequence number
KeepAlive messages do not contain notifications, and do not enter the proper branch in the code to be pulled out. They will not interrupt sequential flow. Since the next message in the sequence with data will "re-use" that sequence number, it can be expected to maintain sequence.
#### Delayed messages
When sequential publishing is enabled, if a message is genuinely missing from the sequence, it will "hold up" the messages until it either arrives or is pulled out of the incoming messages list by being too old. In either case, it is considered "processed" for the sequence purposes at that point and the rest of the messages may proceed.
The automatic republish request mechanism is also leveraged here for that purpose as well.
#### SequenceNumber roll-over after max value
As specified in the OPC UA spec, at 1ms publishing, this would take almost 50 days of uptime. Naturally at any "reasonable" publishing interval this time is even longer. The matter itself is also not addressed throughout the whole existing code that handled it before this change. (Though some consideration can be seen in some places, such as placing messages at the end of the incoming message linked list if no better place was found)
#### Locking on m_cache to get the semaphore
I've considered making a separate locking object for the semaphore management, but decided it is not needed. The message worker would need to obtain m_cache lock later anyway, and momentarily obtaining it is not harmful for the overall flow.
Only in cases of serious contention between new PublishResponses coming in and workers attempting to work would this cause any problems.
#### Callbacks that "take a long time"
This change will naturally suffer from callbacks that take a long time to invoke. In our use case, each callback message is passed into an ActionBlock for handling, so our callbacks are fast. But Other clients may invoke much more code on these callbacks.
Sequential publishing means this capacity is possibly bottlenecked. Sequential publishing should be used for its intended purpose and with callbacks that do not hold up their calling thread for a long time. To ensure proper sequential callbacks, they cannot just be started in order, they have to be processed fully in order.
#### Changing this property while the Subscription is active
There is limited support to on-the-fly changes to this property, and this is not the intended use case. It is meant to be set once and ideally never changed while messages are being processed.
If users need certain items properly sequenced and some items they can accept out-of-order or ignore outdated messages, they can do this by defining two separate Subscription objects, same as they would if they needed different Publishing Intervals.
#### Why semaphore?
At first I had considered using ActionBlock from TPL DataFlow and limiting its concurrency to the number of desired message workers, but I had opted not to add a dependency on DataFlow only for this purpose. (Referencing all of DataFlow just for ActionBlock's automatic concurrency limiting is wasteful.
I had also considered using a Limited Concurrency Task Scheduler implementation and queueing the message workers on that instead of the normal ThreadPool via Task.Run, but that is similarly more complex than it needs to be.
#### Why async Task? What was wrong with Void?
While being on a new Task from the ThreadPool anyway, the worker would still occupy a task from the ThreadPool if it was using the synchronous Wait() method, which is a blocking call. By using WaitAsync, the ThreadPool task goes "back" to the ThreadPool until the semaphore is released.
### Disclaimer - I am not an OPC UA expert
Please do not assume I have "done all my homework" with regard to these changes. I have attempted to learn as much as I can in the time I was working on this change, but I fully expect to have missed a few spots and specifics of the OPC UA protocol.
I have done some testing in our application to verify the out-of-order problem is solved, and that no significant delay is introduced by the additional work done to facilitate this.
- Allow to call SetRSAPublicKey without Issuer being set
- Support to create self signed CA cert with supplied keys
- Test case for RSACng on .NET framework and .NET Core 3.1
Issue reported by @eoursel
fixes#1476
Improve IOP, some recent ModelCompiler versions (starting with errata 1.04.9) produce invalid output and all IsOptional fields have become true by default, causing the complex types client to bail out for Structures created with this ModelCompiler release. However, the IsOptional field can be ignored.
ModelCompiler is fixed in 1.04.10 and greater
fixes#919 and #921
- enable async client stubs in core
- add async versions for subscription management
- APM versions will not be included...
based on PR #919 from @mibos-kremer
Co-authored-by: Michiel <m.bos@kremer.nl>
Martin Regen