# Description Add configurable server resource isolation on top of the merged #4522 foundation. - Coordinate connection, handshake, retained-message, session-establishment, queue, execution, request-cost and parked-request limits through one server-scoped provider. - Keep configured startup, continuity/reconnect, control and trusted-owner reservations inside existing totals. Shared capacity remains usable up to explicit owner ceilings. - Use authoritative Session membership for reassembly continuity capacity without granting unrelated resource or rate privileges. Preserve explicit owner mappings and bounded tracking. - Cache immutable classification evidence while retaining current Session and classifier checks. - Support weighted request ordering or FIFO independently of resource accounting. Installed providers enforce decoded-request limits in both modes. - Preserve fail-fast reassembly, valid Abort handling, fixed deadlines, active-work protection, reverse-connect handoff and bounded OpenAPI processing. - Support custom-handler parking through `IRequestParkingPolicy`, `WithRequestParking(...)`, and `RequestLifetime.ParkSink`. Parking releases execution capacity only, keeping retained-data and parked charges until completion. Built-in Publish and the global opt-out remain unchanged. - Provide direct, fluent and DI configuration, public incoming-request parking support, effective startup limit logs and user-facing documentation. Balanced remains the default. SharedOnly without an explicit provider retains shared behavior for intentionally incompatible/unlimited configurations. Reservations are finite and are not one full-message allowance per Session. ## Related Issues Builds on merged #4522. Fixes #4554. ## Review and CI corrections All three previously missed findings in [review 5326734718](https://github.com/OPCFoundation/UA-.NETStandard/pull/4541#pullrequestreview-5326734718) are addressed in `9e25e4ecc`: 1. A custom FIFO provider receives RequestQueue, RequestQueueBytes, RequestExecution and ParkedRequest accounting. A blocked head cannot be overtaken. 2. `IResourceIsolationReassemblyProvider` replaces the sessionless threshold independently of `UseFairScheduling`. The aggregate byte ceiling remains enforced. 3. Runtime `ReservedReassemblyBytes` includes all configured protected stage floors, including ControlReserved. The review's two already-resolved changes remain intact. The CI collection race was fixed at its source in `e1a994a4b`: `ArrayOf.ToArrayOf` treats a separately read collection count as an initial capacity hint and grows/trims for the actual enumeration. Deterministic tests cover growth, shrinkage and stale-zero counts. The subsequent recovery-test race was fixed in `05753ce77`: fake time advances only after a gated keepalive response has been processed and reset its timer, not merely after request arrival. Recovery-order and timeout assertions remain intact, with additional response-state assertions. Instrumentation reproduced the original failure twice; both fixed variants passed 400 executions per framework. CodeQL alert 388 was verified and dismissed as a false positive. Startup logs contain aggregate numeric capacities, not credentials or caller identities. ## Validation | Scope | Result | | --- | --- | | Latest automated checks at `05753ce7775a58801b4ba9d8f6d7781b709d0b89` | 167 successful, one package-publication check skipped, no failures | | Remaining external prerequisite | `license/cla` pending: Contributor License Agreement is not signed yet | | FIFO/weighted queues, custom parking and server-channel selection | 175 passed on each net10.0/net48 | | Isolation, configuration and parking-hosting selection | 259 passed net10.0; 257 passed net48 | | ArrayOf conversion tests | 73 passed on each net10.0/net48 | | Recovery-audit and keepalive fixtures | 40 passed on each net10.0/net48 | | NativeAOT smoke from preceding updates | Published and executed successfully | No tests, assertions, coverage thresholds, or required checks were weakened. The remaining CLA prerequisite must be completed through [CLA Assistant](https://cla-assistant.io/OPCFoundation/UA-.NETStandard?pullRequest=4541). This is not CTT certification or an unconditional network-DDoS, whole-process-memory or fleet-wide guarantee. ## Checklist - [ ] I have signed the [CLA](https://opcfoundation.org/license/cla/ContributorLicenseAgreementv1.0.pdf) and read the [CONTRIBUTING](https://github.com/OPCFoundation/UA-.NETStandard/blob/master/CONTRIBUTING.md) doc. - [ ] I have added tests that prove my fix is effective or that my feature works and increased code coverage. - [x] I have added all necessary documentation. - [x] I have verified that my changes do not introduce (new) build or analyzer warnings. - [ ] I ran **all** tests locally using the **UA.slnx** solution against at least .net **framework** and .net **10**, and all passed. - [x] I fixed **all** failing and flaky tests in the CI pipelines and **all** CodeQL warnings. - [x] I have addressed **all** PR feedback received. --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: marcschier <11168470+marcschier@users.noreply.github.com>
16 KiB
Rate limiting and admission control
Admission control means checking capacity before accepting more work. The server
applies configurable limits to reject excess connections or Session-establishment
requests early, reducing the work caused by a connection storm (see
Server Session Scalability for the underlying analysis).
Clients respond to busy signals by backing off rather than retrying blindly.
The standard rate-limiting primitives use System.Threading.RateLimiting;
the provider can be replaced through dependency injection.
Rate limiting is on by default with conservative limits. Tune them for the deployment: even legitimate clients can be refused during a sufficiently large burst. Disabling rate limiting does not disable resource isolation or the incomplete-message budget.
Server side
What is limited
- Inbound connections (
opc.tcplistener): a per-second token bucket (with a burst) admits new connections; beyond the burst a connection is shed cheaply so the CPU-bound secure-channel handshake is protected. The listener socket backlog is configurable and defaults to 512 (raised from the previous hard-coded 10) so a burst of simultaneous connects is absorbed rather than dropped by the OS. - Session establishment (
CreateSession/ActivateSession): a concurrency limiter bounds the number of in-flight establishment operations so a connect storm cannot saturate every core and starve steady-state publish delivery. When at capacity the server returnsBadServerTooBusywith a retry-after hint in the fault, before doing the expensive certificate validation / signing. - Incomplete messages: one
ChunkReassemblyBudgetbounds retained intermediate-message buffers across the server's UA-TCP, Kestrel TCP, and UA Secure Conversation WebSocket listeners. This capacity limit is independent ofServerRateLimitOptions.Enabled.
Status codes
| Situation | Status code |
|---|---|
| Session establishment at capacity | BadServerTooBusy (transient — the client should back off and retry) |
Hard session cap reached (MaxSessionCount) |
BadTooManySessions |
| Connection shed at the listener | the connection is dropped; the client sees a transport error and, together with any subsequent BadServerTooBusy, backs off |
| Incomplete-message budget exhausted | BadTcpNotEnoughResources followed by channel closure; if the error cannot be sent, the client observes closure |
| Resource isolation refuses common HTTPS body processing | HTTP 503 (Service Unavailable), distinct from a rate limiter's HTTP 429 |
Configuration
The deterministic limits live in ServerRateLimitOptions:
| Option | Default | Meaning |
|---|---|---|
Enabled |
true |
Switch for this provider's connection and Session-establishment rate limits, not resource isolation or message budgets. |
ListenBacklog |
512 | Listener socket pending-connection backlog. |
ConnectionRateLimitEnabled |
true |
Whether inbound connections are rate limited. |
ConnectionsPerSecond |
500 | Sustained connection admission rate. |
ConnectionBurst |
1000 | Connection burst capacity (token-bucket size). |
SessionRateLimitEnabled |
true |
Whether session establishment is limited. |
MaxConcurrentSessionEstablishment |
max(ProcessorCount * 8, 128) |
Concurrent in-flight CreateSession/ActivateSession. |
SessionEstablishmentQueueLimit |
0 | Waiters before rejecting with BadServerTooBusy (0 = reject immediately). |
Configure it through the DI hosting options:
services.AddOpcUa()
.AddServer(options =>
{
options.ApplicationName = "MyServer";
options.ConfigureRateLimits = limits =>
{
limits.ConnectionsPerSecond = 200;
limits.MaxConcurrentSessionEstablishment = 64;
};
});
To plug in a completely different algorithm, register a custom IServerRateLimiterProvider in DI (it takes precedence over ConfigureRateLimits):
services.AddSingleton<IServerRateLimiterProvider>(sp =>
new MyCustomRateLimiterProvider(...));
Without dependency injection, set the options or provider directly on the server before it starts:
var server = new StandardServer(telemetry);
server.RateLimitOptions = new ServerRateLimitOptions { ConnectionsPerSecond = 200 };
// or: server.RateLimiterProvider = new DefaultServerRateLimiterProvider(options);
Raw TCP, Kestrel TCP, and UACP WebSocket listeners consume the
IConnectionRateLimiter supplied through TransportListenerSettings, injected
by StandardServer.ConfigureTransportListenerSettings. A custom server can
override that hook to supply its own limiter. The ordinary shared
TokenBucketConnectionRateLimiter wraps a
System.Threading.RateLimiting.TokenBucketRateLimiter. With the default
isolation provider in Balanced or TrustedReservations, the default rate provider
instead uses ResourceIsolationConnectionRateLimiter to divide the same totals
between protected and shared traffic, as described below.
Kestrel and UACP WebSocket listeners also enforce their configured channel cap
across concurrent admissions and release capacity with the physical connection,
including reverse handoff. The raw listener retains its existing admission and
unused-channel reclamation behavior.
Incomplete messages
ChunkReassemblyBudget (Opc.Ua.Bindings) counts the backing-array length of
each retained intermediate chunk, including pool rounding and metadata. All
listeners of a server share the same budget. Completing, replacing, aborting,
faulting, or closing a partial message releases its reservation.
The budget applies only while messages await further chunks, not to the whole process heap, socket buffers, final chunks, decoded requests, or outgoing responses. A request that fits in one chunk can still be served while the reassembly budget is full. A chunk that exceeds available capacity is rejected without waiting; the partial message is discarded and its channel is closed.
The budget's default is sixteen times TransportQuotas.MaxMessageSize, clamped to
64 MiB through 1 GiB, then raised if necessary to accommodate four maximum-sized
messages. A configuration with unlimited message size receives a 1 GiB budget.
Both the stack's 2 MiB and the reference server's 4 MiB message limits select
64 MiB. ChunkReassemblyBudget.GetDefaultMaxBytes(maxMessageSize) returns
this value, and ChunkReassemblyBudget.CreateDefault(configuration) creates a
budget from an EndpointConfiguration using the same sizing policy. The budget
can be used with unlimited message sizes in SharedOnly; the default Balanced
isolation provider separately requires finite message and buffer limits.
In SharedOnly, channels without an activated session may reserve only while
total usage stays at or below MaxBytesWithoutSession, which defaults to half of MaxBytes.
Channels with activated sessions can use the remaining headroom. This is a
capacity policy, not an authentication boundary: an activated anonymous session
also qualifies. The two-argument constructor sets the sessionless threshold
explicitly, including the entire budget for sessionless workloads.
The threshold is not reserved space for new clients. For example, with a 100 MiB total and 50 MiB sessionless threshold, 60 MiB retained by activated sessions prevents further sessionless intermediate chunks. Single-chunk requests are still processed, but a large multi-chunk OpenSecureChannel, CreateSession, or ActivateSession may be refused until occupancy drops. The shared budget alone provides neither request scheduling by caller nor reserved startup capacity.
Server resource isolation limits usage by caller and
can keep capacity reserved for verified startup/recovery work inside the same
totals. Direct and hosted StandardServer instances use Balanced by default.
With the default isolation provider active, its byte-class reservations replace
the sessionless occupancy rule above; the shared budget still enforces
MaxBytes. With a 4 MiB message limit and 65,536-byte buffer limit, Balanced
reserves 16.25 MiB each for startup and reconnect from a 64 MiB budget, leaving
31.5 MiB shared. These are conservative retained-array bounds, not payload sizes.
A reserve does not make an unknown caller eligible to use it. A deployment
needs trusted ingress classification for protected access before OPC UA
authentication. Implement IResourceIsolationClassifier using a network
entry path whose access is controlled by your deployment, as shown in the
dedicated trusted-ingress example.
An IP address or previous successful connection is not proof of identity.
Unknown callers can be refused while a reserve is unused because that memory
is available only to its eligible callers, not to every caller.
SharedOnly retains the shared occupancy rule without separate startup reserves.
These policies prevent a busy caller population from taking resources set aside for eligible recovery or control traffic. They cannot guarantee access to every unknown client during a distributed flood. They also do not limit unrelated application allocations. See the limits of this protection when sizing a deployment.
Balanced startup rejects insufficient capacities rather than silently removing
reserves or increasing totals. See the
startup configuration choices and
plan inspection example
for the actual fields and checks. Isolation remains independent of
ServerRateLimitOptions.Enabled.
When the default connection rate limiter is enabled alongside Balanced or
TrustedReservations, it also reserves one burst token and one token per second
for startup, reconnect, and each provisioned trusted owner, inside the existing
ConnectionBurst and ConnectionsPerSecond totals. At least one shared token
must remain in each total. Startup rejects totals that cannot fit. Protected
traffic can also use available shared tokens, but ordinary traffic cannot use
the reserved tokens. Closing a connection does not refund a token. FairShare
and SharedOnly use the ordinary shared token bucket. If you supply your own
rate-limiter provider, the server calls that provider instead. It may reject
a connection even when the isolation policy has reserved space for it, so
configure both policies consistently.
Configure the hosted server through its fluent builder:
services.AddOpcUa()
.AddServer(options => options.ApplicationName = "MyServer")
.WithChunkReassemblyBudget(256L * 1024 * 1024);
For direct construction, assign the budget before startup:
var server = new StandardServer(telemetry)
{
ChunkReassemblyBudget = new ChunkReassemblyBudget(256L * 1024 * 1024)
};
The same instance can be shared by several servers. A host that opens listeners
itself passes it through TransportListenerSettings.ChunkReassemblyBudget;
otherwise each standalone listener creates its own appropriately sized budget.
For Dependency Injection (DI), the two-argument builder overload sets the
sessionless threshold and validates both limits immediately:
services.AddOpcUa()
.AddServer(options => options.ApplicationName = "MyServer")
.WithChunkReassemblyBudget(256L * 1024 * 1024, 64L * 1024 * 1024);
A directly constructed ChunkReassemblyBudget(maxBytes, maxBytesWithoutSession)
can also be registered as a singleton when several servers should share it.
Server channels also enforce a fixed assembly deadline using ChannelLifetime.
Continuation traffic cannot restart it. A shared quota/clock timer schedules
asynchronous cleanup independently of listener inactivity sweeps. Non-positive
lifetimes use the 30-second default for assembly, and expiration reports
BadTimeout before closure when possible; see
incomplete-message resource limits.
HTTPS / Kestrel transport
The HTTPS/WSS transport is Kestrel-hosted and can use the ASP.NET Core rate limiter middleware. Because that listener builds its own isolated web host, attach the limiter through the stack's DI with AddHttpsRateLimiter (net8+):
services.AddOpcUa()
.AddHttpsTransport()
.AddHttpsRateLimiter(); // default: global 100 req/s fixed window, rejects with HTTP 429
Pass an Action<RateLimiterOptions> to fully configure the limiter with any System.Threading.RateLimiting policy. This registers a contributor that calls services.AddRateLimiter(...) and app.UseRateLimiter() on the listener's isolated Kestrel host; rejected requests return HTTP 429.
Client side
A client that gets a "server busy" signal must not hammer the server with retries — that is exactly what amplifies a connect storm. The default ReconnectPolicy (used by ManagedSession) is server-signal-aware: when the previous attempt failed with an overload signal (BadServerTooBusy, BadTcpServerTooBusy, BadTooManySessions, BadTooManyOperations, BadTooManyPublishRequests, or a transient timeout) it backs off more aggressively (4× the computed delay, capped at MaxDelay) and honors a server-provided retry-after hint as a lower bound.
This is exposed through IReconnectPolicy.TryGetNextDelay, which adapts the backoff to the previous attempt's status code and any server-provided retry-after hint. The connection state machine calls it automatically; a policy that returns false opts out of adaptive behavior and the state machine falls back to the basic attempt-based GetNextDelay, so a minimal custom policy only has to implement the plain delay. Because a failed initial connect funnels into the same reconnect loop, the adaptive backoff applies to both initial connects and reconnects.
ReconnectPolicy.IsServerBusySignal(StatusCode) classifies whether a status code is an overload signal, for use in custom policies.
To keep a bulk connect from bursting, a client-wide connect admission gate ramps concurrent initial connects: ManagedSessionBuilder.WithConnectRateLimiter(maxConcurrency) (or a shared RateLimiter / IClientConnectGate) bounds how many ManagedSession.CreateAsync calls establish at once; the excess wait in an unbounded queue rather than being rejected, so many sessions to one server ramp up smoothly instead of stampeding. It is off by default; through DI use the ConnectRateLimiterMaxConcurrency client option.
Server backpressure signals
- Structured retry-after: the server carries a machine-readable retry-after to the client independently of diagnostics — as a
RetryAfterMsvalue inResponseHeader.additionalHeaderon aBadServerTooBusyServiceFault, and as the standard HTTPRetry-Afterheader on the HTTPS 429. The client honors both viaIReconnectPolicy.TryGetNextDelay. The UA-TCP client also honors aRetryAfterMs=Ntoken in a transient server-busyErrormessage reason as a lower bound on channel-reconnect backoff. The legacyRetryAfterMs=NAdditionalInfotoken remains a best-effort compatibility hint. See Server retry-after backpressure for how the client honors these signals during reconnect. - Load-based
Server.ServiceLevel: the reference server computesServer.ServiceLevelfrom session-establishment headroom (255 at low load, scaling toward a floor as sessions approachMaxSessionCount, with hysteresis), so a client can read/subscribe to it as a proactive capacity signal.
See also
- Server Session Scalability — the analysis motivating this work.
- Performance Benchmarks — the session-scalability load test.
- Sessions, Reconnection, and Subscription Engines —
ManagedSessionand reconnect architecture.