1 Commits
Author SHA1 Message Date
romanettandClaude Opus 5.5 6cfb8cf196 Server: remove per-request context copies and lock hot spots from the request path (−16 % allocation per Read) (#4767)
## Description

Five small changes to the per-request server path, found in the CPU and
allocation profiles of the o6 read benchmark (`AsyncCustomNodeManager`,
Server GC, 10 clients × 100 Reads in flight) on master 3a31473cf. Each
change is its own commit and can be reviewed and reverted on its own.
The channel gate (about 40 % of waits) is not touched; that is A1.

### Results

Paired A/B runs against master (0e40e1229), alternating builds per
round, 5 rounds, medians (min..max). The machine is shared with other
sessions: allocated bytes per request are deterministic; CPU per request
is best read from the runs pinned to 4 cores.

| Scenario | master | this PR | Change |
|---|---|---|---|
| o6read 10 × 100, server pinned to 4 cores: CPU / request | 39.1 µs
(35.9..39.8) | 36.9 µs (35.5..37.6) | **−5.6 %** |
| same: allocated / request | 6,198 B | 5,224 B | **−15.7 %** |
| same: req/s | 98.7k (95.4..108.5k) | 104.6k (97.6..105.9k) | +5.9 % |
| o6read 10 × 100, 24 cores: req/s | 183.7k (173.3..198.0k) | 189.8k
(178.9..197.5k) | +3.4 % |
| same: allocated / request | 6,500 B | 5,578 B | −14.2 % |
| o6read 10 × 100 Basic256Sha256, 24 cores: req/s | 159.8k
(149.4..178.7k) | 164.2k (156.7..165.3k) | +2.7 % |
| same: allocated / request | 6,590 B | 5,694 B | −13.6 % |
| `read --nodes 1`, 20 sessions: req/s | 155.3k (141.6..157.6k) | 163.8k
(163.4..166.5k) | +5.5 % |
| same: allocated / request | 5,548 B | 4,689 B | −15.5 % |
| `read --nodes 100`: allocated / request | 74.4 KB | 70.1 KB | −5.8 % |
| `browse`: allocated / request | 39.8 KB | 35.0 KB | −12.2 % |
| `browse`: lock contentions / 1000 requests | 474 | 367 | −23 % |

At 24 cores CPU per request stays within noise, because the time these
changes free now waits at the channel gate (see below).

Per item, pinned to 4 cores (5 rounds; CPU deltas of one item are within
run-to-run noise, allocation deltas are not):

| Item | CPU / request | Allocated / request |
|---|---|---|
| Q1 system context | +1.8 % | −1.7 % |
| Q2 request table | −4.3 % | −1.4 % |
| Q3 request timeouts | −2.8 % | −2.2 % |
| Q4 namespace metadata | −3.5 % | ±0 |
| Q5 read-path allocations | −2.4 % | −10.9 % |

Sampled thread time at 24 cores (cpu trace, first `Opc.Ua` frame above
the wait or copy):

| Frame | master | this PR |
|---|---|---|
| `ServerSystemContext.Copy` (incl. `MemberwiseClone`) | 12.4 % | 0.1 %
|
| `RequestManager` (`ConcurrentDictionary` inside
`EnterValidationScope`, scope `Dispose`, `RequestReceived`,
`CompleteRequest`) | 7.1 % | 0 |
| `RequestLifetime` (CTS registration spin,
ctor/`RegisterExternal`/`DisposeResources`) | 4.8 % | 0.2 % |
| `NamespaceMetadataRegistry.GetAsync` lock | 4.0 % | 0 |
| `RuntimeHelpers.InitializeArray` (`ReadValidationAttributes`) | 2.2 %
| 0 |
| Channel gate (`ChannelGate+AwaitEntryAsync`, inclusive) | 31 % | 46 %
|

## Changes

**Q1: one system context copy per request and node manager.**
`AsyncCustomNodeManager`'s service methods (Read, Write, Browse,
TranslateBrowsePaths, Call, HistoryRead/Update, monitored item and event
services) use the per-request copy from
`OperationContext.GetSystemContext` instead of copying the node
manager's context again. The per-request cache keeps the first node
manager's copy without an extra object and up to eight more, so a
request that touches several node managers no longer evicts the other
one's copy. `ServerSystemContext` copies itself with a copy constructor
instead of `MemberwiseClone` (which allocates through a runtime call)
unless it is a derived type; `SessionSystemContext` gains a protected
copy constructor. Copies per Read went from about three to one.

**Q2: lock-free request registration in `RequestManager`.** #4742 took
the server-wide lock off the request path, but the two
`ConcurrentDictionary`s for executing requests and open validation
scopes stayed the main waiting point after the channel gate (four writes
per request from every worker). `RequestTable<T>` places an entry with a
compare-and-swap into a free slot near a scrambled hash of its key, and
the entry remembers its slot so completion is one more compare-and-swap;
only when every probed slot is taken does an entry go to an overflow set
under a lock. Drains, Cancel, Session close, the timeout sweep and
`Dispose` read the slots one after the other, which keeps the existing
ordering arguments (registration fence vs. Session/Cancel marks, drain
count published before the snapshot). Sharding the dictionaries over
per-processor locks was tried first and only moved about 5 % of waits
into the shard locks.

**Q3: no timer or shared-token registrations per request.**
`RequestLifetime`'s timeout constructor, used by the endpoint for every
request, no longer creates a `Timer` and no longer registers with the
server's request-queue token and the transport token, which every worker
registered with and so contended on. The deadline and the two tokens are
watched by `RequestDeadlineMonitor`: one sweep every 20 ms over
intrusive lists sharded by processor, running only while requests are
monitored. The `params` constructor keeps its synchronous registrations.

**Q4: namespace metadata lookups without a lock.**
`NamespaceMetadataRegistry` reads its caches (found and missing, by URI
and now also by index) without its lock; only filling and invalidating
take it. The generation moves before the caches are cleared, so a lookup
racing an invalidation never reports a stale miss.

**Q5: fewer per-call allocations on the read path.**
`ReadValidationAttributes` fills its attribute-id array element by
element instead of with an array initializer
(`RuntimeHelpers.InitializeArray` per call); single-node Read and Write
requests skip the per-request attribute cache and validate with
`permissionsOnly` (the same three attributes, read once); `ReadAsync`
allocates the list of nodes to validate only when needed. (Review: a
shared static attribute array, a per-thread `Variant[]` scratch array
and a per-thread Read node cache were dropped again because they crossed
virtual calls; the measurements below were taken with them, so Q5's
allocation saving is a little smaller than shown.)

## Behaviour notes

- **Request timeouts (Q3):** a `TimeoutHint` now fires up to 20 ms late
(the sweep period), and cancelling the server's or the transport's token
reaches the request's `CancellationToken` up to 20 ms later instead of
synchronously. A token that is already cancelled cancels the request at
once, and `TryCancel` checks the tokens first, so a token cancelled
before an explicit Cancel still owns the terminal state. The
exactly-once terminal state of #4744 is unchanged: the sweep claims it
through the same compare-and-swap.
- **System contexts (Q1):** a custom node manager that changed fields of
the context it got in a service call now changes the copy shared by that
request's other calls into the same node manager; it never reaches other
requests or the node manager's own `SystemContext`.
- **Request ids (Q2):** `RequestManager.RequestReceived` no longer
throws for a different `OperationContext` with a request id that is
still registered. Requests are tracked by their context; ids come from a
server-wide counter.

## Testing

- New: `ServerSystemContextCopyTests` (every field copied, derived types
kept), `AsyncCustomNodeManagerTests` (one copy per node manager and
request, two managers in one request, not shared across requests;
single- vs multi-node Read verdicts identical incl. a node read twice),
`RequestTableTests` (idempotent add, exactly-once remove, overflow,
concurrent churn while enumerating; drain racing completion on many
threads; Session close and Cancel by handle exactly once while others
register), `RequestDeadlineMonitorTests` (timeout precision with a fake
clock, token propagation, ownership vs. explicit Cancel, no leaked
entries, sweep/cancel/complete race), `NamespaceMetadataRegistryTests`
(cached miss by index invalidated, lookups racing `Invalidate`,
permission changes visible through the cached node).
- Run locally (net10.0): Opc.Ua.Types.Tests, Opc.Ua.Core.Tests (Stack),
Opc.Ua.Server.Tests, Opc.Ua.Sessions.Tests, Opc.Ua.Subscriptions.Tests,
Opc.Ua.Redundancy.Server.Tests, Opc.Ua.Stress.Tests Contract +
Integration. net48 builds.
- Measured with `perf/ServerLoadHarness` scenarios `o6read` (o6 harness,
not part of this PR), `read`, `browse`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 06:25:31 +02:00