Files
romanett 0e40e12296 CI: CTT regression runs, weekly and on PRs labelled 'ctt' (#4760)
## Summary

Adds the **CTT regression** workflow. It runs the OPC UA Compliance Test
Tool against the reference server:

- every Sunday on master;
- manually;
- on pull requests a maintainer labels `ctt`.

The CTT is member-only software, so the run itself happens in the
private repository `OPCFoundation/UA-.NETStandard-CTT`. This workflow
does four things:

1. Starts the private run with `workflow_dispatch`, passing the PR head
ref and the commit that was labelled. A push after labelling is not run;
label again.
2. Waits for the run (about 1.5 hours) and downloads only its public
summary: result counts per part and the test cases that changed against
the baseline. CTT scripts and logs are never downloaded.
3. Posts the result as the `CTT` commit status, as a PR comment (updated
in place) and in the job summary, then removes the label.
4. Feeds a README badge for the weekly master run.

The results are a regression signal for this code base, not a
certification. The summary says so.

## Security

- **Secret access:** `pull_request_target` is used only to get the
access secret. The job never checks out or runs PR code; PR data is
passed through `env`.
- **PR code runs next to the CTT** on the private runner, so it could
read the CTT files. Only label a PR after reviewing it.
- **Access to the private repository:** a GitHub App installed on that
repository only, or a fine-grained token, with Actions read/write. A run
takes longer than an App installation token lives (one hour), so the job
mints a new installation token for each call.

## Setup needed after merge

- [ ] Create a GitHub App, or use an existing one. Permissions: Actions
read/write and Metadata read. Install it on
`OPCFoundation/UA-.NETStandard-CTT` only.
- [ ] Set variable `CTT_APP_ID` and secret `CTT_APP_PRIVATE_KEY` in this
repository. Alternatively, set secret `CTT_TOKEN` with a fine-grained
token that has the same permissions on the same repository.
- [ ] Create the `ctt` label.

## Test plan

- [x] Private workflow on GitHub-hosted `windows-latest`. Silent CTT
install, server build, generated test PKI and all 23 parts ran in about
83 minutes and matched the local baseline. Remaining differences:
Pass→Warning flips from response-timestamp delays, which led to raising
Time Tolerance to 250 ms, and one ResendData timing case.
- [x] Embedded bash scripts pass `bash -n`; App JWT signing checked
against a throwaway RSA key.
- [ ] End to end from this repository after merge, once the secret is
set: run it manually, then label a PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 11:43:53 +02:00
..