Files
UA-.NETStandard/.github/scripts/update-test-durations.ps1
romanett fbaabf5b28 Cut CI runner minutes: duration-packed test batches, CodeQL on Linux/daily, cancellable branch packing (#4694)
Reduces GitHub Actions runner minutes without removing any OS, target
framework, test project or check. All figures come from master CI run
37520359929 (158 jobs, ~1,880 runner-minutes) and the last two weeks of
run history.

## 1. Test batches packed by duration (`get-ci-matrix.ps1`)

Previously the expander picked the **smallest** batch size that fit the
256-job limit, which meant 2 projects per job and 33 jobs per profile.
Job logs show ~45% of the test matrix (674 of ~1,516 min) went to
compiling. Each batch rebuilds the shared dependency graph: the first
project's build took 3.8–4.9 min, the second only 0.4–0.9 min.

- Projects are now packed in path order (which keeps `PubSub.*`,
`Redundancy.*`, `WotCon.*` together) up to a 30-minute weight target,
using `.github/ci-test-durations.json`. Each weight is the slowest PR
profile's test minutes, rounded up, plus 1.
- A batch never holds more projects than the job timeout can budget for:
at most 7, given 20 + n×45 ≤ 350.
- If the matrix would exceed the job limit, the target is raised
instead.
- **PR test matrix: 132 → 60 jobs** (15 per profile; macOS 33 → 15).
Each batch is estimated at ≤36 min on the slowest runner, comparable to
today's slowest jobs (37–43 min), so wall-clock should be about the
same. Expected saving is roughly 300+ runner-min per run, plus less
queueing for the scarce macOS runners.
- The weekly full scope shrinks the same way (still 13 profiles).
- The table only steers balance: a project missing from it gets
`defaultMinutes`, so it is still scheduled.
- `run-dotnet-tests.ps1` now records per-project
`buildSeconds`/`testSeconds` (batch summary and job summary). The new
`update-test-durations.ps1` refreshes the table from any run's
`dotnet-results-*` artifacts.
- `discover` output `batch_size` is renamed to `batch_target_minutes`.
Nothing consumed it.

New `CiMatrixScriptTests` cases:
- each profile runs each of its projects exactly once;
- only a lone project may exceed the target;
- PR profiles average ≥3 projects per job;
- the duration table names only existing projects.

## 2 + 3. CodeQL (`codeql-analysis.yml`)

- **No longer runs on every push to master.** Instead, a daily schedule
on master (was weekly) gives an analysis that always completes. Each
analysis takes ~3 h, and with `cancel-in-progress: true` per branch, 49
of 76 push runs in two weeks were cancelled by the next merge before
uploading anything. `master378`/`release/*`/`develop/*` keep their push
trigger. PRs are still analyzed as before.
- Only PR runs are cancelled when superseded. Push and scheduled runs
always finish.
- **Runs on `ubuntu-latest` instead of `windows-2025`.** Only net10.0 is
built, so the extracted sources are identical. The traced Windows build
alone took 70 min, against 12 min for the untraced Linux `UA.slnx`
net10.0 leg. This PR touches a `.cs` file, so its own CodeQL run is the
first measurement.

## 4. NuGet packing on feature branches (`nuget-publish.yml`)

298 of ~400 runs in two weeks were feature-branch pushes, each about 59
Windows-min across two signed builds.

- A superseded feature-branch run is now cancelled. Protected lines and
tags are never cancelled: they publish previews, and `release.yml`
promotes a run by ID. Each manual (`workflow_dispatch`) run gets its own
concurrency group, so a later push cannot cancel it.
- A new `changes` job skips packing when a feature branch, as a whole,
changes only documentation. It diffs against the merge base with master,
not the previous push, so a docs-only push that cancels a code push
cannot mask that code change. It uses the same `get-path-relevance.ps1`
rule as CI.
- Protected refs, tags, manual runs, and any branch whose merge base
cannot be resolved always pack. Checked locally for docs-only branch
(skip), code branch, protected ref, manual run and unknown commit (all
pack).

## Verification

- `CiMatrixScriptTests`, `CiPathRelevanceTests` and the new
`CiTestDurationsScriptTests` (covering `update-test-durations.ps1`): all
`*.Ci*` tests in `Opc.Ua.Tools.Tests` pass locally, 63/63.
- This PR's own CI run shows the new matrix. Compare its total
runner-minutes and longest test job against run 37520359929.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:00:19 +02:00

96 lines
3.5 KiB
PowerShell

<#
.SYNOPSIS
Refreshes .github/ci-test-durations.json from the results of a CI run.
.DESCRIPTION
get-ci-matrix.ps1 packs test projects into batches by the weights in
.github/ci-test-durations.json. The weights only steer balance - a stale or
missing weight never changes what runs - but batches drift out of balance as
suites grow, so refresh them from a recent run now and then:
gh run download <run-id> --pattern 'dotnet-results-*' --dir ./ci-results
./.github/scripts/update-test-durations.ps1 -ResultsPath ./ci-results -Source 'Run <run-id>'
Each project's weight is its slowest profile's test minutes, rounded up,
plus one minute for its incremental build in a batch that already built the
shared dependencies. The cold build of those dependencies is a fixed cost of
the batch, not of any project, so first-in-batch build time is ignored.
Projects the run did not execute keep their current weight.
.PARAMETER ResultsPath
Directory holding the downloaded 'dotnet-results-*' artifacts. Every
batch-summary.json below it is read.
.PARAMETER Source
Short description of the run the weights came from, recorded in the table.
.PARAMETER DurationsPath
The table to update. Defaults to .github/ci-test-durations.json.
#>
Param(
[Parameter(Mandatory = $true)]
[string] $ResultsPath,
[Parameter(Mandatory = $true)]
[string] $Source,
[string] $DurationsPath = ''
)
$ErrorActionPreference = 'Stop'
if ([string]::IsNullOrWhiteSpace($DurationsPath)) {
$DurationsPath = Join-Path $PSScriptRoot '..' 'ci-test-durations.json'
}
$table = Get-Content -LiteralPath $DurationsPath -Raw | ConvertFrom-Json -AsHashtable
$weights = @{}
foreach ($entry in $table.projects.GetEnumerator()) {
$weights[$entry.Key] = [int]$entry.Value
}
$summaries = @(Get-ChildItem -LiteralPath $ResultsPath -Recurse -File -Filter 'batch-summary.json')
if ($summaries.Count -eq 0) {
throw "No batch-summary.json below '$ResultsPath'. Download the run's 'dotnet-results-*' artifacts first."
}
$slowest = @{}
foreach ($summary in $summaries) {
$document = Get-Content -LiteralPath $summary.FullName -Raw | ConvertFrom-Json
foreach ($record in $document.projects) {
# A failed or timed-out project's duration says nothing about a healthy
# run, and a not-applicable one did not run at all.
if ($record.outcome -ne 'passed' -or $null -eq $record.testSeconds) {
continue
}
$stem = [System.IO.Path]::GetFileNameWithoutExtension($record.project)
if (-not $slowest.ContainsKey($stem) -or $record.testSeconds -gt $slowest[$stem]) {
$slowest[$stem] = [int]$record.testSeconds
}
}
}
if ($slowest.Count -eq 0) {
throw "The summaries below '$ResultsPath' record no passed project with a duration."
}
foreach ($entry in $slowest.GetEnumerator()) {
$weights[$entry.Key] = [int][System.Math]::Ceiling($entry.Value / 60.0) + 1
}
$keys = [System.Collections.Generic.List[string]]::new()
foreach ($key in $weights.Keys) {
$keys.Add($key)
}
$keys.Sort([System.StringComparer]::Ordinal)
$projects = [ordered]@{}
foreach ($key in $keys) {
$projects[$key] = $weights[$key]
}
[ordered]@{
source = $Source
defaultMinutes = [int]$table.defaultMinutes
projects = $projects
} | ConvertTo-Json -Depth 4 | Set-Content -LiteralPath $DurationsPath -Encoding utf8NoBOM
Write-Host "Updated $($slowest.Count) of $($projects.Count) weights in '$DurationsPath' from $($summaries.Count) batch summaries."