Files
UA-.NETStandard/.github/scripts/get-ci-matrix.ps1
romanett fbaabf5b28 Cut CI runner minutes: duration-packed test batches, CodeQL on Linux/daily, cancellable branch packing (#4694)
Reduces GitHub Actions runner minutes without removing any OS, target
framework, test project or check. All figures come from master CI run
37520359929 (158 jobs, ~1,880 runner-minutes) and the last two weeks of
run history.

## 1. Test batches packed by duration (`get-ci-matrix.ps1`)

Previously the expander picked the **smallest** batch size that fit the
256-job limit, which meant 2 projects per job and 33 jobs per profile.
Job logs show ~45% of the test matrix (674 of ~1,516 min) went to
compiling. Each batch rebuilds the shared dependency graph: the first
project's build took 3.8–4.9 min, the second only 0.4–0.9 min.

- Projects are now packed in path order (which keeps `PubSub.*`,
`Redundancy.*`, `WotCon.*` together) up to a 30-minute weight target,
using `.github/ci-test-durations.json`. Each weight is the slowest PR
profile's test minutes, rounded up, plus 1.
- A batch never holds more projects than the job timeout can budget for:
at most 7, given 20 + n×45 ≤ 350.
- If the matrix would exceed the job limit, the target is raised
instead.
- **PR test matrix: 132 → 60 jobs** (15 per profile; macOS 33 → 15).
Each batch is estimated at ≤36 min on the slowest runner, comparable to
today's slowest jobs (37–43 min), so wall-clock should be about the
same. Expected saving is roughly 300+ runner-min per run, plus less
queueing for the scarce macOS runners.
- The weekly full scope shrinks the same way (still 13 profiles).
- The table only steers balance: a project missing from it gets
`defaultMinutes`, so it is still scheduled.
- `run-dotnet-tests.ps1` now records per-project
`buildSeconds`/`testSeconds` (batch summary and job summary). The new
`update-test-durations.ps1` refreshes the table from any run's
`dotnet-results-*` artifacts.
- `discover` output `batch_size` is renamed to `batch_target_minutes`.
Nothing consumed it.

New `CiMatrixScriptTests` cases:
- each profile runs each of its projects exactly once;
- only a lone project may exceed the target;
- PR profiles average ≥3 projects per job;
- the duration table names only existing projects.

## 2 + 3. CodeQL (`codeql-analysis.yml`)

- **No longer runs on every push to master.** Instead, a daily schedule
on master (was weekly) gives an analysis that always completes. Each
analysis takes ~3 h, and with `cancel-in-progress: true` per branch, 49
of 76 push runs in two weeks were cancelled by the next merge before
uploading anything. `master378`/`release/*`/`develop/*` keep their push
trigger. PRs are still analyzed as before.
- Only PR runs are cancelled when superseded. Push and scheduled runs
always finish.
- **Runs on `ubuntu-latest` instead of `windows-2025`.** Only net10.0 is
built, so the extracted sources are identical. The traced Windows build
alone took 70 min, against 12 min for the untraced Linux `UA.slnx`
net10.0 leg. This PR touches a `.cs` file, so its own CodeQL run is the
first measurement.

## 4. NuGet packing on feature branches (`nuget-publish.yml`)

298 of ~400 runs in two weeks were feature-branch pushes, each about 59
Windows-min across two signed builds.

- A superseded feature-branch run is now cancelled. Protected lines and
tags are never cancelled: they publish previews, and `release.yml`
promotes a run by ID. Each manual (`workflow_dispatch`) run gets its own
concurrency group, so a later push cannot cancel it.
- A new `changes` job skips packing when a feature branch, as a whole,
changes only documentation. It diffs against the merge base with master,
not the previous push, so a docs-only push that cancels a code push
cannot mask that code change. It uses the same `get-path-relevance.ps1`
rule as CI.
- Protected refs, tags, manual runs, and any branch whose merge base
cannot be resolved always pack. Checked locally for docs-only branch
(skip), code branch, protected ref, manual run and unknown commit (all
pack).

## Verification

- `CiMatrixScriptTests`, `CiPathRelevanceTests` and the new
`CiTestDurationsScriptTests` (covering `update-test-durations.ps1`): all
`*.Ci*` tests in `Opc.Ua.Tools.Tests` pass locally, 63/63.
- This PR's own CI run shows the new matrix. Compare its total
runner-minutes and longest test job against run 37520359929.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 10:00:19 +02:00

618 lines
28 KiB
PowerShell

<#
.SYNOPSIS
Expands the GitHub Actions build and test workload for a CI scope.
.DESCRIPTION
Produces the job matrices consumed by .github/workflows/buildandtest.yml
(pull-request scope) and .github/workflows/nightly.yml (full scope), plus a
manifest that records every (project x profile) tuple the run is expected to
cover.
The workload is described once, here, as a table of profiles. A profile is a
(runner OS, CustomTestTarget, test-host target framework, configuration,
category filter, tier) tuple. Projects are discovered from the file system
with a '*.Tests.csproj' sweep, so neither matrix has to be hand-maintained
as projects are added or renamed.
CustomTestTarget, not '--framework', is the mechanism that pins the stack to
a single target framework (see targets.props). A profile carries both the
CustomTestTarget it builds with and the framework its tests actually run on.
The project fan-out is batched: each matrix entry carries several projects
that the executor runs in sequence, keeping per-project results. Batches are
sized by measured duration rather than by project count. Every batch pays a
fixed cost of several minutes - checkout, SDK install, and the cold build of
the shared dependency graph - while a further project in the same batch only
adds its incremental build and its tests. Two projects per job spent almost
half of the pull-request matrix compiling the same assemblies over and over,
so projects are packed, in path order, into batches of about
BatchTargetMinutes each. Path order keeps related projects (PubSub.*,
Redundancy.*, WotCon.*) together, so their shared dependencies are built
once per batch.
GitHub Actions also refuses to start a workflow run whose matrices expand
past 256 jobs, and that limit cannot be raised. If the packed matrix does not
fit, the target is raised until it does, so the matrix adapts as test
projects are added instead of silently truncating.
.PARAMETER Scope
'pr' expands the workload that gates a pull request. 'full' expands the
complete workload Azure Pipelines used to run on a schedule.
.PARAMETER ExcludeMacOS
Drop the macOS test profile. The batch target is deliberately computed from
the full profile table regardless of this switch, so excluding macOS does not
reshuffle the batches of the remaining profiles.
.PARAMETER OnlyProject
Restrict the test matrix to the single test project whose path ends with this
value. Used by the protected private-fuzz-corpus job so it covers exactly the
profiles this table already enables for that project, instead of repeating
the profile list in a second place. The non-mainline tiers are dropped, since
they cover named projects of their own.
.PARAMETER MaxEntries
The documented GitHub Actions ceiling on matrix jobs per workflow run.
Exceeding it is an error, never a truncation.
.PARAMETER ReservedEntries
Entries reserved for the jobs that are not part of these matrices - Native
AoT, the fuzz contract and replay jobs, coverage and the summary - plus
headroom.
.PARAMETER MaxJobTimeoutMinutes
Upper bound on a batch job's 'timeout-minutes', below the 360-minute ceiling
GitHub enforces on hosted runners. A batch whose per-project budgets add up
past this is an error rather than a clamp: clamping would let the executor
outlive the job, which is the one case that produces no annotation and no
results. The packer therefore never puts more projects in one batch than
this ceiling can budget for.
.PARAMETER BatchTargetMinutes
The measured duration, in minutes, a batch is packed up to, on top of its
fixed cost. Weights come from DurationsPath and are the slowest pull-request
profile's figures, so batches on the faster runners finish sooner. A project
heavier than the target gets a batch of its own. Raised in steps of five
when the packed matrix would not fit MaxEntries.
.PARAMETER DurationsPath
JSON table of per-project weights in minutes. Defaults to
.github/ci-test-durations.json. A project missing from it is weighted with
the table's defaultMinutes, so a new project is still scheduled; the table
only steers balance, never what runs. Regenerate it with
.github/scripts/update-test-durations.ps1.
.PARAMETER RepositoryRoot
Repository root. Defaults to the root two levels above this script.
.PARAMETER ManifestPath
Optional path to write the expanded workload manifest to. The workflows
upload it so a run's intended coverage can be compared against another CI
system's inventory.
#>
Param(
[ValidateSet('pr', 'full')]
[string] $Scope = 'pr',
[switch] $ExcludeMacOS,
[string] $OnlyProject = '',
[int] $MaxEntries = 256,
[int] $ReservedEntries = 40,
[int] $MaxJobTimeoutMinutes = 350,
[int] $BatchTargetMinutes = 30,
[string] $DurationsPath = '',
[string] $RepositoryRoot = '',
[string] $ManifestPath = ''
)
$ErrorActionPreference = 'Stop'
if ([string]::IsNullOrWhiteSpace($RepositoryRoot)) {
$RepositoryRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot '..' '..')).Path
}
else {
$RepositoryRoot = (Resolve-Path -LiteralPath $RepositoryRoot).Path
}
if ([string]::IsNullOrWhiteSpace($DurationsPath)) {
$DurationsPath = Join-Path $RepositoryRoot '.github' 'ci-test-durations.json'
}
if ($BatchTargetMinutes -lt 1) {
throw "BatchTargetMinutes must be positive, not $BatchTargetMinutes."
}
# Runner images: GitHub-hosted labels.
$RunnerImages = @{
windows = 'windows-latest'
linux = 'ubuntu-latest'
macos = 'macos-latest'
}
# Hosted runner architectures, recorded in the manifest so a comparison against
# another CI system's inventory is not silently architecture-blind.
$RunnerArchitectures = @{
windows = 'x64'
linux = 'x64'
macos = 'arm64'
}
$LatestSdk = '10.0.401'
# The category filter the fast legs use. Tiers that lift it declare so below.
$DefaultFilter = 'TestCategory!=LongRunning&TestCategory!=Stress'
# Projects that are not part of the mainline sweep:
# Aot - published and run as native executables rather than
# by 'dotnet test'; the workflows have their own jobs.
# Stress - opt-in tier owned by .github/workflows/stress-test.yml.
# Subscriptions.Durable - its own tier, run unfiltered below.
# OneFuzz.Validator - pins net10.0 to match the net10.0-only drop it
# validates, so a legacy leg fails with NETSDK1005
# before the no-op shell can take effect. It is covered
# by the fuzz-public-drop job instead.
$MainlineExclusions = @(
'Opc.Ua.Aot.Tests.csproj',
'Opc.Ua.Stress.Tests.csproj',
'Opc.Ua.Subscriptions.Durable.Tests.csproj',
'Opc.Ua.OneFuzz.Validator.Tests.csproj')
$DurableProject = 'Opc.Ua.Subscriptions.Durable.Tests.csproj'
<#
.SYNOPSIS
Sorts strings with an ordinal comparison so the batches a run produces do not
depend on the runner's culture.
#>
function Sort-Ordinal([string[]] $values)
{
$list = [System.Collections.Generic.List[string]]::new()
foreach ($value in $values) {
$list.Add($value)
}
$list.Sort([System.StringComparer]::Ordinal)
return $list.ToArray()
}
<#
.SYNOPSIS
Converts an absolute path below the repository root into a forward-slashed
repository-relative path.
#>
function ConvertTo-RelativePath([string] $fullName)
{
return $fullName.Substring($RepositoryRoot.Length).TrimStart('\', '/').Replace('\', '/')
}
<#
.SYNOPSIS
Discovers files below the repository root, skipping build output.
#>
function Find-RepositoryFile([string] $filter)
{
$found = Get-ChildItem -LiteralPath $RepositoryRoot -Recurse -File -Filter $filter |
Where-Object { $_.FullName -notmatch '[\\/](bin|obj|\.git)[\\/]' } |
ForEach-Object { ConvertTo-RelativePath $_.FullName }
return Sort-Ordinal @($found)
}
<#
.SYNOPSIS
Returns the SDK versions a profile needs, newest last so the latest SDK
drives the build while the older runtime is available to host its tests.
#>
function Get-SdkVersions([string] $framework)
{
switch ($framework) {
'net8.0' { return @('8.0.x', $LatestSdk) }
'net9.0' { return @('9.0.x', $LatestSdk) }
default { return @($LatestSdk) }
}
}
# The workload. Each profile fans out over the project set named by its tier.
#
# customTestTarget - passed as /p:CustomTestTarget for restore, build and test
# framework - the target framework the tests actually execute on
# scopes - which CI scopes include the profile
#
# The Linux net10.0 Release mainline profile is pull-request only on purpose: in
# the full scope the long-running tier runs the identical tuple with the category
# filter lifted, so keeping both would run the same tests twice.
$Profiles = @(
@{ id = 'windows-net48'; os = 'windows'; customTestTarget = 'net48'; framework = 'net48'; configuration = 'Release'; scopes = @('pr', 'full') }
@{ id = 'windows-net10.0'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr', 'full') }
@{ id = 'linux-net10.0'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr') }
@{ id = 'macos-net10.0'; os = 'macos'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr', 'full') }
@{ id = 'windows-net10.0-debug'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Debug'; scopes = @('full') }
@{ id = 'linux-net10.0-debug'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Debug'; scopes = @('full') }
@{ id = 'windows-net9.0'; os = 'windows'; customTestTarget = 'net9.0'; framework = 'net9.0'; configuration = 'Release'; scopes = @('full') }
@{ id = 'linux-net9.0'; os = 'linux'; customTestTarget = 'net9.0'; framework = 'net9.0'; configuration = 'Release'; scopes = @('full') }
@{ id = 'windows-net8.0'; os = 'windows'; customTestTarget = 'net8.0'; framework = 'net8.0'; configuration = 'Release'; scopes = @('full') }
@{ id = 'linux-net8.0'; os = 'linux'; customTestTarget = 'net8.0'; framework = 'net8.0'; configuration = 'Release'; scopes = @('full') }
@{ id = 'linux-long-running'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full')
tier = 'long-running'; filter = ''; hangTimeout = '30m'; coverage = $false
}
@{ id = 'windows-durable'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full')
tier = 'durable'; filter = ''; hangTimeout = '30m'; coverage = $false
}
@{ id = 'linux-durable'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full')
tier = 'durable'; filter = ''; hangTimeout = '30m'; coverage = $false
}
)
# Solution builds. 'solutions' is either '*' for every discovered .slnx or an
# explicit list. Every solution across every
# supported target framework in both configurations for the full scope, narrowed
# for pull requests, plus the Linux leg that proves UA.slnx still compiles for
# every Linux-capable target framework.
$BuildProfiles = @(
@{ id = 'pr-windows'; os = 'windows'; scopes = @('pr'); solutions = '*'
tfms = @('net48', 'net10.0'); configurations = @('Debug', 'Release')
}
@{ id = 'pr-linux'; os = 'linux'; scopes = @('pr'); solutions = @('UA.slnx')
tfms = @('net8.0', 'net9.0', 'net10.0'); configurations = @('Release')
}
@{ id = 'full-windows'; os = 'windows'; scopes = @('full'); solutions = '*'
tfms = @('net48', 'net8.0', 'net9.0', 'net10.0')
configurations = @('Debug', 'Release')
}
@{ id = 'full-linux'; os = 'linux'; scopes = @('full'); solutions = @('UA.slnx')
tfms = @('net8.0', 'net9.0', 'net10.0'); configurations = @('Release')
}
)
# Every batch carries this much on top of its packed weight: checkout, SDK
# install, restore, and the cold build of the dependency graph its first
# project pulls in. Run 37520359929 measured 3.8-4.9 minutes for that first
# build against 0.4-0.9 minutes for each further project in the same batch.
$BatchFixedMinutes = 6
# One combined ceiling per project, shared by that project's build and test
# invocation in run-dotnet-tests.ps1. The job timeout budgets it exactly once
# per project, so the executor must not spend it twice. Keep 45 minutes for
# mainline too: on hosted Windows, the net48 Opc.Ua.Server.Tests build plus
# its otherwise healthy test run normally takes about 25 minutes and has
# exceeded 30 under runner variance.
$PerProjectTimeoutMinutes = 45
# 20 minutes of fixed cost plus one per-project ceiling for each project has to
# stay below MaxJobTimeoutMinutes, which caps how many projects one batch takes.
$MaxProjectsPerBatch = [System.Math]::Floor(($MaxJobTimeoutMinutes - 20) / $PerProjectTimeoutMinutes)
if ($MaxProjectsPerBatch -lt 1) {
throw "MaxJobTimeoutMinutes=$MaxJobTimeoutMinutes cannot budget even one $PerProjectTimeoutMinutes-minute project."
}
if (-not (Test-Path -LiteralPath $DurationsPath -PathType Leaf)) {
throw "The per-project duration table '$DurationsPath' does not exist."
}
$durationTable = Get-Content -LiteralPath $DurationsPath -Raw | ConvertFrom-Json -AsHashtable
$DefaultProjectMinutes = [int]$durationTable.defaultMinutes
if ($DefaultProjectMinutes -lt 1) {
throw "'$DurationsPath' must declare a positive defaultMinutes."
}
$ProjectMinutes = @{}
foreach ($entry in $durationTable.projects.GetEnumerator()) {
if ([int]$entry.Value -lt 1) {
throw "'$DurationsPath' gives '$($entry.Key)' a weight of $($entry.Value); every weight must be at least one minute."
}
$ProjectMinutes[$entry.Key] = [int]$entry.Value
}
<#
.SYNOPSIS
Returns the packing weight of a project, in minutes.
#>
function Get-ProjectMinutes([string] $project)
{
$stem = [System.IO.Path]::GetFileNameWithoutExtension($project)
if ($ProjectMinutes.ContainsKey($stem)) {
return $ProjectMinutes[$stem]
}
return $DefaultProjectMinutes
}
$allTestProjects = Find-RepositoryFile '*.Tests.csproj'
$mainlineProjects = @($allTestProjects | Where-Object { $MainlineExclusions -notcontains (Split-Path $_ -Leaf) })
$durableProjects = @($allTestProjects | Where-Object { (Split-Path $_ -Leaf) -eq $DurableProject })
$solutions = Find-RepositoryFile '*.slnx'
if ($mainlineProjects.Count -eq 0) {
throw "Discovery found no mainline test projects beneath '$RepositoryRoot'. A matrix that runs nothing would report success without testing anything."
}
if ($durableProjects.Count -eq 0) {
throw "Discovery found no '$DurableProject'. The durable-subscription tier would silently disappear."
}
if ($solutions.Count -eq 0) {
throw "Discovery found no '*.slnx' beneath '$RepositoryRoot'."
}
if (-not [string]::IsNullOrWhiteSpace($OnlyProject)) {
$normalized = $OnlyProject.Replace('\', '/')
$mainlineProjects = @($mainlineProjects | Where-Object { $_.EndsWith($normalized, [System.StringComparison]::OrdinalIgnoreCase) })
if ($mainlineProjects.Count -eq 0) {
throw "No discovered test project path ends with '$OnlyProject'. A matrix that runs nothing would report success without testing anything."
}
# The named tiers cover named projects, so narrowing to one project leaves
# them with nothing to run rather than silently re-running it unfiltered.
$durableProjects = @()
}
$projectsByTier = @{
mainline = $mainlineProjects
'long-running' = if ([string]::IsNullOrWhiteSpace($OnlyProject)) { $mainlineProjects } else { @() }
durable = $durableProjects
}
<#
.SYNOPSIS
Expands the solution build matrix for a scope.
#>
function Expand-BuildMatrix([string] $scope)
{
$entries = @()
foreach ($buildProfile in $BuildProfiles) {
if ($buildProfile.scopes -notcontains $scope) {
continue
}
$profileSolutions = $buildProfile.solutions
if ($profileSolutions -eq '*') {
$profileSolutions = $solutions
}
foreach ($solution in $profileSolutions) {
if ($solutions -notcontains $solution) {
throw "Build profile '$($buildProfile.id)' names solution '$solution', which was not discovered."
}
$stem = [System.IO.Path]::GetFileNameWithoutExtension($solution)
foreach ($tfm in $buildProfile.tfms) {
foreach ($configuration in $buildProfile.configurations) {
$entries += [ordered]@{
id = "$($buildProfile.id)-$stem-$tfm-$configuration"
name = "$stem $tfm $configuration ($($buildProfile.os))"
os = $buildProfile.os
runsOn = $RunnerImages[$buildProfile.os]
arch = $RunnerArchitectures[$buildProfile.os]
solution = $solution
customTestTarget = $tfm
configuration = $configuration
dotnet = (Get-SdkVersions $tfm) -join "`n"
}
}
}
}
}
return $entries
}
<#
.SYNOPSIS
Packs an ordered project list into contiguous batches whose weights add up
to at most $targetMinutes, never more than $MaxProjectsPerBatch projects.
.DESCRIPTION
Contiguous rather than best-fit: path order keeps projects that share
dependencies together, and the result only depends on the list and the
table, so a run is reproducible. A project heavier than the target is
placed in a batch of its own rather than split or dropped.
#>
function Split-IntoBatches([string[]] $projects, [int] $targetMinutes)
{
$batches = @()
$current = @()
$currentMinutes = 0
foreach ($project in $projects) {
$minutes = Get-ProjectMinutes $project
if ($current.Count -gt 0 -and
($currentMinutes + $minutes -gt $targetMinutes -or $current.Count -ge $MaxProjectsPerBatch)) {
$batches += , @($current)
$current = @()
$currentMinutes = 0
}
$current += $project
$currentMinutes += $minutes
}
if ($current.Count -gt 0) {
$batches += , @($current)
}
return $batches
}
<#
.SYNOPSIS
Counts the test matrix entries a batch target produces for a scope, ignoring
which optional runners are enabled so the target is stable.
#>
function Measure-TestEntries([string] $scope, [int] $targetMinutes)
{
$total = 0
foreach ($testProfile in $Profiles) {
if ($testProfile.scopes -notcontains $scope) {
continue
}
$tier = 'mainline'
if ($testProfile.ContainsKey('tier')) {
$tier = $testProfile.tier
}
$total += @(Split-IntoBatches $projectsByTier[$tier] $targetMinutes).Count
}
return [int]$total
}
<#
.SYNOPSIS
Expands the test matrix and the per-tuple manifest for a scope.
#>
function Expand-TestMatrix([string] $scope, [int] $targetMinutes, [bool] $includeMacOS)
{
$entries = @()
$manifest = @()
foreach ($testProfile in $Profiles) {
if ($testProfile.scopes -notcontains $scope) {
continue
}
if ($testProfile.os -eq 'macos' -and -not $includeMacOS) {
continue
}
$tier = 'mainline'
if ($testProfile.ContainsKey('tier')) {
$tier = $testProfile.tier
}
$filter = $DefaultFilter
if ($testProfile.ContainsKey('filter')) {
$filter = $testProfile.filter
}
$hangTimeout = '10m'
if ($testProfile.ContainsKey('hangTimeout')) {
$hangTimeout = $testProfile.hangTimeout
}
# coverlet.collector 10.x ships build assets for net8.0 and newer only, so
# a .NET Framework test host cannot load the 'XPlat Code Coverage'
# collector at all - VSTest only warns and writes no report. The
# long-running and durable tiers opt out explicitly:
# they re-run projects the filtered legs already covered, so folding their
# numbers into the merged report would double-count them.
$coverage = -not $testProfile.framework.StartsWith('net4')
if ($testProfile.ContainsKey('coverage')) {
$coverage = [bool]$testProfile.coverage
}
# See $PerProjectTimeoutMinutes for why this is one combined ceiling.
$perProjectTimeout = $PerProjectTimeoutMinutes
$batches = @(Split-IntoBatches $projectsByTier[$tier] $targetMinutes)
for ($index = 0; $index -lt $batches.Count; $index++) {
$batch = $batches[$index]
$batchId = "$($testProfile.id)-$(($index + 1).ToString('00'))"
# 20 minutes of fixed cost - checkout, SDK install, restore - plus
# the combined build-and-test ceiling for every project in the
# batch. It has to stay a true upper bound on what the executor can
# spend, or GitHub cancels the job before run-dotnet-tests.ps1 can
# attribute the overrun to a project.
$jobTimeout = 20 + ($batch.Count * $perProjectTimeout)
if ($jobTimeout -gt $MaxJobTimeoutMinutes) {
throw ("Batch '$batchId' of $($batch.Count) projects needs $jobTimeout minutes, past the " +
"$MaxJobTimeoutMinutes-minute ceiling. Clamping it would let the executor outlive the " +
'job, so lower the per-project ceiling or raise ReservedEntries so smaller batches fit.')
}
$entries += [ordered]@{
id = $batchId
name = "$($testProfile.id) ($($index + 1)/$($batches.Count))"
os = $testProfile.os
runsOn = $RunnerImages[$testProfile.os]
arch = $RunnerArchitectures[$testProfile.os]
profile = $testProfile.id
tier = $tier
customTestTarget = $testProfile.customTestTarget
framework = $testProfile.framework
configuration = $testProfile.configuration
filter = $filter
hangTimeout = $hangTimeout
coverage = $coverage
projects = $batch -join ';'
perProjectTimeout = $perProjectTimeout
timeoutMinutes = $jobTimeout
# The batch's summed weight from the duration table: what the
# packer balanced, on top of the fixed per-batch cost.
packedMinutes = [int]($batch | ForEach-Object { Get-ProjectMinutes $_ } | Measure-Object -Sum).Sum
dotnet = (Get-SdkVersions $testProfile.framework) -join "`n"
}
foreach ($project in $batch) {
# Every automatic entry uses the corpus checked into the
# repository. The private crash corpus is only ever replayed by
# the protected job in nightly.yml.
$corpus = 'none'
if ($project -like 'fuzzing/*') {
$corpus = 'public-checked-in'
}
$manifest += [ordered]@{
project = $project
profile = $testProfile.id
tier = $tier
os = $testProfile.os
runsOn = $RunnerImages[$testProfile.os]
arch = $RunnerArchitectures[$testProfile.os]
customTestTarget = $testProfile.customTestTarget
framework = $testProfile.framework
configuration = $testProfile.configuration
filter = $filter
coverage = $coverage
corpus = $corpus
batch = $batchId
}
}
}
}
return @{ entries = $entries; manifest = $manifest }
}
$buildEntries = @(Expand-BuildMatrix $Scope)
$budget = $MaxEntries - $buildEntries.Count - $ReservedEntries
if ($budget -lt 1) {
throw "The '$Scope' solution build matrix alone needs $($buildEntries.Count) of the $MaxEntries entries GitHub Actions allows per workflow run, leaving no room for tests."
}
# Raising the target shrinks the matrix until it fits. Once every batch is
# already at the per-batch project cap, raising it further changes nothing, so
# that is reported rather than looped on.
$heaviestProject = (@($ProjectMinutes.Values) + $DefaultProjectMinutes | Measure-Object -Maximum).Maximum
$targetCeiling = $MaxProjectsPerBatch * $heaviestProject
$targetMinutes = $BatchTargetMinutes
while ((Measure-TestEntries $Scope $targetMinutes) -gt $budget) {
if ($targetMinutes -ge $targetCeiling) {
throw ("No batch target fits the '$Scope' test workload into $budget matrix entries; " +
"$MaxProjectsPerBatch projects per batch is the most MaxJobTimeoutMinutes allows.")
}
$targetMinutes += 5
}
$expanded = Expand-TestMatrix $Scope $targetMinutes (-not $ExcludeMacOS)
$testEntries = @($expanded.entries)
$totalEntries = $testEntries.Count + $buildEntries.Count
$largestBatch = ($testEntries | ForEach-Object { @($_.projects -split ';').Count } | Measure-Object -Maximum).Maximum
Write-Host "Scope: $Scope"
Write-Host "Repository root: $RepositoryRoot"
Write-Host "Mainline projects: $($mainlineProjects.Count)"
Write-Host "Solutions: $($solutions -join ', ')"
Write-Host "Batch target: $targetMinutes min (+$BatchFixedMinutes fixed), at most $MaxProjectsPerBatch projects"
Write-Host "Largest batch: $largestBatch projects"
Write-Host "Test entries: $($testEntries.Count)"
Write-Host "Build entries: $($buildEntries.Count)"
Write-Host "Reserved entries: $ReservedEntries"
Write-Host "Total matrix jobs: $totalEntries (limit $MaxEntries)"
if ($totalEntries -gt $MaxEntries) {
throw "The '$Scope' workload expands to $totalEntries matrix jobs, past the $MaxEntries GitHub Actions allows per workflow run. Raise BatchTargetMinutes so fewer, larger batches are chosen rather than dropping entries."
}
if (-not [string]::IsNullOrWhiteSpace($ManifestPath)) {
$manifestDirectory = Split-Path -Parent $ManifestPath
if (-not [string]::IsNullOrWhiteSpace($manifestDirectory) -and -not (Test-Path -LiteralPath $manifestDirectory)) {
$null = New-Item -ItemType Directory -Path $manifestDirectory -Force
}
$document = [ordered]@{
scope = $Scope
batchTargetMinutes = $targetMinutes
includeMacOS = (-not $ExcludeMacOS)
builds = $buildEntries
tests = @($expanded.manifest)
}
$document | ConvertTo-Json -Depth 6 | Out-File -LiteralPath $ManifestPath -Encoding utf8
Write-Host "Wrote the workload manifest to '$ManifestPath'."
}
$testsJson = ConvertTo-Json -InputObject $testEntries -Depth 5 -Compress
$buildsJson = ConvertTo-Json -InputObject $buildEntries -Depth 5 -Compress
if (-not [string]::IsNullOrWhiteSpace($env:GITHUB_OUTPUT)) {
"tests=$testsJson" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
"builds=$buildsJson" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
"batch_target_minutes=$targetMinutes" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
"test_entry_count=$($testEntries.Count)" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
"build_entry_count=$($buildEntries.Count)" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
}
else {
Write-Output $testsJson
Write-Output $buildsJson
}