mirror of
https://github.com/OPCFoundation/UA-.NETStandard.git
synced 2026-10-18 14:43:08 +08:00
fbaabf5b28
Reduces GitHub Actions runner minutes without removing any OS, target framework, test project or check. All figures come from master CI run 37520359929 (158 jobs, ~1,880 runner-minutes) and the last two weeks of run history. ## 1. Test batches packed by duration (`get-ci-matrix.ps1`) Previously the expander picked the **smallest** batch size that fit the 256-job limit, which meant 2 projects per job and 33 jobs per profile. Job logs show ~45% of the test matrix (674 of ~1,516 min) went to compiling. Each batch rebuilds the shared dependency graph: the first project's build took 3.8–4.9 min, the second only 0.4–0.9 min. - Projects are now packed in path order (which keeps `PubSub.*`, `Redundancy.*`, `WotCon.*` together) up to a 30-minute weight target, using `.github/ci-test-durations.json`. Each weight is the slowest PR profile's test minutes, rounded up, plus 1. - A batch never holds more projects than the job timeout can budget for: at most 7, given 20 + n×45 ≤ 350. - If the matrix would exceed the job limit, the target is raised instead. - **PR test matrix: 132 → 60 jobs** (15 per profile; macOS 33 → 15). Each batch is estimated at ≤36 min on the slowest runner, comparable to today's slowest jobs (37–43 min), so wall-clock should be about the same. Expected saving is roughly 300+ runner-min per run, plus less queueing for the scarce macOS runners. - The weekly full scope shrinks the same way (still 13 profiles). - The table only steers balance: a project missing from it gets `defaultMinutes`, so it is still scheduled. - `run-dotnet-tests.ps1` now records per-project `buildSeconds`/`testSeconds` (batch summary and job summary). The new `update-test-durations.ps1` refreshes the table from any run's `dotnet-results-*` artifacts. - `discover` output `batch_size` is renamed to `batch_target_minutes`. Nothing consumed it. New `CiMatrixScriptTests` cases: - each profile runs each of its projects exactly once; - only a lone project may exceed the target; - PR profiles average ≥3 projects per job; - the duration table names only existing projects. ## 2 + 3. CodeQL (`codeql-analysis.yml`) - **No longer runs on every push to master.** Instead, a daily schedule on master (was weekly) gives an analysis that always completes. Each analysis takes ~3 h, and with `cancel-in-progress: true` per branch, 49 of 76 push runs in two weeks were cancelled by the next merge before uploading anything. `master378`/`release/*`/`develop/*` keep their push trigger. PRs are still analyzed as before. - Only PR runs are cancelled when superseded. Push and scheduled runs always finish. - **Runs on `ubuntu-latest` instead of `windows-2025`.** Only net10.0 is built, so the extracted sources are identical. The traced Windows build alone took 70 min, against 12 min for the untraced Linux `UA.slnx` net10.0 leg. This PR touches a `.cs` file, so its own CodeQL run is the first measurement. ## 4. NuGet packing on feature branches (`nuget-publish.yml`) 298 of ~400 runs in two weeks were feature-branch pushes, each about 59 Windows-min across two signed builds. - A superseded feature-branch run is now cancelled. Protected lines and tags are never cancelled: they publish previews, and `release.yml` promotes a run by ID. Each manual (`workflow_dispatch`) run gets its own concurrency group, so a later push cannot cancel it. - A new `changes` job skips packing when a feature branch, as a whole, changes only documentation. It diffs against the merge base with master, not the previous push, so a docs-only push that cancels a code push cannot mask that code change. It uses the same `get-path-relevance.ps1` rule as CI. - Protected refs, tags, manual runs, and any branch whose merge base cannot be resolved always pack. Checked locally for docs-only branch (skip), code branch, protected ref, manual run and unknown commit (all pack). ## Verification - `CiMatrixScriptTests`, `CiPathRelevanceTests` and the new `CiTestDurationsScriptTests` (covering `update-test-durations.ps1`): all `*.Ci*` tests in `Opc.Ua.Tools.Tests` pass locally, 63/63. - This PR's own CI run shows the new matrix. Compare its total runner-minutes and longest test job against run 37520359929. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
618 lines
28 KiB
PowerShell
618 lines
28 KiB
PowerShell
<#
|
|
.SYNOPSIS
|
|
Expands the GitHub Actions build and test workload for a CI scope.
|
|
|
|
.DESCRIPTION
|
|
Produces the job matrices consumed by .github/workflows/buildandtest.yml
|
|
(pull-request scope) and .github/workflows/nightly.yml (full scope), plus a
|
|
manifest that records every (project x profile) tuple the run is expected to
|
|
cover.
|
|
|
|
The workload is described once, here, as a table of profiles. A profile is a
|
|
(runner OS, CustomTestTarget, test-host target framework, configuration,
|
|
category filter, tier) tuple. Projects are discovered from the file system
|
|
with a '*.Tests.csproj' sweep, so neither matrix has to be hand-maintained
|
|
as projects are added or renamed.
|
|
|
|
CustomTestTarget, not '--framework', is the mechanism that pins the stack to
|
|
a single target framework (see targets.props). A profile carries both the
|
|
CustomTestTarget it builds with and the framework its tests actually run on.
|
|
|
|
The project fan-out is batched: each matrix entry carries several projects
|
|
that the executor runs in sequence, keeping per-project results. Batches are
|
|
sized by measured duration rather than by project count. Every batch pays a
|
|
fixed cost of several minutes - checkout, SDK install, and the cold build of
|
|
the shared dependency graph - while a further project in the same batch only
|
|
adds its incremental build and its tests. Two projects per job spent almost
|
|
half of the pull-request matrix compiling the same assemblies over and over,
|
|
so projects are packed, in path order, into batches of about
|
|
BatchTargetMinutes each. Path order keeps related projects (PubSub.*,
|
|
Redundancy.*, WotCon.*) together, so their shared dependencies are built
|
|
once per batch.
|
|
|
|
GitHub Actions also refuses to start a workflow run whose matrices expand
|
|
past 256 jobs, and that limit cannot be raised. If the packed matrix does not
|
|
fit, the target is raised until it does, so the matrix adapts as test
|
|
projects are added instead of silently truncating.
|
|
|
|
.PARAMETER Scope
|
|
'pr' expands the workload that gates a pull request. 'full' expands the
|
|
complete workload Azure Pipelines used to run on a schedule.
|
|
|
|
.PARAMETER ExcludeMacOS
|
|
Drop the macOS test profile. The batch target is deliberately computed from
|
|
the full profile table regardless of this switch, so excluding macOS does not
|
|
reshuffle the batches of the remaining profiles.
|
|
|
|
.PARAMETER OnlyProject
|
|
Restrict the test matrix to the single test project whose path ends with this
|
|
value. Used by the protected private-fuzz-corpus job so it covers exactly the
|
|
profiles this table already enables for that project, instead of repeating
|
|
the profile list in a second place. The non-mainline tiers are dropped, since
|
|
they cover named projects of their own.
|
|
|
|
.PARAMETER MaxEntries
|
|
The documented GitHub Actions ceiling on matrix jobs per workflow run.
|
|
Exceeding it is an error, never a truncation.
|
|
|
|
.PARAMETER ReservedEntries
|
|
Entries reserved for the jobs that are not part of these matrices - Native
|
|
AoT, the fuzz contract and replay jobs, coverage and the summary - plus
|
|
headroom.
|
|
|
|
.PARAMETER MaxJobTimeoutMinutes
|
|
Upper bound on a batch job's 'timeout-minutes', below the 360-minute ceiling
|
|
GitHub enforces on hosted runners. A batch whose per-project budgets add up
|
|
past this is an error rather than a clamp: clamping would let the executor
|
|
outlive the job, which is the one case that produces no annotation and no
|
|
results. The packer therefore never puts more projects in one batch than
|
|
this ceiling can budget for.
|
|
|
|
.PARAMETER BatchTargetMinutes
|
|
The measured duration, in minutes, a batch is packed up to, on top of its
|
|
fixed cost. Weights come from DurationsPath and are the slowest pull-request
|
|
profile's figures, so batches on the faster runners finish sooner. A project
|
|
heavier than the target gets a batch of its own. Raised in steps of five
|
|
when the packed matrix would not fit MaxEntries.
|
|
|
|
.PARAMETER DurationsPath
|
|
JSON table of per-project weights in minutes. Defaults to
|
|
.github/ci-test-durations.json. A project missing from it is weighted with
|
|
the table's defaultMinutes, so a new project is still scheduled; the table
|
|
only steers balance, never what runs. Regenerate it with
|
|
.github/scripts/update-test-durations.ps1.
|
|
|
|
.PARAMETER RepositoryRoot
|
|
Repository root. Defaults to the root two levels above this script.
|
|
|
|
.PARAMETER ManifestPath
|
|
Optional path to write the expanded workload manifest to. The workflows
|
|
upload it so a run's intended coverage can be compared against another CI
|
|
system's inventory.
|
|
#>
|
|
|
|
Param(
|
|
[ValidateSet('pr', 'full')]
|
|
[string] $Scope = 'pr',
|
|
[switch] $ExcludeMacOS,
|
|
[string] $OnlyProject = '',
|
|
[int] $MaxEntries = 256,
|
|
[int] $ReservedEntries = 40,
|
|
[int] $MaxJobTimeoutMinutes = 350,
|
|
[int] $BatchTargetMinutes = 30,
|
|
[string] $DurationsPath = '',
|
|
[string] $RepositoryRoot = '',
|
|
[string] $ManifestPath = ''
|
|
)
|
|
|
|
$ErrorActionPreference = 'Stop'
|
|
|
|
if ([string]::IsNullOrWhiteSpace($RepositoryRoot)) {
|
|
$RepositoryRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot '..' '..')).Path
|
|
}
|
|
else {
|
|
$RepositoryRoot = (Resolve-Path -LiteralPath $RepositoryRoot).Path
|
|
}
|
|
if ([string]::IsNullOrWhiteSpace($DurationsPath)) {
|
|
$DurationsPath = Join-Path $RepositoryRoot '.github' 'ci-test-durations.json'
|
|
}
|
|
if ($BatchTargetMinutes -lt 1) {
|
|
throw "BatchTargetMinutes must be positive, not $BatchTargetMinutes."
|
|
}
|
|
|
|
# Runner images: GitHub-hosted labels.
|
|
$RunnerImages = @{
|
|
windows = 'windows-latest'
|
|
linux = 'ubuntu-latest'
|
|
macos = 'macos-latest'
|
|
}
|
|
|
|
# Hosted runner architectures, recorded in the manifest so a comparison against
|
|
# another CI system's inventory is not silently architecture-blind.
|
|
$RunnerArchitectures = @{
|
|
windows = 'x64'
|
|
linux = 'x64'
|
|
macos = 'arm64'
|
|
}
|
|
|
|
$LatestSdk = '10.0.401'
|
|
|
|
# The category filter the fast legs use. Tiers that lift it declare so below.
|
|
$DefaultFilter = 'TestCategory!=LongRunning&TestCategory!=Stress'
|
|
|
|
# Projects that are not part of the mainline sweep:
|
|
# Aot - published and run as native executables rather than
|
|
# by 'dotnet test'; the workflows have their own jobs.
|
|
# Stress - opt-in tier owned by .github/workflows/stress-test.yml.
|
|
# Subscriptions.Durable - its own tier, run unfiltered below.
|
|
# OneFuzz.Validator - pins net10.0 to match the net10.0-only drop it
|
|
# validates, so a legacy leg fails with NETSDK1005
|
|
# before the no-op shell can take effect. It is covered
|
|
# by the fuzz-public-drop job instead.
|
|
$MainlineExclusions = @(
|
|
'Opc.Ua.Aot.Tests.csproj',
|
|
'Opc.Ua.Stress.Tests.csproj',
|
|
'Opc.Ua.Subscriptions.Durable.Tests.csproj',
|
|
'Opc.Ua.OneFuzz.Validator.Tests.csproj')
|
|
|
|
$DurableProject = 'Opc.Ua.Subscriptions.Durable.Tests.csproj'
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Sorts strings with an ordinal comparison so the batches a run produces do not
|
|
depend on the runner's culture.
|
|
#>
|
|
function Sort-Ordinal([string[]] $values)
|
|
{
|
|
$list = [System.Collections.Generic.List[string]]::new()
|
|
foreach ($value in $values) {
|
|
$list.Add($value)
|
|
}
|
|
$list.Sort([System.StringComparer]::Ordinal)
|
|
return $list.ToArray()
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Converts an absolute path below the repository root into a forward-slashed
|
|
repository-relative path.
|
|
#>
|
|
function ConvertTo-RelativePath([string] $fullName)
|
|
{
|
|
return $fullName.Substring($RepositoryRoot.Length).TrimStart('\', '/').Replace('\', '/')
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Discovers files below the repository root, skipping build output.
|
|
#>
|
|
function Find-RepositoryFile([string] $filter)
|
|
{
|
|
$found = Get-ChildItem -LiteralPath $RepositoryRoot -Recurse -File -Filter $filter |
|
|
Where-Object { $_.FullName -notmatch '[\\/](bin|obj|\.git)[\\/]' } |
|
|
ForEach-Object { ConvertTo-RelativePath $_.FullName }
|
|
return Sort-Ordinal @($found)
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Returns the SDK versions a profile needs, newest last so the latest SDK
|
|
drives the build while the older runtime is available to host its tests.
|
|
#>
|
|
function Get-SdkVersions([string] $framework)
|
|
{
|
|
switch ($framework) {
|
|
'net8.0' { return @('8.0.x', $LatestSdk) }
|
|
'net9.0' { return @('9.0.x', $LatestSdk) }
|
|
default { return @($LatestSdk) }
|
|
}
|
|
}
|
|
|
|
# The workload. Each profile fans out over the project set named by its tier.
|
|
#
|
|
# customTestTarget - passed as /p:CustomTestTarget for restore, build and test
|
|
# framework - the target framework the tests actually execute on
|
|
# scopes - which CI scopes include the profile
|
|
#
|
|
# The Linux net10.0 Release mainline profile is pull-request only on purpose: in
|
|
# the full scope the long-running tier runs the identical tuple with the category
|
|
# filter lifted, so keeping both would run the same tests twice.
|
|
$Profiles = @(
|
|
@{ id = 'windows-net48'; os = 'windows'; customTestTarget = 'net48'; framework = 'net48'; configuration = 'Release'; scopes = @('pr', 'full') }
|
|
@{ id = 'windows-net10.0'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr', 'full') }
|
|
@{ id = 'linux-net10.0'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr') }
|
|
@{ id = 'macos-net10.0'; os = 'macos'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr', 'full') }
|
|
@{ id = 'windows-net10.0-debug'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Debug'; scopes = @('full') }
|
|
@{ id = 'linux-net10.0-debug'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Debug'; scopes = @('full') }
|
|
@{ id = 'windows-net9.0'; os = 'windows'; customTestTarget = 'net9.0'; framework = 'net9.0'; configuration = 'Release'; scopes = @('full') }
|
|
@{ id = 'linux-net9.0'; os = 'linux'; customTestTarget = 'net9.0'; framework = 'net9.0'; configuration = 'Release'; scopes = @('full') }
|
|
@{ id = 'windows-net8.0'; os = 'windows'; customTestTarget = 'net8.0'; framework = 'net8.0'; configuration = 'Release'; scopes = @('full') }
|
|
@{ id = 'linux-net8.0'; os = 'linux'; customTestTarget = 'net8.0'; framework = 'net8.0'; configuration = 'Release'; scopes = @('full') }
|
|
@{ id = 'linux-long-running'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full')
|
|
tier = 'long-running'; filter = ''; hangTimeout = '30m'; coverage = $false
|
|
}
|
|
@{ id = 'windows-durable'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full')
|
|
tier = 'durable'; filter = ''; hangTimeout = '30m'; coverage = $false
|
|
}
|
|
@{ id = 'linux-durable'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full')
|
|
tier = 'durable'; filter = ''; hangTimeout = '30m'; coverage = $false
|
|
}
|
|
)
|
|
|
|
# Solution builds. 'solutions' is either '*' for every discovered .slnx or an
|
|
# explicit list. Every solution across every
|
|
# supported target framework in both configurations for the full scope, narrowed
|
|
# for pull requests, plus the Linux leg that proves UA.slnx still compiles for
|
|
# every Linux-capable target framework.
|
|
$BuildProfiles = @(
|
|
@{ id = 'pr-windows'; os = 'windows'; scopes = @('pr'); solutions = '*'
|
|
tfms = @('net48', 'net10.0'); configurations = @('Debug', 'Release')
|
|
}
|
|
@{ id = 'pr-linux'; os = 'linux'; scopes = @('pr'); solutions = @('UA.slnx')
|
|
tfms = @('net8.0', 'net9.0', 'net10.0'); configurations = @('Release')
|
|
}
|
|
@{ id = 'full-windows'; os = 'windows'; scopes = @('full'); solutions = '*'
|
|
tfms = @('net48', 'net8.0', 'net9.0', 'net10.0')
|
|
configurations = @('Debug', 'Release')
|
|
}
|
|
@{ id = 'full-linux'; os = 'linux'; scopes = @('full'); solutions = @('UA.slnx')
|
|
tfms = @('net8.0', 'net9.0', 'net10.0'); configurations = @('Release')
|
|
}
|
|
)
|
|
|
|
# Every batch carries this much on top of its packed weight: checkout, SDK
|
|
# install, restore, and the cold build of the dependency graph its first
|
|
# project pulls in. Run 37520359929 measured 3.8-4.9 minutes for that first
|
|
# build against 0.4-0.9 minutes for each further project in the same batch.
|
|
$BatchFixedMinutes = 6
|
|
|
|
# One combined ceiling per project, shared by that project's build and test
|
|
# invocation in run-dotnet-tests.ps1. The job timeout budgets it exactly once
|
|
# per project, so the executor must not spend it twice. Keep 45 minutes for
|
|
# mainline too: on hosted Windows, the net48 Opc.Ua.Server.Tests build plus
|
|
# its otherwise healthy test run normally takes about 25 minutes and has
|
|
# exceeded 30 under runner variance.
|
|
$PerProjectTimeoutMinutes = 45
|
|
|
|
# 20 minutes of fixed cost plus one per-project ceiling for each project has to
|
|
# stay below MaxJobTimeoutMinutes, which caps how many projects one batch takes.
|
|
$MaxProjectsPerBatch = [System.Math]::Floor(($MaxJobTimeoutMinutes - 20) / $PerProjectTimeoutMinutes)
|
|
if ($MaxProjectsPerBatch -lt 1) {
|
|
throw "MaxJobTimeoutMinutes=$MaxJobTimeoutMinutes cannot budget even one $PerProjectTimeoutMinutes-minute project."
|
|
}
|
|
|
|
if (-not (Test-Path -LiteralPath $DurationsPath -PathType Leaf)) {
|
|
throw "The per-project duration table '$DurationsPath' does not exist."
|
|
}
|
|
$durationTable = Get-Content -LiteralPath $DurationsPath -Raw | ConvertFrom-Json -AsHashtable
|
|
$DefaultProjectMinutes = [int]$durationTable.defaultMinutes
|
|
if ($DefaultProjectMinutes -lt 1) {
|
|
throw "'$DurationsPath' must declare a positive defaultMinutes."
|
|
}
|
|
$ProjectMinutes = @{}
|
|
foreach ($entry in $durationTable.projects.GetEnumerator()) {
|
|
if ([int]$entry.Value -lt 1) {
|
|
throw "'$DurationsPath' gives '$($entry.Key)' a weight of $($entry.Value); every weight must be at least one minute."
|
|
}
|
|
$ProjectMinutes[$entry.Key] = [int]$entry.Value
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Returns the packing weight of a project, in minutes.
|
|
#>
|
|
function Get-ProjectMinutes([string] $project)
|
|
{
|
|
$stem = [System.IO.Path]::GetFileNameWithoutExtension($project)
|
|
if ($ProjectMinutes.ContainsKey($stem)) {
|
|
return $ProjectMinutes[$stem]
|
|
}
|
|
return $DefaultProjectMinutes
|
|
}
|
|
|
|
$allTestProjects = Find-RepositoryFile '*.Tests.csproj'
|
|
$mainlineProjects = @($allTestProjects | Where-Object { $MainlineExclusions -notcontains (Split-Path $_ -Leaf) })
|
|
$durableProjects = @($allTestProjects | Where-Object { (Split-Path $_ -Leaf) -eq $DurableProject })
|
|
$solutions = Find-RepositoryFile '*.slnx'
|
|
|
|
if ($mainlineProjects.Count -eq 0) {
|
|
throw "Discovery found no mainline test projects beneath '$RepositoryRoot'. A matrix that runs nothing would report success without testing anything."
|
|
}
|
|
if ($durableProjects.Count -eq 0) {
|
|
throw "Discovery found no '$DurableProject'. The durable-subscription tier would silently disappear."
|
|
}
|
|
if ($solutions.Count -eq 0) {
|
|
throw "Discovery found no '*.slnx' beneath '$RepositoryRoot'."
|
|
}
|
|
|
|
if (-not [string]::IsNullOrWhiteSpace($OnlyProject)) {
|
|
$normalized = $OnlyProject.Replace('\', '/')
|
|
$mainlineProjects = @($mainlineProjects | Where-Object { $_.EndsWith($normalized, [System.StringComparison]::OrdinalIgnoreCase) })
|
|
if ($mainlineProjects.Count -eq 0) {
|
|
throw "No discovered test project path ends with '$OnlyProject'. A matrix that runs nothing would report success without testing anything."
|
|
}
|
|
# The named tiers cover named projects, so narrowing to one project leaves
|
|
# them with nothing to run rather than silently re-running it unfiltered.
|
|
$durableProjects = @()
|
|
}
|
|
|
|
$projectsByTier = @{
|
|
mainline = $mainlineProjects
|
|
'long-running' = if ([string]::IsNullOrWhiteSpace($OnlyProject)) { $mainlineProjects } else { @() }
|
|
durable = $durableProjects
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Expands the solution build matrix for a scope.
|
|
#>
|
|
function Expand-BuildMatrix([string] $scope)
|
|
{
|
|
$entries = @()
|
|
foreach ($buildProfile in $BuildProfiles) {
|
|
if ($buildProfile.scopes -notcontains $scope) {
|
|
continue
|
|
}
|
|
|
|
$profileSolutions = $buildProfile.solutions
|
|
if ($profileSolutions -eq '*') {
|
|
$profileSolutions = $solutions
|
|
}
|
|
foreach ($solution in $profileSolutions) {
|
|
if ($solutions -notcontains $solution) {
|
|
throw "Build profile '$($buildProfile.id)' names solution '$solution', which was not discovered."
|
|
}
|
|
$stem = [System.IO.Path]::GetFileNameWithoutExtension($solution)
|
|
foreach ($tfm in $buildProfile.tfms) {
|
|
foreach ($configuration in $buildProfile.configurations) {
|
|
$entries += [ordered]@{
|
|
id = "$($buildProfile.id)-$stem-$tfm-$configuration"
|
|
name = "$stem $tfm $configuration ($($buildProfile.os))"
|
|
os = $buildProfile.os
|
|
runsOn = $RunnerImages[$buildProfile.os]
|
|
arch = $RunnerArchitectures[$buildProfile.os]
|
|
solution = $solution
|
|
customTestTarget = $tfm
|
|
configuration = $configuration
|
|
dotnet = (Get-SdkVersions $tfm) -join "`n"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
return $entries
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Packs an ordered project list into contiguous batches whose weights add up
|
|
to at most $targetMinutes, never more than $MaxProjectsPerBatch projects.
|
|
|
|
.DESCRIPTION
|
|
Contiguous rather than best-fit: path order keeps projects that share
|
|
dependencies together, and the result only depends on the list and the
|
|
table, so a run is reproducible. A project heavier than the target is
|
|
placed in a batch of its own rather than split or dropped.
|
|
#>
|
|
function Split-IntoBatches([string[]] $projects, [int] $targetMinutes)
|
|
{
|
|
$batches = @()
|
|
$current = @()
|
|
$currentMinutes = 0
|
|
foreach ($project in $projects) {
|
|
$minutes = Get-ProjectMinutes $project
|
|
if ($current.Count -gt 0 -and
|
|
($currentMinutes + $minutes -gt $targetMinutes -or $current.Count -ge $MaxProjectsPerBatch)) {
|
|
$batches += , @($current)
|
|
$current = @()
|
|
$currentMinutes = 0
|
|
}
|
|
$current += $project
|
|
$currentMinutes += $minutes
|
|
}
|
|
if ($current.Count -gt 0) {
|
|
$batches += , @($current)
|
|
}
|
|
return $batches
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Counts the test matrix entries a batch target produces for a scope, ignoring
|
|
which optional runners are enabled so the target is stable.
|
|
#>
|
|
function Measure-TestEntries([string] $scope, [int] $targetMinutes)
|
|
{
|
|
$total = 0
|
|
foreach ($testProfile in $Profiles) {
|
|
if ($testProfile.scopes -notcontains $scope) {
|
|
continue
|
|
}
|
|
$tier = 'mainline'
|
|
if ($testProfile.ContainsKey('tier')) {
|
|
$tier = $testProfile.tier
|
|
}
|
|
$total += @(Split-IntoBatches $projectsByTier[$tier] $targetMinutes).Count
|
|
}
|
|
return [int]$total
|
|
}
|
|
|
|
<#
|
|
.SYNOPSIS
|
|
Expands the test matrix and the per-tuple manifest for a scope.
|
|
#>
|
|
function Expand-TestMatrix([string] $scope, [int] $targetMinutes, [bool] $includeMacOS)
|
|
{
|
|
$entries = @()
|
|
$manifest = @()
|
|
foreach ($testProfile in $Profiles) {
|
|
if ($testProfile.scopes -notcontains $scope) {
|
|
continue
|
|
}
|
|
if ($testProfile.os -eq 'macos' -and -not $includeMacOS) {
|
|
continue
|
|
}
|
|
|
|
$tier = 'mainline'
|
|
if ($testProfile.ContainsKey('tier')) {
|
|
$tier = $testProfile.tier
|
|
}
|
|
$filter = $DefaultFilter
|
|
if ($testProfile.ContainsKey('filter')) {
|
|
$filter = $testProfile.filter
|
|
}
|
|
$hangTimeout = '10m'
|
|
if ($testProfile.ContainsKey('hangTimeout')) {
|
|
$hangTimeout = $testProfile.hangTimeout
|
|
}
|
|
# coverlet.collector 10.x ships build assets for net8.0 and newer only, so
|
|
# a .NET Framework test host cannot load the 'XPlat Code Coverage'
|
|
# collector at all - VSTest only warns and writes no report. The
|
|
# long-running and durable tiers opt out explicitly:
|
|
# they re-run projects the filtered legs already covered, so folding their
|
|
# numbers into the merged report would double-count them.
|
|
$coverage = -not $testProfile.framework.StartsWith('net4')
|
|
if ($testProfile.ContainsKey('coverage')) {
|
|
$coverage = [bool]$testProfile.coverage
|
|
}
|
|
# See $PerProjectTimeoutMinutes for why this is one combined ceiling.
|
|
$perProjectTimeout = $PerProjectTimeoutMinutes
|
|
|
|
$batches = @(Split-IntoBatches $projectsByTier[$tier] $targetMinutes)
|
|
for ($index = 0; $index -lt $batches.Count; $index++) {
|
|
$batch = $batches[$index]
|
|
$batchId = "$($testProfile.id)-$(($index + 1).ToString('00'))"
|
|
# 20 minutes of fixed cost - checkout, SDK install, restore - plus
|
|
# the combined build-and-test ceiling for every project in the
|
|
# batch. It has to stay a true upper bound on what the executor can
|
|
# spend, or GitHub cancels the job before run-dotnet-tests.ps1 can
|
|
# attribute the overrun to a project.
|
|
$jobTimeout = 20 + ($batch.Count * $perProjectTimeout)
|
|
if ($jobTimeout -gt $MaxJobTimeoutMinutes) {
|
|
throw ("Batch '$batchId' of $($batch.Count) projects needs $jobTimeout minutes, past the " +
|
|
"$MaxJobTimeoutMinutes-minute ceiling. Clamping it would let the executor outlive the " +
|
|
'job, so lower the per-project ceiling or raise ReservedEntries so smaller batches fit.')
|
|
}
|
|
$entries += [ordered]@{
|
|
id = $batchId
|
|
name = "$($testProfile.id) ($($index + 1)/$($batches.Count))"
|
|
os = $testProfile.os
|
|
runsOn = $RunnerImages[$testProfile.os]
|
|
arch = $RunnerArchitectures[$testProfile.os]
|
|
profile = $testProfile.id
|
|
tier = $tier
|
|
customTestTarget = $testProfile.customTestTarget
|
|
framework = $testProfile.framework
|
|
configuration = $testProfile.configuration
|
|
filter = $filter
|
|
hangTimeout = $hangTimeout
|
|
coverage = $coverage
|
|
projects = $batch -join ';'
|
|
perProjectTimeout = $perProjectTimeout
|
|
timeoutMinutes = $jobTimeout
|
|
# The batch's summed weight from the duration table: what the
|
|
# packer balanced, on top of the fixed per-batch cost.
|
|
packedMinutes = [int]($batch | ForEach-Object { Get-ProjectMinutes $_ } | Measure-Object -Sum).Sum
|
|
dotnet = (Get-SdkVersions $testProfile.framework) -join "`n"
|
|
}
|
|
|
|
foreach ($project in $batch) {
|
|
# Every automatic entry uses the corpus checked into the
|
|
# repository. The private crash corpus is only ever replayed by
|
|
# the protected job in nightly.yml.
|
|
$corpus = 'none'
|
|
if ($project -like 'fuzzing/*') {
|
|
$corpus = 'public-checked-in'
|
|
}
|
|
$manifest += [ordered]@{
|
|
project = $project
|
|
profile = $testProfile.id
|
|
tier = $tier
|
|
os = $testProfile.os
|
|
runsOn = $RunnerImages[$testProfile.os]
|
|
arch = $RunnerArchitectures[$testProfile.os]
|
|
customTestTarget = $testProfile.customTestTarget
|
|
framework = $testProfile.framework
|
|
configuration = $testProfile.configuration
|
|
filter = $filter
|
|
coverage = $coverage
|
|
corpus = $corpus
|
|
batch = $batchId
|
|
}
|
|
}
|
|
}
|
|
}
|
|
return @{ entries = $entries; manifest = $manifest }
|
|
}
|
|
|
|
$buildEntries = @(Expand-BuildMatrix $Scope)
|
|
$budget = $MaxEntries - $buildEntries.Count - $ReservedEntries
|
|
if ($budget -lt 1) {
|
|
throw "The '$Scope' solution build matrix alone needs $($buildEntries.Count) of the $MaxEntries entries GitHub Actions allows per workflow run, leaving no room for tests."
|
|
}
|
|
|
|
# Raising the target shrinks the matrix until it fits. Once every batch is
|
|
# already at the per-batch project cap, raising it further changes nothing, so
|
|
# that is reported rather than looped on.
|
|
$heaviestProject = (@($ProjectMinutes.Values) + $DefaultProjectMinutes | Measure-Object -Maximum).Maximum
|
|
$targetCeiling = $MaxProjectsPerBatch * $heaviestProject
|
|
$targetMinutes = $BatchTargetMinutes
|
|
while ((Measure-TestEntries $Scope $targetMinutes) -gt $budget) {
|
|
if ($targetMinutes -ge $targetCeiling) {
|
|
throw ("No batch target fits the '$Scope' test workload into $budget matrix entries; " +
|
|
"$MaxProjectsPerBatch projects per batch is the most MaxJobTimeoutMinutes allows.")
|
|
}
|
|
$targetMinutes += 5
|
|
}
|
|
|
|
$expanded = Expand-TestMatrix $Scope $targetMinutes (-not $ExcludeMacOS)
|
|
$testEntries = @($expanded.entries)
|
|
$totalEntries = $testEntries.Count + $buildEntries.Count
|
|
$largestBatch = ($testEntries | ForEach-Object { @($_.projects -split ';').Count } | Measure-Object -Maximum).Maximum
|
|
|
|
Write-Host "Scope: $Scope"
|
|
Write-Host "Repository root: $RepositoryRoot"
|
|
Write-Host "Mainline projects: $($mainlineProjects.Count)"
|
|
Write-Host "Solutions: $($solutions -join ', ')"
|
|
Write-Host "Batch target: $targetMinutes min (+$BatchFixedMinutes fixed), at most $MaxProjectsPerBatch projects"
|
|
Write-Host "Largest batch: $largestBatch projects"
|
|
Write-Host "Test entries: $($testEntries.Count)"
|
|
Write-Host "Build entries: $($buildEntries.Count)"
|
|
Write-Host "Reserved entries: $ReservedEntries"
|
|
Write-Host "Total matrix jobs: $totalEntries (limit $MaxEntries)"
|
|
|
|
if ($totalEntries -gt $MaxEntries) {
|
|
throw "The '$Scope' workload expands to $totalEntries matrix jobs, past the $MaxEntries GitHub Actions allows per workflow run. Raise BatchTargetMinutes so fewer, larger batches are chosen rather than dropping entries."
|
|
}
|
|
|
|
if (-not [string]::IsNullOrWhiteSpace($ManifestPath)) {
|
|
$manifestDirectory = Split-Path -Parent $ManifestPath
|
|
if (-not [string]::IsNullOrWhiteSpace($manifestDirectory) -and -not (Test-Path -LiteralPath $manifestDirectory)) {
|
|
$null = New-Item -ItemType Directory -Path $manifestDirectory -Force
|
|
}
|
|
$document = [ordered]@{
|
|
scope = $Scope
|
|
batchTargetMinutes = $targetMinutes
|
|
includeMacOS = (-not $ExcludeMacOS)
|
|
builds = $buildEntries
|
|
tests = @($expanded.manifest)
|
|
}
|
|
$document | ConvertTo-Json -Depth 6 | Out-File -LiteralPath $ManifestPath -Encoding utf8
|
|
Write-Host "Wrote the workload manifest to '$ManifestPath'."
|
|
}
|
|
|
|
$testsJson = ConvertTo-Json -InputObject $testEntries -Depth 5 -Compress
|
|
$buildsJson = ConvertTo-Json -InputObject $buildEntries -Depth 5 -Compress
|
|
|
|
if (-not [string]::IsNullOrWhiteSpace($env:GITHUB_OUTPUT)) {
|
|
"tests=$testsJson" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
|
|
"builds=$buildsJson" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
|
|
"batch_target_minutes=$targetMinutes" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
|
|
"test_entry_count=$($testEntries.Count)" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
|
|
"build_entry_count=$($buildEntries.Count)" | Out-File -FilePath $env:GITHUB_OUTPUT -Append
|
|
}
|
|
else {
|
|
Write-Output $testsJson
|
|
Write-Output $buildsJson
|
|
}
|