<# .SYNOPSIS Expands the GitHub Actions build and test workload for a CI scope. .DESCRIPTION Produces the job matrices consumed by .github/workflows/buildandtest.yml (pull-request scope) and .github/workflows/nightly.yml (full scope), plus a manifest that records every (project x profile x shard) workload the run is expected to cover. The workload is described once, here, as a table of profiles. A profile is a (runner OS, CustomTestTarget, test-host target framework, configuration, category filter, tier) tuple. Projects are discovered from the file system with a '*.Tests.csproj' sweep, so neither matrix has to be hand-maintained as projects are added or renamed. CustomTestTarget, not '--framework', is the mechanism that pins the stack to a single target framework (see targets.props). A profile carries both the CustomTestTarget it builds with and the framework its tests actually run on. The project fan-out is batched: each matrix entry carries several projects that the executor runs in sequence, keeping per-project results. Batches are sized by measured duration rather than by project count. Every batch pays a fixed cost of several minutes - checkout, SDK install, and the cold build of the shared dependency graph - while a further project in the same batch only adds its incremental build and its tests. Two projects per job spent almost half of the pull-request matrix compiling the same assemblies over and over, so projects are packed, in path order, into batches of about BatchTargetMinutes each. Path order keeps related projects (PubSub.*, Redundancy.*, WotCon.*) together, so their shared dependencies are built once per batch. GitHub Actions also refuses to start a workflow run whose matrices expand past 256 jobs, and that limit cannot be raised. If the packed matrix does not fit, the target is raised until it does, so the matrix adapts as test projects are added instead of silently truncating. .PARAMETER Scope 'pr' expands the workload that gates a pull request. 'full' expands the complete workload Azure Pipelines used to run on a schedule. .PARAMETER ExcludeMacOS Drop the macOS test profile. The batch target is deliberately computed from the full profile table regardless of this switch, so excluding macOS does not reshuffle the batches of the remaining profiles. .PARAMETER OnlyProject Restrict the test matrix to the single test project whose path ends with this value. Used by the protected private-fuzz-corpus job so it covers exactly the profiles this table already enables for that project, instead of repeating the profile list in a second place. The non-mainline tiers are dropped, since they cover named projects of their own. .PARAMETER MaxEntries The documented GitHub Actions ceiling on matrix jobs per workflow run. Exceeding it is an error, never a truncation. .PARAMETER ReservedEntries Entries reserved for the other full-scope jobs: discovery, four NativeAOT legs, eight crash-corpus replays, coverage, the summary, and one spare. .PARAMETER MaxJobTimeoutMinutes Upper bound on a batch job's 'timeout-minutes', below the 360-minute ceiling GitHub enforces on hosted runners. A batch whose per-project budgets add up past this is an error rather than a clamp: clamping would let the executor outlive the job, which is the one case that produces no annotation and no results. The packer therefore never puts more projects in one batch than this ceiling can budget for. .PARAMETER BatchTargetMinutes The measured duration, in minutes, a batch is packed up to, on top of its fixed cost. Weights come from DurationsPath and are the slowest pull-request profile's figures, so batches on the faster runners finish sooner. A project heavier than the target gets a batch of its own. Raised in steps of five when the packed matrix would not fit MaxEntries. .PARAMETER DurationsPath JSON table of per-project weights in minutes. Defaults to .github/ci-test-durations.json. A project missing from it is weighted with the table's defaultMinutes, so a new project is still scheduled; the table only steers balance, never what runs. Regenerate it with .github/scripts/update-test-durations.ps1. .PARAMETER RepositoryRoot Repository root. Defaults to the root two levels above this script. .PARAMETER ManifestPath Optional path to write the expanded workload manifest to. The workflows upload it so a run's intended coverage can be compared against another CI system's inventory. #> Param( [ValidateSet('pr', 'full')] [string] $Scope = 'pr', [switch] $ExcludeMacOS, [string] $OnlyProject = '', [int] $MaxEntries = 256, [int] $ReservedEntries = 16, [int] $MaxJobTimeoutMinutes = 350, [int] $BatchTargetMinutes = 30, [string] $DurationsPath = '', [string] $RepositoryRoot = '', [string] $ManifestPath = '' ) $ErrorActionPreference = 'Stop' if ([string]::IsNullOrWhiteSpace($RepositoryRoot)) { $RepositoryRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot '..' '..')).Path } else { $RepositoryRoot = (Resolve-Path -LiteralPath $RepositoryRoot).Path } if ([string]::IsNullOrWhiteSpace($DurationsPath)) { $DurationsPath = Join-Path $RepositoryRoot '.github' 'ci-test-durations.json' } if ($BatchTargetMinutes -lt 1) { throw "BatchTargetMinutes must be positive, not $BatchTargetMinutes." } # Runner images: GitHub-hosted labels. $RunnerImages = @{ windows = 'windows-latest' linux = 'ubuntu-latest' macos = 'macos-latest' } # Hosted runner architectures, recorded in the manifest so a comparison against # another CI system's inventory is not silently architecture-blind. $RunnerArchitectures = @{ windows = 'x64' linux = 'x64' macos = 'arm64' } $LatestSdk = '10.0.402' # The category filter the fast legs use. Tiers that lift it declare so below. $DefaultFilter = 'TestCategory!=LongRunning&TestCategory!=Stress' # Projects that are not part of the mainline sweep: # Aot - published and run as native executables rather than # by 'dotnet test'; the workflows have their own jobs. # Stress - opt-in tier owned by .github/workflows/stress-test.yml. # Subscriptions.Durable - its own tier, run unfiltered below. # OneFuzz.Validator - pins net10.0 to match the net10.0-only drop it # validates, so a legacy leg fails with NETSDK1005 # before the no-op shell can take effect. It is covered # by the fuzz-public-drop job instead. $MainlineExclusions = @( 'Opc.Ua.Aot.Tests.csproj', 'Opc.Ua.Stress.Tests.csproj', 'Opc.Ua.Subscriptions.Durable.Tests.csproj', 'Opc.Ua.OneFuzz.Validator.Tests.csproj') $DurableProject = 'Opc.Ua.Subscriptions.Durable.Tests.csproj' <# .SYNOPSIS Sorts strings with an ordinal comparison so the batches a run produces do not depend on the runner's culture. #> function Sort-Ordinal([string[]] $values) { $list = [System.Collections.Generic.List[string]]::new() foreach ($value in $values) { $list.Add($value) } $list.Sort([System.StringComparer]::Ordinal) return $list.ToArray() } <# .SYNOPSIS Converts an absolute path below the repository root into a forward-slashed repository-relative path. #> function ConvertTo-RelativePath([string] $fullName) { return $fullName.Substring($RepositoryRoot.Length).TrimStart('\', '/').Replace('\', '/') } <# .SYNOPSIS Discovers files below the repository root, skipping build output. #> function Find-RepositoryFile([string] $filter) { $found = Get-ChildItem -LiteralPath $RepositoryRoot -Recurse -File -Filter $filter | Where-Object { $_.FullName -notmatch '[\\/](bin|obj|\.git)[\\/]' } | ForEach-Object { ConvertTo-RelativePath $_.FullName } return Sort-Ordinal @($found) } <# .SYNOPSIS Returns the SDK versions a profile needs, newest last so the latest SDK drives the build while the older runtime is available to host its tests. #> function Get-SdkVersions([string] $framework) { switch ($framework) { 'net8.0' { return @('8.0.x', $LatestSdk) } 'net9.0' { return @('9.0.x', $LatestSdk) } default { return @($LatestSdk) } } } # The workload. Each profile fans out over the project set named by its tier. # # customTestTarget - passed as /p:CustomTestTarget for restore, build and test # framework - the target framework the tests actually execute on # scopes - which CI scopes include the profile # # The Linux net10.0 Release mainline profile is pull-request only on purpose: in # the full scope the long-running tier runs the identical tuple with the category # filter lifted, so keeping both would run the same tests twice. $Profiles = @( @{ id = 'windows-net48'; os = 'windows'; customTestTarget = 'net48'; framework = 'net48'; configuration = 'Release'; scopes = @('pr', 'full') } @{ id = 'windows-net10.0'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr', 'full') } @{ id = 'linux-net10.0'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr') } @{ id = 'macos-net10.0'; os = 'macos'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('pr', 'full') } @{ id = 'windows-net10.0-debug'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Debug'; scopes = @('full') } @{ id = 'linux-net10.0-debug'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Debug'; scopes = @('full') } @{ id = 'windows-net9.0'; os = 'windows'; customTestTarget = 'net9.0'; framework = 'net9.0'; configuration = 'Release'; scopes = @('full') } @{ id = 'linux-net9.0'; os = 'linux'; customTestTarget = 'net9.0'; framework = 'net9.0'; configuration = 'Release'; scopes = @('full') } @{ id = 'windows-net8.0'; os = 'windows'; customTestTarget = 'net8.0'; framework = 'net8.0'; configuration = 'Release'; scopes = @('full') } @{ id = 'linux-net8.0'; os = 'linux'; customTestTarget = 'net8.0'; framework = 'net8.0'; configuration = 'Release'; scopes = @('full') } @{ id = 'linux-long-running'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full') tier = 'long-running'; filter = ''; hangTimeout = '30m'; coverage = $false } @{ id = 'windows-durable'; os = 'windows'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full') tier = 'durable'; filter = ''; hangTimeout = '30m'; coverage = $false } @{ id = 'linux-durable'; os = 'linux'; customTestTarget = 'net10.0'; framework = 'net10.0'; configuration = 'Release'; scopes = @('full') tier = 'durable'; filter = ''; hangTimeout = '30m'; coverage = $false } ) # Solution builds. 'solutions' is either '*' for every discovered .slnx or an # explicit list. Every solution across every # supported target framework in both configurations for the full scope, narrowed # for pull requests, plus the Linux leg that proves UA.slnx still compiles for # every Linux-capable target framework. $BuildProfiles = @( @{ id = 'pr-windows'; os = 'windows'; scopes = @('pr'); solutions = '*' tfms = @('net48', 'net10.0'); configurations = @('Debug', 'Release') } @{ id = 'pr-linux'; os = 'linux'; scopes = @('pr'); solutions = @('UA.slnx') tfms = @('net8.0', 'net9.0', 'net10.0'); configurations = @('Release') } @{ id = 'full-windows'; os = 'windows'; scopes = @('full'); solutions = '*' tfms = @('net48', 'net8.0', 'net9.0', 'net10.0') configurations = @('Debug', 'Release') } @{ id = 'full-linux'; os = 'linux'; scopes = @('full'); solutions = @('UA.slnx') tfms = @('net8.0', 'net9.0', 'net10.0'); configurations = @('Release') } ) # Every batch carries this much on top of its packed weight: checkout, SDK # install, restore, and the cold build of the dependency graph its first # project pulls in. Run 37520359929 measured 3.8-4.9 minutes for that first # build against 0.4-0.9 minutes for each further project in the same batch. $BatchFixedMinutes = 6 # Test execution gets a consistent 45-minute ceiling for every project. The job # timeout adds a fixed 20-minute allowance for checkout, restore, and builds. # Keep 45 minutes for mainline too; projects whose test execution exceeds this # standard ceiling are split into filtered shards. $PerProjectTimeoutMinutes = 45 $DedicatedTestProjects = @('Opc.Ua.Server.Tests', 'Opc.Ua.WotCon.Tests', 'Opc.Ua.Types.Tests') # 20 minutes of fixed cost plus one per-project ceiling for each project has to # stay below MaxJobTimeoutMinutes, which caps how many projects one batch takes. $MaxProjectsPerBatch = [System.Math]::Floor(($MaxJobTimeoutMinutes - 20) / $PerProjectTimeoutMinutes) if ($MaxProjectsPerBatch -lt 1) { throw "MaxJobTimeoutMinutes=$MaxJobTimeoutMinutes cannot budget even one $PerProjectTimeoutMinutes-minute project." } if (-not (Test-Path -LiteralPath $DurationsPath -PathType Leaf)) { throw "The per-project duration table '$DurationsPath' does not exist." } $durationTable = Get-Content -LiteralPath $DurationsPath -Raw | ConvertFrom-Json -AsHashtable $DefaultProjectMinutes = [int]$durationTable.defaultMinutes if ($DefaultProjectMinutes -lt 1) { throw "'$DurationsPath' must declare a positive defaultMinutes." } $ProjectMinutes = @{} foreach ($entry in $durationTable.projects.GetEnumerator()) { if ([int]$entry.Value -lt 1) { throw "'$DurationsPath' gives '$($entry.Key)' a weight of $($entry.Value); every weight must be at least one minute." } $ProjectMinutes[$entry.Key] = [int]$entry.Value } <# .SYNOPSIS Returns the packing weight of a project, in minutes. #> function Get-ProjectMinutes([string] $project) { $stem = [System.IO.Path]::GetFileNameWithoutExtension($project) if ($ProjectMinutes.ContainsKey($stem)) { return $ProjectMinutes[$stem] } return $DefaultProjectMinutes } $allTestProjects = Find-RepositoryFile '*.Tests.csproj' $mainlineProjects = @($allTestProjects | Where-Object { $MainlineExclusions -notcontains (Split-Path $_ -Leaf) }) $durableProjects = @($allTestProjects | Where-Object { (Split-Path $_ -Leaf) -eq $DurableProject }) $solutions = Find-RepositoryFile '*.slnx' if ($mainlineProjects.Count -eq 0) { throw "Discovery found no mainline test projects beneath '$RepositoryRoot'. A matrix that runs nothing would report success without testing anything." } if ($durableProjects.Count -eq 0) { throw "Discovery found no '$DurableProject'. The durable-subscription tier would silently disappear." } if ($solutions.Count -eq 0) { throw "Discovery found no '*.slnx' beneath '$RepositoryRoot'." } if (-not [string]::IsNullOrWhiteSpace($OnlyProject)) { $normalized = $OnlyProject.Replace('\', '/') $mainlineProjects = @($mainlineProjects | Where-Object { $_.EndsWith($normalized, [System.StringComparison]::OrdinalIgnoreCase) }) if ($mainlineProjects.Count -eq 0) { throw "No discovered test project path ends with '$OnlyProject'. A matrix that runs nothing would report success without testing anything." } # The named tiers cover named projects, so narrowing to one project leaves # them with nothing to run rather than silently re-running it unfiltered. $durableProjects = @() } $projectsByTier = @{ mainline = $mainlineProjects 'long-running' = if ([string]::IsNullOrWhiteSpace($OnlyProject)) { $mainlineProjects } else { @() } durable = $durableProjects } <# .SYNOPSIS Expands the solution build matrix for a scope. #> function Expand-BuildMatrix([string] $scope) { $entries = @() foreach ($buildProfile in $BuildProfiles) { if ($buildProfile.scopes -notcontains $scope) { continue } $profileSolutions = $buildProfile.solutions if ($profileSolutions -eq '*') { $profileSolutions = $solutions } foreach ($solution in $profileSolutions) { if ($solutions -notcontains $solution) { throw "Build profile '$($buildProfile.id)' names solution '$solution', which was not discovered." } $stem = [System.IO.Path]::GetFileNameWithoutExtension($solution) foreach ($tfm in $buildProfile.tfms) { foreach ($configuration in $buildProfile.configurations) { $entries += [ordered]@{ id = "$($buildProfile.id)-$stem-$tfm-$configuration" name = "$stem $tfm $configuration ($($buildProfile.os))" os = $buildProfile.os runsOn = $RunnerImages[$buildProfile.os] arch = $RunnerArchitectures[$buildProfile.os] solution = $solution customTestTarget = $tfm configuration = $configuration dotnet = (Get-SdkVersions $tfm) -join "`n" } } } } } return $entries } <# .SYNOPSIS Packs an ordered project list into contiguous batches whose weights add up to at most $targetMinutes, never more than $MaxProjectsPerBatch projects. .DESCRIPTION Contiguous rather than best-fit: path order keeps projects that share dependencies together, and the result only depends on the list and the table, so a run is reproducible. A project heavier than the target is placed in a batch of its own rather than split or dropped. #> function Split-IntoBatches([string[]] $projects, [int] $targetMinutes) { $batches = @() $current = @() $currentMinutes = 0 foreach ($project in $projects) { $minutes = Get-ProjectMinutes $project $stem = [System.IO.Path]::GetFileNameWithoutExtension($project) if ($stem -eq 'Opc.Ua.WotCon.Samples.Tests' -or $DedicatedTestProjects -contains $stem) { if ($current.Count -gt 0) { $batches += , @($current) $current = @() $currentMinutes = 0 } $batches += , @($project) continue } if ($current.Count -gt 0 -and ($currentMinutes + $minutes -gt $targetMinutes -or $current.Count -ge $MaxProjectsPerBatch)) { $batches += , @($current) $current = @() $currentMinutes = 0 } $current += $project $currentMinutes += $minutes } if ($current.Count -gt 0) { $batches += , @($current) } return $batches } <# .SYNOPSIS Returns exhaustive, non-overlapping filters for projects whose full test suite exceeds the per-project execution budget. #> function Get-TestProjectShards([string] $project, [string] $framework) { $stem = [System.IO.Path]::GetFileNameWithoutExtension($project) switch ($stem) { 'Opc.Ua.WotCon.Samples.Tests' { if ($framework -eq 'net48') { return @( @{ id = 'all' name = 'all' filter = '' hangTimeout = '' } ) } $prefix = 'Opc.Ua.WotCon.Samples.Tests.' $generationScenario = "${prefix}WotSampleEndToEndTests.RealSamplesAggregateSubscribeAndReplaceGenerationAsync" $pumpComparison = "${prefix}WotPumpAddressSpaceComparisonTests.WotPumpInstanceMatchesNativePumpSubsetAsync" $nativeTypes = "${prefix}WotPumpAddressSpaceComparisonTests.CompanionTypeDefinitionsMatchNativePumpServerAsync" return @( @{ id = 'sample-core' name = 'sample-core' filter = @( "FullyQualifiedName!~$generationScenario" "FullyQualifiedName!~$pumpComparison" "FullyQualifiedName!~$nativeTypes" ) -join '&' hangTimeout = '' } @{ id = 'sample-generation' name = 'sample-generation' filter = "FullyQualifiedName~$generationScenario" hangTimeout = '45m' } @{ id = 'sample-native-comparisons' name = 'sample-native-comparisons' filter = "(FullyQualifiedName~$pumpComparison|FullyQualifiedName~$nativeTypes)" hangTimeout = '30m' } ) } 'Opc.Ua.WotCon.Tests' { $materialization = 'Opc.Ua.WotCon.Tests.Materialization.' if ($framework -ne 'net48') { return @( @{ id = 'wot-materialization' name = 'wot-materialization' filter = "FullyQualifiedName~${materialization}" hangTimeout = '' } @{ id = 'wot-other' name = 'wot-other' filter = "FullyQualifiedName!~${materialization}" hangTimeout = '' } ) } return @( @{ id = 'wot-materialization-core' name = 'wot-materialization-core' filter = "FullyQualifiedName~${materialization}&TestCategory!=Integration" hangTimeout = '' } @{ id = 'wot-materialization-integration' name = 'wot-materialization-integration' filter = "FullyQualifiedName~${materialization}&TestCategory=Integration" hangTimeout = '' } @{ id = 'wot-other-core' name = 'wot-other-core' filter = "FullyQualifiedName!~${materialization}&TestCategory!=Integration" hangTimeout = '' } @{ id = 'wot-other-integration' name = 'wot-other-integration' filter = "FullyQualifiedName!~${materialization}&TestCategory=Integration" hangTimeout = '' } ) } 'Opc.Ua.Server.Tests' { if ($framework -ne 'net48') { return @( @{ id = 'all' name = 'all' filter = '' hangTimeout = '' } ) } $groups = @( @{ id = 'server-node-management' name = 'server-node-management' terms = @('NodeManager', 'SamplingGroup', 'SamplingInterval', 'NodeState', 'MonitoredNode') } @{ id = 'server-historian' name = 'server-historian' terms = @('Historian', 'Aggregate', 'History') } @{ id = 'server-subscriptions' name = 'server-subscriptions' terms = @('Subscription', 'MonitoredItem', 'Publish', 'KeepAlive') } @{ id = 'server-sessions' name = 'server-sessions' terms = @('Session', 'Identity', 'UserManagement', 'UserDatabase', 'Role', 'Authentication') } @{ id = 'server-configuration' name = 'server-configuration' terms = @('Configuration', 'Certificate', 'TrustList', 'KeyCredential', 'Security') } @{ id = 'server-runtime' name = 'server-runtime' terms = @('Fluent', 'Hosting', 'RuntimeNodeSet', 'NodeSet', 'StateMachine', 'ComplexType', 'FileSystem', 'AliasNames', 'Program') } @{ id = 'server-transport' name = 'server-transport' terms = @('NativeTcp', 'ReverseConnect', 'Transport', 'ReferenceServer', 'Discovery', 'Quickstart') } @{ id = 'server-other' name = 'server-other' terms = @() } ) $assignedTerms = @() foreach ($group in $groups) { $filterTerms = @( $assignedTerms | ForEach-Object { "FullyQualifiedName!~$_" } ) if ($group.terms.Count -gt 0) { $groupFilter = @( $group.terms | ForEach-Object { "FullyQualifiedName~$_" } ) -join '|' $filterTerms += "($groupFilter)" } $group.filter = $filterTerms -join '&' $group.hangTimeout = '' $assignedTerms += $group.terms } return $groups } default { return @( @{ id = 'all' name = 'all' filter = '' hangTimeout = '' } ) } } } <# .SYNOPSIS Counts the test matrix entries a batch target produces for a scope, ignoring which optional runners are enabled so the target is stable. #> function Measure-TestEntries([string] $scope, [int] $targetMinutes) { $total = 0 foreach ($testProfile in $Profiles) { if ($testProfile.scopes -notcontains $scope) { continue } $tier = 'mainline' if ($testProfile.ContainsKey('tier')) { $tier = $testProfile.tier } foreach ($batch in @(Split-IntoBatches $projectsByTier[$tier] $targetMinutes)) { if ($batch.Count -eq 1) { $total += @(Get-TestProjectShards $batch[0] $testProfile.framework).Count } else { $total++ } } } return [int]$total } <# .SYNOPSIS Expands the test matrix and the per-tuple manifest for a scope. #> function Expand-TestMatrix([string] $scope, [int] $targetMinutes, [bool] $includeMacOS) { $entries = @() $manifest = @() foreach ($testProfile in $Profiles) { if ($testProfile.scopes -notcontains $scope) { continue } if ($testProfile.os -eq 'macos' -and -not $includeMacOS) { continue } $tier = 'mainline' if ($testProfile.ContainsKey('tier')) { $tier = $testProfile.tier } $filter = $DefaultFilter if ($testProfile.ContainsKey('filter')) { $filter = $testProfile.filter } $hangTimeout = '10m' if ($testProfile.ContainsKey('hangTimeout')) { $hangTimeout = $testProfile.hangTimeout } # coverlet.collector 10.x ships build assets for net8.0 and newer only, so # a .NET Framework test host cannot load the 'XPlat Code Coverage' # collector at all - VSTest only warns and writes no report. The # long-running and durable tiers opt out explicitly: # they re-run projects the filtered legs already covered, so folding their # numbers into the merged report would double-count them. $coverage = -not $testProfile.framework.StartsWith('net4') if ($testProfile.ContainsKey('coverage')) { $coverage = [bool]$testProfile.coverage } # Each project gets one test-execution ceiling; the job timeout also # includes fixed setup/build overhead. $perProjectTimeout = $PerProjectTimeoutMinutes $batches = @(Split-IntoBatches $projectsByTier[$tier] $targetMinutes) for ($index = 0; $index -lt $batches.Count; $index++) { $batch = $batches[$index] $batchHangTimeout = $hangTimeout $shards = @(@{ id = 'all' name = 'all' filter = '' hangTimeout = '' }) if ($batch.Count -eq 1) { $shards = @(Get-TestProjectShards $batch[0] $testProfile.framework) } foreach ($shard in $shards) { $batchId = "$($testProfile.id)-$(($index + 1).ToString('00'))-$($shard.id)" $batchTimeout = $perProjectTimeout $batchFilter = $filter if (-not [string]::IsNullOrWhiteSpace($shard.filter)) { if ([string]::IsNullOrWhiteSpace($batchFilter)) { $batchFilter = $shard.filter } else { $batchFilter = "$batchFilter&$($shard.filter)" } } $shardHangTimeout = $batchHangTimeout if (-not [string]::IsNullOrWhiteSpace($shard.hangTimeout)) { $shardHangTimeout = $shard.hangTimeout } # Twenty minutes of fixed cost - checkout, SDK install, restore, # and builds - plus the test-execution ceiling for every project # in the batch. This leaves each test its full budget while # keeping the job timeout a true upper bound. $jobTimeout = 20 + ($batch.Count * $batchTimeout) if ($jobTimeout -gt $MaxJobTimeoutMinutes) { throw ("Batch '$batchId' of $($batch.Count) projects needs $jobTimeout minutes, past the " + "$MaxJobTimeoutMinutes-minute ceiling. Clamping it would let the executor outlive the " + 'job, so lower the per-project ceiling or raise ReservedEntries so smaller batches fit.') } $name = "$($testProfile.id) ($($index + 1)/$($batches.Count))" if ($shard.id -ne 'all') { $name = "$name - $($shard.name)" } $entries += [ordered]@{ id = $batchId name = $name os = $testProfile.os runsOn = $RunnerImages[$testProfile.os] arch = $RunnerArchitectures[$testProfile.os] profile = $testProfile.id tier = $tier shard = $shard.name customTestTarget = $testProfile.customTestTarget framework = $testProfile.framework configuration = $testProfile.configuration filter = $batchFilter hangTimeout = $shardHangTimeout coverage = $coverage projects = $batch -join ';' perProjectTimeout = $batchTimeout timeoutMinutes = $jobTimeout # The batch's summed weight from the duration table: what the # packer balanced, on top of the fixed per-batch cost. packedMinutes = [int]($batch | ForEach-Object { Get-ProjectMinutes $_ } | Measure-Object -Sum).Sum dotnet = (Get-SdkVersions $testProfile.framework) -join "`n" } foreach ($project in $batch) { # Every automatic entry uses the corpus checked into the # repository. The private crash corpus is only ever replayed by # the protected job in nightly.yml. $corpus = 'none' if ($project -like 'fuzzing/*') { $corpus = 'public-checked-in' } $manifest += [ordered]@{ project = $project profile = $testProfile.id tier = $tier os = $testProfile.os runsOn = $RunnerImages[$testProfile.os] arch = $RunnerArchitectures[$testProfile.os] customTestTarget = $testProfile.customTestTarget framework = $testProfile.framework configuration = $testProfile.configuration filter = $batchFilter shard = $shard.name coverage = $coverage corpus = $corpus batch = $batchId } } } } } return @{ entries = $entries; manifest = $manifest } } $buildEntries = @(Expand-BuildMatrix $Scope) $budget = $MaxEntries - $buildEntries.Count - $ReservedEntries if ($budget -lt 1) { throw "The '$Scope' solution build matrix alone needs $($buildEntries.Count) of the $MaxEntries entries GitHub Actions allows per workflow run, leaving no room for tests." } # Raising the target shrinks the matrix until it fits. Once every batch is # already at the per-batch project cap, raising it further changes nothing, so # that is reported rather than looped on. $heaviestProject = (@($ProjectMinutes.Values) + $DefaultProjectMinutes | Measure-Object -Maximum).Maximum $targetCeiling = $MaxProjectsPerBatch * $heaviestProject $targetMinutes = $BatchTargetMinutes while ((Measure-TestEntries $Scope $targetMinutes) -gt $budget) { if ($targetMinutes -ge $targetCeiling) { throw ("No batch target fits the '$Scope' test workload into $budget matrix entries; " + "$MaxProjectsPerBatch projects per batch is the most MaxJobTimeoutMinutes allows.") } $targetMinutes += 5 } $expanded = Expand-TestMatrix $Scope $targetMinutes (-not $ExcludeMacOS) $testEntries = @($expanded.entries) $totalEntries = $testEntries.Count + $buildEntries.Count $largestBatch = ($testEntries | ForEach-Object { @($_.projects -split ';').Count } | Measure-Object -Maximum).Maximum Write-Host "Scope: $Scope" Write-Host "Repository root: $RepositoryRoot" Write-Host "Mainline projects: $($mainlineProjects.Count)" Write-Host "Solutions: $($solutions -join ', ')" Write-Host "Batch target: $targetMinutes min (+$BatchFixedMinutes fixed), at most $MaxProjectsPerBatch projects" Write-Host "Largest batch: $largestBatch projects" Write-Host "Test entries: $($testEntries.Count)" Write-Host "Build entries: $($buildEntries.Count)" Write-Host "Reserved entries: $ReservedEntries" Write-Host "Total matrix jobs: $totalEntries (limit $MaxEntries)" if ($totalEntries -gt $MaxEntries) { throw "The '$Scope' workload expands to $totalEntries matrix jobs, past the $MaxEntries GitHub Actions allows per workflow run. Raise BatchTargetMinutes so fewer, larger batches are chosen rather than dropping entries." } if (-not [string]::IsNullOrWhiteSpace($ManifestPath)) { $manifestDirectory = Split-Path -Parent $ManifestPath if (-not [string]::IsNullOrWhiteSpace($manifestDirectory) -and -not (Test-Path -LiteralPath $manifestDirectory)) { $null = New-Item -ItemType Directory -Path $manifestDirectory -Force } $document = [ordered]@{ scope = $Scope batchTargetMinutes = $targetMinutes includeMacOS = (-not $ExcludeMacOS) builds = $buildEntries tests = @($expanded.manifest) } $document | ConvertTo-Json -Depth 6 | Out-File -LiteralPath $ManifestPath -Encoding utf8 Write-Host "Wrote the workload manifest to '$ManifestPath'." } $testsJson = ConvertTo-Json -InputObject $testEntries -Depth 5 -Compress $buildsJson = ConvertTo-Json -InputObject $buildEntries -Depth 5 -Compress if (-not [string]::IsNullOrWhiteSpace($env:GITHUB_OUTPUT)) { "tests=$testsJson" | Out-File -FilePath $env:GITHUB_OUTPUT -Append "builds=$buildsJson" | Out-File -FilePath $env:GITHUB_OUTPUT -Append "batch_target_minutes=$targetMinutes" | Out-File -FilePath $env:GITHUB_OUTPUT -Append "test_entry_count=$($testEntries.Count)" | Out-File -FilePath $env:GITHUB_OUTPUT -Append "build_entry_count=$($buildEntries.Count)" | Out-File -FilePath $env:GITHUB_OUTPUT -Append } else { Write-Output $testsJson Write-Output $buildsJson }