Run Integration Tests Skill

Build the MAUI product, install local workloads, and run integration tests.

When to Use

• User asks to "run integration tests"
• User asks to "test templates locally"
• User asks to "validate MAUI build with templates"
• User wants to verify changes don't break template scenarios
• User asks to run specific test categories (WindowsTemplates, Samples, Build, Blazor, etc.)

Available Test Categories

Category	Platform	Description
Build	All	Basic template build tests
WindowsTemplates	Windows	Windows-specific template scenarios
macOSTemplates	macOS	macOS-specific scenarios
Blazor	All	Blazor hybrid templates
MultiProject	All	Multi-project templates
Samples	All	Sample project builds
AOT	macOS	Native AOT compilation
RunOnAndroid	macOS	Build, install, run on Android emulator
RunOniOS	macOS	iOS simulator tests

Scripts

All scripts are in .github/skills/run-integration-tests/scripts/

Run Integration Tests (Full Workflow)

# Run with specific category
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "WindowsTemplates"

# Run with Release configuration
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Samples" -Configuration "Release"

# Run with custom test filter
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "FullyQualifiedName~BuildSample"

# Skip build step (if already built)
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -SkipBuild

# macOS: Skip Xcode version check (for version mismatches)
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "macOSTemplates" -SkipBuild -SkipInstall -SkipXcodeVersionCheck

# Auto-provision SDK if not found (first-time setup)
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -AutoProvision

Parameters

Parameter	Required	Default	Description
-Category	No	-	Test category to run (WindowsTemplates, Samples, Build, etc.)
-TestFilter	No	-	Custom NUnit test filter expression
-Configuration	No	Debug	Build configuration (Debug/Release)
-SkipBuild	No	false	Skip build/pack step if already done
-SkipInstall	No	false	Skip workload installation if already done
-SkipXcodeVersionCheck	No	false	Skip Xcode version validation (macOS)
-AutoProvision	No	false	Automatically provision local SDK if not found
-ResultsDirectory	No	artifacts/integration-tests	Directory for test results

Workflow Steps

The script performs these steps:

1. Build & Pack: .\build.cmd -restore -pack -configuration $Configuration
2. Install Workloads: .dotnet\dotnet build .\src\DotNet\DotNet.csproj -t:Install -c $Configuration
3. Extract Version: Reads MAUI_PACKAGE_VERSION from installed packs
4. Run Tests: .dotnet\dotnet test ... -filter "Category=$Category"

Example Usage

# Run WindowsTemplates tests
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "WindowsTemplates"

# Run Samples tests
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Samples"

# Run multiple categories
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "Category=Build|Category=Blazor"

Prerequisites

• Windows for WindowsTemplates, macOS for macOSTemplates/RunOniOS/RunOnAndroid
• .NET SDK (version from global.json)
• Sufficient disk space for build artifacts
• Local SDK and workloads must be provisioned first

Provisioning the Local SDK (Required First Time)

Before running integration tests, you must provision the local .NET SDK and MAUI workloads:

# Step 1: Restore dotnet tools
dotnet tool restore

# Step 2: Provision local SDK and install workloads (~5 minutes)
dotnet cake --target=dotnet

# Step 3: Install MAUI local workloads
dotnet cake --target=dotnet-local-workloads

Verification:

# Check SDK exists
ls .dotnet/dotnet

# Check MAUI SDK version
ls .dotnet/packs/Microsoft.Maui.Sdk

Note: The old ./build.sh --target=dotnet syntax no longer works. Use dotnet cake directly.

Output

• Test results in TRX format at <ResultsDirectory>/
• Build logs in artifacts/ directory
• Console output with test pass/fail summary

Troubleshooting

Issue	Solution
"MAUI_PACKAGE_VERSION was not set"	Ensure build step completed successfully
"Local .dotnet SDK not found"	Run dotnet tool restore && dotnet cake --target=dotnet && dotnet cake --target=dotnet-local-workloads
Template not found	Workload installation may have failed
Build failures	Check artifacts/log/ for detailed build logs
"Cannot proceed with locked .dotnet folder"	Kill processes using .dotnet: Get-Process | Where-Object { $_.Path -like "*\.dotnet\*" } | ForEach-Object { Stop-Process -Id $_.Id -Force }
Session times out / becomes invalid	Integration tests are long-running (15-60+ min). Run manually in a terminal window instead of via Copilot CLI
Tests take too long	Start with Build category (fastest), then run others. Use -SkipBuild -SkipInstall if workloads are already installed
iOS tests fail with "mlaunch exited with 1"	Simulator state issue. Run individual tests instead of the whole category (see below)
iOS simulator state errors (code 137/149)	Reset simulator: xcrun simctl shutdown all && xcrun simctl erase all or run tests individually

Running Manually (Recommended for Long-Running Tests)

Integration tests can take 15-60+ minutes depending on the category. For best results, run directly in a terminal:

cd D:\repos\dotnet\maui

# Option 1: Use the skill script
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -SkipBuild -SkipInstall

# Option 2: Run dotnet test directly (if workloads already installed)
$env:MAUI_PACKAGE_VERSION = (Get-ChildItem .dotnet\packs\Microsoft.Maui.Sdk -Directory | Sort-Object Name -Descending | Select-Object -First 1).Name
.dotnet\dotnet test src\TestUtils\src\Microsoft.Maui.IntegrationTests --filter "Category=Build"

Running All Categories Sequentially

# Windows categories (run on Windows)
$categories = @("Build", "WindowsTemplates", "Blazor", "MultiProject", "Samples")
foreach ($cat in $categories) {
    Write-Host "Running $cat..." -ForegroundColor Cyan
    pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category $cat -SkipBuild -SkipInstall
}

Running Individual iOS Tests (Recommended)

Running all iOS tests together (-Category "RunOniOS") can cause simulator state issues. For better reliability, run tests individually:

# Available iOS tests
$iosTests = @(
    "RunOniOS_MauiDebug",
    "RunOniOS_MauiRelease",
    "RunOniOS_MauiReleaseTrimFull",
    "RunOniOS_BlazorDebug",
    "RunOniOS_BlazorRelease",
    "RunOniOS_MauiNativeAOT"
)

# Run a specific iOS test
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "FullyQualifiedName~RunOniOS_MauiDebug" -SkipBuild -SkipInstall -SkipXcodeVersionCheck

# Run all iOS tests individually (more reliable than running category)
foreach ($test in $iosTests) {
    Write-Host "Running $test..." -ForegroundColor Cyan
    pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "FullyQualifiedName~$test" -SkipBuild -SkipInstall -SkipXcodeVersionCheck
}