deployment-e2e-testing

5
6
Source

Guide for writing Aspire deployment end-to-end tests. Use this when asked to create, modify, or debug deployment E2E tests that deploy to Azure.

Install

mkdir -p .claude/skills/deployment-e2e-testing && curl -L -o skill.zip "https://mcp.directory/api/skills/download/5258" && unzip -o skill.zip -d .claude/skills/deployment-e2e-testing && rm skill.zip

Installs to .claude/skills/deployment-e2e-testing

About this skill

Aspire Deployment End-to-End Testing

This skill provides patterns and practices for writing end-to-end tests that deploy Aspire applications to real Azure infrastructure.

Overview

Deployment E2E tests extend the CLI E2E testing patterns with actual Azure deployments. They use the Hex1b terminal automation library to drive the Aspire CLI and verify that deployed applications work correctly.

Location: tests/Aspire.Deployment.EndToEnd.Tests/

Supported Platforms: Linux only (Hex1b requirement).

Prerequisites:

  • Azure subscription with appropriate permissions
  • OIDC authentication (CI) or Azure CLI authentication (local)

Relationship to CLI E2E Tests

Deployment tests build on the CLI E2E testing skill. Before working with deployment tests, familiarize yourself with:

Key differences from CLI E2E tests:

AspectCLI E2E TestsDeployment E2E Tests
Duration5-15 minutes15-45 minutes
ResourcesLocal onlyAzure resources
AuthenticationNoneAzure OIDC/CLI
CleanupTemp directoriesAzure resource groups
TriggersPR, pushNightly, manual, deploy-test/*

Key Components

Core Classes

  • DeploymentE2ETestHelpers (Helpers/DeploymentE2ETestHelpers.cs): Terminal factory and environment helpers
  • DeploymentE2EAutomatorHelpers (Helpers/DeploymentE2EAutomatorHelpers.cs): Async extension methods on Hex1bTerminalAutomator for deployment scenarios
  • Hex1bAutomatorTestHelpers (shared): Common async extension methods on Hex1bTerminalAutomator (WaitForSuccessPromptAsync, AspireNewAsync, etc.)
  • AzureAuthenticationHelpers (Helpers/AzureAuthenticationHelpers.cs): Azure auth and resource naming
  • DeploymentReporter (Helpers/DeploymentReporter.cs): GitHub step summary reporting
  • SequenceCounter (Helpers/SequenceCounter.cs): Prompt tracking (same as CLI E2E)

Test Architecture

Each deployment test:

  1. Validates Azure authentication (skip if not available locally)
  2. Generates a unique resource group name
  3. Creates a project using aspire new
  4. Deploys using aspire deploy
  5. Verifies deployed endpoints work
  6. Reports results to GitHub step summary
  7. Cleans up Azure resources

Test Structure

public sealed class MyDeploymentTests(ITestOutputHelper output)
{
    [Fact]
    public async Task DeployMyScenario()
    {
        // 1. Validate prerequisites
        var subscriptionId = AzureAuthenticationHelpers.TryGetSubscriptionId();
        if (string.IsNullOrEmpty(subscriptionId))
        {
            Assert.Skip("Azure subscription not configured.");
        }

        if (!AzureAuthenticationHelpers.IsAzureAuthAvailable())
        {
            if (DeploymentE2ETestHelpers.IsRunningInCI)
            {
                Assert.Fail("Azure auth not available in CI.");
            }
            Assert.Skip("Azure auth not available. Run 'az login'.");
        }

        // 2. Setup
        var resourceGroupName = AzureAuthenticationHelpers.GenerateResourceGroupName("my-scenario");
        var workspace = TemporaryWorkspace.Create(output);
        var startTime = DateTime.UtcNow;

        try
        {
            // 3. Build terminal and run deployment
            using var terminal = DeploymentE2ETestHelpers.CreateTestTerminal();
            var pendingRun = terminal.RunAsync(TestContext.Current.CancellationToken);

            var counter = new SequenceCounter();
            var auto = new Hex1bTerminalAutomator(terminal, defaultTimeout: TimeSpan.FromSeconds(500));

            await auto.PrepareEnvironmentAsync(workspace, counter);
            if (DeploymentE2ETestHelpers.IsRunningInCI)
            {
                await auto.SourceAspireCliEnvironmentAsync(counter);
            }
            await auto.AspireNewAsync("MyProject", counter, useRedisCache: false);
            // ... deployment steps ...

            await auto.TypeAsync("exit");
            await auto.EnterAsync();
            await pendingRun;

            // 4. Report success
            var duration = DateTime.UtcNow - startTime;
            DeploymentReporter.ReportDeploymentSuccess(
                nameof(DeployMyScenario),
                resourceGroupName,
                deploymentUrls,
                duration);
        }
        catch (Exception ex)
        {
            DeploymentReporter.ReportDeploymentFailure(
                nameof(DeployMyScenario),
                resourceGroupName,
                ex.Message);
            throw;
        }
        finally
        {
            // 5. Cleanup Azure resources
            await CleanupResourceGroupAsync(resourceGroupName);
        }
    }
}

Extension Methods

DeploymentE2EAutomatorHelpers Extensions on Hex1bTerminalAutomator

MethodDescription
PrepareEnvironmentAsync(workspace, counter)Sets up the terminal environment with custom prompt and workspace directory
InstallAspireCliFromPullRequestAsync(prNumber, counter)Downloads and installs the Aspire CLI from a PR build artifact
InstallAspireCliReleaseAsync(counter)Installs the latest released Aspire CLI
SourceAspireCliEnvironmentAsync(counter)Adds ~/.aspire/bin to PATH so the aspire command is available

These extend Hex1bTerminalAutomator and are used alongside the shared Hex1bAutomatorTestHelpers methods (WaitForSuccessPromptAsync, AspireNewAsync, etc.) documented in the CLI E2E Testing Skill.

Azure Authentication

In CI (GitHub Actions)

Tests use OIDC (Workload Identity Federation) for authentication:

- name: Azure Login (OIDC)
  uses: azure/login@v2
  with:
    client-id: ${{ secrets.AZURE_CLIENT_ID }}
    tenant-id: ${{ secrets.AZURE_TENANT_ID }}
    subscription-id: ${{ vars.ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTION }}

The test code automatically detects CI and uses DefaultAzureCredential which picks up the OIDC session.

Local Development

Authenticate with Azure CLI before running tests:

# Login to Azure
az login

# Set your subscription
az account set --subscription "your-subscription-id"

# Set environment variable
export ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTION="your-subscription-id"

# Run tests
dotnet test tests/Aspire.Deployment.EndToEnd.Tests/

Authentication Helpers

// Check if auth is available
if (!AzureAuthenticationHelpers.IsAzureAuthAvailable())
{
    Assert.Skip("Azure auth not available");
}

// Get subscription ID
var subscriptionId = AzureAuthenticationHelpers.GetSubscriptionId();

// Generate unique resource group name
var rgName = AzureAuthenticationHelpers.GenerateResourceGroupName("my-test");
// Result: "aspire-e2e-my-test-20240115-abc12345"

// Check auth type
if (AzureAuthenticationHelpers.IsOidcConfigured())
{
    // Using OIDC (CI)
}
else
{
    // Using Azure CLI (local)
}

Resource Group Naming

Resource groups are named with a consistent pattern for easy identification and cleanup:

{prefix}-{testname}-{date}-{runid}

Example: aspire-e2e-aca-starter-20240115-12345678

Components:

  • prefix: From ASPIRE_DEPLOYMENT_TEST_RG_PREFIX (default: aspire-e2e)
  • testname: Sanitized test name (lowercase, alphanumeric, hyphens)
  • date: UTC date in YYYYMMDD format
  • runid: GitHub run ID or random GUID suffix

Reporting Results

GitHub Step Summary

Tests automatically write to the GitHub step summary:

// Report success with URLs
DeploymentReporter.ReportDeploymentSuccess(
    testName: "DeployStarterToACA",
    resourceGroupName: "aspire-e2e-...",
    deploymentUrls: new Dictionary<string, string>
    {
        ["Dashboard"] = "https://dashboard.azurecontainerapps.io",
        ["Web Frontend"] = "https://webfrontend.azurecontainerapps.io"
    },
    duration: TimeSpan.FromMinutes(15));

// Report failure
DeploymentReporter.ReportDeploymentFailure(
    testName: "DeployStarterToACA",
    resourceGroupName: "aspire-e2e-...",
    errorMessage: "Deployment timed out",
    logs: "Full deployment logs...");

Asciinema Recordings

Tests generate recordings for debugging:

var recordingPath = DeploymentE2ETestHelpers.GetTestResultsRecordingPath(nameof(MyTest));
// CI: $GITHUB_WORKSPACE/testresults/recordings/MyTest.cast
// Local: /tmp/aspire-deployment-e2e/recordings/MyTest.cast

Cleanup

Always cleanup Azure resources in a finally block:

private static async Task CleanupResourceGroupAsync(string resourceGroupName)
{
    var process = new Process
    {
        StartInfo = new ProcessStartInfo
        {
            FileName = "az",
            Arguments = $"group delete --name {resourceGroupName} --yes --no-wait",
            RedirectStandardOutput = true,
            RedirectStandardError = true,
            UseShellExecute = false
        }
    };

    process.Start();
    await process.WaitForExitAsync();

    if (process.ExitCode != 0)
    {
        var error = await process.StandardError.ReadToEndAsync();
        throw new InvalidOperationException($"Cleanup failed: {error}");
    }
}

Workflow Triggers

Deployment tests are triggered by:

  1. Nightly schedule (03:00 UTC) - Runs on main
  2. Manual dispatch - Via GitHub Actions UI
  3. Push to deploy-test/* - For rapid iteration

Iterating on Tests

To iterate quickly during development:

# Create a protected branch
git checkout -b deploy-test/my-feature

# Make changes
# ...

# Push to trigger workflow
git push origin deploy-test/my-feature

Environment Variables

VariableRequiredDescription
ASPIRE_DEPLOYMENT_TEST_SUBSCRIPTIONYesAzure subscription ID
ASPIRE_DEPLOYMENT_TEST_RG_PREFIXNoResource group prefix (default: aspire-e2e)
`AZUR

Content truncated.

run-device-tests

dotnet

Build and run .NET MAUI device tests locally with category filtering. Supports iOS, MacCatalyst, Android on macOS; Android, Windows on Windows. Use TestFilter to run specific test categories.

376

performance-benchmark

dotnet

Generate and run ad hoc performance benchmarks to validate code changes. Use this when asked to benchmark, profile, or validate the performance impact of a code change in dotnet/runtime.

385

find-reviewable-pr

dotnet

Finds open PRs in the dotnet/maui and dotnet/docs-maui repositories that are good candidates for review, prioritizing by milestone, priority labels, partner/community status.

235

update-roslyn-version

dotnet

Guide for updating the Roslyn language server version in the vscode-csharp repository. Use this when asked to update Roslyn, bump the Roslyn version, or upgrade the language server version.

194

hypothesis-driven-debugging

dotnet

Investigate compiler failures, test errors, or unexpected behavior through systematic minimal reproduction, 3-hypothesis testing, and verification. Always re-run builds and tests after changes.

82

pr-finalize

dotnet

Finalizes any PR for merge by verifying title/description match implementation AND performing code review for best practices. Use when asked to "finalize PR", "check PR description", "review commit message", before merging any PR, or when PR implementation changed during review. Do NOT use for extracting lessons (use learn-from-pr), writing tests (use write-tests-agent), or investigating build failures (use pr-build-status).

102

You might also like

nano-banana-pro

garg-aayush

Generate and edit images using Google's Nano Banana Pro (Gemini 3 Pro Image) API. Use when the user asks to generate, create, edit, modify, change, alter, or update images. Also use when user references an existing image file and asks to modify it in any way (e.g., "modify this image", "change the background", "replace X with Y"). Supports both text-to-image generation and image-to-image editing with configurable resolution (1K default, 2K, or 4K for high resolution). DO NOT read the image file first - use this skill directly with the --input-image parameter.

2,84260,228

youtube-transcribe-skill

feiskyer

Extract subtitles/transcripts from a YouTube video URL and save as a local file. Use when you need to extract subtitles from a YouTube video.

5127,809

ui-ux-pro-max

nextlevelbuilder

"UI/UX design intelligence. 50 styles, 21 palettes, 50 font pairings, 20 charts, 8 stacks (React, Next.js, Vue, Svelte, SwiftUI, React Native, Flutter, Tailwind). Actions: plan, build, create, design, implement, review, fix, improve, optimize, enhance, refactor, check UI/UX code. Projects: website, landing page, dashboard, admin panel, e-commerce, SaaS, portfolio, blog, mobile app, .html, .tsx, .vue, .svelte. Elements: button, modal, navbar, sidebar, card, table, form, chart. Styles: glassmorphism, claymorphism, minimalism, brutalism, neumorphism, bento grid, dark mode, responsive, skeuomorphism, flat design. Topics: color palette, accessibility, animation, layout, typography, font pairing, spacing, hover, shadow, gradient."

4,3856,730

pdf-to-markdown

aliceisjustplaying

Convert entire PDF documents to clean, structured Markdown for full context loading. Use this skill when the user wants to extract ALL text from a PDF into context (not grep/search), when discussing or analyzing PDF content in full, when the user mentions "load the whole PDF", "bring the PDF into context", "read the entire PDF", or when partial extraction/grepping would miss important context. This is the preferred method for PDF text extraction over page-by-page or grep approaches.

5,3584,439

flutter-development

aj-geddes

Build beautiful cross-platform mobile apps with Flutter and Dart. Covers widgets, state management with Provider/BLoC, navigation, API integration, and material design.

2,4021,792

drawio-diagrams-enhanced

jgtolentino

Create professional draw.io (diagrams.net) diagrams in XML format (.drawio files) with integrated PMP/PMBOK methodologies, extensive visual asset libraries, and industry-standard professional templates. Use this skill when users ask to create flowcharts, swimlane diagrams, cross-functional flowcharts, org charts, network diagrams, UML diagrams, BPMN, project management diagrams (WBS, Gantt, PERT, RACI), risk matrices, stakeholder maps, or any other visual diagram in draw.io format. This skill includes access to custom shape libraries for icons, clipart, and professional symbols.

2,7451,642