Aspire Monitoring

name: aspire-monitoring-eval
description: Evaluates the aspire-monitoring skill for correct diagnostics routing between local
  Aspire CLI and deployed platform-specific tools. Tests the bridge logic that is the skill's
  primary value.
version: "1.0"
type: capability
tags:
  skill: aspire-monitoring
config:
  runs: 3
  timeout: 120s
  executor: copilot-sdk
  model: gpt-5-mini
stimuli:
  - name: mon-browser-logs-001
    prompt: I want to see browser console logs and network requests from my Vite frontend in the Aspire
      Dashboard alongside the API server logs. How does that work?
    tags:
      priority: p1
      area:
        - browser-logs
        - dashboard
        - 13-3
        - routing
    environment:
      files:
        - src: ../../../evals/csharp-apphost/MyApp.AppHost/MyApp.AppHost.csproj
          dest: csharp-apphost/MyApp.AppHost/MyApp.AppHost.csproj
        - src: ../../../evals/csharp-apphost/MyApp.AppHost/Program.cs
          dest: csharp-apphost/MyApp.AppHost/Program.cs
        - src: ../../../evals/csharp-apphost/aspire.config.json
          dest: csharp-apphost/aspire.config.json
    graders:
      - type: prompt
        name: knows_browsers_integration
        config:
          prompt: Does the assistant's response identify the `Aspire.Hosting.Browsers` integration (or
            equivalent description) as the mechanism that captures browser console logs, network
            requests, and screenshots into the dashboard, and `WithBrowserLogs()` as the opt-in call
            on a frontend resource? Answer based on intent.
      - type: prompt
        name: routes_code_change_to_aspireify
        config:
          prompt: Does the assistant's response route the actual code edit (adding `WithBrowserLogs()` to the
            AppHost) to the `aspireify` skill (or otherwise indicate that AppHost authoring is out
            of scope for this monitoring skill)? Answer based on intent — direct AppHost edits from
            the monitoring skill should be discouraged.
      - type: prompt
        name: explains_dashboard_surface
        config:
          prompt: Does the assistant's response explain that once `WithBrowserLogs()` is wired, the browser
            telemetry surfaces in the dashboard alongside server-side telemetry (and is also visible
            via `aspire otel logs <frontend>`)?
      - type: output-contains
        config:
          substring: WithBrowserLogs
  - name: mon-dashboard-run-001
    prompt: I want to view OTLP telemetry from a non-Aspire service in the Aspire Dashboard without
      booting an AppHost. How do I do that and how do I query it from the CLI?
    tags:
      priority: p1
      area:
        - dashboard
        - standalone
        - 13-3
    graders:
      - type: prompt
        name: knows_dashboard_run
        config:
          prompt: Does the assistant's response recommend `aspire dashboard run` as the way to launch the
            Aspire Dashboard standalone (without an AppHost) and capture OTLP from any source?
            Answer based on intent.
      - type: prompt
        name: warns_foreground_blocking
        config:
          prompt: Does the assistant's response warn that `aspire dashboard run` is foreground / blocking and
            must be treated as a long-running background process by agents (e.g., bash mode="async",
            capture the printed dashboard URL and token, do not wait for it to "finish")? This is
            the critical safety note. Answer based on intent.
      - type: prompt
        name: connects_cli_via_dashboard_url
        config:
          prompt: Does the assistant's response show how to connect the Aspire CLI to a standalone dashboard
            using `--dashboard-url` on `aspire otel logs` and/or `aspire otel traces`? The simplest
            form passes the full login URL printed by `aspire dashboard run` (which contains the
            `t=` token) to `--dashboard-url` — login URLs are normalized automatically. `--ap