Agents Optimize

optimize

Measure and improve your AgentCore agent's quality through evaluation, monitoring, and observability.

When to use

• You want to know if your agent is giving good answers
• You want to set up continuous quality monitoring in production
• You want to add a quality gate to your CI/CD pipeline
• You want to understand agent behavior through logs, metrics, and traces
• You want to set up CloudWatch dashboards or X-Ray tracing

Do NOT use for:

• Debugging a specific broken agent (wrong answers, errors) → use agents-debug
• Production security hardening (IAM, auth) → use agents-harden

Input

$ARGUMENTS can be:

• An eval goal: "add a quality gate", "set up monitoring"
• An observability goal: "set up CloudWatch dashboard", "understand my traces"
• A specific evaluator: "llm-as-a-judge", "code-based"
• Empty — the skill will guide based on project context

Process

Step 0: Verify CLI version

Run agentcore --version. This skill requires v0.9.0 or later.

Step 1: Read project context

Read agentcore/agentcore.json to understand existing evaluators, online eval configs, and agent setup.

If agentcore/agentcore.json is not found:

"This skill requires an AgentCore project. Use agents-get-started to create one."

Step 2: Determine the workflow

Step 3: Follow the loaded reference

The reference file contains the full procedure. Follow it step by step.

Cross-references

• After setting up evals, suggest agents-harden for production readiness
• If eval results reveal agent issues, suggest agents-debug for root cause analysis
• If the developer needs to add capabilities first, suggest agents-build

Output

Depends on the workflow — see the loaded reference for specific outputs.

Quality criteria

• Evaluator configuration uses only valid CLI flags
• Online eval sampling rate is appropriate (not 100% in production without discussion)
• CI/CD quality gate has a clear pass/fail threshold
• Observability setup includes both tracing and logging
• The developer understands the eval data delay: ~10 seconds put-to-get, end-to-end — one ingestion step covers both trace reads and eval queries; there is no separate indexing wait

Cost Optimization

Understand what drives AgentCore costs and how to control them. Pricing values are volatile — always verify against the AgentCore pricing page.

Cost components

AgentCore charges for several things independently:

Rates above are published as of the time of writing. Always verify against the AgentCore pricing page — pricing changes between releases.

First-day cost questions

"How much will my agent cost per invocation?"

There's no single number — it depends on:

• Which model (Haiku is ~10x cheaper than Sonnet per token)
• How long the session stays active (Runtime bills by vCPU-hour and GB-hour, not per request — idle sessions cost money)
• Whether it uses tools (gateway calls are $0.005 per 1,000 + any Lambda/API costs)
• Whether memory extraction is running (async, billed separately at $0.25 per 1,000 events)
• How long conversations run (more tokens = more model cost, and longer active sessions = more compute cost)

A simple Haiku-based agent with no memory and no tools costs very little per request — Runtime compute is billed by vCPU-hour ($0.0895) and GB-hour ($0.00945), so a sub-second request on a small environment costs fractions of a cent. A Sonnet agent with semantic memory, 5 gateway tools, and online evals at 10% sampling costs significantly more per request — the model token costs alone can be 10–30x higher, plus memory extraction ($0.25 per 1,000 events), gateway tool calls ($0.005 per 1,000 invocations), and eval model usage. These are published rates as of the time of writing — verify against the AgentCore pricing page for current numbers. If the awsknowledge MCP server is available, use the aws___search_documentation tool to look up current AgentCore pricing.

"How much will this demo/prototype cost me?"

Use the --defaults flags (Strands, Bedrock, no memory) during development. Stay under the free tier where possible. The biggest surprises come from:

• Idle sessions burning compute — Runtime bills by vCPU-hour while the session is active, including idle time before idleRuntimeSessionTimeout reclaims it. Default timeout is 15 minutes. Call StopRuntimeSession when done, or lower the timeout. See agents-harden Session lifecycle management.
• Leaving an online eval config running at 100% sampling
• Forgetting to set CloudWatch log retention (defaults to indefinite)
• Keeping a test memory resource with an expensive strategy (SEMANTIC or EPISODIC)

Cost reduction levers

Model selection

AgentCore supports four model providers — pick the right one for the task, not just the default:

Rules of thumb:

• Haiku or Nova Micro for simple extractive tasks (10–30x cheaper than Sonnet per token)
• Reserve Sonnet/Opus/GPT-5 for reasoning-heavy workflows
• Use different models for agent vs evaluator — a Haiku-based evaluator grading a Sonnet agent is a common cost-effective pattern
• For cost-sensitive customer support or classification agents, start with Nova Lite or Gemini Flash and only upgrade if quality is insufficient

Memory

• Only enable strategies you actually use — each LTM strategy runs extraction on every session
• SEMANTIC is the most expensive strategy. If you only need session summaries, use SUMMARIZATION alone.
• Tune relevance_score up so fewer memory records retrieve per query
• Set --expiry to the shortest duration that serves your use case (default is 30 days)

Online evals

• Start at 1–5% sampling in production, not 100%
• Use agentcore pause online-eval <name> when debugging or iterating — resume when you're ready to measure
• Pick the smallest evaluator set that gives signal

Logs and traces

• Set retention policies on log groups:

  aws logs put-retention-policy \
    --log-group-name /aws/bedrock-agentcore/runtimes/<AGENT_ID>-DEFAULT \
    --retention-in-days 30

• Don't log entire payloads — log structured events with just what you need
• X-Ray sampling is configured automatically; no dial to turn there

Gateway

• Tool calls are per-invocation, not per byte. Volume matters, not payload size.
• If a tool is called on every invocation for the same static data, consider baking that data into the system prompt instead

Container builds

• If you don't need Container, use CodeZip — no ECR storage charge
• If you need Container, keep the image small (see agents-harden Initialization time section)

Cross-references

• For model selection decisions, see `references/evals.md` Path A (evaluator model choice applies the same way)
• For memory strategy decisions, see `agents-build/references/memory.md`
• For log retention (a harden concern), see agents-harden Observability section

evals

Set up evaluation for your AgentCore agent — from a single quality check to a full production monitoring pipeline.

When to use

• You want to know if your agent is giving good answers
• You want continuous monitoring of live traffic
• You want a CI/CD quality gate that fails the build if quality drops
• You want to interpret eval scores you've already run
• You want to compare agent versions

Not for debugging a specific wrong answer — use the agents-debug skill for that.

Input

$ARGUMENTS is optional. If provided, it scopes the skill:

/evals                          # interactive — asks what you want to set up
/evals quick                    # run a quick eval on the most recent session
/evals monitor                  # set up continuous online monitoring
/evals ci                       # generate a CI/CD quality gate script

Process

Step 1: Understand the goal

Ask (or infer from $ARGUMENTS):

"What are you trying to do?

>

1. Run a one-time eval on recent sessions to see how my agent is doing

2. Set up continuous monitoring of live traffic

3. Add a quality gate to my CI/CD pipeline

4. Create a custom evaluator for my specific use case

5. Understand eval scores I've already run"

Step 2: Check prerequisites — and know what actually needs a deploy

Read agentcore/agentcore.json if it exists. Check:

• Is there a deployed runtime? (not always required — see below)
• Are there existing evaluators configured?

If no project context: Ask which runtime they want to evaluate. They can use --runtime-arn for standalone mode.

What actually requires a deployed runtime — and what doesn't:

The local-dev eval loop is a real option. agentcore dev auto-instruments OTEL and ships spans to CloudWatch the same way deployed runtimes do — this isn't a deployed-only feature. You can iterate on evaluators against your own local invocations, with a short round-trip and no AWS CDK churn.

For the dataset runner and online monitoring, deploy is genuinely required. Those paths invoke or ingest from a live AgentCore Runtime agent — there's no local equivalent.

Don't tell the developer to fully deploy before they can make progress on evals. Definition, authoring, and unit-testing are local. Running agentcore run eval is local too, given the prerequisites below.

Evaluating a local dev run

Requirements:

1. AWS credentials available locally (e.g., aws sso login for the account you want spans to land in). 2. CloudWatch Transaction Search enabled on the account. One-time setup — either in the CloudWatch console (Settings → X-Ray traces → Transaction Search) or via:

   aws xray update-trace-segment-destination --destination CloudWatchLogs

3. OTEL is already on. agentcore dev auto-instruments with the AWS OpenTelemetry distro by default. If you've passed --no-traces, remove it. 4. Wait ~10 seconds after invoking — CloudWatch put-to-get latency is ~10s end-to-end (covers both trace reads and eval queries; it's one ingestion step, not two).

The loop:

# Terminal 1 — start local dev with OTEL on (default)
agentcore dev

# Terminal 2 — invoke a few times, noting the session ID
agentcore dev --invoke "What's the weather in Seattle?" --stream
# or: agentcore invoke "..." once dev is running
# Note the session ID from the response / logs.

# Wait ~10 seconds for CloudWatch ingestion, then evaluate
agentcore run eval \
  --runtime MyAgent \
  --session-id <session-id-from-local-run> \
  --evaluator "Builtin.Helpfulness"

The evaluator runs in AWS (it's a managed evaluation service — the model call happens there, not locally), but the agent run being evaluated happened on your laptop. This is the fastest iteration loop for tuning an evaluator's instructions or rating scale.

Hand-constructed spans (no runtime at all)

For the tightest unit-test loop — or when you want to evaluate a saved snapshot without running the agent — call the Evaluate API directly with spans you construct:

import boto3

client = boto3.client("bedrock-agentcore", region_name="<REGION>")

response = client.evaluate(
    evaluatorId="Builtin.Helpfulness",
    sessionSpans=[
        # Minimum shape matches the OTEL span schema for AgentCore traces.
        # Easiest way to produce a fixture: download one real span via
        # `agentcore traces get <traceId> --output trace.json`, then mutate it.
        {"name": "agent.invoke", "attributes": {"gen_ai.prompt": "What's the weather?", "gen_ai.response.content": "Sunny, 72°F."}},
    ],
)
print(response["evaluatorResults"])

The full span schema and field list is in the `Understanding input spans` doc. This path is overkill for a one-time eval but invaluable when a custom code evaluator needs regression-test fixtures that don't depend on CloudWatch at all.

---

Path A: Run a one-time eval

Step A1: Choose an evaluator

Start with built-in evaluators — they require no setup and cover the most common quality dimensions:

Built-in evaluator names may change. Check the AgentCore docs for the current list — new evaluators are added across releases.

Recommendation: Start with Builtin.Helpfulness for a general quality check. Add Builtin.GoalSuccessRate for task completion. Use Builtin.ToolSelectionAccuracy when your agent uses tools. Use Builtin.Correctness or Builtin.Faithfulness when you have ground truth to compare against.

Step A2: Run the eval

# Run against the most recent session (auto-detected from project)
agentcore run eval --evaluator "Builtin.Helpfulness"

# Run against multiple evaluators
agentcore run eval \
  --evaluator "Builtin.Helpfulness" \
  --evaluator "Builtin.GoalSuccessRate"

# Run against a specific runtime (standalone mode, no project needed)
agentcore run eval \
  --runtime-arn arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/myagent-abc123 \
  --evaluator "Builtin.Helpfulness"

# Extend the lookback window (default is 7 days)
agentcore run eval --evaluator "Builtin.Helpfulness" --days 14

Step A3: Interpret the results

Scores are normalized to 0–1:

• 0.8–1.0 — Good. Agent is performing well on this dimension.
• 0.6–0.8 — Acceptable. Worth monitoring but not urgent.
• Below 0.6 — Investigate. Check recent traces for patterns.

Results are saved to agentcore/.cli/eval-runs/. View history:

agentcore evals history
agentcore evals history --limit 10

---

Path B: Set up continuous monitoring

Step B1: Create an evaluator (if needed)

For continuous monitoring, built-in evaluators are usually sufficient. If you need a custom evaluator for your specific use case, see Path D first.

Step B2: Add an online eval config

agentcore add online-eval \
  --name my_quality_monitor \
  --runtime MyAgent \
  --evaluator "Builtin.Helpfulness" \
  --evaluator "Builtin.GoalSuccessRate" \
  --sampling-rate 5

Important naming rule: Config names must use underscores only — no hyphens. my-monitor will fail with a validation error; my_monitor works.

Sampling rate guidance:

• 1–5 — Good for production (1–5% of requests evaluated)
• 10–20 — Good for staging or low-traffic agents
• 100 — Evaluate every request (dev/testing only, adds latency and cost)

Step B3: Deploy to activate

agentcore deploy -y

The online eval config starts in CREATING state and becomes ACTIVE within a few seconds after deploy.

Step B4: View results

Results stream to CloudWatch Logs:

/aws/bedrock-agentcore/evaluations/results/<config-id>

View in the AWS console: CloudWatch → GenAI Observability → Bedrock AgentCore → Evaluations tab.

Stream eval logs from the CLI:

agentcore logs evals --runtime MyAgent --since 1h
agentcore logs evals --follow

Pause/resume without redeploying:

agentcore pause online-eval my_quality_monitor
agentcore resume online-eval my_quality_monitor

---

Path C: CI/CD quality gate

Generate a script that runs evals and fails the build if quality drops below a threshold.

#!/bin/bash
# quality-gate.sh — run after deploy in CI/CD

set -e

RUNTIME="MyAgent"
EVALUATOR="Builtin.Helpfulness"
THRESHOLD="0.7"

echo "Running quality gate eval..."
result=$(agentcore run eval \
  --runtime "$RUNTIME" \
  --evaluator "$EVALUATOR" \
  --days 1 \
  --json)

score=$(echo "$result" | jq -r '.run.results[0].aggregateScore // empty')

if [ -z "$score" ]; then
  echo "⚠️  No eval data found. Has the agent been invoked recently?"
  echo "   Invoke the agent at least once, wait ~10 seconds, then re-run."
  exit 1
fi

echo "Quality score: $score (threshold: $THRESHOLD)"

if awk -v s="$score" -v t="$THRESHOLD" 'BEGIN{exit !(s<t)}'; then
  echo "❌ Quality gate FAILED: score $score < $THRESHOLD"
  exit 1
fi

echo "✅ Quality gate PASSED"

Note: CloudWatch put-to-get latency is ~10 seconds end-to-end — the same ingestion step unlocks both trace reads and eval queries; there's no extra indexing wait. In CI/CD, invoke the agent as part of your integration tests, then add a short sleep 10 (or sleep 15 for headroom) before running the quality gate. The old sleep 300 pattern from earlier skills/docs is 30× longer than needed now.

For standalone mode (no project context in CI):

agentcore run eval \
  --runtime-arn arn:aws:bedrock-agentcore:us-east-1:123456789012:runtime/myagent-abc123 \
  --evaluator "Builtin.Helpfulness" \
  --days 1 \
  --json

---

Path D: Custom evaluator

Use a custom evaluator when built-ins don't cover your specific quality criteria — domain accuracy, tone, format compliance, safety for your use case.

Step D1: Choose the evaluator type

• LLM-as-a-judge — An LLM scores each response against your instructions. Most flexible.
• Code-based — A Lambda function scores responses programmatically. Use for deterministic checks (format validation, required fields, etc.).

Step D2: Create an LLM-as-a-judge evaluator

Choose the right level first:

• SESSION — evaluate the whole conversation (goal completion, overall quality)
• TRACE — evaluate each individual response (helpfulness, accuracy, tone)
• TOOL_CALL — evaluate tool selection and parameters

Check the AgentCore docs for additional evaluator levels — new levels may be added across releases.

agentcore add evaluator \
  --name ResponseQuality \
  --level TRACE \
  --model "global.anthropic.claude-sonnet-4-5-20250929-v1:0" \
  --instructions "Evaluate the assistant's response for helpfulness and accuracy. Context: {context}. Response to evaluate: {assistant_turn}" \
  --rating-scale 1-5-quality

Note: The evaluator model ID above is an example — check the AgentCore docs for current supported evaluator model IDs and cross-region inference profiles.

Placeholder rules by level:

Rating scale presets (pass as literal strings to --rating-scale):

• 1-5-quality — Poor/Fair/Good/Very Good/Excellent (default)
• 1-3-simple — Low/Medium/High
• pass-fail — Pass/Fail
• good-neutral-bad — Good/Neutral/Bad

Custom rating scale:

--rating-scale "0:Incorrect:Factually wrong or misleading, 0.5:Partial:Partially correct, 1:Correct:Accurate and complete"

Step D3: Create a code-based evaluator (for deterministic checks)

agentcore add evaluator \
  --name FormatChecker \
  --level TRACE \
  --type code-based \
  --lambda-arn arn:aws:lambda:<REGION>:<YOUR_ACCOUNT_ID>:function:check-response-format \
  --timeout 30

Your Lambda receives the trace context and must return a score between 0 and 1. Use the SDK's @custom_code_based_evaluator() decorator to handle the Lambda event parsing and response contract for you:

# lambda_function.py
from bedrock_agentcore.evaluation.custom_code_based_evaluators import (
    custom_code_based_evaluator,
    EvaluatorInput,
    EvaluatorOutput,
)

@custom_code_based_evaluator()
def handler(evaluator_input: EvaluatorInput, context) -> EvaluatorOutput:
    # evaluator_input.session_spans contains the trace data
    # Implement your deterministic check (regex, schema validation, rule engine, etc.)
    response_text = _extract_response(evaluator_input.session_spans)

    if _matches_required_format(response_text):
        return EvaluatorOutput(value=1.0, label="Pass")
    return EvaluatorOutput(value=0.0, label="Fail", reasoning="Response did not match expected format")

The decorator handles parsing the raw Lambda event, extracting trace/span IDs, and serializing the response — write your check against typed EvaluatorInput and return a typed EvaluatorOutput.

Step D3.5: Unit-test the evaluator locally before deploying

The @custom_code_based_evaluator function is a plain Python function. Import it directly and exercise the logic with fixtures — no deploy, no AWS credentials needed:

# test_evaluator.py
from bedrock_agentcore.evaluation.custom_code_based_evaluators import EvaluatorInput
from lambda_function import handler  # the decorated function above

def _fake_input(response_text: str) -> EvaluatorInput:
    # Construct the minimum EvaluatorInput shape the handler reads.
    # Use a saved real trace for higher-fidelity fixtures — download one via
    # `agentcore traces get <traceId> --output trace.json` after a single deploy+invoke.
    return EvaluatorInput(
        session_spans=[{"attributes": {"gen_ai.response.content": response_text}}],
        # ...fill remaining fields the SDK expects for your level
    )

def test_matches_format():
    out = handler(_fake_input('{"status": "ok"}'), context=None)
    assert out.value == 1.0

def test_rejects_free_text():
    out = handler(_fake_input("Here's your answer: ok"), context=None)
    assert out.value == 0.0
    assert "did not match" in (out.reasoning or "").lower()

Run with pytest test_evaluator.py. Iterate the logic until the fixtures pass. Only then deploy — the deploy step is about wiring the Lambda into AgentCore, not about debugging the check.

For LLM-as-judge evaluators, there's no equivalent unit-test surface (the model call happens in the eval service), but you can iterate on the instructions against saved traces by dry-running the prompt in Bedrock console or in a one-off script before agentcore add evaluator.

Step D4: Deploy and run against a real trace

agentcore deploy -y
agentcore run eval --evaluator ResponseQuality --days 7

Evaluator name rules: alphanumeric + underscores only, max 48 chars. No hyphens.

---

Troubleshooting

"No spans found for session"

• Wait ~10 seconds after invoking the agent — CloudWatch put-to-get is ~10s end-to-end (there's no separate eval-indexing step beyond that)
• Check that observability was enabled when the agent was deployed
• Extend the lookback: --days 14 or --days 30

"No agent specified" or agent ID not found

• Run from inside your AgentCore project directory, or
• Use --runtime-arn to specify the agent explicitly

Online eval config stuck in CREATING

• Run agentcore status --type online-eval to check status
• Usually resolves within 30 seconds of deploy

remove evaluator fails

• An online eval config is referencing this evaluator
• Remove the online eval config first: agentcore remove online-eval --name <name>
• Then remove the evaluator

Cross-region inference (data residency)

Both built-in and LLM-as-judge evaluators use cross-region inference by default. The data being evaluated stays in your primary region, but the inference call that runs the judge model may execute in another AWS region within the same geography (e.g., us-east-1 → us-east-2/us-west-2; EU stays in EU).

There's no extra cost, and logs don't include the inference region. But if data-residency rules require pinning inference to a single region:

• Built-in evaluators: they're managed by AgentCore and use cross-region inference as-configured. If single-region inference is required, use a custom evaluator instead.
• Custom LLM-as-judge evaluators: pin the model by choosing a region-specific model ID for --model instead of a cross-region inference profile ID. Check the docs for current single-region model IDs in your region.
• Code-based evaluators: not affected. The Lambda runs wherever you deployed it.

See cross-region inference for the current geography/region mapping rather than baking it in here — it expands across releases.

When to use the dataset runner vs. agentcore run eval

Two different tools for two different workflows — developers confuse them.

Use `agentcore run eval` when you're iterating on an evaluator, investigating a specific regression, or running a quality gate against recent traffic. It's fast, cheap, and doesn't invoke the agent itself — it only scores existing traces.

Use `OnDemandEvaluationDatasetRunner` when you have a dataset of scenarios with expected responses / trajectories / assertions and you want to run them as a batch. The runner invokes the agent for each scenario, waits for telemetry ingestion (default 180 seconds, paid once per run not per scenario), then evaluates. This requires a deployed runtime. Typical use: regression pack in CI before promoting a new version.

Use online eval for continuous production monitoring at a sampling rate — not the same as a benchmark.

Output

• CLI commands to run evals or set up monitoring
• Quality gate script (for CI/CD path)
• Evaluator config (for custom evaluator path)
• Interpretation of scores if reviewing existing results

Observability Setup

Set up logging, tracing, and monitoring for your AgentCore agent.

What's auto-enabled

AgentCore automatically enables:

• X-Ray tracing — every invocation generates a trace
• CloudWatch logging — agent logs ship to CloudWatch

These are on by default whether you're running deployed (agentcore deploy + invoke) or locally (agentcore dev). The dev server auto-instruments your agent with the AWS OpenTelemetry distro the same way the deployed runtime does; opt out with agentcore dev --no-traces.

Two prerequisites for the local path to work end-to-end:

1. AWS credentials available locally — the OTEL exporter needs them to ship spans. 2. CloudWatch Transaction Search is enabled on the account (one-time setup per account) — see "Viewing traces" below. Without it, spans are ingested but not searchable, so agentcore traces list and agentcore run eval --session-id return empty.

After deploy, AgentCore Runtime also auto-instruments the container (the default CMD wraps the app with opentelemetry-instrument). You don't need to configure OTEL in your code for either path — but you do need your agent code to be instrumented correctly.

Ensuring logs appear in CloudWatch

Three things must be true for logs to appear:

1. OTEL entrypoint wrapper in Dockerfile

Your Dockerfile CMD must use the OpenTelemetry wrapper:

CMD ["opentelemetry-instrument", "python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

For CodeZip builds, this is handled automatically. For Container builds, you must add it.

2. IAM permissions for CloudWatch and X-Ray

Your runtime execution role needs:

{
  "Effect": "Allow",
  "Action": [
    "logs:CreateLogGroup",
    "logs:CreateLogStream",
    "logs:PutLogEvents",
    "xray:PutTraceSegments",
    "xray:PutTelemetryRecords"
  ],
  "Resource": "*"
}

3. Use the logging module, not print()

AgentCore captures structured logs via the Python logging module. print() statements go to stdout but are not captured by the OTEL pipeline.

import logging
logger = logging.getLogger(__name__)

# Good — captured by CloudWatch
logger.info("Processing request", extra={"session_id": session_id})

# Bad — not captured
print(f"Processing request {session_id}")

Viewing traces

Traces show the full execution path of one agent invocation — model calls, tool calls, and timing.

# List recent traces
agentcore traces list --runtime <AgentName> --since 1h --limit 10

# Get a specific trace
agentcore traces get <traceId> --runtime <AgentName>

Trace delay: Traces appear ~10 seconds after invocation (previously 30–60s). Don't panic if they're not immediate, and don't bake longer waits into scripts — older skills and docs that say "30–60 seconds" or "2–5 minutes" are stale.

Also verify Transaction Search is enabled in CloudWatch — this is a prerequisite for trace visibility in the console.

Viewing logs

# Stream recent logs
agentcore logs --runtime <AgentName> --since 30m

# Filter by level
agentcore logs --runtime <AgentName> --level error --since 1h

# Search for specific text
agentcore logs --runtime <AgentName> --query "timeout" --since 2h

CloudWatch dashboard

For production agents, set up a CloudWatch dashboard with:

• Invocation count and error rate
• P50/P90/P99 latency
• Memory and CPU utilization
• Error log count by type

These metrics are available in the AWS/BedrockAgentCore namespace after deploy.

Multi-account observability

If your agents are spread across accounts (typical setup: separate prod / staging / dev accounts), use CloudWatch cross-account observability to view metrics, traces, and logs from one central monitoring account.

The setup order matters — do it in this sequence or the console won't show source-account data:

1. Pick a monitoring account. This is where you'll view everything. Often a central observability account, not a workload account. 2. Configure the monitoring account first. CloudWatch console → Settings → Monitoring account configuration → Configure. Choose which telemetry types to share (enable Metrics and Logs — traces go through X-Ray's own cross-account mechanism). 3. Link each source account. Either via AWS Organizations (if your accounts are in one) or via individual linking. Source accounts must accept the link. 4. Deploy AgentCore agents in the source accounts with observability enabled — same default OTEL wrap-up as single-account. No code changes needed. 5. View from the monitoring account. AgentCore Observability in the CloudWatch console now shows data from all linked accounts side-by-side, identified by source account ID.

Order-of-operations trap: if you deploy agents in source accounts before linking, the telemetry still flows correctly — it just won't be visible from the monitoring account until the link is active. You don't need to redeploy, just wait a few minutes after linking.

Traces: cross-account trace viewing uses X-Ray's existing cross-account sharing model. If the CloudWatch cross-account link is set up correctly for Logs and Metrics but traces don't show, check X-Ray's cross-account config separately.

IAM: no extra IAM on the agent execution roles for cross-account observability. The cross-account feature operates at the CloudWatch/X-Ray layer, not at the source of the telemetry.

See cross-account observability for the current console flow and edge cases.

Cross-references

• If logs aren't appearing at all, check the three requirements above or use agents-debug
• For production observability setup, see agents-harden
• For measuring agent quality (not just operational health), load `references/evals.md`