Agent Skills

.*
!.gitignore

from .executor import ExperimentAnalyzerExecutor
from .evaluator import ExperimentAnalyzerEvaluator

PROJECT_CONFIG = {
    "name": "llmo-bits-ai-eng-skill-evals",
    "executor": ExperimentAnalyzerExecutor,
    "evaluator": ExperimentAnalyzerEvaluator,
    "description": "Evaluates the llm-obs-experiment-analyzer Claude Code skill",
}

__all__ = ["ExperimentAnalyzerExecutor", "ExperimentAnalyzerEvaluator"]

[
    {
        "input": {
            "llmo_project_name": "cmd-i-skill-evals",
            "baseline": "a4b9d02e-203c-4d62-b8ec-ae986e83cb06",
            "candidate": "a05b32f1-351b-44f0-bf4a-efa85c910de7"
        },
        "labels": {
            "question": "Are there specific scenarios where the notebook skill was loaded in one experiment but not the other?",
            "answer": "Yes, there are 4 scenarios where the candidate loaded the skill and the baseline didn't, and 3 where the baseline loaded the skill and the candidate didn't.",
            "key_points": [
                "2 ValueError crashes in baseline → 0 in candidate, eval pipeline fixed.",
                "Metric recording was silently broken in baseline — 4/6 metrics now visible for the first time (combined_score 72.1%).",
                "Skill loading flat overall (~43%), but candidate loses 3 SQL scenarios it shouldn't — worth watching.",
                "Core task performance unchanged — 1/3 of runs still fail to produce a notebook at all."
            ]
        },
        "metadata": {
            "created_at": "2026-02-25",
            "labeler": "mbl"
        },
        "record_id": "0edcb42a-b3b1-451f-9db5-ac21f8e40ede"
    },
    {
        "input": {
            "llmo_project_name": "bits-sre-judge-alignment",
            "baseline": "21ad232f-b485-40fb-8a73-cf473ddc2c3b",
            "candidate": "2c2c1525-400e-4d6b-9dd2-110fb0036274"
        },
        "labels": {
            "question": "Which judge gives answers that are better aligned with human preferences, as represented in the eval set, the candidate judge or the baseline judge?",
            "answer": "4.1 (the candidate) is better aligned. I think it's because we're giving it pretty heavy and specific instructions, which the 4.1 series is built to follow, rather than more open ended or \"goal oriented\" prompting that reasoning models are better at.",
            "key_points": [
                "IC was way up.",
                "Mean error down.",
                "Bias a little worse than we'd like still.",
                "Significantly better than baseline overall."
            ]
        },
        "metadata": {
            "created_at": "2026-02-25",
            "labeler": "mbl"
        },
        "record_id": "e8810544-348e-4340-bad2-c3d57e4e0b5f"
    },
    {
        "input": {
            "llmo_project_name": "cmdi_o2_tv2_test_trunc",
            "baseline": "0bf91646-55e5-47b8-bb25-e3efeab93154",
            "candidate": "7da2caa0-d0f5-4f21-bd24-002c57a17882"
        },
        "labels": {
            "question": "What's the improvement of adding a warning about truncated content out of tools in the prompt?",
            "answer": "No statistically significant improvement",
            "key_points": [
                "The metrics are better on the surface.",
                "When running deeper analysis these results are not significant.",
                "Context truncation is not a significant factor in the differing results."
            ]
        },
        "metadata": {
            "created_at": "2026-02-26",
            "labeler": "mbl"
        },
        "record_id": "7a857d04-e9d8-4c88-b441-c85983d69ce1"
    },
    {
        "input": {
            "llmo_project_name": "assistant-code-reading",
            "baseline": "f13a28ab-a10e-444c-b598-31b4596f7311",
            "candidate": "fd6bc13b-6e7d-40df-843a-4576dd4b0e2b"
        },
        "labels": {
            "question": "Does the tool \"code inspection\" help getting better results?",
            "answer": "Yes, but the tool is not called, leading to a suspicion that the better results are due to informing the model that it CAN call code.",
            "key_points": [
                "Overall results slightly improve, though not significantly, across all categories. Most gains, suspiciously, occurred in non-sandbox scenarios, suggesting the improvement is due to simply informing the model it can investigate code.",
                "Sandbox use is primarily for code investigation, but results vary significantly from previous experiments, indicating high run-to-run variance. These improvements aren't correlated with sandbox usage."
            ]
        },
        "metadata": {
            "created_at": "2026-03-05",
            "labeler": "mbl"
        },
        "record_id": "08209254-9600-4dab-9ce1-d4301afa95a2"
    }
]

[
    {
        "input": {
            "llmo_project_name": "cmd-i-skill-evals",
            "baseline": "a4b9d02e-203c-4d62-b8ec-ae986e83cb06",
            "candidate": "a05b32f1-351b-44f0-bf4a-efa85c910de7",
            "question": "Are there specific scenarios where the notebook skill was loaded in one experiment but not the other?"
        },
        "labels": {
            "answer": "Yes, there are 4 scenarios where the candidate loaded the skill and the baseline didn't, and 3 where the baseline loaded the skill and the candidate didn't.",
            "key_points": [
                "2 ValueError crashes in baseline → 0 in candidate, eval pipeline fixed.",
                "Metric recording was silently broken in baseline — 4/6 metrics now visible for the first time (combined_score 72.1%).",
                "Skill loading flat overall (~43%), but candidate loses 3 SQL scenarios it shouldn't — worth watching.",
                "Core task performance unchanged — 1/3 of runs still fail to produce a notebook at all."
            ]
        },
        "metadata": {
            "created_at": "2026-03-06",
            "labeler": "mbl",
            "mode": "comparative_qa"
        },
        "record_id": "64877688-a38f-4a81-a19e-89b2a603d208"
    },
    {
        "input": {
            "llmo_project_name": "bits-sre-judge-alignment",
            "baseline": "21ad232f-b485-40fb-8a73-cf473ddc2c3b",
            "candidate": "2c2c1525-400e-4d6b-9dd2-110fb0036274",
            "question": "Which judge gives answers that are better aligned with human preferences, as represented in the eval set, the candidate judge or the baseline judge?"
        },
        "labels": {
            "answer": "4.1 (the candidate) is better aligned. I think it's because we're giving it pretty heavy and specific instructions, which the 4.1 series is built to follow, rather than more open ended or \"goal oriented\" prompting that reasoning models are better at.",
            "key_points": [
                "IC was way up.",
                "Mean error down.",
                "Bias a little worse than we'd like still.",
                "Significantly better than baseline overall."
            ]
        },
        "metadata": {
            "created_at": "2026-03-06",
            "labeler": "mbl",
            "mode": "comparative_qa"
        },
        "record_id": "794981d3-ce25-4cd3-971e-40d75e2adbed"
    },
    {
        "input": {
            "llmo_project_name": "assistant-code-reading",
            "baseline": "f13a28ab-a10e-444c-b598-31b4596f7311",
            "candidate": "fd6bc13b-6e7d-40df-843a-4576dd4b0e2b",
            "question": "Does the tool \"code inspection\" help getting better results?"
        },
        "labels": {
            "answer": "Yes, but the tool is not called, leading to a suspicion that the better results are due to informing the model that it CAN call code.",
            "key_points": [
                "Overall results slightly improve, though not significantly, across all categories. Most gains, suspiciously, occurred in non-sandbox scenarios, suggesting the improvement is due to simply informing the model it can investigate code.",
                "Sandbox use is primarily for code investigation, but results vary significantly from previous experiments, indicating high run-to-run variance. These improvements aren't correlated with sandbox usage."
            ]
        },
        "metadata": {
            "created_at": "2026-03-06",
            "labeler": "mbl",
            "mode": "comparative_qa"
        },
        "record_id": "e7dd93c6-67d6-4704-bb77-d64ed542e4fc"
    }
]

"""LLM judge evaluator for the llm-obs-experiment-analyzer skill."""

import threading
from pathlib import Path
from typing import Any

from ddtrace.llmobs._evaluators import BaseEvaluator, EvaluatorContext, EvaluatorResult
from ddtrace.llmobs._evaluators.llm_judge import LLMJudge
from ddeval import BaseProjectEvaluator, Evaluator
from ddeval.evaluators._ai_gateway import DEFAULT_MODEL, create_ai_gateway_client


# Metrics scored 0–10 by the judge, normalized to 0.0–1.0
_STRUCTURED_OUTPUT = {
    "type": "object",
    "properties": {
        "answer_accuracy": {"type": "number"},
        "key_points_coverage": {"type": "number"},
        "evidence_quality": {"type": "number"},
        "reasoning": {"type": "string"},
    },
    "required": ["answer_accuracy", "key_points_coverage", "evidence_quality", "reasoning"],
    "additionalProperties": False,
}


class _JudgeCache:
    """Thread-safe cache: one LLM call per unique row, shared across all metric evaluators."""

    def __init__(self):
        self._cache: dict[str, dict[str, Any]] = {}
        self._lock = threading.Lock()

    def get(self, key: str) -> dict[str, Any] | None:
        with self._lock:
            return self._cache.get(key)

    def set(self, key: str, scores: dict[str, Any]) -> None:
        with self._lock:
            self._cache[key] = scores


class _SkillJudge(BaseEvaluator):
    def __init__(self, metric_name: str, cache: _JudgeCache):
        super().__init__(name=metric_name)
        self.metric_name = metric_name
        self._cache = cache
        rubric_path = Path(__file__).parent / "prompts" / "judge_rubric.txt"
        self._judge = LLMJudge(
            user_prompt=rubric_path.read_text(),
            client=create_ai_gateway_client(),
            structured_output=_STRUCTURED_OUTPUT,
            model=DEFAULT_MODEL,
        )

    def evaluate(self, context: EvaluatorContext) -> EvaluatorResult:
        if not context.output_data or not context.expected_output:
            return EvaluatorResult(value=None)

        # Fields that uniquely identify a row — prevents redundant LLM calls across metrics
        cache_key = "|".join(
            str(context.input_data.get(f, "")) for f in ["baseline", "candidate", "question"]
        )

        scores = self._cache.get(cache_key)
        if scores is None:
            result = self._judge.evaluate(context)
            scores = result.value
            self._cache.set(cache_key, scores)

        raw = scores.get(self.metric_name)
        return EvaluatorResult(
            value=float(raw) / 10.0 if raw is not None else None,
            reasoning=scores.get("reasoning"),
        )


class ExperimentAnalyzerEvaluator(BaseProjectEvaluator):
    def get_evaluators(self) -> list[Evaluator]:
        cache = _JudgeCache()
        return [
            _SkillJudge("answer_accuracy", cache),
            _SkillJudge("key_points_coverage", cache),
            _SkillJudge("evidence_quality", cache),
        ]

    def get_summary_evaluators(self) -> list:
        return []

{
    "input_parameters": {},
    "executor_config": {
        "timeout_seconds": 900
    }
}

"""Executor for the llm-obs-experiment-analyzer Claude Code skill."""

import os
import sys
from typing import Any

_eval_lib_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "../.."))
if _eval_lib_path not in sys.path:
    sys.path.insert(0, _eval_lib_path)

from _eval_lib import DEFAULT_TIMEOUT_SECONDS, invoke_skill  # noqa: E402
from ddeval import BaseProjectExecutor, Config  # noqa: E402


class ExperimentAnalyzerExecutor(BaseProjectExecutor):
    def execute_single(
        self, input_data: dict[str, Any], config: Config | None
    ) -> dict[str, Any]:
        cfg = config.executor_config if config else {}
        timeout_seconds = cfg.get("timeout_seconds", DEFAULT_TIMEOUT_SECONDS)
        mcp_env = cfg.get("mcp_env", "prod")

        baseline = input_data.get("baseline")
        candidate = input_data.get("candidate")
        experiment_id = input_data.get("experiment_id")

        if not baseline and not candidate and not experiment_id:
            raise ValueError(
                "input_data must contain 'baseline', 'candidate', or 'experiment_id'"
            )

        parts = ["/llm-obs-experiment-analyzer"]
        if baseline:
            parts.append(baseline)
        if candidate:
            parts.append(candidate)
        if not baseline and not candidate and experiment_id:
            parts.append(experiment_id)
        if input_data.get("question"):
            parts.append(input_data["question"])
        parts.append("--output agent")
        prompt = " ".join(parts)

        # prompt is passed to the Claude Agent SDK, not to a shell — not a command injection risk
        report = invoke_skill(prompt, timeout_seconds=timeout_seconds, mcp_env=mcp_env)
        return {"report": report, "word_count": len(report.split()) if report else 0}

You are an expert judge evaluating the quality of an LLM experiment analysis report.

## Experiment Context

Baseline experiment ID: {{input_data.baseline}}
Candidate experiment ID: {{input_data.candidate}}
Question asked: {{input_data.question}}

## Expected Answer

{{expected_output.answer}}

Key points that should be covered:
{{expected_output.key_points}}

## Generated Report

{{output_data.report}}

---

## Scoring Instructions

Score each dimension from 0 to 10. Be strict — reserve 9–10 for exceptional quality.

### 1. answer_accuracy (0–10)

Does the report's conclusion or verdict align with the expected answer?

- 9–10: The report reaches the same conclusion as the expected answer with correct supporting evidence (specific metrics, counts, percentages). Nuances match.
- 7–8: The verdict is correct but misses some nuance or uses approximate rather than exact numbers.
- 5–6: The report is partially correct — gets the direction right but is vague, hedged, or misses a key part of the answer.
- 3–4: The report is mostly wrong or contradicts the expected answer in a meaningful way.
- 0–2: The report does not answer the question, produces no output, or gives the opposite conclusion.

### 2. key_points_coverage (0–10)

Does the report surface the key findings listed in the expected key points?

- 9–10: All key points are explicitly addressed with matching evidence and correct interpretation.
- 7–8: Most key points are covered; one is missing or has incorrect detail.
- 5–6: Half or more key points are present but some are missing or shallow.
- 3–4: Only one or two key points are touched, others are absent or wrong.
- 0–2: Key points are not addressed or the report is empty/trivial.

### 3. evidence_quality (0–10)

Are the report's claims grounded in specific, quantified evidence from the experiment data?

- 9–10: Every major claim cites specific numbers (metric pass rates, event counts, percentages, delta values). Reasoning is traceable to data.
- 7–8: Most claims are quantified; a few are qualitative without supporting numbers.
- 5–6: Some quantification present, but significant portions rely on vague language ("some events", "a few cases", "seems better").
- 3–4: Claims are mostly qualitative with minimal data reference.
- 0–2: No quantification; report is purely speculative or empty.

### Weighted aggregate (for your reasoning only — do not output this score)

Suggested weights: answer_accuracy × 0.5 + key_points_coverage × 0.3 + evidence_quality × 0.2

---

Return a JSON object with exactly these keys:

{
  "answer_accuracy": <integer 0–10>,
  "key_points_coverage": <integer 0–10>,
  "evidence_quality": <integer 0–10>,
  "reasoning": "<2–4 sentences explaining the scores, referencing specific parts of the report>"
}

[]

"""LLM judge evaluator for the llm-obs-trace-rca skill."""

import threading
from pathlib import Path
from typing import Any

from ddtrace.llmobs._evaluators import BaseEvaluator, EvaluatorContext, EvaluatorResult
from ddtrace.llmobs._evaluators.llm_judge import LLMJudge
from ddeval import BaseProjectEvaluator, Evaluator
from ddeval.evaluators._ai_gateway import DEFAULT_MODEL, create_ai_gateway_client


# Metrics scored 0–10 by the judge, normalized to 0.0–1.0
_STRUCTURED_OUTPUT = {
    "type": "object",
    "properties": {
        "diagnosis_accuracy": {"type": "number"},
        "evidence_grounding": {"type": "number"},
        "actionability": {"type": "number"},
        "completeness": {"type": "number"},
        "reasoning": {"type": "string"},
    },
    "required": ["diagnosis_accuracy", "evidence_grounding", "actionability", "completeness", "reasoning"],
    "additionalProperties": False,
}


class _JudgeCache:
    """Thread-safe cache: one LLM call per unique row, shared across all metric evaluators."""

    def __init__(self):
        self._cache: dict[str, dict[str, Any]] = {}
        self._lock = threading.Lock()

    def get(self, key: str) -> dict[str, Any] | None:
        with self._lock:
            return self._cache.get(key)

    def set(self, key: str, scores: dict[str, Any]) -> None:
        with self._lock:
            self._cache[key] = scores


class _LlmObsTraceRcaJudge(BaseEvaluator):
    def __init__(self, metric_name: str, cache: _JudgeCache):
        super().__init__(name=metric_name)
        self.metric_name = metric_name
        self._cache = cache
        rubric_path = Path(__file__).parent / "prompts" / "judge_rubric.txt"
        self._judge = LLMJudge(
            user_prompt=rubric_path.read_text(),
            client=create_ai_gateway_client(),
            structured_output=_STRUCTURED_OUTPUT,
            model=DEFAULT_MODEL,
        )

    def evaluate(self, context: EvaluatorContext) -> EvaluatorResult:
        if not context.output_data or not context.expected_output:
            return EvaluatorResult(value=None)

        # Fields that uniquely identify a row — prevents redundant LLM calls.
        cache_key = "|".join(
            str(context.input_data.get(f, ""))
            for f in ["ml_app", "eval_name", "timeframe", "mode", "failure_filter"]
        )

        scores = self._cache.get(cache_key)
        if scores is None:
            result = self._judge.evaluate(context)
            scores = result.value
            self._cache.set(cache_key, scores)

        raw = scores.get(self.metric_name)
        return EvaluatorResult(
            value=float(raw) / 10.0 if raw is not None else None,
            reasoning=scores.get("reasoning"),
        )


class LlmObsTraceRcaEvaluator(BaseProjectEvaluator):
    def get_evaluators(self) -> list[Evaluator]:
        cache = _JudgeCache()
        return [
            _LlmObsTraceRcaJudge("diagnosis_accuracy", cache),
            _LlmObsTraceRcaJudge("evidence_grounding", cache),
            _LlmObsTraceRcaJudge("actionability", cache),
            _LlmObsTraceRcaJudge("completeness", cache),
        ]

    def get_summary_evaluators(self) -> list:
        return []

{
    "input_parameters": {},
    "executor_config": {
        "timeout_seconds": 1000
    }
}

"""Executor for the llm-obs-trace-rca Claude Code skill."""

import os
import sys
from typing import Any

_eval_lib_path = os.path.abspath(os.path.join(os.path.dirname(__file__), "../.."))
if _eval_lib_path not in sys.path:
    sys.path.insert(0, _eval_lib_path)

from _eval_lib import DEFAULT_TIMEOUT_SECONDS, invoke_skill  # noqa: E402
from ddeval import BaseProjectExecutor, Config  # noqa: E402


class LlmObsTraceRcaExecutor(BaseProjectExecutor):
    def execute_single(
        self, input_data: dict[str, Any], config: Config | None
    ) -> dict[str, Any]:
        cfg = config.executor_config if config else {}
        timeout_seconds = cfg.get("timeout_seconds", DEFAULT_TIMEOUT_SECONDS)
        mcp_env = cfg.get("mcp_env", "prod")

        if input_data.get("from") and input_data.get("to"):
            time_spec = f"from {input_data['from']} to {input_data['to']}"
        else:
            timeframe = input_data.get("timeframe", "now-24h")
            time_spec = f"over the last {timeframe}"

        ml_app = input_data.get("ml_app")
        eval_name = input_data.get("eval_name")
        mode = input_data.get("mode")
        failure_filter = input_data.get("failure_filter")

        # Append explicit mode/filter hints so the skill skips inference when needed.
        hints = []
        if mode:
            hints.append(f"mode={mode}")
        if failure_filter:
            hints.append(f"filter={failure_filter}")
        suffix = f" [{', '.join(hints)}]" if hints else ""

        if ml_app and eval_name:
            prompt = f"/llm-obs-trace-rca Analyze eval failures for {eval_name} on {ml_app} {time_spec}{suffix}"
        elif eval_name:
            prompt = f"/llm-obs-trace-rca Analyze eval failures for {eval_name} {time_spec}{suffix}"
        elif ml_app:
            prompt = f"/llm-obs-trace-rca What's wrong with {ml_app} {time_spec}{suffix}"
        else:
            raise ValueError("input_data must contain either 'ml_app' or 'eval_name'")

        report = invoke_skill(prompt, timeout_seconds=timeout_seconds, mcp_env=mcp_env)
        return {"report": report, "word_count": len(report.split()) if report else 0}

You are evaluating an AI-generated root cause analysis (RCA) report for a production LLM application.

The report was generated by analyzing LLM observability traces for the following application:

**Application**: {{input_data.ml_app}} / {{input_data.eval_name}}
**Timeframe**: {{input_data.timeframe}}
**Mode**: {{input_data.mode}} (eval, errors, or generic — determines what signal was used)

---

## Expected Findings

The following key findings and recommendations were identified by a human analyst as ground truth for this scenario:

**Key findings** (root causes and failure patterns that should be diagnosed):
{{expected_output.key_findings}}

**Expected recommendations** (actionable changes that should be proposed):
{{expected_output.expected_recommendations}}

---

## Generated RCA Report

{{output_data.report}}

---

## Scoring Instructions

Score each dimension from 0 to 10. Use the full range.

### 1. diagnosis_accuracy (0–10)

Does the report correctly identify the root causes described in the expected findings?

- **10**: All key root causes from expected findings are identified, correctly named, and traced to the right span or system component. No significant false diagnoses.
- **7–9**: Most key root causes identified. Minor misattributions or missing one secondary cause.
- **4–6**: Some root causes identified but important ones are missing, or the diagnosis is at the symptom level ("model gave bad answer") rather than tracing to a specific root cause (e.g., "system prompt lacks date format instructions in the parent agent span").
- **1–3**: Very few root causes identified. Report mostly describes what failed without diagnosing why.
- **0**: No root causes identified, or diagnoses are entirely wrong.

### 2. evidence_grounding (0–10)

Are claims backed by trace evidence — span IDs, quoted judge reasoning, specific content from trace fields?

- **10**: Every significant claim references a span ID with a trace link. Judge reasoning is quoted verbatim (Eval Signal) or stack traces are cited (Error Signal). Trace deep-dives show what was fetched (system prompt content, tool parameters, retrieved documents, etc.).
- **7–9**: Most claims are grounded. A few assertions are unsupported.
- **4–6**: Some evidence cited, but key claims are asserted without trace references. Or evidence is generic ("the trace showed issues") rather than specific.
- **1–3**: Little to no trace evidence. Report is mostly assertions.
- **0**: No evidence cited at all.

### 3. actionability (0–10)

Are recommendations specific and immediately actionable, with before/after content from the traces?

- **10**: Each recommendation specifies exactly what to change (system prompt text, tool parameter, routing logic), includes a before/after quote from the actual trace, and explains the causal link to the failure.
- **7–9**: Most recommendations are specific. One or two are somewhat vague or lack before/after text.
- **4–6**: Recommendations are present but generic (e.g., "improve the system prompt") without specific text changes or references to actual trace content.
- **1–3**: Recommendations are vague advice (e.g., "improve retrieval quality") with no specifics.
- **0**: No recommendations, or recommendations are irrelevant.

### 4. completeness (0–10)

Does the report cover all required sections and address the breadth of failure modes?

- **10**: Report includes all expected sections (Signal Summary, Failure Taxonomy, Detailed Analysis per top failure mode, Prioritized Action Plan, Limitations). The Signal Summary matches the mode used (eval health table for Eval Signal, error type table for Error Signal, anomaly table for Generic). Covers all key failure modes from expected findings.
- **7–9**: All major sections present. One failure mode underexplored or one section thin.
- **4–6**: Core sections present but several are shallow. Failure taxonomy is incomplete (e.g., only 1–2 modes when 4+ were expected).
- **1–3**: Large sections missing or severely truncated. Failure analysis is superficial.
- **0**: Report is empty, extremely brief, or entirely off-topic.

---

## Output

Return a JSON object with the following fields:
- `diagnosis_accuracy`: integer 0–10
- `evidence_grounding`: integer 0–10
- `actionability`: integer 0–10
- `completeness`: integer 0–10
- `reasoning`: 2–4 sentences explaining the scores, citing specific strengths and weaknesses in the report

id,preference_type,tag_key,tag_value,owner,confidence,owner_type,handle,exclusion_type,exclusion_resource_type,prompt_text,priority
1,tag_mapping,cost-center,CC-100,team-platform,high,team,,,,,
2,tag_mapping,cost-center,CC-200,team-data-eng,high,team,,,,,
3,tag_mapping,cost-center,CC-300,team-security,high,team,,,,,
4,tag_mapping,project,atlas,team-atlas,medium,team,,,,,
5,tag_mapping,project,hermes,alice@example.com,medium,user,,,,,
6,tag_mapping,env,production,sre-team,low,team,,,,,
7,tag_mapping,managed-by,,team-infra,low,team,,,,,
8,exclusion,,,,,,deploy-bot,,,,
9,exclusion,,,,,,ci-runner,service,,,
10,exclusion,,,,,,github-actions,service,,,
11,exclusion,,,,,,legacy-ops,team,aws_ec2_instance,,
12,prompt_text,,,,,,,,,Our organization assigns ownership by cost center. The cost-center tag is the primary ownership signal for all cloud resources. Team identifiers always use the team- prefix followed by the team name (e.g. team-platform team-data-eng).,high
13,prompt_text,,,,,,,,,Shared infrastructure accounts (deploy-bot ci-runner github-actions) are automation accounts and should never be assigned as resource owners. Look for the human or team that configured the automation instead.,medium
14,prompt_text,,,,,,,,,For container images the repository owner in GitHub is a reliable secondary signal when cost-center tags are missing.,low

MIT License

Copyright (c) 2026, Datadog <info@datadoghq.com>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.