Phoenix Cli
Turn messy trace observations and open-coding notes into named failure categories with counts so you can design evals and prioritize fixes for an LLM or agent product.
Overview
Axial Coding is an agent skill most often used in Ship—Testing (also Validate—scope and Operate—iterate) that groups trace-grounded observations into counted failure taxonomies via Phoenix CLI annotations.
Install
npx skills add https://github.com/arize-ai/phoenix --skill phoenix-cliWhat is this skill?
- Groups open-ended observations into MECE-style failure taxonomies with named categories and counts
- Runs through Phoenix CLI `annotate` using a reused coding annotation identifier (`CODING_ANNOTATION_IDENTIFIER`)
- Chains naturally after open coding but can start from any observation set
- Feeds downstream eval scoping and fix-priority decisions from real traces, not invented top-down labels
- Documents the workflow term vs server annotation name (`coding_session_id`) for UI filter compatibility
Adoption & trust: 674 installs on skills.sh; 10k GitHub stars; 2/3 security scanners passed (skills.sh audits).
What problem does it solve?
You have piles of trace notes and observations but no shared, counted taxonomy to decide which evals to build or which bugs to fix first.
Who is it for?
Solo builders shipping agents or LLM features who use Arize Phoenix, already collected observations, and need MECE-style failure groups before eval and fix planning.
Skip if: Teams without Phoenix CLI access, without real trace observations, or who only want a one-off brainstormed category list with no annotation-backed counts.
When should I use this skill?
Reach for Axial Coding whenever the user has observations and needs structure—e.g., “what categories of failures do we have”, “what should I build evals for”, “how do I prioritize fixes”, “group these notes”, “MECE break
What do I get? / Deliverables
You get a structured failure taxonomy with named categories and counts tied to your Phoenix coding session, ready for eval design and prioritized fixes—typically after open coding when that step was used.
- Named failure categories grouped from observations
- Per-category counts suitable for prioritization
- Annotation-backed session data in Phoenix for eval and fix follow-up
Recommended Skills
Journey fit
Spans multiple journey phases - primary shelf plus alternate fits below.
Canonical shelf is Ship because the skill’s payoff is eval design, QA structure, and fix prioritization grounded in production-like traces—not early ideation alone. Testing subphase fits axial coding’s role: classifying failures before you write eval suites, regression checks, and release gates.
Where it fits
After a prototype spike, you axial-code session notes to see which failure modes block a credible MVP demo.
Before writing eval cases, you group trace failures so each eval maps to a named category with a count baseline.
You turn reviewer observations on agent runs into a shared taxonomy instead of one-off bug comments.
Monthly you re-run axial coding on new traces to see which failure categories grew and deserve the next fix sprint.
You align product analytics themes with trace-derived failure categories to explain quality-related churn signals.
How it compares
Use this for trace-anchored qualitative coding in Phoenix, not a generic spreadsheet or chat-only categorization pass.
Common Questions / FAQ
Who is Axial Coding for?
Indie and solo builders running LLM or agent products with Phoenix who need to turn open coding or ad-hoc observations into countable failure categories for evals and prioritization.
When should I use Axial Coding?
Use it when you have observations and need structure—during Ship when scoping evals and test plans, during Validate when narrowing what to prove before build, or during Operate when ranking production failure themes from recent traces.
Is Axial Coding safe to install?
Treat it like any third-party agent skill: review the Security Audits panel on this Prism detail page and limit Phoenix credentials and network access to environments you trust before running annotate workflows.
Workflow Chain
Requires first: open coding
SKILL.md
READMESKILL.md - Phoenix Cli
# Axial Coding Group open-ended observations into structured failure taxonomies. Axial coding turns notes, trace observations, or open-coding output into named categories with counts, supporting downstream work like eval design and fix prioritization. It works well after [open coding](open-coding.md), but can start from any set of open-ended observations. **Reach for this whenever** the user has observations and needs structure — e.g., "what categories of failures do we have", "what should I build evals for", "how do I prioritize fixes", "group these notes", "MECE breakdown", or any framing that asks for categories or counts grounded in real traces rather than invented top-down. ## Coding annotation identifier (reuse the open-coding value) Reuse the **coding annotation identifier** chosen in open coding — every `annotate` call below passes `--identifier "$CODING_ANNOTATION_IDENTIFIER"` explicitly. In a fresh shell or fresh agent invocation, set `CODING_ANNOTATION_IDENTIFIER` to the same value (recoverable from the wrap-up UI URL or by listing `.px/coding/*.jsonl`); don't mint a new id. See [open-coding.md#coding-annotation-identifier-pick-this-first](open-coding.md#coding-annotation-identifier-pick-this-first) for the rationale and the sanitization rule. > **Workflow term vs. server annotation name.** The skill calls this value the **coding annotation identifier**; the server annotation NAME used for the UI filter stays `coding_session_id` for data compatibility. Don't try to rename the server-side key. ```bash CODING_ANNOTATION_IDENTIFIER="coding-run:chatbot-context-loss-2026-05-06" SLUG=$(echo -n "$CODING_ANNOTATION_IDENTIFIER" | sed 's/[^a-zA-Z0-9_-]/-/g') NOTES_SIDECAR=".px/coding/${SLUG}.jsonl" AXIAL_SIDECAR=".px/coding/${SLUG}-axial.jsonl" ``` ## Choosing the unit Open coding's diagnostic in [open-coding.md#choosing-the-unit-of-analysis](open-coding.md#choosing-the-unit-of-analysis) commits to a unit (trace, span, or session). Axial coding inherits that unit by default — if open coding ran at the session level, axial labels will too; same for trace and span. **An axial label can live at a different level than the note that informed it** — that's a feature, and it works in every direction: - *Trace → span*: a trace-level note "answered shipping when asked about returns" can produce a span-level annotation on the retrieval span once a pattern reveals retrieval as the consistent culprit. - *Trace → session*: a batch of trace-level notes describing single-turn confusion can produce a session-level annotation once you see the pattern is "the agent doesn't track the user's stated context across turns." - *Session → trace*: a session-level note about cross-turn drift may, on closer reading, attribute to one specific turn where the agent dropped the thread; a trace-level annotation can name that turn. Whichever level you write the axial label on, write the matching `coding_session_id` UI-filter annotation on the same entity (see [UI-filter annotation](#ui-filter-annotation) below) so the UI link picks it up. ## Process 1. **Set the coding annotation identifier** — set `CODING_ANNOTATION_IDENTIFIER` to the value used in open coding and re-derive `SLUG`, `NOTES_SIDECAR`, `AXIAL_SIDECAR` (see [Coding annotation identifier](#coding-annotation-identifier-reuse-the-open-coding-value)) 2. **Gather** — read open-coding notes from `$NOTES_SIDECAR` (at the unit committed in open coding); no server round-trip 3. **Pattern** — group notes with common themes 4. **Name** — create actionable category names 5. **Attribute** — decide what level each category lives at; an axial label can move up (trace → session) or down (trace → span) from the source note's level to the level the pattern actually implicates 6. **Record** — `px {trace,span,session} annotate ... --name axial_coding_category --label <cat> --identifier "$CODING_ANNOTATION_IDENTIFIER"`, add/update one JSONL sidecar row for the label, then write the matching `coding_sessi