Sf Ai Agentforce Observability

sf-ai-agentforce-observability: Agentforce Session Tracing Extraction & Analysis

Use this skill when the user needs trace-based observability, not just testing: extract Session Tracing Data Model (STDM) records, work with Parquet datasets, reconstruct session timelines, analyze topic/action latency, or debug agent behavior from Data 360 telemetry.

When This Skill Owns the Task

Use sf-ai-agentforce-observability when the work involves:

• Data 360 / Session Tracing extraction
• .parquet files from Agentforce telemetry
• session timeline reconstruction
• trace-driven debugging of topic routing, action failures, or latency
• Polars / PyArrow-based analysis of large telemetry datasets

Delegate elsewhere when the user is:

• formally testing agents → sf-ai-agentforce-testing
• debugging Apex logs → sf-debug
• authoring or reconfiguring the agent itself → sf-ai-agentforce or sf-ai-agentscript

---

Prerequisites That Must Exist

Before extraction, verify:

• Data 360 is enabled
• Session Tracing is enabled
• the Salesforce Standard Data Model version is sufficient
• Einstein / Agentforce capabilities are enabled in the org
• JWT / ECA auth for Data 360 access is configured

If auth is missing, hand off to:

• sf-connected-apps

Deep setup guide:

• references/auth-setup.md

---

What This Skill Works With

Core storage / analysis model

• extraction via Data 360 APIs
• Parquet for storage efficiency
• Polars for large-scale lazy analysis

Core STDM entities

At minimum, expect work around:

• session
• interaction / turn
• interaction step
• moment
• message

GenAI Trust Layer / audit records may also be relevant for content-quality and generation debugging.

Full schema:

• references/data-model-reference.md

---

Required Context to Gather First

Ask for or infer:

• target org alias
• time window or date range
• agent filter, if any
• whether the goal is extraction, summary analysis, or single-session debugging
• output location for extracted data
• whether the user already has Parquet files on disk

---

Recommended Workflow

1. Verify setup and auth

Confirm Data 360 tracing exists and JWT/ECA auth is working.

2. Choose the extraction mode

3. Extract to Parquet

Use the provided scripts under scripts/ rather than reimplementing extraction logic.

4. Analyze with Polars

Common analysis goals:

• session volume and duration
• topic distribution
• action step failures
• latency hotspots
• abandonment / escalation patterns
• session-level timeline reconstruction

5. Convert findings into next actions

Typical outcomes:

• topic mismatch → improve routing or descriptions
• action failure → inspect Flow / Apex implementation
• latency issue → optimize downstream action path
• test gap → add targeted agent tests

---

High-Signal Operational Rules

• treat STDM as read-only telemetry
• expect ingestion lag; this is not perfect real-time debugging
• use date filters and focused extraction to avoid unnecessary volume / query cost
• prefer Parquet over ad hoc JSON for durable analysis
• use lazy Polars patterns for large datasets

Common pitfalls:

• assuming missing data means no issue, when tracing may simply not be enabled
• running huge broad queries without date or agent filters
• trying to fix the agent inside this skill instead of handing off to authoring / testing skills

---

Output Format

When finishing, report in this order: 1. What data was extracted or analyzed 2. Scope (org, dates, agent filter, session IDs) 3. Key findings 4. Likely root causes 5. Recommended next skill / next action

Suggested shape:

Observability task: <extract / analyze / debug-session>
Scope: <org, dates, agents, session ids>
Artifacts: <directories / parquet files>
Findings: <latency, routing, action, quality, abandonment patterns>
Root cause: <best current explanation>
Next step: <testing, agent fix, flow fix, apex fix>

---

Cross-Skill Integration

---

Reference Map

Start here

• README.md
• references/basic-extraction.md
• references/filtered-extraction.md
• references/cli-reference.md

Data model / querying

• references/data-model-reference.md
• references/query-patterns.md
• references/client-demo-queries.md

Analysis / debugging

• references/analysis-cookbook.md
• references/analysis-examples.md
• references/debugging-sessions.md
• references/polars-cheatsheet.md
• references/agent-execution-lifecycle.md

Auth / troubleshooting

• references/auth-setup.md
• references/troubleshooting.md
• references/billing-and-troubleshooting.md
• references/builder-trace-api.md
• scripts/

---

Score Guide

Credits & Dependencies

Core Dependencies

Data Processing

Authentication

HTTP & CLI

Salesforce APIs Used

Related Salesforce Documentation

• Data Cloud Query API
• Agentforce Session Tracing
• JWT Bearer Flow

Inspiration & Patterns

• sf-data skill: Data extraction patterns and validation scoring
• sf-connected-apps skill: JWT authentication templates
• sf-ai-agentforce-testing skill: Agentforce integration patterns

Acknowledgments

License

This skill is released under the MIT License.

MIT License

Copyright (c) 2024-2026 Jag Valaiyapathy

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

sf-ai-agentforce-observability

Extract and analyze Agentforce session tracing data from Salesforce Data Cloud.

Status: General Availability (GA)

This skill has been validated against live Salesforce orgs and is production-ready.

Features

• High-Volume Extraction: Handle 1-10M records/day via Data Cloud Query API
• Parquet Storage: Efficient columnar storage (10x smaller than JSON)
• Polars Analysis: Lazy evaluation for memory-efficient analysis of 100M+ rows
• Session Debugging: Reconstruct session timelines for troubleshooting
• Incremental Sync: Watermark-based extraction for continuous monitoring
• GenAI Quality Analysis: Extract Trust Layer metrics (toxicity, adherence, resolution)

Quick Start

1. Prerequisites

# Install Python dependencies
pip install polars pyarrow pyjwt cryptography httpx rich click pydantic

# Verify Data Cloud access
sf org display --target-org myorg

2. Configure Authentication

Session tracing extraction requires JWT Bearer auth to the Data Cloud Query API.

# Generate certificate
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
  -keyout ~/.sf/jwt/myorg.key \
  -out ~/.sf/jwt/myorg.crt \
  -subj "/CN=DataCloudAuth"

# Create External Client App (via sf-connected-apps skill)
# Required scopes: cdp_query_api, cdp_profile_api

See references/auth-setup.md for detailed instructions.

3. Extract Session Data

# Extract last 7 days
python3 scripts/cli.py extract --org myorg --days 7 --output ./data

# Extract specific date range
python3 scripts/cli.py extract --org myorg --since 2026-01-01 --until 2026-01-15

# Extract complete session tree for debugging
python3 scripts/cli.py extract-tree --org myorg --session-id "a0x..."

4. Analyze Data

# Session summary
python3 scripts/cli.py analyze --data-dir ./data

# Debug specific session
python3 scripts/cli.py debug-session --data-dir ./data --session-id "a0x..."

# Topic analysis
python3 scripts/cli.py topics --data-dir ./data

# Extract GenAI quality metrics
python3 scripts/cli.py extract-quality --data-dir ./data

5. Use with Python

from scripts.analyzer import STDMAnalyzer
from pathlib import Path

analyzer = STDMAnalyzer(Path("./data"))

# Session summary
print(analyzer.session_summary())

# Step distribution
print(analyzer.step_distribution())

# Message timeline for debugging
print(analyzer.message_timeline("a0x..."))

Data Model

24 DMOs verified via T6 live API testing (January 2026):

Session Tracing DMOs (5)

AIAgentSession (18 fields)
├── AIAgentSessionParticipant (12 fields) - Roles: USER, AGENT
├── AIAgentInteraction (20 fields) - Types: TURN, SESSION_END
│   ├── AIAgentInteractionStep (23 fields) - Types: LLM_STEP, ACTION_STEP, TOPIC_STEP
│   └── AIAgentInteractionMessage (21 fields) - Types: Input, Output
└── AIAgentMoment (13 fields) - Contains Agent API name

GenAI Audit & Feedback DMOs (13) ✅ T6 Verified

GenAIGatewayRequest (30 fields) - LLM call details, token usage
├── GenAIGatewayResponse (8 fields)
├── GenAIGeneration (11 fields) - LLM output text
│   ├── GenAIContentQuality (10 fields) - Trust Layer assessment
│   │   └── GenAIContentCategory (10 fields) - Detector results
│   └── GenAIFeedback (16 fields) - User thumbs up/down
│       └── GenAIFeedbackDetail (10 fields) - Feedback comments
└── GenAIGatewayRequestTag, GenAIGtwyRequestMetadata, GenAIGtwyObjRecord...

Key Enum Values (Live API Verified)

DMOs NOT Found (3)

• GenAIRetrieverResponse__dlm ❌
• GenAIRetrieverRequest__dlm ❌
• GenAIRetrieverQualityMetric__dlm ❌

Important: Data Cloud uses AiAgent (lowercase 'i') in field names, not AIAgent.

See references/data-model-reference.md for full schema.

Output Format

Data is stored in Parquet format:

stdm_data/
├── sessions/data.parquet
├── interactions/data.parquet
├── steps/data.parquet
├── messages/data.parquet
├── generations/data.parquet        # GenAI quality
├── content_quality/data.parquet    # Trust Layer
├── content_categories/data.parquet # Toxicity/Adherence
└── metadata/
    ├── extraction.json
    └── watermark.json

Real-Time Trace Analysis

For real-time agent trace capture and analysis (v1.1 PlanSuccessResponse with 13 step types), use sf-ai-agentforce-testing Phase F via sf agent preview start/send/end. This replaces the previous browser-based capture approach with a simpler CLI workflow.

CLI Reference

See references/cli-reference.md for all options.

Validation

This skill includes comprehensive validation testing:

Total: 260+ tests | 100% pass rate (34 discovery tests skip as expected for non-existent DMOs)

Run validation:

cd validation
source .venv/bin/activate
pytest scenarios/ -v --org YourOrgAlias

Integration with Other Skills

Requirements

• Python 3.10+
• Salesforce org with Data Cloud and Agentforce enabled
• Session Tracing enabled in Agentforce settings
• JWT Bearer auth configured (via External Client App)
• Salesforce Standard Data Model v1.124+

Resources

• Data Model Reference - Full STDM schema
• Query Patterns - SQL examples for Data Cloud
• Analysis Cookbook - Polars analysis patterns
• Polars Cheatsheet - Quick reference

License

MIT License - See LICENSE file.

Author

Jag Valaiyapathy

---

Last updated: January 2026 | Validated against live org

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Agent Execution Lifecycle — 6-Phase Model

How the Agentforce platform processes each user message internally, based on live Builder trace captures. Maps AgentScript syntax to the 13 real-time trace step types.

Visual version: Generated runtime artifact at ~/.agent/diagrams/agent-execution-lifecycle.html with Mermaid flowcharts, timing bars, and side-by-side script-to-trace mapping.

Data source: Live Builder traces from a test agent, sandbox org, 2026-02-25.

---

The 6 Phases

Every user message triggers this pipeline:

Phase 1: Input & Init          (automatic — platform-managed)
Phase 2: Topic Selection        (script-driven — AgentScript start_agent)
    ↓ Decision: LLM invoked go_* tool?
    ├─ Yes → Phase 3 + Phase 4
    └─ No  → Phase 5 (short-circuit)
Phase 3: Topic Transition       (automatic — TransitionStep)
Phase 4: Topic Execution        (LLM + actions — AgentScript topic)
Phase 5: Trust Layer            (automatic — grounding + safety)
Phase 6: Response Delivery      (automatic — final message)

Full path (Turn 1): 1 → 2 → 3 → 4 → 5 → 6 — 46 steps, 4.6s Short-circuit (Turn 2): 1 → 2 → 5 → 6 — 15 steps, 2.6s

The short-circuit occurs when the Topic Selector has enough conversation context to respond directly without routing to a topic.

---

Phase → Step Type Mapping

Phase 1: Input & Init

Phase 2: Topic Selection

Phase 3: Topic Transition

Phase 4: Topic Execution

Phase 5: Trust Layer

Phase 6: Response Delivery

The PlannerResponseStep message is delivered to the user. No additional trace steps.

---

Script → Trace Mapping Quick Reference

---

Timing Analysis

Turn 1 (Full Path — 46 steps, 4,556ms)

Turn 2 (Short-Circuit — 15 steps, 2,564ms)

Key insight: Grounding evaluation costs ~1s on every turn — as expensive as an LLM call. It's the fixed cost of Agentforce's safety guarantees.

---

Conversation Context Growth

The messages_sent[] array in LLMStep grows per turn:

• Turn 1: 3 messages (system prompt, user input, assembled instructions)
• Turn 2: 7 messages (system prompt, Turn 1 user/assistant/tool/assistant, Turn 2 user, instructions)

Each turn adds ~4 messages. After 10 turns, the LLM processes 30+ messages.

---

Dynamic Instruction Assembly

Each set on AgentScriptInternal_agent_instructions generates one VariableUpdateStep that appends a line via template evaluation. The instructions are reset to "" at the start of each phase and rebuilt from scratch.

Topic Selector assembly (Turn 1, 6 steps):

Step  7: "" → "\nCustomer: Test Customer"
Step  8: → + "This agent specializes in product troubleshooting..."
Step  9: → + "ROUTING:"
Step 10: → + "- ANY product issue → Product Help"
Step 11: → + "- Customer requests human agent → Escalation"
Step 12: → + "- ANY non-product request → Escalation"

The final value becomes the last system message in the LLM prompt.

---

State Variables (24 at Entry)

Full state captured at NodeEntryStateStep (Turn 1):

---

STDM Span Types (Testing Center Trace View)

When viewing session traces in Testing Center (not Builder), steps are collapsed into 6 color-coded span types:

This mapping is critical for correlating Builder-level debugging with Testing Center trace analysis. A single TOPIC_STEP span in Testing Center may represent multiple NodeEntryStateStep + TransitionStep events at the Builder level.

---

Related Documents

• builder-trace-api.md — Reverse-engineered API endpoints and capture methodology
• data-model-reference.md — Complete 24-DMO STDM schema (persisted layer)
• query-patterns.md — Data Cloud SQL for persisted trace data

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Analysis Cookbook

Common analysis patterns using Polars for session tracing data.

Getting Started

from pathlib import Path
import polars as pl

# Load data as lazy frames (memory efficient)
data_dir = Path("./stdm_data")

sessions = pl.scan_parquet(data_dir / "sessions" / "**/*.parquet")
interactions = pl.scan_parquet(data_dir / "interactions" / "**/*.parquet")
steps = pl.scan_parquet(data_dir / "steps" / "**/*.parquet")
messages = pl.scan_parquet(data_dir / "messages" / "**/*.parquet")

---

Session Analysis

Basic Statistics

# Session count by agent
sessions.group_by("ssot__AiAgentApiName__c").agg(
    pl.count().alias("session_count")
).sort("session_count", descending=True).collect()

Completion Rate by Agent

sessions.group_by("ssot__AiAgentApiName__c").agg([
    pl.count().alias("total"),
    pl.col("ssot__AiAgentSessionEndType__c")
      .filter(pl.col("ssot__AiAgentSessionEndType__c") == "Completed")
      .count().alias("completed")
]).with_columns([
    (pl.col("completed") / pl.col("total") * 100).round(1).alias("completion_rate")
]).collect()

Session Duration Analysis

# Note: Requires timestamp parsing
sessions.with_columns([
    pl.col("ssot__StartTimestamp__c").str.to_datetime().alias("start"),
    pl.col("ssot__EndTimestamp__c").str.to_datetime().alias("end"),
]).with_columns([
    (pl.col("end") - pl.col("start")).alias("duration")
]).group_by("ssot__AiAgentApiName__c").agg([
    pl.col("duration").mean().alias("avg_duration"),
    pl.col("duration").max().alias("max_duration"),
]).collect()

---

Interaction Analysis

Turns per Session

turns_per_session = (
    interactions
    .filter(pl.col("ssot__AiAgentInteractionType__c") == "TURN")
    .group_by("ssot__AiAgentSessionId__c")
    .agg(pl.count().alias("turn_count"))
)

# Distribution
turns_per_session.group_by("turn_count").agg(
    pl.count().alias("sessions")
).sort("turn_count").collect()

Topic Routing

# Most common topics
interactions.filter(
    pl.col("ssot__AiAgentInteractionType__c") == "TURN"
).group_by("ssot__TopicApiName__c").agg(
    pl.count().alias("turn_count"),
    pl.col("ssot__AiAgentSessionId__c").n_unique().alias("session_count")
).sort("turn_count", descending=True).collect()

Topic Switches (Multi-Topic Sessions)

# Sessions that used multiple topics
topic_counts = (
    interactions
    .filter(pl.col("ssot__AiAgentInteractionType__c") == "TURN")
    .group_by("ssot__AiAgentSessionId__c")
    .agg(pl.col("ssot__TopicApiName__c").n_unique().alias("topic_count"))
)

# Sessions with 2+ topics
topic_counts.filter(pl.col("topic_count") > 1).collect()

---

Step Analysis

LLM vs Action Ratio

steps.group_by("ssot__AiAgentInteractionStepType__c").agg(
    pl.count().alias("count")
).with_columns([
    (pl.col("count") / pl.col("count").sum() * 100).round(1).alias("percentage")
]).collect()

Most Used Actions

steps.filter(
    pl.col("ssot__AiAgentInteractionStepType__c") == "ACTION_STEP"
).group_by("ssot__Name__c").agg(
    pl.count().alias("invocations")
).sort("invocations", descending=True).head(20).collect()

Steps per Turn Distribution

steps_per_turn = (
    steps
    .group_by("ssot__AiAgentInteractionId__c")
    .agg(pl.count().alias("step_count"))
)

steps_per_turn.group_by("step_count").agg(
    pl.count().alias("turns")
).sort("step_count").collect()

Action Input/Output Analysis

# Parse JSON in action inputs (example for specific action)
action_steps = steps.filter(
    (pl.col("ssot__AiAgentInteractionStepType__c") == "ACTION_STEP") &
    (pl.col("ssot__Name__c") == "Get_Order_Status")
).collect()

# Parse inputs
import json
for row in action_steps.iter_rows(named=True):
    input_data = json.loads(row["ssot__InputValueText__c"] or "{}")
    output_data = json.loads(row["ssot__OutputValueText__c"] or "{}")
    print(f"Input: {input_data}, Output: {output_data}")

---

Message Analysis

Message Length Distribution

messages.with_columns([
    pl.col("ssot__ContentText__c").str.len_chars().alias("length")
]).group_by("ssot__AiAgentInteractionMessageType__c").agg([
    pl.col("length").mean().alias("avg_length"),
    pl.col("length").max().alias("max_length"),
]).collect()

Common User Phrases

# Word frequency in user messages (simple)
user_messages = messages.filter(
    pl.col("ssot__AiAgentInteractionMessageType__c") == "INPUT"
).select("ssot__ContentText__c").collect()

# Count words
from collections import Counter
words = Counter()
for row in user_messages.iter_rows(named=True):
    content = row["ssot__ContentText__c"] or ""
    words.update(content.lower().split())

print(words.most_common(20))

---

Time-Based Analysis

Sessions by Date

sessions.with_columns([
    pl.col("ssot__StartTimestamp__c").str.slice(0, 10).alias("date")
]).group_by("date").agg(
    pl.count().alias("sessions")
).sort("date").collect()

Sessions by Hour

sessions.with_columns([
    pl.col("ssot__StartTimestamp__c").str.slice(11, 2).alias("hour")
]).group_by("hour").agg(
    pl.count().alias("sessions")
).sort("hour").collect()

Day of Week Analysis

# Requires datetime conversion
sessions.with_columns([
    pl.col("ssot__StartTimestamp__c")
      .str.to_datetime()
      .dt.weekday()
      .alias("weekday")
]).group_by("weekday").agg(
    pl.count().alias("sessions")
).sort("weekday").collect()

---

Debugging Patterns

Find Failed Sessions

failed = sessions.filter(
    pl.col("ssot__AiAgentSessionEndType__c").is_in(["Escalated", "Abandoned", "Failed"])
).sort("ssot__StartTimestamp__c", descending=True).collect()

for row in failed.head(5).iter_rows(named=True):
    print(f"Session: {row['ssot__Id__c']}")
    print(f"  Agent: {row['ssot__AiAgentApiName__c']}")
    print(f"  End: {row['ssot__AiAgentSessionEndType__c']}")
    print()

Session Timeline Reconstruction

def get_timeline(session_id: str) -> pl.DataFrame:
    """Reconstruct message timeline for a session."""
    # Get interaction IDs
    interaction_df = interactions.filter(
        pl.col("ssot__AiAgentSessionId__c") == session_id
    ).collect()

    interaction_ids = interaction_df["ssot__Id__c"].to_list()

    # Get messages
    msg_df = messages.filter(
        pl.col("ssot__AiAgentInteractionId__c").is_in(interaction_ids)
    ).sort("ssot__MessageSentTimestamp__c").collect()

    return msg_df

# Usage
timeline = get_timeline("a0x1234567890ABC")
for row in timeline.iter_rows(named=True):
    msg_type = row["ssot__AiAgentInteractionMessageType__c"]
    content = row["ssot__ContentText__c"]
    icon = "→" if msg_type == "INPUT" else "←"
    print(f"{icon} {content[:100]}...")

---

Performance Tips

Use Lazy Evaluation

# Good: Lazy evaluation, deferred execution
result = (
    sessions
    .filter(pl.col("ssot__AiAgentApiName__c") == "My_Agent")
    .group_by("ssot__AiAgentSessionEndType__c")
    .agg(pl.count())
    .collect()  # Execute here
)

# Avoid: Eager loading of everything
df = pl.read_parquet(data_dir / "sessions" / "**/*.parquet")  # Loads all data
result = df.filter(...)  # Then filter

Select Only Needed Columns

# Good: Select specific columns
sessions.select([
    "ssot__Id__c",
    "ssot__AiAgentApiName__c",
    "ssot__AiAgentSessionEndType__c"
]).collect()

# Avoid: Select all columns
sessions.collect()  # Includes all columns

Use Streaming for Large Results

# For very large datasets
for batch in sessions.collect(streaming=True):
    # Process batch
    pass

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Analysis Examples

Examples for analyzing extracted STDM data with Polars and the CLI.

---

CLI Analysis Commands

Summary Statistics

Get high-level overview of extracted data:

stdm-extract analyze --data-dir ./stdm_data

Output:

📊 SESSION SUMMARY
═══════════════════════════════════════════════════════════

Agent                        Sessions  Completed  Escalated  Completion %
───────────────────────────────────────────────────────────────────────────
Customer_Support_Agent           450        382         45        84.9%
Order_Tracking_Agent             312        298          8        95.5%
FAQ_Agent                        156        152          2        97.4%
───────────────────────────────────────────────────────────────────────────
TOTAL                            918        832         55        90.6%

Topic Analysis

See which topics handle the most turns:

stdm-extract topics --data-dir ./stdm_data

Output:

📊 TOPIC ROUTING
═══════════════════════════════════════════════════════════

Topic                        Turns  Sessions  Avg Turns/Session
───────────────────────────────────────────────────────────────
Order_Status                  1234       312              4.0
Return_Policy                  856       245              3.5
Account_Help                   654       198              3.3
General_FAQ                    432       156              2.8

Action Analysis

See which actions are invoked most:

stdm-extract actions --data-dir ./stdm_data

Output:

📊 ACTION INVOCATIONS
═══════════════════════════════════════════════════════════

Action                       Count  Avg/Session
─────────────────────────────────────────────────
Get_Order_Status              1856         2.0
Search_Knowledge_Base         1245         1.4
Create_Case                    567         0.6
Update_Contact                 234         0.3

---

Python Analysis

Setup

import polars as pl
from pathlib import Path

data_dir = Path("./stdm_data")

# Lazy loading (memory efficient)
sessions = pl.scan_parquet(data_dir / "sessions" / "**/*.parquet")
interactions = pl.scan_parquet(data_dir / "interactions" / "**/*.parquet")
steps = pl.scan_parquet(data_dir / "steps" / "**/*.parquet")
messages = pl.scan_parquet(data_dir / "messages" / "**/*.parquet")

Session Analysis

Sessions by End Type:

sessions.group_by("ssot__AiAgentSessionEndType__c").agg(
    pl.count().alias("count")
).sort("count", descending=True).collect()

Daily Session Trend:

sessions.with_columns(
    pl.col("ssot__StartTimestamp__c").str.slice(0, 10).alias("date")
).group_by("date").agg(
    pl.count().alias("sessions")
).sort("date").collect()

Hourly Distribution:

sessions.with_columns(
    pl.col("ssot__StartTimestamp__c").str.slice(11, 2).alias("hour")
).group_by("hour").agg(
    pl.count().alias("sessions")
).sort("hour").collect()

Turn Analysis

Turns Per Session:

turns_per_session = (
    interactions
    .filter(pl.col("ssot__AiAgentInteractionType__c") == "TURN")
    .group_by("ssot__AiAgentSessionId__c")
    .agg(pl.count().alias("turns"))
)

# Distribution
turns_per_session.group_by("turns").agg(
    pl.count().alias("sessions")
).sort("turns").collect()

Multi-Topic Sessions (Topic Switches):

topic_counts = (
    interactions
    .filter(pl.col("ssot__AiAgentInteractionType__c") == "TURN")
    .group_by("ssot__AiAgentSessionId__c")
    .agg(pl.col("ssot__TopicApiName__c").n_unique().alias("topics"))
)

# Sessions with 2+ topics
topic_counts.filter(pl.col("topics") > 1).collect()

Step Analysis

LLM vs Action Ratio:

steps.group_by("ssot__AiAgentInteractionStepType__c").agg(
    pl.count().alias("count")
).with_columns(
    (pl.col("count") / pl.col("count").sum() * 100).round(1).alias("percent")
).collect()

Most Used Actions:

steps.filter(
    pl.col("ssot__AiAgentInteractionStepType__c") == "ACTION_STEP"
).group_by("ssot__Name__c").agg(
    pl.count().alias("invocations")
).sort("invocations", descending=True).head(10).collect()

Steps Per Turn:

steps.group_by("ssot__AiAgentInteractionId__c").agg(
    pl.count().alias("steps")
).group_by("steps").agg(
    pl.count().alias("turns")
).sort("steps").collect()

Message Analysis

Average Message Length by Type:

messages.with_columns(
    pl.col("ssot__ContentText__c").str.len_chars().alias("length")
).group_by("ssot__AiAgentInteractionMessageType__c").agg([
    pl.col("length").mean().round(0).alias("avg_length"),
    pl.col("length").max().alias("max_length"),
]).collect()

Common User Phrases:

user_msgs = messages.filter(
    pl.col("ssot__AiAgentInteractionMessageType__c") == "INPUT"
).select("ssot__ContentText__c").collect()

from collections import Counter
words = Counter()
for row in user_msgs.iter_rows(named=True):
    text = row["ssot__ContentText__c"] or ""
    words.update(text.lower().split())

print(words.most_common(20))

---

Using the Analyzer Class

The skill includes a helper class for common analyses:

from scripts.analyzer import STDMAnalyzer

analyzer = STDMAnalyzer(Path("./stdm_data"))

# Quick summary
summary = analyzer.session_summary()
print(summary)

# Step breakdown
steps_df = analyzer.step_distribution()
print(steps_df)

# Topic patterns
topics_df = analyzer.topic_analysis()
print(topics_df)

# Find failed sessions
failed = analyzer.find_failed_sessions(limit=10)
print(failed)

---

Exporting Results

To CSV

result = sessions.group_by(...).agg(...).collect()
result.write_csv("output.csv")

To JSON

result.write_json("output.json")

To Parquet (for further analysis)

result.write_parquet("output.parquet")

---

Visualization (with Matplotlib)

import matplotlib.pyplot as plt

# Daily trend
daily = sessions.with_columns(
    pl.col("ssot__StartTimestamp__c").str.slice(0, 10).alias("date")
).group_by("date").agg(
    pl.count().alias("sessions")
).sort("date").collect()

plt.figure(figsize=(12, 4))
plt.bar(daily["date"], daily["sessions"])
plt.xticks(rotation=45)
plt.title("Sessions per Day")
plt.tight_layout()
plt.savefig("daily_trend.png")

---

See Also

• Polars Cheatsheet - Quick reference
• Analysis Cookbook - More recipes
• Debugging Sessions - Session-level debugging

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Authentication Setup

This skill uses JWT Bearer authentication to access the Data Cloud Query API. Authentication is configured via an External Client App (ECA) in Salesforce.

Prerequisites

1. Salesforce CLI installed and authenticated to your org 2. OpenSSL for certificate generation 3. Data Cloud enabled in your org 4. System Administrator profile (or appropriate permissions for ECA setup)

---

Quick Setup (5 Steps)

Step 1: Generate JWT Certificate

# Create directory for JWT keys
mkdir -p ~/.sf/jwt

# Generate private key and self-signed certificate (valid 1 year)
# Naming convention: {org_alias}-agentforce-observability.key
openssl req -x509 -sha256 -nodes -days 365 \
    -newkey rsa:2048 \
    -keyout ~/.sf/jwt/YourOrg-agentforce-observability.key \
    -out ~/.sf/jwt/YourOrg-agentforce-observability.crt \
    -subj "/CN=AgentforceObservability/O=YourOrg"

# Secure the private key (required - Salesforce rejects world-readable keys)
chmod 600 ~/.sf/jwt/YourOrg-agentforce-observability.key

Step 2: Create External Client App

In Salesforce Setup:

1. Setup → External Client App Manager → New External Client App 2. Fill in the basic details:

3. Click Save

Step 3: Configure OAuth Settings

In the ECA → OAuth Settings tab:

Upload Certificate: 1. Check Use digital signatures 2. Click Choose File 3. Upload ~/.sf/jwt/{org}-agentforce-observability.crt 4. Click Save

Step 4: Configure App Policies

In the ECA → Policies tab:

Add Your User: 1. Click Manage → Manage Profiles or Manage Permission Sets 2. Add your user's profile or an appropriate permission set 3. Click Save

Step 5: Get Consumer Key & Test

1. Go to ECA → OAuth Settings tab 2. Copy the Consumer Key (starts with 3MVG9...) 3. Test authentication:

# Option 1: Pass consumer key directly
python3 scripts/cli.py test-auth \
    --org YourOrg \
    --consumer-key "3MVG9..."

# Option 2: Use environment variable
export SF_CONSUMER_KEY="3MVG9..."
python3 scripts/cli.py test-auth --org YourOrg

Expected output:

Testing Authentication
Org: YourOrg
Key: /Users/you/.sf/jwt/YourOrg-agentforce-observability.key

Getting org info...
Instance: https://your-org.my.salesforce.com
Username: your.user@example.com
Sandbox: True

Testing token generation...
✓ Token generated

Testing Data Cloud access...
✓ Data Cloud accessible

Authentication successful!

---

Key Path Resolution

The CLI resolves the private key path in this order:

1. Explicit `--key-path` argument (highest priority) 2. App-specific key: ~/.sf/jwt/{org_alias}-agentforce-observability.key 3. Generic org key: ~/.sf/jwt/{org_alias}.key (fallback)

Examples

# Uses app-specific key automatically
python3 scripts/cli.py test-auth --org YourOrg --consumer-key "..."
# → Resolves to: ~/.sf/jwt/YourOrg-agentforce-observability.key

# Explicit key path (overrides all defaults)
python3 scripts/cli.py test-auth --org YourOrg \
    --consumer-key "..." \
    --key-path ~/.sf/jwt/custom-key.key

---

File Locations

---

Required OAuth Scopes

---

Troubleshooting

invalid_grant Error

RuntimeError: Token exchange failed: invalid_grant

Causes & Fixes:

Verify certificate expiry:

openssl x509 -enddate -noout -in ~/.sf/jwt/YourOrg-agentforce-observability.crt

403 Forbidden

RuntimeError: Access denied: Ensure ECA has cdp_query_api scope

Fix: Add cdp_query_api scope to the ECA OAuth Settings.

Key Not Found

FileNotFoundError: Private key not found at ~/.sf/jwt/myorg.key

Causes: 1. Key file doesn't exist → Generate with Step 1 2. Wrong org alias → Check sf org list for correct alias 3. Using old naming convention → Rename to {org}-agentforce-observability.key

Permission Denied on Key

PermissionError: [Errno 13] Permission denied: '~/.sf/jwt/...'

Fix:

chmod 600 ~/.sf/jwt/*.key

---

Sandbox vs Production

The skill automatically detects sandbox orgs:

1. Reads isSandbox from sf org display --json 2. Uses https://test.salesforce.com for token exchange (sandboxes) 3. Uses https://login.salesforce.com for production

No additional configuration needed.

---

Security Best Practices

1. Protect private keys: Always chmod 600 ~/.sf/jwt/*.key 2. Rotate certificates annually: Regenerate before expiration 3. Use separate ECAs: One per org/environment for isolation 4. Never commit keys: Add *.key to .gitignore 5. Audit access: Review ECA usage in Setup → Security → Connected Apps OAuth Usage

---

Environment Variables

For automation/CI, set these environment variables:

# Consumer key (org-specific takes priority)
export SF_YOURORG_CONSUMER_KEY="3MVG9..."
# or generic fallback
export SF_CONSUMER_KEY="3MVG9..."

The CLI checks in this order: 1. --consumer-key argument 2. SF_{ORG_ALIAS}_CONSUMER_KEY (uppercase, hyphens to underscores) 3. SF_CONSUMER_KEY

---

See Also

• sf-connected-apps skill - General ECA patterns
• CLI Reference - All command options
• Troubleshooting - More error solutions

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Basic Extraction Examples

Simple examples to get started with STDM extraction.

Prerequisites

1. JWT authentication configured (see Auth Setup) 2. Consumer key set: export SF_CONSUMER_KEY="3MVG9..." 3. Org alias available in sf org list

---

Example 1: Extract Last 7 Days

The simplest extraction - all sessions from the last 7 days:

stdm-extract extract --org prod

Output structure:

./stdm_data/
├── sessions/
│   └── data.parquet
├── interactions/
│   └── data.parquet
├── steps/
│   └── data.parquet
└── messages/
    └── data.parquet

---

Example 2: Extract Last 24 Hours

stdm-extract extract --org prod --days 1

---

Example 3: Custom Output Directory

stdm-extract extract --org prod --output /data/agentforce/prod

---

Example 4: Verbose Mode

See detailed progress and timing:

stdm-extract extract --org prod --verbose

Sample output:

🔐 Authenticating to Data Cloud...
   Instance URL: https://myorg.my.salesforce.com

📊 Extracting sessions...
   Query: SELECT ... FROM ssot__AIAgentSession__dlm WHERE ...
   Records: 1,234 (2.3s)

📊 Extracting interactions...
   Records: 5,678 (4.1s)

📊 Extracting steps...
   Records: 12,345 (8.7s)

📊 Extracting messages...
   Records: 9,876 (5.2s)

✅ Extraction complete!
   Total records: 29,133
   Duration: 20.3s
   Output: ./stdm_data/

---

Example 5: Test Authentication First

Before extracting, verify your setup:

stdm-extract test-auth --org prod

Success:

✅ Authentication successful
   Instance URL: https://myorg.my.salesforce.com
   Token valid for: 3599 seconds

Failure:

❌ Authentication failed
   Error: invalid_grant
   Hint: Check certificate expiration with:
         openssl x509 -enddate -noout -in ~/.sf/jwt/prod.key

---

Example 6: Count Records Before Extraction

Check how much data exists without downloading:

stdm-extract count --org prod --dmo sessions

Output:

📊 Record counts for prod:
   Sessions: 12,345

---

Example 7: Extract from Sandbox

Works the same as production - just use the sandbox alias:

# List your orgs
sf org list

# Extract from sandbox
stdm-extract extract --org mysandbox --days 3

---

What's Next?

• Filtered Extraction - Filter by agent, date range
• Analysis Examples - Analyze extracted data
• Debugging Sessions - Debug specific sessions

---

Quick Reference

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Billing Considerations & Troubleshooting

Billing Considerations

Reference: Billing Considerations for Agentforce Session Tracing

Agentforce Session Tracing consumes Data 360 credits for ingestion, storage, and processing.

Credit Consumption

Cost Estimation

Records per session ≈ Turns × 24 (avg per LLM call)
Daily records ≈ Sessions/day × Avg turns × 24

Example:
  1,000 sessions/day × 4 turns × 24 = 96,000 records/day ingested

Tip: Use Digital Wallet for near real-time consumption tracking.

---

Common Issues & Fixes

See references/troubleshooting.md for detailed solutions.

---

Agentforce Credit Costs (Cross-Reference)

When analyzing traces for performance optimization, understanding credit costs per step type is critical:

Optimization insight: When trace analysis reveals excessive action calls, consider consolidating multiple Flow actions into a single Apex action or using framework-level set operations where possible.

See sf-ai-agentscript/references/production-gotchas.md for the full credit consumption model and optimization strategies.

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

Builder Trace API — Reverse-Engineered Internal Endpoint

How the Agentforce Builder's "Trace" tab renders real-time execution telemetry, the exact internal API endpoints, request/response schemas, and how to capture trace data programmatically.

Status: ✅ Confirmed via live network capture (February 2026).

---

Two-Layer Trace Architecture

The Agentforce Builder's Trace tab is powered by two distinct systems:

Layer 1: Real-Time Execution Trace (Aura Controller)

The live trace rendered during Builder testing uses an Aura controller action — confirmed via CDP network capture:

Layer 2: Persisted Session Tracing (STDM in Data Cloud)

After 5-15 minutes, a subset of trace data is persisted to Data Cloud across 24 DMOs:

See data-model-reference.md and query-patterns.md for full STDM documentation.

---

Confirmed Architecture

The Builder uses a two-phase request pattern per user message:

Phase 1: Message Delivery (SSE Stream)
┌─────────────────────────────────────────────────────────────────┐
│ Builder LWC → POST /einstein/ai-agent/v1/sessions/{id}/        │
│                     messages/stream                              │
│ Auth: Bearer token (JWT)                                        │
│ Body: {"message":{"type":"Text","text":"...","sequenceId":"N"}} │
│                                                                  │
│ Response: SSE stream with events:                                │
│   event: TEXT_CHUNK  → streaming word-by-word response           │
│   event: INFORM      → final message + planId + feedbackId       │
│   event: END_OF_TURN → turn complete                             │
└─────────────────────────────────────────────────────────────────┘
                    │
                    │ planId from INFORM event
                    ▼
Phase 2: Trace Retrieval (Aura Controller)
┌─────────────────────────────────────────────────────────────────┐
│ Builder LWC → POST /aura                                        │
│ Descriptor: AgentAuthoringController/getSimulationPlanTraces    │
│ Auth: Browser cookies (X-SFDC-Page-Cache, X-SFDC-Request-Id)   │
│ Params: { planId, sessionId, version: "1.0" }                  │
│                                                                  │
│ Response: PlanSuccessResponse with plan[] array                  │
│   → 13 step types, full variable state, LLM prompts, timing     │
└─────────────────────────────────────────────────────────────────┘

Key discovery: The planId returned in the SSE INFORM event is the same ID used to fetch the trace. This links the message stream to the execution trace.

---

SSE Stream Schema (/messages/stream)

Request

POST https://api.salesforce.com/einstein/ai-agent/v1/sessions/{sessionId}/messages/stream
Authorization: Bearer {jwt_token}
Content-Type: application/json
x-salesforce-region: us-west-2
X-B3-TraceId: {distributed_trace_id}
X-B3-SpanId: {span_id}

{
  "message": {
    "type": "Text",
    "text": "Where is my order?",
    "sequenceId": "1"
  }
}

SSE Event Types

INFORM Event Schema (Critical — Contains planId)

{
  "timestamp": 1772000000000,
  "originEventId": "1772000000000-REQ",
  "traceId": "00000000000000001234567890abcdef",
  "offset": 0,
  "message": {
    "type": "Inform",
    "feedbackId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "isContentSafe": true,
    "message": "I can help you with that. Let me look into it...",
    "id": "11111111-2222-3333-4444-555555555555",
    "metrics": {},
    "planId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "result": [],
    "citedReferences": []
  }
}

Note: planId and feedbackId are identical in the observed capture.

---

Trace API Schema (getSimulationPlanTraces)

Request (Aura)

POST /aura
Content-Type: application/x-www-form-urlencoded
X-SFDC-Page-Cache: {page_cache_id}
X-SFDC-Request-Id: {request_id}
X-B3-TraceId: {trace_id}

URL-encoded body (decoded):

{
  "actions": [{
    "id": "48;a",
    "descriptor": "serviceComponent://ui.agent.authoring.components.aura.controller.AgentAuthoringController/ACTION$getSimulationPlanTraces",
    "callingDescriptor": "UNKNOWN",
    "params": {
      "planId": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
      "sessionId": "019xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "version": "1.0"
    }
  }]
}

Response: PlanSuccessResponse

{
  "type": "PlanSuccessResponse",
  "planId": "aaaaaaaa-...",
  "sessionId": "019xxxxx-...",
  "intent": "DefaultTopic",
  "topic": "DefaultTopic",
  "plan": [
    { "type": "UserInputStep", ... },
    { "type": "SessionInitialStateStep", ... },
    { "type": "EnabledToolsStep", ... },
    { "type": "LLMStep", ... },
    ...
  ]
}

---

Complete Step Type Taxonomy (13 Types)

For the full execution lifecycle and how these steps map to AgentScript syntax, see agent-execution-lifecycle.md.

Captured from a single agent turn (46 steps total):

Common Fields (All Step Types)

Every step has:

{
  "startExecutionTime": 1772000000000,
  "endExecutionTime": 1772000000000,
  "type": "StepTypeName"
}

Timestamps are Unix epoch milliseconds.

---

Detailed Step Schemas

LLMStep (Most Data-Rich)

Contains the full LLM prompt, tool definitions, and response — the most valuable step for debugging.

{
  "startExecutionTime": 1772000001000,
  "endExecutionTime": 1772000002880,
  "type": "LLMStep",
  "data": {
    "agent_name": "Topic Selector",
    "prompt_name": "topic_selector_prompt",
    "prompt_content": "[full prompt string, 4000+ chars]",
    "prompt_response": null,
    "execution_latency": 1879
  },
  "messages_sent": [
    {"role": "system", "content": "Topic Selector & Safety Router..."},
    {"role": "user", "content": "I need help with my order"},
    {"role": "system", "content": "Customer: Jane Doe..."}
  ],
  "tools_sent": [
    "go_order_support",
    "go_escalation",
    "Inappropriate_Content",
    "Prompt_Injection",
    "Reverse_Engineering"
  ],
  "response_messages": [
    {"role": "assistant", "content": "", "tool_invocation": {"name": "go_order_support", "arguments": "{}"}}
  ]
}

FunctionStep (Action Execution)

Shows Flow/Apex invocation with input/output:

{
  "startExecutionTime": 1772000003000,
  "endExecutionTime": 1772000005675,
  "type": "FunctionStep",
  "function": {
    "name": "Get_Order_Status",
    "input": {"orderId": "ORD-12345"},
    "output": {"status": "Shipped", "trackingNumber": "1Z999..."},
    "errors": null
  },
  "executionLatency": 2675
}

TransitionStep (Topic Routing)

Shows topic-to-topic transitions with full state:

{
  "startExecutionTime": 1772000002900,
  "endExecutionTime": 1772000002900,
  "type": "TransitionStep",
  "data": {
    "from_agent": "Topic Selector",
    "to_agent": "order_support",
    "current_state": {"authenticated": true, "customer_name": "Jane Doe", "...": "..."},
    "transition_type": "ROUTING",
    "transition_mode": "DIRECT",
    "directive_context": "on_message"
  }
}

PlannerResponseStep (Final Output + Safety)

Contains the safety score breakdown:

{
  "startExecutionTime": 1772000006800,
  "endExecutionTime": 1772000006800,
  "type": "PlannerResponseStep",
  "message": "I can help you track your order. Let me look that up...",
  "responseType": "Inform",
  "isContentSafe": true,
  "safetyScore": {
    "safetyScore": {
      "safety_score": 0.99,
      "category_scores": {
        "toxicity": 0, "hate": 0, "identity": 0, "violence": 0,
        "physical": 0, "sexual": 0, "profanity": 0, "biased": 0
      }
    }
  }
}

ReasoningStep (Grounding Evaluation)

The agent's self-assessment of response quality:

{
  "startExecutionTime": 1772000005600,
  "endExecutionTime": 1772000006800,
  "type": "ReasoningStep",
  "category": "GROUNDED",
  "reason": "The response follows the mandatory first question as instructed..."
}

---

Real-Time → STDM Field Mapping (Confirmed)

Key finding: The real-time trace contains 13 step types vs 5 in STDM. The most valuable debugging data (full LLM prompts, variable change reasons, safety scores, grounding evaluations) is not persisted to Data Cloud.

---

Other Discovered Endpoints

postInteractionSummary (Aura)

Called after trace retrieval. Sends the full agent script + conversation back for analytics:

Descriptor: aura://AuthoringAgentFamilyController/ACTION$postInteractionSummary
Params: {
  authoringAgentInteractionSummaryInputRepresentation: {
    projectId: "{project_id}",
    currentAFScript: "[full agent script JSON]",
    ...
  }
}

EvfSdkController/publishEvent (Aura Telemetry)

Client-side observability events tagged COPILOT_O11Y:

Descriptor: aura://EvfSdkController/ACTION$publishEvent
Params: {
  event: {
    contextualData: {
      loggerName: "COPILOT_O11Y",
      loggerAppName: "AgentAuthoring:agentAuthoringBuilder"
    },
    feature: "EINSTEIN_..."
  }
}

---

Auth Patterns (Confirmed)

Important: The trace endpoint (getSimulationPlanTraces) uses cookie-based auth only. It cannot be called with a standalone JWT token. You must maintain a browser session.

---

Capture Methodology

Option 1: CLI Preview Commands (Recommended)

The sf agent preview start/send/end commands provide programmatic access to the same v1.1 trace data without browser automation:

# Start session → send utterance → end and get traces
SESSION_ID=$(sf agent preview start --api-name My_Agent --target-org myOrg --json 2>/dev/null | jq -r '.result.sessionId')
PLAN_ID=$(sf agent preview send --session-id "$SESSION_ID" --api-name My_Agent --utterance "test" --target-org myOrg --json 2>/dev/null | jq -r '.result.messages[-1].planId')
TRACES_PATH=$(sf agent preview end --session-id "$SESSION_ID" --api-name My_Agent --target-org myOrg --json 2>/dev/null | jq -r '.result.tracesPath')

# Analyze trace
jq '.' "$TRACES_PATH/$PLAN_ID.json"

For full workflow details, see sf-ai-agentforce-testing Phase F: trace-enriched preview testing. If you use --authoring-bundle instead of --api-name, remember that preview start now requires an explicit mode flag: --simulate-actions or --use-live-actions.

Option 2: Manual HAR Export (Fallback)

1. Open DevTools (Cmd+Opt+I) → Network tab 2. Enable: ✅ Preserve log, ✅ Disable cache, Filter: Fetch/XHR 3. Send test messages in Builder 4. Right-click → "Save all as HAR with content" 5. Analyze with jq or the trace analyzer in sf-ai-agentforce-testing

Key Findings

• The trace endpoint (getSimulationPlanTraces) returns the full PlanSuccessResponse with 13 step types
• CLI approach (sf agent preview) saves traces as JSON files automatically — no network interception needed
• Trace files contain complete LLM prompts, safety scores, grounding assessments, and action I/O

---

Capture Log

Use this section to track your own capture sessions.

Org:              [your org alias]
Agent:            [agent API name]
Session ID:       [from SSE stream URL]
Plan ID:          [from INFORM event planId]
User message:     [test utterance]
Total requests:   [seen], [captured]
SSE streams:      [count]
Step types found: [count]
Steps in trace:   [count]

Key findings from initial investigation:

• Architecture is Aura-based (Hypothesis A confirmed)
• SSE stream at api.salesforce.com delivers messages; trace is fetched separately via Aura
• 13 step types in real-time vs 5 in persisted STDM
• Full LLM prompts, tool lists, and safety scores available only in real-time trace
• planId from INFORM event links messages to trace
• Cookie auth required for trace (no standalone API access)

---

Programmatic Access Comparison

---

Related Documents

• Data Model Reference — Complete 24-DMO STDM schema (persisted layer)
• Query Patterns — Data Cloud SQL examples for persisted trace data
• Debugging Sessions — Session timeline reconstruction from STDM
• Auth Setup — JWT Bearer configuration for Data Cloud access

<!-- Parent: sf-ai-agentforce-observability/SKILL.md -->

CLI Reference

Complete command reference for the stdm-extract CLI tool.

Global Options

---

Commands

extract

Extract session tracing data for a date range.

stdm-extract extract --org <alias> [options]

Options:

Examples:

# Last 7 days for all agents
stdm-extract extract --org prod

# Specific date range
stdm-extract extract --org prod --since 2026-01-01 --until 2026-01-15

# Filter by agent
stdm-extract extract --org prod --agent Customer_Support_Agent

# Multiple agents
stdm-extract extract --org prod --agent Agent1 --agent Agent2

---

extract-tree

Extract complete session tree for specific session IDs.

stdm-extract extract-tree --org <alias> --session-ids <ids> [options]

Options:

Example:

stdm-extract extract-tree --org prod --session-ids "a0x001,a0x002,a0x003"

---

extract-incremental

Incremental extraction based on last run timestamp.

stdm-extract extract-incremental --org <alias> [options]

Options:

Notes:

• Watermark stored at ~/.sf/observability/{org}/watermark.json
• First run extracts last 24 hours
• Subsequent runs extract since last watermark

Example:

# First run: extracts last 24 hours
stdm-extract extract-incremental --org prod

# Subsequent runs: extracts only new data
stdm-extract extract-incremental --org prod

---

analyze

Generate summary statistics from extracted data.

stdm-extract analyze --data-dir <path> [options]

Options:

Example:

stdm-extract analyze --data-dir ./stdm_data --format table

---

debug-session

Debug a specific session with full message timeline.

stdm-extract debug-session --data-dir <path> --session-id <id> [options]

Options: