sf-ai-agentforce-observability: Agentforce Session Tracing Extraction & Analysis

Expert in extracting and analyzing Agentforce session tracing data from Salesforce Data 360. Supports high-volume data extraction (1-10M records/day), Parquet storage, and Polars-based analysis for debugging agent behavior.

Core Responsibilities

Session Extraction: Extract STDM (Session Tracing Data Model) data via Data 360 Query API
Data Storage: Write to Parquet format with PyArrow for efficient storage
Analysis: Polars-based lazy evaluation for memory-efficient analysis
Debugging: Session timeline reconstruction for troubleshooting agent issues
Cross-Skill Integration: Works with sf-connected-apps for auth, sf-ai-agentscript for fixes

Document Map

Need	Document	Description
Quick start	README.md	Installation & basic usage
Data model	resources/data-model-reference.md	Full STDM schema documentation
Query patterns	resources/query-patterns.md	Data Cloud SQL examples
Analysis recipes	resources/analysis-cookbook.md	Common Polars patterns
CLI reference	docs/cli-reference.md	Complete command documentation
Auth setup	docs/auth-setup.md	JWT Bearer configuration
Troubleshooting	resources/troubleshooting.md	Common issues & fixes

Quick Links:

Data Model Overview
CLI Quick Reference
Analysis Examples
Cross-Skill Integration

CRITICAL: Prerequisites Checklist

Before extracting session data, verify:

Check	How to Verify	Why
Data 360 enabled	Setup → Data 360	Required for Query API
Salesforce Standard Data Model v1.124+	Setup → Apps → Packaging → Installed Packages	Required for session tracing DMOs
Einstein Generative AI enabled	Setup → Einstein Generative AI	Enables agent capabilities
Session Tracing enabled	Setup → Einstein Audit, Analytics, and Monitoring	Must toggle ON to collect data
JWT Auth configured	Use `sf-connected-apps`	Required for Data 360 API

Official Setup Guide: Set Up Agentforce Session Tracing

Auth Setup (via sf-connected-apps)

bash

# 1. Create key directory
mkdir -p ~/.sf/jwt

# 2. Generate certificate (naming convention: {org}-agentforce-observability)
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
  -keyout ~/.sf/jwt/myorg-agentforce-observability.key \
  -out ~/.sf/jwt/myorg-agentforce-observability.crt \
  -subj "/CN=AgentforceObservability/O=MyOrg"

# 3. Secure the private key
chmod 600 ~/.sf/jwt/myorg-agentforce-observability.key

# 4. Create External Client App in Salesforce (see docs/auth-setup.md)
# Required scopes: cdp_query_api, refresh_token/offline_access

Key Path Resolution Order:

Explicit
```
--key-path
```
argument

App-specific:

~/.sf/jwt/{org}-agentforce-observability.key

Generic fallback:
```
~/.sf/jwt/{org}.key
```

See docs/auth-setup.md for detailed instructions.

T6 Live API Discovery Summary ✅

Validated: January 30, 2026 | 24 DMOs Found | 260+ Test Points

Category	DMOs	Status
Session Tracing	5	✅ All Found (Session, Interaction, Step, Message, Participant)
Agent Optimizer	6	✅ All Found (Moment, Tag system)
GenAI Audit	13	✅ All Found (Generation, Quality, Feedback, Gateway)
RAG Quality	3	❌ Not Found (GenAIRetriever* DMOs don't exist)

Key Discoveries:

Field naming: API uses
```
AiAgent
```
(lowercase 'i'), not
```
AIAgent
```
Agent name location: Stored on
```
Moment
```
, not
```
Session
```

Channel types:

E & O

Builder

SCRT2 - EmbeddedMessaging

Voice

NGC

Builder: Voice Preview

Agent types:

EinsteinServiceAgent

AgentforceEmployeeAgent

AgentforceServiceAgent

Employee

Participant roles:
```
USER
```
,
```
AGENT
```
(not Owner/Observer)

GenAI detectors:

TOXICITY

(9 categories),

PII

(4 types),

PROMPT_DEFENSE

InstructionAdherence

Session Tracing Data Model (STDM)

The STDM consists of 5 core DMOs plus 13 GenAI Audit DMOs. Important: Field names use

AiAgent

(lowercase 'i'), not

AIAgent

ssot__AIAgentSession__dlm (SESSION)
├── ssot__Id__c                          # Session ID (UUID)
├── ssot__StartTimestamp__c              # Session start (TimestampTZ)
├── ssot__EndTimestamp__c                # Session end (TimestampTZ)
├── ssot__AiAgentSessionEndType__c       # End type (Completed, Abandoned, etc.)
├── ssot__AiAgentChannelType__c          # Channel (PSTN, Messaging, etc.)
├── ssot__RelatedMessagingSessionId__c   # Linked messaging session
├── ssot__RelatedVoiceCallId__c          # Linked voice call
├── ssot__SessionOwnerId__c              # Owner ID
├── ssot__IndividualId__c                # Data Cloud individual
└── ssot__InternalOrganizationId__c      # Org ID

    └── ssot__AIAgentInteraction__dlm (TURN)  [1:N]
        ├── ssot__Id__c                          # Interaction ID
        ├── ssot__AiAgentSessionId__c            # FK to Session
        ├── ssot__AiAgentInteractionType__c      # TURN or SESSION_END
        ├── ssot__TopicApiName__c                # Topic that handled this turn
        ├── ssot__StartTimestamp__c              # Turn start
        ├── ssot__EndTimestamp__c                # Turn end
        ├── ssot__TelemetryTraceId__c            # Trace ID for debugging
        └── ssot__TelemetryTraceSpanId__c        # Span ID for debugging

            └── ssot__AIAgentInteractionStep__dlm (STEP)  [1:N]
                ├── ssot__Id__c                          # Step ID
                ├── ssot__AiAgentInteractionId__c        # FK to Interaction
                ├── ssot__AiAgentInteractionStepType__c  # LLM_STEP or ACTION_STEP
                ├── ssot__Name__c                        # Action/step name
                ├── ssot__InputValueText__c              # Input to step (JSON)
                ├── ssot__OutputValueText__c             # Output from step (JSON)
                ├── ssot__ErrorMessageText__c            # Error if step failed
                ├── ssot__PreStepVariableText__c         # Variables before
                ├── ssot__PostStepVariableText__c        # Variables after
                ├── ssot__GenerationId__c                # LLM generation ID
                └── ssot__GenAiGatewayRequestId__c       # GenAI Gateway request

ssot__AIAgentMoment__dlm (MOMENT - links to Session, not Interaction)
├── ssot__Id__c                          # Moment ID
├── ssot__AiAgentSessionId__c            # FK to Session (NOT interaction!)
├── ssot__AiAgentApiName__c              # Agent API name (lives here!)
├── ssot__AiAgentVersionApiName__c       # Agent version
├── ssot__RequestSummaryText__c          # User request summary
├── ssot__ResponseSummaryText__c         # Agent response summary
├── ssot__StartTimestamp__c              # Moment start
└── ssot__EndTimestamp__c                # Moment end

Key Schema Notes:

Agent API name is in
```
AIAgentMoment
```
, not
```
AIAgentSession
```
Moments link to sessions via
```
AiAgentSessionId
```
, not interactions
All field names use
```
AiAgent
```
prefix (lowercase 'i')

GenAI Trust Layer DMOs (13) ✅ T6 Verified

GenAIGatewayRequest__dlm (30 fields) - LLM request details
├── gatewayRequestId__c, prompt__c, maskedPrompt__c
├── model__c, provider__c, temperature__c
├── promptTokens__c, completionTokens__c, totalTokens__c
├── enableInputSafetyScoring__c, enableOutputSafetyScoring__c, enablePiiMasking__c
└── sessionId__c, userId__c, appType__c, feature__c

GenAIGeneration__dlm (11 fields) - LLM output
├── generationId__c (FK for Steps)
├── responseText__c, maskedResponseText__c
└── Links to: AIAgentInteractionStep.ssot__GenerationId__c

GenAIContentQuality__dlm (10 fields) - Trust Layer assessment
└── isToxicityDetected__c, parent__c (FK to Generation)

GenAIContentCategory__dlm (10 fields) - Detector results
├── detectorType__c: TOXICITY | PII | PROMPT_DEFENSE | InstructionAdherence
├── category__c: hate, identity, CREDIT_CARD, EMAIL_ADDRESS, High, Low, etc.
└── value__c: Confidence score (0.0-1.0)

GenAIFeedback__dlm (16 fields) - User feedback
├── feedback__c: GOOD | BAD
└── GenAIFeedbackDetail__dlm (10 fields) - Free-text comments

Detector Categories (Live API Verified):

Detector	Categories
`TOXICITY`	`hate` , `identity` , `physical` , `profanity` , `safety_score` , `sexual` , `toxicity` , `violence`
`PII`	`CREDIT_CARD` , `EMAIL_ADDRESS` , `PERSON` , `US_PHONE_NUMBER`
`PROMPT_DEFENSE`	`aggregatePromptAttackScore` , `isPromptAttackDetected`
`InstructionAdherence`	`High` , `Low` , `Uncertain`

See resources/data-model-reference.md for full field documentation.

Workflow (5-Phase Pattern)

Phase 1: Requirements Gathering

Use AskUserQuestion to gather:

#	Question	Options
1	Target org	Org alias from `sf org list`
2	Time range	Last N days / Date range
3	Agent filter	All agents / Specific API names
4	Output format	Parquet (default) / CSV
5	Analysis type	Summary / Debug session / Full extraction

Phase 2: Auth Configuration

Verify JWT auth is configured:

python

from scripts.auth import Data360Auth

auth = Data360Auth(
    org_alias="myorg",
    consumer_key="YOUR_CONSUMER_KEY"
)

# Test authentication
token = auth.get_token()
print(f"Auth successful: {token[:20]}...")

If auth fails, invoke:

Skill(skill="sf-connected-apps", args="Setup JWT Bearer for Data 360")

Phase 3: Extraction

Basic Extraction (last 7 days):

bash

python3 scripts/cli.py extract \
  --org prod \
  --days 7 \
  --output ./stdm_data

Filtered Extraction:

bash

python3 scripts/cli.py extract \
  --org prod \
  --since 2026-01-01 \
  --until 2026-01-28 \
  --agent Customer_Support_Agent \
  --output ./stdm_data

Session Tree (specific session):

bash

python3 scripts/cli.py extract-tree \
  --org prod \
  --session-id "a0x..." \
  --output ./debug_session

Phase 4: Analysis

Session Summary:

python

from scripts.analyzer import STDMAnalyzer
from pathlib import Path

analyzer = STDMAnalyzer(Path("./stdm_data"))

# High-level summary
summary = analyzer.session_summary()
print(summary)

# Step distribution by agent
steps = analyzer.step_distribution(agent_name="Customer_Support_Agent")
print(steps)

# Topic routing analysis
topics = analyzer.topic_analysis()
print(topics)

Debug Specific Session:

bash

python3 scripts/cli.py debug-session \
  --data-dir ./stdm_data \
  --session-id "a0x..."

Phase 5: Integration & Next Steps

Based on analysis findings:

Finding	Next Step	Skill
Topic mismatch	Improve topic descriptions	`sf-ai-agentscript`
Action failures	Debug Flow/Apex	`sf-flow` , `sf-debug`
Slow responses	Optimize actions	`sf-apex`
Missing coverage	Add test cases	`sf-ai-agentforce-testing`

CLI Quick Reference

Extraction Commands

Command	Purpose	Example
`extract`	Extract session data	`extract --org prod --days 7`
`extract-tree`	Extract full session tree	`extract-tree --org prod --session-id "a0x..."`
`extract-incremental`	Resume from last run	`extract-incremental --org prod`

Analysis Commands

Command	Purpose	Example
`analyze`	Generate summary stats	`analyze --data-dir ./stdm_data`
`debug-session`	Timeline view	`debug-session --session-id "a0x..."`
`topics`	Topic analysis	`topics --data-dir ./stdm_data`

Common Flags

Flag	Description	Default
`--org`	Target org alias	Required
`--consumer-key`	ECA consumer key	`$SF_CONSUMER_KEY` env var
`--key-path`	JWT private key path	`~/.sf/jwt/{org}-agentforce-observability.key`
`--days`	Last N days	7
`--since`	Start date (YYYY-MM-DD)	-
`--until`	End date (YYYY-MM-DD)	Today
`--agent`	Filter by agent API name	All
`--output`	Output directory	`./stdm_data`
`--verbose`	Detailed logging	False
`--format`	Output format (table/json/csv)	table

See docs/cli-reference.md for complete documentation.

Analysis Examples

Session Summary

📊 SESSION SUMMARY
════════════════════════════════════════════════════════════════

Period: 2026-01-21 to 2026-01-28
Total Sessions: 15,234
Unique Agents: 3

SESSIONS BY AGENT
────────────────────────────────────────────────────────────────
Agent                          │ Sessions │ Avg Turns │ Avg Duration
───────────────────────────────┼──────────┼───────────┼─────────────
Customer_Support_Agent         │   8,502  │    4.2    │     3m 15s
Order_Tracking_Agent           │   4,128  │    2.8    │     1m 45s
Product_FAQ_Agent              │   2,604  │    1.9    │       45s

END TYPE DISTRIBUTION
────────────────────────────────────────────────────────────────
✅ Completed:    12,890 (84.6%)
🔄 Escalated:     1,523 (10.0%)
❌ Abandoned:       821 (5.4%)

Debug Session Timeline

🔍 SESSION DEBUG: a0x1234567890ABC
════════════════════════════════════════════════════════════════

Agent: Customer_Support_Agent
Started: 2026-01-28 10:15:23 UTC
Duration: 4m 32s
End Type: Completed
Turns: 5

TIMELINE
────────────────────────────────────────────────────────────────
10:15:23 │ [INPUT]  "I need help with my order #12345"
10:15:24 │ [TOPIC]  → Order_Tracking (confidence: 0.95)
10:15:24 │ [STEP]   LLM_STEP: Identify intent
10:15:25 │ [STEP]   ACTION_STEP: Get_Order_Status
         │          Input: {"orderId": "12345"}
         │          Output: {"status": "Shipped", "eta": "2026-01-30"}
10:15:26 │ [OUTPUT] "Your order #12345 has shipped and will arrive by Jan 30."

10:16:01 │ [INPUT]  "Can I change the delivery address?"
10:16:02 │ [TOPIC]  → Order_Tracking (same topic)
10:16:02 │ [STEP]   LLM_STEP: Clarify request
10:16:03 │ [STEP]   ACTION_STEP: Check_Modification_Eligibility
         │          Input: {"orderId": "12345", "type": "address_change"}
         │          Output: {"eligible": false, "reason": "Already shipped"}
10:16:04 │ [OUTPUT] "I'm sorry, the order has already shipped..."

Cross-Skill Integration

Prerequisite Skills

Skill	When	How to Invoke
`sf-connected-apps`	Auth setup	`Skill(skill="sf-connected-apps", args="JWT Bearer for Data Cloud")`

Follow-up Skills

Finding Skill How to Invoke

Topic routing issues

Finding	Skill	How to Invoke
Topic routing issues	`sf-ai-agentscript`	`Skill(skill="sf-ai-agentscript", args="Fix topic: [issue]")`
Action failures	`sf-flow` / `sf-debug`	`Skill(skill="sf-debug", args="Analyze agent action failure")`
Test coverage gaps	`sf-ai-agentforce-testing`	`Skill(skill="sf-ai-agentforce-testing", args="Add test cases")`

sf-ai-agentscript

Skill(skill="sf-ai-agentscript", args="Fix topic: [issue]")

Action failures

sf-flow

sf-debug

Skill(skill="sf-debug", args="Analyze agent action failure")

Test coverage gaps

sf-ai-agentforce-testing

Skill(skill="sf-ai-agentforce-testing", args="Add test cases")

Commonly Used With

Skill	Use Case	Confidence
`sf-ai-agentscript`	Fix agent based on trace analysis	⭐⭐⭐ Required
`sf-ai-agentforce-testing`	Create test cases from observed patterns	⭐⭐ Recommended
`sf-debug`	Deep-dive into action failures	⭐⭐ Recommended

Key Insights

Insight	Description	Action
STDM is read-only	Data 360 stores traces; cannot modify	Use for analysis only
Session lag	Data may lag 5-15 minutes	Don't expect real-time
Volume limits	Query API: 10M records/day	Use incremental extraction
Parquet efficiency	10x smaller than JSON	Always use Parquet for storage
Lazy evaluation	Polars scans without loading	Handles 100M+ rows
~24 records per LLM call	Each round-trip generates ~24 records	Factor into volume estimates
5-minute collection interval	Data collection runs every 5 minutes	Account for processing delay

Billing Considerations

Reference: Billing Considerations for Agentforce Session Tracing

Agentforce Session Tracing consumes Data 360 credits for ingestion, storage, and processing.

Credit Consumption

Usage Type	Digital Wallet Card	Description
Batch Data Pipeline	Data Services	Records ingested via data streams. ~24 records per LLM round-trip. Primary cost driver.
Data Queries	Data Services	Records processed when running queries, reports, dashboards
Streaming Calculated Insights	Data Services	Used for Prompt Builder usage and feedback metrics
Storage Beyond Allocation	Data Storage	Storage consumed above allocated amount

Cost Estimation

Records per session ≈ Turns × 24 (avg per LLM call)
Daily records ≈ Sessions/day × Avg turns × 24

Example:
  1,000 sessions/day × 4 turns × 24 = 96,000 records/day ingested

Tip: Use Digital Wallet for near real-time consumption tracking.

Common Issues & Fixes

Error	Cause	Fix
`401 Unauthorized`	JWT auth expired/invalid	Refresh token or reconfigure ECA
`No session data`	Tracing not enabled	Enable Session Tracing in Agent Settings
`Query timeout`	Too much data	Add date filters, use incremental
`Memory error`	Loading all data	Use Polars lazy frames
`Missing DMO`	Wrong API version	Use API v60.0+

See resources/troubleshooting.md for detailed solutions.

Output Directory Structure

After extraction:

stdm_data/
├── sessions/
│   └── date=2026-01-28/
│       └── part-0000.parquet
├── interactions/
│   └── date=2026-01-28/
│       └── part-0000.parquet
├── steps/
│   └── date=2026-01-28/
│       └── part-0000.parquet
├── messages/
│   └── date=2026-01-28/
│       └── part-0000.parquet
└── metadata/
    ├── extraction.json      # Extraction parameters
    └── watermark.json       # For incremental extraction

Dependencies

Python 3.10+ with:

polars>=1.0.0           # DataFrame library (lazy evaluation)
pyarrow>=15.0.0         # Parquet support
pyjwt>=2.8.0            # JWT generation
cryptography>=42.0.0    # Certificate handling
httpx>=0.27.0           # HTTP client
rich>=13.0.0            # CLI progress bars
click>=8.1.0            # CLI framework
pydantic>=2.6.0         # Data validation

Install:

pip install -r requirements.txt

sf-ai-agentforce-observability

NPX Install

SKILL.md Content

sf-ai-agentforce-observability: Agentforce Session Tracing Extraction & Analysis

Core Responsibilities

Document Map

CRITICAL: Prerequisites Checklist

Auth Setup (via sf-connected-apps)

T6 Live API Discovery Summary ✅

Session Tracing Data Model (STDM)

GenAI Trust Layer DMOs (13) ✅ T6 Verified

Workflow (5-Phase Pattern)

Phase 1: Requirements Gathering

Phase 2: Auth Configuration

Phase 3: Extraction

Phase 4: Analysis

Phase 5: Integration & Next Steps

CLI Quick Reference

Extraction Commands

Analysis Commands

Common Flags

Analysis Examples

Session Summary

Debug Session Timeline

Cross-Skill Integration

Prerequisite Skills

Follow-up Skills

Commonly Used With

Key Insights

Billing Considerations

Credit Consumption

Cost Estimation

Common Issues & Fixes

Output Directory Structure

Dependencies

License