sf-ai-agentforce-observability

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

sf-ai-agentforce-observability: Agentforce Session Tracing Extraction & Analysis

sf-ai-agentforce-observability：Agentforce会话追踪数据提取与分析

Expert in extracting and analyzing Agentforce session tracing data from Salesforce Data 360. Supports high-volume data extraction (1-10M records/day), Parquet storage, and Polars-based analysis for debugging agent behavior.

专业处理从Salesforce Data 360中提取并分析Agentforce会话追踪数据的工作。支持高容量数据提取（每日100万至1000万条记录）、Parquet格式存储，以及基于Polars的分析来调试Agent行为。

Core Responsibilities

核心职责

Session Extraction: Extract STDM (Session Tracing Data Model) data via Data 360 Query API
Data Storage: Write to Parquet format with PyArrow for efficient storage
Analysis: Polars-based lazy evaluation for memory-efficient analysis
Debugging: Session timeline reconstruction for troubleshooting agent issues
Cross-Skill Integration: Works with sf-connected-apps for auth, sf-ai-agentscript for fixes

会话提取：通过Data 360 Query API提取STDM（Session Tracing Data Model，会话追踪数据模型）数据
数据存储：使用PyArrow将数据写入Parquet格式，实现高效存储
数据分析：基于Polars的延迟评估，实现内存高效的数据分析
调试功能：重建会话时间线，排查Agent相关问题
跨技能集成：与sf-connected-apps配合完成认证，与sf-ai-agentscript配合完成问题修复

Document Map

文档地图

Need	Document	Description
Quick start	README.md	Installation & basic usage
Data model	references/data-model-reference.md	Full STDM schema documentation
Query patterns	references/query-patterns.md	Data Cloud SQL examples
Analysis recipes	references/analysis-cookbook.md	Common Polars patterns
CLI reference	references/cli-reference.md	Complete command documentation
Auth setup	references/auth-setup.md	JWT Bearer configuration
Troubleshooting	references/troubleshooting.md	Common issues & fixes
Analysis examples	references/analysis-examples.md	Session summary & debug timeline output
Billing & issues	references/billing-and-troubleshooting.md	Credit consumption & common errors

需求	文档	描述
快速入门	README.md	安装及基础使用说明
数据模型	references/data-model-reference.md	完整的STDM架构文档
查询模式	references/query-patterns.md	Data Cloud SQL示例
分析指南	references/analysis-cookbook.md	常用Polars操作模式
CLI参考	references/cli-reference.md	完整的命令文档
认证设置	references/auth-setup.md	JWT Bearer配置说明
故障排查	references/troubleshooting.md	常见问题及解决方法
分析示例	references/analysis-examples.md	会话摘要与调试时间线输出示例
计费与问题	references/billing-and-troubleshooting.md	额度消耗说明与常见错误解决

CRITICAL: Prerequisites Checklist

重要：前置条件检查清单

Before extracting session data, verify:

Check	How to Verify	Why
Data 360 enabled	Setup → Data 360	Required for Query API
Salesforce Standard Data Model v1.124+	Setup → Apps → Packaging → Installed Packages	Required for session tracing DMOs
Einstein Generative AI enabled	Setup → Einstein Generative AI	Enables agent capabilities
Session Tracing enabled	Setup → Einstein Audit, Analytics, and Monitoring	Must toggle ON to collect data
JWT Auth configured	Use `sf-connected-apps`	Required for Data 360 API

Official Setup Guide: Set Up Agentforce Session Tracing

在提取会话数据前，请验证以下内容：

检查项	验证方式	原因
Data 360已启用	进入Setup → Data 360	是使用Query API的必要条件
Salesforce标准数据模型v1.124+	进入Setup → Apps → Packaging → Installed Packages	是使用会话追踪DMO的必要条件
Einstein生成式AI已启用	进入Setup → Einstein Generative AI	启用Agent功能的必要条件
会话追踪已启用	进入Setup → Einstein Audit, Analytics, and Monitoring	必须开启才能收集数据
JWT认证已配置	使用 `sf-connected-apps`	是访问Data 360 API的必要条件

官方设置指南：Set Up Agentforce Session Tracing

Auth Setup (via sf-connected-apps)

认证设置（通过sf-connected-apps）

bash

undefined

bash

undefined

1. Create key directory

1. 创建密钥目录

mkdir -p ~/.sf/jwt

2. Generate certificate (naming convention: {org}-agentforce-observability)

2. 生成证书（命名规范：{org}-agentforce-observability）

openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048
-keyout ~/.sf/jwt/myorg-agentforce-observability.key
-out ~/.sf/jwt/myorg-agentforce-observability.crt
-subj "/CN=AgentforceObservability/O=MyOrg"

3. Secure the private key

3. 保护私钥

chmod 600 ~/.sf/jwt/myorg-agentforce-observability.key

4. Create External Client App in Salesforce (see references/auth-setup.md)

4. 在Salesforce中创建外部客户端应用（详见references/auth-setup.md）

Required scopes: cdp_query_api, refresh_token/offline_access

所需权限范围：cdp_query_api, refresh_token/offline_access


**Key Path Resolution Order:**
1. Explicit `--key-path` argument
2. App-specific: `~/.sf/jwt/{org}-agentforce-observability.key`
3. Generic fallback: `~/.sf/jwt/{org}.key`

See [references/auth-setup.md](references/auth-setup.md) for detailed instructions.

---


**密钥路径解析顺序：**
1. 显式指定的`--key-path`参数
2. 应用特定路径：`~/.sf/jwt/{org}-agentforce-observability.key`
3. 通用回退路径：`~/.sf/jwt/{org}.key`

详细说明请查看[references/auth-setup.md](references/auth-setup.md)。

---

T6 Live API Discovery Summary

T6实时API发现总结

Validated: January 30, 2026 | 24 DMOs Found | 260+ Test Points

Category	DMOs	Status
Session Tracing	5	All Found (Session, Interaction, Step, Message, Participant)
Agent Optimizer	6	All Found (Moment, Tag system)
GenAI Audit	13	All Found (Generation, Quality, Feedback, Gateway)
RAG Quality	3	Not Found (GenAIRetriever* DMOs don't exist)

Key Discoveries:

Field naming: API uses
```
AiAgent
```
(lowercase 'i'), not
```
AIAgent
```
Agent name location: Stored on
```
Moment
```
, not
```
Session
```

Channel types:

E & O

Builder

SCRT2 - EmbeddedMessaging

Voice

NGC

Participant roles:
```
USER
```
,
```
AGENT
```
(not Owner/Observer)

验证日期：2026年1月30日 | 发现24个DMO | 260+测试点

分类	DMO数量	状态
会话追踪	5	全部发现（Session、Interaction、Step、Message、Participant）
Agent优化器	6	全部发现（Moment、标签系统）
生成式AI审计	13	全部发现（Generation、Quality、Feedback、Gateway）
RAG质量	3	未发现（GenAIRetriever* DMO不存在）

关键发现：

字段命名：API使用
```
AiAgent
```
（小写'i'），而非
```
AIAgent
```
Agent名称位置：存储在
```
Moment
```
中，而非
```
Session
```

渠道类型：

E & O

、

Builder

、

SCRT2 - EmbeddedMessaging

、

Voice

、

NGC

参与者角色：
```
USER
```
、
```
AGENT
```
（而非Owner/Observer）

Session Tracing Data Model (STDM)

会话追踪数据模型（STDM）

See references/data-model-reference.md for the complete field-level schema of all 5 core DMOs + 13 GenAI Audit DMOs.

5 Core DMOs — all field names use

AiAgent

prefix (lowercase 'i'):

DMO	Key	Relationship	Primary Fields
`AIAgentSession__dlm`	`Id__c`	Root	StartTimestamp, EndTimestamp, ChannelType, EndType
`AIAgentInteraction__dlm`	`Id__c`	Session → N Turns	TopicApiName, InteractionType, TraceId
`AIAgentInteractionStep__dlm`	`Id__c`	Turn → N Steps	StepType (LLM/ACTION), InputValue, OutputValue, Error
`AIAgentMoment__dlm`	`Id__c`	Session (NOT Turn)	AgentApiName lives here, RequestSummary, ResponseSummary
`AIAgentMessage__dlm`	`Id__c`	Turn → Messages	Content, Role, Timestamp

13 GenAI Trust Layer DMOs — detectors for toxicity, PII, prompt defense, instruction adherence:

DMO	Purpose	Key Fields
`GenAIGatewayRequest__dlm`	LLM request details	model, provider, tokens, safety flags
`GenAIGeneration__dlm`	LLM output	responseText, links to Steps via GenerationId
`GenAIContentQuality__dlm`	Trust Layer assessment	isToxicityDetected
`GenAIContentCategory__dlm`	Detector results	detectorType, category, confidence value
`GenAIFeedback__dlm`	User feedback	GOOD/BAD + detail comments

所有5个核心DMO + 13个生成式AI审计DMO的完整字段级架构，请查看references/data-model-reference.md。

5个核心DMO —— 所有字段名均使用

AiAgent

前缀（小写'i'）：

DMO	主键	关联关系	核心字段
`AIAgentSession__dlm`	`Id__c`	根节点	StartTimestamp, EndTimestamp, ChannelType, EndType
`AIAgentInteraction__dlm`	`Id__c`	Session → N轮对话	TopicApiName, InteractionType, TraceId
`AIAgentInteractionStep__dlm`	`Id__c`	对话轮次 → N步骤	StepType (LLM/ACTION), InputValue, OutputValue, Error
`AIAgentMoment__dlm`	`Id__c`	关联Session（非对话轮次）	AgentApiName存储于此, RequestSummary, ResponseSummary
`AIAgentMessage__dlm`	`Id__c`	对话轮次 → 消息	Content, Role, Timestamp

13个生成式AI信任层DMO —— 用于检测毒性、PII、提示词防护、指令遵循情况：

DMO	用途	核心字段
`GenAIGatewayRequest__dlm`	LLM请求详情	model, provider, tokens, safety flags
`GenAIGeneration__dlm`	LLM输出	responseText, 通过GenerationId关联Steps
`GenAIContentQuality__dlm`	信任层评估结果	isToxicityDetected
`GenAIContentCategory__dlm`	检测器结果	detectorType, category, confidence value
`GenAIFeedback__dlm`	用户反馈	GOOD/BAD + 详细评论

Workflow (5-Phase Pattern)

工作流（5阶段模式）

Phase 1: Requirements Gathering

阶段1：需求收集

Ask the user to gather:

#	Question	Options
1	Target org	Org alias from `sf org list`
2	Time range	Last N days / Date range
3	Agent filter	All agents / Specific API names
4	Output format	Parquet (default) / CSV
5	Analysis type	Summary / Debug session / Full extraction

询问用户收集以下信息：

序号	问题	选项
1	目标组织	`sf org list` 中的组织别名
2	时间范围	最近N天 / 日期范围
3	Agent筛选	所有Agent / 特定API名称
4	输出格式	Parquet（默认）/ CSV
5	分析类型	摘要 / 会话调试 / 完整提取

Phase 2: Auth Configuration

阶段2：认证配置

Verify JWT auth is configured:

python

from scripts.auth import Data360Auth

auth = Data360Auth(
    org_alias="myorg",
    consumer_key="YOUR_CONSUMER_KEY"
)

验证JWT认证是否已配置：

python

from scripts.auth import Data360Auth

auth = Data360Auth(
    org_alias="myorg",
    consumer_key="YOUR_CONSUMER_KEY"
)

Test authentication

测试认证

token = auth.get_token() print(f"Auth successful: {token[:20]}...")


If auth fails, use the **sf-connected-apps** skill: "Setup JWT Bearer for Data 360"

token = auth.get_token() print(f"Auth successful: {token[:20]}...")


如果认证失败，请使用**sf-connected-apps**技能：“为Data 360配置JWT Bearer认证”

Phase 3: Extraction

阶段3：数据提取

Basic Extraction (last 7 days):

bash

python3 scripts/cli.py extract \
  --org prod \
  --days 7 \
  --output ./stdm_data

Filtered Extraction:

bash

python3 scripts/cli.py extract \
  --org prod \
  --since 2026-01-01 \
  --until 2026-01-28 \
  --agent Customer_Support_Agent \
  --output ./stdm_data

Session Tree (specific session):

bash

python3 scripts/cli.py extract-tree \
  --org prod \
  --session-id "a0x..." \
  --output ./debug_session

基础提取（最近7天）：

bash

python3 scripts/cli.py extract \
  --org prod \
  --days 7 \
  --output ./stdm_data

筛选提取：

bash

python3 scripts/cli.py extract \
  --org prod \
  --since 2026-01-01 \
  --until 2026-01-28 \
  --agent Customer_Support_Agent \
  --output ./stdm_data

会话树提取（特定会话）：

bash

python3 scripts/cli.py extract-tree \
  --org prod \
  --session-id "a0x..." \
  --output ./debug_session

Phase 4: Analysis

阶段4：数据分析

Session Summary:

python

from scripts.analyzer import STDMAnalyzer
from pathlib import Path

analyzer = STDMAnalyzer(Path("./stdm_data"))

会话摘要：

python

from scripts.analyzer import STDMAnalyzer
from pathlib import Path

analyzer = STDMAnalyzer(Path("./stdm_data"))

High-level summary

高级别摘要

summary = analyzer.session_summary() print(summary)

Step distribution by agent

按Agent统计步骤分布

steps = analyzer.step_distribution(agent_name="Customer_Support_Agent") print(steps)

Topic routing analysis

主题路由分析

topics = analyzer.topic_analysis() print(topics)


**Debug Specific Session:**
```bash
python3 scripts/cli.py debug-session \
  --data-dir ./stdm_data \
  --session-id "a0x..."

topics = analyzer.topic_analysis() print(topics)


**特定会话调试：**
```bash
python3 scripts/cli.py debug-session \
  --data-dir ./stdm_data \
  --session-id "a0x..."

Phase 5: Integration & Next Steps

阶段5：集成与后续步骤

Based on analysis findings:

Finding	Next Step	Skill
Topic mismatch	Improve topic descriptions	`sf-ai-agentscript`
Action failures	Debug Flow/Apex	`sf-flow` , `sf-debug`
Slow responses	Optimize actions	`sf-apex`
Missing coverage	Add test cases	`sf-ai-agentforce-testing`

根据分析结果采取对应行动：

发现问题	后续步骤	技能
主题不匹配	优化主题描述	`sf-ai-agentscript`
动作执行失败	调试Flow/Apex	`sf-flow` , `sf-debug`
响应缓慢	优化动作逻辑	`sf-apex`
覆盖范围不足	添加测试用例	`sf-ai-agentforce-testing`

CLI Quick Reference

CLI快速参考

Extraction Commands

提取命令

Command	Purpose	Example
`extract`	Extract session data	`extract --org prod --days 7`
`extract-tree`	Extract full session tree	`extract-tree --org prod --session-id "a0x..."`
`extract-incremental`	Resume from last run	`extract-incremental --org prod`

命令	用途	示例
`extract`	提取会话数据	`extract --org prod --days 7`
`extract-tree`	提取完整会话树	`extract-tree --org prod --session-id "a0x..."`
`extract-incremental`	从上一次运行处恢复提取	`extract-incremental --org prod`

Analysis Commands

分析命令

Command	Purpose	Example
`analyze`	Generate summary stats	`analyze --data-dir ./stdm_data`
`debug-session`	Timeline view	`debug-session --session-id "a0x..."`
`topics`	Topic analysis	`topics --data-dir ./stdm_data`

命令	用途	示例
`analyze`	生成摘要统计数据	`analyze --data-dir ./stdm_data`
`debug-session`	时间线视图	`debug-session --session-id "a0x..."`
`topics`	主题分析	`topics --data-dir ./stdm_data`

Common Flags

通用参数

Flag	Description	Default
`--org`	Target org alias	Required
`--consumer-key`	ECA consumer key	`$SF_CONSUMER_KEY` env var
`--key-path`	JWT private key path	`~/.sf/jwt/{org}-agentforce-observability.key`
`--days`	Last N days	7
`--since` / `--until`	Date range (YYYY-MM-DD)	- / Today
`--agent`	Filter by agent API name	All
`--output`	Output directory	`./stdm_data`
`--verbose`	Detailed logging	False

See references/cli-reference.md for complete documentation.

参数	描述	默认值
`--org`	目标组织别名	必填
`--consumer-key`	ECA消费者密钥	`$SF_CONSUMER_KEY` 环境变量
`--key-path`	JWT私钥路径	`~/.sf/jwt/{org}-agentforce-observability.key`
`--days`	最近N天	7
`--since` / `--until`	日期范围（YYYY-MM-DD）	- / 今日
`--agent`	按Agent API名称筛选	所有Agent
`--output`	输出目录	`./stdm_data`
`--verbose`	详细日志	False

完整文档请查看references/cli-reference.md。

Analysis Examples

分析示例

See references/analysis-examples.md for full session summary and debug timeline output examples.

Session Summary: Shows sessions by agent (count, avg turns, avg duration) and end type distribution (completed/escalated/abandoned).

Debug Timeline: Reconstructs a session step-by-step — input → topic routing → LLM steps → action steps → output — with timestamps and I/O payloads.

完整的会话摘要和调试时间线输出示例，请查看references/analysis-examples.md。

会话摘要：展示各Agent的会话统计（数量、平均对话轮次、平均时长）以及会话结束类型分布（完成/升级/放弃）。

调试时间线：逐步重建会话流程 —— 输入 → 主题路由 → LLM步骤 → 动作步骤 → 输出 —— 包含时间戳和I/O负载。

Cross-Skill Integration

跨技能集成

Skill	When	How to Invoke
`sf-connected-apps`	Auth setup	Use the sf-connected-apps skill: "JWT Bearer for Data Cloud"
`sf-ai-agentscript`	Fix topic routing issues	Use the sf-ai-agentscript skill: "Fix topic: [issue]"
`sf-flow` / `sf-debug`	Debug action failures	Use the sf-debug skill: "Analyze agent action failure"
`sf-ai-agentforce-testing`	Create test cases from patterns	Use the sf-ai-agentforce-testing skill: "Add test cases"

技能	使用场景	调用方式
`sf-connected-apps`	认证设置	使用sf-connected-apps技能：“为Data Cloud配置JWT Bearer认证”
`sf-ai-agentscript`	修复主题路由问题	使用sf-ai-agentscript技能：“修复主题：[问题描述]”
`sf-flow` / `sf-debug`	调试动作执行失败	使用sf-debug技能：“分析Agent动作执行失败问题”
`sf-ai-agentforce-testing`	根据模式创建测试用例	使用sf-ai-agentforce-testing技能：“添加测试用例”

Key Insights

关键见解

Insight	Description	Action
STDM is read-only	Data 360 stores traces; cannot modify	Use for analysis only
Session lag	Data may lag 5-15 minutes	Don't expect real-time
Volume limits	Query API: 10M records/day	Use incremental extraction
Parquet efficiency	10x smaller than JSON	Always use Parquet for storage
Lazy evaluation	Polars scans without loading	Handles 100M+ rows
~24 records per LLM call	Each round-trip generates ~24 records	Factor into volume estimates

见解	描述	行动建议
STDM为只读	Data 360仅存储追踪数据，无法修改	仅用于分析
会话数据延迟	数据可能存在5-15分钟延迟	不要期望实时数据
容量限制	Query API每日最多处理1000万条记录	使用增量提取
Parquet格式高效性	体积比JSON小10倍	始终使用Parquet格式存储
延迟评估	Polars无需加载全部数据即可扫描	可处理1亿+行数据
每次LLM调用生成约24条记录	每轮LLM调用会生成约24条记录	估算数据量时需考虑此因素

Billing & Common Issues

计费与常见问题

See references/billing-and-troubleshooting.md for credit consumption details and error resolution.

Quick reference: Session Tracing consumes Data 360 credits. ~24 records per LLM round-trip. 1,000 sessions/day × 4 turns × 24 = ~96K records/day. Use Digital Wallet for consumption tracking.

Error	Quick Fix
`401 Unauthorized`	Refresh token or reconfigure ECA
`No session data`	Enable Session Tracing in Agent Settings
`Query timeout`	Add date filters, use incremental
`Memory error`	Use Polars lazy frames

额度消耗详情和错误解决方法，请查看references/billing-and-troubleshooting.md。

快速参考：会话追踪会消耗Data 360额度。每次LLM轮次调用约生成24条记录。每日1000个会话 × 4轮对话 ×24 = 约9.6万条记录/天。使用Digital Wallet追踪额度消耗。

错误	快速修复方案
`401 Unauthorized`	刷新令牌或重新配置ECA
`No session data`	在Agent设置中开启会话追踪
`Query timeout`	添加日期筛选，使用增量提取
`Memory error`	使用Polars延迟帧

Output Directory Structure

输出目录结构

stdm_data/
├── sessions/          # date=YYYY-MM-DD/part-0000.parquet
├── interactions/      # date=YYYY-MM-DD/part-0000.parquet
├── steps/             # date=YYYY-MM-DD/part-0000.parquet
├── messages/          # date=YYYY-MM-DD/part-0000.parquet
└── metadata/          # extraction.json + watermark.json (incremental)

stdm_data/
├── sessions/          # date=YYYY-MM-DD/part-0000.parquet
├── interactions/      # date=YYYY-MM-DD/part-0000.parquet
├── steps/             # date=YYYY-MM-DD/part-0000.parquet
├── messages/          # date=YYYY-MM-DD/part-0000.parquet
└── metadata/          # extraction.json + watermark.json（增量提取用）

Dependencies

依赖项

Python 3.10+:

polars>=1.0

pyarrow>=15.0

pyjwt>=2.8

cryptography>=42.0

httpx>=0.27

rich>=13.0

click>=8.1

pydantic>=2.6

Install:

pip install -r requirements.txt

Python 3.10+：

polars>=1.0

pyarrow>=15.0

pyjwt>=2.8

cryptography>=42.0

httpx>=0.27

rich>=13.0

click>=8.1

pydantic>=2.6

安装命令：

pip install -r requirements.txt