Loading...
Loading...
Found 3 Skills
ABSOLUTE MUST to debug and inspect LLM/AI agent traces using PostHog's MCP tools. Use when the user pastes a trace URL (e.g. /llm-observability/traces/<id>), asks to debug a trace, figure out what went wrong, check if an agent used a tool correctly, verify context/files were surfaced, inspect subagent behavior, investigate LLM decisions, or analyze token usage and costs.
A/B test agent variants measuring quality and total session token cost across simple and complex benchmarks. Use when creating compact agent versions, validating agent changes, comparing internal vs external agents, or deciding between variants for production. Use for "compare agents", "A/B test", "benchmark agents", or "test agent efficiency". Do NOT use for evaluating single agents, testing skills, or optimizing prompts without variant comparison.
Analyzes observability signals from customer GenAI applications with DQL. Reads OpenTelemetry GenAI spans and LLM evaluation bizevents. Use for: golden signals (traffic, errors, latency, saturation); LLM signals (model, provider, tokens); cost/token analytics, usage attribution, and prompt caching; agent signals (tool calls, steps, failures, loop detection, Smartscape topology); conversation/session analytics; guardrails (blocked/truncated responses); and evaluation signals (quality, pass/fail). Trigger: "LLM latency", "token usage by model", "cost by model and provider", "cost per conversation", "who is driving token spend", "do I have prompt caching", "failing agent tool calls", "find runaway agents", "responses truncated or blocked", "failed evaluations", "am I hitting rate limits", "token throughput / TPM", "provider throttling or 429s". Do NOT use for: Davis CoPilot/MCP telemetry (dt-platform), generic service metrics (dt-obs-services), logs (dt-obs-logs), or non-GenAI tracing (dt-obs-tracing).