phoenix-evals

Original：🇺🇸 English

Translated

Build and run evaluators for AI/LLM applications using Phoenix.

2installs

Added on2026-02-08

NPX Install

npx skill4agent add arize-ai/phoenix phoenix-evals

Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.

Task	Files
Setup	`setup-python` , `setup-typescript`
Build code evaluator	`evaluators-code-{python\|typescript}`
Build LLM evaluator	`evaluators-llm-{python\|typescript}` , `evaluators-custom-templates`
Run experiment	`experiments-running-{python\|typescript}`
Create dataset	`experiments-datasets-{python\|typescript}`
Validate evaluator	`validation` , `validation-calibration-{python\|typescript}`
Analyze errors	`error-analysis` , `axial-coding`
RAG evals	`evaluators-rag`
Production	`production-overview` , `production-guardrails`

Starting Fresh:

observe-tracing-setup

→

error-analysis

→

axial-coding

→

evaluators-overview

Building Evaluator:

fundamentals

→

evaluators-{code\|llm}-{python\|typescript}

→

validation-calibration-{python\|typescript}

RAG Systems:

evaluators-rag

→

evaluators-code-*

(retrieval) →

evaluators-llm-*

(faithfulness)

Production:

production-overview

→

production-guardrails

→

production-continuous

Prefix	Description
`fundamentals-*`	Types, scores, anti-patterns
`observe-*`	Tracing, sampling
`error-analysis-*`	Finding failures
`axial-coding-*`	Categorizing failures
`evaluators-*`	Code, LLM, RAG evaluators
`experiments-*`	Datasets, running experiments
`validation-*`	Calibrating judges
`production-*`	CI/CD, monitoring

Principle	Action
Error analysis first	Can't automate what you haven't observed
Custom > generic	Build from your failures
Code first	Deterministic before LLM
Validate judges	>80% TPR/TNR
Binary > Likert	Pass/fail, not 1-5