phoenix-evals
Original：🇺🇸 English
Translated
Build and run evaluators for AI/LLM applications using Phoenix.
23installs
Sourcearize-ai/phoenix
Added on
NPX Install
npx skill4agent add arize-ai/phoenix phoenix-evalsTags
Translated version includes tags in frontmatterSKILL.md Content
View Translation Comparison →Phoenix Evals
Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.
Quick Reference
| Task | Files |
|---|---|
| Setup | |
| Build code evaluator | |
| Build LLM evaluator | |
| Run experiment | |
| Create dataset | |
| Validate evaluator | |
| Analyze errors | |
| RAG evals | |
| Production | |
Workflows
Starting Fresh:
→ → →
observe-tracing-setuperror-analysisaxial-codingevaluators-overviewBuilding Evaluator:
→ →
fundamentalsevaluators-{code\|llm}-{python\|typescript}validation-calibration-{python\|typescript}RAG Systems:
→ (retrieval) → (faithfulness)
evaluators-ragevaluators-code-*evaluators-llm-*Production:
→ →
production-overviewproduction-guardrailsproduction-continuousRule Categories
| Prefix | Description |
|---|---|
| Types, scores, anti-patterns |
| Tracing, sampling |
| Finding failures |
| Categorizing failures |
| Code, LLM, RAG evaluators |
| Datasets, running experiments |
| Calibrating judges |
| CI/CD, monitoring |
Key Principles
| Principle | Action |
|---|---|
| Error analysis first | Can't automate what you haven't observed |
| Custom > generic | Build from your failures |
| Code first | Deterministic before LLM |
| Validate judges | >80% TPR/TNR |
| Binary > Likert | Pass/fail, not 1-5 |