AI Agent Failure Analysis and Evals Interview
Use traces to separate tool, turn, and task failures, test root-cause hypotheses, and turn confirmed failures into eval cases and regression gates.
3 docs tagged with "evals"
View all tagsChoosing a Semantic Threshold: Using Jevals and Human Labels for Tool Results
Evaluate whether agents use tool results with jevals and Kev. Add human labels, compare threshold tradeoffs, and apply a chosen threshold to held-out traces.
Coding Agent Evals Guide: Turn Traces into Datasets and Quality Gates
Turn a preserved Coding Agent trace into a local JSONL dataset, deterministic graders, machine-readable reports, and a CI quality gate.