Glossary
Eval
What is an eval?
An eval (evaluation) is a repeatable test that checks an AI system’s output against what “correct” means for a task, using real cases and answers or grading rules the team agrees on.
Also called: Evaluation, AI eval
Why it matters
Evals are how you know an agent works and keeps working. A company’s own evals capture its policies, exceptions and hard-won judgment, which makes them one of its most valuable AI assets.
Example
A refund eval holds 200 past refund requests with the decision an experienced person made, and checks whether a new model reaches the same decision and cites the right policy.
Learn more
Guide · DecideHow to choose an AI model by cost per taskMeasure cost per task, not price per token, and find where a cheaper model does the same job.Guide · DecideWhen to use an AI agent: sort every step into four bucketsDelete, plain code, agent or human: a decision tree for every step, with real data and a worked example.Guide · ProveWhat is shadow mode? Testing AI agents on real work before they go liveRun an agent on real work without sending anything, score it against your people, and promote it one step at a time.Guide · ProveHow to benchmark AI on your own work: people, people with AI, and agentsCompare people, people with AI and agents on your own cases, and pick each workflow’s setup by cost per correct result.Guide · EquipHow to write a PRD for an AI agent, starting from real workA 13-section agent PRD template, built from real work: evals that define “correct,” limits on what the agent may do, and a staged rollout.