Definition
In AI, evaluation (often shortened to eval) is the process of measuring model output quality and task performance using test data, metrics, and human or automated judging.
How it works
You build a test set and scoring criteria — objective metrics, rule-based checks or human review — run the model, and compare results against reference answers or a previous version. Stronger models are also commonly used as judges (LLM-as-a-judge). Because many model outputs have no single correct answer, evaluations usually combine several metrics.
Usage
Evaluation underpins model selection, version comparison before a release, regression testing after prompt or model changes, and picking the right model for a business use case.
Related and confusable terms
A benchmark is a public, standardized test set — one tool of evaluation; evaluation is the broader practice covering data, metrics and process.