Eval Harness — AI Glossary

An eval harness is infrastructure that runs a prompt or model against a fixed test set and computes aggregate scores per metric. It decouples what you test (the eval set) from how you run it (the harness), so the same tests can run against different models, prompt variants, or decoding settings with no code changes.