Terminal-Bench

Terminal-Bench is an evaluation benchmark for AI agents that measures their ability to complete long-horizon, multi-step shell tasks — git operations, build and test loops, file manipulation, system configuration, and recovery from intermediate errors. It targets the kinds of jobs that terminal-native coding agents are asked to do in practice, where success depends on tool reliability and multi-step state rather than single-turn reasoning. Terminal-Bench complements code-patch benchmarks like SWE-Bench by exercising the surrounding environment, not just the diff. Tasks are typically run in disposable containers and scored on whether the agent produces the specified end state, not on how many turns or tokens it took to get there.