Model Cascade

A model cascade is a routing pattern in which each request is first attempted by a cheaper, smaller model and only escalated to a stronger, more expensive model when the small model's confidence is low or its output fails a validation check. The escalation signal can come from the model's own self-assessment, a logprob-based confidence score, a lightweight classifier, or a downstream validator (schema check, tool-call success, judge score). Cascades cut cost on easy requests while preserving quality on hard ones, and they degrade gracefully: a partially correct small-model output is often better than a hard error. The design choice is where to set the escalation threshold — too aggressive and you lose the cost savings, too lax and quality drops.