Semantic Caching — AI Glossary

Semantic caching is a pattern for caching LLM responses keyed by meaning similarity rather than exact prompt match. The incoming prompt is embedded, a vector store is queried for near-duplicates, and if cosine similarity exceeds a threshold, the cached response is returned.