Needle in a Haystack

Needle in a haystack is a long-context evaluation pattern that measures whether a model can retrieve a specific fact (the needle) planted at an arbitrary position inside a long irrelevant passage (the haystack). The evaluator varies two dimensions — the total context length and the depth at which the needle is inserted — and asks a question that can only be answered from the needle. Recovery rates are reported as a grid, which exposes where model attention degrades: near perfect at the start and end, often worse in the middle, sometimes collapsing at long context lengths. Popular in 2023-2024 as an initial check on long-context claims, the benchmark is useful but limited — it tests literal recall of a planted fact, not multi-hop reasoning, aggregation across the corpus, or adversarial distractors. Modern long-context evals add those dimensions rather than replacing NIAH entirely.