Attention Mechanism — AI Glossary

An attention mechanism is a neural network component that allows a model to dynamically weigh the importance of different parts of the input when generating each part of the output. Rather than processing input as a fixed-length summary, attention lets the model "focus" on the most relevant tokens at each generation step. It is the core innovation that makes transformers and modern LLMs possible.