Token — AI Glossary

A token is the basic unit of text that AI models use to process and generate language. Tokens can be whole words, parts of words, or individual characters — for example, the word "prompting" might be split into "prompt" and "ing." Token counts determine context window limits, API costs, and processing speed.