Tokens

LLMs break text into tokens (words/subwords, e.g. “un”, “believe”, “able”).

From 10-AI-Concepts

• Basis for context windows (max tokens processed).