Token space

Plain English. The fixed vocabulary of chunks — tokens — a model can read and write, and the space of all possible sequences over it. A model does not choose among words or ideas; at each step it chooses among the tens of thousands of tokens in its vocabulary, one after another. Everything a model can express is a path through token space. Distinct from latent space: token space is the discrete, visible alphabet; latent space is the continuous, internal map of meaning.

Why it moves money. Token space sets the metering unit and the cost floor. Every product is billed per token (see /invest/dollars-per-million-tokens), every context window is measured in tokens (see /invest/context-window), and vocabulary design decides how many tokens a given language or codebase costs to process — which is why non-English and code workloads can carry a quiet price premium.

What to watch. Tokeniser efficiency as a hidden margin lever: a model that represents the same text in fewer tokens is cheaper to run and to buy, without any change in intelligence. Watch for vocabulary changes framed as capability that are really cost engineering.

Further reading. The GPT tokeniser, explained