What is a token?
You'd assume ChatGPT is reading your English, but it doesn't actually "know" letters. Every sentence you send gets chopped into little fragments first — those fragments are tokens. A short English word is usually one token; a longer word might be two or three.The model works with these tokens, not with words themselves.
Why it matters
It decides cost and lengthAPI pricing, free-tier limits, the maximum context length — all of it is tokens doing the accounting. Models charge per token, and each one has a hard cap on how many it can take at once.
It's the reason some languages are "cheaper"
Languages tokenize differently. A sentence in one language might use fewer tokens than the same idea in another, which quietly changes the bill.
A quick example
The phrase "I love apples" might become three tokens: "I", "love", "apples". "I really love apples" adds one more. The finer the split, the more precisely the model reads — and the more it costs.Bottom line: tokens are the AI's unit of measurement. Every dollar you spend, every inch of context you occupy, is being quietly counted in tokens.
Comments