Artificial intelligence · Exhibit 2

How many tokens is a word?

Updated October 20, 2026

Plan on about 1.3 tokens per English word in ChatGPT, so 1,000 words come to about 1,300 tokens. That’s OpenAI’s own rule of thumb turned around, and my counts of real text came out between 1.15 and 1.4 around it, higher when the text is full of numbers. Other languages cost more for the same meaning, and Greek costs more than twice as much.

Tokens are the pieces a model reads, and they set both the price of a request and how much text fits in one. Type or paste anything into the box and your browser cuts it up the way GPT-4o does. The example is packed with awkward cases, so it shows about 1.5 tokens per word. Nothing you type leaves this page.

What is a token, exactly?

A language model doesn’t read letters. Before your message reaches it, a tokenizer cuts the text into pieces from a fixed list, the model’s vocabulary, and swaps each piece for its number on that list. The list behind GPT-4o and GPT-5 is called o200k_base and has about 200,000 entries. Common words have an entry of their own, usually with the space in front glued on: ” strawberry” is number 101830. Rarer words, typos and capitals are built from smaller pieces. STRAWBERRY in capitals takes four, the typo strawbery takes two, and numbers are cut into groups of up to three digits, so 1234567 becomes 123, 456 and 7.

The usual explanation for the old question about the r’s in strawberry blames tokens. After a space, ” strawberry” arrives as a single number, so the spelling isn’t in what the model receives. But written in quotes, or as the very first word of a message, the word arrives as three pieces, and a 2024 study of many models found that how a word is split barely predicts the mistakes: the models can name a word’s letters and still lose count, most of all when a letter appears more than twice.

How many tokens per word in English?

About 1.3. OpenAI’s help page puts it as one token for about ¾ of a word (0.75 words per token), or 100 tokens for about 75 words, and about 4 characters per token. Google’s Gemini docs say 100 tokens are about 60 to 80 English words. I counted two English texts to see how far that holds.

That post, why ChatGPT keeps picking 17, has grown since I counted it, so a fresh paste won’t match these numbers exactly. As it stood then, it sat almost exactly on both of OpenAI’s rules, at 3.8 characters per token. The Declaration does better than both, at 5.3.

About half of the difference between 1.15 and 1.39 tokens per word is numbers. Declaration, dignity and everyone are one token each, space included. A figure like 2,773 takes four: the space, the 2, the comma and 773. My article is full of figures like that. The only figures in the Declaration are its headings, Article 1 to Article 30. Strip every figure out of my article and it drops from 1.4 to 1.27 tokens per word. The rest is probably punctuation, but I haven’t measured it.

How many tokens is 1,000 words?

For a quick estimate, take the first column. The second shows how far my real texts strayed from it, at 1.15 to 1.4 tokens per word.

Whole books land in the middle of that range. I counted three from Project Gutenberg, with the line breaks inside paragraphs joined up the way you’d paste them. Pride and Prejudice is 159,225 tokens, 1.24 per word. Moby-Dick is 286,225 tokens, 1.32 per word. The King James Bible is 1,089,019 tokens, 1.28 per word, so it doesn’t fit in a million. Pasted with Gutenberg’s original line breaks, Pride and Prejudice costs 7% more, 170,258 tokens.

Which languages cost more tokens?

To compare languages fairly you need the same text in each one, so I counted the whole Declaration in 23 languages. Each bar below is that document in one language, divided by the English count. Pick a different vocabulary and the order changes.

Chinese came in at 1.17 times English, cheaper than Spanish, and Arabic costs about the same as Portuguese. At the far end, Polish costs 81% more than English and Greek more than twice as much.

Some of this comes from the translators. The Italian text is 19% longer than the English in characters, so it would cost more under any counting. Greek is only 17% longer and still costs 2.19 times as much. Petrov and colleagues (2023) and Ahia and colleagues (2023) found gaps like these across many more languages and tokenizers.

Tokens per word is a shakier number outside English. German glues words together into long ones, so it has fewer words for the same text. Chinese and Japanese are written without spaces, and their word counts here are a word splitter’s best guess.

Did GPT-4o make other languages cheaper?

For some of them, by a lot. GPT-4 and GPT-3.5 used cl100k_base, a list half the size. The Hindi Declaration takes 11,284 tokens with it and 3,376 with GPT-4o’s list, so the old list needed 3.3 times as many. For Bengali it needed 3.5 times as many, for Greek and Hebrew about 2.5 times, for Russian 1.8 times. Under GPT-4, the Hindi Declaration cost 5.6 times as much as the English one.

English barely moved: 2,016 tokens to 2,017 here, and 3,045 to 2,980 in my article about 17. The new list isn’t the old one with extras on top: it dropped about 15,000 of the old entries, and of its roughly 115,000 new ones, more than half contain characters outside plain English letters, digits and punctuation.

How many tokens per word in Gemini and Claude?

In Gemini, about 1.18 tokens per English word: the English Declaration came to 2,072 tokens, against 1.15 per word in ChatGPT. Every company builds its own list. Google’s counter is free with a Gemini API key, so I sent the same 23 texts to Gemini’s countTokens for Gemini 3.1 Flash-Lite, then again for Gemini 3.8 Flash, and got identical counts for all 23.

For English and the European languages here, Gemini comes within about 15% of GPT-4o either way. For several Asian scripts it’s clearly cheaper: Japanese takes 2,425 tokens in Gemini against 3,557 in GPT-4o, and Bengali and Hindi follow. Hebrew goes the other way. Each Gemini count carries one extra token, which I left in: Gemini’s counter says 2 for “hello” and 3 for “hello hello”, so a single message gets one token on top of the words, probably the start-of-text marker Gemma’s report tells developers to add. I only tested single messages.

For Claude I have no count of my own. Anthropic says its models since Claude Opus 4.7, which in October 2026 includes Opus 5.5 and the Fable 5.1 and Mythos 5.1 families, count the same text as roughly 30% more tokens than earlier Claude models, and that the exact increase depends on the content. How that compares with ChatGPT I haven’t measured: Claude’s counter needs a Claude API key, which I don’t have yet. The box above is exact for OpenAI’s newer models and close for Gemini in European languages. For Claude I can’t tell you how far off it is.

How many tokens per word in Llama, Qwen, DeepSeek and Gemma?

Open models publish their vocabularies, so I counted the Declaration with nine of them on my own computer, with no API in between. For English they sit close together, 1.14 to 1.19 tokens per word. In other languages they part ways.

Only the Chinese-made models make Chinese clearly cheaper than English: 0.81 times in Qwen3.5 and 0.83 times in DeepSeek, while Gemini is level at 0.99. Vocabularies are also still changing fast. Qwen3 needed 5.2 times as many tokens for the Hindi Declaration as for the English one; Qwen3.5, with a list of about 248,000 entries instead of 152,000, needs 2.2 times. Llama 3.1 counts English exactly like GPT-4, 2,016 tokens each, but needs about half as many tokens for Hindi. Llama 4 moved to a new list of 200,000 entries and cut Hindi again, from 2.95 times English to 1.76. Qwen3.6 kept the Qwen3.5 list byte for byte, so its numbers match.

Some lists don’t change at all. DeepSeek-V3 and V4.1 gave identical counts for all 23 texts, and so did Gemma 3 and Gemma 4. Google says Gemma 3 uses the same tokenizer as Gemini 2.0, and my counts suggest Gemini 3.1 still does: on all 23 texts, Gemini’s counter came out exactly one token above Gemma’s. That’s the extra token from the section above.

Do code and JSON cost more tokens than text?

Code has no words to count, so I measured it the way OpenAI states its rule of thumb, in characters per token, on files that were already on my computer. Code keeps its line breaks, so I compared it with the three books counted with their line breaks too: from 3.79 characters per token for the King James Bible to 4.26 for Pride and Prejudice. Six modules from Python’s own standard library (argparse, csv, textwrap and three more, CPython 3.11.9) came out at 4.76, and none of the six went below 4.08, so Python cost no more than the books. Five files of JavaScript from the semver package came out at 3.42: the same number of characters costs about a quarter more tokens than Jane Austen. Names get cut up: parseOptions is two tokens, and this.semver is three, this, .sem and ver.

JSON pays for its layout. Chrome’s DevTools protocol file, 1.4 million characters with a two-space indent, takes 206,554 tokens. With the indent removed it takes 163,995, a fifth fewer; on one line, 135,405, a third fewer. The indent costs a token on every line, and the space after each colon keeps the quotes apart, where on one line the quote closing a key, the colon and the quote opening its value make a single token. If you paste JSON into a model, put it on one line first. To see it yourself, paste the same JSON into the box above both ways.

Why do emoji and accents cost extra tokens?

Emoji and rare characters pile up tokens too. Each character takes up to 4 bytes in UTF-8, the encoding nearly every web page uses, and the vocabulary only has entries for the common ones, so the rest gets spelled out a byte or two at a time. Many emoji are several characters glued together. You’ve seen the thumbs-up in the box: two characters, a hand and a skin tone, three tokens. A German flag is two letters of a hidden alphabet, 8 bytes, and takes four.

Accents can cost tokens even when you can’t see the difference. The Vietnamese file I started with stored some letters in two parts, a plain e and a separate accent mark on top, which looks exactly the same on screen. Written fully that way (Unicode calls it NFD), the Vietnamese Declaration costs 8,872 tokens. Written the usual way, with each accented letter as one character, it costs 3,093. Same text to your eye, 2.9 times the tokens. I counted every language in the usual form, so the table above isn’t skewed by this.

How does this counter work?

It runs gpt-tokenizer, an open-source JavaScript version of OpenAI’s tiktoken under the MIT license, with the o200k_base list, in a separate thread so a very long line can’t freeze the page. The list weighs about 2 MB, under 1 MB compressed, and your browser doesn’t download it until you change the text; the example was counted in advance.

The count is exact for plain text in GPT-4o and GPT-4.1. For GPT-5, the o-series and gpt-oss I rely on OpenAI’s tiktoken library, which maps them to this list as of October 2026; OpenAI doesn’t publish a tokenizer for each newer model. gpt-oss uses o200k_harmony, this list plus extra control tokens. It isn’t exact for GPT-4, GPT-3.5 or OpenAI’s text-embedding-3 models, which use the older list from the table. In a chat, OpenAI’s cookbook adds about 3 tokens per message for GPT-4o, plus 3 to start the reply. Newer models add formatting tokens of their own, and OpenAI’s guide notes that “model-specific behavior can change tokenization”, so its input-token counter is the only exact count. The box doesn’t count any of those.

What this simplifies

My rule of thumb for everyday English rests on five texts, two counted in detail and three books, so it’s a rule of thumb at best. The language comparison rests on one formal document, translated by different people. A token count tells you what your text costs as input and how much of the context window, the most text the model can take in at once, it takes up. Reasoning models also bill their hidden thinking as output tokens, which no count of your own text can show. And none of this says how well a model understands each language.

Sources

• My counts, October 2026: all 23 languages in one CSV file, the nine open vocabularies in a second, and the three books in a third, with how they were counted. Free to reuse under CC BY 4.0. Cite as: How It Ticks, “How many tokens is a word?”, October 2026, howitticks.com/tokens-per-word/.
• Texts: the Universal Declaration of Human Rights from the udhr2 corpus in nltk_data, which collects the texts of the UDHR in Unicode project. Books from Project Gutenberg, ebooks 1342, 2701 and 10.
• OpenAI: Understanding and counting tokens (the ¾ rule); which model uses which list, tiktoken, model.py; about 4 bytes per token, the tiktoken README; tokens per chat message, How to count tokens with tiktoken.
• Google: Understand and count tokens, Gemini API docs.
• Open vocabularies: the tokenizer.json files of Llama 3.1, Llama 4, Qwen3, Qwen3.5, Qwen3.6, DeepSeek-V3, DeepSeek-V4.1, Gemma 3 and Gemma 4 on Hugging Face, at the revisions listed in the data file, counted with the tokenizers library. The Llama 3.1, Llama 4 and Gemma 3 files come from unsloth’s ungated copies of the same models; the repository for each is in the data file.
• Anthropic: Token counting, Claude docs (free counter, the newer tokenizer since Opus 4.7, the same in Fable 5 and 5.1 and Mythos 5 and 5.1; read October 20, 2026).
• OpenAI: Counting tokens (the input-token counter; model-specific behavior can change tokenization).
• Gemma Team, Google DeepMind, Gemma 3 Technical Report, 2025: Gemma 3 uses the same tokenizer as Gemini 2.0.
• Tairan Fu and colleagues, Why Do Large Language Models (LLMs) Struggle to Count Letters?, 2024.
• Aleksandar Petrov and colleagues, Language Model Tokenizers Introduce Unfairness Between Languages, 2023. Orevaoghene Ahia and colleagues, Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models, 2023.
• The counter: gpt-tokenizer by Bazyli Brzoska, MIT license.