Don't know English? Talking to AI in Hindi will cost you more; a 'language tax' is being applied..
Using AI chatbots to communicate in Hindi—or any language other than English—can end up costing you more. Companies like OpenAI, Anthropic, and Google often present their new AI models as tools that function identically for everyone, regardless of location or language. However, new data from researchers reveals that users interacting with AI in languages such as Hindi, Arabic, and Chinese may actually incur higher costs compared to those communicating in English.
What is the reason for this?
It depends on how the AI model processes language; the same prompt (instruction) in Hindi can generate significantly more "tokens"—the units AI systems use to read and comprehend text—than it would in English. Simply put, the AI consumes more tokens to process the same message in Hindi than in English. Consequently, using AI becomes more expensive for non-English speakers.
The "Language Tax": What is it?
Researchers and developers often refer to this as a "language tax"—essentially a hidden cost resulting from the way AI models process different languages. A few weeks ago, OpenAI researcher Aran Komatsuzaki detailed an experiment examining how tokenizers from OpenAI and Anthropic process text across various languages. Using AI researcher Rich Sutton’s famous article "The Bitter Lesson" as a benchmark, Komatsuzaki translated the text into multiple languages and observed the number of tokens generated by different AI systems.
The results highlighted a significant disparity between English and other languages. The analysis showed that OpenAI’s tokenizer required 1.37 times more tokens for Hindi text compared to English. Meanwhile, for Anthropic’s Claude tokenizer, that figure rose to 3.24 times. Claude required 2.86 times more tokens for Arabic, whereas 1.71 times more tokens were needed for Chinese.
Disclaimer: This content has been sourced and edited from TV9. While we have made modifications for clarity and presentation, the original content belongs to its respective authors and website. We do not claim ownership of the content.
