Nvidia and AI4Bharat are working on a dataset containing 2.8 trillion tokens across 22 Indian languages. The aim is to help developers train AI models that can better understand and respond in Indian languages. The project also includes tools to create and check data used for AI training.
The dataset is called Nemotron-pretraining-Indic-v1. Nvidia and AI4Bharat are developing it together. AI4Bharat is a language research lab at the Indian Institute of Technology Madras (IIT Madras). The team also plans to release the dataset-building tools.
One of the main reasons behind the project is the lack of digital training data for many Indian languages. AI models have access to far more training material in English than in several Indian languages. This gap can affect how well these models understand local languages and answer questions from users.
In an announcement published on October 16, 2026, the team said the dataset covers all 22 languages listed in the Eighth Schedule of the Indian Constitution.
Nvidia’s New AI Dataset Covers 22 Indian Languages
Nvidia’s Nemotron-pretraining-Indic-v1 dataset contains around 2.8 trillion tokens of text in 22 Indian languages. It covers 34 language and script combinations and is intended to help developers train AI models on Indian language data.
The dataset includes text in the native scripts of all 22 languages. It also has Romanized text for 12 languages, where words are written using the English alphabet. This matters because people often use different scripts when communicating online.
For example, someone might write Hindi in English letters rather than Devanagari. Including these variations could help AI models understand the way people write and communicate on the internet. The project’s technical documentation lists around 2.601 trillion tokens in native scripts and 224.6 billion tokens in Romanized text. The figures were calculated using the Gemma-4 tokenizer.
The dataset covers Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi and Odia, along with the other languages listed in the Eighth Schedule of the Indian Constitution.
According to the project announcement, these 22 languages were the first languages of about 97% of India’s population, based on the 2011 Census. The dataset could give developers more material to train AI models that understand the languages spoken by people across the country.
ALSO READ: India’s AI Industry: Key Statistics and Trends (2025–2026)
How Nvidia and AI4Bharat Built the Dataset
Nvidia and AI4Bharat created the dataset by translating English documents into 22 Indian languages. The source material comes from 15 subsets of Nvidia’s existing Nemotron-CC-v2 collection.
The team used Nvidia’s NeMo Data Designer to translate the documents and check the results. The tool helps find poor-quality translations and correct them before adding the text to the dataset.
For 12 languages, the process uses Google’s Gemma-4-31B-it model for translation and quality checks. For the other 10 languages, it uses Bodhan-MT for translation and Gemma-4-31B-it to review the results. Translations that fail to meet the required quality standards are revised and checked again.
The team also works to preserve the original format of each document, including tables, equations, links and code. Each translated record remains linked to its English source, making it easier to trace where the data came from.
Nvidia and AI4Bharat plan to release the dataset along with the tools used to create it. They will also provide a human-verified test suite to help researchers evaluate AI models across Indian languages.
ALSO READ: Open Source AI Statistics
How the Nvidia AI4Bharat Dataset Could Improve Indian-Language AI
Nvidia AI4Bharat Dataset could help developers build AI models that better understand and generate text in Indian languages. It may support tools such as chatbots, translation services and educational platforms.
To test the benefits of additional training data, the researchers continued training Nvidia’s Nemotron 3.5 Super VL model using 10 billion tokens in Indian languages. According to Nvidia’s technical documentation, the model improved its scores across five Indian-language benchmarks after this training.
On the MILU benchmark, the model’s score rose from 72.48 to 77.90. Its score on Indic MMLU-Pro also increased, from 51.83 to 56.81.
The researchers reported gains on three other tests: Indic ARC-Challenge, Indic ARC-Easy and Indic GSM8K. These benchmarks assess different language, reasoning and problem-solving skills.
The results show that adding more Indian-language data to AI training can improve performance on certain tests. However, better benchmark scores do not necessarily mean a model will perform well in every Indian language or everyday situation.
Why AI Models Need More Indian Language Training Data
Building AI models for Indian languages is difficult because each language has its own grammar, vocabulary and script. There is also less digital text available for some languages than for others.
AI models need enough training data to understand how people use a language. But the quality of that data matters just as much as the amount. Poor translations or limited examples can affect how well a model responds.
Nvidia and AI4Bharat aim to address this problem by creating a large dataset of Indian-language text. Their tools could also help researchers prepare more training data from other sources.
However, translated text can contain errors. Checking the data and reviewing its quality will be important before using it to train AI models.The project could give researchers more resources to build and test AI tools for Indian languages.