WanJuan 1.0 Dataset

Scholar·WanJuan 1.0
Multimodal Corpus

Scholar·WanJuan 1.0 Multimodal Corpus open-source version. Released by Shanghai Artificial Intelligence Laboratory, it includes three major types of data: text, images with text, and video, covering various types such as scientific literature, encyclopedias, community Q&A, textbooks, code, etc., with a total size exceeding 2TB, specifically designed for pre-training large language models and multimodal models.

Get Dataset Now Online Preview
~2TB Total Data Multimodal Data 600M+ Text Samples Apache-2.0 License
WanJuan 1.0 Multimodal Corpus
💾
~2TB
Total Data
🔀
Multimodal
Text / Images with Text / Video
📄
600M+
Number of Text Samples
📜
Apache-2.0
Open Source License

Dataset Highlights

A large-scale multimodal pre-training corpus carefully constructed by the Shanghai Artificial Intelligence Laboratory

🌐

Multimodal Coverage

Covering three major modal types: text, images with text, and video, fully supporting the pre-training needs of large language models and multimodal models, with rich and diverse data forms.

📚

High-Quality Text Corpus

The text data includes various types such as scientific literature, encyclopedic knowledge, community Q&A, textbooks, and code, rigorously cleaned and deduplicated, ensuring quality.

📊

Ultra-Large Scale

The total data volume exceeds 2TB, with over 600 million text samples, more than 22 million image-text pairs, and over 1 million videos, meeting large-scale training needs.

🔓

Completely Open Source

Released under the Apache-2.0 license, supporting academic research and commercial use, with no concerns about license restrictions, freely accessible and usable.

🤖

Pre-training Ready

Designed specifically for LLM and multimodal model pre-training, with a unified standardized data format, directly usable in the training process of large models like InternLM.

🏛️

Professionally Built Team

Professionally produced by the Shanghai Artificial Intelligence Laboratory (Shusheng·Puyu InternLM team), with rigorous and reliable data collection, cleaning, and quality control processes.

Applicable Scenarios

Widely applicable to various tasks in the AI field, from model pre-training to academic research

🧠

Pre-training of Large Language Models

Used for training large language models such as InternLM, providing rich and diverse text corpora to support model knowledge acquisition

🖼️

Multimodal Model Training

Utilizing image-text pairs and video data to train visual-language understanding models, enhancing cross-modal capabilities

🔬

Academic Research

Supports research and experiments in fields such as natural language processing, computer vision, and multimodal learning

🎯

Domain Fine-tuning

Extracts domain data from subsets such as scientific literature, textbooks, and code for fine-tuning models in specific domains

Multimodal Pre-training Large Language Models NLP Computer Vision Open Source Data

Data Preview

The following is an overview of the data structure of the Wanjun 1.0 corpus

JSON
{
  "text": [
    {"type": "science",      "samples": "100M+", "format": "jsonl"},
    {"type": "encyclopedia", "samples": "50M+",  "format": "jsonl"},
    {"type": "qa_forum",     "samples": "80M+",  "format": "jsonl"},
    {"type": "textbook",     "samples": "30M+",  "format": "jsonl"},
    {"type": "code",         "samples": "20M+",  "format": "jsonl"}
  ],
  "image_text": {
    "samples": "22M+",
    "format": "tar+jsonl",
    "description": "# Image-text pair data, containing images and corresponding text descriptions"
  },
  "video": {
    "samples": "1M+",
    "format": "tar+jsonl",
    "description": "# Video data, containing video clips and annotation information"
  },
  "license": "Apache-2.0",
  "total_size": "~2TB",
  "publisher": "Shanghai AI Lab (Shanghai Artificial Intelligence Laboratory)"
}

3 Steps to Get Started Quickly

Quickly integrate the Wanjun 1.0 dataset into your workflow from browsing to training

01

Browse the Dataset

View the details of the Wanjun 1.0 dataset on the Ace Data Cloud platform, and understand the metadata such as the scale, format, and licensing agreements of each modality subset.

02

Download or Stream

Download or stream the text, image-text, and video subset data files as needed, with support for selective retrieval by category.

03

Load into Training Pipeline

Load the data into your training pipeline, supporting mainstream frameworks like PyTorch DataLoader, and start model pre-training directly.

Get the Dataset

Start Exploring the Wanjun 1.0 Data

A large-scale multimodal open-source corpus, licensed under Apache-2.0, available now. Whether you are training large language models or multimodal models, Wanjun 1.0 is the ideal data foundation.

Get the Dataset Now