How large language models are trained, stage by stage, in plain English

The short answer: four stages, each fed by different data

Large language models are trained in sequence. First comes pretraining, where the model reads an enormous volume of text and learns to predict the next word. Post-training then turns that raw predictor into a useful assistant: supervised fine-tuning on worked examples, followed by preference and reinforcement training that rewards better responses. Evaluation runs throughout, testing the model on material it has never seen.

Each stage needs a different kind of data, and that is where licensed business records come in. Pretraining wants volume. The later stages want quality: realistic examples of tasks done well, clear outcomes, and tests the model could not have memorized. Records of real company work are scarce on the public web, so they tend to matter most in those later stages.

The training pipeline at a glance

Stage 1: pretraining teaches language from text at scale

Pretraining is the expensive part. Text is split into tokens, which are words or word fragments, and the model repeatedly guesses the next token in a passage, then nudges billions of internal numbers, called weights, to guess better next time. Repeat that across a vast corpus and the model absorbs vocabulary, grammar, facts and patterns of reasoning without anyone labeling anything.

The constraint is supply. Researchers at Epoch AI estimated the effective stock of human-generated public text at roughly 300 trillion tokens and projected that, if current trends continue, language models will have fully used it at some point between 2026 and 2032 (Epoch AI). That forecast carries wide uncertainty, but it explains why developers look beyond the open web.

Stage 2: supervised fine-tuning teaches the model to follow instructions

A pretrained model can continue a passage, but it does not reliably answer questions or carry out requests. Supervised fine-tuning fixes that by training on curated examples: a prompt, followed by the response a skilled person would give. These datasets are far smaller than pretraining corpora, and quality counts for more than quantity.

Fine-tuning is also how developers specialize a model, for example toward customer support, finance operations or software engineering, by adding examples from that field.

Stage 3: preference and reinforcement training shape behavior

Next, the model learns which of its own outputs are better. In preference training, people compare two responses and pick the stronger one, and those choices train a reward signal that steers the model toward helpful, accurate answers. In reinforcement learning, the model attempts tasks, a grader checks the result, and successful attempts are reinforced.

Reinforcement learning grows in importance as models become agents that carry out multi-step work inside software. Developers build practice workspaces for this, described in the explainer on RL environments.

Stage 4: evaluation tests the model on work it has not seen

Evaluation is less a final step than a constant check. Developers run models against held-out test sets to measure accuracy, safety and task completion, and compare versions before release. A test only works if the model did not see its questions during training.

There is a quieter risk as well. As more web text is itself generated by AI, models trained heavily on it can degrade over generations, the problem covered in what model collapse is. Fresh human-created records help on both fronts: they are unseen, and they reflect how people actually work.

What makes a business record usable for training

Useful is not the same as permitted. A record only helps a developer if the company can license it and the developer can document where it came from. Three checks come first:

• Ownership: the US Copyright Office's circular on works made for hire explains that material employees create within the scope of their jobs is generally owned by the employer, while content from outside contractors may not be unless rights were assigned in writing.
• Third-party content: documents that belong to clients, or that were licensed from vendors, usually need the other party's agreement or must be excluded.
• Personal data: names, contact details and sensitive information are redacted or de-identified under rules the company agrees before any work begins.

This is general information, not legal, tax or financial advice. Confirm with your own counsel, tax adviser or professional body before acting.

The explainer on ethical sourcing of AI training data sets out why provenance and consent weigh heavily with developers, and enterprise AI data licensing deals beyond the media headlines shows how operating companies take part.

What this means if you advise companies

You do not need to understand the mathematics to spot a client whose records suit the later training stages. Look for work recorded end to end, with results attached:

• The company has 50+ full-time employees at peak (contractors excluded) and several years of documented operations.
• Work runs through many systems, such as email, chat, CRM, finance, support and engineering tools, and retired systems were archived rather than deleted.
• Records show complete tasks with outcomes, not just final documents.
• The company created the records, and its contracts and notices allow licensing.
• An owner, CEO, CFO or other authorized representative would sponsor the conversation.

The company fit checker runs a preliminary, non-binding version of these checks. Partners earn 25% of the eligible platform fees SourceX actually collects from the referred company's licensing deals, capped at $100,000 per referred company; the reward becomes payable only after the buyer pays and SourceX receives its fee, and no reward is guaranteed. The rewards policy explainer sets out the details in plain language.

Next step

If a client's records read like worked examples with outcomes, register as a partner and send the introduction. SourceX handles qualification, the data inventory, terms and delivery, and the partner never touches the data.