Training data
Training data is the collection of examples, such as text, images, audio or records, that a machine learning model learns from, and it largely decides what the model knows, how well it performs and what biases it carries.
A model is only as good as what it learned from. A large language model is trained on vast amounts of text gathered from the public web, books, code and licensed sources; an image model learns from pictures paired with descriptions.
Training data is different from the information a model looks up while answering. Facts it learned in training are frozen at its knowledge cutoff. Facts it retrieves at question time, through search or RAG, can be current. Both shape answers, but only retrieved facts can change without building a new model.
How it works
Builders collect raw data, remove duplicates, spam and harmful material, and sometimes add labels such as "positive review" or "negative review". The model is then trained on this set, adjusting itself to predict or reproduce the patterns in it.
Web pages often become training data when AI crawlers visit them. Many AI companies publish the names of their crawlers so site owners can allow or block them in robots.txt. Builders rarely publish their training data in full. Some list the broad sources, such as public web pages, licensed content and data from human reviewers; others say little. Because the training set is fixed when training starts, anything that changed afterwards, including your prices, address or team, is missing unless the tool searches the web while answering. Asking an assistant about your business with web search switched off is a simple way to see what it learned in training alone.
For large language models, training happens in stages. Pre-training uses the broad collection of text to learn language and general knowledge. Later stages use much smaller, carefully written datasets, often including examples of good answers and human ratings of which reply is better, to make the model helpful and safe in conversation. Most of what a model knows comes from pre-training; most of how it behaves comes from the later stages.
Example
An accountancy firm wants a model to sort incoming emails into invoices, questions and complaints. It labels a few thousand past emails by hand, which become the training data. If nearly all the complaints in that set came from one big client, the model may struggle to spot complaints written in a different style.
The fix is not more data but better-balanced data: adding a few hundred complaints from smaller clients, checking labels for mistakes and testing the model on a fresh sample of emails it never saw. The firm keeps a short spreadsheet of misfiled emails each month, which doubles as new training examples. Within a few months the error rate on complaints from smaller clients is measured separately, so a drop in quality is visible rather than hidden in an overall average.
Why it matters for your business
Your public website may already be part of some model's training data, which shapes what AI tools say about you. Clear, accurate, consistent pages help. If you train or fine-tune a model on your own data, quality matters more than volume, and customer data brings privacy duties under rules like GDPR. Keeping your own facts consistent across your site, profiles and directories is the part you control.
Common mistakes
- Assuming a model knows recent changes to your business because it once read your website.
- Using customer records for training without a legal basis or without removing personal details.
- Training on a small, unbalanced sample and expecting the model to handle every case fairly.
- Blocking all AI crawlers without weighing the effect on how AI assistants describe your business.
Questions people ask
Is my website used as AI training data?
Possibly. Many AI companies train on publicly available web pages gathered by crawlers. You can check your server logs for known AI crawler names and use robots.txt to allow or block specific ones. Blocking stops future collection by crawlers that respect it, but does not remove pages already gathered.
What is the difference between training data and a prompt?
Training data is what the model learned from before you ever used it, and it shapes the model permanently. A prompt is what you type in a single conversation, and it affects only that answer. Major consumer AI assistants let you control whether your conversations may be used for future training.
Can bad training data cause wrong answers?
Yes. If the training data contains errors, outdated facts or one-sided examples, the model learns them. That is one reason AI tools sometimes repeat common myths or describe a business incorrectly, especially when there is little accurate information about it online.
