When custom language model projects disappoint, the cause is rarely the training algorithm. Much more often, it is the data: too noisy, too narrow, unrepresentative, or full of information that should never have been included. The good news is that data preparation is something your team can start on well before any training begins. This guide walks through the process step by step.
Step 1: Start with the task, not the data
It is tempting to begin by gathering everything you have. Resist that. First, write down precisely what the model should do. For example: "Given a customer email, draft a reply that follows our support guidelines and cites the relevant help article." A clear task tells you which data matters, what format it should take, and how you will measure success.
For each task, note:
- The typical input, such as an email, a contract, a form, or a question.
- The desired output, such as a reply, a label, extracted fields, or a summary.
- Who currently does this work and what "good" looks like to them.
- What mistakes would be most costly.
Step 2: Inventory your data sources
Next, list every place where relevant data lives. Common sources include:
- Support tickets and chat transcripts.
- Internal wikis, manuals, and standard operating procedures.
- Product documentation and knowledge base articles.
- Emails and templates.
- Contracts, reports, and other business documents.
- Spreadsheets and databases with structured records.
For each source, record its approximate size, format, age, owner, and, critically, whether you have the right to use it for training. Some data may be covered by customer contracts, privacy commitments, or third-party licenses that limit its use. Involve your legal or compliance team early.
Step 3: Protect personal and sensitive information
Language models can memorize parts of their training data. Anything you include could, in principle, surface in an output. Before training, decide how to handle:
- Personal information such as names, email addresses, phone numbers, street addresses, and account numbers.
- Confidential business information that should not be visible to every user of the model.
- Credentials and secrets accidentally pasted into tickets or documents.
Typical approaches include removing such data, replacing it with realistic placeholders, or excluding entire documents. Automated detection tools help, but should be combined with sampling and manual review. Also consider who will use the model: a model for internal staff may tolerate different content than one exposed to customers.
Step 4: Clean the data
Raw business data is messy. Cleaning usually involves:
- Deduplication. Remove exact and near-duplicate documents. Repeated content causes the model to over-weight it.
- Boilerplate removal. Strip email signatures, legal footers, navigation menus, and auto-generated text.
- Removing outdated material. Old policies, discontinued products, and superseded procedures teach the model the wrong things.
- Fixing encoding and formatting. Convert documents from scanned files, word processors, and spreadsheets into clean text, preserving meaningful structure such as headings, lists, and tables.
- Filtering low-quality examples. In support data especially, not every past reply is a good one. Exclude replies that were rude, wrong, or escalated as complaints.
Step 5: Choose the right format
How data should be formatted depends on the training method.
For continued pre-training
You need large volumes of clean domain text, structured as documents. Quantity matters more here, but quality still counts.
For instruction tuning
You need pairs of instructions and ideal responses. These can come from historical data, such as a customer question and an approved answer, or be written by subject-matter experts. Keep a consistent structure: if replies should always end with a link to a help article, every example should do so.
For preference alignment
You need sets of responses to the same prompt, ranked from better to worse by people who understand the domain. Clear written guidelines for raters help keep judgments consistent.
For distillation
You need a large, representative set of real inputs. A teacher model generates the target outputs, which are then filtered for quality.
Step 6: Write high-quality examples
For many projects, a few hundred to a few thousand excellent examples create more value than a vast pile of mediocre ones. Ask your best performers to write or review examples. Cover the full range of situations the model will face, including difficult and unusual cases, not just the easy ones. Where the correct behavior is to refuse, escalate, or ask a clarifying question, include examples of that too. Models learn what you show them.
Step 7: Build an evaluation set before you train
An evaluation set is a collection of examples, separate from the training data, used to measure the model's performance. It is the single most important artifact in the project, because every decision depends on it.
- Draw examples from real, recent usage, so they reflect what the model will actually see.
- Include common cases, edge cases, and cases where mistakes are costly.
- Define how each output will be judged: exact match, automatic scoring, rubric-based human review, or a combination.
- Keep the evaluation set strictly separate from training data, and avoid near-duplicates across the two.
- Agree with stakeholders, in writing, what score counts as success.
Step 8: Document everything
Keep a record of where each dataset came from, what filtering was applied, what was removed and why, and which version was used for each training run. This documentation makes results reproducible, supports compliance reviews, and makes future retraining much easier.
Step 9: Plan for ongoing data
Your business will change. New products, policies, and customer questions will appear. Plan from the start how new data will be collected, reviewed, and added, and how often the model will be re-evaluated. A simple quarterly review of a fresh sample of real inputs can catch drift before it becomes a problem.
How much data is enough?
There is no universal number, but some rough guidance helps with planning. For instruction tuning on a focused task, a few hundred carefully reviewed examples can produce a noticeable change, and a few thousand often gets close to what the data can offer. Preference alignment typically needs hundreds to a few thousand ranked comparisons. Continued pre-training benefits from far larger collections, often millions of words of clean domain text. Distillation needs many representative inputs, but those inputs do not need human-written answers.
The better question is usually not "how much?" but "how representative?" A smaller set that covers the real range of situations, including rare and difficult ones, nearly always beats a larger set dominated by easy, repetitive cases.
A quick checklist
- Tasks are written down with clear inputs, outputs, and success criteria.
- All data sources are inventoried, with usage rights confirmed.
- Personal and sensitive information is handled according to a written plan.
- Data is deduplicated, cleaned, and current.
- Training examples follow a consistent format suited to the chosen method.
- Expert-reviewed examples cover common, rare, and difficult cases.
- A separate evaluation set exists and success thresholds are agreed.
- Data sources and processing steps are documented.
- There is a plan for ongoing data collection and re-evaluation.
Conclusion
Good data preparation is not glamorous, but it is where most of the value in a custom model is created. Teams that invest in clear tasks, clean and lawful data, and a solid evaluation set get better models, faster, and with fewer surprises. Even if you plan to work with an outside partner, doing the first few steps yourself will make the project quicker and less expensive.
Learn more about how we handle data on our Responsible AI page, or read about our custom LLM training service and our guide to fine-tuning versus training from scratch.
Want to discuss how this applies to your project? Book a free scoping call.