Data Curation for Fine-Tuning

After this lesson, you will be able to: Build and curate a fine-tuning dataset: sourcing, formatting with chat templates, decontamination against your eval set, and quality filtering.

In fine-tuning, the data is the experiment. This lesson covers how to construct a clean dataset, why decontamination is non-negotiable for a credible result, and the quality filters that matter more than raw size.

Prerequisites:Instruction Tuning and Preference Optimization

Data is the variable that matters most

With LoRA the architecture and recipe are mostly fixed; what you change is the data. So your dataset IS your contribution and your confound. Sources: existing public datasets (Hugging Face Hub), data you collect or annotate, model-generated (synthetic) data, or a mix. Whatever the source, you must be able to describe exactly how it was built, because reviewers will ask, and because the next lesson's evaluation is only meaningful if the training data did not leak into the test set.

Chat templates: format the data the way the model expects

Instruction models expect a specific role/template; the tokenizer applies it.

python

from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B-Instruct")

messages = [
    {"role": "system", "content": "You are a terse assistant."},
    {"role": "user", "content": "Define entropy in one sentence."},
    {"role": "assistant", "content": "Entropy measures uncertainty ..."},
]
# apply_chat_template inserts the model's exact special tokens / format.
text = tok.apply_chat_template(messages, tokenize=False)
# Train on `text`, masking the loss so it's computed only on the assistant turn.
# Using the WRONG template (or none) is a top cause of bad fine-tunes.

Common mistakes only experienced researchers catch

Skipping decontamination and reporting inflated numbers (fatal for a paper). Training without the model's chat template, so the fine-tune fights the format the base model learned. Computing loss over the whole sequence instead of masking to the response. Using synthetic data from a stronger model without disclosing it (a licensing and validity issue). Optimizing dataset size when a smaller, cleaner, deduplicated set would do better. Not setting and reporting a random seed for the train/val/test split, so the split is irreproducible.

Quick Check

Before reporting a fine-tuning result, what must you verify about the training data?

Pick the most critical check.

←Instruction Tuning and Preference Optimization

Back to Fine-Tuning and Adaptation

Compute Realities for Fine-Tuning→