When DeepSeek-R1 landed in January 2025, it proved that chain-of-thought plus reinforcement learning could push open models to near-frontier reasoning on math and code. It also proved something less convenient: you do not run a 671B-parameter MoE for every customer support query. The R1 paper’s most practical contribution is the distillation recipe — generate reasoning traces from the teacher, then supervised fine-tune a small model on them. We ran exactly that playbook for a math-heavy internal tool, distilling into Qwen2.5-7B and Llama-3.1-8B. The gap between a student that parrots <think> tags and one that actually reasons turned out to be almost entirely a data-quality problem.

The data pipeline is the whole game

Distillation here is SFT on reasoning traces, not teacher-forcing a softmax distribution. The student learns to emit a thought block followed by an answer, in the teacher’s style. Whether that transfers real reasoning depends on what you put in front of the student.

Our pipeline, in order:

1. Sample from real workloads. We used typed math and code problems from our own support queues, not just GSM8K. Synthetic questions from the teacher help coverage, but real traffic is what the model will actually see.
2. Generate multiple traces per question from the teacher at temperature ~0.7, capturing the full thought plus final answer.
3. Rejection-sample on correctness. Keep a trace only if the final answer is verifiably correct — unit tests for code, known answers for math. This is the single most important filter. Students are superb at internalizing the error pattern of a teacher that got 30% of its reasoning wrong.
4. Deduplicate near-identical traces with MinHash. We ended up with ~90K rows from 250K raw generations.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import json

def build_distill_dataset(raw_traces, verifier):
    """raw_traces: [{question, thought, answer}]; verifier(answer) -> bool."""
    rows = []
    for t in raw_traces:
        if not verifier(t["answer"]):
            continue
        rows.append({
            "messages": [
                {"role": "user", "content": t["question"]},
                {"role": "assistant",
                 "content": f"<think>{t['thought']}</think>{t['answer']}"},
            ]
        })
    return rows

The thought block lives inside the assistant turn, which keeps it in the same token stream the model will generate at inference time — no scaffolding to load at serve time.

Training

LoRA (r=64, alpha=128) on Qwen2.5-7B and Llama-3.1-8B, learning rate 4e-5 with cosine decay, 2 epochs, effective batch size 256 via gradient accumulation, on 8 x A100-80G with gradient checkpointing. Each run cost about 20 hours. The R1 team’s own numbers are the target to sanity-check against: their Qwen2.5-7B distill hit 55.5% on AIME 2024 and 92.8% on GSM8K. We landed within a couple of points on both — which is where the “was it actually worth it” conversation starts to get interesting.

Evaluating the reasoning, not the style

The classic failure of trace distillation is style mimicry: the student emits a <think> block that paraphrases the question without reasoning, followed by a plausible-but-wrong answer. Per-token loss looks great — the model learned the format — and so does human eyeballing, because the format is what the eye checks first. Loss and eyeballs both lie.

We evaluated on GSM8K, MATH-500, AIME 2024, and an internal verifier set, and graded output correctness, not fluency:

• The answer-filtering fix was worth the most. A student trained on unfiltered traces scored ~12 points lower on AIME than the same model trained on verified traces only.
• “Fewer, better traces” beat “more traces.” Going from 90K to 200K rows without re-filtering didn’t help; the extra rows were mostly repeated error patterns.
• Over-training collapses the model. At 4+ epochs, held-out AIME dropped several points even as train loss fell — the student starts memorizing trace text instead of the underlying reasoning.
• Reinforcement learning (GRPO) on top of the SFT model added a few AIME points, but it is a separate project with its own instability; SFT distillation alone was 80% of the value.

Production lessons

• Ship the student quantized. An AWQ-quantized 7B serves thousands of queries on a single L4 or A10G at 25-40 tok/s decode — the 30-50x cost reduction versus the teacher is the entire point of the exercise.
• Keep the teacher’s traces in a versioned store. They are the asset; the weights are disposable. Every new eval failure becomes “generate more traces in the failing region.”
• Gate on correctness benchmarks, and include an adversarial style check (e.g., “how often does a wrong answer come with a confident <think> block?”). That specific number is how you catch mimicry in a dashboard.
• Track reasoning quality separately from pass@k. An agent that reasons correctly 90% of the time but answers 30% wrong is a different product than one that gets all its wrong answers from style mimicry — and they demand different fixes.