SFT Data Curator

When designing an instruction-tuning dataset for a specific capability (code generation, math, conversation, safety), use this framework to plan data collection, define quality criteria, and structure the training pipeline.

Input Requirements

Provide:

• Target capability (e.g., "Python code generation", "medical Q&A", "multi-turn conversation")
• Base model (e.g., Llama 3 8B, Mistral 7B, Qwen 2.5 72B)
• Budget (annotation hours, API costs for synthetic generation)
• Format preference (Alpaca, ShareGPT, ChatML)

Step 1: Dataset Design

Size Guidelines

Quality Level	Examples Needed	Expected Outcome
Research prototype	1,000-5,000	LIMA-quality: comparable to larger datasets if examples are expert-written
Production v1	10,000-50,000	Stanford Alpaca level: solid instruction following across common tasks
Production v2	50,000-200,000	Vicuna/Llama 2 Chat level: robust multi-turn, domain coverage

Quality always beats quantity. 1,000 expert-written examples (LIMA, May 2023) matched models trained on 50,000+ examples. Prioritize:

1. Diversity -- cover the full range of target capabilities
2. Accuracy -- every response must be factually correct
3. Clarity -- responses should be concise and well-structured
4. Difficulty gradient -- include easy, medium, and hard examples

Diversity Checklist

For a general-purpose assistant:

• Open-ended questions (20%)
• Factual Q&A (20%)
• Creative writing (10%)
• Code generation (15%)
• Reasoning and math (15%)
• Summarization (10%)
• Instruction following with constraints (10%)

Adjust percentages for domain-specific models. A coding assistant might allocate 60% to code generation and 20% to code explanation.

Step 2: Data Format

Alpaca Format (single-turn)

{
  "instruction": "Write a function that reverses a string in Python.",
  "input": "",
  "output": "def reverse_string(s):\n    return s[::-1]"
}

Use when: single-turn tasks, simple instruction-response pairs, rapid prototyping.

ShareGPT Format (multi-turn)

{
  "conversations": [
    {"from": "system", "value": "You are a Python expert."},
    {"from": "human", "value": "How do I reverse a string?"},
    {"from": "gpt", "value": "Use slicing: s[::-1]"},
    {"from": "human", "value": "What about for a list?"},
    {"from": "gpt", "value": "Same syntax works: my_list[::-1]"}
  ]
}

Use when: conversational applications, multi-turn context is important.

ChatML Format (with special tokens)

<|im_start|>system
You are a Python expert.<|im_end|>
<|im_start|>user
How do I reverse a string?<|im_end|>
<|im_start|>assistant
Use slicing: s[::-1]<|im_end|>

Use when: targeting models that use ChatML natively (Qwen, Yi).

Step 3: Quality Criteria

Per-Example Checks

1. Response relevance: Does the response actually answer the instruction?
2. Factual accuracy: Are all claims verifiable and correct?
3. Completeness: Does the response fully address the instruction?
4. Conciseness: Could the same information be conveyed in fewer words?
5. Format consistency: Does the response follow the expected style?

Red Flags (reject the example)

• Response contradicts itself
• Response includes harmful content without refusal
• Response hallucinates facts or citations
• Instruction is ambiguous and response doesn't clarify
• Response is a copy of the instruction rephrased

Dataset-Level Checks

• No more than 5% of examples from any single source/template
• At least 80% of response tokens are meaningful (not filler)
• Average response length is 50-200 tokens (avoid very short or very long)
• System prompt diversity: at least 10 different system prompts represented

Step 4: Training Configuration

Parameter	Recommended Range	Notes
Learning rate	1e-5 to 5e-5	Lower for larger models (1e-5 for 70B, 5e-5 for 7B)
Epochs	1-3	Monitor validation loss, stop at first sign of increase
Batch size	32-128	Scale with gradient accumulation if GPU-limited
Warmup	0-5% of steps	Less critical than pre-training
Weight decay	0.0-0.1	Optional for short fine-tuning runs
Loss masking	Response tokens only	Mask instruction and system prompt tokens
Pre-training data mixing	2-5%	Mix raw text to prevent catastrophic forgetting

Step 5: Evaluation Protocol

After training, evaluate on:

1. Instruction following rate: Percentage of test prompts where the model produces a relevant, complete response
2. Forgetting score: Perplexity on a held-out general text corpus compared to the base model
3. Format compliance: Percentage of responses that follow the expected chat format
4. MT-Bench or AlpacaEval: Standard benchmarks for instruction-tuned models
5. Domain-specific eval: Custom evaluation for your target capability

Warning Signs

• Validation loss increases after epoch 1: you're overfitting, reduce epochs or increase data
• Forgetting score increases > 15%: learning rate too high or too many epochs
• Model reproduces training examples verbatim: severe overfitting, needs more diverse data
• Model refuses benign instructions: over-trained on safety data, rebalance the dataset