The data-curation toolkit — selection, negatives, coverage

The question this chapter answers: given a pile of run logs and a diagnosed gap, what are the actual mechanics — which row to keep, how to mine a negative that teaches the right ranking, how to synthesize what’s missing, how to prove your data has a hole before you find out the expensive way — that turn “we have logs” into “we have a training set”? BLUF: there is no single “clean the data” step. Selection, negative-mining, active acquisition, decontamination, curriculum, synthesis, and coverage-auditing are seven separate instruments, each answering a different sub-question, and the field’s post-2022 LLM literature is almost entirely three much older ideas — curriculum/self-paced learning, active learning, and hard-negative mining for contrastive retrieval — re-derived for autoregressive transformers. Knowing the lineage tells you which tool each new-sounding 2025/2026 paper is actually handing you.

This chapter is the cross-cutting toolkit, not the per-gap playbook. It answers how you select, mine, and audit data; it deliberately does not re-decide which fix (SFT vs DPO vs GRPO) a given failure needs — that routing lives in Matching the fix to the gap and The three gaps, defined, whose K/R/P vocabulary (Knowledge gap / prior-ranking “R” gap / policy-vs-judgment “P” gap) is exactly the taxonomy this chapter’s methods feed: §1–2 and §7–8 are mostly load-bearing for K- and R-gaps (what to inject, how to know it’s missing), §3–4 are mostly load-bearing for R- and P-gaps (how to mine the negative that teaches the ranking). (K/R/P maps onto framework.md’s knowledge/execution/exploration lens as execution = R ∪ P, one continuous axis at two granularities — see taxonomy.md §0 for the full reconciliation, canonical.) The kinds of SFT §6 already covers LIMA/AlpaGasus/IFD/LESS/DEITA and n-gram decontamination at the SFT-row level — this chapter extends that into the full toolkit (production-scale gotchas, hard-negative mining for preference pairs, active/curriculum acquisition, synthesis mechanics, coverage-gap detection) without repeating its worked examples. Per-gap-specific curation (what a confirmed K-gap corpus recipe looks like end to end) is Knowledge-gap curation; which fix to route to per gap is Intervention per gap; preference-method mechanics (the DPO/KTO loss itself) is Preference.

Running example, threaded through every section: your CTF-solving agent’s run logs, where you need to decide (a) which of your run-log rows survive into a training set, (b) how to construct (shallow-tool ≺ expert-tool) and (default-args ≺ expert-args) preference pairs — the agent reaches for nmap/curl by reflex and never reaches for amass (passive subdomain/recon enumeration) even when it’s the objectively better tool for the job — and (c) how you’d prove, before you spend a training run finding out, that your corpus has a coverage hole where amass/passive-recon should be.

Status: every arXiv id below is verified live against arxiv.org/abs/<id> per the source ledger (artifacts/three-gap-survey/{section-F,ledger-F}.md, verification pass 2026-07-02); non-arXiv sources (OpenReview, ACL Anthology, pre-arXiv classics) are cited by venue, not linked to an invented id. Confidence tags follow the ledger: unmarked = CONFIRMED and load-bearing; flagged inline where a claim is single-paper, contested, or too recent to be independently replicated.

0. The genealogy in three lines

Three 2009–2010-era ideas, developed independently for entirely different problems, are what every 2023–2026 LLM-data-curation paper in this chapter re-derives:

Why the lineage matters practically: every time a 2025/2026 paper’s headline sounds novel — “online curriculum for RL,” “prompt-perturbation uncertainty,” “policy’s-own-failures hard-negative mining” — check which of these three roots it’s re-deriving. It tells you the failure mode it doesn’t fix (self-paced curricula still need re-estimation as the policy moves; uncertainty sampling still confounds “unsure of the fact” with “unsure of the phrasing”; hard-negative mining still needs a floor against too-easy or too-hard negatives) before you trust a paper’s improvement as free.

1. Selection — which rows to keep

“Selection” answers one question: given a pool of candidate SFT rows, which subset do you actually train on? The seminal finding, repeatedly confirmed at small-to-medium scale then complicated at production scale, is that more data is not more signal — a smaller, better-chosen set beats a larger, unfiltered one.

What “quality” operationally means, collapsed to one sentence per axis: correctness (AlpaGasus) is “is this response actually right,” informativeness (IFD) is “did the model need this instruction to produce this response,” targeted influence (LESS/BIDS) is “does this row’s gradient point at the specific capability I diagnosed as missing” — three different, all legitimate, all not interchangeable meanings of “good data.”

The production-scale correction

The single most important corrective in this literature: at production scale, most of this collapses toward random. Ivison et al. (AI2, up to 5.8M-row pools) arXiv:2503.01807 found IFD/DEITA/complexity-style selection falls below random selection once you count their own compute cost — only a cheap, gradient-free, judge-free representation-similarity method (RDS+: weighted mean-pooled hidden states of a pretrained-not-instruction-tuned LM, similarity/kNN against a small target set) stays Pareto-optimal at every scale. This is independently corroborated by the Qwen/Alibaba team at million-scale, arXiv:2410.09335: self-scoring methods that beat random at 10K–100K scale become statistically indistinguishable from random at million-scale, on two independent million-scale pools.

This is a genuine, unresolved contested edge, not settled: the seminal small-scale papers (10K–100K pools) all say sophisticated selection wins; the two production-scale studies above say it collapses to or below random once compute is counted. The field has not published a principled crossover pool-size threshold. Regime-gate your method choice (§9) rather than trusting either side unconditionally.

A related complication for your teacher choice, not selection per se: arXiv:2411.07133 found that a stronger model’s own benchmark performance does not monotonically predict how good its responses are as SFT training targets for a given student — don’t assume “biggest teacher available” is automatically the best distillation source without checking student-side downstream fit.

2. The quality / diversity / complexity axes

DEITA arXiv:2312.15685 is the paper that names this precisely, and its most load-bearing finding for your case isn’t the score itself — it’s the failure mode of naive scoring: ranking candidates by quality/complexity alone and taking the top-k reproduces whatever pattern already dominates the pool. If 80% of your run logs are nmap-then-curl trajectories, a pure quality-score top-k selection over that pool gives you a curated set that is still 80% nmap-then-curl — high-scoring rows cluster around the harness’s existing skew unless an explicit diversity/coverage constraint forces otherwise. This is the exact mechanism behind “the model reaches for one salient tool, uses default args”: your existing SFT/rollout pool has that skew, and score-only selection amplifies rather than corrects it.

InsTag arXiv:2308.07074 operationalizes diversity and complexity as two independently measurable axes via open-set LLM tagging: diversity = number of unique tags present, complexity = tags-per-example, both correlating with SFT quality holding dataset size fixed. This is the tagging machinery §7’s synthesis-conditioning and §8’s coverage audit both reuse.

The concrete fix for the “one salient tool” failure, combining §1’s methods with the diversity constraint:

1. Score the pool by quality/informativeness (AlpaGasus or IFD).
2. Walk the score-ranked list; keep a candidate only if its (tool_name, arg_pattern)-distance (or embedding distance, if you don’t have a structured tool signature) to every already-kept row exceeds a threshold τ.
3. This — DEITA’s diversity-dedup applied as a hard constraint on top of, not instead of, quality ranking — is the mechanism that specifically prevents top-k selection from reproducing the prior-collapse it’s supposed to fix.

Note what this does and doesn’t do: it makes your existing good amass rows survive selection instead of being crowded out by a thousand near-duplicate nmap rows. It does not create amass rows that don’t exist in the pool at all — that’s a synthesis problem (§7) or a coverage-gap problem (§8), not a selection problem. Selection can only re-weight what’s already there.

3. Hard-negative mining for preference pairs

This is the P-gap machinery: constructing (chosen, rejected) pairs so that DPO/KTO actually teaches the right ranking, rather than a pair so trivial the gradient vanishes or so extreme it teaches a shortcut.

The lineage, applied

ANCE’s founding result arXiv:2007.00808 — random or in-batch negatives give a vanishing gradient once a model is even moderately good, because the model already ranks them far below the positive; a hard negative has to come from the model’s own current top-ranked wrong candidates, mined globally, not sampled at random. RSO arXiv:2309.06657 ports this to preference optimization directly: both DPO’s static pairs and SLiC’s SFT-policy-only pairs diverge from the maximum-likelihood target, and rejection-sampling toward the estimated optimal policy before pairing closes that gap. DPO itself arXiv:2305.18290 establishes the (prompt, chosen, rejected) triplet objective — but the load-bearing finding for curation, not algorithm choice, is Ivison et al.’s systematic ablation arXiv:2406.09279: across {preference data, algorithm, reward model, training prompts}, preference-data quality dominates outcome variance more than DPO-vs-PPO-vs-KTO algorithm choice. Put engineering budget in the mining pipeline below, not in loss-function tuning.

The worked pipeline, for (shallow-tool ≺ expert-tool)

flowchart TD
  A["Roll out K=4-8 samples per task,<br/>current checkpoint, T≈0.7-1.0"] --> B["Decompose at the tool-call<br/>DECISION POINT, not whole trajectory<br/>(ANCE/RSO principle)"]
  B --> C["Verify each candidate:<br/>tool-exists · arg-schema-valid ·<br/>arg-semantically-correct<br/>(structured rubric, not judge alone)"]
  C --> D["Mine the hard negative =<br/>highest-rubric-scoring WRONG<br/>action at that decision point"]
  D --> E["Reject near-zero-probability<br/>candidates — vanishing gradient<br/>just like random negatives"]
  E --> F["Construct the MINIMAL PAIR:<br/>anchor at the tool-call boundary,<br/>continue BOTH branches with the<br/>POLICY'S OWN rollout"]
  F --> G{"DIAGNOSTIC GATE:<br/>does the policy complete the<br/>'chosen' branch at a non-trivial<br/>rate (mini pass@k)?"}
  G -->|No| H["K-GAP masquerading as a<br/>ranking problem —<br/>route to §7/knowledge-curation,<br/>NOT DPO"]
  G -->|Yes| I["CHES similarity check<br/>(chosen vs rejected embeddings)"]
  I -->|Too similar| J["Discard/diversify —<br/>likelihood-displacement risk"]
  I -->|OK| K["Train; monitor chosen<br/>log-prob directly, not<br/>just the margin"]

Concretely, at the amass-vs-nmap decision point (recon stage, target has a large unenumerated subdomain surface):

// Anchor: same prefix, same decision point. Two branches diverge here.
{
  "prompt_prefix": [
    {"role": "system", "content": "Tools: nmap, amass, curl, submit_flag."},
    {"role": "user", "content": "Target: corp-app.example.com. Find the flag."}
  ],
  "chosen": {
    "tool_calls": [{"function": {"name": "amass", "arguments": "{\"domain\": \"example.com\", \"mode\": \"passive\"}"}}],
    "rationale_for_chosen": "Broad passive enumeration before touching any single host — surfaces staging/internal subdomains nmap alone won't find."
  },
  "rejected": {
    "tool_calls": [{"function": {"name": "nmap", "arguments": "{\"target\": \"corp-app.example.com\", \"ports\": \"1-1000\"}"}}],
    "why_this_is_the_hard_negative": "Not a strawman — nmap is a HIGH-rubric-scoring, plausible, frequently-correct action; it's wrong here specifically because it skips subdomain enumeration on a target with a large unenumerated surface. That's what makes it a hard negative, not a random one."
  }
}

The diagnostic gate (Verified Critical Step Optimization arXiv:2602.03412) is the step people skip and shouldn’t: force the current policy to continue executing from the chosen branch and check it succeeds at a non-trivial rate. If the policy can’t complete the amass-first path even when handed it, that’s not a ranking problem — the model doesn’t know how to use amass effectively (a K-gap), and training DPO on this pair risks amplifying an unfixed knowledge gap rather than fixing a preference ranking. Route to synthesis/knowledge-injection (§7, Knowledge-gap curation) instead.

The ceiling: likelihood displacement

arXiv:2410.08847 (Razin, Malladi, Bhaskar, Chen, Arora, Hanin — Princeton, ICLR 2025) is the hard ceiling on “how minimal can a mined pair be.” A chosen/rejected pair that is too embedding-similar can catastrophically push probability mass onto a third, unintended, semantically-distant completion, rather than genuinely re-ranking chosen above rejected. Diagnosed via a CHES (centered-hidden-embedding-similarity) check between the pair before training; monitored during training via the chosen-response log-probability directly, not just the chosen-minus-rejected margin — a shrinking margin can hide a chosen log-prob that’s also silently falling.

Contested, and worth A/B-testing before you build on-policy mining infra: is on-policy negative sampling (the ANCE/RSO prescription above) always better than static negatives? A 2026 ICLR poster (OpenReview tz9mJmgrdM) found it ranges from 3× better to 0.4× worse depending on model/task, with the flip-sign mechanism uncharacterized. Don’t assume on-policy is automatically correct for your setup — verify on your own model before investing in the rollout infrastructure the pipeline above needs.

4. Active-learning / uncertainty-guided acquisition

Selection (§1) filters a pool you already have. Active learning decides which new example to generate/label next, targeting the diagnosed gap directly instead of hoping a fixed pool happens to contain it. Classic roots: core-set arXiv:1708.00489 (diversity over per-example uncertainty in batch settings), BALD arXiv:1703.02910 (MC-dropout mutual information as an uncertainty proxy).

The crucial LLM-era fix — semantic, not token, entropy. Raw token-level entropy is dominated by paraphrase/lexical variation, not by genuine uncertainty about the underlying fact or decision. Semantic entropy arXiv:2302.09664 (Kuhn, Gal & Farquhar) fixes this: sample k completions, cluster them by bidirectional NLI entailment, compute entropy over the cluster distribution rather than the raw token distribution. This is your direct K-vs-R distinguishing instrument.

Active Instruction Tuning arXiv:2311.00288 operationalizes query-by-committee via prompt-paraphrase disagreement instead of a model ensemble — no second model needed. Runnable today, directly on your tool-choice problem:

1. Take the same scenario (target with a large unenumerated subdomain surface), generate 3–5 paraphrases varying phrasing, not the underlying task.
2. Run the current checkpoint on each paraphrase at T≈0.7–1.0.
3. Measure the tool/argument-choice flip-rate across paraphrases.

RLVR dynamic sampling — active acquisition inside the training loop

On the RL side, the single most-replicated 2025–2026 finding — PCL arXiv:2510.01135, AdaRFT arXiv:2504.05520, SEC (Self-Evolving Curriculum) arXiv:2505.14970, DAPO’s Dynamic Sampling arXiv:2503.14476, and VCRL (OpenReview FBhWTuMTYA) — is that training signal is maximized at intermediate difficulty: pass-rate ≈ 50%, equivalently maximal reward-variance across a rollout group. This is free: the k rollouts GRPO already generates give you pass_rate and reward_std with zero extra inference.

# free — reuses rollouts you already generate
pass_rate = mean(reward for _ in rollouts)          # per prompt, across k samples
reward_std = std(reward for _ in rollouts)
if reward_std ≈ 0:                                   # all-correct or all-wrong: zero advantage signal
    drop_and_resample(prompt)                        # DAPO's Dynamic Sampling, arXiv:2503.14476
oversample(prompts_with(pass_rate in [0.2, 0.6]))    # this is the literature-side justification
                                                       # for keeping a GRPO baseline in the ~30-60% band

Because difficulty is policy-dependent and goes stale as the policy improves, PCL/SEC recompute it online via a learned value model or bandit rather than caching a once-computed label — a static difficulty score is wrong by definition after enough training steps.

Contested, two different decisions, not a contradiction: arXiv:2508.14094 finds that under a fixed data-acquisition budget (new problems to annotate/verify), prioritizing the hardest available examples gives the largest GRPO gains (up to 47%) — the opposite of the intermediate-difficulty rule above. The two rules answer different questions: what to collect (acquisition budget → go hard) vs. what to train on per training step from an already-large pool (batch composition → stay intermediate). Don’t apply one where the other belongs.

5. Decontamination for eval integrity

This is a load-bearing confound on every method above: a curation or synthesis pipeline that accidentally leaks your held-out eval into the training pool doesn’t fail loudly — it inflates the training-set score without inflating real capability, and every diagnostic in this book that reads “the fix worked” is only as trustworthy as this gate.

Stage 1 — cheap, mandatory baseline. N-gram overlap: a token counts as contaminated if it sits in a shared run of more than ~10 tokens between an eval sample and the training set — this is the methodology the Llama 2 report uses (§A.6) and is a standard, reproducible first pass. Pair with exact-substring/suffix-array and MinHash near-duplicate detection across the full corpus, not just the eval set — deduplication of the training corpus itself measurably improves downstream models independent of any eval-leakage concern arXiv:2107.06499.

Stage 2 — mandatory if any rephrase/persona/backtranslate step touched source material (§7 does, by construction). N-gram-only decontamination is trivially bypassed by paraphrase arXiv:2311.04850 (the LLM Decontaminator paper) — exactly the kind of rephrasing your synthesis pipeline does on purpose. An embedding-similarity flag (cosine >~0.80–0.85) plus an LLM-judge confirmation pass (“is this a paraphrase/restatement such that the answer is directly inferable?”) is required on top of, not instead of, n-gram/MinHash dedup.

Stage 3 — RL-specific, and the reason this section isn’t just an SFT concern. A single very recent controlled study arXiv:2601.06103 (Jan 2026, small models 0.5B–4B, not yet independently replicated) found GRPO-style RL on clean data generalizes leaked pretraining contamination to uncontaminated same-family items, whereas SFT only re-inflates the literal contaminated rows. A clean RL environment sitting on a contaminated base model can silently look like genuine capability gain. Before trusting a GRPO/RLVR pass@k jump as real, decontaminate the reward/eval set against the base model’s own pretraining corpus, not just your own synthetic pipeline’s outputs.

Contested — does contamination matter at frontier scale at all? Bordt et al. arXiv:2410.03249 found moderate repeated contamination is effectively “forgotten” via weight decay once pretraining exceeds ~5× Chinchilla-optimal tokens. This describes large, diluted, public benchmarks — it does not license skipping decontamination on a small, high-value held-out set (a handful of CTF flags, a curated eval trace), which is exactly the regime where memorization risk is highest, not lowest.

6. Curriculum / difficulty-ordering

Curriculum learning’s classic root — Bengio et al., ICML 2009 (no arXiv id) — orders/weights training examples by a computable “easiness.” Self-paced learning, Kumar/Packer/Koller, NeurIPS 2010 (no arXiv id), sharpens this: the model itself sets its own pace rather than an external teacher fixing an order in advance. Nearly every RL-curriculum paper in §4’s “RLVR dynamic sampling” subsection (DAPO arXiv:2503.14476, PCL arXiv:2510.01135, AdaRFT arXiv:2504.05520, SEC arXiv:2505.14970, VCRL) is this same self-paced-learning idea, re-derived: the pass-rate/reward-variance signal is the model’s own, computed online, exactly because a static curriculum staled the moment the policy moved past it.

For offline SFT curricula specifically (as opposed to online RL batch composition), order training rows by IFD (§1) or by difficulty-as-judged, low-to-high, on the theory that early-training gradient steps on rows the model already half-knows are more stable than starting on the hardest rows cold. This is weaker evidence than the RL-side finding and is genuinely contested for generative tasks specifically:

Contested — difficulty vs. coverage, for generative fine-tuning. Dataset Cartography’s original framing arXiv:2009.10795 (§8) treats “ambiguous/hard-and-persistently-wrong” as the interesting region for classification-era tasks. A 2025–2026 workshop paper (OpenReview g1DiK2Yi4j) argues that for generative SFT/agent-trajectory tasks specifically, difficulty-based selection narrows output-distribution coverage and underperforms random selection — not yet reconciled with cartography’s classification-era prescription. If your curated set is agentic-trajectory-shaped (it is), don’t import a pure difficulty-ordering rule from classification-era curriculum literature without checking it against a coverage metric (§8) first.

Net practical rule: online RL batch composition → intermediate difficulty, recomputed continuously (§4). Offline SFT-row ordering → don’t lean on difficulty alone; check coverage isn’t collapsing (§2, §8) at the same time.

7. Synthetic-data generation mechanics

Selection (§1–2) and negative-mining (§3) work on data you already have. This section is how you produce rows to fill a gap those methods can’t fill by re-weighting alone — the mechanics behind the corpus you’d curate, not a replacement for curation.

The lineage, in order:

• Self-Instruct arXiv:2212.10560 — bootstrap instruction/I-O triples from ~175 seeds via in-context generation, ROUGE-L<0.7 dedup as the core diversity control.
• Evol-Instruct / WizardLM arXiv:2304.12244 — LLM-driven in-depth/in-breadth rewrite operators escalate instruction complexity beyond what’s cheap to hand-author.
• InsTag arXiv:2308.07074 — the tagging machinery from §2, reused here to condition generation toward under-tagged regions, not just to measure a finished pool.
• Instruction Backtranslation arXiv:2308.06259 — predict the instruction a piece of real web/document text answers (text→instruction direction, not the reverse), then self-curate. Grounds every synthesized row in genuine text — the same “grounded in reality, not imagined” principle §1’s LIMA and this whole chapter’s K-gap concerns rest on.
• WRAP arXiv:2401.16380 — rephrase noisy source docs into a small, fixed set of styles (3–5: Wikipedia-like, Q&A, ELI5), not open-ended paraphrase volume. A small fixed style-set, empirically, beats unbounded paraphrase generation.
• Persona Hub arXiv:2406.20094 — 1B mined personas; injecting a persona into a generation prompt biases sampling toward that persona’s knowledge slice, explicitly demonstrated for synthesizing tools/functions at scale in the source paper.
• Magpie arXiv:2406.08464 — truncate the chat template right before the user turn, let the aligned model autoregress its own query. Cheapest possible query generator, zero seeds needed — but it faithfully reproduces whatever skew already exists in the policy, so it diagnoses prior skew (it’s a good coverage-audit probe, §8) but cannot fix it by construction.

The R-gap fix, concretely: force coverage of amass without fabricating a tool result

1. Do not self-sample (Magpie-style) for this — by definition it reproduces the existing nmap-first skew.
2. Build/reuse a tool taxonomy independent of your current data (InsTag-style tagging over your tool documentation, not over what’s already in your run logs — critical, see §8).
3. Condition synthesis on personas/scenarios that structurally require the underused tool (Persona Hub): “a security researcher who always starts with passive OSINT before touching a single host” → generates a scenario where the winning first move is amass, not nmap.
4. This synthesizes the scenario/query, not the tool result. Cross-reference kinds-of-sft.md §4: fabricating what amass’s output looks like teaches confabulation exactly like any other ungrounded tool result. Run the synthesized scenario for real, against a real target, and keep the executed transcript.
5. Verify with the same three-stage contract ToolACE arXiv:2409.00920 and APIGen-MT arXiv:2504.03601 both independently converge on: format-valid → executes against a real/sandboxed API → LLM-judge/committee semantic-match.
6. Apply DEITA’s diversity-dedup (§2) on the resulting set so it doesn’t just become a second, smaller monoculture.

A very recent inversion worth knowing about, not yet broadly validated: Firefly arXiv:2605.17558 starts from real API outputs and works backward to the query — the opposite direction from ToolACE/APIGen-MT — specifically to fix synthetic-schema drift (where a model-generated tool call is plausible-looking but doesn’t match the real tool’s actual schema/output shape). Same grounding principle as Instruction Backtranslation, applied to tool calls instead of documents.

8. Coverage-gap detection — proving the hole exists

This is the closest thing in the literature to a direct methodological answer to “how do I know what’s missing” — and it’s the step that turns “I suspect we never train on amass” into a defensible, falsifiable claim.

Dataset Cartography arXiv:2009.10795 (Swayamdipta et al.) plots every training example on (confidence, variability-across-epochs) from a single training run — no held-out taxonomy needed to run it — exposing easy / ambiguous / hard-and-persistently-wrong regions. On its own this tells you about examples you have; it says nothing about examples you don’t.

The step that actually finds the hole: cross-reference the hard-and-never-improving bucket against an independent skill taxonomy — built first, from your tool documentation or --help output, via InsTag-style LLM self-tagging arXiv:2308.07074 or metacognitive labeling arXiv:2405.12205 — not derived from what’s already in your run logs, or you’ll define your skill taxonomy in a way that can’t see the hole it’s supposed to find.

Worked example — detecting the amass/passive-recon hole:

1. Build the taxonomy from your tool surface, independent of usage: {nmap, amass, curl, sqlmap, http_get, ...} × {active-scan, passive-recon, exploit, exfil, ...}.
2. Count raw mentions of each tag combination across your run-log corpus — this is the Kandpal-style mention-count check arXiv:2211.08411, which established a causal relationship between corpus mention count and answerability for facts; the same logic applies directly to tool-usage rows.
3. amass × passive-recon appears 0 times across the corpus. That’s not “the model happens to underperform on this” — it’s a directly measurable, confirmed coverage hole, distinguishable from a merely-rare-but-present pattern.
4. Cross-check against capacity, not just presence: Allen-Zhu & Li’s knowledge-capacity scaling laws arXiv:2404.05405 (~2 bits of knowledge per parameter, in controlled synthetic setups, degraded by junk-data dilution) tell you that even after you inject amass coverage, if it’s a vanishingly small token-fraction of a much larger/noisier mix, the effective capacity a given model size allocates to it shrinks — track domain-token-fraction, not just raw row count, once you’ve closed the zero-coverage gap.

Confirming the hole is real, not an artifact of your training pipeline (two independent per-example diagnostics):

• Gekhman et al. arXiv:2405.05904 (Technion/Google): SFT rows containing facts/skills genuinely unknown to the base model, flagged by a pre-SFT in-context-oracle probe, are fit more slowly and, once fit, linearly increase hallucination on unrelated held-out inputs. If you push amass rows into SFT and unrelated tool-choice accuracy gets worse, that’s this mechanism confirming the pre-injection gap was real and structural, not incidental.
• The Incomplete Learning Phenomenon arXiv:2604.10079 (UNSW/Tencent, April 2026, single paper, not yet broadly replicated) shows even converged SFT models fail to reproduce a persistent subset of their own training data, with “missing prerequisite knowledge in the base model” as one of five named, empirically-distinguishable causes (alongside SFT-data inconsistency, sequential-finetuning forgetting, and under-optimized rare patterns) — a per-example diagnostic, not just an aggregate score, for confirming why a specific amass row didn’t stick even after training on it.

Closing the loop end-to-end: STAT arXiv:2510.10023 (Princeton) operationalizes exactly this cycle — tag training data by skill, train, probe the trained model per-skill-tag to build a Missing-Skill-Profile from its own post-training failures, then reweight or synthesize specifically for the failing tags. Reported as complementary to GRPO/RL, not a substitute for it — run STAT’s probe after a training round, not instead of one.

9. Apply these in what order

None of §1–8 is a default pipeline you run start to finish every time — pick the subset the diagnosis calls for. But when you are building a curated/mined set, apply them in this order:

1. Coverage-gap detection first (§8). Before selecting or mining anything, confirm what’s actually missing (tag corpus against an independent taxonomy, check mention counts). Selection and negative-mining can only re-weight what already exists — running them on a pool with a zero-coverage hole just curates around the hole more efficiently.
2. If the gap is confirmed missing (not just rare): synthesize (§7), grounded — condition on personas/taxonomy gaps for scenario diversity, execute for real, verify with the three-stage contract. Do not self-sample (Magpie-style) to fill a confirmed hole; it reproduces the hole by construction.
3. Verifier-filter, always, before anything else touches the pool — this alone removes the majority of harmful noise (per kinds-of-sft §6).
4. Scale-gate your selection method (§1). Target set ≤~50–100K rows (the realistic regime for a targeted gap-patch): run the full quality→difficulty→diversity→targeted-gradient stack (AlpaGasus → IFD → DEITA → LESS/BIDS). Target set 100K+ rows: skip straight to representation-similarity (RDS+-style) or well-stratified random, and validate anything fancier against a compute-matched random baseline before trusting an uplift.
5. Apply the diversity constraint (§2) as a hard filter, not an afterthought, regardless of which selection method above you used — this is what prevents re-collapsing onto the existing skew.
6. If the diagnosed gap is a ranking problem (R/P-gap), mine hard negatives (§3) from the selected, verified pool — on-policy, decision-point-anchored, gated by the K-vs-P diagnostic, checked for likelihood displacement before training.
7. If acquiring genuinely new examples under a fixed budget, go hard (§4’s acquisition rule). If composing an already-large pool into RL batches, stay intermediate-difficulty, recomputed online, not cached.
8. Decontaminate against your eval set before training, every time (§5) — n-gram baseline always, embedding+judge if any synthesis/rephrase step touched source material, base-model-pretraining-corpus check if the downstream training is RL, not just SFT.

flowchart TD
  Start["Diagnosed gap<br/>(K / R / P — see decision.md)"] --> Cov["§8 Coverage-gap detection:<br/>is it actually MISSING,<br/>or just under-weighted?"]
  Cov -->|"Missing (0 mentions)"| Syn["§7 Synthesize, grounded —<br/>persona/taxonomy-conditioned<br/>scenario + REAL execution"]
  Cov -->|"Present but rare/skewed"| Sel["§1-2 Selection + diversity constraint<br/>on the existing pool"]
  Syn --> Ver["Verifier-filter (mandatory)"]
  Sel --> Ver
  Ver --> Gate{"Gap type?"}
  Gate -->|"K — inject off-policy"| Done1["SFT on the synthesized/selected set"]
  Gate -->|"R/P — mis-ranked"| Neg["§3 Hard-negative mining,<br/>on-policy, decision-point-anchored"]
  Neg --> LD["Likelihood-displacement check<br/>before training (CHES + chosen log-prob)"]
  LD --> Done2["DPO/KTO on the mined pairs"]
  Done1 --> Decon["§5 Decontaminate against eval —<br/>always, before trusting any resulting number"]
  Done2 --> Decon

  classDef your fill:#132b22,stroke:#34d399,color:#eafaf3;
  class Cov,Neg,LD your;

Cross-links

• The kinds of SFT §6 — the SFT-row-level version of §1’s selection stack (LIMA/AlpaGasus/IFD/LESS/DEITA), plus the forgetting/replay side this chapter doesn’t repeat.
• Method → Data — which training method (SFT/DPO/KTO/GRPO) consumes which data object; this chapter is the mechanics behind producing/filtering that object, not the method-selection question itself.
• Post-training dataset registry — concrete, proven-by-usage downloadable datasets if you need a general-capability ingredient (tool-calling format anchors, preference mixes) rather than building your own from run logs.
• Diagnosing the gap — the instrumentation (pass@k, Pass@(k,T), Cover@τ, elicitation ladder) that tells you which gap you’re curating for before you pick a method from this chapter.
• The decision — the K/R/P routing tree this chapter’s methods feed into.
• Knowledge-gap curation — the confirmed-K-gap corpus recipe end to end (deferred from §7/§8 here).
• Intervention per gap — which fix routes to which gap type, in full.
• Preference — RLHF · DPO · KTO — the loss mechanics §3’s mined pairs feed into.

Bibliography

Confidence calibration: every arXiv id above is verified live per the source ledger (artifacts/three-gap-survey/ledger-F.md, pass 2026-07-02), not recalled from training memory. High confidence on the genealogy claim (curriculum/active-learning/hard-negative-mining as the three roots) and on the core mechanics of each named method — these are widely-cited, independently-corroborated findings. Explicitly flagged single-paper or very-recent results (Verified Critical Step Optimization, the RL-contamination-generalization study, the Incomplete Learning Phenomenon, Firefly) are marked “promising, not yet validated” throughout and should not be treated as settled. Three genuine contested edges are called out inline rather than resolved: selection-vs-random at production scale, on-policy-vs-static negatives for DPO, and hard-vs-intermediate difficulty for acquisition-vs-batch-composition — the field has not published a reconciliation for any of the three, and this chapter deliberately does not manufacture one.