Proven post-training datasets — a usage-cited registry

Every other chapter in this book asks “which method” (The family map) or “which sequence of stages” (The recipe is a sequence, not a pick). This chapter answers the question those chapters leave open once you’ve picked Sequence B (§2 of the recipe chapter): which concrete, downloadable dataset actually fills each rung, for the six GENERAL (non-cyber) capabilities every Sequence-B rung needs regardless of what cyber-specific data you build on top — chat template, tool/function-calling, instruction following, preference, reasoning, and willingness/refusal calibration. This is a registry, not an essay: the tables are the payload.

1. The inclusion rule (read this before trusting any row)

This project’s permanent stance is never rely on academic cybersecurity-LLM projects — grounding comes from real frontier/open-weight recipes, not single-paper academic artifacts (frontier-cyber-model-path.md §1). This chapter applies the exact dataset-side analog of that stance:

A dataset gets a row ONLY if it is PROVEN-BY-USAGE — i.e. a named, real, released open-weight model or recipe (a tech report, an official model card, or a widely-used/well-cited community fine-tune) is documented as having actually trained on it. Every row below names that model/recipe and cites the source (model card, tech report, or lab blog post) that states the link directly — not inferred, not “this would probably be a good fit.”

Consequences enforced throughout:

• No single-paper-only academic datasets. A dataset whose only appearance is its own authors’ academic ablation, with no third-party or even self-reported shipped model consuming it, is dropped. Each category file below has an explicit “Dropped” list — read it before re-proposing something that already failed the bar.
• No cyber-specific datasets at all — this registry is scoped to GENERAL capabilities on purpose; cyber-specific data is out of scope here by design, not merely unfound.
• Self-proof is weaker than cross-lab proof, and is flagged as such. A dataset proven only via its own authors’ reference model (e.g. Magpie-Reasoning-150K → the Magpie team’s own checkpoints) is marked Medium confidence even though the usage is real; a dataset reused by an independent lab or absorbed into a bigger shipped mixture (e.g. COIG-CQIA → folded into BAAI’s Infinity-Instruct recipe) earns High.
• License ≠ proof-of-usage. Several rows below carry a real, verified training usage but a murky or non-commercial license (ShareGPT, ToolACE, COIG-CQIA, toxic-dpo-v0.2). The PROVEN_IN column tells you it was used; the License column tells you separately whether you can reuse it — don’t conflate them.

Every table row format is: dataset · link · capability · Sequence-B stage · size · license · PROVEN_IN (model/recipe) + citation · language(s) · notes · confidence. Source notes for all six categories, including live-verification method and the full per-dataset detail, are the six research files under artifacts/overnight-datasets/research/ (instruction-chat, tool-calling-agentic, preference-alignment, reasoning-cot, willingness-uncensor-safety, chinese-labs-multilingual) — this chapter distills them; consult those files for the complete citation quotes.

2. Instruction / chat SFT

The CPT→SFT rung: general instruction-following and multi-turn chat behavior — the base every other capability in this chapter sits on top of.

Dropped: raw LMSYS-Chat-1M as a standalone primary SFT set (no lab found training directly+solely on the 1M dump for general chat SFT); raw Magpie-Align dumps as a standalone commercial-usable set (kept only the technique + the Apache-licensed Smol-Magpie-Ultra derivative).

3. Tool / function-calling & agentic trajectories

The SFT rung that teaches the model to use things — required regardless of cyber content, since the CTF agent’s core skill is calling tools correctly across multi-turn trajectories.

Dropped: NexusRaven-V2 training data (only an eval set was released publicly); watt-tool-8B/70B (proprietary/undisclosed dataset, no citable artifact); a standalone Qwen tool-calling SFT row (Qwen never published one — its format adoption of Hermes-style tool use is credited to the Hermes row above instead).

Cross-cutting for Sequence-B: prioritize the multi-turn/agentic sets (ToolACE, APIGen-MT, Agent-FLAN, ToolBench) over single-call sets (xLAM-60k, APIBench, Glaive) — CTF tool use is inherently multi-step and stateful. Agent-FLAN’s explicit negative/anti-hallucination samples are the one artifact in this table that doubles as willingness/refusal-adjacent signal (see §7).

4. Preference (DPO / KTO / RLHF)

Dropped: no cyber-specific preference dataset was in scope; any preference set whose only usage was a single unrelated academic ablation was excluded before it reached the table.

5. Reasoning / CoT (math + code + science distillation)

Dropped: the raw DeepSeek-R1 “800k samples” SFT set was never released as a standalone artifact (only checkpoints were open-sourced) — the community reconstructions above are what’s actually usable and are what’s listed instead; AM-DeepSeek-R1-Distilled-1.4M (no verified third-party adoption beyond its own paper); any cyber-specific reasoning/CTF-reasoning dataset (out of scope by permanent stance).

Read order if you’re pulling data, not just reading the table: NuminaMath-CoT (substrate) → OpenThoughts3-1.2M (best current fully-open ablated set) → OpenMathReasoning (if you need competition-tier math) → OpenCodeReasoning (closest analog to CTF/exploit-reasoning) → Mixture-of-Thoughts/OpenR1-Math-220k (if reproducibility of the training recipe matters more than raw SOTA) → Bespoke-Stratos/Sky-T1 (cheapest pilot).

6. Willingness / uncensor vs. safety / refusal calibration

This category is both directions of refusal/compliance behavior — deliberately, because the target model is a sandboxed offensive-security research agent and over-refusal is a real, documented failure mode (see §8).

6a. Compliance-increasing (uncensor / de-refusal)

6b. Safety-increasing (harmlessness / appropriate refusal / anti-jailbreak / anti-over-refusal)

Dropped: Nous Hermes 3’s compliance-steerability behavior is real but no single named public dataset is credited for it (proprietary blend); Do-Not-Answer/XSTest/OR-Bench are refusal-rate eval sets, not training sets, in any published recipe verified; Llama Guard’s training-data composition is not public.

7. Chinese-labs & multilingual

Dropped: zhihu_rlhf_3k and dikw/hh_rlhf_cn (only scattered small/unlabeled community reward-model usage, no named shipped recipe); a standalone first-party Qwen/DeepSeek/GLM/Yi SFT-or-preference-data row (none of those labs release their actual training data — their footprint here is captured indirectly via AgentInstruct/LongAlign, and via community R1-distillation sets like Congliu’s 110k).

8. Which datasets at which Sequence-B rung

Mapped onto Sequence B’s actual stage order (frontier-recipe-is-a-sequence.md §2): base/instruct choice → (optional) CPT → SFT cold-start → rejection-sampling/on-policy SFT → preference (DPO/KTO) → RLVR/GRPO → iterate.

9. Two honest caveats

Willingness vs. over-refusal, for a sandboxed offensive-security agent. Over-refusal — declining a legitimate pentest/CTF request because it pattern-matches “harmful” — is a real, well-documented failure mode, which is exactly what CoCoNot and WildJailbreak were purpose-built to fix (contrastive harmful-vs-benign-but-scary-looking prompts, §6b). That is the correct tool for this project’s actual problem: precision on the refusal boundary inside a controlled, sandboxed tool-use context — not blanket compliance. The de-refusal-direction datasets in §6a (Dolphin, toxic-dpo-v0.2, wizard_vicuna_70k_unfiltered) are included because they are genuinely proven-by-usage and instructive as the opposite pole of the same axis, but they are a blunter instrument (strip refusal-flavored completions wholesale) than CoCoNot’s calibrated “refuse the actually-unsafe subset, comply with the rest.” Default recommendation: build the willingness rung primarily from the Tulu-3 safety_noncompliance bucket (WildGuardMix + WildJailbreak + CoCoNot together, exactly as Tulu 3 ships it), and treat §6a as a reference recipe pattern rather than a default ingredient.

The off-policy caveat for distilled reasoning data. Nearly every dataset in §5 is CoT distilled from a stronger teacher (DeepSeek-R1, QwQ-32B, Llama-3.1/3.3-70B-Instruct) — the policy model you’d actually train on Sequence B never generated these traces itself. This is the correct, cheap way to bootstrap a reasoning prior before any on-policy stage (exactly what Sky-T1/Bespoke-Stratos/OpenThoughts/OpenR1 all do), but pure SFT-on-distillation caps quality at the teacher’s and plateaus — it does not substitute for an on-policy RL stage afterward. This mirrors the same on/off-policy axis this book treats as the single most load-bearing distinction in post-training generally (foundations/on-off-policy.md) and the same caveat the preference registry (§4) makes explicitly about off-policy DPO mixes: budget an on-policy regeneration-and-rescoring pass against your own SFT checkpoint before treating either the reasoning prior or the preference stage as final, mirroring what Tulu 3 and NVIDIA’s HelpSteer2-Preference both do in practice.

Cross-links

• The recipe is a sequence, not a pick — the Sequence-A/Sequence-B stage skeleton this registry’s §8 mapping is built against.
• The path to a frontier cybersecurity model — the north star this registry serves; the “proven-by-usage, not academic-only” stance is the dataset-side mirror of that chapter’s model/recipe-side stance.
• Method → Data (your real bottleneck) — the method-first framing that explains why RLVR/GRPO in §8 deliberately has no fixed dataset row, and why rejection-sampling FT’s data object is a byproduct you already produce.
• Full per-dataset detail, live-verification method, and additional dropped candidates: artifacts/overnight-datasets/research/{instruction-chat,tool-calling-agentic,preference-alignment, reasoning-cot,willingness-uncensor-safety,chinese-labs-multilingual}.md.