Lab notebook Β· post-training data strategy

Synthetic data ate post-training.
Here’s how to use it.

Pick your data problem. Get the strategy β€” distilled from Nathan Lambert’s Lecture 7 on post-training.

Built from Nathan Lambert’s Lecture 7 announcement Β· watch the lecture Β· Interconnects

TEACHER big Β· slow Β· smart STUDENT small Β· fast Β· cheap on-policy data multi-teacher votes Hinton 2015 β†’ OPD Β· MOPD Β· OPSD
Fig. 1 β€” knowledge distillation, then and now: the teacher writes the answers, the student learns to write them alone.
Step 1

What’s your data problem?

Pick the one that hurts most right now.

Step 2

Two constraints

Scale first, then verifiability.

Strategy card