Training Zeta2: Edit Prediction via Distillation and Settled Data
AI Engineergo watch the original →
the gist
Zed trains its Zeta2 edit prediction model by distilling frontier model outputs, using a repair step for bad predictions, and filtering training examples by comparing model outputs against user-settled code states.
The Distillation and Repair Pipeline
Zed generates training data by capturing opt-in production edit traces and distilling them through a frontier teacher model. Because frontier models can produce inconsistent outputs for the same input, Zed implements a two-stage process to ensure quality. First, the system runs static evaluations to check for heuristics like boundary violations or immediate reversals, where the model undoes the user's last keystroke. If a prediction fails these checks, it is routed to a second frontier model in a repair step to correct the output. This refined output is then formatted into JSONL files, which serve as the base for training student models like Zeta2.
Filtering with Settled Data
To improve training quality, Zed uses "settled data," which is the final code state after a user stops editing a specific region for 10 seconds. Because user intent can change or be overridden by other agents, raw settled data is noisy. To filter this, the team originally generated 10 teacher predictions per example and measured the Levenshtein distance to the settled state. To reduce the cost of this process, they now use the student model itself to generate 50 predictions, which is computationally inexpensive once the student reaches near-teacher quality. Examples that fall into the middle of the Levenshtein distance distribution are prioritized for training, as they represent non-obvious, useful edits that often fall outside the student model's original training cutoff.
Evaluation and Deployment
Zed evaluates models using a held-out test set and tracks metrics including the reversal ratio and kept rate, which measures the character overlap between the prediction and the final settled state. The team also monitors diagnostic error counts by snapshotting the number of errors before and after a prediction. Once a model is ready, it is deployed via a dashboard that allows for incremental traffic sampling, starting at 15% before a full rollout.