VibeThinker-3B: Reasoning via Verifiable Reinforcement Learning

The Breakthrough

VibeThinker-3B, a 3B parameter model based on Qwen-2.5-3B, achieves reasoning performance on math and coding benchmarks comparable to significantly larger proprietary models by utilizing a specialized post-training recipe that emphasizes verifiable reasoning over broad knowledge storage.

What Actually Worked

• Spectrum-to-Signal Training: The team generated synthetic data with diverse solution strategies (the spectrum) and used reinforcement learning to amplify correct reasoning paths (the signal).
• Two-Stage Curriculum: Stage one focused on broad coverage across STEM and chat, while stage two involved retraining exclusively on difficult, long-horizon problems.
• Reasoning Trace Filtering: The training process discarded any reasoning traces shorter than 5,000 tokens and filtered out easy problems to force the model to develop deep, multi-step reasoning capabilities.
• Multi-Domain RL (MGPO): The model uses a variation of Guided Policy Optimization (GPO) to weight training examples, avoiding both overly simple tasks and problems exceeding the model's current capability level.
• Test-Time Compute (CLR): The model employs Claim Level Reliability (CLR), a technique where multiple answers are generated and sampled to identify the most likely correct response, significantly boosting benchmark scores.

Context

The authors propose that intelligence in verifiable domains, such as math and code, relies on search and constraint satisfaction rather than the broad factual memorization required by general-purpose models. By offloading knowledge-heavy tasks and focusing on reasoning engines, the researchers aim to prove that smaller models can achieve high-level performance in specific domains. While VibeThinker-3B excels at long-horizon reasoning, it lacks the general knowledge and flexibility of larger models, often struggling with creative tasks or design-heavy prompts like SVG generation.