ThinkingCap: Reducing LLM Reasoning Tokens by 58%

The Breakthrough

ThinkingCap, a fine-tuned variant of the Qwen3.6-27B model, demonstrates that LLMs can achieve equivalent reasoning performance while significantly reducing compute costs by learning to terminate reasoning processes as soon as a conclusion is reached.

What Actually Worked

• The developers trained the model on a curated dataset spanning multiple domains, explicitly rewarding efficiency alongside correctness to discourage the model from generating redundant reasoning tokens.
• By penalizing verbosity during the reasoning phase, the model learned to identify the point of convergence, effectively eliminating "looping" behaviors where models rephrase the same logic repeatedly.
• The team inadvertently discovered that applying this compression to the final answer output as well as the reasoning trace resulted in more concise, human-preferred responses, leading them to ship this behavior as a feature.

Before / After

• Reasoning token reduction: Average of 45.8% across out-of-domain benchmarks, reaching up to 58% on training-domain benchmarks.
• Accuracy: Remained essentially unchanged, with a slight improvement on the GSM8K benchmark from 93.3% to 96.5%.
• Latency: In a live test calculating the smallest integer n for 100 trailing zeros in n!, the base Qwen3.6-27B model took 140 seconds and over 7,000 reasoning tokens, while ThinkingCap completed the task in significantly less time using under 2,000 total tokens.

Context

Modern reasoning models, such as OpenAI's o1 and DeepSeek R1, often suffer from "overthinking" where they continue to generate reasoning tokens even after a problem is solved. This creates unnecessary latency and cost. BottleCap AI addressed this by fine-tuning the Qwen3.6-27B base model to prioritize efficient reasoning. The result is a drop-in replacement that maintains high accuracy while drastically reducing the token expenditure required for complex tasks.