Researchers Automate LLM Reasoning Strategy Design, Cutting Token Usage by 69.5%

Test-time scaling (TTS) is a method that enhances the performance of large language models (LLMs) by leveraging additional computational resources during inference. However, TTS strategies have traditionally been designed manually, limiting their efficiency and requiring significant time investment from engineers. The new AutoTTS framework automates this process, transforming strategy design into an algorithmic search task.

At the core of AutoTTS is the idea of delegating strategy design to a specialized LLM agent, such as Claude Code. The agent operates within a predefined environment, where search boundaries, optimization goals (balancing accuracy and cost), and feedback mechanisms are established. Instead of manually creating branching rules or pruning unpromising reasoning paths, the agent independently tests and refines controllers—algorithms that manage computational resource allocation.

A key advantage of AutoTTS is its use of an offline environment for strategy testing. Rather than repeatedly running the base model to generate tokens, the agent analyzes pre-collected reasoning trajectories with intermediate signals. This significantly reduces the cost of finding the optimal strategy: the entire process took 160 minutes and cost $39.90. The result was the Confidence Momentum Controller, which employs non-obvious mechanisms such as tracking confidence trends instead of instantaneous values and dynamically distributing resources across reasoning branches.

During testing on models like Qwen3 (ranging from 0.6B to 8B parameters) and DeepSeek-R1, AutoTTS demonstrated impressive results. In resource-saving mode, the framework reduced token consumption by 69.5% compared to the baseline Self-Consistency method while maintaining accuracy. In high-budget mode, AutoTTS outperformed all manual strategies in five out of eight tests. On the GPQA-Diamond benchmark, token usage was reduced from 510K to 151K with a slight increase in accuracy. The solution is already available on GitHub and can serve as a ready-to-use replacement for existing TTS controllers.