After R1 comes S1, a reasoning model trained with a $50 budget | Llm meaning in education | Generative ai examples | Best llm training dataset reddit | Turtles AI

A team of researchers from Stanford and the University of Washington has developed an AI model for reasoning, called s1, at a cost of less than $50. Using distillation from Google’s Gemini 2.0 Flash, the model demonstrated high capabilities in mathematics and coding, raising questions about the sustainability of innovation in the field.

Key points:

• Economic model: s1 was trained on a minimal budget, demonstrating that advanced reasoning capabilities can be replicated without large investments.
• Distillation technique: Researchers extracted cognitive capabilities from an advanced Google model, Gemini 2.0 Flash, using supervised fine-tuning.
• Surprising results: Despite the simplicity of the training, s1 achieved performance comparable to high-level models such as OpenAI’s o1 and DeepSeek’s R1.
• Ethical and strategic implications: The ease with which multimillion-dollar models can be replicated raises questions about competitiveness and intellectual property protection in the AI field.

In the AI research landscape, a recent study by researchers at Stanford and the University of Washington showed that it is possible to obtain a high-level AI reasoning model with minimal investment. The s1 model, developed with less than $50 in cloud computing credits, comes close in performance to advanced solutions such as OpenAI’s o1 and DeepSeek’s R1. The code, available on GitHub, represents a significant breakthrough in the field and an example of efficiency in model training. The key to s1’s success lies in its use of the distillation technique, a process in which a smaller model is trained to emulate the behavior of a more advanced one. In this case, the researchers leveraged Google’s Gemini 2.0 Flash Thinking Experimental, extracting its reasoning capability through a targeted dataset. Distillation has been used before to create low-cost AI models, as demonstrated by a Berkeley team that last month built a similar system for about $450. However, the s1 project takes this trend to the extreme, minimizing computational cost and questioning the need for billion-dollar investments to develop performant models. The training method was based on a dataset reduced to only 1,000 selected questions, each accompanied by the answer and thought process followed by Google’s advanced model. This approach achieved remarkable results, with training completed in less than 30 minutes using 16 Nvidia H100 GPUs. Niklas Muennighoff, one of the researchers involved in the project, said the cost of the computation needed today would be around $20. One particularly interesting detail concerns the inclusion of the word “wait” in the model’s reasoning process: this simple instruction helped improve the accuracy of the answers, suggesting that a longer processing time may benefit the quality of reasoning. The success of s1 has reignited the debate about the democratization of AI and the possibility of replicating complex models without unlimited resources. However, the issue clashes with the interests of large technology companies, which invest billions in training ever more advanced models. OpenAI has already accused DeepSeek of misusing its API data for distillation purposes, and Google itself, owner of Gemini, explicitly forbids the use of its models to create direct competitors. Legal issues aside, the research shows that supervised fine-tuning is a cost-effective and competitive alternative to large-scale reinforcement learning-based approaches, such as the one DeepSeek adopted to develop R1. Meanwhile, giants such as Meta, Google, and Microsoft continue to devote investments of hundreds of billions of dollars to AI infrastructure, aiming to overcome current technological limitations.

While these resources are essential to the advancement of AI, s1 highlights how ingenuity and optimization can provide competitive solutions at drastically reduced costs.