OLMo 2 32B is a fully open LLM outperforming GPT-4o mini | Llm meaning death | Llm training dataset free | Llm sft dataset | Turtles AI

AI2’s OLMo 2 introduces improvements to the open language model architecture, with optimized data and techniques, promoting transparency and efficiency in the AI ​​industry.

Key Points:

• New Architecture: OLMo 2 adopts advanced training techniques and a new data mix, the “Dolmino Mix 1124”.
• Enhanced Models: Available in 7 and 13 billion parameter versions, with improved performance.
• Transparency and Accessibility: All training data, code, and recipes are publicly available.
• Industry Competition: OLMo 2 competes against leading models such as Llama 3.1 and Qwen 2.5.

The Allen Institute for AI (AI2) recently unveiled OLMo 2, the second generation of its open language models. This release introduces improvements to the architecture and training techniques, including a new data mix called the “Dolmino Mix 1124”, which has led to significant performance improvements on several benchmarks. Additionally, OLMo 2-Instruct incorporates best practices from Tülu 3, focusing on permissive data and extending reinforcement learning with verifiable rewards (RLVR). Available in 7 and 13 billion parameter sizes, OLMo 2 models compete with other open models such as Llama 3.1 and Qwen 2.5, offering efficiency and transparency thanks to the complete availability of the training data, code, and recipes used.

This initiative is part of a broader trend in AI, where research institutions and companies are developing increasingly advanced extended language models (LLMs). For example, the Gemma team has introduced Gemma 2, a family of state-of-the-art, lightweight models ranging from 2 to 27 billion parameters. These models apply well-known technical modifications to the Transformer architecture, such as interleaving local and global attention and group-query attention, offering competitive performance compared to larger models.

Likewise, OpenAI, founded in 2015 with the goal of promoting and developing friendly AI, recently announced its transformation into a for-profit company, continuing to contribute significantly to the advancement of AI.

These developments reflect the commitment of the scientific community to make AI more accessible and transparent, promoting open collaboration and innovation in the field.