Meta is preparing for the launch of Llama 4 | Meta Facebook login | Meta AI image generator | Meta login | Turtles AI

Meta is preparing for the launch of Llama 4
Development of an autonomous AI with advanced capacity for reasoning and adaptation
Isabella V29 October 2024

 

 

Meta is set to launch Llama 4 early next year, aiming to develop autonomous AI. Meanwhile, OpenAI may have already achieved significant milestones with AGI while keeping a low profile. Meta’s innovative approach includes the use of techniques such as “Chain of Thought” to improve real-time reasoning and adaptation.

Key points:

  • Meta plans to release Llama 4 in 2025, with new features for autonomous reasoning.
  • OpenAI may have achieved AGI internally, delaying its release pending competitors.
  • Meta is using self-supervised learning and human feedback to improve its models.
  • Llama’s generation of synthetic data is improving language resources in underserved languages.

Meta is preparing to release Llama 4, a model that promises to greatly improve AI reasoning capabilities, with the goal of approaching the results achieved by GPT-4o and OpenAI’s o1. This development is accompanied by growing competition with Chinese models such as Kai-Fu Lee’s 01.AI, which are gaining attention for their performance in reasoning benchmarks. Manohar Paluri, vice president of AI at Meta, said the team is exploring strategies to enable Llama to not only plan, but also evaluate and adapt to decisions in real time, making AI more flexible and responsive to changing circumstances. This approach is based on innovative techniques, including “Chain of Thought,” which aims to give models a deeper understanding of complex situations.

Yann LeCun, head of Meta AI, argues that the development of autonomous AI systems could bring significant everyday benefits to people by requiring an understanding of the physical world and causal relationships. This concept is close to that of AGI, of which OpenAI seems to have an advantage, as suggested by recent statements by Sam Altman, who refuted speculation about an imminent release of Orion (GPT-5). The central question is whether Meta will be able to take Llama’s reasoning to a level comparable to that of GPT-4o and o1, and how this might affect the competitive landscape.

Paluri emphasized the importance of decomposing complex tasks into manageable steps to improve reasoning in “untestable domains.” A practical example is trip planning, which requires dealing with real-time variables such as adverse weather conditions, demonstrating how future versions of Llama can excel at solving practical problems. Recently, Meta introduced Dualformer, a model that can dynamically switch between different thinking styles, improving efficiency in complex tasks.

A key element in Llama’s success is its approach to learning. Meta uses self-supervised learning (SSL) to enable models to learn from huge volumes of unlabeled data, ensuring general understanding. In parallel, reinforcement learning with human feedback (RLHF) helps refine behavior in specific tasks, ensuring that models are not only understanding but also aligned with practical goals. This combination allows Llama to generate high-quality synthetic data, a crucial resource for addressing challenges related to underrepresented languages, such as Indian languages.

During the Cypher 2024 summit, Vivek Raghavan of Sarvam AI illustrated how Llama 3.1 405B was employed to build a model capable of generating Indian language data, highlighting Llama’s potential in creating diverse language resources. Recent innovations include quantized versions of Llama models, improving on-device performance and reducing memory requirements, making the use of such models more accessible and practical.

Meta has already started pre-training for Llama 4, and Mark Zuckerberg’s optimism about AI progress is palpable. Expectations for the release of “next-generation” models by 2025 are high, with the anticipation of advanced features such as memory capacity and multi-modal support, promising a radically improved user experience. With this momentum of innovation, the coming months could prove decisive for the future of AI.

In anticipation of these developments, the field of AI continues to grow and transform, bringing with it unprecedented challenges and opportunities.