Meta Unveils Llama 3.1: A New Era in Open-Source AI
In April, Meta teased an ambitious project: an open-source AI model that could rival the top-tier private models from companies like OpenAI.
Today, that vision is realized with the release of Llama 3.1, the largest open-source AI model to date. Meta asserts that Llama 3.1 surpasses GPT-4 and Anthropic’s Claude 3.5 Sonnet on multiple benchmarks. Alongside this, Meta is expanding its Llama-based AI assistant to more countries and languages and introducing a feature that generates images based on an individual's likeness. CEO Mark Zuckerberg is confident that by year’s end, Meta AI will surpass ChatGPT as the most widely used assistant.
Llama 3.1: A Quantum Leap
Llama 3.1 isn’t just an upgrade; it’s a monumental leap forward. With 405 billion parameters, it stands as Meta’s most complex model, trained on over 16,000 of Nvidia’s high-end H100 GPUs. While Meta hasn’t disclosed the total cost, the expenditure on Nvidia’s hardware alone suggests a price tag in the hundreds of millions.
Despite this significant investment, Meta is committed to an open-source approach, with a license that requires approval only from companies with hundreds of millions of users. Zuckerberg, in a blog post, likened this move to the open-source revolution led by Linux, predicting that open-source AI will soon surpass proprietary models.
Strategic Collaborations and Cost Efficiency
To ensure broad adoption, Meta is partnering with industry leaders such as Amazon, Databricks, Google, and Nvidia. They claim Llama 3.1 is about half as expensive to run as OpenAI’s GPT-4.
Guarded Data and Synthetic Training
Meta has kept the data used for training Llama 3.1 under wraps, citing trade secrets. However, they have disclosed that synthetic data was crucial in training the model. Ahmad Al-Dahle, Meta’s VP of Generative AI, envisions Llama 3.1 serving as a "teacher for smaller models," making AI deployment more efficient.
Meta AI Assistant: Broader Integration
Meta’s Llama-based assistant is set to transform user interactions across Instagram, Facebook, and WhatsApp. Starting this week, it will first be available on WhatsApp and the Meta AI website in the US, followed by Instagram and Facebook. It will also support new languages, including French, German, Hindi, Italian, and Spanish.
Innovative Features and Accessibility
The assistant's most advanced 405-billion parameter model will be freely accessible on Meta AI but will switch to a more manageable 70-billion model after a certain number of prompts, indicating the high operational cost of the larger model. A new “Imagine Me” feature allows users to scan their faces to create personalized images, avoiding deepfake issues by not using profile photos.
Meta AI will also be integrated into the Quest headset, replacing its voice command interface, and will enable real-time identification and information about the surroundings via the headset’s passthrough mode.
Future Prospects
Zuckerberg’s bold prediction that Meta AI will become the leading chatbot by year’s end, surpassing ChatGPT’s 100 million users, highlights Meta’s ambition. Al-Dahle acknowledges that while the industry is still finding its footing, the race in AI is just beginning.
Meta continues to push the boundaries, bringing cutting-edge technology to the forefront of the AI industry.
Editor’s note: This article was written by Jonathan Gasca in collaboration with OpenAI’s GPT-4.0