Llama 3.1 is pitched as a ‘frontier-level’ open-source model, meaning that it can take on the most advanced private models in the market, including OpenAI’s GPT-4.
Mark Zuckerberg has a bold proposition: that open-source AI will eventually become the industry standard in what has become an incredibly competitive space in the global technology sector.
In an open letter posted yesterday (23 July), the Meta CEO and founder presented his case for why open-source AI “is good for the world” and will be around for the long term. This came on the back of the release of Llama 3.1, the company’s biggest AI model yet.
“Until today, open-source large language models have mostly trailed behind their closed counterparts when it comes to capabilities and performance,” Meta wrote in its announcement. “Now, we’re ushering in a new era with open source leading the way.”
Background
Meta first hinted at Llama 3.1 in April, when the Facebook parent said it was working on an open-source AI model that matched the performance of those created by private companies such as OpenAI and Anthropic.
Around the same time, Meta unveiled Llama 3 – its batch of large language models (LLMs) trained on 8bn and 70bn parameters to support a broad range of use cases, with plans to make it multilingual and multimodal.
“Improvements in our post-training procedures substantially reduced false refusal rates, improved alignment and increased diversity in model responses,” Meta said at the time. “We also saw greatly improved capabilities like reasoning, code generation and instruction following making Llama 3 more steerable.”
This came less than a year after Meta released Llama 2 for both research and commercial use.
So what’s new about Llama 3.1?
Meta is pitching Llama 3.1 as its first “frontier-level” open-source AI model, meaning that it can take on the most advanced models currently in the market from private companies, such as GPT-4 and Claude 3.5 Sonnet.
The biggest model in the Llama 3.1 herd is trained on 405bn parameters, putting it in a “class of its own” and giving it capabilities that can help users unlock “new workflows, such as synthetic data generation and model distillation”.
According to evaluations made my Meta, Llama 3.1 405B is competitive when pitted against other language models such as Nemotron 4, GPT-4, GPT-4 Omni and Claude 3.5 Sonnet across a wide range of parameters, including coding, maths and reasoning.
What about drawbacks?
While the decision to open source Llama 3.1 405B can go a long way in democratising access to AI models, Victor Botev, co-founder of Iris.ai, argues that the sheer size of the model “raises important questions about priorities” within AI development.
“Larger models come with substantial drawbacks in terms of computational requirements and energy consumption,” he said.
“This immense scale limits accessibility, as many researchers and organisations lack the infrastructure to effectively utilise such massive models. The environmental impact of training and running these behemoths is also a growing concern that cannot be ignored.”
According to Botev, innovations in model efficiency might benefit the AI community more than “simply scaling up to larger sizes”.
“Achieving similar or superior results with smaller, more manageable models would not only reduce costs and environmental impact but also make advanced AI more accessible to a broader range of users and applications,” he said.
“As we move forward, it’s crucial to balance the pursuit of raw performance with considerations of practicality, accessibility and sustainability in AI development.”
Safety and competition
While Zuckerberg took aim at Apple in his letter – calling the iPhone maker out for the way it taxes developers, the “arbitrary rules” it applies and “all the product innovations” it blocks from shipping – he also addressed concerns around the safety of open-source AI models, arguing that it is safer than alternatives.
“My framework for understanding safety is that we need to protect against two categories of harm: unintentional and intentional … unintentional harm covers the majority of concerns people have around AI – ranging from what influence AI systems will have on the billions of people who will use them to most of the truly catastrophic science fiction scenarios for humanity,” he said.
“On this front, open source should be significantly safer since the systems are more transparent and can be widely scrutinised. Historically, open-source software has been more secure for this reason.”
Earlier today, regulatory bodies from the US, UK and the EU signed a joint statement promising to protect consumers from unfair competition practices within the AI space and promising to work “in the interests of fair, open and competitive markets”.
The statement also highlighted the current “technological inflection point” the world has reached with the evolution of AI, which can introduce new means of competing. “This requires being vigilant and safeguarding against tactics that could undermine fair competition,” the statement said.
Find out how emerging tech trends are transforming tomorrow with our new podcast, Future Human: The Series. Listen now on Spotifyon Apple or wherever you get your podcasts.
Meta CEO Mark Zuckerberg at Facebook’s developer conference in 2019. Image: Anthony Quintano via Flickr (CC BY 2.0)