Meta Platforms Inc has announced the release of a new large language model, LLaMA, which is the core software of their new artificial intelligence system. The company has made the software available to researchers under a non-commercial license, heating up the AI arms race as big tech companies rush to integrate the technology into their products and impress investors.
LLaMA, which stands for Large Language Model Meta AI, is trained on 20 languages with a focus on those with Latin and Cyrillic alphabets, and requires far less computing power than previous offerings. It can mine vast amounts of text in order to summarise information and generate content. The model can answer questions with sentences that read as though they were written by humans.
The announcement by Meta appears to be a step in testing their generative AI capabilities so they can implement them into their products in the future. Generative AI is a new application of AI that Meta has less experience with, but is clearly important for the future of their business.
AI has emerged as a bright spot for investments in the tech industry, whose slowing growth has prompted widespread layoffs and a cutback on experimental bets. Meta said LLaMA could outperform competitors that examine more parameters, or variables that the algorithm takes into account.
Meta also claims that a version of LLaMA with 13 billion parameters can outperform GPT-3, a recent predecessor to the model on which Microsoft-backed OpenAI’s ChatGPT is built. It described its 65-billion-parameter LLaMA model as competitive with Google’s Chinchilla70B and PaLM-540B, which are even larger than the model that Google used to show off its Bard chat-powered search.
Meta attributes the performance to a larger quantity of cleaner data and architectural improvements in the model that enhance training stability. This puts Meta in direct competition with Alphabet Inc and China’s Baidu Inc, which have also been promoting their own offerings in the AI space.
In May last year, Meta released large language model OPT-175B, which formed the basis of a new iteration of its chatbot BlenderBot. It later introduced a model called Galactica, which could write scientific articles and solve math problems, but quickly pulled down the demo after it generated authoritative-sounding false responses.
The release of Meta’s LLaMA model shows that the AI arms race is heating up as big tech companies rush to integrate AI technology into their products and impress investors. Meta’s new language model is a step towards testing their generative AI capabilities, an application of AI that Meta has less experience with, but is important for the future of their business.
If you want to apply for access to LLaMA, please follow this request form.