MOSS, the first large-scale conversational language model in China, is set to become open source software by the end of March 2023, according to its research and development team. An open-source software program is made available to the general public with its source code, allowing users to use, study, modify, and distribute it freely.
Named after the artificial intelligence-based computer in the Chinese sci-fi film The Wandering Earth II, MOSS was the first ChatGPT-like software to be unveiled in China. The platform was made available for public testing on February 20th, and many people quickly tested the software and shared their experiences online. The platform was even suspended for an upgrade due to the overwhelming traffic it received.
“The plan is to have MOSS interact with humans for a month to optimize it. If everything goes well, it will be made open source by the end of March,” said Qiu Xipeng, director of the Natural Language Processing Committee of the Shanghai Computer Society, which is leading the R&D. Qiu, who is also a professor at Fudan University’s School of Computer Science and Technology, said that ChatGPT, developed by OpenAI in the United States, is not an open-source software and its technical solutions have not been made public. Therefore, there have been doubts over whether a ChatGPT-like model, based on instruction learning technology, can be successfully developed in China.
“MOSS still has a lot of room for improvement,” Qiu said. “But its advent proves that the domestic scientific research team has the ability to overcome important technical challenges on the way to developing ChatGPT-like products.”
The domestically developed chatbot has opened up all the technical avenues for generative language models to understand human intentions and have dialogue capabilities, he noted, adding that the project has received strong support from the Shanghai Artificial Intelligence Laboratory.
“Large-scale language models are almost monopolized by foreign countries. They only develop APIs or do not open them to us,” Qiu said. “We want to share MOSS and its model codes and development experiences with everyone, and hope that China can be at the forefront of the world in terms of large-scale language models.”
Making MOSS an open-source software can effectively reduce the threshold for the development and application of pre-trained language models, thus allowing small and medium-sized enterprises to develop various vertical products, such as smart customer service, smart home, and AI lawyers on the basis of it, Qiu said.
“The development threshold of pre-trained language models is very high, requiring a lot of computing power,” Qiu added.
“We look forward to the continued cooperation between the Fudan team and the Shanghai laboratory, through MOSS and subsequent research and exploration, to promote AI inclusiveness and empower the domestic AI industry as soon as possible,” he concluded.
Why is it called MOSS?
The original version was called OpenChat in 2022, but it was found that it was already taken. Later, the movie “The Wandering Earth” was popular, and many AI researchers hoped to create their own strong artificial intelligence. Therefore, it was renamed model MOSS, hoping to encourage to continue iterating and eventually reach the true MOSS level. Also, in the NLP field, researchers often name their models after their favorite characters in movies or TV shows, such as Transformer, BERT, ERNIE, Megatron, etc.
Why use an English interface?
They did both Chinese and English interfaces, but after training the model, it was found that Chinese was more challenging, and the results were not as expected. Therefore, they continue to collect Chinese data and train Chinese models while promoting the beta test primarily in English. The English ability is only to verify the technical route, and the next version of MOSS will significantly enhance Chinese understanding and generation.
How does MOSS compare to ChatGPT?
MOSS has a parameter quantity of billions, so the lack of inference ability and factual knowledge is a difficult defect to overcome, but this defect will quickly alleviate with the increase of parameters. Chinese understanding and generation in MOSS is weaker, but with the increase of Chinese data, the next version of MOSS will significantly enhance this capability. Overall, as a model one order of magnitude smaller than ChatGPT, MOSS is far behind in terms of capability, but it does have some similar abilities. Among the publicly accessible similar demos, MOSS should be one of the most competitive ones, and interested users can compare MOSS, Meta’s BlenderBot 3, and other demos to see which one is closer to ChatGPT.
China’s MOSS, being an open-source AI Chatbot is a massive development for China’s artificial intelligence industry. With the software being open source, it allows for smaller businesses to develop their own AI products, which could benefit businesses by providing more targeted products and services for consumers. MOSS is based on instruction learning technology, and its arrival in China has proven that the domestic scientific research team is capable of overcoming critical technical challenges on the path to developing ChatGPT-like products.
