Rise of the Phoenix | | | | Turtles AI

  In an effort to #democratize language models across multiple languages, a team of researchers from #China have released a paper about a new large language model (#LLM) called ‘#Phoenix’. The model not only performs competitively in #English and #Chinese, but also outperforms existing open-source LLMs in languages (including #Vicuna #13B) with limited resources, covering both #Latin and #non-Latin languages. The researchers believe that their work will be instrumental in making language models more accessible, particularly in countries where people face restrictions from OpenAI or local governments. The team has made their data, code, and models available on GitHub at https://github.com/FreedomIntelligence/LLMZoo. The researchers argue that the current state of AI development, where ChatGPT and its successor GPT-4 are developed and maintained by a single company, is unacceptable for the AI community. They define ‘AI supremacy’ as a company’s absolute leadership and monopoly position in an AI field, which may include exclusive capabilities beyond general artificial intelligence. The researchers believe that the existence of AI supremacy could result in unforeseen consequences, such as a single company controlling the future of humanity, with unclear accountability. They argue that AI should be open again, and their work aims to lower the cost and barrier of ChatGPT training so that more responsible researchers can join the ChatGPT research and share their diverse thoughts on developing large language models in a planet-safe way. The team's approach to democratization involves combining instruction data and conversation data to train models to follow instructions in a chat fashion. The resulting multilingual LLM, ‘Phoenix’, achieves state-of-the-art performance on fully open-source Chinese LLMs. In non-Latin languages, Phoenix outperforms existing open-source LLMs, including Vicuna-13b and Guanaco. The Latin-version of Phoenix, called ‘Chimera’, sets a new state-of-the-art in open-source LLMs, impressing even GPT-4 with 96.6% ChatGPT Quality. The researchers believe that their proposed models will benefit people who face legal restrictions from using ChatGPT or related tools, making AI more open and equal again. However, they acknowledge that their models face limitations, including the lack of common sense, limited knowledge domain, biases, inability to understand emotions, and misunderstandings due to context. The researchers plan to add more automatic and human evaluations in the future to make their models more robust. They emphasize that their goal is not to compete with other models, but rather to assist the community in replicating and advancing ChatGPT and GPT-4. The models are currently accessible only within their university and SRIBD.