2023-07-18 03:09:18
‘FriendliAI’ (CEO Byung-Gon Jeon), a startup that provides generative AI serving engine PeriFlow, is developing ‘BPipe’, which dramatically improves the learning efficiency of super-large language models such as ChatGPT. ‘ announced that the system will be presented at the International Machine Learning Society (ICML 2023) to be held on July 25th. The super-large language model, which has recently become a hot topic due to ChatGPT, is a technology that produces excellent results in various language creation tasks by calculating with hundreds to hundreds of billions of parameters (parameters), and is being used in various fields. As the field of super-large language models develops, the technology to efficiently learn super-large language models is becoming important. In order to train a very large language model, distributed learning is essential because of the huge number of parameters (parameters). ‘Pipeline Parallelization’, one of the distributed learning technologies, is a technology in which different GPUs pipeline different layers in a massive language model and process them. This technology has the disadvantage that imbalances occur in the memory used by each GPU at its maximum. BPipe, developed by Friendly AI, solves memory imbalance with a new pipeline parallelization method that leaves intermediate calculation results that need to be stored during learning to GPUs with available memory. As a result, the same language model can be trained faster and more efficiently. For example, if the existing pipeline parallelization is a method of having several workers carry different weights, BPipe is a method in which a worker with a light load temporarily takes over the load of a worker with a heavy load when needed. The research results were presented in ‘BPipe: Memory-Balanced Pipeline Parallelism for Training Large Language Models’ at the International Conference on Machine Learning (ICML), the top artificial intelligence conference held in Hawaii, USA from July 25 to 27. It will be published under the title of The thesis is scheduled to be published on July 26 after being adopted as an oral presentation given to the top 2.5% of the papers submitted to the conference. On the other hand, Friendly AI is a deep-tech company that solves the pain points of companies related to generative AI serving. Its core technologies of ‘dynamic batch processing’ and ‘selective batch processing’ to improve the efficiency of language models such as GPT-3 last year. The engine that applied the has been announced. Press release courtesy of: Friendly AI beSUCCESS is a professional media company with a particular focus on startups and tech industry | beSUCCESS is a media company that supports domestic companies’ overseas expansion. It not only directly and indirectly provides foreign information needed to expand insights of domestic startups, such as global tech trends, startup news, and entrepreneurship, including Silicon Valley. It serves as a ‘connection’ to support the global success of Korean startups by providing major news in English, such as Korea’s startup ecosystem and launch news, to countries around the world.
1689652566
#Generative #engine #development #Friendly #announced #Bpipe #system #increases #efficiency #superlarge #language #model #learning #International #Machine #Learning #Society