Meta's AI Supercomputer

Contact Counsellor

Last Updated23/02/2022

Facebook-parent Meta announced in January last week that it is building an AI supercomputer, the AI Research SuperCluster (RSC).
The company said that this will be the fastest supercomputer in the world once fully built by mid-2022.
The device is said to accelerate AI research and help in building the metaverse, the next major computing platform.

A supercomputer can perform high-level processing at a faster rate when compared to a normal computer.
Supercomputers are made up of hundreds or thousands of powerful machines which use better artificial intelligence (AI) models to improve operations that process huge amounts of data in less time than normal computers.
They work together to perform complex operations that are not possible with normal computing systems.
Supercomputers require high-speed and specialised chip architectures. The chip performs 660 operations per cycle and thus run up to 230 gigaflops at 350 MHz, Gupta said.
AI supercomputers are built by combining multiple graphic processing units (GPUs) into compute nodes, which are then connected by a high-performance network fabric to allow fast communication between those GPUs, Meta said in their blog.

The supercomputers market is limited to a few major players holding a greater share of the market.
According to Mordor Intelligence, a market intelligence firm, some of the key players include HPE, Atos SE, Dell Inc., Fujitsu Corporation, IBM Corporation, Lenovo Inc., NEC Technologies India Private Limited etc.
The firm estimates the supercomputers market to grow around 9.5% during the 2022 to 2027 period. The firm also considers the increasing use of cloud technology as one of the significant supercomputer market trends with supercomputing centres adopting the cloud, due to the growing workload.
The demand for data centres, AI, and ML (machine learning) among enterprises such as Government and educational entities, is witnessing exponential growth due to the COVID-19 pandemic boosting the demand for supercomputers.
Fast processing of large scale complex data, weather forecast, studying the impact of climate change, predicting and managing natural disasters, advance research on genomes to prevent, diagnose and treat diseases, simulating car crash tests are some of the major areas where supercomputers are used.

The RSC is a powerful supercomputer which can perform tasks like translating text between languages and help identify potentially harmful content on Meta's platform.
It can run computer vision workflows up to 20 times faster, and train large-scale natural language processing models three times faster.
The RSC will help its researchers build better AI models that can work across hundreds of different languages, seamlessly analyse text, images and video together, power real-time voice translations to large groups of people speaking different languages so that they can collaborate on a research project or play an AR game together, and develop new augmented reality tools.
RSC is a powerful supercomputer capable of quintillions of operations per second.
RSC today comprises a total of 760 NVIDIA DGX A100 systems as its compute nodes, for a total of 6,080 GPUs. RSC’s storage tier has 175 petabytes of Pure Storage FlashArray, 46 petabytes of cache storage in Penguin Computing Altus systems, and 10 petabytes of Pure Storage FlashBlade.

Since 2013, we have been making significant strides in AI, including self-supervised learning, where algorithms can learn from vast numbers of examples.
The company however reckons that to fully realise the benefits of advanced AI— self-supervised learning of various domains whether vision, speech, language —will require training large and complex models for critical use cases like identifying harmful content on Meta's platform.
Computer vision, for example, needs to process larger, longer videos with higher data sampling rates.
Speech recognition needs to work well even in challenging conditions with a lot of background noise and needs to understand more languages, dialects, and accents, Meta said.
Therefore it was decided that the best way to accelerate progress was to design a new computing infrastructure, RSC.
There are very large-scale scientific problems that need the right level of depth, accuracy and speed, like modelling all the climate change phenomena, which cannot be handled with the current generation of supercomputers.

Meta said that RSC will help its researchers build better AI models that can learn from trillions of examples, work across hundreds of different languages, seamlessly analyse text, images and video together, power real-time voice translations to large groups of people speaking different languages so that they can collaborate on a research project or play an AR game together, and develop new augmented reality tools.
Researchers will be able to train the largest models needed to develop advanced AI for computer vision, speech recognition.

The AI supercomputers will help build the foundation of metaverse to create artificial-intelligence agents in that environment for rich user interaction mimicking the real world and provide high-performance computing to specific tasks.
The RSC will pave the way toward building technologies for the metaverse where AI-driven applications and products will play an important role.
RSC can keep people safe in the metaverse through its training models that can detect harmful content faster than earlier systems.