Facebook’s parent company Meta is partway through building an artificial intelligence (AI) supercomputer cluster it estimates will be the fastest on Earth when completed.

The AI Research SuperCluster (RSC) is currently kitted out with 6,080 NVIDIA A100 tensor core GPUs, purpose-built for AI research and processing.

By the middle of the year, Meta said in a blog post, the RSC will be connected with a total 16,000 A100 GPUs along with hundreds of petabytes of storage and cache memory that makes the system capable of serving 16 TB/s of training data.

“RSC will help Meta’s AI researchers build new and better AI models that can learn from trillions of examples; work across hundreds of different languages; seamlessly analyze text, images, and video together; develop new augmented reality tools; and much more,” Meta said.

“Our researchers will be able to train the largest models needed to develop advanced AI for computer vision, NLP [natural language processing], speech recognition, and more.

“We hope RSC will help us build entirely new AI systems that can, for example, power real-time voice translations to large groups of people, each speaking a different language, so they can seamlessly collaborate on a research project or play an AR game together.”

An outline of the Meta RSC's architecture. Image: supplied

The RSC marks Meta’s departure from using open source and publicly available data sets for its AI modeling as the cluster has been designed to use data about the company’s products in real-time.

As a result, it is isolated from the internet with data instead being piped directly from Meta’s production data centres.

Meta suggests this isolation is about maintaining the privacy of user data which “must go through a privacy review process” to make sure it has all been “correctly anonymised” before use.

The direct connection to Meta’s products like Facebook and Instagram will give the company greater access to proprietary AI models using data that won’t be as accessible to other research teams.

Meta said access to real-world data will help its AI content moderation systems which notably failed to stop the livestreamed Christchurch massacre in 2019 before the footage could spread.

Another stated use for the direct data pipeline will be in building out Meta’s augmented and virtual reality offerings and to “power the metaverse”.