NCI’s newest supercomputer is Gadi, Australia's peak research supercomputer for 2020 and beyond. A 3,200 node supercomputer comprising the latest generation Intel Cascade Lake and Nvidia V100 processors, Gadi supports diverse workloads with over 9 petaflops of peak performance.

Technical Specifications

Gadi contains a total of 145,152 CPU cores, 567 Terabytes of memory and 640 GPUs.

  • 3,024 nodes each containing two 24-core Intel Xeon Scalable ‘Cascade Lake’ processors and 192 Gigabytes of memory.
    • including 50 nodes each offering 1.5 Terabytes of Intel Optane DC Persistent memory.
  • 160 nodes each containing four Nvidia V100 GPUs and two 24-core Intel Xeon Scalable 'Cascade Lake' processors.
  • Linking the storage systems and Gadi is Mellanox Technologies' latest generation HDR InfiniBand technology in a Dragonfly+ topology, capable of transferring data at up to 200 Gb/s.
  • The storage sub-systems are NetApp enterprise class storage arrays, linked together in a DDN Lustre parallel file system.
  • Altair’s PBSPro software optimises job scheduling and workload management.
  • Gadi uses the latest version of the CentOS 8 operating system.
Black mesh cabinets with silver computer servers, cables and lights visible inside them.

Gadi offers a 7 times increase in computational performance, and a significantly increased number of GPUs compared to its predecessor, the Raijin supercomputer. This provides significant benefits for the largest parallel codes in particular, and all users benefit from shorter queues and higher throughput. 

Gadi first launched to NCI users in mid-November 2019. User documentation about the system can be found in our User Guides.

The Gadi installation was documented on our dedicated blog. Read through to see each step of the progress.

NCI’s previous supercomputer was a hybrid Fujitsu Primergy-Lenovo NeXtScale system called Raijin. Named after the Shinto god of thunder, lightning and storms, Raijin was a high-performance, distributed memory cluster with a peak performance of over 2 petaflops.

Image of HPC system

Raijin was originally installed in 2012 and entered full production in June 2013. Significant updates to the system entered production in November 2016 and again in mid-2017. 

Raijin evolved from a relatively straightforward x86-based cluster to a heterogeneous one made up of many different systems. Over its operational life, Raijin was augmented with a wide variety of x86 and non-x86 based systems, including GPUs, many-core Xeon Phi processors and Power8 processors.

The entire Raijin cluster ran the same operating system, an optimised version of a highly-customised, NCI-developed Linux kernel. 

Raijin had:

  • 89,256 cores in 4,500 Intel Xeon Sandy Bridge, Broadwell and Skylake nodes
  • 128 GPUs in 32 NVIDIA Tesla K80 and P100 nodes
  • 32 64-core Intel Xeon Phi processors in 32 nodes
  • 64 cores in 4 IBM Power8 nodes 
  • 300 terabytes of memory
  • 8 petabytes of operational disk storage
  • Hybrid FDR/EDR Mellanox Infiniband full fat tree interconnect (up to 100 Gb/sec)