iTNews published an article on 28 July 2023 about the second phase of hardware installation for UQ's new supercomputer, Bunya. RCC is adding new-generation GPUs and thousands more CPUs to Bunya.
iTNews interviewed RCC Chief Technology Officer Jake Carroll for the article. Jake told iTnews the additional GPUs will make “Bunya Phase 2.0” more suitable for “accelerated computing, machine learning, artificial intelligence and interactive visual computing.”
Below are Jake’s full answers to iTNews’ questions, published with the publication’s permission. You can read iTNews’ article here.
- What kinds of workloads will be supported by the upgrade?
- The upgrade will add GPUs as well as newer generation CPUs to Bunya.
- The addition of GPUs means the supercomputer will be more suitable for accelerated computing, machine learning, artificial intelligence and interactive visual computing.
- In terms of the codes Bunya supports, we are not overly prescriptive, and we want to help as many researchers accelerate their research outcomes as possible.
- The supercomputer was built to be balanced and to cater to as many research disciplines as possible.
- Bunya is now running codes from almost every research domain including, but not limited to, the humanities, economics, molecular dynamics, genomics, weather and climate modelling, artificial intelligence and astrophysics.
- Whilst we say Bunya is a balanced system, we also built it with some specific research communities in mind. For example, UQ’s bioinformatics researchers make use of Bunya's uniformly very high memory footprint per node.
- What are the “feeds and speeds” details of the upgrade – how many more GPUs are being installed, do they demand extra memory, storage, or networking, and so on?
- Bunya Phase 2.0 will feature a fleet of Nvidia H100 and L40 GPUs as well as AMD's Genoa CPUs.
- Networking is facilitated via Nvidia HDR InfiniBand with a signalling rate of 200Gbit/sec per node.
- Because we are firm believers in fostering the growth of the multi-vendor GPU ecosystem and providing diverse options for different research use cases, we also have more AMD Instinct MI210 GPUs coming online. AMD's Instinct GPUs are gaining momentum in the scientific research community as a viable alternative as the software ecosystem grows stronger.
- Storage is provided by our existing IBM ESS3500 Storage Scale parallel filesystems. This storage system features all flash NVME for I/O acceleration.
- From a memory and throughput perspective: the H100 GPU uses HBM3 memory, and the Genoa CPU uses DDR5 memory. The effect is lower latency and higher throughput inside these nodes when moving data between components.
- How does this upgrade fit in the lifespan of Bunya?
- Bunya is an "evergreen" supercomputer.
- It was designed to be iterative and change as technology does to give researchers the best cutting-edge capabilities as soon as they are practically available.
- This is in contrast to large once-off purchases that can lack agility and stymie access to the very latest technologies.
- We operate on a five-year planning horizon for each new tranche of technology we put in place. We can expect that in five years, we will consider the value of these components that are getting commissioned right now and look carefully at the market to determine where we place our investments next to get the best value and research outcomes for UQ.
- To be clear: we do not by default turn a computer off once it reaches five years of age. We consider carefully the value of it, the computational outcomes per watt of power consumed, compared to what the efficiency looks like in the current market.
- More than ever, we must consider exactly how much power every new component uses and the value the power that goes in provides back to the researcher and the University. The days of power consumption without tight measurement and consideration for the environmental impact are long gone (and rightly so), in our view.
