A range of new advanced computing equipment for UQ has either already been installed or will be installed over the next few months. This includes:
- a significant boost to the imaging-intensive Wiener supercomputer
- a new MeDiCI cache, creating even more data storage and faster data transfers
- a new disk-based storage system at the Polaris Data Centre.
Wiener
RCC will enhance the Wiener supercomputer this year to boost it to a total of 1036 core processing units (CPUs) and 112 Volta graphics processing unit (GPU) cards. The new system will serve a larger UQ research community than originally planned.
The hardware will also include a leading-edge communications system that makes inter-GPU communications even more efficient, and is the first of its type in the world.
UQ’s Research Facilities Infrastructure Grants (RFIG) scheme supported the additional hardware, and includes contributions from the faculties of Science, Medicine and Engineering, Architecture and IT.
Wiener was created in 2016, with funding from the RFIG scheme, as a special purpose supercomputer for processing data streams produced by modern microscopes, such as UQ’s Lattice Light Sheet Microscope (LLSM) at the Institute for Molecular Biosciences (IMB) and electron microscopes in the Centre for Microscopy and Microanalysis (CMM), in real time.
Specifically, Wiener was designed to deliver the image improvement (deconvolution) capacity needed for analysis, and represented an essential new component in the HPC infrastructure required to support cutting-edge microscopy-based research.
Wiener combines a judicious mix of conventional CPUs and GPUs in a small, and relatively inexpensive package, targeting very fast image deconvolution.
Receiving the first NVIDIA Volta class GPUs in Asia-Pacific, Wiener was one of the fastest computers of its type in the region at the time. UQ was one of the first institutions to apply this leading-edge technology to real-time microscopy.
While perfect for deconvolution, the uniqueness of the Wiener design has accelerated the delivery of results for a range of other strategic research projects at the University, including optimising a new treatment for Alzheimer’s disease; automatically categorising skin cancers using artificial intelligence; and discovering new anti-microbial agents for treating drug resistant infections. What’s common about these projects is they apply machine learning algorithms based on deep neural networks. Wiener’s GPU technology provides an extremely efficient way of training such networks.
A new MeDiCI cache
UQ's MeDiCI (Metropolitan Data Caching Infrastructure) data fabric welcomes a new cache, named Ferdinando, which is installed in the Queensland Brain Institute (QBI) and attached to the Wiener supercomputer.
Replacing a former DDN GS7K (with a capacity of 150 TB), the new IBM device, a GH14S, boasts about 1 PB of high-speed, parallel cache storage.
For the technical minded, this cache can deliver more than 60 gigabytes per second of parallel file system bandwidth, and support more than half a million input/output (I/O) operations per second.
The system acts as both a fast scratch storage system for Wiener, and also a cache for UQ Research Data Manager (RDM) collections, meaning researchers can now access their data collections at very high speed on Wiener.
A new disk-based tier for UQ RDM and QRIScloud data collections
RCC is currently finalising a contract that will replace one of the existing tape-based storage tiers in the Polaris Data Centre, which UQ uses for offsite data storage, with high-speed spinning disk.
Using a novel technology, called ZeroWatt, the system can spin down disks to save power, giving it some of the power efficiency of tape. However, since the disks can be spun back up in seconds, the new system will be much faster than the current tape system.
An initial purchase of more than 5 PBs of ZeroWatt storage will mean that all data collections will be held on tape, and can be retrieved into a MeDiCI cache in seconds, providing a significant speedup over the existing tape recalls.
The existing tape silos will be reconfigured to deliver two additional data copies for resilience in the event of a disk failure or data corruption.
We expect the new system to be operational around mid-2019, and envisage a substantial improvement in data transfer performance for data-intensive research projects.
