Supercomputer Bunya was launched this month after a successful user testing phase and is now available for wider use by University of Queensland researchers.
Bunya’s launch will see high-performance computers FlashLite and Tinaroo switched off in 2023. FlashLite will be shut down on Tuesday, 3 January. Tinaroo’s decommissioning date in 2023 has yet to be determined and will be announced via RCC’s usual channels.
Both FlashLite and Tinaroo users are being urged to apply for access to Bunya. “Now is the time to get involved,” said RCC Chief Technology Officer Jake Carroll. See the section further below about how to apply for access to Bunya and for more information about FlashLite’s shutdown.
“Bunya is the culmination of a multi-year planning process, a fresh look at our model for supercomputing, and a group of incredibly talented people working at their peak to deliver something significant to the research community,” said Jake.
“Bunya is a special initiative. We value and encourage the innovative spirit that built it. We are excited by the performance characteristics it exhibits. We are proud that RCC could create something that will carry our community for several years into the future in a scalable, sustainable and progressive way.
“We look forward to the many successes and advantages Bunya will bring to our researchers.”
Bunya test users
Test users of Bunya from UQ’s research community have given RCC very positive feedback about their experience.
RCC selected test users who would benefit from Bunya’s architecture, stress test the system, provide valuable feedback and help indicate opportunities for early improvements.
Bunya's performance in early benchmarks and user testing met and exceeded RCC’s expectations. “Depending upon the code being run, the speedup users are experiencing has been anywhere from almost two times to over four times faster,” said Jake.
“We expect to see further gains, as RCC staff work closely with our users to optimise their codes and software to suit the new environment. Plus, we have started our investigation into compiler optimisations for the new hardware.”
Two researchers within the UQ Centre of Hypersonics — Postdoctoral Research Fellow Dr Nick Gibbons and PhD student Lachlan Whyborn — had been using Bunya for a few weeks when contacted by RCC for feedback.
Nick said Bunya had performed very well in simple benchmarking tests, and he had managed to do some large calculations on the machine.
“The user experience on Bunya has been very similar to the large national supercomputers I am familiar with,” said Nick.
“Our code compiled on the first try, for example, and it looks like we won't need any machine specific tweaks that might confuse our users later on.
“Overall the experience has been very positive, and the RCC team is to be congratulated on a smooth testing phase!”
Nick is one of the developers of his group’s open-source flow simulation code, Eilmer. “We use Eilmer to compute the [gas] flow around fast-moving objects, targeting improvements in the design of supersonic aeroplanes, the safety of re-entering spacecraft, and many other research problems in fundamental fluid mechanics. These simulations use a large amount of compute time, so I was pleased to be invited as an early test user of Bunya,” said Nick.
Meanwhile, Lachlan has been using Bunya on simulations for his PhD work. “These are computational fluid dynamics simulations of very high resolution that allow us to study the physics of hypersonic boundary layers. Specifically, my research is looking at what types of disturbances will upset the stability of a boundary layer and cause it to go turbulent,” said Lachlan.
“The experience with Bunya has been smooth for the most part. The dense core count per node has helped me do simulations that were just not possible on Tinaroo.”
Dr Emily Kahl, a Software Engineer in the Bernhardt Group at the Australian Institute for Bioengineering and Nanotechnology (AIBN), said she is “loving Bunya so far” after using it in its testing phase since the end of September.
In her work, Emily has been developing GPU-accelerated codes to model fluid flow at the molecular level using molecular dynamics.
“This work is closely connected to the project we have with Pawsey's PaCER program, so our work with Bunya will benefit our work on Setonix (Pawsey's new cluster) and vice versa, improving the quality of our research on both,” she said.
“Portability between different GPU vendors is important to us, so the GPUs on Bunya have been a great resource and will continue to be important for simulations as more of us move on to the cluster.”
Emily values RCC’s efforts in making the developer experience “pleasant and efficient” on Bunya. “Developing custom codes has subtly different needs to researchers who use off-the-shelf software, and historically many HPC resources have not put much effort into supporting the needs of software developers. Improving developer productivity substantially reduces turnaround times for research.
She said RCC had provided “first-class support” in installing specific software onto Bunya that her research group requested, making part of her job much easier.
“Additionally, I really like the decision by RCC to manage their own scientific software stack (e.g., compilers and scientific libraries) rather than limiting selection to vendor-supplied software,” said Emily.
“Flexibility in choosing the right software for the job is important for my work and, in my experience, vendor lock-in adds excess friction to the software development process that just makes everything that little bit more annoying to work with. The decisions RCC has made in this regard seem much more developer-friendly.”
Dr Sharif Nada, a Postdoctoral Research Fellow in UQ’s School of Chemistry and Molecular Biosciences, had been using Bunya for just over two weeks when RCC contacted him, and was still in the process of installing and testing the software his research group needs on the machine.
“Thus far, the aspect of Bunya that stands out to me is the heterogeneous nature of the system hardware. It is the first HPC machine I have used in Australia that has some nodes with AMD GPUs and others with NVIDIA GPUs,” said Sharif.
“This will help us to evaluate the performance of our GPU-accelerated code on a range of hardware and more easily identify the resource allocations that give us the best balance between speed and efficiency.”
The Molecular Dynamics Group Sharif is part of plans to use Bunya to perform biomolecular simulations to improve real-world processes ranging from drug design to the manufacture of thin-film semiconductors.
“The combination of Bunya’s powerful CPU and GPU compute capabilities will allow us to refine protocols for automatically generating starting configurations for densely packed polymer systems, which have been considered infeasible to set up manually due to their complexity.”
Bunya is available to all UQ research staff and higher-degree research students to use. For information about how to apply for access please see below.
For more information about Bunya, please read our previous articles about the HPC:
FlashLite’s shutdown
FlashLite will be switched off after 9am on Tuesday, 3 January 2023. There will be no further notice about this.
Users can run jobs as usual until the end but any job found running on 3 January will be killed. Users should make sure to finish their jobs that use $TMPDIR and copy data to /home, /scratch and/or /QRISdata which is accessible via Tinaroo and Bunya before 3 January.
Please email rcc-support@uq.edu.au with the subject line 'Migration to Bunya' and one of our staff will take you through the onboarding procedures. Note that the RCC Support Desk will be shut down over the Christmas and New Year break — please see our RCC Support Desk webpage to view these closure dates.
How to apply for access to Bunya
Users can now apply for access to Bunya — the link to the “Bunya access application form” is below.
Important: there are 8 questions and all are mandatory.
Please copy the questions into the “Request details” box on the application form and answer each under its number. You will need to provide current FoR codes.
On Bunya, every user is required to be in an account group which will enable reporting by organisational units but also individual research groups. Research group leaders would be the owner of these account groups and are able to administer which user belongs under their research group. Research group leaders (group owners) are able to appoint one or more group administrators who will be able to add and remove users from the account group in addition to themselves.
Users will not be able to run a job unless a group leader has set up an account group and has added them to it.
Possible group leaders will be contacted using the information provided by applicants in the application form and will be sent information on how to set up an account group. It is therefore important that all applicants provide information about their supervisor and associated research groups. Group leaders may also contact rcc-support@uq.edu.au to set up an account group.
If you have any questions or concerns, please contact rcc-support@uq.edu.au.
