The power of three

Dr Marlies Hankel wears two hats. One for her work as a UQ-based eResearch Analyst for both RCC and QCIF, the other as a Senior Research Fellow at UQ’s Australian Institute for Bioengineering and Nanotechnology (AIBN).

She is a computational scientist with a background in mathematics, programming, quantum molecular dynamics and high-performance computing (HPC) cluster management.  

She helps UQ and QCIF member researchers access and efficiently use available HPC resources and is involved in the installation and maintenance of software in UQ facilities.

The group she works in at AIBN is one of the biggest HPC usage groups at UQ.

HPC is an important part of Marlies’ everyday work — it’s also essential to her own research.

Her research is on two-dimensional (2D) materials for battery storage and sensor applications. In layperson terms, she is investigating whether very thin membranes can be used to act as poisonous gas sensors or electrodes in everyday batteries.

She uses the following HPC facilities in her research:

• UQ's Tinaroo HPC cluster, operated by RCC
• National Computational Infrastructure’s (NCI) Raijin HPC cluster through QCIF’s NCI share and also via a successful National Computational Merit Allocation Scheme (NCMAS) project bid. Raijin is a national resource, based at the Australian National University in Canberra.
• WA-based Pawsey Supercomputing Centre’s Magnus HPC cluster.

During the past year, Marlies has used all these facilities to computationally investigate mainly carbon nitrogen materials.

To see if a material was suitable as a gas sensor, different gases were placed in different positions on the materials to see if the gas interacted with the material or not. The investigations included five gases: carbon dioxide, sulfur dioxide, methane, ammonia and hydrogen sulfide; and six different materials with a varying carbon nitrogen ratio. This was to determine how the nitrogen content would influence the sensing performance of the material.

In a similar vein, to see if a material was suitable as a battery electrode, three metals — lithium, sodium and calcium — were placed on the materials in increasing amounts to see if they interacted strongly with the material and if it retained its structural integrity. This was done with the same, aforementioned six carbon nitrogen materials, as well as silicon-doped carbon material.

Access to large amounts of HPC resources has been essential for Marlies’ research. During the investigations, she was able to place the calculations for different materials on different clusters so they could all be run together. Also, if one cluster was under maintenance, then calculations did not stop as they could be distributed over other clusters.

Marlies' calculations are large-scale Message Passing Interface (MPI) calculations, so all three clusters — Tinaroo, Raijin and Magnus — are extremely suitable for this work. Most of her calculations use between 48 and 96 cores, with around 0.5–1 GB of memory per core.

Raijin offers the largest range of available software and hardware to suit most types of calculations. “Here, the ability to run large-scale MPI but also small-scale data analysis calculations make this an essential resource for my work,” said Marlies.

Magnus offers a quick turnaround of calculations that use a very large number of compute cores, but calculations are only allowed to run for a maximum of 24 hours.

Marlies’ most-used HPC cluster is Tinaroo. Tinaroo offers much longer wall times (the real-world time elapsed between method entry and method exit) as any other cluster and is therefore essential for calculations that cannot be easily restarted. It also offers access to a large number of cores.

The advantage of Tinaroo over the other two clusters for UQ researchers is that HPC time is not limited by quarter-yearly quotas on usage and that UQ researchers can apply for ‘enhanced access’ on Tinaroo. It is the only cluster that offers the possibility for increased throughput of calculations for a limited time.

“While I need access to all three clusters to do my research, Tinaroo is still the most important and most-used cluster,” said Marlies.

“Access to Tinaroo for UQ researchers is straightforward and free and it has a similar setup to other clusters around the world. Access to Raijin and Magnus requires an application and is based on merit — it’s free as well, but it requires the effort of a group to get in. QCIF offers access to Raijin via a simpler application to help researchers gain first access before applying through the usual [NCMAS] merit-based scheme.”

Marlies’ work on 2D materials has produced interesting results. She found that 2D carbon nitrogen materials are excellent candidates to sense ammonia and hydrogen sulfide. Through careful tuning of the nitrogen content they also make excellent materials for lithium and calcium ion battery anode materials.

The work has also resulted in Marlies giving two invited talks and three talks at international conferences over the past year, including:

• the 12th Pacific Rim Conference on Ceramic and Glass Technology (PACRIM 12), Hawaii, 21–26 May 2017
• Nanostructures for Sensors, Electronics, Energy and Environment (NanoS-E3), Brisbane, 26–29 September 2017
• Royal Australian Chemical Institute (RACI) Centenary Congress, Melbourne, 23–28 July 2017
• the International Conference on BioNano Innovation, Brisbane, 24-27 September 2017, and
• the International Conference on Nanoscience and Nanotechnology (ICONN), Wollongong, 29 January – 2 February 2018.

Marlies is happy to be contacted with questions on any of the HPC resources mentioned in this article, or about her work and possible collaborations: m.hankel@uq.edu.au.

For more information about Tinaroo, including how to access it, please visit RCC’s website. RCC generally runs ‘Introduction to HPC’ workshops on the last Friday of each month for UQ staff and HDR students, which includes help with RCC-managed HPCs, such as Tinaroo, FlashLite and Awoonga.

Visit Pawsey’s website for more information about Magnus. Pawsey tends to run annual HPC training at UQ, which includes help with using Magnus. The next such training is scheduled for 6–7 June 2018, with more information to be announced in future issues of RCC News.

For more information about NCI’s Raijin HPC, please visit the QRIScloud website, which provides links to the NCMAS scheme and to applying for QCIF’s NCI share.