Research Computing Centre

Tinaroo

About Tinaroo

Tinaroo is a traditional high performance computing cluster. It is an SGI rackable HPC system for broad research use across UQ.

Tinaroo features an 80 per cent increase in number of processor cores, four-fold increase in average memory and eight-fold peak performance increase over Barrine, the UQ HPC system it is replacing.

Tinaroo has about 6,000 processor cores, and some 30 TBs of main memory. That means it can solve very large problems by splitting it up across the processors and memory modules.

Tinaroo will be focused on tightly coupled parallel jobs to undertake tasks requiring a very large number of computer cores to be applied to a given problem at the same time. It will augment FlashLite, a data-intensive supercomputer and Awoonga, a conventional high-performance compute cluster. Together these systems will span a wide range of computational templates.

Tinaroo is a UQ HPC resource that has been co-located with QRIScloud and uses QRIScloud's identity management system to ensure interoperability with FlashLite, Awoonga and storage infrastructure. Tinaroo is for UQ staff and students and their research collaborators.

Tinaroo is connected to a file system that provides nearly 0.5 PBs of storage, and has access to QRIScloud and Research Data Services (RDS) data collections. This means researchers can process data they have previously archived using Tinaroo, FlashLite, RDS and/or QRIScloud. The seamless way data can be moved between platforms is very powerful.

Tinaroo, installed in Springfield’s Polaris Data Centre, has access to UQ’s GPFS parallel storage environment, which has been upgraded with an additional capacity of about half-a-petabyte to provide working storage capability.
 

Getting a Tinaroo account

Access to Tinaroo is through QRIScloud. 

To open a Tinaroo account, UQ staff and students should do the following:

  1. Register for a QRIScloud account if they do not already have one (register at QRIScloud). 
    - Click on "Account" and login using their AAF credentials.
    - All UQ users will be able to use their organisational ID (login name) and password as their AAF credential without any further process.
 
    - If you are a student, please also provide the name of your UQ supervisor when you register for a QRIScloud account.
  2. Then apply for Tinaroo access by clicking here and completing the form. 

Once your access to Tinaroo has been activated, your access to Tinaroo will be confirmed via email.
Access to Tinaroo should typically be activated within 2 working days.
 

Tinaroo training 

RCC conducts regular 'Introduction to HPC' training on the last Friday of each month. Please visit RCC's training webpage for further information.

Tinaroo vendor SGI will work with RCC to deliver training annually over the next three years and will provide comprehensive maintenance and technical support. The 2016 session was held 6–9 June.

Please send enquiries via email to rcc-support@uq.edu.au for further information or to register your interest in future Tinaroo sessions.
 

Tinaroo User Guide

A Tinaroo User Guide is available here (from UQ networks including VPN).
 

Tinaroo support

Users of Tinaroo should submit support requests to: rcc-support@uq.edu.au.
 

Tinaroo Enhanced Access Program

RCC launched a new program in October 2016 to provide enhanced access to HPC Tinaroo for UQ research groups and individuals conducting high impact research.

The benefits of the new mechanism include:

  • increased queue limits for single node jobs
  • more flexibility with job array limits
  • increased maximum allowed wall-times
  • the ability to submit jobs to Tinaroo from Nimrod, a scientific workflow system
  • exclusive use of nodes outside of the batch system
  • virtualisation that permits using the guest operating system of your choice, and more.

See the Tinaroo Enhanced Access Guide for further information about benefits. The mechanism is applicable to the workload, storage and batch system configuration "dimensions".

The application process requires a brief strategic and technical justification for using Tinaroo Enhanced Access. To apply, send an email to rcc-support@uq.edu.au with "Tinaroo Enhanced Access Request" as the subject. Please check the Tinaroo Enhanced Access Guide as to what information is required in your email submission.
 

Tinaroo FAQs

Click here to view Tinaroo’s FAQs.
 

Tinaroo technical specifications

The cluster is based on Rocks 6.2; CentOS 6.6; Torque / Maui Resource Manager.

Tinaroo shares the GPFS based storage that is attached to FlashLite.

There are two login nodes and two admin nodes.

There are 61 C2112-4GP3-R-F 2RU (rack-unit) chassis with four server nodes per chassis.

Each of the 244 compute nodes is comprised of:

  • 2 x Intel Xeon E5-2680V3 2.5Ghz, 12 core processors
  • 8 x 16 GB DIMMs (128 GB) [max. 120GB should requested per node]
  • 1 x 1 TB 7.2K SATA drive.

The 244 compute nodes have in aggregate:

  • 488 CPU sockets
  • 5856 CPU cores
  • 31,232 GB of RAM
  • 244 TB of local disk space
  • Licensed as GPFS storage clients
  • 234.2 TFLOP.

The nodes are connected to each other via a FDR infiniband network arranged in a four-dimensional enhanced hypercube topology. Although the network topology favours jobs running across 18 or fewer nodes (i.e. less than 433 cores), the networking penalty associated with running much larger jobs is not too high. The batch system will be tuned to take full advantage of the network topology.

The total system:

  • 5 server racks
  • 154 RUs (rack units)
  • 102.4 kW
  • 2.6 tonnes.