Data Management
UQ has recently established a data management policy. The policy mandates the management and curation of research data for specified periods, depending on ARC/NHMRC funding rules and local statutes on data retention.
High Performance Computing users can ask the RCC for help with data archiving (see the UQ Research Data Archive section below).
The RCC can also help estimate file storage sizes to assist with data management planning and costing.
QRIScloud, which the RCC manages in conjunction with QCIF, can offer extensive data storage for data of national significance.
The RCC deploys tools and technologies to help UQ researchers manage their data well. The RCC currently supports two technologies — OMERO and LiveArc — for the capture and management of imaging data from instruments such as Magnetic Resonance Imaging (MRI) machines and microscopes.
OMERO
Open Microscopy Remote Objects (OMERO) is a modern client-server software platform for visualising, managing, and annotating scientific image data.
OMERO supports the importing and archiving of images, annotation and tagging, recording experimental protocols, and exporting of images in a number of formats. It also makes it possible to collaborate with colleagues anywhere in the world by creating user groups with different permission levels.
XNAT / Australian Imaging Service
XNAT is an extensible open-source imaging informatics software platform dedicated to imaging-based research. The Australian Imaging Service (AIS) builds on XNAT investments at universities and clinical sites across Australia with enhanced data management and analysis capabilities.
AIS will provide a distributed federation linking member institutions, including the UQ XNAT service, with a federated search layer, common community practice, support for expanded data types and a Trusted Data Repository ensuring ongoing ownership and accountability of data.
LiveArc
The RCC has implemented a local LiveArc instance for QRIScloud users to better manage their data collections on the cloud.
LiveArc is a subject-oriented informatics framework and capability developed primarily at the University of Melbourne. It is built with the commercial Mediaflux data operating system, which was developed by Arcitecta, a Melbourne-based company specialising in data management systems for large-scale distributed data.
LiveArc is being used at UQ as a repository to manage mostly MRI imaging data.
UQ Research Data Archive
RCC operates a nearline data archive for UQ research data.
It has multiple petabytes of capacity by utilising hierarchical storage management (HSM) technologies.
HSM is a technology for seamlessly migrating data between high performance, high-cost storage (e.g. hard disks in RAID arrays where you need to read and write data), to low performance, low-cost storage (e.g. magnetic tape drives where it sits when not in active use).
A number of pools of disk and tapes are available to be used depending on the circumstances.
Individual users, projects and individuals with special data needs can use the HSM for their work. The HSM file systems were able to be directly accessed from the UQ Barrine HPC cluster. Although this direct mount access from HPC is no longer possible, your HSM data can be accessed by other means as required.
A remote copy of your data is created automatically and stored offsite, in addition to the copies held at UQ’s St Lucia Campus.
RCC does not operate a traditional data backup service, which involves daily incremental and weekly full backups of data. This means the RCC cannot recover a file you have accidentally deleted.