Research Computing Centre

Unstructured Data Collections

  1. Home
  2. Systems

At UQ, a combination of UQ Information Technology Services (ITS) and RCC take responsibility for the technical aspects of research data storage. 

All researchers can ask RCC for help with data archiving, preservation, management and workflows. RCC provides the large, high-performance and persistent data storage technologies that keep UQ’s mass research data outputs safe.  

RCC can help estimate file storage sizes to assist with data management planning and costing. RCC can also help with complex data-intensive workflows, giving researchers an opportunity to optimise their use of data and compute infrastructure together, to decrease time to discovery. 

For help with data storage, please contact the RCC Service Desk: rcc-support@uq.edu.au

Please note UQ Library is responsible for: 

  • data management plans 

  • metadata design and management 

  • research data publication standards. 
     

MeDiCI  

Since 2015, RCC has developed and successfully deployed MeDiCI (Metropolitan Data Caching Infrastructure), a high-performance data storage fabric for UQ. 

MeDiCI delivers ultra-fast multi-site data access on demand without user involvement, saving researchers valuable time to focus on their work.   

MeDiCI is the backbone of UQ Research Data Manager's (RDM) high-scalability, high-performance computing connected data storage collections. 

MeDiCI provides seamless access to data, regardless of where they are created, manipulated and archived. It connects campus and QRIScloud data storage with the off-site data centre UQ and QRIScloud uses. It speeds up data access between UQ’s main St Lucia campus and the Polaris Data Centre in Springfield, about 30kms south of St Lucia.  

MeDiCI, a distributed file system viewed as a single namespace, holds copies of data on campus until they are not required for some time. It allows data currently being accessed by researchers to be cached on campus, and moves data not accessed for a while back to Polaris. 

MeDiCI provides a number of ways for UQ users to access their data. It can be mounted directly on workstations and clusters on campus, using a number of standard protocols (including NFS, Samba, SMB and IBM’s GPFS native clients). It also offers a synch-and-share mode in which portable devices such as laptops can synchronise files into the same file space. Finally, data can be exposed through an object interface such as S3. 

Underpinning MeDiCI are:

  • HPE's DMF 7 Hierarchical Storage Management Software Suite
  • High-performance and environmentally friendly spin down "ZeroWatt" disk 
  • IBM's high-performance TS series tape media and libraries 
  • DDN GRIDScaler storage and IBM Storage Scale parallel filesystem technologies
  • A 400Gbit/sec ethernet campus backbone from UQ ITS and our sector wide academic network research provider, AARNET. 

UQ’s Queensland Brain Institute (QBI) trialled MeDiCI successfully and the data fabric was rolled out for widespread UQ use, starting in early 2017. 

RCC and QBI also trialled MeDiCI with the University of California San Diego to test the data fabric’s capabilities for international data collaboration. 

UQ has several external partner universities facilitated via QCIF that are using the MeDiCI platform as their data management platform. 

Read more about MeDiCI: