RCC is in the process of installing new software and hardware infrastructure to power the University of Queensland’s next-generation research data archiving and storage platform.
The infrastructure, the latest version of Hewlett Packard Enterprise’s (HPE) Data Management Framework 7 (DMF7), is part of UQ’s ongoing and significant investment in research data storage capabilities.
Unlike the current version of DMF, DMF7 scales to trillions of storage objects. It will provide UQ researchers and the University’s growing Research Data Manager (RDM) platform (Q collections[1]), with a sustainable path for continual data growth. It will enable far more data to be stored and faster data transfers.
“RCC recognised it must provide a step-change in storage scalability to meet our future research challenges,” said RCC Chief Technology Officer Jake Carroll.
"This is one of the most important infrastructure projects RCC has undertaken, and certainly one of the most complex. For our end users, clients and researchers this work is very much 'behind the scenes', so whilst it means a great deal to us as admins, engineers and computer scientists, our many clients across UQ will simply see things working better," said Jake.
RCC will use the DMF7 server and storage platform as part of high-performance computing infrastructure to service the data storage and data analysis needs of UQ researchers.
UQ’s version of DMF7 will be tightly integrated with MeDiCI, the University’s high-performance data storage fabric, which provides seamless access to data assets regardless of where they are created, manipulated and archived. This will enable MeDiCI’s greater performance (including faster data transfers), easier management and a better overall experience for RDM users.
DMF7 provides UQ with options for a diverse range of archiving and preservation backends and greater data security (see below for more technical information about this).
The new infrastructure will allow for more configurable data storage policy management, giving RCC the ability to create rich metadata for each object. This will enable more granular archiving policies and placement, depending upon the requirements of the research project.
DMF7 was released in mid-2019 as a product, but RCC waited a year for HPE to include all the features UQ needed to ensure deep integration with the University’s existing high-performance infrastructure and data storage environment.
DMF7 technical information
- DMF7 provides UQ options for a diverse range of archiving and preservation backends. These backends include but are not limited to:
- On-premise Zero Watt Storage that provides archiving assurance but disk-like latency and recall speeds.
- On-premise exa-scale tape storage technologies that offer the very best storage economics.
- Off-premise public cloud deep archiving infrastructure.
- DMF7 provides UQ new options in continuous data integrity checking, on the fly checksum calculation and “bit rot” detection tools, giving UQ a further level of assurance for its most important research data assets.
- Due to new filesystem design techniques, DMF7 can provide “tier0” capabilities to create dynamic filesystem name spaces, portions of filesystems and views of data for specific needs, then allow the collapse and consolidation of the filesystem when complete.
- Performance: DMF7 will implement UQ’s first HDR Infiniband 200Gbit/sec network — allowing up to 25 gigabytes per second of network traffic to move between storage arrays and archiving devices point to point.
[1] Q collections are established when a researcher ticks the HPC box in RDM. These collections are larger and more performant than the default RDM ones.
