The University of Queensland’s new Natural Resources Innovation and Characterisation Hub (NRICH) is the latest laboratory to receive the UQ Research Computing Centre’s assistance in deploying PITSCHI, a state-of-the-art repository for managing large imaging datasets.
Data from NRICH scientific instruments, such as microscopes, scanning electron microscopes, x-ray and mass spectrometry, will be directly ingested into PITSCHI (Particle Imaging depoT using Storage CacHing Infrastructure), a Clowder-based system.
PITSCHI and Clowder enable users to share, annotate, organise and analyse large collections of electron and correlative microscopy datasets with minimal manual handling of data.
PITSCHI fully integrates with UQ instrument booking capability Research Infrastructure Management System (RIMS), and data storage infrastructure, including MeDiCI, the University’s high-performance data storage fabric. MeDiCI enables PITSCHI users to seamlessly access UQ Research Data Manager (UQRDM) Q data collections.
NRICH is a partnership between UQ’s Centre for Microscopy and Microanalysis (CMM, which co-developed PITSCHI alongside RCC), the Queensland Government, Microscopy Australia and industry groups.
Deploying PITSCHI at NRICH was a joint effort between CMM, Microscopy Australia, RIMS, and RCC.
CMM Director Professor Roger Wepf said deploying PITSCHI at NRICH was “crucial because it streamlines data management, reducing the need to re-enter data across systems.”
“With PITSCHI, each dataset is stored securely in UQ’s RDM system, while metadata is harvested as data is ingested. It eliminates the risk of outdated spreadsheets and avoids inefficient Word or PDF forms by validating and storing data directly,” said Prof. Wepf.
“Integrated with UQ’s RDM Q-collections, PITSCHI also minimises the need to copy large datasets, reducing storage demands and the energy footprint to keep data safe —making it an essential tool for sustainable and efficient research as Q-collections are cached to UQ HPC clusters directly.
“In addition, it allows FAIR [Findable, Accessible, Interoperable and Reusable] data principles to be implemented at the root of data generation, in this case at the microscopes.”
RCC Senior Research Software Developer Dr Mark Endrei said PITSCHI implements an integrated data management approach that aims to empower researchers.
“As cameras and imaging technologies advance, they generate increasingly large volumes of data, making robust data management systems critical. This need is particularly pronounced in large, multi-user facilities such as the CMM and NRICH.”
CMM researchers were the first to use PITSCHI in 2021. So far, 280 researchers, including those outside of CMM, have used PITSCHI to manage 420 TB of data from 50 instruments across 4,000,000 files.
With years of experience working with 3D scientific datasets and exploring PITSCHI, CMM Data Informatics Manager Dr Rubbiya Ali believes PITSCHI can offer far broader benefits than secure data transfer alone.
“By combining AI, structured metadata, and Persistent Identifiers (PIDs), PITSCHI can unlock a new era of accessible, actionable, and impactful data. This convergence of technologies has the potential to transform imaging repositories into engines of innovation. It will drive progress within the scientific community and empower scientists to unlock insights and make discoveries that reshape our understanding of the world,” said Dr Ali.
PITSCHI continues to evolve to meet the needs of UQ researchers and partners. The latest enhancements include support for fee-for-service users, support for instruments with multiple cameras, and improved session recovery and file naming flexibility.
Work required to enable PITSCHI’s deployment at NRICH has been completed and a pilot has commenced as new systems come online. The facility, based at UQ’s Long Pocket campus, is expected to become fully operational by the end of this year.
NRICH will focus on mixed ore bodies for rare minerals and metals essential for modern technologies, advanced batteries, renewable energy and quantum technologies, and recovery and recycling of mining waste and more demanding exploration and extraction opportunities.
PITSCHI was developed as part of the Australian Research Data Commons (ARDC) Australian Characterisation Commons at Scale project, and is built on the open source Clowder data management framework.
Read RCC and CMM’s previous article about PITSCHI.
