ADRUK Ingest Improvement Task and Finish Group: Smoothing the ingest of data

Abstract

Objectives
Analysts ingest data to TREs for research; extract, process and curate public sector data then made accessible to researchers in safe environments. Although differences surround each dataset, the processes can be made broadly replicable. We will summarise data ingest processes across organisations, identify and share lessons learnt and best practices.

Methods
An ADR UK Task and Finish group with a membership comprising representatives from each of the four UK nations has been established. Membership includes the Scottish Government, the ONS, NISRA, the Honest Broker Service, the Welsh Government, Research Data Scotland, UK Data Service, UK LLC, and eDRIS. The focus of the work will be on the following key areas:

• Areas of good practice,


• Technical issues affecting the data ingest process,


• Capability and skills,


• Opportunities for further improvements,


• Key challenges


• The role of automation.

Results
A final report and recommendations including lessons learnt: ‘Smoothing the ingest of data – Comparison across the 4 UK countries’ will be completed in June 2025, in advance of the ADR UK conference. The work included in this report will be a catalyst for further UK wide collaboration and learning opportunities and the ADR UK conference is the ideal opportunity to disseminate the key findings.

Conclusion
This work and report will inform potential improvements and alignments across the 4 UK nations for data ingest processes. There is an intentional focus on reproducibility, creating efficiencies, and robust quality checks, and building cross-national relationships.