Learning Outcomes in Open Online TRE Training: A Mixed-Methods Evaluation

Abstract

Objectives
To assess the effectiveness and impact of open online Trusted Research Environment (TRE) training through mixed-methods analysis. This study aims to understand participant engagement, satisfaction, and perceived learning outcomes while identifying opportunities for improving program evaluation and learner experience for future program iterations.

Methods
A retrospective mixed-methods analysis was conducted using available program data from multiple sources. Data included registration information, usability scales, learning analytics, post-training surveys, participant feedback, and semi-structured interviews. We employed triangulation to validate findings across different data sources. Qualitative data was analysed using thematic and sentimental analysis to identify recurring patterns and learner experience, while descriptive statistics were used to analyse quantitative data from surveys, registration forms, usability scales, and learning analytics extracted from a learning management system.

Results
Preliminary data collection and analysis have already revealed interesting findings. Based on the registration data, over half of the users are academic researchers affiliated with universities. Survey responses submitted by 45 webinar participants yield 98% satisfaction. Sentimental analysis on feedback and interviews demonstrated an overwhelmingly positive learner experience. Interview data provided rich insights into participants’ perceived learning outcomes, with perceived gain as one of the main themes identified for outcome analysis. However, it has also revealed early signs of areas for improvement, including technical support, and dropouts.

Conclusion
This assessment provides valuable insights into program effectiveness and areas for improvement. Preliminary findings support the program's positive impact while highlighting areas for improvement.