Co-producing data-intensive research with an underserved group: a case study and evaluation identifying pathways to impact

Abstract

Introduction
Co-production of research, where researchers and experts by experience work as equal partners throughout a research project, can improve the quality, relevance, implementation and impact of research. However, there is limited evidence on methods for successful co-production in data-intensive research with underserved groups. In partnership with the charity Voice of Young People in Care (VOYPIC) and a group of care experienced young people, the Administrative Data Research Centre Northern Ireland (ADRC NI) piloted and evaluated a co-production approach in a research project that used linked administrative data to examine the association between care experience and mental ill health and mortality.

Objective
The aim of this paper is to report the impact of co-production using the pilot as a case study, and assess the mechanisms involved against published principles of co-production. Additionally, we consider if co-production in this context is a special case that warrants bespoke guidance.

Methods
Two participatory workshops and three semi-structured 1-1 interviews were conducted to collect the perspectives of pilot participants. Deductive thematic analysis was used to sort data into three predetermined categories: 1) impact; 2) barriers; and 3) enablers. To formally assess pathways to implementing co-production and achieving impact, mechanisms were mapped against the five National Institute for Health and Care Research (NIHR) principles of co-production.

Results
Positive impacts were identified for individuals, the research and organisations involved. Common barriers to co-production, like representativeness and resource constraints were identified, alongside challenges specific to data-intensive research, such as balancing power-sharing with data access constraints. Key enablers included genuine power sharing, valuing diverse knowledge, and partnership working. Special considerations needed to support successful co-production in this context include extra effort to achieve inclusion and address support needs. Partnerships with voluntary and community organisations support an inclusive, trauma-informed approach.

Conclusion
This case study and evaluation can be utilised to support co-production with underserved groups in other data-intensive research contexts. Embedding co-production of data research with underserved groups will require changes to the broader research eco-system, including tailored guidance and resources, and funding partnerships rather than only pre-specified research projects.

Introduction

The last decade has seen vast expansion and infrastructure investment in routinely collected health and administrative data for research, with an emphasis on public benefit [1, 2]. This coincides with growing recognition that patient and public involvement (PPI) is essential for improving research quality and relevance, and to make research culture more inclusive [3]. The field of PPI is rapidly evolving, with increasing demand for more equitable approaches such as co-production [4]. Co-production intersects with PPI but goes further than engaging or consulting with the people relevant to a project. A defining feature is the sharing of power, with service users or members of the public working in equal partnership with researchers throughout the research process [5]. The burgeoning interest in co-production has seen a proliferation of general guidance and evaluation frameworks published [6–10]. However, there is limited evidence on the specific features needed to successfully co-produce research in the context of data-intensive research generally, and in particular with underserved groups [11, 12].

Specific challenges of integrating co-production in data-intensive research

The active and equal involvement of service users or members of the public required by co-production represents a fundamental challenge to conventional research culture and practices [5]. In the context of data-intensive research, such as that using linked administrative data or large biobank data, there are additional challenges to integrating co-production. Firstly, data-intensive research involves advanced statistical knowledge and analytical techniques. Explaining data-intensive research processes and outputs in accessible ways to lay audiences is not easy [11, 13] and may require intensive capacity building so that members of the public can participate in a meaningful way [14]. Secondly, because they are typically not involved in data collection, data scientists are often more distanced from the people whose lives they are researching than researchers in many other disciplines [2]. By implication, data scientists may be less likely to have some of the essential relational and facilitation skills that underpin co-production (e.g. facilitating workshops, consensus building) than their counterparts in qualitative or applied research. Thirdly, data-intensive research can often involve secondary analysis of routinely collected administrative datasets which are held in secure physical or digital environments known as Trusted Research Environments (TREs) that can only be accessed by approved researchers. Ideally, co-production occurs at all stages of the research process from inception through to dissemination [6], yet public access to the data analysis stage is unrealistic in this kind of data-intensive research. Perhaps as a result of these additional hurdles, public involvement in data-intensive research is less advanced than in other types of research [15].

Specific challenges of integrating co-production with underserved groups

Administrative data research, which often links population-wide routinely collected datasets, has the well-documented advantage of capturing information on underserved sections of the population [16]. While there is no simple or single definition of an underserved group [17], the term confers people or communities that have traditionally been marginalised or excluded in some way. Co-production places experiential knowledge derived from lived experience of a particular life event or health condition as equally valid and complementary to the knowledge of researchers [18]. While no one is completely defined by a single label or identity [19], lived experience may be synonymous with difficult life experiences, vulnerability or additional needs that can inhibit involvement in co-produced research [20]. These can include barriers related to: 1) accessibility due to language, living environment or physical illness or disability; 2) participation due to education level, employment, caring responsibilities or specific cultural barriers; and 3) past experience, where there is the potential to trigger distress or trauma when engaging in the research subject area [17]. For example, the voices of care experienced young people have been underrepresented in research examining their health and health care needs [21] and there can be care-related barriers to involvement such as frequent placement moves and challenges gaining parental or informed consent [22]. These obstacles, coupled with the inherent difficulties of involvement in data intensive research detailed above, suggest a new, specific approach is necessary to include these voices in data-intensive research.

Challenges evaluating co-production

There is no single standardised way of ‘doing’ co-production, and consequently it is operationalised by researchers in different ways [23]. While flexibility and adaptability to context are considered necessary components of co-production, this does pose challenges for evaluation, including defining success and understanding the necessary conditions [24, 25]. A prerequisite for evaluation is clarity about what constitutes ‘good’ co-production. Progress has been made in the UK, where the National Institute for Health and Care Research (NIHR) has published five key principles of co-produced research [6]. These are: 1) sharing of power; 2) including all perspectives and skills; 3) respecting and valuing the knowledge of all those working together on the research; 4) reciprocity; and 5) building and maintaining relationships. However, challenges remain in understanding how a co-production approach works in practice, and for whom in what circumstances.

One recommended strategy for supporting co-production is for practitioners to share examples of how it has been implemented, including clear and consistent reporting of context, processes and outcomes [23]. Co-produced qualitative and applied research studies have been evaluated against the NIHR principles of co-production, and features that deliver successful co-production identified [26, 27]. However, to date, no co-produced research using population-wide, linked administrative data has been appraised and evaluated in a similar way. There is merit in evaluating co-produced research using the NIHR framework because it provides a recognised standard, ensuring findings are directly relevant to existing guidance.

To support co-production in an area where evidence is currently lacking, this paper reports on the evaluation of co-production in data-intensive research delivered by the Administrative Data Research Centre Northern Ireland (ADRC NI), the charity Voice of Young People in Care (VOYPIC) and a group of care experienced young people. The research itself examined the association between care experience and a range of mental health and mortality outcomes.

The co-production pilot

The VOYPIC/ADRC NI partnership was conceived to provide dedicated mechanisms and support to overcome some of the additional barriers care experienced young people may face co-producing research. Care does not define young people, but it does mean they have to deal with the reasons that brought them into care, not living with their family, and living independently at an early age. The aim of the pilot was intentionally broad, namely to co-produce data-intensive research related to childhood social care and health and mortality outcomes with experts by experience (EBEs), i.e. care experienced young people. This allowed an iterative process that could evolve subject to the interests and preferences of the EBEs. The intention was to test possible entry points for co-production in the data research process, not deliver a pre-determined plan.

Between April 2022 and May 2024, a total of nine participatory workshops were facilitated by ADRC NI, held at a time and location chosen by the young people. VOYPIC facilitated recruitment of the EBE’s and provided practical and emotional support before, during and after each workshop. ADRC NI and VOPYIC held regular planning and review meetings, and VOYPIC led all activity and additional engagement with EBEs between workshops. Workshop content was sensitively planned and flexible in the context of what the young people had going in their lives, which was shared with facilitators either by the young people directly or by the partners in VOYPIC. Psychological/emotional support was provided by VOYPIC during and after each workshop given the potentially triggering content, which included discussions around mental health, self-harm and suicide. There was no renumeration for EBE’s due to budget constraints, although travel expenses were covered and food/refreshments provided. Details of each workshop and related outputs is provided in Supplementary Appendix 1.

Over the course of the pilot EBEs were involved in the following: setting the research agenda; specifying research questions; defining positive outcomes based on measures available in administrative data; interpreting results; dissemination; and evaluation. The co-production pilot also led to a paid 8-week internship for a care experienced young person as an ADRC NI Research Assistant in 2023 and 2024, however the reporting and evaluation of the internship programme is outside of the scope of this paper.

The aims of the evaluation were to: 1) identify if co-producing data-intensive research with care experienced young people (an underserved group) led to demonstrable impact; and 2) to identify the features or mechanisms that enabled or inhibited impact. We also sought to identify aspects where co-production of data-research with an underserved group is a special case. The evaluation addressed the specific research questions:

RQ1. What impact did the co-production pilot achieve?

RQ2. What barriers inhibit the co-production of data-intensive research with an underserved group?

RQ3. What specific enablers support the co-production of data-intensive research with an underserved group?

Methods

Patient and public involvement statement

VOYPIC and the young people were content creators of the evaluation, and co-authors and peer reviewers for the paper. Reporting is in accordance with the GRIPP2 checklist for reporting PPI [28] (Supplementary Appendix 2).

Participants

Participants were sampled purposively for experience of involvement in the co-production pilot from beginning to end (April 2022-August 2024). All staff involved in the pilot took part in the evaluation, comprising three ADRC NI research/public engagement staff (AM, SMcK and EN) and three VOYPIC staff (LK, BK and JI). During the pilot eleven EBEs were involved to varying degrees, however a core group of four EBEs (RB, MM, EI and AS) attended all nine workshops. Only these four young people took part in the evaluation as they were best positioned to reflect on the entire process. EBE’s that had participated in earlier workshops (up to and including workshop 6) were not invited to participate in the evaluation, as they had already opted not to continue their involvement in the pilot.

Design

A qualitative evaluation of a co-production pilot. The evaluation process was facilitated by ADRC NI and designed to capture the perspectives of 1) the EBEs, 2) VOYPIC and 3) ADRC NI research/public engagement staff. A participatory workshop held in March 2024 captured the EBE’s perspectives through facilitated dialogue. The workshop was facilitated jointly by VOYPIC staff (BK and JI) and ADRC NI staff (SMcK, AM, EN). Questions to guide semi-structured discussion were developed by ADRC NI and shared with the EBEs in advance (Supplementary Appendix 3).

A second workshop in May 2024 captured the ADRC NI perspective. The EBEs co-developed a series of questions to guide the dialogue and took turns to facilitate (Supplementary Appendix 4). At both workshops participants individually answered the facilitator’s questions but were also able to talk and interact with each other.

The VOYPIC perspective was captured in three 1-1 semi-structured interviews (April-May 2024) with an ADRC NI researcher (SMcK). The semi-structured interview topic guide was developed by SMcK, who drafted initial questions based on the overall aims of the evaluation. While the questions were not explicitly mapped against the NIHR co-production principles, they were designed to elicit reflections on areas central to those principles, such as mutual respect, power-sharing, and joint decision-making. Two colleagues (AM and EN) reviewed the draft schedule and their feedback informed further refinement of the questions and the addition of specific prompts to encourage deeper reflection (Supplementary Appendix 5). All workshops and interviews were audio recorded, transcribed verbatim and anonymised (SMcK).

Analysis

Data were analysed using hybrid deductive-inductive thematic analysis informed by Bingham’s five-phase model [29]. This model emphasises systematic data management, iterative clustering, collaborative refinement, and the application of theoretical frameworks. Initial deductive thematic analysis (whereby a-priori codes are applied to the data) was carried out jointly by one academic researcher (SMcK) and one EBE (AS). This joint approach was adopted to ensure appropriate support to AS. Training in qualitative data analysis was provided for AS who was employed as Research Assistant under the ADRC NI internship programme.

Three overarching topic codes were used to sort the data into categories based on the research questions: 1) impact, 2) barriers, and 3) enablers. To ensure the analysis stage was accessible and suitable for a young person we used a participatory method informed by qualitative open sorting and grouping techniques, using visual aids including flip charts and sticky notes [30, 31]. SMcK and AS initially reviewed the transcripts and selected excerpts that fit the three main topic codes. These data items were cut out and each stuck on a separate sticky note. Working together both sorters placed sticky notes onto sheets of wall-mounted flipchart paper against the three topic headings. Joint working enabled real-time discussion of interpretations and facilitated shared decision-making about the initial coding of data. Where differences in interpretation emerged, we resolved these through open discussion, reflection on the data, and mutual agreement.

After the initial deductive coding, in line with AS’s preference, SMcK completed the final stages of thematic analysis, with subsequent validation by AS. This inductive stage involved reflection on the data items under the “impact” topic, iterative clustering of related excerpts into three themes (individual, research and organisational impact), and subsequent clustering into sub-themes by grouping conceptually similar items together and assigning a label to each sub-theme. SMcK also applied the NIHR co-production principles [6] as an interpretive framework to data coded under the topic headings ‘barriers’ and ‘enablers’, systematically mapping data excerpts to these principles in line with the fifth phase of Bingham’s model [29], which advocates for theoretical integration at the final stage of analysis. This approach allowed us to identify factors that hinder or enable the delivery of co-production principles in practice [6]. Coded data were transferred into NVivo for storage and cross-validated by a third researcher (AM).

Results

RQ1. Impacts of co-producing data-intensive research with care experienced young people

The impacts of the co-production pilot were captured in three main themes: 1) individual-level; 2) research-level; and 3) organisation-level. These themes and related sub-themes with example quotes are shown in Tables 1–3. There was consensus among all participants (researchers, VOYPIC staff and the EBEs) that the co-production pilot delivered positive benefits for them as individuals (Table 1). All participants reported gaining new skills and knowledge and found the process rewarding (Table 1). Equally, all participants gained new perspectives and insights into each other’s worlds, reducing the gap between academia and lived experience:

Table 1: Individual-level impact achieved by the co-production pilot.

Individual-level impact sub-themes	Example quotes
New skills and knowledge	Now I kind of know what goes into research, it’s pretty cool. (EBE1)
	I learned more of how our data is collected and such. And like how research is done. (EBE2)
	It’s been a massive learning curve. Even our development as workers, and that’s obviously improved our work. (VOYPIC2)
	It’s not what we’re used to, so we’ve had to try new things, get out of our comfort zone, develop new skills. (ADRC1)
Rewarding	So this experience has been like, really really nice. (EBE2)
	When you see it all up there [storyboard of workshops on wall], it is pretty amazing. (EBE4)
	It’s been a really brilliant bit of work to be involved in. (VOYPIC1)
	It’s been rewarding for me, career changing. (ADRC2)
	I just think it’s one of the best things I’ve ever been part of. (ADRC3)
Changed perspectives	Before I felt like research was something so grandiose and above me. (EBE1)
	We come from a very practical side where the work is very different so, it gives a bit more insight. (VOYPIC3)
	It’s changed how I thought about doing research. (ADRC2)
	It becomes less abstract and more human (ADRC1)
Increased confidence	I’m getting more confident in speaking up and giving my opinions and stuff. (EBE3)
	Now I want to do research in the future. So that’s because I know it’s not as hard and scary as I thought it was. (EBE2)
	Because of doing this, the young people have seen it’s okay, they ask questions, it’s okay to get clarification and you’re not stupid. (VOYPIC3)

Table 2: Research-level impact achieved by the co-production pilot.

Research-level impact sub-themes	Example quotes
Greater relevance	It’s actually doing the right research, asking the right questions driven by you. (ADRC2)
	It’s taken the research in directions it wouldn’t necessarily have gone in. (ADRC1)
Incorporating lived experience perspective	There’s been a closing of the gap between research and people. (ADRC3)
	Some of the wording has been changed. (EBE4)
	You had mentioned a lot of comments .... that we hadn’t thought about because we just don’t know or have experience with that. (ADRC2)
	Some of your messaging has been really powerful, things like focus on the positive, and don’t stigmatise care experienced young people. (ADRC1)
	I think it’s good as well because it’s actually being done by people who have the experience and not just done by professionals. (EBE3)
	At the start I didn’t know nothing. But as the workshops went on, we were given enough information at the start to be able to get our input in, and actually put input and the findings an all too. (EBE4)
Increased reach	Our research has reached audiences that we could never have made it reach alone. (ADRC2)
	It’s much more likely to have an impact now in places where it needs to. (ADRC3)
	It’s much more accessible than before. (ADRC2)
	It gives it more credibility to say that this has been done in partnership ... it’s stronger and I think more people will sit up and listen. (ADRC1)

Table 3: Research-level impact achieved by the co-production pilot.

Organisation-level impact sub-themes	Example quotes
Amplifying lived experience	Hearing everybody else’s feedback as well [after EBE conference presentation], hearing the feedback and everyone else saying that they want to do what we’re doing. (EBE4)
	For us it’s very much about putting the voice of the young person at the heart of public discourse. So having that opportunity to do that in a field where perhaps we haven’t had that opportunity before has been a real win for us. (VOYPIC1)
	We have a strategic plan and one of those measures is empowering children and young people and another is influencing change and creating change. And I think this project really delivers. (VOYPIC2)
	We are rolling this approach out with other topics that we’re looking at, so other groups and other research. (ADRC2)
Positive partnerships	Being able to rely on each other has been really helpful and also how we cross-promote what each other’s doing. (VOYPIC1)
	It’s bringing us out of our inward focus to more of an outward focus and building those relationships. (ADRC1)
	The knowledge and what you are finding through the data to help inform some of our thinking and how we as an organisation are prioritising our work and our support for young people. (VOYPIC1)

Before if I’d thought about it, I would just see researchers as so far apart from me that I wouldn’t really be thinking about it, and I probably wouldn’t be thinking of it as a future career type of thing. But I think it just helps more to, like, solidify that it’s not like out of range or anything like that, and it’s not impossible to do. (EBE2)

The young people all described how their confidence had grown over time (Table 1), and this was also observed by VOYPIC staff:

Seeing the difference, you know, just as a VOYPIC worker in these young people who have gone into this two years ago and what they are like now, I think it’s in no small part to being involved in this group. (VOYPIC2)

Impacts were reported for several aspects of the research process, including the generation of community-relevant priorities and research questions, interpretation of results and enhanced dissemination (Table 2). Researchers felt the focus had shifted to issues more relevant to the EBEs:

When you work with data and you’re looking at numbers, sometimes you forget and get obsessed with the statistics and what’s going on and forget that there are real people behind these numbers. But we get to come and sit face to face with the real people behind the numbers and that’s helped with this research. (ADRC2)

A lived experience perspective was incorporated into academic outputs, from subtle changes like revised terminology around care experience, to more profound changes via a complete revision of the focus of the next research programme to exploring predictors of positive trajectories for care leavers (Table 2). One of most tangible areas of impact was increased reach, as EBEs developed lay-friendly versions of academic outputs for dissemination via institutional websites and social media accounts (Table 2).

Co-producing data-intensive research also impacted the organisations involved (Table 3). For VOYPIC the pilot was a further expression of their core aim of giving voice to care experienced young people. Both VOYPIC and ADRC NI benefited from building longer-term working relationships and a positive partnership. For ADRC NI the pilot has provided evidence to support roll-out of the approach to other projects with different underserved groups, as well as other data research centres hoping to replicate the approach.

RQ2. Barriers to co-producing data-intensive research with an underserved group

By applying the NIHR principles to the case study, the evaluation identified a range of factors that enabled or hindered the co-production of data-intensive research with an underserved group. A summary of the barriers identified is shown in Table 4. Several barriers are likely to be universal in co-produced research across contexts, for example achieving representativeness, time and resource pressures, and dealing with varying levels of understanding and ability. Other barriers will be more apparent in data-intensive research and/or research with an underserved group. Reconciling a power sharing agenda with data access restrictions was problematic. Interestingly, most of the young people did not mind being excluded from this stage and were content with shaping the direction of analysis and interpreting findings, with one stating “we obviously don’t have the right permissions for it (EBE4).” One of the young people, however, did feel excluded and expressed the view:

Table 4: Barriers to achieving the principles of co-production in practice mapped to the five NIHR principles of co-production. Note: no barriers related to the principle of reciprocity were identified.

Principle	Key feature, activity or mechanism
Share power and responsibility	Restrictions to data access & analysis stage
	Delays with project applications and researcher access to data
	Securing senior level support
Include all perspectives and skills	Explaining data research to non-data scientists
	Making complex statistical information accessible
	Reflecting diversity of care experience
	Retention of EBEs
	Membership refresh changed group dynamics
	Researcher skills gap e.g. facilitating workshops
Respect and value the knowledge of everyone	Takes time to develop trust and confidence
	Achieving balanced participation
	Differing levels of ability and understanding
Build and maintain relationships	Different organisational cultures
	Lack of time
	Lack of funding
	Workshop timetabling

It’s a bit frustrating you have to be qualified to be able to partake in research [data analysis] because you don’t have to be qualified to have knowledge and experience. (EBE1)

This requires creative solutions, and there was universal support for future access to a synthetic dataset, as a means of understanding the data available to inform topic prioritisation and analysis plans. Researchers found it challenging to balance the young people’s preferences and priorities with the actual, often prolonged, timelines of securing data sharing agreements, and gaining access to health and administrative data sets:

The challenge has been aligning what we decide to do together with delivery because of delays. (ADRC1)

The pilot also highlighted challenges explaining data research and statistical methods in an accessible way, especially at the start when researchers were less experienced in working with the young people:

It wasn’t really explained well to us so we all kind of came into this blind. (EBE1)

I was like I don’t think I’m gonna get this ... because I don’t have that type of background. (EBE4)

As quantitative researchers more used to working with numbers, the researchers were also anxious about the practical challenges of running participatory workshops and the potential to cause harm to vulnerable young people. This culture gap was also recognised by VOYPIC:

It’s academia and youth work coming together to do this ... they both have completely different outcomes, approaches and ways of working. (VOYPIC3)

One of the concerns I had was about our young people live chaotic lives at times and they can go through periods of great stability and then periods of great chaos. And sometimes that’s not necessarily understood outside our world. (VOYPIC1)

RQ3. Enablers of co-produced data-intensive research with an underserved group

The evaluation identified several enablers that participants felt were key to realising co-production principles and achieving impact (Table 5). As with the barriers, many are integral features of a co-production approach, regardless of type of research or setting. There was consensus that there was a genuine effort to share power and decision-making, and value different types of knowledge:

Table 5: Enablers that deliver the principles of co-production in practice mapped to the five NIHR principles of co-production.

Principle	Key feature, activity or mechanism
Share power and responsibility	Provide capacity building to upskill EBEs
	Act on input from EBEs
	Be flexible and adaptable
	Work at level and pace that EBEs are comfortable with
	Share decision making wherever possible
	Be transparent about any constraints or non-negotiables
	Collaboratively plan workshop structure and content
	Regularly update, recap and reflect on progress
Include all perspectives and skills	Provide practical and emotional support for EBEs
	Make information accessible, avoid lecturing or jargon
	Support EBEs to make informed decisions
	Use a variety of facilitation and group work techniques
	Include practical activities with tangible outputs
	Encourage and structure opportunity for questions
	Venue and time chosen by EBEs
Respect and value the knowledge of everyone	Have an open-minded attitude
	Listen to and value opinions of others
	Place equal value of different types of knowledge
	Positive behaviours based on respect and equality
	Genuine intention and desire to co-produce
Reciprocity	Celebrate success and recognise achievements
	Acknowledge contributions of EBEs in research outputs
Build and maintain relationships	Partnerships between research and charitable/advocacy
	organisations
	Regular planning and review meetings
	Allow time to develop relationships/build trust
	Incorporate opportunities to get know each other
	Have a mixture of work and fun in workshops

I felt like, I guess, whatever I would say would actually be listened to. (EBE1)

I always got the impression from how you spoke at the start of each session that we were included in everything. (EBE2)

Like we were always told we don’t have to say yes to anything, we can always say no. (EBE3)

Yeah like we’re all equals. It doesn’t matter if we’re a researcher or not, we’re still getting our say. And we are seeing our say being put into it as well, with a lot of the papers and the wording as well. (EBE4)

I feel this is being left to the group to kind of guide, and although you [the researchers] have had your targets .... you haven’t pushed it down a route that you want it to go. (VOYPIC3)

It’s feeling like they’re important, feeling like their voice and their opinion really matters. (VOYPIC2)

Perhaps the most important enabler was the partnership between ADRC NI and VOYPIC, which bridged the gap between academic research and care experienced young people:

Being able to have those two very different worlds come together. I think we’ve put a lot of work into building that and making sure that it does run smoothly. (VOYPIC1)

We’re all really open to the fact that we can learn from each other ... willingness to learn and accept our own expertise in our own lane and go to each other and say, you know, this isn’t my area, what do you think about this? (VOYPIC2)

The ADRC NI/VOYPIC partnership catalysed positive impacts in a number of ways. The young people all acknowledged that they would not have initially attended without encouragement from VOYPIC or remained involved without their ongoing support. Because the young people trusted VOYPIC, they took a leap of faith:

So it wasn’t “we want you to go to this,” it was “we want you to come with us to this.” So it was bringing the young people to the workshops and having them as part of the VOYPIC team that was going to it. I think that’s how we got them through the door. (VOYPIC1)

VOYPIC is an organisation that has care experienced staff and on this particular project the staff working on it are care experienced ... so I think that’s another level. (VOYPIC3)

VOYPIC routinely advised the research team on workshop content and structure, and how to make complex information accessible to young people:

It’s not easy to digest, so working together we have developed different skills and different tools and different ways to present the information. (VOYPIC3)

Partnership working also provided a means to address some of ethical complexities of co-producing data research on difficult topics. All participants identified that the emotional support for the young people provided by VOYPIC was an essential feature:

Knowing you have somebody there [VOYPIC] makes it a whole lot better and easier. Knowing, alright, you see something that you know it’s gonna be hard on you. You can sit down and listen to it, or you can go out and you can focus on something else and one of the staff members will go out and distract you from it. (EBE4)

They [VOYPIC] know the whole info, and we wouldn’t have to explain the whole thing. We’d just be like, this thing, I just don’t feel comfortable. (EBE2)

Discussion

This study reports the impact of a co-production pilot and identifies key features that inhibit or facilitate delivery of the five principles of co-production in practice. It adds to the existing evidence base in two ways. Firstly, it describes and evaluates the process of co-producing data-intensive research with care experienced young people. Secondly, it identifies the special considerations needed to support successful co-production in this context.

Impact of the co-production pilot

The case study and evaluation show that co-production of data-intensive research with an underserved group is feasible and engenders positive impact. This co-production pilot resulted in: 1) individual-level benefits for the EBE’s, the charity partner and academic researchers alike, bridging the gap between academic data researchers and the people behind the numbers they are exploring; 2) research benefits increasing the accessibility and reach of academic research and its findings, enhancing interpretation of results, developing relevant research questions and sharing knowledge; and 3) organisational benefits with partner organisations delivering on strategic aims to widen participation and build multi-professional relationships.

The findings align with other reports that the impacts of co-produced research are often more intangible and diffuse, such as changes in human relationships, culture shifts, and capacity building [32–34]. As mutual benefit is a core principle of co-production, it should also be key feature of its evaluation [35] and this case study shows the impact on individuals is at least as pertinent as the impact on the research. A universal reflection from the EBEs was that they felt heard and were given a voice in the data research process. The pilot shifted the inherent power balance in research and empowered a group of young people who have not always felt empowered. This does not mean that the co-production process was free of hierarchies, and there were instances where the researchers had to take decisions to put parameters around options offered to EBEs to comply with institutional and research programme constraints. However, transparency and open communication around these issues can be an act of shifting power in itself [5], and in this case supported the realisation of the five principles of co-production.

No negative impacts were identified, although this may be a matter of attitude and perception. For example, all participants found the process challenging at different stages and in different aspects (e.g. emotional, practical, logistical) but did not conceptualise these as negative impacts.

Barriers and enablers - is co-production of data-intensive research with underserved groups a special case?

Most examples of co-produced research are based on qualitative and applied research, and there is much to be learned from testing its application in different settings [25]. Co-production is inherently aspirational, and many of the barriers and enablers to co-produced research identified in this study are commonly reported in other types of research. For example, time constraints, inadequate resources, meeting learning needs (of researchers and EBEs) and reflecting diversity in group membership are commonly cited challenges [5, 11, 23, 36]. Similarly, many of the solutions are not unique to the context of this case study, and features such as transparency about constraints, flexibility, and making information accessible should always be present in co-produced research. However, we identified special considerations when trying to put co-production ideals into practice as some of the ordinary challenges are more acute in data-intensive research, and for EBEs with lived experience of adversity or trauma.

A distinguishing feature of co-production is that research ‘subjects’ share power and decision-making throughout the entire research process [19], although this requirement is contested and can prove challenging in practice [33]. Within this pilot EBEs were excluded from the data analysis stages. Researchers should not assume that EBEs do (or don’t) want to be involved at every stage of the research process. While there should be opportunity for EBEs to attain skills in statistical techniques, it is important to avoid co-production of data-intensive research becoming an elitist endeavour. Lived experience experts do not need to become researchers to make a valuable contribution [37]. This study does show that while it may not be feasible to directly involve EBEs as equal partners across all stages of the data research process, it is feasible within stages. There is growing recognition of the need for pragmatism, and that often only some parts of a project can be co-produced [27]. The complexity of data science and statistics requires extra effort to achieve the principle of inclusion, but this complexity is precisely why co-production is so helpful for translating findings for a broader audience [11].

An additional dimension is managing the emotional and practical support needs of potentially vulnerable communities such as care experienced young people. To be inclusive, co-production processes must consider support measures that enable participation. Vulnerability can be situational and nuanced [38] and we do not wish to generalise about whole groups or communities, but avoidance of harm is a key consideration. In this case study, partnership between the university and a charitable organisation ensured the workshops and wrap-around support were bespoke to the needs of the young people involved. This paper advocates for co-producing data-intensive research with underserved groups in partnership with voluntary or community organisations representing these groups who can bring established relationships and expertise on participatory, trauma-informed practice. This comes with the need for caution about over-asking of partner’s time and resources without proper consideration of resources or clear benefits for the community in question [34].

Future directions to support co-produced data-intensive research with underserved groups

The underpinning principles of co-production have been clearly articulated, although these are constantly being refined and re-invented, perhaps because a universally accepted definition of co-production remains elusive [39]. Gaps remain in understanding how to translate co-production into practice. Some commentators note there is already a confusing array of co-production-related guidance, and the future emphasis should be on financial and practical support [23]. There is limited value in generating multiple iterations of similar and overlapping general guidance, but there is a need for guidance, training and practical toolkits for practitioners that account for the different content and context of data-intensive research. Our findings also highlight the importance of relational and facilitation skills in co-production. These are not secondary to technical expertise but are core competencies essential for impactful research. Balancing technical and interpersonal skills is vital, particularly when working with underserved groups or on sensitive topics. This has implications for team composition, training, and leadership in data-intensive research, and reinforces the importance of working with partner organisations.

A shift from pockets of good practice at the research project level to co-production as ‘business as usual’ also requires advances in the broader data research ecosystem. Crucially, research infrastructure and funding need to evolve to support long-term partnerships that facilitate co-production [23, 40]. Organisations that facilitate and fund data-intensive research have a key role to play in commissioning, hosting and sign-posting data scientists towards such guidance and resources. New partnerships between organisations working with data and statistics such as the Public Engagement in Data Research Initiative (PEDRI) in the UK, have potential to progress public involvement in data research and fill resource and information gaps [41]. NIHR already hosts resources such as lay glossaries of common statistical terms, accessible animations and case studies relevant to data-intensive research [42–44].

Strengths and limitations of our approach

This work provides a unique case study of co-produced data research with an underserved group. To bridge the theory-practice gap we mapped the key features that either inhibit or enable co-produced research and pathways to impact against the NIHR five principles of co-production [6]. The robustness of the findings was considered using Nowell et al.’s criteria for thematic analysis [45]. Collaborative analysis and member checks supported credibility by ensuring that both academic and lived experience perspectives shaped interpretation and write-up. Detailed description of context and methods is provided to aid transferability, and although limited by the case study’s context-specific nature, we provide detailed methodological description to aid assessment of relevance elsewhere. These results warrant testing in other contexts, although the findings are not intended to be prescriptive as co-production is always sensitive to the context in which the research is being co-produced [39]. Dependability is supported by a transparent, systematic analysis process and audit trail, and confirmability is strengthened through ongoing checks for bias in collaborative discussions and manuscript versions.

The evaluation itself was co-produced and included different perspectives, although the small sample and purposive sampling method may have resulted in bias. Involving only EBEs that attended every workshop may have biased the findings, as by definition they would have ‘voted with their feet’ if the experience had not been a positive one overall. We do not have data on reasons why the EBEs that left, did so. Retention of care experienced young people in co-produced research is a pervasive issue [46], and in future we will attempt to follow up with young people that drop out to understand why.

Conclusion

This case study and evaluation can be utilised to inform co-production with underserved groups in other data-intensive research contexts. It challenges the misguided perception that co-production is too complex or difficult in a data-intensive context or with particular groups to be achievable. The nature of data-intensive research requires deliberate effort to address the difficulty of inclusion, but doing so enriches the research and enhances its ability to make a positive impact.

Key to the success of this pilot was the core relationship between the academic researchers and the charitable organisation with expertise in participatory engagement with the underserved group at the heart of the research. These partnerships need to be properly resourced to realise mutual benefit. This also demonstrates a commitment to co-production that goes beyond tokenism and allows learning and expertise to develop and be shared. Embedding co-production of data research with underserved groups will require changes to the broader research eco-system, including tailored guidance and resources and funding partnerships rather than only pre-specified research projects.

Finally, a core message from the participants in this case study is not to attempt co-production for co-production’s sake, or without careful consideration of support for EBEs. It is essential to reflect on purpose and authenticity, and it won’t be right or feasible for every research project [8].

Acknowledgments

The authors would like to acknowledge Professor Dermot O’Reilly’s significant role in the development of the research programme on children’s social care in Northern Ireland. Dermot sadly died in October 2023. He was a dear friend and Director of the Administrative Data Research Centre Northern Ireland (ADRC NI) since 2014. He will be missed; his work on health and social inequalities will continue in his memory.

Funding statement

The data research was supported by the UKRI’s Administrative Data Research Centre Northern Ireland (ES/S00744X/1 & ESW010240/1) and Health Data Research UK/ DATAMIND (MR/W014386/1). The VOYPIC partnership was supported by the ESRC Impact Acceleration Account at Queen’s University (ES/X004600/1). For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any Author Accepted Manuscript version arising from this submission.

Conflict of interests statement

None declared.

Ethics statement

Ethical approval was not required as the information cited in this paper was gathered as part of an evaluation process of ongoing co-production work.

Data availability statement

Evaluation data are not available for sharing as participants have not provided consent for their data to be shared publicly.

Author contributions

Conceptualisation and data curation, all authors. Data analysis SMcK, AS and AM. All authors were involved in drafting the manuscript or revising it critically for important intellectual content and approved the final manuscript.

Supplementary appendices

Supplementary Appendix 1: Details of participatory workshops to co-produce data-intensive research with care experienced young people.

Supplementary Appendix 2: Patient and Public Involvement (PPI) in the evaluation of the co-production pilot using the GRIPP2 Checklist (short form).

Supplementary Appendix 3: Experts by experience perspective – workshop discussion prompts.

Supplementary Appendix 4: ADRC NI staff perspective – workshop discussion prompts.

Supplementary Appendix 5: Voice of Young People in Care (VOYPIC) staff perspective – semi-structured interview questions.

References

1. Mc Grath-Lone L, Jay MA, Blackburn R, Gordon E, Zylbersztejn A, Wiljaars L, et al. What makes administrative data “research-ready”? A systematic review and thematic analysis of published literature. Int. J. Popul. Data Sci. 2022;7(1). 10.23889/ijpds.v7i1.1718

   10.23889/ijpds.v7i1.1718

2. Aitken M, Tully MP, Porteous C, Denegri S, Cunningham-Burley S, Banner N, et al. Consensus statement on public involvement and engagement with data intensive health research. Int. J. Popul. Data Sci. 2019;4(1). 10.23889/ijpds.v4i1.586

   10.23889/ijpds.v4i1.586

3. Medical Research Council. Looking forward: Working with the Medical Research Council towards a public involvement strategy [Internet]. 2022 [cited 31.01.25]; Available from: https://www.ukri.org/publications/public-involvement-review/.

4. Paylor J, McKevitt C. The Possibilities and Limits of “Co-producing” Research. Front. Sociol. 2019;4:23. 10.3389/fsoc.2019.00023

   10.3389/fsoc.2019.00023

5. Farr M, Davies P, Andrews H, Bagnall D, Brangan E, Davies R. Co-producing knowledge in health and social care research: reflections on the challenges and ways to enable more equal relationships. Humanit. Soc. Sci. Commun. 2021;8:1-7. 10.1057/s41599-021-00782-1

   10.1057/s41599-021-00782-1

6. NIHR. Guidance on co-producing a research project [Internet]. 2021 [cited 31.01.25]; Available from: https://www.learningforinvolvement.org.uk/content/resource/nihr-guidance-on-co-producing-a-research-project/.

7. Farr M, Davies R, Davies P, Bagnall D, Brangan E, Andrews H. A map of resources for co-producing research in health and social care [Internet]. 2020 [cited 31.01.25];1. Available from: https://arc-w.nihr.ac.uk/Wordpress/wp-content/uploads/2020/05/Map-of-resources-Web-version-v1.2.pdf.

8. Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Heal. Expect. 2019;22:785–801. 10.1111/hex.12888

   10.1111/hex.12888

9. Popay J, Collins M, PiiAF Study Group. The Public Involvement Impact Assessment Framework Guidance [Internet]. Univ. Lancaster, Liverpool Exet. 2014 [cited 24.09.25];1–8. Available from: https://www.bangor.ac.uk/health-sciences/documents/piiaf-guidance-jan14.pdf.

10. Michele Kok. Guidance Document: Evaluating public involvement in research. Univ. West Engl. Bristol 2018;44:1–18.

11. Vinnicombe S, Bianchim MS, Noyes J. A review of reviews exploring patient and public involvement in population health research and development of tools containing best practice guidance. BMC Public Health 2023;23:1–14. 10.1186/s12889-023-15937-9

    10.1186/s12889-023-15937-9

12. Teodorowski P, Rodgers SE, Fleming K, Tahir N, Ahmed S, Frith L. Exploring how to improve the involvement of Polish and South Asian communities around big data research. A qualitative study using COM-B model. Int. J. Popul. Data Sci. 2023;8(1). 10.23889/ijpds.v8i1.2130

    10.23889/ijpds.v8i1.2130

13. Jewell A, Pritchard M, Barrett K, Green P, Markham S, McKenzie S, et al. The maudsley biomedical research centre (Brc) data linkage service user and carer advisory group: Creating and sustaining a successful patient and public involvement group to guide research in a complex area. Res. Involv. Engagem. 2019;5:1–10. 10.1186/s40900-019-0152-4

    10.1186/s40900-019-0152-4

14. van Voorn GAK, Vemer P, Hamerlijnck D, Ramos IC, Teunissen GJ, Al M, et al. The Missing Stakeholder Group: Why Patients Should be Involved in Health Economic Modelling. Appl. Health Econ. Health Policy 2016;14:129–33. 10.1007/s40258-015-0200-7

    10.1007/s40258-015-0200-7

15. Johnson H, Davies JM, Leniz J, Chukwusa E, Markham S, Sleeman KE. Opportunities for public involvement in big data research in palliative and end-of-life care. Palliat. Med. 2021;35:1724–6. 10.1007/s40258-015-0200-7

    10.1007/s40258-015-0200-7

16. Allnatt G, Elliott M, Scourfield J, Lee A, Griffiths LJ. Use of Linked Administrative Children’s Social Care Data for Research: A Scoping Review of Existing UK Studies. Br. J. Soc. Work 2022;52:3923–44. 10.1093/bjsw/bcac049

    10.1093/bjsw/bcac049

17. NIHR. Improving inclusion of underserved groups in clinical research: Guidance from the NIHR-INCLUDE project [Internet]. 2020 [cited 24.09.25]; Available from: https://www.learningforinvolvement.org.uk/content/resource/nihr-include-guidance-general/.

18. Karazivan P, Dumez V, Flora L, Pomey MP, Del Grande C, Ghadiri DP, et al. The patient-as-partner approach in health care: A conceptual framework for a necessary transition. Acad. Med. 2015;90:437–41. 10.1097/ACM.0000000000000603

    10.1097/ACM.0000000000000603

19. Smith B, Williams O, Bone L. Co-production: A resource to guide co-producing research in the sport, exercise, and health sciences. Qual. Res. Sport. Exerc. Heal. 2023;15:159–87. 10.1080/2159676X.2022.2052946

    10.1080/2159676X.2022.2052946

20. Gray B. Lived experience research: a guide to the barriers and facilitators from a service user’s perspective. Psychosis 2024;16:98–102. 10.1093/schbul/sbac186

    10.1093/schbul/sbac186

21. Smales M, Savaglio M, Webster S, Skouteris H, Pizzirani B, O’Donnell R, et al. Are the voices of young people living in out-of-home care represented in research examining their health?: A systematic review of the literature. Child. Youth Serv. Rev. 2020;113:104966. 10.1016/j.childyouth.2020.104966

    10.1016/j.childyouth.2020.104966

22. Rosenfeld AA, Pilowsky DJ, Fine P, Thorpe M, Fein E, Simms MD, et al. Foster Care: An Update. J. Am. Acad. Child Adolesc. Psychiatry 1997;36:448–57. 10.1097/00004583-199704000-00006

    10.1097/00004583-199704000-00006

23. Smith H, Budworth L, Grindey C, Hague I, Hamer N, Kislov R, et al. Co-production practice and future research priorities in United Kingdom-funded applied health research: a scoping review. Heal. Res. Policy Syst. 2022;20:1–43. 10.1186/s12961-022-00838-x

    10.1186/s12961-022-00838-x

24. Durose C, Richardson L, Perry B. Craft metrics to measure co-production. Nat. Comment 2018;562:32–3. 10.1038/d41586-018-06860-w

    10.1038/d41586-018-06860-w

25. Redman S, Greenhalgh T, Adedokun L, Staniszewska S, Denegri S. Co-production of knowledge: The future. BMJ 2021;372:16–7. 10.1136/bmj.n434

    10.1136/bmj.n434

26. Dewa LH, Lawrence-Jones A, Crandell C, Jaques J, Pickles K, Lavelle M, et al. Reflections, impact and recommendations of a co-produced qualitative study with young people who have experience of mental health difficulties. Heal. Expect. 2021;24:134–46. 10.1111/hex.13088

    10.1111/hex.13088

27. NIHR. Co-production in action: number one [Internet]. 2021 [cited 31.01.25]; Available from: https://www.learningforinvolvement.org.uk/content/resource/nihr-co-production-in-action-number-one/.

28. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: Tools to improve reporting of patient and public involvement in research. BMJ 2017;358. 10.1136/bmj.j3453.

    10.1136/bmj.j3453

29. Bingham AJ. From Data Management to Actionable Findings: A Five-Phase Process of Qualitative Data Analysis. Int. J. Qual. Methods 2023;22:1–11. 10.1177/16094069231183620

    10.1177/16094069231183620

30. Jackson SF. A participatory group process to analyze qualitative data. Prog. Community Health Partnersh. 2008;2:161–70. https://dx.doi.org/10.1353/cpr.0.0010

31. Best P, Badham J, McConnell T, Hunter RF. Participatory theme elicitation: open card sorting for user led qualitative data analysis. Int. J. Soc. Res. Methodol. 2022;25:213–31. 10.1080/13645579.2021.1876616

    10.1080/13645579.2021.1876616

32. Russell J, Fudge N, Greenhalgh T. The impact of public involvement in health research: what are we measuring? Why are we measuring it? Should we stop measuring it? Res. Involv. Engagem. 2020;6:4–11. 10.1186/s40900-020-00239-w

    10.1186/s40900-020-00239-w

33. Beckett K, Farr M, Kothari A, Wye L, Le May A. Embracing complexity and uncertainty to create impact: Exploring the processes and transformative potential of co-produced research through development of a social impact model. Heal. Res. Policy Syst. 2018;16:1–18. 10.1186/s12961-018-0375-0

    10.1186/s12961-018-0375-0

34. Campbell H, Vanderhoven D. N8/ESRC Research Programme Knowledge That Matters: Realising the Potential of Co-Production [Internet]. 2016 [cited 31.01.25]; Available from: https://www.n8research.org.uk/view/5163/Final-Report-Co-Production-2016-01-20.pdf.

35. Boivin A, Richards T, Forsythe L, Grégoire A, L’Espérance A, Abelson J, et al. Evaluating patient and public involvement in research. BMJ. 2018;363. 10.1136/bmj.k5147

    10.1136/bmj.k5147

36. NIHR. Co-production in action: number three [Internet]. 2020 [24.09.25]; Available from: https://www.learningforinvolvement.org.uk/content/resource/co-production-in-action-number-three/.

37. Staley K. There is no paradox with PPI in research. J. Med. Ethics 2013;39:186–7. 10.1136/medethics-2012-100512

    10.1136/medethics-2012-100512

38. Lambert N, Carr S. ‘Outside the Original Remit’: Co-production in UK mental health research, lessons from the field. Int. J. Ment. Health Nurs. 2018;27 (4):1273–81. 10.1111/inm.12499

    10.1111/inm.12499

39. O’Mara-Eves A, Laidlaw L, Vigurs C, Candy B, Collis A, Kneale D. The Value of Co-Production Research Project: A Rapid Critical Review of the Evidence Full Report. 2022. https://cdn.prod.website-files.com/5ffee76a01a63b6b7213780c/63595629676ab598f0e6bf44_ValueCoPro_RapidReviewFull31Oct22.pdf.

40. Beran D, Pesantes MA, Berghusen MC, Hennig BJ, Jacobi J, Lazo-Porras M, et al. Rethinking research processes to strengthen co-production in low and middle income countries. BMJ 2021;372:1–4. 10.1136/bmj.m4785

    10.1136/bmj.m4785

41. Public Engagement in Data Research Inititative (PEDRI) [Internet]. 2024 [cited 31.01.25]; Available from: https://www.pedri.org.uk/.

42. Worboys HM, Broomfield J, Smith A, Stannard R, Tyrer F, Vounzoulaki E, et al. Incorporation of patient and public involvement in statistical methodology research: development of an animation. Res. Involv. Engagem. 2023;9:1–6. 10.1186/s40900-023-00513-7

    10.1186/s40900-023-00513-7

43. NIHR. PPI-SMART: PPI for Statistical Methodology & Research Techniques [Internet]. 2025 [cited 31.01.25]. Available from: https://leicesterbrc.nihr.ac.uk/ppismart/ppismart-case-studies/.

44. NIHR. PPI-SMART Case study 2 – Public contributors help researchers build better statistical models [Internet]. 2025 [cited 31.01.25]; Available from: https://leicesterbrc.nihr.ac.uk/ppismart/ppismart-case-studies/.

45. Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods. 16(1), 1–13. 10.1177/1609406917733847

    10.1177/1609406917733847

46. Kelly B, Friel S, McShane T, Pinkerton J, Gilligan E. “I haven’t read it, I’ve lived it!” The benefits and challenges of peer research with young people leaving care. Qual. Soc. Work 2020;19:108–24. 10.1177/1473325018800370

    10.1177/1473325018800370