Influencing policy and practice is work in progress

Abstract

Focusing on policy impact without a functioning regional devolved government required Administrative Data Research Centre Northern Ireland (ADRC NI) researchers and staff to develop new ways of influencing policy and practice in Northern Ireland. The development of a co-production model with policymakers, service providers and third sector representatives has embedded pathways to impact within research programmes, to great success. This paper describes the efforts of the ADRC NI to influence policy and practice, in Northern Ireland in challenging political circumstances, outlining the context, approach, processes, strengths, challenges and lessons for the future.

Introduction

ADRC NI is a research partnership of Queen’s University Belfast and Ulster University and is funded by the Economic and Social Research Council (ESRC), part of UK Research and Innovation (UKRI), with additional support from the Research and Development Division of Health and Social Care Northern Ireland (HSCR&D). ADRC NI along with the Northern Ireland Research and Statistics Agency (NISRA) are collectively referred to as Administrative Data Research Northern Ireland (ADR NI), which, with sister centres in England, Scotland and Wales, is part of a UK-wide partnership called ADR UK, coordinated by a Strategic Hub [1]. ADR UK’s ethos is to facilitate safe and secure access to linked administrative datasets for research purposes for public benefit and to inform policy decisions and service provision [2]. This mission is shared by ADRC NI, whose objective is to develop policy-led ethical research which provides evidentiary tools to both government and communities to affect policy response.

The evolution of the original Administrative Data Research Network (ADRN) into ADR UK came with a renewed focus on research that would impact on policy, legislation and services. This refocusing required a paradigm shift in the way research programmes were conceived of and delivered; i.e. away from largely curiosity-based research for academic interest and benefit and towards one that necessitated a more direct engagement and involvement of the government departments and data owners who would benefit from better evidence generated by administrative data research, and incorporate this evidence into their policymaking and service provision. This change was, to some extent, dictated by an increasing recognition of the significant burden for most host government departments in the extraction and processing of subsets of their administrative data, as well as supporting the researchers through the provision of expert knowledge and metadata, and the onerous governance framework for data transfers. Aligning the academic and policy agenda increased the likelihood of successful data acquisition, the prerequisite for research success and subsequent research impact.

However, this transition has not been an easy or straightforward experience, but the lessons learned have helped form our current approach, which has brought with it many successes as well as further learning and development, which will be detailed below.

The purpose of this paper is to illustrate the unique approach taken by ADRC NI in seeking to impact policy and practice more robustly, its successes and challenges, and key learning for researchers and impact and engagement professionals moving forward.

Background

The road to policy-engaged research in Northern Ireland has not been smooth. In January 2017 the devolved government at the Stormont Assembly collapsed and devolution was not restored until January 2020. Trying to engage with and influence policy during this vacuum was difficult for the ADRC NI, and our current method of embedding engagement to achieve impact was developed in response to this challenging environment.

Approach

Without a functioning government ADRC NI was forced to reconsider how best to influence policymakers in order to secure research impact. While we did get buy-in in principle at a very senior civil service level, this was not generally sufficient to initiate change, which required a more concerted approach to senior middle management within individual departments. We did this by developing an approach that centres engagement and involvement in the impact process; both with departmental policymakers and service providers, and with voluntary and community groups whose longstanding work on areas of research interest and existing policy and governmental relationships helped us gain access to policy circles. The primary mode for this stakeholder-centric engagement to impact pipeline is the project Steering Committee encompassing a range of key stakeholders, but this is also underlined by a range of broader principles of engagement that have enabled our research to have a wider and deeper impact on policy and practice.

The extent to which ADRC researchers can influence government policy or practice is often largely dependent upon the willingness of departments to engage with those researchers, and sometimes it has been very difficult to form a suitable working relationship. In our experience departments tend towards a distinctive culture that determines their willingness to form working partnerships with the academic community and for the wider access to their administrative data. Some departments and data owners are very facilitating and very welcoming of external (academic) input, but others, at the other end of the spectrum, are more cautious and less open to collaborative research, even when this is at no cost to themselves. This longstanding reticence has not been mitigated by the introduction of the Digital Economy Act in 2017, which was introduced specifically as “…a framework for sharing personal data, for defined purposes across specific parts of the public sector” [3]. These differences are to some extent driven by perceived sensitivities of the data, and in the UK, this can translate to obstacles when attempting to explore and understand socioeconomic issues with potential political ramifications. The reticence of some departments to make their individual-level administrative data available has thus meant that is has been almost impossible for social scientists here to undertake poverty-related research and therefore to monitor and understand the impact of major upheavals such as welfare reform [4] or Brexit [5] or the long period of austerity in 2010-2020 [6]. These differences in departmental culture pre-date the pandemic but have become more apparent during the recent urgent need for data on the effects of the pandemic on the population.

Departmental culture around data-sharing has critical influence on access to data as well as the ability to articulate research needs, so that data-focussed research has become more mature in some areas than in others, and the research findings often have a broader reach amongst those data custodians and policymakers that have championed the wider use of their data for research and service evaluation.

The degree to which departments can articulate their research needs also varies. In Northern Ireland, policy and legislation have shifted in recent years to a focus on Outcomes-Based Accountability, which in turn has produced specific outcomes targeted to the Programme for Government, agreed by the Executive for each Assembly mandate [7]. These targets and outcomes will be the main focus of departmental work. Some will connect these to a research agenda, and some will not. Some departments have a strong internal tradition of research and may have specified Areas of Research Interest signalling the major themes and particular studies they wish to see undertaken to assist them in reaching their targeted outcomes. Other departments have more limited experience and it may take a few worked examples before they appreciate the utility of research and become more proactive in the generation of new research programmes, and in participating in the co-production of data research.

Another overarching difficulty can be a silo mentality of departments who only want research in areas that they influence and for the budgets they control. This challenges a linked data and life-course approach, which draws its strength from investigating and demonstrating the cross-cutting and longitudinal nature of social factors. However, this attitude is waning as evidenced by the development of large-scale linked datasets, such as the Longitudinal Educational Outcomes (LEO) database in England and Wales [8], and the soon to be realised Northern Ireland version of the same, to understand the relationship between educational attainment and earnings and employment.

Steering committees

The central way ADRC NI has approached the difficulties in attaining policy impact during the absence of devolution has been to establish project-specific steering committees (or advisory groups). “The ADRC NI co-production model focuses on representation and participation by diverse stakeholders on Steering Committees to influence research priorities from project initiation through to the interpretation and contextualisation of research. This approach recognises multiplicity of knowledge beyond the academic and its contribution as an adjunct to the understanding of administrative data. Steering Committee representatives are drawn from government departments (as data custodians and policymakers), practitioners or service providers, VCSE organisations (as domain experts), and representation from Northern Ireland Commissioners’ offices [9].” The deliberate inclusion of these diverse voices and needs in the research process, and the successes achieved with the model, demonstrate the critical utility of engaging, particularly policymakers, in the development, design and delivery of research that speaks to policy needs and subsequently achieves impact.

Steering committee composition

Each ADRC NI research project is required to have a constituted steering or advisory committee or group, with a varied membership comprised of at least one departmental representative and one community representative, in addition to the research team. Steering committees typically meet twice yearly, though more frequent meetings may be called where there are issues around data access, analysis, research questions and direction, or knowledge exchange event organisation, to be addressed by the entire Committee. Choice of departmental representation is critical; the ideal candidate will have sufficient authority to facilitate data access and the requisite seniority to ensure that research findings have a high likelihood of being placed on the desks of the apposite policymakers.

Including non-governmental organisations (NGOs) as well as data subjects with lived experience of the issues at hand (experts by experience) also helps to elucidate not only the data and initial findings, but the translation of these findings into useable evidence and a positive policy intervention for communities. Departments, NGOs and Voluntary, Community and Social Enterprises (VCSEs) are often accustomed to working together and have long experience in doing so, and departments in many cases place high value on the views and experiences of those coming from the third sector. This dynamic can help shift attitudes to data sharing and research, when valued critical friends and partners echo the research findings as well as the calls from data-focused researchers for more access to data. The combination of domain experts in VCSE organisations, policy experts from government departments, and data experts within academia, means research exploration is more robust, data safeguarded and exploited to its full potential, and that potential translated into the strongest possible intervention for the people whose data this is: the public.

Potential tensions

While the relationships between VCSE groups and governmental policymakers is mentioned above as a potential positive dynamic and boost toward achieving impact, this is not always the case. Some departments are more hesitant about participating in committees with community or NGO representation, or with specific NGOs who have perhaps been lobbying them on certain issues for some time. These tensions must be managed carefully in order to preserve existing relationships for the researcher and research centre in question with both the policymaker and the VCSE group, while not compromising the integrity or independence of the research. At times difficult choices must be made, and this is where the support of an experienced engagement and impact professional is crucial.

Another important aspect of planning for impact is recognising the fundamental differences between the timescales and needs of the stakeholders and especially between research and policy spheres [10]. While academia is not renowned for its rapid turnaround, policy development and change can be even more protracted and yet often also requires ‘quick and dirty’ responses to urgent departmental questions. Furthermore, academia is motivated and rewarded by publications of new findings in peer-reviewed journals whilst policy is sometimes more concerned with reaffirming previous established findings using local data, which has extra potency for local policy making. To address this, we have employed a dual publication model with Data Insights made available with initial headline findings which are useful to policymakers and VCSE groups, while not prejudicing any forthcoming academic publications.

We have also found from working with policymakers that direct impact on the development of policy and strategy, including citations in the same, are not the only way to influence policymakers and data owners through Steering Committees [11]. Indeed, a large part of engaging directly with policymakers in the design and delivery of research is to underscore the utility of sharing and using data to multiple aspects of their work, including assessing and reporting on the ‘health’ of the datasets, understanding how the data can be combined in a resource-light way to answer questions that previously required dedicated funding to assess [12].

The exchanges at Steering Committees can have other benefits outside those directly related to policy. For example, where departmental representatives have facilitated access to additional datasets that were unknown to the research team and in other situations, synergies between regional government-held data and local government data or between different government departments have produced useful discussions about the sharing of data across authorities, and how collectively these can be used to paint a more complete picture and perhaps inform future research initiatives. We increasingly find that linking data can be a conduit for linking different policy makers.

Third sector advocates

Over the last few years we have rediscovered the power of the third sector as an avenue to impact on the policymaking process. As well as having in-depth, grassroots knowledge of the issues being explored by data-focused researchers [9], these groups often have longstanding relationships with departments and elected representatives through their advocacy work. Through these relationships, VCSE groups that have become directly involved in data research and appreciate the utility of using data and the potential evidentiary and policy impact this work can generate, are often strong advocates for both the use of the data and the use of the findings to address the needs of their constituent communities. The legitimacy they bring as a result of working directly with and hearing the concerns of their communities make them more difficult for government to ignore, and their involvement and support for projects can help overcome the reticence of departments in sharing data and exploring certain policy areas, despite potential tensions as detailed above. Involving VCSE groups directly in the steering committees for ADRC NI research has also resulted in access to elected representatives and fora that would not have been as easily accessible otherwise. One example of this, through the air pollution and health research programme, is explored below.

Policy symposium series

Within an engagement to impact framework such as ADRC NI’s, there is a need for wider dissemination of both research findings and potential research collaborations. Targeted knowledge exchange events are one of the non-academic ways we use our research to kindle a discussion within policy-relevant areas, with an audience wider than just policymakers and researchers. The ADRC NI policy symposium series are knowledge exchange events developed and delivered in partnership with key stakeholders (typically those organisations and departments participating in a research programme’s steering committee as well as identified others with an interest or working in the space). The format begins as a standard research event, but deliberately makes space for ‘responses’ from policymakers, service providers and community groups to discuss the research findings, the usage of data pertaining to their area of work or community, how the research connects or will be useful to their work, and to suggest areas for further exploration and collaboration. Platforming both the research and policy and community interpretations in this way recognises the multiplicity of knowledge involved in delivering impactful research for public benefit, and “supports the idea of stakeholders being key advocates for data research and helping to cascade the benefits of data-focused research to their own stakeholders and partners. During the period when Northern Ireland was without a devolved government (January 2017 – January 2020) these were particularly useful to connect researchers and their evidence with decision makers and communities [9].”

Discussion

Whilst ADRC NI’s approach to impact has been largely successful, particularly in difficult circumstances, there are certain prerequisites that need to be considered when the aim is for academics to undertake research in collaboration with data custodians and policy makers, with the goal of securing input into policy development and evaluation. These include a functioning government, willingness and ability of the departments and data owners to engage, and input from a wider circle of stakeholders to ensure a robust consideration of the stories being told by the data. However, we now have a modus operandi which has helped circumvent many of the obstacles. Arguably, the strength of ADRC NI’s approach to designing research to impact on policy and practice is that we have rejected the notion that there is one avenue to impactful research, and recognise that research with the most wide-ranging and deepest impact explores, supports and exploits all avenues of influence on decisions that impact the lives of people represented in our data (the data subjects). This approach is not without its limitations, but it has demonstrated the possibility of delivering tangible research impact even in challenging situations, such as a failing government or a global pandemic.

Moving forward, we intend to further enhance the co-production model we employ by moving into direct involvement of data subjects in research concerning them. At time of writing this includes the inclusion of a working carer on a steering committee for research looking at the impact of COVID on the mental health of workers and carers, and the development of workshops for care-experienced children and young people on data and research developed in partnership with Voices of Young People in Care (VOYPIC) [13].

Below we discuss some examples of successful impact we have achieved using our approach, as well as the limitations of our methods and ways we think it could be adapted and improved in future.

Examples of success

Mental health and wellbeing of care-experienced young people: a longitudinal study

This study was the first to use linked individual-level Social Services data in Northern Ireland, and the Steering Committee consisted of representatives from the Department of Health and Barnardo’s NI [14], a leading youth charity. Directly engaging with policy colleagues on a regular basis has helped the lead researcher to add additional analyses to her research that directly speak to priorities identified by policymakers on the steering committee. In addition, information from Barnardo’s NI, which provides domiciliary care services, has also made the narrative context around the data analysis more robust. This combination of knowledge and experience has helped place findings from the research within several influential strategies, including the joint Department of Health and Department of Education strategy ‘A Life Deserved: “Caring” for Children and Young People in Northern Ireland [15].’

Furthermore, the analyses undertaken by ADRC NI provides a level of scrutiny that the data have not been subjected to and this informal audit can also impact on the quality and ultimate utility of the data resource with tangible outcomes for data custodians and communities. For example, whilst conducting initial analysis on the social services data, the lead researcher discovered deficiencies in the data capture for one year that had not been previously recognised by the data owners who were able to develop a plan to address any issues with data collection, collation and coding [16].

Impact of exposure to air pollution on health and wellbeing

As with all ADRC NI steering committees, the advisory committee for the study examining the effect of exposure to air pollution on health and wellbeing, and the effect of exposure to air pollution on maternal, foetal and infant health, comprises a wide array of stakeholders (including the Department of Health and the Department for Agriculture, Environment and Rural Affairs), community and voluntary groups (British Heart Foundation NI (BHF NI), Asthma UK), and a local authority (Belfast City Council). At the first meeting, departmental colleagues indicated there were additional datasets available through the departments that would aid in giving a broader and deeper picture of the impact of air pollution, thereby paving the way for even more useful research findings and evidence to be produced and utilised for policymaking.

Value and impact were also added by third sector (VCSE) representatives on the committee. Through Belfast City Council we were directed towards BHF NI who in turn introduced us to Asthma UK, who were looking for research that would help them respond to the public consultation on a Clean Air Strategy for Northern Ireland. The research team were thus invited to present their findings to the All Party Group on Lung Health, which led to further engagement with and submission of an Assembly Written Question on secondary use of data legislation by the Deputy Chair, and a subsequent presentation to the Minister responsible for the new legislation. Further work is ongoing in support of Asthma UK and a backbench MLA to develop a private member’s bill on climate change and air pollution. Without the engagement with the VCSE representatives on the steering committee we would likely not have gained access to these influential circles or been invited back for further briefings (as we have been) [17].

Conclusion

Directly engaging and involving policymakers, data owners and third sector stakeholders in the development and delivery of research programmes, using community stakeholders as data advocates, and developing targeted knowledge exchange activities in partnership with stakeholders, enhances the potential impact the research can have on policy by supporting the design and development of research that answers the questions most relevant to policymakers whilst also maintaining academic independence and integrity. However, it is not a guarantee, and there will be instances where research needs and policy requirements do not dovetail. To address this, what is needed is a continuation of the broader cultural shift within data custodians and departmental policy owners to value and prioritise the use of the public’s data for the public’s benefit.

Conflict of interest

The authors declare that they have no conflicts of interest.

Acknowledgements

The authors acknowledge the support of the staff of the ADRC NI, without whom this work would not be possible.

The authors also acknowledge and thank the ESRC as funders of the ADRC NI and its wider partnership, ADR UK.

Ethics statement

Ethical approval was not required for this paper as it is based on our approach to impact and engagement.

References

1. ‘About Us: Our Partnership.’ https://www.adruk.org/about-us/about-adr-uk/ accessed 13 October 2021.

2. ‘About us: Our Values.’ https://www.adruk.org/about-us/about-adr-uk/ accessed 13 October 2021.

3. Information Commissioner’s Office. ‘Data sharing across the public sector: the Digital Economy Act codes’. https://ico.org.uk/for-organisations/guide-to-data-protection/ico-codes-of-practice/data-sharing-a-code-of-practice/data-sharing-across-the-public-sector-the-digital-economy-act-codes/ accessed 27 October 2021.

4. Welfare Reform Act 2012. https://www.legislation.agov.uk/ukpga/2012/5/contents accessed 27 October 2021.

5. European Union (Withdrawal Agreement) Act 2020. https://www.legislation.gov.uk/ukpga/2020/1/contents/aenacted accessed 27 October 2021.

6. Great Recession, Wikipedia. https://en.wikipedia.org/wiki/Great_Recession accessed 27 October 2021

7. Northern Ireland Assembly, NIAR 362-20: Research and Information Service Research Paper; 2021.

8. ‘Statistics: higher education graduate employment and earnings,’ updated 24 June 2021. https://www.gov.uk/government/collections/statistics-higher-education-graduate-employment-and-earnings accessed 26 October 2021.

9. Nelson, E. and Burns, F. (2020) “Impact through Engagement: Co-production of administrative data research and the approach of the Administrative Data Research Centre Northern Ireland”. International Journal of Population Data Science, 5(3). 10.23889/ijpds.v5i3.1369

   https://doi.org/10.23889/ijpds.v5i3.1369

10. Van Der Graff, P., Cheetham M., McCabe K., Rushmer R. “Localising and tailoring research evidence helps public health decision making.” Health Inf Libr J. 2018:35(3)202–12.

11. What type of impact does ADR UK aim to have? From ‘Our Impact’, https://www.adruk.org/our-mission/our-impact/ accessed 4 October 2021.

12. ‘Improving the lives of farmers,’ ADR NI, https://www.adruk.org/our-mission/our-impact/article/improving-the-lives-of-farmers-192/. Impact case study demonstrating how using linked administrative data saved the NI Department for Agriculture, Environment and Rural Affairs £350,000 on its farmers census survey.

13. VOYPIC https://www.voypic.org/ accessed 27 October 2021.

14. Barnardo’s NI https://www.barnardos.org.uk/northern-ireland accessed 27 October 2021.

15. ‘A Life Deserved: “Caring” for Children and Young People in Northern Ireland,’ Department of Health and Department of Education, Northern Ireland. https://www.health-ni.gov.uk/publications/life-deserved-strategy-looked-after-children accessed 4 October 2021.

16. Maguire, A. (2021) “Improving practice: impact of a social services data quality investigation” ADR UK, ‘Our Impact’. https://www.adruk.org/our-mission/our-impact/improving-practice-impact-of-a-social-services-data-quality-investigation/ accessed 5 January 2022.

17. Rowland, N. (2021) “A breath of fresh air: understanding the impact of air pollution on public health”. ADR UK, ‘Our Impact’. https://www.adruk.org/our-mission/our-impact/article/a-breath-of-fresh-air-understanding-the-impact-of-air-pollution-on-public-health-448/ accessed 27 October 2021.