Governance permissions and data flows for the linkage followed the separation principle [16], whereby identifiers such as names and postcodes were kept separate at all times from attribute data (records from school or hospital records). Figure 1 shows the flow of identifiers and a pseudo-identifier (the anonymised Pupil Matching Reference, aPMR) from the Department for Education to NHS Digital. Separately, education attribute data flowed from the Department for Education to the Office of National Statistics Secure Research Service (ONS SRS). A two-stage linkage process was used to link NPD to HES. Stage 1 linked NPD to the Personal Demographic Service (PDS), which contains all individuals with an NHS number, and stage 2 linked NPD-PDS linked data to HES. At the first stage of linkage (step C in Figure 1), NHS Digital linkers had access only to the identifiers (date of birth, sex, and histories of forenames, surnames and postcodes) but no attribute data. At the second stage of linkage (step D), NHS Digital used the NHS number, date of birth, sex and postcode to link to HES data. The linkage step, pseudonymised HESID and anonymised PMR were transferred (step E) and merged with a University College London (UCL) held extract of HES within the UCL Data Safe Haven (DSH) (step F). Linked HES-PMR records were ultimately transferred to the ONS SRS (step G).

The study population consisted of four cohorts of children born between 1 September and 31 August in the academic years of 1990/91, 1996/97, 1999/00, and 2004/05 (Figure 2). These cohorts were defined separately in NPD and HES, so that linkage created three comparison groups for each of the four cohorts: linked NPD-HES, unlinked NPD, and unlinked HES records. We compared pupil characteristics in the linked and unlinked NPD cohorts at each stage of each linkage process. We used NPD as the inception cohort, as state school is a universal service attended at some point in the school years by at least 95% of all children [2, 18]. On the other hand, not all children attend hospital, unless they were young enough for their birth to be recorded in HES (1997 onwards).

Figure 2 shows that whether a pupil is expected to link to a HES record or not is affected by the start date of the PDS, the NPD and the subsets of HES data. Pupils born in 1990/91 were expected to have the lowest proportion of records in NPD that linked to HES (i.e. linkage rate). These children only appeared in NPD at the first school census collection in 2001/02 at age 10. Their names and postcodes captured each year in NPD from 2001/02 until leaving state school in 2009/10 or earlier, would be linked to names and postcode details recorded prospectively from General Practitioner (GP) registrations and hospital contacts on the PDS from 2004 onwards. These children could link to HES admission records from 1997 onwards (age 6 years), outpatients from age 12, or accident and emergency department from age 16.

Whilst it was expected that most children would have contact with hospital at some point during childhood or adolescence, we did not anticipate complete overlap between the two datasets. We expected children born in 2004/05 to have the best linkage rates of the four cohorts (and for linkage quality to remain constant or improve for subsequent cohorts). Firstly, 97% of children born in England would be expected to have their birth recorded in HES and in PDS [19]. Secondly, their linkage to subsequent health records should be more accurate than earlier cohorts due to immediate allocation by midwives of NHS numbers to babies at birth, a process introduced at the end of 2002 [20].

The data sources are described in detail in the Supplementary Appendices 2 and 3.

At stage 1, between 91% and 95% of pupils linked at the first step of the 8-step algorithm, i.e. exact linkage by first name, surname, date of birth, sex and postcode (Table 2; Supplementary Appendix 6). However, evaluation by ethnic group showed that the additional steps in this algorithm, i.e. from 2-8, captured a greater percentage of ethnic minority groups (11.8% of minority ethnic groups versus 4.2% of white ethnic group).

A considerable percentage of records were linked in years after the first available Spring census (Figure 4). For example, 12% and 21% of records of pupils born in academic years 1990/91 and 1996/97 respectively, were matched after 2004/05 – their first available Spring census when it was possible to link to PDS. Similarly, in academic years 1999/00 and 2004/05, 16% and 9% of pupils were matched after their academic Year 1- their second available Spring census. For pupils born in academic year 1999/00 or after, the majority of records were linked in the first two academic years. In particular, 50% of records in cohort 1999/00 and 51% in 2004/05 were linked in Year 1, while 34% and 40% were linked in reception year (Supplementary Appendix 6).

Linkage at stage 2, from PDS to HES using the NHS Digital internal 7-step algorithm (Table 3) showed a similar pattern to linkage at stage 1. Of the 2,202,823 pairs in NPD linked at stage 2, 81% (n=1,791,480) were linked at step 1 and 18% at step 2 (n=386,579) (Supplementary Table 7.1 in Supplementary Appendix 6). Pupils from ethnic minorities were disproportionately linked at steps 2-8. For example, around 20% of pupils categorized in Black and Chinese ethnic groups were linked at step 2, compared to 17% of white pupils that linked at this step. Of steps 3-8 of the algorithm, step 6 was particularly important for the linkage of ethnic minority groups, linking between 0.7%-1.7% of ethnic minority records (see Supplementary Appendix 6 for more details).

Pupils who linked to HES after both linkage stages and who were merged with HES attribute data comprise the matched dataset used for all subsequent analyses. Linkage rate by region, ethnic group, sex and IDACI deciles are shown in the Supplementary Appendix 7. We found that linkage rates improved over time for all these variables. However, ethnic minorities and pupils living in more deprived areas were less likely to match to HES. The linkage rate for white pupils improved from 94.6% in the 1990/91 cohort to 98.9% in the 2004/05 cohort. In contrast, for ethnic minority pupils in the same cohorts the linkage rate rose from 92.4% to 97.7%, respectively. We found a similar pattern by IDACI deciles. Linkage rates by region provide evidence that London has consistently lower linkage rates than the rest of the country.

Differences in the distribution of sociodemographic and educational characteristics of pupils recorded in NPD who linked or not to HES are shown in Table 4 (and Supplementary Table 9.1–9.4 in Supplementary Appendix 8). Overall, relatively low standardized differences are observed across all variables providing evidence of small or moderate differences between linked and unlinked groups. We considered standardized differences of 0.2, 0.5 and 0.8 as small, moderate and large, respectively [28, 34]. The largest differences were for the AAP/S and persistent authorized absence rate in cohort 1996/97 with values of 0.44 and 0.42. The mean standardized difference across cohort for region and ethnic groups was 0.25 and 0.24 whereas for sex and IDACI deciles was 0.13 and 0.17 (Table 4).

Table 5 shows the results of multivariable logistic models displaying adjusted Odds Ratios (OR) for linkage to HES. Unadjusted models are also shown in Supplementary Appendix 9. OR below 1 indicates lower odds of linkage to HES compared with the reference category. Consistent with linkage rate estimates, we found differences across ethnic groups, deprivation and region. Across all cohorts, we found that relative to pupils of white ethnicity, pupils of ethnic minorities including Asian, Black, Chinese, Mixed and Any other ethnic group were less like to be matched. The odds of linkage to HES for Asian ethnic groups were less than ethnic minority pupils (e.g. 1990/91: Adjusted OR 0.69, 95% CI 0.66 to 0.72, p < 0.01; 2004/05: Adjusted OR 0.51, 95% CI 0.47 to 0.54, p < 0.01). Relative to male pupils, with the exception of pupils born in academic year 1990/91, female pupils were less likely to be matched (e.g. 2004/05: Adjusted OR 0.72, 95% CI 0.69 to 0.75, p < 0.01). Compared to pupils in the fifth IDACI Deciles, pupils living in the most deprived areas were less likely to be matched, whereas pupils living in the most affluent areas were more likely to be matched. Similarly, results for the region of pupil residence show differences for linkage success.

Our finding that the linkage rate was 99% for the youngest cohort is encouraging for future studies using multi-step deterministic algorithms in England. This linkage rate is similar to studies in Scotland, Wales and Australia that used probabilistic linkage methods [11, 13, 14, 37–39]. For instance, linkage rates for the annual Scottish Governments pupils census linked to the community health index database ranged between 86.3% and 95% [14], while two other Scottish studies found linkage rates of 99.7% [13] and 81.8% [11].

We found that between 2.3–7.6% of ethnic minority pupils were not linked to health records. Ethnic differences reported in previous linkage success reflect differences in the quality of registration of Chinese, Asian and Hispanic names [8, 27, 28]. The differences in linkage rates by ethnic minority in linkage steps that relaxed the requirement to agree on exact full name suggest that inconsistencies in forenames and surnames explain the lower linkage rates for ethnic minority pupils. Residential instability may also be relevant: lower rates of linkage for pupils from ethnic minorities at steps 1 and 2 between PDS and HES (i.e. stage 2), could be due to poor recording of postcode, as reported in other studies [40, 41]. It is also estimated that 20% of children aged 0 to 15 years are born outside the UK, which may have a differential impact on linkage success [42]. Additional steps in the deterministic algorithm that incorporate phonetic systems codes for other languages [43, 44], or methods that discriminate partial agreements in string comparisons [45–48], or probabilistic linkage methods could be used to further improve linkage rates for ethnic minorities [40, 48].

We found that pupils living in more deprived neighbourhoods were less likely to link to health records than pupils living in more affluent areas. Previous studies have suggested that families from more affluent areas are more likely to comply with the administrative process [8]. However, pupils living in London were less likely to link to HES records than in other regions, even after accounting for sociodemographic characteristics. This difference may reflect higher rates of international emigration from London, less use of health services, differential use of private health services, or poorer quality of identifiers in London.

Improvements in the quality of recording of identifiers in schools and health data systems likely account for improved linkage rates over time. Changes in health systems governing collection of patient identifiers, such as the implementation of NHS Numbers for Babies (NN4B) service on 29th October 2002, the introduction of Registration ONline system (RON) on 1st July 2009, the correction of a postcode extraction error by NHS Digital on 1st April 2013, have been shown to improve the completeness of identifiers used in the linkage [20]. Retrospective correction of this extraction error and re-linkage by NHS Digital of birth episodes to subsequent HES records, would be expected to improve linkage to NPD in earlier years.

Our study demonstrates very high linkage rates between educational and HES records for pupils attending state schools in England. The governance for this project addressed the challenges of cross-sectoral linkage between health and educational institutions in England whilst avoiding disclosure during the linkage process [16]. Use of multiple steps at each stage of linkage, and of identifiers recorded over multiple years for each child, were critical to achieving high linkage rates. Preliminary findings indicate that two-thirds of the linked HES records related to at least one admission, excluding the birth admission (to be reported elsewhere). The linkage algorithms used for this project are currently being used to link educational and health records for all pupils in England born academic years 1995/96 onwards and will be relevant for other studies linking data to HES or NPD (or both) [17].

Linking educational data with hospital and death records creates new possibilities for studying a wide spectrum of policy-relevant questions. For example, the availability of data across the child life course could enable studies into the impact of health on education and education on health. Linked data for all children will be made available for applications for research from government and academia in 2021 [49, 50].

Record-level indicators of the linkage process (i.e. variables indicating the step in our rule-based linkage algorithms at which a pair of records linked) were shared by NHS Digital to enable us to evaluate linkage biases. We used this information to demonstrate the value of later steps in the algorithm for linking pupils from ethnic minority and deprived areas. However, we did not have information on country of birth, and so could not assess whether linkage rates were lower for children who were born outside England. Future studies should consider sharing information about the completeness or quality of the identifiers to identify whether changes in data entry systems could address missed links in these more vulnerable groups [16].

A limitation and advantage were the system changes in administrative data resulting in improvements in identifier and linkage quality and additional data collections from both services. These changes can introduce variation in linkage error over time, for instance, patients with fewer contacts with health services or more mobile populations could have out-of-date residential information in PDS disproportionately affecting linkage quality, which analysts need to consider when investigating trends.

A further limitation is that since no gold-standard dataset defining true match status was available, we could not derive standard measures of linkage quality (sensitivity/recall, false match rate and positive predictive value/precision). Approaches for estimating rates of false matches in further linkage between HES and NPD could be applied, for example by applying the linkage algorithms to a set of ‘negative controls’ (i.e. NPD records for which we are certain there should be no link in HES or vice versa) and counting how many records were erroneously linked [51, 52]. This would allow an estimation of false match rates, but would not allow identification of which records were falsely matched. Existing ‘gold-standard’ data for health records in England for specific sub populations also have the potential to be used in the future evaluations of linkage quality [53]. Future studies could develop representative gold-standard data using known links from UK cohort studies, such as the Millennium Cohort Study or Next Steps to allow linkage error to be fully measured [54, 55].

We created a de-identified linked database that brings together data from the Department for Education (education and social care) and hospitalisation data for all children – the ECHILD Database. This resource will be made available for approved researchers later in 2021 for purposes that benefit health, wellbeing, education and the provision of health or social care. The ECHILD dataset will enable a step change in the scale and depth of research into the inter-relationships between health, education and social care across the life course, and how services across England vary in their responses.

Our linkage created a de-identified bridging file that combines pseudo-identifiers from education (anonymised Pupil Matching Reference) and HES. This bridging file can be used by the data providers to link to further datasets for approved studies, without the need to link real-world identifiers such as names and postcodes. As the data systems for capturing identifiers change, as is currently happening at NHS Digital [56], our evaluation of linkage success will need to be repeated and linkage metrics provided to researchers.

Researchers addressing questions relating to ethnic minority or deprived groups need to consider whether to adjust for missing data among these groups due to missed links. Statistical techniques include weighting or imputation, depending on the research objectives [57].