IJPDS International Journal of Population Data Science

We have included this clarification on page 4:

“In Ontario, children can be enrolled in non-mandatory, publicly funded kindergarten in September of the year they turn 4 years of age (junior kindergarten), and in September of the year they turn 5 years they can be enrolled in senior kindergarten. Publicly funded compulsory schooling begins at age 6. Approximately 97% of students attending public school in Ontario entered school in kindergarten [22]. Senior kindergarten teachers complete the EDI for each child in their classroom, based on their observation of the child for at least 4 to 6 months (median age of administration in the present dataset is 5.68 years).”

Thanks for this suggestion. Mapping the data by age does make sense, but Ontario uses the calendar year instead of the academic year to place students in the appropriate grade level. Therefore, a student entering Grade 3 for example will be 7 or 8 (they have to turn 8 by Dec 31 of the year they enter Grade 3). Because of this potentially confusing policy, we retained the organization of the data by grade, but included verbiage about the ages for each grade level. Page 5:

“Ontario places children in the appropriate grade based on the calendar year; they must turn a given age by December 3. Thus, Ontario students enter Grade 3 in September at age 7-8, Grade 6 at age 10-11, Grade 9 at age 13-14, and Grade 10 at age 14-15. The EQAO assessment is taken in spring of the academic year.”

We hope the clarification on the age of EDI assessment eases the interpretation of the EDI administration. We also revised as follows on page 5:

“The present Ontario-based dataset consists of the EDI data collected over nine years in three-year cycles [23]: Cycle 1: 2004-2006, Cycle 2: 2007-2009, Cycle 3: 2010-2012. In each Cycle, the EDI was completed for all senior kindergarten students in approximately a third of all publicly funded school board in Ontario; thus by the end of the 3-year cycle the EDI assessment was implemented just once in every school district in Ontario. This roll-out strategy allowed communities to utilize longitudinal data by which they could examine changes in patterns of child vulnerabilities and thereby guide new policy and funding directions, without the cost of administering the EDI every single year.”

We have removed these sentences pertaining to the nationwide CanNECD database: “The index remains a permanent component of the CanNECD EDI database. As of 2014 the database contained 798,987 EDI records spanning the previous decade and including EDI data matched with the SES index from all Canadian provinces and territories except Nunavut [13].”

In this section (page 7), we added that the CanNECD SES Index is a neighbourhood-level measure and that there are 797 unique neighbourhoods in the present dataset.

Page 8 – when you say “As a first step, only students with unique and exact matches on these variables were included.” I assume you later included partial matches, what is the difference here between exact and partial matches?

This is a bit further confused when later on you write “The linking process created a fully matched dataset of the EDI and all four EQAO data points (n=86,778), which is a subset of a partially matched dataset (n=155,082) in which the EDI is matched to all four EQAO datapoints except those with job action or COVID-19 related cancellations.” So, when you refer to partially matched here, do you mean they are exact linked, but do not include data for all assessments?

We have modified the paragraph in question on page 7-8 to be more clear. In the revision, we did not use the term “partial match”, to hopefully avoid confusion.

The paragraph in question on page 9 now reads:

The proportion of EDI participants linked to the four EQAO datapoints ranged from 42-50%, depending on the cohort year (Table 2). For cohorts from 2009-2012, which had fewer EQAO assessments to match due to job action or COVID-19, up to 57% of EDI cases were linked to at least one EQAO administration.

We began the linkage process with 374,239 total EDI cases, to which 174,685 EQAO Grade 3 cases were linked using the process described above. EQAO grades 6, 9, and 10 administrations were then linked to that dataset using a stepwise inner-join linking approach. “Inner-join” refers to a linkage of two datasets that restricts the outcome dataset to participants who exist in both datasets. The result was 155,082 cases when including participants who are missing EQAO administrations due to job action or COVID-19. From there, a fully linked dataset (n=86,778) was created which includes participants who took the EDI and each of the four EQAO assessments (with no missing EQAO datapoints). Figure 1 visually depicts the linking process.

We also added this paragraph on page 8 to clarify the linkage process:

“We merged the subsequent grade 6, 9, and 10 EQAO scores to the EDI-Grade 3 dataset simply by isolating each EDI administration year, linking the datasets on the EQAO student personal identification number, and then re-merging all administration years together. Three versions were created using a stepwise inner join merging strategy that restricted each cohort’s linked data to the EQAO administration with the least number of participants present for each EQAO administration that was available (see Figure 1). The least restrictive version of the dataset captures all students who were present for at least one EQAO administration; the second least restrictive restricts to students present for all four EQAO administrations (3, 6, 9, and 10) except missingness due to job action and COVID-19, and the most restrictive restricts to students who were present for all four EQAO administrations with no missingness. The fewer assessment timepoints included, the greater the size of the dataset, since each additional assessment adds the constraint that the participant be present for the linked dataset with no missingness. For researchers interested in pursuing analyses about specific academic outcomes (i.e., just literacy or numeracy, or only academic outcomes through Grade 6), the size of the dataset will be larger, as literacy is not assessed in Grade 9, and numeracy is not assessed in Grade 10. The rationale for creating a dataset with no missingness is described later in the section on validation of the linkage and discussion below.”

Thank you for this suggestion. We agree Table 2 was confusing. The revised Table 2 (now Table 1) takes the form you recommend (page 9).

Per your recommendation, we excluded CanNECD from this table.

Data access statement

The data set at the centre of this study is held securely at OCCS. Data-sharing agreements prohibit OCCS from making the data set publicly available, but access may be granted to those who meet pre-specified criteria for confidential access by contacting the EDI team at OCCS at https://edi.offordcentre.com/contact. The full data set creation plan and underlying analytic code are available from the authors upon request; programs may rely upon coding templates or macros that are unique to OCCS.

This has been added to the discussion on page 14:

“While several provinces and at least one territory in Canada practice data linkage aiming to enable the study of children’s educational trajectory similar to our study goals (e.g., Manitoba [17], British Columbia [69]), the methods of linking and their success are not directly comparable with ours. The major limitation of our study was lack of common identifiers, which is a specific feature of government administration in Ontario – Canada’s most populous province – where separate identifiers are issued through health authorities and education authorities without an existing cross-walk between the two. In the future, our study may pave the way for connecting educational data to administrative databases (e.g., 64), once legislative obstacles allow the combination of health and education identifiers. In the meantime, however, our study has tested a unique method that allows building useful, linked datasets despite substantial barriers, and creates a potential to catch up to the educational research conducted in other parts of the country.”

Introduction, end of last paragraph. I would move “prohibitive costs” last in the list as the example that follows is on co.

Page 3 - “In Canada, over 1.5 million children have been assessed with the EDI since its inception [12].” Add “inception since xxx year.”

Page 3 – “The other major data-sharing partner was Ontario’s Educational Quality Assessment Office (EQAO) [add: a public authority/government agency ?] EQAO is Ontario’s main provider…”

Page 3 – “home environments and routines outside of school”, what is a routine outside school?

Page 4 – “Vulnerability is assigned using the first baseline dataset: children who score at or below he [add: bottom? 10th percentile?] in a given domain”

Page 5 – the order of your subsections is different, and you have not listed sex and DOB in additional variables.

Page 5-6. I would remove Table 1 or put in a supplement. This is not your core focus and takes up too much space.

Page 6 Overview of research linking EDI-EQAO thus far – I find the discussion of results a bit out of place in this section. It should go in the introduction to justify your work or in the discussion to hint at what can be done later.

Page 8 “which added 51,000 new matches to the 2005, 2008, and 2011 cohorts. ” say that in terms of how did the linkage rate improve, what was the % linked after that?

Page 8 – when you say “As a first step, only students with unique and exact matches on these variables were included.” I assume you later included partial matches? This is a bit further confused when later on you write “The linking process created a fully matched dataset of the EDI and all four EQAO data points (n=86,778), which is a subset of a partially matched dataset (n=155,082) in which the EDI is matched to all four EQAO datapoints except those with job action or COVID-19 related cancellations.”

Page 10- “… can be interpreted like a r2 value: it measures proportions of variance in a continuous variable that is explained by a grouping variable and is standardized between -1 and 1. Effect sizes are interpreted as small at 0.01, medium at 0.06, and large at 0.14 or greater.” Something is not right here, r2 does not range between -1 and 1, but 0-1.