<?xml version="1.0"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20190208//EN" "JATS-journalpublishing1.dtd" [
]>
<article xml:lang="en" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" dtd-version="1.2" article-type="research-article">
    <front>
        <journal-meta>
            <journal-id journal-id-type="publisher-id">IJPDS</journal-id>
            <journal-title-group>
                <journal-title>International Journal of Population Data Science</journal-title>
                <abbrev-journal-title>IJPDS</abbrev-journal-title>
            </journal-title-group>
            <issn pub-type="epub">2399-4908</issn>
            <publisher>
                <publisher-name>Swansea University</publisher-name>
            </publisher>
        </journal-meta>
        <article-meta>
            <article-id pub-id-type="doi">10.23889/ijpds.v11i5.3650</article-id>
            <article-id pub-id-type="publisher-id">11:5:3650</article-id>
            <article-id pub-id-type="pii">S2399490821036508</article-id>
            <article-categories>
                <subj-group subj-group-type="heading">
                    <subject>Population Data Science</subject>
                </subj-group>
            </article-categories>
            <title-group>
                <article-title>Reproducibility and robustness of research conducted with administrative data: lessons learnt from the CBS Replication Games</article-title>
            </title-group>
            <contrib-group>
                <contrib contrib-type="author">
                    <name>
                        <surname>Fitzgerald</surname>
                        <given-names initials="J">Jack</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-1"><sup>1</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Fiala</surname>
                        <given-names initials="L">Lenka</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-2"><sup>2</sup></xref>
                    <xref ref-type="aff" rid="affil-3"><sup>3</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Karpinska</surname>
                        <given-names initials="K">Kasia</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-4"><sup>4</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Maineri</surname>
                        <given-names initials="A">Angelica</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-4"><sup>4</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Alrouh</surname>
                        <given-names initials="H">Hekmat</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-4"><sup>4</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Atav</surname>
                        <given-names initials="T">Tilbe</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-6"><sup>6</sup></xref>
                    <xref ref-type="aff" rid="affil-7"><sup>7</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Bäckmann</surname>
                        <given-names initials="M">Malte Henning</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-5"><sup>5</sup></xref>
                    <xref ref-type="aff" rid="affil-7"><sup>7</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Bekkers</surname>
                        <given-names initials="R">Rene</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-8"><sup>8</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Bernasco</surname>
                        <given-names initials="W">Wim</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-9"><sup>9</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Bles</surname>
                        <given-names initials="P">Per</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-10"><sup>10</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Moneva</surname>
                        <given-names initials="A">Asier</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-9"><sup>9</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Nikkhah Bahrami</surname>
                        <given-names initials="S">Samin</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-11"><sup>11</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Oreel</surname>
                        <given-names initials="T">Tom</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-4"><sup>4</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Prudon</surname>
                        <given-names initials="R">Roger</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-12"><sup>12</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Schouwenaar</surname>
                        <given-names initials="C">Chantal</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-8"><sup>8</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Sivak</surname>
                        <given-names initials="E">Elizaveta</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-13"><sup>13</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Steenbeek</surname>
                        <given-names initials="W">Wouter</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-9"><sup>9</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Veerhoek</surname>
                        <given-names initials="O">Oskar</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-14"><sup>14</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Villalobos</surname>
                        <given-names initials="S">Sofía</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-8"><sup>8</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Vitt</surname>
                        <given-names initials="M">Mats</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-15"><sup>15</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Wang</surname>
                        <given-names initials="G">Guan-Yuan</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-16"><sup>16</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Wang</surname>
                        <given-names initials="K">Keling</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-17"><sup>17</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Zwier</surname>
                        <given-names initials="D">Dieuwke</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-18"><sup>18</sup></xref>
                </contrib>
                <contrib contrib-type="author">
                    <name>
                        <surname>Brodeur</surname>
                        <given-names initials="A">Abel</given-names>
                    </name>
                    <xref ref-type="aff" rid="affil-2"><sup>2</sup></xref>
                </contrib>
                <aff id="affil-1"><label>1</label><institution>VU Amsterdam, Amsterdam, Netherlands</institution></aff>
                <aff id="affil-2"><label>2</label><institution>Institute for Replication (University of Ottawa), Ottawa, Canada</institution></aff>
                <aff id="affil-3"><label>3</label><institution>Tilburg University, Tilburg, Netherlands</institution></aff>
                <aff id="affil-4"><label>4</label><institution>Erasmus School of Social and Behavioral Sciences (ESSB), Rotterdam, Netherlands</institution></aff>
                <aff id="affil-5"><label>5</label><institution>Erasmus University Rotterdam</institution></aff>
                <aff id="affil-6"><label>6</label><institution>Erasmus School of Economics (ESE), Rotterdam, Netherlands</institution></aff>
                <aff id="affil-7"><label>7</label><institution>Erasmus School of Health Policy &amp; Management (ESHPM), Rotterdam, Netherlands</institution></aff>
                <aff id="affil-8"><label>8</label><institution>VU Amsterdam, FSW, Amsterdam, Netherlands</institution></aff>
                <aff id="affil-9"><label>9</label><institution>Nederlands Studiecentrum Criminaliteit en Rechtshandhaving (NSCR), Amsterdam, Netherlands</institution></aff>
                <aff id="affil-10"><label>10</label><institution>Universiteit Maastricht, School of Business and Economics, Researchcentrum Onderwijs en Arbeidmarkt, Maastricht, Netherlands</institution></aff>
                <aff id="affil-11"><label>11</label><institution>Utrecht University, REBO Faculty (Utrecht University School of Economics), Utrecht, Netherlands</institution></aff>
                <aff id="affil-12"><label>12</label><institution>Lancaster University, Lancaster, United Kingdom</institution></aff>
                <aff id="affil-13"><label>13</label><institution>University of Groningen, Groningen, Netherlands</institution></aff>
                <aff id="affil-14"><label>14</label><institution>Radboud University - Nijmegen School of Management, Nijmegen, Netherlands</institution></aff>
                <aff id="affil-15"><label>15</label><institution>Universiteit Maastricht, School of Business and Economics, Maastricht, Netherlands</institution></aff>
                <aff id="affil-16"><label>16</label><institution>Tilburg University, Tilburg School of Economics and Management (TISEM), Tilburg, Netherlands</institution></aff>
                <aff id="affil-17"><label>17</label><institution>Universitair Medisch Centrum Erasmus Rotterdam, Rotterdam, Netherlands</institution></aff>
                <aff id="affil-18"><label>18</label><institution>Utrecht University, Utrecht, Netherlands</institution></aff>
            </contrib-group>
            <pub-date date-type="pub" publication-format="electronic">
                <day></day>
                <month></month>
                <year></year>
            </pub-date>
            <pub-date date-type="collection" publication-format="electronic">
                <year></year>
            </pub-date>
            <volume>11</volume>
            <issue>5</issue>
            <elocation-id>3650</elocation-id>
            <permissions>
                <license license-type="open-access" xlink:href="https://creativecommons.org/licenses/by-nc-nd/4.0/">
                    <license-p>This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.</license-p>
                </license>
            </permissions>
            <self-uri xlink:href="https://ijpds.org/article/view/3650">This article is available from the IJPDS website at: https://ijpds.org/article/view/3650</self-uri>
            <abstract>
                <p>The social sciences are experiencing a renaissance in open science, with mass replication initiatives and transparency policies becoming widespread in many institutions and academic journals. However, the rising use of restricted data is a key barrier to replication in the social sciences. In a sample of 158 articles published in Top 5 economics journals from 2020–2023, over 48% required restricted-use data to produce some main findings (Fitzgerald 2025). The proportions of papers that use restricted data have tripled or quadrupled in economic subdisciplines since 2000 (Garg &amp; Fetzer 2024). The time and costs associated with re-accessing restricted-access data tend to be a significant barrier to preclude reproducibility in practice. In February 2026, ODISSEI and the Institute for Replication will host a Replication Games hackathon where teams of replicators will attempt to reproduce and run robustness checks on six papers published since 2020 in top economics journals with robust transparency policies that are produced entirely from Statistic Netherlands administrative microdata and other public-use data, and have made code available. The results of our reproducibility and robustness analyses can help improve the trustworthiness of scientific analyses using restricted-access data. Difficulties in reproducing findings in these papers can help highlight necessary infrastructural adjustments (e.g., improvements in version control) to help ensure that findings using CBS microdata are reproducible. Conversely, if results in these papers are reasonably reproducible and robust, this can help boost trust in research using restricted-access data. The presentation will outline the preliminary results and lessons learnt from the three-day event.</p>
            </abstract>
        </article-meta>
    </front>
</article>