Using Big Data in Policy Evaluations: the Troubled Families Programme - Knight, Naomi; Han, Lu; Man, Lan-Ho; Taylor, Ricky (International Journal of Population Data Science, 2018)
Background The Ministry for Housing, Communities and Local Government have carried out one of the biggest data linkage exercises in government in order to evaluate the impact of the Troubled Families Programme. Linking individual and family level data across multiple administrative datasets has proven to be both innovative and cost-effective, enabling us to place children in the broader context of their family and household circumstances for our analysis. Objectives To use administrative datasets to measure children’s service use outcomes, for both programme and comparison individuals and families, to assess the impact of the Troubled Families Programme on outcomes for ‘children needing help’. Methods The comparison group provide a counterfactual, used to derive a robust assessment of the programme’s impact on children’s outcomes: in this case child safeguarding. Linked datasets means we can control for both individual and family level characteristics, such as parental employment, benefits, school attendance, children and adult offending and the circumstances of siblings. We have used propensity score matching to control for all covariates impacting on both treatment and outcome status. Findings Preliminary findings show a statistically significant reduction in the number of ‘children in need’ in the 6-12 month period after intervention start compared to the matched comparison group, and a reduction in the number of ‘looked after children’. There was an increase in the number of children with a ‘child protection plan’, but this was not statistically significantly different to the comparison group. Conclusions While access to such a wide range of individual and family characteristics is a key methodological advantage to evaluation, challenges include: missing data, time lags in the datasets and complex variable definitions. We have worked and continue to work with other government departments to overcome these. Whilst still in its earliest stages preliminary results for the programme’s impact are encouraging.