Stretching a Buck: Using Administrative Data to Inform Continuous Quality Improvement

Abstract

While the nFORM administrative data system is used to collect operations and performance data from HHS funded Healthy Marriage and Responsible Fatherhood (HMRF) grantees to evaluate program performance using top line measures like enrollment and attendance of workshops, the rich data collected through the system provides an unparalleled window into the workings of the HMRF programs and the population they serve. This paper describes how raw data exported from the nFORM system have been used to develop econometric models to understand the relationship of demographic and socio-economic characteristics on program enrollment and attendance, the effectiveness of incentives and behavioral nudges on program participation, as well as changes in behaviors and attitudes due to the intervention. Moreover, the paper discusses how patterns revealed through mining the raw nFORM data combined with other administrative data has provided insights into deficiencies in outreach and recruitment efforts, and highlights how the relative effectiveness of steps taken to remedy the deficiencies can also be tracked using the data. Lastly, the paper presents recommendations and best practices in using nFORM and other similar administrative data to inform continuous quality improvement of HMRF programs without stretching the budget.

While predicting child maltreatment risk at the household level is useful for allocating limited child welfare resources, significant privacy, data integration, data governance and legal hurdles make such an algorithm economically and politically difficult to put into production. In this project, we take a different approach to child maltreatment risk prediction, developing machine learning models that predict, not for a household but for a small spatial areal unit, such as the block. The only private health data required for this use case are geocoded maltreatment events. We present the results of a machine learning analysis in Richmond Virginia, including exploratory analysis, feature engineering, model development and validation. We then interpret our models in a resource allocation context.