Representativeness of survey participants in relation to mental disorders: a linkage between national registers and a population-representative survey

Abstract

Introduction
Surveys and registers have provided important insights into the mental health of the community. However, both sources have strengths and limitations. While participation in surveys has been shown to be lower among those with mental disorders, misclassification and limited information on confounders are typical issues for registers.

Objectives
To examine if participants of the Central Denmark Region's 2017 How are you? survey were representative of the general population in terms of mental disorder diagnoses.

Methods
By linking data from the Central Denmark Region's 2017 How are you? survey with the Danish national registers, we compared the frequency of mental disorder diagnoses among (a) participants in the survey (n = 32,417), before and after applying non-response weights, and (b) the entire population who were eligible to participate (n = 1,063,082; 16 years of age or older on 10^th January 2017 and registered as living in the Central Denmark Region). Using logistic regression models, we estimated associations between being diagnosed with any mental disorder and nine general medical conditions to assess whether selection into the survey appeared to bias these associations.

Results
Based on register data, 10.4% (n = 110,492) of the eligible population had received a diagnosis of any mental disorder prior to the date of this survey. Among the unweighted survey sample, 8.2% (n = 2,648) had received a diagnosis; once non-response weights were applied, this corresponded to 9.5%. Representativeness varied by sex, age and type of mental disorder. For example, people with organic disorders or substance use disorders were generally underrepresented among survey participants of all ages; however, representativeness of common disorders such as mood or neurotic disorders was generally good. With respect to the association of any mental disorder and general medical conditions, we found that estimates were similar for survey samples (both weighted and unweighted) compared to the entire eligible population.

Conclusions
People with a previous diagnosis of a mental disorder are slightly underrepresented in the survey. However, this selection bias was minimized when non-response weights were applied. Associations between mental disorders and general medical conditions did not appear to be affected by selection bias. With the application of non-response weights, the survey provided a sample representative of the general population in terms of mental disorder diagnoses.

Introduction

Epidemiology is a foundational science for health research. In particular, it is essential to understand the prevalence of disorders, in order to design appropriate services [1]. Within the epidemiological tool box, surveys and register-based research have provided essential information to guide health planning. On one hand, surveys have played a critical role in mental health research. They can estimate the prevalence of different types of mental disorders, and explore service use and unmet need [2–4]. Whether these studies are multinational (e.g. World Mental Health Surveys [5] and the Wellcome Global Monitor [6]) or single site and smaller, population-based surveys have provided information on the epidemiology of mental health issues. On the other hand, several nations now have access to administrative registers (e.g. the Nordic countries) [7], which are useful to complement survey data. National registers, like electronic healthcare databases, provide large samples which often include the entire population, and thus are not subject to selection bias. However, the registers are administrative and often lack details about relevant exposures or confounders.

Each of these data sources have strengths and limitations. In particular, there are well known factors that influence participation in surveys [8–14]. Disorders associated with premature mortality may be underrepresented in surveys and previous research has also indicated that participation is lower in health surveys among people with poor mental health [15–17]. If selection into the survey is systematically different between those with and without disorders of interest, observed associations could be biased. This could be relevant, for example, in studies that use survey data to look at the associations between mental disorders and general medical conditions. Conversely, identifying cases based only on registers reflect those who seek help and not all health care provided may be included in register-based research [18]. This may mean surveys are a better source of information for mild mental disorders. Additionally, surveys and registers capture different details: while registers identify those who have received a diagnosis from a health care provider, surveys can provide greater depth by measuring symptoms and indicators of impairment [19–21]. Furthermore, the standardization of self-reported measures can be more culturally sensitive for measuring mental disorders [22]. Combining register data with survey data can fill these gaps [23, 24]. Register data can be supplemented by data from surveys on topics that registers do not capture well (e.g. work patterns [25], social relations [26], over-the-counter medication use [27], smoking [28], or alcohol intake) [28], while the registers can provide long follow-up periods or relevant confounders like education, income, health care utilization, comorbid diagnoses etc. Conversely, we can use the national registers to explore factors that influence selection bias in surveys. For example, survey participants can be compared to the general population to assess how representative participants are [29, 30].

When using a data source to find out information about a population, it is important to try to identify the data source’s limitations and, if possible, quantify them. In this article, we attempt to do this with a Danish population-based survey: How are you? (original Danish title Hvordan har du det?). It is also referred to as the Danish National Health Survey [31] and can be used to supplement the register data [31]. This survey has been run in 2010, 2014, 2017 and 2021. It is a national survey based on five regional stratified random samples and one national random sample, and is concerned with health-related quality of life, health behaviour, morbidity, and social relations in the Danish population. Weighting procedures are used in the survey to correct for survey design and non-response in relation to socio-demographic characteristics, visits to general practitioners, and somatic hospital admissions [31]; these weights ensure that participants are representative of the entire population regarding these characteristics. However, it is unknown the extent to which participants are representative regarding mental disorders. Several previous surveys have suggested that non-participants have more mental disorders than participants [9–12, 32, 33]. As data from How are you? is being used by researchers to describe health in Denmark, we believe it would be useful to consider how representative the survey population is. In this article, we aim to assess the extent of any selection bias with regards to mental disorder diagnoses in the 2017 Central Denmark Region How are you? survey, by comparing the frequency of diagnoses among survey participants, with the entire population eligible to participate in the survey. We make these comparisons by sex, age and type of mental disorder. Additionally, we estimate the association between mental disorders and general medical conditions in survey participants and in the eligible population. It is important to note that the Danish national registers are not free of limitations (as has been previously highlighted) [34, 35], however we hope that triangulation can help us to understand whether those with mental disorder diagnoses are well represented in the survey.

Methods

The Danish national registers

The Danish Civil Registration System holds continuously updated information on all individuals residing in Denmark [36]. Each person in the entire population is assigned a unique Central Person Register number, which allows linkage to person-level data in the other Danish national registers, including the health registers - for example, the National Patient Register [37, 38], which contains information on diagnoses made during hospital visits, or the Danish Psychiatric Central Research Register [39], which contains psychiatric diagnoses made during inpatient, outpatient and emergency visits. An introduction to the Danish national registers is provided by Thygesen and Ersboll [40].

The 2017 central Denmark region How are you? survey

The 2017 Central Denmark Region How are you? survey is a questionnaire-based survey, one of six mutually exclusive random subsamples that cover all of Denmark: one in each of the five Danish administrative regions, and one national sample [31, 41]. It is based on stratified random selection of participants who were 16 years of age or older on 10^th January 2017 and were registered as living within the Central Denmark Region on that date, identified from the Danish Civil Registration System (which includes the entire Danish population). The survey includes questions about participants’ health, including physical health, mental health and health behaviours (such as physical activity, smoking and alcohol intake) [31, 42, 43]. Survey data, such as that from this survey, can also be provided to Statistics Denmark to anonymize the data and link it to the national registers.

Approvals

The study was approved by the Danish Data Protection Agency, and data access was agreed by Statistics Denmark and the Danish Health Data Authority. According to Danish law, informed consent is not required for register-based studies. All data were de-identified and not recognizable at an individual level. The survey was reported to the Register of Research Projects of the Central Denmark Region (record number 1 16 02 593 16). All survey participants were informed that their survey data would be linked to the registers. Linkage was carried out by Statistics Denmark.

Study population and design

This cross-sectional study included the 1,063,082 individuals in Denmark who met the eligibility criteria for participating in the 2017 Central Denmark Region How are you? survey. A total of 52,000 persons were invited to participate in the survey. Invitation to participate was random in terms of all factors (e.g. sex, age, education), but stratification ensured coverage of all municipalities in the region (i.e. municipalities with fewer inhabitants were oversampled). The survey was sent out via secure Digital Post (a system used by Danish authorities and businesses to communicate with the population) or by regular postal services to the 10% of the invitees not registered to use digital post. The subsample not registered to use digital post received a paper questionnaire; these were mostly the elderly [31]. Those who did not respond after a reminder were sent a paper questionnaire and two reminders delivered to their home address. Invitees were informed that participation was voluntary and responses would be kept confidential. They were provided with contact details in case they had further questions and informed that participants would be entered into a prize draw. The response rate of 62% resulted in 32,417 participants. Those invited to participate were a random sample of the entire eligible population; however, it is unclear whether those who agreed to participate were representative of the eligible population. Statistics Denmark had access to information on both participants and non-participants, and they calculated non-response weights to correct for differences in selection probabilities (based on municipality) and response rates. These were constructed using a model-based calibration approach [44] based on information from the national registers on participants and non-participants (i.e., sex, age, municipality of residence, social background, and healthcare utilization through visits to general practitioners and somatic hospitals). These weights are made available with the survey data for use in statistical analyses. Once they are applied, the sample is expected to be representative of the target population at least in relation to the characteristics included in the model; however, these weights did not include direct information on mental disorders.

In this study, we explored the representativeness of the How are you? survey participants through several data obtained from national registers: demographic characteristics, proportion of participants who had been diagnosed with mental disorders before the survey was carried out, and associations between mental disorders and general medical conditions. Given that details of those who were invited to participate but declined cannot be provided for ethical reasons (thus, we could not identify non-responders in the registers), we carried out a comparison of survey participants with the entire eligible population.

A summary of the data obtained from each data source is provided in Supplementary Table 1. As mentioned above, all individuals who met the eligibility criteria to participate in the survey were identified in the Civil Registration System [36]. From here, we obtained details about each individual’s date of birth and sex. We linked them to their health and education data in other registers. Information on mental disorder diagnoses prior to the survey date was obtained from the Danish Psychiatric Central Research Register [39], which contains psychiatric diagnoses made in psychiatric inpatient contacts since 1969, and also in outpatient or emergency contacts since 1995, with a range of health care providers. Prior to 1994, diagnoses were coded according to International Classification of Diseases, version 8 (ICD-8); from 1994 onwards, ICD-10 was used. Mental disorders were ascertained in the period 1969 to January 2017. Details of the specific diagnoses and earliest age of diagnosis within each mental disorder group are presented in Supplementary Table 2 [45].

Information on nine categories of general medical conditions (circulatory, endocrine, pulmonary, gastrointestinal, urogenital, musculoskeletal, hematological, cancer and neurological; defined as in previous studies) [46, 47] was obtained. This was retrieved using the National Patient Register [37, 38] (which comprises data on diagnoses made during inpatient contacts since 1977, and also in outpatient or emergency contacts since 1995) and the Danish National Prescription Register [48] (which includes information on all redeemed prescriptions since 1995). General medical conditions were ascertained after individuals reached 1 year of age, from 1995 onwards, when prescription data became available. Criteria for the general medical conditions are presented in Supplementary Table 3.

Finally, information about the education group (primary/ early childhood, secondary, above secondary, missing) in the year prior to the survey was obtained for each individual. This came from Statistics Denmark’s registers on education [49]. Information through responses from the How are you? survey was not used in this study, as those were only available for survey participants.

Statistical analysis

Statistical analyses were carried out in Stata 16 and Microsoft Excel. For all analyses, we considered three populations: the entire population eligible for the survey, survey participants, and survey participants after adjusting for non-response weights provided by Statistics Denmark. We ascertained the number of people within the categories of each of the following variables as of 10 January 2017: sex (male, female), age group (16–24, 25–34, 35–44, 45–54, 55–64, 65–74, 75+ years), and education group (primary/early childhood, secondary, above secondary, missing). Additionally, we determined the number of people who had received a diagnosis of any mental disorder (ICD-10 F codes) and 10 specific types of mental disorders (each F code subgroup, as per Supplementary Table 2). We carried out Chi-squared tests to compare the frequencies of mental disorders observed in the survey sample, with the frequencies expected if the proportions had been the same as in the entire eligible population. For the observed frequencies, we calculated the frequencies as observed (i.e. unweighted, not taking the non-response weights into consideration), as well as the frequencies with the non-response weights (i.e. weighting each individual as per their assigned non-response weight to make the sample representative).

We then carried out the same descriptive analyses to consider the frequency of any mental disorder in each population disaggregated into fourteen groups depending on sex and age group. When looking at specific types of mental disorders in each population by sex and age, however, due to small numbers, we used broader age categories (16–34, 35–54, 55+ years), resulting in six groups.

Finally, we estimated associations between being diagnosed with any mental disorder and the nine general medical conditions (Supplementary Table 3) [46] through logistic regression models to calculate odd ratios (ORs) and 95% confidence intervals (CIs). These were estimated for the three populations to assess whether the associations were different in survey participants. The models were adjusted for sex, age group and education.

Results

The characteristics of the entire eligible population (1,063,082 individuals in Denmark who met the eligibility criteria for participating in the 2017 Central Denmark Region How are you? survey) and 32,417 survey participants before and after weights were applied are shown in Table 1.

Table 1: Characteristics of all those eligible for the 2017 Central Denmark Region How are you? survey and the survey participants (unweighted and weighted according to sampling and participation weights). ^aComparing the frequencies observed in the unweighted and weighted survey participants, with the frequencies expected if the proportions had been the same as in the entire eligible population.

Entire eligible population(n = 1,063,082)			Survey participants (n = 32,417)
			Unweighted			Weighted
Characteristic	Frequency	% (95% CI)	Frequency	% (95% CI)	P value from Chi squared testa	Frequency	% (95% CI)	P value from Chi squared testa
Sex					<0.001			0.973
Male	529,675	49.82 (49.73–49.92)	15,105	46.60 (46.06–47.14)		16,153	49.83 (49.18–50.47)
Female	533,407	50.18 (50.08–50.27)	17,312	53.40 (52.86–53.94)		16,264	50.17 (49.53–50.83)
Age group					<0.001			0.861
16–24	162,736	15.31 (15.24–15.38)	3,791	11.69 (11.35–12.04)		4,962	15.31 (14.80–15.82)
25–34	158,201	14.88 (14.81–14.95)	3,424	10.56 (10.23–10.89)		4,831	14.90 (14.39–15.42)
35–44	163,108	15.34 (15.27–15.41)	4,620	14.25 (13.88–14.64)		4,979	15.36 (14.89–15.84)
45–54	179,893	16.92 (16.85–17.00)	5,915	18.25 (17.83–18.67)		5,477	16.89 (16.43–17.36)
55–64	159,126	14.97 (14.90–15.04)	5,841	18.02 (17.60–18.44)		4,843	14.94 (14.52–15.37)
65–74	142,027	13.36 (13.30–13.42)	5,703	17.59 (17.18–18.01)		4,402	13.58 (13.19–13.97)
75+	97,991	9.22 (9.16–9.27)	3,123	9.63 (9.32–9.96)		2,925	9.02 (8.68–9.38)
Education					<0.001			0.001
Primary/early childhood	6,319	0.59 (0.58–0.61)	74	0.23 (0.18–0.29)		153	0.47 (0.37–0.60)
Secondary	736,507	69.28 (69.19–69.37)	22,430	69.19 (68.69–69.69)		22,249	68.63 (68.03–69.23)
Above secondary	303,099	28.51 (28.43–28.60)	9,597	29.60 (29.11–30.10)		9,506	29.32 (28.74–29.91)
Missing	17,157	1.61 (1.59–1.64)	316	0.97 (0.86–1.09)		510	1.57 (1.38–1.79)

Compared to the entire eligible population, there was underrepresentation of the following groups in the survey participants: males, individuals <45 years of age, and individuals with education up to primary/early childhood levels or missing education data. However, once the sampling weights were applied, characteristics of participants were more similar in terms of sex and age group.

Based on register data, 10.4% (n = 110,492) of the eligible population had received a diagnosis of any mental disorder prior to the date of the survey (see Supplementary Table 4). Among the unweighted survey sample, 8.2% (n = 2,648) had received a diagnosis; once weights were applied, this corresponded to 9.5%. Figure 1 shows the proportions in the entire eligible population, unweighted survey sample and weighted survey sample who had been diagnosed with any mental disorder and 10 specific types of mental disorders. While those with most types of mental disorders were, at least slightly, underrepresented, the largest relative differences were for organic disorders (prevalence of 0.57% in the eligible population and 0.30% in the weighted survey participants) and substance use disorders (prevalence of 1.87% in the eligible population and 1.37% in the weighted survey participants). For eating disorders (prevalence of 0.46% in the eligible population and 0.53% in the weighted survey participants) and developmental disorders (prevalence of 0.53% in the eligible population and 0.54% in the weighted survey participants), the survey sample was in fact slightly overrepresented, although the differences were not statistically significant according to results from the Chi-squared test.

Figure 1: Prevalence of mental disorder diagnoses between all those eligible for the 2017 Central Denmark Region How are you? survey and the survey participants (before and after weights were applied).

Underrepresentation of those with mental disorder diagnoses in the survey sample was not universal (Figure 2). For example, the largest relative differences were observed for females in the 75+ age group (prevalence of 10.63% in the eligible population and 6.5% in the weighted survey participants) and males in the 75+ age group (prevalence of 6.45% in the eligible population and 4.11% in the weighted survey participants). For females, the weighted survey was generally representative for all other age groups. However, males with mental disorders were slightly underrepresented, especially among the middle-aged groups (25–34, 35–44, and 45–54 years). For the majority of sex and age groups, application of the weights improved representativeness (Supplementary Table 5).

Figure 2: Prevalence of mental disorder diagnoses between all those eligible for the 2017 Central Denmark Region How are you? survey and the the survey participants (before and after weights were applied), by sex and age group.

The results for types of mental disorders by sex and age group are shown in Supplementary Figure 1 and Supplementary Table 6. Again, there was variation in representation of types of mental disorders by sex and age group. For example, people with organic disorders or substance use disorders were underrepresented among survey participants; however, representativeness of common disorders such as mood, neurotic, and behavioral disorders was generally good, especially for females.

Logistic regression models were used to explore the odds of general medical condition comorbidity among those with any mental disorder diagnosis, compared to those without a mental disorder diagnosis, in each of the three populations separately. For example, in all three populations, those with a mental disorder were at increased odds of a circulatory medical condition; although the point estimates differed slightly for each population, the conclusion would be similar. This was the case for most general medical conditions: the ORs were similar for the survey samples (both weighted and unweighted) compared to the entire eligible population (Figure 3 and Supplementary Table 7), suggesting that associations among survey participants were not biased. However, for some general medical conditions, differences were observed e.g. the OR for the entire eligible population was lower than the ORs for the survey samples for endocrine conditions and pulmonary conditions.

Figure 3: Odds ratios (and 95% confidence intervals) for associations between any mental disorder and nine general medical conditions, among all those eligible for the 2017 Central Denmark Region How are you? survey and the survey participants (before and after weights were applied). Adjusted models included sex, age group, and education.

Discussion

By combining the Danish register data with data from the 2017 Central Denmark Region How are you? survey, we have shown that people with a previous diagnosis of a mental disorder are underrepresented in the survey. However, this selection problem is minimized when sampling weights are applied. This pattern is not the same across all age groups, or even all mental disorders, with underrepresentation not observed for eating disorders or developmental disorders, and minimal for others (e.g. personality disorders or behavioral disorders). Additionally, while selection may be an issue for some sex- and age- groups for some mental disorder types, associations between mental disorders and general medical conditions do not appear to be strongly affected by selection bias.

The application of non-response weights meant that the survey provided a sample representative of the entire eligible population in terms of a range of characteristics, including sex and age group. While the survey underrepresents those with mental disorder diagnoses, we found that this bias was relatively small. People diagnosed with any mental disorder represented 10.4% of the entire eligible population; among the weighted survey participants, this was less than 1%-point less (9.5%). The weighted survey participants were also fairly representative in terms of specific types of mental disorders: for the most common types of disorders, neurotic disorders, 5.1% of the entire eligible population had received a diagnosis, compared to 4.7% of the weighted survey participants. The slight overrepresentation of those with eating disorders may reflect characteristics observed in individuals with eating disorders [50]. This variation should be considered by researchers when carrying out surveys that are used to investigate mental health. The response rates to surveys have been generally declining [51, 52], which can affect their generalizability and the representativeness compared to the target populations [53]. However, the How are you? surveys have maintained reasonable response rates, with the 2017 Central Denmark Region survey achieving 62%. Bias may be more impactful in surveys that have low response rates, but applying weights may mitigate this [54]. A recent study considering primary health care utilization in Danish registers and the same survey highlights the importance of applying the calibrated survey weights to address non-response; it concluded that applying the weights reduced the bias caused by differential selection [55].

The observation that people with mental disorder diagnoses are slightly underrepresented in the HHDD survey is in line with descriptions of other surveys. Two surveys in the Dutch population reported that depression and anxiety symptoms were more common in non-participants than participants [32, 33]. A study in the United States found that those with records of substance use were underrepresented [11]. A Finnish register based study reported that subjects (across sexes and education levels) with any psychiatric disorder, as identified in the Finnish Hospital Discharge Register, participated less in their survey than those without psychiatric disorders [9]. Haapea et al. [9] discuss this as possibly resulting due to characteristics associated with some mental disorders e.g. passivity or cognitive impairment, which may be supported by the underrepresentation of people with organic disorders in the Danish survey. Studies of individuals with schizophrenia suggested that longer duration [10] and increased severity [10, 12] of schizophrenia were less likely to participate. While some surveys have managed reasonable response rates among people with organic disorders [56, 57], Paganini-Hill et al. [57] reported that there appeared to be a higher proportion of dementia on death records among non-responders to their survey. For those with cognitive impairments (e.g. dementia, intellectual disabilities, etc), participation may not be possible. Therefore, the survey may not be representative of this group. If researchers want to try to increase participation among people with mental disorders, and in particular these subgroups, they may need to take further measures to target them. Stolzmann et al. [58] discuss that pen-and-paper surveys are generally better for conducting surveys among people with mental disorders, as information technology based-methods have varying levels of use and desirability among those with mental health conditions.

There are limitations of our study that should be considered when interpreting the results. First, the Danish national registers, which we compare the survey to, cannot be considered a ‘gold standard’ for identifying people with mental disorders. The Danish Psychiatric Central Research Register includes information on hospital contacts for mental disorders (inpatient, outpatient, and emergency room visits), but we do not have any information on mental disorders treated only in general practice or by independent psychiatrists and psychologists, or on people who do not seek any medical treatment for their mental disorders. As a result, it is likely that the national registers are more likely to identify more severe cases of mental disorders. Therefore, they may not be able to indicate representation of people with milder cases of mental disorders. Second, it is not possible to identify people who have recovered from mental disorders in the Danish Psychiatric Central Research Register. Our study classifies anyone who had received a mental disorder diagnosis prior to survey eligibility as having a mental disorder; however, it is possible that individuals will no longer have the mental disorder by the time they decided whether to respond to the survey. Therefore, misclassification is possible in both directions. However, it should be considered that by combining the surveys and the registers, we are trying to improve our understanding of the limitations of these data sources – research can, and should, continue to consider these, with the aim of providing estimates of mental disorder prevalence [23, 24]. Third, selection bias could also arise from excess mortality if a disorder is associated with increased risk of premature mortality. However, we defined our study population as being eligible to participate in the survey, and this selection mechanism would have affected both survey participants and the eligible population.

Our work could help guide other researchers with access to both survey and administrative data when considering and designing future studies aimed at measuring the representativeness of survey data. Combining data sources may provide opportunities to better understand our data [59]. This type of survey-register linkage could provide more precise non-response weights in relation to mental disorders, or could use quantitative bias analysis techniques to adjust for selection bias [60]. This investigation of survey data is a useful task to understand our data sources and there is a need to replicate this work in other surveys, and especially in child and youth samples. Additionally, there is a need to explore representativeness across other population indicators, for example other health conditions or immigrant status.

Although people with mental disorder diagnoses are underrepresented in the 2017 Central Denmark Region How are you? survey, this appears to be a relatively minor issue after sampling weights are applied. In other surveys, if researchers want to investigate the prevalence of mental disorders, they need to consider the possibility of underrepresentation, which may reduce a survey’s ability to indicate absolute numbers of mental disorders in a population of interest. However, when looking at the association between any mental disorders and general medical conditions, the results for survey participants (both weighted and unweighted) were similar results to those obtained for the entire eligible population – the 2017 Central Denmark Region How are you? survey appears to provide a good sample to do this type of research.

Declaration of interest

The authors have no declarations of interest.

Funding

The project is supported by the Danish National Research Foundation (Niels Bohr Professorship to John McGrath) and the Lundbeck Foundation (Lundbeck Foundation Fellowship R345-2020-1588 to Oleguer Plana-Ripoll). John McGrath is also supported by the John Cade Fellowship (APP1056929) from the National Health and Medical Research Council. The How are you? survey was funded by the Central Denmark Region.

Acknowledgements

We would like to thank researchers at DEFACTUM for feedback on this study.

Ethics statement

The study was approved by the Danish Data Protection Agency. Access to individual-level Denmark data is governed by Danish authorities. These include the Danish Data Protection Agency, the Danish Health Data Authority, the Ethical Committee, and Statistics Denmark. All data were de-identified and not recognizable at an individual level. The survey was reported to the Register of Research Projects of the Central Denmark Region (record number 1 16 02 593 16). All survey participants were informed that their survey data would be linked to the registers. Linkage was carried out by Statistics Denmark.

References

1. McGrath JJ, Mortensen PB, Whiteford HA. Pragmatic Psychiatric Epidemiology-If You Can’t Count It, It Won’t Count. JAMA Psychiatry. 2018;75(2):111–2. 10.1001/jamapsychiatry.2017.4184

   https://doi.org/10.1001/jamapsychiatry.2017.4184

2. Demyttenaere K, Bruffaerts R, Posada-Villa J, Gasquet I, Kovess V, Lepine JP, et al. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA. 2004;291(21):2581–90. 10.1001/jama.291.21.2581

   https://doi.org/10.1001/jama.291.21.2581

3. Harris MG, Kazdin AE, Munthali RJ, Vigo DV, Hwang I, Sampson NA, et al. Perceived helpfulness of service sectors used for mental and substance use disorders: Findings from the WHO World Mental Health Surveys. Int J Ment Health Syst. 2022;16(1):6. 10.1186/s13033-022-00516-z

   https://doi.org/10.1186/s13033-022-00516-z

4. Kessler RC, Demler O, Frank RG, Olfson M, Pincus HA, Walters EE, et al. Prevalence and treatment of mental disorders, 1990 to 2003. N Engl J Med. 2005;352(24):2515–23. 10.1056/NEJMsa043266

   https://doi.org/10.1056/NEJMsa043266

5. Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ustun TB. The WHO World Mental Health (WMH) Surveys. Psychiatrie (Stuttg). 2009;6(1):5–9.

6. Insights from the Wellcome Global Monitor. London, UK: Wellcome Trust; 2020.

7. Laugesen K, Ludvigsson JF, Schmidt M, Gissler M, Valdimarsdottir UA, Lunde A, et al. Nordic Health Registry-Based Research: A Review of Health Care Systems and Key Registries. Clin Epidemiol. 2021;13:533–54. 10.2147/CLEP.S314959

   https://doi.org/10.2147/CLEP.S314959

8. Kekkonen V, Kivimaki P, Valtonen H, Hintikka J, Tolmunen T, Lehto SM, et al. Sample selection may bias the outcome of an adolescent mental health survey: results from a five-year follow-up of 4171 adolescents. Public Health. 2015;129(2):162–72. 10.1016/j.puhe.2014.11.015

   https://doi.org/10.1016/j.puhe.2014.11.015

9. Haapea M, Miettunen J, Laara E, Joukamaa MI, Jarvelin MR, Isohanni MK, et al. Non-participation in a field survey with respect to psychiatric disorders. Scand J Public Health. 2008;36(7):728–36. 10.1177/1403494808092250

   https://doi.org/10.1177/1403494808092250

10. Cheung P, Schweitzer I, Yastrubetskaya O, Crowley K, Tuckwell V. Studies of aggressive behaviour in schizophrenia: is there a response bias? Med Sci Law. 1997;37(4):345–8. 10.1177/002580249703700411

    https://doi.org/10.1177/002580249703700411

11. Fischer EH, Dornelas EA, Goethe JW. Characteristics of people lost to attrition in psychiatric follow-up studies. J Nerv Ment Dis. 2001;189(1):49–55. 10.1097/00005053-200101000-00009

    https://doi.org/10.1097/00005053-200101000-00009

12. Riedel M, Strassnig M, Muller N, Zwack P, Moller HJ. How representative of everyday clinical populations are schizophrenia patients enrolled in clinical trials? Eur Arch Psychiatry Clin Neurosci. 2005;255(2):143–8. 10.1007/s00406-004-0547-5

    https://doi.org/10.1007/s00406-004-0547-5

13. Pietila AM, Rantakallio P, Laara E. Background factors predicting non-response in a health survey of northern Finnish young men. Scand J Soc Med. 1995;23(2):129–36. 10.1177/140349489502300208

    https://doi.org/10.1177/140349489502300208

14. Lewinsohn PM, Hops H, Roberts RE, Seeley JR, Andrews JA. Adolescent psychopathology: I. Prevalence and incidence of depression and other DSM-III-R disorders in high school students. J Abnorm Psychol. 1993;102:133–44. 10.1037/0021-843X.102.1.133

    https://doi.org/10.1037/0021-843X.102.1.133

15. Green SM, Navratil JL, Loeber R, Lahey BB. Potential dropouts in a longitudinal survey: Prevalence, stability, and associated characteristics. Journal of Child and Family Studies. 1994:69–87. 10.1007/BF02233912

    https://doi.org/10.1007/BF02233912

16. Allott K, Chanen A, Yuen HP. Attrition bias in longitudinal research involving adolescent psychiatric outpatients. J Nerv Ment Dis. 2006;194(12):958–61. 10.1097/01.nmd.0000243761.52104.91

    https://doi.org/10.1097/01.nmd.0000243761.52104.91

17. Noll RB, Zeller MH, Vannatta K, Bukowski WM, Davies WH. Potential bias in classroom research: comparison of children with permission and those who do not receive permission to participate. J Clin Child Psychol. 1997;26(1):36–42. 10.1207/s15374424jccp2601_4

    https://doi.org/10.1207/s15374424jccp2601_4

18. Thielen K, Nygaard E, Andersen I, Rugulies R, Heinesen E, Bech P, et al. Misclassification and the use of register-based indicators for depression. Acta Psychiatr Scand. 2009;119(4):312–9. 10.1111/j.1600-0447.2008.01282.x

    https://doi.org/10.1111/j.1600-0447.2008.01282.x

19. Edwards J, Thind A, Stranges S, Chiu M, Anderson KK. Concordance between health administrative data and survey-derived diagnoses for mood and anxiety disorders. Acta Psychiatr Scand. 2020;141(4):385–95. 10.1111/acps.13143

    https://doi.org/10.1111/acps.13143

20. Kessler RC, Abelson J, Demler O, Escobar JI, Gibbon M, Guyer ME, et al. Clinical calibration of DSM-IV diagnoses in the World Mental Health (WMH) version of the World Health Organization (WHO) Composite International Diagnostic Interview (WMHCIDI). Int J Methods Psychiatr Res. 2004;13(2):122–39. 10.1002/mpr.169

    https://doi.org/10.1002/mpr.169

21. Wittchen HU. Reliability and validity studies of the WHO–Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res. 1994;28(1):57–84. 10.1016/0022-3956(94)90036-1

    https://doi.org/10.1016/0022-3956(94)90036-1

22. Pez O, Gilbert F, Bitfoi A, Carta MG, Jordanova V, Garcia-Mahia C, et al. Validity across translations of short survey psychiatric diagnostic instruments: CIDI-SF and CIS-R versus SCID-I/NP in four European countries. Social psychiatry and psychiatric epidemiology. 2010;45(12):1149–59. 10.1007/s00127-009-0158-6

    https://doi.org/10.1007/s00127-009-0158-6

23. Edwards J, Georgiades K. Reading Between the Lines: A Pursuit of Estimating the Population Prevalence of Mental Illness Using Multiple Data Sources. Can J Psychiatry. 2022;67(2):101–3. 10.1177/07067437211016255

    https://doi.org/10.1177/07067437211016255

24. Edwards TH, Stoll S. A Bayesian approach to quantifying uncertainty from experimental noise in DEER spectroscopy. J Magn Reson. 2016;270:87–97. 10.1016/j.jmr.2016.06.021

    https://doi.org/10.1016/j.jmr.2016.06.021

25. Albertsen K, Hannerz H, Nielsen ML, Garde AH. Shift work and use of psychotropic medicine: a follow-up study with register linkage. Scand J Work Environ Health. 2020;46(4):350–5. 10.5271/sjweh.3872

    https://doi.org/10.5271/sjweh.3872

26. Mikkelsen ASB, Lund R, Kristiansen M. Social relations and healthcare utilisation among middle-aged and older people: study protocol for an implementation and register-based study in Denmark. BMC Health Serv Res. 2017;17(1):728. 10.1186/s12913-017-2650-0

    https://doi.org/10.1186/s12913-017-2650-0

27. Liew Z, Ritz B, Virk J, Olsen J. Maternal use of acetaminophen during pregnancy and risk of autism spectrum disorders in childhood: A Danish national birth cohort study. Autism Research. 2016;9(9):951–8. 10.1002/aur.1591

    https://doi.org/10.1002/aur.1591

28. Allesoe K, Lau CJ, Buhelt LP, Aadahl M. Physical activity, self-rated fitness and stress among 55,185 men and women in the Danish Capital Region Health survey 2017. Prev Med Rep. 2021;22:101373. 10.1016/j.pmedr.2021.101373

    https://doi.org/10.1016/j.pmedr.2021.101373

29. Hansen ABG, Hvidtfeldt UA, Gronbaek M, Becker U, Nielsen AS, Tolstrup JS. The number of persons with alcohol problems in the Danish population. Scand J Public Health. 2011;39(2):128–36. 10.1177/1403494810393556

    https://doi.org/10.1177/1403494810393556

30. Bliddal M, Liew Z, Pottegard A, Kirkegaard H, Olsen J, Nohr EA. Examining Nonparticipation in the Maternal Follow-up Within the Danish National Birth Cohort. Am J Epidemiol. 2018;187(7):1511–9. 10.1093/aje/kwy002

    https://doi.org/10.1093/aje/kwy002

31. Christensen AI, Lau CJ, Kristensen PL, Johnsen SB, Wingstrand A, Friis K, et al. The Danish National Health Survey: Study design, response rate and respondent characteristics in 2010, 2013 and 2017. Scand J Public Health. 2022;50(2):180–8. 10.1177/1403494820966534

    https://doi.org/10.1177/1403494820966534

32. der Wiel AB, van Exel E, de Craen AJ, Gussekloo J, Lagaay AM, Knook DL, et al. A high response is not essential to prevent selection bias: results from the Leiden 85-plus study. J Clin Epidemiol. 2002;55(11):1119–25. 10.1016/s0895-4356(02)00505-x

    https://doi.org/10.1016/s0895-4356(02)00505-x

33. Cheung KL, Ten Klooster PM, Smit C, de Vries H, Pieterse ME. The impact of non-response bias due to sampling in public health studies: A comparison of voluntary versus mandatory recruitment in a Dutch national survey on adolescent health. BMC Public Health. 2017;17(1):276. 10.1186/s12889-017-4189-8

    https://doi.org/10.1186/s12889-017-4189-8

34. Musliner KL, Liu X, Gasse C, Christensen KS, Wimberley T, Munk-Olsen T. Incidence of medically treated depression in Denmark among individuals 15-44 years old: a comprehensive overview based on population registers. Acta Psychiatr Scand. 2019;139(6):548–57. 10.1111/acps.13028

    https://doi.org/10.1111/acps.13028

35. Mason K, Thygesen LC, Stenager E, Bronnum-Hansen H, Koch-Henriksen N. Evaluating the use and limitations of the Danish National Patient Register in register-based research using an example of multiple sclerosis. Acta Neurol Scand. 2012;125(3):213–7. 10.1111/j.1600-0404.2011.01558.x

    https://doi.org/10.1111/j.1600-0404.2011.01558.x

36. Pedersen CB. The Danish Civil Registration System. Scand J Public Health. 2011;39(7 Suppl):22–5. 10.1177/1403494810387965

    https://doi.org/10.1177/1403494810387965

37. Andersen TF, Madsen M, Jorgensen J, Mellemkjoer L, Olsen JH. The Danish National Hospital Register. A valuable source of data for modern health sciences. Dan Med Bull. 1999;46(3):263–8.

38. Schmidt M, Schmidt SA, Sandegaard JL, Ehrenstein V, Pedersen L, Sorensen HT. The Danish National Patient Registry: a review of content, data quality, and research potential. Clin Epidemiol. 2015;7:449–90. 10.2147/CLEP.S91125

    https://doi.org/10.2147/CLEP.S91125

39. Mors O, Perto GP, Mortensen PB. The Danish Psychiatric Central Research Register. Scandinavian Journal of Public Health. 2011;39(7 Suppl):54–7. 10.1177/1403494810395825

    https://doi.org/10.1177/1403494810395825

40. Thygesen LC, Ersboll AK. Danish population-based registers for public health and health-related welfare research: introduction to the supplement. Scand J Public Health. 2011;39(7 Suppl):8–10. 10.1177/1403494811409654

    https://doi.org/10.1177/1403494811409654

41. Christensen AI, Lau CJ, Kristensen PL, Poulsen HS, Breinholt Larsen F. 35 Years of health surveys in Denmark: a backbone of public health practice and research. Scand J Public Health. 2022:14034948221083113. 10.1177/14034948221083113

    https://doi.org/10.1177/14034948221083113
42. Copenhagen: Sundhedsstyrelsen; 2022.
43. Aarhus: DEFACTUM, Region Midtjylland; 2018.
44. New York: Wiley; 2005.

45. Pedersen CB, Mors O, Bertelsen A, Waltoft BL, Agerbo E, McGrath JJ, et al. A comprehensive nationwide study of the incidence rate and lifetime risk for treated mental disorders. JAMA Psychiatry. 2014;71(5):573–81. 10.1001/jamapsychiatry.2014.16

    https://doi.org/10.1001/jamapsychiatry.2014.16

46. Momen NC, Plana-Ripoll O, Agerbo E, Benros ME, Borglum AD, Christensen MK, et al. Association between Mental Disorders and Subsequent Medical Conditions. N Engl J Med. 2020;382(18):1721–31. 10.1056/NEJMoa1915784

    https://doi.org/10.1056/NEJMoa1915784

47. Prior A, Fenger-Gron M, Larsen KK, Larsen FB, Robinson KM, Nielsen MG, et al. The Association Between Perceived Stress and Mortality Among People With Multimorbidity: A Prospective Population-Based Cohort Study. Am J Epidemiol. 2016;184(3):199–210. 10.1093/aje/kwv324

    https://doi.org/10.1093/aje/kwv324

48. Kildemoes HW, Sorensen HT, Hallas J. The Danish National Prescription Registry. Scand J Public Health. 2011;39(7 Suppl):38–41. 10.1177/1403494810394717

    https://doi.org/10.1177/1403494810394717

49. Petersson F, Baadsgaard M, Thygesen LC. Danish registers on personal labour market affiliation. Scand J Public Health. 2011;39(7 Suppl):95–8. 10.1177/1403494811408483

    https://doi.org/10.1177/1403494811408483

50. Sundquist J, Ohlsson H, Winkleby MA, Sundquist K, Crump C. School Achievement and Risk of Eating Disorders in a Swedish National Cohort. J Am Acad Child Adolesc Psychiatry. 2016;55(1):41–6 e1. 10.1016/j.jaac.2015.09.021

    https://doi.org/10.1016/j.jaac.2015.09.021

51. Leeper TJ. Where have the respondents gone? Perhaps we ate them all. Public Opinion Quarterly. 2019;83(S1):280–8. 10.1093/poq/nfz010

    https://doi.org/10.1093/poq/nfz010

52. Harrison S, Alderdice F, Henderson J, Redshaw M, Quigley MA. Trends in response rates and respondent characteristics in five National Maternity Surveys in England during 1995–2018. Arch Public Health. 2020;78:46. 10.1186/s13690-020-00427-w

    https://doi.org/10.1186/s13690-020-00427-w

53. Hupkens CL, van den Berg J, van der Zee J. National health interview surveys in Europe: an overview. Health Policy. 1999;47(2):145–68. 10.1016/s0168-8510(99)00015-9

    https://doi.org/10.1016/s0168-8510(99)00015-9

54. Sun JW, Wang R, Li D, Toh S. Use of Linked Databases for Improved Confounding Control: Considerations for Potential Selection Bias. Am J Epidemiol. 2022;191(4):711–23. 10.1093/aje/kwab299

    https://doi.org/10.1093/aje/kwab299

55. Jensen HAR, Lau CJ, Davidsen M, Feveile HB, Christensen AI, Ekholm O. The impact of non-response weighting in health surveys for estimates on primary health care utilization. Eur J Public Health. 2022. 10.1093/eurpub/ckac032

    https://doi.org/10.1093/eurpub/ckac032

56. Fichter MM, Schroppel H, Meller I. Incidence of dementia in a Munich community sample of the oldest old. Eur Arch Psychiatry Clin Neurosci. 1996;246(6):320–8. 10.1007/BF02189026

    https://doi.org/10.1007/BF02189026

57. Paganini-Hill A, Ducey B, Hawk M. Responders versus nonresponders in a dementia study of the oldest old: the 90+ study. Am J Epidemiol. 2013;177(12):1452–8. 10.1093/aje/kws424

    https://doi.org/10.1093/aje/kws424

58. Stolzmann K, Meterko M, Miller CJ, Belanger L, Seibert MN, Bauer MS. Survey Response Rate and Quality in a Mental Health Clinic Population: Results from a Randomized Survey Comparison. J Behav Health Serv Res. 2019;46(3):521–32. 10.1007/s11414-018-9617-8

    https://doi.org/10.1007/s11414-018-9617-8

59. Plana-Ripoll O, Lasgaard M, Mneimneh ZN, McGrath JJ. The Evolution of Psychiatric Epidemiology: Where to Next? Can J Psychiatry. 2021;66(9):774–7. 10.1177/0706743721996110

    https://doi.org/10.1177/0706743721996110

60. Lash TL, Fox MP, MacLehose RF, Maldonado G, McCandless LC, Greenland S. Good practices for quantitative bias analysis. Int J Epidemiol. 2014;43(6):1969–85. 10.1093/ije/dyu149

    https://doi.org/10.1093/ije/dyu149