Skip to main content
  • Research article
  • Open access
  • Published:

Response rates and selection problems, with emphasis on mental health variables and DNA sampling, in large population-based, cross-sectional and longitudinal studies of adolescents in Norway

Abstract

Background

Selection bias is a threat to the internal validity of epidemiological studies. In light of a growing number of studies which aim to provide DNA, as well as a considerable number of invitees who declined to participate, we discuss response rates, predictors of lost to follow-up and failure to provide DNA, and the presence of possible selection bias, based on five samples of adolescents.

Methods

We included nearly 7,000 adolescents from two longitudinal studies of 18/19 year olds with two corresponding cross-sectional baseline studies at age 15/16 (10th graders), and one cross-sectional study of 13th graders (18/19 years old). DNA was sampled from the cheek mucosa of 18/19 year olds. Predictors of lost to follow-up and failure to provide DNA were studied by Poisson regression. Selection bias in the follow-up at age 18/19 was estimated through investigation of prevalence ratios (PRs) between selected exposures (physical activity, smoking) and outcome variables (general health, mental distress, externalizing problems) measured at baseline.

Results

Out of 5,750 who participated at age 15/16, we lost 42% at follow-up at age 18/19. The percentage of participants who gave their consent to DNA provision was as high as the percentage that consented to a linkage of data with other health registers and surveys, approximately 90%. Significant predictors of lost to follow-up and failure to provide DNA samples in the present genetic epidemiological study were: male gender; non-western ethnicity; postal survey compared with school-based; low educational plans; low education and income of father; low perceived family economy; unmarried parents; poor self-reported health; externalized symptoms and smoking, with some differences in subgroups of ethnicity and gender. The association measures (PRs) were quite similar among participants and all invitees, with some minor discrepancies in subgroups of non-western boys and girls.

Conclusions

Lost to follow-up had marginal impact on the estimated prevalence ratios. It is not likely that the invitation to provide DNA influenced the response rates of 18/19 year olds. Non-western ethnicity, male gender and characteristics related to a low social class and general and mental health problems measured at baseline are associated with lost to follow-up and failure to provide DNA.

Peer Review reports

Background

The general decline in the participation rate in epidemiological studies during recent years [1] may introduce errors in the estimations of exposure and disease occurrence and association measures, and is a major concern for researchers. There are several epidemiological studies investigating factors associated with non-response among adolescents, but studies on the effect of self-selection on association measures, both in cross-sectional and longitudinal designs, are scarce. It is difficult to find discussions of response rates in genetic epidemiological studies and the factors associated with non-response or refusal of agreeing to DNA sampling, even though the same threats to internal validity are in operation as for studies with non-genetic information. This is a concern, as the proportion of epidemiological studies that collect biological specimens increases over time, as reviewed by Morton et al. [2]. This review further reported that less than one-third of epidemiological studies yielded separate participation figures for the biological specimen component of the study. Additionally, we have been able to disclose only one study which has investigated a possible selection bias in the association between genotype and outcome [3], and no studies on factors associated with a refusal to agree to DNA sampling.

Studies of pre-adolescents [4] and adults [5, 6] concluded that selection has little or no impact on the association measures, although such information is insufficient, especially in adolescents and children.

As compared with postal-based, school-based research provides a comparatively inexpensive method of obtaining large samples of children and adolescents with high response rates. In surveys of adolescents under the age of 16, parental consent is required in several countries, including Norway. Tiggers [7] reviewed that active parental consent led to parental permission and response rates in the range of 30%-60% for students biased in the direction of an exclusion of minorities, students having problems in school or students engaged in or at risk for problem behaviours. If passive parental consent is required, parental permission is in the range of 93%-100% [7].

Non-genetic epidemiological studies have reported that adolescents with characteristics associated with poor health have the lowest likelihood of participating in health surveys. Those included lower social status [8]; low maternal education or income [9,10 (young adults)]; being born in a third-world country [11] and those with a less favourable lifestyle [12], including substance abuse [13] and smoking [10 (young adults), [1416]]. Furthermore, adolescents with a young mother [10 (young adults)]; with less favourable [1719] or more favourable [20] relations with peers; living in urban areas [10, 21]; with diagnosed ADHD [22]; low cognitive performance [4, 19, 23]; higher levels of psychopathology [19, 24, 25] and serious psychiatric illness [26] had the lowest participating rate in health surveys.

The aims of the present genetic epidemiological study of adolescents aged 15/16 and 18/19 in the multi-cultural city of Oslo and the rural county of Hedmark were to:

1. Describe response rates across genetic and non-genetic epidemiological studies;

2. Identify predictors of lost to three-year follow-up in a genetic epidemiological study of 18/19-year-old adolescents, with a particular emphasis on gender and ethnicity;

3. Identify predictors of failure to provide DNA in a follow-up of 18/19-year-old adolescents;

4. Investigate the magnitude and direction of possible selection bias in a follow-up at age 18/19 through an investigation of association measures (prevalence ratio) between selected exposures and general and mental health outcome variables measured at baseline (aged 15/16).

Methods

15/16 year olds in 2001: two cross-sectional studies of 10thgraders in Oslo and Hedmark

All students in 10th grade in all 60 and 41 primary schools in Oslo (Figure 1, Sample 1) and Hedmark (Figure 1, Sample 2), respectively, were invited to enter the youth portion of the Oslo Health Study and the Hedmark Health Study, respectively. The data collection was performed at the end of the school year in 2001. All parents received written information and the students signed a consent form before beginning participation. The students completed two four-page questionnaires during two school hours. A project assistant was present in the classroom to inform the students about the survey and to administer the questionnaires. Questionnaires were left at school to be completed by students not present on the day of the survey. Those who did not respond received a copy by mail to their home address, together with a pre-stamped return envelope. Ten years at school is compulsory in Norway; hence, the study included all 15/16 year olds in the study areas. A more detailed description has been published elsewhere [27].

Figure 1
figure 1

Flow chart of the study populations. a27 individuals had an unknown address for the postal-based reminder; b61 individuals had an unknown address and 173 did not consent to be invited to a follow-up; c27 had an unknown address and 229 did not consent to be invited to a follow-up. An additional 55 who gave their consent, but did not fill out the questionnaire at baseline, were invited, of whom 18 participated. They are defined as non-responders at baseline, and are not included in the present analyses.

18/19 year olds in 2004: one cross-sectional study of 13thgraders in Oslo and two longitudinal studies in Oslo and Hedmark

A three-year follow-up study of the two cross-sectional samples of 10th graders in Oslo (Figure 1, Sample 4) and Hedmark (Figure 1, Sample 5) was conducted in 2004, partly in school and partly via mail. In both instances, the participants were invited to give consent to and provide DNA through a sampling of cells from cheek mucosa. The data collection was performed before the end of the school year, and the participants were invited to join a lottery with three sums of NOK 15,000 (i.e. USD 2,470/EUR 1,740).

In Oslo, the follow-up was conducted as a school-based study similar to the baseline, inviting all the final year students (13th graders) in all 32 secondary high schools. This was done in order to reach as many of the baseline participants as possible, representing a cross-sectional study of 13th graders in Oslo (Figure 1, Sample 3). In Norway, all young people between the ages of 16 and 19 have a right to three years of upper secondary education and training funded by the government, and the vast majority of them take advantage of this opportunity. The students filled in a four-page questionnaire and provided a cell sample from their cheek mucosa during one school hour. Because a number of students were not present when the study was conducted, schools were visited several times. Those students who were not reached in school were invited by mail and included in the school-based portion of the follow-up study.

Participants from the baseline study who were not enrolled in the final year of secondary high school in Oslo and who had consented to participate in a follow-up were invited by mail (Figure 1). The package included an invitation letter, an information brochure, a consent form, a questionnaire and two cytobrushes, including a container for buccal cell sampling and a pre-stamped return envelope. Two reminders were sent to those who did not respond.

Methods similar to those applied in the postal-based portion in Oslo were applied for the entire three-year follow-up of all participants from the 2001 baseline study in rural Hedmark (Figure 1).

The youth studies are described more thoroughly elsewhere [28].

Measures

DNA

Two samples of cells from the cheek mucosa were collected from both the left and right side, using two cytobrushes (Medscand Medical AB, Malmö, Sweden) in the studies of 18/19 year olds. In the school-based study, the students were instructed on how to perform the rubbing, which was done simultaneously by all students in the classroom. In the postal-based studies an instruction letter was included, and a plastic tube with a cap and two brushes was returned by mail together with the questionnaire and consent form. The cytobrushes were frozen at -20 degrees C.

Mental health

Externalizing problems

We used the self-report version of the Strengths and Difficulties Questionnaire (SDQ) [29, 30], which is a 25-item wide-angle screening questionnaire with five subscales. Each subscale consists of five items, generating scores for emotional symptoms, conduct problems, hyperactivity-inattention, peer problems and prosocial behaviour. Each item can be answered with "not correct" (0), "partly correct" (1) or "completely correct" (2). We used two of the SDQ subscales: conduct problems and hyperactivityinattention to summarize an index of externalizing problems with a cut-off point at the 90th percentile of the study sample [31].

The SDQ self-report is designed and validated for youngsters (11-16 years old), but has also been used as a valuable instrument for older youths [32, 33]. To adapt the instrument to 18/19 year olds, some small linguistic changes were made in the follow-up questionnaire in accordance with the approved Norwegian translation.

Mental distress

The Hopkins Symptom Checklist (HSCL-10) is comprised of 10 questions regarding psychological symptoms of depression and anxiety (mental distress) experienced the previous week [34]. For each question, there are four possible answers ranging from: "not troubled" (1) to "heavily troubled" (4), and the average score of the items is used as a measure of mental distress [34]. The 10-item version has approximately the same sensitivity and specificity for detecting psychological symptoms or global distress as the more widely used HSCL-25 [3537]. The HSCL-25 version has proven to have a satisfactory validity and reliability as a measure of mental distress in adults [38], and the 10-question version performs almost as well as the longer versions, including among subjects aged 16-24 [34]. An average score for all 10 items equal or above 1.85 was used as the cut-off point for mental distress, corresponding to the 1.75 cut-off of HSCL-25 [34].

General health

Self-evaluated general health status was measured from the question: How would you describe your present state of health? (poor, not very good, good, very good). The categories were operationalized into poor/not very good and good/very good.

Physical activity in leisure time

Physical activity was measured by a question on the amount of weekly hours concerning physical activity outside of school "to an extent that made you sweat and/or out of breath". The answers were rated 0, 1-2, 3-4, 5-7, 8-10 or 11 hours or more per week, and were operationalized in the present study into 0-2 hours per week and more than 2 hours.

Smoking

The question: "Do you smoke or have you smoked earlier?" had four alternatives: never smoking; smoking before, but quit; smoking now and then; and smoking daily. The two middle categories were merged together into one category since the initial results showed that these two groups responded in a similar way.

Ethnic background/Country of origin

Ethnicity was self-reported and determined on basis of the parents' country of birth. Statistics Norway's definition of ethnic minorities, which is those having both parents born in a country other than Norway, was applied [39]: western (one or both parents born in Norway or another western country) vs. non-western (both parents born in a non-western country).

Parental educational level, income and marital status

To obtain information on the parental educational level and income, the questionnaire data was linked to socio-demographic information collected by Statistics Norway for all participants [40, 41]. We applied Statistics Norway's register of highest parental education completed as per Oct. 1, 2000.

The educational level was operationalized into three major groups according to the highest attained educational level: university/college; higher secondary and lower secondary education for the father. Income was operationalized into high (above the 75th percentile), medium (25th to 75th percentile) and low (below the 25th percentile) for the father. The family economic status was self-reported as bad, medium, good or very good, based on a question comparing the family economy with other families in Norway.

The marital status of the parents was dichotomized into married (married/living together) versus not married (unmarried/divorced/separated/one or both parents deceased).

Educational plans

The 15/16 year olds recorded the highest future education they had considered, which was operationalized into: university higher (i.e. university or regional college higher degree); other (university or regional college intermediate level; upper secondary school; vocational education at upper secondary school; one year at upper secondary school; other plans) and not decided. Their educational ambition was used as an indicator of social class of destination.

Invitation group

We created a variable "invitation group", dividing participants into three groups based on participation using "postal-based Hedmark", postal-based Oslo" and "school-based Oslo" as an exposure variable and for adjustment in multivariate analyses.

Statistical methods

Response rates are presented as numbers and percentages for the five studies (Figure 1, Samples 1-5). For all studies of 18/19 year olds in 2004, the numbers and percentages for agreeing to link data to other health surveys and registers, as well as agreeing to provide DNA, are presented. Additionally, for the two longitudinal studies, response and consent rates are presented by gender and ethnicity. The relative risk (RRcrude and RRadjusted) with a corresponding 95% Confidence Interval (CI) for lost to follow-up in 2004 is presented for baseline socio-demographic characteristics (in 2001), using Poisson regression analyses. Data from the two study sites (Hedmark and Oslo) are combined, as there were only small differences in results between the two samples, and we have adjusted for "invitation groups". The variables predicting "lost to follow-up" are similar to those predicting "failure to provide DNA". Thus, we do not present results for "failure to provide DNA", but regard analyses of "lost to follow-up" as a proxy.

Crude and adjusted relative risks (RRs), with a 95% CI of lost to follow-up for selected baseline variables, general health, mental health and risk factors, are presented. Adjustments were done for invitation group, gender, ethnicity, family economy, parental marital status, educational plans and father's education and income using Poisson regression.

The possible effects of selection in attendance on the associations were also assessed based on baseline data. The association (prevalence ratio, PR) between selected risk factors (smoking, physical activity) and three outcome variables (general health, mental distress and externalizing symptoms) are presented by gender and ethnicity among both the participants and all invitees. Analyses were conducted separately by gender and ethnicity due to the large differences in the occurrence of mental health problems. The independent variables were chosen since they are well-known risk factors for adverse health effects and one of them (smoking) is associated with non-response, while the other (physical inactivity) is not.

Analyses were performed using SPSS version 14 and STATA version 10.

Ethics

The study protocol was evaluated by the Regional Committee for Medical Research Ethics and approved by the Norwegian Data Inspectorate. The studies carried out in the schools received approval from the school authorities.

Information from public registers in Statistics Norway about the father's education and income was linked with data from the questionnaire/studies through the individual's personal identification number. All personal identification was erased before the data were analysed.

Results

The samples constituting the present study include data from two cross-sectional school-based studies with a total of 5,750 (88.9% response) 15/16-year-old 10th graders in Oslo (Sample 1, Figure 1) and Hedmark (Sample 2, Figure 1) obtained in 2001, and one corresponding cross-sectional study of 3,308 (90.4% response) 18/19-year-old 13th graders in Oslo (Sample 3, Figure 1) obtained in 2004 (DNA from 3,095) (Table 1). Additionally, we have data on 18/19 year olds from the postal follow-up studies in Oslo (n = 466) and Hedmark (n = 933) in 2004, yielding questionnaire data on a total of 4,707 18/19 year olds, with DNA from 4,305 individuals (Table 1). Finally, based on two longitudinal studies from Oslo and Hedmark, we have three-year follow-up data of a total of 3,668 (70.1% response) 18/19 year olds, with DNA from 3,355 individuals, of whom 3,316 agreed to linkage to the baseline data (Sample 4 and 5, Figure 1). All data obtained in the latter two longitudinal studies is derived from some of the participants in the cross-sectional studies mentioned above (Table 1).

Table 1 Number of invitees and number of participants in the cross-sectional studies in 2001 and 2004, and three-year follow-up of samples from 2001

Response rates

In the studies of 10th graders aged 15/16, in which passive parental consent was obtained, the participation rate was similar to rural Hedmark (88.3%) and urban Oslo (89.2%) (Table 1). The response rate was also quite similar in the cross-sectional school-based study of 13th graders aged 18/19 (90.4%), although collection of DNA from each individual was added as part of the survey. The response rate in the pure postal-based study in Hedmark among 18/19 year olds was considerably lower (55.4%) (Table 1). A total of 2,489 (70.1%) out of 3,550 invited 18/19 year olds in Oslo participated in the follow-up and gave their consent to linkage, while in Hedmark 827 (49.1%) out of 1,683 of those invited did so.

Almost all participants in the school-based survey of 18/19 year olds consented to give their DNA (93.6%), and the rate was also high in the postal-based data collections, at 82% and 88.7% in Oslo and Hedmark, respectively (Table 1). The percentage of participants who consented to give their DNA was about the same as the percentage who agreed to link their data to registers and previous health surveys.

Factors associated with follow-up rates

Lost to follow-up was closely related to failure to provide DNA, and analyses revealed that there were similar predictors operating in the two instances. As a result, the factors associated with lost to follow-up presented in the following are valid for failure to provide DNA (data not shown).

For the purpose of linking data from the two time points, 2,489 (65.3%) out of 3,811 15/16 year olds in Oslo who participated in the baseline study in 2001participated and consented to the linkage of data in 2004 (Table 2). Thus, the lost to follow-up in Oslo was 34.7%. In Hedmark, 827 (42.7%) of the participants in 2001 also participated and agreed to linkage in 2004, yielding a lost to follow-up of 57.3%. More girls than boys and more participants with a Norwegian/western than non-western background participated in the follow-up study (Table 2). In Oslo, 97.7% of participants in the follow-up who consented to linkage of data also agreed to provide DNA, which was quite similar to Hedmark (96.3%) (Table 2).

Table 2 Number of participants in the longitudinal studies1 in Oslo and Hedmark according to gender and ethnicity

There were no major differences between Oslo and Hedmark in socioeconomic predictors for lost to follow-up (data not shown). Therefore, the relative risk of selected socio-demographic baseline factors for lost to follow-up for Hedmark and Oslo are presented as combined (Table 3). Significant predictors were: male gender; non-western ethnicity; postal survey compared to school-based; lower educational plans than university/higher education; low education and income of father; low perceived economy in the family and unmarried as compared to married parents (Table 3). Adjustments for "invitation group" resulted in weaker risk estimates, except for non-western ethnicity which increased from RR = 1.17 (95% CI: 1.08-1.26) to RR = 1.31 (95% CI: 1.21-1.43). Separate analyses by western/non-western ethnicity were conducted. There was no major change in risk estimates for western boys and western girls (results not shown). However, in non-western boys and girls, the father's income or education was not a significant predictor for lost to follow-up, and a poor perceived family economic situation was significantly associated with lost to follow-up in girls only (results not shown).

Table 3 Baseline socio-demographic characteristics in 2001, prevalence and relative risk (RR) of lost to follow-up in 2004 in Oslo and Hedmark combined

The relative risk of selected baseline health and health-related factors for lost to follow-up did not differ in separate analyses between Hedmark and Oslo. In the combined data set, poor self-reported health (borderline significantly), externalized symptoms and smoking were significant predictors for lost to follow-up (Table 4). When stratifying ethnic groups, none of the selected factors were significant predictors for lost to follow-up in non-western boys and girls (data not shown).

Table 4 Baseline health characteristics in 2001, prevalence and relative risk of lost to follow-up in 2004 in Oslo and Hedmark combined

In subgroups of western boys, the predictors were similar to the total sample, with mental distress yielding a significant effect, while in western girls only externalized symptoms and smoking were significant predictors for lost to follow-up (data not shown).

Association measures

In order to investigate a possible distortion in association measures due to lost to follow-up, we examined the prevalence ratios (PRs) between baseline exposure data from 2001 (smoking and physical activity) and selected outcome health variables (self-reported health, mental distress and externalized symptoms) among participants and all invitees in the follow-up in 2004.

Regarding the associations between physical activity and self-reported health, mental distress and externalized symptoms, the prevalence ratios were similar in the groups of participants and all invitees except for self-reported health among non-western girls, in which the PR was 3.0 (1.3-6.7) for participants and 2.0 (1.1-3.6) for all invitees (Table 5). Even so, the number of participants in these groups was low and the confidence intervals were wide. The analyses of association between smoking and self-reported health, mental distress and externalized symptoms gave similar prevalence ratios in the groups of participants and all invitees for both western boys and girls (Table 6). In subgroups of non-western boys and girls, there were differences in PRs and CIs between participants and all invitees. Nevertheless, the number of participants in these groups was low and the confidence intervals were wide (Table 6).

Table 5 Associations between baseline physical activity and baseline self-reported health, mental distress and externalized symptoms among participants in 2004 and all invitees
Table 6 Association between baseline smoking habits and baseline self-reported health, mental distress and externalized symptoms among participants in 2004 and all invitees

Discussion

Main findings

We report similarly high response rates in all the school-based surveys - irrespective of age of the adolescents, year of the study or whether the survey was carried out in urban Oslo or rural Hedmark. In the follow-up, the response rate was markedly higher in Oslo when school-based and postal-based data collection were combined in comparison with the pure postal-based data collection in Hedmark. The rate of participants who gave their consent to DNA provision was as high as the rate of those who consented to linkage of data with other health registers and previous surveys. Significant predictors of lost to follow-up and failure to provide DNA samples were: male gender; non-western ethnicity; postal survey compared with school-based; lower educational plans than university/higher education; low education and income of father; low perceived economy in the family; unmarried as compared with married parents; poor self-reported health; externalized symptoms and smoking, with some differences in subgroups of ethnicity and gender. Regarding the association between selected exposures and outcomes, the main finding was that the association measures (PRs) were quite similar in the groups of participants and all invitees. In subgroups of non-western boys and girls, however, we found some differences, though the pattern was inconsistent, the number in the analysis was small and the confidence intervals of the estimated associations were large.

Response rates

When conducting epidemiological studies, we aim to select samples in which all groups are represented in the study sample in the same way as their representation in the general population. Still, almost every study is hampered by a number of invitees who decline to participate. Any sign of selective attendance in which certain exposed groups are grossly under or overrepresented may incur disturbances to the conclusion. Epidemiological studies with a low level of participation are particularly vulnerable to self-selection bias threatening the internal validity. In the three present cross-sectional, school-based studies, the response rate was approximately 90%. In addition to the advantage with the classroom setting, several other factors may have contributed to the high response rates. Active parental consent was not needed, which in other studies has reduced the response rate to the level of 19%-60% [7, 42]. We used no invasive methods and the data collection only took a little of the participants' time [43, 44]: two hours for the 15/16 year olds and one hour for the 18/19 year olds. Face-to-face recruitment instead of a less personal form of contact between the study recruiter and potential participants may also increase the participation rate [45]. Among adults, monetary incentives may increase the response rate, but the effect on differential study participation is mixed [46]. Greater monetary incentives may have a greater impact on minority and low education individuals participating than on those who are non-minority with a higher education, though in contrast, potential responders with a high income or education may have a greater demand to be compensated for their time [46]. In the present study of 18/19 year olds, an incentive was given by letting participants join a lottery consisting of three sums of NOK 15,000 (i.e. USD 2,470/EUR 1,740), but it is not known whether incentives may bias studies of adolescents or had any impact on the response rate in this study.

Predictors of lost to follow-up

About 10% did not agree to the linkage to other health surveys or registries, including their own baseline, thereby contributing to "lost to follow-up". In the consent form, the question of agreeing to a linkage to their own baseline was written in the same sentence as the linkage to registers. We are not able to rule out whether this mix could be the reason why so many adolescents refused the linkage.

Most of the predictors of lost to follow-up found in the present genetic epidemiological study have previously been reported in surveys of adolescents, and are the ones most often supportive of our findings [10, 21]. In addition, we have found the following predictors of lost to follow-up, which as far as the authors are aware, have not previously been reported: postal survey compared with school-based; lower educational plans than university/higher education and low perceived economy in the family. Most studies report that urban area of living, as compared to rural, predict non-response [10, 21], which is in contrast to the findings of the present study, in which we detected no differences between Oslo and Hedmark in the response rate among 15/16-year-old 10th graders. It could be that school-based studies are less sensitive to location than other settings due to oral information about the purpose of the study [45] and a possible team feeling.

Association measures

In the follow-up studies, the response rate was 65% in Oslo when combining the school-based and postal portion and only 43% in Hedmark, which is a concern regarding internal validity. However, in a respiratory health survey in Norway of 15-70-year olds, early responders were compared with late responders after a first and second reminder and telephone follow-up with respect to prevalence estimates and association measures [6]. The response rates increased from 42.7% to 79.9%, but there were only marginal differences in the exposure-disease relationship and prevalence estimates when initial responders were compared with all responders. This is in accordance with the present study of adolescents, in which we found no marked differences in association measures (PRs) among responders and all invitees, when restricted to western girls and boys. In non-western girls and boys, however, there were differences in the prevalence ratios in the association between exposure to smoking and physical activity on selected outcomes, with no discernable pattern. This could be due in part to the low number of participants in these groups or information bias, i.e. a linguistic or cultural problem in understanding the meaning of any of the questions from the questionnaire [47, 48].

We may also draw support from a Dutch study on pre-adolescents by de Winter et al. [4] regarding western boys and girls in the present study, as well as a warning that prevalence estimates of mental health problems may increase with increasing participation rates. They utilized information from community registers, parents, teachers and classmates in order to investigate a possible bias in association measures and prevalence estimates. Responders were compared with late responders and non-responders, demonstrating that extra efforts to increase the sample size from 66% to 76% prevented an underestimation of the prevalence of psychopathology. Nonetheless, even with differences between non-responders and responders on several individual characteristics, no significant differences were found pertaining to associations between these characteristics and psychopathology [4]. In the present study, mental health was associated with lost to follow-up in the western subgroup, and we also detected that after reminders, late responders reported more mental health problems than early responders [49]. For that reason, we may have underreported the occurrence of mental health in the follow-up.

In a two-year follow-up of 15-18-year-old psychiatric outpatients, it was possible to reach all 101 patients except four, using a comprehensive tracking system [26]. Axis I and II disorders at the two-year follow-up were significantly associated with follow-up contact difficulties, while baseline psychopathology and sociodemographic variables were not. Thus, relying on baseline characteristics of adolescents may underestimate the extent of psychopathology at follow-up. Based on the above mentioned studies, there might be a higher rate of mental health problems among those lost to follow-up compared to participants in our study. Even though we found that relying on baseline information yielded a higher overall prevalence of externalized symptoms in those lost to follow-up compared to participants and overall no difference regarding mental distress, we cannot rule out of whether there was any underestimation of psychopathology at follow-up in the present study.

According to Hartge [50], poor response rates may be of little concern if the willingness to participate is essentially unrelated to exposure. Even if willingness differs with exposure, bias will still not result unless the tendency is stronger (or weaker) in different levels of outcome (i.e. in individuals with disease vs. no disease). According to Kleinbaum et al. [51], even if a willingness to participate is unrelated to exposure, this willingness may be stronger (or weaker) associated with baseline exposure by level of outcome, meaning that selection bias may occur. In the present study, we have investigated whether there are differences in associations measures (PRs) between baseline exposures (smoking, physical activity) and baseline health outcomes (self-reported health, mental distress, externalized symptoms) among participants and all invitees. The use of baseline outcome variables must be regarded as a proxy evaluation of selection bias in associations between baseline exposures and outcomes at follow-up. In our study, a willingness to participate was associated with one of the two exposure variables, namely baseline smoking, but not with baseline physical activity (Table 4). For the total material and in Norwegian/western participants, we detected no selection bias in the association measures (PRs) when utilizing baseline smoking and physical activity as the exposures and selected baseline outcomes (mental distress, externalized symptoms, self-reported health), thereby indicating that the associations between willingness to participate and smoking (and physical activity) are similar by level of outcomes [51]. However, in subgroups of non-western youths (especially in boys) the association between willingness to participate and exposure (particularly with smoking) differed by level of outcome (mental distress, externalized symptoms, self-reported health). So in accordance with Kleinbaum [51], the estimated PRs are biased in these subgroups of participants due to selection, which are indicated with different estimates of PRs between participating and all invited non-western boys and girls (Table 6). However, the number of participants in these groups was low and the confidence intervals were wide.

Unfortunately, it is not possible to conclude that the association between baseline exposures (smoking or physical activity) and selected outcomes measured at follow-up (self-reported health, mental distress, externalized symptoms) are free from selection bias, but if we lean on analyses of the present baseline outcome data and reports from previous studies [4, 6], we may be able to say that there is probably no major selection bias. Regarding subgroups of non-western immigrants, it is more likely that the association measure is biased.

DNA

In the present study, it is not possible to directly assess whether the task of providing DNA has affected the rate of lost to follow-up among 18/19 year olds. However, in the school-based study of 13th graders in which DNA was provided, the response rate was high at 90%, which is similar to the school-based surveys of 10th graders in which DNA sampling was not included. Because of this, it is unlikely that a particular fear of providing DNA played a role in the response rate for the school-based portion of the study. In a qualitative study in the UK [52] of 23-67-year-old participants from an epidemiological health study which collected DNA, it was reported that most of the panel had a positive attitude to medical research and that genetic research in particular was seen as being especially rich in the potential for medical advancement. Other reasons for participating in this genetic health study were: a desire to do good; the possibility of a health gain in the form of a health check; confidence in the research process and its governance and a perception of low risk [52]. The study revealed that the participants had these positive attitudes although most of them misunderstood the aim of the genetic epidemiological study, which was explained in information leaflets. It is not known which factors were in operation for adolescents, and this should be further explored.

To the best of our knowledge, the present study is the first of its kind to investigate predictors of failure to provide DNA. We hypothesize from the present data that there are similar personal reasons behind a willingness to provide DNA and a willingness to agree to linkage of data to registers and health surveys, which should also be further explored. The proposed hypothesis is based on detection of a similarly high response rate in the school-based studies that did or did not collect DNA, with a consent rate as high for providing DNA as for linking data to registers and health surveys.

Conclusions

Studies on the effect of self-selection on association measures in epidemiological studies on adolescents are scarce. For this reason, more studies primarily designed to address this topic are needed. Carefully designed studies on self-selection problems may, however, not be a universal answer or truth for all other studies on adolescents. Consequently, epidemiological studies should be carefully planned to allow a judgement of strength and direction of potential errors due to selection. In the present study, associations between selected exposures and health variables measured by prevalence ratios differed somewhat between the participants and all the invitees for non-western boys and girls, although not for western boys and girls. Further studies that aim at validating instruments and questions in a multicultural setting are recommended, as the difference between ethnic groups could be due to linguistic and cultural differences. In general, however, we conclude that the estimated prevalence ratios in the present study were only marginally influenced by lost to follow-up.

As opposed to most studies on adolescents, we did not find a lower response rate in urban as compared with a rural area, and we conclude with similar ratios among 15/16-year-old adolescents in the school-based studies in urban Oslo and rural Hedmark.

As expected, the response rate was considerably higher in school-based surveys than in postal-based surveys. It is not likely that the invitation to provide DNA has influenced the response rates of 18/19 year olds, especially in the school-based survey. We also conclude that the willingness to provide DNA is slightly lower in a postal-based study as compared with a school-based study and that there were similar proportions of participants who consented to provide DNA and who agreed to the linkage of data to other health surveys and health registers. Non-western ethnicity, male gender and characteristics related to low social class and general and mental health problems measured at baseline were associated with lost to follow-up and failure to provide DNA in the present genetic epidemiological study. The predictors were similar to those of non-genetic epidemiological studies of adolescents.

This study is based on both urban and rural samples, and results regarding lost to follow-up, failure to provide DNA and self-selection on association measures were similar across the various samples. The findings may therefore be generalizable to adolescents living in similar urban and rural areas. However, care should be taken in generalizing findings from the group of non-western ethnicity due to low numbers and corresponding wide confidence intervals of estimates.

References

  1. Galea S, Tracy M: Participation rates in epidemiologic studies. Ann Epidemiol. 2007, 17: 643-653. 10.1016/j.annepidem.2007.03.013.

    Article  PubMed  Google Scholar 

  2. Morton LM, Cahill J, Hartge P: Reporting participation in epidemiologic studies: a survey of practice. Am J Epidemiol. 2006, 163: 197-203. 10.1093/aje/kwj036.

    Article  PubMed  Google Scholar 

  3. Bhatti P, Sigurdson AJ, Wang SS, Chen J, Rothman N, Hartge P, Bergen AW, Landi MT: Genetic variation and willingness to participate in epidemiologic research: data from three studies. Cancer Epidemiol Biomarkers Prev. 2005, 14: 2449-2453. 10.1158/1055-9965.EPI-05-0463.

    Article  PubMed  Google Scholar 

  4. de Winter AF, Oldehinkel AJ, Veenstra R, Brunnekreef JA, Verhulst FC, Ormel J: Evaluation of non-response bias in mental health determinants and outcomes in a large sample of pre-adolescents. Eur J Epidemiol. 2005, 20: 173-181. 10.1007/s10654-004-4948-6.

    Article  PubMed  Google Scholar 

  5. Sogaard AJ, Selmer R, Bjertness E, Thelle D: The Oslo Health Study: The impact of self-selection in a large, population-based survey. Int J Equity Health. 2004, 3: 3-10.1186/1475-9276-3-3.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Brøgger J, Bakke P, Eide GE, Gulsvik A: Contribution of follow-up of nonresponders to prevalence and risk estimates: a Norwegian respiratory health survey. Am J Epidemiol. 2003, 157: 558-566. 10.1093/aje/kwg003.

    Article  PubMed  Google Scholar 

  7. Tigges BB: Parental consent and adolescent risk behavior research. J Nurs Scholarsh. 2003, 35: 283-289. 10.1111/j.1547-5069.2003.00283.x.

    Article  PubMed  Google Scholar 

  8. Lewinsohn PM, Hops H, Roberts RE, Seeley JR, Andrews JA: Adolescent psychopathology: I. Prevalence and incidence of depression and other DSM-III-R disorders in high school students. J Abnorm Psychol. 1993, 102: 133-144. 10.1037/0021-843X.102.1.133.

    Article  CAS  PubMed  Google Scholar 

  9. Hille ET, Elbertse L, Gravenhorst JB, Brand R, Verloove-Vanhorick SP: Nonresponse bias in a follow-up study of 19-year-old adolescents born as preterm infants. Pediatrics. 2005, 116: e662-e666. 10.1542/peds.2005-0682.

    Article  CAS  PubMed  Google Scholar 

  10. Pietila AM, Rantakallio P, Laara E: Background factors predicting non-response in a health survey of northern Finnish young men. Scand J Soc Med. 1995, 23: 129-136.

    CAS  PubMed  Google Scholar 

  11. Badawi MA, Eaton WW, Myllyluoma J, Weimer LG, Gallo J: Psychopathology and attrition in the Baltimore ECA 15-year follow-up 1981-1996. Soc Psychiatry Psychiatr Epidemiol. 1999, 34: 91-98. 10.1007/s001270050117.

    Article  CAS  PubMed  Google Scholar 

  12. Mattila VM, Parkkari J, Rimpela A: Adolescent survey non-response and later risk of death. A prospective cohort study of 78,609 persons with 11-year follow-up. BMC Public Health. 2007, 7: 87-10.1186/1471-2458-7-87.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Siddiqui O, Flay BR, Hu FB: Factors affecting attrition in a longitudinal smoking prevention study. Prev Med. 1996, 25: 554-560. 10.1006/pmed.1996.0089.

    Article  CAS  PubMed  Google Scholar 

  14. Cunradi CB, Moore R, Killoran M, Ames G: Survey nonresponse bias among young adults: the role of alcohol, tobacco, and drugs. Subst Use Misuse. 2005, 40: 171-185. 10.1081/JA-200048447.

    Article  PubMed  Google Scholar 

  15. Morrison TC, Wahlgren DR, Hovell MF, Zakarian J, Burkham-Kreitner S, Hofstetter CR, Slymen DJ, Keating K, Russos S, Jones JA: Tracking and follow-up of 16,915 adolescents: minimizing attrition bias. Control Clin Trials. 1997, 18: 383-396. 10.1016/S0197-2456(97)00025-1.

    Article  CAS  PubMed  Google Scholar 

  16. Pirie PL, Murray DM, Luepker RV: Smoking prevalence in a cohort of adolescents, including absentees, dropouts, and transfers. Am J Public Health. 1988, 78: 176-178. 10.2105/AJPH.78.2.176.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Frame CL, Strauss CC: Parental informed consent and sample bias in grade-school children. Journal of Social & Clinical Psychology. 1987, 227-236.

    Google Scholar 

  18. Gerrits MH, van den Oord EJ, Voogt R: An evaluation of nonresponse bias in peer, self, and teacher ratings of children's psychosocial adjustment. J Child Psychol Psychiatry. 2001, 42: 593-602. 10.1111/1469-7610.00755.

    Article  CAS  PubMed  Google Scholar 

  19. Noll RB, Zeller MH, Vannatta K, Bukowski WM, Davies WH: Potential bias in classroom research: comparison of children with permission and those who do not receive permission to participate. J Clin Child Psychol. 1997, 26: 36-42. 10.1207/s15374424jccp2601_4.

    Article  CAS  PubMed  Google Scholar 

  20. Brandenburg NA, Friedman RM, Silver SE: The epidemiology of childhood psychiatric disorders: prevalence findings from recent studies. J Am Acad Child Adolesc Psychiatry. 1990, 29: 76-83. 10.1097/00004583-199001000-00013.

    Article  CAS  PubMed  Google Scholar 

  21. Cottler LB, Zipp JF, Robins LN, Spitznagel EL: Difficult-to-recruit respondents and their effect on prevalence estimates in an epidemiologic survey. Am J Epidemiol. 1987, 125: 329-339.

    CAS  PubMed  Google Scholar 

  22. Green SM, Navratil JL, Loeber R, Lahey BB: Potential dropouts in a longitudinal study: Prevalence, stability, and associated characteristics. Journal of Child and Family Studies. 1994, 69-87. 10.1007/BF02233912.

    Google Scholar 

  23. Esser G, Schmidt MH, Woerner W: Epidemiology and course of psychiatric disorders in school-age children--results of a longitudinal study. J Child Psychol Psychiatry. 1990, 31: 243-263. 10.1111/j.1469-7610.1990.tb01565.x.

    Article  CAS  PubMed  Google Scholar 

  24. Kessler RC, Little RJ, Groves RM: Advances in strategies for minimizing and adjusting for survey nonresponse. Epidemiol Rev. 1995, 17: 192-204.

    CAS  PubMed  Google Scholar 

  25. Haavisto A, Sourander A, Multimaki P, Parkkola K, Santalahti P, Helenius H, Nikolakaros G, Kumpulainen K, Moilanen I, Piha J, et al: Factors associated with depressive symptoms among 18-year-old boys: a prospective 10-year follow-up study. J Affect Disord. 2004, 83: 143-154. 10.1016/j.jad.2004.06.008.

    Article  PubMed  Google Scholar 

  26. Allott K, Chanen A, Yuen HP: Attrition bias in longitudinal research involving adolescent psychiatric outpatients. J Nerv Ment Dis. 2006, 194: 958-961. 10.1097/01.nmd.0000243761.52104.91.

    Article  PubMed  Google Scholar 

  27. The Oslo Health Study (HUBRO) - The Youth part (UNGHUBRO): Methods. Accessed 30.09.10, [http://www.fhi.no/eway/default.aspx?pid=233&trg=MainArea_5661&MainArea_5661=5631:0:15,4130:1:0:0:::0:0]

  28. Youth 2004: Methods. Accessed 30.09.10, [http://www.fhi.no/eway/default.aspx?pid=233&trg=MainArea_5661&MainArea_5661=5631:0:15,4377:1:0:0:::0:0]

  29. Goodman R: Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001, 40: 1337-1345. 10.1097/00004583-200111000-00015.

    Article  CAS  PubMed  Google Scholar 

  30. Information for researchers and professionals about the Strengths & Difficulties Questionnaires. Accessed 30.09.10, [http://www.sdqinfo.org/]

  31. Lien L, Haavet OR, Thoresen M, Heyerdahl S, Bjertness E: Mental health problems, negative life events, perceived pressure and the frequency of acute infections among adolescents. Results from a cross-sectional, multicultural, population-based study. Acta Paediatr. 2007, 96: 301-306. 10.1111/j.1651-2227.2007.00084.x.

    Article  PubMed  Google Scholar 

  32. Van Roy B, Groholt B, Heyerdahl S, Clench-Aas J: Self-reported strengths and difficulties in a large Norwegian population 10-19 years: age and gender specific results of the extended SDQ-questionnaire. Eur Child Adolesc Psychiatry. 2006, 15: 189-198. 10.1007/s00787-005-0521-4.

    Article  PubMed  Google Scholar 

  33. Van RB, Veenstra M, Clench-Aas J: Construct validity of the five-factor Strengths and Difficulties Questionnaire (SDQ) in pre-, early, and late adolescence. Journal of Child Psychology & Psychiatry & Allied Disciplines. 2008, 49: 1304-1312.

    Article  Google Scholar 

  34. Strand BH, Dalgard OS, Tambs K, Rognerud M: Measuring the mental health status of the Norwegian population: a comparison of the instruments SCL-25, SCL-10, SCL-5 and MHI-5 (SF-36). Nord J Psychiatry. 2003, 57: 113-118. 10.1080/08039480310000932.

    Article  PubMed  Google Scholar 

  35. Lipman RS, Covi L, Shapiro AK: The Hopkins Symptom Checklist (HSCL)--factors derived from the HSCL-90. J Affect Disord. 1979, 1: 9-24. 10.1016/0165-0327(79)90021-1.

    Article  CAS  PubMed  Google Scholar 

  36. Hesbacher PT, Rickels K, Morris RJ, Newman H, Rosenfeld H: Psychiatric illness in family practice. J Clin Psychiatry. 1980, 41: 6-10.

    CAS  PubMed  Google Scholar 

  37. Tambs K: Moderate effects of hearing loss on mental health and subjective well-being: results from the Nord-Trondelag Hearing Loss Study. Psychosom Med. 2004, 66: 776-782. 10.1097/01.psy.0000133328.03596.fb.

    Article  PubMed  Google Scholar 

  38. Glass RM, Allan AT, Uhlenhuth EH, Kimball CP, Borinstein DI: Psychiatric screening in a medical clinic. An evaluation of a self-report inventory. Arch Gen Psychiatry. 1978, 35: 1189-1195.

    Article  CAS  PubMed  Google Scholar 

  39. Lie B: Immigration and immigrants 2002. 2002, Oslo: Statistisk sentralbyrå

    Google Scholar 

  40. Norwegian Standard Classification of Education (NOS C 751).

  41. Income and Property Statistics for Households 2002 (NOS D 310).

  42. Geluda K, Bisaglia JB, Moreira V, Maldonado BM, Cunha AJ, Trajman A: Third-party informed consent in research with adolescents: the good, the bad and the ugly. Soc Sci Med. 2005, 61: 985-988. 10.1016/j.socscimed.2004.11.061.

    Article  PubMed  Google Scholar 

  43. Mott DA, Pedersen CA, Doucette WR, Gaither CA, Schommer JC: A national survey of U.S. pharmacists in 2000: assessing nonresponse bias of a survey methodology. AAPS PharmSci. 2001, 3: E33-10.1208/ps030433.

    Article  CAS  PubMed  Google Scholar 

  44. Groves RM: Survey errors and survey costs. 2004, New York: Wiley

    Google Scholar 

  45. Bongers IM, Van Oers JA: Mode effects on self-reported alcohol use and problem drinking: mail questionnaires and personal interviewing compared. J Stud Alcohol. 1998, 59: 280-285.

    Article  CAS  PubMed  Google Scholar 

  46. Singer E: The Use of Incentives to Reduce Nonresponse in Household Surveys. Survey nonresponse. Edited by: Groves RM, Dillman DA, Eltinge J, Little RJA. 2002, New York: Wiley, 163-177.

    Google Scholar 

  47. Pasick RJ, Stewart SL, Bird JA, D'Onofrio CN: Quality of data in multiethnic health surveys. Public Health Rep. 2001, 116 (Suppl 1): 223-243. 10.1093/phr/116.S1.223.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Warnecke RB, Johnson TP, Chavez N, Sudman S, O'Rourke DP, Lacey L, Horm J: Improving question wording in surveys of culturally diverse populations. Ann Epidemiol. 1997, 7: 334-342. 10.1016/S1047-2797(97)00030-6.

    Article  CAS  PubMed  Google Scholar 

  49. Sagatun A, Lien L, Søgaard AJ, Bjertness E, Heyerdahl S: Ethnic Norwegian and ethnic minority adolescents in Oslo, Norway. A longitudinal study comparing changes in mental health. Soc Psychiatry Psychiatr Epidemiol. 2008, 43: 87-95. 10.1007/s00127-007-0275-z.

    Article  PubMed  Google Scholar 

  50. Hartge P: Participation in population studies. Epidemiology. 2006, 17: 252-254. 10.1097/01.ede.0000209441.24307.92.

    Article  PubMed  Google Scholar 

  51. Kleinbaum DG, Kupper LL, Morgenstern H: Epidemiologic research: principles and quantitative methods. 1982, New York: Van Nostrand Reinhold

    Google Scholar 

  52. dixon-Woods M, Ashcroft RE, Jackson CJ, Tobin MD, Kivits J, Burton PR, Samani NJ: Beyond "misunderstanding": written information and decisions about taking part in a genetic epidemiology study. Soc Sci Med. 2007, 65: 2212-2222. 10.1016/j.socscimed.2007.08.010.

    Article  PubMed  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The data collection was carried out and funded by the University of Oslo and the Norwegian Institute of Public Health. The City of Oslo contributed to funding of the baseline study while The Centre for Child and Adolescent Mental Health, Eastern and Southern Norway, Oslo, contributed to funding of the follow-up study.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Espen Bjertness.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

EB was the project manager and responsible for the conception of the follow-up project, conceived the present study, took part in the design of the study and interpretation of the data, and drafted the manuscript. ÅS took part in the planning and design of the follow-up project, coordinated the practical part of the project, took part in the data collection and design of the study, discussed the analysis and interpretation of the data, and reviewed the article critically. KG and LL took part in the planning, design and data collection of the study, discussed the analysis and interpretation of the data, and reviewed the article critically. AJS was project manager of the baseline study in Oslo, participated in the planning and design of the follow-up project, took part in the design of the study, discussed the analysis and interpretation of the data, and reviewed the article critically. RS took part in the collecting of the baseline data, carried out all the statistical analyses of the study, took part in the planning and design of the study as well as the interpretation of the data, and reviewed the article critically. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Bjertness, E., Sagatun, Å., Green, K. et al. Response rates and selection problems, with emphasis on mental health variables and DNA sampling, in large population-based, cross-sectional and longitudinal studies of adolescents in Norway. BMC Public Health 10, 602 (2010). https://doi.org/10.1186/1471-2458-10-602

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2458-10-602

Keywords