In many GAVI-eligible countries, effectiveness of new vaccines will be evaluated by case-control methodology. To inform the design and assess selection bias of a future case-control study of rotavirus vaccine effectiveness (VE) in western Kenya, we performed a sham case-control study evaluating VE of pentavalent vaccine (DTP-Hib-HepB) against rotavirus acute gastroenteritis (AGE).
From ongoing rotavirus surveillance, we defined cases as children 12 weeks to 23 months old with EIA-confirmed rotavirus AGE. We enrolled one community-based and two hospital-based control groups. We collected vaccination status from cards at enrollment, or later in homes, and evaluated VE by logistic regression.
We enrolled 91 cases (64 inpatient, 27 outpatient), 252 non-rotavirus AGE facility-based controls (unmatched), 203 non-AGE facility-based controls (age-matched) and 271 community controls (age-matched). Documented receipt of 3 pentavalent doses was 77% among cases and ranged from 81-86% among controls. One percent of cases and 0-2% of controls had no pentavalent doses. The adjusted odds ratio of three versus zero doses for being a case was 3.27 (95% CI 0.01-1010) for community controls and 0.69 (95% CI 0.06-7.75) for non-rotavirus hospital-based AGE controls, translating to VE of -227% and 31%, respectively, with wide confidence intervals. (No facility-based non-AGE controls were unvaccinated.) Similar results were found for ≥2 pentavalent doses and for severe rotavirus AGE.
The study showed that it is feasible to carry out a real case control in the study area, but this needs to be done as soon as the vaccine is introduced to capture the real impact. Sham case-control or pilot studies before vaccine introduction can be useful in designing case-control VE studies.
Keywords:Rotavirus; Case-control; Vaccine effectiveness
Rotavirus causes approximately 453,000 deaths annually in children, most of them occurring in sub-Saharan Africa and Asia . By December 2012, 13 of 73 countries eligible for funding through the Global Alliances for Vaccines and Immunizations (GAVI) had introduced rotavirus vaccines into their national immunization programs and 20 more countries, including Kenya, had been approved to receive GAVI funding for rotavirus vaccine [2,3].
As more developing countries introduce rotavirus vaccines, there is growing interest in evaluating their impact to support decisions for continued and augmented country-level financing. In many countries with on-going rotavirus surveillance, rotavirus vaccine effectiveness (VE) will be evaluated by case-control methodology [4-6]. Case-control studies for evaluation of VE have the advantage of being relatively quick, not requiring pre-vaccine data or a well-defined denominator population, and being efficient in the setting of rare diseases [6,7]. Case-control studies have been utilized to evaluate rotavirus VE in Nicaragua and El Salvador [8-10], as well as other vaccines in various settings [7,11-13]. While having advantages, case-control studies are inherently at risk of selection bias due to the challenges of selecting a sample of controls representative of the population from which the cases arose [7,14,15]. If controls differ in their propensity to be vaccinated or to seek medical care from the source population of cases, then the association between vaccine and disease can be distorted, leading to biased estimates of VE [7,16]. Besides selection bias, case-control studies can suffer from misclassification of exposure (i.e. vaccination status).
We used ongoing surveillance for rotavirus gastroenteritis in rural western Kenya to perform a case-control study evaluating effectiveness of pentavalent vaccine (diphtheria, pertussis, tetanus Haemophilus influenzae type b (Hib), and hepatitis B) on rotavirus gastroenteritis. Since pentavalent vaccine should have no protection against rotavirus disease, we refer to this study as a “sham” case-control study, and as such, after adjusting for potential confounders, any “vaccine effectiveness” could be ascribed to residual selection bias, and could be useful in interpreting results of a real case-control study of rotavirus VE undertaken after the vaccine’s introduction into Kenya. Besides assessing selection bias, our study aimed to define characteristics of the population (e.g. vaccine coverage and timeliness) and the design (e.g. ability to document vaccine status) that would inform the design and execution of a real case-control study of rotavirus vaccine after its introduction into Kenya.
The study was conducted in Siaya County, western Kenya, in the setting of a long-standing Health and Demographic Surveillance System (HDSS) . The area is rural, poor, has high malaria and HIV prevalence, and a high under-5 mortality ratio [17-19].
Surveillance for acute gastroenteritis (AGE) among children <5 years of age was conducted among inpatients at a district hospital and outpatients at two health centers located in the HDSS area. AGE was defined as ≥3 looser than normal stools and/or >1 episode of unexplained vomiting followed by loose stool within a 24-hour period beginning within seven days before seeking healthcare. Stool was collected in a plastic diaper from which at least 2 ml of stool was scooped into a specimen container. In the outpatient facilities, if the child was unable to provide stool during the visit, a field worker would follow-up at the child’s home to collect stool. All stool samples were collected within 48 hours after presentation, stored and transported in cool boxes on the same day of collection to KEMRI/CDC laboratories, approximately 60 kilometers away. Batch testing for rotavirus using enzyme-immunoassay (EIA, Rotaclone™ Kit) was done.
The case-control study was embedded in ongoing rotavirus surveillance system. Eligible children were 12 weeks (i.e. 2 weeks after the second scheduled Expanded Program for Immunization visit) to 23 months of age who presented with AGE, as these children are eligible to have received pentavalent vaccine. Among these, cases were defined as those with an EIA-confirmed rotavirus-positive stool sample. Severe AGE was defined using the 20-point numerical Vesikari scoring system with scores of ≥11 . Case selection at the sites was consecutive until the desired sample size was reached. Inpatients were enrolled continuously, including during nights and weekends. Outpatients were enrolled during weekday daytime hours.
We enrolled two hospital and one community-based control groups. The first hospital control group was unmatched, consisting of children presenting to the same facility as the cases, who had AGE not caused by rotavirus, as determined by a negative rotavirus EIA test. The second hospital control group was children presenting with a non-AGE illness, excluding those who had other EPI-vaccine-preventable diseases (i.e. measles, pneumonia, meningitis, sepsis, otitis media, bacteraemia, epiglottitis, hepatitis and pertussis). In a 3:1 ratio, controls were age-matched to cases within -15 to +60 days of the case-patient’s birthday. Eligible controls were enrolled starting from the admission closest in date to the case’s admission date, until three controls were enrolled. The third control group was selected from the same villages as the case-patients, but not living in the same compound. Starting at the case’s compound, we went to the nearest compound on either side of the case’s compound to check for eligible children. If no children met eligibility criteria in that compound, we went to the adjacent compound in the same direction until we enrolled a control. After the first control, we skipped the next compound and inquired about eligible controls at the following compound. We repeated this process until we enrolled three controls.
HIV testing was done using parallel rapid test kits on whole blood obtained from finger-prick --Determine HIV-1/2 (Abbott Diagnostic Division, Hoofdorp, Netherlands) and Bioline® (Standard Diagnostics Inc, Korea) - with tie-breaking of discordant results using Unigold™ (Trinitiy Biotech PLC, Ireland). PCR testing confirmed HIV status for children <18 months old who were antibody-positive on at least one rapid test.
During an interview with trained study staff, the parent was asked for each child’s Ministry of Health-issued vaccination card. For children who came to the hospital without their vaccination card, we visited their home and collected the immunization history from available cards. If the card was not available, a verbal report of the child’s vaccination status was collected. Other data collected included history, symptoms and signs of illness for cases, parent’s education level, household characteristics and possessions, and distance to the nearest health facility.
For the sham case-control study, we did not expect to find statistically significant “effectiveness” of the pentavalent vaccine against rotavirus AGE, but rather to determine if the study found a point estimate that gave an indication of potential bias (i.e. the odds ratio was significantly different from 1.0). To estimate enrollment duration for a real case-control study, we determined the sample size based on a rotavirus VE evaluation. While coverage with two Pentavalent doses is 94% in the area , we assumed that in the first year of rotavirus vaccine introduction, coverage among infants would be lower due to obstacles of rolling out the vaccine. We therefore estimated 50% coverage. Using these parameters, we needed 61 cases and183 controls in each of the three control groups (3:1 control to case ratio) to detect a VE of 60% in the first year of life. To account for missing data, we increased the sample size by 10%, to 67 cases and 201 controls per control group.
A dose of pentavalent was considered valid (i.e. immunologically protective) if it was administered ≥14 days before the date of admission for the case, 14 days before the date of admission for the case for the matched controls and 14 days before the control child’s admission for the unmatched hospital controls.
To evaluate socioeconomic status (SES), we constructed quintiles based on eight indicator variables of asset ownership which were scored and reduced to a single score through principal component analysis . We considered risk factors for rotavirus AGE using community controls only because comparison between cases and the two hospital control groups evaluated rotavirus AGE risk against other diagnoses, which has less meaningful epidemiologic interpretation. Comparison between rotavirus AGE cases and hospital controls was done only to identify variables for adjustment in logistic regression.
The primary analysis included only documented reports of vaccine status; having zero doses was determined from either vaccination card or verbal report as it was expected that unvaccinated children may be less likely to have cards, which are usually given at the first vaccination in the clinic. The main analysis compared children who had received three doses versus zero doses. Secondary analyses included children who had received at least two documented doses versus zero doses. For the analysis of the matched control groups, conditional logistic regression was used; all variables with a p value of <0.10 in univariate analysis were incorporated in multivariable analysis using backwards elimination. Variables with more than 5% unknown responses were included in the model with “unknown” coded as a category of a dummy variable. Children with unknown vaccine status were always excluded from the model. VE was calculated as ((1-Odds Ratio) X 100). We also did an unmatched analysis of cases and non-rotavirus facility-based AGE controls, controlling for age in months and seasonality (4 quarters of the year) using unconditional logistic regression. We reran the analysis for only severe cases. Lastly, we evaluated timely vaccination, defined as having received the vaccine between one week before and two weeks after the scheduled date of vaccination, which in Kenya is 6, 10 and 14 weeks for the primary EPI series.
This study was approved by the Kenya Medical Research Institute’s ethics review committee (SSC#1836). Written informed consents were obtained from all participants or their legal guardians prior to enrolment.
Case enrollment occurred from December 1, 2010 through November 30, 2011. There were 507 AGE hospitalizations among children <5 years old and 232 AGE sick visits in the outpatient facilities, of which, 409 (81%) and 176 (76%) were among children 3–23 months of age, respectively (Figure 1). We obtained stool specimens from 289 (71%) age-eligible inpatients and 118 (67%) age-eligible outpatients. Severe AGE occurred in 197 (68%) inpatients and 35 (30%) outpatients (Table 1). Among inpatients, stool samples were more likely to be collected from children <12 months of age and with severe AGE (Table 1). No difference in stool collection was seen among outpatients. Among the 3–23 month old inpatients and outpatients with AGE who had stool collected, 80 (28%) and 32 (27%), respectively, were positive for rotavirus (Figure 1). Five (6%) hospitalized rotavirus-positive cases died during their admission.
Figure 1. Flow chart of rotavirus surveillance and enrollment of cases in case–control study from December 2010 to November 2011.
Table 1. Characteristics of children with AGE with and without stool collection, western Kenya, 2010-11
Among age-eligible rotavirus AGE cases, 64 (80%) inpatients and 27 (84%) outpatients were enrolled (Figure 1). We enrolled 252 AGE hospital controls without rotavirus detected, 203 non-AGE hospital controls and 271 community controls. In multivariable analysis, comparing rotavirus AGE cases to community controls having a father with less education , a mother currently married, and being moderately-to-severely stunted were protective against being a case, while being in the least poor SES quintiles and not attending daycare were significant risk factors (Table 2).
Table 2. Univariate analysis comparing risk factors among rotavirus AGE cases and controls, western Kenya, 2010-11
The percentage of cases with documented vaccination status was 83%, which was lower than among community controls (90%) and higher than hospital controls (74-77%). Among cases, 77% had received 3 doses of pentavalent by documented report and only 1% were unvaccinated (Table 3). Among control groups, this proportion with 3 doses varied slightly -- 86% for community controls, 81% for facility-based non-AGE controls, and 83% for unmatched facility-based AGE controls, according to the vaccine card with only 0% - 2% unvaccinated.
Table 3. Table comparing immunization status among rotavirus AGE cases and controls, western Kenya, 2010–11
The adjusted odds ratio of three doses versus zero doses was 3.27 (95% CI 0.01-1010) for community controls and 0.69 (95% CI 0.06-7.75) for facility-based AGE controls; this translates to VE of -227% and 31%, respectively, with very wide confidence intervals (Table 4). (No facility-based non-AGE controls were unvaccinated, thus this analysis could not be performed.) Similar results were found when evaluating ≥2 doses of pentavalent vaccine versus zero doses and when restricting cases to severe rotavirus AGE (Table 4).
Table 4. Univariate and multivariable analysis comparing immunization status for rotavirus AGE cases versus controls, western Kenya, 2010–11 (documented immunization status only)
Among cases with documented timing of pentavalent immunization, 83%, 70% and 69% received timely vaccination for pentavalent doses 1, 2 and 3, respectively (Figure 2). Similar timing of vaccination was seen among all control groups.
Figure 2. Distribution of age of vaccination for dose 1, 2 & 3 of pentavalent vaccine by cases and controls. a, b, c Vertical dotted line represents scheduled date at 6, 10 & 14 weeks. Grey shaded area is considered timely vaccination.
Our sham case-control study provided several important lessons that can be applied to a case-control study of rotavirus VE in the same setting after vaccine introduction, expected in 2014 . First and foremost, a major challenge was finding very few unvaccinated children, which limited the study’s power to make any conclusions about bias. Given the high coverage levels of routine immunizations in Kenya, unvaccinated children may not be epidemiologically representative of children in this population. An evaluation of rotavirus VE in Nicaragua found that small differences in vaccine coverage resulted in substantially disparate estimates of VE when coverage was high (≥95% vaccinated), but not when coverage was lower (~91% vaccinated) . To avoid this problem in Kenya, we plan to perform the case-control study of rotavirus vaccine during the roll-out phase before full coverage is achieved.
Although case-control studies often employ multiple control groups in an attempt to understand bias, multiple control groups often yield varying effectiveness estimates, leaving researchers with challenges in interpretation of the results. We explored three different source populations for enrolling controls. Children in all three control groups were largely similar to case-patients with regard to potentially confounding socio-demographic characteristics. One exception was that the hospitalized control groups tended to have more severe malnutrition, suggesting children hospitalized with non-rotavirus illnesses tend to be different in their nutritional status, which can potentially confound VE estimates.
The ease of enrollment differed between the three control groups. Matched non-AGE facility-based controls were easy to enroll, but difficult to find sufficient numbers to meet matching criteria and harder to collect vaccination data. Community control enrollment achieved sufficient numbers, but required additional expense and logistical coordination. Children hospitalized for AGE who tested negative for rotavirus offered a time and resource-efficient alternative to the other two more traditional control groups, because they were already being tested through the diarrheal surveillance platform. These children had the same healthcare seeking behavior for diarrheal illness as individuals with rotavirus diarrhea, and had the most similar demographic profile to rotavirus AGE cases, particularly SES which can be related to health-seeking and vaccination. Additionally, study staff were blinded to the child’s case or control status at time of enrollment and during vaccine verification, which occurred before EIA results were available. The rotavirus EIA is highly sensitive and specific and thereby minimizes the misclassification of cases and controls [24-26]. Based on these reasons, we believe in this setting the use of test-negative controls will provide the most resource efficient and epidemiologically unbiased control group, and should be the only control group for the real case-control study.
Another challenge was documenting vaccination status and dates of vaccination. The proportion of children for whom we were unable to document vaccination status ranged from 10% in community controls to 29% in the non-AGE hospital controls. High levels of undocumented vaccination status could bias VE estimates particularly if vaccination status is differentially missing by number of doses received. For the real case-control study, efforts will be made to seek documentation of vaccination for all cases and controls in the home or the immunization clinics, when unavailable in the hospital.
We did not find any significant associations of pentavalent vaccination on rotavirus gastroenteritis in children. There are several possible explanations. First, a lack of association in the absence of selection bias is expected as pentavalent vaccine should confer no immunologic protection against rotavirus disease. Therefore, it is possible that there was no selection bias. Second, as mentioned, our study had very limited power due to the low numbers of both case and control children who received no pentavalent doses. Confidence intervals around VE estimates were very wide and included 1 for all evaluable control groups. Sham case-control studies, like real ones of vaccine effectiveness, are not useful in settings of extremely high vaccine coverage. Third, our study might have been biased because many children were excluded from the analysis because they did not have their vaccination cards available and/or had unknown vaccination status and this percentage varied between cases and controls. If these children systematically tended to be more or less vaccinated, we might have missed an association between pentavalent vaccine and rotavirus AGE.
A similar approach to quantifying bias in case-control studies is the so-called “bias-indicator” study in which the cases are children hospitalized with non-rotavirus AGE, a syndrome for which rotavirus vaccine should have no protective efficacy so that any association between the vaccine and case-status is likely due to selection bias [6,27]. Bias-indicator studies are a different type of sham case-control study than we performed, in that the sham is not the exposure (i.e. vaccine) but rather the outcome (i.e. type of gastroenteritis). This approach was taken in evaluations of effectiveness of oral cholera vaccine in Mozambique and rotavirus vaccine in the U.S., both indicating no measurable selection bias [27,28]. The advantage of the bias-indicator approach is that it can be done contemporaneously with an actual case-control study of rotavirus AGE cases, which cannot be done for a sham case-control study like ours due to the co-administration of most EPI vaccines.
We found almost a third of children received the second and third pentavalent doses late. Late vaccination could have implications for rotavirus vaccine, as young infants who are unvaccinated are at the greatest risk of severe dehydration from rotavirus, particularly early in introduction of vaccine before herd protection occurs . In addition, concerns persist about increased absolute risk of intussusception among children given late rotavirus vaccination, although WHO acknowledges that the risk of severe rotavirus disease outweighs any potential increased risk of intussusception among late vaccines [29-31]. Emphasis not just on vaccination, but on timely vaccination, is important in developing countries, where median ages of vaccination tend to be weeks to months behind schedule . Moreover, for the case-control study, late vaccination could bias towards increased likelihood of vaccination among controls if the age-matching window is too wide. In this study, we allowed children to be matched who were up to 60 days older than the case, which is probably too wide; we will decrease that window for age-matching in the real case-control study.
Monitoring the impact and performance of new vaccines, like rotavirus vaccine, in routine immunization programs is important for countries and can be used to advocate further support for the immunization program. Case-control studies remain one of the more accessible methods for such evaluations, despite their limitations. The findings from this sham case-control study will enable future rotavirus vaccine case-control studies in Kenya to be designed more efficiently. Sham case-control studies of VE or pilot studies of existing vaccination patterns in communities should be considered prior to the start of other case-control studies of the target vaccine to assess the existence of selection bias and improve the study design and efficiency.
AGE: Acute gastroenteritis; EIA: Enzyme immunoassay; GAVI: Global alliance for vaccines and immunization; HDSS: Health and demographic surveillance system; HIV: Human immunodeficiency virus; OR: Odds ratio; PCR: Polymerase chain reaction; VE: Vaccine effectiveness.
The authors declare that they have no competing interests.
SK, JT, UP, FO, KL and DF designed the study. SK, RO and FO collected and managed the data. SK, JT, KL and DF analyzed data and drafted the manuscript. DB and UP revised the manuscript. All authors read and approved the final manuscript.
We would like to acknowledge the HDSS and hospital community where we carry out the surveillance activities. We also acknowledge the support offered by Dr. Omoto, Dr. Onditi and other staff at the Siaya District Hospital. We acknowledge the support received from the Division of Disease Surveillance and Response (DDSR) of the Kenya Ministry of Public Health and Sanitation (MoPHS), staff in the field and data room at KEMRI/CDC and the World Health Organization Regional Office for Africa (WHO/AFRO). This manuscript is published with the approval of the Director of KEMRI. KEMRI/CDC is a member of the INDEPTH Network.
This project was supported by the U.S. Centers for Disease Control and Prevention.
The findings and conclusions of this report are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention (CDC).
Tate JE, Burton AH, Boschi-Pinto C, Steele AD, Duque J, Parashar UD: 2008 estimates of worldwide rotavirus-associated mortality in children younger than 5 years before the introduction of universal rotavirus vaccination programmes: a systematic review and meta-analysis.
[cited 2013 January 2013]; Available from: http://www.jhsph.edu/research/centers-and-institutes/ivac/vims/IVAC-VIMS-Report-2012-12.pdf webcite
Global Alliance for Vaccines and Immunization: Rotavirus vaccine support. [cited 2013 21/01/2013]; Available from: http://www.gavialliance.org/support/nvs/rotavirus/# webcite
Mwenda JM, Ntoto KM, Abebe A, Enweronu-Laryea C, Amina I, McHomvu J, Kisakye A, Mpabalwani EM, Pazvakavambwa I, Armah GE, Seheri LM, Kiulia NM, Page N, Widdowson M-A, Duncan Steele A: Burden and epidemiology of rotavirus diarrhea in selected African countries: preliminary results from the African Rotavirus Surveillance Network.
[cited 2012 October 1st]; WHO/IVB/08.16: [Available from: http://whqlibdoc.who.int/hq/2008/WHO_IVB_08.16_eng.pdf webcite
De Palma O, Cruz L, Ramos H, de Baires A, Villatoro N, Pastor D, de Oliveira LH, Kerin T, Bowen M, Gentsch J: Effectiveness of rotavirus vaccination against childhood diarrhoea in El Salvador: case-control study.
Lee EH-J, Lewis RF, Makumbi I, Kekitiinwa A, Ediamu TD, Bazibu M, Braka F, Flannery B, Zuber PL, Feikin DR: Haemophilus influenza type b conjugate vaccine is highly effective in the Ugandan routine immunization program: a case-control study.
Trop Med Int Health 2008, 13(4):1-8. PubMed Abstract
Baqui AH, El Arifeen S, Saha SK, Persson L, Zaman K, Gessner BD, Moulton LH, Black RE, Santosham M: Effectiveness of Haemophilus influenza type B conjugate vaccine on prevention of pneumonia and meningitis in Bangladeshi children: a case-control study.
Whitney CG, Pilishvili T, Farley MM, Schaffner W, Craig AS, Lynfield R, Nyquist A-C, Gershman KA, Vazquez M, Bennett NM, Reingold A, Thomas A, Glode MP, Zell ER, Jorgensen JH, Beall B, Schuchat A: Effectiveness of seven-valent pneumococcal conjugate vaccine against invasive pneumococcal disease: a matched case-control study.
Odhiambo FO, Laserson KF, Sewe M, Hamel MJ, Feikin DR, Adazu K, Ogwang S, Obor D, Amek N, Bayoh N, Ombok M, Lindblade K, Desai M, Kuile F, Phillips-Howard P, Eijk AM, Rosen D, Hightower A, Ofware P, Muttai H, Nahlen B, DeCock K, Slutsker L, Breiman RF, Vulule JM: Profile: The KEMRI/CDC health and demographic surveillance system—western Kenya.
Adazu K, Lindblade KA, Rosen DH, Odhiambo F, Ofware P, Kwach J, DK M, Van Eijk AM, Pauli A, Diana K, VJ M, Slutsker L: Health and demographic surveillance in rural western Kenya: a platform for evaluating interventions to reduce morbidity and mortality from infectious diseases.
Health Pol Plan 2006, 21(6):459-468. Publisher Full Text
Payne DC, Staat MA, Edwards KM, Szilagyi PG, Gentsch JR, Stockman LJ, Curns AT, Griffin M, Weinberg GA, Hall CB, Fairbrother G, Alexander J, Parashar UD: Active, population-based surveillance for severe rotavirus gastroenteritis in children in the United States.
Amar CF, East CL, Gray J, Iturriza-Gomara M, Maclure EA, McLauchlin J: Detection by PCR of eight groups of enteric pathogens in 4,627 faecal samples: re-examination of the English case-control Infectious Intestinal Disease Study (1993–1996).
Lucas MES, Deen JL, von Seidlein L, Wang X-Y, Ampuero J, Puri M, Ali M, Ansaruzzaman M, Amos J, Macuamule A, Cavailler P, Guerin PJ, Mahoudeau C, Kahozi-Sangwa P, Chaignat C-L, Barreto A, Songane FF, Clemens JD: Effectiveness of mass oral cholera vaccination in Beira, Mozambique.
Velazquez FR, Colindres R, Grajales C, Hernandez MT, Mercadillo MG, Torres FJ, Cervantes-Apolinar M, DeAntonio-Suarez R, Ortega-Barria E, Blum M, Breuer T, Verstraeten T: Post marketing surveillance of intussusception following mass introduction of the attenuated human rotavirus vaccine in Mexico.
Wkly Epidemiol Rec 2009, 84(51–52):533-540. PubMed Abstract
Wkly Epidemiol Rec 2012, 87:53-60. PubMed Abstract
The pre-publication history for this paper can be accessed here: