People often search for information on influenza A(H1N1)v prevention on the web. The extent to which information found on the Internet is consistent with recommendations issued by the World Health Organization is unknown.
We conducted a search for "swine flu" accessing 3 of the most popular search engines through different proxy servers located in 4 English-speaking countries (Australia, Canada, UK, USA). We explored each site resulting from the searches, up to 4 clicks starting from the search engine page, analyzing availability of World Health Organization recommendations for swine flu prevention.
Information on hand cleaning was reported on 79% of the 147 websites analyzed; staying home when sick was reported on 77.5% of the websites; disposing tissues after sneezing on 75.5% of the websites. Availability of other recommendations was lower. The probability of finding preventative recommendations consistent with World Health Organization varied by country, type of website, and search engine.
Despite media coverage on H1N1 influenza, relevant information for prevention is not easily found on the web. Strategies to improve information delivery to the general public through this channel should be improved.
In June 2009 the World Health Organization's director-general Dr. Margaret Chan, raising the pandemic alert level to phase six, officially declared the first flu pandemic of the 21st century .
According with WHO recommendations, during a pandemic the adoption of non-pharmaceutical preventive measures represents one of the most important interventions in mitigating the spread of the infection in the community [2,3]. Of such preventive measures, some are supported by scientific studies (hand washing, isolation of the sick, use of protective equipment), others are recommended by groups of experts (respiratory etiquette ).
However, in order to be effective, non-pharmaceutical preventive measures must gain community acceptance . Thus, in order to achieve a change of behavior in the general public and favor the implementation of such recommendations, health professionals, using communication strategies, have informed the population through different channels, including mass-reach broadcast media advertising (television and radio advertisements or programs), print-based materials, and audiovisual materials .
The role of the Internet as a platform for delivering public health interventions to specific patient groups and to the general public is constantly increasing, due in particular to its disseminating potential : the worldwide penetration of the Internet is increasing and the use of this medium for seeking health information is frequent . Moreover, the Internet potential for individual tailoring and interactivity is superior to that of other high reach-delivery channels .
Integration of the Internet with the more classical media as well as with the novel, alternative formats (email, interactive digital TV, SMS texts, podcasts) for social marketing can allow to reach specific groups whose characteristics are different from those of the general population .
In the evolving context of the swine flu pandemic the World Health Organization (WHO) issued specific recommendations for individuals, addressing a set of behavior changes that can help prevent the spread of the influenza virus. These recommendations are accessible on the WHO website .
Recommendations for H1N1 flu prevention may not be available or easily accessible on the Internet. Chances to get appropriate information about swine flu might also vary with the different geographic areas from which the information is sought.
The objective of our research is to analyze the probability that an average Internet user has to get appropriate information about H1N1 flu prevention in four different English-speaking countries when searching advice through generic web search engines.
We simulated a search for generic information on pandemic flu in four English speaking countries using generic web search engines, and analyzed the contents of the first page hits for presence of recommendations as reported by WHO . Selected countries included Australia, Canada, UK, and USA.
Search string and search strategy
We identified the most used search strings for seeking generic information on influenza A(H1N1)v on the web through "Google Trends", an online application that shows how often a particular search-term is entered relative to the total search-volume across different regions of the world . We compared three different search strings ("influenza", "swine flu", "H1N1 flu"), and verified that swine flu had been the most frequently used search string in the four countries considered in our study during the 30 days prior to the investigation.
On August 13th 2009 we performed multiple searches for "swine flu" using the three most popular search engines (Google, Yahoo, MSN-Bing). The search engines were accessed through four different proxy servers, located in each of the 4 above mentioned countries. Through this strategy, we obtained results that reflected exactly what would be obtained launching the search from the four different countries.
In order to review the web pages as they were found on the date of the search we downloaded all resulting web pages with an off line browser and stored them in CD-ROM supports which were used for the review process.
We analyzed all the Internet pages shown in the first page of the search results.
For each webpage resulting from the searches, we recorded the country where the search was launched from, and the search engine we obtained it from. In the revision process two of the authors reviewed all the web pages content.
We recorded the type of web site (Public Health Agency, university/hospital/research, news, drug company/commercial, other types of website including Wikipedia and blogs), and explored each Internet page and all its links up to 4 clicks starting from the search engine webpage. In order to compare website contents we selected all the recommendations for swine flu prevention listed in the WHO recommendation webpage , and created a checklist of 21 items, each item being relevant to one of the recommendations (Table 1). Through this checklist we verified whether each specific prevention item was available on the reviewed Internet pages. We defined an item as available when the information was reached and consistent with the WHO recommendation.
Table 1. Frequency and percentage of websites reporting recommendations consistent with WHO (total number of websites: 147 )
We calculated the frequency of available items included in the WHO recommendation list by type of web site, country, and search engine, within 4 clicks. Since different search engines may report overlapping results, we calculated proportions excluding multiple overlapping results in the analysis by country. To compare the obtained figures we used the Chi square test. We then performed a logistic regression analysis with the dependent variable being the availability of specific recommendations and the independent variables being country, search engine and type of web site. Associations at the multivariate analysis were expressed with adjusted odds ratios and 95% confidence intervals. All statistical analyses were considered significant at the 0.05 level. We used Stata version 10.0 for the analysis.
We found a total of 147 websites, resulting from the search in the four English-speaking countries with the three web search engines: Australia (30 sites), Canada (40 sites), US (40 sites), UK (37 sites). For a complete list of the URLs resulting from the searches see Additional file 1: URL list.
Most of the websites were from Public Health Agencies and from news providers: these two categories accounted for a proportion of 37% and 40% respectively. In fewer cases the websites were from Universities, Hospitals, Research Institutes, or blogs. Few websites were commercial or from drug industries.
The first hit reported in each search result page was always a website from a Public Health Agency, with the exception of MSN searches from Canada (first hit: news provider) and US (first hit: Wikipedia)
Overall, the information availability differed between each recommendation. We found that recommendations about hand cleaning, staying home when sick, use of tissue when coughing or sneezing and tissue disposal were the most frequently shown. One step of the respiratory etiquette, covering mouth and nose with the sleeve or with the elbow crook when coughing or sneezing, was among the less represented information on the web. Other information rarely reported included airflow improvement, keeping good health habits, informing family and friends when sick, contacting health professionals before travelling to health facilities (Table 1).
In the univariate analysis on availability of recommendations by country, we found that information about resting and drinking fluids when sick, breastfeeding despite illness, no restriction to travel to and from any country during pandemic, and contacting health professionals before travelling to health facilities were more represented in searches conducted from the United Kingdom than in searches conducted from the other countries. Presence of information varied significantly among the four countries also regarding communication to family and friends when sick, and one of the mask-wearing indications.
We performed then a logistic regression model to explore the independent effect of country adjusting for type of website and search engine. The reference country used for this analysis was UK. (Table 2)
Table 2. Adjusted ORs and 95% CI of finding single recommendations during web search by country (multivariable analysis, reference country: UK)
Information about resting and drinking fluids when sick, breastfeeding despite illness, and absence of travel restrictions were significantly less frequent in Australia, Canada, and USA compared with UK (Table 2). Information on hand cleaning, not wearing a mask if not sick, staying home when sick, disposing tissues after use, and cleaning hands after coughing or sneezing were less frequent in Australia compared with UK. Information on sneezing in sleeve or elbow crook was more frequently available in Canada, whereas the recommendation on contacting health professionals before travelling to health facilities was less frequently found in Canada compared with UK (Table 2). Finally, information on wearing a mask when sick if in contact with other people was more frequently found in US compared with UK (Table 2).
The effect of type of website, according to the univariate analysis, showed that pages from Public Health Agencies hosted significantly more information on all items but avoiding touching mouth and nose, improving airflow in living place, practicing good health habits, wearing a mask if sick and in close contact with other people, coughing or sneezing in sleeve or elbow crook, and safety of pork meat.
When adjusting for country and search engine, and taking as reference websites from Public Health Agencies, we found that pages from Universities or Hospitals had a significantly lower probability of reporting recommendations on hand cleaning and staying home when sick (Table 3). Websites from news providers reported less information compared with websites from Health Agencies on all items but improving airflow in the living place, practicing good health habits, wearing a mask when sick and in close contact with other people, informing family and friends about illness, and contacting health professionals before travelling to health facilities. Web pages from drug manufacturers and other commercial agencies less frequently reported information on hand cleaning, reducing time spent in crowds, not needing a mask if not sick, staying home when sick, avoiding contacts with other people, using tissues, disposing tissues and cleaning hands after coughing or sneezing. On the other hand information on improving airflow in the living place was more frequently reported in these web pages compared with those from Public Health Agencies. Pages from other categories, compared with pages from Health Agencies, also less frequently reported information on hand cleaning, staying home when sick, coughing or sneezing in sleeve or elbow crook, not needing to stop breastfeeding, and safety of pork meat; on the other hand information on improving airflow in the living place was more frequently reported in websites from these categories than in pages from Public Health Agencies.
Table 3. Adjusted ORs and 95% CI of finding single recommendations during web search by type of website (multivariable analysis, reference type: Health Agencies)
The univariate analysis by search engine showed that websites resulting from Google and Yahoo searches, compared with those resulting from MSN Bing, more frequently reported information on avoiding touching mouth and nose, practicing good health habits, staying home when sick, avoiding contact with other people when sick, respiratory etiquette, informing family and friends when sick, safety of pork meat, and absence of travelling restrictions.
Adjusting for country and type of website, and taking Google as reference, the multivariate analysis (Table 4) showed that while there was no significant difference in the probability of finding the WHO recommendations between Yahoo and Google, websites obtained with MSN Bing less frequently reported information on almost all items, except for hand cleaning, avoiding contact with people and crowds, information on mask wearing, resting and taking fluids, and breastfeeding.
Table 4. Adjusted ORs and 95% CI of finding single recommendations during the web search by search engine (multivariable analysis, reference search engine: Google)
Our study shows an overall high probability of finding on the web appropriate information regarding hand washing and main steps of the respiratory etiquette consistent with WHO recommendations, whereas other recommendations regarding behaviors that can prevent human-to-human transmission of influenza A(H1N1)v are rarely found during a generic web search on pandemic flu. Moreover, the probability of finding appropriate information for H1N1 influenza prevention is affected by country, type of website, and web search engine.
General information on respiratory etiquette consistent with WHO recommendations is rarely complete and reported, especially when considering information on covering mouth and nose with sleeve or elbow crook when coughing or sneezing.
Recommendations about practicing good health habits, improving airflow in living place, informing family and friends when sick, as well as those about breastfeeding (continue breastfeeding despite illness) and contacting health professionals before travelling to health facilities are also rarely shown, and would deserve better implementation.
In the last years access to the Internet and use of the World Wide Web have been constantly increasing. Users often seek health information on the web  and the Internet has played a central role as an information source on H1N1 influenza, as documented by the volume of searches launched on Google on this subject .
Therefore, giving on the web appropriate advices on prevention to users typically surfing the web for information on pandemic flu should be a privileged strategy for information dissemination. Thus, it is crucial to measure the probability that an average Internet user has to access appropriate information.
In our study we found a higher availability of information on H1N1 influenza for most recommendations in searches performed in the UK compared with the other countries.
Cultural models, local habits, different web dissemination strategies, and different moments in the evolution of the pandemic may explain some of these differences. Moreover, certain specific recommendations were more frequent in some countries, as documented by the higher probability of finding information on sneezing in the sleeve or elbow crook in Canada, and wearing a mask when sick in the US compared with the UK. On the other hand, a lower coverage of WHO recommendations in Australia might be associated with the observation that in this country the pandemic peak was already over when this study was conducted.
We observed that websites obtained through MSN-Bing show less information about pandemic flu prevention compared to websites obtained through Google or Yahoo.
Finally, we found that the majority of the websites were from news providers, followed by pages from Public Health Agencies. However, news websites provide significantly less information regarding preventive measures compared to Public Health Agencies. The same happens for commercial websites and drug companies websites.
The major Internet search engines use complex algorithms to rank search results. Websites from major organizations tend to appear first when a search is carried out, and the risk of exploring sites that provide inaccurate advice should be small . Our search shows that usually the first search hit, in the large majority of cases, is a website from a Public Health Agency. Nevertheless, news providers, often found in the first page of search results, in most cases do not provide adequate advice.
Therefore, taking into account the higher rank that news providers get on the major search engines in a generic search on swine flu, this kind of websites should improve reporting of complete information on influenza prevention, or provide links to websites showing comprehensive sets of adequate recommendations. This could be an effective strategy for a broader dissemination of appropriate influenza prevention behaviors.
The main limitation of our study is that the analysis was performed only once, while information found on the web is constantly changing. Since the pandemic virus started circulating, knowledge on its epidemiological characteristics has improved, and interest towards this subject may have changed. We cannot say if the availability of recommendations on prevention has changed over time. We did not consider the advertised links that appear on the side and on the top of Google search result page, which lead to commercial sites. Another limitation is that our analysis was performed only on English websites from English-speaking countries, and may therefore not be generalized to other countries or to websites in other languages. Although further studies are needed to address this issue, we think that a study of this kind can be useful to assess web availability of recommendations on prevention in specific contexts, and can therefore help improving strategies of public health information delivery to the general public.
The Internet is a rich and efficient source of information about infectious diseases and their prevention. Its role in social marketing can be crucial. Integrating its potentials with those of other media can guarantee a broad spread of information. Moreover, public health interventions transmitted through this medium could target groups with peculiar characteristics .
In conclusion, in order to guarantee an adequate use of this instrument by the public, health professionals, who are still one of the main sources of information regarding health matters, should guide patients to other reliable sources (information prescription [9,10]) at the same time, deeper attention should be paid by Public Health Agencies towards quality of information and search engine optimization; studies like the one presented in this article should be used more often to improve delivery of public health information to the general public.
The authors declare that they have no competing interests.
FG coordinated the study, participated in the webpages review process and in the writing process. MR participated in the design and in the maintenance of the database. EP participated in the webpages review process. CR helped to draft the manuscript. LR performed the statistical analysis. DL helped in the webpages review process. AET conceived the study, participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.
World now at the start of 2009 influenza pandemic [http:/ / www.who.int/ mediacentre/ news/ statements/ 2009/ h1n1_pandemic_phase6_20090611/ en/ index.html] webcite
Aiello AE, Coulborn RM, Aragon TJ, Baker MG, Burrus BB, Cowling BJ, Duncan A, Enanoria W, Fabian MP, Ferng Y, Larson E, Leung GM, Markel H, Milton DK: Research findings from nonpharmaceutical intervention studies for pandemic influenza and current gaps in the research.
Pandemic H1N1 2009 Guidance for individuals [http://www.who.int/csr/disease/swineflu/guidance/individuals/en/index.html] webcite
JRSM 2008, 101(9):447. Publisher Full Text
The pre-publication history for this paper can be accessed here: