Skip to main content

Immediate effects of alcohol marketing communications and media portrayals on consumption and cognition: a systematic review and meta-analysis of experimental studies

Abstract

Background

Restricting marketing of alcoholic products is purported to be a cost-effective intervention to reduce alcohol consumption. The strength of evidence supporting this claim is contested. This systematic review aimed to assess immediate effects of exposure to alcohol marketing on alcoholic beverage consumption and related cognitions.

Methods

Electronic searches of nine databases, supplemented with reference list searches and forward citation tracking, were used to identify randomised, experimental studies assessing immediate effects of exposure to alcohol marketing communications on objective alcohol consumption (primary outcome), explicit or implicit alcohol-related cognitions, or selection without purchasing (secondary outcomes). Study limitations were assessed using the Cochrane Risk of Bias tool. Random and fixed effects meta-analyses were conducted to estimate effect sizes.

Results

Twenty four studies met the eligibility criteria. A meta-analysis integrating seven studies (758 participants, all students) found that viewing alcohol advertisements increased immediate alcohol consumption relative to viewing non-alcohol advertisements (SMD = 0.20, 95 % CI = 0.05, 0.34). A meta-analysis integrating six studies (631 participants, all students) did not find that viewing alcohol portrayals in television programmes or films increased consumption (SMD = 0.16, 95 % CI = −0.05, 0.37). Meta-analyses of secondary outcome data found that exposure to alcohol portrayals increased explicit alcohol-related cognitions, but did not find that exposure to alcohol advertisements influenced explicit or implicit alcohol-related cognitions. Confidence in results is diminished by underpowered analyses and unclear risk of bias.

Conclusions

Viewing alcohol advertisements (but not alcohol portrayals) may increase immediate alcohol consumption by small amounts, equivalent to between 0.39 and 2.67 alcohol units for males and between 0.25 and 1.69 units for females. The generalizability of this finding beyond students and to other marketing channels remains to be established.

Peer Review reports

Background

Alcohol marketing is a prominent feature of an ‘alcogenic’ environment - an environment that reflects and promotes a culture of alcohol use [1]. Alcohol marketing communications have been identified as a potential target for public health intervention due to their proposed influence on harmful patterns of alcohol consumption [2, 3]. The alcohol industry’s position is that marketing raises awareness of certain brands or products, but does not cause overall increased consumption [4, 5].

Findings from three published systematic reviews are discordant with the industry’s position [68]. These reviews investigated relationships between exposure to various forms of alcohol marketing and alcohol consumption among young people. Their findings were based on syntheses of overlapping but not identical sets of primary studies. Two of the reviews synthesised evidence from longitudinal cohort studies only [6, 7], whilst the third also incorporated evidence from cross-sectional studies [8]. All three concluded that exposure to alcohol marketing has a dose-dependent association with initiation of alcohol use and increased alcohol consumption.

Due to their focus on people below the legal drinking age these reviews did not include experimental studies, in which participants are randomised to be exposed either to alcohol marketing or a control stimulus, with alcohol consumption objectively measured post-exposure. Whilst the authors of one review [7] note that such studies lack ecological validity (i.e. their settings and procedures may not reflect the complex nature of real-world advertising exposure), experimental studies have several advantages. They allow for a high degree of control over marketing exposure and, with successful randomisation, minimise the potential for confounding of effects by unmeasured variables – a limitation of some longitudinal studies as acknowledged by previous review authors [68]. Further, objective measurement of alcohol consumption has benefits over self-report measures used in longitudinal studies, which are prone to influence by participant and contextual characteristics [9]. In synthesising observational or experimental research, there is therefore a trade-off between greater ecological validity and greater internal validity with reduced risk of bias. We are unaware of any systematic attempt to date to synthesise the results of experimental studies on this topic. We therefore conducted the systematic review reported here, to assess evidence from randomised, experimental studies for the immediate effects of exposure to alcohol marketing communications on alcohol consumption, and on alcohol-related cognitions.

Synthesising experimental evidence for the immediate effects of alcohol marketing on consumption to address the limitations of observational longitudinal evidence presupposes that immediate and distal effects of alcohol marketing are related. Alcohol marketing has been hypothesised to promote consumption by normalizing alcohol use and highlighting desirable consequences of consumption [10]. Exposure to marketing is thought to stimulate a motivation to consume alcohol via two sets of processes, conscious and non-conscious. The conscious (explicit) processes include making attitudes to alcohol more favourable and increasing positive expectancies of its use [1114]. Non-conscious (implicit) processes include priming, imitation, and associative learning, through which general approach orientations towards alcohol are activated [15]. If these causal theories are correct, any immediate and distal effects of exposure to alcohol marketing on consumption are realised through related, though possibly distinct, psychological processes, with positive alcohol-related cognitions being activated immediately in response to a single exposure, as well as developing over time in response to repeated exposures.

Nevertheless, it is important to note that there are myriad other effects of exposure to alcohol marketing not captured by this theoretical framework. Evidence from the marketing literature shows that whilst the objective of any marketing campaign is to increase sales, individual marketing communications have a low likelihood of stimulating immediate urges to purchase and consume the marketed product [16]. Communications are therefore designed to meet additional intermediate objectives such as raising awareness, interest, and identification with products and brands, associating products with certain emotions and experiences, and increasing the number of contexts in which use of the products is seen as appropriate [17, 18]. These outcomes are subtle in nature and develop gradually, meaning they are unlikely to be observed in studies investigating only the immediate effects of marketing exposure.

Marketing campaigns are also typically targeted at distinct demographic subgroups. Early research into alcohol marketing effectiveness conducted by Anheuser Busch, for example, aimed to increase the potency of advertising by linking products with the personality types of consumers [19, 20]. Relatedly, there are individual-level differences in receptivity to specific marketing communications that affect the degree to which a marketing message will be influential. A framework for understanding these differences, the Message Interpretation Process model, highlights that the effectiveness of any marketing message is dependent on audience characteristics such as identification with the characters and messages being presented, and level of scepticism towards the message [13, 21]. Experimental studies of the immediate effects of alcohol marketing do not typically account for factors such as product targeting, identification with brands, and differences in receptivity. Any observed effects on consumption and cognition may therefore be diluted.

With these caveats in mind, the present systematic review and previous reviews can be seen as examining complementary bodies of evidence that have different strengths and limitations, and that each add to understanding of the complex relationship between alcohol marketing and consumption. Consistency between their respective findings would bolster the claim that alcohol marketing has a causal impact on alcohol consumption. However, we acknowledge that even the combination of these two bodies of evidence is still unable to explain the full breadth of the possible public health consequences of alcohol marketing.

To ascertain whether certain population subgroups are more susceptible to the effects of alcohol marketing, it is important to explore whether any immediate effects vary by demographic, behavioural, or cognitive characteristics. For example, alcohol marketing has been found to promote gender stereotypes [22], and as such there may be gender differences in responses to marketing. Also, heavier drinkers have shown attentional bias towards alcohol-related cues [23] and may be more likely to attend to and be influenced by alcohol marketing. Similarly, individuals with weaker executive function or who are more impulsive may be more susceptible to marketing communications [24].

The present study

The primary objective of this systematic review was to estimate the direction and size of immediate effects of exposure to alcohol marketing communications on alcohol consumption and alcohol-related cognitions, using data collected from individually randomised, experimental studies. The secondary objective was to investigate whether the following factors explain any observed heterogeneity in effects between studies: type of alcoholic drink being marketed or consumed, participant demographic characteristics (gender, age, and level of education), baseline alcohol consumption, and executive function.

Method

This systematic review was conducted according to Cochrane methodological standards for systematic reviews of health interventions [25] and is reported in line with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines [26]. The review protocol was prospectively registered in the PROSPERO database (ID: CRD42013005057 [27]).

Search strategy

Electronic searches of seven research literature databases (MEDLINE, EMBASE, PsycInfo, ASSIA, Science Citation Index Expanded, Social Sciences Citation Index, and Econlit) and two grey literature databases (CPCI – Science, and CPCI - Social Science & Humanities) were conducted up to 21 September 2015. Details of electronic searches are provided in Additional file 1. Searches of eligible studies’ reference lists and forward citation tracking (using Google Scholar and PubMed) were also conducted. No restrictions were imposed for publication date, format, or language.

Inclusion criteria

Individually randomised, laboratory-based experiments with between- or within-participants designs were included. Quasi-experimental and non-experimental studies were excluded. There were no restrictions on types of participants. The eligible intervention was exposure to alcohol marketing communications, defined as any form of advertising, communication or media exposure designed to encourage favourable cognitions towards alcoholic beverages, their purchasing, or consumption. Interventions involving communications negatively valenced towards alcohol (e.g. public health messages) were excluded. The eligible comparator was no exposure to alcohol marketing, or exposure to any other form of alcohol marketing. The primary outcome was objectively measured alcoholic beverage consumption (total amount consumed). Eligible secondary outcomes (proposed mediators of the immediate effect alcohol marketing communications on consumption) were: explicit and implicit alcohol-related cognitions, alcoholic beverage purchasing; and alcoholic beverage selection without purchasing.

Study selection and data extraction

Full details of procedures for selecting eligible studies and extracting data are presented in the systematic review protocol [27]. Retrieved title-abstract records were independently screened by KB, SK, and KS, with IS acting as arbiter in case of disagreements. Full-text screening of potentially eligible study reports was undertaken using the same procedure. Data on the characteristics of included studies were extracted by one reviewer (KS, KB, or SK). Outcome data were extracted in duplicate by two reviewers (two of KS, KB and SK) working independently, with discrepancies resolved by discussion and IS acting as arbiter when needed.

Risk of bias assessment

The Cochrane Risk of Bias tool [28] was used to assess potential bias in studies that included the primary outcome, alcohol consumption. Assessments were conducted in duplicate by two reviewers (two of KS, KB and SK) working independently. Unpublished information needed to inform assessments was sought by contacting study authors. Nine domains were considered: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessors, incomplete outcome data, selective outcome reporting, baseline comparability between groups, consistency in intervention delivery, and whether objective outcome measures had been assessed for validity and reliability. Risk of bias judgements in three key domains (sequence generation, blinding of participants, and baseline comparability between groups), judged by the authors conducting assessments to be most likely to impact on confidence in study-level estimates of alcohol consumption, were used to determine a summary study-level risk of bias. (Non-) blinding of outcome assessors was judged unlikely to impact on confidence in estimates of this effect, due to the inclusion criterion that consumption was objectively measured. Assessments of incomplete outcome data and selective outcome reporting were also judged unlikely to impact on confidence in estimates; in the former case due to expected low rates of attrition among studies using this specific experimental paradigm (with immediate post-exposure outcome measurement), and in the latter case due to the anticipated lack of study pre-registration for use to reliably assess this dimension.

Reporting biases were assessed by visual inspection of funnel plots. Formal statistical tests to investigate the degree of funnel plot asymmetry [29] were not conducted as planned due to the small number of studies incorporated into each meta-analysis.

Data synthesis

As the scales used to measure the primary outcome varied between studies, we computed study-level standardised mean differences (SMD) between comparison groups with 95 % confidence intervals. Where outcome data could not be obtained from either a study report or its authors, replacement standard deviations were imputed using established methods [30].

Meta-analyses [31] were conducted using Review Manager 5.3 [32]. For each meta-analysis, we estimated both random effects and fixed effects models and compared results. Statistical heterogeneity was assessed using the Chi 2 test and the I 2 statistic, and by examining the random effects between study variance (Tau 2). Sensitivity analyses were performed to assess the robustness of results to the exclusion of studies with high risk of bias and studies for which we had imputed standard deviations due to missing data.

Subgroup analysis, planned a priori, was used to explore gender differences in effects. Other planned subgroup analyses (type of alcoholic drink being marketed or consumed, participant age, education, baseline alcohol consumption, and executive function) could not be conducted due to a lack of measurement and/or reporting of outcome data by subgroups in primary studies. A planned subgroup analysis by media channel could not be conducted as only two media channels, alcohol advertisements and alcohol portrayals in television programmes or films, were investigated among included studies. Effects of exposure to alcohol advertisements and alcohol portrayals were analysed separately.

Results

Results of the search

Figure 1 illustrates the flow of studies through the systematic review process. Searches yielded a total of 21,192 study records, of which 5,662 were discarded as duplicates. Three additional records were identified through searches of reference lists. One additional record was a recently published paper by members of the research team. All studies screened were reported in English. Title-abstract screening of the 15,534 unique records identified 42 studies judged potentially eligible. Full-text screening resulted in the exclusion of a further 17 studies and the remaining 24 studies (reported in 25 articles, one of which reported a subgroup analysis of another included study [33]) were accepted into the review. Principal reasons for study exclusion are presented in Additional file 2.

Fig. 1
figure 1

PRISMA flow diagram

Characteristics of included studies

Tables 1 and 2 show the key characteristics of included studies. Detailed information is presented in Additional file 3. Twenty three of the 24 included studies had been published in peer-reviewed journals and the other [34] was a published dissertation. Eleven studies assessed the primary outcome of alcohol consumption [3444], and 14 assessed secondary outcomes [40, 4557], with one study assessing both [40]. Eighteen included studies were conducted in laboratory settings. Of the other six included studies, all assessing secondary outcomes only, four were conducted in a school classroom setting [49, 51, 55, 56], and two online [45, 46]. All but one study had a between-participants design, with the exception using a within-participants design [55]. Participants of included studies were exposed to alcohol advertisements [3538, 40, 4345, 47, 49, 50, 53, 5557], alcohol portrayals in television programmes [34, 42, 43, 48, 51, 54] or films [35, 39, 41, 46, 52], or a combination of both advertisements and portrayals [35, 43]. The advertisements presented were mostly television advertisements, with two exceptions being a study that used magazine advertisements presented as slides [53], and a study that used alcohol advertisements presented within screenshots of the social media website Facebook [45]. The portrayals presented tended to show characters consuming alcohol in positive contexts (e.g. celebration) and with positive consequences (e.g. companionship), often in comedy films or programmes. It was not possible to ascertain whether the alcohol portrayals presented were genuine examples of alcohol marketing (e.g. product placement paid for by an alcohol company) or incidental.

Table 1 Key characteristics of included studies assessing primary outcome - alcohol consumption
Table 2 Key characteristics of included studies assessing secondary outcomes – explicit and implicit alcohol-related cognitions, and alcohol selection without purchasing

Risk of bias

Risk of bias assessments are summarised in Fig. 2. Detailed risk of bias tables are provided in Additional file 4. Study-level risk of bias was ‘unclear’ in nine of eleven studies that measured the primary outcome of alcohol consumption due to partial or unclear reporting of methods and procedures. One study [34] was judged to be at overall ‘low’ risk of bias (i.e. low risk of bias in all key domains) and the other [43] was judged to be at overall ‘high’ risk of bias due to concern about procedures applied to generate the allocation sequence.

Fig. 2
figure 2

Risk of bias summary table. Review authors’ judgements about each risk of bias item for each included study assessing the primary outcome of alcohol consumption. Key: + indicates low risk; ? indicates unclear risk; − indicates high risk

At the domain-level, most studies reported insufficient information regarding randomisation procedures. Procedures for blinding participants were reported in six studies. Six of these studies [34, 37, 38, 40, 41, 43] reported successful blinding, indicated by responses to post-experiment questions regarding awareness of the study purpose, and the other [36] reported a blinding procedure but not whether this succeeded. Regarding blinding of outcome assessors, four of six studies with sufficient information were judged at ‘high’ risk of bias in this domain on the basis that measurement was conducted by assessors who were likely to have had knowledge of both the study hypotheses and participant assignments to conditions [3941, 43, 44]. Eight studies [34, 35, 3843] compared baseline differences between groups on demographic characteristics and typical alcoholic beverage consumption, with all but one finding no differences. The exception [35] found baseline differences in typical alcohol consumption and adjusted for these accordingly. Six studies reported information concerning the validity of the outcome measure [34, 3841, 44]. All found that self-reported typical alcohol consumption was strongly associated with scores on the outcome measure, indicating that these scores were valid proxies of typical alcohol consumption.

Effects on alcohol consumption

All eleven studies assessing the primary outcome of alcohol consumption were conducted using students, with ages ranging between 16 and 45 years. All studies received approval from institutional review boards, and all participants were above the legal drinking age in the country the study was conducted (16 being the legal drinking age in the Netherlands). Quantities of alcohol consumed were measured as millilitres of alcoholic beverage consumed, number of alcoholic drinks consumed (e.g. bottles of beer or glasses of wine), or both. When two measures of alcohol consumption were reported (e.g. number of bottles consumed and amount consumed in millilitres), we retained data for the more granular unit of measurement (e.g. amount consumed in millilitres). Two studies [34, 42] assessed consumption of non-alcoholic beer that participants had been led to believe contained alcohol.

Alcohol advertisements

Seven studies, with 758 participants, investigated the effect of exposure to alcohol advertisements on alcoholic beverage consumption [3538, 40, 43, 44]. A random effects meta-analysis of data from these studies found a summary effect size (SMD) of 0.20 (95 % C.I. = 0.05–0.34), indicating that participants who viewed alcohol advertisements consumed more alcohol than those who did not, the size of this effect being small (Fig. 3).

Fig. 3
figure 3

The effect of alcohol advertisements vs. control group interventions on alcohol consumption

For illustrative purposes, a summary effect size of 0.20 is equivalent to an increase of 1.57 (95 % C.I. = 0.39–2.67) alcohol units consumed by the average male drinker (around half a pint of lager at ABV 5.2 %), or an increase of 0.99 (95 % C.I. = 0.25–1.69) units consumed by the average female drinker (around half a 175 ml glass of wine at ABV 12 %), on the heaviest drinking day of the week.Footnote 1

No statistical heterogeneity was observed (see Fig. 3), indicating that the direction and size of this effect was consistent between studies. Results were insensitive to the exclusion of data from one study [44] judged at high risk of bias (SMD = 0.17, 95 % C.I. = 0.01–0.32), and to the exclusion of two studies [36, 44] for which we had imputed standard deviations (SMD = 0.18, 95 % C. I. = 0.01–0.34). Results of random effects and fixed effects meta-analyses were similar. The number of participants included in this meta-analysis was lower than the threshold optimal information size (OIS) (i.e. the total number of participants generated by a conventional sample size calculation for a single adequately powered randomised controlled trial) when SMD = 0.20 (OIS = 788 at SMD = 0.20, α = 0.05, Power(1-β) = 0.8). Although visual inspection of the corresponding funnel plot did not indicate asymmetry, presence of publication bias cannot be ruled out.

Differences between male and female participants in the effect of exposure to alcohol advertising on consumption were investigated in a planned subgroup analysis. No differences between males and females were found (Fig. 4; p = 0.89), but this analysis was underpowered. Additionally, two studies reported on whether past week alcohol consumption moderated observed within-study effects. One of these studies [38] found that heavier past-week drinkers (>7 alcoholic beverages in past week) consumed more alcohol than lighter drinkers (≤7 drinks) following exposure to alcohol advertisements, whilst another [40] reported no differences in consumption between heavier and lighter drinkers.

Fig. 4
figure 4

Sub-group analysis by gender: alcohol advertisements vs. control group interventions on alcohol consumption

Alcohol portrayals in television programmes or films

Six studies, with 605 participants, investigated the effect of exposure to alcohol portrayals in television programmes or films on immediate alcoholic beverage consumption [34, 35, 39, 4143]. A random effects meta-analysis of these studies (Fig. 5) did not find any difference in quantities of alcoholic beverages consumed between exposed and unexposed participants (SMD = 0.16, 95 % C. I. = −0.05–0.37). The result was insensitive to the exclusion of outcome data from one study [42] that incorporated imputed standard deviations. Results of random and fixed effects meta-analyses were similar. The total number of participants was lower than the threshold optimal information size of 1,230. Visual inspection of the corresponding funnel plot did not indicate asymmetry, but again publication bias cannot be ruled out. A subgroup analysis found no difference in amounts consumed between males and females (Fig. 6; p = 0.47), but this analysis was again underpowered. One study reported whether self-control, a marker of executive function, moderated observed effects [41]. This study found no differences in consumption between high and low self-control participants in the alcohol portrayal condition.

Fig. 5
figure 5

The effect of alcohol portrayals in television programmes or films vs. control group interventions on alcohol consumption

Fig. 6
figure 6

Subgroup analysis by gender: alcohol portrayals in television programmes or films

Effects on alcohol-related cognitions and alcohol selection

Fourteen included studies assessed the effects of exposure to alcohol marketing on at least one of the secondary outcomes of interest (Table 2). Explicit alcohol-related cognitions (explicit attitudes, outcome expectancies, or intentions to consume alcohol) were the most frequently measured construct (12 studies), whilst few included studies measured implicit alcohol-related cognitions (3 studies) or alcohol selection without purchase (4 studies). Three of these studies assessed multiple secondary outcomes [47, 50, 56]. Studies sampled college students (6), adolescent high school students (3), children between the ages of 8 and 12 (3), or adults from the general population (2). No included studies incorporated measures of alcohol purchasing.

Alcohol advertisements

Eight studies examined the impact of exposure to alcohol advertising on explicit alcohol-related cognitions [40, 45, 47, 49, 50, 5557]. One study [49] reported finding an effect, but did not provide data in a form useable for meta-analysis. A random effects meta-analysis of data from the remaining seven studies (1,368 participants; Fig. 7) did not find a difference in positive explicit cognitions between exposed and unexposed participants (SMD = 0.09, 95 % C. I. = −0.04–0.22). The total sample size in this meta-analysis did not exceed the threshold optimal information size of 3,878.

Fig. 7
figure 7

The effect of alcohol advertisements vs. control group interventions on explicit alcohol-related cognitions

Three studies assessed whether exposure to advertisements influenced implicit attitudes towards alcohol [47, 50, 56]. These studies each used different tasks to assess this construct. A random effects meta-analysis of data from these studies (451 participants, Fig. 8) did not find a difference between exposed and unexposed participants (SMD = 0.15, 95 % C. I. = −0.04–0.33). The total sample size in this meta-analysis did not exceed the threshold optimal information size of 1,398.

Fig. 8
figure 8

The effect of alcohol advertisements vs. control group interventions on implicit alcohol-related cognitions

Three studies assessed whether exposure to advertisements influenced alcohol selection without purchasing [47, 53, 56]. One study [56] reported finding no effect on hypothetical selection of an alcoholic versus a non-alcoholic beverage amongst adolescents. One study [47] reported finding no effect on selection of a voucher for an alcohol-related versus non-alcohol related outlet among adults. One [53] tested whether viewing print-based alcohol advertisements influenced the amount of alcohol poured into mixed drinks, and also reported finding no effect. Data from these studies were not combined due to heterogeneity in the outcome measures used, and due to one study not providing useable data [56].

Alcohol portrayals in television programmes or films

Four studies reported results concerning the impact of exposure to alcohol portrayals in television programmes or films on explicit alcohol-related cognitions [46, 48, 51, 52]. One study [51] reported not finding an effect, but did not provide useable data. A random effects meta-analysis of data from the remaining three studies (281 participants; Fig. 9) found that exposed participants reported more positively-valenced explicit cognitions than unexposed participants (SMD = 0.40, 95 % C. I. = 0.07–0.73). At the point estimate of this summary effect size (but not at the lower bound estimate of the 95 % confidence interval), the total sample size exceeded the threshold optimal information size of 200.

Fig. 9
figure 9

The effect of alcohol portrayals in television programmes or films vs. control group interventions on explicit alcohol-related cognitions

One study examined the effect of portrayals on alcohol selection without purchasing [54]. In this study, children (8–11 years old) were given a hypothetical choice of serving either whisky or water to photographs of adults. Participants who viewed a programme showing alcohol consumption were more likely to serve whisky than those who viewed the programme with no alcohol consumption and participants who viewed no television at all.

Discussion

Principal findings

The results of this systematic review suggest that exposure to alcohol advertisements may increase immediate consumption of alcoholic beverages by small amounts, equivalent to between 0.39 and 2.67 alcohol units for males and between 0.25 and 1.69 units for females. We did not find evidence that exposure to alcohol portrayals in television programmes or films had an effect on immediate alcoholic beverage consumption. No eligible studies investigating immediate effects of exposure to other forms of alcohol marketing on consumption were identified.

Confidence in summary estimates of these effects was diminished by three factors. First, our meta-analyses were typically underpowered. Second, the lower bound confidence interval for our estimate of the effect of advertisements on consumption was close to the line of no effect. Third, risk of bias remained unclear in the majority of source studies due to incomplete or ambiguous reporting of study design features, methods and procedures. Taken together, these factors leave open the possibility that the integration of results from further research could substantively change summary estimates of these effect sizes, and hence the principal findings of this review.

Our findings are, however, broadly consistent with those of previous systematic reviews on this topic, which report evidence for a positive association between exposure to alcohol marketing, of any type and over time, and quantities of alcoholic beverages consumed [68]. Our findings support the less emphatic claim that exposure to alcohol advertising, but not alcohol portrayals in television programmes or films, may increase quantities consumed immediately following exposure, by small amounts. This difference in findings might reflect the possibility that a series of discrete exposures, each of which individually causes a relatively small or no immediate increase in consumption, cumulate into higher levels of consumption over time among more exposed people. Alternatively, our findings may be specific to student populations, the only participants in the studies included in this review, so may not generalize to the general populations included in longitudinal studies. The distinction found in this review between the impact of alcohol advertising and that of media portrayals of alcohol use may be due to the media portrayals used not being genuine attempts at alcohol marketing at all, but rather incidental to the storylines of the television programmes and films presented and so not designed to elicit any behavioural response from the viewer. It was not made clear in primary studies whether media portrayals used were genuine examples of alcohol marketing.

Overall, the reviewed body of evidence for the immediate effects of alcohol marketing communications and media portrayals did not contribute much to elucidating differences in effects (moderation) by participant characteristics. First, included studies that measured alcohol consumption were invariably conducted among recruited samples of students. Second, a planned subgroup analysis did not identify any difference in the effect on consumption between males and females but was underpowered; while other planned subgroup analyses were precluded by lack of reporting of relevant outcome data in a useable form. Third, regarding variation by typical drinking habits, one study found that effects of alcohol advertising on alcohol consumption were larger in heavier drinkers, whilst another found no differences between lighter and heavier drinkers; and regarding variation by individual differences in executive function, one study reported no differences between participants high or low in self-control (the ability to resist impulses that conflict with long-term goals, and a marker of executive function [58]) in alcohol consumed after exposure to alcohol portrayals.

Findings were also equivocal concerning the effects of exposure to alcohol marketing communications on alcohol-related cognitions (proposed mediators of the effect on consumption). We found evidence that exposure to alcohol portrayals increased positively valenced explicit alcohol-related cognitions, but did not find evidence that exposure to alcohol advertisements influenced explicit or implicit alcohol-related cognitions. We did not identify any eligible studies that assessed the impact of exposure to alcohol portrayals on implicit alcohol-related cognitions. Viewing media portrayals of alcohol consumption in positive contexts and with positive consequences such as companionship, bonding, and celebration appears to increase general liking of alcohol and expectations that consumption will result in positive outcomes. However, this finding is based on data from only three studies. Additional limitations of meta-analyses for secondary outcomes included heterogeneous participants and specific outcome measures used in primary studies, and lack of statistical precision (especially for the two meta-analyses pertaining to alcohol advertisements). As such, current evidence from experimental studies neither undermines nor convincingly supports the proposal that exposure to alcohol marketing influences subsequent alcohol consumption by first inducing (and then, by repeated exposure, reaffirming) positive alcohol-related cognitions.

Strengths and limitations

The strengths of this systematic review lie in its methodological rigour and its novel focus on experimental studies with objectively measured alcohol consumption as the primary outcome. However, scope to fully address the review’s primary objective was limited by the overall completeness and applicability of the evidence base. In particular, we identified an absence of published experimental research into the effects of alcohol marketing delivered using channels other than visual broadcast media and involving participants other than students. The generalizability of our findings beyond these channels and this specific population subgroup has therefore yet to be established. Scope to address the review’s secondary objective, to investigate factors that might explain observed heterogeneity in effects between studies, was severely hampered by the narrow coverage of possible moderating factors in included studies, coupled with the lack of reporting of outcome data in the disaggregated form needed for planned subgroup analyses. A broader limitation of the evidence base is that individual studies tended to measure general effects of alcohol marketing communications with little or no consideration of how the creative content of individual communications is tailored to influence individuals with specific psychosocial characteristics, or how differences in individual receptivity to marketing content might modify any effects.

Implications for policy

Harmful alcohol consumption is responsible for 5.9 % of all deaths worldwide, accounts for 5.1 % of the global burden of injury and disease, and brings substantial social and economic costs to individuals and societies [59]. Restricting the marketing of alcoholic products, along with increasing price and reducing availability, is purported to be a cost-effective intervention to reduce consumption [60]. There is a lack of high quality evidence for or against the implementation of such interventions [61]. As such, alcohol marketing restrictions are not favoured by policy makers [62]. Whilst the evidence from experimental studies is currently limited, the results of this systematic review do, in our view, lend some qualified support to the public health case for restrictions, bans, or other policies that would reduce exposure to alcohol advertising on visual broadcast media to reduce alcohol consumption. Importantly, whilst the individual-level immediate effects found here may be small, such effects could, if sustained in response to overall reduced exposure over time, have a meaningful impact on consumption at the population level. Whilst the evidence synthesised in the current review concerns only the immediate effects of alcohol marketing and media exposure, the findings can be considered alongside those from systematic reviews of longitudinal studies, which imply that less exposure to alcohol marketing would lead to a later age of starting to drink alcohol and lower alcohol consumption in young people.

Implications for research

There is a need for high quality experimental and quasi-experimental studies into the effects of alcohol marketing on alcohol consumption that focus on media channels other than broadcast media, particularly in populations other than undergraduate students, and powered for main effects and subgroup analyses. In recent years there has been considerable development in the subtlety and range of alcohol marketing communications [6365]. Research into the effects of alcohol marketing has failed to keep pace with these developments. More work is required regarding how effects of alcohol marketing on consumption might be mediated by implicit processes, or modified by individual differences. Finally, we recommend more detailed reporting of future experimental studies of the acute effects of alcohol marketing exposure, so that risk of bias to their results can be comprehensively assessed.

Conclusions

Current evidence from experimental studies suggests that exposure to alcohol advertisements may increase the quantities of alcoholic beverages that people immediately consume by small amounts. Current evidence does not support the claim that exposure to portrayals of alcohol use in films and television programmes increases immediate alcohol consumption. Exposure to alcohol portrayals, but not alcohol advertisements, may increase positive explicit alcohol-related cognitions. It remains to be established whether these findings generalize beyond students and to other marketing channels.

Notes

  1. Calculated by multiplying our summary effect size and 95 % confidence intervals by the standard deviation of mean alcohol unit consumption for males (mean = 8.39 units ± 7.85) and females (mean = 5.33 units ± 4.96) on the heaviest drinking day of the week amongst alcohol users over the age of 18 in the United Kingdom. Consumption data taken from the National Diet and Nutrition Survey, 2014 [71].

Abbreviations

OIS:

Optimal information size

SMD:

Standardized mean difference

References

  1. Murphy A, Roberts B, Ploubidis GB, Stickley A, McKee M. Using multi-level data to estimate the effect of an ‘alcogenic’ environment on hazardous alcohol consumption in the former Soviet Union. Health Place. 2014;27:205–11.

    Article  PubMed  Google Scholar 

  2. Anderson P. Is it time to ban alcohol advertising? Clin Med. 2009;9:121–4.

    Article  Google Scholar 

  3. Babor T, Caetano R, Casswell S, Edwards G, Giesbrecht N, Graham K, et al. Alcohol: no ordinary commodity - research and public policy. 2nd ed. Oxford: Oxford University Press; 2010.

    Book  Google Scholar 

  4. International Centre for Alcohol Policies. Industry views on beverage alcohol advertising and marketing, with special reference to young people. 2003. http://www.icap.org/portals/0/download/all_pdfs/Other_Publications/WHO_paper_annexed.pdf. Accessed 07 Dec 2015.

  5. Portman Group. Alcohol marketing, its impact and its regulation in the UK: A briefing note. London: The Portman Group; 2010. http://www.webcitation.org/5xiTuhyCr. Accessed 07 Dec 2015.

    Google Scholar 

  6. Anderson P, De Bruijn A, Angus K, Gordon R, Hastings G. Impact of alcohol advertising and media exposure on adolescent alcohol use: a systematic review of longitudinal studies. Alcohol Alcohol. 2009;44:229–43.

    Article  PubMed  Google Scholar 

  7. Smith LA, Foxcroft DR. The effect of alcohol advertising, marketing and portrayal on drinking behaviour in young people: systematic review of prospective cohort studies. BMC Public Health. 2009;9:51.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Booth A, Meier P, Stockwell T, Sutton A, Wilkinson A, Wong R, et al. Independent review of the effects of alcohol pricing and promotion. Part A: Systematic reviews. Sheffield: School of Health and Related Research, University of Sheffield; 2008.

    Google Scholar 

  9. Del Boca FK, Darkes J. The validity of self-reports of alcohol consumption: state of the science and challenges for research. Addiction. 2003;98:1–12.

    Article  PubMed  Google Scholar 

  10. Wyllie A, Zhang J, Casswell S. Positive responses to televised beer advertisements associated with drinking and problems reported by 18 to 29-year-olds. Addiction. 1998;93:749–60.

    Article  CAS  PubMed  Google Scholar 

  11. Morgenstern M, Isensee B, Sargent JD, Hanewinkel R. Attitudes as mediators of the longitudinal association between alcohol advertising and youth drinking. Arch Pediat Adol Med. 2011;165:610–6.

    Article  Google Scholar 

  12. Morgenstern M, Isensee B, Sargent JD, Hanewinkel R. Exposure to alcohol advertising and teen drinking. Prev Med. 2011;52:146–51.

    Article  PubMed  Google Scholar 

  13. Austin EW, Chin M-J, Grube JW. How does alcohol advertising influence underage drinking? The role of desirability, identification and skepticism. J Adolescent Health. 2006;38:376–84.

    Article  Google Scholar 

  14. Fleming K, Thornton E, Atkin CK. Alcohol advertising exposure and perceptions: links with alcohol expectancies and intentions to drink or drinking in underaged youth and young adults. J Health Commun. 2004;9:3–29.

    Article  PubMed  Google Scholar 

  15. Unger JB, Schuster D, Zogg J, Dent CW, Stacy AW. Alcohol advertising exposure and adolescent alcohol use: a comparison of exposure measures. Addict Res Theory. 2003;11:177–93.

    Article  Google Scholar 

  16. Wind YJ, Sharp B. Advertising empirical generalizations: Implications for research and action. J Advert Res. 2009;49:246–52.

    Article  Google Scholar 

  17. Sutherland M, Sylvester AK. Advertising and the mind of the consumer: what works, what doesn’t, and why. 3rd ed. Australia: Allen & Unwin; 2008.

    Google Scholar 

  18. Vakratsis D, Ambler T. How advertising works: what do we really know? J Market. 1999;63:26–43.

    Article  Google Scholar 

  19. Ackoff RL, Emshoff JR. Advertising research at Anheuser-Busch, Inc. (1963–68). Sloan Manage Rev. 1975a;16(2):1–15.

    Google Scholar 

  20. Ackoff RL, Emshoff JR. Advertising research at Anheuser-Busch, Inc. (1968–74). Sloan Manage Rev. 1975b;16(3):1–15.

    Google Scholar 

  21. Austin EW, Knaus C. Predicting the potential for risky behaviour among those “too young” to drink as the result of appealing advertising. J Health Commun. 2000;5:13–27.

    Article  CAS  PubMed  Google Scholar 

  22. Austin EW, Hust SJT. Targeting adolescents? The content and frequency of alcoholic and nonalcoholic beverage ads in magazine and video formats November 1999-April 2000. J Health Commun. 2005;10:769–85.

    Article  PubMed  Google Scholar 

  23. Field M, Cox WM. Attentional bias in addictive behaviours: a review of its development, causes, and consequences. Drug Alcohol Depen. 2008;97:1–20.

    Article  Google Scholar 

  24. Folkvord F, Anschütz DJ, Nederkoorn C, Westerik H, Buijzen M. Impulsivity, “advergames,” and food intake. Pediatrics. 2014;133:1007–12.

    Article  PubMed  Google Scholar 

  25. Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological standards for the conduct of new Cochrane Intervention Reviews. http://editorial-unit.cochrane.org/mecir. Accessed 07 Dec 2015.

  26. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA Statement. PLoS Med. 2009;6:e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Brown KG, King SE, Shemilt I, Marteau TM. Immediate effects of alcohol marketing communications on consumption: A systematic review of experimental studies (protocol). PROSPERO International prospective register of systematic reviews; 2013. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42013005057. Accessed 29 Apr 2016.

  28. Higgins JPT, Altman DG, Sterne JAC. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011), The Cochrane Collaboration. 2011. Available from http://handbook.cochrane.org.

    Google Scholar 

  29. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ. 1997;315:629–34.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Furukawa TA, Barbui C, Cipriani A, Brambilla P, Watanabe N. Imputing missing standard deviations in meta-analyses can provide accurate results. J Clin Epidemiol. 2006;59:7–10.

    Article  PubMed  Google Scholar 

  31. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88.

    Article  CAS  PubMed  Google Scholar 

  32. Review Manager (RevMan) [Computer program]. Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration; 2014.

  33. Koordeman R, Kuntsche E, Anschutz DJ, van Baaren RB, Engels RCME. Do we act upon what we see? Direct effects of alcohol cues in movies on young adults’ alcohol drinking. Alcohol Alcohol. 2011;46:393–8 [Sub-group analysis of Reference 39].

    Article  PubMed  Google Scholar 

  34. Sumarta TT. Effects of videos, words and beverage cues on subsequent consumption of beverages. Diss Abstr Int: Section B: Sci Eng. 2000;61:3311.

    Google Scholar 

  35. Engels RCME, Hermans R, van Baaren RB, Hollenstein T, Bot SM. Alcohol portrayal on television affects actual drinking behaviour. Alcohol Alcohol. 2009;44:244–9.

    Article  PubMed  Google Scholar 

  36. Kohn PM, Smart RG. The impact of television advertising on alcohol consumption: an experiment. J Stud Alcohol. 1984;45:295–301.

    Article  CAS  PubMed  Google Scholar 

  37. Kohn PM, Smart RG. Wine, women, suspiciousness and advertising. J Stud Alcohol. 1987;48:161–6.

    Article  CAS  PubMed  Google Scholar 

  38. Koordeman R, Anschutz DJ, Engels RCME. Exposure to alcohol commercials in movie theatres affects actual alcohol consumption in young adult high weekly drinkers: an experimental study. Am J Addiction. 2011a;20:285–91.

    Article  Google Scholar 

  39. Koordeman R, Anschutz DJ, van Baaren RB, Engels RCME. Effects of alcohol portrayals in movies on actual alcohol consumption: an observational experimental study. Addiction. 2011b;106:547–54.

    Article  PubMed  Google Scholar 

  40. Koordeman R, Anschutz DJ, Engels RCME. The effect of alcohol advertising on immediate alcohol consumption in college students: an experimental study. Alcohol Clin Exp Res. 2012;36:874–80.

    Article  PubMed  Google Scholar 

  41. Koordeman R, Anschutz DJ, Engels RCME. Self-control and the effects of movie alcohol portrayals on immediate alcohol consumption in male college students. Front Psychiatry. 2015;5:187.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Roehrich L, Goldman MS. Implicit priming of alcohol expectancy memory processes and subsequent drinking behaviour. Exp Clin Psychopharm. 1995;3:402–10.

    Article  CAS  Google Scholar 

  43. Sobell LC, Sobell MB, Riley DM, Klajner F, Leo GI, Pavan D, Cancilla A. Effect of television programming and advertising on alcohol consumption in normal drinkers. J Stud Alcohol. 1986;47:333–40.

    Article  CAS  PubMed  Google Scholar 

  44. Wilks J, Vardanega AT, Callan VJ. Effect of television advertising of alcohol on alcohol consumption and intentions to drive. Drug Alcohol Rev. 1992;11:15–21.

    Article  CAS  PubMed  Google Scholar 

  45. Alhabash S, McAlister AR, Quilliam ET, Richards JI, Lou C. Alcohol’s getting a bit more social: when alcohol marketing messages on Facebook increase young adults’ intentions to imbibe. Mass Commun Soc. 2015;18:350–75.

    Article  Google Scholar 

  46. Bahk CM. The impact of presence versus absence of negative consequences in dramatic portrayals of alcohol drinking. J Alcohol Drug Educ. 1997;42:18–26.

    Google Scholar 

  47. Brown KG, Stautz K, Hollands GJ, Winpenny EM, Marteau TM. The cognitive and behavioural impact of alcohol promoting and alcohol warning advertisements: an experimental study. Alcohol Alcoholism. 2015;51(3):354–62. agv104.

    Article  PubMed  PubMed Central  Google Scholar 

  48. De Graaf A. Alcohol makes others dislike you: reducing the positivity of teens’ beliefs and attitudes toward alcohol use. Health Commun. 2013;28:435–42.

    Article  PubMed  Google Scholar 

  49. Dunn ME, Yniguez RM. Experimental demonstration of the influence of alcohol advertising on the activation of alcohol expectancies in memory among fourth- and fifth-grade children. Exp Clin Psychopharm. 1999;7:473–83.

    Article  CAS  Google Scholar 

  50. Goodall CE, Slater MD. Automatically-activated attitudes as mechanisms for message effects: the case of alcohol advertisements. Commun Res. 2010;37:620–43.

    Article  Google Scholar 

  51. Kotch B, Coulter ML, Lipsitz A. Does televised drinking influence children’s attitudes toward alcohol? Addict Behav. 1986;11:67–70.

    Article  CAS  PubMed  Google Scholar 

  52. Kulick AD, Rosenberg H. Influence of positive and negative film portrayals of drinking on older adolescents’ alcohol outcome expectancies. J Appl Soc Psychol. 2001;31:1492–9.

    Article  Google Scholar 

  53. McCarty D, Ewing JA. Alcohol consumption while viewing alcoholic beverage advertising. Int J Addict. 1983;18:1011–8.

    Article  CAS  PubMed  Google Scholar 

  54. Rychatrik RG, Fairbank JA, Allen CM, Foy DW, Drabman RS. Alcohol use in television programming: effects on children’s behaviour. Addict Behav. 1983;8:19–22.

    Article  Google Scholar 

  55. Slater MD, Rouner D, Murphy K, Beauvais F, Van Leuven J, Rodriguez MD. Male adolescents’ reactions to TV beer advertisements: the effects of sports content and programming context. J Stud Alcohol. 1996;57:425–33.

    Article  CAS  PubMed  Google Scholar 

  56. Van Hoof JJ, de Jong MDT, Fennis BM, Gosselt JF. There’s alcohol in my soap: Portrayal and effects of alcohol use in a popular television series. Health Educ Res. 2009;24:421–9.

    Article  PubMed  Google Scholar 

  57. Zwarun L, Linz D, Metzger M, Kunkel D. Effects of showing risk in beer commercials to young drinkers. J Broadcast Electron. 2006;50:52–77.

    Article  Google Scholar 

  58. Duckworth AL, Kern ML. A meta-analysis of the convergent validity of self-control measures. J Res Pers. 2011;45:259–68.

    Article  PubMed  PubMed Central  Google Scholar 

  59. World Health Organization. Alcohol: Fact sheet. http://www.who.int/mediacentre/factsheets/fs349/en/. Accessed 07 Dec 2015

  60. Anderson P, Chisholm D, Fuhr DC. Effectiveness and cost-effectiveness of policies and programmes to reduce the harm caused by alcohol. Lancet. 2009;373:2234–46.

    Article  PubMed  Google Scholar 

  61. Siegfried N, Pienaar DC, Ataguba JE, Volmink J, Kredo T, Jere M, Parry CDH. Restricting or banning alcohol advertising to reduce alcohol consumption in adults and adolescents. Cochrane Database Syst Rev. 2014;11:CD010704.

    PubMed  Google Scholar 

  62. UK Government Home Department. The Government’s Alcohol Strategy. London: HM Government; 2012.

    Google Scholar 

  63. Gordon R, Hastings G, Moodie C. Alcohol marketing and young people’s drinking: what the evidence base suggests for policy. J Public Affairs. 2010;10:88–101.

    Article  Google Scholar 

  64. Jernigan DH. The global alcohol industry: an overview. Addiction. 2009;104 Suppl 1:6–12.

    Article  PubMed  Google Scholar 

  65. Winpenny EM, Marteau TM, Nolte E. Exposure of children and adolescents to alcohol marketing on social media websites. Alcohol Alcohol. 2013;49:154–9.

    Article  PubMed  PubMed Central  Google Scholar 

  66. Jajodia A, Earleywine M. Measuring alcohol expectancies with the implicit association test. Psychol Addict Behav. 2003;17:126–33.

    Article  PubMed  Google Scholar 

  67. Payne KB, Govorun O, Arbuckle NL. Automatic attitudes and alcohol: does implicit liking predict drinking? Cognition Emotion. 2008;22:238–71.

    Article  Google Scholar 

  68. Bauman KE, Bryan ES. Subjective expected utility and children’s drinking. J Stud Alcohol. 1980;41:952–8.

    Article  CAS  PubMed  Google Scholar 

  69. Gorsuch RL, Arno DH. The relationship of children’s attitudes toward alcohol to their value development. J Abnorm Child Psych. 1979;7:287–95.

    Article  CAS  Google Scholar 

  70. Fromme K, Stroot E, Kaplan D. Comprehensive effects of alcohol: development and psychometric assessment of a new expectancy questionnaire. Psychol Assess. 1993;5:19–26.

    Article  Google Scholar 

  71. Public Health England and Food Standards Agency. National Diet and Nutrition Survey; 2014. http://www.noo.org.uk/data_sources/Nutrition/NDNS. Accessed 07 Dec 2015

Download references

Acknowledgments

We would like to thank Isla Kuhn for assisting with the search strategy, and Dr Saleem Alhabash, Professor Paul Kohn, and Dr Renske Koordeman for providing additional data and study information.

Funding

The study was jointly funded by the National Institute for Health Research School for Public Health Research, and by the Department of Health Policy Research Program (Policy Research Unit in Behaviour and Health [PR-UN-0409-10109]). The funding bodies had no role in the study design, analysis, decision to publish, or preparation of the manuscript.

Availability of data and materials

Not applicable.

Authors’ contributions

KB, SK, IS, and TM designed the study and prepared the protocol. KS, KB, SK, and IS participated in the searches, study selection, and data extraction. SK and KS carried out the analysis. KS and IS drafted the manuscript. All authors contributed to the critical revision of the manuscript. All authors read and approved the final manuscript.

Competing interests

KS, KB, and TMM were authors on one of the studies included in this review. These authors were not involved in decisions regarding the inclusion of this study or in the extraction of data from this study.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Theresa M. Marteau.

Additional information

Kaidy Stautz and Kyle G. Brown are joint first authors.

Additional files

Additional file 1:

Electronic searches, dates and yields. (DOCX 54 kb)

Additional file 2:

Characteristics of excluded studies. (DOCX 14 kb)

Additional file 3:

Characteristics of included studies. (DOCX 100 kb)

Additional file 4:

Risk of bias judgements. (DOCX 31 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Stautz, K., Brown, K.G., King, S.E. et al. Immediate effects of alcohol marketing communications and media portrayals on consumption and cognition: a systematic review and meta-analysis of experimental studies. BMC Public Health 16, 465 (2016). https://doi.org/10.1186/s12889-016-3116-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-016-3116-8

Keywords