Email updates

Keep up to date with the latest news and content from BMC Public Health and BioMed Central.

Open Access Highly Accessed Research article

Development and evaluation of a youth mental health community awareness campaign – The Compass Strategy

Annemarie Wright1*, Patrick D McGorry1, Meredith G Harris12, Anthony F Jorm1 and Kerryn Pennell1

Author affiliations

1 ORYGEN Research Centre, University of Melbourne, Locked Bag 10, Parkville, Victoria, Australia

2 School of Population Health, University of Queensland, Locked Bag 500, Richlands, Queensland, Australia

For all author emails, please log on.

Citation and License

BMC Public Health 2006, 6:215  doi:10.1186/1471-2458-6-215


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2458/6/215


Received:8 January 2006
Accepted:22 August 2006
Published:22 August 2006

© 2006 Wright et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Early detection and treatment of mental disorders in adolescents and young adults can lead to better health outcomes. Mental health literacy is a key to early recognition and help seeking. Whilst a number of population health initiatives have attempted to improve mental health literacy, none to date have specifically targeted young people nor have they applied the rigorous standards of population health models now accepted as best practice in other health areas. This paper describes the outcomes from the application of a health promotion model to the development, implementation and evaluation of a community awareness campaign designed to improve mental health literacy and early help seeking amongst young people.

Method

The Compass Strategy was implemented in the western metropolitan Melbourne and Barwon regions of Victoria, Australia. The Precede-Proceed Model guided the population assessment, campaign strategy development and evaluation. The campaign included the use of multimedia, a website, and an information telephone service. Multiple levels of evaluation were conducted. This included a cross-sectional telephone survey of mental health literacy undertaken before and after 14 months of the campaign using a quasi-experimental design. Randomly selected independent samples of 600 young people aged 12–25 years from the experimental region and another 600 from a comparison region were interviewed at each time point. A series of binary logistic regression analyses were used to measure the association between a range of campaign outcome variables and the predictor variables of region and time.

Results

The program was judged to have an impact on the following variables, as indicated by significant region-by-time interaction effects (p < 0.05): awareness of mental health campaigns, self-identified depression, help for depression sought in the previous year, correct estimate of prevalence of mental health problems, increased awareness of suicide risk, and a reduction in perceived barriers to help seeking. These effects may be underestimated because media distribution error resulted in a small amount of print material "leaking" into the comparison region.

Conclusion

We believe this is the first study to apply the rigorous standards of a health promotion model including the use of a control region to a mental health population intervention. The program achieved many of its aims despite the relatively short duration and moderate intensity of the campaign.

Background

The mental health literacy of Australian and international populations has been closely examined over the past ten years [1-4]. It is an important litmus test of a population's capacity to recognize mental health problems and seek appropriate help for them. Indeed mental health literacy is a key to promoting early detection and treatment of mental health problems and thus improving longer term outcomes [1,5]. Whilst a number of population health initiatives have attempted to improve mental health literacy with the aim of improving recognition and help seeking and reducing stigma [6-9], none to date have rigorously applied models to guide their development, implementation and evaluation. This is despite such standards being commonplace in other population health areas targeting physical health problems[10,11].

In this paper we describe the use of the Precede-Proceed Health Promotion Planning Model [12] which guided the development of a program designed to improve early identification of mood disorders and psychosis in young people. Most mental disorders typically first present during adolescence and young adulthood and these are often characterized by co-morbidity [13,14]. These specific disorders and age group were considered an important target for improving mental health as early detection and treatment at the time of first onset has been found to improve long-term outcome [15,16] and reduce the risk of future episodes of illness [17,18]. The program, The Compass Strategy, was a mental health literacy community awareness campaign targeting young people aged 12–25 in the western metropolitan Melbourne and Barwon regions of Victoria, Australia. The program was implemented from May 2001 to May 2003. Three key aspects of the program are outlined here – the population assessment, the campaign strategy and the evaluation results.

Method

Model for program planning

The Precede-Proceed Model [12] was selected to guide the development, implementation and evaluation of the program as it is a theoretically robust model that addresses all these phases of program development. As reviewed by Green and Kreuter [10] it has served as a successful model in several rigorously evaluated clinical and field trials, has been widely used by government and specialist health organizations (e.g. Centres for Disease Control and Prevention (CDC)), and has been successfully applied across a range of preventive health promotion programs including early detection initiatives. Its only other application in mental health has been to promote medication compliance amongst people with chronic mental disorders [19].

The model involves nine phases (see Figure 1) and is based on the premise that a thorough assessment (Precede) should be made before planning a health promotion intervention, and evaluation (Proceed) is built in to the process to enable measurement of the effectiveness of interventions. Priority targets for intervention are established through each phase of the assessment process (phases 1–5) on the basis of causal importance in the chain of health determinants, their prevalence and their changeability. The results of this assessment process guide the development of the intervention (phase 6). The evaluation (phases 7–9) then tracks the impact of the intervention on factors identified as important targets in the assessment process.

thumbnailFigure 1. Application of the Precede-Proceed Model to The Compass Strategy.

Setting

The adjacent Victorian regions of western metropolitan Melbourne (population 794,922) and Barwon (population 221,846), were selected for the intervention as they provided the geographic and demographic variability needed to evaluate the generalisability of an initiative such as this to metropolitan, regional and rural settings. This selection was also guided by the capacity of the regions' service system to respond to the needs of young people with mental health problems via specialist youth mental health services and recent General Practitioners' training initiatives in youth mental health.

The metropolitan region of south east outer Melbourne (population 536,718), and the adjoining regional/rural area of Mornington Peninsula and west Gippsland (population 154,697) were selected as a comparison region for the impact evaluation. Selection was based upon similarities in geographic and socio-demographic factors [20] as well as suitable distance from the experimental area to ensure the presence of a buffer zone should the intervention "leak" beyond the experimental region's boundaries.

Population data sources

Focus groups

Four focus groups [21] with young people and parents affected by mental health problems were conducted in early 2001. These were based on focus group theory espoused by Krueger [22]. The aim was to explore ideas about how to convey messages regarding recognition and early help-seeking to young people and the rest of the public.

Service provider consultations

Approximately 30 consultation forums and presentations, with an average of 9 participants per session, were conducted prior to the campaign with workers from a range of key service sectors. Consultations included an overview of the rationale and strategies to be used by Compass, and an exploration of challenges faced by young people and service providers in recognising mental health problems and finding appropriate help.

Telephone survey

A cross-sectional telephone survey using structured interviews was conducted before the campaign [5] with a randomly selected sample of 600 young people aged 12–25 years from the experimental region and another 600 from the comparison region. Participants were presented with vignettes of a person with either depression or psychosis followed by questions related to recognition of disorder, best forms of treatment, perceived outcomes and exposure to the disorder and related media. The survey was repeated with an independent sample of 600 young people from each of these same regions 14 months after the campaign commenced.

Assessment

Phases 1 & 2 – Epidemiological and social assessment

Typically the application of the Precede-Proceed Model commences in phase 1 with the examination of quality of life factors that are of prime importance to the community followed by an examination of health issues connected to the attainment or deprivation of these quality of life factors (phase 2). However, as mental disorders account for 55% of the total injury and disease burden in young Australians aged 15 to 24 years [23] increasing rates of treatment has the potential to substantially reduce disease burden and consequently improve health-related quality of life [24]. Therefore, further identification of health and social targets was not pursued, and the health target was identified as early identification and treatment of depression and psychosis in young people.

However an outstanding issue that usually occurs as part of the social assessment process is engagement of the target population and/or key organizational partners [12], which is vital to the long term success and relevance of an intervention. This sub-assessment required the identification of the key stakeholders likely to have an investment in the mental health of young people, the young people and families themselves who have been affected or may be in the future, and those organizations and services which take responsibility for the treatment of these young people. The stakeholders identified included:

• mental health services

• general practitioners

• community health services

• young consumers of mental health services or their family members

• the education sector

• youth and welfare services

• youth and mental health research programs

• mental health peak bodies

• local, State and Commonwealth government departments.

Elements of the social reconnaissance method [25] were used to determine relevant aspects of the social and cultural context of the community using community leaders (general, local and specialized) as informants or interviewees. A Project Development Group comprised of representatives from the above stakeholder groups was established to guide the process of reviewing relevant research, identify community leaders, representatives or groups who would be key informants and how they might best be approached, then review the interview results and guide further developments. The principal method used to access key informants was a series of service provider forums as well as consultations with key groups via established service provider meetings.

Phase 3 – Behavioural and environmental assessment

The focus of this assessment phase was the identification of the etiological factors, or determinants of health, in the behavioural patterns and environment of the population. Factors of particular interest were those associated with early identification and receipt of treatment for mental health problems.

Behavioural assessment

The behavioural assessment required a systematic analysis of behavioural links to the identified health target of early identification and treatment. A study of pathways to care [26] and results from focus groups suggest four overall steps in the process of early identification and entry into treatment: 1. recognition of the problem; 2. seeking help for the problem; 3. delivery of appropriate treatment and 4. compliance with treatment. The ranking of importance of these problems required an examination of how frequently they occur and whether they are clearly and potently linked to the health problem.

Recognition of depression, and especially psychosis, has been identified as quite limited in a number of studies [5,27] and is an essential step in the process of effective help seeking [1,5]. Help seeking at the earliest possible stage, in turn, is essential to early receipt of treatment [26]. However, rates of help seeking are poor, particularly in relation to mood disorders [28,29]. Indeed, recognition and help seeking were considered particularly worthwhile targets as previous population health interventions have demonstrated some success in increasing them [7], although this occurred over a four to five year period. Furthermore, recognition is likely to change more quickly than help seeking, as the former is knowledge based and the latter requires behaviour change [30].

Delivery of appropriate treatment and the promotion of treatment compliance were not considered priority targets as strategies targeting these issues were already being conducted in the target area by other health and mental health services. Instead liaison with these initiatives to maximize synergies was facilitated through their membership in the Project Development Group.

Environmental assessment

The environmental assessment process focuses on factors in the immediate social and physical environment that could be causally linked to the behavioural targets (Figure 1) or directly to the outcome of interest – viz. early identification and treatment. Social environment factors associated with recognition and help seeking (as revealed through focus groups, telephone survey and a previous study [31]) include social support for help seeking, social norms of expected behaviours and responses to mental health problems, contact with a social network, and the stigma associated with mental illness. Health care environmental factors, including accessibility and availability of services [28] and the quality of treatment by treating professionals [32,33], are important factors but were managed by other health and mental health services.

Social support and social norms for recognition and help seeking were determined to be the most worthy environmental targets, as strategies targeting them have previously demonstrated some success [6,7].

Phase 4 – Educational and ecological assessment

The focus in this phase is on the factors influencing health-related behaviour and conditions of living that require change in order for behavioural and environmental change to occur. The primary premise is that for behaviour change to occur one needs the cognitive and affective building blocks that provide the motivation to act (predisposing factors) as well as resources (enabling factors) and rewards (reinforcing factors).

Predisposing factors

Predisposing factors are the knowledge, attitudes, beliefs, values, perceived needs and abilities related to the motivation to act. The principal predisposing factors challenging young people (as revealed through focus groups, telephone survey and a previous study [31]) were limited knowledge and awareness of signs, symptoms and potential seriousness of depression or psychosis and treatments available, beliefs that they weren't susceptible to these disorders or that they would be stigmatized if they sought help, and a fear of being "locked up" in a psychiatric hospital.

Reinforcing factors

The prime sources of reinforcement identified by young people (as revealed through focus groups, telephone survey and a previous study [31]) in the form of positive feedback and social support for recognition and help seeking were family and friends followed by teachers, counselors, and lay leaders, for example church leaders. Indeed, in many instances young people in the focus groups said they needed those around them to recognize that they were experiencing a mental health problem as they couldn't at the time of onset.

Focus groups and service provider consultations revealed that these social supports faced the same challenges as young people, that is, knowing the signs and symptoms and sources of help, and a lack of belief in the susceptibility to the disorders.

Enabling factors

In regard to enabling factors that facilitate the process of recognition and help seeking, accessibility and availability of quality services are obvious factors and these concerns were raised in service provider consultations. However, as highlighted earlier, these issues were managed concurrently by other programs. Lack of information about mental health problems and appropriate sources of help have been identified as a barrier to effective help seeking by young people and parents (as revealed through focus groups). The development of related skills could further enable recognition and help seeking[12] although this was determined to be a secondary target to other factors already identified that are necessary pre-requisites to skill attainment.

Therefore, the priority target factors identified in this phase for both young people and their social supports was the provision of information to improve knowledge of symptoms and sources of help for depression and psychosis, and related beliefs regarding the severity of these problems, and young people's susceptibility to them.

Phase 5 – Administration and policy assessment

This phase requires an analysis of the policies, resources, and circumstances in an organizational setting and context that either hinder or facilitate the development of a health promotion program. To a large extent this assessment was intrinsic to the establishment of the project. Funding was made available for grant application in the context of the new Australian Government mental health policy focusing on early intervention and mental health promotion [34], which meant that it was likely to complement other initiatives borne of this policy. Furthermore, provision of funding was based on the demonstration of support and capacity in the community and the service system for an initiative such as this.

Intervention

Phase 6 – Implementation

In this phase the predisposing, enabling, and reinforcing factors that influence behaviour conducive to recognition and early help seeking became the factors that shaped the program. The development and implementation of the intervention occurred over a three year period and was guided by the Process Model of Social Marketing Program Development [35]. The campaign itself ran from May 2001 to May 2003.

Strategy development

The development of the overall strategy was undertaken largely through ongoing consultation with the Project Development Group. This helped to ensure that the intervention complemented related mental health promotion programs and facilitated liaison with existing community programs and organizations so that they could assist in the delivery and promotion of the project. As the program progressed, the Project Development Group also provided a means of feedback about how the campaign was being received in the community and suggestions for refinement.

The nature of the target region was also examined to determine opportunities and limitations for health promotion activities. For example, as the intervention was restricted to one geographic area, and a matched area was studied as a comparison region, strategies such as television advertising could not be used as, due to local television network coverage, they would broadcast beyond the target region and risk contaminating the comparison region.

Audience segmentation

Whilst young people and their social supports were identified as key targets during phase 3 of the assessment process, in practical terms the target audience was divided into adults (mainly parents), and young people (self, friend or family member). Young people were further segmented according to socio-economic status and gender. Socio-economic disadvantage and lower literacy have been associated with poorer response to media campaigns, with personalized outreach approaches likely to be more effective [36]. Hence this group required a different approach to minimize the likelihood of increasing health inequalities. Some segmentation according to gender was also undertaken due to lower help-seeking rates in males [37] and higher prevalence of depression in females[38].

Development of key messages

The key messages evolved during the preliminary phase of the campaign and were gradually refined into a core slogan of "get on top of it before it gets on top of you" using a common phrase to bring to life the benefits of early recognition and help seeking. Based on the identified predisposing and reinforcing factors (Figure 1) and the Health Belief Model [39] as it relates to help seeking, this core message was supplemented by five other key messages – that young people are particularly susceptible to mental health problems, that symptoms should be taken seriously, a listing of core symptoms, getting help early is a key to successful treatment and directions to sources of more information (a website and an information line). The message was delivered in gender neutral "third person" so it would be relevant for those potentially affected or those close to them.

Channels of communication

According to Green and Kreuter [12], direct communications to the target population strengthen predisposing factors, indirect communications to key people in social networks strengthen reinforcing factors and training strengthens enabling factors. The selection of channels of communication (see Table 1) was determined by focus group feedback and national survey findings [31] with an emphasis on the use of multiple settings and strategies to maximize the likelihood of the message being received[40].

Table 1. Compass campaign modules

A range of major and minor media were reviewed for their capacity to reach young people and/or parents and their capacity to convey the core messages (see additional files 1, 2, 3, 4 for examples of Compass media campaign material). Media dissemination, whilst at times costly, provides an umbrella for attention to the issue, is able to reach the general population, and assures control over and adds authority to the core messages. It also allows for agents of reinforcement (peers, parents, schools etc.) to be exposed to the same message and therefore potentially support the desired behaviour [41]. The website and information line modules were designed to provide more information related to the core campaign messages, and to address the target enabling factors of skills in recognition and help seeking. The video and Navigator modules were developed for their potential to address all predisposing, enabling and reinforcing factors.

Additional File 1. Compass Brochure

Format: PDF Size: 127KB Download file

This file can be viewed with: Adobe Acrobat ReaderOpen Data

Additional File 2. Example of Compass newspaper advertisement

Format: PDF Size: 163KB Download file

This file can be viewed with: Adobe Acrobat ReaderOpen Data

Additional File 3. Example of Compass Postcards

Format: PDF Size: 472KB Download file

This file can be viewed with: Adobe Acrobat ReaderOpen Data

Additional File 4. Example of Compass Poster

Format: PDF Size: 550KB Download file

This file can be viewed with: Adobe Acrobat ReaderOpen Data

The selection of interventions was also guided by the Stages of Change Model [42] and stages of innovation adoption [36]. Both models have been widely applied to the field of population health [11]. The primary premise in applying these models is that different people will be at different stages of readiness to contemplate mental health issues and the benefits of early help seeking, and will move through these stages at varying rates as part of the process of recognition and help seeking. The overall modular strategy was designed to facilitate progress through these stages.

Pre-testing

Where possible, all elements of the major and minor media were pre-tested with the target audience of young people and adults using previously established health campaign pre-testing methods [43]. The aim was to ascertain message comprehension, appropriateness to youth culture, reading level, user friendliness, accuracy of take out message, capacity to engage the target audience as well as to determine any potential negative effects. The website was pre-tested for functionality and ease of navigation, and the information line underwent a series of "dry runs" prior to commencement. The video module was developed in consultation with the target audience of potential users, mainly education departments and secondary school teachers, to test out concepts and ensure synergy with current curricula and school policies. Navigator was piloted with five groups, refined and then implemented in full.

Implementation strategy

Each module had its own implementation plan to ensure appropriate timing of campaign modules in relation to stage of readiness for change and to ensure effective targeting of high impact periods e.g. for major media this was school holidays and for minor media, service providers and Navigator this was school term. The result was an overall implementation plan that proceeded in waves of intensity with a varying number of modules implemented at any one time. This strategy is common in the commercial market place and is particularly useful in making sure the impact of media strategies isn't reduced through overuse.

Ethical approval

Ethical approval for the focus groups was obtained from North Western Mental Health Behavioural and Psychiatric Research and Ethics Committee and from Barwon Health Research and Ethics Advisory Committee. Focus group participants gave written consent for their participation after reading a plain language statement describing the study.

Ethical approval for the telephone survey was obtained from the Ethics Committee of the Victorian Government Department of Human Services. Survey participants gave oral consent to participation after being read a plain language statement describing the study by the interviewer, and for those aged under eighteen years of age, oral consent was obtained from a parent or guardian.

Results

Evaluation

Phases 7, 8 & 9 – Process, impact and outcome evaluation

The evaluation process was developed concurrently with the intervention. The primary goal was to track whether the program was being implemented effectively and whether it was impacting on the targets set during the assessment process (Figure 1). The evaluation structure was guided by that outlined as part of the Precede-Proceed Model [12] and the framework developed by Hawe and colleagues [44]. The development of the evaluation plan and review of its implementation was guided by an Evaluation Committee comprised of experts in the field of mental health literacy, qualitative and quantitative research methodologies and population health program evaluation. The evaluation results described here are for the first 14 months of the program, and don't include the Navigator and Video modules which were implemented later.

Process evaluation

Process evaluation (Table 2) examined whether the program was being implemented as planned and operating effectively, and allowed for adjustments to be made to the strategy as problems or deficits arose. Regular reviews of the attainment of key performance indicators (KPIs), that is, the pre-defined essential activities linked to the attainment of established program goals in the strategic plan, were undertaken by the Project Development Group. Sixty-five KPIs were developed to measure the achievement of the program objectives, including the development of a multimedia community awareness campaign for young people, their families and community leaders and a complementary school-based awareness campaign, as well as the development of an evaluation strategy to measure campaign awareness and mental health literacy. These objectives were then linked to the goals of increased mental health literacy and earlier help seeking identified in phases 3 and 4.

Table 2. Process evaluation results

Determining whether the target population was being reached was measured in a number of ways. A "pop-up" questionnaire on the home page of the website enabled the measurement of the source of referral to the website (e.g. cinema advertisement, newspaper, school), the age and gender of the user and their postcode. A similar questionnaire was used as part of the information line. This was supplemented by monitoring the demand for minor media resources (e.g. brochures, postcards, posters) by schools and community agencies. Feedback from these services and from website users helped measure the utility of these resources.

The effectiveness of the information line and website were tracked by monitoring hits to the website and calls to the information line. Examination of these data in parallel with the varying intensity of the campaign waves indicated that as the campaign intensity increased so did visits to the website and calls to the information line.

Impact evaluation

The telephone survey was the primary measure for impact evaluation. The baseline survey was repeated with an independent sample of 600 young people in the experimental region and 600 young people from the comparison region. The sample was stratified by age (12–14 years (25%), 15–17 years (25%) and 18–25 years (50%)) and region at both time points. Because there was no existing research to indicate a likely effect size, a power analysis was carried out using Cohen's [45] "small" effect size as the criterion (i.e. h = 0.2 for a difference between independent proportions). For this effect size, a sample of 392 in each region gave 80% power with an alpha of 0.05, and a sample of 584 had the same power with an alpha of 0.01.

The pre-campaign study response rate was 89.7% and the post-campaign response rate was 86.8%. Logistic regression analysis was used to ascertain whether a range of sample characteristics varied across time points or regions. There were no significant region-by-time interaction effects for sample differences in gender, school enrolment, unemployment or language other than English spoken at home. Age was not included in this analysis due to sample stratification.

A series of binary logistic regression analyses were used to measure the association between a range of campaign outcome variables and the predictor variables of region (experimental, comparison) and time (pre-campaign, post-campaign). The regression analysis was carried out controlling for vignette where outcome variables specifically related to the depression or psychosis vignette. The outcome variables considered in the analysis were correct recognition of disorder, form of help recommended, helpfulness rating of professionals and medications, perceived prognosis, perceived likelihood of discrimination, exposure to mental illness in family and friends, self-identified depression, help sought for the first time for self-identified depression during the time of the campaign, likely behaviours associated with the disorder (e.g. suicide), barriers to help seeking for self, correct estimate of prevalence and exposure to mental health media campaigns. The program was judged to have an impact if there were significant region-by-time interaction effects. Results using Wald statistics are reported in Table 3.

Table 3. Percentage of survey respondents reporting outcomes pre- and post-campaign in experimental and comparison regions.

Significant results were observed for the outcome variables of perceived suicide risk, barriers to help seeking ("what others might think" and "thinking that nothing can help"), media exposure, correct prevalence estimate, and self-identified depression. There was an overall significant change in the proportion of those seeking help over time in the experimental region. However, when the help-seeking rate was examined specifically for those identifying themselves as having a mental health problem, the effect was non-significant. Non-significant effects were recorded for all other outcome variables including the key targets of recognition and knowledge of sources of help. A post hoc analysis revealed a greater impact of the campaign on the same outcome variables (Table 3) in the Barwon sub-region.

Outcome evaluation

For this level of evaluation, it was planned to examine changes in service utilisation, duration of untreated illness and number of contacts in pathways to care. However, practical difficulties in gathering data over the whole intervention region meant that this plan could not be achieved.

Discussion

We believe this is the first study to apply the rigorous standards of a health promotion model to a mental health population intervention. The program achieved many of its aims despite the relative short duration and moderate intensity of the campaign.

Process evaluation revealed that the website was a far more frequently used source of information than the information line, and was more effective at attracting use by the target age group of young people. Newspapers, schools, posters and General Practitioners were the most effective website referral sources. Schools were also the greatest source of service provider demand for print materials.

Results of the impact evaluation telephone survey revealed significant changes in the target region across the range of impact targets. The overall recall of mental health campaigns was significantly higher in the target region. This suggests that the enabling factor objective of making mental health information more available was achieved. Three key predisposing and reinforcing factors showed significant change. Beliefs regarding the risk of suicide associated with depression and psychosis increased or stabilised in the target region whilst it declined in the comparison region. "Thinking that nothing can help" decreased as a barrier to help seeking in the target region, whilst it either increased or stabilised in the comparison region, suggesting that young people became more positive toward the potential benefits of treatment. Self-identified depression significantly increased in the experimental region and is reported here as a predisposing factor as it is most likely to be an indication of increased awareness of depression rather than an increase in prevalence per se [9]. This interpretation is supported by the finding that the relative proportion of these self-identifiers who had sought help did not change significantly over time.

In relation to the environmental targets of social supports and social norms, perceived negative evaluation by others regarding help seeking decreased in the experimental area, and perceived prevalence of mental health problems increased. A small but significant increase in the behavioural target of help seeking during the implementation of the campaign was observed in the target area.

The high response rate for the telephone surveys and the comparability of sample characteristics across time and regions strengthens the credibility of the results. The measurement tool itself has established validity [46], although its validity in telephone interview format has not been established. Observed changes may be an underestimate of the impact of the campaign given that a small amount of media distribution error resulted in some of the print material "leaking" into the comparison region and it was not possible to put geographical restriction on website access.

The use of website and information line referral data was a useful process evaluation tool, but several limitations must be considered. The use of visits per month may be an underestimate of individual actual visitors to the website as firewalls on computer networks within services or institutions will only represent this as one visitor when it may be several people from the same network. Also, the collection of referral source data was at times open to ambiguity where a referral source could be categorised under more than one heading e.g. poster, GP (poster in waiting room) or school (poster on premises). Similarly, referral from "school" to the website could be through recommendation from a school counsellor for personal use or research for a school project on mental health. Furthermore, the limitations of the referral source data as a measure of campaign effectiveness must be considered in relation to major and more expensive media such as radio and cinema advertising. Whilst neither of these media rated well as a referral source, it is difficult to ascertain the degree to which they had a role in priming or reinforcing key messages of the campaign. Indeed, the post hoc analysis revealed a greater impact of the campaign in the Barwon region and this was also the only experimental sub-region where radio advertising was implemented and there was a higher media distribution rate per head of population.

This is one of only a small number of studies that have used a comparison region to evaluate a mental health literacy community awareness campaign regarding depression [9,47], and the first study using this methodology in relation to psychosis. Data from the comparison region was critical in establishing the effectiveness of Compass as the strategy was operating concurrently with a range of national mental health community awareness initiatives, including beyondblue: the national depression initiative [48], MindMatters school based initiatives [49], and campaigns by SANE Australia [50].

In regard to time taken to have a significant impact on mental health literacy, comparisons are difficult due to differences in study design and campaign intensity and spread. Hegerl and colleagues [47] reported no significant impact after a 10 month regional campaign, Jorm and colleagues [9] reported significant changes over a nine year period which included a four year national campaign, whilst Compass demonstrated significant changes after a 14 month regional campaign.

Whilst a number of psychosis studies have examined the impact of a community awareness campaign on help seeking, in particular duration of untreated symptoms [51-53], they have not reported on the interim impact of these campaigns on mental health literacy. This may be of particular importance where a community awareness campaign does not demonstrate an impact on duration of untreated illness [52] or help seeking, as the campaign may not have had enough time to take effect. Timely measures of mental health literacy may provide evidence that some change has occurred, heralding longer term impacts on help seeking, so that campaigns can be continued. However, it must be acknowledged that evidence of this study's effects on help seeking have not been fully evaluated.

Conclusion

The Compass Strategy was effective over a relatively short period of time with a moderate intensity campaign. This effectiveness may be attributed to the rigour of the Precede-Proceed Model which facilitated fine tuning of the campaign targets through the population assessment process, the use of evidence based campaign strategies, and the capacity to refine the campaign elements in response to regular reviews of process evaluation findings. Indeed, the Precede-Proceed Model may provide an efficient means for designing effective mental health community awareness campaigns in the future as it is potentially adaptable to all contexts and mental disorders.

The local media materials, website and video developed as part of Compass are currently being disseminated throughout the state of Victoria, Australia. Dissemination of Compass modules internationally would need to be considered in the context of appropriate fit with identified priority targets following the application of the Precede-Proceed Model to the population of interest.

Competing interests

The author(s) declare that they have no competing interests.

Authors' contributions

AW was the Project Manager and was responsible for conception and design of the study; funding acquisition; coordination of the intervention and data acquisition; analysis and interpretation of the data; and drafting of the manuscript. PDM was the Project Director and was involved in the conception, acquisition of funding, project design and management, interpretation of the data and production of the manuscript. MGH was involved in the design, coordination and evaluation of the project and drafting of the manuscript. AFJ was involved in the planning of the evaluation and in the analysis and writing of the paper. KP was involved in the acquisition of funding, design and coordination of the study, and drafting of the manuscript. All authors read and approved the final manuscript.

Acknowledgements

The authors would like to thank the following people and organisations for their contributions to this study : Dr. Tom Callaly, Barwon Health, Clockwork Young People's Health Service, Sarah Spurr, Donna Gee, Sophia Liddy, Robyn Frances, Maureen Joss, David Juriansz, Sue Cotton, Linda Mack, Dr. Stephen Carbone, Margaret Sheehan, Dr. Ruth Parslow, Guillaume Roux, Adam Doherty, focus group and survey participants, and members of the Project Development Group, the Evaluation Committee and the Strategic Steering Committee. The telephone survey was conducted by the Wallis Consulting Group.

Funding for the study was provided by ANZ Trustees, beyondblue: the national depression initiative, the State Government of Victoria Department of Human Services, and the Australian Commonwealth Government Department of Health and Ageing.

AW was funded by ANZ Trustees, beyondblue: the national depression initiative and ORYGEN Research Centre, The University of Melbourne; PDM was funded by ORYGEN Research Centre, The University of Melbourne; MGH was funded by The University of Melbourne; AFJ was funded by the National Health and Medical Research Council; and KP was funded by ORYGEN Research Centre, The University of Melbourne. Beyondblue: the national depression initiative was represented in the Evaluation Committee of The Compass Strategy and, as such, participated in the development of the study design.

References

  1. Jorm AF: Mental health literacy. Public knowledge and beliefs about mental disorders.

    British Journal of Psychiatry 2000, 177:396-401. PubMed Abstract | Publisher Full Text OpenURL

  2. Goldney RD, Fisher LJ, Wilson DH: Mental health literacy: an impediment to the optimum treatment of major depression in the community.

    Journal of Affective Disorders 2001, 64:277-284. PubMed Abstract | Publisher Full Text OpenURL

  3. Highet NJ, Hickie IB, Davenport TA: Monitoring awareness of and attitudes to depression in Australia.

    Medical Journal of Australia 2002, 176:S63-8. PubMed Abstract | Publisher Full Text OpenURL

  4. Lauber C, Nordt C, Falcato L, Rossler W: Do people recognise mental illness? Factors influencing mental health literacy.

    European Archives of Psychiatry and Clinical Neuroscience 2003, 253:248-251. Publisher Full Text OpenURL

  5. Wright A, Harris MG, Wiggers JH, Jorm AF, Cotton SM, Harrigan S, Hurworth RE, McGorry PD: Recognition of depression and psychosis by young Australians and their beliefs about treatment.

    Medical Journal of Australia 2005, 183:18-23. PubMed Abstract | Publisher Full Text OpenURL

  6. Paykel ES, Hart D, Priest RG: Changes in public attitudes to depression during the Defeat Depression Campaign.

    British Journal of Psychiatry 1998, 173:519-522. PubMed Abstract OpenURL

  7. Johannessen JO, McGlashan TH, Larsen TK, Horneland M, Joa I, Mardal S, Kvebaek R, Friis S, Melle I, Opjordsmoen S, Simonsen E, Ulrik H, Vaglum P: Early detection strategies for untreated first-episode psychosis.

    Schizophrenia Research 2001, 51:39-46. PubMed Abstract | Publisher Full Text OpenURL

  8. Regier DA, Hirschfeld RM, Goodwin FK, Burke JD, et al.: The NIMH Depression Awareness, Recognition, and Treatment Program: Structure, aims, and scientific basis.

    American Journal of Psychiatry 1988, 145:1351-1357. PubMed Abstract OpenURL

  9. Jorm AF, Christensen H, Griffiths KM: The impact of beyondblue:the national depression initiative on the Australian public's recognition of depression and beliefs about treatment.

    Australian & New Zealand Journal of Psychiatry 2005, 39:248-254. Publisher Full Text OpenURL

  10. Green LW, Kreuter MW: Health program planning: an educational and ecological approach. 4th edition. New York, McGraw Hill; 2004.

  11. Glanz K, Rimer BK, Lewis FM: Health behaviour and health education. Theory, research and practice. 3rd edition. San Francisco, Jossey-Bass; 2002.

  12. Green LW, Kreuter MW: Health promotion planning: an educational and ecological approach. 3rd edition. Mountain View, Mayfield Publishing Company; 1999.

  13. Kessler RC, Berglund P, Demler O, Jin R, Walters EE: Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey replication.

    Archives of General Psychiatry 2005, 62:593-602. PubMed Abstract | Publisher Full Text OpenURL

  14. Sawyer MG, Kosky RJ, Graetz BW, Arney FM, Zubrick SR, Baghurst PA: The National Survey of Mental Health and Wellbeing: the child and adolescent component.

    Australian & New Zealand Journal of Psychiatry 2000, 34:214-220. Publisher Full Text OpenURL

  15. Kupfer DJ, Frank E, Perel JM: The advantage of early treatment intervention in recurrent depression.

    Archives of General Psychiatry 1989, 46:771-775. PubMed Abstract OpenURL

  16. Loebel AD, Lieberman JA, Alvir JM, Mayerhoff DI, Geisler SH, Szymanski SR: Duration of psychosis and outcome in first-episode schizophrenia.

    American Journal of Psychiatry 1992, 149:1183-1188. PubMed Abstract OpenURL

  17. Johnstone EC, Crow TJ, Johnson AL, MacMillan JF: The Northwick Park Study of first episodes of schizophrenia. I. Presentation of the illness and problems relating to admission.

    British Journal of Psychiatry 1986, 148:115-120. PubMed Abstract OpenURL

  18. Kroll L, Harrington R, Jayson D, Fraser J, Gowers S: Pilot study of continuation cognitive-behavioral therapy for major depression in adolescent psychiatric patients.

    Journal of the American Academy of Child & Adolescent Psychiatry 1996, 35:1156-1161. Publisher Full Text OpenURL

  19. Kelly GR, Scott JE, Mamon J: Medication compliance and health education among outpatients with chronic mental disorders.

    Medical Care 1990, 28:1181-1197. PubMed Abstract OpenURL

  20. Australian Bureau of Statistics: 1996 Census of population and housing. Sydney, ABS; 1996.

  21. Hurworth RE, Wright A: Focus group report concerning help seeking by young people with psychosis or depression. Melbourne, Centre for Program Evaluation, University of Melbourne; 2001.

  22. Krueger R: Focus groups. 2nd edition. Thousand Oaks, California, Sage; 1994.

  23. Mathers C, Vos T, Stevenson C: The burden of disease and injury in Australia. Canberra, The Australian Institute of Health and Welfare.; 1999.

  24. McGorry PD, Edwards J, Mihalopoulos C, Harrigan SM, Jackson HJ: EPPIC: an evolving system of early detection and optimal management.

    Schizophrenia Bulletin 1996, 22:305-326. PubMed Abstract OpenURL

  25. Nix HL, Seerly NR: Community reconnaissance method: a synthesis of functions.

    Journal of Community Development Society 1971, 11:62-69. OpenURL

  26. Lincoln C, Harrigan S, McGorry PD: Understanding the topography of the early psychosis pathways. An opportunity to reduce delays in treatment.

    British Journal of Psychiatry - Supplementum 1998, 172:21-25. OpenURL

  27. Jorm AF, Korten AE, Jacomb PA, Christensen H, Rodgers B, Pollitt P: "Mental health literacy": a survey of the public's ability to recognise mental disorders and their beliefs about the effectiveness of treatment.

    Medical Journal of Australia 1997, 166:182-186. PubMed Abstract | Publisher Full Text OpenURL

  28. Sawyer MG, Arney FM, Baghurst PA, Clark JJ, Graetz BW, Kosky RJ, Nurcombe B, Patton GC, Prior MR, Raphael B, Rey J, Whaites LC, Zubrick SR: Mental Health of Young People in Australia. Canberra, Commonwealth Department of Health and Aged Care; 2000.

  29. McLennan A: Mental health and well-being: Profile of adults, Australia 1997. Canberra, Australian Bureau of Statistics; 1998.

  30. Green LW, Ottoson JM: Community and population health. 8th edition. Boston, WCB / McGraw-Hill; 1999.

  31. KeysYoung: Research and consultation among young people on mental health issues. Final report. Canberra, Commonwealth Department of Health and Family Services; 1997.

  32. National Health and Medical Research Council: Depression in young people. Clinical practice guidelines. Canberra, The Australian Government Publishing Service; 1997.

  33. The University of Melbourne: The Australian Clinical Practice Guidelines for Early Psychosis. Melbourne, EPPIC; 1997.

  34. Australian Health Ministers: Second National Mental Health Plan. Canberra, AGPS; 1998.

  35. Lebvre RC, Rochlin L: Social marketing. In Health behaviour and health education Theory, research and practice. 2nd edition. Edited by Glanz K, Lewis FM and Rimer BK. San Francisco, Jossey-Bass; 1996. OpenURL

  36. Rogers EM: Diffusion of innovations. 4th edition. New York, Free Press; 1995.

  37. Verbrugge LM: Sex differences in complaints and diagnoses.

    Journal of Behavioural Medicine 1980, 3:327-355. Publisher Full Text OpenURL

  38. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Washington, DC, American Psychiatric Association; 2000.

  39. Janz NK, Becker MH: The Health Belief Model: A decade later.

    Health Education Quarterly 1984, 11:1-47. PubMed Abstract OpenURL

  40. Redman S, Spencer EA, Sanson-Fisher RW: The role of mass-media in changing health related behaviour: a critical appraisal of two models.

    Health Promotion International 1990, 5:85-101. OpenURL

  41. Manoff RK: Social marketing: New imperative for public health. New York, Praeger; 1985.

  42. Prochaska JO, Di Clemente CC: Stages and processes of self-change in smoking: Towards an integrative model of change.

    Journal of Consulting and Clinical Psychology 1983, 5:390-395. Publisher Full Text OpenURL

  43. National Cancer Institute: Making health communications work. A planners guide. Bethesda, MD, Office of Cancer Communications, National Cancer Institute; 1989.

  44. Hawe P, Degeling D, Hall J: Evaluating health promotion: A health worker's guide. Sydney, MacLennon and Petty; 1990.

  45. Cohen J: A power primer.

    Psychological Bulletin 1992, 112:155-159. Publisher Full Text OpenURL

  46. Jorm AF, Korten AE, Jacomb PA, Rodgers B, Pollitt P, Christensen H, Henderson S: Helpfulness of interventions for mental disorders: beliefs of health professionals compared with the general public.

    British Journal of Psychiatry 1997, 171:233-237. PubMed Abstract OpenURL

  47. Hegerl U, Althaus D, Stefanek J: Public attitudes toward treatment of depression: effects of an information campaign.

    Pharmacopsychiatry 2003, 36:288-291. PubMed Abstract | Publisher Full Text OpenURL

  48. beyondblue:the national depression initiative: beyondblue:the national depression initiative. [http://www.beyondblue.org.au] webcite

  49. MindMatters: MindMatters. [http://cms.curriculum.edu.au/mindmatters/] webcite

  50. SANE Australia: SANE Australia. [http://www.sane.org] webcite

  51. Chong SA, Mythily S, Verma S: Reducing the duration of untreated psychosis and changing help-seeking behaviour in Singapore.

    Social Psychiatry and Psychiatric Epidemiology 2005, 40:619-621. Publisher Full Text OpenURL

  52. Malla A, Norman R, Scholten D, Manchanda R, McLean T: A community intervention for early identification of First Episode Psychosis.

    Social Psychiatry and Psychiatric Epidemiology 2005, 40:337-344. Publisher Full Text OpenURL

  53. Melle I, Larsen TK, Haahr U, Friis S, Johannessen JO, Opjordsmoen S, Simonsen E, Rund BR, Vaglum P, McGlashan TH: Reducing the duration of untreated first-episode psychosis. Effects on Clinical presentation.

    Archives of General Psychiatry 2004, 61:143-150. PubMed Abstract | Publisher Full Text OpenURL

  54. Kitchener BA, Jorm AF: Mental health first aid training for the public: evaluation of effects on knowledge, attitudes and helping behaviour.

    BMC Psychiatry 2002., 2 PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2458/6/215/prepub