Email updates

Keep up to date with the latest news and content from BMC Research Notes and BioMed Central.

Open Access Project Note

Strengthening organizational performance through accreditation research-a framework for twelve interrelated studies: the ACCREDIT project study protocol

Jeffrey Braithwaite1*, Johanna Westbrook1, Brian Johnston2, Stephen Clark3, Mark Brandon4, Margaret Banks5, Clifford Hughes6, David Greenfield1, Marjorie Pawsey2, Angus Corbett7, Andrew Georgiou1, Joanne Callen1, John Øvretveit8, Catherine Pope9, Rosa Suñol10, Charles Shaw11, Deborah Debono1, Mary Westbrook1, Reece Hinchcliff1 and Max Moldovan1

Author Affiliations

1 University of New South Wales, Kensington, NSW 2052, Australia

2 Australian Council on Healthcare Standards, 5 Macarthur Street, Ultimo NSW 2007, Australia

3 Quality in Practice/Australian General Practice Accreditation Limited, PO Box 2058, Milton QLD 4064, Australia

4 Aged Care Accreditation Agency Limited, PO Box 773, Parramatta NSW 2124, Australia

5 Australian Commission on Safety and Quality in Health Care, GPO Box 5480, Sydney NSW 2001, Australia

6 Clinical Excellence Commission, GPO Box 1614, Sydney, NSW 2001, Australia

7 University of Technology, 15 Broadway, Ultimo NSW 2007, Australia

8 Karolinska Institute, Fakturor, Box 23 109, SE-104 35 Stockholm, Sweden

9 University of Southampton, University Road, Southampton SO17 1BJ UK

10 Avedis Donabedian University Institute, Autonomous University of Barcelona, CIBER Epidemiology and Public Health (CIBERESP), Spain

11 European Society for Quality in Healthcare, St Camillus Hospital, Shelbourne Road, Limerick, Ireland

For all author emails, please log on.

BMC Research Notes 2011, 4:390  doi:10.1186/1756-0500-4-390

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1756-0500/4/390


Received:22 February 2011
Accepted:9 October 2011
Published:9 October 2011

© 2011 Braithwaite et al; licensee BioMed Central Ltd.

This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Service accreditation is a structured process of recognising and promoting performance and adherence to standards. Typically, accreditation agencies either receive standards from an authorized body or develop new and upgrade existing standards through research and expert views. They then apply standards, criteria and performance indicators, testing their effects, and monitoring compliance with them. The accreditation process has been widely adopted. The international investments in accreditation are considerable. However, reliable evidence of its efficiency or effectiveness in achieving organizational improvements is sparse and the value of accreditation in cost-benefit terms has yet to be demonstrated. Although some evidence suggests that accreditation promotes the improvement and standardization of care, there have been calls to strengthen its research base.

In response, the ACCREDIT (Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork) project has been established to evaluate the effectiveness of Australian accreditation in achieving its goals. ACCREDIT is a partnership of key researchers, policymakers and agencies.

Findings

We present the framework for our studies in accreditation. Four specific aims of the ACCREDIT project, which will direct our findings, are to: (i) evaluate current accreditation processes; (ii) analyse the costs and benefits of accreditation; (iii) improve future accreditation via evidence; and (iv) develop and apply new standards of consumer involvement in accreditation. These will be addressed through 12 interrelated studies designed to examine specific issues identified as a high priority. Novel techniques, a mix of qualitative and quantitative methods, and randomized designs relevant for health-care research have been developed. These methods allow us to circumvent the fragmented and incommensurate findings that can be generated in small-scale, project-based studies. The overall approach for our research is a multi-level, multi-study design.

Discussion

The ACCREDIT project will examine the utility, reliability, relevance and cost effectiveness of differing forms of accreditation, focused on general practice, aged care and acute care settings in Australia. Empirically, there are potential research gains to be made by understanding accreditation and extending existing knowledge; theoretically, this design will facilitate a systems view of accreditation of benefit to the partnership, international research communities, and future accreditation designers.

"Accreditation of health-care organisations is a multimillion dollar industry which shapes care in many countries. Recent reviews of research show little evidence that accreditation increases safety or improves quality. It's time we knew about the cost and value of accreditation and about its future direction." [Professor John Øvretveit, Karolinska Institute, Sweden, 7 October 2009]

Background

Service accreditation is a system of organizational improvement centred on a certifying agency (or accrediting body) assessing performance against pre-determined standards, usually by multiple means. Internationally, accreditation is designed to improve organizations by developing new standards or upgrading existing standards through research or expert advice, and by defining criteria and performance indicators and applying these standards, criteria and indicators to organizational processes and outcomes. Although models differ in detail, [1] most accreditation systems assess and rate the performance of organizations and services by evaluating their progress and appraising their compliance with standards, usually via mechanisms such as self-assessment surveys, data review and structured visits by surveyors. Some systems use peer surveyors and others persons whose background is audit methodology. Following training, assessors or surveyors have detailed knowledge of applicable standards. Figure 1 provides a generic accreditation model which illustrates a typical accreditation process from standards development into the cycle of standards application, assessment and award of accreditation and periodic review.

thumbnailFigure 1. Generic accreditation model.

The reach of accreditation is extensive and the investments in it in many sectors are considerable. Industries such as school education, [2] universities, [3,4] software manufacture, [5] the seafood sector, [6] and ambulance services [7] have embraced accreditation, standard setting, and surveying processes. Accreditation has been applied to laboratories, [8] management systems, [9] products,[10] medical curricula, [11] and staff competencies [12]. Essentially, the core concerns addressed by the processes of accreditation are whether organizations satisfy pre-designated standards, are regularly examined and continuously improved, and the extent to which customer satisfaction is met or enhanced [13]. However, we lack convincing evidence of the long-term effects and organizational impact of accreditation processes.

The health sector, as an early adopter of accreditation, has promoted its use since 1951 (the Joint Commission in the United States of America), and in Australia since 1973. Stakeholders recognize its potential to improve organizational performance, quality of care, safety standards and consumer satisfaction. However, despite support for accreditation among informed groups, accreditation has had its share of criticism, including the lack of supporting evidence and concerns about the costs of uncertain benefits and whether it offers value for money [14-16]. There is a desire among stakeholders to strengthen the research base. This project is a response to the need for a program of research into accreditation that links the key industry partners and policy bodies with interested researchers, and plans to produce results which will link with other multi-method, multi-phased studies underway in Europe [17].

The ACCREDIT (Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork) project is a partnership led by researchers in the Centre for Clinical Governance Research and Centre for Health Systems and Safety Research in the Australian Institute of Health Innovation (AIHI) at University of New South Wales with the three major Australian health-sector accreditation agencies (The Australian Council on Healthcare Standards [ACHS], Australian General Practice Accreditation Limited [AGPAL], and Aged Care and Standards Accreditation Agency [ACSAA]), the leading quality improvement policy bodies (the Australian Commission on Safety and Quality in Health Care [ACSQHC] and the Clinical Excellence Commission [CEC]), key Australian investigators, and international collaborators. These partners are dedicated to studying the impact of accreditation and to executing an extended research program, to provide evidence and empirical models for ways in which accreditation can be improved.

Research significance and importance of the problem

Some evidence suggests that accreditation programs can promote change [18] and the standardization of services and organizational processes, including how decisions about care are made [19]. However, the research literature is either inconsistent or does not support the contentions that accreditation directly improves organizational performance, quality of care, and patient satisfaction [14,20-22]. In one of the first studies to attempt to link accreditation with organizational outcomes, we found that accreditation was significantly positively correlated with organizational culture (P = 0.005) and leadership (P = 0.005), but there was weaker statistical evidence on the relation to clinical indicator performance (P = 0.080) [23]. No statistically significant association was observed between accreditation and organizational climate (P = 0.110) or consumer involvement (P = 0.377) [23].

Thirty-four of 89 selected hospitals in the European Methods of Assessing Response to Quality Improvement Strategies (MARQuIS) project [24] were accredited (without International Organization for Standardization (ISO) certification), 10 were ISO9000-certificated without accreditation and 27 had neither accreditation nor certification. On 229 criteria of quality and safety, percentage scores were 66.9, 60.0 and 51.2 respectively. These statistically significant differences suggest that accreditation is a key quality strategy. However, there were confounding factors and a small sample, and the study did not substantially differentiate between accreditation and certification only [25].

To date, work on the costs and benefits of accreditation has been rudimentary [26-28]. Unless the economic benefits are modelled, we cannot make sound policy decisions about the future enhancement of accreditation, develop a new framework for its conduct, or understand its value.

In preparatory work to develop the framework reported here, we examined the literature concerning two initiatives that have recently received policy support: unannounced (short-notice) surveys conducted by surveyors [29] and tracer methodology (i.e., patient journeys) used to assess care [30]. We found no evidence for the benefits of short-notice surveys, whereas the limited studies of patient journeys suggested that they can be useful in evaluating care. Work commissioned by ACSQHC, and undertaken by ACHS, ACSAA and AGPAL in conjunction with ACCREDIT researchers assessing these short-notice surveys and patient journeys trials tentatively indicated that these can be useful tools which complement but do not substitute for existing methods.

A systematic review of the literature conducted by the research team "... reveals a complex picture ... inconsistent findings ... [and] ... insufficient studies by which to draw conclusions."[[14] p.181] An overarching research framework with twelve interrelated studies (Figure 2) aims to address some of these gaps.

thumbnailFigure 2. Research strategy and studies.

Methods and design

General aims

We are funded to execute a multi-method, triangulated research agenda with 12 studies designed by the ACCREDIT stakeholders. The ACCREDIT partners met in August 2007 to draft the conceptual framework and research plan. They subsequently refined the plan and conducted various studies, [14,20,23,26,31-33] evaluation projects, [29,30] literature reviews, [21,34,35] and partnership activities [36] to provide the empirical platforms for this proposal. An International Advisory Group offered strategic counsel to the project, and has an ongoing advisory role.

Detailed research aims

The specific aims of the research address four main areas. These link 12 interrelated studies of issues identified as requiring research evidence as a high priority [14] (Table 1).

Table 1. Research aims, key questions, and related studies

Advancing the knowledge base

The research aims require a multi-method, [37] multi-level approach, [38] incorporating multi-layered data, [39] to provide rigorous answers to the key questions mapped to the four research aims (Table 1) and addressed in the 12 studies (Figure 1). The 12 studies are designed to answer questions to advance the accreditation knowledge base and meet expressed industry needs for empirical information. The proposal's research questions have emerged from extensive reviews [14,21,34] and consultations. Table 2 outlines the 12 interrelated studies that will address key derived questions, linking the 12 studies into four research aims.

Table 2. Twelve interrelated studies of accreditation--overview of approaches and methods

Methods, sample sizes and design features

The samples for the quantitative studies will be based on sample size calculations that ensure sufficient power to answer the questions under investigation. Qualitative studies will involve sample sizes based on saturation methods.

As shown in Table 2 a wide range of research techniques have been designed and will be applied, including objective empirical measurements, ethnographic observations, focus groups, interviews, trials, ranking exercises, and questionnaire surveys, providing a rich database. This will help create the triangulation effect often missing in discrete, project-based research, which has often produced unrelated, fragmented, and incommensurate findings in the past. A systems approach both to the triangulated multi-method design and to interpreting the findings will be taken, facilitating an understanding of the complex knowledge base that twelve interrelated studies will bring.

Discussion

We have established a partnership with the main health-care, general-practice, and aged-care accreditation providers in the country, thereby incorporating the major accreditation domains in the one overarching study. This has allowed us to design policy- and industry-relevant research, e.g., to evaluate current accreditation processes (aim 1: studies 1, 2, 3 and 12) and to improve future accreditation approaches (aim 3: studies, 7-11) (Tables 1 and 2).

There has been no persuasive cost-benefit analysis of accreditation internationally, and we intend to address this oversight in aim 2 via study 6. Insufficient work has been directed towards the assessment of new methods of accreditation, such as short-notice surveys (i.e., testing their validity) and tracking patients on their journeys through the system, whereby services are assessed based on the quality of care delivered longitudinally. These initiatives require novel assessment methods, e.g., studies 9 and 10 (Table 2).

To address aim 4, we will develop and test a new standard for consumer involvement in accreditation, which will be required for the next generation of accreditation designs [23]. The research technologies, which we will use in unique configurations across the studies, include the Delphi method in study 1, following our use of an earlier version of this in previous research; [40] ethnographic mapping in study 3, based on our experience in recent research; [41] and randomized designs applied to health-service organizational research in studies 2, 5, and 12. A mix of studies of this kind is challenging to do, but is needed given the pervasiveness of accreditation and its lack of an evidence base. ACCREDIT results from the 12 studies will facilitate a systems view of accreditation; given its complexity, this seems highly desirable.

Conclusion

The ACCREDIT project has been planned in response to questions that the partners, customers of accreditation services, policy bodies (e.g., ACSQHC and CEC), and public and private funders of health-care have raised for many years about the utility, reliability, and cost-effectiveness of accreditation. Our findings are designed to build on what we already know, fill a number of research gaps, and facilitate the improvement of accreditation and the transparency and credibility of the accreditation, surveying and standards-setting processes.

Abbreviations

ACCREDIT: Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork; ACHS: The Australian Council on Healthcare Standards; ACSAA: Aged Care and Standards Accreditation Agency; ACSQHC: Australian Commission on Safety and Quality in Health Care; AGPAL: Australian General Practice Accreditation Limited; AIHI: Australian Institute of Health Innovation; CEC: Clinical Excellence Commission; ISO: International Organization for Standardization; MARQuIS: Methods of Assessing Response to Quality Improvement Strategies

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

JB and JW are the Chief Investigators of the ACCREDIT project. JB, JW, BJ, SC, MBr, MBa, CH, DG and MP contributed to developing the overarching research strategy. AC, AG and JC are responsible for specified studies. MW provided statistical advice and expertise. DD supported the development of the research proposal and provided advice on securing ethics approval. DG is senior research fellow working across the projects and MP provides expert advice. RH and MM are respectively responsible for qualitative and quantitative research across the projects. JØ, CP, RS and CS are members of the International Advisory Group, and offered strategic counsel on the program of studies. Team members are responsible, respectively, for leading studies 5, 6 and 7 [JB]; study 12 [JW]; study 1 [BJ]; study 3 [SC]; study 2 [MBr]; study 7 [MBa]; study 9 [CH]; studies 4 and 11 [DG and MP]; study 8 [AC]; and study 10 [AG and JC]. All authors read and approved the final manuscript.

Acknowledgements and funding

This research is supported under Australian Research Council's Linkage Projects scheme (project LP100200586). Human Research Ethics Committee approval for its conduct was granted by the University of New South Wales (HREC 10274). We acknowledge the staff of the industry partners (ACHS, AGPAL, ACSAA) and the quality improvement agencies (ACSQHC, CEC) who provided support for the project. We appreciate Ms Danielle Marks' valuable research assistance in the development of this paper.

References

  1. Shaw C, Kutryba B, Braithwaite J, Bedlicki M, Warunek A: Sustainable healthcare accreditation: messages from Europe in 2009.

    Int J Qual Health Care 2010, 22:341-350. PubMed Abstract | Publisher Full Text OpenURL

  2. Stimson WA: Better public schools with ISO 9000:2000.

    Qual Prog 2003, 36:38-45. OpenURL

  3. Skilbeck M, Connell H: Quality Assurance and Accreditation in Australian Higher Education: A National Seminar on Future Arrangements. Canberra: Department of Education, Training and Youth Affairs; 2000. OpenURL

  4. Fiekers T, Littler DA, Dale BG: Applying the ISO 9000 series to higher education: an assessment model.

    International Journal of Continuing Engineering Education and Life-Long Learning 1999, 9:30-44. OpenURL

  5. Jones LG, Price AL: Changes in computer science accreditation.

    Commun ACM 2002, 45:99-103. Publisher Full Text OpenURL

  6. Yacout S, Bourbonnais P, Boudreau J: Integrating ISO 9000 with HACCP programs in seafood processing industry.

    Computers & Industrial Engineering 1998., 35 PubMed Abstract | PubMed Central Full Text OpenURL

  7. Chapleau W: Accreditation of ambulance services: building EMS for the future.

    Emerg Med Serv 2000, 29:73-75. PubMed Abstract OpenURL

  8. Gough LA, Reynolds TM: Is Clinical Pathology Accreditation worth it? A survey of CPA-accredited laboratories.

    British Journal of Clinical Governance 2000, 5:195. OpenURL

  9. Casile M, Davis-Blake A: When accreditation standards change: Factors affecting differential responsiveness of public and private organizations.

    Acad Manage J 2002, 45:180. OpenURL

  10. Casper S, Hancke B: Global quality norms within national production regimes: ISO 9000 standards in French and German car industries.

    Organization Studies 1999, 20:961-986. Publisher Full Text OpenURL

  11. Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S: General competencies and accreditation in graduate medical education.

    Health Aff 2002, 21:103. OpenURL

  12. Hoyle D, OBrien P: ISO 9000-quality systems handbook.

    New technology, work and employment 1996, 11:148-150. OpenURL

  13. Corbett CJ, Luca AM, Pan JN: Global perspectives on global standards: a fifteen-economy survey of ISO 9000 and ISO 14000.

    ISO Management Systems 2003, January-February:31-40. OpenURL

  14. Greenfield D, Braithwaite J: Health sector accreditation research: a systematic review.

    Int J Qual Health Care 2008, 20:172-183. PubMed Abstract | Publisher Full Text OpenURL

  15. Pomey MP, Contandriopoulos AP, Francois P, Bertrand D: Accreditation: a tool for organizational change in hospitals?

    Int J Qual Health Care Assur 2004, 17:113-124. Publisher Full Text OpenURL

  16. Øvretveit J: Does Improving Quality Save Money? A Review of Evidence of which Improvement to Quality Reduce Costs for Health Service Providers. London: The Health Foundation; 2009. OpenURL

  17. Groene O, Klazinga N, Wagner C, Arah O, Thompson A, Bruneau C, Suñol R, DUQuE Research Project: Investigating organizational quality improvement systems, patient empowerment, organizational culture, professional involvement and the quality of care in European hospitals: the 'Deepening our Understanding of Quality Improvement in Europe (DUQuE)' project.

    BMC Health Serv Res 2010, 10:281. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  18. Pomey M-P, Lemieux-Charles L, Champagne F, Angus D, Shabah A, Contandriopoulos A-P: Does accreditation stimulate change? A study of the context and the impact of the accreditation process on Canadian healthcare organisations.

    Implement Sci 2010, 5:31. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  19. Scrivens E, Klein R, Steiner A: Accreditation: what can we learn from the Anglophone model?

    Health Policy 1995, 34:193-204. PubMed Abstract | Publisher Full Text OpenURL

  20. Greenfield D, Braithwaite J: Developing the evidence base for accreditation of healthcare organisations: a call for transparency and innovation.

    Qual Saf Health Care 2009, 18:162-163. PubMed Abstract | Publisher Full Text OpenURL

  21. Greenfield D, Travaglia J, Braithwaite J, Pawsey M: An analysis of the health sector accreditation literature. [http://clingov.med.unsw.edu.au] webcite

    A report for the Australian Accreditation Research Network: Examining future health care accreditation research Sydney: Centre for Clinical Governance Research, UNSW; 2007. OpenURL

  22. Commonwealth of Australia: Evaluation of the impact of accreditation on the delivery of quality of care and quality of life to residents in Australian Government subsidised residential care homes. Canberra: Commonwealth of Australia; 2007. OpenURL

  23. Braithwaite J, Greenfield D, Westbrook J, Pawsey M, Westbrook M, Gibberd R, Naylor J, Nathan S, Robinson M, Runciman W, Jackson M, Travaglia J, Johnston B, Yen D, McDonald H, Low L, Redman S, Johnson B, Corbett A, Hennessy D, Clark J, Lancaster J: Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study.

    Qual Saf Health Care 2010, 19:14-21. PubMed Abstract | Publisher Full Text OpenURL

  24. Vallejo P, Suñol R: MARQuIS: quality improvement strategies for European cross-border healthcare.

    Qual Saf Health Care 2009, 18:i1-i2. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  25. Shaw C, Groene O, Mora N, Suñol R: Accreditation and ISO certification: do they explain differences in quality management in European hospitals.

    Int J Qual Health Care 2010, in press.

    October 8

    OpenURL

  26. Hindle D, Braithwaite J, Travaglia J, Iedema R: [http:/ / www.med.unsw.edu.au/ medweb.nsf/ resources/ Projects7/ $file/ Patient+Safety+report+WEB3.pdf] webcite

    Patient safety: a comparative analysis of eight Inquiries in six countries. Sydney: Centre for Clinical Governance Research, The University of New South Wales; 2006. OpenURL

  27. Braithwaite J, Westbrook J, Pawsey M, Greenfield D, Naylor J, Iedema R, Runciman B, Redman S, Jorm C, Robinson M, Nathan S, Gibberd R: A prospective, multi-method, multi-disciplinary, multi-level, collaborative, social-organisational design for researching health sector accreditation.

    BMC Health Serv Res 2006, 6:113. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  28. Dean Beaulieu N, Epstein AM: National Committee on Quality Assurance health-plan accreditation: predictors, correlates of performance, and market impact.

    Med Care 2002, 40:325-337. PubMed Abstract | Publisher Full Text OpenURL

  29. Johnston B, Hennessy D, McDonald H, Yen D, Clark S, Fahey K, Greenbank S, Petrakis N, Moody J, Braithwaite J, Greenfield D, Brandon M: [http://www.achs.org.au/] webcite

    Piloting innovative accreditation methodology: short notice surveys-final report. Sydney: Australian Council on Healthcare Standards for the Australian Commission on Safety and Quality in Health Care; 2009. OpenURL

  30. Johnston B, Hennessy D, McDonald H, Yen D, Braithwaite J, Greenfield D: [http://www.achs.org.au/] webcite

    Piloting innovative accreditation methodology: patient journeys-final report. Sydney: Australian Council on Healthcare Standards for the Australian Commission on Safety and Quality in Health Care; 2009. OpenURL

  31. Greenfield D, Pawsey M, Naylor J, Braithwaite J: Are healthcare accreditation surveys reliable?

    Int J Health Care Qual Assur 2009, 22:105-116. PubMed Abstract | Publisher Full Text OpenURL

  32. Greenfield D, Braithwaite J, Pawsey M, Johnston B, Robinson M: Distributed leadership to mobilise capacity for accreditation research.

    J Health Organ Manag 2009, 23:255-267. PubMed Abstract | Publisher Full Text OpenURL

  33. Lancaster J, Braithwaite J, Greenfield D: Benefits of participating in accreditation surveying.

    Int J Health Care Qual Assur 2010, 23:141-152. PubMed Abstract OpenURL

  34. Greenfield D, Braithwaite J: [http://clingov.med.unsw.edu.au] webcite

    A review of health sector accreditation research literature. Sydney: Centre for Clinical Governance Research, UNSW; 2007. OpenURL

  35. Greenfield D, Travaglia J, Braithwaite J, Pawsey M: Unannounced surveys and tracer methodology: literature review. [http://clingov.med.unsw.edu.au] webcite

    A Report for the Australian Accreditation Research Network: Examining future health care accreditation research Sydney: Centre for Clinical Governance Research, UNSW; 2007. OpenURL

  36. Adrian A, Pawsey M, Braithwaite J, Greenfield D, Travaglia J: [http://clingov.med.unsw.edu.au] webcite

    Australian Accreditation Research Network Project, final report for the Department of Health and Ageing. Sydney: Australian Council on Healthcare Standards and Centre for Clinical Governance Research, University of New South Wales; 2007. OpenURL

  37. Campbell DT, Fiske DW: Convergent and discriminant validation by the multitrait, multimethod matrix.

    Psychol Bull 1959, 56:81-105. PubMed Abstract OpenURL

  38. Klein KJ, Kozlowski SWJ: Multilevel Theory, Research and Methods on Organizations: Foundations, Extensions and New Directions. San Franciso: Jossey-Bass; 2000. OpenURL

  39. Fink A: Evaluation Fundamentals: Guiding Health Programs, Research and Health Policy. Newbury Park: Sage; 1993. OpenURL

  40. Clay-Williams R, Braithwaite J: Determination of health care teamwork training competencies: a Delphi study.

    Int J Qual Health Care 2009, 21:433-440. PubMed Abstract | Publisher Full Text OpenURL

  41. Nugus P, Braithwaite J: The dynamic interaction of quality and efficiency in the Emergency Department: Squaring the circle?

    Soc Sci Med 2010, 70:511-517. PubMed Abstract | Publisher Full Text OpenURL

  42. Brent RJ: Applied Cost-Benefit Analysis.

    Cheltenham, Edward Elgar 2006, in press. OpenURL