Open Access Research article

Effectiveness of a quality management program in dental care practices

Katja Goetz1*, Stephen M Campbell12, Björn Broge3, Marc Brodowski3, Michel Wensing14 and Joachim Szecsenyi13

Author Affiliations

1 Department of General Practice and Health Services Research, University of Heidelberg, Vosstr. 2, Building 37, Heidelberg D-69115, Germany

2 Centre for Primary Care, Institute of Population Health, University of Manchester, Manchester, UK

3 AQUA-Institute for Applied Quality Improvement and Research in Health Care, Goettingen, Germany

4 Scientific Institute for Quality of Health Care, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands

For all author emails, please log on.

BMC Oral Health 2014, 14:41  doi:10.1186/1472-6831-14-41

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6831/14/41


Received:12 July 2013
Accepted:23 April 2014
Published:28 April 2014

© 2014 Goetz et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.

Abstract

Background

Structured quality management is an important aspect for improving patient dental care outcomes, but reliable evidence to validate effects is lacking. We aimed to examine the effectiveness of a quality management program in primary dental care settings in Germany.

Methods

This was an exploratory study with a before-after-design. 45 dental care practices that had completed the European Practice Assessment (EPA) accreditation scheme twice (intervention group) were selected for the study. The mean interval between the before and after assessment was 36 months. The comparison group comprised of 56 dental practices that had undergone their first assessment simultaneously with follow-up assessment in the intervention group. Aggregated scores for five EPA domains: ‘infrastructure’, ‘information’, ‘finance’, ‘quality and safety’ and ‘people’ were calculated.

Results

In the intervention group, small non-significant improvements were found in the EPA domains. At follow-up, the intervention group had higher scores on EPA domains as compared with the comparison group (range of differences was 4.2 to 10.8 across domains). These differences were all significant in regression analyses, which controlled for relevant dental practice characteristics.

Conclusions

Dental care practices that implemented a quality management program had better organizational quality in contrast to a comparison group. This may reflect both improvements in the intervention group and a selection effect of dental practices volunteering for the first round of EPA practice assessment.

Keywords:
Dental care; Oral health care; Quality of care; Quality improvement; Quality management

Background

Improving the quality of healthcare is a high priority in Western health care systems [1] driven by factors such as reducing adverse events, optimizing efficiency, and enhancing patient satisfaction [2]. An excellent definition of quality in healthcare is given by Mills & Batchelor [3]. In essence, however, quality of care can been defined and evaluated in terms of structure, process and outcomes [4]. Whilst the presence of specific organizational structures does not necessarily result in better clinical processes and outcomes, organizational aspects are certainly enablers of higher performance [4,5]. Little is known about how to improve quality of organizational aspects of primary sector dental care. The majority of literature and the evidence base for defining and measuring quality in primary care come from general medical practice and not from oral health care settings [6]. However, assessing and monitoring the quality of dental care play an important role in quality assurance and quality improvement [7].

In most health care systems, a variety of quality improvement initiatives have been implemented to enhance both health care management broadly speaking and dental health care specifically speaking [3,7-10]. For instance, in the United Kingdom quality indicators were developed for the new National Health Service (NHS) dental contract which targets measuring the quality of patient care as well as performance [3]. In 2005, in Scotland, the “Action Plan for Improving Oral Health and Modernising NHS Dental Services” was announced [11]. Since 1997, in the United States, an assessment instrument developed and initiated by MetLife has been implemented for dental care providers [12]. Particularly countries such as the United Kingdom, the United States and Canada have shown expertise in development and implementation of quality management systems. Quality management means quality assurance: the systematic measurement and monitoring of process, structure and outcome of care and results in a continuous improvement process. For example, the plan-do-study-act cycle, to ensure quality of care [9,13]. In 2006, the German government stipulated that general dental practitioners should implement a system of annual assessment of quality management, in the same way that general medical practices are expected to do [14]. Although, to date, there are no formal sanctions, so participation remains voluntary. The result has been that different quality management systems have become available for health care providers in primary care settings [15]. These different quality management systems measure structure and process of care as well as non-clinical outcomes of patients.

However, while such quality management programs are available for dental care, evidence on their impact and effectiveness is sparse [6], with some exceptions [15,16]. There is an urgent need for validated quality assessment tools for dental care [7]. The European Practice Assessment is a comprehensive, integrative and multifaceted tool for quality assessment and quality improvement in health care in terms of quality management. It is based on quality indicators developed for use in primary medical care settings to evaluate the structure and process of care [17]. The European Practice Assessment tool has shown effectiveness in improving the management of general medical practices [18,19]. The current study focuses on the implementation and repeated measurement of European Practice Assessment tool in primary dental care settings and examined whether improvements occurred in dental care practice that completed the European Practice Assessment twice compared with dental care practices that completed the European Practice Assessment once.

Methods

European practice assessment

In 2004, the European Practice Assessment was developed for general medical care and already in 2005, the content as well the process was adapted for general dental care settings [17,19,20]. An expert group consisting of dentists and employees of the Institute for Applied Quality Improvement and Research in Health Care (AQUA-Institute) adapted and piloted the European Practice Assessment for dental care practices. In both cases, the European Practice Assessment consists of a set of validated quality indicators for external and internal assessment including a patient survey of satisfaction with care and a staff job satisfaction survey, an outreach visit by a trained external facilitator, structured feedback, a team-meeting in the practice and formal accreditation by an external organization [17]. These different indicators are framed within five key conceptual domains: “infrastructure”, “information”, “finance”, “quality and safety” and “people”. Some indicators for dental care practices are different to those for general medical care. The domain “infrastructure” was expanded by dimensions ‘material management’ and ‘laboratory management’; the domain “quality and safety” by dimensions ‘safety of staff and patients, hygiene, infection control’ and ‘provisions for emergency situations’; and the domain “information” by dimensions ‘communication with other health care providers’ and ‘information for patients on practice, practice policy and community resources’. Detailed information about the European Practice Assessment for dental care settings on a website is under construction.

After the implementation of European Practice Assessment in either a general medical care practice or a dental care practice, feedback is given by the trained facilitator using software called Visotool, which shows the results of the assessment. An anonymous comparison between a practice’s score and all other practices that have undertaken the assessment is available, which serves as a catalyst for quality improvement and for benchmarking. The application of the European Practice Assessment is coordinated by the AQUA-Institute based in Goettingen, Germany [21].

Design and participants

The study conforms to the STROBE-Guidelines. A before-after study design was used with an intervention group and a comparison group of dental care practices. We included dental care practices in Germany that had completed the European Practice Assessment as part of a quality management program. For the intervention group, we identified all 45 dental care practices which had completed the European Practice Assessment twice, with their first assessment between April 2005 and December 2008 and their second assessment between April 2008 and January 2012. The interval between first and second assessment was around 36 months. During the period between April 2008 and May 2011 when the intervention practices were undergoing their second assessment, 57 dental care practices were undergoing their first assessment with the European Practice Assessment program but none had yet had a second assessment. These were called upon as our comparison group. One dental care practice was integrated into a general medical care centre and was excluded from the analysis. Therefore, 56 practices in total were included in the comparison group. Figure 1 shows the distribution of dental care practices to intervention and comparison groups respectively.

thumbnailFigure 1. Selection of private dental care practice for the intervention and comparison.

Statistical analysis

The analyses were performed using SPSS version 20.0 (SPSS Inc., IBM). Continuous data were summarized using means and standard deviations. Categorical data were presented as frequency counts and percentages. The practice characteristics of the intervention and comparison groups were compared using Students t-test for continuous data and Chi2 test for categorical data.

Z scores were used to compare the overall mean score across all domains and dimensions of the European Practice Assessment instrument, in the intervention (second assessment) and comparison groups. Z scores were used to normalize raw scores [18,22]. The mean scores of all domains and dimensions were based on the proportion of indicators for which a positive response was achieved by all of the practices, on a scale of 0 to 100. The 95% confidence intervals were calculated for the differences in scores between the first and second assessment and for the differences between the second assessment and the comparison group. Furthermore, linear regression analyses were performed with aggregated scores on each of the five domains as dependent outcomes; practice characteristics as well as the affiliation to intervention (second assessment) or comparison group were handled as potential predictors. An alpha level of p < 0.05 was used for tests of statistical significance.

Ethics approval

Ethical approval was not required because we used secondary data available from the routine implementation of a quality management program in primary dental care sector in Germany [14]. All elements of the European Practice Assessment and the information from the trained external facilitators were anonymized for data analysis in our study. No additional information or data from patients or staff were collected.

Results

The practices included in the intervention (second assessment) and comparison groups did not differ significantly in the relevant covariates of practice characteristics (Table 1).

Table 1. Characteristics of the study sample

Domains of the European practice assessment

Changes in the scores for the five domains and all dimensions for both the intervention group (first and second assessment) and comparison groups are shown in Table 2. We observed an improvement in quality management in the intervention group, with a change in score of 4.4 encompassing all 281 indicators within the five domains. A higher score occurred in the intervention group in all five domains, particularly in the domains “information” (T0 82.6, T1 89.0) and “quality and safety” (T0 82.4, T1 88.7), which showed an improvement over 6.0. Compared with the comparison group at first assessment, the intervention group at second assessment had significantly better scores for the domains “infrastructure” (T1 94.2, Comparison group 84.0) and “quality and safety” (T1 88.7, Comparison group 78.1).

Table 2. Mean scores* for the domains and dimensions of the European practice assessment instrument

Improvements were observed in the intervention group in the dimensions “complaint management” and “communication with other health care providers”. Compared with the comparison group at first assessment, the intervention group at second assessment had significantly better score in the dimensions such as “medical equipment, including drugs”, “staff management”, “prevention” and “complaint management”.

Elements associated with each domain of the European practice assessment

Table 3 shows the results of the regression analyses separated for each of the five domains regarding practice characteristics and affiliation to intervention or comparison group. The five regression models explained between 6.4% and 27.2% of the variance respectively. The only independent variable that showed a significant association with each domain was intervention versus comparison group affiliation with the intervention group demonstrating higher scores in each domain.

Table 3. Impact of the practice characteristics and group affiliation for each domain (results of linear regression analyses, under specification of standardized beta coefficient and 95% confidence interval (CI), α = 5%)

Discussion

To our knowledge, this is the first study that has evaluated and demonstrated quality improvement in primary dental care practices in Germany. In this study, a repeated measurement was used to evaluate the effect of the assessment process using the European Practice Assessment quality management program. The intervention and comparison group practices did not differ remarkably in comparison to general medical practice characteristics. Furthermore, the baseline data and the first assessment of the intervention group showed higher scores than in the comparison group within the five key domains (‘infrastructure’, ‘people’, ‘information’, ‘finance’, and ‘quality and safety’). The comparison of the results of the second assessments in intervention practices with the baseline assessments in comparison group practices showed improvements across all domains, but especially within the domains of ‘quality and safety’ and ‘infrastructure’.

Quality improvement depends on a set of valid and feasible quality indicators that are able to measure quality of care [23]. “Indicators are measurable elements of practice for which there is evidence or consensus that they reflect quality and hence help change the quality of care provided” [23,24]. The implementation of a quality management system in practices can be facilitated by the use of quality indicators. Quality indicators should yield positive assessment on a range of attributes such as clarity, feasibility, reliability, validity and transparency and in order to demonstrate sensitivity to change, benchmarking data are required so that health care providers can assess and compare their own quality of care with others [23,25]. Moreover, for assessment to lead to improvement it must be part of an ongoing process such as the “plan-do-study-act” (PDSA) cycle [26]. A continuous quality improvement is an essential part of quality management programs for health care services, which includes general medical practices and dental care practices in the primary sector [18].

The evaluation of quality of care requires a mixture of objective and subjective measures [27]. The European Practice Assessment consists of a set of objective and subjective quality indicators, which evaluate the structure and process of care from the perspective of practice owners, staff, patients and trained external facilitators [17]. For general medical practices, the effectiveness of the European Practice Assessment in showing higher scores at repeat assessment has already been shown [18,20]. These results regarding general medical practice are comparable to our results regarding the improvement for each domain at dental care practices [18,20]. The improvement of dimensions and domains in dental care practices follows a similar trend to that of the improvement in general medical practices.

There are different quality improvement activities being initiated in oral health services worldwide ranging from measurements of the process of technical restoration procedures to examination of long term health outcomes for the population [9]. Within this range, one important component is the measurement of dental care practice operations including structure, process and outcomes [9]. However, a systematic and organized agenda for quality improvement in dentistry is still in its fledgling stage [7]. The results of this study suggest that the European Practice Assessment provides a much needed mechanism for assessing quality in dental care practices and improving quality and safety [28].

Overall, reliable evidence regarding effectiveness of quality management programs in any field of healthcare is limited [29,30]. Studies on the effectiveness of quality management programs for dental care practices can make an important contribution to the evidence base related to quality in oral health services and also to improving patient outcomes. This is important if oral health services are to stay on par with other health services in terms of quality management [7,31]. Therefore, raising awareness regarding the development and continuous measurement of quality in dental care practices is important for dentists and oral health services policy makers. One opportunity (enabler) would be the introduction of performance-based reimbursement to incentivize good quality of care. The potential role of performance-based reimbursement for dentistry is currently under discussion in the United Kingdom and it is being piloted [3,32]. However, performance-based reimbursement is also associated with unintended consequences [33,34]. For example, the introduction of performance-based reimbursement in general medical care practices has shown short-term gains, but the evidence for its effectiveness long-term is not compelling [35,36]. Therefore, it should be implemented with caution in dental care settings [2,37] and its implementation should only be considered within the context of a system wide quality improvement strategy [9].

Limitations

Our study has the following limitations. The sample of participating dental care practices was small and may not have been generally representative of dental care practices in Germany. However, all practices that had used the European Practice Assessment twice were included. The allocation of practices to an intervention or a comparison group was not randomized and a baseline measurement in the comparison group was lacking. Moreover, the study design has a weakness in that the pre-post measurement was possible with the intervention group, but with only a single set of observations at a second point in time. Although our results showed improvements in the intervention group, this may reflect a selection effect of dental practices volunteering for the first round of the European Practice Assessment. Therefore, the results of the study have to be interpreted carefully and need to be confirmed in further studies. In addition, although it is known, that a multifaceted quality management program motivates practices to change [38], there is no reliable evidence from this study about the impact on clinical outcomes because the data presented concentrates on structure and process of care. Because this was the first study evaluating effectiveness of a quality management program in primary dental care settings in Germany, we have no experience on which to base our assessment as to how clinically relevant our results are. At this time, we have no reference standards. This study provides preliminary results as basis for further studies. The study design was explorative. Therefore, no correction for multiple tests was needed. The observed effects should be examined in further study with a larger sample.

Conclusions

In summary, implementation of quality management in dental care practices requires a paradigm shift: there cannot longer be a singular focus on technical aspects, but also it is necessary to integrate organizational aspects of service delivery and employ a team approach. Based on results from this study, this has the potential to result in better organizational quality in dental care practices. The European Practice Assessment for dental care practices provides such a quality management program, as it focuses on the improvement of structural and organizational aspects to promote high quality of care.

Competing interests

BB and MB are employed by the AQUA-Institute which disseminates the European Practice Assessment in Germany. JS is its director and stockholder. Other authors: No conflict of interest declared.

Authors’ contributions

KG, SC and JS initiated and designed the study. MB and BB coordinated the study. KG carried out data analysis and wrote the manuscript. All authors (KG, SC, MB, BB, MW and JS) read earlier versions of the manuscript, provided critical comments and approved the final manuscript.

Acknowledgements

The authors thank the primary sector dental care practices and outreach facilitators who participated in the study. The authors also thank the European Practice Assessment management team at the AQUA-Institute and Sarah Berger for reviewing English language expression in this paper.

References

  1. World Health Organization (WHO): Quality and accreditation in health services. A global review. Geneva (Switzerland): The Organization; 2003. OpenURL

  2. The Institute of Medicine (IOH): Crossing the quality chasm: a new health system for the 21st century. Washington D.C: National Academies Press; 2001. OpenURL

  3. Mills I, Batchelor P: Quality indicators: the rationale behind their use in NHS dentistry.

    Br Dent J 2011, 211:11-15. PubMed Abstract | Publisher Full Text OpenURL

  4. Donabedian A: The quality of care. How can it be assessed?

    JAMA 1988, 260:1743-1748. PubMed Abstract | Publisher Full Text OpenURL

  5. Campbell SM, Roland MO, Buetow S: Defining quality of care.

    Soc Sci Med 2000, 51:1611-1625. PubMed Abstract | Publisher Full Text OpenURL

  6. Campbell SM, Tickle M: What is quality primary dental care?

    Br Dent J 2013, 215:135-139. PubMed Abstract | Publisher Full Text OpenURL

  7. Bader JD: Challenges in quality assessment of dental care.

    J Am Dent Assoc 2009, 140:1456-1464. PubMed Abstract | Publisher Full Text OpenURL

  8. Greenfield D, Braithwaite J: Health sector accreditation research: a systematic review.

    Int J Qual Health Care 2008, 20:172-183. PubMed Abstract | Publisher Full Text OpenURL

  9. Glassman P: Oral health quality improvement in the era of accountability. Michigan: W.K. Kellogg Foundation; 2011. OpenURL

  10. Kenny DJ, Conway RM, Johnston DH: The development of IS09002 quality management standards for Canadian dental practices.

    J Can Dent Assoc 1999, 65:105-108. PubMed Abstract OpenURL

  11. NHS Scotland: An Action Plan for Improving Oral Health and Modernising NHS Dental Services in Scotland. Scottish Executive Edinburgh; 2005.

    http://www.scotland.gov.uk/Resource/Doc/37428/0012526.pdf webcite. [last accessed April 24, 2014]

    OpenURL

  12. Crall JJ, Spritzer KL, Hays RD: Development and implementation of a dental office assessment program.

    J Am Coll Dent 2012, 79:33-41. OpenURL

  13. Deming EW: Massachusetts Institute of Technology, Centre for Advanced Engineering Study. Cambridge (MA): Out of Crisis; 1986. OpenURL

  14. Federal Joint Committee (GB-A):

    Quality management guideline for dental care. 2006.

    http://www.eazf.de/doc/Dokumente/QM-Richtlinie_GBA.pdf webcite. [last accessed April 24, 2014]

    OpenURL

  15. Bergmann Krauss B, Boehme P: Qualitätsmanagement-Systeme für die Zahnarztpraxis.

    IDZ-Inform 2005, 5:4-38.

    in German

    OpenURL

  16. Mettes TG, van der Sanden WJ, Bronkhorst E, Grol RP, Wensing M, Plasschaert AJ: Impact of guideline implementation on patient care: randomized trial.

    J Dent Res 2009, 89:71-76. OpenURL

  17. Engels Y, Dautzenberg M, Campbell S, Broge B, Boffin N, Marshall M, Elwyn G, Vodopivec-Jamsek V, Gerlach FM, Samuelson M, Grol R: Testing a European set of indicators for the evaluation of the management of primary care practices.

    Fam Pract 2006, 23:137-147. PubMed Abstract | Publisher Full Text OpenURL

  18. Szecsenyi J, Campbell S, Broge B, Laux G, Willms S, Wensing M, Goetz K: Effectiveness of a quality-improvement program in improving management of primary care practices.

    CMAJ 2011, 183:E1326-E1333. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  19. EPA Website:

    European Practice Assessment.

    http://www.epa-qm.de/epa/front_content.php?idart=218 webcite. [last accessed April 24, 2014]

    OpenURL

  20. Götz K, Szecsenyi J, Broge B, Willms S: [Welche Wirkung hat Qualitätsmanagement in Arztpraxen? Ergebnisse aus Entwicklung und Evaluation des Europäischen Praxisassessments (EPA).]. Göttingen: AQUA-Verlag; 2011.

    in German

    OpenURL

  21. AQUA-Website:

    Institute for Applied Quality Improvement and Research in Health Care.

    http://www.aqua-institut.de webcite. [last accessed April 24, 2014]

    OpenURL

  22. Goderis G, Borgermans L, Heyrman J, Van Den Broeke C, Carbonez A, Mathieu C, Verbeke G, Grol R: Monitoring modifiable cardiovascular risk in type 2 diabetes care in general practice. The use of an aggregated z-score.

    Med Care 2010, 48:589-595. PubMed Abstract | Publisher Full Text OpenURL

  23. Campbell SM, Braspenning J, Hutchinson A, Marshall M: Research methods used in developing and applying quality indicators in primary care.

    Br Med J 2003, 326:816-819. Publisher Full Text OpenURL

  24. Lawrence M, Olesen F: Indicators of quality health care.

    Eur J Gen Pract 1997, 3:103-108. Publisher Full Text OpenURL

  25. Braspenning J, Hermans R, Calsbeek H, Westert G, Campbell SM, Grol R: Quality and safety of care: the role of indicators. In Improving patient care: the implementation of change in health care practice. 2nd edition. Edited by Grol R, Wensing M, Eccles M, Davis D. Oxford: UK John Wiley & Sons, Ltd; 2013. OpenURL

  26. Boaden R, Harvey G, Moxham C, Proudlove N: Quality improvement: theory and practice in healthcare. Coventry: NHS Institute for Innovation and Improvement; 2008. OpenURL

  27. Campbell SM, Eriksson T: Multiple strategies for quality improvement and patient safety – money alone is not the answer, nor is trust. Conclusions of the 6th April 2011.

    Euro J Gen Pract 2011, 17:238-240. Publisher Full Text OpenURL

  28. Lester HE, Eriksson T, Dijkstra R, Martinson K, Tomasik T, Sparrow N: Practice accreditation: the European perspective.

    Br J Gen Pract 2012, 62:273-274. OpenURL

  29. Minkman M, Ahaus K, Huijsman R: Performance improvement based on integrated quality management models: what evidence do we have? A systematic literature review.

    Int J Qual Health Care 2007, 19:90-104. PubMed Abstract | Publisher Full Text OpenURL

  30. Schouten LMT, Hulscher MEJL, Everdingen JJE V, Huijsman R, Grol R: Evidence for the impact of quality improvement collaboratives: systematic review.

    BMJ 2008, 336:1491-1494. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  31. Niederman R, Leitch J: “Know what” and “Know how”: knowledge creation in clinical practice.

    J Dent Res 2006, 85:296-297. PubMed Abstract | Publisher Full Text OpenURL

  32. Voinea-Griffin A, Fellows JL, Rindal DB, Barasch A, Gilbert GH, Safford MM: Pay for performance: will dentistry follow?

    BMC Oral Health 2010, 10:9. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  33. Campbell SM, Kontopantelis E, Hannon KL, Burke M, Barber A, Lester HE: Framework and indicator testing protocol for developing and piloting quality indicators for the UK Quality and Outcomes Framework.

    BMC Fam Pract 2011, 12:85. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  34. Tickle M, Campbell SM: How do we measure quality in primary dental care?

    Br Dent J 2013, 215:183-187. PubMed Abstract | Publisher Full Text OpenURL

  35. Petersen LA, Woodard LD, Urech T, Daw C, Sookanan S: Does Pay-for-Performance improve the quality of health care?

    Ann Intern Med 2006, 145:265-272. PubMed Abstract | Publisher Full Text OpenURL

  36. Scott A, Sivey P, Ait Ouakrim D, Willenberg L, Naccarella L, Furler J, Young D: The effect of financial incentives on the quality of health care provided by primary care physicians.

    Cochrane Database Syst Rev 2011, (9):CD008451. OpenURL

  37. Campbell SM, Tickle M: How do we improve quality in primary dental care?

    Br Dent J 2013, 215:239-243. PubMed Abstract | Publisher Full Text OpenURL

  38. Grimshaw J, McAuley LM, Bero LA, Grilli R, Oxman A, Ramsay C, Vale L, Zwarenstein M: Systematic reviews of the effectiveness of quality improvement strategies and programmes.

    Qual Saf Health Care 2003, 12:298-303. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6831/14/41/prepub