Open Access Highly Accessed Research article

The role of deliberate practice in the acquisition of clinical skills

Robbert J Duvivier1*, Jan van Dalen1, Arno M Muijtjens2, Véronique RMP Moulaert3, Cees PM van der Vleuten2 and Albert JJA Scherpbier4

Author affiliations

1 Skillslab, Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands

2 Department of Educational Development and Research, Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands

3 Rehabilitation Foundation Limburg, Adelante Rehabiliation Centre, Hoensbroek, the Netherlands

4 Institute for Medical Education, Faculty of Health Medicine and Life Sciences, Maastricht University, Maastricht, the Netherlands

For all author emails, please log on.

Citation and License

BMC Medical Education 2011, 11:101  doi:10.1186/1472-6920-11-101


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6920/11/101


Received:8 May 2011
Accepted:6 December 2011
Published:6 December 2011

© 2011 Duvivier et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The role of deliberate practice in medical students' development from novice to expert was examined for preclinical skill training.

Methods

Students in years 1-3 completed 34 Likert type items, adapted from a questionnaire about the use of deliberate practice in cognitive learning. Exploratory factor analysis and reliability analysis were used to validate the questionnaire. Analysis of variance examined differences between years and regression analysis the relationship between deliberate practice and skill test results.

Results

875 students participated (90%). Factor analysis yielded four factors: planning, concentration/dedication, repetition/revision, study style/self reflection. Student scores on 'Planning' increased over time, score on sub-scale 'repetition/revision' decreased. Student results on the clinical skill test correlated positively with scores on subscales 'planning' and 'concentration/dedication' in years 1 and 3, and with scores on subscale 'repetition/revision' in year 1.

Conclusions

The positive effects on test results suggest that the role of deliberate practice in medical education merits further study. The cross-sectional design is a limitation, the large representative sample a strength of the study. The vanishing effect of repetition/revision may be attributable to inadequate feedback. Deliberate practice advocates sustained practice to address weaknesses, identified by (self-)assessment and stimulated by feedback. Further studies should use a longitudinal prospective design and extend the scope to expertise development during residency and beyond.

Background

The ultimate goal of medical education is to prepare students to become clinically competent doctors. During their years in medical school and the hospital undergraduate students begin the gradual transition from novice to expert, a process that is by no means an easy one [1-3]. Research in other domains has shown that it requires hard work to become very good in a particular area of expertise. Chase and Simon, for example, argued that it takes ten years of intensive practice to reach expert level in a particular field [4].

Ericsson made a distinction between work, play and training [5]. Work entails activities that primarily lead to immediate monetary and/or social rewards. Play involves activities that have no explicit goal and are inherently enjoyable. For training Ericsson introduced the concept of "deliberate practice", characterising training as a highly structured activity explicitly directed at improvement of performance in a particular domain [6]. Specific tasks are invented to overcome weaknesses and performance is carefully monitored to provide cues for ways to achieve further improvement. Deliberate practice is not mere mindless repetition of a certain task, but a focused approach to training aimed at reaching a well-defined goal. Practical implementation of the theoretical construct is based on several design principles [7]:

(a) repetitive performance of intended cognitive or psychomotor skills

(b) rigorous skills assessment

(c) specific informative feedback

(d) better skills performance

Research has shown that training activities guided by these principles help the acquisition and maintenance of expert performance in sports and music [8], but also in typing [9], economics [10] and chess [11]. Ericsson discussed the use of deliberate practice in medicine, but his theoretical examples were limited to clinical performance of qualified doctors [12,13].

Exactly how the use of the principles of deliberate practice can contribute to the development from novice to expert during undergraduate medical training and beyond remains unclear, although there is evidence that deliberate practice leads to effective performance with high-fidelity medical simulators [14,15].

Moulaert et al. explored whether the theoretical principles of Ericsson could be identified in the study habits of undergraduate medical students [16]. Aspects of deliberate practice were positively correlated with results on knowledge and skills tests.

Moulaert's paper addressed study habits of students in the cognitive component of the curriculum. Although recent developments in undergraduate medical curricula have also emphasised the acquisition of clinical skills, little is known about students' learning strategies with regard to these skills. Physical examination skills are predominantly psychomotor by nature. Since research in other domains has shown that the principles of Ericsson's theory are applicable to training in psychomotor skills such as tennis and golf [17], it seems worthwhile to investigate whether deliberate practice can be of benefit in the area of clinical skills learning too.

Based on Ericsson's theory of deliberate practice and the available evidence of its effectiveness in other domains, we hypothesised that training activities grounded in deliberate practice can be effective in undergraduate clinical skill training.

We further hypothesised that more senior students exhibit more aspects of deliberate practice, due to effective learning habits acquired as they progressed through the curriculum. Based on these hypotheses, we assumed that engaging in deliberate practice would positively affect students' performance in clinical skills. We have operationalized this by comparing scores on formal assessments of clinical skills.

We addressed the following research questions to examine our hypotheses:

1) Which aspects of deliberate practice can be identified in the behaviour of medical students when practising clinical skills?

2) What development can be seen in the use of deliberate practice across different years of study?

3) To what extent does deliberate practice have predictive value for results on Objective Structured Clinical Examinations (OSCE), and which aspects contribute the most?

In order to answer these questions we conducted a quantitative cross-sectional retrospective study, using a questionnaire to investigate students' engagement in different aspects of deliberate practice.

Methods

Setting

The study was carried out at the Faculty of Health, Medicine and Life Sciences, Maastricht University, the Netherlands in 2008. Skill training receives a great deal of attention throughout the six-year medical curriculum. In the first three years the curriculum is organised in thematic blocks of six to ten weeks each focused on a specific group of patient problems or conditions. The main educational format is problem-based small group sessions, with additional lectures, skill training and laboratory sessions. The subjects of skill training are aligned with the themes of the problem-based sessions. Years 4-6 consist of clinical placements, and skill training in these years focuses on revision and remediation. All skill training takes place at the Skillslab, a specialised educational facility. A skills training session for undergraduates (year 1-3) consists of a four-stage process involving tutor demonstration, followed by explanation, practice under supervision with feedback and corrective critique. Most sessions start with a discussion of students' preparatory reading. Next, the teacher demonstrates the skill using a student, a (simulated) patient or a model, after which students practise on models or one another. Finally, the teacher summarises the session and students can ask questions [18].

Students take a skills test at the end of years 1, 2, 3 and 5. This is a performance-based test consisting of multiple stations where students are presented with different clinical scenarios (OSCE). Each student undertakes the same series of tasks, and performance is graded by trained physicians using station-specific standardised checklists. Each checklist contains key items that the student must perform to gain a satisfactory rating. The overall OSCE result is based on the total number of correctly performed key checklist items.

Questionnaire

Moulaert's questionnaire was based on the literature on expertise development and reviewed by eight medical students and eight experts in cognitive psychology and educational science. We adapted this questionnaire to fit skills training, and justified the content validity to guarantee transferability to this domain. It was piloted among four medical students, reviewed by four medical education experts and modified in response to their comments. The final questionnaire took approximately ten minutes to complete and consisted of 34 items requiring a response on a five-point Likert scale (1 = never, 5 = always) (additional file 1). Students received oral and written information about the purpose of the questionnaire. All students gave informed consent.

Additional file 1. Questionnaire. Questionnaire used in study, based on previous work by Moulaert et al and adapted for skills.

Format: PDF Size: 83KB Download file

This file can be viewed with: Adobe Acrobat ReaderOpen Data

Our aim was to include all undergraduate students in the first three years. Immediately after they attended the skill test of curriculum year 2008-2009, all students in years 1-3 were requested to complete the questionnaire. Skills tests are scheduled at the end of the academic year. Participation was voluntary and students received no financial reward for participation. Students received their individual test result 6 weeks after completion of the test, as is standard procedure at Maastricht University. For each participating student we linked OSCE results and questionnaire data. In order to guarantee anonymity, an independent data analyst changed all student ID numbers into random numbers.

Data analysis

All analyses were performed using SPSS Software (version 15.0 for MS Windows). A p-value of 0.05 was the threshold value for significance.

Research Question 1

We performed exploratory factor analysis and reliability analysis to validate the adapted questionnaire. Extracted factors constituted subscales of aspects of deliberate practice.

Research Question 2

In order to answer the second research question, we conducted analysis of variance (ANOVA) to investigate between-year differences in aspects of deliberate practice. After calculating mean item scores for each factor we used Bonferroni correction to safeguard against type I errors.

We calculated effect sizes (ES) for all comparisons expressed as Cohen's d. Hojat and Xi define the practical importance of effects sizes as ES ≈ .20: small effect size of negligible practical importance; ES ≈ .50: medium effect size of moderate practical importance; and ES ≈ .80: large effect size of crucial practical importance [19].

Research Question 3

In order to answer the third research question we used regression analysis to investigate which aspects of deliberate practice were associated with OSCE results.

We calculated mean z-scores for the OSCE results in each of the three years.

We calculated the mean scores for the subscales resulting from the factor analysis and used regression analysis to estimate the correlation with OSCE scores. OSCE scores were available for 93% of the students. We were not able to retrieve OSCE results for all students due to several reasons; wrong or invalid student ID used on the questionnaire, duplicate student ID used, withdrawn OSCE results. To prevent possible confounding by age and gender these variables were included in the regression analysis as additional independent variables.

Ethical Approval

No ethical approval was required for this study according to Dutch law. However, the Department of Educational Development and Research at Maastricht University who reviewed and approved the project proposal carefully considered the ethical issues concerned. There was no potential harm to participants, we guaranteed anonymity of participants and we obtained informed consent of all participants before conducting our study.

Results

Descriptive statistics

We aimed to include all students in Years 1, 2 and 3, but were unable to reach some of them. Of the total of 972 students invited to participate, 875 completed the questionnaire giving an overall response rate of 90%. Respondents were distributed evenly over the three years, with 298 (92%), 292 (87%) and 285 (90%) students in Years 1, 2 and 3, respectively. Twenty-four students did not state their year of training and eight students did not fill in their student ID.

The mean age of the respondents was 20.1 years (SD = 1.24 years) and female students accounted for 68%. This is in concordance with the overall age and gender distribution in the student population of Maastricht medical school.

Research Question 1: Aspects of deliberate practice

We verified whether the consistency of the data justified the use of factor analysis. As the Kaiser-Meyer-Olkin measure of sampling adequacy (MSA) showed that partial correlations among variables were likely to be large (MSA = .86) and Bartlett's test of sphericity was significant (p < 0.001) indicating strong relationships among the variables, we concluded that it was appropriate to perform factor analysis. We conducted factor analysis based on principal component analysis and oblique rotation. We used three indicators to determine the number of factors: Cattell's scree plot, eigenvalues and whether the resulting item clusters represented theoretically meaningful aspects of deliberate practice. Table 1 shows the items in order of descending eigenvalues. The factor analysis yielded four factors:

Table 1. Items of the factors resulting from the factor analysis

1. planning (higher scores indicate a stronger tendency to organise work in a structured way);

2. concentration/dedication (higher scores indicate a shorter attention span);

3. repetition/revision (higher scores indicate a stronger tendency to practise);

4. study style/self reflection (higher scores indicate a stronger tendency to self-regulate learning).

Cronbach's alpha was calculated to investigate the internal consistency of the scales (table 1).

Research Question 2: Comparison between students in different years

The data analysis revealed statistically significant differences between the years for two subscales: planning and repetition/revision (table 2). Scores on the planning scale show an increase in planning behaviour and organisation of work over the years. The differences were of small to medium practical importance (Cohen's d).

Table 2. Comparisons between scores on the subscales in different years

Research Question 3: Relationships between aspects of deliberate practice and test results

Significant positive correlations of factor scores with test results were found for planning in Years 1 and 3 with small to medium effect sizes (table 3). Concentration/dedication scores decreased over the years indicating increased attention span. They showed a significant negative correlation with test results in Year 3 with a small to medium effect. Scores on repetition/revision were significantly positively correlated with test results in Year 1, with a medium effect.

Table 3. Relation between factors and test results (Standard Regression Coefficient (Beta))

Discussion

We explored the use of deliberate practice by medical students in learning clinical skills by identifying study habits related to deliberate practice, whether their use changed across different years of study and whether it had a positive effect on students' scores on formal assessments of clinical skills.

The use of deliberate practice in relation to clinical skills in the first three years of undergraduate medical education showed an increase in several aspects. More specifically, students showed progressively more planning behaviour and an increased tendency to structure their work. Furthermore, we found a positive relationship between OSCE performance and some aspects of deliberate practice: repetition/revision in Year 1, planning in Years 1 and 3 and attention span in Year 3.

Our results reveal a trend of increasing use of deliberate practice as students' progress through the curriculum. Students seem to acquire the ability to structure their study and practice activities, a finding that may be attributable to an increased focus on and awareness of desired study outcomes. Combined with increased concentration/dedication with respect to practising, this increased awareness of desired outcomes may contribute to the effectiveness of practice activities. Thus planning and focusing of attention seem to be aspects of deliberate practice that help students to gradually refine their performance.

Interpreting these combined results, it appears that students gradually learn how to make more efficient use of their time, energy and resources. In short, they seem to learn how to learn.

Research in other domains, mostly by Ericsson and colleagues, has shown that the main determinant of success in expert performers is not the amount of time spent practising, but the amount of time devoted to activities specifically targeted at aspects of performance that need improvement [20]. After initial mastery of basic skills, some types of practice, the proficient execution of routine tasks for example, are unlikely to lead to further improvement. In other words, repetition in itself is not enough. Progress depends on sustained efforts to purposefully enhance particular aspects of performance. This principle might explain why the positive association we found between repetition and test results was not sustained after Year 1. First year students who are just beginning to learn skills are likely to benefit from any practice effort, regardless of structure and organisation, whereas it seems plausible that a lack of focus on identified weaknesses would hamper learning after Year 1. This would suggest that efforts in Year 3 have to be focused and well planned in order to be effective.

As stated in the introduction, a key challenge for students is to acquire appropriate study habits to support their continuing improvement. While we found aspects of deliberate practice that promoted skill performance, the use of feedback has not been discussed so far. Although it did not emerge in this study as one of the important aspects of deliberate practice, it is crucial to the development of effective learning habits and therefore merits attention. Most of the study habits we have described so far are intrinsically self-directed and depend entirely on students' ability to shape their own learning. Feedback is a prerequisite for all identified aspects of deliberate practice and plays a facilitative role in the development of students' practice habits. It has been pointed out, however, that students are not always capable of recognising areas where further practising is needed [21,22], and may require guidance in identifying the next steps to be mastered. Ericsson has argued that coaches, trainers and teachers (whether in sport, music or academia) will always play an essential role in guiding the selection, sequencing and form of practice activities.

This brings us to the practical implications of this study. The tentative conclusion seems justified that educators can facilitate clinical skill development by equipping students with skills to use (aspects of) deliberate practice. Thus the teaching of clinical skills should incorporate the use of efficient strategies to practise them [23,24].

This type of learning is only possible with students' full cooperation. In other words, it requires students' active participation in their learning. Students need to adopt attitudes and strategies that are conducive to the planning of their learning and evaluation of their current and desired skill performance so as to enhance their habits of skill practice. This implies that course designers should accommodate this process by aiming for a learning environment that incorporates these characteristics.

The exact role of the teacher in feedback, and the preferred timing and method of feedback delivery remain to be determined. This is especially important in the clinical years when training moves to the clinical workplace where students are expected to be increasingly able to personally initiate and direct their efforts to acquire knowledge and skills rather than rely on teachers. The use of self-regulated/self-directed learning strategies is an important area for further research. The focus of this research might be on clarifying learning in individuals further along the novice-expert continuum, such as residents or senior physicians [25].

It must be noted here that the concept of deliberate practice (and to a lesser extent self-directed learning) relies on the assumption that learners are able to identify weaknesses in their own performance and knowledge and take measures to address these. These skills need to be acquired through observation and feedback, which can gradually be replaced by self-assessment. Initially, this can be seen to by frequent and adequate assessment. However, for students to become lifelong learners, they need to develop self-assessment skills along the way during their training. The literature on self-assessment has pointed out that medical students, residents and physicians have limited ability to accurately judge their own performance [26-28].

A recent study confirmed the importance of self-assessment for achieving competent performance. Mavis et al. showed that second-year students who were capable of realistic self-appraisal and had high self-efficacy were more likely to perform above average on OSCEs compared to low self-rated students [29]. Further research on optimal planning of the transition from teacher-assessment to self-assessment is recommended.

The cross-sectional design is a limitation of our study. Although the results afford information about groups of students at different stages in the curriculum, it gives no insight into the progress of individual students. The latter would require a longitudinal design. It should also be noted that the correlations we found do not warrant conclusions regarding causal relationships. In other words the findings provide some insights and signal trends, but these require further investigation using an experimental research design.

The strength of our study lies in the large and representative study sample. Our results could be strengthened by extending the scope of further studies to residents in different specialties and practising doctors to capture the next stages of the continuum from novice to expert. Further studies should also investigate whether deliberate practice remains an important factor in improving performance throughout a professional's career. If so, we should teach students to use this approach from an early phase in their education to maximise its beneficial effects. Finally, it would be interesting to examine whether the concept of deliberate practice in medicine can advance our understanding of expertise development in this domain.

Conclusions

This study investigated the role of deliberate practice in medical students' development from novice to expert for preclinical skill training. We used a questionnaire based on previous research, which was completed by 875 students in years 1-3 (90% of total student population). We examined differences between years and the relationship between deliberate practice and skill test results. Factor analysis yielded four factors from the questionnaire: planning, concentration/dedication, repetition/revision, study style/self reflection. Student scores on 'planning' increased over time, score on sub-scale 'repetition/revision' decreased. Student results on the clinical skill test correlated positively with scores on subscales 'planning' and 'concentration/dedication' in years 1 and 3, and with scores on subscale 'repetition/revision' in year 1.

The positive effects of deliberate practice (as measured in our questionnaire) on test results merit further study to clarify the usefulness of deliberate practice in clinical skills training, not only in undergraduate students but also in clinical years and during residency.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

RJD and JvD designed the study, with contributions from AMM, AJJAS and CPMvdV. RJD, VM and JvD developed the questionnaire. AM, AJJAS and CPMvdV reviewed the questionnaire and provided suggestions for improvement. RJD carried out data collection and RJD and AMM analysed the results. All authors discussed the results. RJD wrote the first draft of the paper; all authors provided comments and revisions and approved the final manuscript.

Acknowledgements

The authors thank the students of the Faculty of Health Medicine and Life Sciences, Maastricht University for their participation in this study. Thanks to Bert Zonneveld for granting his permission to distribute the questionnaire and to Desirée Bonckers and Margriet Brorens for their support during the distribution and data collection.

Thanks to Jacqueline Pisters and Paula Rinkens from MEMIC for processing the questionnaires and data transcription. Thanks to Mereke Gorsira for editing the final version of the manuscript and her useful suggestions and corrections. Special thanks to Ron Hoogenboom for statistical support and valuable advice.

Robbert Duvivier is supported by a Kootstra Talent Fellowship from the Faculty of Health Medicine and Life Sciences, Maastricht University

References

  1. Schmidt H, Boshuizen H: On acquiring expertise in medicine.

    Educational Psychology Review 1993, 5:250-221. OpenURL

  2. Prince K, van de Wiel M, Scherpbier A, can der Vleuten CPM, Boshuizen HPA: A qualitative analysis of the transition from theory to practice in undergraduate training in a PBL-medical school.

    Advances in Health Sciences Education 2000, 5:105-116. PubMed Abstract | Publisher Full Text OpenURL

  3. Prince K, Van de Wiel M, Van der Vleuten C, Boshuizen H, Scherpbier A: Junior doctors' opinions about the transition from medical school to clinical practice: a change of environment.

    Educ Health (Abingdon) 2004, 17:323-331. Publisher Full Text OpenURL

  4. Chase WG, Simon HA: Perception in chess.

    Cognitive psychology 1973, 4:55-81. Publisher Full Text OpenURL

  5. Ericsson AK, Ed: The road to excellence. Mahwah NJ: Lawrence Eribaum Associates; 1996.

  6. Ericsson KA, Krampe RT, Tesch-Römer C: The role of deliberate practice in the acquisition of expert performance.

    Psychological Review 1993, 100:363-363. OpenURL

  7. Issenberg SB, McGaghie WC, Gordon DL, Symes S, Petrusa ER, Hart IR, Harden RM: Effectiveness of a cardiology review course for internal medicine residents using simulation technology and deliberate practice.

    Teach Learn Med 2002, 14:223-228. PubMed Abstract | Publisher Full Text OpenURL

  8. Krampe RT, Ericsson KA: Maintaining excellence: deliberate practice and elite performance in young and older pianists.

    J Exp Psychol Gen 1996, 125:331-359. PubMed Abstract | Publisher Full Text OpenURL

  9. Keith N, Ericsson KA: A deliberate practice account of typing proficiency in everyday typists.

    J Exp Psychol Appl 2007, 13:135-145. PubMed Abstract | Publisher Full Text OpenURL

  10. Unger JM, Keith N, Hilling C, Gielnik MM, M F: Deliberate practice among South African small business owners: Relationships with education, cognitive ability, knowledge, and success.

    Jour Occup Org Psych 2009, 82:21-44. Publisher Full Text OpenURL

  11. De Bruin ABH, Smits N, Rikers RMJP, HG S: Deliberate practice predicts performance over time in adolescent chess players and drop-outs: A linear mixed models analysis.

    Brit Jour Psych 2008, 99:473-497. Publisher Full Text OpenURL

  12. Ericsson KA: Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains.

    Academic Medicine 2004, 79:S70. PubMed Abstract | Publisher Full Text OpenURL

  13. Ericsson KA: An expert-performance perspective of research on medical expertise: the study of clinical performance.

    Medical Education 2007, 41:1124. PubMed Abstract | Publisher Full Text OpenURL

  14. Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER, Waugh RA, Brown DD, Safford RR, Gessner IH, et al.: Simulation technology for health care professional skills training and assessment.

    JAMA 1999, 282:861-866. PubMed Abstract | Publisher Full Text OpenURL

  15. Issenberg SB, McGaghie WC, Petrusa ER, Lee Gordon D, Scalese RJ: Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review.

    Med Teach 2005, 27:10-28. PubMed Abstract | Publisher Full Text OpenURL

  16. Moulaert V, Verwijnen MGM, Rikers R, Scherpbier AJ: The effects of deliberate practice in undergraduate medical education.

    Medical education 2004, 38:1044-1052. PubMed Abstract | Publisher Full Text OpenURL

  17. Starkes J, Ericsson AK, Eds: Expert performance in sports: advances in research on sport expertise. Champaign IL: Human Kinetics; 2003.

  18. Duvivier R, Dalen van J, Bartholomeus P, Scherpbier A, Verwijnen G: Skills training. In Lessons from problem-based learning. Edited by van Berkel H, Scherpbier A, Hillen H, Van der vleuten C: Oxford University Press; 2010. OpenURL

  19. Hojat M, Xu G: A Visitor's Guide to Effect Sizes-Statistical Significance Versus Practical (Clinical) Importance of Research Findings.

    Advances in health sciences education 2004, 9:241-249. PubMed Abstract | Publisher Full Text OpenURL

  20. Ericsson AK, Charness N, Feltovich P, Hoffman RR, Eds: Cambridge handbook on expertise and expert performance. Cambridge UK: Cambridge University Press; 2006.

  21. Gordon MJ: A review of the validity and accuracy of self-assessments in health professions training.

    Acad Med 1991, 66:762-769. PubMed Abstract | Publisher Full Text OpenURL

  22. Falchikov N, Boud D: Student self-assessment in higher education: A meta-analysis.

    Review of Educational Research 1989, 59:395-430. OpenURL

  23. Wayne DB, Butter J, Siddall VJ, Fudala MJ, Linquist LA, Feinglass J, Wade LD, McGaghie WC: Simulation-based training of internal medicine residents in advanced cardiac life support protocols: a randomized trial.

    Teach Learn Med 2005, 17:210-216. PubMed Abstract | Publisher Full Text OpenURL

  24. Rolfe IE, Sanson-Fisher RW: Translating learning principles into practice: a new strategy for learning clinical skills.

    Medical Education 2002, 36:345. PubMed Abstract | Publisher Full Text OpenURL

  25. Eva KW, Regehr G: Self-assessment in the health professions: a reformulation and research agenda.

    Academic Medicine 2005, 80:S46. PubMed Abstract | Publisher Full Text OpenURL

  26. Eva KW, Cunnington JPW, Reiter HI, Keane DR, Norman GR: How can I know what I don't know? Poor self assessment in a well-defined domain.

    Advances in Health Sciences Education 2004, 9:211-224. PubMed Abstract | Publisher Full Text OpenURL

  27. Hodges B, Regehr G, Martin D: Difficulties in recognizing one's own incompetence: novice physicians who are unskilled and unaware of it.

    Academic Medicine 2001, 76:S87. PubMed Abstract | Publisher Full Text OpenURL

  28. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L: Accuracy of physician self-assessment compared with observed measures of competence: a systematic review.

    JAMA 2006, 296:1094. PubMed Abstract | Publisher Full Text OpenURL

  29. Mavis B: Self-efficacy and OSCE performance among second year medical students.

    Advances in Health Sciences Education 2001, 6:93-102. PubMed Abstract | Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6920/11/101/prepub