We studied graduates of German medical schools who graduated between 1996 and 2002 using a standardized survey carried out by the independent Centre for Higher Education Development (CHE, Germany). All 37 German medical schools were contacted and questionnaires were sent out to all graduates. Importantly, there was only one medical school between 1996 and 2002 offering a complete PBL-based curriculum in Germany 1718. This medical school at the private University Witten/Herdecke was founded in 1983 with the major goal to provide an alternative in education with self-guided, patient-oriented medical training throughout medical school 18.

The survey was conducted between 2004 and 2005; details of the study procedure are provided elsewhere 19. In brief, the CHE contacted medical graduates by mail. A link to the online questionnaire was given in individually addressed covering letters by the CHE. Graduates were asked to carry out their self-assessment online. An anonymized ID-code for each graduate prevented multiple survey responses. Contact addresses were obtained from the database of the State Medical Chambers in Germany and from alumni associations wherever applicable. According to an official request to the institutional ethics committee of the University Witten/Herdecke, approval by an ethics commission was not necessary for conducting the study, as there was no intervention. All participants in the study participated voluntarily and provided informed consent.

As an independent study core, the CHE developed the standardized questionnaire which consisted of 35 items 19. The questionnaire contained baseline demographic data and professional status. Furthermore, the questionnaire assessed nine competencies as required by current employment and whether these were adequately represented in the respective curriculum (as translated from the German original: "Please evaluate to which extent the following competencies are (A) required in your day-to-day job and (B) were taught in medical school"). The design of the text questions was based on reports describing the general skills necessary for professional practice in medicine 132021. The following nine competencies were included to be evaluated by the graduates: "Independent learning/working", "Practical medical skills", "Psycho-social competence", "Teamwork", "Problem-solving skills", "Medical knowledge", "Interdisciplinary thinking", "Business competence", and "Research competence". These outcome variables were collected using a 6-point Likert scale on which a score of six (6) reflects the positive end ["very intensively required/taught"], and a score of one (1) the negative end ["not at all required/taught"] of the scale. A comparison of the final national board examination grades was performed based on the publications of the German Institute for Medical and Pharmaceutical Exam Questions 22.

Baseline demographics and clinical characteristics are presented as mean ± standard deviation (SD) for continuous variables and percentage [frequency] for categorical variables. Outcomes of interest were the self-assessed competencies as reported by graduates of German medical schools. Graduates were stratified by whether their medical curriculum was PBL-based (University Witten/Herdecke) or conventional (all other German medical schools). To ensure generalizabilty, universities were excluded if less than 50 of their graduates responded.

Differences in baseline demographics between the two stratified cohorts were determined using chi-square and two-tailed t-tests for categorical and continuous variables, respectively. Fisher's exact test was used if subgroups included less than ten subjects.

To determine whether medical school education sufficiently covers the job requirements of physicians, we calculated the mean difference (Δ) between the ratings of competencies as required at the day-to-day work and as taught in medical school by subtracting those from each other. This comparison demonstrates whether an education is perceived to cover job requirements (Figure 1). It also permitted detection of deficits (mean difference [Δ] was negative) and surplus (mean difference [Δ] was positive) of teaching in the different curricula for each competence. To evaluate whether those so calculated deficit or surplus for a competence was significant, we applied a paired two-tailed t-test (Figure 1).

To express effect size, Hedges' g was computed for all significant differences between graduates from the PBL-based curriculum and from the conventional curricula regarding the research questions "Required competencies at the day-to-day work", and "Competencies as taught in medical school" 23. A Hedges' g < 0.5 was defined as a "small", 0.5 to 0.8 as a "moderate" and >0.8 as a "large" effect size 24.

All analyses were performed using SPSS (Version 13, SPSS Inc., Chicago, IL, USA) and SAS (Version 9.2, SAS Institute Inc., Cary, NC, USA). A p-value < 0.05 was considered to indicate statistical significance.

Overall, graduates of both curricula rated the job requirements similarly. The most required competencies at their day-to-day work were "Independent learning/working", "Practical medical skills", and "Psycho-social competence" (5.19 ± 0.9, 5.17 ± 1.2, and 4.89 ± 1.3, respectively; Table 2). In contrast, differences between the two groups were found with respect to "Research competence" and "Medical knowledge". Graduates of the PBL-based curriculum rated the "Research competence" as a significantly higher job requirement as compared to graduates of conventional curricula (3.78 ± 1.6 vs. 2.92 ± 1.5, p < 0.001; respectively). Conversely, graduates of the PBL-based curriculum rated "Medical knowledge" as less required compared to graduates of conventional curricula (4.43 ± 1.2 vs. 4.78 ± 1.0, p = 0.02; respectively). However, both differences demonstrated only a small to moderate effect size, while Hedges' g for differences in "research competence", and "medical knowledge" were 0.6 and 0.4, respectively (Table 2).

Assessment of the competencies as taught in medical school differed significantly between the two groups (Table 3). As such, graduates of the PBL-based curriculum deemed that their curriculum facilitated preparation in "Independent learning", "Practical medical skills", "Psycho-social competence", "Teamwork", "Problem-solving skills", and "Interdisciplinary thinking", whereas graduates of the conventional curriculum rated their teaching as consistently inferior (Table 3, all p < 0.001). The greatest effect size for the differences in ratings between the two groups were observed for "Psycho social competence" (Hedges' g: 2.4) and "Practical medical skills" (Hedges' g: 2.4), followed by "Interdisciplinary thinking" (Hedges' g: 1.9). In contrast, PBL graduates rated "Medical knowledge" and "Research competence" inferior compared to graduates from conventional curricula (4.38 ± 1.0 vs. 4.69 ± 0.9, p = 0.002, and 2.57 ± 1.3 vs. 3.10 ± 1.3, p < 0.001 respectively). However, the difference for these two competencies demonstrated only a small effect size (Hedges' g: 0.3 and 0.4 for "Medical knowledge" and "Research competence", respectively).

"Business competence" was rated to be taught worse than any other competence, with similar results in both groups (4.86 ± 1 vs. 5.65 ± 1; Table 3); however, ratings of graduates of the PBL-based curriculum were significantly superior as compared to graduates of the conventional curriculum (p < 0.001).

The differences between competencies required at the day-to-day work and competencies taught in medical school were illustrated for both groups in figure 1. The ratings given by graduates of the PBL-based curriculum indicated a significant deficit (negative values) in the teaching of "Business competence" (Δ - 1.44, p < 0.001) and "Research competence" (Δ -1.23, p < 0.001). In comparison, the ratings by graduates of the conventional curriculum indicated deficits in all competencies except for "Research competence" (Δ +0.19, p < 0.001). In contrast, significant surplus of medical school education (positive values) as compared to requirements at the day-to-day work were identified for the PBL-based curriculum in "Interdisciplinary thinking" (Δ +0.88, p < 0.001), "Independent learning/working" (Δ +0.57, p < 0.001), "Psycho social competence" (Δ +0.56, p = 0.003), "Teamwork" (Δ +0.39, p = 0.03), and "Problem-solving skills" (Δ +0.36, p = 0.02).

Overall, both groups of graduates assess the value of core competencies as demanded by their day-to-day work similarly. Competencies such as "Practical medical skills" and "Psycho social competence" are related to high clinical value and mirror classic PBL strengths 2. Interestingly, medical doctors overall rate "Independent learning/working" as a non-medical competence to be more required than other medical competencies. One could argue that these findings are in line with the theory introduced by Sir Osler in 1899 emphasizing the need to prepare for lifelong learning 125. Although job requirements are similar for both groups, graduates from the PBL-based curriculum evaluate "Medical knowledge" as less required and "Research competence" as more highly required. While it is possible that PBL graduates are more used to compensating medical knowledge gaps by independently acquiring new information, this ability may be less developed/trained in graduates from a conventional curriculum. This hypothesis is in line with a recent study by Prince et al., which indicates that a lower rating of "possession of profession-relevant knowledge" is associated with PBL-based curricula 8. However, those evaluated differences showed only a small to moderate effect size.

The rating "How well were these competencies taught in medical school?" differs significantly between the two groups of graduates. The PBL-curriculum was rated superior in seven out of nine competencies, all with a large effect size. "Independent learning/working", "Psycho-social competence" and "Practical medical skills" are rated as the top three competencies that were acquired during PBL-based medical school education. These findings are similar to the current study of Koh et al., where the author demonstrates strong evidence that graduates from a PBL-based curriculum benefit especially in social and cognitive competencies 15. Our results confirm the strength of PBL-based curriculum in teaching social and cognitive competencies. Furthermore, our results show that the reformed curriculum based on PBL not only affects the typical PBL-related competencies, but also more general work-related skills which have been shown to be important for success in the medical professions 2. PBL-based graduates in particular rated "Practical medical skills" as being much better taught at medical school compared to their colleagues from conventional curricula. These findings may support the hypothesis that PBL methods directly and indirectly improve medical skills by improving independent learning and communication skills 2826.

In contrast to the described self-perceived benefits of the PBL-based curriculum, the conventional curricula were rated better for teaching of "Medical knowledge" and "Research competence". Case-studies, as used in PBL-based curricula, are time consuming 4. Further, systematic lectures are (partly) replaced by individual seminars where students can raise and work on their own questions. Both effects may lead to impartment less medical knowledge in a PBL-based curriculum as compared to a conventional curriculum in medical school 2. However, no disadvantages were observed regarding performance in the state administered final national board medical examination, in fact graduates from the PBL-based curriculum graduated with better average results in our study cohort as compared to graduates from the conventional curriculum. Other studies comparing USMLE (United States Medical Licensing Examination) Step 1 and 2 mean scores and pass rates showed similar performance of graduates from PBL-based und conventional curricula in the United States 5. Research competence may be another weakness of PBL-based curriculum. This finding was previously shown in a review by Colliver et al. 27. This review in conjunction with our data leads us to the explanation that PBL-based curricula have a distinct disadvantage in losing the ties between educational theory and research 27.

By directly comparing "How much a competence is required at work?" and "How well this competence was taught in medical school?", the curricular performance can be estimated in relation to job requirements. Our data suggests that the PBL-based curriculum fulfilled all job requirements except for two competencies (see below) and moreover, the PBL-based provided a significant educational surplus in five competencies, especially in "Interdisciplinary thinking" and "Independent learning/working" (both p < 0.001, Figure 1).

Our results further demonstrate that "Business competence" is insufficiently taught in both curricula. Thus, it may be of great interest to incorporate aspects of business administration and management which apparently are demanded by the clinical job to enhance both curricula. In fact, in a study by Larson et al., the investigators demonstrate that implementation of joint MD/MBA programs at US universities has considerably increased from six programs in 1993 to 33 programs in 2002 28. In addition, "Research competence" is not sufficiently taught by the PBL-curriculum and needs further improvement to meet job requirements.

Our results need to be evaluated in the context of our study limitations. Most importantly, our findings may be limited by selection bias as our overall response rate was low, although we included nation-wide over 4,800 graduates. Furthermore, the response rate differed between both groups. While only one medical school in Germany (University Witten/Herdecke) offered a PBL-based curriculum during 1996 and 2002, there may be differences with respect to realization of the PBL framework which may limit external validity of the observed findings. Even so our data was collected 2004/05, it has still relevant for the development of medical education systems, especially while the major part of medical schools is still based on a conventional curriculum. In addition, PBL-based curriculum should be developed further under the consideration of the identified weaknesses. Finally, our assessment of competencies required at day-to-day work as well the competencies as taught in medical school was based on self-perception. Some studies suggest that self-perception can differ from objective measures 1429. In this context, an interesting aspect may be that the evaluation of job requirements is very similar, even so great differences with a large effect size are observed between the evaluation of PBL-based and conventional curricula.

Given the limitations of the present analysis, further research to confirm and extend our findings is warranted. It may be of additional value to approach and survey employers or professional colleagues about their perceived skills of PBL and conventional medical graduates. Moreover, to address the limitation of low response rates, it may be more feasible to use personal interviews rather than postal or online questionnaires. Finally, more medical schools should be included which offer PBL-based curricula today. Thus longitudinal follow-up studies and objective assessment of competencies are necessary to fully evaluate the impact of PBL on medical education.