Email updates

Keep up to date with the latest news and content from BMC Medical Education and BioMed Central.

Open Access Research article

Measuring strategies for learning regulation in medical education: Scale reliability and dimensionality in a Swedish sample

Samuel Edelbring

Author affiliations

Centre for Medical Education, Department LIME, Karolinska Institutet, Berzelius väg 3, 171 77, Stockholm, Sweden

Citation and License

BMC Medical Education 2012, 12:76  doi:10.1186/1472-6920-12-76

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1472-6920/12/76


Received:5 January 2012
Accepted:26 July 2012
Published:15 August 2012

© 2012 Edelbring; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The degree of learners’ self-regulated learning and dependence on external regulation influence learning processes in higher education. These regulation strategies are commonly measured by questionnaires developed in other settings than in which they are being used, thereby requiring renewed validation. The aim of this study was to psychometrically evaluate the learning regulation strategy scales from the Inventory of Learning Styles with Swedish medical students (N = 206).

Methods

The regulation scales were evaluated regarding their reliability, scale dimensionality and interrelations. The primary evaluation focused on dimensionality and was performed with Mokken scale analysis. To assist future scale refinement, additional item analysis, such as item-to-scale correlations, was performed.

Results

Scale scores in the Swedish sample displayed good reliability in relation to published results: Cronbach’s alpha: 0.82, 0.72, and 0.65 for self-regulation, external regulation and lack of regulation scales respectively. The dimensionalities in scales were adequate for self-regulation and its subscales, whereas external regulation and lack of regulation displayed less unidimensionality. The established theoretical scales were largely replicated in the exploratory analysis. The item analysis identified two items that contributed little to their respective scales.

Discussion

The results indicate that these scales have an adequate capacity for detecting the three theoretically proposed learning regulation strategies in the medical education sample. Further construct validity should be sought by interpreting scale scores in relation to specific learning activities. Using established scales for measuring students’ regulation strategies enables a broad empirical base for increasing knowledge on regulation strategies in relation to different disciplinary settings and contributes to theoretical development.

Keywords:
Regulation strategies; Self-regulated learning; Approaches to studying; Validation

Background

Students’ independent learning in terms of monitoring and guiding their own learning process is considered fundamental for students’ achievements in higher education [1,2]. With roots in adult learning [3], the concept of self-regulated learning has developed in social cognitive theory [4] and psychology of learning [5]. Self-regulated learning is considered to be a capability that comprises aspects of responsibility for one’s own learning, such as evaluating, regulating, and setting goals for the learning process [1,5]. Theorists describe students’ learning regulation strategies as indirectly influencing the process of learning, thereby influencing learning outcomes [5]. The concept has been found to engage students, educators as well as policy makers [6]. Inspired by incentives for lifelong learning, and student centred education, the large interest for self-regulated learning in higher education has also reached medical education [7,8].

The theoretical constructs of regulation strategies are commonly measured by questionnaire scales for quantification of results. The issue of validity—the degree to which the scale measures what it purports to measure, is of central concern in this practice. Messick [9] stated that validity is a characteristic of the interpretation of results and does not pertain to the instrument itself. Cultural differences, changes in student characteristics over time, varying teaching and study practices between disciplines and national educational systems require renewed assessment and fine tuning of instruments [10]. Hence, whether or not established questionnaires function in new environments is an empirical question, the results need to undergo psychometric evaluation for the population in question [11].

The predominant method for evaluating scale quality is to measure the internal scale consistency with Cronbach’s alpha, which functions as a lower bound of reliability [12]. A more thorough analysis would also look into the dimensionality of the item set—the extent to which scale items relate to one construct—thereby allowing using a sum score to accurately indicate levels of the latent trait. Because it has been shown that high Cronbach’s alpha does not guarantee unidimensionality, additional measures are needed to assess whether items comprise a unidimensional scale regarding the construct it is intended to cover [13].

A fairly recent scale analysis method is the Mokken scale analysis [14,15]. This method presents an alternative to factor analysis and is categorised under the non-parametric item response theory (IRT). Mokken analyses evaluate scale dimensionality based on Loevinger’s definition of homogeneity [16]. In addition to dimensionality, Mokken scale analysis also has methods for testing the IRT models of monotone homogeneity and double monotonicity [17]. Monotone homogeneity means that respondents can be ordered on a latent trait using scale scores. The more restrictive model of double monotonicity means that the scale items also can be hierarchically ordered in relation to the latent trait. Furthermore, the Mokken method makes fewer assumptions on empirical data than does traditional factor analysis (e.g. regarding models and linearity of the item-construct relation) [18-20].

The Inventory of Learning Styles (ILS) [5] was developed in a European setting specifically for the higher education context and has been widely used and reviewed in researching students’ learning patterns [2,21,22]. In addition to regulation strategies the complete ILS covers processing strategies, mental models of learning, and learning orientations. The regulation scales purport to measure how students usually approach studying in terms of regulation strategies. The three main scales—self-regulation, external regulation, and lack of regulation—are represented by 28 items. Self-regulation contains the subscale dimensions learning process and result and learning content, which are concerned with the degree to which students plan processing activities, diagnose the cause of learning problems that occur, and direct themselves toward learning objectives of their own. External regulation contains the subscales learning results and learning processes. External regulation concerns the degree to which students rely on didactic aids, such as formal learning objectives and assignments. Lack of regulation concerns the inability to regulate one’s own learning and perceived lack of external support [5].

Increased knowledge of regulation strategies in student groups contributes both to direct student benefit and to research purposes. Regulation strategy scales have been used as a self-diagnostic tool for stimulating reflection on learning processes and raising awareness about study strategies among teachers and students [2,23,24]. Research on regulation strategies contributes with knowledge regarding the type of guidance students need from the teacher and the course structure, both in larger structures, such as comparisons between traditional and problem-based settings [25], and in relation to specific course activities [26]. This knowledge forms an important base for making full use of students’ motivation to learn and adapting corresponding course design and teacher guidance [27]. Differences in how self-regulated learning is enacted and its consequences for learning highlight the need for investigating these attributes in relation to each discipline [28]. In the field of medical education, concern has been voiced that much of previous efforts to enhance learners’ autonomy have been based on loose assumptions rather than systematic research [29]. The rich body of empirical and theoretical literature from other disciplines cannot replace contextualised empirical findings in medicaleducation [8]. The aim of this study was to psychometrically evaluate a translation of established regulation strategy scales in a Swedish medical education context.

Methods

Participants

The sample consisted of medical students participating in a clinical clerkship preparatory course. The students were approached cross-sectionally during the course, in four hospitals affiliated with the medical university of Karolinska Institutet, Sweden. A pilot was conducted in two courses in autumn 2008 (n = 44, response rate = 67 %) followed by a cross-sectional study in all four courses in spring 2009 (n = 206, response rate = 83 %). At this point, the students were at the verge of entering the more clinical oriented phase of the medical programme.

Materials and procedure

The regulation strategy scales from the 120-item version of the Inventory of Learning Styles (ILS) [5] were translated, piloted, revised, and used in 2008–2009. The 28 items were composed of Likert-type statements, with response alternatives denoting the frequency with which students use the proposed activity: e.g. ‘I add something to the subject matter from other sources’, ‘I do this seldom or never’, ‘I do this sometimes’, ‘I do this regularly’, ‘I do this often’, ‘I do this almost always’ (the answer categories are represented by the numbers 1–5).

The questionnaire was first translated by the author to Swedish. Item wordings were then examined in relation to the Norwegian translation [25] as the two languages are closely related, and then reviewed by two bilingual medical educationalists regarding content validity. After the pilot, item wordings were further refined through telephone interviews and a group interview, with three and five respondents, respectively.

Incomplete responses were approached differently when analysing the data from the pilot and the revised version. In the pilot, non-systematic missing values were approached with “hot deck imputation” —copying values from other respondents with similar but complete response patterns in the scale analysis in order to make full use of the small sample [30]. In the revised setting incomplete responses (10) were discarded from the scale analysis. The original item order from the ILS was retained, although item numbering differs due to the standalone regulation strategy version being shorter. The study has been ethically reviewed by the local ethical board (rn: 2008/822-31/5).

Validation criteria

Internal reliability for the scales was estimated with Cronbach’s alpha. Scale dimensionality was estimated by Mokken scale analysis based on Loevinger’s coefficient H[16]. H > 0.3 means that items have enough in common to trust the ordering of persons by using the scale scores [17]. The common practise for interpreting dimensionality by means of coefficient H is that a scale is considered weak when the H-estimate is between 0.3 and 0.4, medium when between 0.4 and 0.5 and strong when > 0.5. All item pairs within each scale were further investigated regarding covariance (Hij), and individual item relation to the scale (Hi) [14]. The Mokken automated scale partitioning was used to investigate whether the established factor structure would be replicated. Interrelations between scales were calculated by Spearman rank order correlation on summed scale scores due to its ordered data origin. Invariant item ordering is one method used to check for double monotonicity, i.e. the extent to which items can be ordered hierarchically in the scale in relation to the regulation strategy construct. This aspect was estimated by coefficient HT. Proposed interpretations of this coefficient indicate that HT < 0.3 means that the item ordering is inaccurate regarding item ordering; low accuracy between 0.3 and 0.4, medium when between 0.4 and 0.5, and high accuracy when > 0.5 [31]. Statistical analyses were performed in the statistical package R version 2.13.0 [32]. The R Mokken library version 2.5 was used for Mokken scale analysis.

Results

Pilot

Reliability, as measured by Cronbach’s alpha, ranged from 0.60 to 0.80 in the three main scales (Table 1). Mokken dimensionality analysis showed that the two self-regulation subscales were of adequate dimensionality (H > 0.3) (Table 2). The item pair covariance (Hij) ranged from −0.21 to 0.77 and individual item dimensionality (Hi) ranged from 0.07 to 0.46 (Table 3). Despite some negative Hij values, items generally contributed to their respective dimensions.

Table 1. Internal consistency of regulation scales

Table 2. Mokken dimensionality evaluation

Table 3. Individual item dimensionality (Hi) in relation to scales

Exploratory scale partition resulted in three Mokken scales in the pilot (Table 4). The first scale only included items from the established self-regulation scale. Mokken scale 2 corresponded to the external regulation scale, and scale 6 to lack of regulation. Dimensionality coefficient H ranged from 0.15 to 0.36 (Table 2). Data from student interviews indicated that the items required a lot of thought, and that the students perceived several items as being similar to each other.

Table 4. Resulting scales from Mokken exploratory partitioning

Revised version

Based on student interviews, the wordings of the items were refined slightly, after which the questionnaire was distributed to the cohort of medical students taking a clinical clerkship preparatory course at four teaching hospitals. The reliability for main scales ranged from Cronbach’s alpha 0.65 to 0.82 (Table 1).

Evaluation of the established scales showed self-regulation and its subscales to be of adequate (H > 0.3) dimensionality (Table 2). The individual item dimensionality (Hi) ranged from 0.11 to 0.50, with two items (22 and 28) displaying low, but positive, scale dimensionality (Table 2). Pair-wise covariance (Hij) ranged from −0.09 to 0.67. All item pairs, except three, displayed positive Hij values. Negative Hij pairs were 8–14 and 8–28 in scale external regulation and 6–18 in Lack of regulation, ranging from −0.01 to −0.09.

Six Mokken scales were extracted from the responses and compared to the theoretical factor structure (Table 3). The first one included six out of seven items from self-regulation of learning process and results and one (item 22) from the external regulation scales. Mokken scale 2 included three items from external regulation of learning processes and one from external regulation of learning results. The third scale was identical to the self-regulation of learning content. The fourth and sixth scales included three and two items, respectively, from lack of regulation. The fifth scale contains two items from external regulation of learning results.

No internal correlation was found between main scales (Rho ≤ 0.14 in the pilot and ≤ 0.08 in the revised setting). As expected, subscales correlated significantly with each other (p < 0.01) for self-regulation (Rho = 0.47) and external regulation (Rho = 0.39). Three scales (external regulation, sub scale self-regulation of learning content, and external regulation of learning processes) displayed invariant item ordering coefficients >= 0.3 (Table 5).

Table 5. Invariant Item Order (IIO) scale assessment

Discussion

Adequate dimensionality, divergent validity, and scale consistency contribute validity to the self-regulation (SR) scale and its subscales in the medical education setting. However, the analyses indicate that some of the items do not contribute optimally in the regulation scales and that external regulation (ER) and lack of regulation (LR) scales were weak regarding dimensionality in this sample.

Because psychometrics is about measuring the unobservable, there is not one single criterion against which we can assert fulfilment of the goal of trustworthy tools. Therefore, several aspects, taken together, form a basis for assessment and basis for further scale development. internal consistency in this sample was very good in comparison with other studies using the scales. Cronbach’s alpha values for the three main scales (SR: 0.82, ER: 0.72, and LR: 0.65) were exceeding those obtained in a Norwegian medical setting (SR: 0.73, ER: 0.69, and LR: 0.57) [25] and generally exceeded previously reported Cronbach’s alpha of 0.48 to 0.81 in several studies in the Netherlands [22], 0.46 to 0.72 in British settings [2], and 0.69 to 0.75 in a Finnish pharmacy setting [33].

Dimensionality, as assessed by Mokken scale analysis, was higher in the dataset from the revised version, although the estimates were not ideal (Table 2). The strongest regulation scale was the subscale Self-regulation process and results, with coefficient H = 0.41. Following empirically derived rules of thumb of H-interpretation, this falls into the category of moderate dimensionality [17]. Of the main scales, self-regulation got highest value with H = 0.32 which means it is a weak scale regarding dimensionality. External regulation scale displayed lowest dimensionality (H = 0.21). This finding corresponds with this dimension not being detected at all in a Finnish sample [26]. Higher H coefficient values for sub scales are explained by their items being more narrowly connected to each other, whereas the main scales approach the strategy dimensions more broadly. The implication for scales with low dimensionality is that ranking of participants with scores in a narrow range will be less accurate.

On a more detailed level, the Hi estimate contributed with insight into how single items contribute to scale dimensionality (Table 2). In general, all items contributed to their respective scale, except items 22 and 28, which did not seem to contribute much to the scale (external regulation). The characteristic of item 22 is discussed below. Item 28 concerns fulfilment of assignments during the course which makes it less appropriate for a clinically (practically) oriented course. The total dimensionality coefficient (H) for that scale would increase removing these two items. However, that would remove nuances from the scale construct, and hence, from the theoretical base. Since both items score positively on the Hi, they contribute, however little, to the scale construct. The practical implications are that the scale is less accurate in distinguishing between respondents with similar scale scores than if all items contribute highly to the scale construct. When disregarding previous partitioning and exposing all items to exploratory scale partitioning, they risk being regrouped in ways that were not initially intended. However, the theoretical scale structure was broadly replicated in the data. All aspects of the established scales were represented in the six Mokken scales. The largest group is Mokken scale 1, with six of the seven self-regulation of learning process and results items represented. Scale 2 contains items from external regulation, mostly from external regulation of learning process. Scale 3 was identical to self-regulation of learning content. Scale 5 has two items from external regulation of learning results, and scales 4 and 6 correspond with lack of regulation.

With the exception of one item (22), the six exploratory Mokken scales correspond well with the theoretical partitioning established in previous research (Table 3). However, in this sample, item 22 converges with self-regulation of learning process and results, although it belonged to external regulation of learning results in the established partitioning. This item also displayed low item-scale homogeneity (Hi). The item concerns the frequency with which students thoroughly apply themselves to the methods dealt with in a course. Considering the context in which respondents were situated—the clinical preparatory course—the “methods” could be interpreted as diagnostic methods, and thus a core aspect of the course. Training to apply these methods could fit well into the scope of a self-regulated goal. Consequently, this item does not function well in discriminating between self- and externally regulated strategies, but contributes to both constructs in this setting.

Regulation strategies as such are theorised as not only relating to individual preference, but also to the specific learning situation [5,34,35]. Consequently, the situative aspect should be considered in data collection and interpretation. Comparisons between different overarching curricula, such as PBL and traditional ones, should expect variations in how the construct is interpreted [25,36]. Comparisons within the same educational culture and setting, contribute to establish the construct validity of regulation strategies as measured by these scales. When comparing this study’s scale means to other traditional Scandinavian curricula, a similar distribution of pattern is discerned, dominated by external regulation, followed by self-regulation and lack of regulation (Table 6).

Table 6. Regulation strategy mean scores from the Swedish setting compared with Norwegian medical education settings, using problem-based (PBL) and traditional curriculum

As the three regulation strategies are different constructs, they are not expected to interrelate; i.e. they should display divergent validity. Nevertheless, the subscales of the self- and external regulation scales should relate, to some extent, although covering different facets. The non-interrelation identified among the main scales contributes indirectly to their divergent validity. Interrelations between subscales were significant, although somewhat weaker than those reported elsewhere [2]. Judging by their face value, one could assume that self-regulation and external regulation would be each other’s opposites and, hence, correlate negatively; however, in this sample, they did not. Research shows that students can successfully combine the two regulation strategies, suggesting a dynamic relationship where external regulation has modelling and scaffolding functions [23].

Invariant item ordering (IIO) is not a claimed property of the ILS’s regulation strategy scales. Scale items contribute within the regulation strategy scales, even if not ordered in a hierarchy. Nevertheless, the test for IIO contributed additional validation data in disclosing to what extent respondents conceive the items in a similar manner. Three scales displayed adequate IIO accuracy, while the overall low IIO estimates for other scales express variation in how respondents approach the scale items (Table 5).

Implications and further development

The gains of having an established instrument that can be used in different disciplines and in different cultural settings should be considered when choosing and assessing methods to measure regulation strategies. The continued use of these established scales allows researchers to build on prior theoretical bases. In line with the view of validity pertaining to contextualised results, complexity of the construct, and purpose of use of the results, the researcher needs to interpret results on regulation strategies in relation to the students’ study situation [9].

Scale scores should be interpreted in conjunction with other empirical data, such as course activities or other scale scores, thereby contributing to further construct validity.

Further refinement of the wording of questionnaire items is recommended in this translation, as medical students found some items hard to deal with. This work should retain content validity in adhering to the theoretical base and use reported dimensionality coefficients (Hi), and future content validation of the wordings with student groups, preferably from other disciplines as well. The vocational character of the participants’ study environment is currently marginally reflected in the strategy scales. Considering the contextual influence discussed by Richardson [10], more accurate results would perhaps be found in medical and other professional education if the scale items were adapted toward vocational aspects. However, as item adaptation would imply deviation from the original scales, and thus, the theoretical base, a better approach would be to add a vocational oriented scale along with the others. Generalizability of findings of this study beyond third-year students in a traditional Swedish medical curriculum is restricted by the moderate sample size and single discipline context. Data-gathering from the whole cohort, cross-sectionally over four course settings, contribute to the strength of the study, providing a broad base for validation of scale results.

Conclusion

The regulation scales of the ILS measures individual study strategies relating to specific learning environments and, hence, to the culture in which they are used. These scales were developed in a non-medical education setting. Still, when used in a Swedish medical education setting, the scale structure is replicated, and internal consistencies are good. Thorough analysis of scale homogeneity identified areas where these scales can be further improved and possibly more tailored toward the professional aspects of medical education.

Competing interest

The author declare that he have no competing interests.

Acknowledgments

Jan Vermunt is acknowledged for kindly providing the ILS and allowing its translation and use. Thanks to Ulf Brodin for advice on the Mokken analysis. Lars Owe Dahlgren is acknowledged for supervising the study. Tine Nielsen and John T.E. Richardson are acknowledged for critical reading and valuable comments on earlier versions of the manuscript.

Thanks also to Klara Bolander-Laksov, Cormac McGrath, Carl Savage and KI students for participating in the questionnaire translation and revision. Course directors, teachers and administrators are thanked for assistance in data collection: Olle Broström, Peter Henriksson, Magnus Sköld, Daphne Vassiliou, Mini Ruiz, Katarina Arnrup and Anette Boban.

References

  1. Boekaerts ME: Self-regulated learning: where we are today.

    International Journal of Educational Research 1999, 31(6):445-457. Publisher Full Text OpenURL

  2. Boyle EA, Duffy T, Dunleavy K: Learning styles and academic outcome: the validity and utility of Vermunt's Inventory of Learning Styles in a British higher education setting.

    Br J Educ Psychol 2003, 73(Pt 2):267-290. PubMed Abstract OpenURL

  3. Knowles MS: Self-Directed Learning: A Guide for Learners and Teachers. New York: Association Press; 1975. OpenURL

  4. Zimmerman BJ: A Social Cognitive View of Self-Regulated Academic Learning.

    Journal of Educational Psychology 1989, 81(3):329-339. OpenURL

  5. Vermunt JD: The regulation of constructive learning processes.

    British Journal of Educational Psychology 1998, 68(2):149-171. Publisher Full Text OpenURL

  6. Boekaerts M: Self-Regulated Learning: A New Concept Embraced by Researchers, Policy Makers, Educators, Teachers, and Students.

    Learning and Instruction 1997, 7(2):161-186. Publisher Full Text OpenURL

  7. White CB, Gruppen LD: Self-Regulated Learning in Medical Education. In Understanding Medical Education. Edited by Swanwick T. Wiley-Blackwell; 2010:271-282. OpenURL

  8. Brydges R, Butler D: A reflective analysis of medical education research on self-regulation in learning and practice.

    Med Educ 2012, 46(1):71-79. PubMed Abstract | Publisher Full Text OpenURL

  9. Messick S: Validity. In Educational measurement. 3rd edition. Edited by Linn RL. New York: American Council on Education: Macmillan; 1989:xiv, 610. OpenURL

  10. Richardson JTE: Methodological Issues in Questionnaire-Based Research on Student Learning in Higher Education.

    Educational Psychology Review 2004, 16(4):347-358. Publisher Full Text OpenURL

  11. Harkness J: Questionnaire translation. In Cross-cultural survey methods. Edited by Harkness J, Vijver FJR, Mohler PP. New York: Wiley; 2002. OpenURL

  12. Cortina JM: What is coefficient alpha? An examination of theory and applications.

    Journal of Applied Psychology 1993, 78(1):98-104. OpenURL

  13. Schmitt N: Uses and Abuses of Coefficient Alpha.

    Psychological Assessment 1996, 8(4):350-353. OpenURL

  14. Van der Ark LA: Mokken Scale Analysis in R.

    Journal of Statistical Software 2007, 20(11):1-19. OpenURL

  15. Brodin U, Fors U, Laksov K: The application of Item Response Theory on a teaching strategy profile questionnaire.

    BMC Medical Education 2010, 10(1):14. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text OpenURL

  16. Loevinger J: The technic of homogeneous tests compared with some aspects of "scale analysis" and factor analysis.

    Psychological Bulletin 1948, 45(6):507-529. PubMed Abstract OpenURL

  17. Sijtsma K, Molenaar IW: Introduction to nonparametric item response theory. Thousand Oaks, Calif: SAGE Publications; 2002. OpenURL

  18. Hemker BT, Sijtsma K, Molenaar IW: Selection of Unidimensional Scales From a Multidimensional Item Bank in the Polytomous Mokken I RT Model.

    Applied Psychological Measurement 1995, 19(4):337-352. Publisher Full Text OpenURL

  19. Van Schuur WH: Mokken Scale Analysis: Between the Guttman Scale and Parametric Item Response Theory.

    Political Analysis 2003, 11(2):139-163. Publisher Full Text OpenURL

  20. Mokken R: Nonparametric Models for Dichotomous Responses. In Handbook of modern item response theory. Edited by Hambleton RK, Linden WJ. New York, N.Y: Springer; 1997. OpenURL

  21. Richardson JTE: Researching student learning: approaches to studying in campus-based and distance education. Buckingham: Society for Research into Higher Education: Open Univ. Press; 2000. OpenURL

  22. Vermunt JD, Vermetten YJ: Patterns in Student Learning: Relationships Between Learning Strategies, Conceptions of Learning, and Learning Orientations.

    Educational Psychology Review 2004, 16(4):359-384. Publisher Full Text OpenURL

  23. Vermunt JDHM, Van Rijswijk FAWM: Analysis and Development of Students’ Skill in Self-Regulated Learning.

    Higher Education 1988, 17(6):647-682. Publisher Full Text OpenURL

  24. Maclellan E, Soden R: Facilitating Self-Regulation in Higher Education Through Self-Report.

    Learning Environments Research 2006, 9(1):95-110. Publisher Full Text OpenURL

  25. Hofgaard Lycke K, Grøttum P, Strømsø HI: Student learning strategies, mental models and learning outcomes in problem-based and traditional curricula in medicine.

    Medical Teacher 2006, 28(8):717-722. PubMed Abstract | Publisher Full Text OpenURL

  26. Vilppu H, Mikkilä-Erdmann M, Ahopelto I: The Role of Regulation and Processing Strategies in Understanding Science Text Among University Students.

    Scandinavian Journal of Educational Research 2011. Publisher Full Text OpenURL

  27. Vermunt JD, Verloop N: Congruence and friction between learning and teaching.

    Learning and Instruction 1999, 9(3):257-280. Publisher Full Text OpenURL

  28. Vanderstoep SW, Pintrich PR, Fagerlin A: Disciplinary Differences in Self-Regulated Learning in College Students.

    Contemporary Educational Psychology 1996, 21(4):345-362. PubMed Abstract | Publisher Full Text OpenURL

  29. Misch DA: Andragogy and medical education: are medical students internally motivated to learn?

    Adv Health Sci Educ Theory Pract 2002, 7(2):153-160. PubMed Abstract | Publisher Full Text OpenURL

  30. Downey RG, King C: Missing data in Likert ratings: A comparison of replacement methods.

    The Journal of General Psychology 1998, 125(2):175-191. PubMed Abstract | Publisher Full Text OpenURL

  31. Ligtvoet R, Van der Ark LA, Te Marvelde JM, Sijtsma K: Investigating an Invariant Item Ordering for Polytomously Scored Items.

    Educational and Psychological Measurement 2010, 70(4):578-595. Publisher Full Text OpenURL

  32. R Development Core Team: R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2011. OpenURL

  33. Nieminen J, Lindblom-Ylänne S, Lonka K: The Development of Study Orientations and Study Success in Students of Pharmacy.

    Instructional Science 2004, 32(5):387-417. OpenURL

  34. Entwistle NJ, Peterson ER: Conceptions of Learning and Knowledge in Higher Education: Relationships with Study Behaviour and Influences of Learning Environments.

    International Journal of Educational Research 2004, 41:407-428. Publisher Full Text OpenURL

  35. Minnaert A, Janssen PJ: Bias in the assessment of regulation activities in studying at the level of higher education.

    European Journal of Psychological Assessment 1997, 13(2):99-108. Publisher Full Text OpenURL

  36. Van der Veken J, Valcke M, Muijtjens A, De Maeseneer J, Derese A: The potential of the inventory of learning styles to study students' learning patterns in three types of medical curricula.

    Medical Teacher 2008, 30(9–10):863-869. PubMed Abstract | Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1472-6920/12/76/prepub