Email updates

Keep up to date with the latest news and content from BMC Ophthalmology and BioMed Central.

Open Access Commentary

Patient-centred measurement in ophthalmology – a paradigm shift

Konrad Pesudovs

Author Affiliations

NHMRC Centre for Clinical Eye Research, Department of Ophthalmology, Flinders Medical Centre and Flinders University, Bedford Park, South Australia, 5042, Australia

BMC Ophthalmology 2006, 6:25  doi:10.1186/1471-2415-6-25


The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2415/6/25


Received:2 May 2006
Accepted:15 June 2006
Published:15 June 2006

© 2006 Pesudovs; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Ophthalmologists and researchers in ophthalmology understand what a rapidly evolving field ophthalmology is, and that to conduct good research it is essential to use the latest and best methods. In outcomes research, one modern initiative has been to conduct holistic measurement of outcomes inclusive of the patient's point of view; patient-centred outcome. This, of course, means including a questionnaire. However, the irony of trying to improve outcomes research by being inclusive of many measures is that the researcher may not be expert in all measures used. Certainly, few people conducting outcomes research in ophthalmology would claim to be questionnaire experts. Most tend to be experts in their ophthalmic subspecialty and probably simply choose a popular questionnaire that appears to fit their needs and think little more about it. Perhaps, unlike our own field, we assume that the field of questionnaire research is relatively stable. This is far from the case. The measurement of patient-centred outcomes with questionnaires is a rapidly evolving field. Indeed, over the last few years a paradigm shift has occurred in patient-centred measurement.

Text

Measurement of patient-centred health outcomes began in the 1950s, and more than 70 measures of functional status, 2 dozen generic quality of life instruments, and hundreds of disease-specific measures now exist [1-4]. Much of this research has relied on the theory and methods of Classical Test Theory (CTT). Underpinning CTT is simple scoring of responses to questionnaires; for multi-category response scales this is usually summary scoring where response categories are assigned ordinal numbers which are summed across questions to arrive at a total score. This score is assumed to represent measurement of the underlying trait (e.g. quality of life). An alternative approach is Item Response Theory (IRT) whereby items and persons can be scaled according to a series of responses to items. Implicit is that not only can people have different ability, but that items can have different difficulty and both can be estimated. The foundations of IRT were laid down in the 1920s, and great advances were made after 1950, especially with the contribution of Georg Rasch [5,6]. Rasch analysis is a special case of IRT similar to a one-parameter model, but importantly, Rasch models meet the conditions of non-interactive conjoint structures so, unlike IRT models, they are valid measurement models [7]. (See Massof's excellent background paper for further discussion of these methods as applied to ophthalmology [8]). It has only been in the last 25 years that Rasch analysis has been applied in studies of health status. Early applications included mental health and physical rehabilitation in the 1980s [9,10], including low vision rehabilitation [11,12]. Through the 1990s, and into the 21st century, Rasch analysis has penetrated many health outcome fields and is becoming prevalent in ophthalmology.

However, many popular ophthalmic questionnaires use traditional summary scoring [13]. Summary scoring assumes the value of each item represents equal difficulty and therefore scores them equally. In addition, the ordinal integer response scale used for each item assumes uniform changes between response categories. For example, in a summary scored vision disability instrument such as the 'Activities of Daily Vision Scale (ADVS)'[14], a response of "a little difficulty" (score of 4) is used to represent twice the level of ability as "extreme difficulty" (score of 2) which is similarly two times as good as "unable to perform the activity due to vision" (score of 1) for all items. This appears illogical and Rasch analysis has been used to confirm that specific response category calibrations are required to provide a linear scale [15]. Similarly, summary scoring assumes that all items are of equal difficulty. For example, with the ADVS instrument an answer of "a little difficulty" to the question regarding visual difficulties 'driving at night' scores the same as "a little difficulty" with 'driving during the day'. Again, this is illogical and Rasch analysis has been used to confirm that subjects report that 'driving at night' is a more difficult task than 'driving during the day'; Rasch analysis can provide the appropriate weighting for each item to enable linear measurement [15].

By resolving inequities in a scale arising from differential item difficulty, Rasch analysis provides a self-evident benefit in terms of accuracy of scoring. However, this process also removes noise from the measurement which in turn improves sensitivity to change and correlations with other variables [16,17]. Clearly, these are important benefits for outcomes research with implications for sample sizes etc. The assumption of unidimensionality inherent in Rasch analysis also provides unparalleled insight into the dimensionality of a questionnaire. This can be used to advantage in questionnaire development. Therefore, beyond its simple application, Rasch analysis has been used in three important ways in ophthalmology: the development of new questionnaires, shortening or revising existing questionnaires, and test equating.

Across ophthalmology, a number of questionnaires have been developed using Rasch analysis. The field of low vision has led the way with the Veterans Affairs Low-Vision Visual Functioning Questionnaire, the Melbourne Low Vision ADL Index and the Visual Function Questionnaire [18-22]. Two Rasch scaled questionnaires have also been developed for refractive outcomes: one for spectacles, contact lenses and refractive surgery (The Quality of Life Impact of Refractive Correction, QIRC) and one for contact lenses only (Contact lens Impact on Quality of Life, CLIQ) [23-26]. While other sub-fields of ophthalmology are yet to gain questionnaires developed using Rasch analysis, these are likely not far away as a recent major review of quality of life in glaucoma called for the development of a Rasch scaled glaucoma-specific visual disability questionnaire [27].

Existing conventionally validated questionnaires can be rescaled using Rasch analysis. In this way, summary scoring can be converted to a truly linear scale using a simple formula. This has occurred for cataract surgery outcomes [28], refractive surgery outcomes [17], age-related macular degeneration outcomes [29], low vision care [30], and the measurement of ophthalmic pain [31,32]. Some of these approaches have been to simply rescore an existing questionnaire, while others have taken the approach of completely re-engineering a questionnaire to optimize its performance. The latter process takes advantage of the insight into dimensionality of a questionnaire to refine measurement and improve targeting of content to the population. Notable examples of questionnaire re-engineering are the Impact of Visual Impairment (IVI) for low vision outcomes [33,34], the Visual Disability Assessment (VDA) reinvented as the Cataract Outcomes Questionnaire [28,35], and The Visual Functioning 14 (VF-14) [36-38]. Usually, re-engineering of questionnaires involves removal of poorly fitting items to make a shorter questionnaire, although Velozo et al actually added items to the VF-14 and yet were unable to completely satisfactorily improve the instrument [36]. Rasch analysis has also been used to simply confirm the performance of, or detect deficiencies in questionnaires [39-42].

In ophthalmology, the most important patient-centred outcome measure is visual disability. Visual disability is the reason for performing cataract surgery, world wide the most common operation performed, and visual disability is a consequence of many ophthalmic diseases. Accordingly, there are many visual disability questionnaires e.g.: the Visual Activities Questionnaire (VAQ) [43], the Activities of Daily Vision Scale (ADVS) questionnaire [44], the VF-14 [45], the VDA [35], the National-Eye Institute Visual Functioning Questionnaire (NEI-VFQ)[46] and the Catquest questionnaire [47]. These questionnaires are widely used in different part of the world, for instance Catquest is widely used in Europe, the VF-14 in North America, and the VDA in Australia. While these questionnaires all measure the same concept, and have many of the same items, their scores cannot be simply compared. However, Rasch analysis provides a mechanism to equate scores from different questionnaires. Since all these questionnaires measure the same underlying trait, visual disability, they can all be modelled on the same latent variable. Massof has made an important first step in equating visual disability questionnaires implementing the ADVS, VAQ, VF-14 and the NEI-VFQ on a large population, conducting Rasch analysis and providing equations for conversion of summary scores to Rasch scores and conversion between questionnaires [48]. While Rasch analysis is in itself fairly simple, its application is somewhat specialized. Massof's equations provide a mechanism to gain the benefits of Rasch scaling without having to perform Rasch analysis.

Today, the outcomes researcher is faced with many different published questionnaires to choose from; some summary scored, others Rasch scaled. A major recent review by De Boer et al has attempted to make sense of this choice by systematically classifying questionnaires by the quality of their development and validation [49]. Notably, de Boer et al included Rasch scaling as a point of differentiation. A logical extension of this would be to rate highest questionnaires developed using Rasch analysis, followed by those conventionally developed and re-engineered using Rasch analysis and then those simply re-scored using Rasch analysis. There seems to be little advantage in using non-Rasch scaled questionnaires given this will increase noise and therefore reduce statistical power. Despite this, summary scored questionnaires remain popular [50], perhaps due to the simplicity of their scoring. But, many questionnaire developers, and other researchers, provide simple Rasch scale conversion obviating the need for questionnaire users to perform Rasch analysis [22,23,48]. This simple step would have added significant value to the paper by Owen et al published in this edition of BMC Ophthalmology [50]. Hopefully, others will agree that the time has come to abandon summary scoring. Lets just hope that Max Planck's pessimism about scientific change: "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it" does not hold true for the Rasch analysis paradigm shift.

References

  1. McHorney CA: Generic health measurement: past accomplishments and a measurement paradigm for the 21st century.

    Ann Intern Med 1997, 127:743-750. PubMed Abstract OpenURL

  2. McHorney CA: Health status assessment methods for adults: past accomplishments and future challenges.

    Annu Rev Public Health 1999, 20:309-335. PubMed Abstract | Publisher Full Text OpenURL

  3. McHorney CA: Use of item response theory to link 3 modules of functional status items from the Asset and Health Dynamics Among the Oldest Old study.

    Arch Phys Med Rehabil 2002, 83:383-394. PubMed Abstract | Publisher Full Text OpenURL

  4. McHorney CA, Monahan PO: Postscript: Applications of Rasch analysis in health care.

    Med Care 2004, 42:I73-8. PubMed Abstract OpenURL

  5. Bock R: A brief history of itern response theory.

    Educ Measure Issues Pract 1997, 21-33. Publisher Full Text OpenURL

  6. Rasch BG: Probabilistic Models for Some Intelligence and Attaiment Tests. Copenhagen, Denmark, Danmarks Paedogogiske Institut; 1960.

  7. Fisher WPJ: The Rasch debate: Validity and revolution in educational measurement. In Objective measurement: Theory into practice. Volume 2. Edited by Wilson M. Norwood, NJ, Ablex; 1994::36-72. OpenURL

  8. Massof RW: The measurement of vision disability.

    Optom Vis Sci 2002, 79:516-552. PubMed Abstract | Publisher Full Text OpenURL

  9. Barrett PT, Kline P: A comparison between Rasch analysis and factor analysis of item in the EPQ.

    Pers Study Group Behav 1981, 1:11-28. OpenURL

  10. Wright BD, Linacre JM: Observations are always ordinal, measurements must be interval.

    Arch Phys Med Rehabil 1989, 70:857-860. PubMed Abstract OpenURL

  11. Schulz EM: Functional Assessment In Rehabilitation: An Example With The Visually Impaired. , Universirty of Chicago; 1987.

  12. Becker SW, Lambert RW, Schulz EM, Wright BD, Burnet DL: An instrument to measure the activity level of the blind.

    Int J Rehabil Res 1985, 8:415-424. PubMed Abstract OpenURL

  13. Likert R: A technique for the measurement of attitudes.

    Arch of Psychol 1932, 140:1-55. OpenURL

  14. Mangione CM, Phillips RS, Seddon JM, Lawrence MG, Cook EF, Dailey R, Goldman L: Development of the 'Activities of Daily Vision Scale'. A measure of visual functional status.

    Med Care 1992, 30:1111-1126. PubMed Abstract OpenURL

  15. Pesudovs K, Garamendi E, Keeves JP, Elliott DB: The Activities of Daily Vision Scale for cataract surgery outcomes: re-evaluating validity with Rasch analysis.

    Invest Ophthalmol Vis Sci 2003, 44:2892-2899. PubMed Abstract | Publisher Full Text OpenURL

  16. Norquist JM, Fitzpatrick R, Dawson J, Jenkinson C: Comparing alternative Rasch-based methods vs raw scores in measuring change in health.

    Med Care 2004, 42:I25-36. PubMed Abstract OpenURL

  17. Garamendi E, Pesudovs K, Stevens MJ, Elliott DB: The Refractive Status and Vision Profile: Evaluation of psychometric properties and comparison of Rasch and summated Likert-scaling.

    Vision Res 2006, 46:1375-1383. PubMed Abstract | Publisher Full Text OpenURL

  18. Haymes SA, Johnston AW, Heyes AD: The development of the Melbourne low-vision ADL index: a measure of vision disability.

    Invest Ophthalmol Vis Sci 2001, 42:1215-1225. PubMed Abstract | Publisher Full Text OpenURL

  19. Nutheti R, Shamanna BR, Krishnaiah S, Gothwal VK, Thomas R, Rao GN: Perceived visual ability for functional vision performance among persons with low vision in the Indian state of Andhra Pradesh.

    Invest Ophthalmol Vis Sci 2004, 45:3458-3465. PubMed Abstract | Publisher Full Text OpenURL

  20. Massof RW, Hsu CT, Baker FH, Barnett GD, Park WL, Deremeik JT, Rainey C, Epstein C: Visual disability variables. II: The difficulty of tasks for a sample of low-vision patients.

    Arch Phys Med Rehabil 2005, 86:954-967. PubMed Abstract | Publisher Full Text OpenURL

  21. Massof RW, Hsu CT, Baker FH, Barnett GD, Park WL, Deremeik JT, Rainey C, Epstein C: Visual disability variables. I: the importance and difficulty of activity goals for a sample of low-vision patients.

    Arch Phys Med Rehabil 2005, 86:946-953. PubMed Abstract | Publisher Full Text OpenURL

  22. Stelmack JA, Szlyk JP, Stelmack TR, Demers-Turco P, Williams RT, Moran D, Massof RW: Psychometric properties of the Veterans Affairs Low-Vision Visual Functioning Questionnaire.

    Invest Ophthalmol Vis Sci 2004, 45:3919-3928. PubMed Abstract | Publisher Full Text OpenURL

  23. Pesudovs K, Garamendi E, Elliott DB: The Quality of Life Impact of Refractive Correction (QIRC) Questionnaire: development and validation.

    Optom Vis Sci 2004, 81:769-777. PubMed Abstract | Publisher Full Text OpenURL

  24. Pesudovs K, Garamendi E, Elliott DB: A quality of life comparison of people wearing spectacles or contact lenses or having undergone refractive surgery.

    J Refract Surg 2006, 22:19-27. PubMed Abstract OpenURL

  25. Garamendi E, Pesudovs K, Elliott DB: Changes in quality of life after laser in situ keratomileusis for myopia.

    J Cataract Refract Surg 2005, 31:1537-1543. PubMed Abstract | Publisher Full Text OpenURL

  26. Pesudovs K, Garamendi E, Elliott DB: The Contact Lens Impact on Quality of Life (CLIQ) Questionnaire: Development and Validation.

    Invest Ophthalmol Vis Sci 2006, 47:2789-96. PubMed Abstract | Publisher Full Text OpenURL

  27. Spaeth G, Walt J, Keener J: Evaluation of quality of life for patients with glaucoma.

    Am J Ophthalmol 2006, 141:S3-14. PubMed Abstract | Publisher Full Text OpenURL

  28. Pesudovs K, Elliott DB, Coster DJ: The Cataract Outcomes Questionnaire - A Rasch Scaled Measure of Visual Disability.

    Invest Ophthalmol Vis Sci 2005, 45:ARVO e-abstract 3844. OpenURL

  29. Hewitt AW, Jeganathan VS, Kidd JE, Pesudovs K, Verma N: Influence of photodynamic therapy for age related macular degeneration upon subjective vision related quality of life.

    Graefes Arch Clin Exp Ophthalmol 2006, 1-6. PubMed Abstract | Publisher Full Text OpenURL

  30. Smith HJ, Dickinson CM, Cacho I, Reeves BC, Harper RA: A randomized controlled trial to determine the effectiveness of prism spectacles for patients with age-related macular degeneration.

    Arch Ophthalmol 2005, 123:1042-1050. PubMed Abstract | Publisher Full Text OpenURL

  31. Noble BA, Loh RS, MacLennan S, Pesudovs K, Reynolds A, Bridges LR, Burr J, Stewart O, Quereshi S: Comparison of autologous serum eye drops with conventional therapy in a randomised controlled crossover trial for ocular surface disease.

    Br J Ophthalmol 2004, 88:647-652. PubMed Abstract | Publisher Full Text OpenURL

  32. Pesudovs K, Craigie MJ, Roberton G: The visual analogue scale for the measurement of pain is not linear.

    Anaesth Intensive Care 2005, 33:686-7; author reply 687. PubMed Abstract OpenURL

  33. Hassell JB, Weih LM, Keeffe JE: A measure of handicap for low vision rehabilitation: the impact of vision impairment profile.

    Clin Experiment Ophthalmol 2000, 28:156-161. PubMed Abstract | Publisher Full Text OpenURL

  34. Lamoureux EL, Pallant JF, Pesudovs K, Hassell JB, Keeffe JE: The Impact of Vision Impairment Questionnaire: an evaluation of its measurement properties using Rasch analysis.

    Invest Ophthalmol Vis Sci 2006, in press. PubMed Abstract | Publisher Full Text OpenURL

  35. Pesudovs K, Coster DJ: An instrument for assessment of subjective visual disability in cataract patients.

    Br J Ophthalmol 1998, 82:617-624. PubMed Abstract | Publisher Full Text OpenURL

  36. Velozo CA, Lai JS, Mallinson T, Hauselman E: Maintaining instrument quality while reducing items: application of Rasch analysis to a self-report of visual function.

    J Outcome Meas 2000, 4:667-680. PubMed Abstract OpenURL

  37. Valderas JM, Rue M, Guyatt G, Alonso J: The impact of the VF-14 index, a perceived visual function measure, in the routine management of cataract patients.

    Qual Life Res 2005, 14:1743-1753. PubMed Abstract | Publisher Full Text OpenURL

  38. Mallinson T, Stelmack J, Velozo C: A comparison of the separation ratio and coefficient alpha in the creation of minimum item sets.

    Med Care 2004, 42:I17-24. PubMed Abstract OpenURL

  39. Massof RW, Fletcher DC: Evaluation of the NEI visual functioning questionnaire as an interval measure of visual ability in low vision.

    Vision Res 2001, 41:397-413. PubMed Abstract | Publisher Full Text OpenURL

  40. Babcock-Parziale J, McKnight PE, Head DN: Evaluating psychometric properties of a clinical and a self-report blind rehabilitation outcome measure.

    J Rehabil Res Dev 2005, 42:487-498. PubMed Abstract | Publisher Full Text OpenURL

  41. Hart PM, Stevenson MR, Montgomery AM, Muldrew KA, Chakravarthy U: Further validation of the Daily Living Tasks Dependent on Vision: identification of domains.

    Br J Ophthalmol 2005, 89:1127-1130. PubMed Abstract | Publisher Full Text OpenURL

  42. Patel I, Munoz B, Burke AG, Kayongoya A, Mchiwa W, Schwarzwalder AW, West SK: Impact of Presbyopia on Quality of Life in a Rural African Setting.

    Ophthalmology 2006, 113:728-734. PubMed Abstract | Publisher Full Text OpenURL

  43. Sloane ME, Ball K, Owsley C, Bruni SR, Roenkar DL: The visual activities questionnaire: developing an instrument for assessing problems in everyday visual tasks.

    Tech Dig Noninvas Assess Vis Sys 1992, 1:26-29. OpenURL

  44. Mangione CM, Phillips RS, Seddon JM, G LM, Cook EF, Dailey R, Goldman L: Development of the "Activities of Daily Vision Scale". A measure of visual functional status.

    Med Care 1992, 30:1111-1126. PubMed Abstract OpenURL

  45. Steinberg EP, Tielsch JM, Schein OD, Javitt JC, Sharkey P, Cassard SD, Legro MW, Diener-West M, Bass EB, Damiano AM, Steinwachs DM, Sommer A: The VF-14: An index of functional impairment in patients with cataract.

    Arch Ophthalmol 1994, 112:630-638. PubMed Abstract OpenURL

  46. Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD: Development of the 25-item National Eye Institute Visual Function Questionnaire.

    Arch Ophthalmol 2001, 119:1050-1058. PubMed Abstract | Publisher Full Text OpenURL

  47. Lundström M, Roos P, Jensen S, Fregell G: Catquest questionnaire for use in cataract surgery care: description, validity, and reliability.

    J Cataract Refract Surg 1997, 23:1226-1236. PubMed Abstract OpenURL

  48. Massof RW: Application of stochastic measurement models to visual function rating scale questionnaires.

    Ophthalmic Epidemiol 2005, 12:103-124. PubMed Abstract | Publisher Full Text OpenURL

  49. de Boer MR, Moll AC, de Vet HC, Terwee CB, Volker-Dieben HJ, van Rens GH: Psychometric properties of vision-related quality of life questionnaires: a systematic review.

    Ophthalmic Physiol Opt 2004, 24:257-273. PubMed Abstract | Publisher Full Text OpenURL

  50. Owen CG, Rudnicka AR, Smeeth L, Evans JR, Wormald RPL, Fletcher AE: Is the NEI-VFQ-25 a useful tool in identifying visual impairment in an elderly population?

    BMC Ophthalmol 2006, in press. OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2415/6/25/prepub