Email updates

Keep up to date with the latest news and content from BMC Family Practice and BioMed Central.

Open Access Research article

Measuring access to primary care appointments: a review of methods

Wendy Jones2, Glyn Elwyn2*, Peter Edwards1, Adrian Edwards2, Melody Emmerson2 and Richard Hibbs1

Author Affiliations

1 Capricorn Research Network, Croesnewydd Hall, Wrexham Technology Park, Wrexham, LL13 7YP, UK

2 Department of Primary Care, The Clinical School, University of Wales Swansea, SA2 8PP, U.K

For all author emails, please log on.

BMC Family Practice 2003, 4:8  doi:10.1186/1471-2296-4-8

The electronic version of this article is the complete one and can be found online at: http://www.biomedcentral.com/1471-2296/4/8


Received:15 April 2003
Accepted:7 July 2003
Published:7 July 2003

© 2003 Jones et al; licensee BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.

Abstract

Background

Patient access to primary care appointments is not routinely measured despite the increasing interest in this aspect of practice activity. The generation of standardised data (or benchmarks) for access could inform developments within primary care organisations and act as a quality marker for clinical governance. Logically the setting of targets should be based on a sound system of measurement. The practicalities of developing appropriate measures need debate. Therefore we aimed to search for and compare methods that have been published or are being developed to measure patient access to primary care appointments, with particular focus on finding methods using appointment system data.

Method

A search and review was made of the primary care literature from 1990 to 2001, which included an assessment of online resources (websites) and communication with recognised experts. The identified methods were assessed.

Results

The published literature in this specific area was not extensive but revealed emerging interest in the late 1990s. Two broad approaches to the measurement of waiting times to GP appointments were identified. Firstly, appointment systems in primary care organisations were analysed in differing ways to provide numerical data and, secondly, patient perceptions (reports) of access were evaluated using survey techniques. Six different methods were found which were based on appointment systems data.

Conclusion

The two approaches of either using patient questionnaires or appointment system data are methods that represent entirely different aims. The latter method when used to represent patient waiting times for 'routine' elective appointments seems to hold promise as a useful tool and this avoids the definitional problems that surround 'urgent' appointments. The purpose for which the data is being collected needs to be borne in mind and will determine the chosen methods of data retrieval and representation.

Keywords:
access; primary care; measurement; appointments; waiting times.

Background

Primary care is under scrutiny along with other public services to improve access to its users. Access in primary care is typically conceptualised as the achievable access to appointments with clinical professionals, although it is not routinely measured in most practices. It is widely recognised that it represents an important dimension in determining the quality of care [1,2]. Nevertheless, it is also known that waiting times can vary widely in differing localities and countries, from patients being seen the same day to a wait of several weeks. Most primary care organisations have no more than a perception of variable demand and no method of comparing fluctuating levels of access to appointments within or between practices. Measuring patient access could generate useful information for patients, clinicians and practice managers. Demand management initiatives and ways to optimise access could then be audited bearing in mind the impact of such initiatives on opportunity costs [3-5].

There are potential difficulties, recognised by a recent discussion document [6], that have been accentuated by policy and political influences. The NHS Plan in the UK [7] suggested that patients should have access to primary care services within 48 hours, but the concept lacked sufficient definition. The proposed new GMS contract includes an optional 48 hour target for access to GP appointments. But what exactly should we measure? Access can be measured at many different interfaces, from the wait for traditional services such as appointments with a clinician to the alternative solutions of nurse triage, nurse led clinics, telephone advice (including NHS Direct) or electronic mail responses. Differing interpretation of terms can also cause confusion. How soon for example should a problem that is defined by a patient as urgent be seen in general practice? The perception of urgent differs between patients themselves [8] and between doctors and patients. The concept of routine appointments is easier to define and quantify. A proposed measure is the waiting time for the next available routine appointment but this provokes debate about whether this should be practice-based or specified for individual clinicians. If the latter, factors such as part-time working, practitioner popularity, the creation of multiple review appointments are likely to rapidly diminish a clinician's accessibility. How transparent would a practice want to be about such data and how useful or acceptable would it be to publish information at the clinician level? [9]

Recognising the complex nature of this issue, we set out to review the literature. Our main aim was to search the international primary care literature for methods that had been, or were being developed to measure access to GP appointments, focusing on measures using appointment system data. Once identified, existing methods (tools, scales or other instruments) would be compared, with specific attention given to the type and levels of access they aimed to assess.

Method

Preliminary searches indicated that this area did not have an extensive or long-standing research literature; most articles had been published after 1998. A broad but systematic search process was designed to allow for a poorly indexed publication pool. Medline, PubMed, Clin Psyc and ASSIA were searched for relevant publications between 1990 and 2001. The following MeSH terms were used: family practice, health service accessibility (organisation and administration, statistics and numerical data, standards, trends, methods, manpower), appointments and schedules (waiting lists), research design, health service needs and demand, weights and measures, quality of healthcare, management audit, patient satisfaction, health service needs and demand. In addition the following keywords were used: general practice, access, appointments (same day, urgent, routine) appointment systems, measurement, measures, tools, scales, demand, availability, audit and waiting times. Terms were used both singly and in combination. Title searches were used to increase the sensitivity. All citations and abstracts were appraised for relevance and full articles selected for examination by two researchers independently (GE and WJ). Key authors were contacted directly [1,12,16,18,21,25,27] and searches conducted on their previous work. Departments of General Practice in Universities in the United Kingdom were also asked to send details of any relevant research. Conference literature [21] and non-peer reviewed literature obtained from websites was also appraised. Relevant websites were identified using http://omni.ac.uk webcite and the search terms health service delivery, access to primary care and general practice. The following sites were reviewed: the Royal College of General Practitioners [6], the National Primary Care Research and Development Centre [1], The National Primary Care Development Team [10] and the Centre for Innovation in Primary Care [11].

Studies or articles were included in the review if they described tools, scales, questionnaires or other methods of measuring actual patient access to appointments. We also included descriptions of methods that were currently being developed in this field, provided they had undertaken pilot studies and had completed one data collection exercise. Articles were excluded if they were purely editorial.

Results

A total of 1763 citations were initially identified and 38 articles retrieved for detailed assessment from the Pubmed and the Medline searches. Clin Psych and ASSIA searches provided some overlap but no new relevant material. The most helpful pointers to relevant publications were obtained from the website searches and personal communications rather than the traditional search engines. Two broad approaches to the measurement of patient access were identified. Firstly, appointment systems in organisations were analysed in differing ways to provide numerical data and, secondly, patient perceptions (reports) of access were evaluated using survey techniques.

Methods using appointment system data

Table 1 summarises the six identified methods that were based on appointment system data.

Table 1. Comparison of methods based on appointments systems to measure access to primary care

Three methods determine appointment availability and/or the satisfaction of demand on a daily basis but do not measure the days wait for appointments. Campbell measured clinician availability by recording the number of provided appointments at the beginning of each day plus the number of these un-booked at this time [12]. At the end of the day numbers seen and numbers of extras were counted, and adjusted for practice list size. Kendrick and Kerry recorded the number of available appointments at the beginning of the day and the number of extras seen at the end [13]. Ledlow also suggested a categoriations system for recording differing levels of unmet daily demand in a military medical service [14]

Three methods measure access as days waited by representing appointment system data. A computerised package, NEMAS [15,16] enables practices to audit four areas: practice service provision (including 'appointment availability' and patient satisfaction), chronic disease management, drug monitoring and significant event analysis. The method calculates the mean time waited in days plus the minimum and maximum patient waits. Data can be presented for the whole practice or for individual GPs. Data entry involves using electronic forms to record the date of appointment request, the date the patient was seen, whether it was an elective, forced (i.e. next available clinician) or urgent appointment, the clinician requested and the clinician actually consulted. The costs of collecting data has varied considerably depending on which staff member is employed for the task [17]. Transfer of data from the practice system can be automatic with compatible systems (personal communication).

The National Primary Care Development Team is measuring access as part of the adoption of the system of 'Advanced Access', a system developed in the USA [18,19], as a response to patient waiting times of 4 – 6 weeks for routine appointments in primary care [20]. This method collects data on one day per week (which can vary) at 12 noon. The number of days to the third available routine appointment for each clinician is recorded and a median figure calculated to represent an access score for the specified week. Over a month, the average of four median values is taken to represent a monthly access score. The third appointment is chosen, rather than the first, in order to negate the effect of sudden cancellations which otherwise could give false impressions of availability if at the moment of measuring there is one sudden cancellation in a schedule that is otherwise booked for several days ahead. The third appointment has been found by trial and improvement to best represent the actual waits involved (personal communication). Embargoed appointments are not included. A system of weighting has been developed [21] for part time workers to enable their scores to be incorporated.

The Access Response Index (AROS) was developed as a rapidly calculable measure of organisational access [22]. This index is derived by counting the number of days until the next available routine appointment, with any clinician, once during every normal working day. The data is recorded at four pm – a time chosen to avoid the influence of embargoed appointments that many organisations use to maintain urgent same day availability. The results are plotted on a graph, the daily fluctuation represented by the raw data is smoothed to a demonstrate trends by calculating a 5-day moving average (a data point which is the mean of every successive 5-day group). An example of the data produced by this measure is shown in Figure 1.

thumbnailFigure 1. AROS scores for routine appointment availability (data from 11 practices)

Methods using patient questionnaires

Four patient experience questionnaires were found that contained access assessments and the relevant items are outlined in Table 2. The General Practice Assessment Survey (GPAS) [23] was developed by the National Primary Care Research and Development Centre, by adapting the Family Practice Assessment Survey (FPAS)[24]. A second survey named Europep [25,26] has been validated in 10 European countries. Thirdly Baker describes the use of a validated surgery satisfaction questionnaire (SSQ) that he developed [27,28] and finally Grogan's patient Satisfaction Questionnaire contains two sections that ask about access and appointment availability[29].

Table 2. Patient survey instruments: items used to determine access perceptions

Discussion

Principal findings

This review of access measurement reveals the heterogeneous nature of the methods and the lack of any widely accepted conceptualisation of patient access. Identified measures are either practice centred using appointment data or patient orientated via surgery satisfaction questionnaires. It is clear that these two methods represent entirely different aims. It is not possible for episodic patient surveys to provide data that has enough currency or accuracy to inform organisational responses to patient demand.

Also demonstrated are inherent problems over definition of terms – defining what is to be measured and setting of targets. There appear to be three appointment categories over and above emergencies (which by definition cannot be given 'appointments'). Appointments for healthcare could be categorised into urgent, soon, and elective. Urgent appointments are typically seen as requests for same-day consultations. The soon category would fit problems that should be seen within two or three days to prevent escalation or symptom prolongation. Finally, routine or elective appointments suit individuals who value an agreed time window over other factors. The methods that represent patient waiting times for 'routine', i.e. elective appointments seem to hold promise as they avoid the definitional problems that surround 'urgent' appointments, and the different views that patients, clinicians and others have about 'urgency'. The task of deciding whether to represent access profiles for organisations or for individual clinicians also needs careful consideration. It may be more feasible (and less threatening) to routinely measure organisational access, especially if the data is to be used for benchmarking purposes.

To attempt to measure access means to obtain meaningful data from a dynamic system that is not always in equilibrium. The access experienced by an individual varies determined by demand, adequate appointment provision, sudden cancellations and block release of held appointments. We have distinguished two approaches to overcome this, either for a full statistical analysis using compatible systems or data retrieval software, or to use a simple snapshot method, deciding whether data-smoothing methods such as daily moving averages or aggregated weekly median scores are the best portrayal of overall access patterns.

Strengths and weaknesses of the study

Multiple search methods were used to ensure that the breadth of literature and online resources were examined as systematically as possible. The searches proved difficult and reflect the emergent status and the diversity of terms used in this area. We may have overlooked methods developed in other healthcare systems.

Implications of the findings to healthcare services and research

The lack of a widely agreed measurement method to represent patient access to primary care services will make it impossible for practices to compare their response to patient demand with any degree of certainty. The 'third appointment' system is the most widely used method and is currently supported by the National Primary Care Development Team in England (but has no equivalent support in Scotland, Wales or Northern Ireland). It is however a relatively complex manipulation of appointment system data, and incorporates individual clinician availability. It seems from first principles that the important requirements of a tool designed to measure a dynamic concept such as patient access is simplicity and ease of regular data collection, so that longitudinal data patterns capable of indicating trends in organisations can be generated rather than data on individual clinician availability.

A recent survey compared mechanisms used to manage requests for same day appointments[30]. Murray's proposal of doing today's work today (Advanced Access) eliminates appointment categories and the work involved in negotiating urgency by dealing with virtually all demand on the day it arises[31]. Access is an important determinant of healthcare quality but what are the implications of this approach to the balance of overall quality? Continuity of care, whilst not important for some problems is desirable for others [32,33]. Too drastic a shift in favour of access is likely to be at the cost of reduced continuity and a diminution of other services, such as screening and chronic disease management. Measures of organisational quality need to be aware of the dangers of focusing too much on one dimension, and should work towards the creation of measures that balance scores across internal and external requirements [34].

An effective consultation with a well trained clinician who knows the patient and who has access to a well structured longitudinal record will probably remain a gold standard and the issue of immediate 'access' should not be elevated above all the other components in this equation. Nevertheless, it seems prudent to develop measures that provide a better understanding of patient access to organisations with similar resources. Measuring the interval to the next available routine appointment, whilst measuring the measurable [35] will, if a consistent standard can be agreed, provide benchmark data, and mark an important step towards a compendium of methods to assess quality in primary care.

Conclusion

The two approaches of either using patient questionnaires or appointment system data to measure access are methods that represent entirely different aims. The latter method when used to represent patient waiting times for 'routine' elective appointments seems to hold promise as a useful tool and this avoids the definitional problems that surround 'urgent' appointments. The purpose for which the data is being collected needs to be borne in mind and will determine the chosen methods of data retrieval and representation

Competing Interests

The authors of this article have piloted the AROS Index.

Authors' Contributions

Peter Edwards, Melody Emmerson and Glyn Elwyn were involved in the development of the Aros index. Wendy Jones and Glyn Elwyn conducted the Aros pilot study and the literature review with contributions from Adrian Edwards and Richard Hibbs.

Acknowledgements

The partners, manager and receptionists at Ely Bridge Surgery. The AROS Pilot Study Group: New Park Surgery, Talbot Green; Porthcawl Group Practice; Woodlands Medical Centre, Ely, Cardiff; Old School Surgery, Pontyclun; Greenmount Surgery, Ely, Cardiff; Salisbury Rd Surgery, Barry; Llynfi Surgery, Maesteg; Oldcastle Surgery, Bridgend; Park Lane Surgery, Tonyrefail; Meddygfa Teilo, Llandeilo; The Surgery, Overton-on-Dee, Wrexham.

This work was sponsored by CAPRICORN which receives funding from the Welsh Assembly Government

References

  1. Roland M, Holden J, Campbell S: Quality Assessment for General Practice: supporting clinical governance in primary care groups. [http://www.npcrdc.man.ac.uk/] webcite

    Manchester: National Primary Care Research and Development Centre 1998. OpenURL

  2. Campbell SM, Roland MO: Defining quality of care.

    Soc Sci Med 2000, 51:1611-25. PubMed Abstract | Publisher Full Text OpenURL

  3. Rogers A, Entwistle V, Pencheon D: A patient led NHS: managing demand at the interface between lay and primary care.

    BMJ 1998, 316:1816-9. PubMed Abstract | Publisher Full Text OpenURL

  4. Pencheon D: Matching demand and supply fairly and efficiently.

    BMJ 1998, 316:1665-7. PubMed Abstract | Publisher Full Text OpenURL

  5. Gillam S, Pencheon D: Managing demand in general practice.

    BMJ 1998, 316:1895-8. PubMed Abstract | Publisher Full Text OpenURL

  6. Royal College of General Practitioners: Access to general practice based primary care. [http://www.rcgp.org.uk/] webcite

    2001.

    Accessed March 2001. London

  7. The Stationery Office: The NHS Plan.

    London: The Stationery Office 2000. OpenURL

  8. Campbell JL: Patient's perceptions of medical urgency, does deprivation matter?

    Fam Pract 1999, 16:28-32. PubMed Abstract | Publisher Full Text OpenURL

  9. Marshall MN, Shekelle PG, Leatherman S, Brook RH: The public release of performance data: what do we expect to gain?

    JAMA 2000, 283:1866-74. PubMed Abstract | Publisher Full Text OpenURL

  10. The National Primary Care Development Team [http://www.npdt.org/] webcite

  11. Centre for Innovation in Primary Care [http://www.innovate.org.uk] webcite

  12. Campbell JL: General practitioner appointment systems, patient satisfaction, and use of accident and emergency services: a study in one geographical area.

    Fam Pract 1994, 11:438-445. PubMed Abstract OpenURL

  13. Kendrick T, Kerry S: How many surgery appointments should be offered to avoid undesirable number of "extras"?

    BJGP 1999, 49:273-6. OpenURL

  14. Ledlow GR, Bradshaw DM, Shockley C: Primary care access improvement: an empowerment interaction model.

    Military Medicine 2000, 165:390-4. PubMed Abstract OpenURL

  15. NEMAS [program]:

    Glasgow: West of Scotland Postgraduate Medical Education Board and MI Technology Ltd. 2000. OpenURL

  16. Lough M, Wilmot M: Combining audit and information technology in the west of Scotland.

    Practice Computing 1996, Autumn:26-8. OpenURL

  17. Lough JRM, Wilmot M, Murray TS: Supporting practice-based audit: a price to be paid for collecting data.

    BJGP 1999, 49:793-5. OpenURL

  18. Murray M, Tantau C: Same-day appointments: exploding the access paradigm.

    Family Practice Management 2000, 7:45-50. PubMed Abstract OpenURL

  19. Murray M, Tantau C: Redefining open access to primary care.

    Managed Care Quarterly 1999, 7:45-55. PubMed Abstract OpenURL

  20. Murray M, Tantau C: Must patients wait?

    Journal of Quality Improvement 1998, 24:423-5. OpenURL

  21. Oldham J: Advanced Access in Primary Care.

    Manchester: National Primary Care Development Team 2001. OpenURL

  22. Elwyn G, Jones W, Emmerson M, Edwards P, Edwards A: Developing a measure for benchmarking access in primary care: Access Response (AROS).

    Journal of Clinical Evaluation in Clinical Practice 9,1:33-37. OpenURL

  23. Ramsay J, Campbell JL, Schroter S, Green J, Roland M: The General Practice Assessment Survey (GPAS) tests of data quality and Measurement properties.

    Fam Pract 2000, 17:372-9. PubMed Abstract | Publisher Full Text OpenURL

  24. Safran DG, Kosinski M, Tarlov AR, Rogers WH, Taira DH, Lieberman N, Ware JE: The Primary Care Asessment Survey:tests of data quality and measurement performance.

    Med Care 1998, 36:728-39. PubMed Abstract | Publisher Full Text OpenURL

  25. Grol R, Wensing M, Mainz J, Jung HP, Ferreira P, Hearnshaw H, Hjortdahl P, Olesen F, Reis S, Ribacke M, et al.: Patients in Europe evaluate general practice care: an international comparision.

    Br J Gen Pract 2000, 50:882-7. PubMed Abstract OpenURL

  26. Grol R, Wensing M: Patients evaluate family pracitce. In The Europep instrument. Nijmegen: Centre for Health Care Quality; 2000. OpenURL

  27. Baker R: The reliability and criterion validity of a measure of patients satisfaction with their general practice.

    Family Practice 1991, 8:171-7. PubMed Abstract OpenURL

  28. Baker R, Streatfield J: What type of general practice do patients prefer? Exploration of practice characteristics influencing patient satisfaction.

    Br J Gen Pract 1995, 45:654-659. PubMed Abstract OpenURL

  29. Grogan S, Conner M, Willits D, Norman P: Development of a questionnaire to measure patients' satisfaction with general practitioners' services.

    BJGP 1995, 45:525-9. OpenURL

  30. Luthra M, Marshall MN: How do general practices manage requests from patients for "same-day" appointments? A questionnaire survey.

    BJGP 2001, 51:39-41. OpenURL

  31. Murray M: Patient care: access.

    BMJ 2000, 320:1594-6. PubMed Abstract | Publisher Full Text OpenURL

  32. Stott NCH, Kinnersley P, Elwyn G: Measuring general practice-based primary care: generic outcomes.

    Family Practice 1997, 14:486-491. PubMed Abstract | Publisher Full Text OpenURL

  33. Guthrie B, Wyke S: Does continuity in general practice really matter?

    BMJ 2000, 321:734-6. PubMed Abstract | Publisher Full Text OpenURL

  34. Kaplan RS, Norton DP: Using the balanced scorecard as a strategic management system.

    Harvard Business Review 1996, 75-85. OpenURL

  35. Campbell J, Proctor S: A developmental performance framework for primary care.

    International Journal of Healthcare Quality Assurance 1999, 12:279-286. Publisher Full Text OpenURL

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2296/4/8/prepub