In 2002 the Department of Health committed to increasing the permanent capacity for elective surgery (and diagnostic services) in England by establishing more Diagnosis and Treatment Centres 1. The aim was not only to reduce waiting times for routine procedures but also to expand choice and to increase productivity through innovative models of care. Unlike the ten existing NHS centres (including one run jointly with a private provider), expansion was partly to be achieved by encouraging private companies to establish facilities employing staff not currently working for the NHS, thus creating additional capacity in England. The first of these Independent Sector Treatment Centres (ISTCs), focusing on low risk ophthalmic, orthopaedic and day surgery, opened in 2003. By October 2004, six centres had been established and had treated 16 000 patients 2.

Following some concerns about the policy of introducing ISTCs 34, the House of Commons Health Committee undertook a review in spring 2006. In addition to receiving evidence about poor value for money, adverse impact on surgical training, loss of continuity of care, lack of adequate clinical governance, and jeopardising the viability of existing NHS providers, clinicians reported seeing serious complications in patients who had been treated in ISTCs 5678. However, as some of them acknowledged, meaningful comparisons with NHS providers could not be made both because of the poor quality of the data collected by ISTCs 9 and the historical lack of routine data on outcome collected by the NHS. When the Health Committee reported in July 2006, it agreed "that there was no hard, quantifiable evidence to prove that standards in ISTCs differed from those in the NHS" and "recommended that comparable and standardised data be collected" 10. This was welcomed by clinicians 11, some of whom had been advocating such an approach 12.

Meanwhile, in April 2006 the Secretary of State for Health pre-empted the Health Committee's report by announcing the establishment of "a wide-ranging clinical audit" to be carried out by the Healthcare Commission 13. However, inevitably, when it appeared in July 2007, it too confirmed the lack of available routine data with which to assess outcomes 14.

Fortunately, in 2004 the Department of Health's Economic and Operational Research Division foresaw the need for meaningful comparisons of the performance of providers in the emerging world of choice. They commissioned a project to review international evidence on patient-reported outcome measures for five elective surgical procedures and to establish a demonstration project with three objectives: to test the feasibility of data collection in ISTCs and NHS providers; to establish how best to analyse the data; and to explore how best to present inter-provider comparisons. As only two ISTCs agreed to take part, the DH Commercial Directorate subsequently funded data collection from an additional four centres. These combined data provide the first opportunity to address some of the concerns of clinicians, the Health Committee, independent sector providers and policy-makers.

In this paper our objectives are to compare the case-mix of ISTCs and NHS providers (acute hospitals and treatment centres) and patient-reported outcomes in terms of change in health status/quality of life, post-operative complications and overall result of surgery. Comparisons of individual centres will appear elsewhere.

Twenty-six providers (13 NHS hospitals, 7 NHS treatment centres, 6 ISTCs) participated. Ethical approval was granted by the Wales MREC. Consecutive patients (apart from those with cognitive impairment, poor sight, literacy or language comprehension problems) were eligible. Recruitment, which lasted for up to six months between January 2006 and April 2007, was carried out by nursing or administrative staff who explained what the study involved (completion of a pre and post-operative questionnaire) and obtained written consent. This took place either in a pre-operative assessment clinic or on admission for surgery. The pre-operative questionnaire that included: age, sex and postcode (to determine an Index of Multiple Deprivation 15 – higher scores indicate more deprived); duration of symptoms; history of previous similar surgery; general health status (five categories); co-morbidities 16; and the EQ-5D, a generic measure of health-related quality of life 17. In addition, a disease-specific measure of functional status/health-related quality of life was included: the VF14 for cataract surgery 18; the Aberdeen Varicose Vein Questionnaire – AVVQ 19; the Oxford Hip Score – OHS 20; and the Oxford Knee Score – OKS 21. As no specific measure for hernia repair has been validated, the Physical Component Summary (PCS) of the SF-36 was used 22. Higher scores represent better outcomes for the EQ-5D, the PCS and the VF14. The reverse is true for the OHS, OKS and the AVVQ.

Post-operative questionnaires were mailed three (for hernia, cataract and varicose vein surgery) or six months (joint replacement) after surgery to patients' homes from the Royal College of Surgeons of England. Non-responders were sent a reminder letter and replacement questionnaire five weeks after the original mailing. The questionnaires contained: general health status; EQ-5D; a relevant disease-specific instrument; a question relating to complications – "Did you experience any of the following problems after your operation?" (allergy or reaction to drug, and urinary, bleeding or wound problems) 23; and a global evaluation – "How would you describe the results of your operation?".

Summary statistics of pre-operative characteristics are reported for each operation group by treatment sector. Post-operative response bias was investigated by using t-tests to compare the pre-operative disease-specific instrument scores of patients who did and did not respond. The primary outcomes were the post-operative generic and disease-specific PROM scores, adjusted for pre-operative patient characteristics (age, sex, socioeconomic status, comorbidities, duration of symptoms, previous similar surgery, general health status, EQ-5D). Secondary outcomes were the risk-adjusted proportions of patients reporting any complications and rating their operation as excellent, very good, or good.

Risk-adjustment of outcomes was conducted using linear regression (logistic regression for post-operative complication incidence and overall operation success) with pre-operative PROM scores and other patient characteristics (age, sex, general health status, duration of symptoms, number of comorbidities, Index of Multiple Deprivation, previous similar surgery) being potential risk factors.

To account for possible clustering of patient outcomes within hospitals, we used robust standard errors. All P values are 2-sided and P values lower than 0.05 were considered a statistically significant result. Stata software was used for all calculations 24.

The case-mix of patients treated in ISTCs differs from that in NHS providers, in line with the intention of the contracts. Caution is needed in interpreting the observation that patients treated in ISTCs reported slightly better outcomes as very few ISTCs participated, case-mix adjustment might have been insufficient, and patients' reports might have been biased as they were more likely to be satisfied with the way they were treated.

The author(s) declare that they have no competing interests.

JB, NB and JvdM conceived and designed the study; LJ and LC managed the data collection; JL analysed the data; NB drafted the paper and all authors contributed to the final draft. NB acts as guarantor.