Email updates

Keep up to date with the latest news and content from BMC Health Services Research and BioMed Central.

Open Access Highly Accessed Research article

Incorporating statistical uncertainty in the use of physician cost profiles

John L Adams, Elizabeth A McGlynn, J William Thomas and Ateev Mehrotra*

BMC Health Services Research 2010, 10:57  doi:10.1186/1472-6963-10-57

PubMed Commons is an experimental system of commenting on PubMed abstracts, introduced in October 2013. Comments are displayed on the abstract page, but during the initial closed pilot, only registered users can read or post comments. Any researcher who is listed as an author of an article indexed by PubMed is entitled to participate in the pilot. If you would like to participate and need an invitation, please email, giving the PubMed ID of an article on which you are an author. For more information, see the PubMed Commons FAQ.

Rand Study Response from NCQA

Apoorva Stull   (2010-07-27 10:39)  National Committee for Quality Assurance email

To the Editor:

The recent study by Adams et al entitled Incorporating statistical uncertainty in the use of physician cost profiles adds significant information to our understanding of reliability related to the use of measures of cost and resource use in physician practices. NCQA agrees overall with their conclusion that setting standards for reliability of physician cost profiles is important. We developed NCQA’s Physician and Hospital Quality (PHQ) certification program, which aligns with standards for physician level measurement set forth in the Patient Charter for Physician Measurement of the Consumer Purchaser Disclosure Project. This charter resulted from the 2008 New York Attorney General’s settlement of court action with health plans over their doctor profiling programs. While the authors cite the 2008 version of our PHQ standards, their description of the degree of statistical testing is actually from a prior, 2006 version and thus misrepresents current standards.

The PHQ standards require organizations to be transparent about how they use statistical testing and sets the actual standards required for testing. Specifically, PHQ standards require that plans using physician cost profiles as the basis for public reporting, cost sharing and network arrangements must: take quality data into account as well as cost data and take action on cost data when comparisons are based on a confidence interval of 90% or measure reliability of at least .70 based on methodology developed by John Adams and others. For cost measurement, there is no option in our current (2008) standards to use only sample size as a substitute for confidence intervals or a formal test of reliability. NCQA does a formal review of the methodology used by plans in meeting these requirements.

The assertion of Adams et al, that few plans use formal statistical testing in their physician assessment programs is not consistent with the fact that well over a hundred plans have been certified by NCQA as meeting the PHQ standards including the use of confidence intervals or formal reliability testing, which provides significant protection against the misclassification potential that RAND has identified. A complete list of certified organization is available at Thus while the article adds important information on reliability of measures of cost and resource use, the approach of many health plans has changed substantially since the citations used by the authors, at least in part due to the introduction of the NCQA PHQ standards.

Margaret E. O’Kane

Competing interests

None declared


Post a comment