Diagnosis Related Group (DRG) payment is preferred by healthcare reform in various countries but its implementation in resource-limited countries has not been fully explored.
This study was aimed (1) to compare the characteristics of hospitals in Thailand that were audited with those that were not and (2) to develop a simplified scale to measure hospital coding practice.
A questionnaire survey was conducted of 920 hospitals in the Summary and Coding Audit Database (SCAD hospitals, all of which were audited in 2008 because of suspicious reports of possible DRG miscoding); the questionnaire also included 390 non-SCAD hospitals. The questionnaire asked about general demographics of the hospitals, hospital coding structure and process, and also included a set of 63 opinion-oriented items on the current hospital coding practice. Descriptive statistics and exploratory factor analysis (EFA) were used for data analysis.
SCAD and Non-SCAD hospitals were different in many aspects, especially the number of medical statisticians, experience of medical statisticians and physicians, as well as number of certified coders. Factor analysis revealed a simplified 3-factor, 20-item model to assess hospital coding practice and classify hospital intention.
Hospital providers should not be assumed capable of producing high quality DRG codes, especially in resource-limited settings.
Since 2001, the Universal Coverage (UC) scheme has provided health benefits to approximately three quarters of Thailand's citizens. The scheme is financed from general taxation and administered by the National Health Security Office (NHSO), under the supervision of the Public Health Minister. Hospital providers are mainly from the public sector and are paid for outpatient and preventive services based on prospective capitation whereas the Diagnosis Related Group (DRG)-based retrospective payment was used to compensate for the cost of inpatient care.
The DRG system can work well if the diagnosis and procedure codes could reflect both the patient's clinical condition and the actual cost of care incurred by hospital providers. In addition, coding practice in hospitals has to be reliable and capable of producing consistent and reliable coding quality. In an ideal setting, physicians would carefully review relevant clinical information to prepare a discharge summary, which would then be used by certified coders to produce appropriate diagnosis and procedure codes to be submitted for reimbursement. However, such an ideal condition is unlikely, at least in resource-limited settings like Thailand.
In a related qualitative study, we detailed the variation of coding practice in 10 hospitals and then defined the concept of Hospital Coding Practice as comprising elements of both structure and process . In terms of structure, we identified at least eight health care professional disciplines (Medical Statisticians, Nurse, Physician, Public Health Staff/Paramedics, Medical Record Staff, Information Technology Staff, Finance/Accounting Staff, and others) as well as IT infrastructure specifically relevant to coding practice. We also described seven major steps of the coding process (Discharge Summarization, Completeness Checking, Diagnosis and Procedure Coding, Code Checking, Relative Weight Challenging, Coding Report, and Internal Summary and Coding Audit). The findings demonstrated that coding practice is not a simple two-step activity as had been assumed before by payers, but rather a multi-step process that involves many overlapping responsibilities across health care professional disciplines, especially in a resource-limited setting .
Although DRG has been applied in many countries and has been improved over the decades, it still is an imperfect system and a number of concerns have therefore been raised. Discrepancies between the submitted codes and the information in medical records revealed during the coding audit--especially when the submitted codes can result in larger reimbursements than would be consistent with the actual condition--have triggered a number of concerns about quality of medical records, cooperation of physicians, knowledge and skill of coders, as well as hospital intentions to "game the system" (also known as "DRG Creep") . While most of these issues can be objectively verified, the last concern seems to be both the most important and the most difficult to measure.
Hospitals in Thailand are required to check for the accuracy and completeness of the data using the National Health Security Office (NHSO) standard guidelines ; however, errors have been frequently reported. Each hospital is therefore required to check the data before submission, and penalties--financial and non-financial--are imposed if errors are found. In 2008, the Bureau of Claims and Medical Audit (BCMA) conducted the Summary and Coding Audit on 57,828 medical records of 931 hospitals in 75 provinces (SCAD 2008). Hospitals were selected based on pre-specified criteria as presented in Table 1. These 'SCAD hospitals' were then regarded negatively as they were suspected of either having poor data quality or trying to manipulate the system.
Table 1. Inclusion criteria for SCAD 2008
Hospital coding practice can be affected by some factors beyond a hospital's control . Simply because the hospitals were chosen to be audited based on the above inclusion criteria does not necessarily mean they intended to cheat or were incapable of producing quality coding. Understanding what might distinguish SCAD hospitals from the non-SCAD counterparts was an appropriate and feasible next step. This nationwide hospital survey has two main objectives: (1) to describe and compare the characteristics of both groups of hospitals and (2) to develop a simplified scale to measure hospital coding practice.
Based on the case study findings , a questionnaire was developed to comprise three sections. Section 1 asked about general demographics of the respondents and of the hospitals. Section 2 aimed to explore the hospital coding structure; various types of resources in the hospital were asked. Section 3.1 explored how, for each step of the hospital coding process, primary and secondary responsibility was assigned to, or assumed by, a particular health care professional discipline. Face validity of the questionnaire was assessed by presenting the findings to data-coding experts. The general demographic information was verified with national hospital registry to ensure content validity. To ensure comprehensibility and feasibility, a pilot test was conducted among 3-5 graduate students with relevant professional background as well as among a few public and private hospitals; feedback was then used for revision. Forward and backward translation was done to ensure its conceptual and linguistic equivalence.
Section 3.2 contained a set of opinion-oriented items to reflect on the current hospital coding practice. A pool of 63 items were identified from the interviews with the 10 hospitals in the case study . They were categorized into group A to K, according to their relevance to each step of the hospital coding process (A1-A19: Non-specific; B1-B8: Discharge Summarization; C1-C2: Completeness Checking; DE1-DE10: Diagnosis and Procedure Coding; F1-F6: Code Checking; G1-G5: Relative Weight Challenging; H1-H3: Coding Report; I1-I3: Medical Record Audit; J1-J3: Internal Summary Audit; K1-K3: Internal Coding Audit). These items were presented as declarative sentences, followed by response options that indicate varying degree of agreement with the statement. A five-point Likert scale format (-2 = Strongly Disagree; -1 = Disagree; 0 = Neutral; +1 = Agree; +2 = Strongly Agree) was considered the most appropriate, based on the feedback from our pilot test, for the respondents' ability to discriminate meaningfully .
Site & Study Population
The study population of interest initially included all 1,356 hospitals in Thailand, of which 931 hospitals participated in the UC scheme and their audit results were contained in the SCAD 2008. Thirty-five hospitals that provided services to highly specific population groups (such as drug addicts, prisoners, special psychiatric patients, organizational employees) or had special kind of financial support, and 11 hospitals without verified postal address were excluded. After exclusion, there were 1,310 target hospitals, comprising 920 SCAD and 390 non-SCAD hospitals (Figure 1). The respondents targeted for the study were hospital staff who were responsible for the diagnosis and procedure coding process, as identified by the hospital director.
Figure 1. Survey strategy.
The questionnaire, a cover letter, and a prepaid return envelope were mailed to the target hospitals. Hospitals were asked to return the questionnaire within four weeks, after which two follow-up phone calls were done. Non-respondents were defined as those who did not return the questionnaire within two weeks after the second follow-up call. This study was determined by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board as not human subjects research (IRB# 00002096).
For responses to the Section 1 (General Demographics), Section 2 (Hospital Coding Structure), and Section 3.1 (Hospital Coding Process), Pearson's chi-square and Student's t-test were used for analyzing categorical and continuous variables, respectively. Exploratory factor analysis (EFA) was used to analyze the responses in Section 3.2 (Hospital Coding Practice Scale). EFA is a technique used to explain covariance among observed random variables in terms of fewer unobserved random variables named factors. It helps to generate a hypothesis in such a way that the investigation of the relationships between manifest variables and factors is not based on any prior assumptions about which manifest variables are related to which factors .
The factor analysis was done to identify optimal number of factors, determined by Kaiser-Guttman Criterion (Number of eigenvalues > 1) , scree test , and parallel analysis . Items with high uniqueness, defined as larger than 0.70, were removed whereas the remaining items were retained within the factors that showed high factor loadings. According to the 'rule of thumb' in confirmatory factor analysis, loadings should be at least 0.70 because it corresponds to about half of the variance of the variable being explained by the factor of interest. However, some researchers suggest lower levels of cut-points such as 0.60  or as low as 0.40  for EFA. We also performed initial reliability test and item-based statistics in conjunction with EFA . Stata/SE Version 10 (Stata Corp.) was used for all statistical calculations.
The overall response rate was 37.56% with a well-balanced geographical distribution. Of those who responded, 18 hospitals were excluded because they were no longer in operation. The sample therefore consisted of 374 SCAD and 100 non-SCAD hospitals. The SCAD hospitals were significantly more likely to respond than Non-SCAD hospitals (40.87% vs 29.74%; OR 1.63; 95% CI: 1.26-2.12; p = 0.0001). Public hospitals were significantly more likely to respond to our survey than private hospitals (p < 0.001). The geographical distributions were similar between the responders and non-responders (p = 0.663). Larger hospitals were significantly more likely to respond to the survey than smaller ones (p < 0.001).
Characteristics of Hospital Survey Responders
The majority of the contact persons was female (80.08%) with an average age of 37 years and had worked in their hospital for at least 12 years on average (Table 1). Almost half of them (47.97%) were medical statisticians and 31.71% were nurses. Each of the surveyed hospitals was responsible for an average UC population of 60,000. Universal Coverage, moreover, was the major source of hospital revenue for most responders. Under the UC scheme, hospitals have had to set up an adequate number of Primary Care Units (PCU) to provide care for remote populations. In this survey, the mean number of PCUs was 9.20 per hospital. On average, approximately 75% of the beds were occupied. The average length of stay (LOS) was 5.95 days. The mean Relative Weight (RW)--a standard value assigned to each DRG to reflect the cost of its care--and Adjusted Relative Weight (Adjusted RW) were 0.91 and 0.82, respectively. On an average day, the surveyed hospitals took care of 75.50 inpatient and 511.23 outpatient cases. One quarter of the hospitals received full accreditation by the Healthcare Accreditation Institute (similar to the US's Joint Commission on Accreditation of Healthcare Organizations). SCAD hospitals were more likely to be public and smaller than non-SCAD hospitals (p < 0.001). The majority of SCAD hospitals were in Northeast and Central region whereas non-SCAD hospitals were from Bangkok and Central region.
Hospital Coding Structure
The number of computers used specifically for coding purposes in each hospital ranged from 1 to 200 (Mean 7; SD 14.56; N = 376), regardless of SCAD status. We found a significant variation in the types of software that hospitals used. These software programs offered different capability to assist in the coding process. Even the most popular software (HOSxP) was used in only 45% of the hospitals sampled. Majority of SCAD hospitals used HOSxP whereas most of the Non-SCAD hospitals preferred either less common commercial or proprietary software.
We reported at least eight health care professional disciplines (Medical Statisticians, Nurse, Physician, Public Health Staff/Paramedics, Medical Record Staff, Information Technology Staff, Finance/Accounting Staff, and others) were involved in the hospital coding practice . The number of medical statisticians as well as experience of medical statisticians and physicians were statistically significantly different between SCAD and non-SCAD hospitals (p = 0.0256) (Table 2).
Table 2. Human resource for hospital coding practice in SCAD and Non-SCAD hospitals
Only 55 out of 492 hospitals (11.18%) reported that they had at least one formally trained medical statistician (Figure 2). There were at least 572 medical statisticians who were formally trained and received the 2-year certificate program from Kanchanabhisek Institute of Medical and Public Health Technology (KMPHT). Approximately 30% of them continued their study to finish the 4-year Bachelor's of Science Program in Medical Records from the Department of Social Science, Faculty of Social Sciences and Humanities, Mahidol University.
Figure 2. Distribution of medical statisticians with 2- or 4-year degree program.
Regardless of formal training, some of the experienced hospital staff could take an examination to be certified as intermediate and advance coders, who could also be invited to work as external auditors upon request. SCAD hospitals were more likely to have fewer certified coders than non-SCAD hospitals (Table 3).
Table 3. Certified intermediate and advance coders
Hospital Coding Process
Hospital coding process involved different health care professional disciplines in each of the steps. Figure 3 depicts the proportional representation of both primary and secondary responsible staff for all steps of the hospital coding process. The distribution of primary and secondary responsible staff between SCAD and non-SCAD hospitals are not different (results not shown).
Figure 3. Primary and secondary responsible staff in each step of the hospital coding process.
Hospital Coding Practice Scale
Based on the Kaiser-Guttman criterion, scree test, and parallel analysis, our data revealed that the hospital coding practice should comprise 2-4, 10, and 15 factors, respectively. Although parallel analysis has been considered the most accurate as compared to the other two criteria, it suggested a 15-factor model which we thought was not simple enough for our exploration of hospital coding practice.
We then explored the Kaiser-Guttman criterion by re-running the factor analysis with 10 factors specified. The factors were rotated to spread variability more evenly among factors so that all solutions are relatively the same. We used a cut point of 0.70 to drop items with high uniqueness, grouped the retained items into factors, and name the factors based on the member items. We found that 10-factor model was still not clear as some items have similar loadings across 2-3 factors. For example, item A10 (There is a physician responsible for coding practice) had 0.4275 and 0.4613 loadings for factor 2 and 5, respectively.
We therefore tried to follow the scree test approach and re-run the analysis with 4 factors. Orthogonal (varimax) and oblique (promax) rotation gave similar results but we decided to proceed with the latter because of the potential non-independent nature of the factors. After rotation and deletion of items with high uniqueness, we found that Factor 4 did not contain any items. Hence, the findings suggested that 3 factors might be the best solution for our purpose (Table 4).
Table 4. Grouping and naming 3 factors
Factor 1 was named "Data Quality" as hospitals who score high for this factor seemed to pay more attention to the quality of medical record, discharge summary, as well as diagnosis and procedure codes. One could see that hospitals who score high for the second factor paid attention to various aspects of coding practice. Factor 2 was therefore called "Coding Practice". Factor 3 contained three items, all of which suggested relevance to a hospital's interest in reimbursement rather than data quality or coding practice. Hence, we named it "Reimbursement". We also revisited the other items that were dropped because of high uniqueness but also had considerable loadings to Factor 3. Interestingly, they were also suggestive of hospital's reimbursement interest but with the other step of hospital coding process. Table 5 presents the three factors as three different profiles.
Table 5. Hospital intention profiles based on the 3-factor model
As a preferred method for provider payment in both developed and developing countries , DRG assumes that hospitals are well equipped with physicians and certified coders and therefore able to submit diagnosis and procedure codes with high quality. Literature on DRG implementation has been mostly from countries with abundant resource or mainly about its macro-level effects whereas study on how DRG codes are actually produced by hospitals is lacking.
DRG creep has been a major concern in resource-rich setting, in which hospital is suspected of being profit maximizer. As this kind of organizational intention is difficult to assess directly, it is not surprising to see mixed findings on the extent of DRG creep based on surrogate outcome measures [13-18]. While some studies investigated organizational behavior and reported potential influence of hospital management and payer on coding process [19,20], other studies tried to demonstrate potential upcoding of some specific diagnoses such as pneumonia and heart failure [21,22].
Poor coding quality is not only because of DRG creep, but also sicker patients, improvements in coding, and changes instituted by the payer . We added that variation in hospital coding practices in an under-resourced health system is another major determinant of DRG coding quality. It was not fair for a hospital to be assumed 'capable' of producing good codes without qualified physicians and/or coders.
To our knowledge, this study is the first national survey to explore the structural and process components of coding practice, that might affect DRG coding quality. In terms of structure, we found that the use of software, number of medical statisticians, and experience of physicians seemed to be the most important. SCAD hospitals are more likely to have fewer medical statisticians, fewer certified coders, and less experienced physicians. Our previous case study revealed that, with inadequate numbers and inequitable distribution of certified coders, hospitals have tried to survive by using part-time coders from other disciplines, especially nurses . This survey expanded the point further by suggesting that SCAD hospitals are more likely to face such problems than non-SCAD hospitals.
The current production of medical statisticians has been limited whereas the actual task is not necessarily about coding. In Thailand medical statistician is a job position that requires undergraduate-level training and usually is responsible for analyzing patient information . Although the Thai DRG anticipated medical statisticians to be trained and certified to work as coder, a survey of 322 hospitals in 2001 revealed that only 59.87% of the hospital had medical statisticians worked as coders; but as many as 46% of them were considered 'part-time coders' as they had to be responsible for other jobs as well .
Based on the seven steps of the hospital coding process we reported earlier , the cross-function phenomenon also occurs with other health care professional disciplines as well. This study adds to the case study findings by quantifying the number of hospitals that allows such phenomenon to occur. For example, discharge summary has been assumed to be done by physicians and therefore used as a gold standard to see if the codes assigned by hospital coders are correct. However, nurses or medical statisticians are indeed the primarily responsible staff for discharge summary in some hospitals. Nurses have played important roles in almost all steps of the hospital coding process but they have not been formally recognized. While an experienced nurses can become certified coders, their contribution might not be counted toward job promotion in public hospitals.
We also are proposing development of a new tool called Hospital Coding Practice Scale, which can indirectly explore the DRG creep phenomenon. The audit by external peers has been a main mechanism to assess the quality of discharge summary as well as diagnosis and procedure codes but the results cannot be used to judge hospital intention to game the system. By using this measurement model, one can classify hospitals based on the domains they focused on and the profiles they fall into. We hypothesize that DRG creep is more likely among hospitals who focus mainly on reimbursement (profiles 3, 5, 6; Table 3). Further studies are required to ensure the validity, reliability, and feasibility of this tool.
The generalizability of the findings from this study is limited by low response rate. This was actually anticipated because some hospitals might be cautious to provide such detailed and confidential information about coding practices. The difference in both response rates and characteristics between SCAD and non-SCAD hospitals is another limitation of this study that does not allow a direct comparison of various aspects between the two groups. Also, we were unable to explore some other factors that might affect hospital coding practice. For example, various functionality of the software can affect the hospital coding practice.
SCAD and Non-SCAD hospitals were different in many aspects, especially the number of medical statisticians, experience of medical statisticians and physicians, as well as number of certified coders. The findings suggested that hospital providers should not be assumed capable of producing high quality DRG codes, especially in resource-limited settings.
The authors declare that they have no competing interests.
KP conceived of and designed the study, carried out the survey, analyzed the data, and drafted the manuscript. DGW participated in its design, and helped to draft the manuscript. HR helped to revise the manuscript. CR helped to draft and revised the manuscript. All authors read and approved the final manuscript.
This study would not be possible without cooperation from hospital respondents. We also would like to thank Professor Richard H. Morrow for his comments during the preparation of this manuscript and Miss Sudarat Chadsuthi for her help with the production of some figures. This study is a part of the first author's dissertation project "Hospital Coding Practice, Data Quality, and DRG-based Reimbursement under the Thai Universal Coverage Scheme", which received partial financial support from the Health Insurance System Research Office (HISRO), Thailand. He also received the Higher Educational Strategic Scholarships for Frontier Research Network, from the Commission on Higher Education, Thailand.
Educational and Psychological Measurement 1960, 20:141-151. Publisher Full Text
Multivariate Behavioral Research 1966, 1:245-276. Publisher Full Text
Medical Care Research and Review 1991, 48:167-206. Publisher Full Text
JAMA: The Journal of the American Medical Association 1992, 268(7):896-899. Publisher Full Text
Health affairs (Project Hope) 1989, 8(2):35-47. Publisher Full Text
The American journal of cardiology 1999, 84(1):108-109.
A109PubMed Abstract | Publisher Full Text
The pre-publication history for this paper can be accessed here: