Beckman Access versus the Bayer ACS:180 and the Abbott AxSYM cardiac Troponin-I real-time immunoassays: an observational prospective study

Abstract

Background

Reliability of cardiac troponin-I assays under real-time conditions has not been previously well studied. Most large published cTnI trials have utilized protocols which required the freezing of serum (or plasma) for delayed batch cTnI analysis. We sought to correlate the presence of the acute ischemic coronary syndrome (AICS) to troponin-I values obtained in real-time by three random-mode analyzer immunoassay systems: the Beckman ACCESS (BA), the Bayer ACS:180 (CC) and the Abbott AxSYM (AX).

Methods

This was an observational prospective study at a university tertiary referral center. Serum from a convenience sampling of telemetry patients was analyzed in real-time for troponin-I by either the BA-CC (Arm-1) or BA-AX (Arm-2) assay pairs. Presence of the AICS was determined retrospectively and then correlated with troponin-I results.

Results

100 patients were enrolled in Arm-1 (38 with AICS) and 94 in Arm-2 (48 with AICS). The BA system produced 51% false positives in Arm-1, 44% in Arm-2, with negative predictive values of 92% and 100% respectively. In Arm-1, the BA and the CC assays had sensitivities of 97% and 63% and specificities of 18% and 87%. In Arm-2, the BA and the AX assays had sensitivities of 100% and 83% and specificities of 11% and 78%.

Conclusions

In real-time analysis, the performance of the AxSYM and ACS:180 assay systems produced more accurate troponin-I results than the ACCESS system.

Background

Physicians are frequently called upon to care for patients presenting with symptoms suggestive of the acute ischemic coronary syndrome (AICS). Over the past few years a number of cardiac markers, including myoglobin, cardiac troponins and creatine kinase isoenzyme-MB, have become available to assist with the management and triage of patients suspected of having myocardial injury [1, 2]. Cardiac Troponin-I (cTnI) has been shown to be a very sensitive and specific marker in this respect [1–5]. To date, the large published cTnI trials have utilized protocols which required the freezing of serum (or plasma) for delayed batch cTnI analysis. The process of freezing and thawing incorporates extra centrifugation and settling steps that are not performed when the assay is run under real-time conditions which may alter their performance characteristics [6–10]. Additionally, each cTnI assay system currently available utilizes a unique antibody to a specific cTnI epitope, some of which are inherently less specific than others [7, 11, 12].

At our institution, we appeared to have an inappropriately high rate of false positive cTnI values when our assays were performed under real-time conditions, despite strict adherence to the manufacturer's methodology. This phenomenon was apparently not unique to our medical center (verbal communications). From whatever cause, obtaining false positive cTnI values can have a substantial impact on a patient's cardiac evaluation.

There are currently more than half a dozen cTnI quantitative analysis systems available in the U.S. with new generation assays being developed [13]. Despite the literature purporting excellent clinical utility of the cTnI marker, the reliability of these assays, when used under real-time clinical conditions, has not been well studied. Additionally, assays are often inadequately appraised prior to their introduction into clinical use [14]. Cardiac TnI results published from large clinical trials to date have been obtained from assays performed using thawed specimens. This trial was done to assess the level of agreement and clinical accuracy between cTnI values determined in real-time, using fresh serum, in patients undergoing AICS evaluation, by three widely used random-mode (as opposed to batch) analyzer immunoassay systems: the Beckman ACCESS (BA) paramagnetic-particle, chemiluminescent immunoassay versus both the Bayer ACS:180 (CC) chemiluminescent immunoassay and the Abbott AxSYM (AX) microparticle enzyme immunoassay. All three systems use unique cTnI antibodies.

Methods

Subjects and samples

The investigation was performed in accordance with the Declaration of Helsinki with the hospital's institutional review board granting approval of this research project. This was an observational study performed at a university tertiary referral center with prospective blood specimen collection and patient assessments and retrospective determination of the AICS. A convenience sample of adult cardiac telemetry unit patients being treated and/or observed for possible cardiac ischemia, were selected to participate. The diversity of the patient population enrolled at our institution is diverse and not unique when compared with other major medical centers [15]. All patients with hematological samples ordered by the primary care physician for CKMB analysis as part of the patient's myocardial assessment, were eligible. All cTnI specimens were drawn between eight and 24 hours after the onset of symptoms.

Blood samples were obtained in accordance with the standard procedures of the hospital. Blood was drawn into evacuated gel tubes for serum preparation, kept at room temperature to allow clotting, and transported directly to the pathology laboratory for biochemical analysis. In the first phase of the study (Arm-1), samples were processed on the BA and CC analyzers. For the second phase (Arm-2), samples were processed on the BA and AX analyzers. Limited resources restricted comparing all three machines simultaneously. At the time of the study, the hospital was routinely using the BA analyzer. The treating physician was blinded to all other cTnI results obtained as part of the study with the patient's treatment care plans continuing unchanged.

Laboratory analysis

All samples were run simultaneously on the BA analyzer (Access Immunoassay System, Beckman Instruments, Inc. (formerly Sanofi-Synthelabo, Inc.)) and either the CC analyzer (Bayer-Centaur ACS:180 Automated Chemiluninescence System, Bayer Industries (formerly Chiron Diagnostics Corp.)) in Arm-1, or the AX analyzer (AxSYM System, Abbott Laboratories) in Arm-2. Samples were strictly processed according to the manufacturer's specifications with regards to centrifugation, filtering, temperature control and other specimen, reagent and equipment handling procedures [16–18]. Specimens were then split and run on each analyzer pair simultaneously. The analyzers were operated at the manufacturers' specifications with performance reliability maintained within the limits specified by the manufacturer, and were dedicated to running only the cTnI assays. All three first generation systems utilized random-mode (as opposed to batch) sample analysis to obtain real-time results. The manufacturers' predetermined calibrations established limits for negative (lowest limit of detection), indeterminate and positive (based on ROC analysis for the diagnosis of ST elevation MI) values for each assay system. Cutoff values were: <0.03, 0.03–0.15, >0.15 ng/ml for the BA, <0.15, 0.15–1.4, >1.4 ng/ml for the CC, and <0.4, 0.4–1.9, >1.9 ng/ml for the AX assays. All samples were run in duplicate with mean cTnI values used for data analysis. Quality control samples were run at normal and high levels. For this study, all indeterminate values were considered positive in order to incorporate the spectrum of unstable angina (and micro-infarct) into the data analysis [5, 19, 20].

Definitions

The acute ischemic coronary syndrome (AICS) is defined as a continuum of myocardial ischemia ranging from unstable angina, which may be associated with minor myocardial damage, to the extensive tissue necrosis of acute myocardial infarction [21]. The diagnosis of AMI was determined by using CK-MB enzyme assay results and the World Health Organization (WHO) criteria [22, 23]. Unstable angina was defined as new onset, severe or accelerated angina, subacute angina at rest or acute angina at rest (type IIIB in the Braunwald's classification) [24].

Clinical assessment

The likelihood of the patient having the AICS was determined retrospectively. The diagnostic categorization of the AICS as the cardiac endpoint was made if any of the following criteria were present: acute myocardial infarction (AMI), non-Q wave MI, or unstable angina. All parts of the hospital record were reviewed including the ED chart, laboratory results, 12-lead ECG's, the results of any cardiac provocative testing or catheterization performed, and the diagnoses of the clinician responsible for the care of the patient. The chart review covered a period from two weeks prior to the index hospitalization to four weeks after hospital discharge in order to capture any recent prior infarcts or short-term cardiac event occurring within the subsequent thirty days. Of note, our institution is the only tertiary medical center in the region and patients with serious illness are unlikely to have been seen elsewhere. AICS was determined by two independent trained reviewers for each patient. Both reviewers remained blinded to all cTnI assay results. Troponin-I was not used as a criteria for establishing the diagnosis of the AICS or myocardial infarction to avoid biased outcome. A consensus determination was then established when the reviewers disagreed. Demographic data (sex, age and race) were also recorded.

Statistical analysis

All results were entered into an Excel (Microsoft, Redmond, WA) spread sheet. Sensitivity, specificity, positive and negative predictive values, kappa and Bland-Altman plots were generated using Stata v7.0 (StataCorp. 2001. Stata Statistical Software, College Station, TX). There was an enrollment size of 100 samples per group although no a priori statistical determination of sample size was performed.

Results

In Arm-1 (BA vs. CC), a total of 100 patients were enrolled, of which 59% were males, 72% Caucasians, with an average age of 65 years. In Arm-2 (BA vs. AX), a total of 94 patients were enrolled, of which 53% were males, 64% Caucasians, also with an average age of 65 years.

In Arm-1, 38 patients (38%) had a diagnosis consistent with the AICS, and in Arm-2, there were 48 patients (51%). The inter-rater agreement (kappa) of AICS was 0.89 for both arms of the study. Table 1 and Table 2 summarize the cTnI values and the AICS status for patients in each of the two arms in this study. For Arm-1, the percent of negative cTnI values for BA and CC were 12% (95%CI: 6%, 20%) and 68% (95%CI: 58%, 77%) respectively. For Arm-2, the percent of negative cTnI values for BA and AX were 5% (95%CI: 2%, 12%) and 47% (95%CI: 36%, 57%) respectively.

Table 1 cTnI Values and the AICS Clinical Decision: Arm-1

Table 2 cTnI Values and the AICS Clinical Decision: Arm-2

Table 3 summarizes the sensitivity, specificity, negative and positive predictive values of the three assays using the cut-off points of: BA (0.03 ng/ml), CC (0.15 ng/ml), and AX (0.40 ng/ml) to incorporate both the indeterminate and the positive values into the AICS analysis. There was no predominant diagnosis seen among those patients with a false positive cTnI result. Table 4 summarizes the discordant data for those patients who did not meet the criteria for the AICS, yet had a positive cTnI result (false positive value) by both assay pairs. The Bland-Altman plots (Figures 1 and 2) suggest that by performing a linear transformation the analyzers would yield similar results and that the differences in the diagnostic accuracy between the analyzers may have been in part due to differences in the cut-off points chosen by the manufacturers. However, when we recalculated the statistics using the transformed data, no significant changes in sensitivities, specificities, positive or negative predictive values were found.

Table 3 Correlation Between cTnI Results and the Presence of the AICS

Table 4 Patients with False Positive cTnI Results by Both Assay Pairs

Figure 1

Bland-Altman Transformation: BA vs CC (Arm-1). Bland-Altman plot of cTnI for the BA and CC analyzers using transformed data.

Figure 2

Bland-Altman Transformation: BA vs AX (Arm-2). Bland-Altman plot of cTnI for the BA and AX analyzers using transformed data.

The sensitivity and specificity of the cTnI assays found in our study compared to those published by the manufacturers are shown in Table 5, with the manufacturers' cut-offs selected to determine the more specific diagnosis of acute myocardial infarction (instead of the broader diagnosis of the AICS) [16–18].

Table 5 Comparison of cTnI Sensitivity and Specificity Values to the Published Manufacturer's Values

Discussion

Cardiac Troponin-I is frequently used as a marker for the evaluation of patients with AICS [1, 3, 25]. Levels of cTnI are, on average, detectable four to six hours from the onset of chest pain, peaking in twelve hours and remaining elevated for three to ten days, making it an ideal marker for the detection of both acute as well as delayed AICS presentations [26]. Also, cTnI has been shown to be more sensitive than CK-MB mass assays for the detection of subsequent myocardial events in those patients presenting to the ED with chest pain [5, 27].

Unlike Cardiac Troponin-T, which is marketed by a single manufacturer, there are more than half a dozen cTnI quantitative analysis systems currently available [28]. Surprisingly, the clinical reliability of these assays when used in real-time has not been well investigated. As our study shows, the ability of these assays to determine real-time quantitative cTnI values, are not equal. Both the AX and the CC assay systems performed much more accurately than did the BA system, which produced a high number of false positive values. Recently, a second generation ACCESS assay was marketed, improving on some of the limitations of its first generation [13]. As with any assay, its clinical sensitivity and specificity with the fresh specimen cTnI values obtained needs thorough clarification.

When compared to the manufacturer's published sensitivity and specificity data, all three assays yielded lower values with the exception of the sensitivity for the BA assay. This may be due to our study protocol in which we chose for analysis purposes to use the limit of detection (LoD) for the three assays (0.03 ng/ml for the BA, 0.15 ng/ml for the CC, and 0.4 ng/ml for the AX) in order to incorporate both the indeterminate along with the positive cTnI values. This was done in order to more accurately reflect the full spectrum of the AICS from micro infarct and unstable angina to acute MI, whereby replicating how the results are used clinically in the emergency department setting [5, 19, 20, 29]. This was not done in the manufacturers' sponsored studies where the cut-off values used were the upper reference range limits (0.15 ng/ml for the BA, 1.4 ng/ml for the CC, and 1.9 ng/ml for the AX) for the detection of ST elevation MI. Unfortunately, no additional manufacturer data are available to compare against the AICS, a broader diagnostic entity. Despite using the lower cut-off value for the CC analyzer, our study showed a lower sensitivity compared to the manufacturer's value, which cannot be easily explained, although a number of causes of assay interference are discussed below. Similarly, a decreased sensitivity was also seen with the AX analyzer, but in this instance, there was considerable overlap of the 95%CI.

Serum specimens which are frozen for purpose of delayed batch analysis undergo different handling procedures than do fresh serum or plasma samples used for processing in real-time analysis [9, 10]. Freezing entails extra centrifugation and settling steps that allow for potentially more adequate clearance of micro-clots and bubbles from the blood samples, both of which can contribute to an over-estimation of cTnI quantity by the analyzer [30]. This over-counting would result in an abnormally high, or false positive value being obtained. Some of the cTnI assay systems currently available (such as the CC assay system) incorporate clot and bubble detection technology into their processors to help minimize these confounding variables, but their accuracy has not been well studied in real-time clinical trials. Additionally, under real-time laboratory conditions, machines running multiple analytes in random-access mode, may be subject to carry-over effects, assay drift or sub-optimal reagent integrity resulting in further result variation.

All currently marketed cTnI assays use unique antibodies to cTnI epitopes, some of which are inherently less specific than others [7, 11, 12]. The BA recognizes the C-terminal portion (CTP) of cTnI, whereas the other assays recognize the N-terminal portion (NTP) [31]. Shi et al. have shown that there is preferential degradation of the CTP versus the NTP of cTnI, leading to increased assay inaccuracy due to under-estimation of cTnI. Yeo et al. concluded that the superior performance of the AX system (which also lacks the clot and bubble detection technology) over the BA system, is possibly due to the use of a more specific cTnI antibody with less heterophile antibody interference [12].

The numerous cTnI assays currently available do not conform to a recognized uniform standardization, as do the Cardiac Troponin-T (cTnT) assays [28]. Because of unequal reactivity by the different cTnI antibodies, standardization becomes much more difficult. There are significant variations, sometimes greater than 10-fold, in cTnI concentrations when measured by different assays in patients experiencing an AMI [8, 32, 33]. This was also true in comparing the BA with the AX and CC assays where cutoff values were approximately 10× lower for the BA assay. Future standardization of all cTnI assays by use of a common reference standard for cTnI, will be extremely important to achieve comparability of test results between labs [34]. Such efforts are currently under way [35].

Recent studies have revealed a number of other factors which may interfere with the interpretation of elevated cTnI results. Elevated cTnI, especially in the lower ranges, have been seen in patients suffering from acute illness aside from the AICS such as HIV, chronic renal failure, sepsis, cirrhosis, lung diseases, endocrine, muscular and CNS disorders, and non-ischemic dilated cardiomyopathies [36, 37]. Fluid therapy may also interfere with cTnI assays since dilution of the serum with saline, Plasmion (a modified fluid gelatin), hydoxyethyl starch, and 20% human albumin have been shown to interfere with cTnI assay results [38]. Gerhardt et al. have shown that the use of heparinized tubes for serum collection can lower reported cTnI levels, especially early in the course of the AICS disease process [39]. Also, heterophilic antibodies in the serum of some individuals, have led to cases of reported false positive cTnI results [12]. Unfortunately, the routine screening for most of these interfering substances and diagnoses would render the clinical utility of the cTnI assay impractical.

It is unclear as to why the distribution of indeterminate BA-cTnI values for the AICS-positive patients is different in the two study arms (21/38 (55%) in Arm-1 and 8/48 (17%) in Arm-2). This was likely due to an unidentified variation in the patient populations since analysis showed no time differences in assay processing or specimen handling. In both arms of the study, the BA system demonstrated excellent sensitivity but produced a high number of potential false positive (53% and 44% respectfully) results. The implications of a false positive as well as a false negative cTnI result can be dramatic. False positives will increase the costs associated with a patient's cardiac evaluation by necessitating additional testing, potential unnecessary hospitalization, iatrogenic morbidity, missed time from work, and undue angst by all parties involved. False negative results, as was seen with both the CC and AX analyzers, may falsely reassure the treating clinician leading to a missed diagnosis of the AICS with potentially serious medical and legal consequences. In this era of cost containment, a test which demonstrates a 100% negative predictive value, as with the performance of the BA assay system in this trial, is not desirable at the expense of a markedly increased number of false positives which would contribute greatly to the patient's healthcare expenses. Physicians are reminded daily of the balance between risks of inappropriate patient discharge and costs associated with inpatient or observation unit cardiac evaluations. Depending on the clinical circumstances, the treating clinician may have a lower threshold to admit the patient and a higher comfort level with a test demonstrated to produce a higher sensitivity. Reliance on objective measurements of cardiac ischemia, such as cardiac markers, must be taken in context with the patient's entire clinical scenario, and, as better cardiac indicators become available, their use will only increase. As this study shows, there is an additional need for laboratories to more closely collaborate with clinicians and work together to determine the most appropriate troponin clinical cut-off levels and upper reference intervals.

Limitations

There are a number of limitations to this study. All patients were at least eight hours from the onset of their symptoms, and although the pharmacokinetics of cTnI ensures that significantly elevated levels would remain detectable for at least a week, the exact timing of the blood draws were not standardized. Despite variability in phlebotomy times, correlation with the AICS may have more accurately replicated actual clinical use and interpretation of the cTnI marker. Secondly, although we sought to primarily compare the discrepancy of cTnI values obtained, the criteria for determining the presence of the AICS were based on objective as well as subjective assessment, thereby allowing for some cases to be categorized by inference. Despite multiple objective criteria available to the physician, labeling a patient's symptoms as having (or not having) been cardiac in etiology incorporates "the art" into medical practice. Although determining the analytic capabilities of an assay is never without error (with the subsequent calculation of sensitivity and specificity), we feel our results were based on a reasonable AICS cardiac endpoint. Thirdly, the laboratory's handling of the specimens and assays may have resulted in some undetected bias with regards to the manipulation of the samples. Improperly collected or assayed specimens can dramatically alter the results obtained, despite the use of automated assay systems. Fourthly, given the financial restrictions of the project, no AX vs. CC arm was performed, thus slightly limiting the conclusions of this study. Fifthly, as newer generation assays become available, clinicians must be attuned to their deficiencies, as has been demonstrated here with the ACCESS first generation assay system. Direct assay comparisons under real-time conditions are warranted as new cTnI assays are marketed [40]. Sixthly, in this study, each analyzer was dedicated to running only the cTnI assay, and thus not subject to potential confounders that may additionally appear when machines are processing multiple analytes in a random-mode access. Lastly, we did not control for hemodilution, the presence of other illnesses or heterophilic antibodies, all of which may potentially confound the cTnI levels obtained as mentioned in more recent studies.

Conclusions

Assay systems currently available are clearly not equal in their abilities to determine real-time quantitative cTnI values. Our results show that both the AX and the CC assay systems performed much more accurately in their ability to correlate with the diagnosis of the AICS than did the BA system. The BA assay results did not correlate well with clinical outcomes with regard to its positive predictive utility by producing a high number of false positive values.