Department of Intensive Care Medicine, Katholieke Universiteit Leuven; Herestraat 49, B-3000 Leuven, Belgium

Declarative Languages and Artificial Intelligence (DTAI), Department of Computer Science, Katholieke Universiteit Leuven; Celestijnenlaan 200a - bus 2402, B-3001 Heverlee, Belgium

M3-BIORES, Department of Biosystems, Katholieke Universiteit Leuven; Kasteelpark Arenberg 30 - bus 2456, B-3001 Heverlee, Belgium

Abstract

Background

The intensive care unit (ICU) length of stay (LOS) of patients undergoing cardiac surgery may vary considerably, and is often difficult to predict within the first hours after admission. The early clinical evolution of a cardiac surgery patient might be predictive for his LOS. The purpose of the present study was to develop a predictive model for ICU discharge after non-emergency cardiac surgery, by analyzing the first 4 hours of data in the computerized medical record of these patients with Gaussian processes (GP), a machine learning technique.

Methods

Non-interventional study. Predictive modeling, separate development (n = 461) and validation (n = 499) cohort. GP models were developed to predict the probability of ICU discharge the day after surgery (classification task), and to predict the day of ICU discharge as a discrete variable (regression task). GP predictions were compared with predictions by EuroSCORE, nurses and physicians. The classification task was evaluated using aROC for discrimination, and Brier Score, Brier Score Scaled, and Hosmer-Lemeshow test for calibration. The regression task was evaluated by comparing median actual and predicted discharge, loss penalty function (LPF) ((actual-predicted)/actual) and calculating root mean squared relative errors (RMSRE).

Results

Median (P25-P75) ICU length of stay was 3 (2-5) days. For classification, the GP model showed an aROC of 0.758 which was significantly higher than the predictions by nurses, but not better than EuroSCORE and physicians. The GP had the best calibration, with a Brier Score of 0.179 and Hosmer-Lemeshow p-value of 0.382. For regression, GP had the highest proportion of patients with a correctly predicted day of discharge (40%), which was significantly better than the EuroSCORE (p < 0.001) and nurses (p = 0.044) but equivalent to physicians. GP had the lowest RMSRE (0.408) of all predictive models.

Conclusions

A GP model that uses PDMS data of the first 4 hours after admission in the ICU of scheduled adult cardiac surgery patients was able to predict discharge from the ICU as a classification as well as a regression task. The GP model demonstrated a significantly better discriminative power than the EuroSCORE and the ICU nurses, and at least as good as predictions done by ICU physicians. The GP model was the only well calibrated model.

Background

The intensive care unit (ICU) length of stay (LOS) of patients undergoing cardiac surgery may vary considerably. It is often difficult to predict, within the first few hours after admission, which patients will be discharged fast, and which will have a more prolonged ICU stay. This is problematic, not only for counseling patients or their relatives on the expected ICU LOS, but also with regards to bed and resource management in the ICU. Cardiac surgery risk stratification

Computerization in the ICU is on the rise

The goal of this study was to develop and validate a model that predicts ICU discharge of non-emergency adult cardiac surgery patients, by analyzing PDMS data from these patients. Predictions were made as early as possible after admission, by using data known upon admission of the patient, and trend analysis of vital parameters during 4 hours (which is the shortest interval that allows dynamic time series analysis). Machine learning methods were used to develop predictive models for a classification task, to predict the probability of ICU discharge the day after surgery (further referred to as 'second day of discharge prediction'), and for a regression task, to predict the day of ICU dischargeas a discrete variable (further referred to as 'day of discharge prediction'). Models were developed in one patient cohort; the predictive performance was evaluated in a separate and previously unseen validation cohort. The performance of the models was compared against the performance of the EuroSCORE, and against predictions done by the ICU nurses and physicians in the validation cohort, in order to have an indication whether in the future they could have a possible added value as clinical decision support tools.

Methods

Setting

At the Leuven university hospitals, all cardiac surgery patients are admitted postoperatively to the 56 beds surgical ICU, of which 22 beds are dedicated to the care of these patients. Decisions regarding patient discharge are made by the senior ICU medical staff. Once the decision is made that a patient is or will be ready for discharge, a bed on the cardiac surgery ward is always made available on the same day. For reasons of nursing staff on the regular cardiac surgery ward, discharge takes place between 11.00 AM and 19.00 PM. Therefore LOS, or day of discharge, is a discrete and not a continuous variable, and the minimum LOS is 1 day (admission day). The minimum criteria for ICU discharge after cardiac surgery are summarized in table

Guidelines for ICU discharge after cardiac surgery at the university hospitals Leuven

1. Respiratory criteria

The patient is extubated and weaned from mechanical ventilation or other forms of mechanical respiratory support.

His oxygenation is good, or at least comparable to his pre-operative situation. To accomplish this, 3-4 liters of nasal oxygen supplementation is allowed.

The patient has an adequate cough reflex and is able to maintain a safe upper airway.

2. Hemodynamic criteria

Inotropic support and vasopressor therapy have to be stopped upon discharge.

No major arrhythmias compromising hemodynamic stability should be present.

Atrial fibrillation is no contra-indication for discharge, provided an adequate rate control.

Patients, who are depending on external epicardial pacing after surgery, where no background rhythm or implanted pacemaker is present, cannot be discharged to the nursing ward.

3. Neurologic criteria

Patient is awake, capable of communication and has sufficient pain control with his current analgesic therapy regimen.

4. Bleeding

No major bleeding, defined as a persistent need of transfusion of more than 2 units of packed cells per day.

5. Other organ systems

No vital threats to other organ systems (such as the kidneys, the central nervous system,...) are present

Patients

The electronic medical records of scheduled adult (> 18 years) cardiac surgery patients, admitted to the ICU, were used for predictive modeling. Surgery was defined as 'scheduled' when the patient was called from home to undergo surgery. Heart transplant patients, pulmonary thrombendarteriectomy patients, patients on a left or right ventricular assist device or on extracorporeal membrane oxygenation, were excluded. The goal of the study was to predict ICU discharge to a normal ward, but death in the ICU was also regarded as discharge. The data of patients discharged to another ICU or to an intermediate care unit were not used.

The study protocol was approved by the hospital's ethical committee ("Commissie Medische Ethiekvan de Universitaire Ziekenhuizen KULeuven", Belgian reference number B3222006749), and the need for informed consent was waived.

In the medical literature, prospective validation of predictive models in a previously unseen dataset is the most generally accepted method ^{th }of March 2008 and the 8^{th }of January 2009. Over the two time periods, there were no changes in surgical techniques, postoperative management, or in the cardiac surgical and ICU senior medical staff.

The baseline characteristics of validation and test cohorts are presented in table

Baseline characteristics

**Test cohort (n = 462)**

**Validation cohort (n = 499)**

**p-value**

ICU length of stay in days (continuous): median (P25-P75)

1.9 (1.2-3.6)

2.0 (1.2-3.9)

0.277

ICU length of stay in days (discrete): median (P25-P75)

3 (2-5)

3 (2-5)

0.360

Patients discharged on the day after surgery: n (%)

147 (31.8)

149 (29.9)

0.511

ICU mortality: n (%)

8 (1.7)

11 (2.2)

0.599

Hospital mortality: n (%)

15 (3.2)

16 (3.2)

0.972

EuroSCORE (additive): median (P25-P75)

5 (3-7)

5 (3-7)

0.202

EuroSCORE (logistic)%: median (P25-P75)

3.9 (2.1-7.4)

4.5 (2.2-8.1)

0.828

Type of surgery

Isolated OPCAB: n (%)

190 (41.1)

179 (35.9)

0.094

Isolated on-pump CAB: n (%)

1 (0.2)

0 (0)

0.298

Valvular surgery (single, multiple and/or combined with CAB or other surgery): n (%)

250 (54.1)

298 (59.7)

0.079

Other cardiac surgery (congenital, aorta ascendens, myxoma ...): n (%)

21 (4.6)

22 (4.4)

0.919

Repeat cardiac surgery: n (%)

27 (5.8)

49 (9.8)

0.023

Surgery post-endocarditis: n (%)

7 (1.5)

7 (1.4)

0.885

Database setup and input data

For the purpose of this study, a copy of the MetaVision^{® }(iMD-Soft^{®}, Needham, MA, USA) PDMS database (Microsoft^{® }SQL) was made, and all data referring to the identity of the patients were removed from this copy. This included the patient's name, address, hospital identification number, and also all free text fields - because information potentially revealing the identity of the patient could be present in these fields. There were 5 parameter categories:

• ^{®}, were taken from the Hospital Information System (HIS) or the ICU departmental database (FileMaker Pro^{®}, Santa Clara, CA, USA). Missing numerical data were substituted by the population mean for that parameter. Missing categorical data were replaced by the value for that parameter that corresponded to a normal healthy condition.

•

•

• ^{®}, Philips^{®}, Eindhoven, The Netherlands), mechanical ventilator data (Evita 2, 4, and XL, Dräger^{®}, Lübeck, Germany) blood loss, and urine output, registered during the first 4 hours in the ICU. Monitoring and mechanical ventilation data are automatically stored in the database, at a resolution of 1 datapoint per minute. This way, 240 datapoints were present for these parameters. Blood loss and urine output were manually registered by the nurses, at least once every hour. Mean and variance of the monitoring data, and cumulative blood loss and urine output were used.

• _{2}), heart rate, central blood temperature, and systolic pulmonary artery pressure) was analyzed. The length of this interval was chosen in order to have enough information on the dynamic evolution of the patients during their first hours in ICU. Artifacts were removed from the time series using a peak shaving algorithm, values exceeding 2 times the SD of the time series were removed. Missing values were linearly interpolated between two consecutive adjacent known values. Again, mean and variance were determined, but the time series was divided into 6 intervals of 40 minutes. This 40 minutes interval was chosen because by cross-validation in the development cohort, it provided the best predictive results, over a 30 or 60 minutes interval. Approximate Entropy (ApEn) was determined for the 4 hours time series of SpO_{2}, systolic arterial blood pressure, and heart rate

Overall, in the development cohort database 5.18% of data were missing; in the validation cohort the proportion of missing data was 5.74%. Variables with the largest amount of missing data were mainly related to pre-admission assessment of the patients, such as the pre-operative spirometry and pulmonary function tests (31-87% missing), pre-admission heart rate (35% missing), pre-admission blood pressure (27% missing). Intra-operative blood loss was not entered in the database in 50% of the patients.

Predictive modeling

ICU discharge after scheduled cardiac surgery was predicted in two ways.

• Classification task: predict the probability of discharge on the day after surgery ('second day discharge prediction')

• Regression task: predict the day of discharge, which is a discrete variable (day 2, 3 ...) ('day of discharge prediction').

MatLab^{® }(The MathWorks, Natick, MA, USA) was used to build the predictive models.

Models were built for each parameter category separately, and integrated into a single predictive model. In a preparatory phase of the study, a logistic regression (LR) model and several ML models were developed for second day discharge prediction: decision trees (DT), Random Forest (RF), Support Vector Machines (SVM), and Gaussian Processes (GP). A review of the used methods can be found in

The discriminatory power of the LR and DT models was unacceptably low, and GP proved to be the most performant of all models (results not shown). In addition to being highly performant, GP have a number of theoretical advantages such as the ability to deal with very complex datasets, and the possibility to identify the predictive variables (as opposed to black box models such as artificial neural networks)

Validation of the models

The developed models were tested in a previously unseen validation cohort. The performance of the

The performance of the _{actual}) and the predicted day of discharge (D_{predicted}), divided by the actual day of discharge.

This way, the relative error gets a less severe 'penalty' the further in the future this error is made. The LPF will take positive values if the predicted LOS is shorter than the actual LOS and negative values when the predicted LOS is longer than the actual LOS. Third, a prevalence plot was drawn, representing the actual and predicted numbers of patients discharged on each day. Fourth, root mean squared relative errors (RMSRE) were determined, with a value closer to 0 indicating a higher accuracy.

Comparison with EuroSCORE

The additive EuroSCORE was calculated for every patient in the validation cohort using the research calculator available online

Comparison with ICU clinicians

In the validation cohort, the ICU nurse and ICU physician caring for that patient were asked to perform the same two predictive tasks. Their predictions were collected prospectively and recorded in MetaVision^{®}, by the Event Manager^{®}. A pop-up window was programmed to appear 4 hours after admission to the first nurse and physician logging in to the patient's PDMS file, with the following questions. First, "What is the probability that this patient will leave the intensive care unit tomorrow, on the day after surgery?" The answer to this question is a probability between 0% (certainly not discharged on the day after admission) and 100% (certainly discharged on the day after admission). Second, "According to you, on what day will this patient be able to leave the ICU?" The answer to this question is a date. In order not to interfere with the clinical work, the pop-up could be overruled, but was repeated every 15 minutes. In advance, we determined that when the predictions were made later than 6 hours after admission, these results would not be used to compare with the GP models, as this would give the clinicians an unacceptable advantage of more than 2 hours of observation over the computer.

Statistical analyses

Comparison of the baseline characteristics between the development and validation cohort was performed using Stat View^{® }5.0.1 for Windows (SAS Institute, Cary, NC, USA). A Chi-square test was used to compare between nominal variables, a Student t-test was used for the normally distributed, and the Mann-Whitney-U test for non-normally distributed continuous variables. Two sided p-values of less than 0.05 were considered significant.

For the ^{® }was used as statistical software for the comparisons between the different models.

Results

The GP models are accessible at

Second day discharge prediction

The validation results are summarized in table

Classification task

**GP (4 h)**

**EuroSCORE (4 h)**

**EuroSCORE vs. GP**

**Nurses (6 h)**

**Nurses vs. GP**

**ICU physicians (6 h)**

**ICU physicians vs. GP**

**Validation cohort, n = 499**

aROC

0.758

0.726

p = 0.286

X

X

X

X

Brier Score

0.179

0.324

p < 0.001

X

X

X

X

Brier Score Scaled

11%

0%

p < 0.001

X

X

X

X

Hosmer Lemeshow p-value

0.382

< 0.001

X

X

X

X

X

**Nurses answer ≤ 6 h, n = 396**

aROC

0.769

0.726

p = 0.124

0.695

p = 0.018

X

X

Brier Score

0.177

0.326

p < 0.001

0.245

p < 0.001

X

X

Brier Score Scaled

13%

0%

p < 0.001

1.35%

p < 0.001

X

X

Hosmer Lemeshow p-value

0.405

< 0.001

X

< 0.001

X

X

X

**Physicians answer ≤ 6 h, n = 159**

aROC

0.777

0.726

p = 0.334

X

X

0.758

p = 0.719

Brier Score

0.166

0.328

p < 0.001

X

X

0.216

p = 0.055

Brier Score Scaled

14.2%

0%

p < 0.001

X

X

12.5%

p < 0.001

Hosmer Lemeshow p-value

0.696

< 0.001

X

X

X

< 0.001

X

The interpretation of the different validation measures can be found in the methodology section.

In the validation set of 499 patients, the aROC values of the GP submodels built on the different data categories were 0.730 for the admission data, 0.690 for the medication data, 0.640 for the laboratory data, 0.710 for the physiological data and 0.670 for the dynamic data. The integrated GP model had an aROC of 0.758, a Brier Score of 0.179 and a Brier Score Scaled of 11%. The EuroSCORE had an aROC of 0.726, (p = 0.286 as compared to GP) but a Brier score of 0.324 and BSS of 0%. The GP model showed good calibration with a Hosmer-Lemeshow goodness of fit statistic (U-statistic) p-value of 0.382; EuroSCORE had a poor calibration, Hosmer-Lemeshow goodness of fit statistic (U-statistic) p-value was very low (p-value < 0.001).

In 396 of the 499 patients in the validation cohort, the nurses predicted discharge within 6 hours after admission. In this subgroup, the nurses' predictions revealed an aROC of 0.695 (p = 0.018 as compared to GP) and a Brier Score of 0.245. The aROC of the EuroSCORE model was not significantly different from the nurses' predictions (p = 0.379).

Only in 159 of the 499 patients a prediction by the ICU physicians was obtained within 6 hours after admission. In this small subgroup, the aROC for the physician's predictions was 0.758 (p = 0.719 as compared to GP, and p = 0.593 as compared with EuroSCORE), with a Brier Score of 0.216.

The Brier scores of the GP predictions were significantly lower (better) (p < 0.001) than those of the other predictions, except as compared with the physicians (p = 0.055). U-statistic was adequate for the GP models (p = 0.405), the EuroSCORE's, nurses' and physicians' predictions were poorly calibrated (p < 0.001).

The aROC's are depicted in Figure

ROC of the classification task

**ROC of the classification task**.

Reliability diagrams of the classification task

**Reliability diagrams of the classification task**. The reliability curve plots the observed fraction of positives against the predicted fraction of positives. The diagonal indicates a perfect reliability. The dotted horizontal line is the no resolution line, indicating the mean prevalence of the outcome in the population. The Brier score can be expressed as the sum of three terms related to the components of a reliability diagram.

The first term, reliability, is the mean squared difference of the reliability curve to the diagonal. The second term, resolution, is the mean squared difference of the reliability curve to the no resolution line. The third term is a measure of uncertainty. _{k }_{k }_{k }

Day of discharge prediction

The results are summarized in table

Regression task

**Actual**

**GP (4 h)**

**EuroSCORE (4 h)**

**EuroSCORE vs. GP**

**Nurses (6 h)**

**Nurses vs. GP**

**ICU physicians (6 h)**

**ICU physicians vs. GP**

**Validation cohort, n = 499**

LOS: Median (P25-P75)(days)

3 (2-5)

3 (2-3)*

4 (3-5)*

p < 0.001

X

X

X

X

LPF: Median (P25-P75)

X

0 (0-0.4)

-0.3 (-0.5-0)

p = 0.003

X

X

X

X

Patients with LPF = 0: n(%)

X

200 (40)

94 (19)

P < 0.001

X

X

X

X

RMSRE

X

0.408

0.643

X

X

X

X

**Nurses answer ≤ 6 h, n = 396**

LOS: Median (P25-P75)(days)

3 (2-4)

3 (2-3)*

4 (3-5)*

p < 0.001

3 (2-3)*

p = 0.012

X

X

LPF: Median (P25-P75)

X

0 (0-0.4)

-0.3 (-0.5-0)

p = 0.002

0 (0-0.3)

p = 0.567

X

X

Patients with LPF = 0: n(%)

X

181 (46)

86 (22)

P < 0.001

152 (38)

P = 0.044

X

X

RMSRE

X

0.389

0.635

0.522

X

X

**Physicians answer ≤ 6 h, n = 159**

LOS: Median (P25-P75)(days)

3 (2-5)

3 (2-3)*

4 (4-5)*

p < 0.001

X

X

3 (2-3)*

p = 0.578

LPF: Median (P25-P75)

X

0.2 (0-0.5)

-0.25 (-0.5-0.2)

p < 0.001

X

X

0.2 (0-0.4)

p = 0.755

Patients with LPF = 0: n(%)

X

59 (37)

27 (17)

p < 0.001

X

X

49 (31)

P = 0.234

RMSRE

X

0.439

0.631

X

X

0.612

*p-value < 0.001 (as compared to actual LOS)

LOS= Length of stay

LPF= Loss Penalty Function

RMSRE= Root Mean Squared Relative Error

The interpretation of the different validation measures can be found in the methodology section.

In the validation cohort of 499 patients, the actual median (P25-P75) LOS was 3 (2-5) days. Median (P25-P75) LOS predicted by the GP models was 3 (2-3) days (p < 0.001), median (P25-P75) predicted LOS by the EuroSCORE was 4 (3-5) days (p < 0.001). The GP predicted day of discharge was significantly different from that predicted by the EuroSCORE (p < 0.001). The median (P25-P75) LPF of the GP model was 0 (0-0.4), in 40% of the patients the day of discharge was predicted correctly as indicated by an LPF of 0. The EuroSCORE had a significantly worse median (P25-P75) LPF ((-0.3 (-0.5-0), p < 0.001), and a significantly lower proportion of patients with LPF = 0 (19%, p < 0.001). Figure

Distribution of the predictions of the regression task (validation cohort, 499 patients

**Distribution of the predictions of the regression task (validation cohort, 499 patients**. GP versus EuroSCORE). The black bar indicates the actual number of patients discharged on each discrete day. Day 1 is the day of surgery (no patients were discharged on day 1), day 2 is the day after surgery, and so on. The other bars indicate the number of patients predicted to be discharged on that day. The subdivision in these bars indicates the number of true positives (predicted and actually discharged on that day).

The nurses predicted a median (P25-P75) LOS of 3 (2-3) days in the 396 patients where they predicted within 6 hours, which was significantly different from the actual median (P25-P75) LOS in this cohort (3 (2-4) days) (p < 0.001). There was a significant difference between the predictions by the nurses and the predictions by GP (3 (2-3) days, p = 0.012) and EuroSCORE (4 (3-5) days, p < 0.001).

The median LPF of the prediction of the nurses and the GP model was not significantly different (p = 0.567), but the proportion of patients with an LPF = 0 was significantly lower (46% for the GP model, 38% for the nurses' predictions, p = 0.044).

In the 159 patients where the physicians made a prediction within 6 hours after admission, the actual median LOS was 3 (2-5) days, the physicians predicted a median LOS of 3 (2-3) days (p < 0.001), GP predicted 3 (2-3) days (p < 0.001), and EuroSCORE predicted 4 (4-5) days (p < 0.001). There was no significant difference between the predicted LOS by physicians and GP (p = 0.578), but EuroSCORE LOS prediction was significantly different (p < 0.001). There was no significant difference in median LPF or in the proportion of patients with LPF = 0 between GP and physicians.

In comparison to all other predictions, the GP model had the lowest RMSRE: 0.408 for GP compared to 0.643 for EuroSCORE in the 499 patients validation cohort; 0.389 for GP, 0.635 for EuroSCORE, and 0.522 for the nurses in the 396 patients cohort, and 0.439 for GP, 0.631 for EuroSCORE and 0.612 for physicians in the 159 patients cohort.

Discussion

A GP model that used patient data of the first 4 hours after ICU admission, stored in a PDMS, was able to make accurate predictions on the probability of ICU discharge on the day after surgery, and was able to predict the day of discharge. The GP models were constructed in a development cohort, and tested in a previously unseen validation cohort. The GP model showed a significantly better discrimination than EuroSCORE and the ICU nurses, and was at least as discriminative as the ICU physicians. GP models were the best calibrated models, whereas the EuroSCORE, ICU nurses and physician's predictions showed overfitting.

In this era of computerized medical files, a large amount of patient information in the ICU has become available in an assessable format. Until now, only a few applications fully exploited the data-rich environment of intensive care, with its several information sources. Analyzing such large quantity of information, and using it for research purposes, remains a major challenge

Although not set up to perform predictions, our results show that a GP model derived from these data was able to predict ICU discharge. At the population level, calibration and accuracy of the second day discharge predictions was good, with aROC well above and Brier score well below the predefined thresholds. At the individual level, the GP models showed to be the only well calibrated models. This is extremely important in clinical practice, when using the models for patient counseling. The exact day of discharge could be predicted in 40% of the patients, and the GP model showed the lowest RMSRE. Figure

This is the first study describing the use of GP for the prediction of length of stay in the field of medicine and intensive care. Currently, outcome prediction in the cardiac ICU is based on scoring systems that were developed and validated in large multicentre patient databases, such as EuroSCORE for a cardiac surgery patients

The GP models predicted better than the ICU nurses in the classification and the regression task. Although there was a trend towards a better performance than the ICU physicians, this was not statistically significant. When comparing the GP model with ICU nurses and physicians, one should realize that the clinicians had a few major advantages over the computer model. First, we did not obtain predictions within 6 hours in all 499 validation patients. This way, we lost statistical power. On the other hand, not being able to predict within time might be regarded as an extra evaluation criterion, as opposed to the GP model which was able to generate a prediction in every patient. Second, the predictions by physicians and nurses might have been biased in a sense that they could have postponed their predictions in the more difficult to predict patients whereas the GP models have always delivered a prediction within the allotted time, regardless of the uncertainty. Third, nurses and ICU physicians had an advantage of up to 2 hours of data over the GP model. In practice, it turned out to be impossible to respond immediately to the first pop-up window 4 hours after admission. Fourth, nurses and physicians will always have more information at their disposal than is present in the computerized chart of the patient.

A locally derived risk prediction model will not replace or compete with the validated scoring systems with regards to generalizability and benchmarking. Nevertheless, these locally developed models, based on the analysis of the computerized patient chart with machine learning techniques, have the potential to support the clinician in the care for critically ill patients. First, they could be the basis of an early warning monitor, which alerts the clinician when a patient deviates from the expected path with regards to his outcome. Second, in the same sense, they could be of use for the clinician when counseling a patient or his relatives to offer a realistic estimate of the expected clinical course of a particular patient. Third, the ICU LOS of a patient following cardiac surgery has, besides its medical and clinical relevance, also importance on the management level. ICU bed capacity is in many hospitals the bottleneck when planning cardiac surgery. In order to make optimal use of the available capacity, and in order to avoid an excessive number of patients that will die while awaiting cardiac surgery, a locally derived predictive model could be the basis of an ICU capacity planner. GP can take full advantage of the different information sources available in ICU, both static and dynamic, including therapy and the response to therapy. Although the discriminative power of the GP models is not significantly higher than the physicians, they can be of added value because they will deliver their predictions in a more reliable and consistent way (not postponing the predictions in the most difficult cases as the physicians probably did in the present study). In theory, each ICU could build its own predictive models based on its own patient database, because this methodology is able to take into account the specific local situation, and can be adapted and recalibrated over time. The results from this study should first be confirmed in other centers, and preferably in databases from multiple centers.

Conclusions

Gaussian processes, a machine learning technique, are able to fully exploit a PDMS database and use all available data to build a predictive model for ICU discharge after cardiac surgery. This locally learned predictive model performed better than an existing scoring system (EuroSCORE), and better than ICU doctors and nurses. Machine learning offers a general method to learn the most performant predictive model for a specific context, database, or ICU. The predictive models, the used parameters and the weighed coefficients will be different in a different setting.

List of abbreviations

APACHE IV: Acute Physiology and Chronic Health Evaluation IV; ApEn: Approximate Entropy; aROC: area under the Receiver Operating characteristic Curve; BS: Brier Score; DT: Decision Tree; EuroSCORE: European System for Cardiac Operative Risk Evaluation; GP: Gaussian Processes; HIS: Hospital Information System; ICU: Intensive Care Unit; LOS: Length of stay; LPF: Loss Penalty Function; LR: Logistic Regression; PDMS: Patient Data Management System; SD: Standard Deviation; SpO_{2}: Pulse oxymeter arterial Oxygen Saturation; RF: Random Forest; RMSRE: Root Mean Squared Relative Errors; SVM: Support Vector Machine.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

GM and FG conceived and designed the study, drafted the study protocol, performed the statistic analysis, and drafted the manuscript. FG programmed, supervised and analyzed the machine learning algorithms. JR, MB and GVDB participated in the design of the study and helped to draft the manuscript. GM, DC and WDB programmed and designed the PDMS database, programmed the pop-up windows, carried out the database queries and de-identified the data. GM and DC calculated the EuroSCORE. KVL, JMA and DB did the dynamic data analysis. All authors read and approved the final manuscript.

Acknowledgements

The work was supported by the Fund for Scientific Research (FWO), Flanders, Belgium (G.0533.06 and G.0310.08), the Research Council of the Katholieke Universiteit Leuven (GOA/2007/14, GOA/2008/08 and IDO 3M030483), and the Methusalem Programme from the Flemish Government to GVdB. GM was a doctoral Fellow of the Fund for Scientific Research (FWO), Flanders, Belgium (1700809N). JR was a postdoctoral fellow of the FWO, Flanders, Belgium.

Pre-publication history

The pre-publication history for this paper can be accessed here: