Open Access Research article

Comparative effectiveness research on patients with acute ischemic stroke using Markov decision processes

Darong Wu12, Yefeng Cai1, Jianxiong Cai12, Qiuli Liu3, Yuanqi Zhao1, Jingheng Cai4, Min Zhao1, Yonghui Huang4, Liuer Ye4, Yubo Lu12 and Xianping Guo45*

Author Affiliations

1 The 2nd Clinical Medical College of Guangzhou University of Chinese Medicine, the 2nd Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China

2 Guangdong Provincial Academy of Chinese Medical Sciences, Guangzhou, China

3 South China Normal University, Guangzhou, China

4 Sun Yat-Sen University, Guangzhou, China

5 School of Mathematics and Computational Science, Sun Yat-Sen University, 135, Xingangxi Road, Haizhu District, Guangzhou, Guangdong, China

For all author emails, please log on.

BMC Medical Research Methodology 2012, 12:23  doi:10.1186/1471-2288-12-23

Published: 9 March 2012



Several methodological issues with non-randomized comparative clinical studies have been raised, one of which is whether the methods used can adequately identify uncertainties that evolve dynamically with time in real-world systems. The objective of this study is to compare the effectiveness of different combinations of Traditional Chinese Medicine (TCM) treatments and combinations of TCM and Western medicine interventions in patients with acute ischemic stroke (AIS) by using Markov decision process (MDP) theory. MDP theory appears to be a promising new method for use in comparative effectiveness research.


The electronic health records (EHR) of patients with AIS hospitalized at the 2nd Affiliated Hospital of Guangzhou University of Chinese Medicine between May 2005 and July 2008 were collected. Each record was portioned into two "state-action-reward" stages divided by three time points: the first, third, and last day of hospital stay. We used the well-developed optimality technique in MDP theory with the finite horizon criterion to make the dynamic comparison of different treatment combinations.


A total of 1504 records with a primary diagnosis of AIS were identified. Only states with more than 10 (including 10) patients' information were included, which gave 960 records to be enrolled in the MDP model. Optimal combinations were obtained for 30 types of patient condition.


MDP theory makes it possible to dynamically compare the effectiveness of different combinations of treatments. However, the optimal interventions obtained by the MDP theory here require further validation in clinical practice. Further exploratory studies with MDP theory in other areas in which complex interventions are common would be worthwhile.

Markov decision processes; Acute ischemic stoke; Comparative effectiveness research; Traditional Chinese Medicine/integrative medicine