Figure 5.

Log-likelihood evolution during S-EM training. Each column shows the evolution of the log-likelihood for one of the three benchmarks described in the results section. The training procedure was started from two different random seeds (indicated by a solid and a dashed line). The log-likelihood values, log P (D|H_n, θ_n), used in the upper figures are conditional on the states of the sampled hidden nodes (θ_n are the parameter values at iteration n, H_n are the hidden node values at iteration n and D is the observed data). The log-likelihood values in the lower figures, log P (D|θ_n), are computed by summing over all hidden node sequences using the forward algorithm [5]. Note that the forward algorithm can only be used on HMMs and is therefore not applied on the complex benchmark.