The following parameters describe the conventional HMM implementation according to 14:

• A set of states S = {S₁,..., S_N} with q_t being the state visited at time t,

• A set of PDFs B = {b₁(o),..., b_N(o)}, describing the emission probabilities b_j(o_t) = p(o_t|q_t = S_j) for 1 ≤ j ≤ N, where o_t is the observation at time-point t from the sequence of observations O = {o₁,..., o_T},

• The state-transition probability matrix A = {a_{i, j}} for 1 ≤ i, j ≤ N, where a_{i, j} = p(q_{t + 1} = S_j|q_t = S_i),

• The initial state distribution vector Π = {π₁,..., π_N}.

A set of parameters λ = (Π, A, B) completely specifies an HMM. Here we describe the HMM parameter update rules for the EM learning algorithm rigorously derived in 15. The Viterbi algorithm, as shown in Table 2, is a dynamic programming algorithm that runs an HMM to find the most likely sequence of hidden states, called the Viterbi path, that result in an observed sequence. When training the HMM using the Baum-Welch algorithm (an Expectation Maximization procedure), first we need to find the expected probabilities of being at a certain state at a certain time-point using the forward-backward procedure as shown in Table 2. The forward, backward, and Viterbi algorithms take O(TNQ_max) time to execute.

Figure 1 outlines initial, simple transition probability calculations for all possible paths through a "toy" HMM. In Figure 1, to estimate the probability of transition from state 1 to state 2 (1 → 2), we calculate the probability of transition utilization at time intervals 1–2 and 2–3 as:

p(Making transition 1 → 2 at time 1–2) = α₁(1) × a_1,2 × b₂(o₂) × β₂(2), p(Making transition 1 → 2 at time 2–3) = α₂(1) × a_1,2 × b₂(o₃) × β₃(2).

In particular, to estimate the probability of transition 1 → 2 at time interval 1–2 we calculate the sum of probabilities of all possible paths that lead to state 1 at time-point 1 (forward probability α₁(1)). Then we multiply this probability of being in state 1 by the transition a_1,2 and emission b₂(o₂) probabilities.

Further multiplication by the sum of probabilities of all possible paths from state 2 at time 2 until the end of the emission sequence (backward probability β₂(2)), is the expected probability of transition use. The sum of these estimates at time-points 1–2 and 2–3 is equivalent to the transition probability estimate in Table 3 (prior to normalization).

According to 13 t_{i, j}(t, m) is the weighted sum of probabilities of all possible state paths that emit subsequence o₁,..., o_t and finish in state m, taking an i → j transition at least once where the weight of each state path is the number of i → j transitions taken. Processing of the entire t_{i, j}(t, m) recurrence takes memory proportional to O(NQ) and processor time O(TNQQ_max). Initially we have t_{i, j}(1, m) = 0 since no transitions have been made. After initialization, we perform the following recurrence operations:

t i , j ( t , m ) = α t − 1 ( i ) a i , m b m ( o t ) δ ( m = j ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiDaq3aaSbaaSqaaiabdMgaPjabcYcaSiabdQgaQbqabaGccqGGOaakcqWG0baDcqGGSaalcqWGTbqBcqGGPaqkcqGH9aqpcqaHXoqydaWgaaWcbaGaemiDaqNaeyOeI0IaeGymaedabeaakiabcIcaOiabdMgaPjabcMcaPiabdggaHnaaBaaaleaacqWGPbqAcqGGSaalcqWGTbqBaeqaaOGaemOyai2aaSbaaSqaaiabd2gaTbqabaGccqGGOaakcqWGVbWBdaWgaaWcbaGaemiDaqhabeaakiabcMcaPiabes7aKjabcIcaOiabd2gaTjabg2da9iabdQgaQjabcMcaPaaa@53E0@

+ ∑ n = 1 N t i , j ( t − 1 , n ) a n , m b m ( o t ) , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaey4kaSYaaabCaeaacqWG0baDdaWgaaWcbaGaemyAaKMaeiilaWIaemOAaOgabeaakiabcIcaOiabdsha0jabgkHiTiabigdaXiabcYcaSiabd6gaUjabcMcaPiabdggaHnaaBaaaleaacqWGUbGBcqGGSaalcqWGTbqBaeqaaOGaemOyai2aaSbaaSqaaiabd2gaTbqabaGccqGGOaakcqWGVbWBdaWgaaWcbaGaemiDaqhabeaakiabcMcaPaWcbaGaemOBa4Maeyypa0JaeGymaedabaGaemOta4eaniabggHiLdGccqGGSaalaaa@4E04@

where δ(m=j)={1,ifm=j0,otherwise MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeqiTdqMaeiikaGIaemyBa0Maeyypa0JaemOAaOMaeiykaKIaeyypa0ZaaiqaaeaafaqaaeGacaaabaGaeGymaeJaeiilaWcabaqbaeqabeGaaaqaaiabbMgaPjabbAgaMbqaaiabd2gaTjabg2da9iabdQgaQbaaaeaacqaIWaamcqGGSaalaeaacqqGVbWBcqqG0baDcqqGObaAcqqGLbqzcqqGYbGCcqqG3bWDcqqGPbqAcqqGZbWCcqqGLbqzaaaacaGL7baaaaa@4BFF@. Following equation (1), at a certain time-point t we need to score the evidence supporting transition between nodes i and j, which is the sum of probabilities of all possible state paths that emit subsequence o₁,..., o_t-1 and finish in state i (forward probability α_t-1(i)), multiplied by transition a_{i, j} and emission b_j(o_t) probabilities upon arrival to o_t. We extend weighted paths containing evidence of i → j transitions made at previous time-points 1,..., t - 1 further down the trellis in subequation (4). Finally, by the end of the recurrence we marginalize the final state m out of probability t_{i, j}(T, m) to get a weighted sum of state paths taking transition i → j at various time-points that is equivalent to the numerator in the transition probability estimate shown in Table 3. Thus, we estimate transition utilization using:

t i , j E N D = ∑ m = 1 N t i , j ( T , m ) , a i , j = t i , j E N D ∑ j ∈ o u t ( S i ) t i , j E N D , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabdsha0naaDaaaleaacqWGPbqAcqGGSaalcqWGQbGAaeaacqWGfbqrcqWGobGtcqWGebaraaGccqGH9aqpdaaeWbqaaiabdsha0naaBaaaleaacqWGPbqAcqGGSaalcqWGQbGAaeqaaOGaeiikaGIaemivaqLaeiilaWIaemyBa0MaeiykaKIaeiilaWcaleaacqWGTbqBcqGH9aqpcqaIXaqmaeaacqWGobGta0GaeyyeIuoaaOqaaiabdggaHnaaBaaaleaacqWGPbqAcqGGSaalcqWGQbGAaeqaaOGaeyypa0tcfa4aaSaaaeaacqWG0baDdaqhaaqaaiabdMgaPjabcYcaSiabdQgaQbqaaiabdweafjabd6eaojabdseaebaaaeaadaaeqaqaaiabdsha0naaDaaabaGaemyAaKMaeiilaWIaemOAaOgabaGaemyrauKaemOta4KaemiraqeaaaqaaiabdQgaQjabgIGiolabd+gaVjabdwha1jabdsha0jabcIcaOiabdofatnaaBaaabaGaemyAaKgabeaacqGGPaqkaeqacqGHris5aaaakiabcYcaSaaaaaa@6DB1@

where out(S_i) is the set of nodes connected by edges from S_i.

According to 13 e_i(γ, t, m) is the weighted sum of probabilities of all possible state paths that emit subsequence o₁,..., o_t and finish in state m, for which state i emits observation γ at least once where the weight of each state path is the number of γ emissions that it makes from state i.

The following algorithm updates parameters for the set of discrete symbol probability distributions.

Initialization step e_i(γ, 1, m) = α₁(m) δ (i = m) δ (γ = o₁). After initialization we make the recurrence steps, where we correct the emission recurrence presented in 13 [see Additional File 1]:

e i ( γ , t , m ) = α t ( m ) δ ( i = m ) δ ( γ = o t ) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyzau2aaSbaaSqaaiabdMgaPbqabaGccqGGOaakcqaHZoWzcqGGSaalcqWG0baDcqGGSaalcqWGTbqBcqGGPaqkcqGH9aqpcqaHXoqydaWgaaWcbaGaemiDaqhabeaakiabcIcaOiabd2gaTjabcMcaPiabes7aKjabcIcaOiabdMgaPjabg2da9iabd2gaTjabcMcaPiabes7aKjabcIcaOiabeo7aNjabg2da9iabd+gaVnaaBaaaleaacqWG0baDaeqaaOGaeiykaKcaaa@4E82@

= ∑ n = 1 N e i ( γ , t − 1 , n ) a n , m b m ( o t ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeyypa0ZaaabCaeaacqWGLbqzdaWgaaWcbaGaemyAaKgabeaakiabcIcaOiabeo7aNjabcYcaSiabdsha0jabgkHiTiabigdaXiabcYcaSiabd6gaUjabcMcaPiabdggaHnaaBaaaleaacqWGUbGBcqGGSaalcqWGTbqBaeqaaOGaemOyai2aaSbaaSqaaiabd2gaTbqabaGccqGGOaakcqWGVbWBdaWgaaWcbaGaemiDaqhabeaakiabcMcaPaWcbaGaemOBa4Maeyypa0JaeGymaedabaGaemOta4eaniabggHiLdGccqGGUaGlaaa@4E58@

Finally, by the end of the recurrence we marginalize the final state m out of e_i(γ, T, m) and estimate the emission parameters through normalization

e i E N D ( γ ) = ∑ m = 1 N e i ( γ , T , m ) , b ^ j ( γ ) = e j E N D ( γ ) ∑ γ = 1 D e j E N D ( γ ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeqabeGaaaqaaiabdwgaLnaaDaaaleaacqWGPbqAaeaacqWGfbqrcqWGobGtcqWGebaraaGccqGGOaakcqaHZoWzcqGGPaqkcqGH9aqpdaaeWbqaaiabdwgaLnaaBaaaleaacqWGPbqAaeqaaOGaeiikaGIaeq4SdCMaeiilaWIaemivaqLaeiilaWIaemyBa0MaeiykaKIaeiilaWcaleaacqWGTbqBcqGH9aqpcqaIXaqmaeaacqWGobGta0GaeyyeIuoaaOqaaiqbdkgaIzaajaWaaSbaaSqaaiabdQgaQbqabaGccqGGOaakcqaHZoWzcqGGPaqkcqGH9aqpjuaGdaWcaaqaaiabdwgaLnaaDaaabaGaemOAaOgabaGaemyrauKaemOta4KaemiraqeaaiabcIcaOiabeo7aNjabcMcaPaqaamaaqadabaGaemyzau2aa0baaeaacqWGQbGAaeaacqWGfbqrcqWGobGtcqWGebaraaGaeiikaGIaeq4SdCMaeiykaKcabaGaeq4SdCMaeyypa0JaeGymaedabaGaemiraqeacqGHris5aaaakiabc6caUaaaaaa@6B53@

The algorithm for discrete emission parameters estimate B = {b₁(o),..., b_N (o)} takes in O(NED) memory and O(TNEDQ_max) time. As discussed [see Subsection HMM definition, EM learning and Viterbi decoding] the forward sweep takes O(TNQ_max) time, where only the values of α_t-1(i) for 1 ≤ i ≤ N are needed to evaluate αt(i), thus reducing the memory requirement to O(N) for the forward algorithm. Computing e_i(γ, t, m) takes O(NED) previous probabilities of e_i(γ, t - 1, m) for 1 ≤ m ≤ N, 1 ≤ i ≤ E, 1≤ γ ≤ D. Recurrent updating of each e_i(γ, t, m) probability elements takes O(Q_max) summations, totalling O(TNEDQ_max).

Theorem 1 e_i(γ, t, m) is the weighted sum of probabilities of all possible state paths that emit subsequence o₁,..., o_t and finish in state m, for which state i emits observation γ at least once.

Initialization The only chance for a path at a time-point 1 to emit symbol γ at least once from state i and finish in state m is α₁(m) in case i = m and γ = o₁. Therefore, initialization conditions satisfy definition of e_i(γ, t, m).

Induction Suppose e_i(γ, t - 1, m) represents correct weighted sum of probabilities of all possible state paths that emit subsequence o₁,..., o_t-1 and finish in state m, for which state i emits observation γ at least once. We need to prove the above holds for time-point t. Following equation (1) in recurrence part (5) we score the evidence of symbol o_t emission from state i at time-point t, which will be further propagated down the trellis in recurrence part (6). Chances of such event is α_t(m), i.e. sum of probabilities of all possible state paths finishing in state m at time-point t under conditions i = m and γ = o_t. The second part of the recurrence (6) extends the weighted paths containing evidence of γ symbol emission from state i at previous time-points 1,..., t - 1 and finishing in state n further down the trellis through available transitions a_{n, m}. Thus the definition of e_i(γ, t, m) holds true for the time-point t.

At the end of recurrence we marginalize the final state m out of probability e_i(γ, T, m) to get the weighted sum of all state paths making observation γ in state i at various time-points equivalent to the numerator of the discrete emission parameter estimate in Table 3, which is a weighted sum of all possible paths that score emissions evidence at certain time-points. By normalizing these scores we estimate the emission parameters.

The forward sweep strategy was originally formulated in 13 for HMMs with silent Start/End states, and automatically handles the prior probabilities estimates for the states as standard transitions connecting Start with other non-silent states. The prior transition estimates a_{Start, i} are naturally involved within recurrent updates of t_{i, j}(t, m), which takes an additional O(N²) memory if all N non-silent states have non-zero priors with time cost O(TN²Q_max). In order to compute the prior estimates in the conventional HMM formulation we need to know the backward probability at time-point 1, which requires calculation of the entire backward table. Therefore, in the next section we present a linear memory Baum-Welch algorithm modification built around a backward sweep with scaling, which only involves calculation of α₁(i) for 1 ≤ i ≤ N to estimate priors in O(N) time and O(N) memory.

The objective of the algorithm presented in this section is equivalent to that discussed previously [see Section Forward sweep strategy explained] based on forward probabilities of state occupation. However, by using the backward probabilities of state occupation we are able to estimate initial HMM state probabilities much more quickly. In the description that follows we introduce a new set of probabilities:

E_i(γ, t, m) – the weighted sum of probabilities of all possible state paths that emit subsequence o_t,..., o_T and finish in state m, for which state i emits observation γ at least once, where the weight of each state path is the number of γ emissions that it makes from state i.

T_{i, j}(t, m) – the weighted sum of probabilities of all possible state paths that emit subsequence o_t,..., o_T and finish in state m, taking i → j transition at least once, where the weight of each state path is the number of i → j transitions that it takes.

All calculations are based on backward probability β_t(i), but there is inevitably insufficient precision to directly represent these values for significantly long emission sequences. Therefore we scale the backward probability during the recursion.

The linear memory Baum-Welch implementation is shown in Figure 2, where E MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae8hmHueaaa@36AD@ is a set of nodes with free emission parameters and T MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaWenfgDOvwBHrxAJfwnHbqeg0uy0HwzTfgDPnwy1aaceaGae83eXtfaaa@376F@ is a set of nodes with free emanating transitions. Scaling relationships used at every iteration are rigorously proven [see Appendix A]. An alternative to scaling is relation (7) presented in 17 showing how to efficiently add log probabilities

log ⁡ ( ∑ t = 0 N − 1 p i ) = log ⁡ p 0 + log ⁡ ( 1 + ∑ i = 1 N − 1 e log ⁡ p i − log ⁡ p 0 ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGagiiBaWMaei4Ba8Maei4zaC2aaeWaaeaadaaeWbqaaiabdchaWnaaBaaaleaacqWGPbqAaeqaaaqaaiabdsha0jabg2da9iabicdaWaqaaiabd6eaojabgkHiTiabigdaXaqdcqGHris5aaGccaGLOaGaayzkaaGaeyypa0JagiiBaWMaei4Ba8Maei4zaCMaemiCaa3aaSbaaSqaaiabicdaWaqabaGccqGHRaWkcyGGSbaBcqGGVbWBcqGGNbWzdaqadaqaaiabigdaXiabgUcaRmaaqahabaGaemyzau2aaWbaaSqabeaacyGGSbaBcqGGVbWBcqGGNbWzcqWGWbaCdaWgaaadbaGaemyAaKgabeaaliabgkHiTiGbcYgaSjabc+gaVjabcEgaNjabdchaWnaaBaaameaacqaIWaamaeqaaaaaaSqaaiabdMgaPjabg2da9iabigdaXaqaaiabd6eaojabgkHiTiabigdaXaqdcqGHris5aaGccaGLOaGaayzkaaGaeiOla4caaa@671A@

The scoring functions used for the emissions updates are shown in Table 4. With discrete emission we sum all the backward probabilities of state occupation given discrete symbol emission at certain time-points and later we normalize these counts in (8). In the case of a normally distributed continuous PDF we sum emissions and emission deviation from state i mean squared. These sums are scaled by probability of state occupation. We use these counts to estimate the emission mean (9) and variance (10) for a given state.