In order to illustrate the shape of the DFs derived from the nonparametric model, a two dimension model which used Xcorr and ΔCn was investigated at first. Because Xcorr significantly correlate with the charge state (+1, +2, and +3) 15, the matches with different charge states were processed individually. Since a large percentage of correct matches have a double charge, the matches in the control dataset with a double charge are discussed here. Using a trial and error approach, a model with 3 Gaussian functions (18 variables, Table 1) fit the distribution well (χ² goodness of fit test; significance level = 0.05). Figure 1A and Figure 1B show the histogram and density function, respectively. The estimated error for each bin is shown in Figure 1C. The small error (≤ 3.6 × 10^-3) also demonstrates that the fit is accurate.

DFs that can simultaneously reject as many false positives as possible and accept as many true positives as possible are preferred. Thus, the region in the feature space with fewer random matches is more preferred, and the contour lines of the PDF of the random matches are good candidate DFs (Figure 1D). Generally, random matches have a small ΔCn and Xcorr, while correct matches have a large ΔCn and Xcorr. Correct matches with the peptide isoform 44 have a small ΔCn and a large Xcorr. Matches with a small Xcorr and a large ΔCn may be due to the limited search space of the database searching. These matches are rare and more likely to be random matches; they may be localized to the accepted region of the contour line DFs because these results are also rare random events. A new DF of Xcorr was added to exclude such matches: Xcorr > m_Xcorr, where m_Xcorr is the mean of Xcorr of randomized database matches (bold red vertical line in Figure 1D). Given an expected FPRα, a target value f_α can be searched to ensure the calculated FPR (FPR_cal) is less than or equal to α. When searching for f_α, N_N and N_R were counted according to the rules:

∑ i = 1 N P ( i ) f G ( X | i ) ≤ f α MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaabCaeaacqWGqbaucqGGOaakcqWGPbqAcqGGPaqkcqWGMbGzdaWgaaWcbaGaem4raCeabeaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGobGta0GaeyyeIuoakiabcIcaOiabdIfayjabcYha8jabdMgaPjabcMcaPiabgsMiJkabdAgaMnaaBaaaleaaiiGacqWFXoqyaeqaaaaa@4448@

and

Xcorr > m_Xcorr

where X = (Xcorr, ΔCn) is the observation, and N = 3 is the number of Gaussian functions. Many f_α satisfied formula 3 and formula 4. The one with the largest N_N was used in the final DF. Figure 2 shows the DFs for different expected FPRs and different charge states. The shapes of the boundaries were significantly different, which indicates that it is difficult to fit all the distributions of different charge states with a simple distribution. The nonparametric model can provide feasible solutions to this complex problem. Since the resulting DFs are smooth, this method is more robust than the K nearest neighbor method 41.

One obvious advantage of the nonparametric model is that it can easily integrate more scores for validating peptide identifications. By taking into account more features and performing the classification in a high-dimension feature space, a more reasonable DF can be found, and thus, higher sensitivity can be achieved. Here, another powerful parameter called Sim introduced by Zhang 45 in 2004 and discussed by Sun et al. 31 recently was added to the nonparametric model. Sim measures the similarity between the experiment and the predicted MS/MS spectrum which was generated by the kinetic model introduced by Zhang 45 and the mass error tolerance for aligning the ions was specified as 0.5.

For the LCQ control dataset, by trial and error, we found a nonparametric model with 5 component GDFs can work well (65 parameters). We also tried a model with 7 component GDFs, but its performance was not improved and two of the component GDFs had a coefficient P_i near 0 [see Additional file 1]. Thus, we selected 5 component GDFs to build the model. When the expected FPR was 0.05 and 0.01, the actual FPR was 0.044 and 0.012, respectively. The number of peptide matches after filtering was 765 and 699, which were 104 (approximately 15.6%) and 121 (approximately 20.9%) respectively higher than the results of the nonparametric model using Xcorr and ΔCn, respectively. The sensitivity increased to 0.879 and 0.822 respectively, and the specificity did not change. Thus, by incorporating more features, the nonparametric model can provide greater discriminating power. In the following part of this paper, we discussed the nonparametric model with three features: Xcorr, ΔCn and Sim only. All the model parameters used in this paper were provided in Additional file 1.

The control datasets were generated by analyzing a set of known proteins and peptides with MS/MS platforms, which were commonly used to validate the performance of mathematical models for peptide identification 46. Table 2 reports the actual FPR and the number of validated matches at two commonly expected FPRs of 0.05 and 0.01. From Table 2, the following propositions can be made:

(1) In most cases, the FPRs estimated by formula 2 were close to but larger than the actual FPRs. Thus, the quality of the resulting datasets was better than claimed. It facilitates the strict result quality control but some sensitivity is lost.

(2) For little datasets, such as +1 charge state matches of different instruments, the actual FPR was larger than the corresponding estimated FPR. The error of the FPR estimation was also a bit larger. This result agrees with the conclusions of Huttlin et al 33.

(3) The estimated FPRs were not equal but close to the expected FPR. The smaller the resulting datasets, the larger the difference between estimated FPR and expected FPR. This arises from the rounding error in formula 2. For example, with an expected FPR of 0.01, the allowable number of random matches was less than 1 for the +1 charge dataset of LCQ, because only 62 matches were left after filtering. Thus, it is impossible to have an estimated FPR exactly equal to 0.01. A preferred alternative is rounding the estimated FPR to 0 (Table 2).

(4) The error of the FPR estimation at the expected FPR of 0.01 is larger that of 0.05. This result means that some unexpected contaminants exist. For example, in the LCQ control dataset, peptide "HVGDLGNVTADK" was identified with high database scores Xcorr = 4.5837, ΔCn = 0.542204) and the matched percentage of predicted ions reached 91% (Figure 3). This peptide comes from protein sp|P00441| SODC_HUMAN, which is not a protein in the control sample. But this peptide also belongs to protein sp|P00442|SODC_BOVIN, which may be contaminants in the sample because 4 proteins (ALBU_BOVIN, LACB_BOVIN, LCA_BOVIN and CYC_BOVIN) of bovine were added to the control sample.

(5) Manually checking the confirmed matches by the nonparametric model, we found that some results with large Xcorr but very small ΔCn were confirmed. In some cases, the peptide in the second rank was correct. For example, in the LTQ dataset (D2), a peptide "LEAELEK" was identified with Xcorr = 2.4273 and ΔCn = 0.0533 (+1 charge state). The peptide at the second rank was "LEALEEK", a peptide from control protein P62937|PPIA_HUMAN, because of the theoretic mass spectrum similarity between these peptides, which will result in some FPR estimation error.

Two other methods were also be widely used in the proteomic research. The first one (named M1) searches for the optimized cut-off values of Xcorr and ΔCn simultaneously while making the number of confirmed matches reached its maximum given an expected FPR. The resulting accepted region on the Xcorr-ΔCn plane is a rectangle. The second one (named M2) is Peptideprophet (V1.9), which is an empirical statistic model, introduced by Keller et al 15. PeptideProphet provided the estimated error rates (EER) at different probability score cut-offs. EER has similar meaning with FPR, so we used it as the measure of the quality of the resulting dataset and only the probability score cut-offs without additional criterion were used to filter the matches. In order to name it easily, we denote the nonparametric model as M3 in the following part of this paper. For the control datasets, the confirmed matches, the actual FPR and the sensitivity were listed in Table 3 (The filter criteria can be found in Additional file 1). Some conclusions can be drawn:

(1) In each case, the sensitivity of M3 is the highest. The difference in sensitivity of different methods ranges from 3.8% to 27.7%.

(2) For the LCQ and LTQ dataset, the performance of M1 and M2 differs little and Peptideprophet (M2) which was trained by a LCQ control dataset 15 does not seem to work well on the LTQ/FT dataset.

(3) The performance of the nonparametric model differs little on the dataset of different instruments. When the expected FPR is 0.05, the sensitivity is above 0.85 and it is above 0.74 when the expected FPR is 0.01.

(4) FPR estimation errors exist for different methods. In some cases, the error is large. This may be caused by the calculation errors because of unexpected contaminants and random errors.

Shotgun experiments always generate large datasets 8. Thus, the nonparametric model demonstrated to be effective with the control dataset should be validated using large datasets. At first, we investigated the quality of the confirmed matches by the nonparametric model (The filter criteria can be found in Additional file 7). Another 6 parameters which were commonly used to validate the peptide identifications of SEQUEST database search results were calculated for each match. They are maximal continuous b or y ion series length (CSL) 11, the matched percentage of the predicted ions by SEQUEST (Ions) 44, ranked preliminary score (RSp) 44, the continuity of b or y ion series (Cont) 13, the matched percentage of ion intensities in the experiment mass spectrum (iIons) 13 and the matched percentage of the peak number in the experiment mass spectrum (nIons) 23. The percentages of the confirmed results which passed the empirical rules (Table 4) convinced us that most of these matches had a high confidence level. It must be noted that RSp = 1 is a strict rule 44 and some correct matches may be lost if we require RSp = 1. For instance, only 76% correct matches are with RSp = 1 in the LTQ control dataset.

As a case study, we investigated the overlaps of the three methods on the LTQ dataset. More than 90% of the matches confirmed by M1 or M2 were covered by M3 (Figure 4), and 89.1 (FPR = 0.05) and 83.6 (FPR = 0.01) of the matches confirmed by the nonparametric model were covered by M1 ∪ M2. Each method of the three can all provide some matches that are not covered by the other two because they utilize different filter boundaries and different parameters.

Figure 5A shows the mesh grids of a DF of M3 (+2 charge state matches in D5, FPR = 0.01). As it appears, the matches with the smaller Xcorr, ΔCn or Sim were discarded by M3, which agrees with the experience that the matches with large scores (Xcorr, ΔCn or Sim) are more possibly correct. Figure 5B~Figure 5E illustrate the score distributions of the matches uniquely confirmed by M1~M3. It is clear that some matches with small Xcorr, ΔCn and Sim were confirmed by PeptideProphet (red points), which integrated some other parameters, such as preliminary score (Sp). M2 confirmed some matches with middle Xcorr and ΔCn but small Sim (green points). M3 confirmed many matches (4714) with relative smaller Xcorr and ΔCn but large Sim, which were discarded by M1 and M3. These results demonstrated that different filter boundaries with different parameters would generate different results with different sensitivity and integrating more complementary parameters by appropriate methods could improve the sensitivity of database search result validation.

In Table 5, we gave the numbers of confirmed matches, non-redundant peptides, identified proteins (Minimal protein list assembled by DBParser algorithm 47) and the percentage of proteins with at least 2 or 3 peptide hits (The filter criteria can be found in Additional file 7). The nonparametric model can confirm up to 14.5% more proteins than the other two kinds of methods, which indicated that our model has a higher sensitivity. For the same kind of instrument, three methods gave about the same percentage of proteins with at least 2 or 3 peptide hits at different confidence levels. The percentage of proteins with at least 2 peptide hits reaches above 50% for the LCQ or LTQ dataset, but it is about 40% for the LTQ/FT dataset. It is interesting that the percentage of proteins with at least 2 or 3 peptide hits can not be improved by improving the confidence level of the peptide identifications when one method is used.

Six datasets generated by three kinds of mass spectrometry platforms (LCQ, LTQ and LTQ/FT) were used to demonstrate the performance of the nonparametric model. Three control datasets were used to validate the accuracy of the FPR estimation and the improvement of the sensitivity. Since the MS/MS datasets generated by the shotgun technique are always large, we also verified the generality of the nonparametric model on the large real sample datasets. The basic information about the six datasets is listed in Table 6.

The two unpublished LTQ/FT datasets were provided by Beijing Proteome Research Center (BPRC). The samples were digested with trypsin and then analyzed by a 7-Tesla LTQ/FT mass spectrometer (Thermo Electron, San Jose, CA) coupled with an Agilent 1100 nano-flow liquid chromatography system. The reverse phase C18 trap columns (300 μm internal diameter × 5 mm long column) were connected with the 6-port column-switching valve for the on-line desalting. A PicoFritTM tip column (BioBasic C18, 5 μm particle size, 75 μm internal diameter × 10 cm long column, 15 μm internal diameter at spray tip, New Objective, Woburn, MA, USA) was used for the following separation. Elution was solvent A (Milli-Q water, 2 % acetonitrile and 0.1%FA, v/v/v) and solvent B (Milli-Q water, 80% acetonitrile and 0.1%FA, v/v/v). The gradient was 15–40% B in 40 min, 40–100% B in 10 min. One FT full MS scan was followed by 5 data-dependent LTQ MS/MS scans on the five most intense ions. The dynamical excluding time was 45 seconds. Ions were accumulated in linear ion trap controlled by AGC. The AGC values were 5 × 10⁵ charges for FT full MS scan and 1 × 10⁴ charges for LTQ MS/MS scan. The resolution was 10,000 for FT full MS scan at m/z 400. The temperature of the ion transfer tube was set at 200°C and the spray voltage was 1.8 KV. The isolation width was 4Da and normalized collision energy was 35% for MS/MS scan. Mass spectra were acquired over the m/z range from 400 to 2000.

All the MS/MS spectra were extracted from the *.raw files by Extract_MSn.exe which is a console program in Bioworks 3.2 (Thermo Finnigan, San Jose, CA). For the LCQ datasets, the minimal total ion intensity is 10,000. For the LTQ or LTQ/FT datasets, the total ion intensity of each MS/MS spectrum is required to exceed 100. For all the datasets, the spectra must have at least 20 ions. Then the database search was performed on a local TurboSEQUEST (version 2.7) server. The fixed modification of oxidation (15.99Da) on the Met residue and the variable modification of carboxyamidomethylation (57.02Da) on the Cys residue were set. The enzyme was trypsin and the maximal allowed missed cleavage sites was 2. Only the b and y ions were taken into account. For the LCQ or LTQ datasets, the precursor mass error tolerance was 3.0Da, and for the LFQ/FT datasets, it was 15ppm.

For all the datasets except D2, which was searched against the database published by sPRG 55, the searched databases were derived from IPI Human 3.19 60. For the control datasets, the control sequences for dataset D1 and D3 [see Additional file 3, 4 and 5] including the sequences of purified proteins or peptides plus the typical sample contaminants such as keratin and trypsin were added into the IPI Human 3.19. The control sequences for D2 were determined according to the report of sPRG (see Additional file 4) 55. The databases were constructed using the method proposed in one of our previous paper 58 and could be described as: the protein sequences in the normal database were digested in silico (trypsin), and then the amino acid residues (AAR) (except the one on the C-terminal) of the resulting peptides were reshuffled by using a random number generator. Then the reshuffled peptides were spliced to form new protein sequences in the randomized database. Finally, the normal database and the randomized database were merged to form the searched database.

After database searching, the matches with +1, +2 and +3 charge state were extracted (Table 7). For each spectrum, only the first rank match with an assigned peptide with more than 5 AAR was taken into account for further analysis. For the control datasets, the matches which were assigned peptides of control sequences were validated by the following criteria: 1) the b-ion or y-ion series should confirm at least 3 consecutive amino acids of the assigned peptide sequence 12, 2) ranked preliminary score (RSp) ≤ 50. The confirmed matches of control datasets were provided in the supplementary materials [see Additional file 6, 7 and 8).

The workflow of the nonparametric model based method is shown in Figure 7. Firstly, a randomized database was constructed by randomizing the tryptic peptide sequence. Then the MS/MS spectra were searched against the combined database using SEQUEST. Then, matches with an assigned peptide from the randomized database (we call them randomized database matches, RDM) were used to build the nonparametric model. The joint distribution of selected parameters (such as Xcorr, ΔCn and Sim 3145) of random matches was fit with the nonparametric model using the FnPDFe method and the contour lines of the estimated PDF, which are complex nonlinear functions, were used as candidate DFs. The actually used DFs were determined according to the expected FPR and formula 2 for different charge states. Finally, the resulting DFs were used to filter the matches from the normal database. In the model-building step, k-means clustering was used to initialize the EM algorithm procedure.

K-means clustering 59 is commonly used to partition observations into different groups according to defined distance (such as Euclidean distance). The optimization goal of k-means clustering is to find a partition in which objects within each cluster are as close as possible to each other and as far as possible from objects in other clusters. However, in practice, the scale of each feature will significantly affect the clustering results when Euclidean distance is used. In our application, Xcorr and ΔCn were two main features. Xcorr is a float point value whose typical value is 2.5 but may be larger than 10; ΔCn is in the range [0, 1]. When directly using the observed values in the k-means clustering, Xcorr will dominate the partition results (Figure 8) because the distance (formula 5) between two observations (Xcorr_i, ΔCn_i), i = 1, 2, is mainly determined by Xcorr, which has a larger scale.

d = ( X c o r r 1 − X c o r r 2 ) 2 + ( Δ C n 1 − Δ C n 2 ) 2 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemizaqMaeyypa0ZaaOaaaeaacqGGOaakcqWGybawcqWGJbWycqWGVbWBcqWGYbGCcqWGYbGCdaWgaaWcbaGaeGymaedabeaakiabgkHiTiabdIfayjabdogaJjabd+gaVjabdkhaYjabdkhaYnaaBaaaleaacqaIYaGmaeqaaOGaeiykaKYaaWbaaSqabeaacqaIYaGmaaGccqGHRaWkcqGGOaakcqqHuoarcqWGdbWqcqWGUbGBdaWgaaWcbaGaeGymaedabeaakiabgkHiTiabfs5aejabdoeadjabd6gaUnaaBaaaleaacqaIYaGmaeqaaOGaeiykaKYaaWbaaSqabeaacqaIYaGmaaaabeaaaaa@50D2@

Thus, a normalization step, which calculated the z-score of the observed values of each feature, was used to eliminate the scale difference, and thus achieve a more reasonable partition (Figure 8).

The basic objective of nonparametric density estimation is to approximate the distribution of observations using the weighted sum of a series of simple functions, which does not emphasize the physical meaning of the parameters but the accuracy of the approximation. This idea can be implemented using smoothing splines or radial basis function (RBF) neural network to fit the histogram directly 41. Another way to implement the nonparametric model is to fit the distribution with kernel density functions. The optimization goal of the nonparametric model is to minimize the mean integrated squared error of the fit or to maximize the maximum likelihood function of the observations. Many kinds of nonparametric models have been proposed by different researchers 41. The FnPDFe procedure 42 is attractive because it is easy to implement and has a clear statistical explanation. Let X be a d dimension random vector X ∈ R^d. Its PDF can be approximated by a Gaussian mixed model that is defined as the linear combination of N multivariate Gaussian density functions (MGDFs):

f ( X ) = ∑ i = 1 N P ( i ) f G ( X | i ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemOzayMaeiikaGIaemiwaGLaeiykaKIaeyypa0ZaaabCaeaacqWGqbaucqGGOaakcqWGPbqAcqGGPaqkcqWGMbGzdaWgaaWcbaGaem4raCeabeaaaeaacqWGPbqAcqGH9aqpcqaIXaqmaeaacqWGobGta0GaeyyeIuoakiabcIcaOiabdIfayjabcYha8jabdMgaPjabcMcaPaaa@44B2@

where:

f G ( X | i ) = 1 ( 2 π ) d / 2 | Σ i | 1 / 2 e − 1 2 ( X − μ i ) T Σ i − 1 ( X − μ i ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemOzay2aaSbaaSqaaiabdEeahbqabaGccqGGOaakcqWGybawcqGG8baFcqWGPbqAcqGGPaqkcqGH9aqpjuaGdaWcaaqaaiabigdaXaqaamaabmaabaGaeGOmaidcciGae8hWdahacaGLOaGaayzkaaWaaWbaaeqabaGaemizaqMaei4la8IaeGOmaidaamaaemaabaGaeu4Odm1aaSbaaeaacqWGPbqAaeqaaaGaay5bSlaawIa7amaaCaaabeqaaiabigdaXiabc+caViabikdaYaaaaaGccqWGLbqzdaahaaWcbeqaaiabgkHiTKqbaoaalaaabaGaeGymaedabaGaeGOmaidaaSGaeiikaGIaemiwaGLaeyOeI0Iae8hVd02aaSbaaWqaaiabdMgaPbqabaWccqGGPaqkdaahaaadbeqaaiabdsfaubaaliabfo6atnaaDaaameaacqWGPbqAaeaacqGHsislcqaIXaqmaaWccqGGOaakcqWGybawcqGHsislcqWF8oqBdaWgaaadbaGaemyAaKgabeaaliabcMcaPaaaaaa@614C@

and P(i),i = 1,...N satisfies: (1) 0 <P(i) ≤ 1; (2) ∑i=1NP(i)=1 MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaWaaabCaeaacqWGqbaucqGGOaakcqWGPbqAcqGGPaqkcqGH9aqpcqaIXaqmaSqaaiabdMgaPjabg2da9iabigdaXaqaaiabd6eaobqdcqGHris5aaaa@38B7@. μ_i, Σ_i is the mean vector and covariance matrix of the i-th MGDF.

Consider independent and identically distributed observations set{x₁, x₂,......x_n}; the log-likelihood function of the mixed model is:

L ( θ ) = ∑ k = 1 n ln ⁡ f ( x k ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemitaWKaeiikaGccciGae8hUdeNaeiykaKIaeyypa0ZaaabCaeaacyGGSbaBcqGGUbGBcqWGMbGzcqGGOaakcqWG4baEdaWgaaWcbaGaem4AaSgabeaakiabcMcaPaWcbaGaem4AaSMaeyypa0JaeGymaedabaGaemOBa4ganiabggHiLdaaaa@418E@

Generally, MLE can be used to infer the parameters θ in the mixed model. However, the resulting MLE equations cannot be solved analytically. The FnPDFe method uses the EM algorithm to provide iterative solutions for these parameters 43, which can be read as:

(1) Initial step: Initialize the objective parameters μ_i, Σ_i, and P(i) with heuristic knowledge or random values.

(2) E-step: update the posterior distributions:

g t + 1 ( i | x k ) = f G t ( x k | i ) P t ( i ) ∑ j = 1 N f G t ( x k | j ) P t ( j ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaem4zaC2aaWbaaSqabeaacqWG0baDcqGHRaWkcqaIXaqmaaGccqGGOaakcqWGPbqAcqGG8baFcqWG4baEdaWgaaWcbaGaem4AaSgabeaakiabcMcaPiabg2da9KqbaoaalaaabaGaemOzay2aa0baaeaacqWGhbWraeaacqWG0baDaaGaeiikaGIaemiEaG3aaSbaaeaacqWGRbWAaeqaaiabcYha8jabdMgaPjabcMcaPiabdcfaqnaaCaaabeqaaiabdsha0baacqGGOaakcqWGPbqAcqGGPaqkaeaadaaeWbqaaiabdAgaMnaaDaaabaGaem4raCeabaGaemiDaqhaaiabcIcaOiabdIha4naaBaaabaGaem4AaSgabeaacqGG8baFcqWGQbGAcqGGPaqkcqWGqbaudaahaaqabeaacqWG0baDaaGaeiikaGIaemOAaOMaeiykaKcabaGaemOAaOMaeyypa0JaeGymaedabaGaemOta4eacqGHris5aaaaaaa@6373@

(3) M-step: estimate the current parameters:

P t + 1 ( i ) = 1 n ∑ k = 1 n g t ( i | x k ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaemiuaa1aaWbaaSqabeaacqWG0baDcqGHRaWkcqaIXaqmaaGccqGGOaakcqWGPbqAcqGGPaqkcqGH9aqpjuaGdaWcaaqaaiabigdaXaqaaiabd6gaUbaakmaaqahabaGaem4zaC2aaWbaaSqabeaacqWG0baDaaGccqGGOaakcqWGPbqAcqGG8baFcqWG4baEdaWgaaWcbaGaem4AaSgabeaakiabcMcaPaWcbaGaem4AaSMaeyypa0JaeGymaedabaGaemOBa4ganiabggHiLdaaaa@496A@

μ i t + 1 = ∑ k = 1 n g t ( i | x k ) x k ∑ k = 1 n g t ( i | x k ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaacciGae8hVd02aa0baaSqaaiabdMgaPbqaaiabdsha0jabgUcaRiabigdaXaaakiabg2da9KqbaoaalaaabaWaaabCaeaacqWGNbWzdaahaaqabeaacqWG0baDaaGaeiikaGIaemyAaKMaeiiFaWNaemiEaG3aaSbaaeaacqWGRbWAaeqaaiabcMcaPaqaaiabdUgaRjabg2da9iabigdaXaqaaiabd6gaUbGaeyyeIuoacqWG4baEdaWgaaqaaiabdUgaRbqabaaabaWaaabCaeaacqWGNbWzdaahaaqabeaacqWG0baDaaGaeiikaGIaemyAaKMaeiiFaWNaemiEaG3aaSbaaeaacqWGRbWAaeqaaiabcMcaPaqaaiabdUgaRjabg2da9iabigdaXaqaaiabd6gaUbGaeyyeIuoaaaaaaa@59F8@

Σ i t + 1 = ∑ k = 1 n g t ( i | x k ) ( x k − μ i t ) T ( x k − μ i t ) ∑ k = 1 n g t ( i | x k ) MathType@MTEF@5@5@+=feaagaart1ev2aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGacaGaaiaabeqaaeqabiWaaaGcbaGaeu4Odm1aa0baaSqaaiabdMgaPbqaaiabdsha0jabgUcaRiabigdaXaaakiabg2da9KqbaoaalaaabaWaaabCaeaacqWGNbWzdaahaaqabeaacqWG0baDaaGaeiikaGIaemyAaKMaeiiFaWNaemiEaG3aaSbaaeaacqWGRbWAaeqaaiabcMcaPiabcIcaOiabdIha4naaBaaabaGaem4AaSgabeaacqGHsisliiGacqWF8oqBdaqhaaqaaiabdMgaPbqaaiabdsha0baacqGGPaqkdaahaaqabeaacqWGubavaaGaeiikaGIaemiEaG3aaSbaaeaacqWGRbWAaeqaaiabgkHiTiab=X7aTnaaDaaabaGaemyAaKgabaGaemiDaqhaaiabcMcaPaqaaiabdUgaRjabg2da9iabigdaXaqaaiabd6gaUbGaeyyeIuoaaeaadaaeWbqaaiabdEgaNnaaCaaabeqaaiabdsha0baacqGGOaakcqWGPbqAcqGG8baFcqWG4baEdaWgaaqaaiabdUgaRbqabaGaeiykaKcabaGaem4AaSMaeyypa0JaeGymaedabaGaemOBa4gacqGHris5aaaaaaa@6C93@

(4) Repeat steps 2–3 until the change of parameters is very little.

One problem with implementation of the EM algorithm is how to initialize the parameters. Use of an improper starting point may prolong the converging time of the EM algorithm or cause it to reach a local minimum. In this work, k-means clustering was used to partition the observations into subclasses, and the means and covariance matrixes of the component Gaussian distributions were initialized using the means and covariance matrixes of the subclasses.

Another difficulty in implementing the EM algorithm is the selection of the number of component density functions. Generally speaking, inclusion of more functions will approximate the distributions of the observations more accurately, while allowing more parameters to be determined. However, overly complex models may cause the EM algorithm to reach a local minimum and worsen the performance of the resulting model. In this work, a trial and error procedure was used to select the minimum number of component density functions: try numbers from 2 until the change of the likelihood function value is very little (such as less than 1%).