We briefly overview some relevant recent works which have explored different classification, discrimination and clustering techniques to represent the separation between two groups of prognosis signature patients. The studies of van't Veer's group 34 have suggested that a PGL of 70 specifically selected genes has proven accurate in out-of-sample patient prognosis of metastasis.

However, other studies have concluded that the likelihood there exists a 'best' small-size predictive gene list which can be used to reliably improve the ability of patient-specific prognosis using automated pattern processing techniques is unlikely. In very recent work 5, analysing supervised machine learning approaches across several public domain data sets, it was found that many gene sets are capable of predicting molecular phenotypes accurately. Hence it is not surprising that expression profiles identified using different training datasets selected from a larger cohort, should show little agreement. It was also demonstrated that predicting relapse directly from microarray data using supervised machine learning approaches was not viable.

In other work 6, it was shown that the specific example of the van't Veer PGL selection of 70 genes was no more effective at prognosis than the Nottingham Prognostic Indicator (NPI) or a suitably trained artificial neural network using traditional non-genomic biomarkers. This is not surprising from a systems biology perspective, where we would regard cancer as the result of complex interactions between genetic, biological and environmental influences.

In 1, they also found that the top 70 most correlated genes in the van't Veer study can vary significantly depending on the specific training set of patients used. Different randomly selected 70 gene PGL's were selected and shown to have similar prediction ability. They suggested that there is no unique set of genes that can be assumed to be the best or the only set of genes for prognosis accuracy of breast cancer. A follow-up study 2 also suggested a similar conclusion, that we can not create a definitive classifier from a small subset of genes based on the small patient datasets available. Generally, large patient sample sizes are needed to produce viable and robust prediction outcomes of cancer prognosis.

We re-visit the well-known study of van't Veer et.al. 3 in which we focus on 78 sporadic lymph-node negative patients. Of these 78 patients, 34 developed distant metastases within 5 years and 44 remained free of cancer in that period. These are regarded as poor and good-prognosis groups respectively. The interest is whether the information in a gene expression profile alone could be used to perform a patient-specific prognosis separation between those two groups of patients. We will primarily use structure-preserving projective visualisation techniques to investigate this possibility. In the van't Veer study, from an initial set of 24481 human genes synthesised by inkjet microarray technology, about 5000 genes were found to be significantly expressed. They ranked genes by the magnitude of the correlation coefficient and eventually reduced the number of genes to 70, the number of genes which maximised a specific classification model. The centroid-based classifier they constructed could allocate 83% of the patients into the correct prognosis groups with 5 poor prognosis and 8 good-prognosis patients misclassified into the opposite categories.

To illustrate the lack of uniqueness of capability of the van't Veer gene list, which we denote List A in this paper, we compare results on a different gene list, denoted List B, selected on the basis of cross-patient consistency rather than maximising classification accuracy on a specific classification model. Let xⁱ denote the gene expression vector for patient i of the van't Veer PGL. x_G, where G = {1, 2,..., 44} represents a set of expression values across all good prognosis patients, and x_P, where P = {45, 46,..., 78} represents the set of all poor prognosis patients.

The variance of individual gene expression values across each patient group is estimated by

σ L 2 = 〈 ( x i − x ¯ L ) 2 〉 i ∈ L , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeq4Wdm3aa0baaSqaaiabdYeambqaaiabikdaYaaakiabg2da9iabgMYiHlabcIcaOGqabiab=Hha4naaBaaaleaacqWGPbqAaeqaaOGaeyOeI0Iaf8hEaGNbaebadaWgaaWcbaGaemitaWeabeaakiabcMcaPmaaCaaaleqabaGaeGOmaidaaOGaeyOkJe=aaSbaaSqaaiabdMgaPjabgIGiolabdYeambqabaGccqGGSaalaaa@438B@

where L = {G, P} and the average is taken across all patients. Assume RjL MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemOuai1aa0baaSqaaiabdQgaQbqaaiabdYeambaaaaa@2FAD@ is the rank order of the variance of gene j for each patient group. The unique top T ranked genes from each group are extracted,

L G = { j | R j G ≤ T } L P = { j | R j P ≤ T } MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaqbaeaabiqaaaqaaiabdYeamnaaBaaaleaacqWGhbWraeqaaOGaeyypa0Jaei4EaSNaemOAaOMaeiiFaWNaemOuai1aa0baaSqaaiabdQgaQbqaaiabdEeahbaatuuDJXwAK1uy0HMmaeHbfv3ySLgzG0uy0HgiuD3BaGabaOGae8hzIqOaemivaqLaeiyFa0habaGaemitaW0aaSbaaSqaaiabdcfaqbqabaGccqGH9aqpcqGG7bWEcqWGQbGAcqGG8baFcqWGsbGudaqhaaWcbaGaemOAaOgabaGaemiuaafaaOGae8hzIqOaemivaqLaeiyFa0haaaaa@55C7@

The number of T genes is chosen so that List B has a total number of genes equal to 70, the same as List A. Specifically, in this case the 35 lowest non-overlapped variance genes from each patient group were extracted.

List B = {L_G ∪ L_P} - {L_G ∩ L_P}.

This selection criterion emphasises consistency of gene expression across patients, rather than explicitly seeking discrimination (see table 25 for list of genes). Examining the details of the two 70-gene subsets, we observe that there are only five genes in common between the van't Veer study and this alternative gene list. If List A has superior prognostic value, its projective visualisation and discrimination properties should be better than those of List B, since List A was chosen explicitly to maximise discrimination.

A follow-up study by the same group was performed which followed the progression of another set of patients to verify the original study. This follow-up data set contains 295 patients with 106 poor-prognosis and 189 good-prognosis patients. The poor-prognosis patients are categorised into 3 sub-categories.

Patients with metastasis but who did not die as a direct result of the metastasis, patients who died without developing metastasis and the patients who developed metastasis and eventually died. Within these 295 patients, 61 of them are present in the previous study. Those patients will be removed in this paper to ensure we can examine generalisation on a separate set of 234 different patients, 159 of whom are categorised as good-prognosis.

Topographic mappings are mechanisms that map the data in a high dimensional space into a low dimensional space in such a way that preserves the structure of the data. This structure of the data usually means the relative distances between points in the high dimensional space, where a suitable distance function is used to reflect the prior knowledge of the domain. In other words, the points that are close together in a high dimensional space should also stay together in a lower dimensional projection space, and data that lie far apart in a high dimensional feature space should remain significantly separated in the lower dimensional projection space.

Both the large quantity of genes and multiple samples of microarray data make it difficult to represent the entire data set and to identify the interesting genes easily. To make understanding easier, a representation is needed to compress all the information in a lower dimensional space. Moreover, each individual patient can be visualised as to whether that patient has expression values closer to the those patients in different prognosis groups. Topographic projection maps do not assume the existence of clusters, boundaries or classes and so are not subject to the same criticisms as supervised approaches for data mining.

In this paper, three reliable topographic techniques are used for embedding the van't Veer data set: NeuroScale 14, Locally Linear Embedding (LLE)and Stochastic Neighbor Embedding (SNE) 17.

In a Neuroscale 14 topographic map the distribution and trained relative positions of the points in the projection space are determined to reflect the relative dissimilarity between data measurements (gene expression values) in the high-dimensional space, and hence generalises the established Sammon map concept. N measurement vectors x_i in ℝ^p are transformed using a Radial Basis Function (RBF) 1819 network to a corresponding set of feature (visualisation) vectors y_i in ℝ^q. An RBF comprises a single hidden layer of h neurons which represents a set of basis functions, each of which has a centre located at some point in the input space. Generally, q ≪ p as dimension reduction is desired, and typically q = 2 for visualisation. The RBF is a semi-parametric kernel model such that y_i = W Φ (r_i), where the set of weights W can be optimised from a training data set. Thin plate spline functions Φ (r) = r² log(r), where ri2 MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaacbeGae8NCai3aa0baaSqaaiabdMgaPbqaaiabikdaYaaaaaa@2FC2@ = ||x_i - c|| will be used in this experiment. In the Radial Basis Function network trained traditionally for regression problems, the desired output of the network or the target values are already identified through a supervised learning problem. However, in order to use an RBF in the topographic reduction problem, which does not have predefined targets but only considers the distance-preserving nature of the target values, the traditional supervised Radial Basis Function network needs to be modified by using relative supervision where it is only the relative distances between pattern pairs which are important 15. The quality of the projection is measured by the Sammon stress metric (n.b. we are using a reduced form here, neglecting a denominator often employed):

E = ∑ i N ∑ j N ( d i j ∗ − d i j ) 2 , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyrauKaeyypa0ZaaabCaeaadaaeWbqaaiabcIcaOiabdsgaKnaaDaaaleaacqWGPbqAcqWGQbGAaeaacqGHxiIkaaGccqGHsislcqWGKbazdaWgaaWcbaGaemyAaKMaemOAaOgabeaakiabcMcaPmaaCaaaleqabaGaeGOmaidaaaqaaiabdQgaQbqaaiabd6eaobqdcqGHris5aaWcbaGaemyAaKgabaGaemOta4eaniabggHiLdGccqGGSaalaaa@45CD@

where d_ij = ||y_i - y_j|| and dij∗ MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemizaq2aa0baaSqaaiabdMgaPjabdQgaQbqaaiabgEHiQaaaaaa@30FA@ = ||x_i - x_j|| represent the inter-point distances in projection space and data space respectively. The aim of the training process is to set the parameters of the RBF weight matrix to minimise the stress metric and hence capture the the functional relationship between the original data distribution and the projected images. The NeuroScale model is used to visualise and interrogate our data set, considered as subsets of an array of 78 × 70 patterns. Once the functional mapping has been obtained using NeuroScale, the model can be reused without reconstructing the projection over the extended data by just passing the new data points, x_new through the transformation function. y_new = f (x_new, W).

The number and location of the centres needs to be determined. However choices of centres are robust to the outcome 20 since an implicit smoothing regularisation is used as part of the optimisation process. As a normal practice, the number of centres is chosen to be the same as the number of training data points, so that each data point can be used as a centre of the RBF functions. In this experiment the Netlab toolbox 7 is used to help construct the NeuroScale projections.

Locally Linear Embedding (LLE) 16 aims to preserve the local neighbourhood area around a point so that nearby points in high dimensional space remain nearby and similarly co-located with respect to one another in the low dimensional space by preserving the neighbouring distance linearly. Provided there is sufficient data, we expect each data point and its neighbours to lie on or close to a locally linear patch of the manifold. In the simplest formulation of LLE, the method will preserve weights for each data point to the surrounding K nearest neighbours per data point, as measured by Euclidean distance from a point of interest. The optimal weights for each data point to the surrounding K nearest neighbours are given by:

W i j = ∑ k C j k i − 1 ∑ l m C l m i − 1 . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4vaC1aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGH9aqpjuaGdaWcaaqaamaaqababaGaem4qam0aa0baaeaacqWGQbGAcqWGRbWAaeaacqWGPbqAcqGHsislcqaIXaqmaaaabaGaem4AaSgabeGaeyyeIuoaaeaadaaeqaqaaiabdoeadnaaDaaabaGaemiBaWMaemyBa0gabaGaemyAaKMaeyOeI0IaeGymaedaaaqaaiabdYgaSjabd2gaTbqabiabggHiLdaaaOGaeiOla4caaa@48E5@

where Cjki MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4qam0aa0baaSqaaiabdQgaQjabdUgaRbqaaiabdMgaPbaaaaa@3128@ is a covariance matrix within the neighbourhood of x_i and η_j is the neighbour of the data point x_i.

C j k i = ( x i − η j ) T ⋅ ( x i − η k ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4qam0aa0baaSqaaiabdQgaQjabdUgaRbqaaiabdMgaPbaakiabg2da9iabcIcaOGqabiab=Hha4naaBaaaleaacqWGPbqAaeqaaOGaeyOeI0Iaeq4TdG2aaSbaaSqaaiabdQgaQbqabaGccqGGPaqkdaahaaWcbeqaaiabdsfaubaakiabgwSixlabcIcaOiab=Hha4naaBaaaleaacqWGPbqAaeqaaOGaeyOeI0Iaeq4TdG2aaSbaaSqaaiabdUgaRbqabaGccqGGPaqkcqGGUaGlaaa@48F0@

Each high dimensional point x_i is mapped to a low dimensional point y_i in low dimensional space, representing global internal coordinates on the manifold. This is done by choosing d-dimensional coordinates y_i to minimise the cost function in low dimensional space:

Φ ( Y ) = ∑ i N | y i − ∑ j K W i j y j | 2 , MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaeuOPdyKaeiikaGIaemywaKLaeiykaKIaeyypa0ZaaabCaeaacqGG8baFieqacqWF5bqEdaWgaaWcbaGaemyAaKgabeaakiabgkHiTmaaqahabaGaem4vaC1aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqWF5bqEdaWgaaWcbaGaemOAaOgabeaakiabcYha8naaCaaaleqabaGaeGOmaidaaaqaaiabdQgaQbqaaiabdUealbqdcqGHris5aaWcbaGaemyAaKgabaGaemOta4eaniabggHiLdGccqGGSaalaaa@4B3C@

where W_ij is fixed from the high dimensional space. This algorithm has only one free parameter: the number of neighbours per data point, K. The higher the value of K, the more similar to the NeuroScale method this method will be. This K practically, is very hard to find to suit the given data set.

Furthermore, it is hard to find an appropriate value of K which performs well across different choices of data sets. It is typically much smaller than the number of data points. In the experiments here we will show results using K = 5, 20. Furthermore, LLE has a further disadvantage over NeuroScale in that the NeuroScale model can produce a transformation function which can be used for generalisation. Any new patient gene vector can be projected down by using this existing function without recomputing a full new projection, which can be very computational expensive. In addition, the result using LLE is sensitive to the choice of neighbours, K while NeuroScale gives quite consistent results 20.

Stochastic Neighbor Embedding 17 also uses a pairwise similarity but measures similarity using a probabilistic distance approach to preserve the neighbourhood identity. A Gaussian distribution is centred on each object point in the high dimensional space and a probability density is defined over all the potential neighbours of that point. This approach permits a 1-to-many mapping of high dimensional points to projection space.

The high dimensional related probability for each point, i, and each potential neighbour, j, is computed using the asymmetric probability, p_ij, that i would pick j as its neighbour.

p i j = exp ⁡ ( − d i j 2 ) ∑ k ≠ i exp ⁡ ( − d i k 2 ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiCaa3aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGH9aqpjuaGdaWcaaqaaiGbcwgaLjabcIha4jabcchaWjabcIcaOiabgkHiTiabdsgaKnaaDaaabaGaemyAaKMaemOAaOgabaGaeGOmaidaaiabcMcaPaqaamaaqababaGagiyzauMaeiiEaGNaeiiCaaNaeiikaGIaeyOeI0Iaemizaq2aa0baaeaacqWGPbqAcqWGRbWAaeaacqaIYaGmaaGaeiykaKcabaGaem4AaSMaeyiyIKRaemyAaKgabeGaeyyeIuoaaaGccqGGUaGlaaa@513A@

The dissimilarities, d_ij can be based on standard Euclidean distances and scaled by a smoothing factor σ_i which is empirically determined:

d i j = | | x i − x j | | σ i 2 . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemizaq2aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGH9aqpjuaGdaWcaaqaaiabcYha8jabcYha8Hqabiab=Hha4naaBaaabaGaemyAaKgabeaacqGHsislcqWF4baEdaWgaaqaaiabdQgaQbqabaGaeiiFaWNaeiiFaWhabaGaeq4Wdm3aa0baaeaacqWGPbqAaeaacqaIYaGmaaaaaiabc6caUaaa@43F7@

The low dimensional images y_i of the points are used to define a probabilistic density in the mapping space, as:

q i j = exp ⁡ ( − | | y i − y j | | 2 ) ∑ k ≠ i exp ⁡ ( − | | y i − y k | | 2 ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyCae3aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGH9aqpjuaGdaWcaaqaaiGbcwgaLjabcIha4jabcchaWjabcIcaOiabgkHiTiabcYha8jabcYha8Hqabiab=Lha5naaBaaabaGaemyAaKgabeaacqGHsislcqWF5bqEdaWgaaqaaiabdQgaQbqabaGaeiiFaWNaeiiFaW3aaWbaaeqabaGaeGOmaidaaiabcMcaPaqaamaaqababaGagiyzauMaeiiEaGNaeiiCaaNaeiikaGIaeyOeI0IaeiiFaWNaeiiFaWNae8xEaK3aaSbaaeaacqWGPbqAaeqaaiabgkHiTiab=Lha5naaBaaabaGaem4AaSgabeaacqGG8baFcqGG8baFdaahaaqabeaacqaIYaGmaaGaeiykaKcabaGaem4AaSMaeyiyIKRaemyAaKgabeGaeyyeIuoaaaGccqGGUaGlaaa@62DE@

The aim of this SNE method is to match the above two distributions as close as possible. The Kullback-Leibler divergence, which is a measure of dissimilarity between two probabilities is used here as a cost function. This can be achieved by manipulating the coordinates y_i to minimise the cost:

C = ∑ i N ∑ j N p i j log ⁡ p i j q i j = ∑ i N K L ( P i | | Q i ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaem4qamKaeyypa0ZaaabCaeaadaaeWbqaaiabdchaWnaaBaaaleaacqWGPbqAcqWGQbGAaeqaaOGagiiBaWMaei4Ba8Maei4zaCwcfa4aaSaaaeaacqWGWbaCdaWgaaqaaiabdMgaPjabdQgaQbqabaaabaGaemyCae3aaSbaaeaacqWGPbqAcqWGQbGAaeqaaaaaaSqaaiabdQgaQbqaaiabd6eaobqdcqGHris5aaWcbaGaemyAaKgabaGaemOta4eaniabggHiLdGccqGH9aqpdaaeWbqaaiabdUealjabdYeamjabcIcaOiabdcfaqnaaBaaaleaacqWGPbqAaeqaaOGaeiiFaWNaeiiFaWNaemyuae1aaSbaaSqaaiabdMgaPbqabaGccqGGPaqkcqGGUaGlaSqaaiabdMgaPbqaaiabd6eaobqdcqGHris5aaaa@5C75@

The SNE model can be extended for multiple projections of a single object by using a mixture of densities, which produces a probabilistic density in the mapping space:

q i j = ∑ b π i b ∑ c π j c exp ⁡ ( − | | y i b − y j c | | 2 ) ∑ k ∑ d π k d exp ⁡ ( − | | y i b − y k d | | 2 ) . MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xI8qiVKYPFjYdHaVhbbf9v8qqaqFr0xc9vqFj0dXdbba91qpepeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyCae3aaSbaaSqaaiabdMgaPjabdQgaQbqabaGccqGH9aqpdaaeqbqaaiabec8aWnaaBaaaleaacqWGPbqAdaWgaaadbaGaemOyaigabeaaaSqabaaabaGaemOyaigabeqdcqGHris5aOWaaabuaKqbagaadaWcaaqaaiabec8aWnaaBaaabaGaemOAaO2aaSbaaeaacqWGJbWyaeqaaaqabaGagiyzauMaeiiEaGNaeiiCaaNaeiikaGIaeyOeI0IaeiiFaWNaeiiFaWNaemyEaK3aaSbaaeaacqWGPbqAdaWgaaqaaiabdkgaIbqabaaabeaacqGHsislcqWG5bqEdaWgaaqaaiabdQgaQnaaBaaabaGaem4yamgabeaaaeqaaiabcYha8jabcYha8naaCaaabeqaaiabikdaYaaacqGGPaqkaeaadaaeqaqaamaaqababaGaeqiWda3aaSbaaeaacqWGRbWAdaWgaaqaaiabdsgaKbqabaaabeaaaeaacqWGKbazaeqacqGHris5aiGbcwgaLjabcIha4jabcchaWjabcIcaOiabgkHiTiabcYha8jabcYha8jabdMha5naaBaaabaGaemyAaK2aaSbaaeaacqWGIbGyaeqaaaqabaGaeyOeI0IaemyEaK3aaSbaaeaacqWGRbWAdaWgaaqaaiabdsgaKbqabaaabeaacqGG8baFcqGG8baFdaahaaqabeaacqaIYaGmaaaabaGaem4AaSgabeGaeyyeIuoacqGGPaqkaaaaleaacqWGJbWyaeqaniabggHiLdGccqGGUaGlaaa@7DA0@

The number of clusters in the mixture also needs to be determined empirically. Each data point x_i can be projected to many locations of the mixtures yib MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyEaK3aaSbaaSqaaiabdMgaPnaaBaaameaacqWGIbGyaeqaaaWcbeaaaaa@305C@ or yjc MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemyEaK3aaSbaaSqaaiabdQgaQnaaBaaameaacqWGJbWyaeqaaaWcbeaaaaa@3060@. However, for the experiments in this paper, only one projection per data will be used. The main advantage of SNE is its probabilistic approach but the results of the SNE are strongly dependent on the chosen σ. If the chosen σ is too large, the projecting data is likely to collapse to a single point. The suggested σ is σ = log(K), where K is the number of neighbours used to define a local cluster. To be consistent with the LLE method, which used K = 5, 20, we therefore choose σ = log(5), log(20) in the experiments in this paper as representative examples.

Purely for comparison to previous works in this area, we additionally superimpose the results from a discriminative classifier on the projective maps. Classifiers are created to determine the performance of the discrimination patients into good or poor prognosis from each topographic projection images by trying to compare the performance using different techniques and gene lists. The classifiers are built on the two dimensional input of the visualisation space using a separate RBF nonlinear classifier. The output, instead of crisply divided the results into good or poor prognosis patients produces an analogue value indicating the likelihood of good prognosis typically varying from 0 to 1. The classifier uses 2 coordinate input values and produces 2 output values indicating good and poor prognosis likelihood respectively and is trained using the original 78 patients only as a training set. Specifically, the desired target value is T = {T₁, T₂} where T ∈ {[1, 0], [0, 1]} represents good and poor prognosis patients respectively. The two dimensional output of the RBF network is: y_i = W Φ (x_i) where the basis functions constituting Φ are selected using cross validation.

The outputs of the RBF network are then transformed using the softmax function, giving a vector prognosis indicator for each patient P=exp⁡(y)∑jexp⁡(yj) MathType@MTEF@5@5@+=feaafiart1ev1aaatCvAUfKttLearuWrP9MDH5MBPbIqV92AaeXatLxBI9gBaebbnrfifHhDYfgasaacPC6xNi=xH8viVGI8Gi=hEeeu0xXdbba9frFj0xb9qqpG0dXdb9aspeI8k8fiI+fsY=rqGqVepae9pg0db9vqaiVgFr0xfr=xfr=xc9adbaqaaeGaciGaaiaabeqaaeqabiWaaaGcbaGaemiuaaLaeyypa0tcfa4aaSaaaeaacyGGLbqzcqGG4baEcqGGWbaCcqGGOaakieqacqWF5bqEcqGGPaqkaeaadaaeqaqaaiGbcwgaLjabcIha4jabcchaWjabcIcaOiabdMha5naaBaaabaGaemOAaOgabeaacqGGPaqkaeaacqWGQbGAaeqacqGHris5aaaaaaa@4210@. One of the two scales outputs which represents the good prognosis class is used as an indicator and contours of the indicator values are superimposed on the projection map space to show the likelihood of the good prognosis indicators.

Patients with predicted prognosis values in the range 0.3 → 0.7 are considered as ambiguously classified. Removing these 'low-confidence' patients from measures of classification performance figures are likely to improve classification rates since the low confidence patients fall in the overlap between the good and poor prognosis groups as will be seen in the latter results.

Figure 1 is the result from a 2-dimensional NeuroScale projection using List A and Figure 2 is the result using List B. The group of poor-prognosis and good-prognosis patients are labelled differently, with black diamonds and grey circles respectively. The results similarly show some separation between the two groups of patients with a few patients wrongly mapped into the opposite class. This projection is using each patient data point as a separate centre in the RBF model internal to NeuroScale. However, to emphasize, no class information is used to construct the projection map. The different symbols are simply to allow easier identification of the two patient groups. This projection appears to support the previous result of van't Veer et. al. that List A appears to have some discriminatory capability, although it is evident from these figures that any discrimination is on a graded and overlapping scale rather than providing separable distributions.

Figures 1 and 2, also show the classification model contour lines superimposed on the projection map of NeuroScale. The prognosis indicators vary on the level of overlap between two prognosis groups. The areas where there is large overlap between the two patient groups reflects ambiguity of any likely class membership. Therefore, we regard these patients as low confidence samples as far as determining class information and we regard them as 'unclassifiable'.

Table 1 shows the classification result of the NeuroScale projection using List A with 0.5 prognosis indicator as a threshold boundary between good and poor prognosis signatures of all patients. The overall classification rate is 83.33%, while if only high confidence patients are used for consideration the classification result increases to 100% accuracy, the result shown in Table 2. However, there are only 30 patients that fall in the high confidence regions, less than one half of all patients. Similarly the results using List B, shown in Tables 3 and 4 gave the same classification rate as using the list A but with 4 fewer high confidence patients. The results of the projection maps and the classifications support the previous propositions in the literature regarding the lack of uniqueness of a single PGL.

The Locally Linear Embedding results using two different gene sets are projected down and shown in Figures 3 and 4 using K = 5 and Figures 5 and 6 using K = 20 together with the classification contour lines of the good prognosis indicator. K is the number of neighbours used to construct the mapping which has to be chosen empirically. With K = 5, within a good prognosis cluster of both gene lists, there are four obvious poor prognosis patients. Three of them are common across both List projections. These four patients remain in the wrong place even after the number of neighbours increases. Between 13 and 16 poor prognosis patients are likely to be misclassified as good prognosis patients in the 'boundary layer'.

The best representation seems to be K = 20 with List A giving slightly better separation with fewer patients misclassified, with 7 good prognosis and 4 poor prognosis patients likely to be misclassified, from inspection of the figures without the classification results. However, some regions can be classified better when the classifier are trained on the particular data set. For example, few good prognosis patients in Figure 5 are on the right of the projection while most of the good prognosis patients are supposed to be on the left side. Those few patients create the region where patients are likely to be good prognosis even though this could be the result of these few outliers.

Both List projections have a separability of the modes of the two groups even though some patients appear in the wrong relative positions for their prognosis groups. Nevertheless, the difficulty for LLE is choosing the appropriate value for K. The result shows better separation of the training data with K = 20. For Figure 5, poor prognosis patients P45, P55, P54 are isolated from the other patients. However, having these three patients correctly classified could result in poor generalisation across new data. Other than this, the LLE projections reflect some similarities to the NeuroScale projections.

Similar to the NeuroScale classification results, the classification results of LLE are provided in misclassification matrices showing results for all the patients and also using only high confidence patients, with different choices for the number of neighbours, K. Tables 5 and 6 show the classification results of LLE with K = 5 using List A with classification results on all patients and high confidence patients respectively. Similarly for list B, the classification results for K = 5 are shown in Tables 7 and 8. Visually, List B gives a more distinct projection than List A with more clusters of good prognosis patients separated without overlap of many poor prognosis patients. The classification results confirm this. When only high confidence patients are retained, no patients are misclassified using either gene list although the number of high confidence patients using List B is more than using List A by 8 patients.

For K = 20, the classification results are shown in Tables 9 and 10 for list A and Tables 11 and 12 for list B. Contrary to the K = 5 case, the results using List A with K = 20 gives better classification performance. The classification rate is quite high with 93.58% accuracy with larger numbers of high confidence patients compared to the other methods, but this could result from the overfitting of this particular model. As can be seen in Figure 5, the gap of the contours between 0.4 to 0.7 is quite narrow. Choosing the exact boundary that determines the prognosis signature of each patient is therefore critical. As a result, if patient values contain uncertain information or noisy data, the resulting classification outcome of such patients is likely to be effectively random. Therefore the data of such uncertain patients should not be taken into account in representing performance results. We investigate generalisation of these results later in the paper.

The projection maps of Stochastic Neighbor Embedding reflect different results of qualitative projections for σ = log(5) shown in Figures 7 and 8, and σ = log(20) shown in Figures 9 and 10. From these figures it can be seen that the relative distributions of the patient projections are quite different for differing choices of the value of σ and this value is quite hard to determine. With σ = log(20), patients from both gene groups are mostly overlapped. The separation is not as good as in the previous two models.

Figures 7 and 8, show the classification contour lines superimposed on the SNE projection maps using the two different gene Lists with σ = log(5), and Figures 9 and 10 with σ = log(20).

Tables 13 and 14 show the classification results of the SNE with σ = log(5) using List A, with classification results shown on all patients and also those high confidence selected patients respectively. For List B, the classification results are shown in Tables 15 and 16. With σ = log(5), List B gives better overall performance but when only high confidence patients are being measured, no patients are misclassified with almost the same number of high confidence patients using both gene lists. Again, this supports the proposition that equivalent performance can be obtained on dissimilar gene lists.