[Python] Implement PCA using SVD

I am trying to figure out the differences between PCA using Singular Value Decomposition as oppossed to PCA using Eigenvector-Decomposition.

Picture the following matrix:

B = np.array([ [1, 2],
[3, 4],
[5, 6] ])


When computing the PCA of this matrix B using eigenvector-Decomposition, we follow these steps:

1. Center the data (entries of B) by substracting the column-mean from each column
2. Compute the covariance matrix C = Cov(B) = B^T * B / (m -1), where m = # rows of B
3. Find eigenvectors of C
4. PCs = X * eigen_vecs

When computing the PCA of matrix B using SVD, we follow these steps:

1. Compute SVD of B: B = U * Sigma * V.T
2. PCs = U * Sigma

I have done both for the given matrix.

With eigenvector-Decomposition I obtain this result:

[[-2.82842712 0. ]
[ 0. 0. ]
[ 2.82842712 0. ]]


With SVD I obtain this result:

[[-2.18941839 0.45436451]
[-4.99846626 0.12383458]
[-7.80751414 -0.20669536]]


The result obtained with eigenvector-Decomposition is the result given as solution. So, why is the result obtained with the SVD different?

I know that: C = Cov(B) = V * (Sigma^2)/(m-1)) * V.T and I have a feeling this might be related to why the two results are different. Still. Can anyone help me understand better?

Continue reading...