dimensionality reduction for non square matrix? - matlab

Question

Im going to do dimensionality reduction by using PCA/SVD for my extracted features.
Suppose if I want to do classification using SIFT as the features and SVM as the classifier.
I have 3 images for training and I arrange them in a different row..
So 1st row for 1st images, 2nd row for second images and 3rd row for the 3rd image...
And the columns represents the features
A=[ 1 2 3 4 5
4 5 6 7 8
0 0 1 9 0]
To do dimensionality reduction (from my 3x5 matrix/non square matrix), we have to do A*EigenVector
Now I have to extract the eigen vector from my A matrix And from what I understand, that SVD is for non square matrix, so to perform PCA (eigs function), I need to make it square by multiplying it with it's transpose)
Here is if I do SVD directly
[u1,s1,v1] = svd(A);
I got
u1 =
-0.4369 0.1426 0.8882
-0.8159 0.3530 -0.4580
-0.3788 -0.9247 -0.0379
v1 =
-0.2229 0.2206 -0.7088 -0.6070 -0.1754
-0.2984 0.2910 -0.3857 0.4705 0.6754
-0.3966 0.2301 -0.0910 0.5382 -0.7012
-0.6547 -0.7495 0.0045 -0.0598 0.0779
-0.5248 0.5020 0.5836 -0.3419 0.1233
and when I use PCA (eigs function) {as I arrange the feat of images in different row, so I need to do A*A'}, I got
c=A*A'
[e1 e2]=eigs(c);
e1 =
0.4369 0.1426 0.8882
0.8159 0.3530 -0.4580
0.3788 -0.9247 -0.0379
My question is:
is that right that I used it in SVD or in the PCA (by converting t into A*A' matrix) will give me he same eig vectoe (e1 and u1)??
As I arrange my images in different rows and the features for each images in different column. and PCA/SVD is suing to extract the eig vector which represent the relation between the variable.. So in this case the variable would be the row (images) or the columns (features)??
Do I have to convert my matrix into covariance matrix by using (Cov function) if I want to use eigs function?? Or it will be done by eigs function manually??
I do really appriciate any answer

Answer 1

Suppose you have n-dimension samples and you want to reduce it to d-dimension data by PCA.
Suppose your data are stored in matrix AnxN (N is the number of samples(here images)).
here n=3 and N=5.
We define a correlation matrix R = A*A' (nxn). You can use the covariance matrix instead.
Calculate the eignen vectors of R and corresponding eigen values:
R = A*A';
[eigVec, eigVal] = eig(R)
eigVec =
0.8882 0.1426 0.4369
-0.4580 0.3530 0.8159
-0.0379 -0.9247 0.3788
eigVal =
1.7728 0 0
0 49.6457 0
0 0 275.5815
Note that columns of eigVec are the eigen vectors of R.
Some of the eigen values will be zero or if not, you can take a threshold. So you can eliminate the corresponding eigen vectors:
T = eigVec(:, 2:3)
T =
0.1426 0.4369
0.3530 0.8159
-0.9247 0.3788
now T is a nxd matrix.
For any row vector X1xn the X1xn*Tnxd will result a Y1xd output.
The final answer;
B = A'*T;