I want to get the first principal component for an image using the built-in function pca. How can I do that?
I have tried the following code:
[COEFF, SCORE] = pca(image);
SCORE(1:size(SCORE,1),:)=0;
reconstructed_image = SCORE / COEFF + repmat(mean(image,1),size(image,1), 1);
I=reshape(reconstructed_image,[256,256]);
figure
imshow(I,[0 255])
I only get the fist row of the image. Any idea how can I do that correctly?
You can't "PCA one image". What this did is not give you the first row, it used all rows as observations and your columns as parameters, like you'd usually set up your measurements. So it calculated the variance through all parameters, giving you a vector with the length being equal to your number of columns. You'd probably want more images to do this instead.
Please read the following answer of mine though before continuing, since I explain the main pitfalls of PCA in MATLAB there.
PCA in matlab selecting top n components
Related
I have a training set with the size of (size(X_Training)=122 x 125937).
122 is the number of features
and 125937 is the sample size.
From my little understanding, PCA is useful when you want to reduce the dimension of the features. Meaning, I should reduce 122 to a smaller number.
But when I use in matlab:
X_new = pca(X_Training)
I get a matrix of size 125973x121, I am really confused, because this not only changes the features but also the sample size? This is a big problem for me, because I still have the target vector Y_Training that I want to use for my neural network.
Any help? Did I badly interpret the results? I only want to reduce the number of features.
Firstly, the documentation of the PCA function is useful: https://www.mathworks.com/help/stats/pca.html. It mentions that the rows are the samples while the columns are the features. This means you need to transpose your matrix first.
Secondly, you need to specify the number of dimensions to reduce to a priori. The PCA function does not do that for you automatically. Therefore, in addition to extracting the principal coefficients for each component, you also need to extract the scores as well. Once you have this, you simply subset into the scores and perform the reprojection into the reduced space.
In other words:
n_components = 10; % Change to however you see fit.
[coeff, score] = pca(X_training.');
X_reduce = score(:, 1:n_components);
X_reduce will be the dimensionality reduced feature set with the total number of columns being the total number of reduced features. Also notice that the number of training examples does not change as we expect. If you want to make sure that the number of features are along the rows instead of the columns after we reduce the number of features, transpose this output matrix as well before you proceed.
Finally, if you want to automatically determine the number of features to reduce to, one method to do so is to calculate the variance explained of each feature, then accumulate the values from the first feature up to the point where we exceed some threshold. Usually 95% is used.
Therefore, you need to provide additional output variables to capture these:
[coeff, score, latent, tsquared, explained, mu] = pca(X_training.');
I'll let you go through the documentation to understand the other variables, but the one you're looking at is the explained variable. What you should do is find the point where the total variance explained exceeds 95%:
[~,n_components] = max(cumsum(explained) >= 95);
Finally, if you want to perform a reconstruction and see how well the reconstruction into the original feature space performs from the reduced feature, you need to perform a reprojection into the original space:
X_reconstruct = bsxfun(#plus, score(:, 1:n_components) * coeff(:, 1:n_components).', mu);
mu are the means of each feature as a row vector. Therefore you need add this vector across all examples, so broadcasting is required and that's why bsxfun is used. If you're using MATLAB R2018b, this is now implicitly done when you use the addition operation.
X_reconstruct = score(:, 1:n_components) * coeff(:, 1:n_components).' + mu;
According to libsvm faqs, the following one-line code scale each feature to the range of [0,1] in Matlab
(data - repmat(min(data,[],1),size(data,1),1))*spdiags(1./(max(data,[],1)-min(data,[],1))',0,size(data,2),size(data,2))
so I'm using this code:
v_feature_trainN=(v_feature_train - repmat(mini,size(v_feature_train,1),1))*spdiags(1./(maxi-mini)',0,size(v_feature_train,2),size(v_feature_train,2));
v_feature_testN=(v_feature_test - repmat(mini,size(v_feature_test,1),1))*spdiags(1./(maxi-mini)',0,size(v_feature_test,2),size(v_feature_test,2));
where I use the first one to train the classifier and the second one to classify...
In my humble opinion scaling should be performed by:
i.e.:
v_feature_trainN2=(v_feature_train -min(v_feature_train(:)))./(max(v_feature_train(:))-min((v_feature_train(:))));
v_feature_test_N2=(v_feature_test -min(v_feature_train(:)))./(max(v_feature_train(:))-min((v_feature_train(:))));
Now I compared the classification results using these two scaling methods and the first one outperforms the second one.
The question are:
1) What exactly does the first method? I didn't understand it.
2) Why the code suggested by libsvm outperforms the second one (e.g. 80% vs 60%)?
Thank you so much in advance
First of all:
The code described in the libsvm does something different than your code:
It maps every column independently onto the interval [0,1].
Your code however uses the global min and max to map all the columns using the same affine transformation instead of a separate transformation for each column.
The first code works in the following way:
(data - repmat(min(data,[],1),size(data,1),1))
This subtracts each column's minimum from the entire column. It does this by computing the row vector of minima min(data,[],1) which is then replicated to build a matrix the same size as data. Then it is subtracted from data.
spdiags(1./(max(data,[],1)-min(data,[],1))',0,size(data,2),size(data,2))
This generates a diagonal matrix. The entry (i,i) of this matrix is 1 divided by the difference of the maximum and the minimum of the ith column: max(data(:,i))-min(data(:,i)).
The right multiplication of this diagonal matrix means: Multiply each column of the left matrix with the corresponding diagonal entry. This effectively divides column i by max(data(:,i))-min(data(:,i)).
Instead of using a sparse diagonal matrix, you could do this even more efficiently with bsxfun:
bsxfun(#rdivide, ...
bsxfun(#minus, ...
data, min(data,[],1)), ...
max(data,[],1)-min(data,[],1))
Which is the matlab way of writing:
Divide:
The difference of:
each column and its respective minimum
by the difference of each column's max and min.
I know this has already been answered correctly, but I would like to present another solution that I think is also correct and I found more intuitive/shorther then the one presented by knedlsepp. I am new to matlab and as I was studying knedlsepp solution, I found it more intuitive to solve this problem with the following formula:
function [ output ] = feature_scaling( y)
output = (y - repmat(min(y),size(y,1),1)) * diag(1./(max(y) - min(y)));
end
I find it a bit easier to use diag this way instead of spdiags, but I believe it produces the same result for the purpose of this excercise.
Multiplying the first term by the second, effectively divides each member of the matrix (Y-min(Y)) by the scalar value 1/(max(y)-min(y)), achieving the desired result.
In case someone prefers a shorter version, maybe this can be of help.
I have successfully written the MATLAB code for finding the n-order hadamard matrix.
Then I found the transpose of that matrix.
Then I found the basis images,
e.g., A(1,3)th basis image = hadamard_matrix(:,1)*hadamard_matrix(:,2)'
But whenever I try to print it using imshow() function in matlab, it shows just a completely dark image for all the basis images.
So what is the correct approach to show such basis images in matlab ?
Thanks in advance!
The only reasonable explanation I can think of right now, is that your resulting matrix contains only of values smaller than, say, 0.05.
Instead of the default bounds 0 and 1, try other high/low values. For instance:
imshow(A,[min(min(A)) max(max(A))]);
I have an array named Area, which contains a set of values.
The histogram of the array looks like this
The bin width is 60 in this case. I'd like to fit two gaussians to the two peaks here (even if it won't be a great fit).
So I used:
options = statset('Display','final');
obj = gmdistribution.fit(area,2,'Options',options);
gausspdf = pdf(obj, xaxis);
A = sum(gausspdf);
gausspdf = gausspdf/A;
But when I try to plot the two fitted Gaussians, the resulting curve looks like this:
I'm quite confused, as there should be two peaks appearing in the plot?
The gmdistribution.fit method fits data according to maximum-likelihood criterion; that is, it tries to find parameters which maximize the likelihood given the data. It will not necessarily fit what you see or expect visually. Still, there is the possibility that the algorithm converged to a "bad" local minimum. You can try and set the initial conditions according to what you want to get, practically 'helping' the algorithm to converge to the desired result. You do this using the Start option to the fit method, which enables you to give it either an initial guess, in which case you should try and estimate the parameters from the histogram, or an initial component index for each data sample. See the documentation for more details.
I think that your peaks are too close and the function can't distinguish them. so maybe you should change the options for gmdistribution or apply a non-linear function to your data first to get more separate peaks in histogram.
I want to compute a general PCA matrix for a dataset, and I will use it to reduce dimensions of sift descriptors. I have already found some algorithms to compute it, but I couldn't find a way to compute it by using MATLAB.
Can someone help me?
[coeff, score] = princomp(X)
is the right thing to do, but knowing how to use it is a little tricky.
My understanding is that you did something like:
sift_image = sift_fun(img)
which gives you a binary image: sift_feature?
(Even if not binary, this still works.)
Inputs, formulating X:
To use princomp/pca formulate X so that each column is a numel(sift_image) x 1 vector (i.e. sift_image(:))
Do this for all your images and line them up as columns in X. So X will be numel(sift_image) x num_images.
If your images aren't the same size (e.g. pixel dimensions different, more or less of a scene in the images), then you'll need to bring them into some common space, which is a whole different problem.
Unless your stuff is binary, you'll probably want to de-mean/normalize X, both in the column direction (i.e. normalizing each individual image) and row direction (de-meaning the whole dataset).
Outputs
score is the set of eigen vectors: it will be num_pixels * num_images.
To get, say the first eigen vector back into an image shape, do:
first_component = reshape(score(:,1),size(im));
And so on for the rest of the components. There are as many components as input images.
Each row of coeff is the num_images (equal to num_components) set of weights that can be applied to generate each input image. i.e.
input_image_1 = reshape(score * coeff(:,1) , size(original_im));
where input_image_1 is the correct, original shape
coeff(1,:) is a vector (num_images x 1)
score is pixels x num_images
(Disclaimer: I may have the columns/rows mixed up, but the descriptions are correct.)
Does that help?
If you have access to Statistics Toolbox, you can use the command princomp, or in recent versions the command pca.