I have a little problem with SVM classifier in Matlab. I have a 61200x59 matrix of features in which each row represent a set of features extracted from an image (all double values). All those features are associated with a 61200x1 matrix containing 2 labels: 0 and 1 (as double variables). Now I want to train a linear classifier and I've used the following function:
SVM_Model = fitcsvm(train_features, train_labels, 'KernelFunction', 'linear')
If I take a look to the details of this line I obtain this result:
SVM_Model =
ClassificationSVM
ResponseName: 'Y'
CategoricalPredictors: []
ClassNames: [0 1]
ScoreTransform: 'none'
NumObservations: 61200
Alpha: [40956×1 double]
Bias: 0.9998
KernelParameters: [1×1 struct]
BoxConstraints: [61200×1 double]
ConvergenceInfo: [1×1 struct]
IsSupportVector: [61200×1 logical]
Solver: 'SMO'
But when I call the predict function on test set ([label, score] = predict(SVM_Model, test_features(i, :));) it predicts always the label 1 (on over than 15000 test, so it seems a little bit suspicious) and on all the class 0 object I have a classification error. Can anyone tell me what can be the problem? Is necessary to rescale the features because SVM cannot handle high-dimensional points? Or another problem is present (like misconfiguration is SVM learning)?
With HIK kernel everything works fine. Also using RBF kernel gives good result on this problem.
Related
The mvnpdf function in Matlab takes the default mean and std as 0 and I (Identity matrix).
When I type this:
mvnpdf([0,0])
ans = 0.1592
mvnpdf([0,0], [0,0], [1,0;0,1])
ans = 0.1592
mvnpdf([1,1])
ans = 0.0585
mvnpdf([1,1], [0,0], [1,0;0,1])
ans = 0.0585
So the function works when we specify the Mean and Correlation Matrix outselves, however, when I do this, it no longer works
mvnpdf([Inf,Inf])
ans = 0
mvnpdf([Inf,Inf], [0,0], [1,0;0,1])
ans = NaN
Why is this happening? How can I fix this?
These bivariate normal pdfs are a small part of a larger expression which I am optimizing over "rho", so I need an expression like
mvnpdf([Gamma(i+1) Tau(j+1)],[0 0],[1,rho;rho 1])
to work for some unkown $\rho$ and in additional inputs $\Gamma$ and $\Tau$ which can possible be $\infty$ or $-\infty$. Ok LaTeX doesnt seem to work.
Anyway, I would like to be able to provide a correlation Matrix with an unknown parameter Rho and additional inputs Gamma's and Tau's which may run over a large range and could possibly be negative or positive infinity.
I'm using HOG in order to extract a set of features trough an Image A.
the HOG returns a features' vector of 1xN elements.
However the linear SVM accept only 2 features for each sample i.e the training data matrix's size is Mx2. so how i can adapt the HOG vector to be trained on linear SVM.
Please help me.
Thanks
What do you mean by "the linear SVM accept only 2 features for each sample"? You may be confused on how the SVM function accepts its training data. Here's a quick example of how I use it:
First, lets train an SVM model using fitcsvm using 500 samples of random data (the rows in the training data matrix), each with 1000 elements (the columns in the training data matrix), where the first 250 samples are in class 1 (first 250 rows of training labels), and the last 250 samples are in class 0 (last 250 rows of training labels):
>> training_data = rand(500, 1000);
>> training_labels = [ones(250,1); zeros(250,1)];
>>
>> svm_model = fitcsvm(training_data, testing_data)
svm_model =
ClassificationSVM
PredictorNames: {1x1000 cell}
ResponseName: 'Y'
ClassNames: [0 1]
ScoreTransform: 'none'
NumObservations: 500
Alpha: [418x1 double]
Bias: 2.3217
KernelParameters: [1x1 struct]
BoxConstraints: [500x1 double]
ConvergenceInfo: [1x1 struct]
IsSupportVector: [500x1 logical]
Solver: 'SMO'
Properties, Methods
We can generate some random test data for 10 test samples, each with 1000 elements, and create some predictions from it:
>> test_data = rand(10, 1000);
>> predicted_classes = predict(svm_model, test_data)
predicted_classes =
1
1
1
1
1
0
0
0
1
0
Does that answer your question?
I am using the matlab's svm classify function. My train and test data have the following dimensions:
>> size(TrainV)
ans =
99192 705
>> size(TestV)
ans =
246 705
I have a function that trains a one-versus-one classify with 10 classes (45 binary classifiers). The model can be trained by calling the function below:
Models = SVM_multitrain (TrainV(:, 2:end), TrainV(:, 1), 10);
I am sending the feature vectors (TrainV(:, 2:end)) and the labels (TrainV(:, 1)) and I am asking the Models to train the combination of couples for 45 classifiers (10). The function runs ok and I can have the following information after the training. For example, I will show the models for the 3rd and 45th binary classifiers.
> Models(3)
ans =
SupportVectors: [9x704 double]
Alpha: [9x1 double]
Bias: -2.3927 - 0.0001i
KernelFunction: #linear_kernel
KernelFunctionArgs: {}
GroupNames: [20117x1 double]
SupportVectorIndices: [9x1 double]
ScaleData: [1x1 struct]
FigureHandles: []
>> Models(45)
ans =
SupportVectors: [10x704 double]
Alpha: [10x1 double]
Bias: -2.7245 + 0.0000i
KernelFunction: #linear_kernel
KernelFunctionArgs: {}
GroupNames: [22087x1 double]
SupportVectorIndices: [10x1 double]
ScaleData: [1x1 struct]
FigureHandles: []
The problem is when I call the function to classify a feature vector, for example, for the first binary classifier.
>> TestAttribBin = svmclassify(Models(1), TestV(:,2:end))
Subscript indices must either be real positive integers or logicals.
Error in svmclassify (line 140)
outclass = glevels(outclass(~unClassified),:);
What could be the problem? when I apply the same classification procedure to feature vectors extracted in another way this problem does not happen.
The likely cause of this error is passing complex data to svmclassify. svmclassify only accepts real feature vectors. Indeed, passing a complex data to svmclassify results in outclass being complex and complex values cannot be used for indexing as stated by the error message.
One option may be to encode the imaginary part of your vector into the feature, for example by doubling the length of your feature vectors.
In fact the vast majority of machine learning models are based on the assumption of feature vectors being real eg. artificial neural net, regression trees, svm, etc. Although there might be some extensions in some case.
In Matlab I've got a big matrix (512x512x100). In order to analyze it I have used blockproc. Due to the fact that blockproc only accepts 2D matrices I have transformed my original matrix into a 2D matrix using mat2cell and some other consequent steps. The blockproc function search for the pixel with the highest value in each sub-matrix and returns its coordinates referenced to that very sub-matrix.
After a couple of steps I've got the coordinates of those maximum pixels referenced to the first mat2cell transformation. Now I would like to convert those coordinates to global coordinates, that is, absolute coordinates in the large original 3D matrix.
I've try some things with this example:
d=rand(4,4,4)
d(:,:,1) =
0.0451 0.8044 0.0784 0.7859
0.8911 0.3481 0.4636 0.9806
0.4887 0.5677 0.0999 0.9488
0.7822 0.0467 0.5569 0.2256
d(:,:,2) =
0.8131 0.8880 0.8066 0.8103
0.8240 0.3358 0.8422 0.2552
0.8364 0.2759 0.3753 0.7741
0.9853 0.8297 0.3745 0.5936
d(:,:,3) =
0.1200 0.1841 0.3897 0.0894
0.0747 0.7485 0.4866 0.4722
0.8387 0.9523 0.0166 0.5013
0.8210 0.3107 0.6935 0.1286
d(:,:,4) =
0.6424 0.7322 0.4631 0.1684
0.5523 0.0953 0.0168 0.6231
0.1073 0.5530 0.8504 0.9304
0.3482 0.7804 0.7657 0.2496
After doing the mat2cell I've got:
cell(:,:,1) =
[2x2x2 double] [2x2x2 double]
[2x2x2 double] [2x2x2 double]
cell(:,:,2) =
[2x2x2 double] [2x2x2 double]
[2x2x2 double] [2x2x2 double]
These are the resulting coordinates. Each row of the matrix from below represents the coordinates of the maximum pixel in each "sub-cell":
max_px =
2 1
2 3
2 2
1 2
2 2
1 2
2 4
1 4
by doing "cell{1}(2,1)" you get:
ans =
0.8911
wich is the value of the maximum pixel in the first 3D sub-batrix (first "sub-cell")
I guess the is a pretty straight forward way to convert those relative coordinates to global, but I don't know how. Thank you in advance.
You don't really have to use blockproc. I don't know why anyone ever does...
dcell=mat2cell(d,...);
mask=cell2mat( cellfun(#(c) eq(c,max(c(:))) , dcell, 'uni',0) ) ;
[imax,jmax,kmax]=ind2sub(size(d), find(mask(:)));
This assumes that there is a unique max in each sub-array, which you also appear to have assumed.
This uses the max_px that you already have. The 4th and 5th line just convert max_px to more natural 3D coordinates, so maybe those could be omitted, and the code shortened, if you can get it in that form to begin with.
dcell=mat2cell(d,...);
[outer_dims{1:3}]=size(dcell); outer_dims=[outer_dims{:}],
[inner_dims{1:3}]=size(dcell{1}); inner_dims=[inner_dims{:}],
idx=sub2ind(inner_dims,max_px(:,1),max_px(:,2));
[ii,jj,kk]= ind2sub(inner_dims,idx); %get 3-inner max coodinates
[X,Y,Z]=ndgrid(1:outer_dims(1),1:outer_dims(2), 1:outer_dims(3));
dims6=[inner_dims; outer_dims];
idx6=sub2ind(dims6(:).', ii,X(:),jj,Y(:),kk,Z(:));
[imax,jmax,kmax] = ind2sub(size(d),idx6);
I'm trying to code a program that solves systems of equations in MATLAB. I was wondering if there is a way to get MATLAB to group like terms and put their coefficients into a matrix? I realize that I can just enter the coefficients in by hand but I want to hopefully repurpose this small program to perform nodal analysis.
You could always use my sympoly tools to do much of the work for you. Since this set of tools will give you direct access to the parsed result, this will make your life easier, as well as do much symbolic manipulation of an expression. For example...
>>sympoly x y z
>> P = 3*x + 2*x*y - 2.75*z^2
P =
-2.75*z^2 + 3*x + 2*x*y
>> struct(P)
ans =
Var: {'x' 'y' 'z'}
Exponent: [3x3 double]
Coefficient: [3x1 double]
>> P.Exponent
ans =
0 0 2
1 0 0
1 1 0
>> P.Coefficient
ans =
-2.75
3
2
Find sympoly on the file exchange.
It would be easy enough to write a parser in order to do this functionality yourself. Parse out the number and then the variable with its power. Good luck.