I'm using HOG in order to extract a set of features trough an Image A.
the HOG returns a features' vector of 1xN elements.
However the linear SVM accept only 2 features for each sample i.e the training data matrix's size is Mx2. so how i can adapt the HOG vector to be trained on linear SVM.
Please help me.
Thanks
What do you mean by "the linear SVM accept only 2 features for each sample"? You may be confused on how the SVM function accepts its training data. Here's a quick example of how I use it:
First, lets train an SVM model using fitcsvm using 500 samples of random data (the rows in the training data matrix), each with 1000 elements (the columns in the training data matrix), where the first 250 samples are in class 1 (first 250 rows of training labels), and the last 250 samples are in class 0 (last 250 rows of training labels):
>> training_data = rand(500, 1000);
>> training_labels = [ones(250,1); zeros(250,1)];
>>
>> svm_model = fitcsvm(training_data, testing_data)
svm_model =
ClassificationSVM
PredictorNames: {1x1000 cell}
ResponseName: 'Y'
ClassNames: [0 1]
ScoreTransform: 'none'
NumObservations: 500
Alpha: [418x1 double]
Bias: 2.3217
KernelParameters: [1x1 struct]
BoxConstraints: [500x1 double]
ConvergenceInfo: [1x1 struct]
IsSupportVector: [500x1 logical]
Solver: 'SMO'
Properties, Methods
We can generate some random test data for 10 test samples, each with 1000 elements, and create some predictions from it:
>> test_data = rand(10, 1000);
>> predicted_classes = predict(svm_model, test_data)
predicted_classes =
1
1
1
1
1
0
0
0
1
0
Does that answer your question?
Related
I have a little problem with SVM classifier in Matlab. I have a 61200x59 matrix of features in which each row represent a set of features extracted from an image (all double values). All those features are associated with a 61200x1 matrix containing 2 labels: 0 and 1 (as double variables). Now I want to train a linear classifier and I've used the following function:
SVM_Model = fitcsvm(train_features, train_labels, 'KernelFunction', 'linear')
If I take a look to the details of this line I obtain this result:
SVM_Model =
ClassificationSVM
ResponseName: 'Y'
CategoricalPredictors: []
ClassNames: [0 1]
ScoreTransform: 'none'
NumObservations: 61200
Alpha: [40956×1 double]
Bias: 0.9998
KernelParameters: [1×1 struct]
BoxConstraints: [61200×1 double]
ConvergenceInfo: [1×1 struct]
IsSupportVector: [61200×1 logical]
Solver: 'SMO'
But when I call the predict function on test set ([label, score] = predict(SVM_Model, test_features(i, :));) it predicts always the label 1 (on over than 15000 test, so it seems a little bit suspicious) and on all the class 0 object I have a classification error. Can anyone tell me what can be the problem? Is necessary to rescale the features because SVM cannot handle high-dimensional points? Or another problem is present (like misconfiguration is SVM learning)?
With HIK kernel everything works fine. Also using RBF kernel gives good result on this problem.
I am using the matlab's svm classify function. My train and test data have the following dimensions:
>> size(TrainV)
ans =
99192 705
>> size(TestV)
ans =
246 705
I have a function that trains a one-versus-one classify with 10 classes (45 binary classifiers). The model can be trained by calling the function below:
Models = SVM_multitrain (TrainV(:, 2:end), TrainV(:, 1), 10);
I am sending the feature vectors (TrainV(:, 2:end)) and the labels (TrainV(:, 1)) and I am asking the Models to train the combination of couples for 45 classifiers (10). The function runs ok and I can have the following information after the training. For example, I will show the models for the 3rd and 45th binary classifiers.
> Models(3)
ans =
SupportVectors: [9x704 double]
Alpha: [9x1 double]
Bias: -2.3927 - 0.0001i
KernelFunction: #linear_kernel
KernelFunctionArgs: {}
GroupNames: [20117x1 double]
SupportVectorIndices: [9x1 double]
ScaleData: [1x1 struct]
FigureHandles: []
>> Models(45)
ans =
SupportVectors: [10x704 double]
Alpha: [10x1 double]
Bias: -2.7245 + 0.0000i
KernelFunction: #linear_kernel
KernelFunctionArgs: {}
GroupNames: [22087x1 double]
SupportVectorIndices: [10x1 double]
ScaleData: [1x1 struct]
FigureHandles: []
The problem is when I call the function to classify a feature vector, for example, for the first binary classifier.
>> TestAttribBin = svmclassify(Models(1), TestV(:,2:end))
Subscript indices must either be real positive integers or logicals.
Error in svmclassify (line 140)
outclass = glevels(outclass(~unClassified),:);
What could be the problem? when I apply the same classification procedure to feature vectors extracted in another way this problem does not happen.
The likely cause of this error is passing complex data to svmclassify. svmclassify only accepts real feature vectors. Indeed, passing a complex data to svmclassify results in outclass being complex and complex values cannot be used for indexing as stated by the error message.
One option may be to encode the imaginary part of your vector into the feature, for example by doubling the length of your feature vectors.
In fact the vast majority of machine learning models are based on the assumption of feature vectors being real eg. artificial neural net, regression trees, svm, etc. Although there might be some extensions in some case.
I am training a one vs all svm classifier. I used a 200 by 459 matrix to train the classifier using VLFeat svm classifier. (http://www.vlfeat.org/matlab/vl_svmtrain.html)
[W B] = vl_svmtrain(train_image_feats', tmp', .00001);
where train_image_feats' is a 200 by 459 matrix, and tmp' is the label matrix which is 1 by 459 vector.
The above command trains the svm with no problem, but then to classify the scores obtained on the test matrix I get an error. The test matrix is obviously not of the same size as that of the training matrix.
scores(i, :) = W'*test_image_feats' + B;
Where test_image_feats' is a 200 by 90 matrix. scores is a 9 by 459 matrix. 9 Because there are 9 categories(labels) to classify and 459 are the number of training images.
The above command gives the error:
Subscripted assignment dimension mismatch.
Error in svm_classify (line 56) scores(i, :) = W'*test_image_feats'
+ B;
Edit: Full code added..
categories = unique(train_labels);
num_categories = length(categories);
scores = zeros([num_categories size(train_labels, 1)]); %train_labels is 459 by 1 size
for i=1:num_categories %there are 9 categories
tmp = strcmp(train_labels, categories{i});
tmp = tmp - (1-tmp);
[W B] = vl_svmtrain(train_image_feats', tmp', .00001);
scores(i, :) = W'*test_image_feats' + B;
end
predicted_categories = cell(size(train_labels));
parfor i=1:size(test_image_feats,1)
image_scores = scores(:, i);
label_index = find(image_scores==max(image_scores));
predicted_categories{i}=categories(label_index);
end
Conceptually you are training a model with 459 training samples to predict the scores of 90 test samples.
scores = zeros([num_categories size(train_labels, 1)]);
isn't right as it will be the size of the training set. In fact you don't have to care at all about the size of the training set, you could train the model with 20 or 20000 images the prediction step shouldn't be any different.
scores have to be defined with the test case in mind
scores = zeros([num_categories size(test_labels, 1)]);
When you used 459 for both it only worked because size(test_labels, 1) was equal to size(train_labels, 1)
The problem is not with your right hand side of the assignment, but with score(i,:): you are trying to assign a 9-by-90 size matrix into a single row of score - this simply won't fit.
I would like to perform simple LDA on my small data set (65x8). I have 65 instances (samples) , 8 features (attributes) and 4 classes. Any matlab code for LDA , as I know Matlab Toolbox does not have LDA function So I need to write own code. Any help?
I find on web this code
load /Data;
All_data= Data(:,1:8);
All_data_label= Data(:,9);
testing_ind = [];
for i = 1:length(Data)
if rand>0.8
testing_ind = [testing_ind, i];
end
end
training_ind = setxor(1:length(Data), testing_ind);
[ldaClass,err,P,logp,coeff] = classify(Data(testing_ind,:),...
Data((training_ind),:),Data_label(training_ind,:),'linear');
[ldaResubCM,grpOrder] = confusionmat(All_data_label(testing_ind,:),ldaClass)
Then I got this results
ldaClass =
3
2
3
2
1
4
3
3
1
2
1
1
2
err =
0.2963
P =
0.0001 0.0469 0.7302 0.2229
0.1178 0.5224 0.3178 0.0419
0.0004 0.2856 0.4916 0.2224
0.0591 0.6887 0.1524 0.0998
0.8327 0.1637 0.0030 0.0007
0.0002 0.1173 0.3897 0.4928
0.0000 0.0061 0.7683 0.2255
0.0000 0.0241 0.5783 0.3976
0.9571 0.0426 0.0003 0.0000
0.2719 0.5569 0.1630 0.0082
0.9999 0.0001 0.0000 0.0000
0.9736 0.0261 0.0003 0.0000
0.0842 0.6404 0.2634 0.0120
coeff =
4x4 struct array with fields:
type
name1
name2
const
linear
ldaResubCM =
4 0 0 0
0 3 1 0
0 1 1 0
0 0 2 1
grpOrder =
1
2
3
4
So I have 65 Instances, 8 Attributes and 4 classes (1,2,3,4). So dont know how to interpret these results. Any help?
The interpretation of the results derives directly from the documentation of classify.
classify trains a classifier based on the training data and labels (second and third argument), and applies the classifier to the test data (first argument).
ldaClass gives the classes chosen for the test data points, based on the classifier that has been trained using the training data points and labels.
err is the training error rate, the fraction of training data points that are incorrectly classified using the classifier which was trained using that data. The training error rate underestimates the error to be expected on independent test data.
P gives the posterior probabilities. I.e. for each test data point (rows) it gives for each class (columns) the probability that the data point belongs to that class. Probabilities sum to 1 across classes (for each row). The definite classification in ldaClass derives from the posterior probabilities such that for each test data point the class with the highest probability is chosen: [~, ind] = max(P') results in ind = ldaClass'.
coeff contains details about the trained classifier. In order to use this, you have to study in detail how the classifier works.
confusionmat compares the classes assigned by the classifier to the test data with the known true classes, and makes a table of the results, a confusion matrix. Each row corresponds to the true class of a test data point, each column to the class assigned by the classifier. Numbers on the diagonal indicate correct classifications; in your result, you have a test error of 1 - sum(diag(confusionmat)) / sum(confusionmat(:)) of 0.308. In particular, the confusion matrix shows you that of the 4 test data points that belong to class two, three have been classified correctly and 1 incorrectly (as belonging to class three).
grpOrder just gives the explicit class labels for the four classes numbered 1 to 4; in your case, indices and labels are identical.
When doing:
load training.mat
training = G
load testing.mat
test = G
and then:
>> knnclassify(test.Inp, training.Inp, training.Ltr)
??? Error using ==> knnclassify at 91
The length of GROUP must equal the number of rows in TRAINING.
Since:
>> size(training.Inp)
ans =
40 40 2016
And:
>> length(training.Ltr)
ans =
2016
How can I give the second parameter of knnclassify (TRAINING) the training.inp 3-D matrix so that the number of rows will be 2016 (the third dimension)?
Assuming that your 3D data is interpreted as 40-by-40 matrix of features for each of the 2016 instances (third dimension), we will have to re-arrange it as a matrix of size 2016-by-1600 (rows are samples, columns are dimensions):
%# random data instead of the `load data.mat`
testing = rand(40,40,200);
training = rand(40,40,2016);
labels = randi(3, [2016 1]); %# a class label for each training instance
%# (out of 3 possible classes)
%# arrange data as a matrix whose rows are the instances,
%# and columns are the features
training = reshape(training, [40*40 2016])';
testing = reshape(testing, [40*40 200])';
%# k-nearest neighbor classification
prediction = knnclassify(testing, training, labels);