I am using Support Vector Regression(SVR) in libsvm package to predict outputs. Kernel : RBF
Train set size : 729x40
Test set size : 137x40
The output of train set seems fine when measured against ground truth. But the predictions on test set are all the same. It outputs same values.
After checking the related posts, I normalized the data and played with the values of gamma(10-100000) but still the problem persists.
trainGT=games(((games(:,46)>=2010) & (games(:,46)<2015) & (games(:,1)~=8)),43);
featuresTrain=lastGame(games,true,1);
testGT=games((games(:,46)>=2015 & (games(:,1)~=8)),43);
featureTest=lastGame(games,false,1);
eval(['model = svmtrain( trainGT, featuresTrain,''-s 4 -t 2 -c 10 -g 10 ' ''');']);
w = (model.sv_coef' * full(model.SVs));
b = -model.rho;
predictionsTrain = svmpredict(trainGT, featuresTrain,model);
predictionsTest = svmpredict(zeros(length(testGT),1), featureTest, model);
My output is as follows
optimization finished, #iter = 1777
epsilon = 0.630588
obj = -19555.036253, rho = -17.470386
nSV = 681, nBSV = 118
Mean squared error = 305.214 (regression)
Squared correlation coefficient = -1.#IND (regression)
All my predictionTest values are 17.4704 (which is same as the rho value in the output). Can someone please help me on this? Thanks.
Related
Training the 50000 training images with feature vectors of 32x32x3 = 3072 dimensionality is making my computer get stuck. Is there a work around I'm missing to use libSVM efficiently for multiclass SVM classification? A day passes and the SVM is still running for only one class in a one-vs-all training framework.
*I am aware that using pixel values is a terrible way of optimal classification, yet I still want to run this as a lower bound benchmark for a study.
Code:
clc;close all;clear all;
addpath(genpath('./libsvm-3.21'));
addpath(genpath('./liblinear-2.1'));
%load all images:
M1 = load('../Data/cifar-10-batches-mat/data_batch_1.mat');
M2 = load('../Data/cifar-10-batches-mat/data_batch_2.mat');
M3 = load('../Data/cifar-10-batches-mat/data_batch_3.mat');
M4 = load('../Data/cifar-10-batches-mat/data_batch_4.mat');
M5 = load('../Data/cifar-10-batches-mat/data_batch_5.mat');
M = [M1.data; M2.data; M3.data; M4.data; M5.data];
M_labels = [M1.labels; M2.labels; M3.labels; M4.labels; M5.labels];
M_labels_double = double(M_labels);
M_double = double(M)/255.0;
%M_double is the dataset of [50000x3072]
%M_labels_double are the labels and has size of [50000x1]
model=cell(10,1);
for i=1:10
model{i} = svmtrain(double(M_labels_double==i),M_double,'-t 0 -c 1 -g 0.2 -b 1 -m 4000');
end
I'm using K-Fold Cross-validation to get the error rate of a SVM Classifier. This is the code with wich I'm getting the error rate for 8-Fold Cross-validation:
data = load('Entrenamiento.txt');
group = importdata('Grupos.txt');
CP = classperf(group);
N = length(group);
k = 8;
indices = crossvalind('KFold',N,k);
single_error = zeros(1,k);
for j = 1:k
test = (indices==j);
train = ~test;
SVMModel_1 = fitcsvm(data(train,:),group(train,:),'BoxConstraint',1,'KernelFunction','linear');
classification = predict(SVMModel_1,data(test,:));
classperf(CP,classification,test);
single_error(1,j) = CP.ErrorRate;
end
confusion_matrix = CP.CountingMatrix
VP = confusion_matrix(1,1);
FP = confusion_matrix(1,2);
FN = confusion_matrix(2,1);
VN = confusion_matrix(2,2);
mean_error = mean(single_error)
However, the mean_error changes each time I run the script. This is due to crossvalind, which generates random cross-validation indices, so each time I run the script, it generates different random indices.
What should I do to calculate the true error rate? Should I calculate the mean error rate of n code executions? Or what value should I use?
You can check wiki,
In k-fold cross-validation, the original sample is randomly partitioned into k equal size subsamples.
and
The k results from the folds can then be averaged (or otherwise combined) to produce a single estimation.
So no worries about different error rates of randomly selecting folds.
Of course the results will be different.
However if your error rate is in wide range then increasing k would help.
Also rng can be used to get fixed results.
I am doing a project on Image processing in Matlab and wish to implement LIBSVM for supervised learning.
I am encountering a problem in data preparation.
I have the data in CSV format and when i try to convert it into libsvm format by using the information provided in LIBSVM faq:-
matlab> SPECTF = csvread('SPECTF.train'); % read a csv file
matlab> labels = SPECTF(:, 1); % labels from the 1st column
matlab> features = SPECTF(:, 2:end);
matlab> features_sparse = sparse(features); % features must be in a sparse matrix
matlab> libsvmwrite('SPECTFlibsvm.train', labels, features_sparse);
I get the data in the following form:
3.0012 1:2.1122 2:0.9088 ......
[value 1] [index 1]:[value 2] [index 2]:[value 3]
That is the first value takes no index and the value following the index 1 is value 2.
From what i had read, the data should be in the following format:
[label] [index 1]:[value 1] [index 2]:[value 2]......
[label] [index 1]:[value 1] [index 2]:[value 2]......
I need help to make this right.
And also if anyone would give me a clue about how to give labels it will be really helpful.
Thanking you in advance,
Sidra
You don't have to write data to a file, you can instead use the Matlab interface to LIBSVM. This interface consists of two functions, svmtrain and svmpredict. Each function prints a help text if called without arguments:
Usage: model = svmtrain(training_label_vector, training_instance_matrix, 'libsvm_options');
libsvm_options:
-s svm_type : set type of SVM (default 0)
0 -- C-SVC
1 -- nu-SVC
2 -- one-class SVM
3 -- epsilon-SVR
4 -- nu-SVR
-t kernel_type : set type of kernel function (default 2)
0 -- linear: u'*v
1 -- polynomial: (gamma*u'*v + coef0)^degree
2 -- radial basis function: exp(-gamma*|u-v|^2)
3 -- sigmoid: tanh(gamma*u'*v + coef0)
4 -- precomputed kernel (kernel values in training_instance_matrix)
-d degree : set degree in kernel function (default 3)
-g gamma : set gamma in kernel function (default 1/num_features)
-r coef0 : set coef0 in kernel function (default 0)
-c cost : set the parameter C of C-SVC, epsilon-SVR, and nu-SVR (default 1)
-n nu : set the parameter nu of nu-SVC, one-class SVM, and nu-SVR (default 0.5)
-p epsilon : set the epsilon in loss function of epsilon-SVR (default 0.1)
-m cachesize : set cache memory size in MB (default 100)
-e epsilon : set tolerance of termination criterion (default 0.001)
-h shrinking : whether to use the shrinking heuristics, 0 or 1 (default 1)
-b probability_estimates : whether to train a SVC or SVR model for probability estimates, 0 or 1 (default 0)
-wi weight : set the parameter C of class i to weight*C, for C-SVC (default 1)
-v n : n-fold cross validation mode
-q : quiet mode (no outputs)
and
Usage: [predicted_label, accuracy, decision_values/prob_estimates] = svmpredict(testing_label_vector, testing_instance_matrix, model, 'libsvm_options')
Parameters:
model: SVM model structure from svmtrain.
libsvm_options:
-b probability_estimates: whether to predict probability estimates, 0 or 1 (default 0); one-class SVM not supported yet
Returns:
predicted_label: SVM prediction output vector.
accuracy: a vector with accuracy, mean squared error, squared correlation coefficient.
prob_estimates: If selected, probability estimate vector.
Example code for training a linear SVM on a data set of four points with three features:
training_label_vector = [1 ; 1 ; -1 ; -1];
training_instance_matrix = [1 2 3 ; 3 4 5 ; 5 6 7; 7 8 9];
model = svmtrain(training_label_vector, training_instance_matrix, '-t 0');
Applying the resulting model to test data
testing_instance_matrix = [9 5 1; 2 9 5];
predicted_label = svmpredict(nan(2, 1), testing_instance_matrix, model)
results in
predicted_label =
-1
-1
You can also pass the true testing_label_vector to svmpredict so that it directly computes the accuracy; I here replaced the true labels by NaNs.
Please note that there is also a function svmtrain in Matlab's Statistics Toolbox which is incompatible with the one from LIBSVM – make sure you call the correct one.
As #A.Donda answers, you don't have to transfer the data to 'libsvm' format, if you can do the training and predicting in the matlab.
When you want to do the training and predicting work in windows or linux, you have to make the data in 'libsvm' format.
From your mistake, I think you didn't give the label in every line of'data features'. You should add the label in front of the features in every line of the data.
matlab> SPECTF = csvread('SPECTF.train'); % read a csv file
matlab> features = SPECTF(:, :); % because there are no labels in your csv file
matlab> labels = [??];% to add the label as your plan
matlab> features_sparse = sparse(features); % features must be in a sparse matrix
matlab> libsvmwrite('SPECTFlibsvm.train', labels, features_sparse);
You should provide more about your data, so we can help you with the label. BTW, label data is usually set by the user at the beginning. And you can set the label data any integer to one kind of data as you like.
can someone help me to solve this?
I want to test whether this classification is already good or not. So, I try with data testing=data training. it will give 100% (acc) if the classification is good.
this is the code that I found from this site:
data= [170 66 ;
160 50 ;
170 63 ;
173 61 ;
168 58 ;
184 88 ;
189 94 ;
185 88 ]
labels=[-1;-1;-1;-1;-1;1;1;1];
numInst = size(data,1);
numLabels = max(labels);
testVal = [1 2 3 4 5 6 7 8];
trainLabel = labels(testVal,:);
trainData = data(testVal,:);
testData=data(testVal,:);
testLabel=labels(testVal,:);
numTrain = 8; numTest =8
%# train one-against-all models
model = cell(numLabels,1);
for k=1:numLabels
model{k} = svmtrain(double(trainLabel==k), trainData, '-c 1 -t 2 -g 0.2 -b 1');
end
%# get probability estimates of test instances using each model
prob = zeros(numTest,numLabels);
for k=1:numLabels
[~,~,p] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1');
prob(:,k) = p(:,model{k}.Label==1); %# probability of class==k
end
%# predict the class with the highest probability
[~,pred] = max(prob,[],2);
acc = sum(pred == testLabel) ./ numel(testLabel) %# accuracy
C = confusionmat(testLabel, pred) %# confusion matrix
and this is the results:
optimization finished, #iter = 16
nu = 0.645259 obj = -2.799682,
rho = -0.437644 nSV = 8, nBSV = 1 Total nSV = 8
Accuracy = 100% (8/8) (classification)
acc =
0.3750
C =
0 5
0 3
I dont know why there's two accuracy, and its different. the first one is 100% and the second one is 0.375. is my code false? it should be 100% not 37.5%. Can u help me to correct this code??
If your using libsvm then you should change the name of the MEX file since Matlab already has a svm toolbox with the name svmtrain. However, the code is running so it seems you did change the name just not on the code you provided.
The second one is wrong, don't know exactly why. However, I can tell you that you will almost always get 100% accuracy if you use the test_Data = training_Data. That result really does not mean anything, since the algorithm can be overfit and not be shown in your results. Test your algorithm against new data and that will give you a realistic accuracy.
Is that the code you're using? I don't think your svmtrain invocation is valid. You should have svmtrain(MAT, VECT, ...) where MAT is a matrix of data, and VECT is a vector with the labels of each row of MAT. The remaining parameters are string-value pairs, meaning you'll have a string identifier and its corresponding valie.
When I ran your code (Linux, R2011a) I got an error on the svmtrain call. Running with svmtrain(trainData, double(trainLabel==k)) gave a valid output (for that line). Of course, it appears that you're not using pure matlab, as your svmpredict call isn't native matlab, but rather a matlab binding from LIBSVM...
C = confusionmat(testLabel, pred)
swap their positions
C= confusionmat(pred,testLabel)
or use this
[ConMat,order] = confusionmat(pred,testLabel)
shows the confusion matrix and the class order
The problem is in
[~,~,p] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1');
p does not contain the predicted labels, it has the probability estimates of the labels being correct. LIBSVM's svmpredict already calculates accuracy for you correctly, that's why it says 100% in the debug output.
The fix is simple:
[p,~,~] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1');
According to LIBSVM's Matlab bindings README:
The function 'svmpredict' has three outputs. The first one,
predictd_label, is a vector of predicted labels. The second output,
accuracy, is a vector including accuracy (for classification), mean
squared error, and squared correlation coefficient (for regression).
The third is a matrix containing decision values or probability
estimates (if '-b 1' is specified). If k is the number of classes
in training data, for decision values, each row includes results of
predicting k(k-1)/2 binary-class SVMs. For classification, k = 1 is a
special case. Decision value +1 is returned for each testing instance,
instead of an empty vector. For probabilities, each row contains k values
indicating the probability that the testing instance is in each class.
Note that the order of classes here is the same as 'Label' field
in the model structure.
I am sorry to tell that all answers are totally wrong!!
The main error done in the code is:
numLabels = max(labels);
because it returns (1), although it should return 2 if the labels are positive numbers, and then svmtrain/svmpredict will loop twice.
Anyway, change line labels=[-1;-1;-1;-1;-1;1;1;1];
to labels=[2;2;2;2;2;1;1;1];
and it will work successfully ;)
I'm currently working on classifying images with different image-descriptors. Since they have their own metrics, I am using precomputed kernels. So given these NxN kernel-matrices (for a total of N images) i want to train and test a SVM. I'm not very experienced using SVMs though.
What confuses me though is how to enter the input for training. Using a subset of the kernel MxM (M being the number of training images), trains the SVM with M features. However, if I understood it correctly this limits me to use test-data with similar amounts of features. Trying to use sub-kernel of size MxN, causes infinite loops during training, consequently, using more features when testing gives poor results.
This results in using equal sized training and test-sets giving reasonable results. But if i only would want to classify, say one image, or train with a given amount of images for each class and test with the rest, this doesn't work at all.
How can i remove the dependency between number of training images and features, so i can test with any number of images?
I'm using libsvm for MATLAB, the kernels are distance-matrices ranging between [0,1].
You seem to already have figured out the problem... According to the README file included in the MATLAB package:
To use precomputed kernel, you must include sample serial number as
the first column of the training and testing data.
Let me illustrate with an example:
%# read dataset
[dataClass, data] = libsvmread('./heart_scale');
%# split into train/test datasets
trainData = data(1:150,:);
testData = data(151:270,:);
trainClass = dataClass(1:150,:);
testClass = dataClass(151:270,:);
numTrain = size(trainData,1);
numTest = size(testData,1);
%# radial basis function: exp(-gamma*|u-v|^2)
sigma = 2e-3;
rbfKernel = #(X,Y) exp(-sigma .* pdist2(X,Y,'euclidean').^2);
%# compute kernel matrices between every pairs of (train,train) and
%# (test,train) instances and include sample serial number as first column
K = [ (1:numTrain)' , rbfKernel(trainData,trainData) ];
KK = [ (1:numTest)' , rbfKernel(testData,trainData) ];
%# train and test
model = svmtrain(trainClass, K, '-t 4');
[predClass, acc, decVals] = svmpredict(testClass, KK, model);
%# confusion matrix
C = confusionmat(testClass,predClass)
The output:
*
optimization finished, #iter = 70
nu = 0.933333
obj = -117.027620, rho = 0.183062
nSV = 140, nBSV = 140
Total nSV = 140
Accuracy = 85.8333% (103/120) (classification)
C =
65 5
12 38