When I run the below mentioned SVM classifier (One against all) code I get the following error at the line : model{k} = svmtrain(double(trainLabel==k), trainData, '-c 1 -g 0.2 -b 1');
In matlab:Y must be a vector or a character array.
Can anyone help me?
The code is:
%# Fisher Iris dataset
load fisheriris
[~,~,labels] = unique(species); %# labels: 1/2/3
data = zscore(meas); %# scale features
numInst = size(data,1);
numLabels = max(labels);
%# split training/testing
idx = randperm(numInst);
numTrain = 100; numTest = numInst - numTrain;
trainData = data(idx(1:numTrain),:); testData = data(idx(numTrain+1:end),:);
trainLabel = labels(idx(1:numTrain)); testLabel = labels(idx(numTrain+1:end));
Here is the implementation for the one-against-all approach for multi-class SVM:
%# train one-against-all models
model = cell(numLabels,1);
for k=1:numLabels
model{k} = svmtrain(double(trainLabel==k), trainData, '-c 1 -g 0.2 -b 1');
end
%# get probability estimates of test instances using each model
prob = zeros(numTest,numLabels);
for k=1:numLabels
[~,~,p] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1');
prob(:,k) = p(:,model{k}.Label==1); %# probability of class==k
end
%# predict the class with the highest probability
[~,pred] = max(prob,[],2);
acc = sum(pred == testLabel) ./ numel(testLabel) %# accuracy
C = confusionmat(testLabel, pred) %# confusion matrix
You are mixing up the use of the function svmtrain from the libsvm library with the function of the same name included in Matlab's Statistics Toolbox.
In the libsvm library, the third input to svmtrain is a string that contains command line-like options. In Matlab's version, options are set using Name/Value pairs. The string you use, '-c 1 -g 0.2 -b 1', means cost parameter = 1, gamma parameter of the radial basis function kernel = 0.2, and you are requesting probability estimates. To use the same SVM in the Matlab implementation, I think you have to specify the options 'boxconstraint', 1, 'kernel_function' 'rbf', 'rbf_sigma', 0.2. The Matlab implementation does not support probability estimates though.
But since the code you want to use is apparently based on the libsvm library, the easiest way to make it work would be to install libsvm.
Related
I try to code a NN that predicts the cross product of 2 vectors in R^3.
XTrain has 3:2:1000 dimensions and is an array of matrices 3x2 (or basically 2 vectors 3x1). YTrain is a 3:1000 array. Here is my code:
Xtrain = zeros(3,2,1000);
Ytrain = zeros(3,1000);
for c = 1:1000
v1 = -1 + (1+1)*rand(3,1);
v2 = -1 + (1+1)*rand(3,1);
C = cross(v1,v2);
m = [v1, v2];
Xtrain(:,:,c) = m;
Ytrain(:,c) = C;
end
net = feedforwardnet([3 2]);
net.layers{1}.transferFcn = 'tansig';
net.layers{2}.transferFcn = 'purelin';
net = train(net, Xtrain, Ytrain);
However Matlab gives 'Inputs X is not two-dimensional.' error. What is the best solution here?
In feedforward neural networks inputs are always 1D.
So unroll your 2D input into a 1D array, and just train with that!
I am trying to predict the energy output (y), based on two predictors (X).
I have a total sample of 7034 samples (Xtot and ytot), corresponding to nearly 73 days of records.
I selected a week period within the data.
Then, I used the fitlm to create the MLR model.
Proceeded to the prediction.
Is this right? Is this the way that it should be used to obtain a 48 steps ahead prediction?
Thank you!
Xtot = dadosPVPREV(2:3,:);%predictors
ytot = dadosPVPREV(1,:);%variable to be predicted
Xtot = Xtot';
ytot = ytot';
X = Xtot(1:720,:);%period into consideration - predictors
y = ytot(1:720,:);%period into considaration - variable to be predicted
lmModel = fitlm(X, y, 'linear', 'RobustOpts', 'on'); %MLR fit
Xnew = Xtot(720:769,:); %new predictors of the y
ypred = predict(lmModel, Xnew); %predicted values of y
yreal = ytot(720:769); %real values of the variable to be predicted
RMSE = sqrt(mean((yreal-ypred).^2)); %calculation of the error between the predicted and real values
figure; plot(ypred);hold; plot(yreal)
I see that over the past few days you have been struggling to train a prediction model. The following is an example of training such a model using linear regression. In this example, the values of the previous few steps are used to predict 5 steps ahead. The Mackey-Glass function is used as a data set to train the model.
close all; clc; clear variables;
load mgdata.dat; % importing Mackey-Glass dataset
T = mgdata(:, 1); % time steps
X1 = mgdata(:, 2); % 1st predictor
X2 = flipud(mgdata(:, 2)); % 2nd predictor
Y = ((sin(X1).^2).*(cos(X2).^2)).^.5; % response
to_x = [-21 -13 -8 -5 -3 -2 -1 0]; % time offsets in the past, used for predictors
to_y = +3; % time offset in the future, used for reponse
T_trn = ((max(-to_x)+1):700)'; % time slice used to train model
i_x_trn = bsxfun(#plus, T_trn, to_x); % indices of steps used to construct train data
X_trn = [X1(i_x_trn) X2(i_x_trn)]; % train data set
Y_trn = Y(T_trn+to_y); % train responses
T_tst = (701:(max(T)-to_y))'; % time slice used to test model
i_x_tst = bsxfun(#plus, T_tst, to_x); % indices of steps used to construct train data
X_tst = [X1(i_x_tst) X2(i_x_tst)]; % test data set
Y_tst = Y(T_tst+to_y); % test responses
mdl = fitlm(X_trn, Y_trn) % training model
Y2_trn = feval(mdl, X_trn); % evaluating train responses
Y2_tst = feval(mdl, X_tst); % evaluating test responses
e_trn = mse(Y_trn, Y2_trn) % train error
e_tst = mse(Y_tst, Y2_tst) % test error
Also, using data transformation technique to generate new features in some models can reduce the prediction error:
featGen = #(x) [x x.^2 sin(x) exp(x) log(x)]; % feature generator
mdl = fitlm(featGen(X_trn), Y_trn)
Y2_trn = feval(mdl, featGen(X_trn)); % evaluating train responses
Y2_tst = feval(mdl, featGen(X_tst)); % evaluating test responses
I have code for naive Bayes classifier that implement the concept of the naive Bayes, but the accuracy that this algorithm gives me is about 48% and it much lower than MATLAB build-in function for Naive Bayes (84%). Can anybody help me where is the problem?
here is my code:
function [conf, confMat] = NaiveBayesClassifier(train, test)
Att_cnt = size(train, 2) - 1;
% training set
x = train(:, 1:Att_cnt);
y = train(:, Att_cnt+1);
% test set
u = test(:, 1:Att_cnt);
v = test(:, Att_cnt+1);
yu = unique(y);
nc = length(yu); % number of classes
ni = size(x,2); % independent variables
ns = length(v); % test set
% compute class probability
for i = 1 : nc
fy(i) = sum(double(y==yu(i)))/length(y);
end
% normal distribution
% parameters from training set
[mu, sigma] = MLE(train);
% probability for test set
for j = 1 : ns
fu = normcdf(ones(nc,1)*u(j,:), mu, sigma);
P(j,:)= fy.*prod(fu,2)';
end
% get predicted output for test set
[pv0, id] = max(P,[],2);
for i = 1 : length(id)
pv(i,1) = yu(id(i));
end
% compare predicted output with actual output from test data
confMat = confusionmat(v,pv);
conf = sum(pv==v)/length(pv);
end
I just solve it. instead of this line
fu = normcdf(ones(nc,1)*u(j,:), mu, sigma);
i have to write
fu = normpdf(ones(nc,1)*u(j,:), mu, sigma);
so the accuracy is the same as matlab build in function.
I have a "training set" of images. I have formed the 'Eigenspace'. Now i need to label the projections to train the SVM. The projections of "face 1" to the Eigenspace has to be labelled +1 and the projections of all the other faces to the Eigenspace has to be labelled -1.
I don't know how to do this.Any suggestions would be really helpful!
I formed the eigenspace using the following :
function [signals,V] = pca2(data)
[M,N] = size(data);
data = reshape(data, M*N,1); % subtract off the mean for each dimension
mn = mean(data,2);
data = bsxfun(#minus, data, mean(data,1));
% construct the matrix Y
Y = data'*data / (M*N-1);
[V D] = eigs(Y, 10); % reduce to 10 dimension
% project the original data
signals = data * V;
label = ones(N,1);% N samples in total, +1 represents face 1
for i=1:N
% For each face image, you run
[signals,V] = pca2(data); % ith data
if .... % other faces than face 1
label(i) = -1;
end
face(i,:) = reshape(signals,1,[]);
end
model = svmtrain(label,face);
If you are trying to recognize more than one person, you have to create one separate data file for each person, and one sepparate SVM for each person. This is because SVM are focused on two-class separation.
This is an example using libsvm for Matlab (here is the full code), supposing you have the data in a file:
[person1_label, person1_inst] = libsvmread('../person1');
[person2_label, person2_inst] = libsvmread('../person2');
[person3_label, person3_inst] = libsvmread('../person3');
model1 = svmtrain(person1_label, person1_inst, '-c 1 -g 0.07 -b 1');
model2 = svmtrain(person2_label, person2_inst, '-c 1 -g 0.07 -b 1');
model3 = svmtrain(person3_label, person3_inst, '-c 1 -g 0.07 -b 1');
To test one face, you need to apply all the models and get the max output (when using svmpredict you have to use '-b 1' to obtain the probability estimates.
Additionally, in Matlab you don't need to use svmread or svmwrite, you can pass directly the data:
training_data = [];%Your matrix that contains 4 feature vectors
person1_label =[1,1,-1,-1];
person2_label = [-1,-1,1,-1];
person3_label = [-1,-1,-1,1];
model1 = svmtrain(person1_label, person_inst, '-c 1 -g 0.07 -b 1');
model2 = svmtrain(person2_label, person_inst, '-c 1 -g 0.07 -b 1');
model3 = svmtrain(person3_label, person_inst, '-c 1 -g 0.07 -b 1');
It seems that you cannot train the SVM... This is an example on how you can train a Matlab SVM:
%Generate 100 positive points (the data is a circle)
r = sqrt(rand(100,1)); % radius
t = 2*pi*rand(100,1); % angle
dataP = [r.*cos(t), r.*sin(t)]; % points
%Generate 100 negative points (the data is a circle)
r2 = sqrt(3*rand(100,1)+1); % radius
t2 = 2*pi*rand(100,1); % angle
dataN = [r2.*cos(t2), r2.*sin(t2)]; % points
data3 = [dataN;dataP];
theclass = ones(200,1);
theclass(1:100) = -1; %First 100 points are negative
%Train the SVM
cl = svmtrain(data3,theclass,'Kernel_Function','rbf');
I want to do a 10-fold cross-validation in my one-against-all support vector machine classification in MATLAB.
I tried to somehow mix these two related answers:
Multi-class classification in libsvm
Example of 10-fold SVM classification in MATLAB
But as I'm new to MATLAB and its syntax, I didn't manage to make it work till now.
On the other hand, I saw just the following few lines about cross validation in the LibSVM README files and I couldn't find any related example there:
option -v randomly splits the data into n parts and calculates cross
validation accuracy/mean squared error on them.
See libsvm FAQ for the meaning of outputs.
Could anyone provide me an example of 10-fold cross-validation and one-against-all classification?
Mainly there are two reasons we do cross-validation:
as a testing method which gives us a nearly unbiased estimate of the generalization power of our model (by avoiding overfitting)
as a way of model selection (eg: find the best C and gamma parameters over the training data, see this post for an example)
For the first case which we are interested in, the process involves training k models for each fold, and then training one final model over the entire training set.
We report the average accuracy over the k-folds.
Now since we are using one-vs-all approach to handle the multi-class problem, each model consists of N support vector machines (one for each class).
The following are wrapper functions implementing the one-vs-all approach:
function mdl = libsvmtrain_ova(y, X, opts)
if nargin < 3, opts = ''; end
%# classes
labels = unique(y);
numLabels = numel(labels);
%# train one-against-all models
models = cell(numLabels,1);
for k=1:numLabels
models{k} = libsvmtrain(double(y==labels(k)), X, strcat(opts,' -b 1 -q'));
end
mdl = struct('models',{models}, 'labels',labels);
end
function [pred,acc,prob] = libsvmpredict_ova(y, X, mdl)
%# classes
labels = mdl.labels;
numLabels = numel(labels);
%# get probability estimates of test instances using each 1-vs-all model
prob = zeros(size(X,1), numLabels);
for k=1:numLabels
[~,~,p] = libsvmpredict(double(y==labels(k)), X, mdl.models{k}, '-b 1 -q');
prob(:,k) = p(:, mdl.models{k}.Label==1);
end
%# predict the class with the highest probability
[~,pred] = max(prob, [], 2);
%# compute classification accuracy
acc = mean(pred == y);
end
And here are functions to support cross-validation:
function acc = libsvmcrossval_ova(y, X, opts, nfold, indices)
if nargin < 3, opts = ''; end
if nargin < 4, nfold = 10; end
if nargin < 5, indices = crossvalidation(y, nfold); end
%# N-fold cross-validation testing
acc = zeros(nfold,1);
for i=1:nfold
testIdx = (indices == i); trainIdx = ~testIdx;
mdl = libsvmtrain_ova(y(trainIdx), X(trainIdx,:), opts);
[~,acc(i)] = libsvmpredict_ova(y(testIdx), X(testIdx,:), mdl);
end
acc = mean(acc); %# average accuracy
end
function indices = crossvalidation(y, nfold)
%# stratified n-fold cros-validation
%#indices = crossvalind('Kfold', y, nfold); %# Bioinformatics toolbox
cv = cvpartition(y, 'kfold',nfold); %# Statistics toolbox
indices = zeros(size(y));
for i=1:nfold
indices(cv.test(i)) = i;
end
end
Finally, here is simple demo to illustrate the usage:
%# laod dataset
S = load('fisheriris');
data = zscore(S.meas);
labels = grp2idx(S.species);
%# cross-validate using one-vs-all approach
opts = '-s 0 -t 2 -c 1 -g 0.25'; %# libsvm training options
nfold = 10;
acc = libsvmcrossval_ova(labels, data, opts, nfold);
fprintf('Cross Validation Accuracy = %.4f%%\n', 100*mean(acc));
%# compute final model over the entire dataset
mdl = libsvmtrain_ova(labels, data, opts);
Compare that against the one-vs-one approach which is used by default by libsvm:
acc = libsvmtrain(labels, data, sprintf('%s -v %d -q',opts,nfold));
model = libsvmtrain(labels, data, strcat(opts,' -q'));
It may be confusing you that one of the two questions is not about LIBSVM. You should try to adjust this answer and ignore the other.
You should select the folds, and do the rest exactly as the linked question. Assume the data has been loaded into data and the labels into labels:
n = size(data,1);
ns = floor(n/10);
for fold=1:10,
if fold==1,
testindices= ((fold-1)*ns+1):fold*ns;
trainindices = fold*ns+1:n;
else
if fold==10,
testindices= ((fold-1)*ns+1):n;
trainindices = 1:(fold-1)*ns;
else
testindices= ((fold-1)*ns+1):fold*ns;
trainindices = [1:(fold-1)*ns,fold*ns+1:n];
end
end
% use testindices only for testing and train indices only for testing
trainLabel = label(trainindices);
trainData = data(trainindices,:);
testLabel = label(testindices);
testData = data(testindices,:)
%# train one-against-all models
model = cell(numLabels,1);
for k=1:numLabels
model{k} = svmtrain(double(trainLabel==k), trainData, '-c 1 -g 0.2 -b 1');
end
%# get probability estimates of test instances using each model
prob = zeros(size(testData,1),numLabels);
for k=1:numLabels
[~,~,p] = svmpredict(double(testLabel==k), testData, model{k}, '-b 1');
prob(:,k) = p(:,model{k}.Label==1); %# probability of class==k
end
%# predict the class with the highest probability
[~,pred] = max(prob,[],2);
acc = sum(pred == testLabel) ./ numel(testLabel) %# accuracy
C = confusionmat(testLabel, pred) %# confusion matrix
end