So I'm trying to implement a multi-class SVM.
Matlab didn't like having more than two classes to classify data into, so I'm using a Binary Decision Tree to classify data.
I have three classes and am splitting the data into two and one, I'll then classify the first results using a SVM and then classify the results of this to the one unclassified class.
However, when using LIBSVM, I'm getting an error when using svmpredict:
td= a{1,1};
tc = b{1,1};
td1 = a{1,2}; %data to test svm
testdatatest = td1(1:30,1:4); %data to test svm
data = td(1:80, 1:4); %split data
target = tc(1:80); %split data
model = svmtrain(data, target); %train
[predicted_label, accuracy, decision_values]=svmpredict(testdatatest,target, model);
The error I get is:
Undefined function 'svmpredict' for input arguments of type 'struct'.
Any suggestions would be great, thanks.
you must download and make libsvm, open the zip file and select your langauges like Matlab etc. then make it! it would give you two files, now you are using Matlab SVM not libsvm.
good luck
Related
I am trying to classify my input features into two classes using SVM. I want to use 10-fold cross-validation to train an SVM classifier. I am using MATLAB inbuilt functions. But while using predict() function along with crossval() function, I am getting an error:
No valid system or dataset was specified.
Does anyone knows how to resolve this issue?
Training_Features = X;
Training_Labels = Y;
SVMModel =
fitcsvm(Training_Features,Training_Labels,'KernelFunction','RBF');
CVSVMModel = crossval(SVMModel);
[Predict_Labels,Predict_Scores] =
predict(CVSVMModel,Training_Features);
I think you understood the cross validation functionality wrong. Your CVSVMModel is a so called ClassificationPartitionedModel which has no function predict() since Cross Validation is meant for testing the generalisation of your model BEFORE you train it with the WHOLE dataset (not cross-validated).
I suggest the following:
Call [Predict_Labels,Predict_Scores] = kfoldPredict(CVSVMModel); to see how well it does on each validation dataset during the cross-validation
If you're satisfied train a new SVMModel with the whole dataset and predict with that one.
EDIT:
A ClassificationPartitionedModel is a collection of models (in your case 10 different ones). You can access and even call predict() on them by for example:
[Predict_Labels,Predict_Scores] = predict(CVSVMModel.Trained{1, 1},X);
I'm relatively new to using SVM and I have a question regarding how to use the results of the SVM regression. I have found many easy-to-understand documentation on SVM classification, and I can understand how to use the result of SVM for binary classification (i.e. data on one side of the support vector is labeled as one, data on the other side of the support vector is labeled as another), but I have no been able to find such hints on SVM regression- which is why I have run into the following question:
Using both libsvm package and the fitrsvm function in MATLAB, I was able to successfully generate models that are capable of fitting the abalone data set. the result of the libsvm (using svmtrain function) was used along with svmpredict to the successfully predict with new input parameters as followed:
model=svmtrain(age_train,X_train,['-s 3 -t 2 -c 2 -g 2']);
[prediction,accuracy,~]=svmpredict(age_eval,X_eval,model);
Also, as I've said, I was able to achieve the same results using the fitrsvm function as followed:
model1=fitrsvm(X_train,age_train,'OptimizeHyperparameters','auto',...
'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName',...
'expected-improvement-plus'),'KernelFunction','rbf');
age_predict1=predict(model1,X_eval);
Now my question is, how do the svmpredict function (in the case of libsvm package) and the predict function (in the case of fitrsvm function in MATLAB) take the values within the trained models and apply them to the new input data? For example, is there a mathematical equation in which I apply the parameters of the trained model (such as the 'Mu' and the 'Sigma' parameters in the fitrsvm result) to the new input data to obtain the results?
It would be greatly appreciated if someone could help me with this or refer me to someone/somewhere who can help me, thank you very much in advance.
I am classifying gender using a KNN classifier.
I want to add an SVM classifier instead of KNN classifier with the same labels of 0 and 1 (0 for women and 1 for men)
I have a matrix of test examples, sample, a matrix of training examples, training, and a vector with the labels for the training examples group. I want class, a vector of the labels for the test examples.
class = knnclassify(sample, training, group);
if class==1
x='Male';
else
x='Female';
end
How can I change this code to find class using an SVM?
To train an SVM, you will need the Statistics and Machine Learning Toolbox.
The biggest difference between the knnclassify and using an SVM classifier is that training and classifying new labels will be two separate steps.
1. Train your SVM : fitcsvm
This step teaches the classifier how to distinguish between your two classes. It is learning a linear separator (or a weighted combination of the features) which has the largest margin between positive and negative examples. All the examples you give it need to have ground truth labels.
SVM's have many tunable parameters that you can adjust during the training step. There are several good tutorials in the Matlab documentation which describe the differences, but for the most basic version, you can just use your training examples
model = fitcsvm(training,group);
This model will be used in the next step.
2. Classify new examples : predict
To classify your new example, run
class = predict(sample, model);
Notes:
Using your model, you can also run cross-fold validation, useful for accuracy analysis.
cvModel = crossval(model);
classError = kfoldLoss(cvModel);
You can also save your model, like any other Matlab variable for future use.
save('model.m', 'model');
knnclassify comes from the bioinformatics toolbox. In the Statistics and Machine Learning Toolbox, there is also a KNN model which you train with fitcknn and classify with predict. The benefit is that you can reuse your KNN model with several sets of data, compare cross-validation results, and save it for future use.
I trained a ensemble model (RUSBoost) for a binary classification problem by the function fitensemble() in Matlab 2014a. The training by this function is performed 10-fold cross-validation through the input parameter "kfold" of the function fitensemble().
However, the output model trained by this function cannot be used to predict the labels of new data if I use the predict(model, Xtest). I checked the Matlab documents, which says we can use kfoldPredict() function to evaluate the trained model. But I did not find any input of the new data through this function. Also, I found the structure of the trained model with cross-validation is different from that model without cross-validation. So, could anyone please advise me how to use the model, which is trained with cross-validation, to predict labels of new data? Thanks!
kfoldPredict() needs a RegressionPartitionedModel or ClassificationPartitionedEnsemble object as input. This already contains the models and data for kfold cross validation.
The RegressionPartitionedModel object has a field Trained, in which the trained learners that are used for cross validation are stored.
You can take any of these learners and use it like predict(learner, Xdata).
Edit:
If k is too large, it is possible that there is too little meaningful data in one or more iteration, so the model for that iteration is less accurate.
There are no general rules for k, but k=10 like in the MATLAB default is a good starting point to play around with it.
Maybe this is also interesting for you: https://stats.stackexchange.com/questions/27730/choice-of-k-in-k-fold-cross-validation
I'm using libsvm in matlab, and it seems there no existing method to save the model formed from svmtrain. Instead, the functions provided forces me to retrain everytime. Just saving the svmtrain model variable in a .mat does not work. What should I be doing?
The nice thing about SVM, unlike NN is that training is fast and in some cases can be even done online. I have a multiclass SVM and I save the training vectors, classes and the kernel parameters into a txt file.