I am working on image classification project. I utilized Lib-SVM and Vl_feat SVM implementation train a linear kernel. Both classifiers returns different result can some one explain what is the different between two libraries.
From a quick glance at the websites for the two implementations, they use different algorithms to solve the SVM problem. That is, both are SVM's but one uses one trick to find the weights, and the other uses a different trick. The results should both be similar, but not exactly the same.
Another possible difference is the parameters you are passing in to the implementations. The different libraries may have different default settings for certain parameters you are not explicitly setting.
Related
I am trying to detect the faces using the Matlab built-in viola jones face detection. Is there anyway that I can combine two classification models like "FrontalFaceCART" and "ProfileFace" into one in order to get a better result?
Thank you.
You can't combine models. That's a non-sense in any classification task since every classifier is different (works differently, i.e. different algorithm behind it, and maybe is also trained differently).
According to the classification model(s) help (which can be found here), your two classifiers work as follows:
FrontalFaceCART is a model composed of weak classifiers, based on classification and regression tree analysis
ProfileFace is composed of weak classifiers, based on a decision stump
More infos can be found in the link provided but you can easily see that their inner behaviour is rather different, so you can't mix them or combine them.
It's like (in Machine Learning) mixing a Support Vector Machine with a K-Nearest Neighbour: the first one uses separating hyperplanes whereas the latter is simply based on distance(s).
You can, however, train several models in parallel (e.g. independently) and choose the model that better suits you (e.g. smaller error rate/higher accuracy): so you basically create as many different classifiers as you like, give them the same training set, evaluate each accuracy (and/or other parameters) and choose the best model.
One option is to make a hierarchical classifier. So in a first step you use the frontal face classifier (assuming that most pictures are frontal faces). If the classifier fails, you try with the profile classifier.
I did that with a dataset of faces and it improved my overall classification accuracy. Furthermore, if you have some a priori information, you can use it. In my case the faces were usually in the middle up part of the picture.
To further improve your performance, without using the two classifiers in MATLAB you are using, you would need to change your technique (and probably your programming language). This is the best method so far: Facenet.
I was trying to use fitcsvm to train and classify my data. However, I notice - correct me if I'm wrong - that fitcsvm could only be used with 2 classes (groups).
My data have more than 2 classes. Is there away to do classify them in matlab?
I did some googling and I read that some recommend to use fitcecoc, while others recommend to use out of the box code multisvm
Morover, other recommend to use discriminant analysis
Please, advise on best approach to go.
you are correct, fitcsvm is for one or two classes, you may use svmtrain which is matlab's svm classifier for more then two classes, also there is a famous toolbox named libsvm, if you google it would be found easily.
https://github.com/cjlin1/libsvm
Recently I saw some new method for multiply svm classifer named DSVM, it's nice new method,this would be found in matlab's files exchange.
http://www.mathworks.com/matlabcentral/fileexchange/48632-multiclass-svm-classifier
good luck
I am trying different classifiers of Weka on my data set. I have small dataset and I am classifying my data into five classes.
My problem is that when I apply cross validation or percentage split classification by different classifiers, I get very different results.
For example, when I use NaiveBayse or BayseNet classifiers, I have an F-score of around 40 for all classes, but using SMO I get an F-score of 20. The worse result is obtained when I use LibLinear classifier which gives me a F-scores of around 15.
Maybe I should mention that since LibLinear classifier doesn't accept nominals, I assign a code to each of the possible nominal values and use them as Numeric values in my dataset.
Can anybody tell me why I get such different results? I expected all classifiers to have roughly similar results.
In addition, when I use LibLinear on my test set, I have all data classified under one of the classes and there is no instances in the other four classes.
Thanks in advance,
Why would you expect similar results? For small data set especially I think different methods could easily lead to different predictions. Also linear model has tolerance threshold that would cause early termination before convergence. It's something you can play with in LibLINEAR or SMO for instance.
I'm trying to combine multiple classifiers (ANN, SVM, kNN, ... etc.) using ensemble learning (viting, stacking ...etc.) .
In order to make a classifier, I'm using more than 20 types of explanatory variables.
However, each classifier has the best subset of explanatory variables. Thus, seeking the best combination of explanatory variables for each classifier in wrapper method,
I would like to combine multiple classifiers (ANN, SVM, kNN, ... etc.) using ensemble learning (viting, stacking ...etc.) .
By using the meta-learning with weka, I should be able to use the ensemble itself.
But I can not obtain the best combination of explanatory variables since wrapper method summarizes the prediction of each classifier.
I am not stick to weka if it can be solved easier in maybe matlab or R.
With ensemble approaches, best results have been achieved with very simple classifiers. Which on the other hand can be pretty fast, to make up for the ensemble cost.
This may seem counterintuitive at first: one would exepect a better input classifier to produce a better output. However, there are two reasons why this does not work.
First of all, with simple classifiers, you can usually tweak them more to get a diverse set of input classifiers. A full-dimensional method + feature bagging gives you a diverse set of classifiers. A classifier that internally does feature selection or reduction makes feature bagging largely disfunct for getting variety. Secondly, a complex method such as SVM is more likely to optimize/converge towards the very same result. After all, the complex methods are supposed to go through a much larger search space and find the best result in this search space. But that also means, you are more likely to get the same result again.
Last but not least, when using very primivite classifiers, the errors are better behaved and more likely to even out on ensemble combination.
I am currently looking for a multilabel AdaBoost implementation for MATLAB or a technique for efficiently using a two-label implementation for the multilabel case. Any help in that matter would be appreciated.
You can use the same approach used in Support Vector Machines. SVMs are originally binary classifiers, several approaches were proposed for handling multiclass data:
one-against-all: construct one binary classifier per class, and train with instances in this class as positive cases and all other instances as negative cases (ie: 1-vs-not1, 2-vs-not2, 3-vs-not3). Finally use the posterior probability of each classifier to predict the class.
one-against-one: construct several binary classifiers for each pair of classes (ie: 1-vs-2, 1-vs-3, 2-vs-3, ..) by simply training over the instances from both classes. Then you can combine the individual results using a majority vote.
Error Correcting Output Codes: based on the theory of error correction (Hamming code and such), it relies on coding the output of several binary classifier using some redundancy to increase accuracy.
Note these are generic method and can applied to any binary classifier.
Otherwise you can search for a specific implementation of multiclass Adaboost, which I'm sure there are plenty out there.. A quick search revealed this one: Multiclass GentleAdaboosting
You can use Adaboost.M2, its a multiclass adaboost, you can found an implementation in Balu toolbox here the command is Bcl_adaboost this toolbox has other useful stuff, just remember to reference. Hope it helps.
Theoretically speaking, the only correct multi-class boosting is the one defined in A theory of multiclass boosting