Currently for face detection I am using svm classifier over HOG feature set.But I need to implement other classifiers over those HOG feature set and compare the results between them .What other classifiers can I use other than svm?
There are plenty of other classification algorithms, a simple logistic regression could be a starting point. You could use logistic regression; implement gaussian process based classifier or random forrests/decision trees and many others with respective pro's and cons.
See e.g. this link for an overview
Related
My question is can we use CNN for feature extraction and then can we use this extracted feature as an input to another classification algorithm like SVM.
Thanks
Yes, this has already been done and well documented in several research papers, like CNN Features off-the-shelf: an Astounding Baseline for Recognition and How transferable are features in deep neural networks?. Both show that using CNN features trained on one dataset, but tested on a different one usually perform very well or beat the state of the art.
In general you can take the features from the layer before the last, normalize them and use them with another classifier.
Another related technique is fine tuning, where after training a network, the last layer is replaced and retrained, but previous layers' weights are kept fixed.
I have read this line about neural networks :
"Although the perceptron rule finds a successful weight vector when
the training examples are linearly separable, it can fail to converge
if the examples are not linearly separable.
My data distribution is like this :The features are production of rubber ,consumption of rubber , production of synthetic rubber and exchange rate all values are scaled
My question is that the data is not linearly separable so should i apply ANN on it or not? is this a rule that it should be applied on linerly separable data only ? as i am getting good results using it (0.09% MAPE error) . I have also applied SVM regression (fitrsvm function in MATLAB)so I have to ask can SVM be used in forecasting /prediction or it is used only for classification I haven't read anywhere about using SVM to forecast , and the results for SVM are also not good what can be the possible reason?
Neural networks are not perceptrons. Perceptron is on of the oldest ideas, which is at most a single building block of neural networks. Perceptron is designed for binary, linear classification and your problem is neither the binary classification nor linearly separable. You are looking at regression here, where neural networks are a good fit.
can SVM be used in forecasting /prediction or it is used only for classification I haven't read anywhere about using SVM to forecast , and the results for SVM are also not good what can be the possible reason?
SVM has regression "clone" called SVR which can be used for any task NN (as a regressor) can be used. There are of course some typical characteristics of both (like SVR being non parametric estimator etc.). For the task at hand - both approaches (as well as any another regressor, there are dozens of them!) is fine.
I have a question about the SVM MATLAB toolbox 2009b! the question is:
How I can train SVM classifier for classifying multi-classes datasets in MATLAB toolbox 2009b?
I just want to work with MATLAB toolbox, so please answer it if there is a way to implement it. For example, the below code is for classifying two classes datasets:
svmtrain( training data, ...
labels of training data, ...
'Kernel_Function', ...
'rbf', ...
'RBF_Sigma', ...
sigma value, ...
'Method', ...
'LS', ...
'BoxConstraint', ...
C ...
);
I want to know is there a way for training SVM for multi-classes dataset with writing a code such as above code, or should I write some code for training a SVM for each class versus the other classes?
It means, should I consider 1 for the label of the selected class and set the label of the other classes to 0, and train a SVM with above code, and do it for all classes!?
Thanks for your consideration :-)
I have not used SVM in Matlab, so other people can likely provide a more informed response, but I will share what I have learned.
Matlab Bioinformatics Toolbox SVM
From reading the documentation, the SVM in the Bioinformatics Toolbox appears to only support binary classification. As suggested in the question, a binary classifier can, with some effort, be used to classify into multiple classes. There is some discussion on approaches for doing this in the context of SVM here.
Alternate options
LIBSVM does support multi-class classification and comes with a Matlab interface. You could try installing and using it.
Additionally, while looking into this, I did come across several other Matlab toolboxes with SVM implementations. If LIBSVM is not a good option for you, it may be worth looking around to see if a different SVM implementation fits your needs.
If you have MATLAB release R2014b or later you can use the fitcecoc function in the Statistics and Machine Learning Toolbox to train a multi-class SVM.
Yup, the way for solving your problem - is to implement one vs all strategy. One of the SVM's lacks is that it has no direct multiclassification implementation.
But you can implement it through the binary classification.
I didn't see any function for svm multi classification in matlab. But i think it is not hard to implement it by yourself
Do you know an example that makes feature selection using a perceptron, maybe an implementation on matlab...
The perceptron is a binary linear classifier, that is, it can classify n-dimensional data that look like this:
but not like this:
into two distinct categories. Just like any other neural network, it first needs to be trained on a training set, and then only it can be used to classify new data points.
The perceptron can therefore be applied to classify any linearly separable dataset. A Matlab implementation is available in the Neural Network toolbox (see the documentation). An excellent toolbox for pattern recognition in general, with excellent classifiers, is PRTools, which is kind of the open source variant of the commercial toolbox PRSD Studio.
I have a dataset for text classification ready to be used in MATLAB. Each document is a vector in this dataset and the dimensionality of this vector is extremely high. In these cases peopl usually do some feature selection on the vectors like the ones that you have actually find the WEKA toolkit. Is there anything like that in MATLAB? if not can u suggest and algorithm for me to do it...?
thanks
MATLAB (and its toolboxes) include a number of functions that deal with feature selection:
RANDFEATURES (Bioinformatics Toolbox): Generate randomized subset of features directed by a classifier
RANKFEATURES (Bioinformatics Toolbox): Rank features by class separability criteria
SEQUENTIALFS (Statistics Toolbox): Sequential feature selection
RELIEFF (Statistics Toolbox): Relief-F algorithm
TREEBAGGER.OOBPermutedVarDeltaError, predictorImportance (Statistics Toolbox): Using ensemble methods (bagged decision trees)
You can also find examples that demonstrates usage on real datasets:
Identifying Significant Features and Classifying Protein Profiles
Genetic Algorithm Search for Features in Mass Spectrometry Data
In addition, there exist third-party toolboxes:
Matlab Toolbox for Dimensionality Reduction
LIBGS: A MATLAB Package for Gene Selection
Otherwise you can always call your favorite functions from WEKA directly from MATLAB since it include a JVM...
Feature selection depends on the specific task you want to do on the text data.
One of the simplest and crudest method is to use Principal component analysis (PCA) to reduce the dimensions of the data. This reduced dimensional data can be used directly as features for classification.
See the tutorial on using PCA here:
http://matlabdatamining.blogspot.com/2010/02/principal-components-analysis.html
Here is the link to Matlab PCA command help:
http://www.mathworks.com/help/toolbox/stats/princomp.html
Using the obtained features, the well known Support Vector Machines (SVM) can be used for classification.
http://www.mathworks.com/help/toolbox/bioinfo/ref/svmclassify.html
http://www.autonlab.org/tutorials/svm.html
You might consider using the independent features technique of Weiss and Kulikowski to quickly eliminate variables which are obviously unimformative:
http://matlabdatamining.blogspot.com/2006/12/feature-selection-phase-1-eliminate.html