I am looking for a package that provides a single neuron/perceptron for classification in R. So far I only found multi-layer perceptrons. Where can I find one?
Related
I am trying to detect the faces using the Matlab built-in viola jones face detection. Is there anyway that I can combine two classification models like "FrontalFaceCART" and "ProfileFace" into one in order to get a better result?
Thank you.
You can't combine models. That's a non-sense in any classification task since every classifier is different (works differently, i.e. different algorithm behind it, and maybe is also trained differently).
According to the classification model(s) help (which can be found here), your two classifiers work as follows:
FrontalFaceCART is a model composed of weak classifiers, based on classification and regression tree analysis
ProfileFace is composed of weak classifiers, based on a decision stump
More infos can be found in the link provided but you can easily see that their inner behaviour is rather different, so you can't mix them or combine them.
It's like (in Machine Learning) mixing a Support Vector Machine with a K-Nearest Neighbour: the first one uses separating hyperplanes whereas the latter is simply based on distance(s).
You can, however, train several models in parallel (e.g. independently) and choose the model that better suits you (e.g. smaller error rate/higher accuracy): so you basically create as many different classifiers as you like, give them the same training set, evaluate each accuracy (and/or other parameters) and choose the best model.
One option is to make a hierarchical classifier. So in a first step you use the frontal face classifier (assuming that most pictures are frontal faces). If the classifier fails, you try with the profile classifier.
I did that with a dataset of faces and it improved my overall classification accuracy. Furthermore, if you have some a priori information, you can use it. In my case the faces were usually in the middle up part of the picture.
To further improve your performance, without using the two classifiers in MATLAB you are using, you would need to change your technique (and probably your programming language). This is the best method so far: Facenet.
I know this is a generic question but I would like some feedback.
So, I have an image classification task, which I am trying to solve with neural networks. I already have my feature vectors.
There are 3 salient feature vectors (point coordinates in 2d space), each of which can belong to three classes.
These classes are not the same among these 3 features. (for instance, one could be "up down straight" the other "round square triangular").
These are combined in an overall classification task in 6 output classes (not 27 as the number of combinations would maybe imply).
So, what's the question:
Are deep learning networks / convolutional networks a good solution to this? Or would you prefer something else? (I have just began learning about these and cant make up my mind yet, so I would appreciate some feedback from more experienced users)
As I understood, your problem is not so complicated to implement deep learning. Because you have low dimensional features and few class outputs. Deep learning is mostly for highly nonlinear and large-size classification problems. I suggest you you to use AdaBoost or SVM classifier for this kind of simple classification tasks. Matlab has great tools for above techniques.
i have a training set and i want to use a classification method for classifying other documents according to my training set.my document types are news and categories are sports,politics,economic and so on.
i understand naive bayes and KNN completely but SVM and decision tree are vague and i dont know if i can implement this method by myself?or there is applications for using this methods?
what is the best method i can use for classifying docs in this way?
thanks!
Naive Bayes
Though this is the simplest algorithm and everything is deemed independent, in real text classification case, this method work great. And I would try this algorithm first for sure.
KNN
KNN is for clustering rather than classification. I think you misunderstand the conception of clustering and classification.
SVM
SVM has SVC(classification) and SVR(Regression) algorithms to do class classification and prediction. It sometime works good, but from my experiences, it has bad performance in text classification, as it has high demands for good tokenizers (filters). But the dictionary of the dataset always has dirty tokens. The accuracy is really bad.
Random Forest (decision tree)
I've never try this method for text classification. Because I think decision tree need several key nodes, while it's hard to find "several key tokens" for text classification, and random forest works bad for high sparse dimensions.
FYI
These are all from my experiences, but for your case, you have no better ways to decide which methods to use but to try every algorithm to fit your model.
Apache's Mahout is a great tool for machine learning algorithms. It integrates three aspects' algorithms: recommendation, clustering, and classification. You could try this library. But you have to learn some basic knowledge about Hadoop.
And for machine learning, weka is a software toolkit for experiences which integrates many algorithms.
Linear SVMs are one of the top algorithms for text classification problems (along with Logistic Regression). Decision Trees suffer badly in such high dimensional feature spaces.
The Pegasos algorithm is one of the simplest Linear SVM algorithms and is incredibly effective.
EDIT: Multinomial Naive bayes also works well on text data, though not usually as well as Linear SVMs. kNN can work okay, but its an already slow algorithm and doesn't ever top the accuracy charts on text problems.
If you are familiar with Python, you may consider NLTK and scikit-learn. The former is dedicated to NLP while the latter is a more comprehensive machine learning package (but it has a great inventory of text processing modules). Both are open source and have great community suport on SO.
I'm new in Classification so I'm asking for some advice on how to start.
I've created a Matlab script which create two matrices, one is the class identifier, meaning 100x1 which contains the group from where the data is. group one (1) or group two (2).
The second matrix contains the features 100x40 with 40 features for each point.
What's the best way to start, I'm really lost. Does Matlab has some functions I can use?
I would really appreciate some help.
Thank you.
It depends on what version of MATLAB you are using, but the best starting point would be to look at statistics toolbox for supervised learning. Here are some starting tips for MATLAB 2013a:
http://www.mathworks.co.uk/help/stats/supervised-learning.html
Let's assume that your data is
classes: 100x1
features: 100x40
For each method, the first line shows you how to fit your classification model and the second lines shows how to classify the first row of data in features.
Statistics Toolbox
Naive Bayes Classification
Wikipedia: https://en.wikipedia.org/wiki/Naive_Bayes_classifier
myClassifier = NaiveBayes.fit(features, classes)
myClassifier.predict(features(1,:))
Nearest Neighbors
Wikipedia: https://en.wikipedia.org/wiki/Nearest_neighbour_classifiers
myClassifier = ClassificationKNN.fit(features, classes)
myClassifier.predict(features(1,:))
Classification Trees
Wikipedia: https://en.wikipedia.org/wiki/Classification_tree
myClassifier = ClassificationTree.fit(features, classes)
myClassifier.predict(features(1,:))
Support Vector Machines
Wikipedia: https://en.wikipedia.org/wiki/Support_vector_machine
Note that Support Vector Machines moved into 2013a from Bioinformatics toolbox and it only supports classification into two groups.
myClassifier = svmtrain(features, classes)
svmclassify(myClassifier, features(1,:))
Discriminant Analysis
Wikipedia: https://en.wikipedia.org/wiki/Discriminant_analysis
myClassifier = ClassificationDiscriminant.fit(features, classes)
myClassifier.predict(features(1,:))
Neural Network Toolbox:
If you only have two classes, you could use Neural Network Toolbox for pattern recognition by typing nnstart
I am currently looking for a multilabel AdaBoost implementation for MATLAB or a technique for efficiently using a two-label implementation for the multilabel case. Any help in that matter would be appreciated.
You can use the same approach used in Support Vector Machines. SVMs are originally binary classifiers, several approaches were proposed for handling multiclass data:
one-against-all: construct one binary classifier per class, and train with instances in this class as positive cases and all other instances as negative cases (ie: 1-vs-not1, 2-vs-not2, 3-vs-not3). Finally use the posterior probability of each classifier to predict the class.
one-against-one: construct several binary classifiers for each pair of classes (ie: 1-vs-2, 1-vs-3, 2-vs-3, ..) by simply training over the instances from both classes. Then you can combine the individual results using a majority vote.
Error Correcting Output Codes: based on the theory of error correction (Hamming code and such), it relies on coding the output of several binary classifier using some redundancy to increase accuracy.
Note these are generic method and can applied to any binary classifier.
Otherwise you can search for a specific implementation of multiclass Adaboost, which I'm sure there are plenty out there.. A quick search revealed this one: Multiclass GentleAdaboosting
You can use Adaboost.M2, its a multiclass adaboost, you can found an implementation in Balu toolbox here the command is Bcl_adaboost this toolbox has other useful stuff, just remember to reference. Hope it helps.
Theoretically speaking, the only correct multi-class boosting is the one defined in A theory of multiclass boosting