I'm new in Classification so I'm asking for some advice on how to start.
I've created a Matlab script which create two matrices, one is the class identifier, meaning 100x1 which contains the group from where the data is. group one (1) or group two (2).
The second matrix contains the features 100x40 with 40 features for each point.
What's the best way to start, I'm really lost. Does Matlab has some functions I can use?
I would really appreciate some help.
Thank you.
It depends on what version of MATLAB you are using, but the best starting point would be to look at statistics toolbox for supervised learning. Here are some starting tips for MATLAB 2013a:
http://www.mathworks.co.uk/help/stats/supervised-learning.html
Let's assume that your data is
classes: 100x1
features: 100x40
For each method, the first line shows you how to fit your classification model and the second lines shows how to classify the first row of data in features.
Statistics Toolbox
Naive Bayes Classification
Wikipedia: https://en.wikipedia.org/wiki/Naive_Bayes_classifier
myClassifier = NaiveBayes.fit(features, classes)
myClassifier.predict(features(1,:))
Nearest Neighbors
Wikipedia: https://en.wikipedia.org/wiki/Nearest_neighbour_classifiers
myClassifier = ClassificationKNN.fit(features, classes)
myClassifier.predict(features(1,:))
Classification Trees
Wikipedia: https://en.wikipedia.org/wiki/Classification_tree
myClassifier = ClassificationTree.fit(features, classes)
myClassifier.predict(features(1,:))
Support Vector Machines
Wikipedia: https://en.wikipedia.org/wiki/Support_vector_machine
Note that Support Vector Machines moved into 2013a from Bioinformatics toolbox and it only supports classification into two groups.
myClassifier = svmtrain(features, classes)
svmclassify(myClassifier, features(1,:))
Discriminant Analysis
Wikipedia: https://en.wikipedia.org/wiki/Discriminant_analysis
myClassifier = ClassificationDiscriminant.fit(features, classes)
myClassifier.predict(features(1,:))
Neural Network Toolbox:
If you only have two classes, you could use Neural Network Toolbox for pattern recognition by typing nnstart
Related
I am classifying gender using a KNN classifier.
I want to add an SVM classifier instead of KNN classifier with the same labels of 0 and 1 (0 for women and 1 for men)
I have a matrix of test examples, sample, a matrix of training examples, training, and a vector with the labels for the training examples group. I want class, a vector of the labels for the test examples.
class = knnclassify(sample, training, group);
if class==1
x='Male';
else
x='Female';
end
How can I change this code to find class using an SVM?
To train an SVM, you will need the Statistics and Machine Learning Toolbox.
The biggest difference between the knnclassify and using an SVM classifier is that training and classifying new labels will be two separate steps.
1. Train your SVM : fitcsvm
This step teaches the classifier how to distinguish between your two classes. It is learning a linear separator (or a weighted combination of the features) which has the largest margin between positive and negative examples. All the examples you give it need to have ground truth labels.
SVM's have many tunable parameters that you can adjust during the training step. There are several good tutorials in the Matlab documentation which describe the differences, but for the most basic version, you can just use your training examples
model = fitcsvm(training,group);
This model will be used in the next step.
2. Classify new examples : predict
To classify your new example, run
class = predict(sample, model);
Notes:
Using your model, you can also run cross-fold validation, useful for accuracy analysis.
cvModel = crossval(model);
classError = kfoldLoss(cvModel);
You can also save your model, like any other Matlab variable for future use.
save('model.m', 'model');
knnclassify comes from the bioinformatics toolbox. In the Statistics and Machine Learning Toolbox, there is also a KNN model which you train with fitcknn and classify with predict. The benefit is that you can reuse your KNN model with several sets of data, compare cross-validation results, and save it for future use.
I just want to know if a neural network can be trained with a single class of data set. I have a set of data that I want to train a neural network with. After training it, I want to give new data(for testing) to the trained neural network to check if it can recognize it as been similar to the training sample or not.
Is this possible with neural network? If yes, will that be a supervised learning or unsupervised.
I know neural networks can be used for classification if there are multiple classes but I have not seen with a single class before. A good explanation and link to any example will be much appreciated. Thanks
Of course it can be. But in this case it will only recognize this one class that you have trained it with. And depending on the expected output you can measure the similarity to the training data.
An NN, after training, is just a function. For classification problems you can imagine it as a function that takes data as input and returns an integer indicating to which class it belongs to. That being said, if you have only one class that can be represented by an integer value 1, and if training data is not similar to that class, you will get something like 1.555; It will not tel you that it belongs to another class, because you have introduced only one, but it will definitely give you a hint about its similarity.
NNs are considered to be supervised learning, because before training you have to provide both input and target, i. e. the expected output.
If you train a network with only a single class of data then It is popularly known as One-class Classification. There are various algorithms developed in the past like One-class SVM, Support Vector Data Description, OCKELM etc. Tax and Duin developed a MATLAB toolbox for this and it supports various one-class classifiers.
DD Toolbox
One-class SVM
Kernel Ridge Regression based or Kernelized ELM based or LSSVM(where bias=0) based One-class Classification
There is a paper Anomaly Detection Using One-Class Neural Networks
which combines One-Class SVM and Neural Networks.
Here is source code. However, I've had difficulty connecting the source code and the paper.
I am trying to find a way to visualize the data with high-dimensional input for two-class classification in SVM, before analysis to decide which kernel to use. In documents online, the visualization of data is given only for two dimensional inputs (I mean two attributes).
Another question rises: What if I have multi-class and more than two attributes?
To visualize, the data should be represented by 3 or less dimension.
Simply PCA can be applied to reduce dimension.
use pre-image using MDS.
refer to a paper The pre-image problem in kernel methods and its matlab code in http://www.cse.ust.hk/~jamesk/publication.html
Do you know an example that makes feature selection using a perceptron, maybe an implementation on matlab...
The perceptron is a binary linear classifier, that is, it can classify n-dimensional data that look like this:
but not like this:
into two distinct categories. Just like any other neural network, it first needs to be trained on a training set, and then only it can be used to classify new data points.
The perceptron can therefore be applied to classify any linearly separable dataset. A Matlab implementation is available in the Neural Network toolbox (see the documentation). An excellent toolbox for pattern recognition in general, with excellent classifiers, is PRTools, which is kind of the open source variant of the commercial toolbox PRSD Studio.
I have a dataset for text classification ready to be used in MATLAB. Each document is a vector in this dataset and the dimensionality of this vector is extremely high. In these cases peopl usually do some feature selection on the vectors like the ones that you have actually find the WEKA toolkit. Is there anything like that in MATLAB? if not can u suggest and algorithm for me to do it...?
thanks
MATLAB (and its toolboxes) include a number of functions that deal with feature selection:
RANDFEATURES (Bioinformatics Toolbox): Generate randomized subset of features directed by a classifier
RANKFEATURES (Bioinformatics Toolbox): Rank features by class separability criteria
SEQUENTIALFS (Statistics Toolbox): Sequential feature selection
RELIEFF (Statistics Toolbox): Relief-F algorithm
TREEBAGGER.OOBPermutedVarDeltaError, predictorImportance (Statistics Toolbox): Using ensemble methods (bagged decision trees)
You can also find examples that demonstrates usage on real datasets:
Identifying Significant Features and Classifying Protein Profiles
Genetic Algorithm Search for Features in Mass Spectrometry Data
In addition, there exist third-party toolboxes:
Matlab Toolbox for Dimensionality Reduction
LIBGS: A MATLAB Package for Gene Selection
Otherwise you can always call your favorite functions from WEKA directly from MATLAB since it include a JVM...
Feature selection depends on the specific task you want to do on the text data.
One of the simplest and crudest method is to use Principal component analysis (PCA) to reduce the dimensions of the data. This reduced dimensional data can be used directly as features for classification.
See the tutorial on using PCA here:
http://matlabdatamining.blogspot.com/2010/02/principal-components-analysis.html
Here is the link to Matlab PCA command help:
http://www.mathworks.com/help/toolbox/stats/princomp.html
Using the obtained features, the well known Support Vector Machines (SVM) can be used for classification.
http://www.mathworks.com/help/toolbox/bioinfo/ref/svmclassify.html
http://www.autonlab.org/tutorials/svm.html
You might consider using the independent features technique of Weiss and Kulikowski to quickly eliminate variables which are obviously unimformative:
http://matlabdatamining.blogspot.com/2006/12/feature-selection-phase-1-eliminate.html