I have been using libsvm in Matlab for our research. The accuracies of the model when used in cross-validation results are already quite good. However, I am having a small problem when trying to use the (already-trained) model for prediction. The svmpredict command requires us to specify the correct testing label (to be used for measuring accuracy) however I want deploy the model to classify new data (with unknown label), so of course there is no label for this testing data. Can I call svmpredict without specifying the (target) label of the testing data? How do I deploy the models that I have obtained in a production setting?
I don't know about matlab, but in python you have the same situation. It really doesn't matter and you can predict with that value, just ignore the accuracy value on the output.
Related
I am using MATLAB to build a prediction model which the target is binary.
The problem is that those negative observations in my training data may indeed are positives but are just not detected.
I started with a logistic regression model assuming the data is accurate and the results are less than satisfactory. After some research, I moved to one class learning hoping that I can focus on the only the part of data (the positives) that I am certain with.
I looked up the related materials from MATLAB documentation and found that I can use fitcsvm to proceed.
My current problem is:
Am I on the right path? Can one class learning solve my problem?
I tried to use fitcsvm to create a ClassificationSVM using all the positive observations that I have.
model = fitcsvm(Instance,Label,'KernelScale','auto','Standardize',true)
However, when I try to use the model to predict
[label,score] = predict(model,Test)
All the labels predicted for my Test cases are 1. I think I did something wrong. So should I feed the svm only the positive observations that I have?
If not what should I do?
I'm trying to get started using neural networks for a classification problem. I chose to use the Encog 3.x library as I'm working on the JVM (in Scala). Please let me know if this problem is better handled by another library.
I've been using resilient backpropagation. I have 1 hidden layer, and e.g. 3 output neurons, one for each of the 3 target categories. So ideal outputs are either 1/0/0, 0/1/0 or 0/0/1. Now, the problem is that the training tries to minimize the error, e.g. turn 0.6/0.2/0.2 into 0.8/0.1/0.1 if the ideal output is 1/0/0. But since I'm picking the highest value as the predicted category, this doesn't matter for me, and I'd want the training to spend more effort in actually reducing the number of wrong predictions.
So I learnt that I should use a softmax function as the output (although it is unclear to me if this becomes a 4th layer or I should just replace the activation function of the 3rd layer with softmax), and then have the training reduce the cross entropy. Now I think that this cross entropy needs to be calculated either over the entire network or over the entire output layer, but the ErrorFunction that one can customize calculates the error on a neuron-by-neuron basis (reads array of ideal inputs and actual inputs, writes array of error values). So how does one actually do cross entropy minimization using Encog (or which other JVM-based library should I choose)?
I'm also working with Encog, but in Java, though I don't think it makes a real difference. I have similar problem and as far as I know you have to write your own function that minimizes cross entropy.
And as I understand it, softmax should just replace your 3rd layer.
I have set up a model using SimMechanics. It outputs data at the times where the solver steps to. Is there any possibility to have some kind of dense output such that it is possible to interpolate these data to get the solution at arbitrary points without losing the high order of the integrator?
In Matlab this is easily possible using the function deval after the integration of one of the built-in ODE integrators.
In SimMechanics I can select these integrators, too. Is there some kind of analouge way to deval?
Yes, it's possible, although it's a Simulink functionality, not specific to SimMechanics. In the Configuration Parameters of the model, you can set the model to Produce Specified Output Only (see http://www.mathworks.co.uk/help/simulink/gui/data-import-export-pane.html#bq9_fhw-1), under Data Import/Export. This way, only the outputs you specify will be produced regardless of the time steps taken by the solver.
When I tried to train a SVM(trainsvm function) with RBF kernel,
The libSVM library outputs "Line search fails in two-class probability estimates" during training.
After training, the training accuracy of the model is just 20%.
I think I might miss something and it is related to the message.
For more information about my project,
I'm dealing with PASCAL VOC action classification problem.
I'm trying to follow this method.
http://www.ifp.illinois.edu/~jyang29/papers/CVPR09-ScSPM.pdf
There are 1300 training images and 11 classes.
After making codebooks and sparse coding,
The dimension of feature vector is 2688.
The number of training example is 1370.
You need to do a grid search, either using cross validation, or using a separate validation data set to get good values for C and gamma. Libsvm has a script called grid.py that is useful for this. I noticed you tagged this with matlab, using grid.py needs command line tools and a python installation (IMO this generally works out better than with matlab, especially if you have a some big machines to run many jobs in parallel).
I recommend that you read the libsvm guide if you haven't already done so: http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf.
I also suggest you initially use the same dataset as used for the paper as occasionally published algorithms only work well on the dataset chosen for the paper.
Lastly, you could contact the authors of the paper.
I asked about this warning the author of LIBSVM, and he replied that this warning can be ignored.
I'm doing some preliminary testing with 2 classes of vectors, trying to separate them with libsvm. I get a 78.2% correct ID rate in Matlab and at the cmd line (using libsvm), but in Weka I get around 95%.
No cross-validation was done in Weka; just trained model and then read in test dataset and classified it.
Can anyone offer an explanation? Thanks in advance.
If you didn't provide a separate Test Data , the validation Folds should be set, 10 or desired value. however, be sure that the same SVMType and kerneltype are being used in both program. by default Weka uses C-SVC with radial basis function.