I am a student who studies depth estimation.
I am studying by referring to monodepth2, but the difference in score between validation dataset and test dataset is too severe.
Like monodepth2, learning used train dataset of eigen_zhou, and validation also used split of eigen_zhou.
Why is the test score so different from the validation score?
After the training, the test used test dataset, which is the split of eigen.
train, val
test
Thanks.
Related
I am running a few classification models like logistic regression and catboosting. I have taken away part of the train set as unseen data .
When I train both on train and unseen data and get the metrics using log regression I am getting all the metrics like accuracy , AUC,F1,Recall all greater than 0.90. As it's a class imbalance problem i have even balanced the classes using smote .And also I have used z score to normalise all variables
Where the model performs well on train and unseen data and on test data , when I actually run on the set of data ( unlabelled) which I want to predict model is only giving me 10 1s. And rest 150k 0s
Could there be really an issue with my model or it is indeed the way the data is ?
I am working on Multi-Class classification problem, where I have designed
SVM classifier
for
31 different classes
After the classification process I got the predicted labels now I want to calculate the accuracy rate for each distinct class. Although I have calculated the overall accuracy of model but I want more detail.
Can some one provide me Matlab code for this purpose or guide me to calculate the desired result.
I am using Weka's IBk for performing classification on text (tweets). I am converting the training and test data to vector space, and when I am performing the classification on test data, the best result comes from K=1. The training and testing data are separate from each other. Why does K=1 give the best accuracy?
Because you are using vectors; so at k=1 the value you get for proximity (for k=1) is more important than what the common class is in case of k=n (ex: when k=5)
I am trying to classify the four groups of images using SVM method, by randomly selecting training and testing data each time. When T run the program the performance varies due to randomly selecting data. How to get accurate performance of my algorithm and also how to calculate training and testing accuracy?
The formula I am using for performance is
Performance = sum(PredictedLabels == test_labels) / numel(PredictedLabels)
I am using multisvm function for classification.
My suggestion:
Actually the performance measure is acceptable, though there are some other slightly better choices like #Dan has mentioned.
More importantly, you need to deal with randomness.
1) Everytime you select your training data, test the trained model with multiple randomized test data and average the accuracy. (e.g. 10 times or so)
2) Use multiple trained model and average the performance to get general performance.
Remark:
1) You need to make sure the training data and test data do not overlap. Or it is no longer test data.
2) It is better to have the training data have the same number of samples from each class label. This means you can partition your dataset in advance.
I have trained a NN with Back Propagation algorithm and calculated the MSE. Now I want to find the percentage of correctly classified results (i am facing a classification problem). Any help?
It depends on your dataset whether you generate the data or whether you are given a dataset with samples.
In the first case you feed your NN with a generated sample and check whether NN predicts the correct class. You repeat it let say 100 times. And for each correctly classified sample you increment the counter CorrectlyClassified by one. Then the percentage of correctly classified results is equal to CorrectlyClassified. For higher accuracy you may not generate 100 samples, but X samples (where X is bigger than 100). Then the percentage of correctly classified results is:
CorrectlyClassified/X*100.
If you are given a dataset you should use cross-validation. See MATLAB documentation for an example.