I have an unlabeled dataset that I want to classify with my newly trained classifier using NaiveBayes classification in Weka. So actually when in the Classify mode in weka if i give the option Supplied Test set, then it accepts the test set only if it is labelled and evaluates and gives the accuracy.
But what I want is to train it using a train.csv or train.arff file and then give it a new unseen and unlabelled test.csv or test.arff file and classify it and give it labels depending on classes in the training file. But if I provide an unlabelled file as test file to wweka it gives:
ERROR: Train and Test set not compatible
Sample format of my Train and test files are as below:
Train.csv file:
article story .......hockey class
1 0 ...... 0 politics
0 0 .......1 sports
.
.
.
.
. sports
and Test.csv file:
article story .......hockey class
0 1 ...... 0
1 0 .......1
.
.
.
.
.
So how do I classify an unlabelled dataset in Weka using NaiveBayes classifier??
It seems you are missing the class label. Weka requires training and test set to have the exact same attributes in the same order. Now there are two cases:
You know the classes of your test set
The performance is calculated by comparing the actual class labels with the predicted ones. You need to supply the class labels in your test set like you did in your training set.
You DON'T know the classes of your test set
To calculate a performance, Weka needs to compare the predicted classes with the actual classes. If you don't have the actual classes, you cannot calculate the performance. You can only predict classes.
You have to add a class label with missing values for your test instances if you just want prediction.
Even if your test set is labelled, Weka will not see it at first stage. It will use the classifier you developed with training data and then will apply the classifier on the test set you supply. The classifier then predicts each instance class and Weka then keeps track of a correct or incorrect classification. So, what you are doing here is exactly what you are trying to achieve. The error is telling that the training and test sets are not compatible because I believe you have removed the "class" label from the test set. Don't worry. Keep it as it is and the accuracy you are getting from Weka is the actual performance of the classifier. Hope that helps.
you cant leave it all empty, you need to set at least one each class label on the class field (as some kind of "clue" for the weka)
article story .......hockey class
0 1 ...... 0 politics
1 0 .......1 sport
1 1 .......1 ?
1 1 .......1 ?
the two first row will provide weka an example of the prediction class. Then you can predict as much as instance with no class (?) using your trained model
Related
I just created a model that does a binary classification and has a dense layer of 1 unit at the end. I used Sigmoid activation. However, I get this error now when I wanna convert it to CoreML.
I tried to change the number of units to 2 and activation to softmax but still didn't work.
import coremltools as ct
#1. define input size
image_input = ct.ImageType(scale=1/255)
#2. give classifier
classifier_config = coremltools.ClassifierConfig(class_labels=[0, 1]) #ERROR here
#3. convert the model
coreml_model = coremltools.convert("mask_detection_model_surgical_mask.h5",
inputs=[image_input], classifier_config=classifier_config)
#4. load and resize an example image
example_image = Image.open("Unknown3.jpg").resize((256, 256))
# Make a prediction using Core ML
out_dict = coreml_model.predict({mymodel.input_names[0]: example_image})
print(out_dict["classLabels"])
# save to disk
#coreml_model.save("FINALLY.mlmodel")
I found the answer to my question.
Use Softmax activation and 2 Dense units as the final layer with either loss='binary_crossentropy' or `loss='categorical_crossentropy'
Good luck to hundreds of people who posted a similar question but received no answer.
I created a NaiveBayes model in Weka. I exported the model to disk. I now want to inject this model into MATLAB 2018, so that I can check how it performs via some data that I am receiving.
I load my model in MATLAB, by stating something like this:
loadedModel = weka.core.SerializationHelper.read('myweka.model');
I then create a Weka Instance object, and let it contain this data:
instance = infrequent,low,high,medium-high,high,medium,medium,low,low
If I run these two commands:
loadedModel.distributionForInstance(instance)
loadedModel.classifyInstance(instance)
I see the following output:
0.0001
0.9999
1
This is odd to me because if I observe the same record in WEKA ui, I see the same instance with probabilities 0.993 and 0.007, classified as '2'. (I can load the same model multiple times from disk in WEKA, and reproduce this behavior, which is correct) After further investigation, I noticed that regardless of the sequence of attributes my Instance object has, I always get the same probability output and the same classification by invoking the model via MATLAB.
There are some posts on the net that share the same problem, like these:
Always getting the same output
Weka - Classifier returns the same distribution for any input
However, the recommended solution to call 'instance.setClassMissing()' did not solve my issue. Is there anything I am missing, or can try to do in order to further troubleshoot the issue?
Does your test instance has same structure as your train set? If not, you need yo provide the same structure.
Weka indexes nominal attributes and stores the indices internally. So the nominal attributes order in train file is important. For example if your attribute is mapped as low=>0, high=>1 in training, you need to map them like this in your test set. Usually this is achieved by serializing the train header with the model.
Sample code for creating train header:
Instances trainHeader = new Instances(instances, 0);
trainHeader.setClassIndex(instances.classIndex());
When creating a new instance set its dataset:
Instance instance = ...
instance.setDataset(trainHeader);
I am working on a inter-class and intra-class classification problem with one CNN such as first there is two classes Cat and Dog than in Cat there is a classification three different breeds of cats and in Dog there are 5 different breeds dogs.
I haven't tried the coding yet just working on feasibility if that works.
My question is what will be the feasible design for this kind of problem.
I am thinking to design for the training, first CNN-1 network that will differentiate cat and dog and gather the image data of all the training images. After the separation of cat and dog, CNN-2 and CNN-3 will train these images further for each breed of dog and cat. I am just not sure how the testing will work in this situation.
I have approached a similar problem previously in Python. Hopefully this is helpful and you can come up with an alternative implementation in Matlab if that is what you are using.
After all was said and done, I landed on a single model for all predictions. For your purpose you could have one binary output for dog vs. cat, another multi-class output for the dog breeds, and another multi-class output for the cat breeds.
Using Tensorflow, I created a mask for the irrelevant classes. For example, if the image was of a cat, then all of the dog breeds are irrelevant and they should not impact model training for that example. This required a customized TF Dataset (that converted 0's to -1 for the mask) and a customized loss function that returned 0 error when the mask was present for that example.
Finally for the training process. Specific to your question, you will have to create custom accuracy functions that can handle the mask values how you want them to, but otherwise this part of the process should be standard. It was best practice to evenly spread out the classes among the training data but they can all be trained together.
If you google "Multi-Task Training" you can find additional resources for this problem.
Here are some code snips if you are interested:
For the customize TF dataset that masked irrelevant labels...
# Replace 0's with -1 for mask when there aren't any labels
def produce_mask(features):
for filt, tensor in features.items():
if "target" in filt:
condition = tf.equal(tf.math.reduce_sum(tensor), 0)
features[filt] = tf.where(condition, tf.ones_like(tensor) * -1, tensor)
return features
def create_dataset(filepath, batch_size=10):
...
# **** This is where the mask was applied to the dataset
dataset = dataset.map(produce_mask, num_parallel_calls=cpu_count())
...
return parsed_features
Custom loss function. I was using binary-crossentropy because my problem was multi-label. You will likely want to adapt this to categorical-crossentropy.
# Custom loss function
def masked_binary_crossentropy(y_true, y_pred):
mask = backend.cast(backend.not_equal(y_true, -1), backend.floatx())
return backend.binary_crossentropy(y_true * mask, y_pred * mask)
Then for the custom accuracy metrics. I was using top-k accuracy, you may need to modify for your purposes, but this will give you the general idea. When comparing this to the loss function, instead of converting all to 0, which would over-inflate the accuracy, this function filters those values out entirely. That works because the outputs are measured individually, so each output (binary, cat breed, dog breed) would have a different accuracy measure filtered only to the relevant examples.
backend is keras backend.
def top_5_acc(y_true, y_pred, k=5):
mask = backend.cast(backend.not_equal(y_true, -1), tf.bool)
mask = tf.math.reduce_any(mask, axis=1)
masked_true = tf.boolean_mask(y_true, mask)
masked_pred = tf.boolean_mask(y_pred, mask)
return top_k_categorical_accuracy(masked_true, masked_pred, k)
Edit
No, in the scenario I described above there is only one model and it is trained with all of the data together. There are 3 outputs to the single model. The mask is a major part of this as it allows the network to only adjust weights that are relevant to the example. If the image was a cat, then the dog breed prediction does not result in loss.
My project is to forecast the wti crude oil price using ann. I already have the dataset and I divided it into 70% training data and 30% testing data. That's the only basic thing I know and I did for my project. Now I dunno what to do next since I don't have any tutorial or guidance I can refer to. Can anyone tell me what to do next?
Consider that you have TrainData, TargetTrain, TestData and TargetTest.
TrainData and TestData samples are in row and features are in column.
TargetTrain and TargetTest are two classes and are 0 or 1
InputNum=size(TrainData,2);
OutputNum=2; % two class problem
Xtr=TrainData;
Ytr=full(ind2vec(double(TargetTrain+1)));
Xts=TestData;
Yts=full(ind2vec(double(TargetTest+1)));
%% Network Structure
net = feedforwardnet(11);
%% Training
net.trainParam.showWindow=1;
net.trainParam.max_fail=7;
net = train(net,Xtr',Ytr);
For evaluation you can test:
out_train=net(Xtr');
out_test=net(Xts');
This code create ANN with 11 hidden nets.
everybody.I'd like to use caffe to train a 5 classes detection task with "SSD: Single Shot MultiBox Detector", so I changed the num_classes from 21 to 6.However,I get an following error:
"Check failed: num_priors_ * num_classes_ == bottom[1]->channels() (52392 vs. 183372) Number of priors must match number of confidence predictions."
I can understand this error,and I found 52392/6=183372/21,namely why I changed num_classes to 6,but the number of confidence predictions is still 183372. So how to solve this problem. Thank you very much!
Since SSD depends on the number of labels not only for the classification output, but also for the BB prediction, you would need to change num_output in several other places in the model.
I would strongly suggest you wouldn't do that manually, but rather use the python scripts provided in the 'examples/ssd' folder. For instance, you can change line 277 in 'examples/ssd/ssd_pascal_speed.py' to:
num_classes = 5 # instead of 21
And then use the model files this script provides.