My model is a SegNet with cross-entrophy loss. It is for liver and liver vessel medical segmentation.
And my problem is, when I use 725 cases for input, the accuracy increase normally.
The result of liver is good, but the result of vessel still need to be improve.
But when I increase my input data to 34,868 cases, the accuracy doesn't increase at the beginning, and then jump up to about 95% accuracy.
Even the accuracy is about 95%, the result of my segmentation is very poor.
The result got no liver nor liver vessel, but full of background.
The training process with input 725 cases
The training process with input 34,868 cases
I am wondering if my model is too complicated, or I should modified the loss function.
I have change my loss function to focal loss and dice+focal loss, but they doesn't solve the problem or improve the result.
Related
Accuracy of training according to ML.NET is 97%. But when I'm trying to predict the class it always returns the same value, no matter what input data is provided. And it doesn't make much sense, because it's clearly not 97%, but 0%. So I wanted to ask is it normal or maybe I need to leave it for 10 hours of training so it reaches higher than 97%.
Training data is Parkinson's Disease (PD) classification from kaggle.
I am training a DNN in MATLAB , while optimizing my network, I am observing a decrement in accuracy while increasing the epochs. Is it possible?
The loss values in on the other hand decreases during training while increasing epochs. Please guide.
tldr; absolutely.
When entire training dataset is seen once by the model (feed forwarded once), it's termed as 1 epoch.
The below graph shows the general behaviour of accuracy with the number of epochs. Training on more number of epochs can result in low accuracy on validation, even though loss will continue to reduce (training accuracy will be high). This is termed as overfitting.
No. of epochs to train also a hyperparameter that needs fine tuning.
It is absolutely possible:
Especially when you are training in batches
When your learning rate is too high
I'm learing how LSTM works by practicing with time series training data(input is a list of features and output is a scalar).
There is a problem that i couldnt understand when calculating loss for RNN/LSTM:
How loss is calculated? Is it calculated at each time i give the nn new input or acummulated through all the given inputs and then be backprop
#seed Answer is correct. However, in LSTM, or any RNN architecture, the loss for each instance, across all time steps, is added up. In other words, you'll have (L0#t0, L1#t1, ... LT#tT) for each sample in your input batch. Add those losses separately for each instance in the batch. Finally average the losses of each input instance to get the average loss for a current batch
For more information please visit: https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-recurrent-neural-networks
The answer does not depend on the neural network model.
It depends on your choice of optimization method.
If you are using batch gradient descent, the loss is averaged over the whole training set. This is often impractical for neural networks, because the training set is too big to fit into RAM, and each optimization step takes a lot of time.
In stochastic gradient descent, the loss is calculated for each new input. The problem with this method is that it is noisy.
In mini-batch gradient descent, the loss is averaged over each new minibatch - a subsample of inputs of some small fixed size. Some variation of this method is typically used in practice.
So, the answer to your question depends on the minibatch size you choose.
(Image is from here)
Just some general questions about training. I used a convolutional neural network for binary classification of text on a dataset of about 10,000 samples. The dataset was pretty unbalanced with about 80% of the samples in class 1. The very last image shows a model on a balanced dataset of about a few million samples doing a 14-way classification task. All models use nn.ClassNLLCriterion, momentum of 0.9, dropout, and weight decay of 0.00001:
Here's the code for more details
For loss, I got values over 1 for validation. How bad is that? Is a loss over one big or is it reasonable?
Is the error y-axis unit usually in percent? So here, for the error, would it range from 0% to 0.16% or is it 0% to 16%?
For the graphs below, the loss and error graphs look about the same shape. In general, should the loss and error graphs always have the same shape?
Are the error and loss usually on different scales?
I've made digit recognition (56x56 digits) using Neural Networks, but I'm getting 89.5% accuracy on test set and 100% on training set. I know that it's possible to get >95% on test set using this training set. Is there any way to improve my training so I can get better predictions? Changing iterations from 300 to 1000 gave me +0.12% accuracy. I'm also file size limited so increasing number of nodes can be impossible, but if that's the case maybe I could cut some pixels/nodes from the input layer.
To train I'm using:
input layer: 3136 nodes
hidden layer: 220 nodes
labels: 36
regularized cost function with lambda=0.1
fmincg to calculate weights (1000 iterations)
As mentioned in the comments, the easiest and most promising way is to switch to a Convolutional Neural Network. But with you current model you can:
Add more layers with less neurons each, which increases learning capacity and should increase accuracy by a bit. Problem is that you might start overfitting. Use regularization to counter this.
Use batch Normalization (BN). While you are already using regularization, BN accelerates training and also does regularization, and is a NN specific algorithm that might work better.
Make an ensemble. Train several NNs on the same dataset, but with a different initialization. This will produce slightly different classifiers and you can combine their output to get a small increase in accuracy.
Cross-entropy loss. You don't mention what loss function you are using, if its not Cross-entropy, then you should start using it. All the high accuracy classifiers use cross-entropy loss.
Switch to backpropagation and Stochastic Gradient Descent. I do not know the effect of using a different optimization algorithm, but backpropagation might outperform the optimization algorithm you are currently using, and you could combine this with other optimizers such as Adagrad or ADAM.
Other small changes that might increase accuracy are changing the activation functions (like ReLU), shuffle training samples after every epoch, and do data augmentation.