I am using an Echo-state-net code implemented on PyTorch, and even though the prediction is accurate enough, the training fit looks shifted, can someone help to understand why this can happen? The shift function is deactivated in the code.
Example image
Related
Neural network newbie here. I've been testing Niftynet and achieved decent single-class 3D segmentation predictions on an own MRI data set with dense_vnet. However, I ran out of luck when I tried to add a second label. The network seems to spot the correct organs but can't get rid of additional artifacts as if it cannot get out of a local minimum or it doesn't have enough degrees of freedom or something. This is one of the better looking prediction slices which does show some correct labels but also additional noise.
Why would a single-class segmentation work better than a multi-class segmentation? Is it even reasonable to expect good multi-class 3D segmentation results out of DenseVnet? If yes, is there a specific approach to improve the results?
P.S.
Niftynet's site refers to stackoverflow for general questions.
Apparently, DenseVnet does handle multi-class segmentation okay. They have provided a ready model with a Dice loss extension. It worked with my MRI data without any pre-processing even though it's been designed for CT images and Hounsfield units.
I am using a UNet model to train a segmentation algorithm with roughly 1,000 grayscale medical images and 1,000 corresponding masks where the section of interest in the medical image is white pixel and the background is black.
I am using dice loss and a similar dice score as an accuracy metric to account for the fact that my white pixels are generally less in number than the black background pixels. But I am still having a few problems when training
1) The loss converges too fast. If I have my SGD optimizer's learning rate at 0.01 for example, at around 2 epochs the loss (training and validation) will drop to 0.00009 and the accuracy shoots up and settles at 100% in proportion. Testing on an unseen set gives blank images.
Assumption - Overfitting:
I assumed this was due to overfitting, so I augmented the dataset as much as possible with rigid transformations - flipping and rotating, but still no help.
Also if I test the model against the same data I used to train it, it still predicts blank images. So does this mean it isn't a case of overfitting?
2)Model doesn't look like it's even training. I was able to check the model before it reduced all the test data to blackness, but even then the results would look like blurry versions of the original without segmenting the features highlighted by my training mask
3) The loss vs epochs and accuracy vs epochs output charts are very smooth: They present none of the oscillating behaviour that I expect to see when doing semantic segmentation. According to this related post a smooth chart usually occurs when there is only one class. I however assumed that my model would see the training masks (white pixels vs black pixels) and see that as a two class problem. Am I wrong in this assumption?
4) According to this post Dice is good for an unbalanced training set. I have also tried to get precision/recall/F1 results as they suggest, but was unable to do it and assuming it might be related to my 3rd issue where the model sees my segmentation task as a single class problem.
TLDR: How can I fix the black output results I am getting? Can you please help me clarify if my learning model is actually seeing my white and black pixels in each mask as two separate classes and if not what is it actually doing?
Your model is only predicting one class (the background/back pixels) because of the class imbalance.
The loss converges too fast. If I have my SGD optimizer's learning rate at 0.01 for example, at around 2 epochs the loss (training and validation) will drop to 0.00009 and the accuracy shoots up and settles at 100% in proportion. Testing on an unseen set gives blank images.
Lower your learning rate. 0.01 is really high, so try something like 3e-5 for your learning and see how your model performs.
Also, having a 100% accuracy (supposedly you're using dice?) suggests that you're still using accuracy, so I believe that your model does not recognize that you're using dice/dice loss for training and evaluation(code snippets would be appreciated).
Example:
model.compile(optimizer=Adam(lr=TRAIN_SEG_LEARNING_RATE),
loss=dice_coef_loss,
metrics=[dice_coef])
Also if I test the model against the same data I used to train it, it still predicts blank images. So does this mean it isn't a case of overfitting?
Try using model.evaluate(test_data, test_label). If the evaluated performance is good (dice should be extremely low if you're only predicting 0s), then either your labels are messed in some way or there is something wrong with your pipeline.
Possible Solutions if all else fails:
make sure to go through all the sanity checks in this article
You might not have enough data, so try to use a patchwise approach with random crops.
Add more regularization (dropout, BatchNormalization, InstanceNormalization, increasing input image size, etc.)
I am calculating the natural frequencies and the corresponding FRF of a test sample that is excited with a hammer with an accelerometer attached to the tip as input signal and another accelerometer attached to the test sample to measure the response. The calculation of the natural frequencies works fine, however, when I am plotting the normalized FRF of 40 independent measurements, the noise makes it impossible to discern the natural frequencies as you can see below.
Then I found out about the hanning window and I applied this to my data, which gave me the following FRF:
This is an upgrade, but still way too much noise :-(
I am hoping someone can help me what to do next to remove as much noise as possible! Any help would be appreciated!
I am not sure this is a StackOverflow question, but I will still try to help you:
I would try to use autocorrelation or fft. This question could also help you:Determine frequency from signal data in MATLAB
I try to implement a facial feature detector with a CLM following the paper of Cootes et. al. ('Feature Detection and Tracking with Constrained Local Models' and 'Automatic feature localization with constrained local models').
The training of the model worked out well and I got the expected results. Now I'm stuck with the building of the response image. According to the paper I have to calculate the normalized correlation response of every template patch with the underlying image around his current position ( I used the normxcorr2 function from matlab).
I did this. But the result doesn't look like the pictures in the papers. And the Nelder-Meade optimization doesn't converge either.
Did I missed something? Has anyone implemented this algorithm yet and could anyone give me some hints? Is the normxcorr2 function a bad choice?
Thanks!
I'm working on a feed-forward backpropagation network in C++ but cannot seem to make it work properly. The network I'm basing mine on is using the cross-entropy error function. However, I'm not very familiar with it and even though I'm trying to look it up I'm still not sure. Sometimes it seems easy, sometimes difficult. The network will solve a multinomial classification problem and as far as I understand, the cross-entropy error function is suitable for these cases.
Someone that knows how it works?
Ah yes, good 'ole backpropagation. The joy of it is that it doesn't really matter (implementation wise) what error function you use, so long as it differentiable. Once you know how to calculate the cross entropy for each output unit (see the wiki article), you simply take the partial derivative of that function to find the weights for the hidden layer, and once again for the input layer.
However, if your question isn't about implementation, but rather about training difficulties, then you have your work cut out for you. Different error functions are good at different things (best to just reason it out based on the error function's definition) and this problem is compounded by other parameters like learning rates.
Hope that helps, let me know if you need any other info; your question was a lil vague...