Given a waveform that varies in shape over time (see attached) what Neural Network architecture would be best suited to give results with limited training data (less than 100 samples)?
Below is an example representation of the data we would be looking to classify. We have evaluated a 3D CNN model and are currently looking at a LSTM model but both require a large amount of training data to get reasonable results.
Related
I have time series data(one instance of 30 seconds) as shown in the figure, I would like to know the what kind of classifying algorithms I could use.
This is how the data looks in time and frequency domain
In the image we have 2 classes(one represented in blue and the other in orange).On the left section of the image we have data represented in the time-domain and on the right its equivalent Fourier-Transform.
I am thinking of using LSTM to train the data for both domains and also converting the above representations into an image and use CNNs to train.
Any Suggestion such as a better algorithm or a better representation of data would help.
One architecture suited to your needs is WaveNet.
The WaveNet architecture is constructed to deal with very long sequences (your sequences are reasonably long) and has been shown to outperform LSTM based RNNs on several tasks in the original paper.
I am not sure what you mean by
converting the above representations into an image and use CNNs to train
so I would suggest sticking to recurrent models or WaveNet for sequence classification.
im new to neural netwworks, and through the readings i often come across where heat maps are used in the network along with the ground truth provided with the dataset to (as far as my understanding is) evaluate the accuracy of network performance.
to be specific, consider the application of a crowd density estimation network, the dataset provide the crowd images, each image hase a corresponding ground truth .mat file, this file has:
a matrix of X and Y coordinations representing the appearanse of human head in the image.
the total number of human heads in the image (crowd count) which is equal to the matrix rows.
my current understanding is that one image will get through the network and the result is a going to be compared with the given ground-truth (either the head locations, ore the crowd count),
SO, how and at which point and is the crowd density map or heat map is used? do we generate one for the image while training an compare it with the one generated from the ground truth? how is this done?
all the papers i've read neglect describing this process.
any clear explanation will be appreciated.
The counting-by-density CNN approache, which you describe are fully convolutional regression network where the output is a density (heat map). That mean the training ground-truth data is a density map too. In general a MSE loss is used for training. The ground-truth density maps are in general precomputed from the head detection ground-truth.
E.g. if you look at the MCNN code preparation folder you will find get_density_map_gaussian.m file which performs the density estimation from the ground-truth annotated head.
To the best of my knowledge the normalization mainly facilitates the training(optimization) phase. If I train a network with normalized data sets but later feed it unnormalized test data, what will be the theoretical consequence? e.g. Will the test accuracy be bad?
I've done an experiment of this on cifar-10 and found that the network still gives good results on those unnormalized test data.
When using an AlexNet neural network, be it with caffe or CNTK, it needs a mean file as input. What is this mean file for ? How does it affect the training ? How is it generated, only from training sample ?
Mean subtraction removes the DC component from images. It has the geometric interpretation of centering the cloud of data around the origin along every dimension. It reduces the correlation between images which improves training. From my experience I can say that it improves the training accuracy significantly. It is computed from the training data. Computing mean from the testing data makes no sense.
I would like to ask for ideas what options there is for training a MATLAB ANN (artificial neural network) continuously, i.e. not having a pre-prepared training set? The idea is to have an "online" data stream thus, when first creating the network it's completely untrained but as samples flow in the ANN is trained and converges.
The ANN will be used to classify a set of values and the implementation would visualize how the training of the ANN gets improved as samples flows through the system. I.e. each sample is used for training and then also evaluated by the ANN and the response is visualized.
The effect that I expect is that for the very first samples the response of the ANN will be more or less random but as the training progress the accuracy improves.
Any ideas are most welcome.
Regards, Ola
In MATLAB you can use the adapt function instead of train. You can do this incrementally (change weights every time you get a new piece of information) or you can do it every N-samples, batch-style.
This document gives an in-depth run-down on the different styles of training from the perspective of a time-series problem.
I'd really think about what you're trying to do here, because adaptive learning strategies can be difficult. I found that they like to flail all over compared to their batch counterparts. This was especially true in my case where I work with very noisy signals.
Are you sure that you need adaptive learning? You can't periodically re-train your NN? Or build one that generalizes well enough?