Say I have a training set with 50 vectors. I split this set into 5 sets each with 10 vectors and then I scale the vectors in each subset and normalise the subsets. Then I train my ANN with each vector from each subset.
After training is complete, I group my test set into subsets of 10 vectors each, scale the features of the vectors in each subset and normalise each subset and then feed it to the neural network to attempt to classify it.
Is this the right approach? Is it right to scale and normalise each subset, each with its own minimum, maximum, mean and standard deviation?
Related
I have a task where I need to train a machine learning model to predict a set of outputs from multiple inputs. My inputs are 1000 iterations of a set of 3x 1 vectors, a set of 3x3 covariance matrices and a set of scalars, while my output is just a set of scalars. I cannot use regression learner app because these inputs need to have the same dimensions, any idea on how to unify them?
One possible way to solve this is to flatten the covariance matrix into a vector. Once you did that, you can construct a 1000xN matrix where 1000 refers to the number of samples in your dataset and N is the number of features. For example if your features consist of a 3x1 vector, a 3x3 covariance matrix and lets say 5 other scalars, N could be 3+3*3+5=17. You then use this matrix to train an arbitrary model such as a linear regressor or more advanced models like a tree or the like.
When training machine learning models it is important to understand your data and exploit its structure to help the learning algorithms. For example we could use the fact that a covariance matrix is symmetric and positive semi-definite and thus lives in a closed convex cone. Symmetry of the matrix implies that it lives in a subspace of the set of all 3x3 matrices. In fact the dimension of the space of 3x3 symmetric matrices is only 6. You can use that knowledge to reduce redundancy in your data.
I'm working on a neural network with Keras using TensorFlow as the backend right now, and my model takes 5 inputs, all normalized to 0 to 1. The inputs' units vary from m/s to meters to m/s/s. So, for example, one input could vary from 0 m/s to 30 m/s, while another input could vary from 5 m to 200 m in the training dataset.
Is it better to individually and independently normalize all inputs so that I have different scales for each unit/input? Or would normalizing all inputs to one scale (mapping 0-200 to 0-1 for the example above) be better for accuracy?
Normalize individualy each input. Because if you normalize everything by dividing 200 some inputs will affect your network less than others. If one input vary between 0-30, after dividing by 200 you get 0-0.15 scale and scale for input which vary 0-200 will be 0-1 after division. So 0-30 input will have less numbers and you tell your network that input is not so relevant as one whith 0-200.
I calculate the fft for a given soundfile and get an array of the shape e.g. (100,257) with 100 rows and 257 frequency bins. I want to use this as an input vector for a neural network but before I want to normalize with librosa lib
https://librosa.github.io/librosa/generated/librosa.util.normalize.html#librosa.util.normalize
so should I normalize over axis=0 or axis=1? axis=0 normalizes the columns aggregated over the rows and axis=1 normalizes every row or should I normalize over every value independent of rows and columns?
The way how you normalize the fft depends on your application and the final performance. There isn't a general normalization scheme.
In one of my application, I didn't normalize and input the raw fft to the neural network. One common way to normalize is taking the logarithm. This operation can reduce the dynamic range.
My aim is to classify types of cars (Sedans,SUV,Hatchbacks) and earlier I was using corner features for classification but it didn't work out very well so now I am trying Gabor features.
code from here
Now the features are extracted and suppose when I give an image as input then for 5 scales and 8 orientations I get 2 [1x40] matrices.
1. 40 columns of squared Energy.
2. 40 colums of mean Amplitude.
Problem is I want to use these two matrices for classification and I have about 230 images of 3 classes (SUV,sedan,hatchback).
I do not know how to create a [N x 230] matrix which can be taken as vInputs by the neural netowrk in matlab.(where N be the total features of one image).
My question:
How to create a one dimensional image vector from the 2 [1x40] matrices for one image.(should I append the mean Amplitude to square energy matrix to get a [1x80] matrix or something else?)
Should I be using these gabor features for my purpose of classification in first place? if not then what?
Thanks in advance
In general, there is nothing to think about - simple neural network requires one dimensional feature vector and does not care about the ordering, so you can simply concatenate any number of feature vectors into one (and even do it in random order - it does not matter). In particular if you have same feature matrices you also concatenate each of its row to create a vectorized format.
The only exception is when your data actually has some underlying geometrical dependicies, for example - matrix is actualy a pixels matrix. In such case architectures like PyraNet, Convolutional Neural Networks and others, which apply some kind of receptive fields based on this 2d structure - should be better. But those implementations simply accept 2d feature vector as an input.
I want to define a function approximation by RBF neural network in MATLAB.
RBF needs there parameters as "unit centers", "sigma" and "weight". I have a dataset by 1000 records and 10 features.
first question: these three parameters should be in an array format? or can be in matrix format?
second question: I defined "unit centers" by k-means clustering over dataset. This is three cluster centers.
For "sigma" and "weight" parameters, i should define a matrix same as the "unit centers" size?
unit centers are matrix by 3*10 size. Other two RBF parameters should assign in 3in10 size? Or can i define them in 1in10 or 2in10 size?
Centers are of course in the form of a matrix, you have 10 features, you are calculating the centers by distance based on these 10 dimension. and you have more than one centers so it is a matrix of shape: (#centers,#features).
Sigma is just a single number for every center so it is in the shape of: (#centers,1), hence it is a 1D-array
Weights depend on the size of hidden layer(Centers) and with one output neuron it is of the shape: (#centers,1), it is a 1D-array
one last thing to mention here is that, number of your centers is small compare to your input size which is 1000. try 100, 200 or even 500 centers if you did not get good accuracy on test set.