I want to define a function approximation by RBF neural network in MATLAB.
RBF needs there parameters as "unit centers", "sigma" and "weight". I have a dataset by 1000 records and 10 features.
first question: these three parameters should be in an array format? or can be in matrix format?
second question: I defined "unit centers" by k-means clustering over dataset. This is three cluster centers.
For "sigma" and "weight" parameters, i should define a matrix same as the "unit centers" size?
unit centers are matrix by 3*10 size. Other two RBF parameters should assign in 3in10 size? Or can i define them in 1in10 or 2in10 size?
Centers are of course in the form of a matrix, you have 10 features, you are calculating the centers by distance based on these 10 dimension. and you have more than one centers so it is a matrix of shape: (#centers,#features).
Sigma is just a single number for every center so it is in the shape of: (#centers,1), hence it is a 1D-array
Weights depend on the size of hidden layer(Centers) and with one output neuron it is of the shape: (#centers,1), it is a 1D-array
one last thing to mention here is that, number of your centers is small compare to your input size which is 1000. try 100, 200 or even 500 centers if you did not get good accuracy on test set.
Related
I'm working on a neural network with Keras using TensorFlow as the backend right now, and my model takes 5 inputs, all normalized to 0 to 1. The inputs' units vary from m/s to meters to m/s/s. So, for example, one input could vary from 0 m/s to 30 m/s, while another input could vary from 5 m to 200 m in the training dataset.
Is it better to individually and independently normalize all inputs so that I have different scales for each unit/input? Or would normalizing all inputs to one scale (mapping 0-200 to 0-1 for the example above) be better for accuracy?
Normalize individualy each input. Because if you normalize everything by dividing 200 some inputs will affect your network less than others. If one input vary between 0-30, after dividing by 200 you get 0-0.15 scale and scale for input which vary 0-200 will be 0-1 after division. So 0-30 input will have less numbers and you tell your network that input is not so relevant as one whith 0-200.
Say I have a training set with 50 vectors. I split this set into 5 sets each with 10 vectors and then I scale the vectors in each subset and normalise the subsets. Then I train my ANN with each vector from each subset.
After training is complete, I group my test set into subsets of 10 vectors each, scale the features of the vectors in each subset and normalise each subset and then feed it to the neural network to attempt to classify it.
Is this the right approach? Is it right to scale and normalise each subset, each with its own minimum, maximum, mean and standard deviation?
I am trying to implement a classification NN in Matlab.
My inputs are clusters of coordinates from an image. (Corresponding to delaunay triangulation vertexes)
There are 3 clusters (results of the optics algorithm) in this format:
( Not all clusters are of the same size.). Elements represent coordinates in euclidean 2d space . So (110,12) is a point in my image and the matrix depicted represents one cluster of points.
Clustering was done on image edges. So coordinates refer to logical values (always 1s in this case) on the image matrix.(After edge detection there are 3 "dense" areas in an image, and these collections of pixels are used for classification). There are 6 target classes.
So, my question is how can I format them into single column vector inputs to use in a neural network?
(There is a relevant answer here but I would like some elaboration if possible. ( I am probably too tired right now from 12 hours of trying stuff and dont get it 100% :D :( )
Remember, there are 3 different coordinate matrices for each picture, so my initial thought was, create an nn with 3 inputs (of different length). But how to serialize this?
Here's a cluster with its tags on in case it helps:
For you to train the classifier, you need a matrix X where each row will correspond to an image. If you want to use a coordinate representation, this means all images will have to be of the same size, say, M by N. So, the row of an image will have M times N elements (features) and the corresponding feature values will be the cluster assignments. Class vector y will be whatever labels you have, that is one of the six different classes you mentioned through the comments above. You should keep in mind that if you use a coordinate representation, X can get very high-dimensional, and unless you have a large number of images, chances are your classifier will perform very poorly. If you have few images, consider using fractions of pixels belonging to clusters that I suggested in one of the comments: this can give you a shorter feature description that is invariant to rotation and translation, and may yield better classification.
Suppose I have a bunch of instances and I want to find the closest K instances to a particular instance. Moreover, I have some weights showing the strengths of each dimension as we computing the distances. How can I incorporate these weights with the KNN finding process in MATLAB?
There are two methods that can allow you to do this. Looking at the knnsearch documentation, you can either use the seuclidean flag where this performs the standardized Euclidean distance. Each co-ordinate difference between two points is scaled by dividing by a corresponding scale value in S. S by default is the standard deviation for each co-ordinate. You can manually specify each of these scales by specifying the Scale parameter, then specifying a vector where each component will scale each dimension for you instead of the standard deviation in each dimension.
As such, the more contribution a co-ordinate has, the larger the scale should be, as you want to aggregate co-ordinates and will allow distances that are larger to have a smaller Euclidean distance. This is essentially the same thing as weighting the strengths in each dimension.
Alternatively, you can provide your own function that computes the distance between two vectors. You can define what these weights are in your workspace before hand, then create an anonymous function wrapper that accesses these weights when computing whatever distance measure you want yourself. The anonymous function can only take in two vectors, corresponding to two different co-ordinate vectors in KNN. As such, use this anonymous function to access the weights that should be already defined in the workspace then go from there.
Check out: http://www.mathworks.com/help/stats/knnsearch.html
My aim is to classify types of cars (Sedans,SUV,Hatchbacks) and earlier I was using corner features for classification but it didn't work out very well so now I am trying Gabor features.
code from here
Now the features are extracted and suppose when I give an image as input then for 5 scales and 8 orientations I get 2 [1x40] matrices.
1. 40 columns of squared Energy.
2. 40 colums of mean Amplitude.
Problem is I want to use these two matrices for classification and I have about 230 images of 3 classes (SUV,sedan,hatchback).
I do not know how to create a [N x 230] matrix which can be taken as vInputs by the neural netowrk in matlab.(where N be the total features of one image).
My question:
How to create a one dimensional image vector from the 2 [1x40] matrices for one image.(should I append the mean Amplitude to square energy matrix to get a [1x80] matrix or something else?)
Should I be using these gabor features for my purpose of classification in first place? if not then what?
Thanks in advance
In general, there is nothing to think about - simple neural network requires one dimensional feature vector and does not care about the ordering, so you can simply concatenate any number of feature vectors into one (and even do it in random order - it does not matter). In particular if you have same feature matrices you also concatenate each of its row to create a vectorized format.
The only exception is when your data actually has some underlying geometrical dependicies, for example - matrix is actualy a pixels matrix. In such case architectures like PyraNet, Convolutional Neural Networks and others, which apply some kind of receptive fields based on this 2d structure - should be better. But those implementations simply accept 2d feature vector as an input.