i want to create a Neural Network with "three (2D) Matrices" as a inputs , and
the output is a 1 (2D) Matrix , so the three inputs is :
1-2D Matrix Contains ( X ,Y ) Coordinates From a device
2-2D Matrix Contains ( X ,Y ) Coordinates From another different Device
3-2D Matrix Contains the True exact( X , Y ) Coordinates that i already
measured it ( i don't know if that exact include from the inputs or What??)
***Note that each input have his own Error and i want to make the Neural Network
to Minimize that error and choose the best result depends on the true exact (X,Y)***
**Notice that : im Working on object tracking that i extracting (x,y) Coordinate
from the camera and the other device is same the data so for example
i will simulate the Coordinates as Follows:
{ (1,2), (1,3), (1,4), (1,5) , (1,6).......}
and so on
For Sure the Output is one 2D Matrix the best or the True Exact (x,y) ,So im a
beginner and i want to understand how to create this Network with this Different
inputs and choose the best training method to have the Best Results ...?!
thanks in Advance
It sounds like what you want is a HxWx2 input where the first channel (depthwise layer) is your 1st input and the 2nd channel is your 2nd input. Your "true exact" coordinates would be the target that your net output is compared to, rather than being an input.
Note that neural nets don't really handle regression (real-valued outputs) very well - you may get better results dividing your coordinate range into buckets and then treating it as a classification problem instead (use softmax loss vs regression mean-squared error loss).
Expanding on the regression vs classification point:
A regression problem is one where you want the net to output a real value such as a coordinate value in range 0-100. A classification problem is one where you want the net to output a set of propabilities that your input belongs to a given class it was trained on (e.g. you train a net on images belonging to classes "cat" "dog" and "rabbit").
It turns out that modern neural nets are much better at classification than regression, because the way they work is basically by subdividing the N-dimensional input spaces into sub-regions corresponding to the outputs they are being trained to make. What they are naturally doing is classifying.
The obvious way to turn a regression problem into a classification problem, which may work better, is to divide your desired output range into sub-ranges (aka buckets) which you treat as classes. e.g. instead of training your net to output a single (or multiple) value in range 0-100, instead train it to output class probabilities representing, for example, each of 10 separate sub-ranges (classes) 0-9, 10-19, 20-20, etc.
Related
I’m trying to use a Neural Network for purposes of binary classification. It consist of three layers. The first layer has three input neurons, the hidden layer has two neurons, and the output layer has three neurons that output a binary value of 1 or 0. Actually the output is usually a floating point number, but it typically rounds up to a whole number.
If the network only outputs vectors of 3, then shouldn't my input vectors be the same size? Otherwise, for classification, how else do you map the output to the input?
I wrote the neural network in Excel using VBA based on the following article: https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/
So far it works exactly as described in the article. I don’t have access to a machine learning library at the moment so I’ve chosen to give this a try.
For example:
If the output of the network is [n, n ,n], does that mean that my input data has to be [n, n, n] also?
From what I read in here: Neural net input/output
It seems that's the way it should be. I'm not entirely sure though.
To speak simple,
for regression task, your output usually has the dimension [1] (if you predict single value).
For the classification task, your output should have the same number of dimensions equal to the number of classes you have (outputs are probabilities, the sum of them = 1).
So, there is no need to have equal dimensions of input and output. NN is just a projection of one dimension to another.
For example,
regression, we predict house prices: input is [1, 10] (to features of the property), the output is [1] - price
classification, we predict class (will be sold or not): input is [1, 11] (same features + listed price), output is [1, 2] (probability of class 0 (will be not sold) and 1 (will be sold); for example, [1; 0], [0; 1] or [0.5; 0.5] and so on; it is binary classification)
Additionally, equality of input-output dimensions exists in more specific tasks, for example, autoencoder models (when you need to present your data in other dimension and then represent it back, to the original dimension).
Again, the output dimension is the size of outputs for 1 batch. Only one, not of the whole dataset.
The theme here is the use of neural network in learning time histories.
Lets consider for clarity Y = f(X) where X is a vector 1xN and Y 1xN.
In most of the models I can find or test online, X is directly the time vectorised with regular timesteps (X=T).
The prediction task performed on the time history is done therefore using the output Y, and using a sequence of this let say Y(i:i+Nsample) as an Neural Network input and then the output is Y(i+Nsample+1). Then the prediction is performed moving the window one step at the time ( i= i+1).
Now my question is the following. In the case where we have a vector X which is a generic function whose values are known, the problem to model with neural network is:
knowing X(i:i+Nsample+1) and Y(i:i+Nsample) we want to predict Y(i:i+Nsample+1)
then we can do i=i+1 and proceed forward.
What are the best solutions to design such a system, is there an example with Keras or other project from which I could be inspired?
I see several solutions but without being convinced.
a)Set a multidimensional vector (2xNsample) as input [X(i:i+Nsample ) ; Y(i-1:i+Nsample-1)] and predict Y(:i+Nsample) (treat the output as a second input)
b) set two separate lstm for X and Y and then concatenate them in some way
I have tried to train a neural network in matlab,first of all I have build the ANN as follow
net = feedforwardnet([30 20 20 ]);
[net ,tr] = train(net , XTRAIN , temp);
which produce an ANN with the following architecture:
then I test my neural network as follow
outputsOfTest = sim(net , XTEST);
the outputsOfTest is a vector represent the output of neural network testing, usually some the elements ofoutputsOfTest are negative values , for example the outputsOfTest will be something like this [-.34 1.17 .17].
So How to interpret this output? what are negative values indicate to? which class the testing data will belong based on this output?
Should I take the greatest value as an indicator to the class that testing data will belong to?
for example if I have the output vector [-2 .5 1] , which is the greatest value is 1, So the class that testing data belong to is class 3
Should I take the greatest value in magnitude (taking the absolute value) ? for example if I have the output vector [-2 .5 1] , which is the greatest value in it's magnitude is the first element, So the class that testing data belong to is class 1.
Note: sometimes the sum of the elements ofoutputsOfTest exceed one, the sum of the elements may reach 2.5, does this normal?
Your output layer seems to have a linear activation function. Therefor your output vectors components have values that are not restricted to be between 0 and 1. For classification you should use a softmax activation function:
(Source)
The use of softmax results in vector components which have values between 0 and 1 and which sum to 1 for each vector. So basically you get a probability distribution over your classes. The Matlab help has an image showing the effects (left input, right after softmax):
There's more information about it in the UFDL Tutorial.
From what I could find, the following code change might work in Matlab:
net = feedforwardnet([30 20 20]);
net.layers{4}.transferFcn='softmax';
[net ,tr] = train(net , XTRAIN , temp);
In my project I have two vectors of [200x1] that should be used in Neural Network and trained in a way that it gives me the subject's fingerprint gender.
I think I should provide a target vector based on the subject's gender from the ground truth data (like 1 for female and 0 for male) But I am not sure if this consideration is correct.
Any idea for this?
it sounds right. actually you are training a classification to classify two groups male and female.
Your input is two vectors each has size 200x1 and the output should have 2 nodes. y=1[male], y=0[female].
So, just merge two vectors into one vector and put it as input. You just need to know for each element in the input vector the corresponding output value (i.e. either 1 or 0). So using indexing you can shuffle the two vectors into one vector and give it to the NN as the input vector (e.g. input size = 400x1) and get two vectors as output.
So you can also use the same input output set to try with the SVM algorithm alongside to test the result. It is provided in MATLAB and the usage is pretty easy.
example:
v1 = rand(1,200); % I am setting these values as random, but you should use the values you know
v2 = rand(1,200);
input = [v1 v2]';
y = randi(2,400,1)-1; % make a random vector including 0 and 1 for female and male (you should use your data)
net = newff(...) % or whatever NN you are using
net = train(net,input,y) % train the NN using the data
My aim is to classify types of cars (Sedans,SUV,Hatchbacks) and earlier I was using corner features for classification but it didn't work out very well so now I am trying Gabor features.
code from here
Now the features are extracted and suppose when I give an image as input then for 5 scales and 8 orientations I get 2 [1x40] matrices.
1. 40 columns of squared Energy.
2. 40 colums of mean Amplitude.
Problem is I want to use these two matrices for classification and I have about 230 images of 3 classes (SUV,sedan,hatchback).
I do not know how to create a [N x 230] matrix which can be taken as vInputs by the neural netowrk in matlab.(where N be the total features of one image).
My question:
How to create a one dimensional image vector from the 2 [1x40] matrices for one image.(should I append the mean Amplitude to square energy matrix to get a [1x80] matrix or something else?)
Should I be using these gabor features for my purpose of classification in first place? if not then what?
Thanks in advance
In general, there is nothing to think about - simple neural network requires one dimensional feature vector and does not care about the ordering, so you can simply concatenate any number of feature vectors into one (and even do it in random order - it does not matter). In particular if you have same feature matrices you also concatenate each of its row to create a vectorized format.
The only exception is when your data actually has some underlying geometrical dependicies, for example - matrix is actualy a pixels matrix. In such case architectures like PyraNet, Convolutional Neural Networks and others, which apply some kind of receptive fields based on this 2d structure - should be better. But those implementations simply accept 2d feature vector as an input.