I’m trying to use a Neural Network for purposes of binary classification. It consist of three layers. The first layer has three input neurons, the hidden layer has two neurons, and the output layer has three neurons that output a binary value of 1 or 0. Actually the output is usually a floating point number, but it typically rounds up to a whole number.
If the network only outputs vectors of 3, then shouldn't my input vectors be the same size? Otherwise, for classification, how else do you map the output to the input?
I wrote the neural network in Excel using VBA based on the following article: https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/
So far it works exactly as described in the article. I don’t have access to a machine learning library at the moment so I’ve chosen to give this a try.
For example:
If the output of the network is [n, n ,n], does that mean that my input data has to be [n, n, n] also?
From what I read in here: Neural net input/output
It seems that's the way it should be. I'm not entirely sure though.
To speak simple,
for regression task, your output usually has the dimension [1] (if you predict single value).
For the classification task, your output should have the same number of dimensions equal to the number of classes you have (outputs are probabilities, the sum of them = 1).
So, there is no need to have equal dimensions of input and output. NN is just a projection of one dimension to another.
For example,
regression, we predict house prices: input is [1, 10] (to features of the property), the output is [1] - price
classification, we predict class (will be sold or not): input is [1, 11] (same features + listed price), output is [1, 2] (probability of class 0 (will be not sold) and 1 (will be sold); for example, [1; 0], [0; 1] or [0.5; 0.5] and so on; it is binary classification)
Additionally, equality of input-output dimensions exists in more specific tasks, for example, autoencoder models (when you need to present your data in other dimension and then represent it back, to the original dimension).
Again, the output dimension is the size of outputs for 1 batch. Only one, not of the whole dataset.
Related
Each element of my dataset has a multi-label tensor like [1, 0, 0, 1] with varying combinations of 1's and 0's. In this scenario, since I have 4 tensors, I have the output layer of my neural network to be 4. In doing so with BCEWithLogitsLoss, I obtain an output tensor like [3, 2, 0, 0] when I call model(inputs) which is in the range of (0, 3) as I specified with the output layer to have 4 output neurons. This does not match the format of what the target is expected to be, although when I change the number of output neurons to 2, I get a shape mismatch error. What needs to be done to fix this?
When using BCEWithLogitsLoss you make a 1D prediction per output binary label.
In your example, you have 4 binary labels to predict, and therefore, your model outputs 4d vector, each entry represents the prediction of one of the binary labels.
Using BCEWithLogitsLoss you implicitly apply Sigmoid to your outputs:
This loss combines a Sigmoid layer and the BCELoss in one single class.
Therefore, if you want to get the predicted probabilities of your model, you need to add a torch.sigmoid on top of your prediction. The sigmoid function will convert your predicted logits to probabilities.
i want to create a Neural Network with "three (2D) Matrices" as a inputs , and
the output is a 1 (2D) Matrix , so the three inputs is :
1-2D Matrix Contains ( X ,Y ) Coordinates From a device
2-2D Matrix Contains ( X ,Y ) Coordinates From another different Device
3-2D Matrix Contains the True exact( X , Y ) Coordinates that i already
measured it ( i don't know if that exact include from the inputs or What??)
***Note that each input have his own Error and i want to make the Neural Network
to Minimize that error and choose the best result depends on the true exact (X,Y)***
**Notice that : im Working on object tracking that i extracting (x,y) Coordinate
from the camera and the other device is same the data so for example
i will simulate the Coordinates as Follows:
{ (1,2), (1,3), (1,4), (1,5) , (1,6).......}
and so on
For Sure the Output is one 2D Matrix the best or the True Exact (x,y) ,So im a
beginner and i want to understand how to create this Network with this Different
inputs and choose the best training method to have the Best Results ...?!
thanks in Advance
It sounds like what you want is a HxWx2 input where the first channel (depthwise layer) is your 1st input and the 2nd channel is your 2nd input. Your "true exact" coordinates would be the target that your net output is compared to, rather than being an input.
Note that neural nets don't really handle regression (real-valued outputs) very well - you may get better results dividing your coordinate range into buckets and then treating it as a classification problem instead (use softmax loss vs regression mean-squared error loss).
Expanding on the regression vs classification point:
A regression problem is one where you want the net to output a real value such as a coordinate value in range 0-100. A classification problem is one where you want the net to output a set of propabilities that your input belongs to a given class it was trained on (e.g. you train a net on images belonging to classes "cat" "dog" and "rabbit").
It turns out that modern neural nets are much better at classification than regression, because the way they work is basically by subdividing the N-dimensional input spaces into sub-regions corresponding to the outputs they are being trained to make. What they are naturally doing is classifying.
The obvious way to turn a regression problem into a classification problem, which may work better, is to divide your desired output range into sub-ranges (aka buckets) which you treat as classes. e.g. instead of training your net to output a single (or multiple) value in range 0-100, instead train it to output class probabilities representing, for example, each of 10 separate sub-ranges (classes) 0-9, 10-19, 20-20, etc.
I have tried to train a neural network in matlab,first of all I have build the ANN as follow
net = feedforwardnet([30 20 20 ]);
[net ,tr] = train(net , XTRAIN , temp);
which produce an ANN with the following architecture:
then I test my neural network as follow
outputsOfTest = sim(net , XTEST);
the outputsOfTest is a vector represent the output of neural network testing, usually some the elements ofoutputsOfTest are negative values , for example the outputsOfTest will be something like this [-.34 1.17 .17].
So How to interpret this output? what are negative values indicate to? which class the testing data will belong based on this output?
Should I take the greatest value as an indicator to the class that testing data will belong to?
for example if I have the output vector [-2 .5 1] , which is the greatest value is 1, So the class that testing data belong to is class 3
Should I take the greatest value in magnitude (taking the absolute value) ? for example if I have the output vector [-2 .5 1] , which is the greatest value in it's magnitude is the first element, So the class that testing data belong to is class 1.
Note: sometimes the sum of the elements ofoutputsOfTest exceed one, the sum of the elements may reach 2.5, does this normal?
Your output layer seems to have a linear activation function. Therefor your output vectors components have values that are not restricted to be between 0 and 1. For classification you should use a softmax activation function:
(Source)
The use of softmax results in vector components which have values between 0 and 1 and which sum to 1 for each vector. So basically you get a probability distribution over your classes. The Matlab help has an image showing the effects (left input, right after softmax):
There's more information about it in the UFDL Tutorial.
From what I could find, the following code change might work in Matlab:
net = feedforwardnet([30 20 20]);
net.layers{4}.transferFcn='softmax';
[net ,tr] = train(net , XTRAIN , temp);
In my project I have two vectors of [200x1] that should be used in Neural Network and trained in a way that it gives me the subject's fingerprint gender.
I think I should provide a target vector based on the subject's gender from the ground truth data (like 1 for female and 0 for male) But I am not sure if this consideration is correct.
Any idea for this?
it sounds right. actually you are training a classification to classify two groups male and female.
Your input is two vectors each has size 200x1 and the output should have 2 nodes. y=1[male], y=0[female].
So, just merge two vectors into one vector and put it as input. You just need to know for each element in the input vector the corresponding output value (i.e. either 1 or 0). So using indexing you can shuffle the two vectors into one vector and give it to the NN as the input vector (e.g. input size = 400x1) and get two vectors as output.
So you can also use the same input output set to try with the SVM algorithm alongside to test the result. It is provided in MATLAB and the usage is pretty easy.
example:
v1 = rand(1,200); % I am setting these values as random, but you should use the values you know
v2 = rand(1,200);
input = [v1 v2]';
y = randi(2,400,1)-1; % make a random vector including 0 and 1 for female and male (you should use your data)
net = newff(...) % or whatever NN you are using
net = train(net,input,y) % train the NN using the data
I have a data set with 5 inputs and one output. I want to train this data set with Neural Network modeling in MATLAB. When i input data without normalization, the MSE is very large around 1e+3. But, when i normalize the input data, the output error becomes around 1e-4. So, as i understand, normalization is an important task.
My 2 questions:
1- my real target (output) before training process is in range of [0 1000] or [50 800], but after normalization, the neural network gives me a value in range of [0 1] due to normalization. I mean, i cannot get any value in my real range [0 1000] or [50 800]. How can i convert the output from neural network to its correct target in range of [0 1000] or [50 800]? Is it logic to do it? When my real (target) output should be [0 1000] or [50 800], what can i do with the values in range of [0 1]?
2- I want to test the trained NN with one new input pattern. When i normalized the input data in training phase, this new input pattern should be normalized. yes? How? I mean, my input data in training phase, was around 1000 data and i normalized them with (x-min)/(max-min). Should i normalized my one new input pattern in this min and max range?
Well, assuming the normalization is linear (it probably is), you can take the outputs, multiply them by 1000 and round that.
You can also play around with different transfer functions. Sigmoid has many desirable properties, but it is not capable of producing anything larger than 1 (which is sort of necessary if your outputs run up to 1000). I think that often the last layer has linear transfer function - but since your outputs are this large, it may not be sufficient in this particular case.