I am using four parameters temperature, rainfall, humidity and date for the prediction. I am trying to predict a single parameter temperature. I am trying to use back propagation algorithm for training. What might be the best network structure for this purpose?
You could start by setting up a Multilayer Perceptron with 4 input nodes, a single hidden layer(with multiple nodes) and one output node.
Train your network by feeding your testset (for example as .cvs) so that the first input node receives the temperature value, second the rainfall value and so on.
Note that can't use a date as input! Try to convert your date into a numeric value by for example just using the month of the year {1,..,12}, the week {1,..,52} or the day {1,..,365}.
I would also try to normalize my input value to the range of your activation function. So if you use the logistic function, normalize your data to the range [0,1] and for Tanh [-1,1] and so on.
Your output value will be in the same range so you have to denormalize it afterwards. It's important that you choose a bijective function for the normalization process.
Related
I'm using MATLAB to predict a trend with a machine learning approach.
My data file is an .xlsx file containing a timeline in one column (various sampling timestamps, i.e. numbers that represents seconds), and in the other columns I have some integers representing my trend.
My .xlsx file is pretty much like this:
0,0100 | 0
0,0110 | 1
0,0135 | 5
And so on.
I used "|" to distinguish between columns. The sampling time is not regular.
Given 10 values of the trend taken from 10 consecutive timestamps, I'd like to predict the 11th value at a given timestamp. For example, if the 9th value is at 34,010 and the 10th value is at 34,568s I'd like to know the value at 37,431s.
How can I do it?
I've found this link: Time Series Forecasting Using Deep Learning, but there the sampling time is regular.
Should I interpolate my trend values and re-sample them with a constant sampling time?
I would distinguish the forecasting problem from the data sampling time problem. You are dealing substantially with missing data.
Forecasting problem: You may use any machine learning technique just ignoring missing data. If you are not familiar with machine learning, I would suggest you to use LASSO (least absolute shrinkage and selection operator), which has been demonstrated to have predicting power (see "Sparse Signals in the Cross-Section of Returns" by ALEX CHINCO, ADAM D. CLARK-JOSEPH, and MAO YE).
Missing imputation problem: In the first place you should consider the reason why you have missing data. Sometime it makes no sense to impute values because the information that the value is missing is itself important and should not be overridden. Otherwise you have multiple options, other than linear interpolation, to estimate the missing values. For example check the MATLAB function fillmissing.
If I use the trained RNN(or LSTM) to generate series data(name the network with generated-RNN), then I use the series data to train the RNN(i.e. the same structure with the generated-RNN) from scratch, is it possible to get the same trained network(the same trained weights) with the generated-RNN?
You take series with X as input and Y as output to train a model, then with that model you generate series O.
Now you want to recreate the Ws from O=sigmoid(XWL1+b)...*WLN+bn with O as input and X as output.
Is it possible?
Unless X=O, then most likely not. I can't give a formal mathematical proof but multiplying forward through a network isn't equal to multiplying backwards, mainly due to the activation function. If you removed the activation function or took the inverse of the activation function you would more likely approach the Ws you desired, though another set of weights might also get you the the same output for a given input.
Also, this question would be better received in stats.stackexchange than stackoverflow.
I am trying to predict Solar Energy value at a particular date.For this purpose I am using the Artificial Neural Networks model.I am having problem in deciding the correct activation function. Since sigmoid function gives me output 0-1, I want to have and output like 256.33. So I thought to apply sigmoid for hidden layer and ReLu for output layer to keep non-linearity in networks.Can you suggest me what is the way to do this? Is my approach correct?
About my Architecture-I am using 3 layers, from which one is hidden-layer.(1) I tried to apply sigmoid for both the layers as activation function.(2)Then I applied ReLU activation for both the function. These two methods were failure. Now I am trying to apply ReLU on output layer and Sigmoid for hidden layer.
One solution would be to choose some value for the maximum possible solar energy that can be generated in one day. Such as the maximum solar energy ever generated in one day or maximum solar energy possible in the best case scenario. Then use that value to scale the output of the Sigmoid function.
f(x) = Sigmoid(x) * MAX_ENERGY
I used ntstool to create NAR (nonlinear Autoregressive) net object, by training on a 1x1247 input vector. (daily stock price for 6 years)
I have finished all the steps and saved the resulting net object to workspace.
Now I am clueless on how to use this object to predict the y(t) for example t = 2000, (I trained the model for t = 1:1247)
In some other threads, people recommended to use sim(net, t) function - however this will give me the same result for any value of t. (same with net(t) function)
I am not familiar with the specific neural net commands, but I think you are approaching this problem in the wrong way. Typically you want to model the evolution in time. You do this by specifying a certain window, say 3 months.
What you are training now is a single input vector, which has no information about evolution in time. The reason you always get the same prediction is because you only used a single point for training (even though it is 1247 dimensional, it is still 1 point).
You probably want to make input vectors of this nature (for simplicity, assume you are working with months):
[month1 month2; month2 month 3; month3 month4]
This example contains 2 training points with the evolution of 3 months. Note that they overlap.
Use the Network
After the network is trained and validated, the network object can be used to calculate the network response to any input. For example, if you want to find the network response to the fifth input vector in the building data set, you can use the following
a = net(houseInputs(:,5))
a =
34.3922
If you try this command, your output might be different, depending on the state of your random number generator when the network was initialized. Below, the network object is called to calculate the outputs for a concurrent set of all the input vectors in the housing data set. This is the batch mode form of simulation, in which all the input vectors are placed in one matrix. This is much more efficient than presenting the vectors one at a time.
a = net(houseInputs);
Each time a neural network is trained, can result in a different solution due to different initial weight and bias values and different divisions of data into training, validation, and test sets. As a result, different neural networks trained on the same problem can give different outputs for the same input. To ensure that a neural network of good accuracy has been found, retrain several times.
There are several other techniques for improving upon initial solutions if higher accuracy is desired. For more information, see Improve Neural Network Generalization and Avoid Overfitting.
strong text
I want to know that data normalization that is required whether it must be applied to whole part of training set both input and output or input segment is enough.
Whether you should normalize output depends on the type of neurons you use in your neural network and what type of output you expect. Find out the range of possible outputs for the given type of cell, and check that your target outputs falls within this range. If not, you will need to normalize.
The 'standard' neural network uses the sigmoid function which outputs a value between 0 and 1, so if the desired output doesn't fall in this range, you'll need to normalize.