I have been working with the matlab neural network toolkit. Here I am using the NARX network. I have a dataset consisting of prices of an object as well as the quantity of the object purchased over a period of time. Essential this network does one step prediction which is defined mathematically as follows:
y(t)= f (y(t −1),y(t −2),...,y(t −ny),x(t −1),x(t −2),...,x(t −nx))
Here y(t) is the price at time t and x is the amount. So the input features I am using are price and amount and the target is the price at time t+1. Suppose I have 100 records of such transactions and each transaction consists of the price and the amount.Then essentially my neural network can predict the price of the 101st transaction. This works fine for one step predictions. However, if i want to do multiple step predictions, so say i want to predict 10 transactions ahead(110th transaction), then I assume that i do a one step prediction of the price and then feed this back into the neural network. I keep doing this until I reach the 110th prediction. However, in this scenario, after i predict the 101st price , I can feed this price into the neural network to predict the 102nd price, however, I do not know the amount of the object at the 101st transaction. How do I go about this ? I was thinking about setting my targets to be the prices of transactions that are 10 transactions ahead of the current one, so that when I predict the 101st transaction, I am essentially predicting the price of the 110th transaction. Is this a viable solution or am i going about this in a completely wrong manner. Thanks in advance for any help
Similar to what kostas said, once you have the predicted 101 price, you can use all your data to predict the 101 amount, then use that to predict the 102 price, then use the 102 price to predict the 102 amount, etc. However, this compounds any error in your predictions for each variable. To mitigate that, you can add several other features, like a tapering discount on past values or a measure of error to use in the prediction (search temporal difference learning for similar ideas in the reinforcement learning realm).
I guess you can use a separate neural network to do time series prediction for x in order to produce x(t+1) up to x(t+10) and then use these values to feed another ANN to predict y(t).
Related
For a time series forecasting problem, I noticed some people tried to predict the difference or the quotient. For instance, in trading, we can try to predict the price difference P_{t-1} - P_t or the price quotient P_{t-1}/P_t. So we get a more stationary problem. With a recurrent neural network for a regression problem, trying to predict the price difference can be a real pain if the price does not change sufficiently fast because it will predict mostly zero at each step.
Questions :
What are the advantages and inconveniences of using the difference or the quotient instead of the whole quantity?
What can a nice tool to get rid of the repetitive zeros in a problem like trying to predict the price movement?
If the assumption is that the price is stationary (*Pt=Cte), then predict the whole quantity.
If the assumption is that the price increase ()is stationary (Pt= Pt-1+Cte), then predict the absolute difference Pt-Pt-1. (Note: thie is the ARIMA model with a degree of differencing=1)
If the assumption is that the price growth (in percentage) is stationary (Pt=Pt-1 +Cte * Pt-1), then predict the relative difference Pt/Pt-1.
If the price changes rarely (i.e. the absolute or relative difference is most often zero), then try to predict the time interval between tow changes rather than the price itself.
I am trying to create a pollution prediction LSTM. I've seen an example on the web to cater for a Multivariate LSTM to predict the pollution levels for one city (Beijing), but what about more than one city? I don't really want a separate network for every city, I'd like a single generalised model/network for all x cities. But how do I feed that data into the LSTM?
Say I have the same data for each city, do I...
1) Train on all data for one city, then the next city, and so on until all cities are done.
2) Train data for all cities on date t, then data for all cities on t+1, then t+2 etc.
3) Something completely different.
Any thoughts?
I would try first the (1).
Also, you can try a multi inputs / multi outputs network. I mean you have 10 cities. Therefore, your network would have 10 RNN inputs and 10 outputs.
Here is a great tutorial on how to do it with Keras: https://keras.io/getting-started/functional-api-guide/
I'm not sure if it will work, but you can give it a try.
If I have 1000 observations in my dataset with 15 features and 1 label, how is the data in input neurons fed for forward pass and back propagation? Is it fed row wise for 1000 observations (one at a time) and weights are updated with each observation fed or full data is given in terms of input matrix and then with number of epochs, the network learns corresponding weight values? Also if it is fed one at time, what is epochs in that case?
Thanks
Assuming that the data is formatted into rows (1000 instances with 16 features each, with the last one being the label), you would feed in the first 15 features row by row and use the last "feature"/label as the target. This is called online learning. Online learning requires you to feed the data in one example at a time and conduct the back propagation and the weight update for every example. As you can imagine this can get quite intensive due to the backpropagation and update for every instance of your data.
The other option that you mentioned is feeding in the entire data into the network. This performs poorly in practice as the convergence is extremely slow.
In practice, mini-batches are used. This involves sending a small subset of the dataset through and then doing the back propagation and weight update. This provides the benefit of relatively frequent weight updates to speed up learning but is less intensive than the online learning. For more information on mini-batches see this University of Toronto Lecture by Geoffrey Hinton
Finally, an epoch is always 1 run through all of your data. It doesn't matter if you feed it in one at a time or all at once.
I hope this clarified your questions.
There was made an order of food in a certain time from a certain place. I have the following info:
Location - delivery address
Date and time of the order
Day of the week
The weather on that day
The aim of that neural network is to train it and then predict the future orders. E.g. the places where the most of the orders will be tomorrow.
What type of the neural network should be used to make achieve that goal?
What framework or lib is better to be used for such kind of neural network?
It would be great to have an example of a working similar neural network!
Your data is a time series and such can be tackled by e.g. a Recurrent Neural Network (RNN). The popular choice here is LSTM.
You should consider what granularity your location system should have. The exact coordinates are irrelevant, you should translate it to address or, better yet, districts. You can try square grid, but something hand-crafted will work better. Visualise your training data, see how orders are clustering and based on this you could create districts.
Expect seasonality of various forms. Your ANN should for sure look at least at full month, if not a year (depending on climate). You can start with a week though.
Let's say I want to model the amount of visitors at an arbitrary lake at specific time.
Given Data:
Time Series of Amount of Visitors for 12 lakes.
Weather Time Series for the 12 lakes
Number of Trees at lake
Percentage of grass/stone ground of the beach.
Hereby I want to use a Neural Network (NN) to model the amount of visitors and I have some essential questions which I want to introduce step by step. Note that the visitor time series shall not be used!
1) we only use the Inputs:
Time of Day
Day of Week
So there is two inputs and one output. I read of a rule of thumb which says that the hidden neurons should be chosen as
#input>=neurons>=#output.
Is the number of inputs here 2 or is it an estimate of the real amount of dependent variables (as weather, mood of persons, economical situation, ....). If yes so I should choose my hidden neurons as 1 or 2, correct?
2) If I want to include lake specific parameters as the number of treas or the ground ratio, can I just add these as additional inputs (constant for each of the twelve lakes) or would that not help for some reason? How could I assure that there is a causal connection between these inputs and the output?
3) For weather since it is a time series which weather values should I use. How do I get the optimal delay for example. Would Granger Causality be a mean to determine that?
Hope you can help. I just wanna discuss on the strength of NNs for modeling and want to hear your opinion. I would use Matlabs Neural Network Toolbox for this.
Thanks in advance.