I have the following problem in StreamInsight. I have a query where new tasks from an order came in and trigger an output adapter to make an prediction. The outputadapter writes the predicted task cycle time to a table (in Windows Azure). The prediction is based on neural networks and is plugged in in the outputadapter. After the prediction is written in the table I want to do something else with all the predicted times. So in a second query I want to count the number of written tasks in a time window of 5 minutes. When the number of predicted values saved in the table is equal to the number of tasks in an order, I want to get all the predicted values from the table and make a prediction of the order cycle time.
For this idea I need to make a new event in my outputadapter to know the predicted time is writen in the table. But I don't thinks its possible to enqueue new events in the streaminsight server from an outputadapter.
Maybe this figure makes the problem clear:
http://i40.tinypic.com/4h4850.jpg
Hope someone can help me.
Thanks Carlo
First off, I'm assuming you are using pre-2.1 StreamInsight based on your use of the term "output adapter".
From what you've posted, I would strongly recommend that your adapters do either input or output, but not both. This cuts down on the complexity, makes the implementation easier, and depending on how you wrote the adapter, you now have a reusable piece of code in your solution.
If you are wanting to send data from StreamInsight to your neural network prediction engine, you will need to write an output adapter to do that. Then I would create an input adapter that will get the results from the neural network prediction engine and enqueue the data into StreamInsight. After creating your stream from the neural network prediction engine input adapter, you can use dynamic query composition to share the stream to a Windows Azure storage output adapter and your next query.
If your neural network prediction engine can "push" data to your input adapter, that would be the way to do. If not, you'll have to poll for results.
There is a lot more to this, but it's difficult to drill in to more specifics without more details.
Hope this helps.
Related
I am trying to build a complex neural network using Computation Graph implementation in Deeplearning4J. I need to have multiple outputs so that's why I can't go with the generic MultiLayerConfiguration.
However, my problem is that in this case I do not know how to do the evaluation of my model and I would like at least to know the accuracy.
Has anybody worked with Comp Graphs in dl4j?
First of all yes: tons of people use computation graph. They usually start from our existing examples though and tend to mainly use it for things like seq2seq.
As for your question on evaluation, it's conceptually the same as multi layer network. How you evaluate is likely going to be task specific though. If you think about where evaluation happens, it's always tied to a task (classification,regression,binary classification,..) with an output layer . In the most common case usually you only have 1 output which outputs a classification. In that case you can just use the first array it outputs.
Otherwise for multiple outputs..you'd have to define what you're evaluating. Usually tasks merge to 1 path.
If they don't, you'd have multiple output layers where you want to do an evaluation object per output.
Computation graphs and multi layer network both use a .output method to give you raw arrays. That is typically what you pass to eval.eval.
I am trying to develop a MATLAB Simulink model that will help me study the load of my department.
The model works, however one of the blocks goes right over my head when it comes to understanding, as I used the internet to help me with it.
Here is the main block:
The Scope Displays the Voltage, Current & Power
The "dept01" block inputs the data from .csv file and contains only [Time,Power].
Here is what goes inside of the "Electrical Department" Block:
I have no problem understanding this part, I'm simply splitting the total power into three portions.
NOTE: I am also assuming that ultimately Q=0 so Total Power = Real Power
This is the Second Step of the "Electrical Department" block which I cannot understand in any way. Maybe my concepts are weak but this part makes no sense to me.
Can someone please explain it to me that how is the block calculating Voltage & Current using just the Power??? Also how does it imitate the function of a Load so that the Energy Meter sees it as a load?
Thanks!
Load can be emulated by connecting a current source in series with the voltage source as it is seen the diagram. In your case, the controlled current source has been used. It also looks like the dependent current source is derived from the voltage. I request you to give the details of relational operator used and the sigma block. Without which you can not derive the relation ship. If the current is dependent on the voltage like the case, voltage and current can be calculated simply from the power.
I'm trying to make a neural network try to figure out the meaning of input(keyboard keys in this case) according to the user.
I have multiple possible output "commands" that the NN can interpret the inputs to mean, and at each state certain outputs can count as beneficial while others are a detriment
When the NN starts up for the first time, no input should have any particular meaning to it but as time goes on I want the NN to be able to figure out what the user most likely meant.
I've tried a Multilayer perceptron NN that has as many input nodes as there are physical inputs and as many output nodes as there are commands and a number of nodes equal to the sum of the other two layers as a single hidden layer, in this case it is then a 5,15,10.
The NN assumes that the user will only make moves that are in the NN's best interest.
So far it seems the NN is just figuring out what is the command it can take that will most likely result in a beneficial move, regardless of the input key rather than trying to figure out what key should result in what move according to the user.
Because of this I'm wondering (most likely wrong) if I should produce a separate NN for each input to try and figure out the current output according to the user.
Is there a different type of NN I should look into that will work better, and is there a recommended configuration for this problem?
I'll be happy with some recommendations of reading material that would help in this particular problem.
I'm at best an amateur in NN and would like to learn a lot more about the whole field, But I'm trying to focus my efforts on this problem for now.
Accordng to me you want the output to be according to behaviour of the player as number of inputs are more than in actual case. So according to me there should be some type of memory for the actions taken by the player in order to find the patterns.This can be done using Long Short term Memory.
I have been using the Encog Neural Net workbench (version 3.2) to run the sunspot prediction routine and have noticed that when changing the future prediction window to greater than 1 the results in the sunspot_output.csv appear to be time offset so that the output when the network evaluates at t=0 are not really (t+1), (t+2), (t+3) etc. It's very likely I'm not understanding how the workbench is displaying the results so perhaps someone could clarify this for me.
As I understand it if you use a past window of 30 and a future window of 14 then the network will look at the last 30 records and predict forward from the last available record (in this case lets say 11/1/1951 is the last available record). So an evaluation on 11/1/1951 will look back 30 records to 5/1/1949 and use this information to feed through the trained network to predict data for 12/1/1951 (t+1), 1/1/1952 (t+2), 2/1/1952 (t+3), etc. However, looking at the result file this does not appear to be the case. The "prediction" really appears to be a repeat of the pattern from the previous 14 records. So that (t+1) is really more representative of (t-14) 08/01/1950 than the next record forward from (t=0) which would be on 12/1/1951.
I have an image that shows this but unfortunately I don't appear have the reputation points to post it yet. To reproduce this issue I suggest using the Encog workbench and using a past window of 30, future window of 14 and training error of 1 or 2%.
To Summarize:
Has anyone else noticed this issue when looking at the predictive network results, particularly for greater than one time step ahead?
Why do the workbench results show that the encog predictive neural network is not properly predicting into the future when you look at the dates associated with the outputs.
Thank you for any thoughts you may have!
That's not an issue is how a sliding window time series forecaster works.
I would suggest you to deepen here https://www.cs.rutgers.edu/~pazzani/Publications/survey.pdf
It really depends on how you tune the neural network.
If you want more predictive power you have to extract features or syntetize new features (for example I would use wavelet extraction and denoising).
Pay attention to normalization. Use range normalization if you know that there are known ranges otherwise z-normalization.
Use the proper activation function: Sigmoid if the normalized range is 0,1 or tanh if the range is -1,1.
But before ending that the neural network is not predicting I would suggest you to use SVR (Support Vector Regression) included in encog.
It guarantees (if it is present) to reach the global minimum.
See if the SVR predicts better than the ANN.
If not use my firsts suggestions ;-)
Vincenzo
I have a set of data for the past 5 years. Approx 7000 rows of data with features that are binary {yes/no} or are multi-classed {product A, B, C} A total of about 20+ features.
I am trying to make a program (or one time analysis project) to determine (predict) the product shipdate(shipping delay days) based on this historical data. I have 2 columns that indicate when a product was planned to be shipped and another column of when it was actually shipped! Currently.
I'm wondering how I can make a prediction program that determines based on the historic data when new data input of a product will expect to ship. I don't care about a getting a specific date but even just a program that can tell me number of delay days to add...
I took an ML class a while back and I wasn't sure how to start something like this. Any advice? Plus the closest thing to this I can think of is an image recognition assignment using NN. but that was too easy here I have to deal with a date instead of pixel white/black.... I used Matlab back in the day (I still know how to use it) but I just downloaded Weka data mining tool.
I was thinking of a neural network but I'm not sure how to set it up to have my program give me a the expected delay time (# of days/month) from the inputed ship date.
Basically,
I want to input (size = 5, prod = A, ....,expected ship date = jan 1st)
and the program returns the number of days to add as a delay onto my expected ship date given the historical trends...
Would appreciate any any help on how start something like this the correct/easiest/best way... Thanks in advance.
If you use weka, then get your input/label data into the arff format and then you try out all the different regressors (this is a regression problem after all). To avoid having to do too much programming quite yet (if you are just in an exploratory phase), use the weka experimenter which has a GUI for trying out a whole bunch of regressors on your dataset.
Then when you find one that does something expected and you want to do some more data analysis using MATLAB, then you can use a weka/matlab interface.