Memory in an Artificial Neural Network - neural-network

I am just starting to get into the world of Neural networks, using my own objects, and vectors, in the language of C++.
However, I was thinking that a neural network in a game like Super Mario would have a difficult time based on just the screen for input, and the keyboard (or controller) for output, since the net has no way of finding out the velocity of the player, or any of it's past moves.
I thought that if a neural network had "memory", like where some of the output of the previous feed-forward were fed back into the net, along with the normal inputs, this could work to give the neural net a slight sense of what it could have been doing, and thus affect it's current output.
Is There a better way of giving a neural net "memory" by feeding part of it's output into it's input?
I would appreciate any helpful links, Thanks in advance.
PS: My choice of language is c++.


Recurrent Neural Networks (RNN) do just what you described. You can read more about them here: RNNs

Related

Can Neural Networks be used to maximize the output without knowing the expected output?

I'm currently doing research in Combinatorial Game Theory and I'm trying to develop an Artificial Intelligence using a Neural Network. My initial approach to this would be to take statistics of the game and use those as inputs, and train my Neural Network to develop the optimal weight configuration for those inputs in order to get a MAXIMAL output value for those inputs. Each set of inputs represents a move and by passing each move (input x weights) through the neural net I can find out which move has the maximal value. Thus, that move would be the best move to make.
This is all in theory and I'm just curious if constructing a neural net while not knowing the expected value is at all possible. If this doesn't seem reasonable are there any other algorithms I should look into for this sort of problem?
I appreciate any feedback, thank you in advance.

Selection of informative features using neural network

The goal is to reduce the number of input parameters to the neural network, because it is assumed that some of them are not informative (little influence on the output values). I know that there are built-in function http://www.mathworks.com/help/nnet/ref/processpca.html, but I need to perform reduction by using a neural network.
Is there a ready-made solution? If not, then we can tell something in this direction, and algorithm steps etc.

Deep Learning Neural Networks for Time Series Prediction

I'm starting a work on Internet traffic prediction (time series prediction) using artificial neural networks, but I have few experience with the matter.
Does anyone knows which method is the best for that? (which type
of neural network to use for time series prediction)
Is Deep Learning with unsupervised training a good idea for time
series learning?

You can do time-series prediction with neural nets, but it can get pretty tricky.
1) The obvious choice is a recurrent neural network (RNN). However, these can be really difficult to train, and I would not recommend RNNs if this is your first time using neural nets. Recently there has been some interesting work on easing the training of RNNs (e.g. Hessian-free optimization), but again - it's probably not for beginners ;-) Alternatively, you could try a scheme where you use a standard neural net (i.e. not a RNN), and try to predict the next frame of data from the previous? That might work.
2) This question is too general, there is no categorical right answer. Yes, you can use unsupervised feature learning as part of your solution (e.g. pre-training your model), but if your end goal is time-series prediction you will need to do some supervised learning too.
Good luck!

OCR and Neuron Network?

I am trying to code an OCR for shop tickets (in Java), I have good results with image dictionary distance, but not for skewed texts or bad scans.
I heard that neuronal networks are perfect for this.
Question: which type of neuronal network do you recommand for shop tickets character detection ?
Thks

Neural networks will not magically solve the problem for you. They will have similar problems that your current approach has. Most likely you will have to detect skew and correct it before sending it to a classifier.
Similarly with bad scans. It depends what exactly a bad scan is. For example, some neural networks are amazingly efficient at correcting blurs (unfocused image, blur by move, ...).
Have a look at some papers about OCR and neural networks. It is a classical topic so there are many. For example The Anatomy of Bangla OCR System for Printed Texts Using Back Propagation Neural Network also tries to solve the problem of skewed images before running a neural network.
I know that recurrent neural networks can be used for OCR. Even a very simple one will easily recognize simple characters. There is a recent paper that improves upon them: High-Performance OCR forPrinted English and Fraktur using LSTM Networks. They even include text-line normalization which may be very useful in your case.
Notice that there is an answer here about training a normal Feed-forward backpropagation neural network for OCR too: training feedforward neural network for OCR

"Convolutional Neural Networks" with "Deep Learning" have been shown to give some of the best results in OCR (specifically on the MNIST database).
A good starting point is this tutorial.

Are neural networks really abandonware?

I am planning to use neural networks for approximating a value function in a reinforcement learning algorithm. I want to do that to introduce some generalization and flexibility on how I represent states and actions.
Now, it looks to me that neural networks are the right tool to do that, however I have limited visibility here since I am not an AI expert. In particular, it seems that neural networks are being replaced by other technologies these days, e.g. support vector machines, but I am unsure if this is a fashion matter or if there is some real limitation in neural networks that could doom my approach. Do you have any suggestion?
Thanks,
Tunnuz

It's true that neural networks are no longer in vogue, as they once were, but they're hardly dead. The general reason for them falling from favor was the rise of the Support Vector Machine, because they converge globally and require fewer parameter specifications.
However, SVMs are very burdensome to implement and don't naturally generalize to reinforcement learning like ANNs do (SVMs are primarily used for offline decision problems).
I'd suggest you stick to ANNs if your task seems suitable to one, as within the realm of reinforcement learning, ANNs are still at the forefront in performance.
Here's a great place to start; just check out the section titled "Temporal Difference Learning" as that's the standard way ANNs solve reinforcement learning problems.
One caveat though: the recent trend in machine learning is to use many diverse learning agents together via bagging or boosting. While I haven't seen this as much in reinforcement learning, I'm sure employing this strategy would still be much more powerful than an ANN alone. But unless you really need world class performance (this is what won the netflix competition), I'd steer clear of this extremely complex technique.

It seems to me that neural networks are kind of making a comeback. For example, this year there were a bunch of papers at ICML 2011 on neural networks. I would definitely not consider them abandonware. That being said, I would not use them for reinforcement learning.

Neural networks are a decent general way of approximating complex functions, but they are rarely the best choice for any specific learning task. They are difficult to design, slow to converge, and get stuck in local minima.
If you have no experience with neural networks, then you might be happier to you use a more straightforward method of generalizing RL, such as coarse coding.

Theoretically it has been proved that Neural Networks can approximate any function (given an infinite number of hidden neurons and the necessary inputs), so no I don't think the neural networks will ever be abandonwares.
SVM are great, but they cannot be used for all applications while Neural Networks can be used for any purpose.
Using neural networks in combination with reinforcement learning is standard and well-known, but be careful to plot and debug your neural network's convergence to check that it works correctly as neural networks are notoriously known to be hard to implement and learn correctly.
Be also very careful about the representation of the problem you give to your neural network (ie: the inputs nodes): could you, or could an expert, solve the problem given what you give as inputs to your net? Very often, people implementing neural networks don't give enough informations for the neural net to reason, this is not so uncommon, so be careful with that.