EncogModel is extremely useful thanks to its use of VersatileMLDataSet and the ability to perform cross validation.
However I can't see a way to create an Elman, Jordan or other RNN. Is this possible using EncogModel?
EncogModel does not support recurrent neural networks. It would be a decent change to make it do this, because recurrent neural networks are time-series, the EncogModel class would need to be extended to support sequences.
Related
I'm doing a language recognizer, I had planned to classify my i-vectors with neural networks, but I've read a lot of papers and they always use other methods like SVM or PLDA, can someone explain to me why? or it's fine to do it with neural networks?
Neural networks are good for complex non-linear multifeature input. I-vectors by design map speaker space to very simple space where speakers are easily separated with logistic regression or SVM.
If you want to try with neural networks, try something end-to-end like https://github.com/FlashTek/vggvox-pytorch
I am trying to code an OCR for shop tickets (in Java), I have good results with image dictionary distance, but not for skewed texts or bad scans.
I heard that neuronal networks are perfect for this.
Question: which type of neuronal network do you recommand for shop tickets character detection ?
Thks
Neural networks will not magically solve the problem for you. They will have similar problems that your current approach has. Most likely you will have to detect skew and correct it before sending it to a classifier.
Similarly with bad scans. It depends what exactly a bad scan is. For example, some neural networks are amazingly efficient at correcting blurs (unfocused image, blur by move, ...).
Have a look at some papers about OCR and neural networks. It is a classical topic so there are many. For example The Anatomy of Bangla OCR System for Printed Texts Using Back Propagation Neural Network also tries to solve the problem of skewed images before running a neural network.
I know that recurrent neural networks can be used for OCR. Even a very simple one will easily recognize simple characters. There is a recent paper that improves upon them: High-Performance OCR forPrinted English and Fraktur using LSTM Networks. They even include text-line normalization which may be very useful in your case.
Notice that there is an answer here about training a normal Feed-forward backpropagation neural network for OCR too: training feedforward neural network for OCR
"Convolutional Neural Networks" with "Deep Learning" have been shown to give some of the best results in OCR (specifically on the MNIST database).
A good starting point is this tutorial.
I am planning to use neural networks for approximating a value function in a reinforcement learning algorithm. I want to do that to introduce some generalization and flexibility on how I represent states and actions.
Now, it looks to me that neural networks are the right tool to do that, however I have limited visibility here since I am not an AI expert. In particular, it seems that neural networks are being replaced by other technologies these days, e.g. support vector machines, but I am unsure if this is a fashion matter or if there is some real limitation in neural networks that could doom my approach. Do you have any suggestion?
Thanks,
Tunnuz
It's true that neural networks are no longer in vogue, as they once were, but they're hardly dead. The general reason for them falling from favor was the rise of the Support Vector Machine, because they converge globally and require fewer parameter specifications.
However, SVMs are very burdensome to implement and don't naturally generalize to reinforcement learning like ANNs do (SVMs are primarily used for offline decision problems).
I'd suggest you stick to ANNs if your task seems suitable to one, as within the realm of reinforcement learning, ANNs are still at the forefront in performance.
Here's a great place to start; just check out the section titled "Temporal Difference Learning" as that's the standard way ANNs solve reinforcement learning problems.
One caveat though: the recent trend in machine learning is to use many diverse learning agents together via bagging or boosting. While I haven't seen this as much in reinforcement learning, I'm sure employing this strategy would still be much more powerful than an ANN alone. But unless you really need world class performance (this is what won the netflix competition), I'd steer clear of this extremely complex technique.
It seems to me that neural networks are kind of making a comeback. For example, this year there were a bunch of papers at ICML 2011 on neural networks. I would definitely not consider them abandonware. That being said, I would not use them for reinforcement learning.
Neural networks are a decent general way of approximating complex functions, but they are rarely the best choice for any specific learning task. They are difficult to design, slow to converge, and get stuck in local minima.
If you have no experience with neural networks, then you might be happier to you use a more straightforward method of generalizing RL, such as coarse coding.
Theoretically it has been proved that Neural Networks can approximate any function (given an infinite number of hidden neurons and the necessary inputs), so no I don't think the neural networks will ever be abandonwares.
SVM are great, but they cannot be used for all applications while Neural Networks can be used for any purpose.
Using neural networks in combination with reinforcement learning is standard and well-known, but be careful to plot and debug your neural network's convergence to check that it works correctly as neural networks are notoriously known to be hard to implement and learn correctly.
Be also very careful about the representation of the problem you give to your neural network (ie: the inputs nodes): could you, or could an expert, solve the problem given what you give as inputs to your net? Very often, people implementing neural networks don't give enough informations for the neural net to reason, this is not so uncommon, so be careful with that.
I am a student working on an internship project where in we are using Bayesian networks to predict a possible outcome from a given set of discrete parent variables.We now intend to use artificial neural network to do the task.So could any one please help me out with the similarities and differences between Bayesian networks and artificial neural network?Any suggestions as how to proceed with migration would be helpful.
Thanks
Similarity
Both use directed graphs.
Difference
In Bayesian networks the vertices and edges have meaning- The network structure itself gives you valuable information about conditional dependence between the variables. With Neural Networks the network structure does not tell you anything.
I'm looking for computationally heavy tasks to implement with CUDA and wonder if neural networks or bayesian networks might apply. This is not my question, though, but rather what the relation between the two network types is. They seem very related, especially if you look at bayesian networks with a learning capability (which the article on wikipedia mentions). At a glance, bayesian networks look at bit like a specific type of neural networks. Can anyone sum up their relationship, and if there is any connection beyond the apparent similarity?
Bayesian networks represent independence (and dependence) relationships between variables. Thus, the links represent conditional relationships in the probabilistic sense. Neural networks, generally speaking, have no such direct interpretation, and in fact the intermediate nodes of most neural networks are discovered features, instead of having any predicate associated with them in their own right.
It is reported that Bayesian networks are more resistant to the "overtraining" that is seen in some neural networks. In other words some neural networks become so "trained" to the observed measurements used in training that they aren't useful for the general cases.
Indeed they are. I see a bayesian network as a neural network applying the Baye's Theorem on large scale, but I don't remember details. I know where you can find them and I recommend this book for that.