Encog Hopfield network training - neural-network

Question

I am trying to recognize a single character from an image.
The image data has been cleaned and has a clear distinct letter visible.
So when I add the trainigset to the hopfield network, it goed great as log as i only add 2.
Once I add more, the patterns it has trained start to verlap and blur.
How can I Prevent this?
training the network:
public void trainNetwork() {
network.reset();
System.out.println("Training hopfield network");
long startTimeLong = System.nanoTime();
for (double[] ds : trainingInput) {
network.addPattern(doubleArrayToBiPolarNeuralData(ds));
}
long endTimeLong = System.nanoTime();
double durationInSec = (double) ((endTimeLong - startTimeLong) / Math.pow(10, 9));
System.out.println("Finished training network in: " + durationInSec);
}
private BiPolarNeuralData doubleArrayToBiPolarNeuralData(double[] data) {
BiPolarNeuralData patternData = new BiPolarNeuralData(neuroncount);
if (data.length != neuroncount) {
IndexOutOfBoundsException e = new IndexOutOfBoundsException("the size of the traingsinputs is different from the amount of input neurons");
logger.error(e.getMessage(), e);
throw e;
}
patternData.setData(data);
return patternData;
}
Result when training 2 characters:
Cycles until stable(max 100): 1, result=
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
OOOOOOOOOOO -> OOOOOOOOOOO
OOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOO
OOOOOOO OOOOOOOOO -> OOOOOOO OOOOOOOOO
OOOOO OOOOOOO -> OOOOO OOOOOOO
OOOOO OOOOOOO -> OOOOO OOOOOOO
OOOOOO -> OOOOOO
OOOOOOOOOO -> OOOOOOOOOO
OOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOO
OOOOOOOOO OOOOOO -> OOOOOOOOO OOOOOO
OOOOOOO OOOOOOO -> OOOOOOO OOOOOOO
OOOOOO OOOOOOO -> OOOOOO OOOOOOO
OOOOOOO OOOOOOOO -> OOOOOOO OOOOOOOO
OOOOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOOO
OOOOOOOOO OOOOOOO -> OOOOOOOOO OOOOOOO
->
->
->
->
->
->
->
->
->
->
->
->
->
Results when training all characters:
Cycles until stable(max 100): 3, result=
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
->
OOOOOOOOOOO -> OOOOOOOOOOOOOOO
OOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOOOO
OOOOOOOOO OOOOOOOOO -> OOOOOOOOOOOOOOOO
OOOOOOOO OOOOOOO -> OOOOOO OOOOOO
OOOOOOO OOOOOOO -> OOOOOO OOOOO
OOOOOOO -> OOOOOO OOOOO
OOOOOOO -> OOOOOO OOOOO
OOOOOO -> OOOOOO OOOOOOO
OOOOOO -> OOOOOO OOOOO
OOOOOOO -> OOOOOO OOOOOO
OOOOOOO -> OOOOOOO OOOOOO
OOOOOOO OOOOOOO -> OOOOOOO OOOOOO
OOOOOOO OOOOOOO -> OOOOOOO OOOOOO
OOOOOOOO OOOOOOOO -> OOOOOOO OOOOOOO
OOOOOOOOO OOOOOOOOO -> OOOOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOO
OOOOOOOOOOOOOOOO -> OOOOOOOOOOOOOOOO
OOOOOOOOOOOOOO -> OOOOOOOOOOOOO
OOOOOOOOOOO -> OOOOOOOOO O
->
->
->
->
->
->
->
->
->
->
->
->
->
It always resolves to this same composite blob. What am I doing wrong?

Answer 1

The maximum amount of patterns that can be learned in a hopfield network while giving an acceptable number of mistakes is called its capacity. The capacity is a function of the logarithm of the total number of neurons in the net, meaning that if you want more patterns, you have to increase the amount of neurons in the network.
Also, the composite blobs are called mixed states (or sometimes spin-glass states, depending on what type of blob is shown). When a Hopfield network is put into a starting state, it tends to drive itself to the local energy minimum. Sometimes that minimum is not the trained pattern, but a state that is a mix of several patterns that were used in the training. Usually, these mixed states have a higher energy than the trained patterns, but if the starting state is nearer the mixed state, it will tend to drive itself to that local minimum. Sometimes including some noise to the network could avoid these local minimums and get the network to the most approximate trained state. You could include noise by generating a random number and only perform the sign operation if that number is above certain threshold.
In conclusion, adding neurons and noise to the network could help you solve your problem.