The goal is to reduce the number of input parameters to the neural network, because it is assumed that some of them are not informative (little influence on the output values). I know that there are built-in function http://www.mathworks.com/help/nnet/ref/processpca.html, but I need to perform reduction by using a neural network.
Is there a ready-made solution? If not, then we can tell something in this direction, and algorithm steps etc.
Related
Wanted to ask the opinion of SO experts about the type of neural network I should use to teach it make yes/no answers on the combination of over fifty parameters. Essentially I have a valuation that may produce up to fifty different warnings or errors that are present in what’s being evaluated. So far I’ve been using mean average with coefficients to produce yes/no threshold, but wanted to learn more about applying it through supervised neural network, which I can feed different results and teach it to give final verdict. Which neural network I can use for such undertaking? There are quite a few there and as I’m entering the field of artificial learning, I wanted to which direction I should start looking to.
EDIT
What I'm starting to lean towards is employing some kind of back-propagation to adjust coefficients for each of the rule, where the decision of whether barcode data is correct or not will influence those coefficients. I'm pretty sure this can be achieved using a NN, but not exactly sure which one to use.
I'm currently doing research in Combinatorial Game Theory and I'm trying to develop an Artificial Intelligence using a Neural Network. My initial approach to this would be to take statistics of the game and use those as inputs, and train my Neural Network to develop the optimal weight configuration for those inputs in order to get a MAXIMAL output value for those inputs. Each set of inputs represents a move and by passing each move (input x weights) through the neural net I can find out which move has the maximal value. Thus, that move would be the best move to make.
This is all in theory and I'm just curious if constructing a neural net while not knowing the expected value is at all possible. If this doesn't seem reasonable are there any other algorithms I should look into for this sort of problem?
I appreciate any feedback, thank you in advance.
This is an on-going venture and some details are purposefully obfuscated.
I have a box that has several inputs and one output. The output voltage changes as the input voltages are changed. The desirability of the output sequence cannot be evaluated until many states pass and a look back process is evaluated.
I want to design a neural network that takes a number of outputs from the box as input and produce the correct input settings for the box to produce the optimal next output.
I cannot train this network using backpropagation. How do I train this network?
Genetic algorithm would be a good candidate here. A chromosome could encode the weights of the neural network. After evaluation you assign a fitness value to the chromosomes based on their performance. Chromosomes with higher fitness value have a higher chance to reproduce, helping to generate better performing chromosomes in the next generation.
Encoding the weights is a relatively simple solution, more complex ones could even define the topology of the network.
You might find some additional helpful information here:
http://en.wikipedia.org/wiki/Neuroevolution
Hillclimbing is the simplest optimization algorithm to implement. Just randomly modify the weights, see if it does better, if not reset them and try again. It's also generally faster than genetic algorithms. However it is prone to getting stuck in local optima, so try running it several times and selecting the best result.
I've recently been delving into artificial neural networks again, both evolved and trained. I had a question regarding what methods, if any, to solve for inputs that would result in a target output set. Is there a name for this? Everything I try to look for leads me to backpropagation which isn't necessarily what I need. In my search, the closest thing I've come to expressing my question is
Is it possible to run a neural network in reverse?
Which told me that there, indeed, would be many solutions for networks that had varying numbers of nodes for the layers and they would not be trivial to solve for. I had the idea of just marching toward an ideal set of inputs using the weights that have been established during learning. Does anyone else have experience doing something like this?
In order to elaborate:
Say you have a network with 401 input nodes which represents a 20x20 grayscale image and a bias, two hidden layers consisting of 100+25 nodes, as well as 6 output nodes representing a classification (symbols, roman numerals, etc).
After training a neural network so that it can classify with an acceptable error, I would like to run the network backwards. This would mean I would input a classification in the output that I would like to see, and the network would imagine a set of inputs that would result in the expected output. So for the roman numeral example, this could mean that I would request it to run the net in reverse for the symbol 'X' and it would generate an image that would resemble what the net thought an 'X' looked like. In this way, I could get a good idea of the features it learned to separate the classifications. I feel as it would be very beneficial in understanding how ANNs function and learn in the grand scheme of things.
For a simple feed-forward fully connected NN, it is possible to project hidden unit activation into pixel space by taking inverse of activation function (for example Logit for sigmoid units), dividing it by sum of incoming weights and then multiplying that value by weight of each pixel. That will give visualization of average pattern, recognized by this hidden unit. Summing up these patterns for each hidden unit will result in average pattern, that corresponds to this particular set of hidden unit activities.Same procedure can be in principle be applied to to project output activations into hidden unit activity patterns.
This is indeed useful for analyzing what features NN learned in image recognition. For more complex methods you can take a look at this paper (besides everything it contains examples of patterns that NN can learn).
You can not exactly run NN in reverse, because it does not remember all information from source image - only patterns that it learned to detect. So network cannot "imagine a set inputs". However, it possible to sample probability distribution (taking weight as probability of activation of each pixel) and produce a set of patterns that can be recognized by particular neuron.
I know that you can, and I am working on a solution now. I have some code on my github here for imagining the inputs of a neural network that classifies the handwritten digits of the MNIST dataset, but I don't think it is entirely correct. Right now, I simply take a trained network and my desired output and multiply backwards by the learned weights at each layer until I have a value for inputs. This is skipping over the activation function and may have some other errors, but I am getting pretty reasonable images out of it. For example, this is the result of the trained network imagining a 3: number 3
Yes, you can run a probabilistic NN in reverse to get it to 'imagine' inputs that would match an output it's been trained to categorise.
I highly recommend Geoffrey Hinton's coursera course on NN's here:
https://www.coursera.org/course/neuralnets
He demonstrates in his introductory video a NN imagining various "2"s that it would recognise having been trained to identify the numerals 0 through 9. It's very impressive!
I think it's basically doing exactly what you're looking to do.
Gruff
My project is to recognize ancient coins. I have used David Lowe's SIFT algorithm to extract features of images.
[siftImage, descriptors, locs] = sift(filteredImg);
Now I want to give these features to a neural network for training images.
1) What value should I feed to Neural network as input? (descriptors vector or locs)
2) How can I use it for neural network?
Can someone please help me? Thanks a lot in advance.
You need to manually categorise some of your data and perform a statistical analysis of the features, so understand which are going to give you the best chance.
This can go from a basic histogram overlap, of feature frequency distribution by category, to a more complex multi-dimensional cluster behaviour analysis.
This will enable you to find the features that seem be most suitable for the neural network to use for classification.
You should not make assumptions about which will be most useful before analysing the data, as you often find unexpected features give useful information in a new domain.