I'm developing a project for the university. I have to create a classifier for a disease. The data-set i have contains several inputs (symptoms) and each of them is associated to a multiplicative probability factor (e.g. if patient has the symptom A, he has a double probability to have that disease).
So, how can i do this type of classifier? Is there any type of neural network or other instrument to do this??
Thanks in advance
You should specify how much labeled data and unlabeled data you have.
Let's assume you have only labeled data. Then you could use neural networks, but IMHO, SVM or random forests are the best techniques for a first try.
Note that if you use machine learning techniques, your prior information about symptoms (multiplicative coefficients) are not used because the labels are used instead. If you want to use these coefficients, it's no more machine learning.
You can use neural network for this purpose also. If to speak about your situation, with binding symptom A to more chances for decease B, that is what neural network should be able to accomplish. To bind connection weights from input A ( symptom A ) to desease B. From your side, you can engrain such classification rule in case if you'll have enough training data in your training data set. Also I propose you to try two different approaches: 1. neural network with N outputs (N = number of deseases to clasif). 2. Create for each desease neural network.
Related
I just want to know if a neural network can be trained with a single class of data set. I have a set of data that I want to train a neural network with. After training it, I want to give new data(for testing) to the trained neural network to check if it can recognize it as been similar to the training sample or not.
Is this possible with neural network? If yes, will that be a supervised learning or unsupervised.
I know neural networks can be used for classification if there are multiple classes but I have not seen with a single class before. A good explanation and link to any example will be much appreciated. Thanks
Of course it can be. But in this case it will only recognize this one class that you have trained it with. And depending on the expected output you can measure the similarity to the training data.
An NN, after training, is just a function. For classification problems you can imagine it as a function that takes data as input and returns an integer indicating to which class it belongs to. That being said, if you have only one class that can be represented by an integer value 1, and if training data is not similar to that class, you will get something like 1.555; It will not tel you that it belongs to another class, because you have introduced only one, but it will definitely give you a hint about its similarity.
NNs are considered to be supervised learning, because before training you have to provide both input and target, i. e. the expected output.
If you train a network with only a single class of data then It is popularly known as One-class Classification. There are various algorithms developed in the past like One-class SVM, Support Vector Data Description, OCKELM etc. Tax and Duin developed a MATLAB toolbox for this and it supports various one-class classifiers.
DD Toolbox
One-class SVM
Kernel Ridge Regression based or Kernelized ELM based or LSSVM(where bias=0) based One-class Classification
There is a paper Anomaly Detection Using One-Class Neural Networks
which combines One-Class SVM and Neural Networks.
Here is source code. However, I've had difficulty connecting the source code and the paper.
Is it possible to train a Network with 2 inputs : one is the data and the other is a constant that we define.
We train the network with one set of datas and set the second input to '10' for example
then once it has converged, we train with another set of data and set the second input to '20' this time.
Now what if i input test data with the second parameter set to '15', will it automatically extrapolate between the two learned states?
If not, how do i do if i want to do what i explained above : extrapolate between two training states?
thanks a lot
Jeff
It is possible to add another input as a parameter into the neural network, but I am unsure what benefit you are trying to achieve by adding this input.
You would need to train the neural network with this input so that it would estimate the value between the two trained networks. This would involve training each individual network first and then training a second network that would extrapolate between states.
If you are trying to modularise each trained Neural Network for specific roles or classifications, and these classifications represent some kind of continuous relationship (for example, weather predictions that specialise for no rain, light rain, moderate rain and heavy rain), then perhaps this input could be used in some way to encourage the output of a particular network.
If you would like to adjust the weights of each network so that some Neural Networks have more preference than others, perhaps an Ensemble approach can assist with different weights for each network (static and dynamic options are available). If you just want to map the differences between the two networks with different weights, perhaps a linear or non-linear function can be applied between the two networks and mapped to detail the changes between the two trained networks.
Recently, I am trying to using Matlab build-in neural networks toolbox to accomplish my classification problem. However, I have some questions about the parameter settings.
a. The number of neurons in the hidden layer:
The example on this page Matlab neural networks classification example shows a two-layer (i.e. one-hidden-layer and one-output-layer) feed forward neural networks. In this example, it uses 10 neurons in the hidden layer
net = patternnet(10);
My first question is how to define the best number of neurons for my classification problem? Should I use cross-validation method to get the best performed number of neurons using a training data set?
b. Is there a method to choose three-layer or more multi-layer neural networks?
c. There are many different training method we can use in the neural networks toolbox. A list can be found at Training methods list. The page mentioned that the fastest training function is generally 'trainlm'; however, generally speaking, which one will perform best? Or it totally depends on the data set I am using?
d. In each training method, there is a parameter called 'epochs', which is the training iteration for my understanding. For each training method, Matlab defined the maximum number of epochs to train. However, from the example, it seems like 'epochs' is another parameter we can tune. Am I right? Or we just set the maximum number of epochs or leave it as default?
Any experience with Matlab neural networks toolbox is welcome and thanks very much for your reply. A.
a. You can refer to How to choose number of hidden layers and nodes in neural network? and ftp://ftp.sas.com/pub/neural/FAQ3.html#A_hu
Surely you can do cross-validation to determine the parameter of best number of neurons. But it's not recommended as it's more suitable to use it in the stage of weights training of a certain network.
b. Refer to ftp://ftp.sas.com/pub/neural/FAQ3.html#A_hl
And for more layers of neural network, you can refer to Deep Learning, which is very hot in recent years and gets state-of-the-art performances in many of the pattern recognition tasks.
c. It depends on your data. trainlm performs better on function fitting (nonlinear regression) problems than on pattern recognition problems while training large networks and pattern recognition networks, trainscg and trainrp are good choices. Generally, Gradient Descent and Resilient Backpropagation is recommended. More detailed comparison can be found here: http://www.mathworks.cn/cn/help/nnet/ug/choose-a-multilayer-neural-network-training-function.html
d. Yes, you're right. We can tune the epochs parameter. Generally you can output the recognition results/accuracy at every epoch and you will see that it is promoting more and more slowly, and the more epochs the more computing time. You can make a compromise between the accuracy and computation time.
For part b of your question:
You can use like this code:
net = patternnet([10 15 20]);
This script create a network with 3 hidden layer that first layer has 10 neurons, second layer has 15 neurons and 3th layer has 20 neurons.
Is supervised training of a neural network with 2 unknown outputs possible where there is a relation such as y=a.x^b between known parameters (y,x) and unknowns (a,b). here (a,b) are the outputs of network!!!
The direct consequence of the universal approximation theorem is that any continous function from the compact subset of R^d onto k-dimensional hypercube can be approximated with standard feed forward neural network with given error bound eps.
So in simple words - in fact every function can be trained using neural network, which does not mean that in practise any algorithm will actually do this (it is purely existantional proof, which gives no intuition "where to look").
So if your question is "is it possible to train a network that will aproximate my function?" the answer is yes, if the question is "is it possible to make neural network represent exactly my function" then the answer is yes, but given a custom activation function.
I have created a neural network with 2 inputs nodes, 4 hidden nodes and 3 output nodes. The initial weights are random between -1 to 1. I used backpropagation method to update the network with TD error. However, the performance is not good.
I want to know, where the problem might be?
1. Is a bias node necessary?
2. Are eligibility traces necessary?
If anyone can provide me any sample code, I'm very grateful.
Yes, you should include the bias nodes, and yes you should use eligibility traces. The bias nodes just give one additional tunable parameter. Think of the neural network as a "function approximator" as described in Sutton and Barto's book (free online). If the neural network has parameters theta (a vector containing all of the weights in the network), then the Sarsa update is just (using LaTeX notation):
\delta_t = r_t + \gamma*Q(s_{t+1},a_{t+1},\theta_t) - Q(s_t,a_t, \theta_t)
\theta_{t+1} = \theta_t + \alpha*\delta_t*\frac{\partial Q(s,a,\theta)}{\partial \theta}
This is for any function approximator Q(s,a,\theta), which estimates Q(s,a) by tuning its parameters, \theta.
However, I must ask why you're doing this. If you're just trying to get Q learning working really well, then you should use the Fourier Basis instead of a neural network:
http://all.cs.umass.edu/pubs/2011/konidaris_o_t_11.pdf
If you really want to use a neural network for RL, then you should use a natural actor-critic (NAC). NACs follow something called the "natural gradient," which was developed by Amari specifically to speed up learning using neural networks, and it makes a huge difference.
We need more information. What is the problem domain. What are the inputs? What are the outputs?
RL can take a very long time to train and, depending on how you're training, can go from good to great to good to not-so-good during training. Therefore, you should plot the performance of your agent during learning, not just the end result.
You always should use bias nodes. Eligibility traces? Probably not.