Creating a neural network to evaluate a logical function - neural-network

Question

I have been trying to learn neural networking and all the examples I saw on the internet gave examples of emulating logic gates say XOR gate. But what I want to do is create a network that can be trained to emulate functions say the x^2 or e^x. Is this possible? What changes in the network do I need to make?
Here's my code for a neural network consisting of 1 input node, one hidden layer consisting of 3 nodes, and one output node.
#include <iostream.h>
#include <fstream.h>
#include <math.h>
#include <time.h>
const double eeta=0.9;
const int n=5;
struct Net_elem
{
double weights1[3];
double weights2[3];
double bias1,bias2;
};//structure to store network paramenters
Net_elem net_elem;
double sigma(double input)
{
return 1/(1+exp(-input));
}
void show_net_elem()
{
cout.precision(15);
for(int i=0;i<3;i++)
{
cout<<"weights1["<<i<<"]="<<net_elem.weights1[i];
cout<<endl;
}
for(int i=0;i<3;i++)
{
cout<<"weights2["<<i<<"]="<<net_elem.weights2[i];
cout<<endl;
}
cout<<"bias1="<<net_elem.bias1<<" bias2="<<net_elem.bias2<<endl;
system("pause");
system("cls");
}
//function to train the network
void train(double input,double expected)
{
double Output,output[3],Delta,delta[3],delta_bias1,delta_bias2;
//Propogate forward
double sum=0;
for(int i=0;i<3;i++)
output[i]=sigma(input*net_elem.weights1[i]+net_elem.bias1);
sum=0;
for(int i=0;i<3;i++)
sum=sum+output[i]*net_elem.weights2[i];
Output=sigma(sum+net_elem.bias2);
cout<<"Output="<<Output<<endl;
//Backpropogate
Delta=expected-Output;
for(int i=0;i<3;i++)
delta[i]=net_elem.weights2[i]*Delta;
delta_bias2=net_elem.bias2*Delta;
//Update weights
for(int i=0;i<3;i++)
net_elem.weights1[i]=net_elem.weights1[i]+eeta*delta[i]*output[i]*(1-output[i])*input;
for(int i=0;i<3;i++)
net_elem.weights2[i]=net_elem.weights2[i]+eeta*Delta*Output*(1-Output)*output[i];
net_elem.bias2=net_elem.bias2+eeta*delta_bias2;
double sum1=0;
for(int i=0;i<3;i++)
sum1=sum1+net_elem.weights1[i]*delta[i];
net_elem.bias1=net_elem.bias1+eeta*sum1;
show_net_elem();
}
void test()
{
cout.precision(15);
double input,Output,output[3];
cout<<"Enter Input:";
cin>>input;
//Propogate forward
double sum=0;
for(int i=0;i<3;i++)
output[i]=sigma(input*net_elem.weights1[i]+net_elem.bias1);
for(int i=0;i<3;i++)
sum=sum+output[i]*net_elem.weights2[i];
Output=sigma(sum+net_elem.bias2);
cout<<"Output="<<Output<<endl;
}
I have tried to run it to emulate the square root function. But the output simply jumps between 0 and 1, alternating.
Main:
int main()
{
net_elem.weights1[0]=(double)(rand()%100+0)/10;
net_elem.weights1[1]=(double)(rand()%100+0)/10;
net_elem.weights1[2]=(double)(rand()%100+0)/10;
net_elem.weights2[0]=(double)(rand()%100+0)/10;
net_elem.weights2[1]=(double)(rand()%100+0)/10;
net_elem.weights2[2]=(double)(rand()%100+0)/10;;
net_elem.bias1=(double)(rand()%100+0)/10;
net_elem.bias2=(double)(rand()%100+0)/10;
double output[n],input[n];
int ch;
for(int i=1;i<n;i++)
{
input[i]=100;
output[i]=sqrt(input[i]);
}
do
{
cout<<endl<<"1. Train"<<endl;
cout<<"2. Test"<<endl;
cout<<"3. Exit"<<endl;
cin>>ch;
switch(ch)
{
case 1:for(int i=1;i<n;i++)
{
train(input[i],output[i]);
}
break;
case 2:test();break;
case 3:break;
default:cout<<"Enter Proper Choice"<<endl;
}
}while(ch!=3);
}

Answer 1

I think you are missing the point of using a neural network. Neural networks don't imitate known functions. They separate areas in an unknown vector space. The XOR problem is often given as an example, because it is the minimal non-linearly separable problem: A simple perceptron is simply a line separating two areas in you problem
In this case, the blue dots can be separated from the red dots using a simple line (the problem is linearly separable). However, in the XOR problem, the dots are situated like this:
Here, a single line (a perceptron) is not enough. However, a multi-layer perceptron (most probably the type of neural network you are using) can use multiple perceptrons, (in this case two) to separate the blue and red dots. In a similar manner, a neural network can separate any space.
However, the XOR problem produces two types of output, and we use a neural network to separate them. On the other hand, x^2 produces a continuous lines of points, so there's nothing to separate. Also, keep in mind that imitating the XOR function is given as an example of such problems. In practice, nobody ever uses a neural network to replace the XOR function. If you want to use a function, just use it, instead of building something that approximates it.
PS: If you still want to emulate the x^2 function for practice, you need regression. What you are doing is classification (since you are using a sigma function in you output). However, for practicing you'd better stick with classification problems. They are by far more common. Also, for such problems try Matlab, or, if you want to write in C++ use some linear algebra library (eg EIGEN 3) to make it easier writing without a thousand for loops.