MATLAB Neural Network - Forward Propagation - matlab

I am creating a Forward Propagation In the feedforward step, an input pattern is propagated through the network to obtain an output. I have written this in pseudo code and currently attempting to implement this within MATLAB.
There are two errors I currently receive.
Patterns = x'; Desired = y; NHIDDENS = 1; prnout=Desired;
% Patterns become x so number of inputs becomes size of patterns
[NINPUTS,NPATS] = size(Patterns); [NOUTPUTS,NP] = size(Desired);
%apply the backprop here...
LearnRate = 0.15; Momentum = 0; DerivIncr = 0; deltaW1 = 0; deltaW2 = 0;
% Keeps the tan ordering of the examples of x
Inputs1= [Patterns;ones(1,NPATS)]; %Inputs1 = [ones(1,NPATS); Patterns];
% Weight initialisation
Weights1 = 0.5*(rand(NHIDDENS,1+NINPUTS)-0.5);
Weights2 = 0.5*(rand(1,1+NHIDDENS)-0.5);
TSS_Limit = 0.02;
for epoch = 1:10
% FORWARD LOOP
size(NOUTPUTS)
size(NPATS)
for ii = 0: ii < length(NINPUTS)
NOUTPUTS(ii+1) = NPATS(ii);
% Sets bias to 1
NOUTPUTS(1) = 1;
end
for ii = NHIDDENS: ii < NINPUTS
sum = 0;
for ij = 0: ij < ii
sum = sum + deltaW1(ii,ij) * NOUTPUTS(ij);
NOUTPUTS(ii) = tanh(sum);
end
end
Unable to perform assignment because the
left and right sides have a different
number of elements.
Error in mlpts (line 66)
NOUTPUTS(i+1) = NPATS(i);
i am still new to MATLAB and trying to become use to it.
After iterating through the loop
NOUTPUTS = 0 and the error is displayed. I am confused as I am trying to increment NOUTPUTS with ii by 1 through each loop.

I have been able to create the forward propagation with a loop.
for i =3:NNODES
summ = 0;
for j=1:i-1
summ = summ + weights(i,j) * Node_outputs(j);
end
if i == NNODES
Node_outputs(i) = summ
else
Node_outputs(i) = sigmoid(summ);
end
end
Out = Node_outputs(NNODES);
% BOut = ((Node_outputs(NNODES)) * (1 - Node_outputs));
BOut=zeros(1,6);
DeltaWeight = zeros(6,6);

Related

single perceptron not converging

I am programming a simple perceptron in matlab but it is not converging and I can't figure out why.
The goal is to binary classify 2D points.
%P1 Generate a dataset of two-dimensional points, and choose a random line in
%the plane as your target function f, where one side of the line maps to +1 and
%the other side to -1. Let the inputs xn 2 R2 be random points in the plane,
%and evaluate the target function f on each xn to get the corresponding output
%yn = f(xn).
clear all;
clc
clear
n = 20;
inputSize = 2; %number of inputs
dataset = generateRandom2DPointsDataset(n)';
[f , m , b] = targetFunction();
signs = classify(dataset,m,b);
weights=ones(1,2)*0.1;
threshold = 0;
fprintf('weights before:%d,%d\n',weights);
mistakes = 1;
numIterations = 0;
figure;
plotpv(dataset',(signs+1)/2);%mapping signs from -1:1 to 0:1 in order to use plotpv
hold on;
line(f(1,:),f(2,:));
pause(1)
while true
mistakes = 0;
for i = 1:n
if dataset(i,:)*weights' > threshold
result = 1;
else
result = -1;
end
error = signs(i) - result;
if error ~= 0
mistakes = mistakes + 1;
for j = 1:inputSize
weights(j) = weights(j) + error*dataset(i,j);
end
end
numIterations = numIterations + 1
end
if mistakes == 0
break
end
end
fprintf('weights after:%d,%d\n',weights);
random points and signs are fine since plotpv is working well
The code is based on that http://es.mathworks.com/matlabcentral/fileexchange/32949-a-perceptron-learns-to-perform-a-binary-nand-function?focused=5200056&tab=function.
When I pause the infinite loop, this is the status of my vairables:
I am not able to see why it is not converging.
Additional code( it is fine, just to avoid answers asking for that )
function [f,m,b] = targetFunction()
f = rand(2,2);
f(1,1) = 0;
f(1,2) = 1;
m = (f(2,2) - f(2,1));
b = f(2,1);
end
function dataset = generateRandom2DPointsDataset(n)
dataset = rand(2,n);
end
function values = classify(dataset,m,b)
for i=1:size(dataset,1)
y = m*dataset(i,1) + b;
if dataset(i,2) >= y, values(i) = 1;
else values(i) = -1;
end
end
end

Numerically solving a pde with changing domain in Matlab

I'm trying to implement a changing domain into my Matlab code which solves a pde numerically. In the given problem, I am modelling the baking of bread. It is a 1D problem where there is a single heat source from above. I am using the heat diffusion equation and for a fixed domain, I have provided my code below which solves it. However, I need to also model the rising of the bread, so over time my domain, 'L' in my code, changes. I am told that I can assume it changes linearly over an hour and that the bread initially fills 80% of the tin. What suggestions do you guys have for how I can account for the rising?
close all
clear all
clc
Tend = 3600; %Time to end after
dt = 1; %Time step size
Nstep = (Tend/dt)+1; %Number of time points
M = 1000; %Number of x points
dx = 1/M; %Size of the x steps
L = 1; %Length of domain
x = [0:dx:L]; %Creating the x points
kdiff = 2.7e-4; %Diffusion constant value
R = kdiff*dt/(dx*dx);
for i = 1:M+1
u(i,1) = 20;
end
for n = 1:Nstep
DIG(1) = 1;
b(1) = 200; %Boundry condition
DIG(M+1) = 1;
LF(1) = 0;
RT(1) = 0;
LF(M+1) = -1; %Insulated boundry
RT(M+1) = 0;
b(M+1) = 0; %Boundry condition
for m = 2:M
DIG(m) = 2+(2/R);
LF(m) = -1;
RT(m) = -1;
b(m) = u(m+1,n) + u(m-1,n) + (-2 + (2/R))*u(m,n);
end
u(:,n+1) = tdma(LF,DIG,RT,b,M+1);
end
plot(x,u(:,1:600:end)

What's wrong with my matlab programming of a hopfield neural network?

Sorry if the code is sloppy. The code is supposed to set up a Hopfield network from memory vectors of firing rates (a cross, a square, etc), converting between membrane potential and firing rate with the function f(ui) = tanh(beta * ui) and evolving using the evolution rule indicated. But for some reason it's not working: memories are not retrieved and instead the image disappears in some way. Please help.
w = zeros(100);
beta = 4;
% Memory 1: a cross
m1 = ones(10,10);
m1(:, 4) = 0;
m1(4, :) = 0;
init = m1(1:4,:);
m1 = (m1*2-1);
% Memory 2: a square
m2 = ones(10,10);
m2(5:7, 5:7) = 0;
m2 = (m2*2-1);
% Memory 3: a hollow square
m3 = ones(10,10);
m3(:, 1) = 0;
m3(:, 10) = 0;
m3(1, :) = 0;
m3(10, :) = 0;
m3 = (m3*2-1);
% build weight network
for i = 1:100
for j = 1:100
if i ~= j
w(i,j) = w(i,j) + m1(i) * m1(j) + m2(i) * m2(j) + m3(i) * m3(j);
end
end
end
% Initial condition
f1 = [init; rand(6,10)];
f1 = f1*2-1;
f1 = 0.9*f1;
%invert to produce membrane potential
u1 = atanh(f1)/beta;
% evolve according to equation 1
weight = 0;
rate = 0;
evolve = u1;
for t = 1:1
u1 = evolve;
for i = 1:10
for j = 1:10
weight = w(i,:); % weighted connection w/ every other neuron
rate = tanh(beta*u1(:));
evolve(i,j) = u1(i,j) + .05* (-u1(i,j) + (weight * rate));
end
end
end
u1 = evolve;
f1 = tanh(beta * u1);
imshow(reshape(f1, 10, 10), [-1 , 1], 'InitialMagnification', 'fit');

How to use Neural network for non binary input and output

I tried to use the modified version of NN back propagation code by Phil Brierley
(www.philbrierley.com). When i try to solve the XOR problem it works perfectly. but when i try to solve a problem of the form output = x1^2 + x2^2 (ouput = sum of squares of input), the results are not accurate. i have scaled the input and ouput between -1 and 1. I get different results every time i run the same program (i understand its due to random wts initialization), but results are very different. i tried changing learning rate but still results converge.
have given the code below
%---------------------------------------------------------
% MATLAB neural network backprop code
% by Phil Brierley
%--------------------------------------------------------
clear; clc; close all;
%user specified values
hidden_neurons = 4;
epochs = 20000;
input = [];
for i =-10:2.5:10
for j = -10:2.5:10
input = [input;i j];
end
end
output = (input(:,1).^2 + input(:,2).^2);
output1 = output;
% Maximum input and output limit and scaling factors
m1 = -10; m2 = 10;
m3 = 0; m4 = 250;
c = -1; d = 1;
%Scale input and output
for i =1:size(input,2)
I = input(:,i);
scaledI = ((d-c)*(I-m1) ./ (m2-m1)) + c;
input(:,i) = scaledI;
end
for i =1:size(output,2)
I = output(:,i);
scaledI = ((d-c)*(I-m3) ./ (m4-m3)) + c;
output(:,i) = scaledI;
end
train_inp = input;
train_out = output;
%read how many patterns and add bias
patterns = size(train_inp,1);
train_inp = [train_inp ones(patterns,1)];
%read how many inputs and initialize learning rate
inputs = size(train_inp,2);
hlr = 0.1;
%set initial random weights
weight_input_hidden = (randn(inputs,hidden_neurons) - 0.5)/10;
weight_hidden_output = (randn(1,hidden_neurons) - 0.5)/10;
%Training
err = zeros(1,epochs);
for iter = 1:epochs
alr = hlr;
blr = alr / 10;
%loop through the patterns, selecting randomly
for j = 1:patterns
%select a random pattern
patnum = round((rand * patterns) + 0.5);
if patnum > patterns
patnum = patterns;
elseif patnum < 1
patnum = 1;
end
%set the current pattern
this_pat = train_inp(patnum,:);
act = train_out(patnum,1);
%calculate the current error for this pattern
hval = (tanh(this_pat*weight_input_hidden))';
pred = hval'*weight_hidden_output';
error = pred - act;
% adjust weight hidden - output
delta_HO = error.*blr .*hval;
weight_hidden_output = weight_hidden_output - delta_HO';
% adjust the weights input - hidden
delta_IH= alr.*error.*weight_hidden_output'.*(1-(hval.^2))*this_pat;
weight_input_hidden = weight_input_hidden - delta_IH';
end
% -- another epoch finished
%compute overall network error at end of each epoch
pred = weight_hidden_output*tanh(train_inp*weight_input_hidden)';
error = pred' - train_out;
err(iter) = ((sum(error.^2))^0.5);
%stop if error is small
if err(iter) < 0.001
fprintf('converged at epoch: %d\n',iter);
break
end
end
%Output after training
pred = weight_hidden_output*tanh(train_inp*weight_input_hidden)';
Y = m3 + (m4-m3)*(pred-c)./(d-c);
% Testing for a new set of input
input_test = [6 -3.1; 0.5 1; -2 3; 3 -2; -4 5; 0.5 4; 6 1.5];
output_test = (input_test(:,1).^2 + input_test(:,2).^2);
input1 = input_test;
%Scale input
for i =1:size(input1,2)
I = input1(:,i);
scaledI = ((d-c)*(I-m1) ./ (m2-m1)) + c;
input1(:,i) = scaledI;
end
%Predict output
train_inp1 = input1;
patterns = size(train_inp1,1);
bias = ones(patterns,1);
train_inp1 = [train_inp1 bias];
pred1 = weight_hidden_output*tanh(train_inp1*weight_input_hidden)';
%Rescale
Y1 = m3 + (m4-m3)*(pred1-c)./(d-c);
analy_numer = [output_test Y1']
plot(err)
This is the sample output i get for problem
state after 20000 epochs
analy_numer =
45.6100 46.3174
1.2500 -2.9457
13.0000 11.9958
13.0000 9.7097
41.0000 44.9447
16.2500 17.1100
38.2500 43.9815
if i run once more i get different results. as can be observed for small values of input i get totally wrong ans (negative ans not possible). for other values accuracy is still poor.
can someone tell what i am doing wrong and how to correct.
thanks
raman

Matlab debugging - beginner level

I am a total beginner in Matlab and trying to write some Machine Learning Algorithms in Matlab. I would really appreciate it if someone can help me in debugging this code.
function y = KNNpredict(trX,trY,K,X)
% trX is NxD, trY is Nx1, K is 1x1 and X is 1xD
% we return a single value 'y' which is the predicted class
% TODO: write this function
% int[] distance = new int[N];
distances = zeroes(N, 1);
examples = zeroes(K, D+2);
i = 0;
% for(every row in trX) { // taking ONE example
for row=1:N,
examples(row,:) = trX(row,:);
%sum = 0.0;
%for(every col in this example) { // taking every feature of this example
for col=1:D,
% diff = compute squared difference between these points - (trX[row][col]-X[col])^2
diff =(trX(row,col)-X(col))^2;
sum += diff;
end % for
distances(row) = sqrt(sum);
examples(i:D+1) = distances(row);
examples(i:D+2) = trY(row:1);
end % for
% sort the examples based on their distances thus calculated
sortrows(examples, D+1);
% for(int i = 0; i < K; K++) {
% These are the nearest neighbors
pos = 0;
neg = 0;
res = 0;
for row=1:K,
if(examples(row,D+2 == -1))
neg = neg + 1;
else
pos = pos + 1;
%disp(distances(row));
end
end % for
if(pos > neg)
y = 1;
return;
else
y = -1;
return;
end
end
end
Thanks so much
When working with matrices in MATLAB, it is usually better to avoid excessive loops and instead use vectorized operations whenever possible. This will usually produce faster and shorter code.
In your case, the k-nearest neighbors algorithm is simple enough and can be well vectorized. Consider the following implementation:
function y = KNNpredict(trX, trY, K, x)
%# euclidean distance between instance x and every training instance
dist = sqrt( sum( bsxfun(#minus, trX, x).^2 , 2) );
%# sorting indices from smaller to larger distances
[~,ord] = sort(dist, 'ascend');
%# get the labels of the K nearest neighbors
kTrY = trY( ord(1:min(K,end)) );
%# majority class vote
y = mode(kTrY);
end
Here is an example to test it using the Fisher-Iris dataset:
%# load dataset (data + labels)
load fisheriris
X = meas;
Y = grp2idx(species);
%# partition the data into training/testing
c = cvpartition(Y, 'holdout',1/3);
trX = X(c.training,:);
trY = Y(c.training);
tsX = X(c.test,:);
tsY = Y(c.test);
%# prediction
K = 10;
pred = zeros(c.TestSize,1);
for i=1:c.TestSize
pred(i) = KNNpredict(trX, trY, K, tsX(i,:));
end
%# validation
C = confusionmat(tsY, pred)
The confusion matrix of the kNN prediction with K=10:
C =
17 0 0
0 16 0
0 1 16