I am programming a simple perceptron in matlab but it is not converging and I can't figure out why.
The goal is to binary classify 2D points.
%P1 Generate a dataset of two-dimensional points, and choose a random line in
%the plane as your target function f, where one side of the line maps to +1 and
%the other side to -1. Let the inputs xn 2 R2 be random points in the plane,
%and evaluate the target function f on each xn to get the corresponding output
%yn = f(xn).
clear all;
clc
clear
n = 20;
inputSize = 2; %number of inputs
dataset = generateRandom2DPointsDataset(n)';
[f , m , b] = targetFunction();
signs = classify(dataset,m,b);
weights=ones(1,2)*0.1;
threshold = 0;
fprintf('weights before:%d,%d\n',weights);
mistakes = 1;
numIterations = 0;
figure;
plotpv(dataset',(signs+1)/2);%mapping signs from -1:1 to 0:1 in order to use plotpv
hold on;
line(f(1,:),f(2,:));
pause(1)
while true
mistakes = 0;
for i = 1:n
if dataset(i,:)*weights' > threshold
result = 1;
else
result = -1;
end
error = signs(i) - result;
if error ~= 0
mistakes = mistakes + 1;
for j = 1:inputSize
weights(j) = weights(j) + error*dataset(i,j);
end
end
numIterations = numIterations + 1
end
if mistakes == 0
break
end
end
fprintf('weights after:%d,%d\n',weights);
random points and signs are fine since plotpv is working well
The code is based on that http://es.mathworks.com/matlabcentral/fileexchange/32949-a-perceptron-learns-to-perform-a-binary-nand-function?focused=5200056&tab=function.
When I pause the infinite loop, this is the status of my vairables:
I am not able to see why it is not converging.
Additional code( it is fine, just to avoid answers asking for that )
function [f,m,b] = targetFunction()
f = rand(2,2);
f(1,1) = 0;
f(1,2) = 1;
m = (f(2,2) - f(2,1));
b = f(2,1);
end
function dataset = generateRandom2DPointsDataset(n)
dataset = rand(2,n);
end
function values = classify(dataset,m,b)
for i=1:size(dataset,1)
y = m*dataset(i,1) + b;
if dataset(i,2) >= y, values(i) = 1;
else values(i) = -1;
end
end
end
Related
I am creating a Forward Propagation In the feedforward step, an input pattern is propagated through the network to obtain an output. I have written this in pseudo code and currently attempting to implement this within MATLAB.
There are two errors I currently receive.
Patterns = x'; Desired = y; NHIDDENS = 1; prnout=Desired;
% Patterns become x so number of inputs becomes size of patterns
[NINPUTS,NPATS] = size(Patterns); [NOUTPUTS,NP] = size(Desired);
%apply the backprop here...
LearnRate = 0.15; Momentum = 0; DerivIncr = 0; deltaW1 = 0; deltaW2 = 0;
% Keeps the tan ordering of the examples of x
Inputs1= [Patterns;ones(1,NPATS)]; %Inputs1 = [ones(1,NPATS); Patterns];
% Weight initialisation
Weights1 = 0.5*(rand(NHIDDENS,1+NINPUTS)-0.5);
Weights2 = 0.5*(rand(1,1+NHIDDENS)-0.5);
TSS_Limit = 0.02;
for epoch = 1:10
% FORWARD LOOP
size(NOUTPUTS)
size(NPATS)
for ii = 0: ii < length(NINPUTS)
NOUTPUTS(ii+1) = NPATS(ii);
% Sets bias to 1
NOUTPUTS(1) = 1;
end
for ii = NHIDDENS: ii < NINPUTS
sum = 0;
for ij = 0: ij < ii
sum = sum + deltaW1(ii,ij) * NOUTPUTS(ij);
NOUTPUTS(ii) = tanh(sum);
end
end
Unable to perform assignment because the
left and right sides have a different
number of elements.
Error in mlpts (line 66)
NOUTPUTS(i+1) = NPATS(i);
i am still new to MATLAB and trying to become use to it.
After iterating through the loop
NOUTPUTS = 0 and the error is displayed. I am confused as I am trying to increment NOUTPUTS with ii by 1 through each loop.
I have been able to create the forward propagation with a loop.
for i =3:NNODES
summ = 0;
for j=1:i-1
summ = summ + weights(i,j) * Node_outputs(j);
end
if i == NNODES
Node_outputs(i) = summ
else
Node_outputs(i) = sigmoid(summ);
end
end
Out = Node_outputs(NNODES);
% BOut = ((Node_outputs(NNODES)) * (1 - Node_outputs));
BOut=zeros(1,6);
DeltaWeight = zeros(6,6);
For my studies I had to write a PDE solver for the Poisson equation on a disc shaped domain using the finite difference method.
I already passed the Lab exercise. There is one issue in my code I couldn't fix. Function fun1 with the boundary value problem gun2 is somehow oscillating at the boundary. When I use fun2 everything seems fine...
Both functions use at the boundary gun2. What is the problem?
function z = fun1(x,y)
r = sqrt(x.^2+y.^2);
z = zeros(size(x));
if( r < 0.25)
z = -10^8*exp(1./(r.^2-1/16));
end
end
function z = fun2(x,y)
z = 100*sin(2*pi*x).*sin(2*pi*y);
end
function z = gun2(x,y)
z = x.^2+y.^2;
end
function [u,A] = poisson2(funame,guname,M)
if nargin < 3
M = 50;
end
%Mesh Grid Generation
h = 2/(M + 1);
x = -1:h:1;
y = -1:h:1;
[X,Y] = meshgrid(x,y);
CI = ((X.^2 +Y.^2) < 1);
%Boundary Elements
Sum= zeros(size(CI));
%Sum over the neighbours
for i = -1:1
Sum = Sum + circshift(CI,[i,0]) + circshift(CI,[0,i]) ;
end
%if sum of neighbours larger 3 -> inner note!
CI = (Sum > 3);
%else boundary
CB = (Sum < 3 & Sum ~= 0);
Sum= zeros(size(CI));
%Sum over the boundary neighbour nodes....
for i = -1:1
Sum = Sum + circshift(CB,[i,0]) + circshift(CB,[0,i]);
end
%If the sum is equal 2 -> Diagonal boundary
CB = CB + (Sum == 2 & CB == 0 & CI == 0);
%Converting X Y to polar coordinates
Phi = atan(Y./X);
%Converting Phi R back to cartesian coordinates, only at the boundarys
for j = 1:M+2
for i = 1:M+2
if (CB(i,j)~=0)
if j > (M+2)/2
sig = 1;
else
sig = -1;
end
X(i,j) = sig*1*cos(Phi(i,j));
Y(i,j) = sig*1*sin(Phi(i,j));
end
end
end
%Numberize the internal notes u1,u2,......,un
CI = CI.*reshape(cumsum(CI(:)),size(CI));
%Number of internal notes
Ni = nnz(CI);
f = zeros(Ni,1);
k = 1;
A = spalloc(Ni,Ni,5*Ni);
%Create matix A!
for j=2:M+1
for i =2:M+1
if(CI(i,j) ~= 0)
hN = h;hS = h; hW = h; hE = h;
f(k) = fun(X(i,j),Y(i,j));
if(CB(i+1,j) ~= 0)
hN = abs(1-sqrt(X(i,j)^2+Y(i,j)^2));
f(k) = f(k) + gun(X(i,j),Y(i+1,j))*2/(hN^2+hN*h);
A(k,CI(i-1,j)) = -2/(h^2+h*hN);
else
if(CB(i-1,j) ~= 0) %in negative y is a boundry
hS = abs(1-sqrt(X(i,j)^2+Y(i,j)^2));
f(k) = f(k) + gun(X(i,j),Y(i-1,j))*2/(hS^2+h*hS);
A(k,CI(i+1,j)) = -2/(h^2+h*hS);
else
A(k,CI(i-1,j)) = -1/h^2;
A(k,CI(i+1,j)) = -1/h^2;
end
end
if(CB(i,j+1) ~= 0)
hE = abs(1-sqrt(X(i,j)^2+Y(i,j)^2));
f(k) = f(k) + gun(X(i,j+1),Y(i,j))*2/(hE^2+hE*h);
A(k,CI(i,j-1)) = -2/(h^2+h*hE);
else
if(CB(i,j-1) ~= 0)
hW = abs(1-sqrt(X(i,j)^2+Y(i,j)^2));
f(k) = f(k) + gun(X(i,j-1),Y(i,j))*2/(hW^2+h*hW);
A(k,CI(i,j+1)) = -2/(h^2+h*hW);
else
A(k,CI(i,j-1)) = -1/h^2;
A(k,CI(i,j+1)) = -1/h^2;
end
end
A(k,k) = (2/(hE*hW)+2/(hN*hS));
k = k + 1;
end
end
end
%Solve linear system
u = A\f;
U = zeros(M+2,M+2);
p = 1;
%re-arange u
for j = 1:M+2
for i = 1:M+2
if ( CI(i,j) ~= 0)
U(i,j) = u(p);
p = p+1;
else
if ( CB(i,j) ~= 0)
U(i,j) = gun(X(i,j),Y(i,j));
else
U(i,j) = NaN;
end
end
end
end
surf(X,Y,U);
end
I'm keeping this answer short for now, but may extend when the question contains more info.
My first guess is that what you are seeing is just numerical errors. Looking at the scales of the two graphs, the peaks in the first graph are relatively small compared to the signal in the second graph. Maybe there is a similar issue in the second that is just not visible because the signal is much bigger. You could try to increase the number of nodes and observe what happens with the result.
You should always expect to see numerical errors in such simulations. It's only a matter of trying to get their magnitude as small as possible (or as small as needed).
I'm writing the code in Matlab to find interest point using DoG in the image.
Here is the main.m:
imTest1 = rgb2gray(imread('1.jpg'));
imTest1 = double(imTest1);
sigma = 0.6;
k = 5;
thresh = 3;
[x1,y1,r1] = DoG(k,sigma,thresh,imTest1);
%get the interest points and show it on the image with its scale
figure(1);
imshow(imTest1,[]), hold on, scatter(y1,x1,r1,'r');
And the function DoG is:
function [x,y,r] = DoG(k,sigma,thresh,imTest)
x = []; y = []; r = [];
%suppose 5 levels of gaussian blur
for i = 1:k
g{i} = fspecial('gaussian',size(imTest),i*sigma);
end
%so 4 levels of DoG
for i = 1:k-1
d{i} = imfilter(imTest,g{i+1}-g{i});
end
%compare the current pixel in the image to the surrounding pixels (26 points),if it is the maxima/minima, this pixel will be a interest point
for i = 2:k-2
for m = 2:size(imTest,1)-1
for n = 2:size(imTest,2)-1
id = 1;
compare = zeros(1,27);
for ii = i-1:i+1
for mm = m-1:m+1
for nn = n-1:n+1
compare(id) = d{ii}(mm,nn);
id = id+1;
end
end
end
compare_max = max(compare);
compare_min = min(compare);
if (compare_max == d{i}(m,n) || compare_min == d{i}(m,n))
if (compare_min < -thresh || compare_max > thresh)
x = [x;m];
y = [y;n];
r = [r;abs(d{i}(m,n))];
end
end
end
end
end
end
So there's a gaussian function and the sigma i set is 0.6. After running the code, I find the position is not correct and the scales looks almost the same for all interest points. I think my code should work but actually the result is not. Anybody know what's the problem?
I am writing a graphical representation of numerical stability of differential operators and I am having trouble removing a nested for loop. The code loops through all entries in the X,Y, plane and calculates the stability value for each point. This is done by finding the roots of a polynomial of a size dependent on an input variable (length of input vector results in a polynomial 3d matrix of size(m,n,(lenght of input vector)). The main nested for loop is as follows.
for m = 1:length(z2)
for n = 1:length(z1)
pointpoly(1,:) = p(m,n,:);
r = roots(pointpoly);
if isempty(r),r=1e10;end
z(m,n) = max(abs(r));
end
end
The full code of an example numerical method (Trapezoidal Rule) is as follows. Any and all help is appreciated.
alpha = [-1 1];
beta = [.5 .5];
Wind = 2;
Wsize = 500;
if numel(Wind) == 1
Wind(4) = Wind(1);
Wind(3) = -Wind(1);
Wind(2) = Wind(4);
Wind(1) = Wind(3);
end
if numel(Wsize) == 1
Wsize(2) = Wsize;
end
z1 = linspace(Wind(1),Wind(2),Wsize(1));
z2 = linspace(Wind(3),Wind(4),Wsize(2));
[Z1,Z2] = meshgrid(z1,z2);
z = Z1+1i*Z2;
p = zeros(Wsize(2),Wsize(1),length(alpha));
for n = length(alpha):-1:1
p(:,:,(length(alpha)-n+1)) = alpha(n)-z*beta(n);
end
for m = 1:length(z2)
for n = 1:length(z1)
pointpoly(1,:) = p(m,n,:);
r = roots(pointpoly);
if isempty(r),r=1e10;end
z(m,n) = max(abs(r));
end
end
figure()
surf(Z1,Z2,z,'EdgeColor','None');
caxis([0 2])
cmap = jet(255);
cmap((127:129),:) = 0;
colormap(cmap)
view(2);
title(['Alpha Values (',num2str(alpha),') Beta Values (',num2str(beta),')'])
EDIT::
I was able to remove one of the for loops using the reshape command. So;
for m = 1:length(z2)
for n = 1:length(z1)
pointpoly(1,:) = p(m,n,:);
r = roots(pointpoly);
if isempty(r),r=1e10;end
z(m,n) = max(abs(r));
end
end
has now become
gg = reshape(p,[numel(p)/length(alpha) length(alpha)]);
r = zeros(numel(p)/length(alpha),1);
for n = 1:numel(p)/length(alpha)
temp = roots(gg(n,:));
if isempty(temp),temp = 0;end
r(n,1) = max(abs(temp));
end
z = reshape(r,[Wsize(2),Wsize(1)]);
This might be one for loop, but I am still going through the same number of elements. Is there a way to use the roots command on all of my rows at the same time?
I am a total beginner in Matlab and trying to write some Machine Learning Algorithms in Matlab. I would really appreciate it if someone can help me in debugging this code.
function y = KNNpredict(trX,trY,K,X)
% trX is NxD, trY is Nx1, K is 1x1 and X is 1xD
% we return a single value 'y' which is the predicted class
% TODO: write this function
% int[] distance = new int[N];
distances = zeroes(N, 1);
examples = zeroes(K, D+2);
i = 0;
% for(every row in trX) { // taking ONE example
for row=1:N,
examples(row,:) = trX(row,:);
%sum = 0.0;
%for(every col in this example) { // taking every feature of this example
for col=1:D,
% diff = compute squared difference between these points - (trX[row][col]-X[col])^2
diff =(trX(row,col)-X(col))^2;
sum += diff;
end % for
distances(row) = sqrt(sum);
examples(i:D+1) = distances(row);
examples(i:D+2) = trY(row:1);
end % for
% sort the examples based on their distances thus calculated
sortrows(examples, D+1);
% for(int i = 0; i < K; K++) {
% These are the nearest neighbors
pos = 0;
neg = 0;
res = 0;
for row=1:K,
if(examples(row,D+2 == -1))
neg = neg + 1;
else
pos = pos + 1;
%disp(distances(row));
end
end % for
if(pos > neg)
y = 1;
return;
else
y = -1;
return;
end
end
end
Thanks so much
When working with matrices in MATLAB, it is usually better to avoid excessive loops and instead use vectorized operations whenever possible. This will usually produce faster and shorter code.
In your case, the k-nearest neighbors algorithm is simple enough and can be well vectorized. Consider the following implementation:
function y = KNNpredict(trX, trY, K, x)
%# euclidean distance between instance x and every training instance
dist = sqrt( sum( bsxfun(#minus, trX, x).^2 , 2) );
%# sorting indices from smaller to larger distances
[~,ord] = sort(dist, 'ascend');
%# get the labels of the K nearest neighbors
kTrY = trY( ord(1:min(K,end)) );
%# majority class vote
y = mode(kTrY);
end
Here is an example to test it using the Fisher-Iris dataset:
%# load dataset (data + labels)
load fisheriris
X = meas;
Y = grp2idx(species);
%# partition the data into training/testing
c = cvpartition(Y, 'holdout',1/3);
trX = X(c.training,:);
trY = Y(c.training);
tsX = X(c.test,:);
tsY = Y(c.test);
%# prediction
K = 10;
pred = zeros(c.TestSize,1);
for i=1:c.TestSize
pred(i) = KNNpredict(trX, trY, K, tsX(i,:));
end
%# validation
C = confusionmat(tsY, pred)
The confusion matrix of the kNN prediction with K=10:
C =
17 0 0
0 16 0
0 1 16