I am stuck on implementation of the Hopfiled network with perceptron learning rule. The idea here is to learn the weights for a pattern (binary vector) using single-layer perceptron, and then perform associative memory task using standard Hopfield algorithm. However, after I try to recall a stored vector, the output does not converge to the correct pattern.
See the link, Section 13.4, for more detail on Perceptron implementation the code follows.
w = rand(1,21); %weights
d = [0 0 0 0 0 0]; %desired output
eta = .7; %learning rate
x =[1 1 0 0 0 0]; %memorized pattern
z = zeros(6, 21);
z(1,:) = [x(2) x(3) x(4) x(5) x(6) 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0 0];
z(2,:) = [x(1) 0 0 0 0 x(3) x(4) x(5) x(6) 0 0 0 0 0 0 0 -1 0 0 0 0];
z(3,:) = [0 x(1) 0 0 0 x(2) 0 0 0 x(4) x(5) x(6) 0 0 0 0 0 -1 0 0 0];
z(4,:) = [0 0 x(1) 0 0 0 x(2) 0 0 x(3) 0 0 x(5) x(6) 0 0 0 0 -1 0 0];
z(5,:) = [0 0 0 x(1) 0 0 0 x(2) 0 0 x(3) 0 x(4) 0 x(6) 0 0 0 0 -1 0];
z(6,:) = [0 0 0 0 x(1) 0 0 0 x(2) 0 0 x(3) 0 x(4) x(5) 0 0 0 0 0 -1];
for t = 1:100
index = randperm(6);
for i=1:6
Y(t,index(i)) = z(index(i),:)*w';
y(t,index(i)) = sgn(Y(t,index(i)));
w = w + eta*(d(index(i)) - y(t,index(i)))*z(index(i),:);
end
end
n = 6;
probe = input('Enter the probe vector: ');
signal_vector = 2*probe-1; % Convert probe to bipolar form
flag = 0; % Initialize flag
old = signal_vector; %save original input
while flag ~= 6; % test if old vector is same as new vector
index = randperm(n); %make sequence for asynchronous update
for j = 1:n
v =z(index(j),:)*w';
if v > 0
signal_vector(index(j)) = 1;
x(index(j)) = 1;
elseif v < 0
signal_vector(index(j)) = -1;
x(index(j)) = -1;
end
end
flag = signal_vector*old';
end
disp('The recalled vector is ')
0.5*(signal_vector + 1)
function y = sgn(x)
if x > 0
y = 1;
else
y = -1;
end
end
Related
I am working on a MATLAB code that involves deep learning using neural networks.
The image or the data is being fed in the form of matrices.
But I am getting an error "Matrix dimensions must agree".
Can Someone please help me with this problem?
I tried to solve this problem by using .* in stead of matrix multiplication * but the method didn't work.
Function Deeplearningoriginal:
function [w1,w2,w3,w4] = Deeplearningoriginal(w1,w2,w3,w4,input_Image,correct_output)
alpha=0.01;
N=5;
for k = 1:N
input_Image = reshape( input_Image( :, :,k ),25 ,1);
input_of_hidden_layer1 = w1* input_Image;
output_of_hidden_layer1 = ReLU(input_of_hidden_layer1);
input_of_hidden_layer2 = w2* output_of_hidden_layer1;
output_of_hidden_layer2 = ReLU( input_of_hidden_layer2);
input_of_hidden_layer3 = w3* output_of_hidden_layer2;
output_of_hidden_layer3 = ReLU(input_of_hidden_layer3);
input_of_output_node = w4* output_of_hidden_layer3;
final_output = Softmax(input_of_output_node);
correct_output_transpose = correct_output(k,:);
error = correct_output_transpose - final_output;
delta4 = error;
error_of_hidden_layer3 = w4'* delta4;
delta3 = (input_of_hidden_layer3>0).*error_of_hidden_layer3;
error_of_hidden_layer2 = w3'* delta3;
delta2 = (input_of_hidden_layer2>0).* error_of_hidden_layer2;
error_of_hidden_layer1 = w2'*delta2;
delta1 = (input_of_hidden_layer1>0).* error_of_hidden_layer1;
adjustment_of_w4 = alpha*delta4*output_of_hidden_layer3';
adjustment_of_w3 = alpha*delta3*output_of_hidden_layer2';
adjustment_of_w2 = alpha*delta2*output_of_hidden_layer1';
adjustment_of_w1 = alpha*delta1*reshaped_input_image';
w1 = w1 + adjustment_of_w1;
w2 = w2 + adjustment_of_w2;
w3 = w3 + adjustment_of_w3;
w4 = w4 + adjustment_of_w4;
end
end
Training network:
input_Image = zeros (5,5,5);
input_Image(:,:,1) = [ 1 0 0 1 1;
1 1 0 1 1;
1 1 0 1 1;
1 1 0 1 1;
1 0 0 0 1;
];
input_Image(:,:,2) = [ 0 0 0 0 1;
1 1 1 1 0;
1 0 0 0 1;
0 1 1 1 1;
0 0 0 0 0;
];
input_Image(:,:,3) = [ 0 0 0 0 1;
1 1 0 0 1;
1 0 1 0 1;
0 0 0 0 0;
1 1 1 0 1;
];
input_Image(:,:,4) = [ 1 1 1 0 1;
1 1 0 0 1;
1 0 1 0 1;
0 0 0 0 0;
1 1 1 0 1;
];
input_Image(:,:,5) = [ 0 0 0 0 0;
0 1 1 1 1;
0 0 0 0 1;
1 1 1 1 0;
0 0 0 0 1;
];
correct_output = [ 1 0 0 0 0;
0 1 0 0 0;
0 0 1 0 0;
0 0 0 1 0;
0 0 0 0 1;
];
w1 = 2* rand(20,25) -1;
w2 = 2* rand(20,20) -1;
w3 = 2* rand(20,20) -1;
w4 = 2* rand(5,20) -1;
for epoch = 1:100
[w1,w2,w3,w4] = Deeplearningoriginal(w1,w2,w3,w4,input_Image,correct_output);
end
I expected this code to run but because of the error I am not able to proceed.
The problem is the reshape (actually, two problems). After the
input_image = reshape(input_image(:,:,k), 25,1);
input_image is an array with 25 rows and 1 column, whereas w2, w3, and w4 have only 20 columns. To do the matrix multiplication A*B, A must have as many columns as B has rows.
The other problem with the reshape as written is that after the first pass through the loop, input_image is no longer a 5x5x5 array, it is a 25x1 array that contains only the elements of input_image(:,:,1). It is necessary to use a different name on the left-hand-side of the assignment (and throughout the rest of the loop) to avoid loosing the content of input_image.
Hope this helps,
JAC
I am trying to make a parity check matrix from non-systematic to systematic. Hence, I am attaching my code below. Somewhat it is correct, but there are some problems. It would be really great if someone could help me in this.
Subject: Information theory and coding. I am working on LDPC coding and decoding. Please check the code below
MATLAB CODE:
H = [1 0 1 1 0; 0 0 1 0 1; 1 0 0 1 0; 1 0 1 1 1]
[m,n] = size(H);
k = n-m;
for i = k+1:n
%H(:,i)
ind = find(H(:,i),1,'last');
% exchanging (ind)th row and (i-k)th row
if ind < i-k
continue;
end
if ind ~= i-k
temp = H(ind,:);
H(ind,:) = H(i-k,:);
H(i-k,:) = temp;
end
I = find(H(:,i));
% Guassian elimination
for j = 1:length(I)
if I(j) ~= i-k
H(I(j),:) = mod(H(I(j),:)+H(i-k,:),2);
end
end
end
Hsys = H
For e.g.
This is my H matrix:
H =
1 0 1 1 0
0 0 1 0 1
1 0 0 1 0
1 0 1 1 1
I want to have an identity matrix inside the matrix. The dimension on H matrix here is (mxn) which is (4x5).
Generally we use Gaussian elimination method to make the Identity matrix.hence, we make operations between rows. This is how we make it systematic.
I should have matrix as this in the result:
Hsys =
0 1 0 0 0
0 0 1 0 0
1 0 0 1 0
0 0 0 0 1
I should have an identity matrix of dimension m.
Here is how I'd do it (using Gauss-Jordan elimination):
% Not your matrix since it does not have any ones in the second column.
H=[1 1 0 1 1 0 0 1 0 0;
0 1 1 0 1 1 1 0 0 0;
0 0 0 1 0 0 0 1 1 1;
1 1 0 0 0 1 1 0 1 0;
0 0 1 0 0 1 0 1 0 1];
rows = size(H, 1);
cols = size(H, 2);
r = 1;
for c = cols - rows + 1:cols
if H(r,c) == 0
% Swap needed
for r2 = r + 1:rows
if H(r2,c) ~= 0
tmp = H(r, :);
H(r, :) = H(r2, :);
H(r2, :) = tmp;
end
end
end
% Ups...
if H(r,c) == 0
error('H is singular');
end
% Forward substitute
for r2 = r + 1:rows
if H(r2, c) == 1
H(r2, :) = xor(H(r2, :), H(r, :));
end
end
% Back Substitution
for r2 = 1:r - 1
if H(r2, c) == 1
H(r2, :) = xor(H(r2, :), H(r, :));
end
end
% Next row
r = r + 1;
end
I have a 6X4 matrix M1 containing only zeros.
I also have two 1D arrays Y1 and Y2 each with length 4.The two arrays contain the desired index values. Now, I want to set(convert to 1) the elements of matrix M1 such that
M1(Y1:Y2) is equal to 1
for ex: Y1=[1 2 2 1] and Y2=[3 4 5 3]
then, M1 should be
1 0 0 1
1 1 1 1
1 1 1 1
0 1 1 0
0 0 1 0
0 0 0 0
I can do this using for loop. But is there any optimised way to do it? (I intend to use much bigger matrices)
use cumsum!
>> szM = size(M1);
>> M1( sub2ind( szM, Y1, 1:szM(2) ) ) = 1
M1 =
1 0 0 1
0 1 1 0
0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
>> M1( sub2ind( szM, Y2+1, 1:szM(2) ) ) = -1
M1 =
1 0 0 1
0 1 1 0
0 0 0 0
-1 0 0 -1
0 -1 0 0
0 0 -1 0
>> M = cumsum(M,1)
M =
1 0 0 1
1 1 1 1
1 1 1 1
0 1 1 0
0 0 1 0
0 0 0 0
A pitfall:
If any of Y2 equals 6 than setting Y2+1 to -1 will exceed matrix dimension.
To fix this you can add two lines before setting to -1 the elements of M:
>> cols = 1:szM(2);
>> sel = Y2 < szM(1);
>> M1( sub2ind( szM, Y2(sel)+1, cols(sel) ) ) = -1
A spin-off for Pavan Yalamanchili's answer using bsxfun: (hover to see:)
using bsxfun without offsets:
M1 = bsxfun( #ge, (1:size(M1,1))', Y1 ) & bsxfun( #le, (1:size(M1,1))', Y2 );
There may be other techniques, but this uses element wise operations which are insanely parallel.
A very simple solution. Thanks #Shai
>> [rows, cols] = size(M);
>> Y1=[1 2 2 1]; Y2=[3 4 5 3];
>> M = bsxfun(#ge, (1:rows)', Y1) & bsxfun(#le, (1:rows)', Y2)
M =
1 0 0 1
1 1 1 1
1 1 1 1
0 1 1 0
0 0 1 0
0 0 0 0
Unnecessarily complicated code
[rows, cols] = size(M);
offsets = ((1 : cols) - 1) * rows
Y1 = offsets + Y1;
Y2 = offsets + Y2;
M = reshape(1:numel(M), rows, cols);
M = bsxfun(#ge, M, Y1) & bsxfun(#le, M, Y2);
I have a matrix m = zeros(1000, 1000). Within this matrix I want to draw an estimate of the line which passes through 2 points from my matrix. Let's say x = [122 455]; and y = [500 500];.
How can I do this in Matlab? Are there any predefined functions to do this? I am using Matlab 2012b.
I'll denote the two endpoints as p1 and p2 because I'm planning to use x and y for something else. I'm also assuming that the first coordinate of p1 and p2 is x and the second is y. So here's a rather simple way to do it:
Obtain the equation of the line y = ax + b. In MATLAB, this can be done by:
x = p1(1):p2(1)
dx = p2(1) - p1(1);
dy = p2(2) - p1(2);
y = round((x - p1(1)) * dy / dx + p1(2));
Convert the values of x and y to indices of elements in the matrix, and set those elements to 1.
idx = sub2ind(size(m), y, x);
m(idx) = 1;
Example
Here's an example for a small 10-by-10 matrix:
%// This is our initial conditon
m = zeros(10);
p1 = [1, 4];
p2 = [5, 7];
%// Ensure the new x-dimension has the largest displacement
[max_delta, ix] = max(abs(p2 - p1));
iy = length(p1) - ix + 1;
%// Draw a line from p1 to p2 on matrix m
x = p1(ix):p2(ix);
y = round((x - p1(ix)) * (p2(iy) - p1(iy)) / (p2(ix) - p1(ix)) + p1(iy));
m(sub2ind(size(m), y, x)) = 1;
m = shiftdim(m, ix > iy); %// Transpose result if necessary
The result is:
m =
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 1 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Update: I have patched this algorithm to work when dy > dx by treating the dimension with the largest displacement as if it were the x-dimension, and then transposing the result if necessary.
Neither of the provided answers work for displacements in y greater than in x (dy > dx).
As pointed out, Bresenham's line algorithm is exactly meant for that.
The matlab file provided here works similarly than the examples provided in the other answers but covers all the use-cases.
To relate to the previously provided example, the script can be used like this:
% initial conditions
m = zeros(10);
p1 = [1, 4];
p2 = [5, 10];% note dy > dx
% use file provided on file exchange
[x y] = bresenham(p1(1),p1(2),p2(1),p2(2));
% replace entries in matrix m
m(sub2ind(size(m), y, x)) = 1;
result looks like this:
m =
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 1 0 0 0 0 0 0 0 0
0 0 1 0 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0 0
0 0 0 0 1 0 0 0 0 0
For me (matlab R2013b) following line did not work, when p1(1)>p2(2) (":" can not count backwards):
x = p1(1):p2(1);
E.G.:
1:10
1 2 3 4 5 6 7 8 9 10
10:1
Empty matrix: 1-by-0
But it worked when I used linspac instead:
x = linspace(p1(1), p2(1), abs(p2(1)-p1(1))+1);
I am writing an algorithm in matlab to for bicubic interpolation of a surface Psi(x,y). I have a bug in the code and cannot seem to track it down. I am trying a test case with Psi=X^2-0.25 so that its easier to track down the bug. It seems as if my interpolation has an offset. My comments are included in my code. Any help would be appreciated.
Plot of Psi=X^2 in blue and interpolation in red
Countour lines of Psi are plotted and the red dot is the point I am computing the interpolation about. The thick red line is the interpolation which is offset quite a bit from the red dot.
function main()
epsilon=0.000001;
xMin=-1+epsilon;
xMax= 1+epsilon;
yMin=-1+epsilon;
yMax= 1+epsilon;
dx=0.1; Nx=ceil((xMax-xMin)/dx)+1;
dy=0.1; Ny=ceil((yMax-yMin)/dy)+1;
x=xMin:dx:xMax; x=x(1:Nx);
y=yMin:dy:yMax; y=y(1:Ny);
[XPolInX,XPolInY]=GetGhostMatricies(Nx,Ny); %Linear extrapolation matrix
[D0x,D0y]=GetDiffMatricies(Nx,Ny,dx,dy); %derivative matricies: D0x is central differencing in x
[X,Y]=meshgrid(x,y);
Psi=X.^2-0.25; %Note that my algorithm is being written for a Psi that may not have a analytic representation. This Psi is only a test case.
psi=zeros(Nx+2,Ny+2); %linearly extrapolate psi (for solving differential equation not shown here)
psi(2:(Nx+1),2:(Ny+1))=Psi';
psi=(XPolInY*(XPolInX*psi)')';
%compute derivatives of psi
psi_x =D0x*psi; psi_x =(XPolInY*(XPolInX*psi_x)')';
psi_y =(D0y*psi')'; psi_y =(XPolInY*(XPolInX*psi_y)')';
psi_xy=D0x*psi_y; psi_xy=(XPolInY*(XPolInX*psi_xy)')';
% i have verified that my derivatives are computed correctly
biCubInv=GetBiCubicInverse(dx,dy);
i=5; %lets compute the bicubic interpolation at this x(i), y(j)
j=1;
psiVoxel=[psi( i,j),psi( i+1,j),psi( i,j+1),psi( i+1,j+1),...
psi_x( i,j),psi_x( i+1,j),psi_x( i,j+1),psi_x( i+1,j+1),...
psi_y( i,j),psi_y( i+1,j),psi_y( i,j+1),psi_y( i+1,j+1),...
psi_xy(i,j),psi_xy(i+1,j),psi_xy(i,j+1),psi_xy(i+1,j+1)]';
a=biCubInv*psiVoxel; %a=[a00 a01 ... a33]; polynomial coefficients; 1st index is power of (x-xi), 2nd index is power of (y-yj)
xi=x(5); yj=y(1);
clear x y
x=(xi-.2):.01:(xi+.2); %this is a local region about the point we are interpolating
y=(yj-.2):.01:(yj+.2);
[dX,dY]=meshgrid(x,y);
Psi=dX.^2-0.25;
figure(2) %just plotting the 0 level contour of Psi here
plot(xi,yj,'.r','MarkerSize',20)
hold on
contour(x,y,Psi,[0 0],'r','LineWidth',2)
set(gca,'FontSize',14)
axis([x(1) x(end) y(1) y(end)])
grid on
set(gca,'xtick',(xi-.2):.1:(xi+.2));
set(gca,'ytick',(yj-.2):.1:(yj+.2));
xlabel('x')
ylabel('y')
[dX dY]=meshgrid(x-xi,y-yj);
%P is my interpolating polynomial
P = a(1) + a(5) *dY + a(9) *dY.^2 + a(13) *dY.^3 ...
+ a(2)*dX + a(6)*dX .*dY + a(10)*dX .*dY.^2 + a(14)*dX .*dY.^3 ...
+ a(3)*dX.^2 + a(7)*dX.^2.*dY + a(11)*dX.^2.*dY.^2 + a(15)*dX.^2.*dY.^3 ...
+ a(4)*dX.^3 + a(8)*dX.^3.*dY + a(12)*dX.^3.*dY.^2 + a(16)*dX.^3.*dY.^3 ;
[c h]=contour(x,y,P)
clabel(c,h)
figure(3)
plot(x,x.^2-.25) %this is the exact function
hold on
plot(x,P(1,:),'-r*')
%See there is some offset here
end
%-------------------------------------------------------------------------
function [XPolInX,XPolInY]=GetGhostMatricies(Nx,Ny)
XPolInX=diag(ones(1,Nx+2),0);
XPolInY=diag(ones(1,Ny+2),0);
XPolInX(1,1) =0; XPolInX(1,2) =2; XPolInX(1,3) =-1;
XPolInY(1,1) =0; XPolInY(1,2) =2; XPolInY(1,3) =-1;
XPolInX(Nx+2,Nx+2)=0; XPolInX(Nx+2,Nx+1)=2; XPolInX(Nx+2,Nx)=-1;
XPolInY(Ny+2,Ny+2)=0; XPolInY(Ny+2,Ny+1)=2; XPolInY(Ny+2,Ny)=-1;
fprintf('Done GetGhostMatricies\n')
end
%-------------------------------------------------------------------------
function [D0x,D0y]=GetDiffMatricies(Nx,Ny,dx,dy)
D0x=diag(ones(1,Nx-1),1)-diag(ones(1,Nx-1),-1);
D0y=diag(ones(1,Ny-1),1)-diag(ones(1,Ny-1),-1);
D0x(1,1)=-3; D0x(1,2)=4; D0x(1,3)=-1;
D0y(1,1)=-3; D0y(1,2)=4; D0y(1,3)=-1;
D0x(Nx,Nx)=3; D0x(Nx,Nx-1)=-4; D0x(Nx,Nx-2)=1;
D0y(Ny,Ny)=3; D0y(Ny,Ny-1)=-4; D0y(Ny,Ny-2)=1;
%pad with ghost cells which are simply zeros
tmp=D0x; D0x=zeros(Nx+2,Nx+2); D0x(2:(Nx+1),2:(Nx+1))=tmp; tmp=0;
tmp=D0y; D0y=zeros(Ny+2,Ny+2); D0y(2:(Ny+1),2:(Ny+1))=tmp; tmp=0;
%scale appropriatley by dx & dy
D0x=D0x/(2*dx);
D0y=D0y/(2*dy);
end
%-------------------------------------------------------------------------
function biCubInv=GetBiCubicInverse(dx,dy)
%p(x,y)=a00+a01(x-xi)+a02(x-xi)^2+...+a33(x-xi)^3(y-yj)^3
%biCubic*a=[psi(i,j) psi(i+1,j) psi(i,j+1) psi(i+1,j+1) psi_x(i,j) ... psi_y(i,j) ... psi_xy(i,j) ... psi_xy(i+1,j+1)]
%here, psi_x is the x derivative of psi
%I verified that this matrix is correct by setting dx=dy=1 and comparing to the inverse here http://en.wikipedia.org/wiki/Bicubic_interpolation
biCubic=[
%00 10 20 30 01 11 21 31 02 12 22 32 03 13 23 33
1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0;
1 dx dx^2 dx^3 0 0 0 0 0 0 0 0 0 0 0 0;
1 0 0 0 dy 0 0 0 dy^2 0 0 0 dy^3 0 0 0;
1 dx dx^2 dx^3 dy dx*dy dx^2*dy dx^3*dy dy^2 dx*dy^2 dx^2*dy^2 dx^3*dy^2 dy^3 dx*dy^3 dx^2*dy^3 dx^3*dy^3;
0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0;
0 1 2*dx 3*dx^2 0 0 0 0 0 0 0 0 0 0 0 0;
0 1 0 0 0 dy 0 0 0 dy^2 0 0 0 dy^3 0 0;
0 1 2*dx 3*dx^2 0 dy 2*dx*dy 3*dx^2*dy 0 dy^2 2*dx*dy^2 3*dx^2*dy^2 0 dy^3 2*dx*dy^3 3*dx^2*dy^3;
0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0;
0 0 0 0 1 dx dx^2 dx^3 0 0 0 0 0 0 0 0;
0 0 0 0 1 0 0 0 2*dy 0 0 0 3*dy^2 0 0 0;
0 0 0 0 1 dx dx^2 dx^3 2*dy 2*dx*dy 2*dx^2*dy 2*dx^3*dy 3*dy^2 3*dx*dy^2 3*dx^2*dy^2 3*dx^3*dy^2;
0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0;
0 0 0 0 0 1 2*dx 3*dx^2 0 0 0 0 0 0 0 0;
0 0 0 0 0 1 0 0 0 2*dy 0 0 0 3*dy^2 0 0;
0 0 0 0 0 1 2*dx 3*dx^2 0 2*dy 4*dx*dy 6*dx^2*dy 0 3*dy^2 6*dx*dy^2 9*dx^2*dy^2];
biCubInv=inv(biCubic);
end
%-------------------------------------------------------------------------
I found my error. I pad my matricies with ghost cells, however I forget that now the i in Psi without ghost cells is i+1 in psi with ghost cells. Hence, I should be evaluating my interpolating polynomial P at xi=x(6); yj=y(2), not xi=x(5); yj=y(1).