I'm trying to implement a neural network with sigmoid function
But the following code doesnot work
This is the training part of neural network.
It doesnot update the weights properly
What is wrong in this code?
clc; clear all; close all;
% load train_data1
train_data1=[-0.498800000000000,-0.257500000000000;-0.492800000000000,-0.274300000000000;-0.470300000000000,-0.282600000000000;-0.427400000000000,-0.474000000000000;-0.420400000000000,-0.518000000000000;-0.326300000000000,-1.13230000000000;-0.317300000000000,-0.875300000000000;-0.295000000000000,-1.02770000000000;-0.267600000000000,-0.882800000000000;-0.260500000000000,-0.976500000000000;-0.216100000000000,-0.970400000000000;-0.207000000000000,-0.813800000000000;-0.164000000000000,-0.696600000000000;-0.159900000000000,-0.793300000000000;-0.122000000000000,-0.764400000000000;-0.0729000000000000,-0.435300000000000;-0.00640000000000000,-0.0546000000000000;0.132200000000000,0.710300000000000;0.137100000000000,0.587000000000000;0.160300000000000,0.819200000000000;0.230600000000000,0.989200000000000;0.286800000000000,0.737700000000000;0.334000000000000,0.943500000000000;0.375200000000000,0.688200000000000;0.429700000000000,0.567800000000000];
train_data1 = sortrows(train_data1);
% normalize data to [0,1]
data1=[train_data1];
max1=max(max(data1));
min1=min(min(data1));
train_data2 = (train_data1 - min1) / ( max1 - min1);
x = train_data2(:,1); % train input data
r = train_data2(:,2); % train output data
hidden_neurons = 2;
maxepochcount = 1000;
datacount1 = size(x,1);
% add a bias as an input
bias = ones(datacount1,1);
% x = [x bias];
% read how many inputs
inputcount = size(x,2);
% ---------- data loaded -----------
% ---------- set weights -----------
% set initial random weights
WI = (randn(inputcount,hidden_neurons) - 0.5)/10;
WO = (randn(1,hidden_neurons) - 0.5)/10;
%-----------------------------------
%--- Learning Starts Here! ---------
%-----------------------------------
eta1 = 0.5;
eta2 = eta1/5;
% do a number of epochs
for iter = 1:maxepochcount
% loop through the data
for j = 1:datacount1
% read the current sample
I = x(j,:);
D = r(j,1);
% calculate the error for this sample
H = (sigmoid(I * WI))';
O = H' * WO';
error = D-O;
% adjust weight between hidden & output
delta_i = O.*(1-O).*(D-O); % D actual, O calculated output
% Calculate error for each node in layer_(n-1)
delta_j = H.*(1-H).*(WO.'*delta_i); % H.' is the output of hidden layer
% Adjust weights in matrices sequentially
WO = WO + eta2.*delta_i*(H.') % H.' is the output of hidden layer
WI = WI + eta1.*(delta_j*(I))' % I.' is the inputs
% % adjust weight between hidden & output
% delta_HO = error.*eta2 .* hidden_val;
% WO = WO - delta_HO';
% % adjust the weights between input & hidden
% delta_IH = eta1 .* error .* WO' .* (1 - (H .^ 2)) * I;
% WI = WI - delta_IH';
end
O = sigmoid(WO*sigmoid(x * WI)');
% error(iter) = (sum(error .^ 2)) ^ 0.5;
if rem(iter,100)==0 % Every 100 epochs, show how training is doing
plot(x,O, 'color','red','linewidth',2); hold on;
drawnow;
iter
end
% return
end
only the output values are needed to be scaled to the activation function.
If we use tanh we must scale them to [-1,1], in case of sigmoid [0,1].
The code is working fine but sometimes it needs more epochs.
Related
I am trying to calculate the received optical power for an optical communication system for 30 times. When I run the code, I get a 41 x 41 double matrix of H0_LoS and power (P_r_LOS & P_rec_dBm) as shown below
The case above is only for one iteration.
I would like to get a matrix or a cell with results up to 30 times that of the implemented code.
Any assistance, please?
My try below
close all;
clear variables;
clc;
%% Simulation Parameters
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%% Main Simulation Parameters %%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%-------------------------%
% NUMBER OF LIGHT SOURCES %
%-------------------------%
N_t = 1; % Number of light sources
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%% AP Parameters %%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%------------------------%
% LIGHT SOURCES GEOMETRY %
%------------------------%
L = 20; W = 20; H = 3; % Length, width and height of the room (m)
theta_half = 60;
m = -log(2)./log(cosd(theta_half)); % Lambertian order of emission
coord_t = [0 0 0]; % Positions of the light sources
n_t_LED = [0, 0, -1]; n_t_LED = n_t_LED/norm(n_t_LED); % Normalized normal vector of each light source
n_t = repmat(n_t_LED, N_t, 1); % Normalized normal vectors of the light sources
%-------------------------------------%
% LIGHT SOURCES ELECTRICAL PARAMETERS %
%-------------------------------------%
P_LED = 2.84; % Average electrical power consummed by each light source (W)
% P_LED = 1;
param_t = {coord_t, n_t, P_LED, m};
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%% Rx Parameters %%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%--------------------------%
% PHOTODETECTOR PARAMETERS %
%--------------------------%
A_det = 1e-4; % Photoreceiver sensitive area (m²)
FOV = 60*pi/180; % Fielf-of-view of the photoreceiver
T_s = 1; % Gain of the optical filter (ignore if not used)
index = 1.5; % Refractive index of the Rx concentrator/lens (ignore if not used)
G_Con = (index^2)/sin(FOV); % Gain of an optical concentrator; ignore if no lens is used
n_r = [0, 0, 1]; % Normal vector of the photoreceiver
n_r = n_r/norm(n_r); % Normal vector of the photoreceiver (normalized)
%---------------------------%
% RECEIVER PLANE PARAMETERS %
%---------------------------%
step = 0.5; % Distance between each receiving point (m)
X_r = -L/2:step:L/2; % Range of Rx points along x axis
Y_r = -W/2:step:W/2; % Range of Rx points along y axis
N_rx = length(X_r); N_ry = length(Y_r); % Number of reception points simulated along the x and y axis
z_ref = 0.85; % Height of the receiver plane from the ground (m)
z = z_ref-H; % z = -1.65; % Height of the Rx points ("-" because coordinates system origin at the center of the ceiling)
if( abs(z) > H )
fprintf('ERROR: The receiver plane is out of the room.\n');
return
end
param_r = {A_det, n_r, FOV}; % Vector of the Rx parameters used for channel simulation
%% LOS received optical power calculation
H0_LOS = zeros(N_rx,N_ry,N_t);
T = param_t{1}(1,:);
P_t = param_t{3};
for iter = 1:30
for r_x = 1:N_rx
for r_y = 1:N_ry
for i_t = 1:N_t
x = X_r(r_x); y = Y_r(r_y);
R = [x,y,z];
v_tr = (R-T)./norm(R-T);
d_tr = sqrt(dot(R-T,R-T));
phi = -5 + (5+5)*rand(1);
psi = -5 + (5+5)*rand(1);
H0_LOS(r_x,r_y,i_t) = (m+1)*A_det/(2*pi*d_tr^2)*cosd(phi)^m*cosd(psi);
end
end
end
end
P_r_LOS = P_t.*H0_LOS.*T_s.*G_Con;
P_rec_dBm = 10*log10(P_r_LOS*1000);
I have written a simple code to calculate the phase and magnitude of a signal, based on the sinusoidal input given in my problem. I have already determined the magnitude of the signal corresponding to different values of w. More specifically, the phase I want is a vector calculated for different values of w. Notice that the signal I'm talking about is the output signal of a linear process. As matter of fact, I want the phase difference between the input u and the output y, defined for all values of w for all time steps. I have created the time and w vector in my code. Here is the main code I have written in MATAB2021a:
clc;clear;close all;
%{
Problem 2 Simulation, By M.Sajjadi
%}
%% Predifined Parameters
tMin = 0;
tMax = 50;
Ts = 0.01; % Sample Time
n = tMax/Ts; % #Number of iterations
t = linspace(tMin,tMax,n);
% Hint: Frequency Domain
wMin = 10^-pi;
wMax = 10^pi;
Tw = 10;
w = wMin:Tw:wMax; % Vector of Frequency
Nw = length(w);
a1 = 1.8;
a2 = -0.95;
a3 = 0.13;
b1 = 1.3;
b2 = -0.5;
%% Input Generation
M = numel(w);
N = length(t);
U = zeros(M,N);
Y = U; % Response to the sinusoidal Input, Which means the initial conditions are set to ZERO.
U(1,:) = sin(w(1)*t);
U(2,:) = sin(w(2)*t);
U(3,:) = sin(w(3)*t);
Order = 3; % The Order of the Differential Equation, Delay.
%% Main Loop for Amplitude and Phase
Amplitude = zeros(Nw,1); % Amplitude Values
for i=1:numel(w)
U(i,:) = sin(w(i)*t);
for j=Order+1:numel(t)
Y(i,j) = a1*Y(i,j-1) + a2*Y(i,j-2) + a3*Y(i,j-3) + ...
b1*U(i,j-1) + b2*U(i,j-2);
end
Amplitude(i) = max(abs(Y(i,:)));
end
I know I should use fft or findpeaks function in MATLAB, but I do not know how I should do it.
I want to plot the step response. I know that I can use step function with state space equations, but I try to get same results using plot function. Here is my sample of code:
for i=1:201
u(i) = 1;
x1(i+1) = (-(b/J)*x1(i) + (K/J)*x2(i));
x2(i+1) = (-(K/L)*x1(i) - (R/L)*x2(i) + (1/L)*u(i));
y(i) = x1(i);
end
and this is the state space equations:
A = [-b/J K/J
-K/L -R/L];
B = [0
1/L];
C = [1 0];
D = 0;
If i do:
t = 0:1:200;
plot(t, y)
it is not working and I want to have the same results like the step function below:
sys = ss(A,B,C,D);
step(sys)
You can find my state space equation here.
The reason for the mismatch is that sys is a continuous time model, whereas the computation of y treats it as a discrete-time system.
The following is a way of estimating the step-response of a continuous-time system in the discrete-time domain:
% Given from the problem statement
A = [-b/J K/J
-K/L -R/L];
B = [0
1/L];
C = [1 0];
D = 0;
% this is your continuous-time model
sys = ss(A,B,C,D);
% define the sample rate of the equivalent discrete-time model
Ts = 1/10;
% this needs to be something smaller than the time-constants in your model,
% so that you have enough resolution to represent the continuous-time
% signal.
% convert the system to the equivalent discrete-time model
sysd = c2d(sys,Ts);
% define how long a step response you'd like to compute
T = 7;
% this should be long enough to cover the length of the step response
t = 0:Ts:T; % time-grid for the plot
nSmp = length(t); % total number of samples to be computed
% initializations
y = NaN(1, nSmp); % output vector
u = ones(1, nSmp); % unit step input
X = [0; 0]; % state vector, initialized to 0
% compute the samples of the step-response
% (i prefer to use vectorized form to keep the code concise)
for i=1:nSmp
y(i) = sysd.C * X + sysd.D * u(i);
X = sysd.A * X + sysd.B * u(i);
end
% plot continous-time step response
figure;
step(sys);
% plot simulated discrete-time step response
figure;
plot(t, y, 'r')
xlabel('Time (s)');
ylabel('Amplitude');
title('Simulated Step Response');
I want to plot multiple realizations of a stochastic process in matlab. For a single realization I have the following code:
N = 80;
T = dt*N;
dWt = zeros(1,N);
S= repmat(S0,1,N);
S(1) = S0;
dWt = sqrt(dt) * randn;
for t=2:N
dWt(t) = sqrt(dt)*randn;
dSt = k*(mu-S(t-1))*dt + sigma*S(t-1)*dWt(t);
S(t) = S(t-1)+dSt;
end
plot(handles.pMeasure, [0:dt:T],[S0,S]);
I want to replicate this loop n times and plot the results in one plot.
You could add an additional for loop, but it would be best to vectorize everything and calculate all n instances at once:
k = ...
mu = ...
sigma = ...
S0 = ... % Initial condition
dt = ... % Time step
n = ... % Number of instances
N = 80; % Number of time steps, not counting initial condition
T = dt*N; % Final time
rng(1); % Always seed random number generator
dWt = sigma*sqrt(dt)*randn(n,N); % Calculate Wiener increments
S = zeros(n,N+1); % Allocate
S(:,1) = S0; % Set initial conditions
for t = 2:N+1
S(:,t) = S(:,t-1) + k*(mu-S(:,t-1))*dt + S(:,t-1).*dWt(:,t-1);
end
plot(handles.pMeasure,0:dt:T,S)
There are further ways to optimize this if want or you can also try sde_euler in my SDETools Matlab toolbox:
k = ...
mu = ...
sigma = ...
dt = ... % Time step
n = ... % Number of instances
N = 80; % Number of time steps, not counting initial condition
T = dt*N; % Final time
f = #(t,y)k*(mu-y); % Diffusion function
g = #(t,y)sigma*y; % Drift function
t = 0:dt:T; % Time vector
S0 = zeros(n,1); % Initial conditions
opts = sdeset('RandSeed',1,...
'SDEType','Ito'); % Set random seed, specify Ito SDE
S = sde_euler(f,g,t,S0,opts); % Simulate
plot(t,S)
This is going to be a long question :
I have written a code in MATLAB for updating the weights of MLP with one hidden layer . Here is the code :
function [ weights_1,weights_2 ] = changeWeights( X,y,weights_1,weights_2,alpha )
%CHANGEWEIGHTS updates the weight matrices
% This function changes the weight of the weight matrix
% for a given value of alpha using the back propogation algortihm
m = size(X,1) ; % number of samples in the training set
for i = 1:m
% Performing the feed-forward step
X_i = [1 X(i,1:end)] ; % 1-by-5 input
z2_i = X_i*weights_1' ; % 1-by-4
a2_i = sigmoid(z2_i) ; % 1-by-4
a2_i = [1 a2_i] ; % 1-by-5
z3_i = a2_i*weights_2' ; % 1-by-3
h_i = sigmoid(z3_i) ; % 1-by-3
% Calculating the delta_output_layer
delta_output_layer = ( y(i)' - h_i' )...
.*sigmoidGradient(z3_i') ; % 3-by-1 matrix
% Calculating the delta_hidden_layer
delta_hidden_layer = (weights_2'*delta_output_layer)...
.*sigmoidGradient([1;z2_i']) ; % 5-by-1 matrix
delta_hidden_layer = delta_hidden_layer(2:end) ;
% Updating the weight matrices
weights_2 = weights_2 + alpha*delta_output_layer*a2_i ;
weights_1 = weights_1 + alpha*delta_hidden_layer*X_i ;
end
end
Now I wanted to test it on the fisheriris dataset given in MATLAB which can be accesed by load fisheriris command . I renamed meas to X and changed species to a 150-by-3 matrix where each row depicts the name of species (as for example first row is [1 0 0])
I compute error of the output layer using the following function :
function [ g ] = costFunction( X,y,weights_1,weights_2 )
%COST calculates the error
% This function calculates the error in the
% output of the neural network
% Performing the feed-forward propogation
m = size(X,1) ;
X_temp = [ones([m 1]) X] ; % 150-by-5 matrix
z2 = X_temp*weights_1' ; % 150-by-5-by-5-by-4
a2 = sigmoid(z2) ;
a2 = [ones([m 1]) a2] ; % 150-by-5
z3 = a2*weights_2' ; % 150-by-3
h = sigmoid(z3) ; % 150-by-3
g = 0.5*sum(sum((y-h).^2)) ;
g = g/m ;
end
Now in the course the prof gave an example of toy network with 3 iterations , I tested this on that network and it gives the right values but when I test it on the fisheriris data the cost keeps on increasing . And I am not able to understand where it is going wrong .