Training a single neuron perceptron on Matlab - matlab

I am required to train a single neuron perceptron which classifies two different classes.
I wrote this code on matlab.
function error = train_perc( PercepClassTraining,eta,weights )
%initialization:
xZero=1;
wZero=0.1;
t=1; % round number
x=[];
y=[];
error=[];
epoch_error=[];
c=1;
n=1;
x1=PercepClassTraining(n,1);
x2=PercepClassTraining(n,2);
y(n)=(xZero*wZero)+(x1*weights(1))+(x2*weights(2));
error(n)=PercepClassTraining(n,3)-y(n);
while(error(n)~=0)
weights(1)=weights(1)+(eta*error(n)*x1);
weights(2)=weights(2)+(eta*error(n)*x2);
if(n==2000)
epoch_error(c)=mean(error);
error=[];
c=c+1
n=1;
else
n=n+1;
end
y(n)=(xZero*wZero)+(x1*weights(1))+(x2*weights(2));
error(n)=PercepClassTraining(n,3)-y(n);
end
Where eta is learning rate, and PercepClassTraining is the training set (2000 inputs).
It converges at epoch #58 when I use 0.1 for learning rate and [0.01 0.01] for initial weights, and that doesn't use all the training set.
When I set eta to 0.01 and weights to 0.1, 0.1 it loops forever! Is there anything wrong in the code? How about the stopping criteria?

Related

How can find the Fourier Coefficents of this DFT?

I have a data which contains 16 elements:
x=[8.57837e-08, 2.07482e-06, 4.43796e-06, 7.66462e-06, 1.10232e-05, 1.35811e-05, 1.27958e-05, 5.94217e-06, 2.49168e-08, -6.58389e-06, -1.30551e-05, -1.345e-05, -1.07471e-05, -7.38637e-06, -4.42876e-06, -1.88811e-06 ];
A = length(x)
I do DTFT-DFT like dirac signals:
n=0:A;
syms w
X_w=0;
for i=1:length(x)
X_w=X_w+x(i)*exp(-j*w*n(i));
end
figure;fplot(angle(X_w),[0 2*pi]),title('DTFT phase graph')
figure;fplot(abs(X_w),[0 2*pi]),title('DTFT amplitude graph')
hold on
%DFT
N=50;
k=0:N-1;
DFT_X=[];
for k=0:N-1
Xk=0;
for i=1:length(x)
Xk=Xk+x(i).*exp(-j*(2*pi/N).*k.*n(i));
end
DFT_X=[DFT_X Xk];
end
w=2*pi/N*(0:N-1);
stem(w,abs(DFT_X))`
The problem is I want to write this signal with cosinus and sinus components. But I don't really know how can I do.
Thank you all,
Emre.
Direct computation of the Fourier coefficients might be a better option than trying to relate the DFT to the DFS. Looking at the continuous time formulas:
all you'd need to do is sum the input signal x multiplied (element wise) by a cosine over it's domain for the real coefficient and sum the input signal x multiplied (element wise) by a sine over its domain for the imaginary coefficients.
Secondly, you could potentially use conjugate symmetry and these formulas to calculate the relationship between your Xk and the desired An and Bn coefficients.
Xk = (An-j*Bn)/2
and
Xk* = (An+j*Bn)/2)

why frequency is wrong?

I want to find fft of cos(2pi10*n). I wrote this code but it has two problems. First, the estimated frequency is not 10 Hz. Second, from theory I expected that the amplitude of frequency should be pi. I found that the amplitude is dependent to size of fft. I appreciate any help.
n = 0:1:9;
x = cos(2*pi*10*n);
xdft = fft(x);
w=0:2*pi/(10):2*pi-2*pi/(10);
plot(abs(xdft) )
You must have overlooked the fact that, since the frequency of your signal is 10 Hz, you must sample it at no less than 20 Hz, lest it appear aliased during spectral analysis:
for i=1:4
if i==1
fs=1; %this is the value originally considered in OP's code
elseif i==2
fs=8;
elseif i==3
fs=20;
else
fs=40;
end
n=0:(1/fs):9;
x=cos(2*pi*10*n);
xdft=fft(x);
f=(0:(numel(x)-1))/numel(x)*fs;
subplot(4,1,i);
stem(f,abs(xdft),'.-');
xlabel('Frequency [Hz]');
end

Why sigmoid layer gives worse result than tanh layer in 0-1 regression task?

I'm working with regression to predict an array with 0-1 value (array of bit). The neural network specification is the following (MATLAB):
layers = [
imageInputLayer([1 16 2],'Normalization','none')
fullyConnectedLayer(512)
batchNormalizationLayer
reluLayer
fullyConnectedLayer(64)
batchNormalizationLayer
reluLayer
% sigmoidLayer
tanhLayer
regressionLayer
];
I've used the following code to implement Sigmoid Layer:
classdef sigmoidLayer < nnet.layer.Layer
methods
function layer = sigmoidLayer(name)
% Set layer name
if nargin == 2
layer.Name = name;
end
% Set layer description
layer.Description = 'sigmoidLayer';
end
function Z = predict(layer,X)
% Forward input data through the layer and output the result
Z = exp(X)./(exp(X)+1);
end
function dLdX = backward(layer, X ,Z,dLdZ,memory)
% Backward propagate the derivative of the loss function through
% the layer
dLdX = Z.*(1-Z) .* dLdZ;
end
end
end
The output is only 0 or 1. So why sigmoid is worse than tanh, instead of equal or better?
It depends on what you call "worse". Without more details it's hard to answer clearly.
However one of the key differences is the function's derivative. As the gradient update's magnitude depends on the derivative of the function, it can become close to 0 (and the network can't learn anymore) when the derivative saturates.
The sigmoid saturates at 1 and 0, when x->+/- inf, sigmoid -> 1/0 and d(sigmoid)/dx -> 0 and therefore depending on your data this might cause slower or "worse" learning. On the contrary, though it does saturate when going to 1, tanh does not saturate (actually it's a maxima for its derivative) around 0 so learning in this region is not problematic.
You might also want to look into label smoothing

Display BPSK signal waveform in MATLAB

I want to display a signal waveform passed through AWGN channel,so I followed these block diagrams and referenced this website then finished this program.
(http://drmoazzam.com/matlab-code-bpsk-modulation-and-demodulation-with-explanation/)
I sent a bit stream 1 and 0, just two bits. Set symbol energy per bit Es=1, Bit rate Ts=1 and carrier frequency fc=2. I coded the bit stream to biplolar non-return-to-zero form, then modulated it to BPSK signal.
When the signal was received, used coherent detection to demodulate it. To verify my program ( It seems to be right from displaying the waveform. After modulating 1 and 0, the phase of two waveform shifted by 180ยบ each other,as follow image. ), I calculated the BER. The BER varies with SNR. However, I set SNR=1dB, the BER is still 0 (SNR=1,finalber=0). It doesn't make sense. I don't know why result is that. Who can help me to check? Thanks
clc;
clear all;
% BPSK trapz
for kk=1:1000
bitstream=[1 0 ];
%% modulate: BPSK
Es=1;% symbol energy per bit
Ts=1; %Bit rate is assumed to be 1 bit/s;
fc=2; % carrier frequency
d=0.01; % sampled interval
tc=[0:d:0.99];
n=1;
symbolbits=1; % how many bits in a symbol, BPSK=1,QPSK=2
NRZ=2*bitstream-1; % non-return-to-zero line code,bit 1 ->1 , bit 0 ->-1
for i=1:symbolbits:length(bitstream)
s(1,(n-1)*length(tc)+1:n*length(tc))=NRZ(i)*sqrt(2/Ts)*cos(2*pi*fc*tc);
n=n+1;
end
%% through AWGN channnel
SNR=20; % SNR in dB
snr=10^(SNR/10);
E=sum(abs(s).^2)/length(s); %signal energy
N0=E/snr; % noise spectral density
noise=sqrt(N0)*(randn(1,length(s))); % s is real value signal
received=s+noise;
%% coherent detection
for i=1:length(received)/length(tc)
Qr(1,(i-1)*length(tc)+1:i*length(tc))=received((i-1)*length(tc)+1:i*length(tc)).*(sqrt(2/Ts)*cos(2*pi*fc*tc));
end
m=1;
y=[];
time=[0:d:1.99];
for i=1:2
y(1,i)=trapz(time((i-1)*100+1:i*100),Qr((i-1)*100+1:i*100));
end
detected=y;
for i=1:length(detected)
if detected(i)>0
demod(i)=1;
else
demod(i)=0;
end
end
ber(1,kk)=sum(demod~=bitstream)/length(bitstream);
end
finalber=sum(ber)/1000;
figure(1)
time=[0:d:1.99];
subplot(3,1,1); plot(time,s); title('transmitted BPSK signal')
subplot(3,1,2); plot(time,received); title('received BPSK signal')
subplot(3,1,3); plot(time,Qr)

calculate discrete S transform for given discrete time series

let us consider following Page :
http://djj.ee.ntu.edu.tw/S_Transform.pdf
paragraph 2.3 The Discrete S Transform
let say that we have sampled version of signal x, and given sampling frequency fs,i have calculated discrete Fourier transform using following code
function y=DFT(x);
N=length(x);
D=zeros(N,N);
for k=1:N
for n=1: N
D(k,n)=exp((-j*(k-1)*2*pi*(n-1))/N);
end
end
y=D*x'/N;
end
and started to estimate discrete S transform
function [S]=discrete_s_transform(x,fs);
%compute discrete s transform
%fs-sampling frequency
N=length(x); % length of signal
T=1/fs; % sampling period
Y=DFT(X);
how can i continue related to this part ?
clearly loops are not problem to implement,just they go from 1 to N instead of 0 to N-1 because of matlab vectors are 1 based,but what about main code?multiplication to exponential? coudl you please help me to finish S transform ?
Your sum depends only on m so I assume that other parameters as well as function H((m+n)/(NT)) are defined. Via for-loops the simplest one is:
function [S]=discrete_s_transform(x);
N=length(x); % length of signal
S=0;
m=0;
for i=1:N
S=S+H((m+n)/(NT))*exp(a)*exp(b*m/N);
m=m+1;
end
Hope that helps!