From this site,
The output node has a "threshold" t.
Rule:
If summed input ≥ t, then it "fires" (output y = 1).
Else (summed input < t) it doesn't fire (output y = 0).
How y equals to zero. Any Ideas appreciated.
Neural networks have a so called "activation function", it's usually some form of a sigmoid-like function to map the inputs into separate outputs.
http://zephyr.ucd.ie/mediawiki/images/b/b6/Sigmoid.png
For you it happens to be either 0 or 1 and using a comparison instead of a sigmoid function,
so your activation curve will be even sharper than the graph above. In the said graph, your t, the threshold, is 0 on the X axis.
So as pseudo code :
sum = w1 * I1 + w2 + I2 + ... + wn * In
sum is the weighted sum of all in the inputs the neuron, now all you have to do is compare that sum to t, the threshold :
if sum >= t then y = 1 // Your neuron is activated
else y = 0
You can use the last neuron's output as the networks output to predict something into 1/0, true/false etc.
If you're studying NNs, I'd suggest you start with the XOR problem, then it will all make sense.
Related
I am working on developing a suite classifiers for EEG signals and I will be needing a zero-crossings around mean function, defined in the following manner:
Ideally if I have some vector with a range of values representing a sinusoid or any time varying signal, I will want to return a vector of Booleans of the same size as the vector saying if that particular value is a mean crossing. I have the following Matlab implementation:
ZX = #(x) sum(((x - mean(x)>0) & (x - mean(x)<0)) | ((x - mean(x)<0) & (x - mean(x)>0)));
Testing it on toy data:
[0 4 -6 9 -20 -5]
Yields:
0
EDIT:
Yet I believe it should return:
3
What am I missing here?
An expression like:
((x-m)>0) & ((x-m)<0)
is always going to return a vector of all zeros because no individual element of x is both greater and less than zero. You need to take into account the subscripts on the xs in the definition of ZX:
((x(1:end-1)-m)>0) & ((x(2:end)-m)<0)
You can use the findpeaks function on -abs(x), where x is your original data, to find the peak locations. This would give you the zero crossings in general for continuous signals which do not have zero as an actual maximum of the signal.
t = 0:0.01:10;
x = sin(pi*t);
plot(t,x)
grid
y = -abs(x);
[P,L] = findpeaks(y,t);
hold on
plot(L,P,'*')
A simple solution is to use movprod, and count the products which are negative, i.e.,
cnt = sum(sign(movprod(x-mean(x),2))<0);
With your toy example, you will get cnt = 3.
I have some confusion about the terminologies and simulation of an FIR system. I shall appreciate help in rectifying my mistakes and informing what is correct.
Assuming a FIR filter with coefficient array A=[1,c2,c3,c4]. The number of elements are L so the length of the filter L but the order is L-1.
Confusion1: Is the intercept 1 considered as a coefficient? Is it always 1?
Confusion2: Is my understanding correct that for the given example the length L= 4 and order=3?
Confusion3: Mathematically, I can write it as:
where u is the input data and l starts from zero. Then to simulate the above equation I have done the following convolution. Is it correct?:
N =100; %number of data
A = [1, 0.1, -0.5, 0.62];
u = rand(1,N);
x(1) = 0.0;
x(2) = 0.0;
x(3) = 0.0;
x(4) = 0.0;
for n = 5:N
x(n) = A(1)*u(n) + A(2)*u(n-1)+ A(3)*u(n-3)+ A(4)*u(n-4);
end
Confusion1: Is the intercept 1 considered as a coefficient? Is it always 1?
Yes it is considered a coefficient, and no it isn't always 1. It is very common to include a global scaling factor in the coefficient array by multiplying all the coefficients (i.e. scaling the input or output of a filter with coefficients [1,c1,c2,c2] by K is equivalent to using a filter with coefficients [K,K*c1,K*c2,K*c3]). Also note that many FIR filter design techniques generate coefficients whose amplitude peaks near the middle of the coefficient array and taper off at the start and end.
Confusion2: Is my understanding correct that for the given example the length L= 4 and order = 3?
Yes, that is correct
Confusion3: [...] Then to simulate the above equation I have done the following convolution. Is it correct? [...]
Almost, but not quite. Here are the few things that you need to fix.
In the main for loop, applying the formula you would increment the index of A and decrement the index of u by 1 for each term, so you would actually get x(n) = A(1)*u(n) + A(2)*u(n-1)+ A(3)*u(n-2)+ A(4)*u(n-3)
You can actually start this loop at n=4
The first few outputs should still be using the formula, but dropping the terms u(n-k) for which n-k would be less than 1. So, for x(3) you'd be dropping 1 term, for x(2) you'd be dropping 2 terms and for x(1) you'd be dropping 3 terms.
The modified code would look like the following:
x(1)=A(1)*u(1);
x(2)=A(1)*u(2) + A(2)*u(1);
x(3)=A(1)*u(3) + A(2)*u(2) + A(3)*u(1);
for n = 4:N
x(n) = A(1)*u(n) + A(2)*u(n-1)+ A(3)*u(n-2)+ A(4)*u(n-3);
end
I am trying to implement a map / function which has the equation Bernoulli Shift Map
x_n+1 = 2* x_n mod 1
The output of this map will be a binary number which will be either 0/1.
So, I generated the first sample x_1 using rand. The following is the code. The problem is I am getting real numbers. When using a digital calculator, I can get binary, whereas when using Matlab, I am getting real numbers. Please help where I am going wrong. Thank you.
>> x = rand();
>> x
x =
0.1647
>> y = mod(2* x,1)
y =
0.3295
The dyadic transformation seems to be a transformation from [0,1) continuous to [0,1) continuous. I see nothing wrong with your test code if you are trying to implement the dyadic mapping. You should be expecting output in the [0,1)
I misunderstood your question because I focused on the assumption you had that the output should be binary [0 or 1], which is wrong.
To reproduce the output of the dyadic transformation as in the link you provided, your code works fine (for 1 value), and you can use this function to calculate N terms (assuming a starting term x0) :
function x = dyadic(x0,n)
x = zeros(n,1) ; %// preallocate output vector
x(1) = x0 ; %// assign first term
for k=2:n
x(k) = mod( 2*x(k-1) , 1) ; %// calculate all terms of the serie
end
Note that the output does not have to be binary, it has to be between 0 and 1.
In the case of integers, the result of mod(WhateverInteger,1) is always 0, but in the case of Real numbers (which is what you use here), the result of mod(AnyRealNumber,1) will be the fractional part, so a number between 0 and 1. (1 is mathematically excluded, 0 is possible by the mod(x,1) operation, but in the case of your serie it means all the successive term will be zero too).
I'm translating some MATLAB code to Haskell using the hmatrix library. It's going well, but
I'm stumbling on the pos function, because I don't know what it does or what it's Haskell equivalent will be.
The MATLAB code looks like this:
[U,S,V] = svd(Y,0);
diagS = diag(S);
...
A = U * diag(pos(diagS-tau)) * V';
E = sign(Y) .* pos( abs(Y) - lambda*tau );
M = D - A - E;
My Haskell translation so far:
(u,s,v) = svd y
diagS = diag s
a = u `multiply` (diagS - tau) `multiply` v
This actually type checks ok, but of course, I'm missing the "pos" call, and it throws the error:
inconsistent dimensions in matrix product (3,3) x (4,4)
So I'm guessing pos does something with matrix size? Googling "matlab pos function" didn't turn up anything useful, so any pointers are very much appreciated! (Obviously I don't know much MATLAB)
Incidentally this is for the TILT algorithm to recover low rank textures from a noisy, warped image. I'm very excited about it, even if the math is way beyond me!
Looks like the pos function is defined in a different MATLAB file:
function P = pos(A)
P = A .* double( A > 0 );
I can't quite decipher what this is doing. Assuming that boolean values cast to doubles where "True" == 1.0 and "False" == 0.0
In that case it turns negative values to zero and leaves positive numbers unchanged?
It looks as though pos finds the positive part of a matrix. You could implement this directly with mapMatrix
pos :: (Storable a, Num a) => Matrix a -> Matrix a
pos = mapMatrix go where
go x | x > 0 = x
| otherwise = 0
Though Matlab makes no distinction between Matrix and Vector unlike Haskell.
But it's worth analyzing that Matlab fragment more. Per http://www.mathworks.com/help/matlab/ref/svd.html the first line computes the "economy-sized" Singular Value Decomposition of Y, i.e. three matrices such that
U * S * V = Y
where, assuming Y is m x n then U is m x n, S is n x n and diagonal, and V is n x n. Further, both U and V should be orthonormal. In linear algebraic terms this separates the linear transformation Y into two "rotation" components and the central eigenvalue scaling component.
Since S is diagonal, we extract that diagonal as a vector using diag(S) and then subtract a term tau which must also be a vector. This might produce a diagonal containing negative values which cannot be properly interpreted as eigenvalues, so pos is there to trim out the negative eigenvalues, setting them to 0. We then use diag to convert the resulting vector back into a diagonal matrix and multiply the pieces back together to get A, a modified form of Y.
Note that we can skip some steps in Haskell as svd (and its "economy-sized" partner thinSVD) return vectors of eigenvalues instead of mostly 0'd diagonal matrices.
(u, s, v) = thinSVD y
-- note the trans here, that was the ' in Matlab
a = u `multiply` diag (fmap (max 0) s) `multiply` trans v
Above fmap maps max 0 over the Vector of eigenvalues s and then diag (from Numeric.Container) reinflates the Vector into a Matrix prior to the multiplys. With a little thought it's easy to see that max 0 is just pos applied to a single element.
(A>0) returns the positions of elements of A which are larger than zero,
so forexample, if you have
A = [ -1 2 -3 4
5 6 -7 -8 ]
then B = (A > 0) returns
B = [ 0 1 0 1
1 1 0 0]
Note that we have ones corresponding to an elemnt of A which is larger than zero, and 0 otherwise.
Now if you multiply this elementwise with A using the .* notation, then you are multipling each element of A that is larger than zero with 1, and with zero otherwise. That is, A .* B means
[ -1*0 2*1 -3*0 4*1
5*1 6*1 -7*0 -8*0 ]
giving finally,
[ 0 2 0 4
5 6 0 0 ]
So you need to write your own function that will return positive values intact, and negative values set to zero.
And also, u and v does not match in dimension, for a generall SVD decomposition, so you actually would need to REDIAGONALIZE pos(diagS - Tau), so that u* diagnonalized_(diagS -tau) agrres to v
Is there a way in matlab to create a low pass filter, I know i can use the filter function but not sure how to use it, I've been given the following formula for my low pass H(z) = 1 (1 - z^-4)^2 / 16 (1 - z^-1)^2 with a 20Hz cutoff frequency
The filter function allows you to apply a filter to a vector. You still need to provide the filter coefficients. If you look at the documentation for filter, you see that you need to specify two vectors b and a whose elements are coefficients of z in descending powers, where z is the frequency domain variable in a z-transform. Since you have an expression for your filter given as a z-transform, the coefficients are easy to find. First, let's write the numerator of your filter:
(1/16)*(1 - z^-4)^2 = (1/16)*(1 - 2z^-4 + z^-16)
= (1/16)*(1 + 0z^-1 + 0z^-2 + 0z^-3 - 2z^-4 + 0z^5 + 0z^-6 ... + z^-16)
So the b vector is b = (1/16)*[1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1]. Similarly, the a vector is a = [1 -2 1]. So now you can filter your data vector x to get a result y by simply doing y = filter(b,a,x);.
Having said all that, the H(z) you specify above is definitely not a low pass filter. It's more like some weird cascade of a comb filter with itself.
If you want to design your own filter, and assuming you have the Signal Processing Toolbox, the absolute simplest thing to do is design a filter using Matlab's fir1 function:
h = fir1(N, 20/(Fs/2)); %# N is filter length, Fs is sampling frequency (Hz)
which you can then use in the filter function:
y = filter(h, 1, x); %# second param is 1 because this is an FIR filter
You will need to pick N yourself. Generally, larger N values make for better filters, where better = rejects more frequencies above 20 Hz. If your N value starts getting so big that it causes weird behavior (computational errors, slow implementations, unacceptable startup/ending transients in the resulting data) you might consider a more complicated filter design. The Mathworks documentation has an overview of the various digital filter design techniques.
The formula you have given: H(z) = 1 (1 - z^-4)^2 / 16 (1 - z^-1)^2 is the filter's Z-transform. It is a rational function, which means your filter is a recursive (IIR) filter.
Matlab has a function called filter(b,a,X). The b are the coefficients of the numerator with decreasing power of z, i.E. in your case: (1*z^-0 + 0*z^-1 + 0*z^-2 + 0*z^-3 + 0*z^-4)^2, you can use conv() for quantity square:
b = [1 0 0 0 -1]
b = conv(b,b)
and the coefficients of the denominator are:
a = [1 -1]
a = 16 * conv(a,a)
Then you call the filter y = filter(b,a,x), where x is your input data.
You can also check your filter's frequency response with freqz(b,a)
Hope that helped.