I am working on some experimental data which, at some point, need to be time-integrated and then high-pass filtered (to remove low frequency disturbancies introduced by integration and unwanted DC component).
The aim of my work is not related to filtering, but still I would like to analyze more in detail the filters I am using to give some justification (for example to motivate why I chosed to use a 4th order filter instead of a higher/lower one).
This is the filter I am using:
delta_t = 1.53846e-04;
Fs = 1/delta_t;
cut_F = 8;
Wn = cut_F/(Fs/2);
ftype = 'high';
[b,a] = butter(4,Wn,ftype);
filtered_signal = filtfilt(b,a,signal);
I already had a look here: High-pass filtering in MATLAB to learn something about filters (I never had a course on signal processing) and I used
fvtool(b,a)
to see the impulse response, step response ecc. of the filter I have used.
The problem is that I do not know how to "read" these plots.
What do I have to look for?
How can I understand if a filter is good or not? (I do not have any specification about filter performances, I just know that the lowest frequency I can admit is 5 Hz)
What features of different filters are useful to be compared to motivate the choice?
I see you are starting your Uni DSP class on filters :)
First thing you need to remember is that Matlab can only simulate using finite values, so the results you see are technically all discrete. There are 4 things that will influence your filtering results(or tell you if your filter is good or bad) which you will learn about/have to consider while designing a Finite response filter:
1, the Type of the filter (i.e. Hamming, Butterworth (the one you are using), Blackman, Hanning .etc)
2, the number of filter Coefficients (which determines your filter resolution)
3, the sampling frequency of the original signal (ideally, if you have infinite sampling frequency, you can have perfect filters; not possible in Matlab due to reason above, but you can simulate its effect by setting it really high)
4, the cut-off frequency
You can play around with the 4 parameters so that your filter does what you want it to.
So here comes the theory:
There is a trade-off in terms of the width of your main lobe vs the spectrum leakage of your filter. The idea is that you have some signal with some frequencies, you want to filter out the unwanted (i.e. your DC noise) and keep the ones you want, but what if your desired signal frequency is so low that it is very close to the DC noise. If you have a badly designed filter, you will not be able to filter out the DC component. In order to design a good filter, you will need to find the optimal number for your filter coefficients, type of filter, even cut-off frequency to make sure your filter works as you wanted.
Here is a low-pass filter that I wrote back in the days, you can play around with filters a lot by filtering different kinds of signals and plotting the response.
N = 21; %number of filter coefficients
fc = 4000; %cut-off frequency
f_sampling = fs; %sampling freq
Fc = fc/f_sampling;
n = -(N-1)/2:(N-1)/2;
delta = [zeros(1,(N-1)/2) 1 zeros(1,(N-1)/2)];
h = delta - 2*Fc*sinc(2*n*Fc);
output = filter(h,1,yoursignal);
to plot the response, you want to plot your output in the frequency domain using DFT or FFT(in Matlab) and see how the signal has been distorted due to the leakage and etc.
NFFT=256; % FFT length
output=1/N*abs(fft(output,NFFT)).^2; % PSD estimate using FFT
this gives you what is known as a periodigram, when you plot, you might want to do the 10*log10 to it, so it looks nicer
Hope you do well in class.
Related
I was studying for a signals & systems project and I have come across this code on high and low pass filters for an audio signal on the internet. Now I have tested this code and it works but I really don't understand how it is doing the low/high pass action.
The logic is that a sound is read into MATLAB by using the audioread or wavread function and the audio is stored as an nx2 matrix. The n depends on the sampling rate and the 2 columns are due to the 2 sterio channels.
Now here is the code for the low pass;
[hootie,fs]=wavread('hootie.wav'); % loads Hootie
out=hootie;
for n=2:length(hootie)
out(n,1)=.9*out(n-1,1)+hootie(n,1); % left
out(n,2)=.9*out(n-1,2)+hootie(n,2); % right
end
And this is for the high pass;
out=hootie;
for n=2:length(hootie)
out(n,1)=hootie(n,1)-hootie(n-1,1); % left
out(n,2)=hootie(n,2)-hootie(n-1,2); % right
end
I would really like to know how this produces the filtering effect since this is making no sense to me yet it works. Also shouldn't there be any cutoff points in these filters ?
The frequency response for a filter can be roughly estimated using a pole-zero plot. How this works can be found on the internet, for example in this link. The filter can be for example be a so called Finite Impulse Response (FIR) filter, or an Infinite Impulse Response (IIR) filter. The FIR-filters properties is determined only from the input signal (no feedback, open loop), while the IIR-filter uses the previous signal output to control the current signal output (feedback loop or closed loop). The general equation can be written like,
a_0*y(n)+a_1*y(n-1)+... = b_0*x(n)+ b_1*x(n-1)+...
Applying the discrete fourier transform you may define a filter H(z) = X(z)/Y(Z) using the fact that it is possible to define a filter H(z) so that Y(Z)=H(Z)*X(Z). Note that I skip a lot of steps here to cut down this text to proper length.
The point of the discussion is that these discrete poles can be mapped in a pole-zero plot. The pole-zero plot for digital filters plots the poles and zeros in a diagram where the normalized frequencies, relative to the sampling frequencies are illustrated by the unit circle, where fs/2 is located at 180 degrees( eg. a frequency fs/8 will be defined as the polar coordinate (r, phi)=(1,pi/4) ). The "zeros" are then the nominator polynom A(z) and the poles are defined by the denominator polynom B(z). A frequency close to a zero will have an attenuation at that frequency. A frequency close to a pole will instead have a high amplifictation at that frequency instead. Further, frequencies far from a pole is attenuated and frequencies far from a zero is amplified.
For your highpass filter you have a polynom,
y(n)=x(n)-x(n-1),
for each channel. This is transformed and it is possble to create a filter,
H(z) = 1 - z^(-1)
For your lowpass filter the equation instead looks like this,
y(n) - y(n-1) = x(n),
which becomes the filter
H(z) = 1/( 1-0.9*z^(-1) ).
Placing these filters in the pole-zero plot you will have the zero in the highpass filter on the positive x-axis. This means that you will have high attenuation for low frequencies and high amplification for high frequencies. The pole in the lowpass filter will also be loccated on the positive x-axis and will thus amplify low frequencies and attenuate high frequencies.
This description is best illustrated with images, which is why I recommend you to follow my links. Good luck and please comment ask if anything is unclear.
can someone help me with another matlab project:
Is it possible to create a simple low pass filter like in RC circuits? For instance if we create a sine wave like y=10*sin(2*pift).
Can i just filter the signal with a cutoff frequency to be able to see how the filtered signal looks like(for the amplitude decay)?
So when i enter some information for example f = cutoff and plot the filtered signal this must reduce the amplitude somewhere at 30% from the input signal ?
I've been looking for the functions like butter and cheby but they seem to do other kind of filtering like removing the noise from a signal. I just want some simple plots(input and output) which will show the principle of RC, RL low and high pass filters.
An RC circuit is a fist-order filter. To approximate a first order hardware filter, I generally use a IIR filter. A butterworth filter is usually my first choice for IIR, but for a first-order response, it doesn't really matter.
Here's an example with the cutoff frequency being the same as the signal frequency, so the filtered signal should be 3 dB down...
%first, make signal
fs = 1000; %your sample rate, Hz
dur_sec = 1; %what is the duration of your signal, seconds
t_sec = ([1:dur_sec*fs]-1)/fs; %here is a vctor of time
freq_Hz = 25; %what frequency do you want your sign wave
y = sin(2*pi*freq_Hz*t_sec); %make your sine wave
%Second, make your filter
N = 1; %first order
cutoff_Hz = 25; %should be 3dB down at the cutoff
[b,a]=butter(N,cutoff_Hz/(fs/2),'lowpass'); %this makes a lowpass filter
%Third, apply the filter
y_filt = filter(b,a,y);
%Last, plot the results
figure;
plot(t_sec,y,t_sec,y_filt);
xlabel('Time (sec)');
ylabel('Amplitude');
ylim([-1 1]);
legend('Raw','Filtered');
title(['1st-Order Filter with Cutoff at ' num2str(cutoff_Hz) ' Hz']);
As an alternative to using the built-in filter design functions such as butter, you could choose model the circuit itself. A simple first order RC (or RL) circuit would result a first order differential equation. For an RC circuit, you'd then integrate the equation through time, given your sine wave as stimulation. That would work fine, but could be more of a hassle, depending upon your background.
For a simple, first-order hardware filter that is properly buffered by an op-amp on either end of the RC, I think that you'd find that the result using the first order butter filter is going to be awfully close (the same?) as modeling the circuit. The butter filter is way easier to implement in software (because i just gave you the code above), so I'd go that route.
When you move to 2nd order hardware filters, however, that's where you have to be more careful. You have a few options:
1) Continue to model your 2nd order hardware using one of the built-in filter functions. A second order butter is trivial to implement (alter the N in the code above), but this might not model the specific hardware filter that you've created. You'll have to choose the right kind of IIR filter to match the architecture of your hardware filter.
2) If you haven't picked your architecture for your hardware filter, you could choose the architecture to follow one of the canonical filter types, just so that it is easy to model via butter, cheby1, or whatever.
3) You could go back to modeling the circuit with differential equations. This would let you model any filter circuit, whether it followed a canonical type or not. You could also put non-linear effects in here, if you wanted.
For the first order RC filter, though, I think that any of the built-in filter types will be a decent enough model for an RC filter. I'd suggest that you play with my example code above. I think that it will satisfy your need.
Chip
I have a template. I compute the impulse response of the matched filter by taking the inverse Fourier Transform of the conjugate of the Fourier transform of my template. And I would like to perform the matched filtering operation on one of my available EEG channels using the 'filter' command in Matlab. Using the filter command the coefficient 'b' is my impulse response? Moreover, I would like to implement Matlab code to threshold the output of the matched filter to detect peaks.How can I achieve it?
Here is a start for you,
% A template is given
temp = randn(100,1);
% Create a matched filter based on the template
b = flipud(temp(:));
% For testing the matched filter, create a random signal which
% contains a match for the template at some time index
x = [randn(200,1); temp(:); randn(300,1)];
n = 1:length(x);
% Process the signal with the matched filter
y = filter(b,1,x);
% Set a detection threshold (exmaple used is 90% of template)
thresh = 0.9
% Compute normalizing factor
u = temp.'*temp;
% Find matches
matches = n(y>thresh*u);
% Plot the results
plot(n,y,'b', n(matches), y(matches), 'ro');
% Print the results to the console
display(matches);
As Andreas mentions in his answer, there is no need for the fourier transform. If you have a time-domain template, then its matched filter is simply a time-reversed version of itself (which I achieve with flipud). As you go along, you will find that there are many nuances to be worked out. This code works great because I am in control from start to finish, but once you start working with real data, things get much more complicated. Choosing an appropriate threshold value for example will require some knowledge about the data that you will be working with.
In truth, peak detection can be a very non-trivial task depending on the nature of your signals, etc. In my case, peak detection was easy because my signal was completely uncorrelated with the template, except at the point in the middle, and I also knew exactly what amplitude I was expecting to see. All of these assumptions are over-simplifications of the problem which I used to demonstrate the concepts.
Practically, you do this
y = filter( h, 1, x )
with h the impuse response and x and y input and output signals.
The matched filter is nothing else than a correlator that correlates with a given signal pattern.
It has a impulse response which is just the time reverse of the signal pattern you try to look for.
So by the way: If you have a measured signal pattern, reverse it and set this as impulse response of a FIR filter. There is no need to do this in the frequency domain if you have measurement in the time domain (both approaches are equivalent but one is more error prone then the other)
I'm very much a novice at signal processing techniques, but I am trying to apply the fast fourier transform to a daily time series to remove the seasonality present in the data. The example I am working with is from here:
http://www.mathworks.com/help/signal/ug/frequency-domain-linear-regression.html
While I understand how to implement the code as it is written in the example, I am having a hard time adapting it to my specific application. What I am trying to do is create a preprocessing function which deseasonalizes the training data using similar code to the above example. Then, using the same estimated coefficients from the in-sample data, deseasonalize the out-of-sample data to preserve its independence from the in-sample data. Basically, once the coefficients are estimated, I will normalize each new data point using the same coefficients. I suspect this is akin to estimating a linear trend, then removing it from the in-sample data, and then using the same linear model on unseen data to detrend it i the same manner.
Obviously, when I estimate the fourier coefficients, the vector I get out is equal to the length of the in-sample data. The out-of-sample data is comprised of much fewer observations, so directly applying them is impossible.
Is this sort of analysis possible using this technique or am I going down a dead end road? How should I approach that using the code in the example above?
What you want to do is certainly possible, you are on the right track, but you seem to misunderstand a few points in the example. First, it is shown in the example that the technique is the equivalent of linear regression in the time domain, exploiting the FFT to perform in the frequency domain an operation with the same effect. Second, the trend that is removed is not linear, it is equal to a sum of sinusoids, which is why FFT is used to identify particular frequency components in a relatively tidy way.
In your case it seems you are interested in the residuals. The initial approach is therefore to proceed as in the example as follows:
(1) Perform a rough "detrending" by removing the DC component (the mean of the time-domain data)
(2) FFT and inspect the data, choose frequency channels that contain most of the signal.
You can then use those channels to generate a trend in the time domain and subtract that from the original data to obtain the residuals. You need not proceed by using IFFT, however. Instead you can explicitly sum over the cosine and sine components. You do this in a way similar to the last step of the example, which explains how to find the amplitudes via time-domain regression, but substituting the amplitudes obtained from the FFT.
The following code shows how you can do this:
tim = (time - time0)/timestep; % <-- acquisition times for your *new* data, normalized
NFpick = [2 7 13]; % <-- channels you picked to build the detrending baseline
% Compute the trend
mu = mean(ts);
tsdft = fft(ts-mu);
Nchannels = length(ts); % <-- size of time domain data
Mpick = 2*length(NFpick);
X(:,1:2:Mpick) = cos(2*pi*(NFpick-1)'/Nchannels*tim)';
X(:,2:2:Mpick) = sin(-2*pi*(NFpick-1)'/Nchannels*tim)';
% Generate beta vector "bet" containing scaled amplitudes from the spectrum
bet = 2*tsdft(NFpick)/Nchannels;
bet = reshape([real(bet) imag(bet)].', numel(bet)*2,1)
trend = X*bet + mu;
To remove the trend just do
detrended = dat - trend;
where dat is your new data acquired at times tim. Make sure you define the time origin consistently. In addition this assumes the data is real (not complex), as in the example linked to. You'll have to examine the code to make it work for complex data.
I'm having a conceptual issue with notch filters. As far as I understand it, a notch filter output is equal to the sum of highpass and lowpass filter outputs. However, a quick test in MATLAB doesn't show this. Here's some testing code:
% Load a simple signal and specify constants
load handel; % this loads the signal, y, and its sampling rate, Fs
nData=y;
nFreq=[55 65];
nOrder=4;
% Create a lowpassed signal
[b a]=butter(nOrder,nFreq(1)/(Fs/2),'low');
nLP_Data=filter(b,a,nData);
% Create a highpassed signal
[b a]=butter(nOrder,nFreq(2)/(Fs/2),'high');
nHP_Data=filter(b,a,nData);
% Combine LP and HP signals
nCombinedHPLP=nLP_Data+nHP_Data;
% Create a notch filtered signal
[b a]=butter(nOrder/2,nFreq/(Fs/2),'stop'); % The notch filter uses twice the first input argument for its order, hence the "/2"
nN_Data=filter(b,a,nData);
% Plot each and output the total difference in the signals to the Command Window
plot(nN_Data,'r')
hold all
plot(nCombinedHPLP,'b')
legend({'Notched signal','Combined HP, LP signals'});
sum(abs(nCombinedHPLP-nN_Data))
I'm thinking that the difference is due to phase effects of the filters, as changing filter to filtfilt yields two identical signals. Is there not a way (using `filter') of recreating the effect of a notch filter with a highpass and lowpass filter? Thanks for reading.
Rogare,
You HAVE created a notch filter; you just created one with different phase than what a single butter filter would do, as you suspected.
Both your BUTTER filters will have their own phase responses, and there's no guarantee that the processing of your data with both of those filters will have the same net phase as processing your data ONCE with a hand-made band-pass filter. There's no rule that says butter will spend time and energy trying to get the phase of a low-pass and a high-pass filter to behave like the phase of a band-pass filter (in fact, it would be weird if it did!)
However, the MAGNITUDE of the FFT of the resulting data agrees quite nicely:
subplot(2,1,1);
plot(abs(fftshift(fft(nCombinedHPLP))));
title('My Notch')
subplot(2,1,2);
plot(abs(fftshift(fft(nN_Data))));
title('Real Notch')
You do have a conceptual problem: in the opening paragraph of your question you say
As far as I understand it, a notch filter output is equal to the sum
of highpass and lowpass filter outputs
If that were the case, then any signal that passes either the lowpass or the highpass will get through (at low frequency, the lowpass lets it through, so the sum of the outputs of lowpass + highpass lets low frequencies through. Ditto for high frequencies, if you swap the words "low" and "high"...).
There's a difference between applying in series vs summing outputs. I think that is where your confusion is coming from. And yes - depending on how a filter is constructed, how steep it is, etc, you will get all kinds of effects when you apply them in series...
As the order of the notch filter is twice of the order of LP or HP filters, that should indicate that a NF can also be derived as conv(LP,HP).
The concept that you asking is that of superposition, which would presuppose equal length filters.