Does anyone know how to use filters in MATLAB?
I am not an aficionado, so I'm not concerned with roll-off characteristics etc — I have a 1 dimensional signal vector x sampled at 100 kHz, and I want to perform a high pass filtering on it (say, rejecting anything below 10Hz) to remove the baseline drift.
There are Butterworth, Elliptical, and Chebychev filters described in the help, but no simple explanation as to how to implement.
There are several filters that can be used, and the actual choice of the filter will depend on what you're trying to achieve. Since you mentioned Butterworth, Chebyschev and Elliptical filters, I'm assuming you're looking for IIR filters in general.
Wikipedia is a good place to start reading up on the different filters and what they do. For example, Butterworth is maximally flat in the passband and the response rolls off in the stop band. In Chebyschev, you have a smooth response in either the passband (type 2) or the stop band (type 1) and larger, irregular ripples in the other and lastly, in Elliptical filters, there's ripples in both the bands. The following image is taken from wikipedia.
So in all three cases, you have to trade something for something else. In Butterworth, you get no ripples, but the frequency response roll off is slower. In the above figure, it takes from 0.4 to about 0.55 to get to half power. In Chebyschev, you get steeper roll off, but you have to allow for irregular and larger ripples in one of the bands, and in Elliptical, you get near-instant cut off, but you have ripples in both bands.
The choice of filter will depend entirely on your application. Are you trying to get a clean signal with little to no losses? Then you need something that gives you a smooth response in the passband (Butterworth/Cheby2). Are you trying to kill frequencies in the stopband, and you won't mind a minor loss in the response in the passband? Then you will need something that's smooth in the stop band (Cheby1). Do you need extremely sharp cut-off corners, i.e., anything a little beyond the passband is detrimental to your analysis? If so, you should use Elliptical filters.
The thing to remember about IIR filters is that they've got poles. Unlike FIR filters where you can increase the order of the filter with the only ramification being the filter delay, increasing the order of IIR filters will make the filter unstable. By unstable, I mean you will have poles that lie outside the unit circle. To see why this is so, you can read the wiki articles on IIR filters, especially the part on stability.
To further illustrate my point, consider the following band pass filter.
fpass=[0.05 0.2];%# passband
fstop=[0.045 0.205]; %# frequency where it rolls off to half power
Rpass=1;%# max permissible ripples in stopband (dB)
Astop=40;%# min 40dB attenuation
n=cheb2ord(fpass,fstop,Rpass,Astop);%# calculate minimum filter order to achieve these design requirements
[b,a]=cheby2(n,Astop,fstop);
Now if you look at the zero-pole diagram using zplane(b,a), you'll see that there are several poles (x) lying outside the unit circle, which makes this approach unstable.
and this is evident from the fact that the frequency response is all haywire. Use freqz(b,a) to get the following
To get a more stable filter with your exact design requirements, you'll need to use second order filters using the z-p-k method instead of b-a, in MATLAB. Here's how for the same filter as above:
[z,p,k]=cheby2(n,Astop,fstop);
[s,g]=zp2sos(z,p,k);%# create second order sections
Hd=dfilt.df2sos(s,g);%# create a dfilt object.
Now if you look at the characteristics of this filter, you'll see that all the poles lie inside the unit circle (hence stable) and matches the design requirements
The approach is similar for butter and ellip, with equivalent buttord and ellipord. The MATLAB documentation also has good examples on designing filters. You can build upon these examples and mine to design a filter according to what you want.
To use the filter on your data, you can either do filter(b,a,data) or filter(Hd,data) depending on what filter you eventually use. If you want zero phase distortion, use filtfilt. However, this does not accept dfilt objects. So to zero-phase filter with Hd, use the filtfilthd file available on the Mathworks file exchange site
EDIT
This is in response to #DarenW's comment. Smoothing and filtering are two different operations, and although they're similar in some regards (moving average is a low pass filter), you can't simply substitute one for the other unless it you can be sure that it won't be of concern in the specific application.
For example, implementing Daren's suggestion on a linear chirp signal from 0-25kHz, sampled at 100kHz, this the frequency spectrum after smoothing with a Gaussian filter
Sure, the drift close to 10Hz is almost nil. However, the operation has completely changed the nature of the frequency components in the original signal. This discrepancy comes about because they completely ignored the roll-off of the smoothing operation (see red line), and assumed that it would be flat zero. If that were true, then the subtraction would've worked. But alas, that is not the case, which is why an entire field on designing filters exists.
Create your filter - for example using [B,A] = butter(N,Wn,'high') where N is the order of the filter - if you are unsure what this is, just set it to 10. Wn is the cutoff frequency normalized between 0 and 1, with 1 corresponding to half the sample rate of the signal. If your sample rate is fs, and you want a cutoff frequency of 10 Hz, you need to set Wn = (10/(fs/2)).
You can then apply the filter by using Y = filter(B,A,X) where X is your signal. You can also look into the filtfilt function.
A cheapo way to do this kind of filtering that doesn't involve straining brain cells on design, zeros and poles and ripple and all that, is:
* Make a copy of the signal
* Smooth it. For a 100KHz signal and wanting to eliminate about 10Hz on down, you'll need to smooth over about 10,000 points. Use a Gaussian smoother, or a box smoother maybe 1/2 that width twice, or whatever is handy. (A simple box smoother of total width 10,000 used once may produce unwanted edge effects)
* Subtract the smoothed version from the original. Baseline drift will be gone.
If the original signal is spikey, you may want to use a short median filter before the big smoother.
This generalizes easily to 2D images, 3D volume data, whatever.
Related
I need to design an FIR high-pass filter to attenuate frequencies from 20Hz and below. I need the order to be around 8 since I will be implementing the filter on a microcontroller. When using MATLAB FDAtool, there are only Fpass and Fstop as the input parameters. Is there an option to input Fc only (since Fpass and Fstop are not known)? or is there a way to determine Fpass and Fstop from the order number and the cut-off frequency?
It may be a bit naive to think about implementing a filter and consider only the cut-off frequency. In reality, there are several parameters to a filter which must be considered. An ideal filter that only has cut-off frequency would be infinite order, and therefore impossible to implement.
Look at this image to get an idea of the different considerations present in real filter design.
But I'll go ahead and try to read the tea leaves, here:
Select highpass. You will probably want to choose 20Hz as the value for Fstop, though you should be aware that frequencies slightly above this value will also be slightly (but less-so) attenuated. Fpass is the frequency at which the attenuation effectively stops.
Your pass and stop bands will have an amplitude ripple that you will also probably want to specify. Tightening these will increase the order of your filter.
I have a signal consisting of fast oscillating AC part and slowly varying DC part. I need to calcuate its DC part (and envelope, but that's not important now).
I could use the STFT, filter and transform it back, but it's a little inefficient cause I am not looking for a whole spectrum. Any other ideas?
I have read the articles on MathWorks and my maths is good enough to design something general and complicated, but I am looking for tips, hints or smart and elegant simple solutions. Thanks in advance!
DC isolation is basically low-pass filtering. You want to remove all the high-frequency components, while preserving the low. There are tons of approaches to this problem, but it looks like you're looking for something quick and simple.
One of the simplest forms of low-pass filter is the moving average. It's messy in the frequency domain, but for many simple applications it is good enough, trivial to implement, and incredibly fast.
For reasons I'll get to later on, it isn't actually a very good low-pass filter when you consider the frequency domain, but it is a very good smoothing filter in the time domain, which is what it sounds like you need.
Update: After posting this, I realized this was asked with the MATLAB tag, not just signal-processing. I've added a bit at the end showing the simplest way to do it in MATLAB.
Moving Average Filter
Parameters:
N: Filter order, number of samples to average
Input:
xt: Input at time t. For negative t assume x is zero.
Output:
yt: Output at time t. This is the average of the previous N samples.
Algorithm
Initialization (t = 0)
y0 = x0 / N
At each time step (t > 0)
yt = *yt-1 + (xt - xt-N ) / N
Filter Frequency Response
The impulse response of the filter is a rectangular pulse the width of the filter order. This gives a straightforward frequency response:
H[f] = | sin(πfN) / (Nsin(πf)) |
This shows how horrible the frequency response is, but for many smoothing applications it doesn't really matter. The key thing to notice is that the first zero happens at f = 1/N. This gives you a good idea of what order filter to use in order to smooth out components above a specific frequency, as described in the next section.
Parameter Selection
Given a sampling rate fs and a cut-off frequency fc, the normalized cutoff frequency is simply f = fc / fs. This, plus the frequency response above, gives a simple way to come up with the filter order, although tweaking it slightly by hand may improve results:
N = 1/f = fs / fc
Another way to read this formula is that it is important that the filter order is smaller than the fastest DC feature that you wish to preserve. Otherwise it will be partially filtered out.
Time-Domain Response
To illustrate the time-domain response, I'll work through some examples. I'll be using a sample rate of fs = 1000.
I generated a 40ms rectangular pulse with noise added to demonstrate the effect of increasing the filter order.
As you can see, increasing the filter order smooths out the noise, but it has other effects as well. Higher orders mean a longer delay, and a slower response to step changes. Once the order gets too high, the results are worhtless. The N = 51 example shows what happens when the filter order is wider than the event that you want to capture.
Although those sloping edges may look bad, the moving average filter actually gives the steepest possible edges for a given amount of noise reduction that is possible for a linear filter of the same order, and due to the incredibly simple algorithm it is also the fastest linear filter to implement. Because it has only one parameter, it is also simple to tweak to find the optimal order for your application.
Other Tweaks and Implementation Tips
Multipass Filter
The simplest way to improve the moving average filter's response in the frequency domain is to run multiple passes. After a few runs it becomes a very good approximation of a Gaussian filter due ot the central limit theorem. This slows down the step response a bit, but it also rounds it off. Higher frequencies will be attenuated further, although it also affects the roll-off in the low-frequency end. The performance hit for adding a few more passes is minimal.
Circular Buffer
If you are processing a stream of data, then you don't always have an easy way to lookup xt-N. By using a circular buffer to keep track of the last N samples you end up with a simple way to look up the sample you need with only a few steps added to the algorithm - it preserves the constant-time per step regardless of filter order.
Roundoff Error
If you're working with fixed-point or integer values the roundoff can be completely eliminated. Simply don't divide by N in the filter algorithm:
at = *at-1 + xt - xt-N
yt = *at / N
However, you need to take care to make sure that this won't cause an overflow in a.
With floating-point values, the tiny rounding errors could add up over time. The simplest solution for a long-running filter is to calculate y by averaging the entire buffer of the last N samples every so often. This will reset you to a point where there is no accumulated error, while letting you use the fast algorithm for the vast majority of time steps.
Doing it in MATLAB
A moving average is a linear FIR filter with each weight set the same. This means the numerators are all the same and sum to one, and the denominator is 1. It's simple to use the filter function to perform a moving average:
y = filter(ones(1,N)/N, 1, x);
However, that isn't going to use the fast algorithm discussed above. If performance is an issue then you'll want to implement the algorithm yourself.
I am new to matlab and signal processing methods, but i am trying to use its filter properties over a set of data I have. I have a collection of amplitude values obtained at different timestamps. When this is plotted, I get a waveform with several peaks that I can identify. I then perform calculations to derive the time between each consecutive peak and I want to eliminate the rates that are around the range of 48-52peaks per second.
What would be the correct way to go about processing this data step by step? Would a bandstop or notch filter be better if I want to eliminate those frequencies and not attenuate it simply? I am completely lost in the parameters required to feed into the filters for this. Please help...
periodogram is OK, but I would suggest using pwelch instead. It makes a more reasonable PSD estimate and the default parameters are well thought out (Hann windows, 50% overlap of segments, etc.)
If what you want is to remove signals in a wide band (e.g. 48-52 Hz) equally, rather than a single and unchanging frequency, than a bandstop filter is ideal. For example:
fs = 2048;
y = rand(fs*8, 1);
[b,a] = ellip(4, 2, 40, [46 54]/(fs/2));
yy = filter(b,a,y);
This will use a 4th order elliptic bandstop filter to filter the random data variable 'y'. filtfilt.m is also a nice function; it applies the filter forwards and backwards so you get twice the filter action and none of the phase lag or dispersion.
I am currently doing something similar to what you are doing.
I am processing a lot of signals from the Inertial Measurement Unit and motor drives. They all are asynchronously obtained, i.e. they all have very different timestamp and also very different acquisition frequency.
First thing I did was to interpolate all signals data in order to have all signals with same timestamp. You can use the matlab function interp to do this.
After this, you will have all signals with same sample frequency and also timestamp, which will be good in further analysis.
Ok, another thing you can do to analyse the frequency of the peaks is to perform the fourier transform. For beginners i advice the use of periodogram function and not the fft function.
Imagine you signal is x and your sample frequency (after interpolation) is Fs.
You can now use the function periodogram available in matlab like this:
[P,f] = periodogram(x,[],[length(t)],Fs);
This will give the power vs frequency of your signal. After that you will be able to plot and take a look at the frequencies of your signal. In other words, you be able to see the frequencies of the signals that make your acquired signal.
Plot the data this way:
plot(f,P); or semilogy(f,P);
The second is the same thing as the first, but with a logarithmic scale.
After this analysis you can use the Filter Desing and Analysis Tool to design you filter. Just type fdatool in matlab and it will open the design window. Choose the filter type, the cut and pass frequencies and click in design. This tool is very intuitive.
After designing you can export the filter to workspace.
Finally you can use the filter you designed in your signal to see if its what you wanted.
Use the functions filter os filtfilt for this.
Search in the web of the matlab help for the functions I wrote to get more details.
There are a lot of examples availables too.
I hope I could help you.
Good luck.
I am working with the same image and I also need to remove the texture from the image posted in this link
How can I remove the texture from an image using matlab?
Discussions were made on this and I'm quite confused which filter(gaussian LPF or ideal lowpass) is really needed and what is the reason behind this.Which frequencies contribute for this texture????please can someone explain me!
An ideal low pass filter will keep all spatial frequencies below a nominal spatial frequency, and remove all spatial frequencies above it. Unfortunately, a true ideal low pass filter has infinite support (i.e., has an infinitely large non-zero spatial extend). Even a practical approximation to an ideal low pass filter has large spatial support.
A Gaussian, on the other hand, isn't ideal in terms of which frequencies it filters out. A Gaussian in the spatial domain turns out to be a Gaussian in the spatial frequency domain. That is, it doesn't produce very sharp spatial frequency selectivity. The advantage though is that the spatial support of the filter is small. People use Gaussian filters for this because they are convenient mostly. Filtering with a Gaussian tends to look "natural" compared to ideal low pass filters, which can generate ringing artifacts.
A Lanczos filter (windowed sinc filter) is also another choice as it will have a small spatial support and will approximate an ideal filter better than a Gaussian.
However, which is better for your image largely depends on what you want to do. While there's significant theory behind it, qualitative choices like this in image processing are largely an art.
The type of filter you are looking for is ideally nonlinear:
smoothing in areas without large-scale gradients (edges), and
little smoothing close to edges to be preserved.
Here are two alternatives:
The Kuwahara filter:
http://homepage.tudelft.nl/e3q6n/publications/1999/PAA99DRBDPVLV/PAA99DRBDPVLV.pdf
Enhanced shortening flow (Figure 8) in:
http://www.cs.jhu.edu/~misha/Fall07/Papers/intro-to-scalespace.pdf
In the second filter (Enhanced shortening flow), you can
vary the scale parameter and the nonlinear function,
h(Lw) on page 17. Thus, more trimming possibilities.
Ideally, the filter is completely isotropic
(same frequency effect on each possible angle).
Michael
I have FFT outputs that look like this:
At 523 Hz is the maximum value. However, being a messy FFT, there are lots of little peaks that are right near the large peaks. However, they're irrelevant, whereas the peaks shown aren't. Are the any algorithms I can use to extract the maxima of this FFT that matter; I.E., aren't just random peaks cropping up near "real" peaks? Perhaps there is some sort of filter I can apply to this FFT output?
EDIT: The context of this is that I am trying to take one-hit sound samples (like someone pressing a key on a piano) and extract the loudest partials. In the image below, the peaks above 2000 Hz are important, because they are discrete partials of the given sound (which happens to be a sort of bell). However, the peaks that are scattered about right near 523 seem to be just artifacts, and I want to ignore them.
If the peak is broad, it could indicate that the peak frequency is modulated (AM, FM or both), or is actually a composite of several spectral peaks, themselves each potentially modulated.
For instance, a piano note may be the result of the hammer hitting up to 3 strings that are all tuned just a tiny fraction differently, and they all can modulate as they exchange energy between strings though the piano frame. Guitar strings can change frequency as the pluck shape distortion smooths out and decays. Bells change shape after they are hit, which can modulate their spectrum. Etc.
If the sound itself is "messy" then you need a good definition of what you mean by the "real" peak, before applying any sort of smoothing or side-band rejection filter. e.g. All that "messiness" may be part of what makes a bell sound like a real bell instead of an electronic sinewave generator.
Try convolving your FFT (treating it as a signal) with a rectangular pulse( pulse = ones(1:20)/20; ). This might get rid of some of them. Your maxima will be shifted by 10 frequency bins to teh right, to take that into account. You would basically be integrating your signal. Similar techniques are used in Pan-Tompkins algorithm for heart beat identification.
I worked on a similar problem once, and choosed to use savitsky-golay filters for smoothing the spectrum data. I could get some significant peaks, and it didn't messed too much with the overall spectrum.
But I Had a problem with what hotpaw2 is alerting you, I have lost important characteristics along with the lost of "messiness", so I truly recommend you hear him. But, if you think you won't have a problem with that, I think savitsky-golay can help.
There are non-FFT methods for creating a frequency domain representation of time domain data which are better for noisy data sets, like Max-ent recontruction.
For noisy time-series data, a max-ent reconstruction will be capable of distinguising true peaks from noise very effectively (without adding any artifacts or other modifications to suppress noise).
Max ent works by "guessing" an FFT for a time domain specturm, and then doing an IFT, and comparing the results with the "actual" time-series data, iteratively. The final output of maxent is a frequency domain spectrum (like the one you show above).
There are implementations in java i believe for 1-d spectra, but I have never used one.