I have a Pressure signal at 10 kHz that is somewhat noisy. I would therefore like to use the decimate function to reduce the sampling rate and thus reducing the noise.
However, I discovered that depending on the parameters that I put as input to the function, it can happen that the decimate function slightly reduce the amplitude of the signal as well as the amplitude of the noise. I would therefore like to know if somebody could explain me why it is happening in order to ensure that I will be using decimate correctly in the future
As an example, here is different example of the same signal decimated with different parameters. Here I'm just varying the decimation factor r and the order of the Chebyshev Type I lowpass filter n (standard filter used with decimate with n = 8).
if I use n = 8 (standard filter) and r = 4, the output decimated signal (red) has a reduced noise but also a reduced amplitude compared to the original signal (blue):
figure;n = 8;r=4;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
Actually, every even order for the filter (so n = 2*k, k€int) give me the same results, whereas for the odd order (n = 2*k+1) the amplitude is conserved:
graph with n = 9 and r = 4:
figure;n = 9;r=4;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
also, the same holds true if I increase the decimation factor up to 90
n = 9 and r = 90:
figure;n = 9;r=90;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
if now I increase the decimation factor above 90, the amplitude is decreased for any decimation factor and any filter order, except for the 3rd order:
n = 9 and r = 91 (amplitude decreased):
figure;n = 9;r=91;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
n = 6 and r = 100 (amplitude decreased):
figure;n = 6;r=100;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
n = 3 and r = 100 (amplitude conserved!):
figure;n = 3;r=100;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
Finally, for very big decimation factor (>500), the amplitude is conserved no matter which order I use for the filter. I tried to vary the length of the source vector and this holds true.
I'm sure that I'm missing some theoretical background on how filters work and how those parameters can influence the output, but I find a bit weird that the output of the function can be "wrong" (ie with here lower amplitude of the signal, along with the noise) just depending on the parameters I use.
Especially weird is the fact that Matlab tells you in the "help decimate" that for better results at large decimation factor (ie r>13) it is recommended to break r up into its factors and calling decimate several time. So for a decimation factor of 1000, matlab suggest to decimate the signal by 2 three times and by 5 three time (2*2*2*5*5*5 = 1000). if I do that the distance between the source signal and the decimated vector increase even more, whereas with directly a ratio of 1000, the output signal has the same amplitude!
Comparison with recursive call or directly with r = 1000, n = 6:
figure;n = 6;plot(TimeFyr,Pup,decimate(decimate(decimate(decimate(TimeFyr,4,n),10,n),5,n),5,n),decimate(decimate(decimate(decimate(Pup,4,n),10,n),5,n),5,n),'r')
figure;n = 6;r=1000;plot(TimeFyr,Pup,decimate(TimeFyr,r,n),decimate(Pup,r,n),'r')
Can anybody help?
The problem is in the choice of filter. Chebyshev 1, if that is what you are using, does attenuate at low frequencies https://en.wikipedia.org/wiki/Chebyshev_filter.
Chebyshev 2 or Butterworth would be a better choice since they do not have much of a ripple in a passband. Pleased, see the Wikipedia about the filters, and it will be clear. Yes, the choice of filter is extremely important when doing decimation.
I am using MATLAB tutorial Frequency-Domain Linear Regression. There is one part of code, where is necessary to determine "frequency bin of the positive and negative frequency", for fft and ifft, this part of code:
freqbin = 72/12;
freqbins = [freqbin 72-freqbin]+1;
tsfit = zeros(72,1);
tsfit(freqbins) = tsdft(freqbins);
tsfit = ifft(tsfit);
mu = mean(ts);
tsfit = mu+tsfit;
Length of time series is 72, and there is 12 months is one cycle. How there can be only one frequency bin for positive frequency, how do we know that is exactly 72/12+1 (+1 is because first bin is for zero frequency), is that some formula or what?
Let's start with some general notes. If the sample rate is Fs, and the number of samples in the FFT is N, then the frequency of the bin with MATLAB index k is:
f_k = (k-1) * Fs / N ;
Or, conversely,
k = f_k * N / Fs + 1;
Hence, the elements of the FFT vector correspond to frequencies 0, Fs/N, 2*Fs/N, ..., (N-1)*Fs/N.
Note that the frequencies are invariant to shifts by Fs. In other words, f and f+Fs and f-Fs are all equivalent. So, you could consider the first half of the FFT vector to correspond to positive frequencies (index: 1 to N/2; frequency: 0 to (N/2-1)/N*Fs), and the second half to correspond to negative frequencies (index: N/2+1 to N; frequency: -Fs/2 to -Fs/N). Here I include 0 in the set of positive frequencies for brevity.
Now, coming to your specific example:
N = 72
Fs = 12 ; % samples per year
So, the positive frequency range is from 0 to Fs/2 = 6 cycles per year, corresponding to indices 1 to 36, respectively.
You are interested in finding the FFT sample corresponding to the frequency 1 cycle per year. That corresponds to index 1 * 72 / 12 + 1 = 7.
Since the FFT is symmetric for real signals, you are also interested in finding the FFT sample corresponding to the frequency -1 cycle per year, which, using the shift-invariance property, is also equivalent to -1 + Fs = 11 cycles per year. The corresponding index is (12-1) * 72 / 12 + 1 = 67.
In summary,
There are 36 positive frequencies (including 0), not 1.
The algorithm is interested in only one positive frequency with the highest magnitude.
That frequency happens to be 1 cycle per year. The corresponding MATLAB index is 7.
The mirror image of the frequency is -1 cycle per year, which corresponds to MATLAB index 67.
Suppose there is a real signal Amp_time (Amplitude as a function of time) at Sample Rate SampRate.
I can transform it into frequency domain with
fftLength = length(Amp_time);
FT_freq = fft(Amp_time,fftLength); % signal as a function of frequency
FT_freq = FT_freq(1:fftLength/2); % throw away symmetric part
and the corresponding frequencies:
freq = [0:fftLength-1].*(SampRate/fftLength);
freq = freq(freq < SampRate/2); % throw away symmetric part
To transform it back into time domain I simply use
Amp_time2 = ifft(FT_freq,'symmetric');
Question
But how about if I am changing the length of FT_freq?
Suppose I want to reduce length(FT_freq) by averaging FT_freq to a coarser frequency vector (i.e. 10^3 instead of 10^5 frequencies).
[freq2,FT_freq2] = special_average(freq,FT_freq) % reducing length of FT_freq
I don't think that it is possible to take the version above (because how can Matlab possibly know to which frequency the signal is corresponding now).
What am I missing?
Do I have to scale the coarse FT_freq2 up again in order to get the right IFFT?
(bad in terms of memory, ...)
The reason I wanted to reduce the degrees of freedom of the Fourier Transform is to avoid having matrices with size of order 10^5 x 10^3 (FFTing every tenth of a second of some song) and thereafter to compare freq-spectra/tones.
So if my sampling rate is 8000, and I make a sine wave of frequency 1000 like so:
n = 0:255;
fs = 8000;
fa = 50;
x = sin(2*pi*(fa/fs)*n);
resolution = n * (fs/length(n));
stem(resolution, abs(fft(x)));
Everything is fine, and the plot shows the single peak at 1000Hz.
However, if I set a frequency fa = 50, I get a smoother rise up to the peak, however the peak location is incorrect. It's at 62.5 Hz.
So the obvious solution is reduce the sampling rate, but what if that's not possible, such as in my case?
I think that the above observation/result makes sense given the choice of your sampling rate fs and block size N of 8000 and 256 respectively. Note that
fs/N = 8000/256 = 31.25
The discrete outputs from the FFT N-point transform can only be associated with frequencies that are multiples of the above
m*fs/N = m*31.25 for m=0,1,2,….,255
There does not exist any m such that m*fs/N is equal to 50, so the closest is the 62.5 (2*31.25).
If you cannot change the sampling rate, then how about the block size, N such that 50=fs/N? If N were chosen to be any multiple of 160 (=8000/50) then you should be fine
n = 0:159; % or any multiple of 160 less one
(You mention the smoother rise up to the peak which is just leakage - it causes any input signal whose frequency is not exactly at a FFT bin centre to leak into all of the other FFT output bins.)
Note that everything is fine for the input signal with a 1000 Hz frequency since 1000=32*31.25 when fs=8000 and N=256 (so m=32).
The maximum value of your discrete Fourier transform is at 62.5Hz. However, if you interpolate, you'll find that the maximum value is at 50 Hz. See http://www.dspguru.com/dsp/howtos/how-to-interpolate-fft-peak for details.
I have a relatively clean sine signal (from thin-film interference) - two data vectors:X-axis (difference in optical path length) and Y-axis (illuminate).
I want to find the sine's frequency using Fourier transform (in matlab). How do I do that?
thanks!
Firts do you need to use one Window (hamming, hann) in your Signal and now all that you need is get the maximun value from first half Magnitude squared DATA, to find the Frequency calculate "sample rate * maximun index / length DATA:
t = [ 0 : 1 : 100000];
f = 200; % F0 here
Fs = 44100;
data = sin(2*pi*f/Fs*t)';
data = data .* hanning(length(data));
Y = fft(data);
Mag=abs(Y(1:length(data)/2)).^2;
[a,b]=max(Mag);
% Result
Fs*b/length(data)