My goal is to find the maximum values of wave heights and wave lengths.
dwcL01 though dwcL10 is arrays of <3001x2 double> with output from a numerical wave model.
Part of my script:
%% Plotting results from SWASH
% Examination of phase velocity on deep water with different number of layers
% Wave height 3 meters, wave peroid 8 sec on a depth of 30 meters
clear all; close all; clc;
T=8;
L0=1.56*T^2;
%% Loading results tabels.
load dwcL01.tbl; load dwcL02.tbl; load dwcL03.tbl; load dwcL04.tbl;
load dwcL05.tbl; load dwcL06.tbl; load dwcL07.tbl; load dwcL08.tbl;
load dwcL09.tbl; load dwcL10.tbl;
M(:,:,1) = dwcL01; M(:,:,2) = dwcL02; M(:,:,3) = dwcL03; M(:,:,4) = dwcL04;
M(:,:,5) = dwcL05; M(:,:,6) = dwcL06; M(:,:,7) = dwcL07; M(:,:,8) = dwcL08;
M(:,:,9) = dwcL09; M(:,:,10) = dwcL10;
%% Finding position of wave crest using diff and sign.
for i=1:10
Tp(:,1,i) = diff(sign(diff([M(1,2,i);M(:,2,i)]))) < 0;
Wc(:,:,i) = M(Tp,:,i);
L(:,i) = diff(Wc(:,1,i))
end
This works fine for finding the maximum values, if the data is "smooth". The following image shows a section of my data. I get all peaks, when I only need the one around x = 40. How do I filter so I only get the "real" wave crests. The solution needs to be general so that it still works if I change the domain size, wave height or wave period.
If you're basically trying to fit this curve of data to a sine wave, have you considered performing Fourier analysis (FFT in Matlab), then checking the magnitude of that fundamental frequency? The frequency will tell you the wave spacing, and the magnitude the height, and when used over multiple periods will find an average.
See the Matlab help page for an example of the usage
but the basic gist is:
y = [...] %vector of wave data points
N=length(y); %Make sure this is an even number
Y = fft(y); %Convert into frequency domain
figure;
plot(y(1:N)); %Plot original wave data
figure;
plot(abs(Y(1:N/2))./N); %Plot only the lower half of frequencies to hide aliasing
I have one more solution that might work for you. It involves computing the 2nd-order derivative using a 5-point central difference instead of the 2-point finite differences. When using diff twice you are performing two first-order derivatives consecutively (finite 2-point differences) which are very susceptible to noise/oscillations. The advantage of using a higher-order approximation is that the neighboring points help filter out the small oscillations, and this may work for your case.
Let f(:) = squeeze(M(:,2,i)) be the array of data points, and h is the uniform spacing distance between the points:
%Better approximation of the 2nd derivative using neighboring points:
for j=3:length(f)-2
Tp(j,i) = (-f(j-2) + 16*f(j-1) - 30*f(j) + 16*f(j+1) - f(j+2))/(12*h^2);
end
Note that since this 2nd-order derivative requires the 2 neighboring points to the left and right, that the range of the loop must start at the 3rd index and end 2 short of the array length.
Related
I have two sets of data taken from experiments, and they look very similar, except there is a horizontal offset between them, which I believe is due to some bugs in the instrument setting. Suppose they have the form y1=f(x1) and y2=f(x2)= f(x1+c), what's the best way to determine c so that I can take into account the offset to superimpose two data sets to become one data set?
Edit:
let's say my data sets (index 1 and 2) have the form:
x1 = 0:0.2:10;
y1 = sin(x1)
x2 = 0:0.3:10;
y2 = sin(x2+0.5)
Of course, the real data will have some noise, but say the best fit functions have the above forms. How do I find the offset c=0.5? I have looked into the cross-correlation, but I'm not sure if they can handle two data sets with different number of data (and different step sizes). Also, what if the offset values actually fall between two data points? Cross-correlation only returns the index of the data in the array, not something in between if I understand correctly.
This Matlab script calculates the random offset from -pi/2 to +pi/2 between two sine waves:
clear;
C= pi*(rand-0.5); % should be between -pi/2 and +pi/2
N=200; % should be large enough for acceptable sampling rate
N1=20; % fraction for Ts1
N2=30; % fraction for Ts2
Ts1=abs(C*N1/N); % fraction of C for accuracy
Ts2=abs(C*N2/N); % fraction of C for accuracy
Ts=min(Ts1,Ts2); % select highest sampling rate (smaller period)
fs=1/Ts;
P=4; % number of periods should be large enough for well defined correlation plot
x1 = 0:Ts:P*2*pi;
y1 = sin(x1);
x2 = 0:Ts:P*2*pi;
y2 = sin(x2+C);
subplot(3,1,1)
plot(x1,y1);
subplot(3,1,2);
plot(x2,y2);
[cor,lag]=xcorr(y1,y2);
subplot(3,1,3);
plot(lag,cor);
[~,I] = max(abs(cor));
lagdiff = lag(I);
timediff=lagdiff/fs;
In the particular case below, C = timediff = 0.5615:
write a function which takes the time shift as an input and calculates rms between overlapping portions of the two data sets. Then find the minimum of this function using optimization (fminbnd)
I have two main questions, but starting from the beginning:
I wanted to know how FFT (fast Fourier transform) works on real examples. I created two sinusoidal waves (for example Wave1 = 5 Hz and Wave2 = 15 Hz amplitude), then I added those two and made FFT from third "Wave3". It looks OK - I saw my "peaks" around 5 and 15 Hz.
Blue = 5 Hz, Red = 15 Hz, Yellow = Blue + Red. FFT from "Yellow" Wave, looks good:
OK, and then I have changed the data. Now I have two waves with identical amplitudes but opposite phases. If I add them, their amplitude is 0 - and that seems correct to me.
Two waves with opposite phases. Yellow - Wave1+Wave2 + Strange FFT of the yellow wave:
And now is the part that I don't understand at all. Here are my questions:
1) Even if I see on the picture that this third Yellow wave has an amplitude equal to 0, it's not like that in data tables. After adding two main waves (and they have opposite data!) I get the strange result.
Example: 5 first points in the data
Wave 1:
0,0627905195293128
0,125333233564304
0,187381314585724
0,248689887164855
0,309016994374947
Wave 2:
-0,0627905195293134
-0,125333233564304
-0,187381314585724
-0,248689887164855
-0,309016994374947
Wave 3 (Sum) :
-5,68989300120393e-16
-1,11022302462516e-16
-1,11022302462516e-16
3,05311331771918e-16
-1,11022302462516e-16
Why the sum of these waves is not equal 0, as it is shown in the picture? Why FFT looks so strange? Is there even a possibility that FFT will show us the real amplitudes of two identical waves with opposite phases? I thought it will not, but what's the truth?
Here is my MATLAB code:
THREE WAVES:
D = 1; % 1 second
S = 1000; % sampling rate
P = 0.5; % phase
T = 1/S; % sampling period
t = [T:T:D]; % time
myphi=2*pi*P;
myphi2=2*pi*1;
syn3 = sin(2*10*t*pi+myphi); % first wave
syn2 = sin(2*10*t*pi+myphi2); % second wave
sinmax=syn2+syn3; % yellow wave
figure; plot(t,syn3,t,syn2,t,sinmax,'LineWidth',2); grid on;
xlabel('Time (seconds)');
ylabel('Amplitude');
FFT CODE:
L = length(sinmax);
myFFT = fft(sinmax,S);
myFFT=myFFT/L; %scale the output to 1
freq = S/2*linspace(0,1,S/2);
figure; stem(freq,abs(myFFT(1:length(freq))));
xlabel('Frequency (Hz)');
ylabel('Amplitude');
Thank you very much in advance...
Mary
First things first, your calculations are correct.
Because the two graphs are auto-resized, it is easy to make a mistake interpreting them but the amplitudes are way smaller on the 2nd one than the 1st (10e-16 vs. 10e1, respectively).
On the 2nd graph, the one which leaves you puzzled, you are just victim of numerical errors : these numbers can be interpreted as 0.
From there, you have two simple solutions :
you are fine with the results you have, and you can just tweak the figures to display results on the same scale to avoid any misleading interpretation
you would like to have "proper zero values" instead of "small ones"
1) Set a limit for the Y-axis
You can just add something like this line (it is just an example - change it to meet your needs) when plotting your figures :
ymax = max(abs(my_fft));
ymin = - ymax;
ylim([ymin ymax])
2) Set a limit for filtering numerical errors
Like in a lot of numerical methods algorithms, you might want to consider as 0 values which are in between 0 and small interval, often called epsilon :
abs_fft = abs(my_fft);
epsilon = 10e-12 % your threshold
abs_fft(abs_fft < epsilon) = 0;
You might want to check out eps which is a built-in Matlab variable, meant for this kind of cases.
I'm working on a small code to learn signal processing on Matlab. I have got a .wav sound with some noise and I just want to remove the noise. I tried the code under but noise is not removed correctly. My idea is to do a cut band filter to remove the different noise components on the fft. After a lot of researches, I don't understant where is my problem. Here my code :
clear all;
clc;
close all;
% We load and plot the signal
[sig,Fs] = audioread('11.wav');
N = length(sig);
figure,plot(sig); title 'signal'
% FFT of the signal
fft_sig = abs(fft(sig));
% Normalisation of the frequencies for the plot
k = 0 : length(sig) - 1;
f = k*Fs/length(sig);
plot(f,fft_sig);
% Loop with 2 elements because there are 2 amplitudes (not sure about
% this)
for i = 1:2
% I put noise components in an array
[y(i),x(i)] = max(fft_sig);
% Normalisation of the frequencies to eliminate
f(i) = x(i) * (Fs/N);
% Cut band filter with elimination of f from f-20 to f+20 Hz
b = fir1(2 , 2 * [f(i)-20 f(i)+20] / Fs, 'stop')
sig = filter(b,1,sig);
figure,plot(abs(fft(sig)));title 'fft of the signal'
end
Here the images I got, the fft plot is exactly the same before and after applying the filter, there is a modification only on the x axis :
The sampling frequency is Fs = 22050.
Thanks in advance for your help, I hope I'm clear enough in my description
Since you haven't explicitly said so, the code you provided essentially defines your noise as narrowband interference at two frequencies (even though that interference may look less 'noisy').
The first thing to notice is that the value x(i) obtained from max(fft_sig) corresponds to the 1-based index of the located maximum. It won't make a huge difference for large N, but it may have an impact for smaller values especially when trying to design a very narrow notch filter. The corresponding frequency would then be:
freq = (x(i)-1) * (Fs/N);
Also if you are going to iteratively attenuate frequency components you would need to update fft_sig which you use to pick the frequency components to be attenuated (otherwise you will always pick the same component). The simplest would be to recompute fft_sig from the filtered sig:
sig = filter(b,1,sig);
fft_sig = abs(fft(sig));
Finally, since you are trying to attenuate a very narrow frequency range, you may find that you need to increase the FIR filter order by a few orders of magnitudes to get a good attenuation at the desired frequency without attenuating everything else. As was pointed out in comments such a narrow notch filter can often be better implemented using an IIR filter.
The updated code would then look somewhat like:
% Loop with 2 elements because there are 2 amplitudes
for i = 1:2
% I put noise components in an array
% Limit peak search to the lower-half of the spectrum since the
% upper half is symmetric
[y(i),x(i)] = max(fft_sig(1:floor(N/2)+1));
% Normalisation of the frequencies to eliminate
freq = (x(i)-1) * (Fs/N);
% Cut band filter with elimination of f from f-20 to f+20 Hz
b = fir1(2000 , 2 * [freq-20 freq+20] / Fs, 'stop')
sig = filter(b,1,sig);
fft_sig = abs(fft(sig));
%figure;plot(f, abs(fft_sig));title 'fft of the signal'
figure;plot(f, 20*log10(abs(fft_sig)));title 'fft of the signal'
end
As far as the last FFT plot showing a different x-axis, it is simply because you omitted to provide an x-axis variable (in occurrence f used in the previous plot), so the plot shows the array index (instead of frequencies) as the x-axis.
I'm working on sound signals of a walking pattern, which has obvious regular patterns:
Then I thought I can get the frequency of walking (approximately 1.7Hz from the image) using FFT function:
x = walk_5; % Walking sound with a size of 711680x2 double
Fs = 48000; % sound frquency
L=length(x);
t=(1:L)/Fs; %time base
plot(t,x);
figure;
NFFT=2^nextpow2(L);
X=fft(x,NFFT);
Px=X.*conj(X)/(NFFT*L); %Power of each freq components
fVals=Fs*(0:NFFT/2-1)/NFFT;
plot(fVals,Px(1:NFFT/2),'b','LineSmoothing','on','LineWidth',1);
title('One Sided Power Spectral Density');
xlabel('Frequency (Hz)')
ylabel('PSD');
But then it doesn't give me what I expected:
FFT result:
zoom image has lots of noises:
and there is no information near 1.7Hz
Here is the graph from log domain using
semilogy(fVals,Px(1:NFFT));
It's pretty symmetric though:
I couldn't find anything wrong with my code. Do you have any solutions to easily extract the 1.7Hz from the walking pattern?
here is the link for the audio file in mat
https://www.dropbox.com/s/craof8qkz9n5dr1/walk_sound.mat?dl=0
Thank you very much!
Kai
I suggest you to forget about DFT approach since your signal is not appropriate for this type of analysis due to many reasons. Even by looking on the spectrum in range of frequencies that you are interested in, there is no easy way to estimate the peak:
Of course you could try with PSD/STFT and other funky methods, but this is an overkill. I can think of two, rather simple methods, for this task.
First one is based simply on the Auto Correlation Function.
Calculate the ACF
Define the minimum distance between them. Since you know that expected frequency is around 1.7Hz, then it corresponds to 0.58s. Let's make it 0.5s as the minimum distance.
Calculate the average distance between peaks found.
This gave me an approximate frequency of 1.72 Hz .
Second approach is based on the observation to your signal already has some peaks which are periodic. Therefore we can simply search for them using findpeaks function.
Define the minimum peak distance in a same way as before.
Define the minimum peak height. For example 10% of maximum peak.
Get the average difference.
This gave me an average frequency of 1.7 Hz.
Easy and fast method. There are obviously some things that can be improved, such as:
Refining thresholds
Finding both positive and negative peaks
Taking care of some missing peaks, i.e. due to low amplitude
Anyway that should get you started, instead of being stuck with crappy FFT and lazy semilogx.
Code snippet:
load walk_sound
fs = 48000;
dt = 1/fs;
x = walk_5(:,1);
x = x - mean(x);
N = length(x);
t = 0:dt:(N-1)*dt;
% FFT based
win = hamming(N);
X = abs(fft(x.*win));
X = 2*X(1:N/2+1)/sum(win);
X = 20*log10(X/max(abs(X)));
f = 0:fs/N:fs/2;
subplot(2,1,1)
plot(t, x)
grid on
xlabel('t [s]')
ylabel('A')
title('Time domain signal')
subplot(2,1,2)
plot(f, X)
grid on
xlabel('f [Hz]')
ylabel('A [dB]')
title('Signal Spectrum')
% Autocorrelation
[ac, lag] = xcorr(x);
min_dist = ceil(0.5*fs);
[pks, loc] = findpeaks(ac, 'MinPeakDistance', min_dist);
% Average distance/frequency
avg_dt = mean(gradient(loc))*dt;
avg_f = 1/avg_dt;
figure
plot(lag*dt, ac);
hold on
grid on
plot(lag(loc)*dt, pks, 'xr')
title(sprintf('ACF - Average frequency: %.2f Hz', avg_f))
% Simple peak finding in time domain
[pkst, loct] = findpeaks(x, 'MinPeakDistance', min_dist, ...
'MinPeakHeight', 0.1*max(x));
avg_dt2 = mean(gradient(loct))*dt;
avg_f2 = 1/avg_dt2;
figure
plot(t, x)
grid on
hold on
plot(loct*dt, pkst, 'xr')
xlabel('t [s]')
ylabel('A')
title(sprintf('Peak search in time domain - Average frequency: %.2f Hz', avg_f2))
Here's a nifty solution:
Take the absolute value of your raw data before taking the FFT. The data has a ton of high frequency noise that is drowning out whatever low frequency periodicity is present in the signal. The amplitude of the high frequency noise gets bigger every 1.7 seconds, and the increase in amplitude is visible to the eye, and periodic, but when you multiply the signal by a low frequency sine wave and sum everything you still end up with something close to zero. Taking the absolute value changes this, making those amplitude modulations periodic at low frequencies.
Try the following code comparing the FFT of the regular data with the FFT of abs(data). Note that I took a few liberties with your code, such as combining what I assume were the two stereo channels into a single mono channel.
x = (walk_5(:,1)+walk_5(:,2))/2; % Convert from sterio to mono
Fs = 48000; % sampling frquency
L=length(x); % length of sample
fVals=(0:L-1)*(Fs/L); % frequency range for FFT
walk5abs=abs(x); % Take the absolute value of the raw data
Xold=abs(fft(x)); % FFT of the data (abs in Matlab takes complex magnitude)
Xnew=abs(fft(walk5abs-mean(walk5abs))); % FFT of the absolute value of the data, with average value subtracted
figure;
plot(fVals,Xold/max(Xold),'r',fVals,Xnew/max(Xnew),'b')
axis([0 10 0 1])
legend('old method','new method')
[~,maxInd]=max(Xnew); % Index of maximum value of FFT
walkingFrequency=fVals(maxInd) % print max value
And plotting the FFT for both the old method and the new, from 0 to 10 Hz gives:
As you can see it detects a peak at about 1.686 Hz, and for this data, that's the highest peak in the FFT spectrum.
This is the first time I'm using the fft function and I'm trying to plot the frequency spectrum of a simple cosine function:
f = cos(2*pi*300*t)
The sampling rate is 220500. I'm plotting one second of the function f.
Here is my attempt:
time = 1;
freq = 220500;
t = 0 : 1/freq : 1 - 1/freq;
N = length(t);
df = freq/(N*time);
F = fftshift(fft(cos(2*pi*300*t))/N);
faxis = -N/2 / time : df : (N/2-1) / time;
plot(faxis, real(F));
grid on;
xlim([-500, 500]);
Why do I get odd results when I increase the frequency to 900Hz? These odd results can be fixed by increasing the x-axis limits from, say, 500Hz to 1000Hz. Also, is this the correct approach? I noticed many other people didn't use fftshift(X) (but I think they only did a single sided spectrum analysis).
Thank you.
Here is my response as promised.
The first or your questions related to why you "get odd results when you increase the frequency to 900 Hz" is related to the Matlab's plot rescaling functionality as described by #Castilho. When you change the range of the x-axis, Matlab will try to be helpful and rescale the y-axis. If the peaks lie outside of your specified range, matlab will zoom in on the small numerical errors generated in the process. You can remedy this with the 'ylim' command if it bothers you.
However, your second, more open question "is this the correct approach?" requires a deeper discussion. Allow me to tell you how I would go about making a more flexible solution to achieve your goal of plotting a cosine wave.
You begin with the following:
time = 1;
freq = 220500;
This raises an alarm in my head immediately. Looking at the rest of the post, you appear to be interested in frequencies in the sub-kHz range. If that is the case, then this sampling rate is excessive as the Nyquist limit (sr/2) for this rate is above 100 kHz. I'm guessing you meant to use the common audio sampling rate of 22050 Hz (but I could be wrong here)?
Either way, your analysis works out numerically OK in the end. However, you are not helping yourself to understand how the FFT can be used most effectively for analysis in real-world situations.
Allow me to post how I would do this. The following script does almost exactly what your script does, but opens some potential on which we can build . .
%// These are the user parameters
durT = 1;
fs = 22050;
NFFT = durT*fs;
sigFreq = 300;
%//Calculate time axis
dt = 1/fs;
tAxis = 0:dt:(durT-dt);
%//Calculate frequency axis
df = fs/NFFT;
fAxis = 0:df:(fs-df);
%//Calculate time domain signal and convert to frequency domain
x = cos( 2*pi*sigFreq*tAxis );
F = abs( fft(x, NFFT) / NFFT );
subplot(2,1,1);
plot( fAxis, 2*F )
xlim([0 2*sigFreq])
title('single sided spectrum')
subplot(2,1,2);
plot( fAxis-fs/2, fftshift(F) )
xlim([-2*sigFreq 2*sigFreq])
title('whole fft-shifted spectrum')
You calculate a time axis and calculate your number of FFT points from the length of the time axis. This is very odd. The problem with this approach, is that the frequency resolution of the fft changes as you change the duration of your input signal, because N is dependent on your "time" variable. The matlab fft command will use an FFT size that matches the size of the input signal.
In my example, I calculate the frequency axis directly from the NFFT. This is somewhat irrelevant in the context of the above example, as I set the NFFT to equal the number of samples in the signal. However, using this format helps to demystify your thinking and it becomes very important in my next example.
** SIDE NOTE: You use real(F) in your example. Unless you have a very good reason to only be extracting the real part of the FFT result, then it is much more common to extract the magnitude of the FFT using abs(F). This is the equivalent of sqrt(real(F).^2 + imag(F).^2).**
Most of the time you will want to use a shorter NFFT. This might be because you are perhaps running the analysis in a real time system, or because you want to average the result of many FFTs together to get an idea of the average spectrum for a time varying signal, or because you want to compare spectra of signals that have different duration without wasting information. Just using the fft command with a value of NFFT < the number of elements in your signal will result in an fft calculated from the last NFFT points of the signal. This is a bit wasteful.
The following example is much more relevant to useful application. It shows how you would split a signal into blocks and then process each block and average the result:
%//These are the user parameters
durT = 1;
fs = 22050;
NFFT = 2048;
sigFreq = 300;
%//Calculate time axis
dt = 1/fs;
tAxis = dt:dt:(durT-dt);
%//Calculate frequency axis
df = fs/NFFT;
fAxis = 0:df:(fs-df);
%//Calculate time domain signal
x = cos( 2*pi*sigFreq*tAxis );
%//Buffer it and window
win = hamming(NFFT);%//chose window type based on your application
x = buffer(x, NFFT, NFFT/2); %// 50% overlap between frames in this instance
x = x(:, 2:end-1); %//optional step to remove zero padded frames
x = ( x' * diag(win) )'; %//efficiently window each frame using matrix algebra
%// Calculate mean FFT
F = abs( fft(x, NFFT) / sum(win) );
F = mean(F,2);
subplot(2,1,1);
plot( fAxis, 2*F )
xlim([0 2*sigFreq])
title('single sided spectrum')
subplot(2,1,2);
plot( fAxis-fs/2, fftshift(F) )
xlim([-2*sigFreq 2*sigFreq])
title('whole fft-shifted spectrum')
I use a hamming window in the above example. The window that you choose should suit the application http://en.wikipedia.org/wiki/Window_function
The overlap amount that you choose will depend somewhat on the type of window you use. In the above example, the Hamming window weights the samples in each buffer towards zero away from the centre of each frame. In order to use all of the information in the input signal, it is important to use some overlap. However, if you just use a plain rectangular window, the overlap becomes pointless as all samples are weighted equally. The more overlap you use, the more processing is required to calculate the mean spectrum.
Hope this helps your understanding.
Your result is perfectly right. Your frequency axis calculation is also right. The problem lies on the y axis scale. When you use the function xlims, matlab automatically recalculates the y scale so that you can see "meaningful" data. When the cosine peaks lie outside the limit you chose (when f>500Hz), there are no peaks to show, so the scale is calculated based on some veeeery small noise (here at my computer, with matlab 2011a, the y scale was 10-16).
Changing the limit is indeed the correct approach, because if you don't change it you can't see the peaks on the frequency spectrum.
One thing I noticed, however. Is there a reason for you to plot the real part of the transform? Usually, it is abs(F) that gets plotted, and not the real part.
edit: Actually, you're frequency axis is only right because df, in this case, is 1. The faxis line is right, but the df calculation isn't.
The FFT calculates N points from -Fs/2 to Fs/2. So N points over a range of Fs yields a df of Fs/N. As N/time = Fs => time = N/Fs. Substituting that on the expression of df you used: your_df = Fs/N*(N/Fs) = (Fs/N)^2. As Fs/N = 1, the final result was right :P