I'll try to be more specific: I have several time histories of a signal which have pretty much all the same behaviour (sine waves) but all start at a different point in time. How do I automatically detect the initial time lag and delete it such that all sine waves start at the same instant in time?
If you have two signals, x and y, each being a n x 1 matrix where y is a shifted version of x:
[c,lags] = xcorr(x,y); % c is the correlation, should have a clear peak
s = lags(c==max(c)); % s is the shift you need
y2 = circshift(y,s); % y2 should now overlap x
(Demo purposes only - I don't suggest you circshift your actual data). The shift you are looking for in this case should ideally be relatively small compared to the length of x and y. A lot depends on the noise level and the nature of the offset.
The following works pretty well under low noise conditions and fast sampling and may do depending on your requirements for accuracy. It uses a simple threshold and thus is subject to inaccuracy when things get noisy. Adjust thresh to a low value above the noise.
Nwav = 3;
Np = 100;
tmax = 50;
A = 1000;
Nz = Np/5;
%%%%%%%%%%%%%%
thresh = A/50;
%%%%%%%%%%%%%%
% generate some waveforms
t = [0:tmax/(Np-1):tmax].';
w = rand(1,Nwav);
offs = round(rand(1,Nwav)*100);
sig = [A*sin(t(1:end-Nz)*w) ; zeros(Nz,Nwav)] + randn(Np,Nwav);
for ii=1:Nwav
sig(:,ii) = circshift(sig(:,ii),round(rand()*Nz));
end
figure, plot(t,sig)
hold on, plot(t,repmat(thresh,length(t),1),'k--')
% use the threshold and align the waveforms
for ii=1:Nwav
[ir ic] = find(sig(:,ii)>thresh,1)
sig(:,ii) = circshift(sig(:,ii),-ir+1);
end
figure, plot(t,sig)
hold on, plot(t,repmat(thresh,length(t),1),'k--')
There is room for improvement (noise filtering, slope detection) but this should get you started.
I also recommend you look into waveform processing toolboxes, in matlab central for instance.
Related
I want to generate a sine signal y(t) with a time dependent frequency f(t) in Matlab.
I've already tried to realise this using the sine function of Matlab:
h = 0.0001;
npoints = 150/h;
for i = 1:1:npoints
f(i) = 2 - 0.01*i*h;
y(i) = 0.5*sin(2*3.1415*f(i)*i*h)+0.5;
end
where the frequency is decreasing with time and h is the time step width.
My problem:
The signal y(t) doesn't look I expected it to look like. There appears a bump in the amplitude at a distinct time (have a look at the plot below).
Does anyone know why this happens and how to generate this sine signal correctly?
what about
y(i) = 0.5*real(exp(1i*2*pi*f(i)*i*h))+0.5;
You will get the plot below
If you just need a chirp signal starting from 2Hz down to 0.5Hz, the following should do the job
f_start = 2; % start frequency
f_end = 0.5; % end frequency
endtime = 150; % seconds
timestep = 0.0001;
times = timestep:timestep:endtime;
y = chirp(times,f_start,endtime,f_end);
and if you plot it you get
figure(2);plot(times,y);
You can achieve the same manually using below
f_start = 2; % start frequency
f_end = 0.5; % end frequency
timestep = 0.0001;
T = 150;
rate_of_change = (f_start - f_end)/T;
times = timestep:timestep:T;
y = sin(2*pi*(f_start*times - times.^2*rate_of_change/2));
It might be useful to read the following Wikipedia page on Chirp signal.
At 100 you have sin(2*pi*N), which is 0. Change f a little bit, say to 2.0123-... and it goes to the top.
As for the general probably unexpected shape, consider what function you are using in the end (= substitute f back in the formula). You see that you have something of the form y = ...sin(Ai-B*i^2)..., which has a minimum at 100.
The easiest solution here is to simply offset frequency a little more, and use something like f(i) = 3.1 - ..., which has a minimum outside of your considered range.
It looks like there isn't actually a "bump in frequency", but at the 100 value on the x-axis the entire signal is shifted by 180 degrees. Bear in mind that the amplitude still reaches 0 and does not become smaller (e.g. from 0.25 to 0.75)
This is because the i value becomes so high that the value of f(i) changes sign.
Another indicator of this is that the frequency starts to increase again after the shift instead of gradually becoming even lower.
Why do you start off with the value of 2 in f(i)?
Sorry for asking for clarification here, but I cannot post it as a comment.
I have a series of 2D measurements (time on x-axis) that plot to a non-smooth (but pretty good) sawtooth wave. In an ideal world the data points would form a perfect sawtooth wave (with partial amplitude data points at either end). Is there a way of calculating the (average) period of the wave, using OCTAVE/MATLAB? I tried using the formula for a sawtooth from Wikipedia (Sawtooth_wave):
P = mean(time.*pi./acot(tan(y./4))), -pi < y < +pi
also tried:
P = mean(abs(time.*pi./acot(tan(y./4))))
but it didn't work, or at least it gave me an answer I know is out.
An example of the plotted data:
I've also tried the following method - should work - but it's NOT giving me what I know is close to the right answer. Probably something simple and wrong with my code. What?
slopes = diff(y)./diff(x); % form vector of slopes for each two adjacent points
for n = 1:length(diff(y)) % delete slope of any two points that form the 'cliff'
if abs(diff(y(n,1))) > pi
slopes(n,:) = [];
end
end
P = median((2*pi)./slopes); % Amplitude is 2*pi
Old post, but thought I'd offer my two-cent's worth. I think there are two reasonable ways to do this:
Perform a Fourier transform and calculate the fundamental
Do a curve-fitting of the phase, period, amplitude, and offset to an ideal square-wave.
Given curve-fitting will likely be difficult because of discontinuities in saw-wave, so I'd recommend Fourier transform. Self-contained example below:
f_s = 10; # Sampling freq. in Hz
record_length = 1000; # length of recording in sec.
% Create noisy saw-tooth wave, with known period and phase
saw_period = 50;
saw_phase = 10;
t = (1/f_s):(1/f_s):record_length;
saw_function = #(t) mod((t-saw_phase)*(2*pi/saw_period), 2*pi) - pi;
noise_lvl = 2.0;
saw_wave = saw_function(t) + noise_lvl*randn(size(t));
num_tsteps = length(t);
% Plot time-series data
figure();
plot(t, saw_wave, '*r', t, saw_function(t));
xlabel('Time [s]');
ylabel('Measurement');
legend('measurements', 'ideal');
% Perform fast-Fourier transform (and plot it)
dft = fft(saw_wave);
freq = 0:(f_s/length(saw_wave)):(f_s/2);
dft = dft(1:(length(saw_wave)/2+1));
figure();
plot(freq, abs(dft));
xlabel('Freqency [Hz]');
ylabel('FFT of Measurement');
% Estimate fundamental frequency:
[~, idx] = max(abs(dft));
peak_f = abs(freq(idx));
peak_period = 1/peak_f;
disp(strcat('Estimated period [s]: ', num2str(peak_period)))
Which outputs a couple of graphs, and also the estimated period of the saw-tooth wave. You can play around with the amount of noise and see that it correctly gets a period of 50 seconds till very high levels of noise.
Estimated period [s]: 50
I'm attempting to use scale space implementation to fit n Gaussian curves to peaks in a noisy time series digital signal (measuring voltage).
To test it I created the following sample sum of three gaussians with noise (0.2*rand, sorry no picture, i'm new here)
amp = [2; 0.9; 1.3];
mu = [19; 23; 28];
sigma = [4.8; 1.3; 2.5];
x = linspace(1,50,1000);
for n=1:3, y(n,:) = A(n)*exp(-(x-B(n)).^2./(2*C(n)^2)); end
noisysignal = y(1,:) + y(2,:) + y(3,:) + 0.2*rand(1,numel(x))
I found this article http://www.engineering.wright.edu/~agoshtas/GMIP94.pdf posted by user355856 answer to thread "Peak decomposition"!
I believe my code generates the correct result for plotting the zero crossings as a function of the gaussian filter resolution sigma, but I have two issues. The first is that it seems yet another fitting routine would be needed to identify the approximate location of the arch intercepts for approximating the initial peak sigma and mu values. The second is that the edges of the scale space plot have substantial arches that definitely do not correspond to any peak. I'm not sure how to screen these out effectively. Last thing is that is used a spacing of 50 when calculating the second derivative central finite difference since too much more destroyed feature, and to much less results in a forest of zero crossings. Would there be a better way to filter that to control random zero crossings in the gaussian peak tails?
function [crossing] = scalespace(x, y, sigmalimit)
figure; hold on; ylim([0 sigmalimit]);
for sigma = 1:sigmalimit %
yconv = convkernel(sigma, y); %convolve with kernel
xconv = linspace(x(1), x(end), length(yconv));
yconvpp = d2centralfinite(xconv, yconv, 50); % 50 was empirically chosen
num = 0;
for i = 1 : length(yconvpp)-1
if sign(yconvpp(i)) ~= sign(yconvpp(i+1))
crossing(sigma, num+1) = xconv(i);
num = num+1;
end
end
plot(crossing(sigma, crossing(sigma, :) ~= 0),...
sigma*ones(1, numel(crossing(sigma, crossing(sigma, :) ~= 0))), '.');
end
function [yconv] = convkernel(sigma, y)
t = sigma^2;
C = 3; % for kernel truncation
M = C*round(sqrt(t))+1;
window = (-M) : (+M);
G = zeros(1, length(window));
G(:) = (1/(2*pi()*t))*exp(-(window.^2)./(2*t));
yconv = conv(G, y);
This is my first post and I apologize in advance for any issues in style. I'm fairly new to programming, so any advice regarding the programming style or information provided in this question would be much appreciated. I also read through Amro's answer about matlab's GMM function! if anyone feels that would be a more efficient approach to modeling multiple gaussians in a digital signal.
Thank you!
This is the first time I'm using the fft function and I'm trying to plot the frequency spectrum of a simple cosine function:
f = cos(2*pi*300*t)
The sampling rate is 220500. I'm plotting one second of the function f.
Here is my attempt:
time = 1;
freq = 220500;
t = 0 : 1/freq : 1 - 1/freq;
N = length(t);
df = freq/(N*time);
F = fftshift(fft(cos(2*pi*300*t))/N);
faxis = -N/2 / time : df : (N/2-1) / time;
plot(faxis, real(F));
grid on;
xlim([-500, 500]);
Why do I get odd results when I increase the frequency to 900Hz? These odd results can be fixed by increasing the x-axis limits from, say, 500Hz to 1000Hz. Also, is this the correct approach? I noticed many other people didn't use fftshift(X) (but I think they only did a single sided spectrum analysis).
Thank you.
Here is my response as promised.
The first or your questions related to why you "get odd results when you increase the frequency to 900 Hz" is related to the Matlab's plot rescaling functionality as described by #Castilho. When you change the range of the x-axis, Matlab will try to be helpful and rescale the y-axis. If the peaks lie outside of your specified range, matlab will zoom in on the small numerical errors generated in the process. You can remedy this with the 'ylim' command if it bothers you.
However, your second, more open question "is this the correct approach?" requires a deeper discussion. Allow me to tell you how I would go about making a more flexible solution to achieve your goal of plotting a cosine wave.
You begin with the following:
time = 1;
freq = 220500;
This raises an alarm in my head immediately. Looking at the rest of the post, you appear to be interested in frequencies in the sub-kHz range. If that is the case, then this sampling rate is excessive as the Nyquist limit (sr/2) for this rate is above 100 kHz. I'm guessing you meant to use the common audio sampling rate of 22050 Hz (but I could be wrong here)?
Either way, your analysis works out numerically OK in the end. However, you are not helping yourself to understand how the FFT can be used most effectively for analysis in real-world situations.
Allow me to post how I would do this. The following script does almost exactly what your script does, but opens some potential on which we can build . .
%// These are the user parameters
durT = 1;
fs = 22050;
NFFT = durT*fs;
sigFreq = 300;
%//Calculate time axis
dt = 1/fs;
tAxis = 0:dt:(durT-dt);
%//Calculate frequency axis
df = fs/NFFT;
fAxis = 0:df:(fs-df);
%//Calculate time domain signal and convert to frequency domain
x = cos( 2*pi*sigFreq*tAxis );
F = abs( fft(x, NFFT) / NFFT );
subplot(2,1,1);
plot( fAxis, 2*F )
xlim([0 2*sigFreq])
title('single sided spectrum')
subplot(2,1,2);
plot( fAxis-fs/2, fftshift(F) )
xlim([-2*sigFreq 2*sigFreq])
title('whole fft-shifted spectrum')
You calculate a time axis and calculate your number of FFT points from the length of the time axis. This is very odd. The problem with this approach, is that the frequency resolution of the fft changes as you change the duration of your input signal, because N is dependent on your "time" variable. The matlab fft command will use an FFT size that matches the size of the input signal.
In my example, I calculate the frequency axis directly from the NFFT. This is somewhat irrelevant in the context of the above example, as I set the NFFT to equal the number of samples in the signal. However, using this format helps to demystify your thinking and it becomes very important in my next example.
** SIDE NOTE: You use real(F) in your example. Unless you have a very good reason to only be extracting the real part of the FFT result, then it is much more common to extract the magnitude of the FFT using abs(F). This is the equivalent of sqrt(real(F).^2 + imag(F).^2).**
Most of the time you will want to use a shorter NFFT. This might be because you are perhaps running the analysis in a real time system, or because you want to average the result of many FFTs together to get an idea of the average spectrum for a time varying signal, or because you want to compare spectra of signals that have different duration without wasting information. Just using the fft command with a value of NFFT < the number of elements in your signal will result in an fft calculated from the last NFFT points of the signal. This is a bit wasteful.
The following example is much more relevant to useful application. It shows how you would split a signal into blocks and then process each block and average the result:
%//These are the user parameters
durT = 1;
fs = 22050;
NFFT = 2048;
sigFreq = 300;
%//Calculate time axis
dt = 1/fs;
tAxis = dt:dt:(durT-dt);
%//Calculate frequency axis
df = fs/NFFT;
fAxis = 0:df:(fs-df);
%//Calculate time domain signal
x = cos( 2*pi*sigFreq*tAxis );
%//Buffer it and window
win = hamming(NFFT);%//chose window type based on your application
x = buffer(x, NFFT, NFFT/2); %// 50% overlap between frames in this instance
x = x(:, 2:end-1); %//optional step to remove zero padded frames
x = ( x' * diag(win) )'; %//efficiently window each frame using matrix algebra
%// Calculate mean FFT
F = abs( fft(x, NFFT) / sum(win) );
F = mean(F,2);
subplot(2,1,1);
plot( fAxis, 2*F )
xlim([0 2*sigFreq])
title('single sided spectrum')
subplot(2,1,2);
plot( fAxis-fs/2, fftshift(F) )
xlim([-2*sigFreq 2*sigFreq])
title('whole fft-shifted spectrum')
I use a hamming window in the above example. The window that you choose should suit the application http://en.wikipedia.org/wiki/Window_function
The overlap amount that you choose will depend somewhat on the type of window you use. In the above example, the Hamming window weights the samples in each buffer towards zero away from the centre of each frame. In order to use all of the information in the input signal, it is important to use some overlap. However, if you just use a plain rectangular window, the overlap becomes pointless as all samples are weighted equally. The more overlap you use, the more processing is required to calculate the mean spectrum.
Hope this helps your understanding.
Your result is perfectly right. Your frequency axis calculation is also right. The problem lies on the y axis scale. When you use the function xlims, matlab automatically recalculates the y scale so that you can see "meaningful" data. When the cosine peaks lie outside the limit you chose (when f>500Hz), there are no peaks to show, so the scale is calculated based on some veeeery small noise (here at my computer, with matlab 2011a, the y scale was 10-16).
Changing the limit is indeed the correct approach, because if you don't change it you can't see the peaks on the frequency spectrum.
One thing I noticed, however. Is there a reason for you to plot the real part of the transform? Usually, it is abs(F) that gets plotted, and not the real part.
edit: Actually, you're frequency axis is only right because df, in this case, is 1. The faxis line is right, but the df calculation isn't.
The FFT calculates N points from -Fs/2 to Fs/2. So N points over a range of Fs yields a df of Fs/N. As N/time = Fs => time = N/Fs. Substituting that on the expression of df you used: your_df = Fs/N*(N/Fs) = (Fs/N)^2. As Fs/N = 1, the final result was right :P
I have a vector of data, which contains integers in the range -20 20.
Bellow is a plot with the values:
This is a sample of 96 elements from the vector data. The majority of the elements are situated in the interval -2, 2, as can be seen from the above plot.
I want to eliminate the noise from the data. I want to eliminate the low amplitude peaks, and keep the high amplitude peak, namely, peaks like the one at index 74.
Basically, I just want to increase the contrast between the high amplitude peaks and low amplitude peaks, and if it would be possible to eliminate the low amplitude peaks.
Could you please suggest me a way of doing this?
I have tried mapstd function, but the problem is that it also normalizes that high amplitude peak.
I was thinking at using the wavelet transform toolbox, but I don't know exact how to reconstruct the data from the wavelet decomposition coefficients.
Can you recommend me a way of doing this?
One approach to detect outliers is to use the three standard deviation rule. An example:
%# some random data resembling yours
x = randn(100,1);
x(75) = -14;
subplot(211), plot(x)
%# tone down the noisy points
mu = mean(x); sd = std(x); Z = 3;
idx = ( abs(x-mu) > Z*sd ); %# outliers
x(idx) = Z*sd .* sign(x(idx)); %# cap values at 3*STD(X)
subplot(212), plot(x)
EDIT:
It seems I misunderstood the goal here. If you want to do the opposite, maybe something like this instead:
%# some random data resembling yours
x = randn(100,1);
x(75) = -14; x(25) = 20;
subplot(211), plot(x)
%# zero out everything but the high peaks
mu = mean(x); sd = std(x); Z = 3;
x( abs(x-mu) < Z*sd ) = 0;
subplot(212), plot(x)
If it's for demonstrative purposes only, and you're not actually going to be using these scaled values for anything, I sometimes like to increase contrast in the following way:
% your data is in variable 'a'
plot(a.*abs(a)/max(abs(a)))
edit: since we're posting images, here's mine (before/after):
You might try a split window filter. If x is your current sample, the filter would look something like:
k = [L L L L L L 0 0 0 x 0 0 0 R R R R R R]
For each sample x, you average a band of surrounding samples on the left (L) and a band of surrounding samples on the right. If your samples are positive and negative (as yours are) you should take the abs. value first. You then divide the sample x by the average value of these surrounding samples.
y[n] = x[n] / mean(abs(x([L R])))
Each time you do this the peaks are accentuated and the noise is flattened. You can do more than one pass to increase the effect. It is somewhat sensitive to the selection of the widths of these bands, but can work. For example:
Two passes:
What you actually need is some kind of compression to scale your data, that is: values between -2 and 2 are scale by a certain factor and everything else is scaled by another factor. A crude way to accomplish such a thing, is by putting all small values to zero, i.e.
x = randn(1,100)/2; x(50) = 20; x(25) = -15; % just generating some data
threshold = 2;
smallValues = (abs(x) <= threshold);
y = x;
y(smallValues) = 0;
figure;
plot(x,'DisplayName','x'); hold on;
plot(y,'r','DisplayName','y');
legend show;
Please do not that this is a very nonlinear operation (e.g. when you have wanted peaks valued at 2.1 and 1.9, they will produce very different behavior: one will be removed, the other will be kept). So for displaying, this might be all you need, for further processing it might depend on what you are trying to do.
To eliminate the low amplitude peaks, you're going to equate all the low amplitude signal to noise and ignore.
If you have any apriori knowledge, just use it.
if your signal is a, then
a(abs(a)<X) = 0
where X is the max expected size of your noise.
If you want to get fancy, and find this "on the fly" then, use kmeans of 3. It's in the statistics toolbox, here:
http://www.mathworks.com/help/toolbox/stats/kmeans.html
Alternatively, you can use Otsu's method on the absolute values of the data, and use the sign back.
Note, these and every other technique I've seen on this thread is assuming you are doing post processing. If you are doing this processing in real time, things will have to change.