I need to do frequency analysis on an integrating process where input is torque and output is position. If the input is a sinusoid, the output becomes something like this:
The code I use to extract amplitude ratio and phase looks like this:
freq = 40;
freq_rad = freq * 2 * pi
phase_offset_rad = 30 * pi / 180
gain = 0
fs = 500;
L = 100;
t = (0:L-1)*(1/fs);
in = 2 * sin(freq * 2 * pi * t);
pos_in = [];
vel = 0;
pos = 0;
for i = 1:length(t)
vel = vel + in(i);
pos = pos + vel;
pos_in = [pos_in; pos];
end
out = pos_in;
%out = (2 + gain) * sin(freq * 2 * pi * t + phase_offset_rad);
fft_in = fft(in);
fft_out = fft(out);
[mag_in idx_in] = max(abs(fft_in));
[mag_out idx_out] = max(abs(fft_out));
phase = angle(fft_out(idx_out)) - angle(fft_in(idx_in))
phase_deg = phase / (pi / 180)
ratio = mag_out / mag_in
If I run it on perfectly straight sinusoidal signals then it works perfectly. But as soon as I add a distortion like above, both phase and amplitude values are not right. I think I need to somehow "flatten" the signal. But I'm not sure how to extract correct amplitude from it. What is the amplitude? I would say in the output it is ~45 measured from one "plateau" to the next since that's how far the thing moves. That would be ratio of ~22.5. However the result of the calculation is 196.
Maybe I'm thinking about it wrong? I want to eventually derive a transfer function from the torque input to the positional output using experimental data. Perhaps someone can show how to do that instead?
I have been thinking that what I could do is record amplitude ratio and phase and then make a bode plot and from that easily extract the transfer function. I have so far not been able to get as far as getting a bode plot out of running tests with varying input frequencies.
Because FFT assumes you performing frequency analysis of perfectly periodical signal (exactly one period of the signal) your fft(out) will contain very big power disturbance (see the Periodicity and Shift theorem).
I believe, your case you can avoid of FFT analysis artifacts by performing some system modification. Instead of estimating the transfer function of the system, you can estimate the transfer function of the system + filter. I.e. you have to pass the out signal of your system through the high-pass filter:
out = filter([1 -1], 1, out);
Then, you can perform your analysis.
The frequency response of the filter you can estimate trough the freqz function (or just H = fft([1 -1], length(out)); in my case). Then you can eliminate the filter influence in the frequency domain by inverse response applying fft_out = fft_out ./ H(:);. Also, do not forget to nullify the 0-th frequency fft_out(1) = 0; before maximum estimation.
By the way, the phase difference estimation of the different frequencies looks strange (in your code phase = angle(fft_out(idx_out)) - angle(fft_in(idx_in))). Looks like you must use idx_in (or idx_out, depends what estimation is more reliable) for bout angles.
Note: this answer is not the complete guide and some real life enhancements possible will be required.
P.S. Try applying windowing for frequency response estimation in real world applications (for example the Hamming window).
P.P.S Try ask your question at the https://dsp.stackexchange.com/
Update:
In some cases, instead of neglecting filter influence, you can perform same linear input signal transformation of the input signal: in = filter([1 -1], 1, in);
Related
how come the spectrogram of this code is at its max at approximately 4400Hz instead of 2400Hz in the last timestep of the stft? (frequencyCourse(end) = 100, meshingOrder = 24 -> f = 2400)
startTime = 0; %s
endTime = 30; %s
startIAS = 15; %Hz
endIAS = 100; %Hz
meshingOrder = 24;
fs = 100000; %Hz
t = startTime:1/fs:endTime-1/fs;
frequencyCourse = linspace(startIAS, endIAS, length(t));
signal = cos(2*pi*meshingOrder*frequencyCourse.*t);
spectrogram(signal, hanning(2^13), 0, 2^14, fs, 'yaxis')
Here's a picture:
It works fine as long as I use chirp instead of my self constructed signal, it's not an option though, since more specific courses are to come.
Summary
The problem is that the instantaneous phase is the integral of the instantaneaous frequency with respect to time, not the instantaneous frequency multiplied by time.
You should compute the signal as
signal = cos(2*pi*meshingOrder*cumtrapz(t, frequencyCourse));
What your code does
In your example, you seem to want to generate a linear chirp with initial frequency meshingOrder*startIAS and final frequency meshingOrder*endIAS. But that's not the code is doing.
In your computed signal, the instantaneous phase is the argument to the cos function:
2*pi*meshingOrder*frequencyCourse.*t
Since the variable frequencyCourse increases from meshingOrder*startIAS at start time (which is 0) to meshingOrder*endIAS at end time, this can be expressed as
2*pi*(A+B*t).*t
where A = meshingOrder*startIAS and B = meshingOrder*(endIAS-startIAS)/endTime. Differentiating the instantanous phase with respect to t gives an instantaneous frequency
A + 2*B*t
that is
meshingOrder*startIAS + 2*meshingOrder*(endIAS-startIAS)/endTime * t
As you can see, the problem is the factor 2 here. At end time the instantaneous frequency is
meshingOrder*startIAS + 2*meshingOrder*(endIAS-startIAS)
that is
2*meshingOrder*endIAS - meshingOrder*startIAS
In your example this is 4440 Hz, which is in accordance with your observed value.
What the code should do
For a linear chirp (or a chirp with any other simple frequency variation, such as a quadratic or exponential) you could compute the correct instantaneous phase that gives rise to the desired instantaneous frequency. See for example here. This is also what the chirp function internally does.
But you seem to be want to deal with arbitrary frequency courses. To do that, given arbitrary t, just compute the argument of the cos as the cumulative integral of frequencyCourse with respect to t. This is easily done with cumtrapz:
signal = cos(2*pi*meshingOrder*cumtrapz(t, frequencyCourse));
Changing this line in your example gives the following figure, which has the expected frequency variation from 360 Hz to 2400 Hz:
I have generated three identical waves with a phase shift in each. For example:
t = 1:10800; % generate time vector
fs = 1; % sampling frequency (seconds)
A = 2; % amplitude
P = 1000; % period (seconds), the time it takes for the signal to repeat itself
f1 = 1/P; % number of cycles per second (i.e. how often the signal repeats itself every second).
y1 = A*sin(2*pi*f1*t); % signal 1
phi = 10; % phase shift
y2 = A*sin(2*pi*f1*t + phi); % signal 2
phi = 15; % phase shift
y3 = A*sin(2*pi*f1*t + phi); % signal 3
YY = [y1',y2',y3'];
plot(t,YY)
I would now like to use a method for detecting this phase shift between the waves. The point of doing this is so that I can eventually apply the method to real data and identify phase shifts between signals.
So far I have been thinking of computing the cross spectra between each wave and the first wave (i.e. without the phase shift):
for i = 1:3;
[Pxy,Freq] = cpsd(YY(:,1),YY(:,i));
coP = real(Pxy);
quadP = imag(Pxy);
phase(:,i) = atan2(coP,quadP);
end
but I'm not sure if this makes any sense.
Has anyone else done something similar to this? The desired outcome should show a phase shift at 10 and 15 for waves 2 and 3 respectively.
Any advice would be appreciated.
There are several ways that you can measure the phase shift between signals. Between your response, the comments below your response, and the other answers, you've gotten most of the options. The specific choice of technique is usually based on issues such as:
Noisy or Clean: Is there noise in your signal?
Multi-Component or Single-Component: Are there more than one type of signal within your recording (multiple tones at multiple frequencies moving in different directions)? Or, is there just a single signal, like in your sine-wave example?
Instantaneous or Averaged: Are you looking for the average phase lag across your entire recording, or are you looking to track how the phase changes throughout the recording?
Depending on your answer to these questions, you could consider the following techniques:
Cross-Correlation: Use the a command like [c,lag]=xcorr(y1,y2); to get the cross-correlation between the two signals. This works on the original time-domain signals. You look for the index where c is maximum ([maxC,I]=max(c);) and then you get your lag value in units of samples lag = lag(I);. This approach gives you the average phase lag for the entire recording. It requires that your signal of interest in the recording be stronger than anything else in your recording...in other words, it is sensitive to noise and other interference.
Frequency Domain: Here you convert your signals into the frequency domain (using fft or cpsd or whatever). Then, you'd find the bin that corresponds to the frequency that you care about and get the angle between the two signals. So, for example, if bin #18 corresponds to your signal's frequency, you'd get the phase lag in radians via phase_rad = angle(fft_y1(18)/fft_y2(18));. If your signals have a constant frequency, this is an excellent approach because it naturally rejects all noise and interference at other frequencies. You can have really strong interference at one frequency, but you can still cleanly get your signal at another frequency. This technique is not the best for signals that change frequency during the fft analysis window.
Hilbert Transform: A third technique, often overlooked, is to convert your time-domain signal into an analytic signal via the Hilbert transform: y1_h = hilbert(y1);. Once you do this, your signal is a vector of complex numbers. A vector holding a simple sine wave in the time domain will now be a vector of complex numbers whose magnitude is constant and whose phase is changing in sync with your original sine wave. This technique allows you to get the instantaneous phase lag between two signals...it's powerful: phase_rad = angle(y1_h ./ y2_h); or phase_rad = wrap(angle(y1_h) - angle(y2_h));. The major limitation to this approach is that your signal needs to be mono-component, meaning that your signal of interest must dominate your recording. Therefore, you may have to filter out any substantial interference that might exist.
For two sinusoidal signal the phase of the complex correlation coefficient gives you what you want. I can only give you an python example (using scipy) as I don't have a matlab to test it.
x1 = sin( 0.1*arange(1024) )
x2 = sin( 0.1*arange(1024) + 0.456)
x1h = hilbert(x1)
x2h = hilbert(x2)
c = inner( x1h, conj(x2h) ) / sqrt( inner(x1h,conj(x1h)) * inner(x2h,conj(x2h)) )
phase_diff = angle(c)
There is a function corrcoeff in matlab, that should work, too (The python one discard the imaginary part). I.e. c = corrcoeff(x1h,x2h) should work in matlab.
The Matlab code to find relative phase using cross-correlation:
fr = 20; % input signal freq
timeStep = 1e-4;
t = 0:timeStep:50; % time vector
y1 = sin(2*pi*t); % reference signal
ph = 0.5; % phase difference to be detected in radians
y2 = 0.9 * sin(2*pi*t + ph); % signal, the phase of which, is to be measured relative to the reference signal
[c,lag]=xcorr(y1,y2); % calc. cross-corel-n
[maxC,I]=max(c); % find max
PH = (lag(I) * timeStep) * 2 * pi; % calculated phase in radians
>> PH
PH =
0.4995
With the correct signals:
t = 1:10800; % generate time vector
fs = 1; % sampling frequency (seconds)
A = 2; % amplitude
P = 1000; % period (seconds), the time it takes for the signal to repeat itself
f1 = 1/P; % number of cycles per second (i.e. how often the signal repeats itself every second).
y1 = A*sin(2*pi*f1*t); % signal 1
phi = 10*pi/180; % phase shift in radians
y2 = A*sin(2*pi*f1*t + phi); % signal 2
phi = 15*pi/180; % phase shift in radians
y3 = A*sin(2*pi*f1*t + phi); % signal 3
The following should work:
>> acos(dot(y1,y2)/(norm(y1)*norm(y2)))
>> ans*180/pi
ans = 9.9332
>> acos(dot(y1,y3)/(norm(y1)*norm(y3)))
ans = 0.25980
>> ans*180/pi
ans = 14.885
Whether or not that's good enough for your "real" signals, only you can tell.
Here is the little modification of your code: phi = 10 is actually in degree, then in sine function, phase information is mostly expressed in radian,so you need to change deg2rad(phi) as following:
t = 1:10800; % generate time vector
fs = 1; % sampling frequency (seconds)
A = 2; % amplitude
P = 1000; % period (seconds), the time it takes for the signal to repeat itself
f1 = 1/P; % number of cycles per second (i.e. how often the signal repeats itself every second).
y1 = A*sin(2*pi*f1*t); % signal 1
phi = deg2rad(10); % phase shift
y2 = A*sin(2*pi*f1*t + phi); % signal 2
phi = deg2rad(15); % phase shift
y3 = A*sin(2*pi*f1*t + phi); % signal 3
YY = [y1',y2',y3'];
plot(t,YY)
then using frequency domain method as mentioned chipaudette
fft_y1 = fft(y1);
fft_y2 = fft(y2);
phase_rad = angle(fft_y1(1:end/2)/fft_y2(1:end/2));
phase_deg = rad2deg(angle(fft_y1(1:end/2)/fft_y2(1:end/2)));
now this will give you a phase shift estimate with error = +-0.2145
If you know the frequency and just want to find the phase, rather than use a full FFT, you might want to consider the Goertzel algorithm, which is a more efficient way to calculate the DFT for a single frequency (an FFT will calculate it for all frequencies).
For a good implementation, see: https://www.mathworks.com/matlabcentral/fileexchange/35103-generalized-goertzel-algorithm and https://asp-eurasipjournals.springeropen.com/track/pdf/10.1186/1687-6180-2012-56.pdf
If you use an AWGN signal with delay and apply your method it works, but if you are using a single tone frequency estimation will not help you. because there is no energy in any other frequency but the tone. You better use cross-correlation in the time domain for this - it will work better for a fixed delay. If you have a wideband signal you can use subbands domain and estimate the phase from that (it is better than FFT due to low cross-frequency dependencies).
I have a piece of code that gets the FFT of a part of the signal and I'm now trying to get the PSD,
Fs = 44100;
cj = sqrt(-1);
%T=.6;
dt = 1/Fs;
left=test(:,1);
right=test(:,2);
time = 45;
interval =.636;
w_range = time*Fs: (time+interval)*Fs-1;
I = left(w_range);
Q = right(w_range);
n = interval * Fs;
f = -Fs/2:Fs/n:Fs/2-Fs/n;
s = I+cj.*Q;
% Smooth the signal ss = smooth(s,201);
sf = (fftshift(fft(ss(1:n)))); %FFT of signal
figure(1) plot(f,((20*log10((abs(sf))./max(abs(sf))))))
From my understanding, in order to get the PSD I just need to raise sf to the power of 2, or is there anything more I need to perform?
Technically yes, you can obtain the power-spectral density (PSD) of a periodic signal by taking the squared-magnitude of its FFT. Note that if you are going to plot it on a logarithmic decibel scale, there is really no difference between 20*log10(abs(sf)) or 10*log10(abs(sf).^2).
There is however generally more to it in the sense that the PSD estimate computed in this way tends to have a fairly large variance. There are a number of techniques which can be used to improve the estimate. A simple one consists of applying a window to sections of data, perform the FFT, then averaging the resulting PSDs (i.e. averaging the squared-magnitudes).
You are perfectly right. You just have to built the square of the absolute values.
So I have had a few posts the last few days about using MatLab to perform a convolution (see here). But I am having issues and just want to try and use the convolution property of Fourier Transforms. I have the code below:
width = 83.66;
x = linspace(-400,400,1000);
a2 = 1.205e+004 ;
al = 1.778e+005 ;
b1 = 94.88 ;
c1 = 224.3 ;
d = 4.077 ;
measured = al*exp(-((abs((x-b1)./c1).^d)))+a2;
%slit
rect = #(x) 0.5*(sign(x+0.5) - sign(x-0.5));
rt = rect(x/width);
subplot(5,1,1);plot(x,measured);title('imported data-super gaussian')
subplot(5,1,2);plot(x,(real(fftshift(fft(rt)))));title('transformed slit')
subplot(5,1,3);plot(x,rt);title('slit')
u = (fftshift(fft(measured)));
l = u./(real(fftshift(fft(rt))));
response = (fftshift(ifft(l)));
subplot(5,1,4);plot(x,real(response));title('response')
%Data Check
check = conv(rt,response,'full');
z = linspace(min(x),max(x),length(check));
subplot(5,1,5);plot(z,real(check));title('check')
My goal is to take my case, which is $measured = rt \ast signal$ and find signal. Once I find my signal, I convolve it with the rectangle and should get back measured, but I do not get that.
I have very little matlab experience, and pretty much 0 signal processing experience (working with DFTs). So any advice on how to do this would be greatly appreciated!
After considering the problem statement and woodchips' advice, I think we can get closer to a solution.
Input: u(t)
Output: y(t)
If we assume the system is causal and linear we would need to shift the rect function to occur before the response, like so:
rt = rect(((x+270+(83.66/2))/83.66));
figure; plot( x, measured, x, max(measured)*rt )
Next, consider the response to the input. It looks to me to be first order. If we assume as such, we will have a system transfer function in the frequency domain of the form:
H(s) = (b1*s + b0)/(s + a0)
You had been trying to use convolution to and FFT's to find the impulse response, "transfer function" in the time domain. However, the FFT of the rect, being a sinc has a zero crossing periodically. These zero points make using the FFT to identify the system extremely difficult. Due to:
Y(s)/U(s) = H(s)
So we have U(s) = A*sinc(a*s), with zeros, which makes the division go to infinity, which doesn't make sense for a real system.
Instead, let's attempt to fit coefficients to the frequency domain linear transfer function that we postulate is of order 1 since there are no overshoots, etc, 1st order is a reasonable place to start.
EDIT
I realized my first answer here had a unstable system description, sorry! The solution to the ODE is very stiff due to the rect function, so we need to crank down the maximum time step and use a stiff solver. However, this is still a tough problem to solve this way, a more analytical approach may be more tractable.
We use fminsearch to find the continuous time transfer function coefficients like:
function x = findTf(c0,u,y,t)
% minimize the error for the estimated
% parameters of the transfer function
% use a scaled version without an offset for the response, the
% scalars can be added back later without breaking the solution.
yo = (y - min(y))/max(y);
x = fminsearch(#(c) simSystem(c,u,y,t),c0);
end
% calculate the derivatives of the transfer function
% inputs and outputs using the estimated coefficient
% vector c
function out = simSystem(c,u,y,t)
% estimate the derivative of the input
du = diff([0; u])./diff([0; t]);
% estimate the second derivative of the input
d2u = diff([0; du])./diff([0; t]);
% find the output of the system, corresponds to measured
opt = odeset('MaxStep',mean(diff(t))/100);
[~,yp] = ode15s(#(tt,yy) odeFun(tt,yy,c,du,d2u,t),t,[y(1) u(1) 0],opt);
% find the error between the actual measured output and the output
% from the system with the estimated coefficients
out = sum((yp(:,1) - y).^2);
end
function dy = odeFun(t,y,c,du,d2u,tx)
dy = [c(1)*y(3)+c(2)*y(2)-c(3)*y(1);
interp1(tx,du,t);
interp1(tx,d2u,t)];
end
Something like that anyway should get you going.
x = findTf([1 1 1]',rt',measured',x');
I'm testing the phase output of an fft of a sin signal and a cos signal.
The script below creates the signals and performs an FFT on them. Bins who's amplitude is below a threshold are zeroed for the phase spectrum because I am only interested in the phase of the signals.
% 10khz 10 second long time interval
t = 0:1 / 10000:10;
%1khz cos
c = cos(2 * pi * 1000 .* t);
%1khz sin
s = sin(2 * pi * 1000 .* t);
%ffts
C = fft(c)/length(c);
S = fft(s)/length(s);
%magnitude and phases of ffts
CA = abs(C); %cos magnitude
SA = abs(S); %sin magnitude
Cthresh = max(CA) * 0.5;
Sthresh = max(SA) * 0.5;
%find all indeces below the threshold
Crange = find(CA < Cthresh);
Srange = find(SA < Sthresh);
%set the indeces below the threshold to 0 - phase will be meaningless for
%noise values
CP = angle(C);
CP(Crange) = 0;
SP = angle(S);
SP(Srange) = 0;
If you plot CP - the phase of the cos - you will get a phase of 0.3142 in the bins corresponding to the frequency of the cos signal and zeros elsewhere. This is pi/10. I'm expecting to get pi. Why is this?
If you plot SP you get values of 1.2566. I'm expecting to get pi/2 or 1.5708. 80% of the expected value. What is causing these errors?
If your input signal is not perfectly periodic in the FFT aperture length (an exact integer number of full periods), the sinusoids will be discontinuous across the ends of the FFT aperture. Thus you will get a phase that is the average of the two different phases at both ends of the FFT input vector.
If you want a more sensible phase, reference the phase of your sinusoids to the center of the FFT input vector, and do an fft shift before the FFT. This will result in a continuous sinusoid at the zero phase reference position, with a single phase instead of a weird average value.
Also note that matlab may reference the phase to the second point in a sampled sinusoid, e.g. vectorElement[i=1], not the first, vectorElement[i=0]. This will have a phase of pi/10 for a sinusoid of period = 20 samples.
The issue you have is exactly what hotpaw2 has stated. You have 100001 samples in t, so you do not have a perfectly periodic signal, therefore you have leakage. That means that you've got a sin()/sin() function from your implicit rectangular window convolved with your solution. That's what changes your phase.
If instead you try the following:
t = 0:1 / 10000:9.9999;
c = cos(2 * pi * 1000 .* t);
%1khz sin
s = sin(2 * pi * 1000 .* t);
%ffts
C = fft(c)/length(c);
S = fft(s)/length(s);
you would find that the phase of the cosine is zero (which is what you would expect) and that the phase of sine is pi/2.
Performing a linear shift in the time domain (using fftshift) will merely introduce a linear phase term in the frequency domain, and will not resolve the original problem.
In practice, rather than trying to set the length of the sequence precisely to match the period of the signal, windowing should be applied if the signal is to be examined in the frequency domain. In that case you really should make sure that your signals are appropriately aligned so that the window attenuates the end points, thus smoothing out the discontinuity. This has the effect of broadening the main lobe of the FFT, but it also controls the leakage.