Here is a mass-spring-damper system with an impulse response, h and an arbitrary forcing function, f (cos(t) in this case). I am trying to use Matlab's FFT function in order to perform convolution in the frequency domain. I am expecting for the output (ifft(conv)) to be the solution to the mass-spring-damper system with the specified forcing, however my plot looks completely wrong! So, i must be implementing something wrong. Please help me find my errors in my code below! Thanks
clear
%system parameters
m=4;
k=256;
c=1;
wn=sqrt(k/m);
z=c/2/sqrt(m*k);
wd=wn*sqrt(1-z^2);
w=sqrt(4*k*m-c^2)/(2*m);
x0=0; %initial conditions
v0=0;
%%%%%%%%%%%%%%%%%%%%%%%%%
t=0:.01:2*pi ;%time vector
f=[cos(t),zeros(1,length(t)-1)]; %force f
F=fft(f);
h=[1/m/wd*exp(-z*wn*t).*sin(wd*t),zeros(1,length(t)-1)]; %impulse response
H=fft(h);
conv=H.*F; %convolution is multiplication in freq domain
plot(0:.01:4*pi,ifft(conv))
To see what is expected run this code. Enter in cos(t); 4; 1; 256 for the inputs. You can see that it reaches a steady state at an amplitude much different than the plot generated from the above FFT code.
%%%FOR UNDERDAMPED SYSTEMS
func=input('enter function of t---> ','s');
m=input('mass ');
c=input('c ');
k=input('k ');
z=c/2/sqrt(k*m);
wn=sqrt(k/m);
wd=wn*sqrt(1-z^2);
dt=.001;
tMax=20;
x0=0;
v0=0;
t0=0;
t=0:dt:tMax;
X(:,1)=[x0;v0;t0];
functionForce=inline(func);
tic
for i=1:length(t)-1
a=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[X(1,i);X(2,i);X(3,i)]+[0;functionForce(X(3,i));0]);
Xtemp=X(:,i)+[0;0;dt/2] + a*dt/2;
b=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[Xtemp(1);Xtemp(2);Xtemp(3)]+[0;functionForce(X(3,i));0]);
Xtemp=Xtemp+[0;0;dt/2] + b*dt/2;
c=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[Xtemp(1);Xtemp(2);Xtemp(3)]+[0;functionForce(X(3,i));0]);
Xtemp=Xtemp + [0;0;dt] +c*dt;
d=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[Xtemp(1);Xtemp(2);Xtemp(3)]+[0;functionForce(X(3,i));0]);
X(:,i+1)=X(:,i)+(a+2*b+2*c+d)*dt/6+[0;0;dt];
end
toc
figure(1)
axis([0 tMax min(X(1,:)) max(X(1,:))])
plot(t,X(1,:))
The initial transient will appear in the FFT method, so you will need to increase the time span (eg t=0:0.01:15*pi) to ensure the result includes the steady state. Incidentally since you truncate h after the same duration, increasing the time span also makes your impulse response h a better approximation to the actual infinite length impulse response.
So, updating your code to:
T=15*pi;
N=floor(T/.01);
t=[0:N-1]*0.01; ;%time vector
...
plot([0:2*N-2]*0.01, real(ifft(conv)));
axis([0 T]); % only show the duration where the driving force is active
would correspondingly show the following response:
which shows the initial transient, and eventually the steady state. You may notice that the plot is similar to your alternate implementation up to a scaling factor.
This difference in scaling comes from two factors. The first one is simply due to the fact that the convolution in your FFT based implementation computes a sum which isn't weight by the time step (as compare with the dt scaling used in your second implementation). The second one comes from the fact that the second implementation does not account for the mass m for the effect of the driving force.
After accounting for those two factors, you should get the following responses:
where the blue curve represents the FFT based implementation (same as the above curve but scaled down by dt=0.01), and the red curve represents your alternate implementation (with the functionForce divided by m).
Related
I've been running a variation of the doseResponse function downloaded from here to generate dose-response sigmoid curves. However, I've had trouble with one of my datasets generating a linear curve instead. By running the following code, I get the following error and produce the following graph. I also uploaded the data called dose2.csv and resp2.csv to google drive here. Does anyone know how I can fix this? Thanks.
Code to generate graph
% Plotting Dose-Response Curve
response = resp2;
dose = dose2;
% Deal with 0 dosage by using it to normalise the results.
normalised=0;
if (sum(dose(:)==0)>0)
%compute mean control response
controlResponse=mean(response(dose==0));
%remove controls from dose/response curve
response=response(dose~=0)/controlResponse;
dose=dose(dose~=0);
normalised=1;
end
%hill equation sigmoid
sigmoid=#(beta,x)beta(1)+(beta(2)-beta(1))./(1+(x/beta(3)).^beta(4));
%calculate some rough guesses for initial parameters
minResponse=min(response);
maxResponse=max(response);
midResponse=mean([minResponse maxResponse]);
minDose=min(dose);
maxDose=max(dose);
%fit the curve and compute the values
%[coeffs,r,J]=nlinfit(dose,response,sigmoid,[minResponse maxResponse midResponse 1]); % nlinfit doesn't work as well
beta_new = lsqcurvefit(sigmoid,[minResponse maxResponse midResponse 1],dose,response);
[coeffs,r,J]=nlinfit(dose,response,sigmoid, beta_new);
ec50=coeffs(3);
hillCoeff=coeffs(4);
%plot the fitted sigmoid
xpoints=logspace(log10(minDose),log10(maxDose),1000);
semilogx(xpoints,sigmoid(coeffs,xpoints),'Color',[1 0 0],'LineWidth',2)
hold on
%notate the EC50
text(ec50,mean([coeffs(1) coeffs(2)]),[' \leftarrow ' sprintf('EC_{50}=%0.2g',ec50)],'FontSize',20,'Color',[1 0 0]);
%plot mean response for each dose with standard error
doses=unique(dose);
meanResponse=zeros(1,length(doses));
stdErrResponse=zeros(1,length(doses));
for i=1:length(doses)
responses=response(dose==doses(i));
meanResponse(i)=mean(responses);
stdErrResponse(i)=std(responses)/sqrt(length(responses));
%stdErrResponse(i)=std(responses);
end
errorbar(doses,meanResponse,stdErrResponse,'o','Color',[1 0 0],'LineWidth',2,'MarkerSize',12)
Warning Message
Solver stopped prematurely.
lsqcurvefit stopped because it exceeded the function evaluation limit,
options.MaxFunctionEvaluations = 4.000000e+02.
Warning: Iteration limit exceeded. Returning results from final iteration.
Graph (looking to generate a sigmoid curve not linear)
You also need to optimize your initial value [minResponse maxResponse midResponse 1] for lsqcurvefit. Don't just simply start with minimum or maximum values of given values. Instead, you may first start with your equations to estimate your coefficients.
Given the sigmoid model of sigmoid=#(beta,x)beta(1)+(beta(2)-beta(1))./(1+(x/beta(3)).^beta(4)). As x gets arbitrarily close to inf, equation will return beta(2). And as x gets arbitrarily close to 0, equation will return beta(1). Therefore, initial estimation of minResponse, maxResponse, and midResponse seems reasonable enough. Actually your problem lies in your initial estimation of 1. beta(4) can be roughly estimated with the inclination of your log graph. To my rough sketch it was around 1/4 and therefore you may conclude that your initial estimation of 1 was too large for convergence.
beta_new = lsqcurvefit(sigmoid,[minResponse maxResponse midResponse 1/4],dose,response);
I'm working in the space of biosignal acquisition. I made a experiment as detailed below, and am now trying to obtain some results from the data.
I have a text file of a signal in Matlab. I loaded the signal onto a waveform generator, then I recorded the generator output on an oscilloscope.
I imported the recorded signal from the oscilloscope back into Matlab.
The Pearson's correlation coefficient between the original signal and the oscilloscope signal is 0.9958 (obtained using corrcoeff function).
I want to compute the SNR of the oscilloscope signal (what I'm calling my signal plus whatever noise is introduced through the digital-to-analog conversion and visa-versa). I have attached a snippet of the 2 signals for reference.
So my original signal is X and oscilloscope signal is X + N.
I used the snr function to compute SNR as follows.
snr(original, (oscilloscope - original))
The result I got was 20.44 dB.
This seems off to me as I would have thought with such a high correlation, that the SNR should be much higher?
Or is it not appropriate to try and compute SNR in this sort of situation?
All help is appreciated.
Thanks
Edit: Graph of a couple of results vs Sleutheye's simulated relationship
You might be surprised at just how even such moderate SNR can still result in fairly high correlations.
I ran an experiment to illustrate the approximate relation between correlation and signal-to-noise-ratio estimate. Since I did not have your specific EEG signal, I just used a reference constant signal and some white Gaussian noise. Keep in mind that the relationship could be affected by the nature of the signal and noise, but it should give you an idea of what to expect. This simulation can be executed with the following code:
SNR = [10:1:40];
M = 10000;
C = zeros(size(SNR));
for i=1:length(SNR)
x = ones(1,M);
K = sqrt(sum(x.*x)/M)*power(10, -SNR(i)/20);
z = x + K*randn(size(x));
C(i) = xcorr(x,z,0)./sqrt(sum(x.*x)*sum(z.*z));
end
figure(1);
hold off; plot(SNR, C);
corr0 = 0.9958;
hold on; plot([SNR(1) SNR(end)], [corr0 corr0], 'k:');
snr0 = 20.44;
hold on; plot([snr0 snr0], [min(C) max(C)], 'r:');
xlabel('SNR (dB)');
ylabel('Correlation');
The dotted black horizontal line highlights your 0.9958 correlation measurement, and the dotted red vertical line highlights your 20.44 dB SNR result.
I'd say that's a pretty good match!
In fact, for this specific case in my simulation (x = 1; z = x + N(0,σ)) if we denote C(x,z) to be the correlation between x and z, and σ as the noise standard deviation, we can actually show that:
Given a correlation value of 0.9958, this would yield an SNR of 20.79dB, which is consistent with your results.
Lets say I have a mass-spring-damper system...
here is my code (matlab)...
% system parameters
m=4; k=256; c=1; wn=sqrt(k/m); z=c/2/sqrt(m*k); wd=wn*sqrt(1-z^2);
% initial conditions
x0=0; v0=0;
%% time
dt=.001; tMax=2*pi; t=0:dt:tMax;
% input
F=cos(t); Fw=fft(F);
% impulse response function
h=1/m/wd*exp(-z*wn*t).*sin(wd*t); H=fft(h);
% convolution
convolution=Fw.*H; sol=ifft(convolution);
% plot
plot(t,sol)
so I can successfully retrieve a plot, however I am getting strange responses I also programmed a RK4 method that solves the system of differential equations so I know how the plot SHOULD look like, and the plot I am getting from using FFT has an amplitude of a like 2 when it should have an amplitude of like .05.
So, how can I solve for the steady state response for this system using FFT. I want to use FFT because it is about 3 orders of magnitude faster than numerical integration methods.
Keep in mind I am defining my periodic input as cos(t) which has a period of 2*pi that is why I only used FFT over the time vector that spanned 0 to 2*pi (1 period). I also noticed if I changed the tMax time to a multiple of 2*pi, like 10*pi, I got a similar looking plot but the amplitude was 4 rather than 2, either way still not .05!. maybe there is some kind of factor I need to multiply by?
also I plotted : plot(t,Fw) expecting to see one peak at 1 since the forcing function is cos(t), yet I did not see any peaks (maybe I shouldn't be plotting Fw vs t)
I know it is possible to solve for the steady state response using fourier transform / fft, I am just missing something! I need help and understanding!!
The original results
Running the code you provided and comparing the result with the RK4 code posted in your other question, we get the following responses:
where the blue curve represents the FFT based implementation, and the red curve represents your alternate RK4 implementation. As you have pointed out, the curves are quite different.
Getting the correct response
The most obvious problem is of course the amplitude, and the main sources of the amplitude discrepancies of the code posted in this question are the same as the ones I indicated in my answer to your other question:
The RK4 implementation performs a numeric integration which correctly scales the summed values by the integration step dt. This scaling is lacking from the FFT based implementation.
The impulse response used in the FFT based implementation is consistent with the driving force being scaled by the mass m, a factor which was missing from the RK4 implementation.
Fixing those two issues results in responses which are a little closer, but still not identical. As you probably found out given the changes in the posted code of your other question, another thing that was lacking was zero padding of the input and of the impulse response, without which you were getting a circular convolution rather than a linear convolution:
f=[cos(t),zeros(1,length(t)-1)]; %force f
h=[1/m/wd*exp(-z*wn*t).*sin(wd*t),zeros(1,length(t)-1)]; %impulse response
Finally the last element to ensure the convolution yields good result is to use a good approximation to the infinite length impulse response. How long is long enough depends on the rate of decay of the impulse response. With the parameters you provided, the impulse response would have died down to 1% of its original values after approximately 11*pi. So extending the time span to tMax=14*pi (to include a full 2*pi cycle after the impulse response has died down), would probably be enough.
Obtaining the steady-state response
The simplest way to obtain the steady-state response is then to discard the initial transient. In this process we discard a integer number of cycles of the reference driving force (this of course requires knowledge of the driving force's fundamental frequency):
T0 = tMax-2*pi;
delay = min(find(t>T0));
sol = sol(delay:end);
plot([0:length(sol)-1]*dt, sol, 'b');
axis([0 2*pi]);
The resulting responses are then:
where again the blue curve represents the FFT based implementation, and the red curve represents your alternate RK4 implementation. Much better!
An alternate method
Computing the response for many cycles waiting for the transient response to die down and extracting the remaining samples corresponding
to the steady state might appear to be a little wasteful, despite the fact that the computation is still fairly fast thanks to the FFT.
So, let's go back a little and look at the problem domain. As you are probably aware,
the mass-spring-damper system is governed by the differential equation:
where f(t) is the driving force in this case.
Note that the general solution to the homogeneous equation has the form:
The key is then to realize that the general solution in the case where c>0 and m>0 vanishes in the steady state (t going to infinity).
The steady-state solution is thus only dependent on the particular solution to the non-homogenous equation.
This particular solution can be found by the method of undetermined coefficients, for a driving force of the form
by correspondingly assuming that the solution has the form
Substituting in the differential equation yields the equations:
thus, the solution can be implemented as:
EF0 = [wn*wn-w*w 2*z*wn*w; -2*z*wn*w wn*wn-w*w]\[1/m; 0];
sol = EF0(1)*cos(w*t)+EF0(2)*sin(w*t);
plot(t, sol);
where w=2*pi in your case.
Generalization
The above approach can be generalized for more arbitrary periodic driving forces by expressing the driving force as a
Fourier Series (assuming the driving force function satisfies the Dirichlet conditions):
The particular solution can correspondingly be assumed to have the form
Solving for the particular solution can be done in a way very similar to the earlier case. This result in the following implementation:
% normalize
y = F/m;
% compute coefficients proportional to the Fourier series coefficients
Yw = fft(y);
% setup the equations to solve the particular solution of the differential equation
% by the method of undetermined coefficients
k = [0:N/2];
w = 2*pi*k/T;
A = wn*wn-w.*w;
B = 2*z*wn*w;
% solve the equation [A B;-B A][real(Xw); imag(Xw)] = [real(Yw); imag(Yw)] equation
% Note that solution can be obtained by writing [A B;-B A] as a scaling + rotation
% of a 2D vector, which we solve using complex number algebra
C = sqrt(A.*A+B.*B);
theta = acos(A./C);
Ywp = exp(j*theta)./C.*Yw([1:N/2+1]);
% build a hermitian-symmetric spectrum
Xw = [Ywp conj(fliplr(Ywp(2:end-1)))];
% bring back to time-domain (function synthesis from Fourier Series coefficients)
x = ifft(Xw);
A final note
I purposely avoided the undamped c=0 case in the above derivation. In this case, the oscillation never die down and the general solution to the homogeneous equation does not have to be the trivial one.
The final "steady-state" in this case may or may not have the same period as the driving force. As a matter of fact, it may not be periodic at all if the period oscillations from the general solution is not related to the period of the driving force through a rational number (ratio of integers).
I have attempted to write a code in order to solve the following coupled partial differential EM wave equations:
The code employs finite difference time domain using the Yee algorithm which can be read about in the following two online documents:
http://www.eecs.wsu.edu/~schneidj/ufdtd/ufdtd.pdf
http://www.eecs.wsu.edu/~schneidj/ufdtd/chap3.pdf
I want my source at the left hand side boundary to be a sinusoidal wave with parameters as such:
Ex(0,t) = E0 sin(2πft) for 0 ≤ t ≤ 1/f
The code updates the electric and magnetic field properties of the wave with each loop.
My initial code is as follows:
%FDTD Yee algorithm to solve coupled EM wave equations
clear
clc
G=50; %Specify size of the grid
f=10^3; %choose initial frequency of wave in hertz
e=1; %specify permitivity and permeability (normalised condition)
u=1;
Nt=150; %time steps
E0=1; %Electric Field initial amplitude
%Specify step sizes using corruant condition
c=3*10^8;
dx=0.01;
dt=dx/2*c;
%make constant terms
c1=-dt/(dx*e);
c2=-dt/(dx*u);
%create vgector place holders
Ex=zeros(1,G);
Hy=zeros(1,G);
%create updating loop
M=moviein(Nt);
for t=1:Nt
% Spatial Ex
for k=2:G-1
Ex(k)=Ex(k)+c1*(Hy(k)-Hy(k-1));
end
Ex(G)=0; %PEC boundary condition
%E Source at LHS boundary
Ex(1)=E0*sin(2*pi*f*t);
%Spatial Hy
for n=1:G-1
Hy(n)=Hy(n)+c2*(Ex(n)-Ex(n+1));
end
Hy(G)=0; %PMC boundary condition
plot(Ex);
M(:,t) = getframe;
end
movie(M,1);
Basically I want to create an updating movie which shows the sinusoidal wave propagating to the right hand side boundary coded as a perfect electrical conductor; therefore reflecting the wave, and then propagating back to the left hand side boundary coded as a perfect insulator; absorbing the wave.
The problems I have are as follows:
1) I'm not sure how to properly implement my desired source. It don't appear to be purely sinusoidal.
2) The wave I've coded begins to propagate but it quickly disappears for the majority of the simulation. I do not know why this is occurring
3) I do not know much about running a movie simulation and the plot oscillates as the solution is being run. How can I stop this?
Your wave attenuates because the diference equations are not correctly implemented; instead:
Ex(k)=Ex(k)+c1*(Hy(k)-Hy(k-1));
you should use
Ex1(k)=Ex(k)+c1*(Hy(k)-Hy(k-1));
and instead of:
Hy(n)=Hy(n)+c2*(Ex(n)-Ex(n+1));
you should use:
Hy1(n)=Hy(n)+c2*(Ex(n)-Ex(n+1));
and, in the end of the loop update the current "dataframe":
Hy = Hy1;
Ey = Ey1;
(you should take care also the boundary conditions).
If you want a fixed plot frame that doesn't change when your data changes, create first a axis where you can plot into, with a defined xmin/max and ymin/max, see http://www.mathworks.com/help/matlab/ref/axis.html
You should set the Courant number closer to 1 say 0.995. Thus delta_t = 0.995*delta_x/c.
Also assuming delta_x is in METRIC units then e and u should be in metric units.
I do not know about the specific coding language used but in c or c++ there is no need for intermediate variable Ey1 etc.
Also there should be at least 10 samples per wavelength for accuracy ( preferably 60). Thus wavelength = 60*delta_x and thus the frequency equals roughly of the order 10 to power of 9. Also, I think the sinesoidal source should be E0 * sin(2* pi * f * t * delta_t). You need to adjust your constants, and try it again
I've been playing around a little with the Exocortex implementation of the FFT, but I'm having some problems.
Whenever I modify the amplitudes of the frequency bins before calling the iFFT the resulting signal contains some clicks and pops, especially when low frequencies are present in the signal (like drums or basses). However, this does not happen if I attenuate all the bins by the same factor.
Let me put an example of the output buffer of a 4-sample FFT:
// Bin 0 (DC)
FFTOut[0] = 0.0000610351563
FFTOut[1] = 0.0
// Bin 1
FFTOut[2] = 0.000331878662
FFTOut[3] = 0.000629425049
// Bin 2
FFTOut[4] = -0.0000381469727
FFTOut[5] = 0.0
// Bin 3, this is the first and only negative frequency bin.
FFTOut[6] = 0.000331878662
FFTOut[7] = -0.000629425049
The output is composed of pairs of floats, each representing the real and imaginay parts of a single bin. So, bin 0 (array indexes 0, 1) would represent the real and imaginary parts of the DC frequency. As you can see, bins 1 and 3 both have the same values, (except for the sign of the Im part), so I guess bin 3 is the first negative frequency, and finally indexes (4, 5) would be the last positive frequency bin.
Then to attenuate the frequency bin 1 this is what I do:
// Attenuate the 'positive' bin
FFTOut[2] *= 0.5;
FFTOut[3] *= 0.5;
// Attenuate its corresponding negative bin.
FFTOut[6] *= 0.5;
FFTOut[7] *= 0.5;
For the actual tests I'm using a 1024-length FFT and I always provide all the samples so no 0-padding is needed.
// Attenuate
var halfSize = fftWindowLength / 2;
float leftFreq = 0f;
float rightFreq = 22050f;
for( var c = 1; c < halfSize; c++ )
{
var freq = c * (44100d / halfSize);
// Calc. positive and negative frequency indexes.
var k = c * 2;
var nk = (fftWindowLength - c) * 2;
// This kind of attenuation corresponds to a high-pass filter.
// The attenuation at the transition band is linearly applied, could
// this be the cause of the distortion of low frequencies?
var attn = (freq < leftFreq) ?
0 :
(freq < rightFreq) ?
((freq - leftFreq) / (rightFreq - leftFreq)) :
1;
// Attenuate positive and negative bins.
mFFTOut[ k ] *= (float)attn;
mFFTOut[ k + 1 ] *= (float)attn;
mFFTOut[ nk ] *= (float)attn;
mFFTOut[ nk + 1 ] *= (float)attn;
}
Obviously I'm doing something wrong but can't figure out what.
I don't want to use the FFT output as a means to generate a set of FIR coefficients since I'm trying to implement a very basic dynamic equalizer.
What's the correct way to filter in the frequency domain? what I'm missing?
Also, is it really needed to attenuate negative frequencies as well? I've seen an FFT implementation where neg. frequency values are zeroed before synthesis.
Thanks in advance.
There are two issues: the way you use the FFT, and the particular filter.
Filtering is traditionally implemented as convolution in the time domain. You're right that multiplying the spectra of the input and filter signals is equivalent. However, when you use the Discrete Fourier Transform (DFT) (implemented with a Fast Fourier Transform algorithm for speed), you actually calculate a sampled version of the true spectrum. This has lots of implications, but the one most relevant to filtering is the implication that the time domain signal is periodic.
Here's an example. Consider a sinusoidal input signal x with 1.5 cycles in the period, and a simple low pass filter h. In Matlab/Octave syntax:
N = 1024;
n = (1:N)'-1; %'# define the time index
x = sin(2*pi*1.5*n/N); %# input with 1.5 cycles per 1024 points
h = hanning(129) .* sinc(0.25*(-64:1:64)'); %'# windowed sinc LPF, Fc = pi/4
h = [h./sum(h)]; %# normalize DC gain
y = ifft(fft(x) .* fft(h,N)); %# inverse FT of product of sampled spectra
y = real(y); %# due to numerical error, y has a tiny imaginary part
%# Depending on your FT/IFT implementation, might have to scale by N or 1/N here
plot(y);
And here's the graph:
The glitch at the beginning of the block is not what we expect at all. But if you consider fft(x), it makes sense. The Discrete Fourier Transform assumes the signal is periodic within the transform block. As far as the DFT knows, we asked for the transform of one period of this:
This leads to the first important consideration when filtering with DFTs: you are actually implementing circular convolution, not linear convolution. So the "glitch" in the first graph is not really a glitch when you consider the math. So then the question becomes: is there a way to work around the periodicity? The answer is yes: use overlap-save processing. Essentially, you calculate N-long products as above, but only keep N/2 points.
Nproc = 512;
xproc = zeros(2*Nproc,1); %# initialize temp buffer
idx = 1:Nproc; %# initialize half-buffer index
ycorrect = zeros(2*Nproc,1); %# initialize destination
for ctr = 1:(length(x)/Nproc) %# iterate over x 512 points at a time
xproc(1:Nproc) = xproc((Nproc+1):end); %# shift 2nd half of last iteration to 1st half of this iteration
xproc((Nproc+1):end) = x(idx); %# fill 2nd half of this iteration with new data
yproc = ifft(fft(xproc) .* fft(h,2*Nproc)); %# calculate new buffer
ycorrect(idx) = real(yproc((Nproc+1):end)); %# keep 2nd half of new buffer
idx = idx + Nproc; %# step half-buffer index
end
And here's the graph of ycorrect:
This picture makes sense - we expect a startup transient from the filter, then the result settles into the steady state sinusoidal response. Note that now x can be arbitrarily long. The limitation is Nproc > 2*min(length(x),length(h)).
Now onto the second issue: the particular filter. In your loop, you create a filter who's spectrum is essentially H = [0 (1:511)/512 1 (511:-1:1)/512]'; If you do hraw = real(ifft(H)); plot(hraw), you get:
It's hard to see, but there are a bunch of non-zero points at the far left edge of the graph, and then a bunch more at the far right edge. Using Octave's built-in freqz function to look at the frequency response we see (by doing freqz(hraw)):
The magnitude response has a lot of ripples from the high-pass envelope down to zero. Again, the periodicity inherent in the DFT is at work. As far as the DFT is concerned, hraw repeats over and over again. But if you take one period of hraw, as freqz does, its spectrum is quite different from the periodic version's.
So let's define a new signal: hrot = [hraw(513:end) ; hraw(1:512)]; We simply rotate the raw DFT output to make it continuous within the block. Now let's look at the frequency response using freqz(hrot):
Much better. The desired envelope is there, without all the ripples. Of course, the implementation isn't so simple now, you have to do a full complex multiply by fft(hrot) rather than just scaling each complex bin, but at least you'll get the right answer.
Note that for speed, you'd usually pre-calculate the DFT of the padded h, I left it alone in the loop to more easily compare with the original.
Your primary issue is that frequencies aren't well defined over short time intervals. This is particularly true for low frequencies, which is why you notice the problem most there.
Therefore, when you take really short segments out of the sound train, and then you filter these, the filtered segments wont filter in a way that produces a continuous waveform, and you hear the jumps between segments and this is what generates the clicks you here.
For example, taking some reasonable numbers: I start with a waveform at 27.5 Hz (A0 on a piano), digitized at 44100 Hz, it will look like this (where the red part is 1024 samples long):
So first we'll start with a low pass of 40Hz. So since the original frequency is less than 40Hz, a low-pass filter with a 40Hz cut-off shouldn't really have any effect, and we will get an output that almost exactly matches the input. Right? Wrong, wrong, wrong – and this is basically the core of your problem. The problem is that for the short sections the idea of 27.5 Hz isn't clearly defined, and can't be represented well in the DFT.
That 27.5 Hz isn't particularly meaningful in the short segment can be seen by looking at the DFT in the figure below. Note that although the longer segment's DFT (black dots) shows a peak at 27.5 Hz, the short one (red dots) doesn't.
Clearly, then filtering below 40Hz, will just capture the DC offset, and the result of the 40Hz low-pass filter is shown in green below.
The blue curve (taken with a 200 Hz cut-off) is starting to match up much better. But note that it's not the low frequencies that are making it match up well, but the inclusion of high frequencies. It's not until we include every frequency possible in the short segment, up to 22KHz that we finally get a good representation of the original sine wave.
The reason for all of this is that a small segment of a 27.5 Hz sine wave is not a 27.5 Hz sine wave, and it's DFT doesn't have much to do with 27.5 Hz.
Are you attenuating the value of the DC frequency sample to zero? It appears that you are not attenuating it at all in your example. Since you are implementing a high pass filter, you need to set the DC value to zero as well.
This would explain low frequency distortion. You would have a lot of ripple in the frequency response at low frequencies if that DC value is non-zero because of the large transition.
Here is an example in MATLAB/Octave to demonstrate what might be happening:
N = 32;
os = 4;
Fs = 1000;
X = [ones(1,4) linspace(1,0,8) zeros(1,3) 1 zeros(1,4) linspace(0,1,8) ones(1,4)];
x = ifftshift(ifft(X));
Xos = fft(x, N*os);
f1 = linspace(-Fs/2, Fs/2-Fs/N, N);
f2 = linspace(-Fs/2, Fs/2-Fs/(N*os), N*os);
hold off;
plot(f2, abs(Xos), '-o');
hold on;
grid on;
plot(f1, abs(X), '-ro');
hold off;
xlabel('Frequency (Hz)');
ylabel('Magnitude');
Notice that in my code, I am creating an example of the DC value being non-zero, followed by an abrupt change to zero, and then a ramp up. I then take the IFFT to transform into the time domain. Then I perform a zero-padded fft (which is done automatically by MATLAB when you pass in an fft size bigger than the input signal) on that time-domain signal. The zero-padding in the time-domain results in interpolation in the frequency domain. Using this, we can see how the filter will respond between filter samples.
One of the most important things to remember is that even though you are setting filter response values at given frequencies by attenuating the outputs of the DFT, this guarantees nothing for frequencies occurring between sample points. This means the more abrupt your changes, the more overshoot and oscillation between samples will occur.
Now to answer your question on how this filtering should be done. There are a number of ways, but one of the easiest to implement and understand is the window design method. The problem with your current design is that the transition width is huge. Most of the time, you will want as quick of transitions as possible, with as little ripple as possible.
In the next code, I will create an ideal filter and display the response:
N = 32;
os = 4;
Fs = 1000;
X = [ones(1,8) zeros(1,16) ones(1,8)];
x = ifftshift(ifft(X));
Xos = fft(x, N*os);
f1 = linspace(-Fs/2, Fs/2-Fs/N, N);
f2 = linspace(-Fs/2, Fs/2-Fs/(N*os), N*os);
hold off;
plot(f2, abs(Xos), '-o');
hold on;
grid on;
plot(f1, abs(X), '-ro');
hold off;
xlabel('Frequency (Hz)');
ylabel('Magnitude');
Notice that there is a lot of oscillation caused by the abrupt changes.
The FFT or Discrete Fourier Transform is a sampled version of the Fourier Transform. The Fourier Transform is applied to a signal over the continuous range -infinity to infinity while the DFT is applied over a finite number of samples. This in effect results in a square windowing (truncation) in the time domain when using the DFT since we are only dealing with a finite number of samples. Unfortunately, the DFT of a square wave is a sinc type function (sin(x)/x).
The problem with having sharp transitions in your filter (quick jump from 0 to 1 in one sample) is that this has a very long response in the time domain, which is being truncated by a square window. So to help minimize this problem, we can multiply the time-domain signal by a more gradual window. If we multiply a hanning window by adding the line:
x = x .* hanning(1,N).';
after taking the IFFT, we get this response:
So I would recommend trying to implement the window design method since it is fairly simple (there are better ways, but they get more complicated). Since you are implementing an equalizer, I assume you want to be able to change the attenuations on the fly, so I would suggest calculating and storing the filter in the frequency domain whenever there is a change in parameters, and then you can just apply it to each input audio buffer by taking the fft of the input buffer, multiplying by your frequency domain filter samples, and then performing the ifft to get back to the time domain. This will be a lot more efficient than all of the branching you are doing for each sample.
First, about the normalization: that is a known (non) issue. The DFT/IDFT would require a factor 1/sqrt(N) (apart from the standard cosine/sine factors) in each one (direct an inverse) to make them simmetric and truly invertible. Another possibility is to divide one of them (the direct or the inverse) by N, this is a matter of convenience and taste. Often the FFT routines do not perform this normalization, the user is expected to be aware of it and normalize as he prefers. See
Second: in a (say) 16 point DFT, what you call the bin 0 would correspond to the zero frequency (DC), bin 1 low freq... bin 4 medium freq, bin 8 to the highest frequency and bins 9...15 to the "negative frequencies". In you example, then, bin 1 is actually both the low frequency and medium frequency. Apart from this consideration, there is nothing conceptually wrong in your "equalization". I don't understand what you mean by "the signal gets distorted at low frequencies". How do you observe that ?