doseResponse function MATLAB iteration limit exceeded warning - matlab

I've been running a variation of the doseResponse function downloaded from here to generate dose-response sigmoid curves. However, I've had trouble with one of my datasets generating a linear curve instead. By running the following code, I get the following error and produce the following graph. I also uploaded the data called dose2.csv and resp2.csv to google drive here. Does anyone know how I can fix this? Thanks.
Code to generate graph
% Plotting Dose-Response Curve
response = resp2;
dose = dose2;
% Deal with 0 dosage by using it to normalise the results.
normalised=0;
if (sum(dose(:)==0)>0)
%compute mean control response
controlResponse=mean(response(dose==0));
%remove controls from dose/response curve
response=response(dose~=0)/controlResponse;
dose=dose(dose~=0);
normalised=1;
end
%hill equation sigmoid
sigmoid=#(beta,x)beta(1)+(beta(2)-beta(1))./(1+(x/beta(3)).^beta(4));
%calculate some rough guesses for initial parameters
minResponse=min(response);
maxResponse=max(response);
midResponse=mean([minResponse maxResponse]);
minDose=min(dose);
maxDose=max(dose);
%fit the curve and compute the values
%[coeffs,r,J]=nlinfit(dose,response,sigmoid,[minResponse maxResponse midResponse 1]); % nlinfit doesn't work as well
beta_new = lsqcurvefit(sigmoid,[minResponse maxResponse midResponse 1],dose,response);
[coeffs,r,J]=nlinfit(dose,response,sigmoid, beta_new);
ec50=coeffs(3);
hillCoeff=coeffs(4);
%plot the fitted sigmoid
xpoints=logspace(log10(minDose),log10(maxDose),1000);
semilogx(xpoints,sigmoid(coeffs,xpoints),'Color',[1 0 0],'LineWidth',2)
hold on
%notate the EC50
text(ec50,mean([coeffs(1) coeffs(2)]),[' \leftarrow ' sprintf('EC_{50}=%0.2g',ec50)],'FontSize',20,'Color',[1 0 0]);
%plot mean response for each dose with standard error
doses=unique(dose);
meanResponse=zeros(1,length(doses));
stdErrResponse=zeros(1,length(doses));
for i=1:length(doses)
responses=response(dose==doses(i));
meanResponse(i)=mean(responses);
stdErrResponse(i)=std(responses)/sqrt(length(responses));
%stdErrResponse(i)=std(responses);
end
errorbar(doses,meanResponse,stdErrResponse,'o','Color',[1 0 0],'LineWidth',2,'MarkerSize',12)
Warning Message
Solver stopped prematurely.
lsqcurvefit stopped because it exceeded the function evaluation limit,
options.MaxFunctionEvaluations = 4.000000e+02.
Warning: Iteration limit exceeded. Returning results from final iteration.
Graph (looking to generate a sigmoid curve not linear)

You also need to optimize your initial value [minResponse maxResponse midResponse 1] for lsqcurvefit. Don't just simply start with minimum or maximum values of given values. Instead, you may first start with your equations to estimate your coefficients.
Given the sigmoid model of sigmoid=#(beta,x)beta(1)+(beta(2)-beta(1))./(1+(x/beta(3)).^beta(4)). As x gets arbitrarily close to inf, equation will return beta(2). And as x gets arbitrarily close to 0, equation will return beta(1). Therefore, initial estimation of minResponse, maxResponse, and midResponse seems reasonable enough. Actually your problem lies in your initial estimation of 1. beta(4) can be roughly estimated with the inclination of your log graph. To my rough sketch it was around 1/4 and therefore you may conclude that your initial estimation of 1 was too large for convergence.
beta_new = lsqcurvefit(sigmoid,[minResponse maxResponse midResponse 1/4],dose,response);

Related

How do I calculate an exponentially weighted moving mean using DSP signal processing toolbox

I am using MATLAB R2020a with MacOS. I am trying to find the exponentially weighted moving mean of the cycle period of an ECG signal, and have used the dsp.MovingAverage function from the DSP signal processing toolbox, and called the commands shown. However, I am not sure how to specify how many of the elements of the vector to include in the weighted mean. At the moment, is it just adding a weight to all of the elements and then finding the moving mean?
movavgExp = dsp.MovingAverage('Method', 'Exponential weighting', 'ForgettingFactor', 0.1);
Whenever I call the 'WindowLength' command as specified in the DSP documentation, it produces an error:
movavgExp = dsp.MovingAverage(10, 'Method', 'Exponential weighting', 'ForgettingFactor', 0.1);
Warning: The WindowLength property is not relevant in this configuration of the System
object.
I would really appreciate any suggestions for this, thanks in advance!
From the Mathworks page for dsp.MovingAverage:
"Exponential weighting — The block multiplies the samples by a set of weighting factors. The magnitude of the weighting factors decreases exponentially as the age of the data increases, but the magnitude never reaches zero. To compute the average, the algorithm sums the weighted data."
So there is no real averaging window as you use all your signal up to time t (exponentially weighted) for the mean value at that instant.
Of course older samples are weighted less than newer ones, and the parameter for that is that ForgettingFactor. I guess you could then define an "effective" averaging window as the number of samples whose weight is larger than a threshold.
Unfortunately it doesn't seem like dsp.MovingAverage can return the weights itself, but you can calculate them yourself. From the Mathworks page,
where is the weight for the Nth sample and is your forgetting factor. Remember to initialize the weight for the first sample to 1, so that you could have something like:
w = zeros(length(x),1); % where x is your signal
w(1) = 1; % initialize the weight for the first sample
for i = 2:length(x)
w(i) = lambda*w(i-1) + 1; % calculate the successive weights
end
To have then the averaging window for the N-th sample I would probably then normalize the weights from 1 to N with respect to the their sum:
thr = 1.e-3; % your threshold, you'll probably have to play with this a bit
lengthAveragWdw = zeros(length(x),1);
for i = 1:length(x)
wi = w(1:i); % weights used to calculate the moving average up to the i-th sample
wi = wi./sum(wi); % normalize the weights
lengthAveragWdw(i) = sum(wi >= thr); % count the number of samples whose weight is greater than the threshold
end
where thr is a threshold value that you have to decide beforehand.

Matlab: computing signal to noise ratio (SNR) of two highly correlated time domain signals

I'm working in the space of biosignal acquisition. I made a experiment as detailed below, and am now trying to obtain some results from the data.
I have a text file of a signal in Matlab. I loaded the signal onto a waveform generator, then I recorded the generator output on an oscilloscope.
I imported the recorded signal from the oscilloscope back into Matlab.
The Pearson's correlation coefficient between the original signal and the oscilloscope signal is 0.9958 (obtained using corrcoeff function).
I want to compute the SNR of the oscilloscope signal (what I'm calling my signal plus whatever noise is introduced through the digital-to-analog conversion and visa-versa). I have attached a snippet of the 2 signals for reference.
So my original signal is X and oscilloscope signal is X + N.
I used the snr function to compute SNR as follows.
snr(original, (oscilloscope - original))
The result I got was 20.44 dB.
This seems off to me as I would have thought with such a high correlation, that the SNR should be much higher?
Or is it not appropriate to try and compute SNR in this sort of situation?
All help is appreciated.
Thanks
Edit: Graph of a couple of results vs Sleutheye's simulated relationship
You might be surprised at just how even such moderate SNR can still result in fairly high correlations.
I ran an experiment to illustrate the approximate relation between correlation and signal-to-noise-ratio estimate. Since I did not have your specific EEG signal, I just used a reference constant signal and some white Gaussian noise. Keep in mind that the relationship could be affected by the nature of the signal and noise, but it should give you an idea of what to expect. This simulation can be executed with the following code:
SNR = [10:1:40];
M = 10000;
C = zeros(size(SNR));
for i=1:length(SNR)
x = ones(1,M);
K = sqrt(sum(x.*x)/M)*power(10, -SNR(i)/20);
z = x + K*randn(size(x));
C(i) = xcorr(x,z,0)./sqrt(sum(x.*x)*sum(z.*z));
end
figure(1);
hold off; plot(SNR, C);
corr0 = 0.9958;
hold on; plot([SNR(1) SNR(end)], [corr0 corr0], 'k:');
snr0 = 20.44;
hold on; plot([snr0 snr0], [min(C) max(C)], 'r:');
xlabel('SNR (dB)');
ylabel('Correlation');
The dotted black horizontal line highlights your 0.9958 correlation measurement, and the dotted red vertical line highlights your 20.44 dB SNR result.
I'd say that's a pretty good match!
In fact, for this specific case in my simulation (x = 1; z = x + N(0,σ)) if we denote C(x,z) to be the correlation between x and z, and σ as the noise standard deviation, we can actually show that:
Given a correlation value of 0.9958, this would yield an SNR of 20.79dB, which is consistent with your results.

Linear convolution using fft for system output

Here is a mass-spring-damper system with an impulse response, h and an arbitrary forcing function, f (cos(t) in this case). I am trying to use Matlab's FFT function in order to perform convolution in the frequency domain. I am expecting for the output (ifft(conv)) to be the solution to the mass-spring-damper system with the specified forcing, however my plot looks completely wrong! So, i must be implementing something wrong. Please help me find my errors in my code below! Thanks
clear
%system parameters
m=4;
k=256;
c=1;
wn=sqrt(k/m);
z=c/2/sqrt(m*k);
wd=wn*sqrt(1-z^2);
w=sqrt(4*k*m-c^2)/(2*m);
x0=0; %initial conditions
v0=0;
%%%%%%%%%%%%%%%%%%%%%%%%%
t=0:.01:2*pi ;%time vector
f=[cos(t),zeros(1,length(t)-1)]; %force f
F=fft(f);
h=[1/m/wd*exp(-z*wn*t).*sin(wd*t),zeros(1,length(t)-1)]; %impulse response
H=fft(h);
conv=H.*F; %convolution is multiplication in freq domain
plot(0:.01:4*pi,ifft(conv))
To see what is expected run this code. Enter in cos(t); 4; 1; 256 for the inputs. You can see that it reaches a steady state at an amplitude much different than the plot generated from the above FFT code.
%%%FOR UNDERDAMPED SYSTEMS
func=input('enter function of t---> ','s');
m=input('mass ');
c=input('c ');
k=input('k ');
z=c/2/sqrt(k*m);
wn=sqrt(k/m);
wd=wn*sqrt(1-z^2);
dt=.001;
tMax=20;
x0=0;
v0=0;
t0=0;
t=0:dt:tMax;
X(:,1)=[x0;v0;t0];
functionForce=inline(func);
tic
for i=1:length(t)-1
a=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[X(1,i);X(2,i);X(3,i)]+[0;functionForce(X(3,i));0]);
Xtemp=X(:,i)+[0;0;dt/2] + a*dt/2;
b=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[Xtemp(1);Xtemp(2);Xtemp(3)]+[0;functionForce(X(3,i));0]);
Xtemp=Xtemp+[0;0;dt/2] + b*dt/2;
c=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[Xtemp(1);Xtemp(2);Xtemp(3)]+[0;functionForce(X(3,i));0]);
Xtemp=Xtemp + [0;0;dt] +c*dt;
d=([0, 1, 0; -wn^2, -2*z*wn, 0; 0,0,0]*[Xtemp(1);Xtemp(2);Xtemp(3)]+[0;functionForce(X(3,i));0]);
X(:,i+1)=X(:,i)+(a+2*b+2*c+d)*dt/6+[0;0;dt];
end
toc
figure(1)
axis([0 tMax min(X(1,:)) max(X(1,:))])
plot(t,X(1,:))
The initial transient will appear in the FFT method, so you will need to increase the time span (eg t=0:0.01:15*pi) to ensure the result includes the steady state. Incidentally since you truncate h after the same duration, increasing the time span also makes your impulse response h a better approximation to the actual infinite length impulse response.
So, updating your code to:
T=15*pi;
N=floor(T/.01);
t=[0:N-1]*0.01; ;%time vector
...
plot([0:2*N-2]*0.01, real(ifft(conv)));
axis([0 T]); % only show the duration where the driving force is active
would correspondingly show the following response:
which shows the initial transient, and eventually the steady state. You may notice that the plot is similar to your alternate implementation up to a scaling factor.
This difference in scaling comes from two factors. The first one is simply due to the fact that the convolution in your FFT based implementation computes a sum which isn't weight by the time step (as compare with the dt scaling used in your second implementation). The second one comes from the fact that the second implementation does not account for the mass m for the effect of the driving force.
After accounting for those two factors, you should get the following responses:
where the blue curve represents the FFT based implementation (same as the above curve but scaled down by dt=0.01), and the red curve represents your alternate implementation (with the functionForce divided by m).

MATLAB using FFT to find steady state response to a periodic input force (mass spring damper system)

Lets say I have a mass-spring-damper system...
here is my code (matlab)...
% system parameters
m=4; k=256; c=1; wn=sqrt(k/m); z=c/2/sqrt(m*k); wd=wn*sqrt(1-z^2);
% initial conditions
x0=0; v0=0;
%% time
dt=.001; tMax=2*pi; t=0:dt:tMax;
% input
F=cos(t); Fw=fft(F);
% impulse response function
h=1/m/wd*exp(-z*wn*t).*sin(wd*t); H=fft(h);
% convolution
convolution=Fw.*H; sol=ifft(convolution);
% plot
plot(t,sol)
so I can successfully retrieve a plot, however I am getting strange responses I also programmed a RK4 method that solves the system of differential equations so I know how the plot SHOULD look like, and the plot I am getting from using FFT has an amplitude of a like 2 when it should have an amplitude of like .05.
So, how can I solve for the steady state response for this system using FFT. I want to use FFT because it is about 3 orders of magnitude faster than numerical integration methods.
Keep in mind I am defining my periodic input as cos(t) which has a period of 2*pi that is why I only used FFT over the time vector that spanned 0 to 2*pi (1 period). I also noticed if I changed the tMax time to a multiple of 2*pi, like 10*pi, I got a similar looking plot but the amplitude was 4 rather than 2, either way still not .05!. maybe there is some kind of factor I need to multiply by?
also I plotted : plot(t,Fw) expecting to see one peak at 1 since the forcing function is cos(t), yet I did not see any peaks (maybe I shouldn't be plotting Fw vs t)
I know it is possible to solve for the steady state response using fourier transform / fft, I am just missing something! I need help and understanding!!
The original results
Running the code you provided and comparing the result with the RK4 code posted in your other question, we get the following responses:
where the blue curve represents the FFT based implementation, and the red curve represents your alternate RK4 implementation. As you have pointed out, the curves are quite different.
Getting the correct response
The most obvious problem is of course the amplitude, and the main sources of the amplitude discrepancies of the code posted in this question are the same as the ones I indicated in my answer to your other question:
The RK4 implementation performs a numeric integration which correctly scales the summed values by the integration step dt. This scaling is lacking from the FFT based implementation.
The impulse response used in the FFT based implementation is consistent with the driving force being scaled by the mass m, a factor which was missing from the RK4 implementation.
Fixing those two issues results in responses which are a little closer, but still not identical. As you probably found out given the changes in the posted code of your other question, another thing that was lacking was zero padding of the input and of the impulse response, without which you were getting a circular convolution rather than a linear convolution:
f=[cos(t),zeros(1,length(t)-1)]; %force f
h=[1/m/wd*exp(-z*wn*t).*sin(wd*t),zeros(1,length(t)-1)]; %impulse response
Finally the last element to ensure the convolution yields good result is to use a good approximation to the infinite length impulse response. How long is long enough depends on the rate of decay of the impulse response. With the parameters you provided, the impulse response would have died down to 1% of its original values after approximately 11*pi. So extending the time span to tMax=14*pi (to include a full 2*pi cycle after the impulse response has died down), would probably be enough.
Obtaining the steady-state response
The simplest way to obtain the steady-state response is then to discard the initial transient. In this process we discard a integer number of cycles of the reference driving force (this of course requires knowledge of the driving force's fundamental frequency):
T0 = tMax-2*pi;
delay = min(find(t>T0));
sol = sol(delay:end);
plot([0:length(sol)-1]*dt, sol, 'b');
axis([0 2*pi]);
The resulting responses are then:
where again the blue curve represents the FFT based implementation, and the red curve represents your alternate RK4 implementation. Much better!
An alternate method
Computing the response for many cycles waiting for the transient response to die down and extracting the remaining samples corresponding
to the steady state might appear to be a little wasteful, despite the fact that the computation is still fairly fast thanks to the FFT.
So, let's go back a little and look at the problem domain. As you are probably aware,
the mass-spring-damper system is governed by the differential equation:
where f(t) is the driving force in this case.
Note that the general solution to the homogeneous equation has the form:
The key is then to realize that the general solution in the case where c>0 and m>0 vanishes in the steady state (t going to infinity).
The steady-state solution is thus only dependent on the particular solution to the non-homogenous equation.
This particular solution can be found by the method of undetermined coefficients, for a driving force of the form
by correspondingly assuming that the solution has the form
Substituting in the differential equation yields the equations:
thus, the solution can be implemented as:
EF0 = [wn*wn-w*w 2*z*wn*w; -2*z*wn*w wn*wn-w*w]\[1/m; 0];
sol = EF0(1)*cos(w*t)+EF0(2)*sin(w*t);
plot(t, sol);
where w=2*pi in your case.
Generalization
The above approach can be generalized for more arbitrary periodic driving forces by expressing the driving force as a
Fourier Series (assuming the driving force function satisfies the Dirichlet conditions):
The particular solution can correspondingly be assumed to have the form
Solving for the particular solution can be done in a way very similar to the earlier case. This result in the following implementation:
% normalize
y = F/m;
% compute coefficients proportional to the Fourier series coefficients
Yw = fft(y);
% setup the equations to solve the particular solution of the differential equation
% by the method of undetermined coefficients
k = [0:N/2];
w = 2*pi*k/T;
A = wn*wn-w.*w;
B = 2*z*wn*w;
% solve the equation [A B;-B A][real(Xw); imag(Xw)] = [real(Yw); imag(Yw)] equation
% Note that solution can be obtained by writing [A B;-B A] as a scaling + rotation
% of a 2D vector, which we solve using complex number algebra
C = sqrt(A.*A+B.*B);
theta = acos(A./C);
Ywp = exp(j*theta)./C.*Yw([1:N/2+1]);
% build a hermitian-symmetric spectrum
Xw = [Ywp conj(fliplr(Ywp(2:end-1)))];
% bring back to time-domain (function synthesis from Fourier Series coefficients)
x = ifft(Xw);
A final note
I purposely avoided the undamped c=0 case in the above derivation. In this case, the oscillation never die down and the general solution to the homogeneous equation does not have to be the trivial one.
The final "steady-state" in this case may or may not have the same period as the driving force. As a matter of fact, it may not be periodic at all if the period oscillations from the general solution is not related to the period of the driving force through a rational number (ratio of integers).

Finding the phase from FFT on MATLAB

I know the fundamental frequency of my signal and therefore I also know the other frequencies for the harmonics, I have used the FFT command to compute the first 5 harmonics (for which I know their frequencies). Is it possible for me to find the phase with this available information?
Please note I cant be sure my signal is only one period and therefore need to calculate the phase via the known frequency values.
Code seems to be working:
L = length(te(1,:)); % Length of signal
x = te(1,:);
NFFT = 2^nextpow2(L); % Next power of 2 from length of y
Y = fft(x,NFFT)/L;
f = linspace(1,5,5);
Y(1) = []; % First Value is a sum of all harmonics
figure(1);
bar(f,2*abs(Y(1:5)), 'red')
title('Transmission Error Harmonics')
xlabel('Harmonic')
ylabel('|Y(f)|')
figure(2);
bar(f,(angle(Y(1:5))))
title('Transmission Error Phase')
xlabel('Harminic')
ylabel('Angle (radians)')
Note that if your fundamental frequency is not exactly integer periodic in the fft length, then the resulting phase (atan2(xi,xr)) will be flipping signs between adjacent bins due to the discontinuity between the fft ends (or due to the rectangular window convolution), making phase interpolation interesting. So you may want to re-reference the FFT phase estimation to the center of the data window by doing an fftshift (pre, by shift/rotating elements, or post, by flipping signs in the fft result), making phase interpolation look more reasonable.
In general your Fourier transformed is complex. So if you want to know the phase of a certain frequency you calculate it with tan(ImaginaryPart(Sample)/RealPart(Sample)). This can be done by using angle().
In your case you
1- calculate fft()
2- calculate angle() for all samples of the FFT or for the samples you are interested in (i.e. the sample at your fundamental frequency/harmonic)
EDIT: an example would be
t = [0 0 0 1 0 0 0];
f = fft(t);
phase = angle(f);
phase = angle(f(3)); % If you are interested in the phase of only one frequency
EDIT2: You should not mix up a real valued spectrum [which is basically abs(fft())] with a complex fourier transformed [which is only fft()]. But as you wrote that you calculated the fft yourself I guess you have the 'original' FFT with the complex numbers.