Say I have a signal that looks a bit like this:
that I am processing in MatLab, what functions would I have to use to get rid of the flattish area in the middle? is there any functions that can do that, or do I need to program it in myself? Currently I just have a blank function as I don't know where to start:
function removala = removal(a, b)
end
Is there any quick functions that can remove it or do I just have to search for all values below a threshold and remove them myself? For reference a and b are vectors of amplitude points.
use findpeaks:
% generating signal
x = 1:0.1:10;
y = rand(size(x))*0.5;
y([5,25,84]) = [6,-5.5,7.5];
z = y;
thresh = 0.75; % height threshold
% find peaks
[pks,locs] = findpeaks(z,'MinPeakProminence',thresh);
% remove signal noise between peaks
for ii = 1:length(locs)-1
zz = z(locs(ii)+1:locs(ii+1)-1);
zz(abs(zz) < thresh) = 0;
z(locs(ii)+1:locs(ii+1)-1) = zz;
end
% plot
plot(x,y);
hold on
plot(x,z);
plot(x(locs),pks,'og');
legend('original signal','modified signal','peaks')
You probably want to remove the signal whose absolute power is less than a certain threshold.
So the two input of your function would be the raw signal, and the threshold. The function will output a variable "cleanSignal"
function cleanSignal = removal(rawSignal,threshold)
simplest implementation. remove the data below threshold. If rawSignal is a matrix the resulting variable will be a vector concatenating all the epochs above threshold.
ind = abs(rawSignal)<threshold;
rawSignal(ind) = [];
cleanSignal = rawSignal;
This might not be the behavior that you want, since you want to preserve the original shape of your rawSignal matrix. So you can just "nan" the values below threshold.
ind = abs(rawSignal)<threshold;
rawSignal(ind) = nan;
cleanSignal = rawSignal;
end
Notice that this does not really removes flat signal, but signal which is close to 0.
If you really want to remove flat signal you should use
ind = abs(diff(rawSignal))<threshold;
Related
I created the following code in order to find the natural frequencies of a test sample which is excited by use of an impact hammer and has an accelerometer attached to it. However, I got stuck at interp_accelerance_dB_first. This interpolation creates a set of NaN values and I don't know why. I find it strange since interp_accelerance worked fine. I hope someone can help me!
N = 125000;
fs = 1/(x(2)-x(1));
ts = 1/fs;
f = -fs/2:fs/(N-1):fs/2;
% Set x-axis of graph
x_max = (N-1)*ts;
x_axis=0:ts:x_max;
% find the first natural frequency between these boundaries
First_lower_boundary = 15;
First_upper_boundary = 30;
Input = abs(fft(y)); %FFT input force
Output = abs(fft(o)); %FFT output acceleration
Accelerance = Output./Input;
bin_vals = [0 : N-1];
fax_Hz = bin_vals*fs/N;
N_2 = ceil(N/2);
% Interpolate accelerance function in order to be able to average all accelerance functions
Interp_accelerance = interp1(fax_Hz(1:N_2),Accelerance(1:N_2),x_axis);
% --- Find damping ratio of first natural frequency
% Determine the x-axis (from the boundries at the beginning of this script)
x_axis_first_peak = First_lower_boundary:ts:First_upper_boundary;
% Accelerance function with a logarithmic scale [dB]
Accelerance_dB_first = 20*log10(Accelerance(First_lower_boundary:First_upper_boundary));
% Interpolate the accelerance function [dB]
Interp_accelerance_dB_first = interp1(fax_Hz(First_lower_boundary:First_upper_boundary),Accelerance_dB_first,x_axis_first_peak);
Hard to say for sure without knowing what x,y,o are, but generally interp1 returns NaN when you try to interpolate outside of the bounds of the data's x-axis. Append the following at the end of your code:
[min(fax_Hz(First_lower_boundary:First_upper_boundary)),max(fax_Hz(First_lower_boundary:First_upper_boundary))]
[min(x_axis_first_peak),max(x_axis_first_peak)]
If the second segment doesn't fall inside of the first segment, then you've found your problem.
Incidentally, I think that interp_accelerance may be susceptible to the same error, again depending on the input parameters' exact nature.
Hi i'm having a problem where I have a dataset which ranges between -10^3 to 10^3
I need to be able to plot this as with a log scale but semilogy cannot plot negative values
Say for example my data is:
x = [-3,-2,-1,0,1,2,3];
y = [-1000,-100,-10,1,10,100,1000];
(or in general y=sign(x).*10.^abs(x);)
How can I plot this in MATLAB with a log scale? If possible It would be great if the log scale ticks could be on the Y-axis too
Use your actual data as labels, but scale the plotted data with log10.
% data
x = -3:0.1:3;
y = sign(x).*10.^abs(x);
% scaling function
scale = #(x) sign(x).*log10(abs(x));
N = 7; % number of ticks desired
% picking of adequate values for the labels
TickMask = linspace(1,numel(y),N);
YTickLabels = y(TickMask);
% scale labels and plotdata, remove NaN ->inconsistency, do you really want that?
YTick = scale( YTickLabels );
Y = scale(y);
YTick(isnan(YTick)) = 0;
Y(isnan(Y)) = 0;
% plot
plot(x,Y)
set(gca,'YTick',YTick,'YTickLabels',YTickLabels)
grid on
For N = 7:
For N = 11
How to find a valid value for N?
The following function (thanks to gnovice) will return all possible values you could choose for N:
n = numel(x);
N = find(rem(n./(1:n), 1) == 0) + 1;
about the semilogy-style labels: by adding the following line before the plot:
YTickLabels = cellfun(#(x) ['10^' num2str(x)], num2cell(YTick),'UniformOutput',false)
you could at least achieve something like this:
not beautiful and not generic, but a good point to start for you.
The reason you can't make a logarithmic axis that crosses zero, is that it doesn't make sense!
Since a logarithmic scale is generally displayed as eg. 100 - 10 - 1 - 1/10 - 1/100 - ..., you would need an infinite amount of space to make the axis cross zero.
How about this:
x=logspace(-3,3);
y=sign(x).*10.^abs(x);
loglog(x,y)
#thewaywewalk has already given a beautiful solution to it. The one I'm suggesting is an epsilon improvement on it. If you make two changes
(a) Define a new MATLAB function signia that basically extracts the sign before a number.
function value = signia(x)
if(x>=0)
value = '';
else
value = '-';
end
and (b) make this little change that instead of
YTickLabels = cellfun(#(x) ['10^' num2str(x)], num2cell(YTick),'UniformOutput',false)
you use
YTickLabels = cellfun(#(x) [signia(x) '10^{' num2str(x) '}'], num2cell(YTick),'UniformOutput',false);
(notice the presence of curly braces), you'll get an improvement in the Y ticks display. I got the following.
enter image description here
I am having difficulty with calculating 2D area of contours produced from a Kernel Density Estimation (KDE) in Matlab. I have three variables:
X and Y = meshgrid which variable 'density' is computed over (256x256)
density = density computed from the KDE (256x256)
I run the code
contour(X,Y,density,10)
This produces the plot that is attached. For each of the 10 contour levels I would like to calculate the area. I have done this in some other platforms such as R but am having trouble figuring out the correct method / syntax in Matlab.
C = contourc(density)
I believe the above line would store all of the values of the contours allowing me to calculate the areas but I do not fully understand how these values are stored nor how to get them properly.
This little script will help you. Its general for contour. Probably working for contour3 and contourf as well, with adjustments of course.
[X,Y,Z] = peaks; %example data
% specify certain levels
clevels = [1 2 3];
C = contour(X,Y,Z,clevels);
xdata = C(1,:); %not really useful, in most cases delimters are not clear
ydata = C(2,:); %therefore further steps to determine the actual curves:
%find curves
n(1) = 1; %n: indices where the certain curves start
d(1) = ydata(1); %d: distance to the next index
ii = 1;
while true
n(ii+1) = n(ii)+d(ii)+1; %calculate index of next startpoint
if n(ii+1) > numel(xdata) %breaking condition
n(end) = []; %delete breaking point
break
end
d(ii+1) = ydata(n(ii+1)); %get next distance
ii = ii+1;
end
%which contourlevel to calculate?
value = 2; %must be member of clevels
sel = find(ismember(xdata(n),value));
idx = n(sel); %indices belonging to choice
L = ydata( n(sel) ); %length of curve array
% calculate area and plot all contours of the same level
for ii = 1:numel(idx)
x{ii} = xdata(idx(ii)+1:idx(ii)+L(ii));
y{ii} = ydata(idx(ii)+1:idx(ii)+L(ii));
figure(ii)
patch(x{ii},y{ii},'red'); %just for displaying purposes
%partial areas of all contours of the same plot
areas(ii) = polyarea(x{ii},y{ii});
end
% calculate total area of all contours of same level
totalarea = sum(areas)
Example: peaks (by Matlab)
Level value=2 are the green contours, the first loop gets all contour lines and the second loop calculates the area of all green polygons. Finally sum it up.
If you want to get all total areas of all levels I'd rather write some little functions, than using another loop. You could also consider, to plot just the level you want for each calculation. This way the contourmatrix would be much easier and you could simplify the process. If you don't have multiple shapes, I'd just specify the level with a scalar and use contour to get C for only this level, delete the first value of xdata and ydata and directly calculate the area with polyarea
Here is a similar question I posted regarding the usage of Matlab contour(...) function.
The main ideas is to properly manipulate the return variable. In your example
c = contour(X,Y,density,10)
the variable c can be returned and used for any calculation over the isolines, including area.
I have a data distributed in non-equidistant 1D space and I need to convolve this with a Gaussian filter,
gaussFilter = sqrt(6.0/pi*delta**2)*exp(-6.0*x**2 /delta**2);
where delta is a constant and x corresponds to space.
Can anyone hint how to perform a good integration (2nd order) as the data is not equally spaced taking care of the finite end? I intend to write the code in Fortran, but a Matlab example is also welcome.
use this:
function yy = smooth1D(x,y,delta)
n = length(y);
yy = zeros(n,1);
for i=1:n;
ker = sqrt(6.0/pi*delta^2)*exp(-6.0*(x-x(i)).^2 /delta^2);
%the gaussian should be normalized (don't forget dx), but if you don't want to lose (signal) energy, uncomment the next line
%ker = ker/sum(ker);
yy(i) = y'*ker;
end
end
Found something which works.
Though not sure if this is very accurate way as the integration (trapz) is of first order.
function [fbar] = gaussf(f,x,delta )
n = length(f);
fbar = zeros(n,1);
for i=1:n;
kernel = sqrt(6/(pi*delta^2))*exp(-6*((x - x(k))/delta).^2);
kernel = kernel/trapz(x,kernel);
fbar(i) = trapz(x,f.*kernel);
end
end
I recently tried to implement on matlab a simple example of interpolation method using zéro padding in the fourier domain.
But I am not able to get this work properly, I always have a small frequency shift, barely not visible in fourier space, but thay generates a huge error in time space.
As zéro padding in the fourier space seems to be a common (and fast) interpolation method, I assume that there is something I am missing:
Here is the matlab code:
clc;
clear all;
close all;
Fe = 3250;
Te = 1/Fe;
Nech = 100;
F1 = 500;
F2 = 1000;
FMax = 1500;
time = [0:Te:(Nech-1)*Te];
timeDiscrete = [1:1:Nech];
frequency = (timeDiscrete/Nech)*Fe;
signal = cos(2*pi*F1*(time))+cos(2*pi*F2*(time))+cos(2*pi*FMax*(time));
%Compute the FFT
spectrum=zeros(1,Nech);
for k = timeDiscrete
for l = timeDiscrete
spectrum(k) = spectrum(k) + signal(l)*exp(-2*pi*j*l*k/Nech);
end
end
%Compute de inverse FFT
reconstruction=zeros(1,Nech);
for k = timeDiscrete
for l = timeDiscrete
reconstruction(k) = reconstruction(k) + spectrum(l)*exp(2*pi*j*l*k/Nech);
end
end
reconstruction=reconstruction/Nech;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%% Now interpolation will take place %%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Finterp = 6*Fe;
Tinterp = 1/Finterp;
TimeInterp = [0:Tinterp:(Nech-1)*Te];
[m,n] = size(TimeInterp);
NechInterp = n;
TimeInterpDiscrete = [1:1:NechInterp];
%Compute original signal value without any interpolation
signalResampled = cos(2*pi*F1*(TimeInterp))+cos(2*pi*F2*(TimeInterp))+cos(2*pi*FMax*(TimeInterp));
%Compute original signal interpolation by padding the fft and performing
%inverse fft on the result
semipaddedsize=floor(NechInterp/2);
padded_spectrum0 = zeros(1,semipaddedsize);
padded_spectrum0 = padarray(spectrum(1:Nech/2),[0 semipaddedsize-(Nech/2)],0,'post');
padded_spectrum = zeros(1,NechInterp);
padded_spectrum(1:semipaddedsize) = padded_spectrum0;
padded_spectrum(semipaddedsize+1:NechInterp-1) = conj(fliplr(padded_spectrum0));
% padded_spectrum = padarray(spectrum,[0 NechInterp-Nech],0,'post');
padded_timeDiscrete = [1:1:NechInterp];
padded_reconstruction = zeros(1,NechInterp);
for k = padded_timeDiscrete
for l = padded_timeDiscrete
padded_reconstruction(k) = padded_reconstruction(k) + padded_spectrum(l)*exp(2*pi*j*l*k/NechInterp);
end
end
padded_reconstruction=padded_reconstruction/(1*Nech);
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%% Let's print out the result %%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
spectrumresampled=zeros(1,NechInterp);
for k = TimeInterpDiscrete
for l = TimeInterpDiscrete
spectrumresampled(k) = spectrumresampled(k) + signalResampled(l)*exp(-2*pi*j*l*k/NechInterp);
end
end
figure(2);
plot(abs(spectrumresampled)/6,'g');
hold on;
plot(abs(padded_spectrum),'b');
figure(3);
% Ground truth : deterministic signal is recomputed
plot(TimeInterp,signalResampled,'g');
hold on;
% linear interpolation between subsampled points (matlab tracing tool)
plot(time,(reconstruction),'c');
hold on;
% Padding the spectrum and reconstructing it
plot(TimeInterp,real(padded_reconstruction),'m');
hold on;
xlabel('Time in s','FontSize',16)
ylabel('Signal value (no unit)','FontSize',16)
title('\it{ Various signal reconstruction from fourier transform }','FontSize',16)
legend('True signal', 'Reconstruction with linear interpolation', 'Reconstruction with padded spectrum');
I am not able to post images of the result because of my reputation, but, graph are easy to generates, through matlab.
I would really appreciate a comment on either this code or zero padding fft for interpolation in general.
Thank you in advance
Thank you very much for both of you for those advices, I decided to respond my own question because the was not enough space available in coment box:
#Try hard I indeed defined a wrong discrete Time vector, as #Frederick also pointed out, I had a problem in padding my vector, thank you for giving me the right "matlab way" to do it, I should not have been so afraid of fftshift/ifftshift, in addition, the use of padarray with 'both' would also have done the job, as mentionned by #Frederick .
Collapsing the for loop was also an essential step for proper matlab implementation, that I don't use in training purpose to ease my understanding and bound checking.
An additionnal very interesting point #Try hard mentionned in its first sentence, and that I did not realised in the first place, is the fact, that zero padding is just the equivalent of convoluting my data in time domain by a sinc function.
Actually I think that it is literraly equivalent to a convolution with an aliased sinc function, also called dirichlet kernel, which limits, when sampling frequency increase towards infinity, is the classic sinc function (see http://www.dsprelated.com/dspbooks/sasp/Rectangular_Window_I.html#sec:rectwinintro)
I did not posted the whole code here, but the purpose of my original program was to compare dirichlet kernel convolution formula, that I demonstrated in a different framework (theoretical demonstration using fourier series discrete expression) , sinc convolution Whittaker–Shannon interpolation formula, and zero padding, so I should be given with a very similar result.
To the apodization question, I think that the true answer is that, if your signal is bandlimited, you don't need other apodization function than rectangular window.
If your signal is not bandlimited, or aliased regarding the sampling rate, you will need to reduce the contribution of aliased part of the spectrum, which is done by filtering them out with a frequency filter = apodizing window in frequency domain, wich turns into a specific interpolation kernels in time domain.
OK. One problem was the way you were doing the IDFT for padded_reconstruction. The way that you defined TimeInterp and thus NechInterp made the elements of the complex exponent incorrect. That accounts for the incorrect frequencies.
Then there was an issue with including the midpoint in the fourier domain (pt 50) twice. It was near zero, so it wasn't making a hugely obvious problem, but it should only be included once. I rewrote this section because I was having a hard time working it through exactly the way you did it. I kept it very close though. If I were doing this, I would use fftshift and then padarray(...,'both'), which would save the work of having to put the zeros in the middle. If you are doing this for a learning experience and trying not to use matlab tools (e.g. fftshift), then nevermind.
I also redid the way you define time, but to be fair, I think it could work your way.
I've indicated changes with %<<<<<<<<<<
Fe = 3250;
Te = 1/Fe;
Nech = 100;
F1 = 500;
F2 = 1000;
FMax = 1500;
time = [Te:Te:(Nech)*Te]; %<<<<<<<<<<
timeDiscrete = [1:1:Nech];
frequency = (timeDiscrete/Nech)*Fe;
signal = cos(2*pi*F1*(time))+cos(2*pi*F2*(time))+cos(2*pi*FMax*(time));
%Compute the FFT
spectrum=zeros(1,Nech);
for k = timeDiscrete
for l = timeDiscrete
spectrum(k) = spectrum(k) + signal(l)*exp(-2*pi*j*l*k/Nech);
end
end
%Compute de inverse FFT
reconstruction=zeros(1,Nech);
for k = timeDiscrete
for l = timeDiscrete
reconstruction(k) = reconstruction(k) + spectrum(l)*exp(2*pi*j*l*k/Nech);
end
end
reconstruction=reconstruction/Nech;
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%% Now interpolation will take place %%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Finterp = 6*Fe;
Tinterp = 1/Finterp;
TimeInterp = [Tinterp:Tinterp:(Nech*6)*Tinterp]; %<<<<<<<<<<
[m,n] = size(TimeInterp);
NechInterp = n;
TimeInterpDiscrete = [1:1:NechInterp];
%Compute original signal value without any interpolation
signalResampled = cos(2*pi*F1*(TimeInterp))+cos(2*pi*F2*(TimeInterp))+cos(2*pi*FMax*(TimeInterp));
%Compute original signal interpolation by padding the fft and performing
%inverse fft on the result
padded_spectrum = zeros(1,NechInterp); %<<<<<<<<<<
padded_spectrum(1:floor(Nech/2-1)) = spectrum(1:floor(Nech/2-1)); %<<<<<<<<<<
padded_spectrum(end-floor(Nech/2)+1:end) = spectrum(floor(Nech/2)+1:end); %<<<<<<<<<<
padded_reconstruction = zeros(1,NechInterp);
for k = TimeInterpDiscrete %<<<<<<<<<<(no reason for new variable)
for l = TimeInterpDiscrete %<<<<<<<<<<(no reason for new variable)
padded_reconstruction(k) = padded_reconstruction(k) + padded_spectrum(l)*exp(2*pi*j*l*k/NechInterp);
end
end
padded_reconstruction=padded_reconstruction/(1*Nech);
The result you observe in the time domain is ringing due to convolution of a sinc function with your original data. This is the equivalent in the time domain of multiplication by a rectangular window in the frequency domain, which is in effect what you are doing when you zero fill. Don't forget to apodize!
I repost your code after collapsing the loops (which accelerates the computation significantly), redefining the ranges of the time and frequency variables (see the definition of DFT to see why), and removing one of the padding operations you perform which I frankly did not understand the point off.
clc;
clear all;
close all;
Fe = 3250;
Te = 1/Fe;
Nech = 100;
mlt = 10;
F1 = 50;
F2 = 100;
FMax = 150;
time = [0:Te:(Nech-1)*Te];
%timeDiscrete = [1:1:Nech];
timeDiscrete = [0:1:Nech-1];
frequency = (timeDiscrete/Nech)*Fe;
signal = cos(2*pi*F1*(time))+cos(2*pi*F2*(time))+cos(2*pi*FMax*(time));
spectrum = signal*exp(-2*pi*j*timeDiscrete'*timeDiscrete/Nech);
fspec = [0:Nech-1]*Fe/Nech;
reconstruction = spectrum*exp(2*pi*j*timeDiscrete'*timeDiscrete/Nech)/Nech;
figure
plot(time,signal)
hold on
plot(time,reconstruction,'g:')
% **** interpolation ****
Finterp = 6*Fe;
Tinterp = 1/Finterp;
TimeInterp = [0:Tinterp:(Nech-1)*Te];
NechInterp = length(TimeInterp);
%TimeInterpDiscrete = [1:NechInterp];
TimeInterpDiscrete = [0:NechInterp-1];
%Compute original signal value without any interpolation
signalResampled = cos(2*pi*F1*(TimeInterp))+cos(2*pi*F2*(TimeInterp))+cos(2*pi*FMax*(TimeInterp));
%Compute original signal interpolation by padding the fft and performing
%inverse fft on the result
padded_spectrum0 = spectrum;
padded_spectrum0(NechInterp) = 0;
fspecPadded = [0:NechInterp-1]*Finterp/NechInterp;
padded_reconstruction = padded_spectrum0*exp(2*pi*j*TimeInterpDiscrete'*TimeInterpDiscrete/NechInterp)/(1*Nech);
spectrumresampled = signalResampled*exp(-2*pi*j*TimeInterpDiscrete'*TimeInterpDiscrete/NechInterp);
fresampled = [0:NechInterp-1]*Fe/NechInterp;
% **** print out ****
figure(2);
hold on;
plot(fspec,abs(spectrum),'c');
plot(fresampled,abs(spectrumresampled)/6,'g--');
plot(fspecPadded,abs(padded_spectrum0),'m:');
xlabel('Frequency in Hz','FontSize',16)
ylabel('Signal value (no unit)','FontSize',16)
legend('True signal', 'Reconstruction with resampled spectrum', 'Padded spectrum');
figure(3);
% Ground truth : deterministic signal is recomputed
plot(TimeInterp,signalResampled,'g');
hold on;
% Padding the spectrum and reconstructing it
plot(TimeInterp,real(padded_reconstruction),'m:');
xlabel('Time in s','FontSize',16)
ylabel('Signal value (no unit)','FontSize',16)
title('\it{ Various signal reconstruction from fourier transform }','FontSize',16)
legend('True signal', 'Reconstruction with padded spectrum');
And here's an image of a horribly distorted signal due to padding in the frequency domain:
Some improvement is possible by first applying fftshift to center the spectrum and padding on alternate sides of the centered spectrum, then inverting the fftshift operation:
Nz = NechInterp-Nech;
padded_spectrum0 = ifftshift([ zeros(1,floor(Nz/2)) fftshift(spectrum) zeros(1,floor(Nz/2)+rem(Nz,2)) ]); % replaces (NechInterp) = 0;
fspecPadded = [0:NechInterp-1]*Finterp/NechInterp;
Then you arrive at this much nicer interpolated time domain signal because the padding operation does not result in such abrupt drop-offs in the spectrum (some improvements may still be possible however with further tinkering):