First of all I asked this question in Stack Exchange and I am getting only concept related answers and not implementation oriented. So, my problem is I am trying to create high pass filter and I implemented using Python.
from numpy import cos, sin, pi, absolute, arange
from scipy.signal import kaiserord, lfilter, firwin, freqz, firwin2
from pylab import figure, clf, plot, xlabel, ylabel, xlim, ylim, title, grid, axes, show
# Nyquist rate.
nyq_rate = 48000 / 2
# Width of the roll-off region.
width = 500 / nyq_rate
# Attenuation in the stop band.
ripple_db = 12.0
num_of_taps, beta = kaiserord(ripple_db, width)
# Cut-off frequency.
cutoff_hz = 5000.0
# Estimate the filter coefficients.
if num_of_taps % 2 == 0:
num_of_taps = num_of_taps + 1
taps = firwin(num_of_taps, cutoff_hz/nyq_rate, window=('kaiser', beta), pass_zero='highpass')
w, h = freqz(taps, worN=1024)
plot((w/pi)*nyq_rate, absolute(h), linewidth=2)
xlabel('Frequency (Hz)')
ylabel('Gain')
title('Frequency Response')
ylim(-0.05, 1.05)
grid(True)
show()
By looking at the frequency response I am not getting the stop band attenuation as expected. I want 12dB attenuation and I am not getting that. What am I doing wrong?
Change the pass_zero argument of firwin to False. That argument must be a boolean (i.e. True or False). By setting it to False, you are selecting the behavior of the filter to be a high-pass filter (i.e. the filter does not pass the 0 frequency of the signal).
Here's a variation of your script. I've added horizontal dashed lines that show the desired attenuation in the stop band (cyan) and desired ripple bounds in the pass band (red) as determined by your choice of ripple_db. I also plot vertical dashed lines (green) to indicate the region of the transition from the stop band to the pass band.
import numpy as np
from scipy.signal import kaiserord, lfilter, firwin, freqz, firwin2
import matplotlib.pyplot as plt
# Nyquist rate.
nyq_rate = 48000 / 2
# Width of the roll-off region.
width = 500 / nyq_rate
# Attenuation in the stop band.
ripple_db = 12.0
num_of_taps, beta = kaiserord(ripple_db, width)
if num_of_taps % 2 == 0:
num_of_taps = num_of_taps + 1
# Cut-off frequency.
cutoff_hz = 5000.0
# Estimate the filter coefficients.
taps = firwin(num_of_taps, cutoff_hz/nyq_rate, window=('kaiser', beta), pass_zero=False)
w, h = freqz(taps, worN=4000)
plt.plot((w/np.pi)*nyq_rate, 20*np.log10(np.abs(h)), linewidth=2)
plt.axvline(cutoff_hz + width*nyq_rate, linestyle='--', linewidth=1, color='g')
plt.axvline(cutoff_hz - width*nyq_rate, linestyle='--', linewidth=1, color='g')
plt.axhline(-ripple_db, linestyle='--', linewidth=1, color='c')
delta = 10**(-ripple_db/20)
plt.axhline(20*np.log10(1 + delta), linestyle='--', linewidth=1, color='r')
plt.axhline(20*np.log10(1 - delta), linestyle='--', linewidth=1, color='r')
plt.xlabel('Frequency (Hz)')
plt.ylabel('Gain (dB)')
plt.title('Frequency Response')
plt.ylim(-40, 5)
plt.grid(True)
plt.show()
Here is the plot that it generates.
If you look closely, you'll see that the frequency response is close to the corners of the region that defines the desired behavior of the filter.
Here's the plot when ripple_db is changed to 21:
Related
I am processing Long term afib dataset - https://physionet.org/content/ltafdb/1.0.0/
When I test the 30s strips of this data, my model is not correcting predicting the signals. So I am trying to deal with noise in this dataset. Here how it looks
Here is the code to plot -
def plot_filter_graphs(data,xmin,xmax,order):
from numpy import sin, cos, pi, linspace
from numpy.random import randn
from scipy import signal
from scipy.signal import lfilter, lfilter_zi, filtfilt, butter
from matplotlib.pyplot import plot, legend, show, grid, figure, savefig,xlim
lowcut=1
highcut=35
nyq = 0.5 * 300
low = lowcut / nyq
high = highcut / nyq
b, a = signal.butter(order, [low, high], btype='band')
# Apply the filter to xn. Use lfilter_zi to choose the initial condition
# of the filter.
z = lfilter(b, a,data)
# Use filtfilt to apply the filter.
y = filtfilt(b, a, data)
y = np.flipud(y)
y = signal.lfilter(b, a, y)
y = np.flipud(y)
# Make the plot.
figure(figsize=(16,5))
plot(data,'b',linewidth=1.75)
plot(z, 'r--', linewidth=1.75)
plot( y, 'k', linewidth=1.75)
xlim(xmin,xmax)
legend(('actual',
'lfilter',
'filtfilt'),
loc='best')
grid(True)
show()
I am using butter band pass filter to filter the noise. I also checked with filtfilt and lfilt but that is also not giving good result.
Any suggestion, how noise can be removed so that signal accuracy is good and hense it can be used for model prediction
I have a 360x360 image I want to remove lines in it
the portion on it has periodic noisy lines I am working on MATLAB
I tried median filter, but not working how to denoise this image and remove lines?
I tried this
%image 360x360
[rows, columns, numberOfColorChannels] = size(image);
subplot(2, 2, 1);
imshow(image,[]);
horizontalProfile = mean(image);
subplot(2, 2, [2, 4]);
plot(horizontalProfile, 'b-');
grid on;
bottomEnvelope = movmin(horizontalProfile, 10);
upperEnvelope = movmax(horizontalProfile, 10);
deltaGL = mean(upperEnvelope- bottomEnvelope)
hold on;
plot(bottomEnvelope, 'r-', 'LineWidth', 2);
plot(upperEnvelope, 'r-', 'LineWidth', 2);
% Compute midline
midline = (bottomEnvelope + upperEnvelope) / 2;
plot(midline, 'm-', 'LineWidth', 2);
columnsToDim = horizontalProfile > midline;
image(:, columnsToDim) = image(:, columnsToDim) - deltaGL;
subplot(2, 2, 3);
imshow(image, []);
But that did not work better
I've uploaded the image data to Google Drive
This is a perfect use case for the Fast Fourier Transform (FFT).
FFT converts an image in the spatial domain to its frequency domain. The frequency domain can be used to smoothen particular noises (vertical lines in your case) in the spatial domain by removing the corresponding high frequency signals. There are tons of sources you can inform yourself about it, so I leave this part to you.
Here is my approach.*
import cv2
import numpy as np
import matplotlib.pyplot as plt
img = cv2.imread('coin.png',0)
# get the frequency domain
f = np.fft.fft2(img)
fshift = np.fft.fftshift(f)
magnitude_spectrum = 20*np.log(np.abs(fshift))
# smoothen the vertical lines in the spatial domain =
# remove the high frequency signals (i.e. horizontal lines) in the frequency domain
rows, cols = img.shape
crow,ccol = rows//2 , cols//2
fshift[crow-5:crow+6, 0:ccol-10] = 0
fshift[crow-5:crow+6, ccol+11:] = 0
magnitude_spectrum_no_vertical = 20*np.log(np.abs(fshift))
# get the spatial domain back
f_ishift = np.fft.ifftshift(fshift)
img_back = np.fft.ifft2(f_ishift)
img_back = np.real(img_back)
This is the output image only.
Feel free to play around with different approaches: Applying a gaussian filter before FFT to improve the outcome, masking background and so on.
*: Sorry I have no MATLAB. It should be easy to port my Python script to MATLAB, though.
I am trying to estimate the amplitude, frequency, and phase of an incoming signal of about 50Hz based on measurement of only a few cycles. The frequency needs to be precise to .01 Hz. Since the signal itself is going to be a pretty clear sine wave, I am trying parameter fitting with SciPy's curve_fit. I've never used it before, so I wrote a quick test function.
I start by generating samples of a single cycle of a dummy cosine wave
from math import *
import numpy as np
from scipy.optimize import curve_fit
import matplotlib.pyplot as plt
fs = 1000 # Sampling rate (Hz)
T = .1 # Length of collection (s)
windowlength = int(fs*T) # Number of samples
f0 = 10 # Fundamental frequency of our wave (Hz)
wave = [0]*windowlength
for x in range(windowlength):
wave[x] = cos(2*pi*f0*x/fs)
t = np.linspace(0,T,int(fs*T)) # This will be our x-axis for plotting
Then I try to fit those samples to a function, adapting the code from the official example provided by scipy: https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html
# Define function to fit
def sinefit(x, A, ph, f):
return A * np.sin(2*pi*f * x + ph)
# Call curve_fit
popt,cov = curve_fit(sinefit, t, wave, p0=[1,np.pi/2,10])
# Plot the result
plt.plot(t, wave, 'b-', label='data')
plt.plot(t, sinefit(t, *popt), 'r-', label='fit')
print("[Amplitude,phase,frequency]")
print(popt)
This gives me popt = [1., 1.57079633, 9.9] and the plot
plot output
My question is: why is my frequency off? I initialized the curve_fit function with the exact parameters of the cosine wave, so shouldn't the first iteration of the LM algorithm realize that there is zero residual and that it has already arrived at the correct answer? That seems to be the case for amplitude and phase, but frequency is 0.1Hz too low.
I expect this is a dumb coding mistake, since the original wave and the fit are clearly lined up in the plot. I also confirmed that the difference between them was zero across the entire sample. If they really were .1 Hz out of phase, there would be a phase shift of 3.6 degrees over my 100ms window.
Any thoughts would be very much appreciated!
The problem is that your array t is not correct. The last value in your t is 0.1, but with a sampling period of 1/fs = 0.001, the last value in t should be 0.099. That is, the times of the 100 samples are [0, 0.001, 0.002, ..., 0.098, 0.099].
You can create t correctly with either
t = np.linspace(0, T, int(fs*T), endpoint=False)
or
t = np.arange(windowlength)/fs # Use float(fs) if you are using Python 2
I've 2 raw signals X and Y measuring vibrations of a rotating shaft at const. speed 633.33 Hz. My goal is to extract only specific frequency component (say 1X or .35X)and plot orbits (X signal plotted against Y signal) for them. I took the raw signal and I applied the low pass filter using Butterworth filter. it gave me smooth signal in the time domain. Now when I'm trying to apply the butterworth band pass filter between frequencies (630 Hz to 640 Hz) its not working properly. I don't know if I'm doing it right.
The following picture is after the application of low pass filter ( butterworth).
This is another after I applied butterworth low pass and band pass filters. There is complete change in the original signal.
I'm expecting the filter to do something like this a cleaner orbit for 1X frequency component.
My MATLAB code is as follows.
L = length(X); % length of signal
fs= 2e6; % sampling frequency
df = fs/L; % Frequency window
dt = 1/df; % time window
%calculate time axis
T = (0:dt:(L-1)*dt)';
subplot(3,2,1);
plot(T,X);
title('before filtering X signal')
subplot (3,2,2);
plot(T,Y);
title('before filtering Y signal')
subplot(3,2,5);
plot(X,Y);
title('Orbits before filtering')
X = detrend(X,0); % Removing DC Offset
Y = detrend(Y,0); % Removing DC Offset
% Butterworth low pass filter to remove high frequency components
[b2,a2] = butter(6,5*633/(fs/2),'low');
dataInX = X;
X = filter(b2,a2,dataInX); %filter command filters
dataInY = Y;
Y = filter(b2,a2,dataInY);
% butter worth band pass to only plot for 1X frequency component
[b1,a1] = butter(1,[633/(fs/2) 640/(fs/2)],'bandpass');
dataInX = X;
X = filter(b1,a1,dataInX); %filter command filters
dataInY = Y;
Y = filter(b1,a1,dataInY);
subplot(3, 2 ,3);
plot(T,X);
axis tight
title('X signal after filtering')
subplot(3,2,4);
plot(T,Y);
axis tight
title('Y signal after filtering')
subplot(3,2,6);
plot(X,Y);
title('Orbit after filtering')
axis tight
I'm also attaching my data file for reference.
I'm new into the world of filters and DSP. Could someone help to fix this one with suggestions or hints or ideas.
After lowpassing the signal ( i.e. [b2,a2] = butter(6,5*633/(fs/2),'low');) you can down sample from 2 MHz to ~ 4 kHZ and wont see much change in the result. Downsampling in this case will not change any resolution that the filter not already has reduced.
You could e. use resample(x,1,500) to downsample your 2 MHz signal to 4 kHz AFTER applying the lowpass. Then you should have no trouble with your narrow bandpass. Make sure to pass the new samplingrate onto the filter.
Also, if you have the full signal available, use filtfilt instead of filt to avoid phase distortion. In this case, you can lower the filter order a bit, but 4th order for the lowpass and the bandpass (after resampling) should just work fine.
I have to:
Read 'cameraman.tif' and convert it to a double, then filter it with a highpass Butterworth filter (in a frequency domain) , cutoff frequency 40 px and level 5. Binarize (level 0.1, highpass). Find all the pixels 20 px above the closest 'true' pixel. Count the number of isolated territories and their areas. Is it correct?
img = imread('cameraman.tif');
img = double(img)/255;
fft = fftshift(fft2(img));
N = 5; D0 = 40;
D = zeros(256); H = zeros(256);
for kz=1:256
for kx=1:256
%Butterworth
D(kz,kx) = sqrt( (kz-128)^2 + (kx-128)^2);
H(kz,kx) = 1/(1+(D(kz,kx)/D0)^2*N);
end
end
filtered = fft.*(1-H);
result = ifft2(ifftshift(filtered));
figure; subplot(121); imshow(img); title('Oryginalny');
subplot(122); imshow(result); title('Przefiltrowany');
binarized = im2bw(real(result), 0.1);
ilosc= bwdist(binarized)>20 ;
ilosc=sum(ilosc(:))
figure; imshow(binarized); title('Zbinaryzowany');
w = binarized == 1;
b = binarized == 0;
biale = sum(w(:));
czarne = sum(b(:));
[blab, blobs] = bwlabel(binarized);
Given your description of what you want to accomplish, your code looks correct.
This does define the Butterworth filter in frequency domain, then the high pass is just subtracting 1 from every coefficient in the low-pass version. Given your comments, you want to count only those pixels which are approximately 20 pixels above the closest true pixel. Assuming the Euclidean distance, then this is correct.
Back to your Butterworth filter: Bear in mind that your for loops at the beginning to define the filter and the filter centre will change depending on the dimensions of your image. You'll need to change these if you want to move to another image.
If I can make one minor suggestion, I would make the creation of your filter vectorized instead of using for loops. It's actually faster, and you'll notice a huge difference when creating filters of larger sizes. As such, replace your Butterworth construction code with this:
[kx,kz] = meshgrid(1:256, 1:256);
D = sqrt((kz - 128).^2 + (kx - 128).^2);
H = 1 ./ ((1 + (D/D0)).^(2*N));