I want to use a box filter but I think it's causing "Ringing Artifiacts" (could be something else). Is there a connection between them- I think I remember my teacher mentioning it but I'm not totally sure what I'm seeing is "Ringing Artifacts" but that's the term he used. Is there a connection between the two? Or am I just witnessing the result of something else?
Ringing is an artifact that occurs when the kernel in the spatial domain has oscillations. Although the box filter has much oscillations in the Fourier domain, it is not the case in the spatial/temporal domain so you should be fine if you directly convolve in the spatial domain.
For instance, if you have a dirac and convolve it with your box filter, you'll strictly obtain a box, which is the expected result. Note that due to the infinite extent of the spectrum of the box kernel in the Fourier domain, this will not remove all the high frequencies (ie., you will still have high frequencies in your final signal, as illustrated with the box example).
However, if you filter with a box in the frequency domain, this corresponds to filtering with a sinc kernel in the spatial domain, which will produce ringing artifacts, but will perfectly remove high frequencies.
For this reason, people tend to make a compromise between keeping as little as possible high frequencies and not having oscillations. In any case, you cannot remove both at the same time (think of Heisenberg uncertainty principle : the product of the variance in the spatial domain with the variance in the frequency domain is bounded from below).
Such tradeoff can be one of the following (non-exhaustively) :
the Prolate kernel : it is optimal in a certain sense with respect to minimizing ringing while minimizing the amount of high frequencies. However, it's not easy to compute.
the Gabor kernel : it is also optimal in some other sense (a somewhat different criterion), but much easier to compute
the Gaussian / Hanning / Hamming kernels : they are not optimal, but most commonly used as they are pretty cheap and easy to analyze.
Ringing artifacts is a known expression.
Since there is a direct connection between filtering in the spatial domain and filtering in the frequency domain, you should start by considering how a box is represented in the later. That will cause the artifacts you see. So there is in fact a direct connection between the two (box filter and ringing artifacts).
Related
I tried to implement the simple vanilla policy gradient (REINFORCE) in a continuous control problem by adapting this pytorch implementation to the continuous case and I stumbled upon the following issue.
Usually, when the action space is discrete, the output of the policy network is bounded in (0,1)^n by the softmax function which gives the probability that the agent would pick a certain action given the state (input to the network). However, when the action space is continuous, for example if we have K action such that each action ak has lower and upper bounds lk anduk, I haven't found a way (empirical or theoretical) to limit the output of the network (which is usually the means and the standard deviations of the action probability distribution given the state) using lk and uk.
From the few trials I made, without constraining the output of the policy network, it was very hard, if not impossible, to learn a good policy, but i might be doing something wrong since i am new to reinforcement learning.
My intuition suggests me to limit the means and the standard deviations output of the policy network using, for example, a sigmoid and then scaling them with the absolute difference between lk and uk. I'm not quite sure how to do it properly though, considering also that the sampled action could exceed whatever bound you impose on the distribution parameters when using, for example, a gaussian distribution.
Am I missing something? Are there established ways to limit the output of the policy network for continuous action spaces or there's no need to do that at all?
I am not sure this is the right place for this question, if not I will be glad if you point to me a better place.
I have a set of experimental data s(t) which consists of a vector (with 81 points as a function of time t).
From the physics, this is the result of the convolution of the system response e(t) with a probe p(t), which is a Gaussian (actually a laser pulse). In terms of vector, its FWHM covers approximately 15 points in time.
I want to deconvolve this data in Matlab using the convolution theorem: FT{e(t)*p(t)}=FT{e(t)}xFT{p(t)} (where * is the convolution, x the product and FT the Fourier transform).
The procedure itself is no problem, if I suppose a Dirac function as my probe, I recover exactly the initial signal (which makes sense, measuring a system with a Dirac gives its impulse response)
However, the Gaussian case as a probe, as far as I understood turns out to be a critical one. When I divide the signal in the Fourier space by the FT of the probe, the wings of the Gaussian highly amplifies those frequencies and I completely loose my initial signal instead of having a deconvolved one.
From your experience, which method could be used here (like Hamming windows or any windowing technique, or...) ? This looks rather pretty simple but I did not find any easy way to follow in signal processing and this is not my field.
You have noise in your experimental data, do you? The problem is ill-posed then (non-uniquely solvable) and you need regularization.
If the noise is Gaussian the keywords are Tikhonov regularization or Wiener filtering.
Basically, add a positive regularization factor that acts as a lowpass filter. In your notation the estimation of the true curve o(t) then becomes:
o(t) = FT^-1(FT(e)*conj(FT(p))/(abs(FT(p))^2+l))
with a suitable l>0.
You're trying to do Deconvolution process by assuming the Filter Model is Gaussian Blur.
Few notes for doing Deconvolution:
Since your data is real (Not synthetic) data it includes some kind of Noise.
Hence it is better to use the Wiener Filter (Even with the assumption of low variance noise). Otherwise, the "Deconvolution Filter" will increase the noise significantly (As it is an High Pass basically).
When doing the division in the Fourier Domain zero pad the signals to the correct size or better yet create the Gaussian Filter in the time domain with the same number of samples as the signal.
Boundaries will create artifact, Windowing might be useful.
There are many more sophisticated methods for Deconvolution by defining a more sophisticated model on the signal and the noise. If you have more prior data about them, you should look for this kind of framework.
You can always set a threshold on the amplification level for certain frequencies, do that if needed.
Use as much samples as you can.
I hope this will assist you.
I currently work in an experimental rock mechanics lab, and when I conduct an experiment, I record the output signals such as effective torque, normal force and motor velocity. However, the latter quantity causes significant cross-talk over the recorded channels, and I want to filter this out. Let me give an example:
Here the upper plot is the strong signal (motor velocity), and the lower is an idle signal that is affected by the cross-talk (blue is raw signal, red is median filtered). The idle channel is only recording noise. We see three effects here. When the motor voltage changes:
the amplitude of the noise increases
the idle signal's median shifts
there is a spike that lasts approximately 0.1 seconds
If we zoom in on the first spike that occurs at around 115 seconds, we get the following plot. This does not seem to be your typical delta-function type of spike, but rather some kind of electronic "echo".
I have seen much work on blind source separation through independent component analysis (ICA), but that did not prove to be effective in my situation. However, since I know the shape of the signal that is causing the cross-talk, there may be better ways to include this information. My question is this: is there a filter or a combination of filters that can tackle the effects mentioned above?
As I am a geologist and not an electrician or mathematician, I don't have a proper background for this kind of material, so please bear with me. I write Python, MATLAB and C++ quite well, so suggested algorithms written in any of those languages is preferred (but not required).
The crosstalk you encounter, results from a parasitic transmission line. Just think of your typical FM-receiver - where the wires equal the antennae. These effects include parasitic and inductive coupling, and form an oscillator (which is the reason, why you cannot see, the theoretically ideal delta spike)
I recognize two different approaches:
use a hardware filtering circuit
use a software-implemented filter
ad 1:
depending on the needed bandwith (maximum frequency/rate of change) on the idle channel, you can determine the corner frequency, as well as the required filter-order, for a given rate of suppression
ad 2:
you can implement several types of filters (IIF, FIR) which resemble these circuits.
Additionally, if you are measuring the aggressive signal anyways, you can use the measurement on the idle channel to determine system-parameters for a mathematical model of the crosstalk. With this model you'd be able, to exclude the interference by calculation
I am working with the same image and I also need to remove the texture from the image posted in this link
How can I remove the texture from an image using matlab?
Discussions were made on this and I'm quite confused which filter(gaussian LPF or ideal lowpass) is really needed and what is the reason behind this.Which frequencies contribute for this texture????please can someone explain me!
An ideal low pass filter will keep all spatial frequencies below a nominal spatial frequency, and remove all spatial frequencies above it. Unfortunately, a true ideal low pass filter has infinite support (i.e., has an infinitely large non-zero spatial extend). Even a practical approximation to an ideal low pass filter has large spatial support.
A Gaussian, on the other hand, isn't ideal in terms of which frequencies it filters out. A Gaussian in the spatial domain turns out to be a Gaussian in the spatial frequency domain. That is, it doesn't produce very sharp spatial frequency selectivity. The advantage though is that the spatial support of the filter is small. People use Gaussian filters for this because they are convenient mostly. Filtering with a Gaussian tends to look "natural" compared to ideal low pass filters, which can generate ringing artifacts.
A Lanczos filter (windowed sinc filter) is also another choice as it will have a small spatial support and will approximate an ideal filter better than a Gaussian.
However, which is better for your image largely depends on what you want to do. While there's significant theory behind it, qualitative choices like this in image processing are largely an art.
The type of filter you are looking for is ideally nonlinear:
smoothing in areas without large-scale gradients (edges), and
little smoothing close to edges to be preserved.
Here are two alternatives:
The Kuwahara filter:
http://homepage.tudelft.nl/e3q6n/publications/1999/PAA99DRBDPVLV/PAA99DRBDPVLV.pdf
Enhanced shortening flow (Figure 8) in:
http://www.cs.jhu.edu/~misha/Fall07/Papers/intro-to-scalespace.pdf
In the second filter (Enhanced shortening flow), you can
vary the scale parameter and the nonlinear function,
h(Lw) on page 17. Thus, more trimming possibilities.
Ideally, the filter is completely isotropic
(same frequency effect on each possible angle).
Michael
Does anyone know how to use filters in MATLAB?
I am not an aficionado, so I'm not concerned with roll-off characteristics etc — I have a 1 dimensional signal vector x sampled at 100 kHz, and I want to perform a high pass filtering on it (say, rejecting anything below 10Hz) to remove the baseline drift.
There are Butterworth, Elliptical, and Chebychev filters described in the help, but no simple explanation as to how to implement.
There are several filters that can be used, and the actual choice of the filter will depend on what you're trying to achieve. Since you mentioned Butterworth, Chebyschev and Elliptical filters, I'm assuming you're looking for IIR filters in general.
Wikipedia is a good place to start reading up on the different filters and what they do. For example, Butterworth is maximally flat in the passband and the response rolls off in the stop band. In Chebyschev, you have a smooth response in either the passband (type 2) or the stop band (type 1) and larger, irregular ripples in the other and lastly, in Elliptical filters, there's ripples in both the bands. The following image is taken from wikipedia.
So in all three cases, you have to trade something for something else. In Butterworth, you get no ripples, but the frequency response roll off is slower. In the above figure, it takes from 0.4 to about 0.55 to get to half power. In Chebyschev, you get steeper roll off, but you have to allow for irregular and larger ripples in one of the bands, and in Elliptical, you get near-instant cut off, but you have ripples in both bands.
The choice of filter will depend entirely on your application. Are you trying to get a clean signal with little to no losses? Then you need something that gives you a smooth response in the passband (Butterworth/Cheby2). Are you trying to kill frequencies in the stopband, and you won't mind a minor loss in the response in the passband? Then you will need something that's smooth in the stop band (Cheby1). Do you need extremely sharp cut-off corners, i.e., anything a little beyond the passband is detrimental to your analysis? If so, you should use Elliptical filters.
The thing to remember about IIR filters is that they've got poles. Unlike FIR filters where you can increase the order of the filter with the only ramification being the filter delay, increasing the order of IIR filters will make the filter unstable. By unstable, I mean you will have poles that lie outside the unit circle. To see why this is so, you can read the wiki articles on IIR filters, especially the part on stability.
To further illustrate my point, consider the following band pass filter.
fpass=[0.05 0.2];%# passband
fstop=[0.045 0.205]; %# frequency where it rolls off to half power
Rpass=1;%# max permissible ripples in stopband (dB)
Astop=40;%# min 40dB attenuation
n=cheb2ord(fpass,fstop,Rpass,Astop);%# calculate minimum filter order to achieve these design requirements
[b,a]=cheby2(n,Astop,fstop);
Now if you look at the zero-pole diagram using zplane(b,a), you'll see that there are several poles (x) lying outside the unit circle, which makes this approach unstable.
and this is evident from the fact that the frequency response is all haywire. Use freqz(b,a) to get the following
To get a more stable filter with your exact design requirements, you'll need to use second order filters using the z-p-k method instead of b-a, in MATLAB. Here's how for the same filter as above:
[z,p,k]=cheby2(n,Astop,fstop);
[s,g]=zp2sos(z,p,k);%# create second order sections
Hd=dfilt.df2sos(s,g);%# create a dfilt object.
Now if you look at the characteristics of this filter, you'll see that all the poles lie inside the unit circle (hence stable) and matches the design requirements
The approach is similar for butter and ellip, with equivalent buttord and ellipord. The MATLAB documentation also has good examples on designing filters. You can build upon these examples and mine to design a filter according to what you want.
To use the filter on your data, you can either do filter(b,a,data) or filter(Hd,data) depending on what filter you eventually use. If you want zero phase distortion, use filtfilt. However, this does not accept dfilt objects. So to zero-phase filter with Hd, use the filtfilthd file available on the Mathworks file exchange site
EDIT
This is in response to #DarenW's comment. Smoothing and filtering are two different operations, and although they're similar in some regards (moving average is a low pass filter), you can't simply substitute one for the other unless it you can be sure that it won't be of concern in the specific application.
For example, implementing Daren's suggestion on a linear chirp signal from 0-25kHz, sampled at 100kHz, this the frequency spectrum after smoothing with a Gaussian filter
Sure, the drift close to 10Hz is almost nil. However, the operation has completely changed the nature of the frequency components in the original signal. This discrepancy comes about because they completely ignored the roll-off of the smoothing operation (see red line), and assumed that it would be flat zero. If that were true, then the subtraction would've worked. But alas, that is not the case, which is why an entire field on designing filters exists.
Create your filter - for example using [B,A] = butter(N,Wn,'high') where N is the order of the filter - if you are unsure what this is, just set it to 10. Wn is the cutoff frequency normalized between 0 and 1, with 1 corresponding to half the sample rate of the signal. If your sample rate is fs, and you want a cutoff frequency of 10 Hz, you need to set Wn = (10/(fs/2)).
You can then apply the filter by using Y = filter(B,A,X) where X is your signal. You can also look into the filtfilt function.
A cheapo way to do this kind of filtering that doesn't involve straining brain cells on design, zeros and poles and ripple and all that, is:
* Make a copy of the signal
* Smooth it. For a 100KHz signal and wanting to eliminate about 10Hz on down, you'll need to smooth over about 10,000 points. Use a Gaussian smoother, or a box smoother maybe 1/2 that width twice, or whatever is handy. (A simple box smoother of total width 10,000 used once may produce unwanted edge effects)
* Subtract the smoothed version from the original. Baseline drift will be gone.
If the original signal is spikey, you may want to use a short median filter before the big smoother.
This generalizes easily to 2D images, 3D volume data, whatever.