I've been making a routine which measures the phase difference between two spectra using NumPy/Scipy.
I already had the routine written in Matlab, so I basically re-implemented the function and the corresponding unit test using NumPy. However, I found that the unit test fails because scipy.fftpack.fft is introducing some small numerical errors:
import numpy as np
import scipy.fftpack.fft
x = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 3.0, 2.0, 1.0])
X = scipy.fftpack.fft(x)
In this case, since the time-domain signal is symmetric, the expected output is
[16.0000 -6.8284 0 -1.1716 0 -1.1716 0 -6.8284]
as shown in the following Matlab code:
>> x = [0.0, 1.0, 2.0, 3.0, 4.0, 3.0, 2.0, 1.0];
>> X = fft(x)
X =
16.0000 -6.8284 0 -1.1716 0 -1.1716 0 -6.8284
The result should not contain any imaginary components, based on DSP theory. However, the scipy result is as follows:
array([ 16.00000000 +0.00000000e+00j, -6.82842712 -2.22044605e-16j,
0.00000000 -0.00000000e+00j, -1.17157288 -2.22044605e-16j,
0.00000000 +0.00000000e+00j, -1.17157288 +2.22044605e-16j,
0.00000000 +0.00000000e+00j, -6.82842712 +2.22044605e-16j])
Why does scipy.fftpack.fft introduce small imaginary components? I really want to avoid this issue. Could anyone give me a suggestion?
For one thing, scipy.fftpack.fft is guaranteed to always return a complex result, whereas the result of MATLAB's fft function is sometimes real and sometimes complex, depending on whether there is a non-zero imaginary component. However, that doesn't explain why the result of scipy.fftpack.fft actually contains non-zero imaginary components, whereas the result of MATLAB's fft function does not.
I suspect that the underlying reason for the difference has to do with the fact that MATLAB's fft function is apparently based on FFTW, whereas scipy and numpy use FFTPACK due to licensing restrictions.
pyfftw, however, does provide Python bindings to FFTW. If we compare the imaginary components of the results for FFTPACK and FFTW:
from pyfftw.interfaces import scipy_fftpack as fftw
Fx1 = fftpack.fft(x)
print(Fx1.imag)
# [ 0.00000000e+00 -2.22044605e-16 -0.00000000e+00 -2.22044605e-16
# 0.00000000e+00 2.22044605e-16 0.00000000e+00 2.22044605e-16]
print(Fx1.imag == 0)
# [ True False True False True False True False]
Fx2 = fftw.fft(x)
print(Fx2.imag)
# [ 0. 0. 0. 0. 0. 0. 0. 0.]
print(Fx2.imag == 0)
# [ True True True True True True True True]
we see that the imaginary component of the FFTW result compares exactly equal to zero, whereas FFTPACK has a tiny amount of floating-point rounding error.
Beyond that, I have no idea why FFTW's implementation suffers less from rounding error than FFTPACK, but in any case it's important to note that these rounding errors are small enough that they normally don't cause problems (you know you shouldn't be testing for exact equality between float values, right?).
Usually you would simply take the real component of the result, e.g.:
scipy.fftpack.fft(x).real
If these errors are a problem then you could switch to using pyfftw instead of numpy/scipy, but if your code is that sensitive to rounding error then it probably means you're doing something wrong anyway.
Related
I'd like to build a GP with marginalized hyperparameters.
I have seen that this is possible with the HMC sampler provided in gpflow from this notebook
However, when I tried to run the following code as a first step of this (NOTE this is on gpflow 0.5, an older version), the returned samples are negative, even though the lengthscale and variance need to be positive (negative values would be meaningless).
import numpy as np
from matplotlib import pyplot as plt
import gpflow
from gpflow import hmc
X = np.linspace(-3, 3, 20)
Y = np.random.exponential(np.sin(X) ** 2)
Y = (Y - np.mean(Y)) / np.std(Y)
k = gpflow.kernels.Matern32(1, lengthscales=.2, ARD=False)
m = gpflow.gpr.GPR(X[:, None], Y[:, None], k)
m.kern.lengthscales.prior = gpflow.priors.Gamma(1., 1.)
m.kern.variance.prior = gpflow.priors.Gamma(1., 1.)
# dont want likelihood be a hyperparam now so fixed
m.likelihood.variance = 1e-6
m.likelihood.variance.fixed = True
m.optimize(maxiter=1000)
samples = m.sample(500)
print(samples)
Output:
[[-0.43764571 -0.22753325]
[-0.50418501 -0.11070128]
[-0.5932655 0.00821438]
[-0.70217714 0.05077999]
[-0.77745654 0.09362291]
[-0.79404456 0.13649446]
[-0.83989415 0.27118385]
[-0.90355789 0.29589641]
...
I don't know too much in detail about HMC sampling but I would expect that the sampled posterior hyperparameters are positive, I've checked the code and it seems maybe related to the Log1pe transform, though I failed to figure it out myself.
Any hint on this?
It would be helpful if you specified which GPflow version you are using - especially given that from the output you posted it looks like you are using a really old version of GPflow (pre-1.0), and this is actually something that got improved since. What is happening here (in old GPflow) is that the sample() method returns a single array S x P, where S is the number of samples, and P is the number of free parameters [e.g. for a M x M matrix parameter with lower-triangular transform (such as the Cholesky of the covariance of the approximate posterior, q_sqrt), only M * (M - 1)/2 parameters are actually stored and optimised!]. These are the values in the unconstrained space, i.e. they can take any value whatsoever. Transforms (see gpflow.transforms module) provide the mapping between this value (between plus/minus infinity) and the constrained value (e.g. gpflow.transforms.positive for lengthscales and variances). In old GPflow, the model provides a get_samples_df() method that takes the S x P array returned by sample() and returns a pandas DataFrame with columns for all the trainable parameters which would be what you want. Or, ideally, you would just use a recent version of GPflow, in which the HMC sampler directly returns the DataFrame!
I want to run a regression analysis on below data, here x1 and x2 produce y value. But in that case, y value is fixed in all time. So regression will not happen. But why? Need explanation.
Your training set shows that the coefficients are all ~0 and the constant is 5. There's no more information in that dataset, you don't need regression to show that.
You did not specify what kind of regression you are running. Depending on the type of regression you are using, you will need the matrices to be invertible and not be related linearly.
It seems to work using normal equation (with expected results):
import numpy as np
import matplotlib.pyplot as plt
input = np.array([
[2,3,5],
[1,2,5],
[4,2,5],
[1,7,5],
[1,9,5]
])
m = len(input)
X = np.array([np.ones(m), input[:, 0],input[:, 1]]).T # Add Constant to X
y = np.array(input[:, 2]).reshape(-1, 1) # Get the dependant values
betaHat = np.linalg.solve(X.T.dot(X), X.T.dot(y)) # Calculate coefficients
print(betaHat) # Show Constant and coefficients (in that order)
[[ 5.00000000e+00]
[ 5.29208238e-16]
[ 4.32685981e-17]]
I'm a bit confused about the angle() function in Matlab, in particular when applied to an array of real numbers.
The angle() function should give me the phase of a complex number. Example: y = a + bi, ==> phase = arctan(b/a). Indeed, the following works:
for t=1:1000
comp(t) = exp(1i*(t/10));
end
phase_good_comp1 = unwrap(angle(comp)); %this gives me the right answer
b = imag(comp);
a = real(comp);
phase_good_comp2 = atan(b./a); %this gives me the right answer too, but
wrapped (not sure if there is a way to unwrap this, but unwrap() does not
work)
figure(1)
plot(phase_good_comp1)
hold on
plot(phase_good_comp2,'--r')
legend('good phase1', 'good phase2')
title('complex number')
Here's the plot for the complex numbers --
Note that I can use either the angle() function, or the explicit definition of phase, as I have shown above. Both yield good results (I can't unwrap the latter, but that's not my issue).
Now if I apply the same logic to an array of real numbers, I should get a constant phase everywhere, since no imaginary part exists, so arctan(b/a) = arctan(0) = 0. This works if I use the explicit definition of phase, but I get a weird result if I use angle():
for t=1:1000
ree(t) = cos((t/10));
end
phase_bad_re = unwrap(angle(ree)); %this gives me an unreasonable (?) answer
b = imag(ree);
a = real(ree);
phase_good_re = atan(b./a); %this gives me the right answer
figure(1)
plot(phase_bad_re)
hold on
plot(phase_good_re,'--r')
legend('bad phase', 'good phase')
title('real number')
Here's the plot for the real numbers --
Why the oscillation when I use angle()???
The Matlab documentation tells you how to compute this:
The angle function can be expressed as angle(z) = imag(log(z)) = atan2(imag(z),real(z)).
https://www.mathworks.com/help/matlab/ref/angle.html
Note that they define it with atan2 instead of atan.
Now your data is in the range of cosine, which includes both positive and negative numbers. The angle on the positive numbers should be 0 and the angle on the negative numbers should be an odd-integer multiple of pi in general. Using the specific definition that they've chosen to get a unique answer, it is pi. That's what you got. (Actually, for the positive numbers, any even-integer multiple of pi will do, but 0 is the "natural" choice and the one that you get from atan2.)
If you're not clear why the negative numbers don't have angle = 0, plot it out in the complex plane and keep in mind that the radial part of the complex number is positive by definition. That is z = r * exp(i*theta) for positive r and theta given by this angle you're computing.
Since sign of cosine function is periodically changed, angle() is also oscillated.
Please, try this.
a=angle(1);
b=angle(-1);
Phase of 1+i*0 is 0, while phase of -1+i*0 is 3.14.
But, in case of atan, b/a is always 0, so that the result of atan() is all 0.
I have some data points with errors in both the x and y coordinates on these data points. I therefore want to use python's ODR tool to compute the best-fit slope and the error on this slope. I have tried doing it for my actual data but do not find good results. Therefore, I have first tried to use ODR with a simple example as follows:
import numpy as np
import matplotlib.pyplot as plt
from scipy.odr import *
def linear_func(B, x):
return B[0]*x+B[1]
x_data=np.array([0.0, 1.0, 2.0, 3.0])
y_data=np.array([0.0, 1.0, 2.0, 3.0])
x_err=np.array([1.0, 1.0, 1.0, 1.0])
y_err=np.array([5.0, 5.0, 5.0, 5.0])
linear=Model(linear_func)
data=RealData(x_data, y_data, sx=x_err, sy=y_err)
odr=ODR(data, linear, beta0=[1.0, 0.0])
out=odr.run()
out.pprint()
The pprint() line gives:
Beta: [ 1. 0.]
Beta Std Error: [ 0. 0.]
Beta Covariance: [[ 5.20000039 -7.80000026]
[ -7.80000026 18.1999991 ]]
Residual Variance: 0.0
Inverse Condition #: 0.0315397386692
Reason(s) for Halting:
Sum of squares convergence
The resutling Beta values are shown to be 1.0 and 0.0, which I would epect. But why are the standard errors, Beta Std Error, also both zero if my errors on the data points are quite large? Can anyone offer some insight?
I see no discrepancy here. Your example model fits your data perfectly, so the weights you pass to the data do not matter. Moreover, your initial guess beta0=[1.0, 0.0] is a parameter vector giving an optimal solution, so the ODR machinery can not find an iterative improvement of the parameters and quits after zero iterations. The associated errors are zero because for a given data the solution found is infinitely better than any other solution possible because your sum of squares at B=[1, 0] is zero.
To see the what actually happens inside ODR.run() function, add odr.set_iprint(init=2, iter=2, final=2) before you run the regression. In particular, the following output confirms that ODR reaches the stopping condition immediately:
--- STOPPING CONDITIONS:
INFO = 1 ==> SUM OF SQUARES CONVERGENCE.
NITER = 0 (NUMBER OF ITERATIONS)
Note how the errors will not be zero, and NITER will be an integer number if either your x_data is unequal to y_data or if beta0 does not match the optimal solution. In that case, the errors returned by ODR will be nonzero, although still incredibly small.
I am having a hard time understanding what should be a simple concept. I have constructed a vector in MATLAB that is real and symmetric. When I take the FFT in MATLAB, the result has a significant imaginary component, even though the symmetry rules of the Fourier transform say that the FT of a real symmetric function should also be real and symmetric. My example code:
N = 1 + 2^8;
k = linspace(-1,1,N);
V = exp(-abs(k));
Vf1 = fft(fftshift(V));
Vf2 = fft(ifftshift(V));
Vf3 = ifft(fftshift(V));
Vf4 = ifft(ifftshift(V));
Vf5 = fft(V);
Vf6 = ifft(V);
disp([isreal(Vf1) isreal(Vf2) isreal(Vf3) isreal(Vf4) isreal(Vf5) isreal(Vf6)])
Result:
0 0 0 0 0 0
No combinations of (i)fft or (i)fftshift result in a real symmetric vector. I've tried with both even and odd N (N = 2^8 vs. N = 1+2^8).
I did try looking at k+flip(k) and there are some residuals on the order of eps(1), but the residuals are also symmetric and the imaginary part of the FFT is not coming out as fuzz on the order of eps(1), but rather with magnitude comparable to the real part.
What blindingly obvious thing am I missing?
Blindingly obvious thing I was missing:
The FFT is not an integral over all space, so it assumes a periodic signal. Above, I am duplicating the last point in the period when I choose an even N, and so there is no way to shift it around to put the zero frequency at the beginning without fractional indexing, which does not exist.
A word about my choice of k. It is not arbitrary. The actual problem I am trying to solve is to generate a model FTIR interferogram which I will then FFT to get a spectrum. k is the distance that the interferometer travels which gets transformed to frequency in wavenumbers. In the real problem there will be various scaling factors so that the generating function V will yield physically meaningful numbers.
It's
Vf = fftshift(fft(ifftshift(V)));
That is, you need ifftshift in time-domain so that samples are interpreted as those of a symmetric function, and then fftshift in frequency-domain to again make symmetry apparent.
This only works for N odd. For N even, the concept of a symmetric function does not make sense: there is no way to shift the signal so that it is symmetric with respect to the origin (the origin would need to be "between two samples", which is impossible).
For your example V, the above code gives Vf real and symmetric. The following figure has been generated with semilogy(Vf), so that small as well as large values can be seen. (Of course, you could modify the horizontal axis so that the graph is centered at 0 frequency as it should; but anyway the graph is seen to be symmetric.)
#Yvon is absolutely right with his comment about symmetry. Your input signal looks symmetrical, but it isn't because symmetry is related to origin 0.
Using linspace in Matlab for constructing signals is mostly a bad choice.
Trying to repair the results with fftshift is a bad idea too.
Use instead:
k = 2*(0:N-1)/N - 1;
and you will get the result you expect.
However the imaginary part of the transformed values will not be perfectly zero.
There is some numerical noise.
>> max(abs(imag(Vf5)))
ans =
2.5535e-15
Answer to Yvon's question:
Why? >> N = 1+2^4 N = 17 >> x=linspace(-1,1,N) x = -1.0000 -0.8750 -0.7500 -0.6250 -0.5000 -0.3750 -0.2500 -0.1250 0 0.1250 0.2500 0.3750 0.5000 0.6250 0.7500 0.8750 1.0000 >> y=2*(0:N-1)/N-1 y = -1.0000 -0.8824 -0.7647 -0.6471 -0.5294 -0.4118 -0.2941 -0.1765 -0.0588 0.0588 0.1765 0.2941 0.4118 0.5294 0.6471 0.7647 0.8824 – Yvon 1
Your example is not a symmetric (even) function, but an antisymmetric (odd) function. However, this makes no difference.
For a antisymmetric function of length N the following statement is true:
f[i] == -f[-i] == -f[N-i]
The index i runs from 0 to N-1.
Let us see was happens with i=2. Remember, count starts with 0 and ends with 16.
x[2] = -0.75
-x[N-2] == -x[17-2] == -x[15] = (-1) 0.875 = -0.875
x[2] != -x[N-2]
y[2] = -0.7647
-y[N-2] == -y[15] = (-1) 0.7647
y[2] == y[N-2]
The problem is, that the origin of Matlab vectors start at 1.
Modulo (periodic) vectors start with origin 0.
This difference leads to many misunderstandings.
Another way of explanation why linspace(-1,+1,N) is not correct:
Imagine you have a vector which holds a single period of a periodic function,
for instance a Cosinus function. This single period is one of a infinite number of periods.
The first value of your Cosinus vector must not be same as the last value of your vector.
However,that is exactly what linspace(-1,+1,N) does.
Doing so, results in a sequence where the last value of period 1 is the same value as the first sample of the following period 2. That is not what you want.
To avoid this mistake use t = 2*(0:N-1)/N - 1. The distance t[i+1]-t[i] is 2/N and the last value has to be t[N-1] = 1 - 2/N and not 1.
Answer to Yvon's second comment
Whatever you put in an input vector of a DFT/FFT, by theory it is interpreted as a periodic function.
But that is not the point.
DFT performs an integration.
fft(m) = Sum_(k=0)^(N-1) (x(k) exp(-i 2 pi m k/N )
The first value x(k=0) describes the amplitude of the first integration interval of length 1/N. The second value x(k=1) describes the amplitude of the second integration interval of length 1/N. And so on.
The very last integration interval of the symmetric function ends with same value as the first sample. This means, the starting point of the last integration interval is k=N-1 = 1-1/N. Your input vector holds the starting points of the integration intervals.
Therefore, the last point of symmetry k=N is a point of the function, but it is not a starting point of an integration interval and so it is not a member of the input vector.
You have a problem when implementing the concept "symmetry". A purely real, even (or "symmetric") function has a Fourier transform function that is also real and even. "Even" is the symmetry with respect to the y-axis, or the t=0 line.
When implementing a signal in Matlab, however, you always start from t=0. That is, there is no way to "define" the signal starting from before the origin of time.
Searching the Internet for a while lead me to this -
Correct use of fftshift and ifftshift at input to fft and ifft.
As Luis has pointed out, you need to perform ifftshift before feeding the signal into fft. The reason has never been documented in Matlab, but only in that thread. For historical reasons, outputs AND inputs of fft and ifft are swapped. That is, instead of ordered from -N/2 to N/2-1 (the natural order), the signal in time or frequency domain is ordered from 0 to N/2-1 and then -N/2 to -1. That means, the correct way to code is fft( ifftshift(V) ), but most people ignore this at most times. Why it's got silently ignored rather than raising huge problems is that most concerns have been put on the amplitude of signal, not phase. Since circular shifting does not affect amplitude spectrum, this is not a problem (even for the Matlab guys who have written the documentations).
To check the amplitude equality -
Vf2 = fft(ifftshift(V));
Vf5 = fft(V);
Va2 = abs(fftshift(Vf2));
Va5 = abs(fftshift(Vf5));
>> min(abs(Va2-Va5)<1e-10)
ans =
1
To see how badly wrong in phase -
Vp2 = angle(fftshift(Vf2));
Vp5 = angle(fftshift(Vf5));
Anyway, as I wrote in the comment, after copy&pasting your code into a fresh and clean Matlab, it gives 0 1 0 1 0 0.
To your question about N=even and N=odd, my opinion is when N=even, the signal is not symmetric, since there are unequal number of points on either side of the time origin.
Just add the following line after "k = linspace(-1,1,N);"
k(end)=[];
it will remove the last element of the array. This is defined to be symmetric array.
also consider that isreal(complex(1,0)) is false!!!
The isreal function just checks for the memory storage format. so 1+0i is not real in the above example.
You have define your function in order to check for real numbers (like this)
myisreal=#(x) all((abs(imag(x))<1e-6*abs(real(x)))|(abs(x)<1e-8));
Finally your source code should become something like this:
N = 1 + 2^8;
k = linspace(-1,1,N);
k(end)=[];
V = exp(-abs(k));
Vf1 = fft(fftshift(V));
Vf2 = fft(ifftshift(V));
Vf3 = ifft(fftshift(V));
Vf4 = ifft(ifftshift(V));
Vf5 = fft(V);
Vf6 = ifft(V);
myisreal=#(x) all((abs(imag(x))<1e-6*abs(real(x)))|(abs(x)<1e-8));
disp([myisreal(Vf1) myisreal(Vf2) myisreal(Vf3) myisreal(Vf4) myisreal(Vf5) myisreal(Vf6)]);