I'm trying to use OpenCV 2.3 python bindings to calibrate a camera. I've used the data below in matlab and the calibration worked, but I can't seem to get it to work in OpenCV. The camera matrix I setup as an initial guess is very close to the answer calculated from the matlab toolbox.
import cv2
import numpy as np
obj_points = [[-9.7,3.0,4.5],[-11.1,0.5,3.1],[-8.5,0.9,2.4],[-5.8,4.4,2.7],[-4.8,1.5,0.2],[-6.7,-1.6,-0.4],[-8.7,-3.3,-0.6],[-4.3,-1.2,-2.4],[-12.4,-2.3,0.9], [-14.1,-3.8,-0.6],[-18.9,2.9,2.9],[-14.6,2.3,4.6],[-16.0,0.8,3.0],[-18.9,-0.1,0.3], [-16.3,-1.7,0.5],[-18.6,-2.7,-2.2]]
img_points = [[993.0,623.0],[942.0,705.0],[1023.0,720.0],[1116.0,645.0],[1136.0,764.0],[1071.0,847.0],[1003.0,885.0],[1142.0,887.0],[886.0,816.0],[827.0,883.0],[710.0,636.0],[837.0,621.0],[789.0,688.0],[699.0,759.0],[768.0,800.0],[697.0,873.0]]
obj_points = np.array(obj_points)
img_points = np.array(img_points)
w = 1680
h = 1050
size = (w,h)
camera_matrix = np.zeros((3, 3))
camera_matrix[0,0]= 2200.0
camera_matrix[1,1]= 2200.0
camera_matrix[2,2]=1.0
camera_matrix[2,0]=750.0
camera_matrix[2,1]=750.0
dist_coefs = np.zeros(4)
results = cv2.calibrateCamera(obj_points, img_points,size,
camera_matrix, dist_coefs)
First off, your camera matrix is wrong. If you read the documentation, it should look like:
fx 0 cx
0 fy cy
0 0 1
If you look at yours, you've got it the wrong way round:
fx 0 0
0 fy 0
cx cy 1
So first, set camera_matrix to camera_matrix.T (or change how you construct camera_matrix. Remember that camera_matrix[i,j] is row i, column j).
camera_matrix = camera_matrix.T
Next, I ran your code and I see that "can't seem to get it to work" means the following error (by the way - always say what you mean by "can't seem to get it to work" in your questions - if it's an error, post the error. If it runs but gives you weirdo numbers, say so):
OpenCV Error: Assertion failed (ni >= 0) in collectCalibrationData, file /home/cha66i/Downloads/OpenCV-2.3.1/modules/calib3d/src/calibration.cpp, line 3161
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
cv2.error: /home/cha66i/Downloads/OpenCV-2.3.1/modules/calib3d/src/calibration.cpp:3161: error: (-215) ni >= 0 in function collectCalibrationData
I then read the documentation (very useful by the way) and noticed that obj_points and img_points have to be vectors of vectors, because it is possible to feed in sets of object/image points for multiple images of the same chessboard(/calibration points).
Hence:
cv2.calibrateCamera([obj_points], [img_points],size, camera_matrix, dist_coefs)
What? I still get the same error?!
Then, I had a look at the OpenCV python2 samples (in the folder OpenCV-2.x.x/samples/python2), and noticed a calibration.py showing me how to use the calibration functions (never underestimate the samples, they're often better than the documentation!).
I tried to run calibration.py but it doesn't run because it doesn't supply the camera_matrix and distCoeffs arguments, which are necessary. So I modified it to feed in a dummy camera_matrix and distCoeffs, and hey, it works!
The only difference I can see between my obj_points/img_points and theirs, is that theirs has dtype=float32, while mine doesn't.
So, I change my obj_points and img_points to also have dtype float32 (the python2 interface to OpenCV is funny like that; often functions don't work when matrices don't have a dtype):
obj_points = obj_points.astype('float32')
img_points = img_points.astype('float32')
Then I try again:
>>> cv2.calibrateCamera([obj_points], [img_points],size, camera_matrix, dist_coefs)
OpenCV Error: Bad argument
(For non-planar calibration rigs the initial intrinsic matrix must be specified)
in cvCalibrateCamera2, file ....
What?! A different error at least. But I did supply an initial intrinsic matrix!
So I go back to the documentation, and notice the flags parameter:
flags – Different flags that may be zero or a combination of the
following values:
CV_CALIB_USE_INTRINSIC_GUESS cameraMatrix contains valid initial
values of fx, fy, cx, cy that are optimized further
...
Aha, so I have to tell the function explicitly to use the initial guesses I provided:
cv2.calibrateCamera([obj_points], [img_points],size, camera_matrix.T, dist_coefs,
flags=cv2.CALIB_USE_INTRINSIC_GUESS)
Hurrah! It works!
(Moral of the story - read the OpenCV documentation carefully, and use the newest version (i.e. on opencv.itseez.com) if you're using the Python cv2 interface. Also, consult the examples in the samples/python2 directory to supplement the documentation. With these two things you should be able to work out most problems.)
After the help from mathematical.coffee I have got this 3d calibration to run.
import cv2
from cv2 import cv
import numpy as np
obj_points = [[-9.7,3.0,4.5],[-11.1,0.5,3.1],[-8.5,0.9,2.4],[-5.8,4.4,2.7],[-4.8,1.5,0.2],[-6.7,-1.6,-0.4],[-8.7,-3.3,-0.6],[-4.3,-1.2,-2.4],[-12.4,-2.3,0.9],[-14.1,-3.8,-0.6],[-18.9,2.9,2.9],[-14.6,2.3,4.6],[-16.0,0.8,3.0],[-18.9,-0.1,0.3],[-16.3,-1.7,0.5],[-18.6,-2.7,-2.2]]
img_points = [[993.0,623.0],[942.0,705.0],[1023.0,720.0],[1116.0,645.0],[1136.0,764.0],[1071.0,847.0],[1003.0,885.0],[1142.0,887.0],[886.0,816.0],[827.0,883.0],[710.0,636.0],[837.0,621.0],[789.0,688.0],[699.0,759.0],[768.0,800.0],[697.0,873.0]]
obj_points = np.array(obj_points,'float32')
img_points = np.array(img_points,'float32')
w = 1680
h = 1050
size = (w,h)
camera_matrix = np.zeros((3, 3),'float32')
camera_matrix[0,0]= 2200.0
camera_matrix[1,1]= 2200.0
camera_matrix[2,2]=1.0
camera_matrix[0,2]=750.0
camera_matrix[1,2]=750.0
dist_coefs = np.zeros(4,'float32')
retval,camera_matrix,dist_coefs,rvecs,tvecs = cv2.calibrateCamera([obj_points],[img_points],size,camera_matrix,dist_coefs,flags=cv.CV_CALIB_USE_INTRINSIC_GUESS)
The only problem I have now is why is the dist_coefs vector is 5 elements long when returned from the calibration function. the documentation says " if the vector contains four elements, it means that K3=0". But in fact K3 is is used, no matter the length of dist_coefs (4 or 5). Furthermore I can't seem to get flag CV_CALIB_FIX_K3 to work, tied to use that flag to force K3 to be zero. cashes saying an integer is required. this could be because I don't know how to do multiple flags at once, I'm just doing this, flags = (cv.CV..., cv.CV...).
Just to compare, from the matlab camera cal routine the results are...
Focal length: 2210. 2207.
principal point: 781. 738.
Distortions: 4.65e-2 -9.74e+0 3.9e-3 6.74e-3 0.0e+0
Rotation vector: 2.36 0.178 -0.131
Translation vector: 16.016 2.527 69.549
From this code,
Focal length: 1647. 1629.
principal point: 761. 711.
Distortions: -2.3e-1 2.0e+1 1.4e-2 -9.5e-2 -172e+2
Rotation vector: 2.357 0.199 -0.193
Translation vector: 16.511 3.307 48.946
I think if I could figure out how to force k3=0, the rest of the values would align right up.
For what it is worth, the following code snippet currently works under 2.4.6.1:
pattern_size = (16, 12)
pattern_points = np.zeros( (np.prod(pattern_size), 3), np.float32)
pattern_points[:, :2] = np.indices(pattern_size).T.reshape(-1, 2).astype(np.float32)
img_points = pattern_points[:, :2] * 2 + np.array([40, 30], np.float32)
print(cv2.calibrateCamera([pattern_points], [img_points], (400, 400), flags=cv2.CALIB_USE_INTRINSIC_GUESS))
Note that camera_matrix and dist_coefs are not needed.
Make dist_coeffs vector as 5 dimensional zero vector and then use CV_CALIB_FIX_K3 flag. You can see that last element in the vector (K3) will be zero.
When it comes to using multiple flags, you can OR them.
Example : cv.CV_CALIB_USE_INTRINSIC_GUESS | cv.CV_CALIB_FIX_K3
Use Point3f and Point2f instead of Point3d and Point2d to define object_points and image_points and it will work.
Related
I'd like to build a GP with marginalized hyperparameters.
I have seen that this is possible with the HMC sampler provided in gpflow from this notebook
However, when I tried to run the following code as a first step of this (NOTE this is on gpflow 0.5, an older version), the returned samples are negative, even though the lengthscale and variance need to be positive (negative values would be meaningless).
import numpy as np
from matplotlib import pyplot as plt
import gpflow
from gpflow import hmc
X = np.linspace(-3, 3, 20)
Y = np.random.exponential(np.sin(X) ** 2)
Y = (Y - np.mean(Y)) / np.std(Y)
k = gpflow.kernels.Matern32(1, lengthscales=.2, ARD=False)
m = gpflow.gpr.GPR(X[:, None], Y[:, None], k)
m.kern.lengthscales.prior = gpflow.priors.Gamma(1., 1.)
m.kern.variance.prior = gpflow.priors.Gamma(1., 1.)
# dont want likelihood be a hyperparam now so fixed
m.likelihood.variance = 1e-6
m.likelihood.variance.fixed = True
m.optimize(maxiter=1000)
samples = m.sample(500)
print(samples)
Output:
[[-0.43764571 -0.22753325]
[-0.50418501 -0.11070128]
[-0.5932655 0.00821438]
[-0.70217714 0.05077999]
[-0.77745654 0.09362291]
[-0.79404456 0.13649446]
[-0.83989415 0.27118385]
[-0.90355789 0.29589641]
...
I don't know too much in detail about HMC sampling but I would expect that the sampled posterior hyperparameters are positive, I've checked the code and it seems maybe related to the Log1pe transform, though I failed to figure it out myself.
Any hint on this?
It would be helpful if you specified which GPflow version you are using - especially given that from the output you posted it looks like you are using a really old version of GPflow (pre-1.0), and this is actually something that got improved since. What is happening here (in old GPflow) is that the sample() method returns a single array S x P, where S is the number of samples, and P is the number of free parameters [e.g. for a M x M matrix parameter with lower-triangular transform (such as the Cholesky of the covariance of the approximate posterior, q_sqrt), only M * (M - 1)/2 parameters are actually stored and optimised!]. These are the values in the unconstrained space, i.e. they can take any value whatsoever. Transforms (see gpflow.transforms module) provide the mapping between this value (between plus/minus infinity) and the constrained value (e.g. gpflow.transforms.positive for lengthscales and variances). In old GPflow, the model provides a get_samples_df() method that takes the S x P array returned by sample() and returns a pandas DataFrame with columns for all the trainable parameters which would be what you want. Or, ideally, you would just use a recent version of GPflow, in which the HMC sampler directly returns the DataFrame!
I'm simply trying to find the exact minimum of a simple function in MATLAB. I've been experimenting with the use of built-in functions such as "fminbnd" and inline function definition, but I don't think I quite know what I'm doing.
My code is below. I want to find the x and y of Error's minimum.
clear all
A = 5;
tau = linspace(1,4,500); %Array of many tau values between 1 and 4
E1 = qfunc(((-tau) + 5) /(sqrt(2.5)));
E0 = qfunc((tau)/(sqrt(2.5)));
Error = 0.5*E0 + 0.5*E1;
figure
subplot (311), plot(tau, E0);
xlabel('Threshold (Tau)'), ylabel('E0')
title('Error vs. Threshold (E0, 1 <= T <= 4)')
subplot (312), plot(tau, E1);
xlabel('Threshold (Tau)'), ylabel('E1')
title('Error vs. Threshold (E1, 1 <= T <= 4)')
subplot (313), plot(tau, Error);
xlabel('Threshold (Tau)'), ylabel('Pr[Error]');
title('Error vs. Threshold (Pr[Error], 1 <= T <= 4)')
I mean, I can use the cursor when the function is graphed to get close (though not right at the point where it occurs (Threshold = 2.5), but there must be a method just to print the number to the window. So far I have tried:
fminbnd('Error', 'E0', 'E1')
And many other variants. Also tried using anonymous and inline function definitions with no luck.
Can anyone point me in the right direction? Feel foolish for being stuck with this simple problem... Any help greatly appreciated!
See fminbnd
You should try something like this:
Error =#(tau) 0.5*qfunc(((-tau) + 5) /(sqrt(2.5))) + 0.5*qfunc((tau)/(sqrt(2.5)));
x = fminbnd(Error,0,10)
The first argument of fminbnd(f,x1,x2) is the function and the other arguments are the bounds. I did f=Error, x1=0 and x2=10.
Output:
x=2.5000
Another way is to save your error function in .m file. See the webpage above.
I don't understand why you're using E0 and E1 as the limits of the range where the minimum should be found. Or am I misunderstanding something in your code?
Maybe if you have your function as a discrete collection of samples (as seems implied from your way of constructing it, error is going to be a matrix, I think), you could use the "min" command: http://www.mathworks.es/es/help/matlab/ref/min.html
Hope this helped!
EDIT: After some more testing and a response form the scipy mailing list, the issue appears to be with fspecial(). To get the same output I need to generate the same kind of kernel in Python as the Matlab fspecial command is producing. For now I will try to export the kernel from matlab and work from there. Added as a edit since question has been "closed"
I am trying to port the following MATLAB code to Python. It seems to work but the output is different form MATLAB. I think the problem is with apply a "mean" filter to the log(amplituide). Any help appreciated.
The MATLAB code is from: http://www.klab.caltech.edu/~xhou/projects/spectralResidual/spectralresidual.html
%% Read image from file
inImg = im2double(rgb2gray(imread('1.jpg')));
inImg = imresize(inImg, 64/size(inImg, 2));
%% Spectral Residual
myFFT = fft2(inImg);
myLogAmplitude = log(abs(myFFT));
myPhase = angle(myFFT);
mySpectralResidual = myLogAmplitude - imfilter(myLogAmplitude, fspecial('average', 3), 'replicate');
saliencyMap = abs(ifft2(exp(mySpectralResidual + i*myPhase))).^2;
%% After Effect
saliencyMap = mat2gray(imfilter(saliencyMap, fspecial('gaussian', [10, 10], 2.5)));
imshow(saliencyMap);
Here is my attempt in python:
from skimage import img_as_float
from skimage.io import imread
from skimage.color import rgb2gray
from scipy import fftpack, ndimage, misc
from scipy.ndimage import uniform_filter
from matplotlib.pyplot as plt
# Read image from file
image = img_as_float(rgb2gray(imread('1.jpg')))
image = misc.imresize(image, 64.0 / image.shape[0])
# Spectral Residual
fft = fftpack.fft2(image)
logAmplitude = np.log(np.abs(fft))
phase = np.angle(fft)
avgLogAmp = uniform_filter(logAmplitude, size=3, mode="nearest") #Is this same a applying "mean" filter
spectralResidual = logAmplitude - avgLogAmp
saliencyMap = np.abs(fftpack.ifft2(np.exp(spectralResidual + 1j * phase))) ** 2
# After Effect
saliencyMap = ndimage.gaussian_filter(sm, sigma=2.5)
plt.imshow(sm)
plt.show()
For completness here is a input image and the output from MATLAB and python.
I doubt anyone will be able to give you a firm answer on this. It could be any number of things... Could be that one FFT is 0-centered while the other isn't, could be a float vs double somewhere, could be mishandling of absolute value, could be a filter setting, ...
If I were you, I'd write out some intermediate values for both computations and find a way to compare them. Start in the middle, if they compare well then move down, if they don't compare well then move up. Maybe write an intermediate value from the python script out to a file, import into matlab, take the element-wise difference, and graph. If they're not the same dimensions, that's clue #1.
I am trying to use Octave's fminsearch function, which I have used in MATLAB before. The function seems not sufficiently documented (for me at least), and I have no idea how to set to options such that it would actually minimize.
I tried fitting a very simple exponential function using the code at the end of this message. I want the following:
I want the function to take as input the x- and y-values, just like MATLAB would do. Furthermore, I want some control over the options, to make sure that it actually minimizes (i.e. to a minimum!).
Of course, in the end I want to fit functions that are more complicated than exponential, but I want to be able to fit exponentials at least.
I have several problems with fminsearch:
I tried handing over the x- and y-value to the function, but a matlab-style thing like this:
[xx,fval]=fminsearch(#exponential,[1000 1],x,y);
or
[xx,fval]=fminsearch(#exponential,[33000 1],options,x,y)
produces errors:
error: options(6) does not correspond to known algorithm
error: called from:
error: /opt/local/share/octave/packages/optim-1.0.6/fmins.m at line 72, column 16
error: /opt/local/share/octave/packages/optim-1.0.6/fminsearch.m at line 29, column 4
Or, respectively (for the second case above):
error: `x' undefined near line 4 column 3
error: called from:
error: /Users/paul/exponential.m at line 4, column 2
error: /opt/local/share/octave/packages/optim-1.0.6/nmsmax.m at line 63, column 6
error: /opt/local/share/octave/packages/optim-1.0.6/fmins.m at line 77, column 9
error: /opt/local/share/octave/packages/optim-1.0.6/fminsearch.m at line 29, column 4
Apparently, the order of arguments that fminsearch takes is different from the one in MATLAB. So, how is this order??
How can I make fminsearch take values and options?
I found a workaround to the problem that the function would not take values: I defined the x- and y values as global. Not elegant, but at least then the values are available in the function.
Nonetheless, fminsearch does not minimize properly.
This is shown below:
Here is the function:
function f=exponential(coeff)
global x
global y
X=x;
Y=y;
a= coeff(1);
b= coeff(2);
Y_fun = a .* exp(-X.*b);
DIFF = Y_fun - Y;
SQ_DIFF = DIFF.^2;
f=sum(SQ_DIFF);
end
Here is the code:
global x
global y
x=[0:1:200];
y=4930*exp(-0.0454*x);
options(10)=10000000;
[cc,fval]=fminsearch(#exponential,[5000 0.01])
This is the output:
cc =
4930.0 5184.6
fval = 2.5571e+08
Why does fminsearch not find the solution?
There is an fminsearch implementation in the octave-forge package "optim".
You can see in its implementation file that the third parameter is always an options vector, the fourth is always a grad vector, so your ,x,y invocations will not work.
You can also see in the implementation that it calls an fmins implementation.
The documentation of that fmins implementation states:
if options(6)==0 && options(5)==0 - regular simplex
if options(6)==0 && options(5)==1 - right-angled simplex
Comment: the default is set to "right-angled simplex".
this works better for me on a broad range of problems,
although the default in nmsmax is "regular simplex"
A recent problem of mine would solve fine with matlab's fminsearch, but not with this octave-forge implementation. I had to specify an options vector [0 1e-3 0 0 0 0] to have it use a regular simplex instead of a 'right-angled simplex'. The octave default makes no sense if your coefficients differ vastly in scale.
The optimization function fminsearch will always try to find a minimum, no matter what the options are. So if you are finding it's not finding a minimum, it's because it failed to do so.
From the code you provide, I cannot determine what goes wrong. The solution with the globals should work, and indeed does work over here, so something else on your side must be going awry. (NOTE: I do use MATLAB, not Octave, so those two functions could be slightly different...)
Anyway, why not do it like this?
function f = exponential(coeff)
x = 0:1:200;
y = 4930*exp(-0.0454*x);
a = coeff(1);
b = coeff(2);
Y_fun = a .* exp(-x.*b);
f = sum((Y_fun-y).^2);
end
Or, if you must pass x and y as external parameters,
x = [0:1:200];
y = 4930*exp(-0.0454*x);
[cc,fval] = fminsearch(#(c)exponential(c,x,y),[5000 0.01])
function f = exponential(coeff,x,y)
a = coeff(1);
b = coeff(2);
Y_fun = a .* exp(-x.*b);
f = sum((Y_fun-y).^2);
end
I am currently using the Toolbox Graph on the Matlab File Exchange to calculate curvature on 3D surfaces and find them very helpful (http://www.mathworks.com/matlabcentral/fileexchange/5355). However, the following error message is issued in “compute_curvature” for certain surface descriptions and the code fails to run completely:
> Error in ==> compute_curvature_mod at 75
> dp = sum( normal(:,E(:,1)) .* normal(:,E(:,2)), 1 );
> ??? Index exceeds matrix dimensions.
This happens only sporadically, but there is no obvious reason why the toolbox works perfectly fine for some surfaces and not for others (of a similar topology). I also noticed that someone had asked about this bug back in November 2009 on File Exchange, but that the question had gone unanswered. The post states
"compute_curvature will generate an error on line 75 ("dp = sum(
normal(:,E(:,1)) .* normal(:,E(:,2)), 1 );") for SOME surfaces. The
error stems from E containing indices that are out of range which is
caused by line 48 ("A = sparse(double(i),double(j),s,n,n);") where A's
values eventually entirely make up the E matrix. The problem occurs
when the i and j vectors create the same ordered pair twice in which
case the sparse function adds the two s vector elements together for
that matrix location resulting in a value that is too large to be used
as an index on line 75. For example, if i = [1 1] and j = [2 2] and s
= [3 4] then A(1,2) will equal 3 + 4 = 7.
The i and j vectors are created here:
i = [face(1,:) face(2,:) face(3,:)];
j = [face(2,:) face(3,:) face(1,:)];
Just wanted to add that the error I mentioned is caused by the
flipping of the sign of the surface normal of just one face by
rearranging the order of the vertices in the face matrix"
I have tried debugging the code myself but have not had any luck. I am wondering if anyone here has solved the problem or could give me insight – I need the code to be sufficiently general-purpose in order to calculate curvature for a variety of surfaces, not just for a select few.
The November 2009 bug report on File Exchange traces the problem back to the behavior of sparse:
S = SPARSE(i,j,s,m,n,nzmax) uses the rows of [i,j,s] to generate an
m-by-n sparse matrix with space allocated for nzmax nonzeros. The
two integer index vectors, i and j, and the real or complex entries
vector, s, all have the same length, nnz, which is the number of
nonzeros in the resulting sparse matrix S . Any elements of s
which have duplicate values of i and j are added together.
The lines of code where the problem originates are here:
i = [face(1,:) face(2,:) face(3,:)];
j = [face(2,:) face(3,:) face(1,:)];
s = [1:m 1:m 1:m];
A = sparse(i,j,s,n,n);
Based on this information removal of the repeat indices, presumably using unique or similar, might solve the problem:
[B,I,J] = unique([i.' j.'],'rows');
i = B(:,1).';
j = B(:,2).';
s = s(I);
The full solution may look something like this:
i = [face(1,:) face(2,:) face(3,:)];
j = [face(2,:) face(3,:) face(1,:)];
s = [1:m 1:m 1:m];
[B,I,J] = unique([i.' j.'],'rows');
i = B(:,1).';
j = B(:,2).';
s = s(I);
A = sparse(i,j,s,n,n);
Since I do not have a detailed understanding of the algorithm it is hard to tell whether the removal of entries will have a negative effect.