I am trying to artificially manipulate a 2D image using a rigid 3D transformation (T). Specifically, I have an image and I want to transform using T it to determine the image if captured from a different location.
Here's what I have so far:
The problem reduces to determining the plane-induced homography (Hartley and Zisserman Chapter 13) - without camera calibration matrices this is H = R-t*n'/d.
I am unsure, however, how to define n and d. I know that they help to define the world plane, but I'm not sure how to define them in relation to the first image plane (e.g. the camera plane of the original image).
Please advise! Thanks! K
Not sure what you mean by "first image plane": the camera's?
The vector n and the scalar d define the equation of the plane defining the homography:
n X + d = 0, or, in coordinates, n_x * x + n_y * y + n_z * z + d = 0, for every point X = (x, y, z) belonging to the plane.
There are various ways to estimate the homography. For example, you can map a quadrangle on the plane to a rectangle of known aspect ratio.
Or you can estimate the locations of vanishing points (this comes handy when you have, say, an image of a skyscraper, with nice rows of windows). In this case, if p and q are the homogeneous coordinates of the vanishing points of two orthogonal lines on a plane in the scene, then normal to the plane in camera coordinates is simply given by (p X q) / (|p| |q|)
Related
EDIT:
What I have: camera intrinsics, extrinsic from calibration, 2D image and depth map
What I need: 2D virtual view image
I am trying to generate a novel view(right view) for Depth Image Based Rendering. Reason for this is that only the left image and depth map are available at the receiver which has to reconstruct the right view (see image).
I want to know if these steps will give me the desired result or what I should be doing instead,
First, by using Camera Calibration toolbox for MATLAB by CalTech, the intrinsic, extrinsic matrices can be obtained.
Then, they can be mapped to 3D world points using calibration parameters by this method "http://nicolas.burrus.name/index.php/Research/KinectCalibration#tocLink2"
Now, I want to back project this to a new image plane(right view). The right view is simply a translation of left and no rotation because of the setup.
How do I do this reconstruction?
Also, Can I estimate R and T from MATLAB stereo calibration tool and transform every point in original left view to right view using P2 = R*P1+T,
P1 and P2 are image points of 3D world point P in the respective planes.
Any idea and help are highly appreciated, would rephrase/add details if the question is not clear.
(Theoretic answer*)
You have to define what R and T means. If I understand, is the Roto-translation of your (main) left camera. If you can map a point P (like your P1 or P2) in 3D space, the correspondance with a point m (I not call it p to avoid confusion) in your left camera is (unless you use a different convention (pseudocode)
m = K[R|t]*P
which
P1 = (X,Y,Z,1)
m = (u',v',w)
but you want 2D coordinates so the coordinates in your left camera are:
u = u'/w
v = v'/w
if you already roto-translated P1 into P2 (not very useful) is equal to (pseudocode)
1 0 0 0
m = K[I|0]*P = K*[0 1 0 0] * P2
0 0 1 0
Assume this is the theoretical relationship with a 3D point P with his 2D point in an image m, you may think to have you right camera in a different position. If there is only translation with respect to left camera, the right camera is translated of T2 with respect to the left camera and roto-translated of R/T+T2 with respect to the centre of the world.
So the m' proiected point in your right camera should be (assuming that the cameras are equal means the have the same intrinsics K)
m' = K[R|T+T2]*P = K[I|T2]*P2
I is the identity matrix.
If you want to transform m directly into m' withot using 3D points you have to implement epipolar geometry.
If cameras are different with different K, if the calibration of R and T has not the same standard of calibration of K, this equation may not work.
If calibration is not well-done, it could work but with errors.
I have given 3d points of a scene or a subset of these points comprising one object of the scene. I would like to create a depth image from these points, that is the pixel value in the image encodes the distance of the corresponding 3d point to the camera.
I have found the following similar question
http://www.mathworks.in/matlabcentral/newsreader/view_thread/319097
however the answers there do not help me, since I want to use MATLAB. To get the image values is not difficult (e.g. simply compute the distance of each 3d point to the camera's origin), however I do not know how to figure out the corresponding locations in the 2d image.
I could only imagine that you project all 3d points on a plane and bin their positions on the plane in discrete, well, rectangles on the plane. Then you could average the depth value for each bin.
I could however imagine that the result of such a procedure would be a very pixelated image, not being very smooth.
How would you go about this problem?
Assuming you've corrected for camera tilt (a simple matrix multiplication if you know the angle), you can probably just follow this example
X = data(:,1);
Y = data(:,1);
Z = data(:,1);
%// This bit requires you to make some choices like the start X and Z, end X and Z and resolution (X and Z) of your desired depth map
[Xi, Zi] = meshgrid(X_start:X_res:X_end, Z_start:Z_res:Z_end);
depth_map = griddata(X,Z,Y,Xi,Zi)
I am using ray tracing and at the beginning I assumed a plane surface so I used the equation of the plane surface which is :
Ax + BY + CZ +d = 0
while A,B and C are the component of the normal vector of the Plane Normal = [A B C]
and using the Ray equation : Ray = Source + t*Direction
And then solve it for t and I can find the intersection points.
My question now that I have function in matlab to read the surface of the object but the object may not be plane surface and I am getting the data of the surface [X Y Z] of the surface but I don't know which equation should I use to find t and then the intersection point. And I even have a function to give me the normal vector at each point
If you can edit the tags the get the right ones please do it.
It might not be a plane, but you can always calculate a normal vector at each point. You'll just have to work harder to do it. Take two partial derivatives in planar coordinates, cross those vectors, and that's the normal at the point.
If your surface is defined as a height Z on a some X-Y grid, you can solve it easily using fzero. This would exclude some complex shapes, but might work for standard optics problems like a ray hitting a lens. Assume that X, Y and Z are 2-d matrices with the same shape. You can then do a 2d interpolation like
z_interp = interp2(X,Y,Z,x_interp,y_interp)
If this is not the case, you should try to define your own function that can calculate z based on x and y.
For the line, we have
x_ray = x_source + t * x_dir
y_ray = y_source + t * y_dir
z_ray = z_source + t * z_dir
So you can now define a function that calculates the height above the surface as a function of t as
height_above_plane = #(t) z_source + t * z_dir - interp2(X, Y, Z, ...
x_source + t*x_dir, y_source + t*y_dir)
Note that this might not be the shortest distance from the point to the plane, it is just the height measured along the z-direction. The time the ray hits the surface can now be found by searching for the t for which the height is zero. This can be done for arbitrary functions using fzero:
t_intercept = fzero(height_above_plane, 0);
This should work well for simple cases where the function defining the surface is relatively smooth and the ray crosses the surface only once. It might be possible to transform cases with more complex geometry into such a simple case.
If you can get the X Y Z of the surface and you said you can get the normal vector in each point so what is your problem now?
The X Y Z of the surface are the intersection points and if you have the normal vector in each point so you can calculate whatever you want ( the reflected or the refracted rays).
I think you have no troubles at all
I want to calculate the magnetic field from a given image using biot savarts law. For example if I have a picture of a triangle, I say that this triangle forms a closed wire carrying current. Using image derivatives I can get the co-ordinates and direction of the current (normals included). I am struggling implementing this...need a bit of help with logic too. Here is what I have:
Img = imread('littletriangle.bmp');
Img = Img(:,:,1);
Img = double(Img);
[x,y] = size(Img);
[Ix, Iy] = gradient(Img);
biot savart equation is:
b = mu/4*pi sum(Idl x rn / r^2)
where mu/4pi is const, I is current magnitude, rn distance unit vector between a pixel and current, r^2 is the squared magnitude of the displacement between a pixel and the current.
So just to start off, I read the image in, turn it into a binary and then take the image gradient. This gives me the location and orientation of the 'current'. I now need to calculate the magnetic field from this 'current' at every pixel in the image. I am only interested in getting the magnetic field in the x-y plane. anything just to start me off would be brilliant!
For wire
B = mu * I /(2*pi*r)
B is vector and has. Direction is perpendicular on line between wire an point of interest. Fastest way to rotate vector by 90° is just swapping (x.y) so it becomes (y,x) vector
What about current? If you deal whit current then current is homogenous inside of wire (can be triangle) and I in upper direction is just normalized I per point and per Whole I.
So how to do this?
Get current per pixel (current / number of pixel in shape)
For each point calculate B using (r calculated form protagora) as sum of all other mini wires expressed as pixel using upper equation. (B is vector and has also direction, so keep track of B as (x,y) )
having picture of 100*100 will yield (100*100)*(100*100) calculations of B equation or something less if you will not calculate filed from empty space.
B is at the end instead of just mu * I /(2*pi*r) sum of all wire and I becomes dI
You do not need to apply any derivatives, just integration (sum)
I'm trying to obtain the 3D metric reconstruction of the points I have in two different views of my scene by means of a pair of iPhone 4S (a rudimental stereo system).
To do so, I did calibrate the cameras, estimate the fundamental matrix and obtained an estimate of the essential matrix. Now, in Hartley&Zisserman "Multiple View Geometry in CV" book, I see that to any given E, they correspond 4 canonical cameras pairs, of which only one reconstructs as the "actual" stereo configuration.
The problem is that they say [cit.]"...the reconstructed point will be in front of both cameras in one of these four solutions only. Thus, testing with a single point to determine if it is in front of both cameras is sufficient to decide between the four different solutions for the camera matrix P'. ..."
Given that I know F, K_left and K_right, how does one establish whether a 3D point is in front of both cameras or not?
Thanks,
Riccardo
You can get the camera rotation and translation from essential matrix. So, you have the camera matrix P = [R, -R*t]. To test if point X = (x, y, z, 1) is in front of the camera, compute X' = P*X. X' = (x', y', z') will be position of point if camera is in the origin, looking toward z direction. So, if z' is positive, then X' is in front of the camera and X is in front of camera P. Else X is behind P.