Related
i understood using perlin-noise to make a pattern with cubes but what i want to is a random terrain
plain land such as this game called muck (that has been devolped by unity)
i've tried doing:
noise = PerlinNoise(octaves=3, seed=2007)
amp = 3
freq = 24
width = 30
for po in range(width*width):
s = randint(200, 255)
q = Entity(model="cube", collider="box", texture="white_cube",
color=color.rgb(s, s, s))
q.x = floor(po/width)
q.z = floor(po % width)
q.y = floor(noise([q.x/freq, q.z/freq]) * amp)
so this would give a -almost- perfect random terrain but i want my terrain look more realistic than cubic
thanks in advance ;)
Unsurprisingly, your landscape looks cubic because you're constructing it from cubes. So to make it more realistic, use a more flexible Mesh for it. You will have to assign each point of the surface separately and connect it to its surrounding via triangles:
level_parent = Entity(model=Mesh(vertices=[], uvs=[]), color=color.white, texture='white_cube')
for x in range(1, width):
for z in range(1, width):
# add two triangles for each new point
y00 = noise([x/freq, z/freq]) * amp
y10 = noise([(x-1)/freq, z/freq]) * amp
y11 = noise([(x-1)/freq, (z-1)/freq]) * amp
y01 = noise([x/freq, (z-1)/freq]) * amp
level_parent.model.vertices += (
# first triangle
(x, y00, z),
(x-1, y10, z),
(x-1, y11, z-1),
# second triangle
(x, y00, z),
(x-1, y11, z-1),
(x, y01, z-1)
)
level_parent.model.generate()
level_parent.model.project_uvs() # for texture
level_parent.model.generate_normals() # for lighting
level_parent.collider = 'mesh' # for collision
However, this will be quite slow to generate, mostly due to the Perlin noise implementation used (I'm assuming this one). A faster option can be found in this answer.
I wrote a program that takes in entry some points, expressed in 3D coordinates and that must be drawn in a 2D canvas. I use perspective projection, homogeneous coordinates and similar triangles to do that. However, my program does not work and I actually don't know why.
I followed two tutorials. I really understood the geometrical definitions and properties I have read. However, my implementation fails... I will write references to these both courses little by little, to make your reading more confortable :).
Overview : geometrical reminders
The perspective projection is done following this workflow (cf. these 2 courses - I wrote pertinent links about the latter (HTML anchors) further down, in this post) :
Definition of the points to draw, expressed according to the world's coordinates system ; Definition of the matrix of projection, which is a matrix of transformation that "converts" a point expressed according to the world coordinates system into a point expressed according to the camera's coordinates system (NB : I believe this matrix can be understood as being the 3D object "camera")
Product of these points with this matrix (as defined in the adequat part, below, in this document) : the product of these world-expressed points results in the conversion of these points to the camera's coordinates system. Note that points and matrix are expressed in 4D (concept of homogenous coordinates).
Use of similar triangles concept to project (only computing is done at this step) on the canvas the in-camera-expressed points (using their 4D coordinates). After this operation, the points are now expressed in 3D (the third coordinate is computed but not actually used on the canvas). The 4th coordinate is removed because not useful. Note that the 3rd coordinate won't be useful, except to handle z-fighting (though, I don't want to do that).
Last step : rasterization, to actually draw the pixels on the canvas (other computing AND displaying are done at this step).
First, the problem
Well, I want to draw a cube but the perspective doesn't work. The projected points seem to be drawn withtout perspective.
What result I should expect for
The result I'm expecting is the cube displayed in "Image" part of this below PNG :
What I'm outputting
The faces of my cube are odd, as if perspective wasn't well used.
I guess I know why I'm having this problem...
I think my projection matrix (i.e. : the camera) doesn't have the good coefficients. I'm using a very simple projection matrix, without the concepts of fov, near and far clipping planes (as you can see belower).
Indeed, to get the expected result (as previouslyt defined), the camera should be placed, if I'm not mistaken, at the center (on axes x and y) of the cube expressed according to the world coordinate system and at the center (on axes x and y) of the canvas, which is (I make this assumption) placed 1 z in front of the camera.
The Scastie (snippet)
NB : since X11 is not activated on Scastie, the window I want to create won't be shown.
https://scastie.scala-lang.org/N95TE2nHTgSlqCxRHwYnxA
Entries
Perhaps the problem is bound to the entries ? Well, I give you them.
Cube's points
Ref. : myself
val world_cube_points : Seq[Seq[Double]] = Seq(
Seq(100, 300, -4, 1), // top left
Seq(100, 300, -1, 1), // top left z+1
Seq(100, 0, -4, 1), // bottom left
Seq(100, 0, -1, 1), // bottom left z+1
Seq(400, 300, -4, 1), // top right
Seq(400, 300, -1, 1), // top right z+1
Seq(400, 0, -4, 1), // bottom right
Seq(400, 0, -1, 1) // bottom right z+1
)
Transformation (Projection) matrix
Ref. : https://www.scratchapixel.com/lessons/3d-basic-rendering/perspective-and-orthographic-projection-matrix/building-basic-perspective-projection-matrix , End of the Part. "A Simple Perspective Matrix"
Note that I'm using the simplest perspective projection matrix : I don't use concept of fov, near and far clipping planes.
new Matrix(Seq(
Seq(1, 0, 0, 0),
Seq(0, 1, 0, 0),
Seq(0, 0, -1, 0),
Seq(0, 0, -1, 0)
))
Consequence of this matrix : each point P(x;y;z;w) producted with this matrix will be : P'(x;y;-z;-z).
Second, the first operation my program does : a simple product of a point with a matrix.
Ref. : https://github.com/ssloy/tinyrenderer/wiki/Lesson-4:-Perspective-projection#homogeneous-coordinates
/**
* Matrix in the shape of (use of homogeneous coordinates) :
* c00 c01 c02 c03
* c10 c11 c12 c13
* c20 c21 c22 c23
* 0 0 0 1
*
* #param content the content of the matrix
*/
class Matrix(val content : Seq[Seq[Double]]) {
/**
* Computes the product between a point P(x ; y ; z) and the matrix.
*
* #param point a point P(x ; y ; z ; 1)
* #return a new point P'(
* x * c00 + y * c10 + z * c20
* ;
* x * c01 + y * c11 + z * c21
* ;
* x * c02 + y * c12 + z * c22
* ;
* 1
* )
*/
def product(point : Seq[Double]) : Seq[Double] = {
(0 to 3).map(
i => content(i).zip(point).map(couple2 => couple2._1 * couple2._2).sum
)
}
}
Then, use of similar triangles
Ref. 1/2 : Part. "Of the Importance of Converting Points to Camera Space
" of https://www.scratchapixel.com/lessons/3d-basic-rendering/computing-pixel-coordinates-of-3d-point/mathematics-computing-2d-coordinates-of-3d-points
Ref. 2/2 : https://github.com/ssloy/tinyrenderer/wiki/Lesson-4:-Perspective-projection#time-to-work-in-full-3d
NB : at this step, the entries are points expressed according to the camera (i.e. : they are the result of the precedently defined product with the precedently defined matrix).
class Projector {
/**
* Computes the coordinates of the projection of the point P on the canvas.
* The canvas is assumed to be 1 unit forward the camera.
* The computation uses the definition of the similar triangles.
*
* #param points the point P we want to project on the canvas. Its coordinates must be expressed in the coordinates
* system of the camera before using this function.
* #return the point P', projection of P.
*/
def drawPointsOnCanvas(points : Seq[Seq[Double]]) : Seq[Seq[Double]] = {
points.map(point => {
point.map(coordinate => {
coordinate / point(3)
}).dropRight(1)
})
}
}
Finally, the drawing of the projected points, onto the canvas.
Ref. : Part. "From Screen Space to Raster Space" of https://www.scratchapixel.com/lessons/3d-basic-rendering/computing-pixel-coordinates-of-3d-point/mathematics-computing-2d-coordinates-of-3d-points
import java.awt.Graphics
import javax.swing.JFrame
/**
* Assumed to be 1 unit forward the camera.
* Contains the drawn points.
*/
class Canvas(val drawn_points : Seq[Seq[Double]]) extends JFrame {
val CANVAS_WIDTH = 820
val CANVAS_HEIGHT = 820
val IMAGE_WIDTH = 900
val IMAGE_HEIGHT = 900
def display = {
setTitle("Perlin")
setSize(IMAGE_WIDTH, IMAGE_HEIGHT)
setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE)
setVisible(true)
}
override def paint(graphics : Graphics): Unit = {
super.paint(graphics)
drawn_points.foreach(point => {
if(!(Math.abs(point.head) <= CANVAS_WIDTH / 2 || Math.abs(point(1)) <= CANVAS_HEIGHT / 2)) {
println("WARNING : the point (" + point.head + " ; " + point(1) + ") can't be drawn in this canvas.")
} else {
val normalized_drawn_point = Seq((point.head + (CANVAS_WIDTH / 2)) / CANVAS_WIDTH, (point(1) + (CANVAS_HEIGHT / 2)) / CANVAS_HEIGHT)
graphics.fillRect((normalized_drawn_point.head * IMAGE_WIDTH).toInt, ((1 - normalized_drawn_point(1)) * IMAGE_HEIGHT).toInt, 5, 5)
graphics.drawString(
"P(" + (normalized_drawn_point.head * IMAGE_WIDTH).toInt + " ; "
+ ((1 - normalized_drawn_point(1)) * IMAGE_HEIGHT).toInt + ")",
(normalized_drawn_point.head * IMAGE_WIDTH).toInt - 50, ((1 - normalized_drawn_point(1)) * IMAGE_HEIGHT).toInt - 10
)
}
})
}
}
Question
What's wrong with my program ? I understood the geometrical concepts explained by these both tutorials that I read carefully. I'm pretty sure my product works. I think either the rasterization, or the entries (the matrix) could be wrong...
Note that I'm using the simplest perspective projection matrix : I don't use concept of fov, near and far clipping planes.
I think that your projection matrix is too simple. By dropping the near and far clipping planes, you are dropping perspective projection entirely.
You do not have to perform the z-clipping step, but you need to define a view frustum to get perspective to work. I believe that your projection matrix defines a cubic "view frustrum", hence no perspective.
See http://www.songho.ca/opengl/gl_projectionmatrix.html for a discussion of how the projection matrix works.
Quoting the Scratchapixel page:
... If we substitute these numbers in the above equation, we get:
Where y' is the y coordinate of P'. Thus:
This is probably one the simplest and most fundamental relation in computer graphics, known as the z or perspective divide. The exact same principle applies to the x coordinate. ...
And in your code:
def drawPointsOnCanvas(points : Seq[Seq[Double]]) : Seq[Seq[Double]] = {
points.map(point => {
point.map(coordinate => {
coordinate / point(3)
^^^^^^^^
...
The (3) index is the 4th component of point, i.e. its W-coordinate, not its Z-coordinate. Perhaps you meant coordinate / point(2)?
I would like to draw a line on a 2DTexture within unity. This works well within the editor. However, in compiled builds it will flicker as it fills the pixels of the texture with clear colour then re-draws the line.
Here's my drawling code:
public void DrawLineWithMouse (Vector2 _middleVector)
{
tex.SetPixels(fillPixels); // Fills texture with an array of clear pixels
Vector3 newMousePos = Input.mousePosition;
newMousePos.z = 0;
int x0 = ((int)newMousePos.x)/RenderingManager.pixelSize / thickness; //Screen.width-
int y0 = ((int)newMousePos.y)/RenderingManager.pixelSize / thickness; //Screen.height-
int x1 = (int)_middleVector.x - (x0 - (int)_middleVector.x );
int y1 = (int)_middleVector.y - (y0 - (int)_middleVector.y );
DrawLineHelper(tex, x0, y0, x1, y1, Color.yellow); // redraws the line
tex.Apply();
}
I do not get flickering when I create a new texture then apply it. However, this is very, very expensive and for testing purposes only.
public void DrawLineWithMouse (Vector2 _middleVector)
{
tex = new Texture2D(Screen.width/RenderingManager.pixelSize / thickness, Screen.height/RenderingManager.pixelSize / thickness);
tex.SetPixels(fillPixels);
tex.SetPixels(fillPixels);
tex.filterMode = FilterMode.Point;
Vector3 newMousePos = Input.mousePosition;
newMousePos.z = 0;
int x0 = ((int)newMousePos.x)/RenderingManager.pixelSize / thickness; //Screen.width-
int y0 = ((int)newMousePos.y)/RenderingManager.pixelSize / thickness; //Screen.height-
int x1 = (int)_middleVector.x - (x0 - (int)_middleVector.x );
int y1 = (int)_middleVector.y - (y0 - (int)_middleVector.y );
DrawLineHelper(tex, x0, y0, x1, y1, Color.yellow);
GetComponent<Renderer>().material.mainTexture = tex;
It's flicking because apply is an expensive operation. The documentation states:
This is a potentially expensive operation, so you'll want to change as many pixels as possible between Apply calls.
You have to find a way to do less calls to DrawLineWithMouse. What's the best solution, that's up to you and depends on what you are doing.
One alternative is to accumulate deltas of mouse movement and draw after x milliseconds. Another alternative is to draw a mesh on top of the texture instead of changing the texture, and then only change the texture when necessary, if ever (note that you can use a RenderTexture here to easily draw the mesh in the texture).
Finally, lowering the resolution of the texture makes Apply run faster.
BOUNTY STATUS UPDATE:
I discovered how to map a linear lens, from destination coordinates to source coordinates.
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
1). I actually struggle to reverse it, and to map source coordinates to destination coordinates. What is the inverse, in code in the style of the converting functions I posted?
2). I also see that my undistortion is imperfect on some lenses - presumably those that are not strictly linear. What is the equivalent to-and-from source-and-destination coordinates for those lenses? Again, more code than just mathematical formulae please...
Question as originally stated:
I have some points that describe positions in a picture taken with a fisheye lens.
I want to convert these points to rectilinear coordinates. I want to undistort the image.
I've found this description of how to generate a fisheye effect, but not how to reverse it.
There's also a blog post that describes how to use tools to do it; these pictures are from that:
(1) : SOURCE Original photo link
Input : Original image with fish-eye distortion to fix.
(2) : DESTINATION Original photo link
Output : Corrected image (technically also with perspective correction, but that's a separate step).
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
My function stub looks like this:
Point correct_fisheye(const Point& p,const Size& img) {
// to polar
const Point centre = {img.width/2,img.height/2};
const Point rel = {p.x-centre.x,p.y-centre.y};
const double theta = atan2(rel.y,rel.x);
double R = sqrt((rel.x*rel.x)+(rel.y*rel.y));
// fisheye undistortion in here please
//... change R ...
// back to rectangular
const Point ret = Point(centre.x+R*cos(theta),centre.y+R*sin(theta));
fprintf(stderr,"(%d,%d) in (%d,%d) = %f,%f = (%d,%d)\n",p.x,p.y,img.width,img.height,theta,R,ret.x,ret.y);
return ret;
}
Alternatively, I could somehow convert the image from fisheye to rectilinear before finding the points, but I'm completely befuddled by the OpenCV documentation. Is there a straightforward way to do it in OpenCV, and does it perform well enough to do it to a live video feed?
The description you mention states that the projection by a pin-hole camera (one that does not introduce lens distortion) is modeled by
R_u = f*tan(theta)
and the projection by common fisheye lens cameras (that is, distorted) is modeled by
R_d = 2*f*sin(theta/2)
You already know R_d and theta and if you knew the camera's focal length (represented by f) then correcting the image would amount to computing R_u in terms of R_d and theta. In other words,
R_u = f*tan(2*asin(R_d/(2*f)))
is the formula you're looking for. Estimating the focal length f can be solved by calibrating the camera or other means such as letting the user provide feedback on how well the image is corrected or using knowledge from the original scene.
In order to solve the same problem using OpenCV, you would have to obtain the camera's intrinsic parameters and lens distortion coefficients. See, for example, Chapter 11 of Learning OpenCV (don't forget to check the correction). Then you can use a program such as this one (written with the Python bindings for OpenCV) in order to reverse lens distortion:
#!/usr/bin/python
# ./undistort 0_0000.jpg 1367.451167 1367.451167 0 0 -0.246065 0.193617 -0.002004 -0.002056
import sys
import cv
def main(argv):
if len(argv) < 10:
print 'Usage: %s input-file fx fy cx cy k1 k2 p1 p2 output-file' % argv[0]
sys.exit(-1)
src = argv[1]
fx, fy, cx, cy, k1, k2, p1, p2, output = argv[2:]
intrinsics = cv.CreateMat(3, 3, cv.CV_64FC1)
cv.Zero(intrinsics)
intrinsics[0, 0] = float(fx)
intrinsics[1, 1] = float(fy)
intrinsics[2, 2] = 1.0
intrinsics[0, 2] = float(cx)
intrinsics[1, 2] = float(cy)
dist_coeffs = cv.CreateMat(1, 4, cv.CV_64FC1)
cv.Zero(dist_coeffs)
dist_coeffs[0, 0] = float(k1)
dist_coeffs[0, 1] = float(k2)
dist_coeffs[0, 2] = float(p1)
dist_coeffs[0, 3] = float(p2)
src = cv.LoadImage(src)
dst = cv.CreateImage(cv.GetSize(src), src.depth, src.nChannels)
mapx = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
mapy = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
cv.InitUndistortMap(intrinsics, dist_coeffs, mapx, mapy)
cv.Remap(src, dst, mapx, mapy, cv.CV_INTER_LINEAR + cv.CV_WARP_FILL_OUTLIERS, cv.ScalarAll(0))
# cv.Undistort2(src, dst, intrinsics, dist_coeffs)
cv.SaveImage(output, dst)
if __name__ == '__main__':
main(sys.argv)
Also note that OpenCV uses a very different lens distortion model to the one in the web page you linked to.
(Original poster, providing an alternative)
The following function maps destination (rectilinear) coordinates to source (fisheye-distorted) coordinates. (I'd appreciate help in reversing it)
I got to this point through trial-and-error: I don't fundamentally grasp why this code is working, explanations and improved accuracy appreciated!
def dist(x,y):
return sqrt(x*x+y*y)
def correct_fisheye(src_size,dest_size,dx,dy,factor):
""" returns a tuple of source coordinates (sx,sy)
(note: values can be out of range)"""
# convert dx,dy to relative coordinates
rx, ry = dx-(dest_size[0]/2), dy-(dest_size[1]/2)
# calc theta
r = dist(rx,ry)/(dist(src_size[0],src_size[1])/factor)
if 0==r:
theta = 1.0
else:
theta = atan(r)/r
# back to absolute coordinates
sx, sy = (src_size[0]/2)+theta*rx, (src_size[1]/2)+theta*ry
# done
return (int(round(sx)),int(round(sy)))
When used with a factor of 3.0, it successfully undistorts the images used as examples (I made no attempt at quality interpolation):
Dead link
(And this is from the blog post, for comparison:)
If you think your formulas are exact, you can comput an exact formula with trig, like so:
Rin = 2 f sin(w/2) -> sin(w/2)= Rin/2f
Rout= f tan(w) -> tan(w)= Rout/f
(Rin/2f)^2 = [sin(w/2)]^2 = (1 - cos(w))/2 -> cos(w) = 1 - 2(Rin/2f)^2
(Rout/f)^2 = [tan(w)]^2 = 1/[cos(w)]^2 - 1
-> (Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
However, as #jmbr says, the actual camera distortion will depend on the lens and the zoom. Rather than rely on a fixed formula, you might want to try a polynomial expansion:
Rout = Rin*(1 + A*Rin^2 + B*Rin^4 + ...)
By tweaking first A, then higher-order coefficients, you can compute any reasonable local function (the form of the expansion takes advantage of the symmetry of the problem). In particular, it should be possible to compute initial coefficients to approximate the theoretical function above.
Also, for good results, you will need to use an interpolation filter to generate your corrected image. As long as the distortion is not too great, you can use the kind of filter you would use to rescale the image linearly without much problem.
Edit: as per your request, the equivalent scaling factor for the above formula:
(Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
-> Rout/f = [Rin/f] * sqrt(1-[Rin/f]^2/4)/(1-[Rin/f]^2/2)
If you plot the above formula alongside tan(Rin/f), you can see that they are very similar in shape. Basically, distortion from the tangent becomes severe before sin(w) becomes much different from w.
The inverse formula should be something like:
Rin/f = [Rout/f] / sqrt( sqrt(([Rout/f]^2+1) * (sqrt([Rout/f]^2+1) + 1) / 2 )
I blindly implemented the formulas from here, so I cannot guarantee it would do what you need.
Use auto_zoom to get the value for the zoom parameter.
def dist(x,y):
return sqrt(x*x+y*y)
def fisheye_to_rectilinear(src_size,dest_size,sx,sy,crop_factor,zoom):
""" returns a tuple of dest coordinates (dx,dy)
(note: values can be out of range)
crop_factor is ratio of sphere diameter to diagonal of the source image"""
# convert sx,sy to relative coordinates
rx, ry = sx-(src_size[0]/2), sy-(src_size[1]/2)
r = dist(rx,ry)
# focal distance = radius of the sphere
pi = 3.1415926535
f = dist(src_size[0],src_size[1])*factor/pi
# calc theta 1) linear mapping (older Nikon)
theta = r / f
# calc theta 2) nonlinear mapping
# theta = asin ( r / ( 2 * f ) ) * 2
# calc new radius
nr = tan(theta) * zoom
# back to absolute coordinates
dx, dy = (dest_size[0]/2)+rx/r*nr, (dest_size[1]/2)+ry/r*nr
# done
return (int(round(dx)),int(round(dy)))
def fisheye_auto_zoom(src_size,dest_size,crop_factor):
""" calculate zoom such that left edge of source image matches left edge of dest image """
# Try to see what happens with zoom=1
dx, dy = fisheye_to_rectilinear(src_size, dest_size, 0, src_size[1]/2, crop_factor, 1)
# Calculate zoom so the result is what we wanted
obtained_r = dest_size[0]/2 - dx
required_r = dest_size[0]/2
zoom = required_r / obtained_r
return zoom
I took what JMBR did and basically reversed it. He took the radius of the distorted image (Rd, that is, the distance in pixels from the center of the image) and found a formula for Ru, the radius of the undistorted image.
You want to go the other way. For each pixel in the undistorted (processed image), you want to know what the corresponding pixel is in the distorted image.
In other words, given (xu, yu) --> (xd, yd). You then replace each pixel in the undistorted image with its corresponding pixel from the distorted image.
Starting where JMBR did, I do the reverse, finding Rd as a function of Ru. I get:
Rd = f * sqrt(2) * sqrt( 1 - 1/sqrt(r^2 +1))
where f is the focal length in pixels (I'll explain later), and r = Ru/f.
The focal length for my camera was 2.5 mm. The size of each pixel on my CCD was 6 um square. f was therefore 2500/6 = 417 pixels. This can be found by trial and error.
Finding Rd allows you to find the corresponding pixel in the distorted image using polar coordinates.
The angle of each pixel from the center point is the same:
theta = arctan( (yu-yc)/(xu-xc) ) where xc, yc are the center points.
Then,
xd = Rd * cos(theta) + xc
yd = Rd * sin(theta) + yc
Make sure you know which quadrant you are in.
Here is the C# code I used
public class Analyzer
{
private ArrayList mFisheyeCorrect;
private int mFELimit = 1500;
private double mScaleFESize = 0.9;
public Analyzer()
{
//A lookup table so we don't have to calculate Rdistorted over and over
//The values will be multiplied by focal length in pixels to
//get the Rdistorted
mFisheyeCorrect = new ArrayList(mFELimit);
//i corresponds to Rundist/focalLengthInPixels * 1000 (to get integers)
for (int i = 0; i < mFELimit; i++)
{
double result = Math.Sqrt(1 - 1 / Math.Sqrt(1.0 + (double)i * i / 1000000.0)) * 1.4142136;
mFisheyeCorrect.Add(result);
}
}
public Bitmap RemoveFisheye(ref Bitmap aImage, double aFocalLinPixels)
{
Bitmap correctedImage = new Bitmap(aImage.Width, aImage.Height);
//The center points of the image
double xc = aImage.Width / 2.0;
double yc = aImage.Height / 2.0;
Boolean xpos, ypos;
//Move through the pixels in the corrected image;
//set to corresponding pixels in distorted image
for (int i = 0; i < correctedImage.Width; i++)
{
for (int j = 0; j < correctedImage.Height; j++)
{
//which quadrant are we in?
xpos = i > xc;
ypos = j > yc;
//Find the distance from the center
double xdif = i-xc;
double ydif = j-yc;
//The distance squared
double Rusquare = xdif * xdif + ydif * ydif;
//the angle from the center
double theta = Math.Atan2(ydif, xdif);
//find index for lookup table
int index = (int)(Math.Sqrt(Rusquare) / aFocalLinPixels * 1000);
if (index >= mFELimit) index = mFELimit - 1;
//calculated Rdistorted
double Rd = aFocalLinPixels * (double)mFisheyeCorrect[index]
/mScaleFESize;
//calculate x and y distances
double xdelta = Math.Abs(Rd*Math.Cos(theta));
double ydelta = Math.Abs(Rd * Math.Sin(theta));
//convert to pixel coordinates
int xd = (int)(xc + (xpos ? xdelta : -xdelta));
int yd = (int)(yc + (ypos ? ydelta : -ydelta));
xd = Math.Max(0, Math.Min(xd, aImage.Width-1));
yd = Math.Max(0, Math.Min(yd, aImage.Height-1));
//set the corrected pixel value from the distorted image
correctedImage.SetPixel(i, j, aImage.GetPixel(xd, yd));
}
}
return correctedImage;
}
}
I found this pdf file and I have proved that the maths are correct (except for the line vd = *xd**fv+v0 which should say vd = **yd**+fv+v0).
http://perception.inrialpes.fr/CAVA_Dataset/Site/files/Calibration_OpenCV.pdf
It does not use all of the latest co-efficients that OpenCV has available but I am sure that it could be adapted fairly easily.
double k1 = cameraIntrinsic.distortion[0];
double k2 = cameraIntrinsic.distortion[1];
double p1 = cameraIntrinsic.distortion[2];
double p2 = cameraIntrinsic.distortion[3];
double k3 = cameraIntrinsic.distortion[4];
double fu = cameraIntrinsic.focalLength[0];
double fv = cameraIntrinsic.focalLength[1];
double u0 = cameraIntrinsic.principalPoint[0];
double v0 = cameraIntrinsic.principalPoint[1];
double u, v;
u = thisPoint->x; // the undistorted point
v = thisPoint->y;
double x = ( u - u0 )/fu;
double y = ( v - v0 )/fv;
double r2 = (x*x) + (y*y);
double r4 = r2*r2;
double cDist = 1 + (k1*r2) + (k2*r4);
double xr = x*cDist;
double yr = y*cDist;
double a1 = 2*x*y;
double a2 = r2 + (2*(x*x));
double a3 = r2 + (2*(y*y));
double dx = (a1*p1) + (a2*p2);
double dy = (a3*p1) + (a1*p2);
double xd = xr + dx;
double yd = yr + dy;
double ud = (xd*fu) + u0;
double vd = (yd*fv) + v0;
thisPoint->x = ud; // the distorted point
thisPoint->y = vd;
This can be solved as an optimization problem. Simply draw on curves in images that are supposed to be straight lines. Store the contour points for each of those curves. Now we can solve the fish eye matrix as a minimization problem. Minimize the curve in points and that will give us a fisheye matrix. It works.
It can be done manually by adjusting the fish eye matrix using trackbars! Here is a fish eye GUI code using OpenCV for manual calibration.
I have an RGB-D image and am trying to get a 3D visualization in matlab. Currently I am doing:
depth = imread('img_031_depth.png');
depth = double(depth);
img = imread('img_031.png');
surf(depth, img, 'FaceColor', 'texturemap', 'EdgeColor', 'none' )
view(158, 38)
Which gives me an image like:
I have two questions:
1) how can I save the image without it blurring as above
2) As you can see some edges show lined going to zero (e.g. the top of the coffee cup) I would like to remove these.
What I'm trying to produce is a 3D looking pointcloud, as these are only 2.5D I must show them from the right angle.
Any help is appreciated
EDIT: added images (note depth image needs to be normalized for visualization)
If you are only interested in a point cloud, you might want to consider scatter3.
You can select which points to plot (discard those with depth == 0).
You need to have explicit x-y coordinates though.
[y x] = ndgrid( 1:size(img,1), 1:size(img,2) );
sel = depth > 0 ; % which points to plot
% "flatten" the matrices for scatter plot
x = x(:);
y = y(:);
img = reshape( img, [], 3 );
depth = depth(:);
scatter3( x(sel), y(sel), depth(sel), 20, img( sel, : ), 'filled' );
view(158, 38)
Edit: sampled version
[y x] = ndgrid( 1:2:size(img,1), 1:2:size(img,2) );
sel = depth( 1:2:end, 1:2:end ) > 0;
x = x(:);
y = y(:);
img = reshape( img( 1:2:end, 1:2:end, : ), [], 3 );
depth = depth( 1:2:end, 1:2:end );
scatter( x(sel), y(sel), depth(sel), 20, img( sel, : ), 'filled' );
view( 158, 38 );
Alternatively, you can directly manipulate sel mask.
i suggest you first restore x=zu/f and y=zv/f, to obtain x, y, z, where f is your camera focal length;
then apply whatever rotation, translation you want before displaying them [x’,y’,z’] = R[x, y, z] + t;
then project them back using col = xf/z+w/2, row = h/2-yf/z to get a simple image that you can display fast; you can add a depth buffer to the last operation to guarantee
proper occlusions by writing depth at each pixel there and checking that repetitive writing happens only if new z is smaller (that is a new pixel is close to the viewer). The resulting image will still have holes due to the nature of point clouds. You can interpolate in those holes but this means you have to trace rays from every pixels in the image to your point cloud and find a closest neighbor to the ray which probably takes forever in Matlab.
I am also doing some 3D image restoring and reconstructing. The first question is easy. Your photo is taken by a camera. So you need to transform the position to camera coordinate system. In other words, you need to know some intrinsic value of your camera! Or you can never recover it with a single image. Google 'kinect intrinsic value' you can get the focal length etc.
Also, change your view.
Try this! And if it's not working, ask again.