Related
I'm trying to use Mapbox Terrain RGB to get elevation for specific points in space. I used mercantile.tile to get the coordinates of the tile containing my point at zoom level 15, which for -43º, -22º (for simplicity sake) is 12454, 18527, then mercantile.xy to get the corresponding world coordinates: -4806237.7150042495, -2621281.2257876047.
Shouldn't the integer part of -4806237.7150042495 / 256 (tile size) equal the x coordinate of the tile containing the point, that is, 12454? If this calculation checked out I'd figure that I'm looking for the pixel column (x axis) corresponding to the decimal part of the result, like column 127(256 * 0,5) for 12454,5. However, the division results in -18774.366, (which is curiously close to the tile y coordinate, but it looks like a coincidence). What am I missing here?
As an alternative, I thought of using mercantile.bounds, assigning the first and last pixel columns to the westmost and eastmost longitudes, and finding my position with interpolation, but I wanted to check if I'm doing this the right/recommended way. I'm interested in point elevations, so everything said here goes for the Y axis as well.
Here's what I got so far:
def correct_altitude_mode(kml):
with open(kml, "r+") as f:
txt = f.read()
if re.search("(?<=<altitudeMode>)relative(?=<\/altitudeMode>)", txt):
lat = round(float(find_with_re("latitude", txt)), 5)
lng = round(float(find_with_re("longitude", txt)), 5)
alt = round(float(find_with_re("altitude", txt)), 5)
z = 15
tile = mercantile.tile(lng, lat, z)
westmost, southmost, eastmost, northmost = mercantile.bounds(tile)
pixel_column = np.interp(lng, [westmost, eastmost], [0,256])
pixel_row = np.interp(lat, [southmost, northmost], [256, 0])
response = requests.get(f"https://api.mapbox.com/v4/mapbox.terrain-rgb/{z}/{tile.x}/{tile.y}.pngraw?access_token=pk.eyJ1IjoibWFydGltcGFzc29zIiwiYSI6ImNra3pmN2QxajBiYWUycW55N3E1dG1tcTEifQ.JFKSI85oP7M2gbeUTaUfQQ")
buffer = BytesIO(response.content)
tile_img = png.read_png_int(buffer)
_,R,G,B = (tile_img[int(pixel_row), int(pixel_column)])
print(tile_img[int(pixel_row), int(pixel_column)])
height = -10000 + ((R * 256 * 256 + G * 256 + B) * 0.1)
print(f"R:{R},G:{G},B:{B}\n{height}")
plt.hlines(pixel_row, 0.0, 256.0, colors="r")
plt.vlines(pixel_column, 0.0, 256.0, colors="r")
plt.imshow(tile_img)
I have a static image with only center point lat/long (for example https://api.mapbox.com/styles/v1/mapbox/light-v9/static/-78.4649,42.5128,5,0,0/300x200) and I want to put on this map some markers(lat.long) with the help of canvas.
But I need to calculate somehow the xy coordinates for those markers.
So I know the center of map(lat/long) and the lat/long marker coordinates. Is there any way to convert lat/long to xy knowing only zoom level and center?
Or if I know the xy of the center lat/long(it always be the same 150px * 100px) and zoom level, could I calculate the xy for other markers?
I have a lot of markers (>200, and they all are custom svg generated and so on) to place it on this map. I can't use mapbox mapbox static map because of the markers limitation and so on.
UPD: Based on the comments I updated the question.
How to calculate it for 256px square tiles?
Based on the OP comment I'm assuming that the requested image is square, for the sake of simplicity (TILE_SIZE could be decomposed in a TILE_SIZE_X and TILE_SIZE_Y component). I'm also assuming that the image is 256-pixels wide TILE_SIZE=256
I'm giving both the pixel coordinates relative to the center of the image (distanceInPixels function), and to the Lower Left Corner (imageCoordinates function). Changing to the Upper Left Corner in case that's necessary should be trivial (X will be equal and Y = TILE_SIZE -Y).
<!DOCTYPE html>
<html>
<body>
<p id="demo"></p>
<script>
var latLngMarker = {};
var latLngCenter = {};
// Image dimensions in pixels
var TILE_SIZE = 256;
var zoom = 5;
// Coordinates of the marker to be projected on the image
latLngMarker.lat = 41.850;
latLngMarker.lng = -87.650;
// Coordinates of the image center
latLngCenter.lat = 41.850;
latLngCenter.lng = -87.650;
// Coordinates projected on the cartographic plane (Mercator)
var centerProjected = project(latLngCenter);
var markerProjected = project(latLngMarker);
// The result should be X=Y=0, because I made Marker Lat Lng = Center Lat Lng
var distanceFromCenter = distanceInPixels(centerProjected, markerProjected);
alert("X: " + distanceFromCenter.x + " Y: " + distanceFromCenter.y);
// The result should be X=Y=256/2=128 for the same reason
var coords = imageCoordinates(centerProjected, markerProjected);
alert("X: " + coords.x + " Y: " + coords.y);
// The horizontal distance represented by one pixel for a given latitude and zoom level
function pixelResolution (latLng, zoom) {
var radius = 6378137.0 // semi-axis of WGS84 ellipsoid
var circumference = 2 * Math.PI * radius;
var distancePerImage = circumference * Math.cos(latLng.lat * Math.PI / 180.0) / Math.pow(2,zoom);
var distancePerPixel = distancePerImage / TILE_SIZE;
return distancePerPixel
}
// Web mercator projection.
function project(latLng) {
var siny = Math.sin(latLng.lat * Math.PI / 180);
siny = Math.min(Math.max(siny, -0.9999), 0.9999);
var xy = {};
xy.x = TILE_SIZE * (0.5 + latLng.lng / 360);
xy.y = TILE_SIZE * (0.5 - Math.log((1 + siny) / (1 - siny)) / (4 * Math.PI));
return xy
}
// Marker pixel coordinates relative to the image Center
function distanceInPixels(centerProjected, markerProjected) {
var delta = {};
var spacing = pixelResolution(latLngCenter, zoom);
delta.x = Math.round((centerProjected.x - markerProjected.x)/spacing);
delta.y = Math.round((centerProjected.y - markerProjected.y)/spacing);
return delta
}
// Marker pixel coordinates relative to the Lower Left Corner
function imageCoordinates(centerProjected, markerProjected) {
var pixelCoordinates = {};
var spacing = pixelResolution(latLngCenter, zoom);
var deltaPixels = distanceInPixels(centerProjected, markerProjected);
pixelCoordinates.x = TILE_SIZE / 2 - deltaPixels.x;
pixelCoordinates.y = TILE_SIZE / 2 - deltaPixels.y;
return pixelCoordinates
}
</script>
</body>
</html>
Note: I can confirm that the pixelResolution function only works with square image tiles with dimensions of powers of 2. The Math.pow(2,zoom); snippet gives the game away!
Web Mercator function based on:
https://developers-dot-devsite-v2-prod.appspot.com/maps/documentation/javascript/examples/map-coordinates
Horizontal distance represented by one pixel from :
https://wiki.openstreetmap.org/wiki/Zoom_levels
See also:
https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames#Resolution_and_Scale
If you're going to linearly interpolate you'd need to know the lat/long & x/y for 2 points. It wouldn't be possible with only the center point unless you also have a conversion metric for pixels - ie. 50 pixels is .1 delta lat/long.
If you have the lat/long & x/y for two points you can create the ratio as y1 - y2 / lat1-lat2 or x1-x2/long1-long2 each of which should result in the same ratio
Then it'd be relatively easy, assume the ratio is 5 meaning 5px/l so you had a point that was (3,-4) away from that center point you'd simply multiple to find the pixel offset (15,-20) and add that to the center = (165, 80).
Since all of your images are zoomed the same amount you could manually calculate the ratio once and store it as a constant.
sudo/untested python:
def getRatio(latlongs=[(1,1),(0,0)], xys=[(5,5),(0,0)]:
return (xys[0][1]-xys[1][1]) / (latlongs[0][0] - latlongs[1][0])
centerLatLong = (5,5)
centerXY = (150, 100)
def getCoord(lat,long,ratio):
y = (lat-centerLatLong[0])*ratio + centerXY[1]
x = (long-centerLatLong[1])*ratio + centerXY[0]
return x, y
I'm trying to calculate the principal axis via principal component analysis. I have a pointcloud and use for this the Point Cloud Library (pcl). Furthermore, I try to visualize the principal axis I calculated in rviz with markers. Here is the code snipped I use:
void computePrincipalAxis(const PointCloud& cloud, Eigen::Vector4f& centroid, Eigen::Matrix3f& evecs, Eigen::Vector3f& evals) {
Eigen::Matrix3f covariance_matrix;
pcl::computeCovarianceMatrix(cloud, centroid, covariance_matrix);
pcl::eigen33(covariance_matrix, evecs, evals);
}
void createArrowMarker(Eigen::Vector3f& vec, int id, double length) {
visualization_msgs::Marker marker;
marker.header.frame_id = frameId;
marker.header.stamp = ros::Time();
marker.id = id;
marker.type = visualization_msgs::Marker::ARROW;
marker.action = visualization_msgs::Marker::ADD;
marker.pose.position.x = centroid[0];
marker.pose.position.y = centroid[1];
marker.pose.position.z = centroid[2];
marker.pose.orientation.x = vec[0];
marker.pose.orientation.y = vec[1];
marker.pose.orientation.z = vec[2];
marker.pose.orientation.w = 1.0;
marker.scale.x = length;
marker.scale.y = 0.02;
marker.scale.z = 0.02;
marker.color.a = 1.0;
marker.color.r = 1.0;
marker.color.g = 1.0;
marker.color.b = 0.0;
featureVis.markers.push_back(marker);
}
Eigen::Vector4f centroid;
Eigen::Matrix3f evecs;
Eigen::Vector3f evals;
// Table is the pointcloud of the table only.
pcl::compute3DCentroid(*table, centroid);
computePrincipalAxis(*table, centroid, evecs, evals);
Eigen::Vector3f vec;
vec << evecs.col(0);
createArrowMarker(vec, 1, evals[0]);
vec << evecs.col(1);
createArrowMarker(vec, 2, evals[1]);
vec << evecs.col(2);
createArrowMarker(vec, 3, evals[2]);
publish();
This results in the following visualization:
I'm aware that the scale is not very perfect. The two longer arrows are much too long. But I'm confused about a few things:
I think the small arrow should go either up, or downwards.
What does the value orientation.w of the arrow's orientation mean?
Do you have some hints what I did wrong?
Orientations are represented by Quaternions in ROS, not by directional vectors. Quaternions can be a bit unintuitive, but fortunately there are some helper functions in the tf package, to generate quaternions, for example, from roll/pitch/yaw-angles.
One way to fix the marker would therefore be, to convert the direction vector into a quaternion.
In your special case, there is a much simpler solution, though: Instead of setting origin and orientation of the arrow, it is also possible to define start and end point (see ROS wiki about marker types). So instead of setting the pose attribute, just add start and end point to the points attribute:
float k = 1.0; // optional to scale the length of the arrows
geometry_msgs::Point p;
p.x = centroid[0];
p.y = centroid[1];
p.z = centroid[2];
marker.points.push_back(p);
p.x += k * vec[0];
p.y += k * vec[1];
p.z += k * vec[2];
marker.points.push_back(p);
You can set k to some value < 1 to reduce the length of the arrows.
BOUNTY STATUS UPDATE:
I discovered how to map a linear lens, from destination coordinates to source coordinates.
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
1). I actually struggle to reverse it, and to map source coordinates to destination coordinates. What is the inverse, in code in the style of the converting functions I posted?
2). I also see that my undistortion is imperfect on some lenses - presumably those that are not strictly linear. What is the equivalent to-and-from source-and-destination coordinates for those lenses? Again, more code than just mathematical formulae please...
Question as originally stated:
I have some points that describe positions in a picture taken with a fisheye lens.
I want to convert these points to rectilinear coordinates. I want to undistort the image.
I've found this description of how to generate a fisheye effect, but not how to reverse it.
There's also a blog post that describes how to use tools to do it; these pictures are from that:
(1) : SOURCE Original photo link
Input : Original image with fish-eye distortion to fix.
(2) : DESTINATION Original photo link
Output : Corrected image (technically also with perspective correction, but that's a separate step).
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
My function stub looks like this:
Point correct_fisheye(const Point& p,const Size& img) {
// to polar
const Point centre = {img.width/2,img.height/2};
const Point rel = {p.x-centre.x,p.y-centre.y};
const double theta = atan2(rel.y,rel.x);
double R = sqrt((rel.x*rel.x)+(rel.y*rel.y));
// fisheye undistortion in here please
//... change R ...
// back to rectangular
const Point ret = Point(centre.x+R*cos(theta),centre.y+R*sin(theta));
fprintf(stderr,"(%d,%d) in (%d,%d) = %f,%f = (%d,%d)\n",p.x,p.y,img.width,img.height,theta,R,ret.x,ret.y);
return ret;
}
Alternatively, I could somehow convert the image from fisheye to rectilinear before finding the points, but I'm completely befuddled by the OpenCV documentation. Is there a straightforward way to do it in OpenCV, and does it perform well enough to do it to a live video feed?
The description you mention states that the projection by a pin-hole camera (one that does not introduce lens distortion) is modeled by
R_u = f*tan(theta)
and the projection by common fisheye lens cameras (that is, distorted) is modeled by
R_d = 2*f*sin(theta/2)
You already know R_d and theta and if you knew the camera's focal length (represented by f) then correcting the image would amount to computing R_u in terms of R_d and theta. In other words,
R_u = f*tan(2*asin(R_d/(2*f)))
is the formula you're looking for. Estimating the focal length f can be solved by calibrating the camera or other means such as letting the user provide feedback on how well the image is corrected or using knowledge from the original scene.
In order to solve the same problem using OpenCV, you would have to obtain the camera's intrinsic parameters and lens distortion coefficients. See, for example, Chapter 11 of Learning OpenCV (don't forget to check the correction). Then you can use a program such as this one (written with the Python bindings for OpenCV) in order to reverse lens distortion:
#!/usr/bin/python
# ./undistort 0_0000.jpg 1367.451167 1367.451167 0 0 -0.246065 0.193617 -0.002004 -0.002056
import sys
import cv
def main(argv):
if len(argv) < 10:
print 'Usage: %s input-file fx fy cx cy k1 k2 p1 p2 output-file' % argv[0]
sys.exit(-1)
src = argv[1]
fx, fy, cx, cy, k1, k2, p1, p2, output = argv[2:]
intrinsics = cv.CreateMat(3, 3, cv.CV_64FC1)
cv.Zero(intrinsics)
intrinsics[0, 0] = float(fx)
intrinsics[1, 1] = float(fy)
intrinsics[2, 2] = 1.0
intrinsics[0, 2] = float(cx)
intrinsics[1, 2] = float(cy)
dist_coeffs = cv.CreateMat(1, 4, cv.CV_64FC1)
cv.Zero(dist_coeffs)
dist_coeffs[0, 0] = float(k1)
dist_coeffs[0, 1] = float(k2)
dist_coeffs[0, 2] = float(p1)
dist_coeffs[0, 3] = float(p2)
src = cv.LoadImage(src)
dst = cv.CreateImage(cv.GetSize(src), src.depth, src.nChannels)
mapx = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
mapy = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
cv.InitUndistortMap(intrinsics, dist_coeffs, mapx, mapy)
cv.Remap(src, dst, mapx, mapy, cv.CV_INTER_LINEAR + cv.CV_WARP_FILL_OUTLIERS, cv.ScalarAll(0))
# cv.Undistort2(src, dst, intrinsics, dist_coeffs)
cv.SaveImage(output, dst)
if __name__ == '__main__':
main(sys.argv)
Also note that OpenCV uses a very different lens distortion model to the one in the web page you linked to.
(Original poster, providing an alternative)
The following function maps destination (rectilinear) coordinates to source (fisheye-distorted) coordinates. (I'd appreciate help in reversing it)
I got to this point through trial-and-error: I don't fundamentally grasp why this code is working, explanations and improved accuracy appreciated!
def dist(x,y):
return sqrt(x*x+y*y)
def correct_fisheye(src_size,dest_size,dx,dy,factor):
""" returns a tuple of source coordinates (sx,sy)
(note: values can be out of range)"""
# convert dx,dy to relative coordinates
rx, ry = dx-(dest_size[0]/2), dy-(dest_size[1]/2)
# calc theta
r = dist(rx,ry)/(dist(src_size[0],src_size[1])/factor)
if 0==r:
theta = 1.0
else:
theta = atan(r)/r
# back to absolute coordinates
sx, sy = (src_size[0]/2)+theta*rx, (src_size[1]/2)+theta*ry
# done
return (int(round(sx)),int(round(sy)))
When used with a factor of 3.0, it successfully undistorts the images used as examples (I made no attempt at quality interpolation):
Dead link
(And this is from the blog post, for comparison:)
If you think your formulas are exact, you can comput an exact formula with trig, like so:
Rin = 2 f sin(w/2) -> sin(w/2)= Rin/2f
Rout= f tan(w) -> tan(w)= Rout/f
(Rin/2f)^2 = [sin(w/2)]^2 = (1 - cos(w))/2 -> cos(w) = 1 - 2(Rin/2f)^2
(Rout/f)^2 = [tan(w)]^2 = 1/[cos(w)]^2 - 1
-> (Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
However, as #jmbr says, the actual camera distortion will depend on the lens and the zoom. Rather than rely on a fixed formula, you might want to try a polynomial expansion:
Rout = Rin*(1 + A*Rin^2 + B*Rin^4 + ...)
By tweaking first A, then higher-order coefficients, you can compute any reasonable local function (the form of the expansion takes advantage of the symmetry of the problem). In particular, it should be possible to compute initial coefficients to approximate the theoretical function above.
Also, for good results, you will need to use an interpolation filter to generate your corrected image. As long as the distortion is not too great, you can use the kind of filter you would use to rescale the image linearly without much problem.
Edit: as per your request, the equivalent scaling factor for the above formula:
(Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
-> Rout/f = [Rin/f] * sqrt(1-[Rin/f]^2/4)/(1-[Rin/f]^2/2)
If you plot the above formula alongside tan(Rin/f), you can see that they are very similar in shape. Basically, distortion from the tangent becomes severe before sin(w) becomes much different from w.
The inverse formula should be something like:
Rin/f = [Rout/f] / sqrt( sqrt(([Rout/f]^2+1) * (sqrt([Rout/f]^2+1) + 1) / 2 )
I blindly implemented the formulas from here, so I cannot guarantee it would do what you need.
Use auto_zoom to get the value for the zoom parameter.
def dist(x,y):
return sqrt(x*x+y*y)
def fisheye_to_rectilinear(src_size,dest_size,sx,sy,crop_factor,zoom):
""" returns a tuple of dest coordinates (dx,dy)
(note: values can be out of range)
crop_factor is ratio of sphere diameter to diagonal of the source image"""
# convert sx,sy to relative coordinates
rx, ry = sx-(src_size[0]/2), sy-(src_size[1]/2)
r = dist(rx,ry)
# focal distance = radius of the sphere
pi = 3.1415926535
f = dist(src_size[0],src_size[1])*factor/pi
# calc theta 1) linear mapping (older Nikon)
theta = r / f
# calc theta 2) nonlinear mapping
# theta = asin ( r / ( 2 * f ) ) * 2
# calc new radius
nr = tan(theta) * zoom
# back to absolute coordinates
dx, dy = (dest_size[0]/2)+rx/r*nr, (dest_size[1]/2)+ry/r*nr
# done
return (int(round(dx)),int(round(dy)))
def fisheye_auto_zoom(src_size,dest_size,crop_factor):
""" calculate zoom such that left edge of source image matches left edge of dest image """
# Try to see what happens with zoom=1
dx, dy = fisheye_to_rectilinear(src_size, dest_size, 0, src_size[1]/2, crop_factor, 1)
# Calculate zoom so the result is what we wanted
obtained_r = dest_size[0]/2 - dx
required_r = dest_size[0]/2
zoom = required_r / obtained_r
return zoom
I took what JMBR did and basically reversed it. He took the radius of the distorted image (Rd, that is, the distance in pixels from the center of the image) and found a formula for Ru, the radius of the undistorted image.
You want to go the other way. For each pixel in the undistorted (processed image), you want to know what the corresponding pixel is in the distorted image.
In other words, given (xu, yu) --> (xd, yd). You then replace each pixel in the undistorted image with its corresponding pixel from the distorted image.
Starting where JMBR did, I do the reverse, finding Rd as a function of Ru. I get:
Rd = f * sqrt(2) * sqrt( 1 - 1/sqrt(r^2 +1))
where f is the focal length in pixels (I'll explain later), and r = Ru/f.
The focal length for my camera was 2.5 mm. The size of each pixel on my CCD was 6 um square. f was therefore 2500/6 = 417 pixels. This can be found by trial and error.
Finding Rd allows you to find the corresponding pixel in the distorted image using polar coordinates.
The angle of each pixel from the center point is the same:
theta = arctan( (yu-yc)/(xu-xc) ) where xc, yc are the center points.
Then,
xd = Rd * cos(theta) + xc
yd = Rd * sin(theta) + yc
Make sure you know which quadrant you are in.
Here is the C# code I used
public class Analyzer
{
private ArrayList mFisheyeCorrect;
private int mFELimit = 1500;
private double mScaleFESize = 0.9;
public Analyzer()
{
//A lookup table so we don't have to calculate Rdistorted over and over
//The values will be multiplied by focal length in pixels to
//get the Rdistorted
mFisheyeCorrect = new ArrayList(mFELimit);
//i corresponds to Rundist/focalLengthInPixels * 1000 (to get integers)
for (int i = 0; i < mFELimit; i++)
{
double result = Math.Sqrt(1 - 1 / Math.Sqrt(1.0 + (double)i * i / 1000000.0)) * 1.4142136;
mFisheyeCorrect.Add(result);
}
}
public Bitmap RemoveFisheye(ref Bitmap aImage, double aFocalLinPixels)
{
Bitmap correctedImage = new Bitmap(aImage.Width, aImage.Height);
//The center points of the image
double xc = aImage.Width / 2.0;
double yc = aImage.Height / 2.0;
Boolean xpos, ypos;
//Move through the pixels in the corrected image;
//set to corresponding pixels in distorted image
for (int i = 0; i < correctedImage.Width; i++)
{
for (int j = 0; j < correctedImage.Height; j++)
{
//which quadrant are we in?
xpos = i > xc;
ypos = j > yc;
//Find the distance from the center
double xdif = i-xc;
double ydif = j-yc;
//The distance squared
double Rusquare = xdif * xdif + ydif * ydif;
//the angle from the center
double theta = Math.Atan2(ydif, xdif);
//find index for lookup table
int index = (int)(Math.Sqrt(Rusquare) / aFocalLinPixels * 1000);
if (index >= mFELimit) index = mFELimit - 1;
//calculated Rdistorted
double Rd = aFocalLinPixels * (double)mFisheyeCorrect[index]
/mScaleFESize;
//calculate x and y distances
double xdelta = Math.Abs(Rd*Math.Cos(theta));
double ydelta = Math.Abs(Rd * Math.Sin(theta));
//convert to pixel coordinates
int xd = (int)(xc + (xpos ? xdelta : -xdelta));
int yd = (int)(yc + (ypos ? ydelta : -ydelta));
xd = Math.Max(0, Math.Min(xd, aImage.Width-1));
yd = Math.Max(0, Math.Min(yd, aImage.Height-1));
//set the corrected pixel value from the distorted image
correctedImage.SetPixel(i, j, aImage.GetPixel(xd, yd));
}
}
return correctedImage;
}
}
I found this pdf file and I have proved that the maths are correct (except for the line vd = *xd**fv+v0 which should say vd = **yd**+fv+v0).
http://perception.inrialpes.fr/CAVA_Dataset/Site/files/Calibration_OpenCV.pdf
It does not use all of the latest co-efficients that OpenCV has available but I am sure that it could be adapted fairly easily.
double k1 = cameraIntrinsic.distortion[0];
double k2 = cameraIntrinsic.distortion[1];
double p1 = cameraIntrinsic.distortion[2];
double p2 = cameraIntrinsic.distortion[3];
double k3 = cameraIntrinsic.distortion[4];
double fu = cameraIntrinsic.focalLength[0];
double fv = cameraIntrinsic.focalLength[1];
double u0 = cameraIntrinsic.principalPoint[0];
double v0 = cameraIntrinsic.principalPoint[1];
double u, v;
u = thisPoint->x; // the undistorted point
v = thisPoint->y;
double x = ( u - u0 )/fu;
double y = ( v - v0 )/fv;
double r2 = (x*x) + (y*y);
double r4 = r2*r2;
double cDist = 1 + (k1*r2) + (k2*r4);
double xr = x*cDist;
double yr = y*cDist;
double a1 = 2*x*y;
double a2 = r2 + (2*(x*x));
double a3 = r2 + (2*(y*y));
double dx = (a1*p1) + (a2*p2);
double dy = (a3*p1) + (a1*p2);
double xd = xr + dx;
double yd = yr + dy;
double ud = (xd*fu) + u0;
double vd = (yd*fv) + v0;
thisPoint->x = ud; // the distorted point
thisPoint->y = vd;
This can be solved as an optimization problem. Simply draw on curves in images that are supposed to be straight lines. Store the contour points for each of those curves. Now we can solve the fish eye matrix as a minimization problem. Minimize the curve in points and that will give us a fisheye matrix. It works.
It can be done manually by adjusting the fish eye matrix using trackbars! Here is a fish eye GUI code using OpenCV for manual calibration.
I am working on a Unity3D project which relies on a 3D texture momentarily.
The problem is, Unity only allows Pro users to make use of Texture3D. Hence I'm looking for an alternative to Texture3D, perhaps a one dimensional texture (although not natively available in Unity) that is interpreted as 3 dimensional in the shader (which uses the 3D texture).
Is there a way to do this whilst (preferably) keeping subpixel information?
(GLSL and Cg tags added because here lies the core of the problem)
Edit: The problem is addressed here as well: webgl glsl emulate texture3d
However this is not yet finished and working properly.
Edit: For the time being I disregard proper subpixel information. So any help on converting a 2D texture to contain 3D information is appreciated!
Edit: I retracted my own answer as it isn't sufficient as of yet:
float2 uvFromUvw( float3 uvw ) {
float2 uv = float2(uvw.x, uvw.y / _VolumeTextureSize.z);
uv.y += float(round(uvw.z * (_VolumeTextureSize.z - 1))) / _VolumeTextureSize.z;
return uv;
}
With initialization as Texture2D(volumeWidth, volumeHeight * volumeDepth).
Most of the time it works, but sometimes it shows wrong pixels, probably because of subpixel information it is picking up on. How can I fix this? Clamping the input doesn't work.
I'm using this for my 3D clouds if that helps:
float SampleNoiseTexture( float3 _UVW, float _MipLevel )
{
float2 WrappedUW = fmod( 16.0 * (1000.0 + _UVW.xz), 16.0 ); // UW wrapped in [0,16[
float IntW = floor( WrappedUW.y ); // Integer slice number
float dw = WrappedUW.y - IntW; // Remainder for intepolating between slices
_UVW.x = (17.0 * IntW + WrappedUW.x + 0.25) * 0.00367647058823529411764705882353; // divided by 17*16 = 272
float4 Value = tex2D( _TexNoise3D, float4( _UVW.xy, 0.0, 0.0 ) );
return lerp( Value.x, Value.y, dw );
}
The "3D texture" is packed as 16 slices of 17 pixels wide in a 272x16 texture, with the 17th column of each slice being a copy of the 1st column (wrap address mode)...
Of course, no mip-mapping allowed with this technique.
Here's the code I'm using to create the 3D texture if that's what bothering you:
static const NOISE3D_TEXTURE_POT = 4;
static const NOISE3D_TEXTURE_SIZE = 1 << NOISE3D_TEXTURE_POT;
// <summary>
// Create the "3D noise" texture
// To simulate 3D textures that are not available in Unity, I create a single long 2D slice of (17*16) x 16
// The width is 17*16 so that all 3D slices are packed into a single line, and I use 17 as a single slice width
// because I pad the last pixel with the first column of the same slice so bilinear interpolation is correct.
// The texture contains 2 significant values in Red and Green :
// Red is the noise value in the current W slice
// Green is the noise value in the next W slice
// Then, the actual 3D noise value is an interpolation of red and green based on the W remainder
// </summary>
protected NuajTexture2D Build3DNoise()
{
// Build first noise mip level
float[,,] NoiseValues = new float[NOISE3D_TEXTURE_SIZE,NOISE3D_TEXTURE_SIZE,NOISE3D_TEXTURE_SIZE];
for ( int W=0; W < NOISE3D_TEXTURE_SIZE; W++ )
for ( int V=0; V < NOISE3D_TEXTURE_SIZE; V++ )
for ( int U=0; U < NOISE3D_TEXTURE_SIZE; U++ )
NoiseValues[U,V,W] = (float) SimpleRNG.GetUniform();
// Build actual texture
int MipLevel = 0; // In my original code, I build several textures for several mips...
int MipSize = NOISE3D_TEXTURE_SIZE >> MipLevel;
int Width = MipSize*(MipSize+1); // Pad with an additional column
Color[] Content = new Color[MipSize*Width];
// Build content
for ( int W=0; W < MipSize; W++ )
{
int Offset = W * (MipSize+1); // W Slice offset
for ( int V=0; V < MipSize; V++ )
{
for ( int U=0; U <= MipSize; U++ )
{
Content[Offset+Width*V+U].r = NoiseValues[U & (MipSize-1),V,W];
Content[Offset+Width*V+U].g = NoiseValues[U & (MipSize-1),V,(W+1) & (MipSize-1)];
}
}
}
// Create texture
NuajTexture2D Result = Help.CreateTexture( "Noise3D", Width, MipSize, TextureFormat.ARGB32, false, FilterMode.Bilinear, TextureWrapMode.Repeat );
Result.SetPixels( Content, 0 );
Result.Apply( false, true );
return Result;
}
I followed Patapoms response and came to the following. However it's still off as it should be.
float getAlpha(float3 position)
{
float2 WrappedUW = fmod( _Volume.xz * (1000.0 + position.xz), _Volume.xz ); // UW wrapped in [0,16[
float IntW = floor( WrappedUW.y ); // Integer slice number
float dw = WrappedUW.y - IntW; // Remainder for intepolating between slices
position.x = ((_Volume.z + 1.0) * IntW + WrappedUW.x + 0.25) / ((_Volume.z + 1.0) * _Volume.x); // divided by 17*16 = 272
float4 Value = tex2Dlod( _VolumeTex, float4( position.xy, 0.0, 0.0 ) );
return lerp( Value.x, Value.y, dw );
}
public int GetPixelId(int x, int y, int z) {
return y * (volumeWidth + 1) * volumeDepth + z * (volumeWidth + 1) + x;
}
// Code to set the pixelbuffer one pixel at a time starting from a clean slate
pixelBuffer[GetPixelId(x, y, z)].r = color.r;
if (z > 0)
pixelBuffer[GetPixelId(x, y, z - 1)].g = color.r;
if (z == volumeDepth - 1 || z == 0)
pixelBuffer[GetPixelId(x, y, z)].g = color.r;
if (x == 0) {
pixelBuffer[GetPixelId(volumeWidth, y, z)].r = color.r;
if (z > 0)
pixelBuffer[GetPixelId(volumeWidth, y, z - 1)].g = color.r;
if (z == volumeDepth - 1 || z == 0)
pixelBuffer[GetPixelId(volumeWidth, y, z)].g = color.r;
}