I am trying to build a custom shader on the face in metal. As a result, I need to manually apply scene lighting from the spherical harmonics provided by ARKit.
I have implemented a function based on 3.2 (equation 13) from https://cseweb.ucsd.edu/~ravir/papers/envmap/envmap.pdf. I have the following metal function.
inline float estimateLightWithSHCoefficients(const float3 surfaceNormal,
const float L00,
const float L1n1, const float L10, const float L11,
const float L2n2, const float L2n1, const float L20, const float L21, const float L22) {
return (c1*L22*(pow(surfaceNormal.x, 2) - pow(surfaceNormal.y, 2)) + c3*L20*pow(surfaceNormal.z, 2) + c4*L00 - c5*L20
+ 2*c1*(L2n2*surfaceNormal.x*surfaceNormal.y + L21*surfaceNormal.x*surfaceNormal.z + L2n1*surfaceNormal.y*surfaceNormal.z)
+ 2*c2*(L11*surfaceNormal.x + L1n1*surfaceNormal.y + L10*surfaceNormal.z));
}
The constants are said as described in the paper. The coefficients (note that L2n1 corresponds to L2-2 in 3.2 - using n to indicate negative in the naming convention). The coefficients are set from the spherical harmonics output converted to floats (and then first 9 values are used as coefficients for red, next 9 for green, and last 9 for blue - I am not sure if this is the current order, and the inputs are given to the function in the same order as specified in the definition - would really appreciate any clarification if this is correct).
The vertex normal is obtained from ARSCNFaceGeometry's normal source but normalized. I am not sure if this is the correct way to do this.
I have been running some tests and I get negative light values in some positions (which should be clearly a bug). I would appreciate any advice as to why. I believe for this method to work both normal and spherical harmonics needs to be on the same coordinate system with the spherical harmonics centred at origin. I suspect that there might be an issue in where/how I am getting the normals/coefficients but that's just a guess. What's wrong with this approach?
If you have any pointers of existing implementations for calculating light using second-order spherical harmonics, I would highly appreciate it!
Thanks!
Related
For a masking object, I am trying to scale each triangle individually. If I scale the object as a whole, the points further away from the center will get moved too far and I just want the object to have 'more body'. Since I use it as a mask, it doesn't matter if the triangles end up overlapping.
Although looking at this might hurt someone deep inside, this is actually what I'm trying to achieve:
I thought this was best done in a shader and I thought this could be achieved in the geometry shader since I need to know the center of the triangle. I came up with the code below, but things keep acting... strange.
float3 center = (IN[0].vertex.xyz + IN[1].vertex.xyz + IN[2].vertex.xyz) / 3;
for (int i = 0; i < 3; i++)
{
float3 distance = IN[i].vertex.xyz - center.xyz;
float3 normal = normalize(distance);
distance = abs(distance);
float scale = 1;
float3 pos = IN[i].vertex.xyz + (distance * normal.xyz * (scale - 1));
o.pos.xyz = pos.xyz;
o.pos.w = IN[i].vertex.w;
tristream.Append(o);
}
My plan was to calculate the center of the triangle and than calculate the distance between the center and each point. I would than take the normal of this distance to know in which direction I would have to move the vertex and change the position by adding the distance * normal(direction) * scale to the original position of the vertex. Yet, it seems the triangles change when you rotate the camera, so I would doubt it if this is right. Does anyone know what could be wrong?
(Just some notes:
the mesh is basically 2D, only changing across the x- and z-axis (if this matters).
I did abs(distance) since I thought it would cancel out the normal if both would be negative. I'm not sure if this is necessary.
I did scale -1 since a scale of 1 would result in the mesh staying the same. A scale of 2 should result in all triangles being twice as big.
I have no clue on what to do with the w value, but keeping the old value at least doesn't screw up that much. Perhaps here lays the problem? I thought this value should always be 1 for matrix multiplications.
)
Oke, so besides using a way to 'complex' formula to calculate the new position of each point. (Better way at https://math.stackexchange.com/questions/1563249/how-do-i-scale-a-triangle-given-its-cartesian-cooordinates). I found out that it somehow indeed had to do with the w-value. As I always thought this was mainly a helper variable, it would be awesome if someone could explain how that values screwed things over.
Anyways, including that value in the equation it works fine.
float4 center = (IN[0].vertex.xyzw + IN[1].vertex.xyzw + IN[2].vertex.xyzw) / 3;
for (int i = 0; i < 3; i++)
{
float scale = 2;
float4 pos = (IN[i].vertex.xyzw * scale) - center.xyzw;
o.pos.xyzw = pos.xyzw;
tristream.Append(o);
}
This works just fine :)
I am trying to determine a players depth position on a plane, which defines the walkable ground in a 2D brawler game. The problem is depictured in the following drawing:
C represents the players current position. I need to find the magnitude of vector V. Since I am not strong on linear algebra, the one thing I can think of is: determining the intersection point P of L1 and L2, and then take the magnitude from AP. However, I get the feeling there must be an easier way to find V, since I already know the angle the vector should have, given by vector from AB.
Any input would be appreciated, since I am looking forward to step up my linear algebra game.
Edit: As it is unclear thanks to my lack of drawing skills: the geometry depicted above is a parallelogram. The vector V I am looking for is parallel to the left and right side of the parallelogram. Depth does not mean, that I am looking for the vector perpendicular to the top side, but it refers to the fake depth of a purely 2D game. The parallelogram is therefore used as a means for creating the feeling of walking along a z axis.
The depth of your player (length of V) as measured from the top line in your drawing, is just the difference between A.y and C.y. This is seperate from the slant in the parralelogram, as we're just looking at depth.
example:
float v;
Vector2 a = new Vector2(100, 100); //The point you're measuring from
Vector2 c = new Vector2(150, 150); //Your character position
v = c.y - a.y; // This is the length of V.
//In numbers: 50 = 150 - 100
Illustrated: image not to scale
This works for any coördinate in your plane.
Now if you'd want to get the length of AC is when you'd need to apply some pythagoras, which is a² + b² = c². In the example that would mean in code:
Vector2 a = new Vector2(100, 100);
Vector2 c = new Vector2(150, 150);
float ac1 = Mathf.Sqrt(Mathf.Pow(c.x - a.x, 2) + Mathf.Pow(c.y - a.y, 2));
Now that is quite a sore to have to type out every time, and looks quite scary. But Unity has you covered! There is a Vector method called Distance
float ac2 = Vector2.Distance(a, c);
Which both return 70.71068 which is the length of AC.
This works because for any point c in your area you can draw a right angled triangle from a to c.
Edit as per comment:
If you want your "depth" vector to be parallel with the sides of the paralellogram we can just create a triangle in the parallelogram of which we calculate the hypotenuse.
Since we want the new hypotenuse of our triangle to be parallel to the parallelogram we can use the same angle θ as point B has in your drawing (indicated by pink in mine), of which I understood you know the value.
We also know the length of the adjacent (indicated in blue) side of this new triangle, as that is the height we calculated earlier (c.y - a.y).
Using these two values we can use cosine to find the length of hypotenuse (indicated in red) of the triangle, which is equal to the vector V, in parallel with the parallelogram.
the formula for that is: hypotenuse = adjacent/cos(θ)
Now if we were to put some numbers in this, and for my example I took 55 for the angle θ. It would look like this
float v = 50/(cos(55));
image not to scale
Let's call the lower right vertex of the parallelogram D.
If the long sides of the parallelogram are horizontal, you can find magnitude of V vector by:
V.magnitude = (c.y - a.y) / sin(BAD)
Or if you prefer:
V.magnitude = AB.magnitude * (c.y - a.y)/(b.y - a.y)
I have a list of X,Y coordinates that represents a road. For every 5 meters, I need to calculate the angle of the tangent on this road, as I have tried to illustrate in the image.
My problem is that this road is not represented by a mathematical function that I can simply derive, it is represented by a list of coordinates (UTM33N).
In my other similar projects we use ArcGIS/ESRI libraries to perform geographical functions such as this, but in this project I need to be independent of any software that require the end user to have a license, so I need to do the calculations myself (or find a free/open source library that can do it).
I am using a cubic spline function to make the line rounded between the coordinates, since all tangents on a line segment would just be parallell to the segment otherwise.
But now I am stuck. I am considering simply calculating the angle between any three points on the line (given enough points), and using this to find the tangents, but that doesn't sound like a good method. Any suggestions?
In the end, I concluded that the points were plentiful enough to give an accurate angle using simple geometry:
//Calculate delta values
var dx = next.X - curr.X;
var dy = next.Y - curr.Y;
var dz = next.Z - curr.Z;
//Calculate horizontal and 3D length of this segment.
var hLength = Math.Sqrt(dx * dx + dy * dy);
var length = Math.Sqrt(hLength * hLength + dz * dz);
//Calculate horizontal and vertical angles.
hAngle = Math.Atan(dy/dx);
vAngle = Math.Atan(dz/hLength);
After researching for hours and hours I am just not able to linearly interpolate two animation curve,I need something like Mathf.lerp for animation curve,I had try with inTangent and OutTangent with keyframes of the animation curve but it does not store the values,am I missing something ?
Need guidance,this would help me a lot in my project,thanks.
Seems to me you are looking to "morph" one animation curve into another. This imho depends on your use case.
Assuming both curves are defined between [0.0, 1.0], I'd calculate the curve value for t in both, and interpolate linearly both results over the same time.
/*
* Evaluates two AnimationCurve objects, and interpolates linearly
* between the results, effectively "morphing" from curve A to curve
* B in a linear fashion.
* Both curves must be defined between 0.0 and 1.0, and t must be
* within the same 0.0-1.0 range too.
*/
public float EvaluateLerpTwoCurves(AnimationCurve a, AnimationCurve b, float t) {
float valueA = a.Evaluate (t);
float valueB = b.Evaluate (t);
float result = Mathf.Lerp(valueA, valueB, t);
return result;
}
You may wish to interpolate between both AnimationCurve using a different approach (not linearly like Mathf.Lerp does), you could even use a third AnimationCurve to define how to morph between both curves.
Note that the suggested code above requires both curves to be defined between 0.0 and 1.0, and t must be within the same 0.0-1.0 range too.
I am developing an application for the iPhone using opencv. I have to use the method solvePnPRansac:
http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html
For this method I need to provide a camera matrix:
__ __
| fx 0 cx |
| 0 fy cy |
|_0 0 1 _|
where cx and cy represent the center pixel positions of the image and fx and fy represent focal lengths, but that is all the documentation says. I am unsure what to provide for these focal lengths. The iPhone 5 has a focal length of 4.1 mm, but I do not think that this value is usable as is.
I checked another website:
http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
which shows how opencv creates camera matrices. Here it states that focal lengths are measured in pixel units.
I checked another website:
http://www.velocityreviews.com/forums/t500283-focal-length-in-pixels.html
(about half way down)
it says that focal length can be converted from units of millimeters to pixels using the equation: fx = fy = focalMM * pixelDensity / 25.4;
Another Link I found states that fx = focalMM * width / (sensorSizeMM);
fy = focalMM * length / (sensorSizeMM);
I am unsure about these equations and how to properly create this matrix.
Any help, advice, or links on how to create an accurate camera matrix (especially for the iPhone 5) would be greatly appreciated,
Isaac
p.s. I think that (fx/fy) or (fy/fx) might be equal to the aspect ratio of the camera, but that might be completely wrong.
UPDATE:
Pixel coordinates to 3D line (opencv)
using this link, I can figure out how they want fx and fy to be formatted because they use it to scale angles relative to their distance from the center. therefore, fx and fy are likely in pixels/(unit length) but im still not sure what this unit length needs to be, can it be arbitrary as long as x and y are scaled to each other?
You can get an initial (rough) estimate of the focal length in pixel dividing the focal length in mm by the width of a pixel of the camera' sensor (CCD, CMOS, whatever).
You get the former from the camera manual, or read it from the EXIF header of an image taken at full resolution. Finding out the latter is a little more complicated: you may look up on the interwebs the sensor's spec sheet, if you know its manufacturer and model number, or you may just divide the overall width of its sensitive area by the number of pixels on the side.
Absent other information, it's usually safe to assume that the pixels are square (i.e. fx == fy), and that the sensor is orthogonal to the lens's focal axis (i.e. that the term in the first row and second column of the camera matrix is zero). Also, the pixel coordinates of the principal point (cx, cy) are usually hard to estimate accurately without a carefully designed calibration rig, and an as-carefully executed calibration procedure (that's because they are intrinsically confused with the camera translation parallel to the image plane). So it's best to just set them equal to the geometrical geometrical center of the image, unless you know that the image has been cropped asymmetrically.
Therefore, your simplest camera model has only one unknown parameter, the focal length f = fx = fy.
Word of advice: in your application is usually more convenient to carry around the horizontal (or vertical) field-of-view angle, rather than the focal length in pixels. This is because the FOV is invariant to image scaling.
The "focal length" you are dealing with here is simply a scaling factor from objects in the world to camera pixels, used in the pinhole camera model (Wikipedia link). That's why its units are pixels/unit length. For a given f, an object of size L at a distance (perpendicular to the camera) z, would be f*L/z pixels.
So, you could estimate the focal length by placing an object of known size at a known distance of your camera and measuring its size in the image. You could aso assume the central point is the center of the image. You should definitely not ignore the lens distortion (dist_coef parameter in solvePnPRansac).
In practice, the best way to obtain the camera matrix and distortion coefficients is to use a camera calibration tool. You can download and use the MRPT camera_calib software from this link, there's also a video tutorial here. If you use matlab, go for the Camera Calibration Toolbox.
Here you have a table with the spec of the cameras for iPhone 4 and 5.
The calculation is:
double f = 4.1;
double resX = (double)(sourceImage.cols);
double resY = (double)(sourceImage.rows);
double sensorSizeX = 4.89;
double sensorSizeY = 3.67;
double fx = f * resX / sensorSizeX;
double fy = f * resY / sensorSizeY;
double cx = resX/2.;
double cy = resY/2.;
Try this:
func getCamMatrix()->(Float, Float, Float, Float)
{
let format:AVCaptureDeviceFormat? = deviceInput?.device.activeFormat
let fDesc:CMFormatDescriptionRef = format!.formatDescription
let dim:CGSize = CMVideoFormatDescriptionGetPresentationDimensions(fDesc, true, true)
// dim = dimensioni immagine finale
let cx:Float = Float(dim.width) / 2.0;
let cy:Float = Float(dim.height) / 2.0;
let HFOV : Float = format!.videoFieldOfView
let VFOV : Float = ((HFOV)/cx)*cy
let fx:Float = abs(Float(dim.width) / (2 * tan(HFOV / 180 * Float(M_PI) / 2)));
let fy:Float = abs(Float(dim.height) / (2 * tan(VFOV / 180 * Float(M_PI) / 2)));
return (fx, fy, cx, cy)
}
Old thread, present problem.
As Milo and Isaac mentioned after Milo's answer, there seems to be no "common" params available for, say, the iPhone 5.
For what it is worth, here is the result of a run with the MRPT calibration tool, with a good old iPhone 5:
[CAMERA_PARAMS]
resolution=[3264 2448]
cx=1668.87585
cy=1226.19712
fx=3288.47697
fy=3078.59787
dist=[-7.416752e-02 1.562157e+00 1.236471e-03 1.237955e-03 -5.378571e+00]
Average err. of reprojection: 1.06726 pixels (OpenCV error=1.06726)
Note that dist means distortion here.
I am conducting experiments on a toy project, with these parameters---kind of ok. If you do use them on your own project, please keep in mind that they may be hardly good enough to get started. The best will be to follow Milo's recommendation with your own data. The MRPT tool is quite easy to use, with the checkerboard they provide. Hope this does help getting started !