I would like to convert some Vector3 points into 2d, projecting them onto a plane just like a camera would, with perspective.
I am starting with 3d points, a camera position and look direction, and fov.
My attempt:
void Update()
{
Matrix4x4 proj = Matrix4x4.LookAt(FakeCam.transform.position, Vector3.zero, FakeCam.transform.up) *
Matrix4x4.Perspective(fov, aspect, zNear, zFar);
for (int i=0; i<8; i++)
{
var point = proj.MultiplyPoint(Points[i]);
Dots.SetPosition(i, new Vector3(point.x, 0, point.z));
}
Dots.Draw(transform.localToWorldMatrix);
}
The LookAt matrix seems to work as expected here. If I only use the LookAt matrix, I get my object shown (with an orthographic projection). If I add the perspective matrix, it is twisted in the middle (the back face of the cube is rotated 180 ??)
Related
I want to display a 2D arrow on the screen that always moves to point to a 3D object in the AR scene.
The issue is how to measure the angle that should the arrow rotate to point the desired 3D object.
Thanks in advance.
One strategy is to project the position of your object into screen space.
Then calculate the vector between the position of your arrow and that projected position. You can use this vector to calculate an angle of rotation from, for example, a vertical direction, using the Dot Product, followed by an acos.
Finally, you'd need to do a little cross-product calculation to decide whether the above rotation is clockwise or anticlockwise.
Here is some sample code:
public GameObject Target;
RectTransform rt;
void Start()
{
rt = GetComponent<RectTransform>();
}
void Update()
{
// Get the position of the object in screen space
Vector3 objScreenPos = Camera.main.WorldToScreenPoint(Target.transform.position);
// Get the directional vector between your arrow and the object
Vector3 dir = (objScreenPos - rt.position).normalized;
// Calculate the angle
// We assume the default arrow position at 0° is "up"
float angle = Mathf.Rad2Deg * Mathf.Acos(Vector3.Dot(dir, Vector3.up));
// Use the cross product to determine if the angle is clockwise
// or anticlockwise
Vector3 cross = Vector3.Cross(dir, Vector3.up);
angle = -Mathf.Sign(cross.z) * angle;
// Update the rotation of your arrow
rt.localEulerAngles = new Vector3(rt.localEulerAngles.x, rt.localEulerAngles.y, angle);
}
For the above code, I suppose that:
You are only using one main camera, you may need to change this
Your arrow is on the Canvas, by default pointing upwards (when its rotation is (0, 0, 0))
You are using a Canvas in Render Mode: Screen Space - Overlay. The above code would be different if the Canvas were in World Space.
As a high-level overview:
Find direction from UI/view-plane centre to 3D object
Project direction onto UI/view-plane (using forward as normal vector), and normalize
Point 2D arrow toward projected direction
Thank You all Guys, I got an Answer for two Situations :
First One: When the two objects are in 3D
public GameObject flesh;
public GameObject target;
// Start is called before the first frame update
void Start()
{
flesh.transform.position = Camera.main.ScreenToWorldPoint(new Vector3( Screen.width/2, Screen.height/2,1));
}
// Update is called once per frame
void Update()
{
var dir = Camera.main.WorldToScreenPoint(target.transform.position) -
Camera.main.WorldToScreenPoint(flesh.transform.position);
var angle = Mathf.Atan2(dir.y, dir.x) * Mathf.Rad2Deg;
flesh.transform.rotation = Quaternion.AngleAxis(angle, Vector3.forward);
}
Second: When the Flesh is an Image that Has RectTransform,
, this solution Inspirational from #kevernicus
public GameObject Target;
RectTransform rt;
void Start()
{
rt = GetComponent<RectTransform>();
}
void Update()
{
// Get the position of the object in screen space
Vector3 objScreenPos = Camera.main.WorldToScreenPoint(Target.transform.position);
// Get the directional vector between your arrow and the object
Vector3 dir = (objScreenPos - rt.position).normalized;
float angle = Mathf.Rad2Deg * Mathf.Atan2(dir.y, dir.x);
rt.rotation = Quaternion.AngleAxis(angle, Vector3.forward);
}
Use Unity's built in Transform.LookAt function
I have an image UI in a canvas with Screen Space - Camera render mode. What I like to do is move my LineRenderer to the image vertical position by looping through all the LineRenderer positions and changing its y axis. My problem is I cant get the correct position of the image that the LineRenderer can understand. I've tried using ViewportToWorldPoint and ScreenToWorldPoint but its not the same position.
Vector3 val = Camera.main.ViewportToWorldPoint(new Vector3(image.transform.position.x, image.transform.position.y, Camera.main.nearClipPlane));
for (int i = 0; i < newListOfPoints.Count; i++)
{
line.SetPosition(i, new Vector3(newListOfPoints[i].x, val.y, newListOfPoints[i].z));
}
Screenshot result using Vector3 val = Camera.main.ScreenToWorldPoint(new Vector3(image.transform.localPosition.x, image.transform.localPosition.y, -10));
The green LineRenderer is the result of changing the y position. It should be at the bottom of the square image.
Wow, this was annoying and complicated.
Here's the code I ended up with. The code in your question is the bottom half of the Update() function. The only thing I changed is what was passed into the ScreenToWorldPoint() method. That value is calculated in the upper half of the Update() function.
The RectTransformToScreenSpace() function was adapted from this Unity Answer post1 about getting the screen space coordinates of a RectTransform (which is exactly what we want in order to convert from screen space coordinates back into world space!) The only difference is that I was getting inverse Y values, so I changed from Screen.height - transform.position.y to just transform.position.y which did the trick perfectly.
After that it was just a matter of grabbing that rectangle's lower left corner, making it a Vector3 instead of a Vector2, and passing it back into ScreenToWorldPoint(). The only trick there was because of the perspective camera, I needed to know how far away the line was from the camera originally in order to maintain that same distance (otherwise the line moves up and down the screen faster than the image). For an orthographic camera, this value can be anything.
void Update () {
//the new bits:
float dist = (Camera.main.transform.position - newListOfPoints[0]).magnitude;
Rect r = RectTransformToScreenSpace((RectTransform)image.transform);
Vector3 v3 = new Vector3(r.xMin, r.yMin, dist);
//more or less original code:
Vector3 val = Camera.main.ScreenToWorldPoint(v3);
for(int i = 0; i < newListOfPoints.Count; i++) {
line.SetPosition(i, new Vector3(newListOfPoints[i].x, val.y, newListOfPoints[i].z));
}
}
//helper function:
public static Rect RectTransformToScreenSpace(RectTransform transform) {
Vector2 size = Vector2.Scale(transform.rect.size, transform.lossyScale);
Rect rect = new Rect(transform.position.x, transform.position.y, size.x, size.y);
rect.x -= (transform.pivot.x * size.x);
rect.y -= ((1.0f - transform.pivot.y) * size.y);
return rect;
}
1And finding that post from a generalized search on "how do I get the screen coordinates of a UI object" was not easy. A bunch of other posts came up and had some code, but none of it did what I wanted (including converting screen space coordinates back into world space coordinates of the UI object which was stupid easy and not reversibe, thanks RectTransformUtility!)
In Unity, say you have a 3D object,
Of course, it's trivial to get the AABB, Unity has direct functions for that,
(You might have to "add up all the bounding boxes of the renderers" in the usual way, no issue.)
So Unity does indeed have a direct function to give you the 3D AABB box instantly, out of the internal mesh/render pipeline every frame.
Now, for the Camera in question, as positioned, that AABB indeed covers a certain 2D bounding box ...
In fact ... is there some sort of built-in direct way to find that orange 2D box in Unity??
Question - does Unity have a function which immediately gives that 2D frustrum box from the pipeline?
(Note that to do it manually you just make rays (or use world to screen space as Draco mentions, same) for the 8 points of the AABB; encapsulate those in 2D to make the orange box.)
I don't need a manual solution, I'm asking if the engine gives this somehow from the pipeline every frame?
Is there a call?
(Indeed, it would be even better to have this ...)
My feeling is that one or all of the
occlusion system in particular
the shaders
the renderer
would surely know the orange box, and perhaps even the blue box inside the pipeline, right off the graphics card, just as it knows the AABB for a given mesh.
We know that Unity lets you tap the AABB 3D box instantly every frame for a given mesh: In fact does Unity give the "2D frustrum bound" as shown here?
As far as I am aware, there is no built in for this.
However, finding the extremes yourself is really pretty easy. Getting the mesh's bounding box (the cuboid shown in the screenshot) is just how this is done, you're just doing it in a transformed space.
Loop through all the verticies of the mesh, doing the following:
Transform the point from local to world space (this handles dealing with scale and rotation)
Transform the point from world space to screen space
Determine if the new point's X and Y are above/below the stored min/max values, if so, update the stored min/max with the new value
After looping over all vertices, you'll have 4 values: min-X, min-Y, max-X, and max-Y. Now you can construct your bounding rectangle
You may also wish to first perform a Gift Wrapping of the model first, and only deal with the resulting convex hull (as no points not part of the convex hull will ever be outside the bounds of the convex hull). If you intend to draw this screen space rectangle while the model moves, scales, or rotates on screen, and have to recompute the bounding box, then you'll want to do this and cache the result.
Note that this does not work if the model animates (e.g. if your humanoid stands up and does jumping jacks). Solving for the animated case is much more difficult, as you would have to treat every frame of every animation as part of the original mesh for the purposes of the convex hull solving (to insure that none of your animations ever move a part of the mesh outside the convex hull), increasing the complexity by a power.
3D bounding box
Get given GameObject 3D bounding box's center and size
Compute 8 corners
Transform positions to GUI space (screen space)
Function GUI3dRectWithObject will return the 3D bounding box of given GameObject on screen.
2D bounding box
Iterate through every vertex in a given GameObject
Transform every vertex's position to world space, and transform to GUI space (screen space)
Find 4 corner value: x1, x2, y1, y2
Function GUI2dRectWithObject will return the 2D bounding box of given GameObject on screen.
Code
public static Rect GUI3dRectWithObject(GameObject go)
{
Vector3 cen = go.GetComponent<Renderer>().bounds.center;
Vector3 ext = go.GetComponent<Renderer>().bounds.extents;
Vector2[] extentPoints = new Vector2[8]
{
WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y-ext.y, cen.z-ext.z)),
WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y-ext.y, cen.z-ext.z)),
WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y-ext.y, cen.z+ext.z)),
WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y-ext.y, cen.z+ext.z)),
WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y+ext.y, cen.z-ext.z)),
WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y+ext.y, cen.z-ext.z)),
WorldToGUIPoint(new Vector3(cen.x-ext.x, cen.y+ext.y, cen.z+ext.z)),
WorldToGUIPoint(new Vector3(cen.x+ext.x, cen.y+ext.y, cen.z+ext.z))
};
Vector2 min = extentPoints[0];
Vector2 max = extentPoints[0];
foreach (Vector2 v in extentPoints)
{
min = Vector2.Min(min, v);
max = Vector2.Max(max, v);
}
return new Rect(min.x, min.y, max.x - min.x, max.y - min.y);
}
public static Rect GUI2dRectWithObject(GameObject go)
{
Vector3[] vertices = go.GetComponent<MeshFilter>().mesh.vertices;
float x1 = float.MaxValue, y1 = float.MaxValue, x2 = 0.0f, y2 = 0.0f;
foreach (Vector3 vert in vertices)
{
Vector2 tmp = WorldToGUIPoint(go.transform.TransformPoint(vert));
if (tmp.x < x1) x1 = tmp.x;
if (tmp.x > x2) x2 = tmp.x;
if (tmp.y < y1) y1 = tmp.y;
if (tmp.y > y2) y2 = tmp.y;
}
Rect bbox = new Rect(x1, y1, x2 - x1, y2 - y1);
Debug.Log(bbox);
return bbox;
}
public static Vector2 WorldToGUIPoint(Vector3 world)
{
Vector2 screenPoint = Camera.main.WorldToScreenPoint(world);
screenPoint.y = (float)Screen.height - screenPoint.y;
return screenPoint;
}
Reference: Is there an easy way to get on-screen render size (bounds)?
refer to this
It needs the game object with skinnedMeshRenderer.
Camera camera = GetComponent();
SkinnedMeshRenderer skinnedMeshRenderer = target.GetComponent();
// Get the real time vertices
Mesh mesh = new Mesh();
skinnedMeshRenderer.BakeMesh(mesh);
Vector3[] vertices = mesh.vertices;
for (int i = 0; i < vertices.Length; i++)
{
// World space
vertices[i] = target.transform.TransformPoint(vertices[i]);
// GUI space
vertices[i] = camera.WorldToScreenPoint(vertices[i]);
vertices[i].y = Screen.height - vertices[i].y;
}
Vector3 min = vertices[0];
Vector3 max = vertices[0];
for (int i = 1; i < vertices.Length; i++)
{
min = Vector3.Min(min, vertices[i]);
max = Vector3.Max(max, vertices[i]);
}
Destroy(mesh);
// Construct a rect of the min and max positions
Rect r = Rect.MinMaxRect(min.x, min.y, max.x, max.y);
GUI.Box(r, "");
Suppose you have a camera projection matrix, i.e. camera translation vector + rotation quaternion, like every typical camera, it is able to move and rotate in any direction. And independent of it's rotation whether it is looking forward, upward or downward I need to show a compass-like gauge pointing where the camera is targeted at.
The problem is that when the camera is pointed downwards the rotation of camera around it's optical center defines the value of the compass, but when the camera points forward, the rotation of camera around it's center no longer affects the value of compass, in this case the direction of camera defines the value of compass.
It get's more ugly when the camera is tilted downwards only 45 degrees, in this case it is not even clear whether the rotation around camera center affects rotation of compass.
So is there an elegant way of getting the compass value based on arbitrary camera projection matrix / quaternion?
Thank you in advance!
If you want just an arrow pointing at the target its:
Transform camera = Camera.main.transform;
Transform target = Target.transform;
Vector3 relativePosition = target.position - camera.position;
Vector3 targetRelative = Vector3.ProjectOnPlane(relativePosition, camera.forward);
float angle = Angle360(camera.up, targetRelative, camera.forward);
Compass.transform.rotation = Quaternion.Euler(0, 0, angle);
The angle function is:
float Angle360(Vector3 from, Vector3 to, Vector3 normal)
{
float dot = Vector3.Dot(from, to);
float det = Vector3.Dot(normal, Vector3.Cross(from, to));
return Mathf.Atan2(det, dot)*Mathf.Rad2Deg;
}
Here is how you can get the direction of the compass in worldspace:
Project the camera direction and target position on the XZ plane
Transform camera = Camera.main.transform;
Transform target = Target.transform;
Vector3 cameraWorldDirXZ = Vector3.ProjectOnPlane(camera.forward, Vector3.up).normalized;
Vector3 targetWorldDirXZ = Vector3.ProjectOnPlane(target.position, Vector3.up).normalized;
The angle between the cameraWorldDirXZ and targetWorldDirXZ is the angle of your compass needle.
But i don't think this will behave like you think it will. This gives you the angle that you need to rotate the camera.forward vector around the y axis to face the target. If you rotate around camera.forward you don't change either the camera.forward vector or the y axis so the compass wont change.
You might want to try a compass in local space. For that you project onto the camera XZ plane:
Vector3 cameraLocalDirXZ = camera.forward;
Vector3 targetLocalDirXZ = Vector3.ProjectOnPlane(target.position, camera.up).normalized;
Again the angle between the cameraLocalDirXZ and targetLocalDirXZ is the angle of your compass needle. This gives you the angle you need to rotate camera.forward around camera.up to face the target. Note that when you rotate around camera.forward it will change camera.up so it will change the compass direction.
If anyone stumbles upon this problem, the solution (thanks to #Pluto) is very simple, multiply your camera quaternion over three axis vectors (0,0,1), (0,1,0), (1,0,0), you will get three vectors defining coordinate system of your camera, project those three vectors onto your plane, find centroid of your three projected points and voila you have compass direction.
Here's the piece of code for that:
var rotation = /* Your quaternion */;
var cameraOrtX = rotation * new Vector3 (1, 0, 0);
var cameraOrtY = rotation * new Vector3 (0, 1, 0);
var cameraOrtZ = rotation * new Vector3 (0, 0, 1);
var cameraOrtPX = Vector3.ProjectOnPlane(cameraOrtX, new Vector3(0, 1, 0));
var cameraOrtPY = Vector3.ProjectOnPlane(cameraOrtY, new Vector3(0, 1, 0));
var cameraOrtPZ = Vector3.ProjectOnPlane(cameraOrtZ, new Vector3(0, 1, 0));
var centroid = (cameraOrtPX + cameraOrtPY + cameraOrtPZ) / 3.0f;
I am using the following code to handle rotating my player model to the position of my mouse.
void Update() {
// Generate a plane that intersects the transform's position with an upwards normal.
Plane playerPlane = new Plane(Vector3.up, transform.position);
// Generate a ray from the cursor position
Ray ray = Camera.main.ScreenPointToRay(Input.mousePosition);
// Determine the point where the cursor ray intersects the plane.
// This will be the point that the object must look towards to be looking at the mouse.
// Raycasting to a Plane object only gives us a distance, so we'll have to take the distance,
// then find the point along that ray that meets that distance. This will be the point
// to look at.
float hitdist = 0f;
// If the ray is parallel to the plane, Raycast will return false.
if (playerPlane.Raycast(ray, out hitdist)) {
// Get the point along the ray that hits the calculated distance.
var targetPoint = ray.GetPoint(hitdist);
// Determine the target rotation. This is the rotation if the transform looks at the target point.
Quaternion targetRotation = Quaternion.LookRotation(targetPoint - transform.position);
// Smoothly rotate towards the target point.
transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, speed * Time.deltaTime); // WITH SPEED
//transform.rotation = Quaternion.Slerp(transform.rotation, targetRotation, 1); // WITHOUT SPEED!!!
}
I would like to be able to determine if the rotation is clockwise or counter-clockwise for animation purposes. What would be the best way of handling this? I'm fairly unfamiliar with quaternions so I'm not really sure how to approach this.
Angles between quaternions are unsigned. You will always get the shortest distance, and there's no way of defining "counter-clockwise" or "clockwise" unless you actively specify an axis (a point of view).
What you CAN do, however, is to take the axis that you're interested in (I assume it's the normal to your base plane.. perhaps the vertical of your world?) and take the flat 2D components of your quaternions, map them there and compute a simple 2D angle between those.
Quaternion A; //first Quaternion - this is your desired rotation
Quaternion B; //second Quaternion - this is your current rotation
// define an axis, usually just up
Vector3 axis = new Vector3(0.0f, 1.0f, 0.0f);
// mock rotate the axis with each quaternion
Vector3 vecA = A * axis;
Vector3 vecB = B * axis;
// now we need to compute the actual 2D rotation projections on the base plane
float angleA = Mathf.Atan2(vecA.x, vecA.z) * Mathf.Rad2Deg;
float angleB = Mathf.Atan2(vecB.x, vecB.z) * Mathf.Rad2Deg;
// get the signed difference in these angles
var angleDiff = Mathf.DeltaAngle( angleA, angleB );
This should be it. I never had to do it myself and the code above is not tested. Similar to: http://answers.unity3d.com/questions/26783/how-to-get-the-signed-angle-between-two-quaternion.html
This should work even if A or B are not Quaternions, but one of them is an euler-angle rotation.
Two dimensional quaternions (complex numbers) have a signed angle. But, the more correct way to think about complex numbers is with an unsigned angle which is relative to either the XY oriented plane or the YX oriented plane. I.E. a combination of an unsigned angle an an oriented plane of rotation.
In 2D there are only two oriented planes of rotation so the idea of a "signed angle" is really just a trick to get both the unsigned angle and the oriented plane of rotation packed into a single number.
For a quaternion the "signed angle" trick cannot be used because in 3D you have an infinite number of oriented planes you can rotate in, so a single signed angle cannot encode all the rotation information like it can in the 2D case.
The only way for a signed angle to make sense in 3D is with reference to a particular oriented plane, such as the XY oriented plane.
-- UPDATE --
This is pretty easy to solve as a method on a quaternion class. If all you want to know is "is this counter clockwise", then since we know the rotation angle is from 0 to 180, a positive dot product between the quat's axis of rotation and the surface normal should indicate that we're rotating counter clockwise from the perspective of that surface. And a negative dot product indicates the opposite. Ignoring the zero case, this should do the trick with much less work:
public bool IsCounterClockwise( in Vector3 normal ) => I*normal.X + J*normal.Y + K*normal.Z >= 0;