I have a plane with four vertices. It can be rotate around z-axis (0, 0,1).(achieve using model matrix in metal).Model matrix is changed base on rotation gesture.
So what I need to do is rotate plane around z-axis through arbitrary (x,y) where x,y not equal to zero.It means rotate plane around an axis which is perpendicular to xy plane an going through (x,y) point.
Any sugestion please?
This works for me.Here dragCanvas method change translation in model matix while rotateCanvas change its rotation.You may implement your own which does the same. Metod convertCoodinates maps coordinate system to suit as describe in https://developer.apple.com/documentation/metal/hello_triangle
#objc func rotate(rotateGesture: UIRotationGestureRecognizer){
guard rotateGesture.view != nil else { return }
let location = rotateGesture.location(in: self.view)
var rotatingAnchorPoint = convertCoodinates(tapx:location.x , tapy:location.y )
if rotateGesture.state == UIGestureRecognizerState.changed {
print("rotation:\(rotateGesture.rotation)")
renderer?.dargCanvas(axis:float3(Float(rotatingAnchorPoint.x) ,Float(rotatingAnchorPoint.y ),0))
renderer?.rotateCanvas(rotation:Float(rotateGesture.rotation))
renderer?.dargCanvas(axis:float3(Float(-rotatingAnchorPoint.x ) ,Float(-rotatingAnchorPoint.y ),0))
rotateGesture.rotation = 0
} else if rotateGesture.state == UIGestureRecognizerState.began {
}else if rotateGesture.state == UIGestureRecognizerState.ended{
}
}
Related
I'm trying to get the bone rotations related to their parents, but I end up getting pretty weird angles.
I've tried everything, matrix multiplications, offsets, axis swapping, and no luck.
guard let bodyAnchor = anchor as? ARBodyAnchor else { continue }
let skeleton = bodyAnchor.skeleton
let jointTransforms = skeleton.jointLocalTransforms
for (i, jointTransform) in jointTransforms.enumerated() {
//RETRIEVE ANGLES HERE
}
In //RETRIEVE ANGLES HERE I've tried different approaches:
let n = SCNNode()
n.transform = SCNMatrix4(jointTransform)
print(n.eulerAngles)
In this try, I set the jointTransformation to a SCNNode.transform so I can retrieve the eulerAngles to make them human readable and try to understand what's happening.
I get to work some joints, but I think it's pure coincidence or luck, because the rest of the bones rotate very weird.
In other try I get them using jointModelTransforms (Model, instead of Local) so all transforms are relative to the Root bone of the Skeleton.
With this approach I do matrix multiplications like this:
LocalMatrix = Inverse(JointModelMatrix) * (ParentJointModelMatrix)
To get the rotations relative to its parent, but same situation, some bones rotate okay other rotate weird. Pure coincidence I bet.
Why do I want to get the bone rotations?
I'm trying build a MoCap app with my phone that passes to Blender the rotations, trying to build .BVH files from this, so I can use them on Blender.
This is my own rig:
I've done this before with Kinect, but I've been trying for days to do it on ARKit 3 with no luck :(
Using simd_quatf(from:to:) with the right input should do it. I had trouble with weird angles until i started normalising the vectors:
guard let bodyAnchor = anchor as? ARBodyAnchor else { continue }
let skeleton = bodyAnchor.skeleton
let jointTransforms = skeleton.jointLocalTransforms
for (i, jointTransform) in jointTransforms.enumerated() {
// First i filter out the root (Hip) joint because it doesn't have a parent
let parentIndex = skeleton.definition.parentIndices[i]
guard parentIndex >= 0 else { continue } // root joint has parent index of -1
//RETRIEVE ANGLES HERE
let jointVectorFromParent = simd_make_float3(jointTransform.columns.3)
let referenceVector: SIMD3<Float>
if skeleton.definition.parentIndices[parentIndex] >= 0 {
referenceVector = simd_make_float3(jointTransforms[parentIndex].columns.3)
} else {
// The parent joint is the Hip joint which should have
// a vector of 0 going to itself
// It's impossible to calculate an angle from a vector of length 0,
// So we're using a vector that's just pointing up
referenceVector = SIMD3<Float>(x: 0, y: 1, z: 0)
}
// Normalizing is important because simd_quatf gives weird results otherwise
let jointNormalized = normalize(jointVectorFromParent)
let referenceNormalized = normalize(referenceVector)
let orientation = simd_quatf(from: referenceNormalized, to: jointNormalized)
print("angle of joint \(i) = \(orientation.angle)")
}
One important thing to keep in mind though:
ARKit3 tracks only some joints (AFAIK the named joints in ARSkeleton.JointName). The other joints are extrapolated from that using a standardized skeleton. Which means, that the angle you get for the elbow for example won't be the exact angle the tracked persons elbow has there.
Just a guess… does this do the job?
let skeleton = bodyAnchor.skeleton
let jointTransforms = skeleton.jointLocalTransforms
for (i, jointTransform) in jointTransforms.enumerated() {
print(Transform(matrix: jointTransform).rotation)
}
I'm trying to keep an SCNNode always one meter away from the front of the camera, and manipulate the node so that the X and Z axes are always parallel to the ground, while the node rotates around the Y-axis so that the node is always facing the camera.
The code below achieves my goal for the most part, but when turning more than 90˚ clockwise or counterclockwise, the node starts turning. How can I fix that?
override func viewDidLoad() {
super.viewDidLoad()
boxParent.position = (sceneView.pointOfView?.position)!
boxParent.orientation = (sceneView.pointOfView?.orientation)!
boxParent.eulerAngles = (sceneView.pointOfView?.eulerAngles)!
sceneView.scene.rootNode.addChildNode(boxParent)
boxOrigin.position = SCNVector3(0,0,-1)
boxParent.addChildNode(boxOrigin)
box = SCNNode(geometry: SCNBox(width: 0.5, height: 0.2, length: 0.3, chamferRadius: 0))
box.geometry?.firstMaterial?.diffuse.contents = UIColor.blue
box.position = SCNVector3(0,0,0)
boxOrigin.addChildNode(box)
}
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
boxParent.eulerAngles = (sceneView.pointOfView?.eulerAngles)!
boxParent.orientation = (sceneView.pointOfView?.orientation)!
boxParent.position = (sceneView.pointOfView?.position)!
box.position = boxOrigin.worldPosition
box.eulerAngles.y = (sceneView.pointOfView?.eulerAngles.y)!
print(box.eulerAngles)
sceneView.scene.rootNode.addChildNode(box)
}
You're simultaneously using two types of rotation. It's wrong!
boxParent.orientation = (sceneView.pointOfView?.orientation)! //quaternion
This variable uses the node’s orientation, expressed as quaternion (4 components: x, y, z, w).
boxParent.eulerAngles = (sceneView.pointOfView?.eulerAngles)!
The node’s rotation, expressed as pitch, yaw, and roll angles, in radians (3 components: x, y, z).
You need to decide which var you'll be using: orientation or eulerAngles. I suppose you'll choose orientation.
Read this useful article and this one about Quaternions and what a Gimbal Lock is.
Also, use SCNLookAtConstraint object (node's negative z-axis points toward the constraint's target node) or SCNBillboardConstraint object (automatically adjusts a node's orientation so that its local z-axis always points toward the node's pointOfView) for automatically adjusting a node’s orientation, so you camera'll be always pointing toward another node.
so the quest is this, I got an ARPointCloud with a bunch of 3d points and I'd like to select them based on a 2d frame from the perspective of the camera / screen.
I was thinking about converting the 2d frame to a 3d frustum and check if the points where the 3d frustum box, not sure if this is the ideal method, and not even sure how to do that.
Would anyone know how to do this or have a better method of achieving this?
Given the size of the ARKit frame W x H and the camera intrinsics we can create planes for the view frustum sides.
For example using C++ / Eigen we can construct our four planes (which pass
through the origin) as
std::vector<Eigen::Vector3d> _frustumPlanes;
frustumPlanes.emplace_back(Eigen::Vector3d( fx, 0, cx - W));
frustumPlanes.emplace_back(Eigen::Vector3d(-fx, 0, -cx));
frustumPlanes.emplace_back(Eigen::Vector3d( 0, fy, cy - H));
frustumPlanes.emplace_back(Eigen::Vector3d( 0, -fy, -cy));
We can then clip a 3D point by checking its position against the z < 0
half-space and the four sides of the frustum:
auto pointIsVisible = [&](const Eigen::Vector3d& P) -> bool {
if (P.z() >= 0) return false; // behind camera
for (auto&& N : frustumPlanes) {
if (P.dot(N) < 0)
return false; // outside frustum plane
}
return true;
};
Note that it is best to perform this clipping in 3D (before the projection) since points behind or near the camera or points far outside
the frustum can have unstable projection values (u,v).
I need to find out which direction a camera is looking at, e.g. if it is looking towards Z+, Z-, X+, or X-.
I've tried using eulerAngles, but the range for yaw goes 0 -> 90 -> 0 -> -90 -> 0 which means I can only detect if the camera is looking towards Z or X, not if it's looking towards the positive or negative directions of those axes.
You can create an SCNNode that place it in worldFront property to get a vector with the x, y, and z direction.
Another way you could do it is like how this project did it:
// Credit to https://github.com/farice/ARShooter
func getUserVector() -> (SCNVector3, SCNVector3) { // (direction, position)
if let frame = self.sceneView.session.currentFrame {
let mat = SCNMatrix4(frame.camera.transform) // 4x4 transform matrix describing camera in world space
let dir = SCNVector3(-1 * mat.m31, -1 * mat.m32, -1 * mat.m33) // orientation of camera in world space
let pos = SCNVector3(mat.m41, mat.m42, mat.m43) // location of camera in world space
return (dir, pos)
}
return (SCNVector3(0, 0, -1), SCNVector3(0, 0, -0.2))
}
The image below shows a rotated box that should be moved horizontally on the X and Z axes. Y should stay unaffected to simplify the scenario. The box could also be the SCNNode of the camera, so I guess a projection does not make sense at this point.
So lets say we want to move the box in the direction of the red arrow. How to achieve this using SceneKit?
The red arrow indicates -Z direction of the box. It also shows us it is not parallel to the camera's projection or to the global axes that are shown as dark grey lines of the grid.
My last approach is the product of a translation matrix and a rotation matrix that results in a new transformation matrix. Do I have to add the current transform to the new transform then?
If yes, where is the SceneKit function for the addition of matrices like SCNMatrix4Mult for multiplication or do I have to write it myself using Metal?
If no, what I'm missing out with the matrix calculations?
I don't want to make use of GLKit.
So my understanding is that you want to move the Box Node along its own X axis (not it's parents X axis). And because the Box Node is rotated, its X axis is not aligned with its parent's one, so you have the problem to convert the translation between the two coordinate systems.
The node hierarchy is
parentNode
|
|----boxNode // rotated around Y (vertical) axis
Using Transformation Matrices
To move boxNode along its own X axis
// First let's get the current boxNode transformation matrix
SCNMatrix4 boxTransform = boxNode.transform;
// Let's make a new matrix for translation +2 along X axis
SCNMatrix4 xTranslation = SCNMatrix4MakeTranslation(2, 0, 0);
// Combine the two matrices, THE ORDER MATTERS !
// if you swap the parameters you will move it in parent's coord system
SCNMatrix4 newTransform = SCNMatrix4Mult(xTranslation, boxTransform);
// Allply the newly generated transform
boxNode.transform = newTransform;
Please Note: The order matters when multiplying matrices
Another option:
Using SCNNode coordinate conversion functions, looks more straight forward to me
// Get the boxNode current position in parent's coord system
SCNVector3 positionInParent = boxNode.position;
// Convert that coordinate to boxNode's own coord system
SCNVector3 positionInSelf = [boxNode convertPosition:positionInParent fromNode:parentNode];
// Translate along own X axis by +2 points
positionInSelf = SCNVector3Make(positionInSelf.x + 2,
positionInSelf.y,
positionInSelf.z);
// Convert that back to parent's coord system
positionInParent = [parentNode convertPosition: positionInSelf fromNode:boxNode];
// Apply the new position
boxNode.position = positionInParent;
Building on #Sulevus's correct answer, here's an extension to SCNNode that simplifies things by using the convertVector rather than the convertPosition transformation, in Swift.
I've done it as a var returning a unit vector, and supplied an SCNVector3 overload of multiply so you can say things like
let action = SCNAction.move(by: 2 * cameraNode.leftUnitVectorInParent, duration: 1)
public extension SCNNode {
var leftUnitVectorInParent: SCNVector3 {
let vectorInSelf = SCNVector3(x: 1, y: 0, z: 0)
guard let parent = self.parent else { return vectorInSelf }
// Convert to parent's coord system
return parent.convertVector(vectorInSelf, from: self)
}
var forwardUnitVectorInParent: SCNVector3 {
let vectorInSelf = SCNVector3(x: 0, y: 0, z: 1)
guard let parent = self.parent else { return vectorInSelf }
// Convert to parent's coord system
return parent.convertVector(vectorInSelf, from: self)
}
func *(lhs: SCNVector3, rhs: CGFloat) -> SCNVector3 {
return SCNVector3(x: lhs.x * rhs, y: lhs.y * rhs, z: lhs.z * rhs)
}
func *(lhs: CGFloat, rhs: SCNVector3) -> SCNVector3 {
return SCNVector3(x: lhs * rhs.x, y: lhs * rhs.y, z: lhs * rhs.z)
}
}
The far easier way this is usually done:
The usual, normal, and extremely easy way to do this in any game engine or 3D engine is:
You simply have a wrapper node, which, holds the node in question.
This is indeed the entire point of transforms, they enable you to abstract out a certain motion.
That's the whole point of 3D engines - the GPU just multiplies out all the quaternions on the way down to the object; it's wholly pointless to (A) figure out in your head the math and (B) do it manually (indeed in the CPU).
In Unity it's "game objects", in scene kit it's "nodes" and so on.
In all 3D engines, including scene kit, almost everything has one or more "holders" around it.
To repeat, the reasons for this are (A) it's the entire raison d'etre of a game engine, to achieve performance in multiplying out the quaternions of every vertex and (B) sheer convenience and code solidity.
One of a million examples ...
Of course you can trivially do it in code,
cameraHolder.addChildNode(camera)
In the OP's example. It looks like you would use cameraHolder only to rotate the camera. And then for the motion the OP is asking about, simply move camera.
It's perfectly normal to have a chain of a number of nodes to get to an object.
This is often used for "effects". Say you have an object, which sometimes has to "vibrate up and down". You can have one node which only does that movement. Note that then, all the animations etc for that movement only have to be on that node. And critically, they can run independently of any other animations or movements. (And indeed you can just use the node elsewhere to jiggle something else.)