This little method that I wrote, changes spotlight1's position to the unexpected value.
If I understand well, setPosition method should set spotlight's translation relative to the tv's position
TV's translation: [0.0, 0.0, -5.0]
setPosition to [0.0, 5.0, 0.5] relative to Tv's translation.
So:
[0.0 + 0, 0.0 + 5, -5.0 + 0.5] = [0.0, 5.0, -4.5]
But what I get is:
[0.0, 0.9999994, -4.9]
Am I missing some important information here?
func loadLights() {
arView.scene.addAnchor(lightAnchor)
lightAnchor.addChild(spotlight1)
print(tv?.position) // 0.0, 0.0, -5.0
spotlight1.setPosition([0, 5, 0.5], relativeTo: tv)
if let tv = tv {
spotlight1.look(at: tv.position,
from: spotlight1.position,
relativeTo: nil)
}
print(spotlight1.position) // 0.0, 0.99999994, -4.99
}
Reference Coordinate Frame
As paradoxical as it may sound, RealityKit did everything right. You need to take into account the frame of reference (frame of your tv model). As far as I understand, you reduced the scale of the tv model by five times. Am I right? However, reference's transform matters.
In other words, you've scaled down the local coordinate frame (a.k.a. local coordinate system) of the TV model, you're trying to position spotLight1 relative to.
About relativeTo parameter
Read this post and this post to explore possible issues.
Related
I am trying to use ARCamera matrix to do the conversion of 3D point to 2D in ARkit/Scenekit. Previously, I used projectpoint to get the projected x and y coordinate which is working fine. However, the app is significantly slowed down and would crash for appending long recordings.
So I turn into another approach: using the ARCamera parameter to do the conversion on my own. The Apple document for projectionMatrix did not give much. So I looked into the theory about projection matrix The Perspective and Orthographic Projection Matrix and Metal Tutorial. From my understanding that when we have a 3D points P=(x,y,z), in theory we should be able to just get the 2D point like so: P'(2D)=P(3D)*projectionMatrix.
I am assuming that's would be the case, so I did:
func session(_ session: ARSession, didUpdate frame: ARFrame) {
guard let arCamera = session.currentFrame?.camera else { return }
//intrinsics: a matrix that converts between the 2D camera plane and 3D world coordinate space.
//projectionMatrix: a transform matrix appropriate for rendering 3D content to match the image captured by the camera.
print("ARCamera ProjectionMatrix = \(arCamera.projectionMatrix)")
print("ARCamera Intrinsics = \(arCamera.intrinsics)")
}
I am able to get the projection matrix and intrinsics (I even tired to get intrinsics to see whether it changes) but they are all the same for each frame.
ARCamera ProjectionMatrix = simd_float4x4([[1.774035, 0.0, 0.0, 0.0], [0.0, 2.36538, 0.0, 0.0], [-0.0011034012, 0.00073593855, -0.99999976, -1.0], [0.0, 0.0, -0.0009999998, 0.0]])
ARCamera Intrinsics = simd_float3x3([[1277.3052, 0.0, 0.0], [0.0, 1277.3052, 0.0], [720.29443, 539.8974, 1.0]])...
I am not too sure I understand what is happening here as I am expecting that the projection matrix will be different for each frame. Can someone explain the theory here with projection matrix in scenekit/ARKit and validate my approach? Am I using the right matrix or do I miss something here in the code?
Thank you so much in advance!
You'd likely need to use the camera's transform matrix as well, as this is what changes between frames as the user moves around the real world camera and the virtual camera's transform is updated to best match that. Composing that together with the projection matrix should allow you to get into screen space.
Blocks just crumble apart.
How can this problem be solved?
Initializing blocks:
var boxNode = SCNNode(geometry: SCNBox(width: 0.75, height: 0.15, length: 0.25, chamferRadius: 0))
boxNode.position = SCNVector3(x: x1, y: y, z: z1)
boxNode.geometry?.firstMaterial = SCNMaterial()
boxNode.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "wood.jpg")
boxNode.physicsBody = SCNPhysicsBody(type: .dynamic, shape: nil)
boxNode.eulerAngles.y = Float(Double.pi / 2) * rotation
boxNode.physicsBody?.friction = 1
boxNode.physicsBody?.mass = 0.5
boxNode.physicsBody?.angularDamping = 1.0
boxNode.physicsBody?.damping = 1
picture
video
full code
I won't be able to tell you how to fix it as I have the exact same problem which I wasn't able to solve. However, as I played around I figured out a couple of things (which you may find useful):
The same problem hasn't happened to me in pure SceneKit, hence I think it's a bug in ARKit
Node with physics has to be added to the rootNode of the scene, otherwise odd stuff happens (elements passing through each other, gravity behaving in an inconsistent way)
If you pass nil as shape parameter, SceneKit will figure bounding box based on the geometry of the node. This hasn't worked properly for me so what I've done (using SceneKit editor) was to duplicate the geometry and then set it as a custom shape for the bounding box (have a look at the attached image)
Overall I've found physics simulation in SceneKit when used with ARKit to be extremely buggy and I spent a lot of time "tricking" it into working more-or-less how I wanted it to work.
I have plugged in my GKBehavior/GKAgent2D/GKComponent system for some behavioral routines and am having trouble fixing this bug I can't figure out.
Let me first be descriptive:
My current Entity/Component system is based on apple's ECS where:
1-entitymanager:GKEntity manages all components and contains all entities in the game and uses updateWithDelta which goes into all GKComponentSystems which themselves update every component within their system.
Currently, the update order is as follows:
behavior> node> playerNode> sound> interface> physics
2-Each system is a GKComponentSystem which handles updating all elements within.
I have two extensions of NSViewController called GameRenderer and GameControls which both are delegates of SCNSceneRendererDelegate and a custom protocol called KBAndTouchDelegate which update render on screen and control respectively (will be changing this to be incorporated in the ECS once my order updates feel right).
Now, in the behavior componentSystem, it updates a MoveComponent which contains a few functions for updating (willUpdate and didUpdate). Both have a 2DAgent position that is first taken from an SCNNode's position (converted from SCNVector3 to Float2 with just x and z coordinates being changed) then reflected back onto the node after the behavioral algorithm is evaluated.
Here are snippets for the updates:
func agentWillUpdate(agent: GKAgent) {
guard let nodeComponent = entity?.componentForClass(SCNNodeComponent.self)
else{
return
}
let nodePos = nodeComponent.node.position
position = float2(Float(nodePos.x), Float(nodePos.z))
}
func agentDidUpdate(agent: GKAgent) {
guard let nodeComponent = entity?.componentForClass(SCNNodeComponent.self)
else{
return
}
let pos = SCNVector3(x: CGFloat(position.x), y: nodeComponent.node.position.y, z: CGFloat(position.y))
print("\(nodeComponent.node.name) changed to \(pos)")
nodeComponent.node.position = pos
}
The #override updateWithDeltaTime routine checks for the party in which the moveComponent's owner is and placed in an array for getting the closest enemy distance and return the enemy.
The behavior has very low maxSpeed and maxVelocity factors.
Running the application, I printed a set of movements for two nodes which are enemies and monitor each other's position. The enemy search routine works fine as they reflect each other’s position in alternate sequence. The result is below:
Optional("zombie") changed to SCNVector3(x: -20.0, y: 0.0, z: -20.0)
enemy position is: (0.0, 0.0)
Optional("player") changed to SCNVector3(x: 0.0, y: 0.0, z: 0.0)
enemy position is: (-20.0, -20.0)
Optional("zombie") changed to SCNVector3(x: -20.0, y: 0.0, z: 41567.44140625)
enemy position is: (0.0, 0.0)
Optional("player") changed to SCNVector3(x: 0.0, y: 0.0, z: 0.0)
enemy position is: (-20.0, 41567.44140625)
HRTF loaded
Optional("zombie") changed to SCNVector3(x: 30.136764526367188, y: 0.0, z: -20.2421875)
enemy position is: (0.0, 0.0)
Optional("player") changed to SCNVector3(x: 0.0, y: 0.0, z: 0.0)
enemy position is: (30.1367645263672, -20.2421875)
Optional("zombie") changed to SCNVector3(x: -38636.87109375, y: 0.0, z: 15289.939453125)
If looking at the first positions of the "zombie" you will see it starts at -20, -20 then goes to impossibly big values for both x and z positions (again, x and z in the scene's world). The player doesn't move as I don't use any controls just to see if the zombie is able to find the player with the following GKBehavior call
init(targetSpeed: Float, seek: GKAgent, avoid: [GKAgent]){
super.init()
if targetSpeed>0{
setWeight(0.1, forGoal: GKGoal(toReachTargetSpeed: targetSpeed))
setWeight(0.5, forGoal: GKGoal(toSeekAgent: seek))
setWeight(1.0, forGoal: GKGoal(toAvoidAgents: avoid, maxPredictionTime: 1.0))
}
}
The above behavior is called in MoveComponent for the zombie's enemy (the player). Meaning the zombie will seek for the player.
Can anyone tell me what the hell these ginormous values for x and z are ? The zombie and player are about sqrt(800) or 28 meters away. But the zombie goes to stuff like 35800 meters away on x then switches back to about 50 then back and forth at each frame.
There’s a definite pattern and I’m not sure whether this GKBehavior uses A star at which point the probabilistic roadmap sets some very odd coordinate samples but nowhere in my code is there any scalar that would move the zombie so far away and flip the distances back to something reasonably normal every frame.
For anyone who might be having the same issue, as of yesterday 21-03-2016, the issue has been resolved by apple with their newest public update of x-code 7.3.
I am trying to manipulate the player node using physicsBody.velocity by adding or subtracting velocity to the axis for different directions. The problem is that I'm having trouble finding the method for applying that to local space or at least applying the velocity in relation to the direction the object is facing. In other words, it works fine if I have not rotated the object's node. If I do it will still add the velocity to the unrotated space.
I know there is a way to add velocity to the current SCNVector3, but I cannot figure it out.
if isZThrustPositive {
if let velocity = self.physicsBody?.velocity {
if velocity.z * 100 <= 8000 {
thrustDirection = SCNVector3(
x: velocity.x,
y: velocity.y,
z: velocity.z + kPlayerShipMainThrust)
self.physicsBody?.velocity = thrustDirection
}
}
}
I am also trying to rotate the node using angularVelocity in a similar fashion. That works fine as long as the node has not moved or rotated. If it has, it seems to be using world space as well.
if isYRotatingPositive {
if let angularVel = self.physicsBody?.angularVelocity {
self.physicsBody?.angularVelocity = SCNVector4(
x: angularVel.x,
y: angularVel.y + kPlayerShipRotationSpeed,
z: angularVel.z,
w: angularVel.w + kPlayerShipRotationMagnitude)
}
}
After the physics are simulated, I am updating the node's position and rotation.
I have also tried convertPosition, but could not figure out a way to make this work. (I got some really crazy results with this).
Any help is much appreciated.
UPDATE I got the playerShip object to add velocity in the proper directions when it is rotated, but I am still unable to perform multiple rotations without it not turning in the correct direction.
For the ship velocity I did this:
if isXThrustPositive {
thrustDirection = self.convertPosition(SCNVector3(x: kPlayerShipMainThrust, y: 0.0, z: 0.0), toNode: self.parentNode!)
thrustDirection.x = thrustDirection.x - self.position.x
thrustDirection.y = thrustDirection.y - self.position.y
thrustDirection.z = thrustDirection.z - self.position.z
self.physicsBody?.applyForce(thrustDirection, impulse: true)
}
When trying to use a similar method for rotation (I know it would be doomed), the resulting SCNVector3 is just a position, not the direction the node is currently facing. Is there any information on convertTransform and how I might use that?
As it turns out, I had to get the proper position from the rootNode of the scene to perform a proper rotation based on the current orientation of the SCNNode.
xAxis = self.convertPosition(SCNVector3Make(1.0, 0.0, 0.0), toNode: self.parentNode!)
yAxis = self.convertPosition(SCNVector3Make(0.0, 1.0, 0.0), toNode: self.parentNode!)
zAxis = self.convertPosition(SCNVector3Make(0.0, 0.0, 1.0), toNode: self.parentNode!)
if isXRotatingPositive {
self.physicsBody?.applyTorque(
SCNVector4(
x: sin(kPlayerShipRotationSpeed/2.0) * (xAxis.x - self.position.x),
y: sin(kPlayerShipRotationSpeed/2.0) * (xAxis.y - self.position.y),
z: sin(kPlayerShipRotationSpeed/2.0) * (xAxis.z - self.position.z),
w: cos(kPlayerShipRotationSpeed/2.0) * kPlayerShipRotationMagnitude),
impulse: true)
}
Then I just used the standard quaternion rotation formula to get the rotation based on the new axes from the current position.
I hope this helps someone else (and that more information on SceneKit is forthcoming)
If any SceneKit experts want to comment on this or offer suggestions, they are much appreciated. :)
I'm trying to display a pyramid that points following the z axis and then rotates on itself around z too.
As my camera is on the z axis, I'm expecting to see the pyramid from above. I managed to rotate the pyramid to see it this way but when I add the animation it seems to rotate on multiple axis.
Here is my code:
// The following create the pyramid and place it how I want
let pyramid = SCNPyramid(width: 1.0, height: 1.0, length: 1.0)
let pyramidNode = SCNNode(geometry: pyramid)
pyramidNode.position = SCNVector3(x: 0, y: 0, z: 0)
pyramidNode.rotation = SCNVector4(x: 1, y: 0, z: 0, w: Float(M_PI / 2))
scene.rootNode.addChildNode(pyramidNode)
// But the animation seems to rotate aroun 2 axis and not just z
var spin = CABasicAnimation(keyPath: "rotation")
spin.byValue = NSValue(SCNVector4: SCNVector4(x: 0, y: 0, z: 1, w: 2*Float(M_PI)))
spin.duration = 3
spin.repeatCount = HUGE
pyramidNode.addAnimation(spin, forKey: "spin around")
Trying to both manually set and animate the same property can cause issues. Using a byValue animation makes the problem worse -- that concatenates to the current transform, so it's harder to keep track of whether the current transform is what the animation expects to start with.
Instead, separate the fixed orientation of the pyramid (its apex is in the -z direction) from the animation (it spins around the axis it points in). There's two good ways to do this:
Make pyramidNode the child of another node that gets the one-time rotation (π/2 around x-axis), and apply the spin animation directly to pyramidNode. (In this case, the apex of the pyramid will still point in the +y direction of its local space, so you'll want to spin around that axis instead of the z-axis.)
Use the pivot property to transform the local space of pyramidNode's contents, and animate pyramidNode relative to its containing space.
Here's some code to show the second approach:
let pyramid = SCNPyramid(width: 1.0, height: 1.0, length: 1.0)
let pyramidNode = SCNNode(geometry: pyramid)
pyramidNode.position = SCNVector3(x: 0, y: 0, z: 0)
// Point the pyramid in the -z direction
pyramidNode.pivot = SCNMatrix4MakeRotation(CGFloat(M_PI_2), 1, 0, 0)
scene.rootNode.addChildNode(pyramidNode)
let spin = CABasicAnimation(keyPath: "rotation")
// Use from-to to explicitly make a full rotation around z
spin.fromValue = NSValue(SCNVector4: SCNVector4(x: 0, y: 0, z: 1, w: 0))
spin.toValue = NSValue(SCNVector4: SCNVector4(x: 0, y: 0, z: 1, w: CGFloat(2 * M_PI)))
spin.duration = 3
spin.repeatCount = .infinity
pyramidNode.addAnimation(spin, forKey: "spin around")
Some unrelated changes to improve code quality:
Use CGFloat when explicit conversion is required to initialize an SCNVector component; using Float or Double specifically will break on 32 or 64 bit architecture.
Use .infinity instead of the legacy BSD math constant HUGE. This type-infers to whatever the type of spin.repeatCount is, and uses a constant value that's defined for all floating-point types.
Use M_PI_2 for π/2 to be pedantic about precision.
Use let instead of var for the animation, since we never assign a different value to spin.
More on the CGFloat error business: In Swift, numeric literals have no type until the expression they're in needs one. That's why you can do things like spin.duration = 3 -- even though duration is a floating-point value, Swift lets you pass an "integer literal". But if you do let d = 3; spin.duration = d you get an error. Why? Because variables/constants have explicit types, and Swift doesn't do implicit type conversion. The 3 is typeless, but when it gets assigned to d, type inference defaults to choosing Int because you haven't specified anything else.
If you're seeing type conversion errors, you probably have code that mixes literals, constants, and/or values returned from functions. You can probably just make the errors go away by converting everything in the expression to CGFloat (or whatever the type you're passing that expression to is). Of course, that'll make your code unreadable and ugly, so once you get it working you might start removing conversions one at a time until you find the one that does the job.
SceneKit includes animation helpers which are much simpler & shorter to use than CAAnimations. This is ObjC but gets across the point:
[pyramidNode runAction:
[SCNAction repeatActionForever:
[SCNAction rotateByX:0 y:0 z:2*M_PI duration:3]]];
I changed byValue to toValue and this worked for me. So change the line...
spin.byValue = NSValue(SCNVector4: SCNVector4(...
Change it to...
spin.toValue = NSValue(SCNVector4: SCNVector4(x: 0, y:0, z:1, w: 2*float(M_PI))