In Unity, there is an event (Rendered.OnBecameVisible) that you can subscribe to to know when an object (in SceneKit's parlance, an SCNNode) becomes visible "by any camera".
Does Scenekit have an equivalent method/functionality/event subscription/or anything remotely similar I can use to achieve the same-ish functionality? (calling a method when an SCNNode is "visible" by a camera)
https://docs.unity3d.com/ScriptReference/Renderer.OnBecameVisible.html
There is no delegate method you could implement, but your SCNSceneRenderer, typically your SCNView, has a isNode(_:insideFrustumOf:) method:
Use this method to test whether a node lies within the viewing frustum defined by another node (which may or may not be the scene renderer’s current pointOfView node). For example, in a game scene containing multiple camera nodes, you could use this method to determine which camera is currently best for viewing a moving player character.
Related
I want to render two SpriteKit SKViews at the same time which share a common SKScene. I'd like each SKView to show a different part of the scene (e.g. from a different SKCameraNode). Is this possible?
What I've tried: I've instantiated two SKViews and called .presentScene(mySharedScene) on both of them. I can render those views simultaneously and animations work just fine. But since the camera position is set on the SKScene itself via the .camera property, I can't assign a different camera to each SKView.
Ultimately, I'd like to create a simple bouncing ball that leverages SpriteKit's physics engine. Each SKView will be displayed on a different physical monitor, and the ball should be able to bounce between them. I'm doing this purely as a learning exercise.
Yes, you can, I have done split screens before, but let me tell you. It is a real pain. All of your updates get called twice, so you are going to have to develop a system that works around it. Instead, for a simple experience I recommend copying your scene to your 2nd View then updating your camera to your new location.
func didFinishUpdate()
{
let copy = scene.copy()!
view2.presentScene(copy)
copy2.camera!.position = newPosition
}
I'm looking to change the field of view for the rendered content in my AR session. Obviously we can't change the raw camera FOV, but I think it should be possible to change the field of view for the rendered SceneKit content.
Changing the camera field of view is trivial in a raw SceneKit SCNCamera... but I don't know how to do this within an ARSCNView.
You might be able to access the pointOfView property of your ARSCNView (and then retrieve the active SCNCamera).
If that doesn't work (ARKit changing the camera property every frame etc.), you can always go the path of writing the code yourself by using ARSession directly with SCNView.
Note that unless you have a 3D scene covering the entire camera stream, changing the FoV of your virtual camera would break the AR registration (alignment).
The developer documentation for ARSession suggests "If you build your own renderer for AR content, you'll need to instantiate and maintain an ARSession object yourself."
This repo does this: https://github.com/hanleyweng/iOS-ARKit-Headset-View
This code will retrieve the camera from an ARSCNView if there is one:
sceneView.scene.rootNode.childNodes.first(where: { $0.camera != nil})
Note that this will return the camera's associated node, which you may need if you want to control its position or angle directly. The SCNCamera itself is stored in the node's camera property.
It's best not to touch the AR camera if you can avoid it as it will mess up the association between the world model and the scene. I occasionally use this technique if I want a control system that can optionally use AR to track device motion and angle, but which doesn't have to translate into real-world coordinates (i.e. VR apps that don't display the camera feed).
Essentially, I'd only do this if you're using AROrientationTrackingConfiguration or similar.
EDIT I should probably mention that ARKit overrides the camera's projectionTransform property, so you probably won't be able to set fieldOfView manually. I've had some success setting xFov and yFov, but since these are deprecated you shouldn't rely on them.
I am new to iPhone app development (but experienced developer in other languages) and I am making an ARKit application in which I need to track an image's position and display a rectangle around this image.
I could do it in C++ with OpenCV and make the appropriate "bridging" classes to be able to call this code from swift. Now, I need to get the images from ARKit and pass them to this function.
How do I suscribe a function which handles the ARFrames from the ARKit scene? I found that I could get some ARFrame from sceneView.session.currentFrame but I did not find how to make a function that would be called for each frame (or each time my function has ended and is ready to receive the next frame).
Also, I discovered the Vision Framework but it seems to only be able to track an element on which the user tapped. Is that right or is there a combination of predefined functions which could be used to this purpose?
You can get each captured frame by the camera with the session(:didUpdate:) method from the ARSessionDelegate protocol. Note that this method might be useful even if you don't use the provided ARFrame; for instance, I use it to "poll" if there's any virtual content added currently visible.
Regarding the tracking image question, I believe you could create an ARAnchor or SCNNode and position it over the image and ARKit would track it
In ARKit, I have found 2 ways of inserting a node after the hitTest
Insert an ARAnchor then create the node in renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode?
let anchor = ARAnchor(transform:hit.worldTransform)
sceneView.session.add(anchor:anchor)
Insert the node directly
node.position = SCNVector3(hit.worldTransform.columns.3.x, hit.worldTransform.columns.3.y, hit.worldTransform.columns.3.z)
sceneView.scene.rootNode.addChildNode(node)
Both look to work for me, but why one way or the other?
Update: As of iOS 11.3 (aka "ARKit 1.5"), there is a difference between adding an ARAnchor to the session (and then associating SceneKit content with it through ARSCNViewDelegate callbacks) and just placing content in SceneKit space.
When you add an anchor to the session, you're telling ARKit that a certain point in world space is relevant to your app. ARKit can then do some extra work to make sure that its world coordinate space lines up accurately with the real world, at least in the vicinity of that point.
So, if you're trying to make virtual content appear "attached" to some real-world point of interest, like putting an object on a table or wall, you should see less "drift" due to world-tracking inaccuracy if you give that object an anchor than if you just place it in SceneKit space. And if that object moves from one static position to another, you'll want to remove the original anchor and add one at the new position afterward.
Additionally, in iOS 11.3 you can opt in to "relocalization", a process that helps ARKit resume a session after it gets interrupted (by a phone call, switching apps, etc). The session still works while it's trying to figure out how to map where you were before to where you are now, which might result in the world-space positions of anchors changing once relocalization succeeds.
(On the other hand, if you're just making space invaders that float in the air, perfectly matching world space isn't as important, and thus you won't really see much difference between anchor-based and non-anchor-based positioning.)
See the bit around "Use anchors to improve tracking quality around virtual objects" in Apple's Handling 3D Interaction and UI Controls in Augmented Reality article / sample code.
The rest of this answer remains historically relevant to iOS 11.0-11.2.5 and explains some context, so I'll leave it below...
Consider first the use of ARAnchor without SceneKit.
If you're using ARSKView, you need a way to reference positions / orientations in 3D (real-world) space, because SpriteKit isn't 3D. You need ARAnchor to keep track of positions in 3D so that they can get mapped into 2D.
If you're building your own engine with Metal (or GL, for some strange reason)... that's not a 3D scene description API — it's a GPU programming API — so it doesn't really have a notion of world space. You can use ARAnchor as a bridge between ARKit's notion of world space and whatever you build.
So in some cases you need ARAnchor because that's the only sensible way to refer to 3D positions. (And of course, if you're using plane detection, you need ARPlaneAnchor because ARKit will actually move those relative to scene space as it refined its estimates of where planes are.)
With ARSCNView, SceneKit already has a 3D world coordinate space, and ARKit does all the work of making that space match up to the real-world space ARKit maps out. So, given a float4x4 transform that describes a position (and orientation, etc) in world space, you can either:
Create an ARAnchor, add it to the session, and respond to ARSCNViewDelegate callback to provide SceneKit content for each anchor, which ARKit will add to and position in the scene for you.
Create an SCNNode, set its simdTransform, and add it as a child of the scene's rootNode.
As long as you have a running ARSession, there's no difference between the two approaches — they're equivalent ways to say the same thing. So if you like doing things the SceneKit way, there's nothing wrong with that. (You can even use SCNVector3 and SCNMatrix4 instead of SIMD types if you want, but you'll have to convert back and forth if you're also getting SIMD types from ARKit APIs.)
The one time these approaches differ is when the session is reset. If world tracking fails, you resume an interrupted session, and/or
you start a session over again, "world space" may no longer line up with the real world in the same way it did when you placed content in the scene.
In this case, you can have ARKit remove anchors from the session — see the run(_:options:) method and ARSession.RunOptions. (Yes, all of them, because at this point you can't trust any of them to be valid anymore.) If you placed content in the scene using anchors and delegate callbacks, ARKit will nuke all the content. (You get delegate callbacks that it's being removed.) If you placed content with SceneKit API, it stays in the scene (but most likely in the wrong place).
So, which to use sort of depends on how you want to handle session failures and interruptions (and outside of that there's no real difference).
SCNVector3 is just "a representation of a three-component vector." SCNVector3 docs.
When using ARAnchor, you have access to a three-component vector, but also you are able "to track the positions and orientations of real or virtual objects relative to the camera" ARAnchor docs. And that's why you use the session to add the anchor instead of using the scene.
See the docs and you can see the difference in terms of the API :)
Hope it helps.
I am aware of how to use the SpriteKit physics engine to detect when a collision occurs between two nodes, however, I'd like to be able to detect this and also get the direction relative to one of the objects that the collision occurred. Incase anyone is curious, I'm asking this so I can make a platform game and I need to detect when something is resting on the top of the floor. Any help is appreciated, thanks!
Implement the SKPhysicsContactDelegate on one of your objects (perhaps your scene object). Set your scene's physicsWorld.contactDelegate to the object.
In the contact delegate's didBeginContact(_:) method, you are given an SKPhysicsContact object. The contact's contactNormal property should be the direction you're interested in.