In ARKit, I have found 2 ways of inserting a node after the hitTest
Insert an ARAnchor then create the node in renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode?
let anchor = ARAnchor(transform:hit.worldTransform)
sceneView.session.add(anchor:anchor)
Insert the node directly
node.position = SCNVector3(hit.worldTransform.columns.3.x, hit.worldTransform.columns.3.y, hit.worldTransform.columns.3.z)
sceneView.scene.rootNode.addChildNode(node)
Both look to work for me, but why one way or the other?
Update: As of iOS 11.3 (aka "ARKit 1.5"), there is a difference between adding an ARAnchor to the session (and then associating SceneKit content with it through ARSCNViewDelegate callbacks) and just placing content in SceneKit space.
When you add an anchor to the session, you're telling ARKit that a certain point in world space is relevant to your app. ARKit can then do some extra work to make sure that its world coordinate space lines up accurately with the real world, at least in the vicinity of that point.
So, if you're trying to make virtual content appear "attached" to some real-world point of interest, like putting an object on a table or wall, you should see less "drift" due to world-tracking inaccuracy if you give that object an anchor than if you just place it in SceneKit space. And if that object moves from one static position to another, you'll want to remove the original anchor and add one at the new position afterward.
Additionally, in iOS 11.3 you can opt in to "relocalization", a process that helps ARKit resume a session after it gets interrupted (by a phone call, switching apps, etc). The session still works while it's trying to figure out how to map where you were before to where you are now, which might result in the world-space positions of anchors changing once relocalization succeeds.
(On the other hand, if you're just making space invaders that float in the air, perfectly matching world space isn't as important, and thus you won't really see much difference between anchor-based and non-anchor-based positioning.)
See the bit around "Use anchors to improve tracking quality around virtual objects" in Apple's Handling 3D Interaction and UI Controls in Augmented Reality article / sample code.
The rest of this answer remains historically relevant to iOS 11.0-11.2.5 and explains some context, so I'll leave it below...
Consider first the use of ARAnchor without SceneKit.
If you're using ARSKView, you need a way to reference positions / orientations in 3D (real-world) space, because SpriteKit isn't 3D. You need ARAnchor to keep track of positions in 3D so that they can get mapped into 2D.
If you're building your own engine with Metal (or GL, for some strange reason)... that's not a 3D scene description API — it's a GPU programming API — so it doesn't really have a notion of world space. You can use ARAnchor as a bridge between ARKit's notion of world space and whatever you build.
So in some cases you need ARAnchor because that's the only sensible way to refer to 3D positions. (And of course, if you're using plane detection, you need ARPlaneAnchor because ARKit will actually move those relative to scene space as it refined its estimates of where planes are.)
With ARSCNView, SceneKit already has a 3D world coordinate space, and ARKit does all the work of making that space match up to the real-world space ARKit maps out. So, given a float4x4 transform that describes a position (and orientation, etc) in world space, you can either:
Create an ARAnchor, add it to the session, and respond to ARSCNViewDelegate callback to provide SceneKit content for each anchor, which ARKit will add to and position in the scene for you.
Create an SCNNode, set its simdTransform, and add it as a child of the scene's rootNode.
As long as you have a running ARSession, there's no difference between the two approaches — they're equivalent ways to say the same thing. So if you like doing things the SceneKit way, there's nothing wrong with that. (You can even use SCNVector3 and SCNMatrix4 instead of SIMD types if you want, but you'll have to convert back and forth if you're also getting SIMD types from ARKit APIs.)
The one time these approaches differ is when the session is reset. If world tracking fails, you resume an interrupted session, and/or
you start a session over again, "world space" may no longer line up with the real world in the same way it did when you placed content in the scene.
In this case, you can have ARKit remove anchors from the session — see the run(_:options:) method and ARSession.RunOptions. (Yes, all of them, because at this point you can't trust any of them to be valid anymore.) If you placed content in the scene using anchors and delegate callbacks, ARKit will nuke all the content. (You get delegate callbacks that it's being removed.) If you placed content with SceneKit API, it stays in the scene (but most likely in the wrong place).
So, which to use sort of depends on how you want to handle session failures and interruptions (and outside of that there's no real difference).
SCNVector3 is just "a representation of a three-component vector." SCNVector3 docs.
When using ARAnchor, you have access to a three-component vector, but also you are able "to track the positions and orientations of real or virtual objects relative to the camera" ARAnchor docs. And that's why you use the session to add the anchor instead of using the scene.
See the docs and you can see the difference in terms of the API :)
Hope it helps.
Related
Long-winded question out of the way, I'll provide a diagram of what I am going for:
The red square represents the character, the blue rectangle represents the camera, the green dot represents the center of the "stage", and the black circle is the stage itself.
What I desire is to essentially lock the player's movement around the "center" of the stage, so that anytime you move left or right you are more or less rotating around said center. However, I also want the player to be able to move forwards and backwards to/from the center as well. Keep in mind I want the camera to always stay directly behind the player. I have tried many different methods, and the latest is the following:
I took a default actor, attached a spring arm, attached a child actor to that (gets possessed to become the playable character), attached another spring arm, and finally the camera to that. I then added the blueprint code to the first spring arm so that it was the one being controlled by the left/right controls. However, upon hitting play, the only thing that moves is the camera, and it can only move forwards and backwards.
I'm admittedly pretty new to Unreal Blueprints, so any help would be appreciated.
Alright, I figured it out.
Here's the setup needed if anyone else wants something similar.
For the player themselves, you'll need something like this:
The important thing is to center the root mesh where you want to rotate around. The spring arm's target arm length will be affected for the player mesh movement, giving the illusion you are physically moving the character. The second spring arm isn't necessary unless you wish to have more control over the camera to player distance.
For the rotation Blueprint, you'll need this:
The target is whatever you named the root mesh. (Mine was called Center) Drag and drop it from the hierarchy.
For the forward/backward movement, you'll need this:
The target is what you named the spring arm. (I left mine as the default "SpringArm") Again, just drag and drop it from the hierarchy.
Adjustments in Project Settings:
Yes, my inputs are backwards from what you'd think. I felt it was quicker just to reverse the inputs instead of adjusting whatever was causing the movement to be backwards in the first place. (It's probably just the sphere orientation.) Also, you'll notice I have the w and s inputs set to 5 or -5 instead of 1 or -1. This is due to the fact the movement was slow otherwise. I'm sure there's a fix that doesn't involve changing the input axis scale, but honestly I won't really have a reason to alter the values at any point in my project. If it ever comes up where I do need to, I'm sure there's a bypass to change the values from within blueprints anyways.
End result:
End Result Video
If I remember correctly, child actor components are a bit different from other components in that they are transformation independent, that is they do not update their transformation when their parent component moves around.
I find it a bit strange that you would separate your player actor and the camera component. Normally, the player "pawn" contains the mesh and camera components for one player.
I would suggest you do the following:
Create a player actor (e.g. a "pawn" or "character" class)
Create the following component hierarchy:
Root Scene -> Spring arm -> Skeletal or static mesh -> spring arm -> camera
Your root scene is the green center in your drawing. You can then basically use the blueprint you already have to rotate and move your player.
What is the point of a "pure" ARAnchor (that isn't an ARPlaneAnchor)?
Does ARKit ever automatically update/track or add/detect a pure ARAnchor? (Like how it does with ARPlaneAnchor?)
Why would anyone want to add a custom ARAnchor to ARKit, if they are not being tracked?
Can a pure ARAnchor ever be automatically "upgraded" to an ARPlaneAnchor by ARKit?
The main reason why you might use ARAnchor.....
ARAnchor is suitable for non-SceneKit engine (SpriteKit, Metal, etc), for SceneKit it is the same (since ARKit derived from SceneKit and has the same world coordinate)
If you're using ARSKView, you need a way to reference positions / orientations in 3D (real-world) space, because SpriteKit isn't 3D. You need ARAnchor to keep track of positions in 3D so that they can get mapped into 2D.
If you're building your own non-SceneKit engine with Metal (or GL)... that's not a 3D scene description API — it's a GPU programming API — so it doesn't really have a notion of world space. You can use ARAnchor as a bridge between ARKit's notion of world space and whatever you build.
What's the difference between using ARAnchor to insert a node and directly insert a node?
Is there a possibility with a google tango camera to create a situation, that my player goes on the table and if he comes out of the table he falls? Has anyone ever done anything similar and has references or ideas on how to do it?
in order to implement the functionality that you described, you will need to find different planes from the real world and translate their position into Unity scene. There is a class in Tango SDK, called TangoPointCloud which contains several methods for recognizing planes and translate their position into unity scene points. By knowing the positions of the table and the floor, you might be able to implement the feature you want. In my case, TangoPointCloud helped me find the walls from a room and their position relative to unity scene units.
In Unity, there is an event (Rendered.OnBecameVisible) that you can subscribe to to know when an object (in SceneKit's parlance, an SCNNode) becomes visible "by any camera".
Does Scenekit have an equivalent method/functionality/event subscription/or anything remotely similar I can use to achieve the same-ish functionality? (calling a method when an SCNNode is "visible" by a camera)
https://docs.unity3d.com/ScriptReference/Renderer.OnBecameVisible.html
There is no delegate method you could implement, but your SCNSceneRenderer, typically your SCNView, has a isNode(_:insideFrustumOf:) method:
Use this method to test whether a node lies within the viewing frustum defined by another node (which may or may not be the scene renderer’s current pointOfView node). For example, in a game scene containing multiple camera nodes, you could use this method to determine which camera is currently best for viewing a moving player character.
First, I just want to introduce to you guys my problem, because it is really complex so you need this to understand it properly.
I am trying to do something with Scene Kit and Swift : I want to reproduce what we can see in the TV Show Doctor Who where the Doctor's spaceship is bigger on the inside, as you can see in this video.
Of course the Scene Kit Framework doesn't support those kind of unreal dimensions so we need to do some sort of hackery to do achieve that.
Now let's talk about my idea in plain english
In fact, what we want to do is to display two completely different dimensions at the same place ; so I was thinking to :
A first dimension for the inside of the spaceship.
A second dimension for the outside of the spaceship.
Now, let's say that you are outside of the ship, you would be in the outside dimension, and in this outside dimension, my goal would be to display a portion of the inside dimension at the level of the door to give this effect where the camera is outside but where we can clearly see that the inside is bigger :
We would use an equivalent principle from the inside.
Now let's talk about the game logic :
I think that a good way to represent these dimensions would be two use two scenes.
We will call outsideScene the scene for the outside, and insideScene the scene for the inside.
So if we take again the picture, this would give this at the scene level :
To make it look realistic, the view of the inside needs to follow the movements of the outside camera, that's why I think that all the properties of these two cameras will be identical :
On the left is the outsideScene and on the right, the insideScene. I represent the camera field of view in orange.
If the outsideScene camera moves right, the insideScene camera will do exactly the same thing, if the outsideScene camera rotates, the insideScene camera will rotate in the same way... you get the principle.
So, my question is the following : what can I use to mask a certain portion of a certain scene (in this case the yellow zone in the outsideView) with what the camera of another view (the insideView) "sees" ?
First, I thought that I could simply get an NSImage from the insideScene and then put it as the texture of a surface in the outsideScene, but the problem would be that Scene Kit would compute it's perspective, lighting etc... so It would just look like we was displaying something on a screen and that's not what I want.
there is no super easy way to achieve this in SceneKit.
If your "inside scene" is static and can be baked into a cube map texture you can use shader modifiers and a technique called interior mapping (you can easily find examples on the web).
If you need a live, interactive "inside scene" you can use the sane technique but will have to render your scene in a texture first (or renderer your inside scene and outer scene one after the other with stencils). This can be done by leveraging SCNTechnique (new in Yosemite and iOS 8). On older versions you will have to write some OpenGL code in SCNSceneRenderer delegate methods.
I don't know if it's 'difficult'. As we have to in iOS , a lot of times the simplest answer ..is the simplest answer.
Maybe consider this:
Map a texture onto a cylinder sector prescribed by the geometry of the Tardis cube shape. Make sure the cylinder radius is equal of the focal point of the camera. Make sure you track the camera to the focal point.
The texture will be distorted because it is a cylinder making onto a cube. The actors' nodes in the Tardis will react properly to the camera but there should be two groups of light sources...One set for the Tardis and one outside the Tardis.