arkit anchor or node visible in camera and sitting to left or right of frustum - swift

Question

How can i detect if an ARAnchor is currently visible in the camera, i need to test when the camera view changes.
I want to put arrows on the edge of the screen that point in the direction of the anchor when not on screen. I need to know if the node sits to the left or right of the frustum.
I am now doing this but it says pin is visible when it is not and the X values seem not right? Maybe the renderer frustum does not match the screen camera?
var deltaTime = TimeInterval()
public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
deltaTime = time - lastUpdateTime
if deltaTime>1{
if let annotation = annotationsByNode.first {
let node = annotation.key.childNodes[0]
if !renderer.isNode(node, insideFrustumOf: renderer.pointOfView!)
{
print("Pin is not visible");
}else {
print("Pin is visible");
}
let pnt = renderer.projectPoint(node.position)
print("pos ", pnt.x, " ", renderer.pointOfView!.position)
}
lastUpdateTime = time
}
}
Update: The code works to show if node is visible or not, how can i tell which direction left or right a node is in relation to the camera frustum?
update2! as suggested answer from Bhanu Birani
let screenWidth = UIScreen.main.bounds.width
let screenHeight = UIScreen.main.bounds.height
let leftPoint = CGPoint(x: 0, y: screenHeight/2)
let rightPoint = CGPoint(x: screenWidth,y: screenHeight/2)
let leftWorldPos = renderer.unprojectPoint(SCNVector3(leftPoint.x,leftPoint.y,0))
let rightWorldPos = renderer.unprojectPoint(SCNVector3(rightPoint.x,rightPoint.y,0))
let distanceLeft = node.position - leftWorldPos
let distanceRight = node.position - rightWorldPos
let dir = (isVisible) ? "visible" : ( (distanceLeft.x<distanceRight.x) ? "left" : "right")
I got it working finally which uses the idea from Bhanu Birani of the left and right of the screen but i get the world position differently, unProjectPoint and also get a scalar value of distance which i compare to get the left/right direction. Maybe there is a better way of doing it but it worked for me
public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
deltaTime = time - lastUpdateTime
if deltaTime>0.25{
if let annotation = annotationsByNode.first {
guard let pointOfView = renderer.pointOfView else {return}
let node = annotation.key.childNodes[0]
let isVisible = renderer.isNode(node, insideFrustumOf: pointOfView)
let screenWidth = UIScreen.main.bounds.width
let screenHeight = UIScreen.main.bounds.height
let leftPoint = CGPoint(x: 0, y: screenHeight/2)
let rightPoint = CGPoint(x: screenWidth,y: screenHeight/2)
let leftWorldPos = renderer.unprojectPoint(SCNVector3(leftPoint.x, leftPoint.y,0))
let rightWorldPos = renderer.unprojectPoint(SCNVector3(rightPoint.x, rightPoint.y,0))
let distanceLeft = node.worldPosition.distance(vector: leftWorldPos)
let distanceRight = node.worldPosition.distance(vector: rightWorldPos)
//let pnt = renderer.projectPoint(node.worldPosition)
//guard let pnt = renderer.pointOfView!.convertPosition(node.position, to: nil) else {return}
let dir = (isVisible) ? "visible" : ( (distanceLeft<distanceRight) ? "left" : "right")
print("dir" , dir, " ", leftWorldPos , " ", rightWorldPos)
lastDir=dir
delegate?.nodePosition?(node:node, pos: dir)
}else {
delegate?.nodePosition?(node:nil, pos: lastDir )
}
lastUpdateTime = time
}
extension SCNVector3
{
/**
* Returns the length (magnitude) of the vector described by the SCNVector3
*/
func length() -> Float {
return sqrtf(x*x + y*y + z*z)
}
/**
* Calculates the distance between two SCNVector3. Pythagoras!
*/
func distance(vector: SCNVector3) -> Float {
return (self - vector).length()
}
}

Answer 1

Project the ray from the from the following screen positions:
leftPoint = CGPoint(0, screenHeight/2) (centre left of the screen)
rightPoint = CGPoint(screenWidth, screenHeight/2) (centre right of the screen)
Convert CGPoint to world position:
leftWorldPos = convertCGPointToWorldPosition(leftPoint)
rightWorldPos = convertCGPointToWorldPosition(rightPoint)
Calculate the distance of node from both world position:
distanceLeft = node.position - leftWorldPos
distanceRight = node.position - rightWorldPos
Compare distance to find the shortest distance to the node. Use the shortest distance vector to position direction arrow for object.
Here is the code from tsukimi to check if the object is in right side of screen or on left side:
public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
deltaTime = time - lastUpdateTime
if deltaTime>0.25{
if let annotation = annotationsByNode.first {
guard let pointOfView = renderer.pointOfView else {return}
let node = annotation.key.childNodes[0]
let isVisible = renderer.isNode(node, insideFrustumOf: pointOfView)
let screenWidth = UIScreen.main.bounds.width
let screenHeight = UIScreen.main.bounds.height
let leftPoint = CGPoint(x: 0, y: screenHeight/2)
let rightPoint = CGPoint(x: screenWidth,y: screenHeight/2)
let leftWorldPos = renderer.unprojectPoint(SCNVector3(leftPoint.x, leftPoint.y,0))
let rightWorldPos = renderer.unprojectPoint(SCNVector3(rightPoint.x, rightPoint.y,0))
let distanceLeft = node.worldPosition.distance(vector: leftWorldPos)
let distanceRight = node.worldPosition.distance(vector: rightWorldPos)
//let pnt = renderer.projectPoint(node.worldPosition)
//guard let pnt = renderer.pointOfView!.convertPosition(node.position, to: nil) else {return}
let dir = (isVisible) ? "visible" : ( (distanceLeft<distanceRight) ? "left" : "right")
print("dir" , dir, " ", leftWorldPos , " ", rightWorldPos)
lastDir=dir
delegate?.nodePosition?(node:node, pos: dir)
}else {
delegate?.nodePosition?(node:nil, pos: lastDir )
}
lastUpdateTime = time
}
Following is the class to help performing operations on vector
extension SCNVector3 {
init(_ vec: vector_float3) {
self.x = vec.x
self.y = vec.y
self.z = vec.z
}
func length() -> Float {
return sqrtf(x * x + y * y + z * z)
}
mutating func setLength(_ length: Float) {
self.normalize()
self *= length
}
mutating func setMaximumLength(_ maxLength: Float) {
if self.length() <= maxLength {
return
} else {
self.normalize()
self *= maxLength
}
}
mutating func normalize() {
self = self.normalized()
}
func normalized() -> SCNVector3 {
if self.length() == 0 {
return self
}
return self / self.length()
}
static func positionFromTransform(_ transform: matrix_float4x4) -> SCNVector3 {
return SCNVector3Make(transform.columns.3.x, transform.columns.3.y, transform.columns.3.z)
}
func friendlyString() -> String {
return "(\(String(format: "%.2f", x)), \(String(format: "%.2f", y)), \(String(format: "%.2f", z)))"
}
func dot(_ vec: SCNVector3) -> Float {
return (self.x * vec.x) + (self.y * vec.y) + (self.z * vec.z)
}
func cross(_ vec: SCNVector3) -> SCNVector3 {
return SCNVector3(self.y * vec.z - self.z * vec.y, self.z * vec.x - self.x * vec.z, self.x * vec.y - self.y * vec.x)
}
}
extension SCNVector3{
func distance(receiver:SCNVector3) -> Float{
let xd = receiver.x - self.x
let yd = receiver.y - self.y
let zd = receiver.z - self.z
let distance = Float(sqrt(xd * xd + yd * yd + zd * zd))
if (distance < 0){
return (distance * -1)
} else {
return (distance)
}
}
}
Here is the code snippet to convert tap location or any CGPoint to world transform.
#objc func handleTap(_ sender: UITapGestureRecognizer) {
// Take the screen space tap coordinates and pass them to the hitTest method on the ARSCNView instance
let tapPoint = sender.location(in: sceneView)
let result = sceneView.hitTest(tapPoint, types: ARHitTestResult.ResultType.existingPlaneUsingExtent)
// If the intersection ray passes through any plane geometry they will be returned, with the planes
// ordered by distance from the camera
if (result.count > 0) {
// If there are multiple hits, just pick the closest plane
if let hitResult = result.first {
let finalPosition = SCNVector3Make(hitResult.worldTransform.columns.3.x + insertionXOffset,
hitResult.worldTransform.columns.3.y + insertionYOffset,
hitResult.worldTransform.columns.3.z + insertionZOffset
);
}
}
}
Following is the code to get hit test results when there's no plane found.
// check what nodes are tapped
let p = gestureRecognize.location(in: scnView)
let hitResults = scnView.hitTest(p, options: [:])
// check that we clicked on at least one object
if hitResults.count > 0 {
// retrieved the first clicked object
let result = hitResults[0]
}

Answer 2

This answer is a bit late but can be useful for someone needing to know where a node is in camera space relatively to the center (e.g. top left corner, centered ...).
You can get your node position in camera space using scene.rootNode.convertPosition(node.position, to: pointOfView).
In camera space,
(isVisible && (x=0, y=0)) means that your node is in front of the camera.
(isVisible && (x=0.1)) means that the node is a little bit on the right.
Some sample code :
public func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
deltaTime = time - lastUpdateTime
if deltaTime>0.25{
if let annotation = annotationsByNode.first {
guard let pointOfView = renderer.pointOfView else {return}
let node = annotation.key.childNodes[0]
let isVisible = renderer.isNode(node, insideFrustumOf: pointOfView)
// Translate node to camera space
let nodeInCameraSpace = scene.rootNode.convertPosition(node.position, to: pointOfView)
let isCentered = isVisible && (nodeInCameraSpace.x < 0.1) && (nodeInCameraSpace.y < 0.1)
let isOnTheRight = isVisible && (nodeInCameraSpace.x > 0.1)
// ...
delegate?.nodePosition?(node:node, pos: dir)
}else {
delegate?.nodePosition?(node:nil, pos: lastDir )
}
lastUpdateTime = time
}