hitTest(_:options:) don't recognize nodes behind ARKit planes - swift

I place an object on a wall, then try to recognize tap on it, but hit test returns 0 objects. When I change Z position of the object and place it a little bit closer to cam, it's recognized well, but this isn't a solution, because planes are always changing and it can cover the object in any moment. How can I made hitTest work correctly and recognize my nodes behind planes? Or, maybe, I use the wrong method?
fileprivate func addNode(atPoint point: CGPoint) {
let hits = sceneView.hitTest(point, types: .existingPlaneUsingExtent)
if hits.count > 0, let firstHit = hits.first, let originNode = originNode {
let node = originNode.clone()
sceneView.scene.rootNode.addChildNode(node)
node.position = SCNVector3Make(firstHit.worldTransform.columns.3.x, firstHit.worldTransform.columns.3.y, firstHit.worldTransform.columns.3.z)
let resize = simd_float4x4(SCNMatrix4MakeScale(0.2, 0.2, 0.2))
let rotation = simd_float4x4(SCNMatrix4MakeRotation(.pi / 2, -1, 0, 0))
let transform = simd_mul(firstHit.worldTransform, resize)
let finalTransform = simd_mul(transform, rotation)
node.simdTransform = finalTransform
addedNodes.insert(node)
}
}
func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
guard let touch = touches.first else {
print("Unable to identify touches on any plane. Ignoring interaction...")
return
}
let touchPoint = touch.location(in: sceneView)
let hits = sceneView.hitTest(touchPoint, options: [SCNHitTestOption.boundingBoxOnly: true])
let filtered = hits.filter({ addedNodes.contains($0.node) })
print("\(hits.count) vs \(filtered.count), \(hits.first?.node.name ?? "no name")")
if let node = filtered.first?.node {
node.removeFromParentNode()
addedNodes.remove(node)
return
}
addPictureToPlane(atPoint: touchPoint)
}
addedNodes - set with added objects. When I added translating transform with changing Z coordinate at least on 0.05 (close to the camera) detecting working good. At least before plane changing and moving ahead the node.

I believe what you need to do is change your SCNHitTestSearchModeparameter which allows you to set:
Possible values for the searchMode option used with hit-testing
methods.
static let searchMode: SCNHitTestOption
Whereby:
The value for this key is an NSNumber object containing the raw
integer value of an SCNHitTestSearchMode constant.
From the Apple Docs there are three possible options you can use here:
case all
The hit test should return all possible results, sorted from nearest
to farthest.
case any
The hit test should return only the first object found, regardless of
distance.
case closest
The hit test should return only the closes object found.
Based on your question therefore, you would likely need to to utilise the all case.
As such your hitTest function would probably need to look something like this (remembering that self.augmentedRealityView refers to an ARSCNView):
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
//1. Get The Current Touch Location
guard let currentTouchLocation = touches.first?.location(in: self.augmentedRealityView) else { return }
//2. Perform An SCNHitTest Setting The SearchMode To 1 (All) Which Returns A List Of Results Sorted From Nearest To Farthest
if #available(iOS 11.0, *) {
let hitTestResults = self.augmentedRealityView.hitTest(currentTouchLocation, options: [SCNHitTestOption.searchMode: 1])
//3. Loop Through The Results & Get The Nodes
for index in 0..<hitTestResults.count{
let node = hitTestResults[index]
print(node)
}
}
}

Related

RealityKit – Why AnchorEntity position is always (x: 0, y: 0, z: 0)

I am using the following to to place an object on a plane. It works but the position of the anchor is always coming out to be 0,0,0 even though it is based on the raycast result.
let results = arView.raycast(from: tappedLocation,
allowing: .estimatedPlane,
alignment: .horizontal)
if let result = results.first {
print(result.worldTransform) // contains values
let anchor = AnchorEntity(raycastResult: result)
print(anchor.position) // why 0,0,0
}
Any ideas?
It seems that position becomes zero when anchor is nil (transform becomes identity matrix).
If we call raycast with .existingPlaneGeometry, anchor will not be nil.
AnchorEntity has world space position inside.
This is not included in transform.
If we want to know world space position, we can calculate it by the following code. (It seems that there is no property that can be obtained...)
let anchorWorldTransform = anchorEntity.convert(transform: .identity, to: nil)
Alternative approach
If .init(raycastResult:) doesn't work, use alternative approach with .init(world:) initializer.
fileprivate func raycaster() {
guard let query = arView.makeRaycastQuery(from: arView.center,
allowing: .existingPlaneInfinite,
alignment: .horizontal)
else { return }
if let result = arView.session.raycast(query).first {
let anchor = AnchorEntity(world: result.worldTransform)
anchor.addChild(self.yourModel)
arView.scene.anchors.append(anchor)
}
}
Then:
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
self.raycaster()
}
Modifying your code
Also, you may modify your code adding these lines:
let anchor = AnchorEntity(raycastResult: result)
let transform = Transform(matrix: result.worldTransform)
anchor.transform = transform

swift Mac OS: make trackpad reflect display size for NSTouch events

I am trying to make touch events take the trackpad as the display when moving it.
What I mean in this is:
Consider a Mac laptop display, and imagine it to be exactly the size of the trackpad, or so this is how I want the touch events to be implemented. There was a normalised x y coordinate property before in NSEvent which may have given me this, but it seems to have been deprecated.
Below are touchBegan and touchMoved overrides and what I mean:
override func touchesBegan(with event: NSEvent) {
CGDisplayMoveCursorToPoint(CGMainDisplayID(), event.location(in: overlayScene))
let player = SCNAudioPlayer(source: tick)
sineNode.addAudioPlayer(player)
}
override func touchesMoved(with event: NSEvent) {
let location = event.location(in: overlayScene)
guard let node = overlayScene.nodes(at: location).first else{
if currentSKNode != nil{
currentSKNode = nil
}
return
}
if currentSKNode != node{
currentSKNode = node
print(node.name)
let player = SCNAudioPlayer(source: tick)
sineNode.addAudioPlayer(player)
}
}
Imagine a fgraph reader where x and y axes are centred at exactly width/2 and height/2 of the trackpad. When I touch anywhere on it, this should reflect exactly on the apps window which is set to full screen.
Currently, the mouse cursor seems to go only partially across the display when I make a full left to right move, hence trying to reposition the mouse cursor to the position of the NSView, or in this case the SKScene.
Ok, so it turns out, as Kent mentioned, normalised position is not deprecated, but I was looking into NSEvent not NSTouch.
Here's the code for anyone who would stumble upon the same requirements. Works fine, just need to figure out now how to make this as responsive as possible.
override func touchesMoved(with event: NSEvent) {
DispatchQueue.main.async {
let touch = event.touches(matching: .moved, in: self.sceneView).first
let normalised = touch!.normalizedPosition
let translated = CGPoint(x: self.width*normalised.x - self.widthCenter, y: self.height*normalised.y - self.heightCenter)
guard let node = self.overlayScene.nodes(at: translated).first else{
self.currentSKNode = nil
return
}
if self.currentSKNode != node{
self.currentSKNode = node
let player = SCNAudioPlayer(source: self.tick)
self.sineNode.addAudioPlayer(player)
}
}
}

How to remove last placed child node?

I have touchesBegan discover and then place a node when a user taps the screen.
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
if let touchLocation = touches.first?.location(in: sceneView)
{
let hitTestResults = sceneView.hitTest(touchLocation, types: .featurePoint)
if let hitResult = hitTestResults.first
{
addDot(at: hitResult)
}
}
}
func addDot(at hitResult : ARHitTestResult)
{
let imagePlane = SCNPlane(width: 0.1, height: 0.1)
imagePlane.firstMaterial?.diffuse.contents = UIImage(named: "helmet_R.png")
let planeNode = SCNNode(geometry: imagePlane)
planeNode.position = SCNVector3(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
planeNode.constraints = [SCNBillboardConstraint()]
sceneView.scene.rootNode.addChildNode(planeNode)
However, I'm confused as to removing the last placed node in the view.
I attempted a simple
planeNode.removeFromParentNode()
but that did not achieve the results I was hoping for. Therefore, how would I achieve the desired result? I'm aware that removing the parent node is arbitrary, but i'm unsure how to call the last child node placed.
You should create a reference to the last added node in your class.
var lastNode: SCNNode?
Then, let's say for example you want to remove the last node everytime you add a new one. In this case, you would create and add a new one, remove the lastNode and assign your newly created node as the last added node. So in your case, in the method addDot:
sceneView.scene.rootNode.addChildNode(planeNode)
lastNode?.removeFromParentNode()
lastNode = planeNode

TouchesMoved Lag with Pathfinding

I'm making a game where the player follows a touch. The game has obstacles and pathfinding, and I'm experiencing extreme lag with touches moved. Upon loading the game, everything works, but after a few seconds of dragging my finger around (especially around obstacles) the game freezes to the point where moving a touch will freeze all physics (0 fps) until the touch is ended.
I'm assuming the culprit is my function makeGraph(), which finds a path for the player from player.position:CGPoint() to player.destination:CGPoint(), storing a path in player.goto:[CGPoint()], the update function then takes care of moving to the next goto point.
This function is called every cycle of touchesmoved, so the player can switch directions if the finger moves over an obstacle
My question is: 1) what in this code is causing the unbearable lag, 2) how can I make this code more efficient?
My code:
initializing vars:
var obstacles = [GKPolygonObstacle(points: [float2()])]
var navgraph = GKObstacleGraph()
setting up a graph (called at start of level):
func setupGraph(){
var wallnodes :[SKSpriteNode] = [SKSpriteNode]()
self.enumerateChildNodes(withName: "wall") {
node, stop in
wallnodes.append(node as! SKSpriteNode)
}
self.enumerateChildNodes(withName: "crate") {
node, stop in
wallnodes.append(node as! SKSpriteNode)
}
obstacles = SKNode.obstacles(fromNodeBounds: wallnodes)
navgraph = GKObstacleGraph(obstacles: obstacles, bufferRadius: Float(gridSize/2))
}
touchesmoved: I keep track of multiple touches using strings. only one finger may act as the "MoveFinger." this string is created at touchesbegan and emptied at touchesended
override func touchesMoved(_ touches: Set<UITouch>, with event: UIEvent?) {
touchloop: for touch in touches {
if MoveFinger == String(format: "%p", touch) {
let location = touch.location(in: self)
player.destination = location
makeGraph()
player.moving = true
player.UpdateMoveMode()
continue touchloop
}
// .... more if statements ....
}
}
and graph function (called every touchesmoved cycle):
func makeGraph(){
let startNode = GKGraphNode2D(point: float2(Float(player.position.x), Float(player.position.y)))
let endNode = GKGraphNode2D(point: float2(Float(player.destination.x), Float(player.destination.y)))
let graphcopy = navgraph
graphcopy.connectUsingObstacles(node: startNode)
graphcopy.connectUsingObstacles(node: endNode)
let path = graphcopy.findPath(from: startNode, to: endNode)
player.goto = []
for node:GKGraphNode in path {
if let point2d = node as? GKGraphNode2D {
let point = CGPoint(x: CGFloat(point2d.position.x), y: CGFloat(point2d.position.y))
player.goto.append(point)
}
}
if player.goto.count == 0 {
//if path is empty, go straight to destination
player.goto = [player.destination]
} else {
//if path is not empty, remove first point (start point)
player.goto.remove(at: 0)
}
}
any ideas?

How to make SKSpriteNode with fixed position (after adding physics)?

I have several programmatically made SKSpriteNode's. Some of them I want to move around, some I want to be static (have a fixed position). When adding physics to the nodes (need that to be able to do collision detection, right?) and set physicsBodyXXXX.dynamic = false they stay in the same position when moving other object over them. That's fine!
But, I'm still able to grab the node I want to be statically positioned, and move them around. How can I mask out the node I don't want to move in touches function? Or is there another solution?
Tried to find a property like static which made the node's position fixed, but can't find it...
Here's my code for auto generating nodes (in override func didMoveToView(view: SKView):
for Character in englishWord{
// Make letters:
let letterToMove = SKSpriteNode(imageNamed: "\(Character)")
//then setting size and position
var physicsBodyLetterToMove = SKPhysicsBody(rectangleOfSize: letterToMove.size)
physicsBodyLetterToMove.affectedByGravity = false
physicsBodyLetterToMove.allowsRotation = false
physicsBodyLetterToMove.dynamic = false
letterToMove.physicsBody = physicsBodyLetterToMove
self.addChild(letterToMove)
// Make empty boxes for the letters:
let letterRecBox = SKSpriteNode(imageNamed: "EmptyBox")
//then setting size and position
var physicsBodyLetterRecBox = SKPhysicsBody(rectangleOfSize: letterRecBox.size)
physicsBodyLetterToMove.affectedByGravity = false
physicsBodyLetterRecBox.dynamic = false
letterRecBox.physicsBody = physicsBodyLetterRecBox
self.addChild(letterRecBox)
}
So the touches func's:
var selected: [UITouch: SKNode] = [:]
override func touchesBegan(touches: NSSet, withEvent event: UIEvent) {
/* Called when a touch begins */
selected = [:]
for touch: AnyObject in touches {
let location = touch.locationInNode(self)
selected[touch as UITouch] = nodeAtPoint(location)
}
}
override func touchesMoved(touches: NSSet, withEvent event: UIEvent) {
for touch: AnyObject in touches {
let location = touch.locationInNode(self)
for (touch, node) in selected{
if !contains([self], node){
let action = SKAction.moveTo(location, duration: 0.1)
node.runAction(SKAction.repeatAction(action, count: 1))
}
}
}
}
Any idea?
Setting dynamic to false will make the node unaffected by physics. SKActions and touch events are not considered physics so they will still affect your nodes that are not dynamic.
You could do something like:
YourSpriteNode.name = #"staticNode"; //Right after you create it
Then alter your touch method:
for (touch, node) in selected{
if !contains([self], node){
if(![node.name isEqualToString:#"staticNode"])
{
let action = SKAction.moveTo(location, duration: 0.1)
node.runAction(SKAction.repeatAction(action, count: 1))
}
}
}
The newly introduced if statement will prevent any node named "staticNode" from getting moved due to your SKActions. Other nodes will move as expected.
Thanx, that worked.
Only, I had to write it like this:
letterRecBox.name = "staticNode"
...
if !(node.name == "staticNode"){
...then do something...
}