Add SCNNode after rotating rootNode - swift

I'm trying to add a node (a sphere) to a body model but it doesn't work properly after I rotate the model through a pan gesture.
Here's how I'm adding the node (using a long tap gesture):
func addSphere(sender: UILongPressGestureRecognizer) {
switch sender.state {
case .Began:
let location = sender.locationInView(bodyView)
let hitResults = bodyView.hitTest(location, options: nil)
if hitResults.count > 0 {
let result = hitResults.first!
let secondSphereGeometry = SCNSphere(radius: 0.015)
secondSphereGeometry.firstMaterial?.diffuse.contents = UIColor.redColor()
let secondSphereNode = SCNNode(geometry: secondSphereGeometry)
let vpWithZ = SCNVector3(x: Float(result.worldCoordinates.x), y: Float(result.worldCoordinates.y), z: Float( result.worldCoordinates.z))
secondSphereNode.position = vpWithZ
bodyView.scene!.rootNode.addChildNode(secondSphereNode)
}
break
default:
break
}
}
Here is how I rotate the view:
func rotateGesture(sender: UIPanGestureRecognizer) {
let translation = sender.translationInView(sender.view)
var newZAngle = (Float)(translation.x)*(Float)(M_PI)/180.0
newZAngle += currentZAngle
bodyView.scene!.rootNode.transform = SCNMatrix4MakeRotation(newZAngle, 0, 0, 1)
if sender.state == .Ended {
currentZAngle = newZAngle
}
}
And to load the 3D model I just do:
bodyView.scene = SCNScene(named: "male_body.dae") // bodyView is a SCNView in the storyboard
I found something related to the worldTransform property and also the function convertPosition:toNode: but couldn't find an example that works well.
The problem is that, if I rotate the model, the sphere are not positioned properly. They're always positioned as if the model was in its initial state.
If I turn the body and add long tap his arm (on the side), the sphere is added somewhere floating in front of the body, as you can see above.
I don't know how to fix this. Appreciate if someone can help me. Thanks!

Related

How to move and rotate SCNode using ARKit and Gesture Recognizer?

I am working on an AR based iOS app using ARKit(SceneKit). I used the Apple sample code https://developer.apple.com/documentation/arkit/handling_3d_interaction_and_ui_controls_in_augmented_reality as base for this. Using this i am able to move or rotate the whole Virtual Object.
But i want to select and move/rotate a Child Node in Virtual object using user finger, similar to how we move/rotate the whole Virtual Object itself.
I tried the below two links but it is only moving the child node in particular axis but not freely moving anywhere as the user moves the finger.
ARKit - Drag a node along a specific axis (not on a plane)
Dragging SCNNode in ARKit Using SceneKit
Also i tried replacing the Virtual Object which is a SCNReferenceNode with SCNode so that whatever functionality present for existing Virtual Object applies to Child Node as well, it is not working.
Can anyone please help me on how to freely move/rotate not only the Virtual Object but also the child node of a Virtual Object?
Please find the code i am currently using below,
let tapPoint: CGPoint = gesture.location(in: sceneView)
let result = sceneView.hitTest(tapPoint, options: nil)
if result.count == 0 {
return
}
let scnHitResult: SCNHitTestResult? = result.first
movedObject = scnHitResult?.node //.parent?.parent
let hitResults = self.sceneView.hitTest(tapPoint, types: .existingPlane)
if !hitResults.isEmpty{
guard let hitResult = hitResults.last else { return }
movedObject?.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
}
To move an object:
Perform a hitTest to check where you have touched, and detect which plane you touched and get a position. Move your SCNNode to that position by changing the node.position value with an SCNVector3.
Code:
#objc func panDetected(recognizer: UIPanGestureRecognizer){
let hitResult = self.arSceneView.hitTest(loc, types: .existingPlane)
if !hitResult.isEmpty{
guard let hitResult = hitResult.last else { return }
self.yourNode.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
}
The above code is enough to move your node over a detected plane, anywhere you touch, and not just in a single axis.
Rotating a node according to your gesture is a very difficult task and I have worked on a solution for quite sometime, never reaching a perfect output.
But, I have come across this repository in GitHub which allows you to do just that with a very impressive result.
https://github.com/Xartec/ScreenSpaceRotationAndPan
The Swift version of the code you require to rotate your node using your gesture would be :
var previousLoc: CGPoint?
var touchCount: Int?
#objc func panDetected(recognizer: UIPanGestureRecognizer){
let loc = recognizer.location(in: self.view)
var delta = recognizer.translation(in: self.view)
if recognizer.state == .began {
previousLoc = loc
touchCount = recognizer.numberOfTouches
}
else if gestureRecognizer.state == .changed {
delta = CGPoint.init(x: 2 * (loc.x - previousLoc.x), y: 2 * (loc.y - previousLoc.y))
previousLoc = loc
if touchCount != recognizer.numberOfTouches {
return
}
var rotMatrix: SCNMatrix4!
let rotX = SCNMatrix4Rotate(SCNMatrix4Identity, Float((1.0/100) * delta.y), 1, 0, 0)
let rotY = SCNMatrix4Rotate(SCNMatrix4Identity, Float((1.0 / 100) * delta.x), 0, 1, 0)
rotMatrix = SCNMatrix4Mult(rotX, rotY)
let transMatrix = SCNMatrix4MakeTranslation(yourNode.position.x, yourNode.position.y, yourNode.position.z)
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, SCNMatrix4Invert(transMatrix))
let parentNoderanslationMatrix = SCNMatrix4MakeTranslation((self.yourNode.parent?.worldPosition.x)!, (self.yourNode.parent?.worldPosition.y)!, (self.yourNode.parent?.worldPosition.z)!)
let parentNodeMatWOTrans = SCNMatrix4Mult((self.yourNode.parent?.worldTransform)!, SCNMatrix4Invert(parentNoderanslationMatrix))
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, parentNodeMatWOTrans)
let camorbitNodeTransMat = SCNMatrix4MakeTranslation((self.arSceneView.pointOfView?.worldPosition.x)!, (self.arSceneView.pointOfView?.worldPosition.y)!, (self.arSceneView.pointOfView?.worldPosition.z)!)
let camorbitNodeMatWOTrans = SCNMatrix4Mult((self.arSceneView.pointOfView?.worldTransform)!, SCNMatrix4Invert(camorbitNodeTransMat))
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, SCNMatrix4Invert(camorbitNodeMatWOTrans))
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, rotMatrix)
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, camorbitNodeMatWOTrans)
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, SCNMatrix4Invert(parentNodeMatWOTrans))
self.yourNode.transform = SCNMatrix4Mult(self.yourNode.transform, transMatrix)
}
}

iOS 11 PDFKit not updating annotation position

I'm building an app to edit PDFs on iPad.
I'm trying to implement the dragging of the annotations with a panGesture recognizer that I added to the superview of the PDFView. The problem is that the new rect bounds of the annotation gets assigned but the changes doesn't reflect on screen.
Here is my code:
#objc func handlePanGesture(panGesture: UIPanGestureRecognizer) {
let touchLocation = panGesture.location(in: pdfView)
guard let page = pdfView.page(for: touchLocation, nearest: true) else {
return
}
let locationOnPage = pdfView.convert(touchLocation, to: page)
switch panGesture.state {
case .began:
guard let annotation = page.annotation(at: locationOnPage) else {
return
}
currentlySelectedAnnotation = annotation
case .changed:
guard let annotation = currentlySelectedAnnotation else {
return
}
let initialBounds = annotation.bounds
annotation.bounds = CGRect(origin: locationOnPage,
size: initialBounds.size)
print("move to \(locationOnPage)")
case .ended, .cancelled, .failed:
break
default:
break
}
}
Hope you can help me.
Well, Since nobody replied. I think there is a bug in the framework, so I'll post what worked for me, after some time of trial and error.
let initialBounds = annotation.bounds
annotation.bounds = CGRect(
origin: locationOnPage,
size: initialBounds.size)
page.removeAnnotation(annotation)
page.addAnnotation(annotation)
It's not elegant, but it does the job
Using a Bezierpath the whole bezierpath moves on bounds change.
PDF's builtin line type does not get moved with changing bounds though, so have to set startPoint and endPoint at every change.
I added a line to your code so that when dragging, it puts the annotation center where your finger is dragging
case .changed:
guard let annotation = currentlySelectedAnnotation else {
return
}
let initialBounds = annotation.bounds
// Set the center of the annotation to the spot of our finger
annotation.bounds = CGRect(x: locationOnPage.x - (initialBounds.width / 2), y: locationOnPage.y - (initialBounds.height / 2), width: initialBounds.width, height: initialBounds.height)
print("move to \(locationOnPage)")
case .ended, .cancelled, .failed:
currentlySelectedAnnotation = nil
default:
break
}
}

ARKit dragging object changes scale

İ am trying to move the SCNNode object which i placed on to a surface. İt moves but the scale changes and it becomes smaller, when i first start to move.
This is what i did;
#IBAction func dragBanana(_ sender: UIPanGestureRecognizer) {
guard let _ = self.sceneView.session.currentFrame else {return}
if(sender.state == .began) {
let location = sender.location(in: self.sceneView)
let hitTestResult = sceneView.hitTest(location, options: nil)
if !hitTestResult.isEmpty {
guard let hitResult = hitTestResult.first else {return}
movedObject = hitResult.node
}
}
if (sender.state == .changed) {
if(movedObject != nil) {
let location = sender.location(in: self.sceneView)
let hitTestResult = sceneView.hitTest(location, types: .existingPlaneUsingExtent)
guard let hitResult = hitTestResult.first else {return}
let matrix = SCNMatrix4(hitResult.worldTransform)
let vector = SCNVector3Make(matrix.m41, matrix.m42, matrix.m43)
movedObject?.position = vector
}
}
if (sender.state == .ended) {
movedObject = nil
}
}
My answer is probably very late, but I faced this issue myself and it took me a while to kind of figure out why this might happen. I'll share my experience and maybe you can relate to it.
My problem was that I was trying to change the position of the node after changing its scale at runtime (most of my 3D assets were very large when added, I scale them down with a pinch gesture). I noticed changing the scale was the cause of the position change not working as expected.
I found a very simple solution to this. You simply need to change this line:
movedObject?.position = vector
to this:
movedObject?.worldPosition = vector
According to SCNNode documentation, the position property determines the position of the node relative to its parent. While worldPosition is the position of the node relative to the scene's root node (i.e. the world origin of ARSCNView)
I hope this answers your question.
It's because you're moving the object on the 3 axis and Z changes that's why it feels like it scales but it's only getting closer to you.

ARKit – How to put 3D Object on QRCode?

I'm actually trying to put a 3D Object on QRCode with ARKit
For that I use a AVCaptureDevice to detect a QRCode and establish the area of the QRCode that gives me a CGRect.
Then, I make a hitTest on every point of the CGRect to get the average 3D coordinates like so :
positionGiven = SCNVector3(0, 0, 0)
for column in Int(qrZone.origin.x)...2*Int(qrZone.origin.x + qrZone.width) {
for row in Int(qrZone.origin.y)...2*Int(qrZone.origin.y + qrZone.height) {
for result in sceneView.hitTest(CGPoint(x: CGFloat(column)/2,y:CGFloat(row)/2), types: [.existingPlaneUsingExtent,.featurePoint]) {
positionGiven.x+=result.worldTransform.columns.3.x
positionGiven.y+=result.worldTransform.columns.3.y
positionGiven.z+=result.worldTransform.columns.3.z
cpts += 1
}
}
}
positionGiven.x=positionGiven.x/cpts
positionGiven.y=positionGiven.y/cpts
positionGiven.z=positionGiven.z/cpts
But the hitTest doesn't detect any result and freeze the camera while when I make a hitTest with a touch on screen it works.
Do you have any idea why it's not working ?
Do you have an other idea that can help me to achieve what I want to do ?
I already thought about 3D translation with CoreMotion that can give me the tilt of the device but that seems really tedious.
I also heard about ARWorldAlignmentCamera that can locked the scene coordinate to match the orientation of the camera but I don't know how to use it !
Edit : I try to move my 3D Object every time I touch the screen and the hitTest is positive, and it's pretty accurate ! I really don't understand why hitTest on an area of pixels doesn't work...
Edit 2 : Here is the code of the hitTest who works with 2-5 touches on the screen:
#objc func touch(sender : UITapGestureRecognizer) {
for result in sceneView.hitTest(CGPoint(x: sender.location(in: view).x,y: sender.location(in: view).y), types: [.existingPlaneUsingExtent,.featurePoint]) {
//Pop up message for testing
alert("\(sender.location(in: view))", message: "\(result.worldTransform.columns.3)")
//Moving the 3D Object to the new coordinates
let objectList = sceneView.scene.rootNode.childNodes
for object : SCNNode in objectList {
object.removeFromParentNode()
}
addObject(SCNVector3(result.worldTransform.columns.3.x,result.worldTransform.columns.3.y,result.worldTransform.columns.3.z))
}
}
Edit 3 :
I manage to resolve my problem partially.
I take the transform matrix of the camera (session.currentFrame.camera.transform) so that the object is in front of the camera.
Then I apply a translation on (x,y) with the position of the CGRect.
However i can't translate the z-axis because i don't have enough informations.
And I will probably need a estimation of z coordinate like the hitTest do..
Thanks in advance ! :)
You could use Apple's Vision API to detect the QR code and place an anchor.
To start detecting QR codes, use:
var qrRequests = [VNRequest]()
var detectedDataAnchor: ARAnchor?
var processing = false
func startQrCodeDetection() {
// Create a Barcode Detection Request
let request = VNDetectBarcodesRequest(completionHandler: self.requestHandler)
// Set it to recognize QR code only
request.symbologies = [.QR]
self.qrRequests = [request]
}
In ARSession's didUpdate Frame
public func session(_ session: ARSession, didUpdate frame: ARFrame) {
DispatchQueue.global(qos: .userInitiated).async {
do {
if self.processing {
return
}
self.processing = true
// Create a request handler using the captured image from the ARFrame
let imageRequestHandler = VNImageRequestHandler(cvPixelBuffer: frame.capturedImage,
options: [:])
// Process the request
try imageRequestHandler.perform(self.qrRequests)
} catch {
}
}
}
Handle the Vision QR request and trigger the hit test
func requestHandler(request: VNRequest, error: Error?) {
// Get the first result out of the results, if there are any
if let results = request.results, let result = results.first as? VNBarcodeObservation {
guard let payload = result.payloadStringValue else {return}
// Get the bounding box for the bar code and find the center
var rect = result.boundingBox
// Flip coordinates
rect = rect.applying(CGAffineTransform(scaleX: 1, y: -1))
rect = rect.applying(CGAffineTransform(translationX: 0, y: 1))
// Get center
let center = CGPoint(x: rect.midX, y: rect.midY)
DispatchQueue.main.async {
self.hitTestQrCode(center: center)
self.processing = false
}
} else {
self.processing = false
}
}
func hitTestQrCode(center: CGPoint) {
if let hitTestResults = self.latestFrame?.hitTest(center, types: [.featurePoint] ),
let hitTestResult = hitTestResults.first {
if let detectedDataAnchor = self.detectedDataAnchor,
let node = self.sceneView.node(for: detectedDataAnchor) {
let previousQrPosition = node.position
node.transform = SCNMatrix4(hitTestResult.worldTransform)
} else {
// Create an anchor. The node will be created in delegate methods
self.detectedDataAnchor = ARAnchor(transform: hitTestResult.worldTransform)
self.sceneView.session.add(anchor: self.detectedDataAnchor!)
}
}
}
Then handle adding the node when the anchor is added.
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
// If this is our anchor, create a node
if self.detectedDataAnchor?.identifier == anchor.identifier {
let sphere = SCNSphere(radius: 1.0)
sphere.firstMaterial?.diffuse.contents = UIColor.redColor()
let sphereNode = SCNNode(geometry: sphere)
sphereNode.transform = SCNMatrix4(anchor.transform)
return sphereNode
}
return nil
}
Source

Agent following path moves forward indefinitely

I am trying to make a sprite move towards a point while avoiding some obstacles. The graph that I am using is a GKObstacleGraph, obtained from this scene:
For simplicity I decided not to use a circular physics body for the obstacles, but it's enough to use the sprites' bounds for now. So this is how I created the graph:
lazy var graph:GKObstacleGraph? =
{
guard let spaceship = self.spaceship else { return nil }
let obstacles = SKNode.obstacles(fromNodeBounds: self.rocks)
let graph = GKObstacleGraph(obstacles: obstacles, bufferRadius: Float(spaceship.size.width))
return graph
}()
When the user taps on any location of the scene, the spaceship should start moving toward that position by avoiding the obstacles:
func tap(locationInScene location:CGPoint)
{
guard let graph = self.graph else { return }
guard let spaceship = self.spaceship else { return }
let startNode = GKGraphNode2D(point: vector_float2(withPoint: spaceship.position))
let endNode = GKGraphNode2D(point: vector_float2(withPoint: location))
graph.connectUsingObstacles(node: startNode)
graph.connectUsingObstacles(node: endNode)
let path = graph.findPath(from: startNode, to: endNode)
print(path)
let goal = GKGoal(toFollow: GKPath(graphNodes: path, radius: Float(spaceship.size.width)) , maxPredictionTime: 1.0, forward: true)
let behavior = GKBehavior(goal: goal, weight: 1.0)
let agent = GKAgent2D()
agent.behavior = behavior
graph.remove([startNode, endNode])
spaceship.entity?.addComponent(agent)
// This is necessary. I don't know why, but if I don't do that, I
// end up having a duplicate entity in my scene's entities array.
self.entities = [spaceship.entity!]
}
But when I tap on a point on the scene, the spaceship starts moving indefinitely upwards. I tried to print the position of the GKAgent at every frame, and this is what I got:
With very low values, the spaceship still keeps moving upwards without stopping, ever.
This is my project on GitHub.