Creating a box with SceneKit and ARKit - swift

I am trying to create a primitive with SceneKit and ARKit. For whatever reason, it is not working.
let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
let node = SCNNode(geometry: box)
node.position = SCNVector3(0,0,0)
sceneView.scene.rootNode.addChildNode(node)
Do I need to take in the camera coordinates as well?

Your code looks good and it should work. I have tried it as the below code: after creating a new app with ARKit template, I have replaced the function viewDidLoad.
override func viewDidLoad() {
super.viewDidLoad()
// Set the view's delegate
sceneView.delegate = self
let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
let node = SCNNode(geometry: box)
node.position = SCNVector3(0,0,0)
sceneView.scene.rootNode.addChildNode(node)
}
It creates a box at the original point (0, 0, 0). Unfortunately your device is inside the box thus you cannot see that box straightly. To see the box, move your device far aways a bit.
The attached image is the box after moving my device:
If you want to see it immediately, move the box to front a bit, add colour and make the first material be double side (to see it even in or out side):
let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
box.firstMaterial?.diffuse.contents = UIColor.red
box.firstMaterial?.isDoubleSided = true
let boxNode = SCNNode(geometry: box)
boxNode.position = SCNVector3(0, 0, -1)
sceneView.scene.rootNode.addChildNode(boxNode)

You should get the location tapped and use the world coordinates to place the cube properly. I'm not sure (0,0,0) is a normal location for ARKit. You can try something like this:
Put this in your viewDidLoad:
let tapGestureRecognizer = UITapGestureRecognizer(target: self, action: #selector(handleTapFrom))
tapGestureRecognizer.numberOfTapsRequired = 1
self.sceneView.addGestureRecognizer(tapGestureRecognizer)
Then add this method:
#objc func handleTapFrom(recognizer: UITapGestureRecognizer) {
let tapPoint = recognizer.location(in: self.sceneView)
let result = self.sceneView.hitTest(tapPoint, types: ARHitTestResult.ResultType.existingPlaneUsingExtent)
if result.count == 0 {
return
}
let hitResult = result.first
let box = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
let node = SCNNode(geometry: box)
node.physicsBody = SCNPhysicsBody(type: SCNPhysicsBodyType.static, shape: nil)
node.position = SCNVector3Make(hitResult.worldTransform.columns.3.x, hitResult.worldTransform.columns.3.y, hitResult.worldTransform.columns.3.z)
sceneView.scene.rootNode.addChildNode(node)
}
Then when you tap on a plane surface detected, it will add a box on the plane where you tapped.

Related

swift SceneKit calculate node angle

i trying to add a node (pin) to sphere node (when taping on sphere node), but can't correctly calculate an angle, can u help me pls ?
private func addPinToSphere(result: SCNHitTestResult) {
guard let scene = SCNScene(named: "art.scnassets/tree.scn") else { return }
guard let treeNode = scene.rootNode.childNode(withName: "Trunk", recursively: true) else { return }
let x = CGFloat(result.worldCoordinates.x)
let y = CGFloat(result.worldCoordinates.y)
treeNode.position = result.worldCoordinates
treeNode.eulerAngles = SCNVector3((x) * .pi / CGFloat(180), 0, (y - 90) * .pi / CGFloat(180)) // y axis is ignorable
result.node.addChildNode(treeNode)
}
when run code i have this
but want like this
You can do it similar to this code. This is a missile launcher, but it's basically the same thing. Your pin = my fireTube, so create a base node, then add a subnode with your materials to it and rotate it into place. You'll have to experiment with where to place the tube initially, but once you get it into the right place the lookat constraint will always point it in the right direction.
case .d1, .d2, .d3, .d4, .d5, .d6, .d7, .d8, .d9, .d10:
let BoxGeometry = SCNBox(width: 0.8, height: 0.8, length: 0.8, chamferRadius: 0.0)
let vNode = SCNNode(geometry: BoxGeometry)
BoxGeometry.materials = setDefenseTextures(vGameType: vGameType)
let tubeGeometry = SCNTube(innerRadius: 0.03, outerRadius: 0.05, height: 0.9)
let fireTube = SCNNode(geometry: tubeGeometry)
tubeGeometry.firstMaterial?.diffuse.contents = data.getTextureColor(vTheme: 0, vTextureType: .barrelColor)
fireTube.position = SCNVector3(0, 0.2, -0.3)
let vRotateX = SCNAction.rotateBy(x: CGFloat(Float(GLKMathDegreesToRadians(-90))), y: 0, z: 0, duration: 0)
fireTube.runAction(vRotateX)
vNode.addChildNode(fireTube)
return vNode
Then set target on your base node and your subnode will rotate with it:
func setTarget()
{
node.constraints = []
let vConstraint = SCNLookAtConstraint(target: targetNode)
vConstraint.isGimbalLockEnabled = true
node.constraints = [vConstraint]
}
In your case, target equals center mass of your sphere and the pin will always point to it, provided you built and aligned your box and pin correctly.

3D Positional Audio – Move SCNAudioPlayer along Y and Z Axis

Using SceneKit, I can move audioNode from left to right on x axis, but I'm having problem moving on y and z axis. I'm wearing headphone, so I can hear the binaural (3d audio) effects. Also I'm running this on MacOS.
My testing code is below. Could someone let me know what I'm missing? I'd appreciate it!
import Cocoa
import SceneKit
class ViewController: NSViewController {
#IBOutlet weak var sceneView: SCNView!
override func viewDidLoad() {
super.viewDidLoad()
let path = Bundle.main.path(forResource: "Sounds/Test.mp3",
ofType: nil)
let url = URL(fileURLWithPath: path!)
let source = SCNAudioSource(url:url)!
source.loops = true
source.shouldStream = false
source.isPositional = true
source.load()
let player = SCNAudioPlayer(source: source)
let box = SCNBox(width: 100.0,
height: 100.0,
length: 100.0,
chamferRadius: 100.0)
let boxNode = SCNNode(geometry: box)
let audioNode = SCNNode()
boxNode.addChildNode(audioNode)
let scene = SCNScene()
scene.rootNode.addChildNode(boxNode)
sceneView.scene = scene
audioNode.addAudioPlayer(player)
let avm = player.audioNode as! AVAudioMixing
avm.volume = 1.0
let up = SCNAction.moveBy(x: 0, y: 100, z: 0, duration: 5)
let down = SCNAction.moveBy(x: 0, y: -100, z: 0, duration: 5)
let sequence = SCNAction.sequence([up, down])
let loop = SCNAction.repeatForever(sequence)
boxNode.runAction(loop)
// Do any additional setup after loading the view.
}
}
Updated.
You're casting the player.audioNode to AVAudioMixing protocol:
let avm = player.audioNode as! AVAudioMixing
But instead of it, you have to cast it to a class. A code looks like this:
let avm = player.audioNode as? AVAudioEnvironmentNode
Any node that conforms to the AVAudioMixing protocol (for example, AVAudioPlayerNode) can act as a source in this environment. The environment has an implicit listener. By controlling the listener’s position and orientation, the application controls the way the user experiences the virtual world. This node also defines properties for distance attenuation and reverberation that help characterize the environment.
And take into account !
Only inputs with a mono channel connection format to the environment node are spatialized. If the input is stereo, the audio is passed through without being spatialized. Inputs with connection formats of more than two channels aren't supported.
And, of course, you need to implement AVAudio3DMixing protocol.
Here's a working code:
import SceneKit
import AVFoundation
class ViewController: NSViewController, AVAudio3DMixing {
// YOU NEED MONO AUDIO !!!
var renderingAlgorithm = AVAudio3DMixingRenderingAlgorithm.sphericalHead
var rate: Float = 0.0
var reverbBlend: Float = 0.5
var obstruction: Float = -100.0
var occlusion: Float = -100.0
var position: AVAudio3DPoint = AVAudio3DPoint(x: 0, y: 0, z: -100)
override func viewDidLoad() {
super.viewDidLoad()
let scene = SCNScene()
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.camera?.zFar = 200
cameraNode.position = SCNVector3(x: 0, y: 0, z: 40)
scene.rootNode.addChildNode(cameraNode)
let sceneView = self.view as! SCNView
sceneView.scene = scene
sceneView.backgroundColor = NSColor.black
sceneView.autoenablesDefaultLighting = true
let path = Bundle.main.path(forResource: "Test_Mono", ofType: "mp3")
let url = URL(fileURLWithPath: path!)
let source = SCNAudioSource(url: url)!
source.loops = true
source.shouldStream = false // MUST BE FALSE
source.isPositional = true
source.load()
let player = SCNAudioPlayer(source: source)
let audioNode = SCNNode()
let box = SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0.2)
let boxNode = SCNNode(geometry: box)
boxNode.addChildNode(audioNode)
scene.rootNode.addChildNode(boxNode)
audioNode.addAudioPlayer(player)
let avm = player.audioNode as? AVAudioEnvironmentNode
avm?.reverbBlend = reverbBlend
avm?.renderingAlgorithm = renderingAlgorithm
avm?.occlusion = occlusion
avm?.obstruction = obstruction
let up = SCNAction.moveBy(x: 0, y: 0, z: 70, duration: 5)
let down = SCNAction.moveBy(x: 0, y: 0, z: -70 , duration: 5)
let sequence = SCNAction.sequence([up, down])
let loop = SCNAction.repeatForever(sequence)
boxNode.runAction(loop)
avm?.position = AVAudio3DPoint(
x: Float(boxNode.position.x),
y: Float(boxNode.position.y),
z: Float(boxNode.position.z))
}
}
After researching and experimenting for a hwile, I finally figured it out. There were two things that I needed to fix.
I had to change the default renderingAlgorithm for SCNAudioPlayer.AVAudioNode from equalPowerPanning to either HRTF or HRTFHQ. However, AVAudioNode does not have renderingAlgorithm property. However, I was able to cast SCNAudioPlayer.AVAudioNode as AVAudioPlayerNode, and AVAudioPlayerNode does have renderingAlgorithm property. Here's the relevant code.
if let apn = player.audioNode as? AVAudioPlayerNode {
apn.renderingAlgorithm = .HRTFHQ
}
I had to assign a node with SCNCamera to pointOfView for SCNView. Also I had to change the position of the camera node further away from the audioNode. Otherwise, I heard the drastic movement in the beginning. Here's the relevant code.
let cameraNode = SCNNode(geometry: SCNBox(width:1, height:1, length:1, chamferRadius: 0.1))
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(x: 0, y: 0, z: -10)
sceneView.pointOfView = cameraNode
My scene.rootNode is a box geometry with 100x100x100 dimension. Inside scene.rootNode, I have a boxNode with 50x5050 dimension. Then inside the boxNode, I have audioNode generating sound with 1x1x1 dimension as well as cameraNode with 1x1x1 dimension. AudioNode's start position is 0,0,0, and the position for the cameraNode is 0,0,-20.
Finally here's the entire working code.
import Cocoa
import AVFoundation
import SceneKit
class ViewController: NSViewController {
#IBOutlet weak var sceneView: SCNView!
override func viewDidLoad() {
super.viewDidLoad()
let path = Bundle.main.path(forResource: "Sounds/Test_mono.mp3", ofType: nil)
let url = URL(fileURLWithPath: path!)
let source = SCNAudioSource(url: url)!
source.loops = true
source.shouldStream = false
source.isPositional = true
source.load()
let player = SCNAudioPlayer(source: source)
if let apn = player.audioNode as? AVAudioPlayerNode {
apn.renderingAlgorithm = .HRTFHQ
}
let audioNode = SCNNode(geometry: SCNBox(width:1, height:1, length:1, chamferRadius: 0.1))
let cameraNode = SCNNode(geometry: SCNBox(width:1, height:1, length:1, chamferRadius: 0.1))
cameraNode.camera = SCNCamera()
let boxNode = SCNNode(geometry: SCNBox(width:50, height:50, length:50, chamferRadius: 1))
boxNode.addChildNode(audioNode)
audioNode.position = SCNVector3(x: 0, y: 0, z: 0)
boxNode.addChildNode(cameraNode)
cameraNode.position = SCNVector3(x: 0, y: 0, z: -10)
let scene = SCNScene()
scene.rootNode.geometry = SCNBox(width:100, height:100, length:100, chamferRadius: 0.1)
scene.rootNode.addChildNode(boxNode)
boxNode.position = SCNVector3(x: 0, y: 0, z: 0)
sceneView.scene = scene
sceneView.pointOfView = cameraNode
sceneView.audioListener = cameraNode
audioNode.addAudioPlayer(player)
let move = SCNAction.moveBy(x:1, y:0, z:0, duration: 1)
let sequence = SCNAction.sequence([move])
let loop = SCNAction.repeatForever(sequence)
audioNode.runAction(loop)
// Do any additional setup after loading the view.
}
override var representedObject: Any? {
didSet {
// Update the view, if already loaded.
}
}
}

Swift: ARKit place a rectangle based on 2 nodes

My idea is that i want to place 2 sphere nodes at selected locations. From that point i basically want to draw a rectangle that will adjust the height with a slider. Basically that means that the 2 spheres will represent all 4 corners in the beginning. But when testing the code i use 1 meter height as a test.
The problem i have is that i cant seem to place the rectangle at the correct location as illustrated in the image below:
the rectangle in the image has a higher y-point than the points, it's rotated slightly and its center point is above node 2 and not in between the nodes. I don't want to use node.rotation as it wont work dynamically.
This is the code i use to place the 2 nodes and draw + add the rectangle.
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
guard let location = touches.first?.location(in: sceneView) else {return}
let hitTest = sceneView.hitTest(location, types: [ARHitTestResult.ResultType.featurePoint])
guard let result = hitTest.last else {return}
// Converts the matrix_float4x4 to an SCNMatrix4 to be used with SceneKit
let transform = SCNMatrix4.init(result.worldTransform)
// Creates an SCNVector3 with certain indexes in the matrix
let vector = SCNVector3Make(transform.m41, transform.m42, transform.m43)
let node = addSphere(withPosition: vector)
nodeArray.append(node)
sceneView.scene.rootNode.addChildNode(node)
if nodeArray.count == 2 {
let node1 = sceneView.scene.rootNode.childNodes[0]
let node2 = sceneView.scene.rootNode.childNodes[1]
let bezeierPath = UIBezierPath()
bezeierPath.lineWidth = 0.01
bezeierPath.move(to: CGPoint(x: CGFloat(node1.position.x), y: CGFloat(node1.position.y)))
bezeierPath.addLine(to: CGPoint(x: CGFloat(node2.position.x), y: CGFloat(node2.position.y)))
bezeierPath.addLine(to: CGPoint(x: CGFloat(node2.position.x), y: CGFloat(node2.position.y+1.0)))
bezeierPath.addLine(to: CGPoint(x: CGFloat(node1.position.x), y: CGFloat(node1.position.y+1.0)))
bezeierPath.close()
bezeierPath.fill()
let shape = SCNShape(path: bezeierPath, extrusionDepth: 0.02)
shape.firstMaterial?.diffuse.contents = UIColor.red.withAlphaComponent(0.8)
let node = SCNNode.init(geometry: shape)
node.position = SCNVector3(CGFloat(abs(node1.position.x-node2.position.x)/2), CGFloat(abs((node1.position.y)-(node2.position.y))/2), CGFloat(node1.position.z))
sceneView.scene.rootNode.addChildNode(node)
}
}
Also note that this is not my final code. It will be refactored once i get everything working :).
This is currently working. I used the wrong calculation to set it in the middle of the 2 points.
override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
// Check if a window is placed already and fetch location in sceneView
guard debugMode, let location = touches.first?.location(in: sceneView) else {return}
// Fetch targets at the current location
let hitTest = sceneView.hitTest(location, types: [ARHitTestResult.ResultType.featurePoint])
guard let result = hitTest.last else {return}
// Create sphere are position. getPosition() is an extension
createSphere(withPosition: result.getPosition())
// When 2 nodes has been placed, create a window and hide the spheres
if nodeArray.count == 2 {
let firstNode = nodeArray[nodeArray.count-1]
let secondNode = nodeArray[nodeArray.count-2]
firstNode.isHidden = true
secondNode.isHidden = true
createWindow(firstNode: firstNode, secondNode: secondNode)
sceneView.debugOptions = []
}
}
func createSphere(withPosition position: SCNVector3) {
// Create an geometry which you will apply materials to and convert to a node
let object = SCNSphere(radius: 0.01)
let material = SCNMaterial()
material.diffuse.contents = UIColor.red
object.materials = [material]
let node = SCNNode(geometry: object)
node.position = position
nodeArray.append(node)
sceneView.scene.rootNode.addChildNode(node)
}
func createWindow(firstNode: SCNNode, secondNode: SCNNode) {
// Create an geometry which you will apply materials to and convert to a node
let shape = SCNBox(width: CGFloat(abs(firstNode.worldPosition.x-secondNode.worldPosition.x)), height: 0.01, length: 0.02, chamferRadius: 0)
let material = SCNMaterial()
material.diffuse.contents = UIColor.red.withAlphaComponent(0.8)
// Each side of a cube can have different materials
// https://stackoverflow.com/questions/27509092/scnbox-different-colour-or-texture-on-each-face
shape.materials = [material]
let node = SCNNode(geometry: shape)
let minX = min(firstNode.worldPosition.x, secondNode.worldPosition.x)
let maxX = max(firstNode.worldPosition.x, secondNode.worldPosition.x)
// The nodes pivot Y-axis should be split in half of the nodes height, this will cause the node to
// only scale in one direction, when scaling it
// https://stackoverflow.com/questions/42568420/scenekit-understanding-the-pivot-property-of-scnnode/42613002#42613002
node.position = SCNVector3(((maxX-minX)/2)+minX, firstNode.worldPosition.y, firstNode.worldPosition.z)
node.pivot = SCNMatrix4MakeTranslation(0, 0.005, 0)
node.scale = SCNVector3Make(1, -50, 1)
node.name = "window"
sceneView.scene.rootNode.addChildNode(node)
}
Also added:
#objc func resetSceneView() {
// Clear all the nodes
sceneView.scene.rootNode.enumerateHierarchy { (node, stop) in
node.childNodes.forEach({
$0.removeFromParentNode()
})
node.removeFromParentNode()
}
// Reset the session
sceneView.session.pause()
sceneView.debugOptions = [ARSCNDebugOptions.showFeaturePoints]
sceneView.session.run(configuration, options: .resetTracking)
let matrix = sceneView.session.currentFrame!.camera.transform
sceneView.session.setWorldOrigin(relativeTransform: matrix)
nodeArray = []
}
To make sure that the world alignment is reset to your camera position

How to add an UITextField to SCNNode geometry material in Swift?

I'm trying to make an UITextField as a SCNNode geometry material. This code works well:
func createBox(transform: SCNMatrix4) {
let box = SCNBox(width: 0.5, height: 0.5, length: 0.5, chamferRadius: 1)
let textField = UITextField(frame: CGRect(x: 0, y: 0, width: 60, height: 50))
textField.text = "Hello"
let sides = [
textField, // Front
UIColor.black, // Right
UIColor.black, // Back
UIColor.black, // Left
UIColor.black, // Top
UIColor.black // Bottom
]
let materials = sides.map { (side) -> SCNMaterial in
let material = SCNMaterial()
material.diffuse.contents = side
material.locksAmbientWithDiffuse = true
return material
}
box.materials = materials
let boxNode = SCNNode(geometry: box)
boxNode.position = SCNVector3Make(transform.m41, transform.m42, transform.m43)
boxNode.transform = transform
sceneView.scene.rootNode.addChildNode(boxNode)
}
But when I tap on the text field the keyboard doesn't appear. Why is it happening? Is there ways to fix it? I need the user can enter some text in the field. Thanks.

Using SceneKit for hitTesting not returning a hit with SCNNode

The documentation in XCode clearly states that hitTesting a geometry in SceneKit can be done with SCNRender, SCNView or the SCNNode themselves when one plans to test a 3D line segment. I have a use for SCNScene with its nodes without a renderer or a view, therefore I am planning to use SCNNode hitTesting. I create a SCNScene, put a SCNNode in it and test a simple ray that goes through, but I always get an empty hitList and I don't understand why:
import Swift
import SceneKit
let boxGeometry = SCNBox(width: 1.0, height: 1.0, length: 1.0, chamferRadius: 0)
let boxNode = SCNNode(geometry: boxGeometry)
var scene = SCNScene()
scene.rootNode.addChildNode(boxNode)
let from = SCNVector3(x: 0, y: -2, z: 0)
let to = SCNVector3(x: 0, y: 2 , z: 0)
var hits = scene.rootNode.hitTestWithSegmentFromPoint(from, toPoint: to, options:nil) // this is always empty
if hits != nil {
if hits!.count > 0 {
var hit = (hits!.first as! SCNHitTestResult).node as SCNNode
}
}
I have tried passing various forms of options but nothing changes.
SCNHitTestFirstFoundOnlyKey: yes or no does not change anything
SCNHitTestSortResultsKey: yes or no does not change anything
SCNHitTestClipToZRangeKey: invalid for SCNNode
SCNHitTestBackFaceCullingKey: yes or no does not change anything
SCNHitTestBoundingBoxOnlyKey: yes or no does not change anything
SCNHitTestRootNodeKey: rootNOde of scene or boxNode does not change
anything
SCNHitTestIgnoreHiddenNodesKey: yes or no does not change anything
What am I doing wrong?
I have found the answer, which is either a bug or a feature: using SCNScene and its nodes SCNNode for 3D hitTesting, in particular the method: "hitTestWithSegmentFromPoint(toPoint:options:)" does not return a hit unless the scene is included in an SCNView. It appears it cannot be used offscreen. My guess is yours for why this is the case, although I can imagine it has something to do with performing some of these quite expensive calculations on the graphics card.
I have tested this using the GameView SCNScene starter project. The critical line is self.gameView!.scene = scene
override func awakeFromNib(){
let scene = SCNScene()
let boxGeometry = SCNBox(width: 1.0, height: 1.0, length: 1.0, chamferRadius: 0.0)
let boxNode = SCNNode(geometry: boxGeometry)
boxNode.position=SCNVector3(x: 0, y: 0, z: 0)
scene.rootNode.addChildNode(boxNode)
// create and add a camera to the scene
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
scene.rootNode.addChildNode(cameraNode)
// place the camera
cameraNode.position = SCNVector3(x: 0, y: 0, z: 15)
// create and add a light to the scene
let lightNode = SCNNode()
lightNode.light = SCNLight()
lightNode.light!.type = SCNLightTypeOmni
lightNode.position = SCNVector3(x: 0, y: 10, z: 10)
scene.rootNode.addChildNode(lightNode)
// create and add an ambient light to the scene
let ambientLightNode = SCNNode()
ambientLightNode.light = SCNLight()
ambientLightNode.light!.type = SCNLightTypeAmbient
ambientLightNode.light!.color = NSColor.darkGrayColor()
scene.rootNode.addChildNode(ambientLightNode)
// set the scene to the view
// uncomment this to fail
self.gameView!.scene = scene
// allows the user to manipulate the camera
self.gameView!.allowsCameraControl = true
// show statistics such as fps and timing information
self.gameView!.showsStatistics = true
// configure the view
self.gameView!.backgroundColor = NSColor.blackColor()
let hitList = scene.rootNode.hitTestWithSegmentFromPoint(SCNVector3(x:-10,y:0,z:0), toPoint: SCNVector3(x:10,y:0,z:0), options:[SCNHitTestBackFaceCullingKey:false, SCNHitTestSortResultsKey:true, SCNHitTestIgnoreHiddenNodesKey:false])
if hitList?.count > 0 {
println("Hit found: \n\n\( hitList![0] )") // assign self.gameView!.scene = scene to reach this point.
} else {
println("No hit") // uncomment self.gameView!.scene = scene to reach this point.
}
}
I've also had trouble with hitTestWithSegmentFromPoint.
I was calling it in viewDidLoad() and it returned a 0 elements array, though I was sure there was a hit.
Calling it in viewDidAppear() (or later) solved my problem.