3D Positional Audio – Move SCNAudioPlayer along Y and Z Axis - swift

Question

Using SceneKit, I can move audioNode from left to right on x axis, but I'm having problem moving on y and z axis. I'm wearing headphone, so I can hear the binaural (3d audio) effects. Also I'm running this on MacOS.
My testing code is below. Could someone let me know what I'm missing? I'd appreciate it!
import Cocoa
import SceneKit
class ViewController: NSViewController {
#IBOutlet weak var sceneView: SCNView!
override func viewDidLoad() {
super.viewDidLoad()
let path = Bundle.main.path(forResource: "Sounds/Test.mp3",
ofType: nil)
let url = URL(fileURLWithPath: path!)
let source = SCNAudioSource(url:url)!
source.loops = true
source.shouldStream = false
source.isPositional = true
source.load()
let player = SCNAudioPlayer(source: source)
let box = SCNBox(width: 100.0,
height: 100.0,
length: 100.0,
chamferRadius: 100.0)
let boxNode = SCNNode(geometry: box)
let audioNode = SCNNode()
boxNode.addChildNode(audioNode)
let scene = SCNScene()
scene.rootNode.addChildNode(boxNode)
sceneView.scene = scene
audioNode.addAudioPlayer(player)
let avm = player.audioNode as! AVAudioMixing
avm.volume = 1.0
let up = SCNAction.moveBy(x: 0, y: 100, z: 0, duration: 5)
let down = SCNAction.moveBy(x: 0, y: -100, z: 0, duration: 5)
let sequence = SCNAction.sequence([up, down])
let loop = SCNAction.repeatForever(sequence)
boxNode.runAction(loop)
// Do any additional setup after loading the view.
}
}

Answer 1

Updated.
You're casting the player.audioNode to AVAudioMixing protocol:
let avm = player.audioNode as! AVAudioMixing
But instead of it, you have to cast it to a class. A code looks like this:
let avm = player.audioNode as? AVAudioEnvironmentNode
Any node that conforms to the AVAudioMixing protocol (for example, AVAudioPlayerNode) can act as a source in this environment. The environment has an implicit listener. By controlling the listener’s position and orientation, the application controls the way the user experiences the virtual world. This node also defines properties for distance attenuation and reverberation that help characterize the environment.
And take into account !
Only inputs with a mono channel connection format to the environment node are spatialized. If the input is stereo, the audio is passed through without being spatialized. Inputs with connection formats of more than two channels aren't supported.
And, of course, you need to implement AVAudio3DMixing protocol.
Here's a working code:
import SceneKit
import AVFoundation
class ViewController: NSViewController, AVAudio3DMixing {
// YOU NEED MONO AUDIO !!!
var renderingAlgorithm = AVAudio3DMixingRenderingAlgorithm.sphericalHead
var rate: Float = 0.0
var reverbBlend: Float = 0.5
var obstruction: Float = -100.0
var occlusion: Float = -100.0
var position: AVAudio3DPoint = AVAudio3DPoint(x: 0, y: 0, z: -100)
override func viewDidLoad() {
super.viewDidLoad()
let scene = SCNScene()
let cameraNode = SCNNode()
cameraNode.camera = SCNCamera()
cameraNode.camera?.zFar = 200
cameraNode.position = SCNVector3(x: 0, y: 0, z: 40)
scene.rootNode.addChildNode(cameraNode)
let sceneView = self.view as! SCNView
sceneView.scene = scene
sceneView.backgroundColor = NSColor.black
sceneView.autoenablesDefaultLighting = true
let path = Bundle.main.path(forResource: "Test_Mono", ofType: "mp3")
let url = URL(fileURLWithPath: path!)
let source = SCNAudioSource(url: url)!
source.loops = true
source.shouldStream = false // MUST BE FALSE
source.isPositional = true
source.load()
let player = SCNAudioPlayer(source: source)
let audioNode = SCNNode()
let box = SCNBox(width: 1, height: 1, length: 1, chamferRadius: 0.2)
let boxNode = SCNNode(geometry: box)
boxNode.addChildNode(audioNode)
scene.rootNode.addChildNode(boxNode)
audioNode.addAudioPlayer(player)
let avm = player.audioNode as? AVAudioEnvironmentNode
avm?.reverbBlend = reverbBlend
avm?.renderingAlgorithm = renderingAlgorithm
avm?.occlusion = occlusion
avm?.obstruction = obstruction
let up = SCNAction.moveBy(x: 0, y: 0, z: 70, duration: 5)
let down = SCNAction.moveBy(x: 0, y: 0, z: -70 , duration: 5)
let sequence = SCNAction.sequence([up, down])
let loop = SCNAction.repeatForever(sequence)
boxNode.runAction(loop)
avm?.position = AVAudio3DPoint(
x: Float(boxNode.position.x),
y: Float(boxNode.position.y),
z: Float(boxNode.position.z))
}
}

Answer 2

After researching and experimenting for a hwile, I finally figured it out. There were two things that I needed to fix.
I had to change the default renderingAlgorithm for SCNAudioPlayer.AVAudioNode from equalPowerPanning to either HRTF or HRTFHQ. However, AVAudioNode does not have renderingAlgorithm property. However, I was able to cast SCNAudioPlayer.AVAudioNode as AVAudioPlayerNode, and AVAudioPlayerNode does have renderingAlgorithm property. Here's the relevant code.
if let apn = player.audioNode as? AVAudioPlayerNode {
apn.renderingAlgorithm = .HRTFHQ
}
I had to assign a node with SCNCamera to pointOfView for SCNView. Also I had to change the position of the camera node further away from the audioNode. Otherwise, I heard the drastic movement in the beginning. Here's the relevant code.
let cameraNode = SCNNode(geometry: SCNBox(width:1, height:1, length:1, chamferRadius: 0.1))
cameraNode.camera = SCNCamera()
cameraNode.position = SCNVector3(x: 0, y: 0, z: -10)
sceneView.pointOfView = cameraNode
My scene.rootNode is a box geometry with 100x100x100 dimension. Inside scene.rootNode, I have a boxNode with 50x5050 dimension. Then inside the boxNode, I have audioNode generating sound with 1x1x1 dimension as well as cameraNode with 1x1x1 dimension. AudioNode's start position is 0,0,0, and the position for the cameraNode is 0,0,-20.
Finally here's the entire working code.
import Cocoa
import AVFoundation
import SceneKit
class ViewController: NSViewController {
#IBOutlet weak var sceneView: SCNView!
override func viewDidLoad() {
super.viewDidLoad()
let path = Bundle.main.path(forResource: "Sounds/Test_mono.mp3", ofType: nil)
let url = URL(fileURLWithPath: path!)
let source = SCNAudioSource(url: url)!
source.loops = true
source.shouldStream = false
source.isPositional = true
source.load()
let player = SCNAudioPlayer(source: source)
if let apn = player.audioNode as? AVAudioPlayerNode {
apn.renderingAlgorithm = .HRTFHQ
}
let audioNode = SCNNode(geometry: SCNBox(width:1, height:1, length:1, chamferRadius: 0.1))
let cameraNode = SCNNode(geometry: SCNBox(width:1, height:1, length:1, chamferRadius: 0.1))
cameraNode.camera = SCNCamera()
let boxNode = SCNNode(geometry: SCNBox(width:50, height:50, length:50, chamferRadius: 1))
boxNode.addChildNode(audioNode)
audioNode.position = SCNVector3(x: 0, y: 0, z: 0)
boxNode.addChildNode(cameraNode)
cameraNode.position = SCNVector3(x: 0, y: 0, z: -10)
let scene = SCNScene()
scene.rootNode.geometry = SCNBox(width:100, height:100, length:100, chamferRadius: 0.1)
scene.rootNode.addChildNode(boxNode)
boxNode.position = SCNVector3(x: 0, y: 0, z: 0)
sceneView.scene = scene
sceneView.pointOfView = cameraNode
sceneView.audioListener = cameraNode
audioNode.addAudioPlayer(player)
let move = SCNAction.moveBy(x:1, y:0, z:0, duration: 1)
let sequence = SCNAction.sequence([move])
let loop = SCNAction.repeatForever(sequence)
audioNode.runAction(loop)
// Do any additional setup after loading the view.
}
override var representedObject: Any? {
didSet {
// Update the view, if already loaded.
}
}
}