Xcode 8 Swift 3 Pitch-altering sounds - swift

I'm trying to make a simple game with a hit sound that has a different pitch whenever you hit something. I thought it'd be simple, but it ended up with a whole lot of stuff (most of which I completely copied from someone else):
func hitSound(value: Float) {
let audioPlayerNode = AVAudioPlayerNode()
audioPlayerNode.stop()
engine.stop() // This is an AVAudioEngine defined previously
engine.reset()
engine.attach(audioPlayerNode)
let changeAudioUnitTime = AVAudioUnitTimePitch()
changeAudioUnitTime.pitch = value
engine.attach(changeAudioUnitTime)
engine.connect(audioPlayerNode, to: changeAudioUnitTime, format: nil)
engine.connect(changeAudioUnitTime, to: engine.outputNode, format: nil)
audioPlayerNode.scheduleFile(file, at: nil, completionHandler: nil) // File is an AVAudioFile defined previously
try? engine.start()
audioPlayerNode.play()
}
Since this code seems to stop playing any sounds currently being played in order to play the new sound, is there a way I can alter this behaviour so it doesn't stop playing anything? I tried removing the engine.stop and engine.reset bits, but this just crashes the app. Also, this code is incredibly slow when called frequently. Is there something I could do to speed it up? This hit sound is needed very frequently.

You're resetting the engine every time you play a sound! And you're creating extra player nodes - it's actually much simpler than that if you only want one instance of the pitch shifted sound playing at once:
// instance variables
let engine = AVAudioEngine()
let audioPlayerNode = AVAudioPlayerNode()
let changeAudioUnitTime = AVAudioUnitTimePitch()
call setupAudioEngine() once:
func setupAudioEngine() {
engine.attach(self.audioPlayerNode)
engine.attach(changeAudioUnitTime)
engine.connect(audioPlayerNode, to: changeAudioUnitTime, format: nil)
engine.connect(changeAudioUnitTime, to: engine.outputNode, format: nil)
try? engine.start()
audioPlayerNode.play()
}
and call hitSound() as many times as you like:
func hitSound(value: Float) {
changeAudioUnitTime.pitch = value
audioPlayerNode.scheduleFile(file, at: nil, completionHandler: nil) // File is an AVAudioFile defined previously
}
p.s. pitch can be shifted two octaves up or down, for a range of 4 octaves, and lies in the numerical range of [-2400, 2400], having the unit "cents".
p.p.s AVAudioUnitTimePitch is very cool technology. We definitely didn't have anything like it when I was a kid.
UPDATE
If you want multi channel, you can easily set up multiple player and pitch nodes, however you must choose the number of channels before you start the engine. Here's how you'd do two (it's easy to extend to n instances, and you'll probably want to choose your own method of choosing which channel to interrupt when all are playing):
// instance variables
let engine = AVAudioEngine()
var nextPlayerIndex = 0
let audioPlayers = [AVAudioPlayerNode(), AVAudioPlayerNode()]
let pitchUnits = [AVAudioUnitTimePitch(), AVAudioUnitTimePitch()]
func setupAudioEngine() {
var i = 0
for playerNode in audioPlayers {
let pitchUnit = pitchUnits[i]
engine.attach(playerNode)
engine.attach(pitchUnit)
engine.connect(playerNode, to: pitchUnit, format: nil)
engine.connect(pitchUnit, to:engine.mainMixerNode, format: nil)
i += 1
}
try? engine.start()
for playerNode in audioPlayers {
playerNode.play()
}
}
func hitSound(value: Float) {
let playerNode = audioPlayers[nextPlayerIndex]
let pitchUnit = pitchUnits[nextPlayerIndex]
pitchUnit.pitch = value
// interrupt playing sound if you have to
if playerNode.isPlaying {
playerNode.stop()
playerNode.play()
}
playerNode.scheduleFile(file, at: nil, completionHandler: nil) // File is an AVAudioFile defined previously
nextPlayerIndex = (nextPlayerIndex + 1) % audioPlayers.count
}

Related

AVAudioPlayerNode jump to a specific time of the audio file

I found similar questions here, here, and here but with none of the answers I have been able to solve the problem. Simply put, the audio does not jump at a specific moment but instead starts from scratch.
func seekTo(time: Double) {
player.stop()
let startSample = Double(time * audioSampleRate)
let lengthSamples: AVAudioFramePosition = AVAudioFramePosition(Double(audioLengthSamples) - startSample)
let frameCount = AVAudioFrameCount(audioLengthSamples - lengthSamples)
if currentPosition < audioLengthSamples {
player.scheduleSegment(audioFile!, startingFrame: AVAudioFramePosition(startSample), frameCount: AVAudioFrameCount(frameCount), at: nil, completionHandler: nil)
let wasPlaying = player.isPlaying
if wasPlaying {
player.play()
}
}
}
The time variable is where it receives the specific time to which the audio should skip.
Any help?

Calling stop() on AVAudioPlayerNode after finished playing causes crash

I have an AVAudioPlayerNode object which sends a callback once it is finished playing. That callback triggers many other functions in the app, one of which sends out a courtesy stop() message. For some reason, calling stop() at the moment the AVAudioPlayerNode finishes causes a crash. In the code here, I have abbreviated it so that the AVAudioPlayerNode just calls stop immediately to demonstrate the effect (rather than including my whole application). You can see clearly that it crashes. I don't understand why. Either a) the node is still playing and stop() stops it or b) it is done playing and stop can be ignored.
My guess is that this is some edge case where it is at the very end of the file buffer, and it is in an ambiguous state where has no more buffers remaining, yet is technically still playing. Perhaps calling stop() tries to flush the remaining buffers and they are empty?
func testAVAudioPlayerNode(){
let testBundle = Bundle(for: type(of: self))
let url = URL(fileURLWithPath: testBundle.path(forResource: "19_lyrics_1", ofType: "m4a")!)
let player = AVAudioPlayerNode()
let engine = AVAudioEngine()
let format = engine.mainMixerNode.outputFormat(forBus: 0)
engine.attach(player)
engine.connect(player, to: engine.mainMixerNode, format: format)
let delegate = FakePlayMonitorDelegate()
do {
let audioFile = try AVAudioFile(forReading: url)
let length = audioFile.length
player.scheduleFile(audioFile, at: nil, completionCallbackType: AVAudioPlayerNodeCompletionCallbackType.dataPlayedBack, completionHandler: {(completionType) in
print("playing = \(player.isPlaying)")
player.stop()
})
try engine.start()
let expectation = self.expectation(description: "playback")
delegate.expectation = expectation
player.play()
self.waitForExpectations(timeout: 6.0, handler: nil)
} catch {
}
}
In my case calling the .stop() method from the Main thread (you can use another) has resolved the issue.
DispatchQueue.main.async {
player.stop()
}
It can be the deadlock. I have noticed that the completionHandler is invoked from the background queue which is a serial queue, let's call it a "render queue".
Seems that the .stop() method is trying to do some work on a "render queue" but at the moment a "render queue" is busy with the completionHandler.

DispatchGroup On Do, Try, Catch?

I'm working with the AudioKit framework and looking to use DispatchGroup to make a method work async. I'd like for the player.load method to run only after the audioFile has been created; right now it's throwing an error ~50% of the time and I suspect it's due to timing. I've used DispatchGroup with success in other circumstances, but never in a do/try/catch. Is there a way to make this part of the function work with it? If not, is there a way to set up a closure? Thanks!
func createPlayer(fileName: String) -> AKPlayer {
let player = AKPlayer()
let audioFile : AKAudioFile
player.mixer >>> mixer
do {
try audioFile = AKAudioFile(readFileName: "\(fileName).mp3")
player.load(audioFile: audioFile)
print("AudioFile \(fileName), \(audioFile) loaded")
} catch { print("No audio file read, looking for \(fileName).mp3")
}
player.isLooping = false
player.fade.inTime = 2 // in seconds
player.fade.outTime = 2
player.stopEnvelopeTime = 2
player.completionHandler = {
print("Completion")
self.player.detach()
}
player.play()
return player
}

Swift - Recorded Video is Mirrored on Front Camera - How to flip?

I'm trying to mirror the recorded video from a capture session. The video preview for front facing camera shows a mirrored version, however, when I go to save the file and play it back, the captured video is actually mirrored. I'm using Apple's AVCam demo as a reference and can't seem to figure this out! Please help.
I've tried creating an AVCaptureConnection and trying to set the .isVideoMirrored parameter. However, I get this error:
cannot be added to the session because the source and destination media types are incompatible'
I would have thought mirroring the video would be much easier. I think I may be creating my connection incorrectly. The code below doesn't actually "Add connection" when I call the .canAddConnection check.
var captureSession: AVCaptureSession!
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
captureSession = AVCaptureSession()
//Setup Camera
if let dualCameraDevice = AVCaptureDevice.default(.builtInDualCamera, for: .video, position: .front) {
defaultVideoDevice = dualCameraDevice
} else if let frontCameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .front) {
// If the rear wide angle camera isn't available, default to the front wide angle camera.
defaultVideoDevice = frontCameraDevice
}
guard let videoDevice = defaultVideoDevice else {
print("Default video device is unavailable.")
// setupResult = .configurationFailed
captureSession.commitConfiguration()
return
}
let videoDeviceInput = try AVCaptureDeviceInput(device: videoDevice)
if captureSession.canAddInput(videoDeviceInput) {
captureSession.addInput(videoDeviceInput)
}
let movieOutput = AVCaptureMovieFileOutput()
//Video Input variable for AVCapture Connection
let videoInput: [AVCaptureInput.Port] = videoDeviceInput.ports
if captureSession.canAddOutput(movieOutput) {
captureSession.beginConfiguration()
captureSession.addOutput(movieOutput)
captureSession.sessionPreset = .medium
Then I try to setup the AVCapture connection and try to set the parameters for mirroring. Please tell me if there is an easier way to mirror the output / playback.
avCaptureConnection = AVCaptureConnection(inputPorts: videoInput, output: movieOutput)
avCaptureConnection.isEnabled = true
//Mirror the capture connection?
avCaptureConnection.automaticallyAdjustsVideoMirroring = false
avCaptureConnection.isVideoMirrored = false
//Check if we can add a connection
if captureSession.canAddConnection(avCaptureConnection) {
//Add the connection
captureSession.addConnection(avCaptureConnection)
}
captureSession.commitConfiguration()
self.movieOutput = movieOutput
setupLivePreview()
}
}
Somewhere else in the code, connected to an IBAaction, I initialize the recording
// Start recording video to a temporary file.
let outputFileName = NSUUID().uuidString
let outputFilePath = (NSTemporaryDirectory() as NSString).appendingPathComponent((outputFileName as NSString).appendingPathExtension("mov")!)
print("Recording in tap function")
movieOutput.startRecording(to: URL(fileURLWithPath: outputFilePath), recordingDelegate: self)
I think I'm using AVCaptureConnection incorrectly, especially because of the error stating media types are incompatible. If there is a proper way to implement this function please do let me know. Also open to hearing suggestions for an easier way to mirror the playback. Thank you!

swift AVFoundation pitched audio memory issue

Hi still new to swift and programming in general.
I have this function that plays a piece of audio at a specified pitch. It gets called by a NStimer so plays once a second. (function contained in a SoundPlayer class and then NStimer setup and used in viewController)
func playPitchedAudio(audioFile: AVAudioFile, pitch: Float){
audioEngine.stop()
audioEngine.reset()
let audioPlayerNode = AVAudioPlayerNode()
let changePitchEffect = AVAudioUnitTimePitch()
changePitchEffect.pitch = pitch
audioEngine.attachNode(audioPlayerNode)
audioEngine.attachNode(changePitchEffect)
audioEngine.connect(audioPlayerNode, to: changePitchEffect, format: nil)
audioEngine.connect(changePitchEffect, to: audioEngine.outputNode, format: nil)
audioPlayerNode.scheduleFile(audioFile, atTime: nil, completionHandler: nil)
do {
try audioEngine.start()
} catch {
print("error")
}
audioPlayerNode.play()
}
It runs fine and works but adds a few mb to memory every time its called and never regains the space. Did some research on memory leaks but couldn't find anything that helps my my specific scenario so hoping someone can point me in the right direction.
I assumed it was something to do with creating a new node and TimePitch everytime its called so moved them into the class that contains the function but got a "libc++abi.dylib: terminating with uncaught exception of type NSException" error when the sound attempted to play for the second time.
any help much appreciated, Thanks!
extra stuff.
// defined in class to be used by function
var pitchedAudioPlayer = AVAudioPlayerNode()
var audioEngine = AVAudioEngine()
//Timer Start
self.timer.invalidate()
self.timer = NSTimer.scheduledTimerWithTimeInterval(tempo, target: self, selector: #selector(ViewController.timeTriggerPointer), userInfo: nil, repeats: true)
//Timer calls... (along with some other unrelated stuff)
func timeTriggerPointer() {
soundPlayer.playPitchedAudio(pitchFilePath, pitch: -1000.0)
}
Solution -
import AVFoundation
class SoundPlayer {
var pitchedAudioPlayer = AVAudioPlayerNode()
var audioEngine = AVAudioEngine()
let audioPlayerNode = AVAudioPlayerNode()
let changePitchEffect = AVAudioUnitTimePitch()
init() {
audioEngine.attachNode(audioPlayerNode)
audioEngine.attachNode(changePitchEffect)
audioEngine.connect(audioPlayerNode, to: changePitchEffect, format: nil)
audioEngine.connect(changePitchEffect, to: audioEngine.outputNode, format: nil)
}
func playPitchedAudio(audioFile: AVAudioFile, pitch: Float){
audioPlayerNode.stop()
changePitchEffect.pitch = pitch
audioPlayerNode.scheduleFile(audioFile, atTime: nil, completionHandler: nil)
do {
try audioEngine.start()
} catch {
print("error")
}
audioPlayerNode.play()
}
}
A "leak" is a highly technical thing: a piece of unreferenced memory which can never be released (because it is unreferenced). I doubt that you have a leak. You just have more and more objects, that's all.
You are repeating this code over and over (once each time the timer fires):
let audioPlayerNode = AVAudioPlayerNode()
let changePitchEffect = AVAudioUnitTimePitch()
audioEngine.attachNode(audioPlayerNode)
audioEngine.attachNode(changePitchEffect)
The audio engine itself, however, remains in place (it's declared outside the function, as a property of your view controller). So you're adding two more nodes to this same audio engine every time the timer fires. Nodes take up memory, so your memory keeps rising. No surprise here. If you don't want that to happen, don't do that.
I assumed it was something to do with creating a new node and TimePitch everytime its called
There you go. So you already solved your own problem.
Think for a moment about what you're trying to do. You have an audio engine with nodes. The only thing that might to change on each run is the pitch of the AVAudioUnitTimePitch node and the file to be played. So create the whole audio engine and nodes beforehand, and leave it in place. Keep a reference to the AVAudioUnitTimePitch as a property. In the timer function, just change that pitch value!
let theTimePitchNode = AVAudioUnitTimePitch()
// and you have also added that node to the engine in your setup
func playPitchedAudio(audioFile: AVAudioFile, pitch: Float){
audioPlayerNode.stop()
theTimePitchNode.pitch = pitch
audioPlayerNode.scheduleFile(audioFile, atTime: nil, completionHandler: nil)
do {
try audioEngine.start()
} catch {
print("error")
}
audioPlayerNode.play()
}