How to play multiple footstep sounds without blocking animations? - swift

Am coding a game with scenes that a man runs in different distance and durations. Due to different running speeds and durations(0.009sec, 0.02sec, 0.05sec...), it's not a good way to playing a sound with multiple steps inside. By now, It is set to play one step sound when footstep moves once.
I tried the following code:
Steps play one by one, but the animation thread blocked. Then man runs unsmoothly
DispatchQueue.main.async {
Sound.shared.play(.step)
}
The animation goes smoothly, but some step sounds missing, it could be two or more sounds play at the same time, so they sound like one step sound.
DispatchQueue.global(qos: .background).async {
Sound.shared.play(.step)
}
Same as the second
let soundQueue = DispatchQueue.init(label: "soundQueue", qos: .default, attributes: [.concurrent], autoreleaseFrequency: .inherit, target: nil)
soundQueue.async {
Sound.shared.play(.step)
}
Supplement Info: First 30 step sounds AVAudioPlayer are stored in an array so that sounds can be played in a fast way.
func playSound(_ type: SoundType) {
switch type {
case .key:
case .step:
self.doPlayStepSound()
//....
default:
break
}
var stepPlayerArr = [AVAudioPlayer?]()
func doPlayStepSound() {
if stepCount >= 30 {
stepCount = 0
}
if let player = stepPlayerArr[stepCount] {
player.play()
}
stepCount += 1
}
How can I play each step sound when the man's footstep moves without blocking the animations?
Thanks.

Related

Syncing AVPlayer play (start) time with Timecode

I need my app to play a video (avplayer) at a specific time. Time in my app is represented through time code ("23:59:59:23").
I have a videos that need to play at specific timecode - for example
video.mp4 start at 00:01:03:22 (minute 1, second 3, frame 22)
What I have tried so far:
I have successfully setup my app to observe states within my avplayer based on this:media playback state
From what I've read I can achieve what I want by using AVPlayer.setRate. Although I really can't get my head around if this is what I need/how to use it.
My code below. Where I'm now stuck is what to do with everything you see from mulFactor to player.setRate. All taken from here. The below code plays all three of the videos in my app immediately. Assuming this is in fact the right way to achieve what I need, I'm guessing my next step will be replacing hostTime: with my timecodesource and time: with the time of playback????
if keyPath == #keyPath(AVPlayerItem.status) {
let status: AVPlayerItem.Status
if let statusNumber = change?[.newKey] as? NSNumber {
status = AVPlayerItem.Status(rawValue: statusNumber.intValue)!
} else {
status = .unknown
}
// Switch over status value
switch status {
case .readyToPlay:
// Player item is ready to play.
print("item is ready to play")
player.preroll(atRate: 1.0, completionHandler: {_ in print("prerolling avplayer")})
player.automaticallyWaitsToMinimizeStalling = false
let mulFactor = 24.0
let timeScale = CMTimeScale(mulFactor)
let seekCMTime = CMTime(value: CMTimeValue(CGFloat(1000.0)), timescale: timeScale)
let syncTime = CMClockGetHostTimeClock()
let hostTime = CMClockGetTime(syncTime)
player.setRate(1.0, time: seekCMTime, atHostTime: hostTime)
break
case .failed: break
// Player item failed. See error.
case .unknown: break
// Player item is not yet ready.
#unknown default:
print("FATAL ERROR IN AVPLAYER")
}
}

AVPlayerItem.seek failing

I'm playing back audio (mp3) downloaded from a URL using an AVQueuePlayer, and have two problems that might be related:
I've implemented a playback bar, but when that tries to change the currently playing time, it never succeeds. This is the problem I'm asking about here, but it might be linked to the second issue.
After a spell, the audio begins looping. I'm testing this with a recording of background noise I made that is 7m23 long. I hear something at 43s, and then again at 4m06, 5m11 and 6m19 (I might have missed it a couple of other times)
Files are being downloaded from Twilio, which I think is backed by AWS. I create the audio player with:
static var audioPlayer: AVQueuePlayer = AVQueuePlayer()
class func startPlayingFromUrl(url: String){
if let audioURL = URL(string: url){
let audioAsset = AVURLAsset(url: audioURL)
let newPlayerItem = AVPlayerItem(asset: audioAsset)
newPlayerItem.canUseNetworkResourcesForLiveStreamingWhilePaused = true
audioPlayer.insert(newPlayerItem, after: nil)
NotificationCenter.default.addObserver(self, selector:#selector(self.playerDidFinishPlaying(_:)),name: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: audioPlayer.currentItem)
audioPlayer.play()
}
}
The seeking is done with
class func seekToTime(seconds: Int){
let targetTime = CMTimeMakeWithSeconds(Float64(seconds), preferredTimescale: 600)
if let currentItem: AVPlayerItem = audioPlayer.currentItem{
print("Found the current item, trying to skip forward to \(targetTime), currentItem duration is \(currentItem.duration)")
audioPlayer.seek(to: targetTime){(someBoolean) -> () in
print("The completion handler is \(someBoolean)")
}
}
}
The output of the debug message when I'm trying to skip to 205 seconds is:
Found the current item, trying to skip forward to CMTime(value: 123000, timescale: 600, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0), currentItem duration is CMTime(value: 0, timescale: 0, flags: __C.CMTimeFlags(rawValue: 17), epoch: 0)
The completion handler is false
So I think the seek time is being constructed correctly, but perhaps the problem is related to the duration of the currentItem? I don't know what CMTimeFlags means, the values as specified in the Apple docs (https://developer.apple.com/documentation/coremedia/cmtimeflags) match up to the integer values as specified on the Microsoft docs (https://learn.microsoft.com/en-us/dotnet/api/coremedia.cmtime.flags?view=xamarin-ios-sdk-12) so nobody is laying claim to the number 17...
I tried to wrap the creation, setup and playing of the AVPlayerItem inside audioAsset.loadValuesAsynchronously(forKeys: ["duration", "playable"]) {} but it made no difference - duration seems to be being loaded just fine, so maybe the value is legit?

AudioKit Creating Sinewave Tone When Returning from Background

I'm using AudioKit to run an AKSequencer() that plays both mp3 and wav files using AKMIDISampler(). Everything works great, except in cases when the app has entered background state for more than 30+ min, and then brought back up again for use. It seems to then lose all of it's audio connections and plays the "missing file" sinewave tone mentioned in other threads. The app can happily can enter background momentarily, user can quit, etc without the tone. It seems to only happen when left in background for long periods of time and then brought up again.
I've tried changing the order of AudioKit.start() and file loading, but nothing seems to completely eliminate the issue.
My current workaround is simply to prevent the user's display from timing out, however that does not address many use-cases of the issue occurring.
Is there a way to handle whatever error I'm setting up that creates this tone? Here is a representative example of what I'm doing with ~40 audio files.
//viewController
override func viewDidLoad() {
sequencer.setupSequencer()
}
class SamplerWav {
let audioWav = AKMIDISampler()
func loadWavFile() {
try? audioWav.loadWav("some_wav_audio_file")
}
class SamplerMp3 {
let audioMp3 = AKMIDISampler()
let audioMp3_akAudioFile = try! AKAudioFile(readFileName: "some_other_audio_file.mp3")
func loadMp3File() {
try? audioMp3.loadAudioFile(audioMp3_akAudioFile)
}
class Sequencer {
let mixer = AKMixer()
let subMix = AKMixer()
let samplerWav = SamplerWav()
let samplerMp3 = SamplerMp3()
var callbackTrack: AKMusicTrack!
let callbackInstr = AKMIDICallbackInstrument()
func setupSequencer{
AudioKit.output = mixer.mixer
try! AudioKit.start()
callbackTrack = sequencer.newTrack()
callbackTrack?.setMIDIOutput(callbackInstr.midiIn)
samplerWav.loadWavFile()
samplerMp3.loadMp3File()
samplerWav.audioWav >>> subMix
samplerMp3.audioMp3 >>> submix
submix >>> mixer
}
//Typically run from a callback track
func playbackSomeSound(){
try? samplerWav.audioWav.play(noteNumber: 60, velocity: 100, channel: 1)
}
}
Thanks! I'm a big fan of AudioKit.
After some trial and error, here's a workflow that seems to address the issue for my circumstance:
-create my callback track(s) -once- from viewDidLoad
-stop AudioKit, and call .detach() on all my AKMIDISampler tracks and any routing in willResignActive
-start AudioKit (again), and reload and reroute all of the audio files/tracks from didBecomeActive

Audiokit crashes when changing AKPlayer file

I have recently done the migration from AudioKit 3.7 to 4.2 (using Cocoapods), needed for XCode 9.3. I followed the migration guide and changed AKAudioPlayer to AKPlayer.
The issue
When AKPlayer plays an audio file, AudioKit is crashing with this error:
2018-04-17 09:32:43.042658+0200 hearfit[3509:2521326] [avae] AVAEInternal.h:103:_AVAE_CheckNoErr: [AVAudioEngineGraph.mm:3632:UpdateGraphAfterReconfig: (AUGraphParser::InitializeActiveNodesInOutputChain(ThisGraph, kOutputChainFullTraversal, *conn.srcNode, isChainActive)): error -10875
2018-04-17 09:32:43.049372+0200 hearfit[3509:2521326] *** Terminating app due to uncaught exception 'com.apple.coreaudio.avfaudio', reason: 'error -10875'
*** First throw call stack:
(0x1847d6d8c 0x1839905ec 0x1847d6bf8 0x18a0ff1a0 0x18a11bf58 0x18a12aab0 0x18a128cdc 0x18a1a1738 0x18a1a160c 0x10519192c 0x10519d2f4 0x10519d64c 0x10503afdc 0x10507c4a0 0x10507c01c 0x104f6d9cc 0x1852233d4 0x18477faa8 0x18477f76c 0x18477f010 0x18477cb60 0x18469cda8 0x18667f020 0x18e67d78c 0x10504dfd4 0x18412dfc0)
libc++abi.dylib: terminating with uncaught exception of type NSException
Sometimes it happens on the first play, and sometimes the first play is done correctly, but not the second one.
Everything was working great before the migration. I also tried to keep AKAudioPlayer: sounds are played correctly but AKFrequencyTracker does not work anymore.
Context
This is my setup:
Quick explanation:
AKPlayer 1 plays short audio files (between 1 and 5 seconds)
AKFrequencyTracker is used to display a plot
AKPlayer 2 plays background sound (volume must be configurable)
AKWhiteNoise allows to do some manual volume measurements (using AKMixer 2 volume property)
Use case example
The user starts an exercise. A sound is played continuously (with looping) using AKPlayer 2 and the user listens a word (played with AKPlayer 1), the plot is displayed. Next, several words are displayed on screen and the user must pick the right one. And a new word is listened... and so on.
So I have to change dynamically the played file of AKPlayer 1. All the code is written in a dedicated class, a singleton. All the nodes are setup in the init() function.
// singleton
static let main = AudioPlayer()
private init() {
let silenceUrl = Bundle.main.url(forResource: "silence", withExtension: "m4a", subdirectory: "audio")
self.silenceFile = silenceUrl!
self.mainPlayer = AKPlayer(url: self.silenceFile)!
self.mainPlayer.volume = 1.0
self.freqTracker = AKFrequencyTracker(self.mainPlayer, hopSize: 256, peakCount: 10)
let noiseUrl = Bundle.main.url(forResource: "cocktail-party", withExtension: "m4a", subdirectory: "audio")
self.noiseFile = noiseUrl!
self.noisePlayer = AKPlayer(url: self.noiseFile)!
self.noisePlayer.volume = 1.0
self.noisePlayer.isLooping = true
let mixer = AKMixer(self.freqTracker, self.noisePlayer)
self.whiteNoise = AKWhiteNoise(amplitude: 1.0)
self.whiteNoiseMixer = AKMixer(self.whiteNoise)
self.whiteNoiseMixer.volume = 0
self.mixer = AKMixer(mixer, self.whiteNoiseMixer)
AudioKit.output = self.mixer
do {
try AudioKit.start()
} catch (let error) {
print(error)
}
// stop directly the white noise mixer
self.whiteNoise.stop()
self.whiteNoiseMixer.volume = self.whiteNoiseVolume
self.mainPlayer.completionHandler = {
DispatchQueue.main.async {
if let timer = self.timer {
timer.invalidate()
self.timer = nil
}
if let completion = self.completionHandler {
Timer.scheduledTimer(withTimeInterval: 0.5, repeats: false, block: { (_) in
completion()
self.completionHandler = nil
})
}
}
}
}
To change the AKPlayer 1 audio file, I use this function, on the same class:
func play(fileUrl: URL, tracker: #escaping TrackerCallback, completion: (() -> Void)?) throws {
self.completionHandler = completion
let file = try AKAudioFile(forReading: fileUrl)
self.mainPlayer.load(audioFile: file)
self.mainPlayer.preroll()
self.timer = Timer.scheduledTimer(withTimeInterval: self.trackerRefreshRate, repeats: true) { (timer) in
tracker(self.freqTracker.frequency, self.freqTracker.amplitude)
}
self.mainPlayer.play()
}
Thank you.
I'm not sure what you are replacing into the player, but if the format of the file is different from what you had before, channels, samplerate, etc -- you should create a new AKPlayer instance rather than load into the same one. If your files are all the same format then it should work ok.
That said, I haven't seen the crash you show.
Another thing that is dangerous in your code is force unwrapping those optionals - you should guard against things being nil. AKPlayer actually uses AVAudioFile, no need for AKAudioFile.
guard let akfile = try? AVAudioFile(forReading: url) else { return }
if akfile.channelCount != player?.audioFile?.processingFormat.channelCount ||
akfile.sampleRate != player?.audioFile?.processingFormat.sampleRate {
AKLog("Need to create new player as formats have changed.")
}

For Loop wait and delay - Swift 4

I have a nested for loop, and am trying to make it so that the outer loop will only continue once the inner loop and its code is completed, and also add a 1 second delay before performing the next loop.
for _ in 0...3 {
for player in 0...15 {
// CODE ADDING MOVEMENTS TO QUEUE
}
updateBoardArray()
printBoard()
// NEED TO WAIT HERE FOR 1 SEC
}
So I wan't the 0...3 For loop to progress only once the inner loop and update and print functions have completed their cycle, and also a 1 second wait time.
At the moment it all happens at once and then just prints all 4 boards instantly, even when I put that 1 second wait in using DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 1) {}.
Have tried other answers to similar questions but can't seem to get it to work.
As I understand, what you tried to do is the following:
for _ in 0...3 {
for player in 0...15 {
// CODE ADDING MOVEMENTS TO QUEUE
}
updateBoardArray()
printBoard()
DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 1) {}
}
This will NOT work since what you are doing is add a task (that will do nothing) to the main queue that will get trigged after 1 second, and after adding it, the code continues with the for, without waiting.
Solution
What you could do is simply use sleep(1), but bear in mind that this will freeze the main thread (if you are executing this in the main thread).
To avoid freezing the app you could do is:
DispatchQueue.global(qos: .default).async {
for _ in 0...3 {
for player in 0...15 {
// CODE ADDING MOVEMENTS TO QUEUE
}
DispatchQueue.main.async {
updateBoardArray()
printBoard()
}
sleep(1)
}
}
}
Just keep in mind that any UI action you do, must be done in the main thread
asyncAfter() function takes DispatchTime. DispatchTime is in nanosecond. Following is prototype:
func asyncAfter(deadline: DispatchTime, execute: DispatchWorkItem)
Following is extract of docs from Apple Developer webpage:
DispatchTime represents a point in time relative to the default clock with nanosecond precision. On Apple platforms, the default clock is based on the Mach absolute time unit
To solve the problem, use:
DispatchQueue.main.asyncAfter(deadline: DispatchTime.now() + 1000) {}
You can use scheduledTimer:
for i in 0..3{
Timer.scheduledTimer(withTimeInterval: 0.2 * Double(i), repeats: false) { (timer) in
//CODE
}
}
More Info about scheduledTimer.