How to play looping compressed soundtrack without using to much ram? - swift

Question

Right now I'm using AVAudioEngine, with AVAudioPlayer, AVAudioFile, AVAudioPCMBuffer to play couple a compressed soundtrack (m4a). My problem is that if the soundtrack is 40MB uncompressed and 1.8 in m4a when I load the sound in the buffer, the memory usage jump by 40MB (the uncompressed size of the file). How can I optimise that to use as little memory as possible?
Thanks.
let loopingBuffer : AVAudioPCMBuffer!
do{ let loopingFile = try AVAudioFile(forReading: fileURL)
loopingBuffer = AVAudioPCMBuffer(pcmFormat: loopingFile.processingFormat, frameCapacity: UInt32(loopingFile.length))!
do {
try loopingFile.read(into: loopingBuffer)
} catch
{
print(error)
}
} catch
{
print(error)
}
// player is AVAudioPlayerNode
player.scheduleBuffer(loopingBuffer, at: nil, options: [.loops])

Answer 1

Well, as a workaround, I decided to create a wrapper to split the audio into chunk of few second and playing and buffering them one at the time into the AVAudioPlayerNode.
As a result only a few seconds are RAM (twice that when buffering) at any time.
It brung the memory usage for my use case from 350Mo to less than 50Mo.
Here is the code, don't hesitate to use it or improve it (it's a first version). Any comments are welcome!
import Foundation
import AVFoundation
public class AVAudioStreamPCMPlayerWrapper
{
public var player: AVAudioPlayerNode
public let audioFile: AVAudioFile
public let bufferSize: TimeInterval
public let url: URL
public private(set) var loopingCount: Int = 0
/// Equal to the repeatingTimes passed in the initialiser.
public let numberOfLoops: Int
/// The time passed in the initialisation parameter for which the player will preload the next buffer to have a smooth transition.
/// The default value is 1s.
/// Note : better not go under 1s since the buffering mecanism can be triggered with a relative precision.
public let preloadTime: TimeInterval
public private(set) var scheduled: Bool = false
private let framePerBuffer: AVAudioFrameCount
/// To identify the the schedule cycle we are executed
/// Since the thread work can't be stopped when they are scheduled
/// we need to be sure that the execution of the work is done for the current playing cycle.
/// For exemple if the player has been stopped and restart before the async call has executed.
private var scheduledId: Int = 0
/// the time since the track started.
private var startingDate: Date = Date()
/// The date used to measure the difference between the moment the buffering should have occure and the actual moment it did.
/// Hence, we can adjust the next trigger of the buffering time to prevent the delay to accumulate.
private var lastBufferingDate = Date()
/// This class allow us to play a sound, once or multiple time without overloading the RAM.
/// Instead of loading the full sound into memory it only reads a segment of it at a time, preloading the next segment to avoid stutter.
/// - Parameters:
/// - url: The URL of the sound to be played.
/// - bufferSize: The size of the segment of the sound being played. Must be greater than preloadTime.
/// - repeatingTimes: How many time the sound must loop (0 it's played only once 1 it's played twice : repeating once)
/// -1 repeating indéfinitly.
/// - preloadTime: 1 should be the minimum value since the preloading mecanism can be triggered not precesily on time.
/// - Throws: Throws the error the AVAudioFile would throw if it couldn't be created with the URL passed in parameter.
public init(url: URL, bufferSize: TimeInterval, isLooping: Bool, repeatingTimes: Int = -1, preloadTime: TimeInterval = 1)throws
{
self.url = url
self.player = AVAudioPlayerNode()
self.bufferSize = bufferSize
self.numberOfLoops = repeatingTimes
self.preloadTime = preloadTime
try self.audioFile = AVAudioFile(forReading: url)
framePerBuffer = AVAudioFrameCount(audioFile.fileFormat.sampleRate*bufferSize)
}
public func scheduleBuffer()
{
scheduled = true
scheduledId += 1
scheduleNextBuffer(offset: preloadTime)
}
public func play()
{
player.play()
startingDate = Date()
scheduleNextBuffer(offset: preloadTime)
}
public func stop()
{
reset()
scheduleBuffer()
}
public func reset()
{
player.stop()
player.reset()
scheduled = false
audioFile.framePosition = 0
}
/// The first time this method is called the timer is offset by the preload time, then since the timer is repeating and has already been offset
/// we don't need to offset it again the second call.
private func scheduleNextBuffer(offset: TimeInterval)
{
guard scheduled else {return}
if audioFile.length == audioFile.framePosition
{
guard numberOfLoops == -1 || loopingCount < numberOfLoops else {return}
audioFile.framePosition = 0
loopingCount += 1
}
let buffer = AVAudioPCMBuffer(pcmFormat: audioFile.processingFormat, frameCapacity: framePerBuffer)!
let frameCount = min(framePerBuffer, AVAudioFrameCount(audioFile.length - audioFile.framePosition))
print("\(audioFile.framePosition/48000) \(url.relativeString)")
do
{
try audioFile.read(into: buffer, frameCount: frameCount)
DispatchQueue.global().async(group: nil, qos: DispatchQoS.userInteractive, flags: .enforceQoS) { [weak self] in
self?.player.scheduleBuffer(buffer, at: nil, options: .interruptsAtLoop)
self?.player.prepare(withFrameCount: frameCount)
}
let nextCallTime = max(TimeInterval( Double(frameCount) / audioFile.fileFormat.sampleRate) - offset, 0)
planNextPreloading(nextCallTime: nextCallTime)
} catch
{
print("audio file read error : \(error)")
}
}
private func planNextPreloading(nextCallTime: TimeInterval)
{
guard self.player.isPlaying else {return}
let id = scheduledId
lastBufferingDate = Date()
DispatchQueue.global().asyncAfter(deadline: .now() + nextCallTime, qos: DispatchQoS.userInteractive) { [weak self] in
guard let self = self else {return}
guard id == self.scheduledId else {return}
let delta = -(nextCallTime + self.lastBufferingDate.timeIntervalSinceNow)
self.scheduleNextBuffer(offset: delta)
}
}
}