I have AVAudioPCMBuffer, I set it as a source to the AVAudioPlayerNode
internal func play() {
guard let buf = pcmBuf else {
logger?.log(severity: .error, msg: "Sound pcmBuf is nil.")
return
}
if !isPlaying {
player.scheduleBuffer(buf, at: nil, options: .loops)
player.play()
}
}
It is possible that the user seeks forward, in order to do it I need to offset the buffer and set it again, like this
internal func play() {
guard let buf = pcmBuf else {
logger?.log(severity: .error, msg: "Sound pcmBuf is nil.")
return
}
buf.offset = 10 // Eg: in sec
if !isPlaying {
player.scheduleBuffer(buf, at: nil, options: .loops)
player.play()
}
}
So, it will look like I set a new buffer, but with the needed offset, and the playback will start from the required point.
The problem is that there is no offset method...
How to do it?
I don't think you need to offset the buffer. You should pass your play function a valid second parameter instead of nil.
https://developer.apple.com/documentation/avfaudio/avaudioplayernode/1388422-schedulebuffer
"Schedules the playing samples from an audio buffer at the time and playback options you specify."
So pass in a valid AVAudioTime.
https://developer.apple.com/documentation/avfaudio/avaudioplayernode#1669195
init(sampleTime: AVAudioFramePosition,
atRate sampleRate: Double)
I am guessing this is probably the initialiser you want.
Related
I'm subclassing InputStream from iOS Foundation SDK for my needs. I need to implement functionality that worker thread can sleep until data appear in the stream. The test I'm using to cover the functionality is below:
func testStreamWithRunLoop() {
let inputStream = BLEInputStream() // custom input stream subclass
inputStream.delegate = self
let len = Int.random(in: 0..<100)
let randomData = randData(length: len) // random data generation
let tenSeconds = Double(10)
let oneSecond = TimeInterval(1)
runOnBackgroundQueueAfter(oneSecond) {
inputStream.accept(randomData) // input stream receives the data
}
let dateInFuture = Date(timeIntervalSinceNow: tenSeconds) // time in 10 sec
inputStream.schedule(in: .current, forMode: RunLoop.Mode.default) //
RunLoop.current.run(until: dateInFuture) // wait for data appear in input stream
XCTAssertTrue(dateInFuture.timeIntervalSinceNow > 0, "Timeout. RunLoop didn't exit in 1 sec. ")
}
Here the overriden methods of InputStream
public override func schedule(in aRunLoop: RunLoop, forMode mode: RunLoop.Mode) {
self.runLoop = aRunLoop // save RunLoop object
var context = CFRunLoopSourceContext() // make context
self.runLoopSource = CFRunLoopSourceCreate(nil, 0, &context) // make source
let cfloopMode: CFRunLoopMode = CFRunLoopMode(mode as CFString)
CFRunLoopAddSource(aRunLoop.getCFRunLoop(), self.runLoopSource!, cfloopMode)
}
public func accept(_ data: Data) {
guard data.count > 0 else { return }
self.data += data
delegate?.stream?(self, handle: .hasBytesAvailable)
if let runLoopSource {
CFRunLoopSourceSignal(runLoopSource)
}
if let runLoop {
CFRunLoopWakeUp(runLoop.getCFRunLoop())
}
}
But calling CFRunLoopSourceSignal(runLoopSource) and CFRunLoopWakeUp(runLoop.getCFRunLoop()) not get exit from runLoop.
Does anybody know where I'm mistaking ?
Thanks all!
PS: Here the Xcode project on GitHub
Finally I figured out some issues with my code.
First of all I need to remove CFRunLoopSource object from run loop CFRunLoopRemoveSource(). In according with documentation if RunLoop has no input sources then it exits immediately.
public func accept(_ data: Data) {
guard data.count > 0 else { return }
self.data += data
delegate?.stream?(self, handle: .hasBytesAvailable)
if let runLoopSource, let runLoop, let runLoopMode {
CFRunLoopRemoveSource(runLoop.getCFRunLoop(), runLoopSource, runLoopMode)
}
if let runLoop {
CFRunLoopWakeUp(runLoop.getCFRunLoop())
}
}
Second issue is related that I used XCTest environment and it's RunLoop didn't exit for some reasons (Ask the community for help).
I used real application environment and created Thread subclass to check my implementation. The thread by default has run loop without any input sources attached to it. I added input stream to it. And using main thread emulated that stream received data.
Here the Custom Thread implement that runs and sleep until it receive signal from BLEInputStream
class StreamThread: Thread, StreamDelegate {
let stream: BLEInputStream
init(stream: BLEInputStream) {
self.stream = stream
}
override func main() {
stream.delegate = self
stream.schedule(in: .current, forMode: RunLoop.Mode.default)
print("start()")
let tenSeconds = Double(10)
let dateInFuture = Date(timeIntervalSinceNow: tenSeconds)
RunLoop.current.run(until: dateInFuture)
print("after 10 seconds")
}
override func start() {
super.start()
}
func stream(_ aStream: Stream, handle eventCode: Stream.Event) {
if eventCode == .errorOccurred {
print("eventCode == .errorOccurred")
}
else if eventCode == .hasBytesAvailable {
print("eventCode == .hasBytesAvailable")
}
}
}
Here the some UIViewController methods which runs from main thread
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
let baseDate = Date.now
let thread = StreamThread(stream: stream, baseDate: baseDate)
thread.start()
print("main thread pauses at \(Date.now.timeIntervalSince(baseDate))")
Thread.sleep(forTimeInterval: 2)
print("stream accepts Data \(Date.now.timeIntervalSince(baseDate))")
stream.accept(Data([1,2,3]))
}
Here the result:
Everything works as expected - the thread sleeps until input stream receive data. No processor resources consuming.
Although it's allowed to subclass InputStream, there is no good explanation in the documentation how to correctly implement custom InputStream
I am implementing a recorder in my application using an AVAudioRecorder, but I’m encountering a strange behavior when an interruption is triggered by the system.
Indeed, when an interrupt is caught thanks to the AVAudioSession.interruptionNotification, I call the following function:
#objc private func handleInterruption(notification: Foundation.Notification) {
guard let interruptionTypeValue = notification.userInfo?[AVAudioSessionInterruptionTypeKey] as? UInt,
let interruptionType = AVAudioSession.InterruptionType(rawValue: interruptionTypeValue)
else { return }
switch interruptionType {
case .began:
pause()
case .ended:
guard let optionsValue = notification.userInfo?[AVAudioSessionInterruptionOptionKey] as? UInt else { return }
let options = AVAudioSession.InterruptionOptions(rawValue: optionsValue)
if options.contains(.shouldResume) {
resume()
} else {
// TODO:
}
#unknown default:
break
}
}
At the beginning of the interruption, I pause the recorder and at the end of it resume the recorder if needed.
However when I resume the recorder after the interruption, it restart the record, deleting the file previously created since I receive a callback from audioRecorderDidFinishRecording(_ recorder: AVAudioRecorder, successfully flag: Bool).
How to get around this problem?
Thanks
I found similar questions here, here, and here but with none of the answers I have been able to solve the problem. Simply put, the audio does not jump at a specific moment but instead starts from scratch.
func seekTo(time: Double) {
player.stop()
let startSample = Double(time * audioSampleRate)
let lengthSamples: AVAudioFramePosition = AVAudioFramePosition(Double(audioLengthSamples) - startSample)
let frameCount = AVAudioFrameCount(audioLengthSamples - lengthSamples)
if currentPosition < audioLengthSamples {
player.scheduleSegment(audioFile!, startingFrame: AVAudioFramePosition(startSample), frameCount: AVAudioFrameCount(frameCount), at: nil, completionHandler: nil)
let wasPlaying = player.isPlaying
if wasPlaying {
player.play()
}
}
}
The time variable is where it receives the specific time to which the audio should skip.
Any help?
Right now I'm using AVAudioEngine, with AVAudioPlayer, AVAudioFile, AVAudioPCMBuffer to play couple a compressed soundtrack (m4a). My problem is that if the soundtrack is 40MB uncompressed and 1.8 in m4a when I load the sound in the buffer, the memory usage jump by 40MB (the uncompressed size of the file). How can I optimise that to use as little memory as possible?
Thanks.
let loopingBuffer : AVAudioPCMBuffer!
do{ let loopingFile = try AVAudioFile(forReading: fileURL)
loopingBuffer = AVAudioPCMBuffer(pcmFormat: loopingFile.processingFormat, frameCapacity: UInt32(loopingFile.length))!
do {
try loopingFile.read(into: loopingBuffer)
} catch
{
print(error)
}
} catch
{
print(error)
}
// player is AVAudioPlayerNode
player.scheduleBuffer(loopingBuffer, at: nil, options: [.loops])
Well, as a workaround, I decided to create a wrapper to split the audio into chunk of few second and playing and buffering them one at the time into the AVAudioPlayerNode.
As a result only a few seconds are RAM (twice that when buffering) at any time.
It brung the memory usage for my use case from 350Mo to less than 50Mo.
Here is the code, don't hesitate to use it or improve it (it's a first version). Any comments are welcome!
import Foundation
import AVFoundation
public class AVAudioStreamPCMPlayerWrapper
{
public var player: AVAudioPlayerNode
public let audioFile: AVAudioFile
public let bufferSize: TimeInterval
public let url: URL
public private(set) var loopingCount: Int = 0
/// Equal to the repeatingTimes passed in the initialiser.
public let numberOfLoops: Int
/// The time passed in the initialisation parameter for which the player will preload the next buffer to have a smooth transition.
/// The default value is 1s.
/// Note : better not go under 1s since the buffering mecanism can be triggered with a relative precision.
public let preloadTime: TimeInterval
public private(set) var scheduled: Bool = false
private let framePerBuffer: AVAudioFrameCount
/// To identify the the schedule cycle we are executed
/// Since the thread work can't be stopped when they are scheduled
/// we need to be sure that the execution of the work is done for the current playing cycle.
/// For exemple if the player has been stopped and restart before the async call has executed.
private var scheduledId: Int = 0
/// the time since the track started.
private var startingDate: Date = Date()
/// The date used to measure the difference between the moment the buffering should have occure and the actual moment it did.
/// Hence, we can adjust the next trigger of the buffering time to prevent the delay to accumulate.
private var lastBufferingDate = Date()
/// This class allow us to play a sound, once or multiple time without overloading the RAM.
/// Instead of loading the full sound into memory it only reads a segment of it at a time, preloading the next segment to avoid stutter.
/// - Parameters:
/// - url: The URL of the sound to be played.
/// - bufferSize: The size of the segment of the sound being played. Must be greater than preloadTime.
/// - repeatingTimes: How many time the sound must loop (0 it's played only once 1 it's played twice : repeating once)
/// -1 repeating indéfinitly.
/// - preloadTime: 1 should be the minimum value since the preloading mecanism can be triggered not precesily on time.
/// - Throws: Throws the error the AVAudioFile would throw if it couldn't be created with the URL passed in parameter.
public init(url: URL, bufferSize: TimeInterval, isLooping: Bool, repeatingTimes: Int = -1, preloadTime: TimeInterval = 1)throws
{
self.url = url
self.player = AVAudioPlayerNode()
self.bufferSize = bufferSize
self.numberOfLoops = repeatingTimes
self.preloadTime = preloadTime
try self.audioFile = AVAudioFile(forReading: url)
framePerBuffer = AVAudioFrameCount(audioFile.fileFormat.sampleRate*bufferSize)
}
public func scheduleBuffer()
{
scheduled = true
scheduledId += 1
scheduleNextBuffer(offset: preloadTime)
}
public func play()
{
player.play()
startingDate = Date()
scheduleNextBuffer(offset: preloadTime)
}
public func stop()
{
reset()
scheduleBuffer()
}
public func reset()
{
player.stop()
player.reset()
scheduled = false
audioFile.framePosition = 0
}
/// The first time this method is called the timer is offset by the preload time, then since the timer is repeating and has already been offset
/// we don't need to offset it again the second call.
private func scheduleNextBuffer(offset: TimeInterval)
{
guard scheduled else {return}
if audioFile.length == audioFile.framePosition
{
guard numberOfLoops == -1 || loopingCount < numberOfLoops else {return}
audioFile.framePosition = 0
loopingCount += 1
}
let buffer = AVAudioPCMBuffer(pcmFormat: audioFile.processingFormat, frameCapacity: framePerBuffer)!
let frameCount = min(framePerBuffer, AVAudioFrameCount(audioFile.length - audioFile.framePosition))
print("\(audioFile.framePosition/48000) \(url.relativeString)")
do
{
try audioFile.read(into: buffer, frameCount: frameCount)
DispatchQueue.global().async(group: nil, qos: DispatchQoS.userInteractive, flags: .enforceQoS) { [weak self] in
self?.player.scheduleBuffer(buffer, at: nil, options: .interruptsAtLoop)
self?.player.prepare(withFrameCount: frameCount)
}
let nextCallTime = max(TimeInterval( Double(frameCount) / audioFile.fileFormat.sampleRate) - offset, 0)
planNextPreloading(nextCallTime: nextCallTime)
} catch
{
print("audio file read error : \(error)")
}
}
private func planNextPreloading(nextCallTime: TimeInterval)
{
guard self.player.isPlaying else {return}
let id = scheduledId
lastBufferingDate = Date()
DispatchQueue.global().asyncAfter(deadline: .now() + nextCallTime, qos: DispatchQoS.userInteractive) { [weak self] in
guard let self = self else {return}
guard id == self.scheduledId else {return}
let delta = -(nextCallTime + self.lastBufferingDate.timeIntervalSinceNow)
self.scheduleNextBuffer(offset: delta)
}
}
}
I'm trying to use Apple's AVMIDIPlayer object for playing a MIDI file. It seems easy enough in Swift, using the following code:
let midiFile:NSURL = NSURL(fileURLWithPath:"/path/to/midifile.mid")
var midiPlayer: AVMIDIPlayer?
do {
try midiPlayer = AVMIDIPlayer(contentsOf: midiFile as URL, soundBankURL: nil)
midiPlayer?.prepareToPlay()
} catch {
print("could not create MIDI player")
}
midiPlayer?.play {
print("finished playing")
}
And it plays for about 0.05 seconds. I presume I need to frame it in some kind of loop. I've tried a simple solution:
while stillGoing {
midiPlayer?.play {
let stillGoing = false
}
}
which works, but ramps up the CPU massively. Is there a better way?
Further to the first comment, I've tried making a class, and while it doesn't flag any errors, it doesn't work either.
class midiPlayer {
var player: AVMIDIPlayer?
func play(file: String) {
let myURL = URL(string: file)
do {
try self.player = AVMIDIPlayer.init(contentsOf: myURL!, soundBankURL: nil)
self.player?.prepareToPlay()
} catch {
print("could not create MIDI player")
}
self.player?.play()
}
func stop() {
self.player?.stop()
}
}
// main
let myPlayer = midiPlayer()
let midiFile = "/path/to/midifile.mid"
myPlayer.play(file: midiFile)
You were close with your loop. You just need to give the CPU time to go off and do other things instead of constantly checking to see if midiPlayer is finished yet. Add a call to usleep() in your loop. This one checks every tenth of a second:
let midiFile:NSURL = NSURL(fileURLWithPath:"/Users/steve/Desktop/Untitled.mid")
var midiPlayer: AVMIDIPlayer?
do {
try midiPlayer = AVMIDIPlayer(contentsOfURL: midiFile, soundBankURL: nil)
midiPlayer?.prepareToPlay()
} catch {
print("could not create MIDI player")
}
var stillGoing = true
while stillGoing {
midiPlayer?.play {
print("finished playing")
stillGoing = false
}
usleep(100000)
}
You need to ensure that the midiPlayer object exists until it's done playing. If the above code is just in a single function, midiPlayer will be destroyed when the function returns because there are no remaining references to it. Typically you would declare midiPlayer as a property of an object, like a subclassed controller.
Combining Brendan and Steve's answers, the key is sleep or usleep and sticking the play method outside the loop to avoid revving the CPU.
player?.play({return})
while player!.isPlaying {
sleep(1) // or usleep(10000)
}
The original stillGoing value works, but there is also an isPlaying method.
.play needs something between its brackets to avoid hanging forever after completion.
Many thanks.