I have 3 audioQueueBufferRef to fill my audioqueue and everyting works nice when data comes fast enough. But sometimes the outputCallback ask to fill a buffer when there is no more data. In this case i don't enqueue something and the callback is no more called. (normal ?)
After a this i run with 2 buffers and later (after another lag) 1 buffer and at the very last no more buffers have callbacks resulting in no sound.
I have try to store the empty audioQueueBufferRef in a array and call them when i have data to fill. But since data don't comes fast enough, the buffer just "eat" my low amount of data that remain and the sound is laggy.
What is the best way to go when running low on data to fill ?
The best i have foun is to let the buffer just dead without callback and let the remaining buffer do the job until every buffer are empty. After this i stop the audioQueue, store some data and play again. Notice that i have to call AudioqueueStop() because when every buffer are empty and i refill them, no sound is comming out. Is this normal ?
In general does i do it the good way or is there a better approach ?
And is there a callback that detect that all my audioqueueBufferRef are dead (without callback running) ?
2.
I tried this in the playroutine to test a solution for you:
cnt++;
if (cnt > 5) {
inBuffer->mAudioDataByteSize = 4;
AudioQueueEnqueueBuffer(inAQ, inBuffer, 0, NULL);
cnt = 0;
return;
}
It simulates that data arrives too late every 5th time AudioQueue asks for a buffer. Note that I sat the mAudioDataByteSize to size 4, which is the SMALLEST sample size for a 2 channel 16 bit wave file. If you set the size to zero, the queue will die and return the error message 'kAudioQueueErr_BufferEmpty', so you do need to feed data, although you don't have any to feed. If you think about it, it makes sense.
Maybe you could even adapt the code to estimate how much longer you need to wait to get data, let us say you will get more in about 2ms. Then you could just set the buffer size to the number of samples a tad less than that time. If you are playing a 2 channel 16 bit 44100 Hz wave song, 2ms would amount to a buffer size of 44100 Hz * 0.002 seconds = 88, 88 * 2 channels * (16 bits / 8) = 353 bytes.
I think the callback is called in a frequency you set , If you can't fill the data in time, the sound is abnormal. Does your data is from on network?
I have met the same condition as you and my solution is to record one packet data which is silence and when I do not have audio data to fill in, I use silence data instead.
Related
I'm running a test to measure the basic latency of my iPhone app, and the result was disappointing: 50ms for a play-through test app. The app just picks up mic input and plays it out using the same render callback, no other audio units or processing involved. Therefore, the results seemed too bad for such a basic scenario. I need some pointers to see if the result makes sense or I had design flaws in my test.
The basic idea of the test was to have three roles:
My finger snap as the reference sound source.
A simple iOS play-thru app (using built-in mic) as the first
listener to #1.
A Mac (with a USB mic and Audacity) as the second listener to #1 and
the only listener to the iOS output (through a speaker connected via
iOS headphone jack).
Then, with Audacity in recording mode, the Mac would pick up both the sound from my fingers and its "clone" from the iOS speaker in close range. Finally I simply visually observe the waveform in Audacity's recorded track and measure the time interval between the peaks of the two recorded snaps.
This was by no means a super accurate measurement, but at least the innate latency of the Mac recording pipeline should have been cancelled out this way. So that the error should mainly come from the peak distance measurement, which I assume should be much smaller than the audio pipeline latency and can be ignored.
I was expecting 20ms or lower latency, but clearly the result gave me 50~60ms.
My ASBD uses kAudioFormatFlagsCanonical and kAudioFormatLinearPCM as format.
50 mS is about 4 mS more than the duration of 2 audio buffers (one output, one input) of size 1024 at a sample rate of 44.1 kHz.
17 mS is around 5 mS more than the duration of 2 buffers of length 256.
So it looks like the iOS audio latency is around 5 mS plus the duration of the two buffers (the audio output buffer duration plus the time it takes to fill the input buffer) ... on your particular iOS device.
A few iOS devices may support even shorter audio buffer sizes of 128 samples.
You can use core audio and set up the audio session to have a very low latency.
You can set the buffer size to be smaller using AudioSessionSetProperty(kAudioSessionProperty_PreferredHardwareIOBufferDuration,...
Using smaller buffers causes the audio callback to happen more often while grabbing smaller chunks of audio. Keep in mind that this is merely a suggestion to the audio system. iOS will use a callback time suitable value based on your sample rate and integer powers of 2.
Once you set the buffer duration, you can get the actual buffer duration that the system will use using AudioSessionGetProperty(kAudioSessionProperty_CurrentHardwareIOBufferDuration,...
I'll summarize Paul R's comments as the answer, which has solved my problem:
50 ms corresponds to a total buffer size of around 2048 at a 44.1 kHz sample rate, which doesn't seem unreasonable given that you have both a record and a playback path.
I don't know that the buffer size is 2048, and there may be more than one buffer in your record-playback loopback test, but it seems that the effective total buffer size in you test is probably of the order of 2048, which doesn't seem unreasonable. Of course if you're only interested in record latency, as the title of your question suggests, then you'll need to find a way to tease that out separately from playback latency.
I've been working on a very specific project for iOS, lately, and my researches lead me to an almost final code. I've solved all the extreme difficulties I've found until now, but on this one I don't seem to have a clue (about the reason nor the possibility of solving it).
I set up my audioqueue (sample rate 44100, format LinearPCM, 16 bits per channel, 2 bytes per frame, 1 channel per frame...) and start recording the sound with 12 audio buffers. However, there seems to be a delay after every 4 callbacks.
The situation is the following: the first 4 callbacks are called with an interval each of about 2 ms. However, between the 4th and the 5th, there is a delay of about 60ms. The same thing happens between the 8th and the 9th, the 12th and 13th and on...
There seems to be a relation between the bytes per frame and the moment of the delay. I know this because if I change to 4 bytes per frame, I start having the delay between the 8th and the 9th, then between the 16th and the 17th, the 24th and the 25th... Nonetheless, there doesn't seem to be any relation between the moment of the delay and the number of buffers.
The callback function does only two things: store the audio data (inBuffer->mAudioData) on a array my class can use; and call another AudioQueueEnqueueBuffer, to put the current buffer back on the queue.
Did anyone go through this problem already? Does anyone know, at least, what could be the cause of it?
Thank you in advance.
The Audio Queue API seems to run on top of the RemoteIO Audio Unit API, who's real audio buffer size is probably unrelated to, and in your example larger than, whatever size your Audio Queue buffers are. So whenever a RemoteIO buffer is ready, a bunch of your smaller AQ buffers quickly get filled from it. And then you get a longer delay waiting for some larger buffer to be filled with samples.
If you want better controlled (more evenly spaced) buffer latency, try using the RemoteIo Audio Unit directly.
Regardless of the size of the buffers I provide the callback provided to AudioQueueNewInput occurs at roughly the same time interval.
For example:
If you have .05 second buffers and are recording at 44k the callback first called about at .09 seconds and then a second call occurs right after (.001 seconds). Then you wait again for ~.09 seconds. If your buffer size was .025. You would wait .09 seconds and then see 3 more buffers nearly instantly.
Changing the sample rate increases the latency.
Recording 16 bit 8k audio results in .5 seconds of latency between buffer floods.
So I suspect that there is an 8000 byte buffer that is being used behind the scenes. When it's filled my callback gets run with the given buffers until it is emptied.
I want to record 16k 16 bit audio with as little latency as possible. Given the above I always see about a quarter of a second of latency. Is there a way to decrease the latency? Is there an audio session property to set the internal buffer size? I've tried kAudioSessionProperty_PreferredHardwareIOBufferDuration but it does not seem to help.
thanks!
The Audio Queue API looks like it is built on top of the Audio Unit RemoteIO API. Small Audio Queue buffers are probably being used to fill a larger RemoteIO buffer behind the scenes. Perhaps even some rate resampling might be taking place (on the original 2G phone).
For lower latency, try using the RemoteIO Audio Unit API directly, and then requesting the audio session to provide your app a smaller lower latency buffer size.
I am streaming a MP3 over network using custom feeding code, not AVAudioPlayer (which only works with URLs) using APIs like AudioFileStreamOpen and etc.
Is there any way to estimate a length of the stream? I know that I can get a 'elapsed' property using:
if(AudioQueueGetCurrentTime(queue.audioQueue, NULL, &t, &b) < 0)
return 0;
return t.mSampleTime / dataFormat.mSampleRate;
But what about total duration to create a progress bar? Is that possible?
P.S. Clarification - I do know the actual size of the MP3 file, don't know if that can be used... I'll even settle for solution that just gives me a progress bar, not the actual time of play/duration.
If you know the total size of the MP3 file, you can calculate the bits per second, and therefore calculate the duration of the stream. If it's VBR, you'll probably have to average several MPEG frames. For CBR, you can simply use the bitrate of one packet.
Already finished implementing the player. I want to implement the progress bar. But I wonder if that's possible to do since we are streaming the music. Unless we are provided the length of the song before hand.
Please, I need your advice on this.
Unless you are given the size (or time length) of the audio file beforehand, there's no way you can set the range of your progress bar (you'll have no max).
Actually, I found a way to do that and I succeeded. That you calculate the number of frames per packet of the song, and the average packet's size (in bytes). Use NSHTTPConnection to get the file size and use this formular:
totalFrames = (fileSize * framesPerPacket / average packet size)
when you have total frames, just divide it by the bitrate, then u get urself the total time!!