I'm currently developing an app which plays an audio file (mp3, but can change to WAV to reduce decoding time), and records audio at the same time.
For synchronization purposes, I want to estimate the exact time when audio started playing.
Using AudioQueue to control each buffer, I can estimate the time when the first buffer was drained. My questions are:
What is the hardware delay between AudioQueue buffers being drained and them actually being played?
Is there a lower level API (specifically, AudioUnit), that has better performance (in hardware latency measures)?
Is it possible to place an upper limit on hardware latency using AudioQueue, w or w/o decoding the buffer? 5ms seems something that I can work with, more that that will require a different approach.
Thanks!
The Audio Queue API runs on top of Audio Units, so the RemoteIO Audio Unit using raw uncompressed audio will allow a lower and more deterministic latency. The minimum RemoteIO buffer duration that can be set on some iOS devices (using the Audio Session API) is about 6 to 24 milliseconds, depending on application state. That may set a lower limit on both play and record latency, depending on what events you are using for your latency measurement points.
Decoding compressed audio can add around an order of magnitude or two more latency from decode start.
Related
I am attempting to stream live audio from an iOS device to a web browser. The iOS device sends small, mono wav files (as they are recorded) through a web socket. Once the client receives the wav files, I have the Web Audio API decode and schedule them accordingly.
This gets me about 99% of the way there, except I can hear clicks between each audio chunk. After some reading around, I have realized the likely source of my problem: the audio is being recorded at a sample rate of only 4k and this cannot be changed. It appears that the Web Audio API's decodeAudioData() function does not handle sample rates other than 44.1k with exact precision resulting in gaps between chunks.
I have tried literally everything I could find about this problem (scriptProcessorNodes, adjusting the timing, creating new buffers, even manually upsampling) and none of them have worked. At this point I am about to abandon the Web Audio API.
Is the Web Audio API appropriate for this?
Is there a better alternative for what I am trying to accomplish?
Any help/suggestions are appreciated, thanks!
Alas! AudioFeeder.js works wonders. I just specify the sampling rate of 4k, feed it raw 32 bit pcm data and it outputs a consistent stream of seamless audio! Even has built in buffer handling events, so no need to set any loops or timeouts to schedule chunk playback. I did have to tweak it a bit, though, to connect it to the rest of my web audio nodes and not just context.destination.
Note: AudioFeeder does automatically upsample to the audio context sampling rate. Going from 4k to 44.1k did introduce some pretty gnarly sounding artifacts in the highend, but a 48db lowpass filter (4 x 12db's) at 2khz got rid of them. I chose 2khz because, thanks to Harry Nyquist, I know that a sampling rate of 4k couldn't have possibly produced frequencies above 2khz in the original file.
All hail Brion Vibbers
What is the lowest input->output audio passthru latency possible with iPhone 4 / iOS 4.2? I just want to take input from the mic and play it over the headphones with the smallest delay. I'd like to know the theoretical minimum and minimum actually observed, if possible.
An app can usually configure Audio Unit RemoteIO input/record and output/play buffers of length 256 at a 44.1k sample rate. Thus 6 to 12 mS is probably a lowest bound, just from the minimum iOS API buffer filling/handling delay (and not including OS, driver, DAC, amp, speaker, and speed-of-sound-in-air time-of-flight delays).
I need to implement audio streaming on iPhone with latency lower than 50 millisecond .
Any ideas on how I can make it work?
I bumped into:
http://cocoawithlove.com/2009/06/revisiting-old-post-streaming-and.html
But it's very important to me to know that the latency will be very low.
thanks
One way to minimize latency on the receiving end is to use the RemoteIO Audio Unit with very short buffers, and feed it from raw PCM audio or a decompressor for an audio format that requires extremely low computational complexity to decode, as well as small packets.
You pretty much need complete control over the entire network source and path, including hand picking all the equipment, as any router or access point can completely destroy latency by buffeting packets or prioritizing other traffic, etc.
You probably want to use UDP for the IP protocol, with a packet size tuned to your network equipment and to the audio buffer size.
I want to apply an audio filter on the users voice in iPhone.
The filter is quite heavy and needs many audio samples to get the desired quality. I do not want to apply the filter in realtime but I want to have an almost realtime performance. I would like the processing to happen in parrallel with the recording when the nessesary samples are collected and when the user stops recording to hear (after a few seconds) the distorted sound.
My questions are:
1. Which is the right technology layer for this task e.g. audio units?
2. Which are the steps involved?
3. Which are the key concepts and API methods to use?
4. I want to capture the users voice. Which are the right recording settings for this? If my filter alter alters the frequency should I use a wider range?
5. How can I collect the necessary samples for my filter? How can I handle the audio data? I mean depending on the recording settings how the data are packed?
6. How can I wright the final audio recording to a file?
Thanks in advance!
If you find a delay of over a hundred milliseconds acceptable, you can use the Audio Queue API, which is a bit simpler than using the RemoteIO Audio Unit, for both capture and audio playback. You can process the buffers in your own NSOperationQueue as the come in from the audio queue, and either save the processed results to a file or just kept in memory if there is room.
For Question 4: If your audio filter is linear, then you won't need any wider frequency range. If you are doing non-linear filtering, all bets are off.
I want to playback an mp3 rather than an uncompressed file using RemoteIO / AudioUnit.
Using uncompressed files obviously uses far too much disk space (30MB vs 3MB for mp3).
Is this even possible? If so, can you provide a little code headstart?
Thanks a million.
How low-level do you want to go? You could use the AudioHardware API:
err = AudioDeviceAddIOProc(deviceID, ioProc, self);
and in your ioProc fill the buffers yourself, listen for hardware changes, and deal with real-time threading and a lot of other low-level stuff.
I'm assuming you're already decoding the mp3 data with AudioConverter and know how to do that.
Yes, it is possible. But it requires multiple audio API use and multiple threads.
Due to the real-time constraints of the Audio Unit buffer callback thread, you will have do the conversion of compressed files to raw PCM samples outside the Audio Unit callback. You could use Extended Audio File Services or AVAssetReader to do the conversion to uncompressed samples outside the Audio Unit callback. However, you don't need to uncompress the entire file at once. A short uncompressed buffer of a fraction of a second will likely do, as long as you keep filling it far enough ahead of the Audio Unit callback buffer consumption rate. This can be done in a separate timer driven thread that monitors buffer consumption, and decompresses just enough audio accordingly, perhaps into a ring buffer or circular FIFO.
What you will end up with will be similar to a rewrite of the Audio Queue API layer, but with more customizability.