I'm struggling with an AudioQueue audio player I implemented. I initially thought it was truncating the 1st 1/2 of audio that it played but upon loading larger files I notice gaps every other 1/2-1 second. I've run it in debug and I've confirmed that I'm loading the queue correctly with audio. (There are no big zero regions loaded in the queue.) It plays without issue (no gaps) on the simulator but on device I get gaps as if its missing every other chunk of audio. In my app I decompress then pull audio from a memory NSMutableData object. I feed this data into the audio queue. I have a corresponding implementation in the same app that plays wave audio and this example works without issue on long and short audio clips. I'm comparing the wave implementation to the other which does the decompression. the only difference between the two is how I discover the audio meta data and where I get the audio samples for enqueuing. In the wave implementation I use AudioFileGetProperty and AudioFileReadPackets to get this data. In the other case I derive the data before hand using cached ivars loaded during callbacks from my decompressor. The meta data matches for both compressed and wave implementations. I've run the code in instruments and I don't see anything taking more than 1ms in my audio packet delivery/enqueuing logic during playback. I'm completely lost. Please speak up if you have any idea how to solve the situation.
I finally resolved this issue. I found that if I skipped the 1st 44 bytes (the exact size of a wave header) of audio then it plays correctly on the device. It pays correctly on the sim regardless of wether I skip 44 or not. Strange and I'm not sure why but that's the way it works.
Related
We have a VOIP application, that records and plays audio. As such, we are using PlayAndRecord (kAudioSessionCategory_PlayAndRecord) audio session category.
So far, we have used it successfully with iPhone 4/4s/5 with both iOS 6 and iOS 7 where call audio and tones played clearly and were audible.
However, with iPhone 5s, we observed that both the call audio and tones sound robotic/garbled in speaker mode. When using earpiece/bluetooth/headset, sound is clear and audible.
iOS Version used with iPhone 5s: 7.0.4
We are using audiounits for recording/playing of call audio.
When setting audio properties like session category, audio route, session mode etc., we tried both the older (deprecated) AudioSessionSetProperty() and AVAudioSession APIs.
For playing tones, we are using AVAudioPlayer. Playing of tones during the VOIP call and also when pressing keypad controller within the app produces robotic sound.
When instantiating the audio component using AudioComponentInstanceNew, we set componentSubType to kAudioUnitSubType_VoiceProcessingIO.
When replacing kAudioUnitSubType_VoiceProcessingIO with kAudioUnitSubType_RemoteIO, we noticed that the sound of call audio and tones was no longer robotic, it was quite clear, but the volume level was very low when using speaker mode.
In summary, keeping all the other audio APIs the same:
kAudioUnitSubType_VoiceProcessingIO: Volume is high (desirable) but sound of tones and call audio was robotic in speaker mode.
kAudioUnitSubType_RemoteIO: Sound of tones and call audio was clear but it is not audible.
STEPS TO REPRODUCE
- Set audio session category to playAndRecord.
- Set audio route to speaker
- Set all the other audio properties like starting audio unit, activating the audio session, instantiating the audio components.
- Set the input and render callbacks
- Try both options
1. Play tones using AVAudioPlayer
2. Play call audio
Any suggestions on how to get over this issue. Raised as an issue with Apple but no response yet from them.
i have shared the code here github link
The only difference between kAudioUnitSubType_VoiceProcessingIO and kAudioUnitSubType_RemoteIO is that voiceProcessing includes code to tune out acoustic echo i.e. tunes out the noise from the speaker so the microphone doesn't pick it up. Its been a long time since I've played with the audio framework but I remember that to sound off there could be any number of things,
Are you doing any work in the audio callbacks that could be taking a long time?
The callbacks run on realtime threads. if your processing takes too long you can miss data. Would be helpful to track the data over a fixed period of time to see are you capturing it all. Use something like wireShark to sniff the network. Record the number of packets and see did the phone capture the same.
Are you modifying any of the audio?
Do you have a circular buffer that might be causing an issue?
I've had several issues doing this and one was using a third party circular buffer that was described as low latency and efficient ... it wasn't. I answered my own question here and included my circular buffer implementation that greatly improved my audio as the issue was I was skipping data.
Give this a go and let me know:
iOS UI are causing a glitch in my audio stream
Please be aware that some of this code is unique to the audio format ALaw, 0xD5 is a byte of silence in ALaw, if you are using linear PCM or any other that will probably be a noise of some kind.
I'm making a game for iOS platform and i want to use openAL for playing audio effects in the game (except the background music). I want to have for example 20-30 sounds with duration of 1-3 seconds each. Because i want these sounds to be played with no delay i have to load from file, decompress and store in the memory. But decompressed audio (as i understand) uses a lot of memory. So am I on the right way? Or there is another way ? Thanks
Loading sounds does incur a delay, and can cause the rest of the app to jitter. For best performance you definitely want to preload sound effects.
Memory use is a concern, but as long as you're not loading too much audio data into memory at a time, you'll be OK.
44KHz mono audio data will occupy 88,000 bytes per second when uncompressed. Stereo is double that, but usually for sound effects you don't want stereo anyway. So if you had 30 sounds loaded, each of 3 second duration, you'd have 90 seconds of sound using 7.5MB of memory. You can of course halve that memory usage by using a 22050 Hz source before compressing it to AAC (which preserves the source sample rate).
What I do is maintain a cache of audio buffers that I can flush when the app starts to use too much memory like so: https://github.com/kstenerud/ObjectAL-for-iPhone/blob/master/ObjectAL/ObjectAL/OALSimpleAudio.m#L441
For the lowest latency sound effects, you will want to play uncompressed audio already in memory. 2 or 3 MBytes is not a lot of memory to allocate for low latency sound effects.
I've been playing around with Apple's aurioTouch demo which is sample code for their Audio Unit tutorial. This application allows simultaneous input/output from the mic. to speaker. It also renders a stereograph of the inputted sound from the mic.
At a really high-level of this low-level process, the sample code defines an AudioComponent (in this case RemoteIO which allows for simultaneous input/output) and there is a render callback for this Audio Unit. In the callback they do some audio filtering (a DC Rejection Filter) and visualization of the stereograph based on the AudioBuffer sound data from the mic.
My ultimate goal is to create my own custom sound distortion Audio Unit based on the input from the mic. I think the proper way to do this based on the Audio Unit tutorial is to make a second Audio Unit and connect them with an Audio Processing Graph. However, I've read that iOS doesn't allow you to register your own custom Audio Units. My questions are:
Can I do direct manipulation on the AudioBufferList that I have access to in the render callback from the remoteIO Audio Unit (since they already seem to be doing this and applying an audio filter on it) and create my own custom sound distortion there?
I've tried assigning the AudioBufferList data to a constant (a value I've seen it hold from a sample run and logging of the AudioBufferList), but it appears to do nothing.
The answer to your first question is yes. That is generally how it is done.
I believe that you need to manipulate the data in the pointer directly, rather than reassigning. You may want to take a look at the code in openframeworks that handles assigning buffers and passing them to a callback: https://github.com/openframeworks/openFrameworks/blob/master/addons/ofxiPhone/src/sound/ofxiPhoneSoundStream.mm
There is other code out there that you can look at, nick collins has a basic application for getting sound off the microphone and out the speaker, whist processing inbetween: http://www.cogs.susx.ac.uk/users/nc81/code.html. He also has code there that gets sample buffers out of a iPod track that may be useful to you.
It is true that you can not add your own custom AudioUnits to the iPhone.
The way it works it like this: The speaker drives the pull-chain of data through the system. You add a render callback to the ioUnit, as you have already done.
The callback runs whenever the speaker (bus #0) is hungry and it is your job to fill in as many samples as it has requested, in the buffer that the speaker has provided. The size of the provided buffer will be a power of two that is as close as possible to the Preferred IO Buffer Duration you specified when configuring the AudioSession.
The simplest way to do that is to take the AudioBufferList you were given and pass it to AudioUnitRender on the the microphone (bus #1). After you fill in the buffer with Render(), but before the callback returns, you can manipulate the data any way you like. For example, AurioTouch zeroes it out to mute it.
The important thing to remember is that the speaker is going to read from the actual data buffer that it passed you. It is not going to look at the AudioBufferList descriptor and check if you pointed to another databuffer. If you start changing the AudioBufferList that you were given, you will run into problems. At best, you will be ignored. At worst, you will run into memory management issues.
If you don't want to be constrained to working in just the ioData buffer, then you can use your own AudioBufferList, allocated any way you like, in any size, and ask the microphone to Render() into that. Then you can do all the manipulation you like so long as in the end you copy the results into the buffer provided by the callback (i.e. ioData->mBuffers[0].mData as it was at the time the Callback was invoked).
I'm making an iPhone game application using Core Audio, Extended Audio File Services.
It works OK, but when I first call AudioOutputUnitStart, it takes about 1-2 seconds.
After the second call, no problem.
For a game application, 1-2 seconds is very noticeable.
(I tested this on iPhone simulator, and iPhone 3GS)
Also, if I leave the game for about 10 seconds, first call of AudioOutputUnitStart after the leave also takes time.
I'm using a 5-second AAC 160kbps file to play.
For a 5-second file, why not convert that file into a linear PCM wav or aif file? It shouldn't be that much bigger, plus you don't have to involve the AAC decoder in the process.
I'm working on a project which is using Audio Toolbox for recording and playback of PCM data, and I'm having trouble with playback. In the simulator, I can record and play audio just fine, using a custom class to handle storing and sourcing PCM bytes for the recording and playback buffers as needed. On device (iPhone (3.0.1) and iPod 2G (3.1.2)) recording works fine, the audio files produced are correct, but in-app playback stutters, like it's only playing part of each playback buffer. My buffers are one second long, and I've got 3 buffers, which are preloaded before playback starts; stuttering occurs during those first 3 seconds as well, which I think rules out a latency problem.
I've written Audio Toolbox code before that worked, and I'm not doing anything strange here except that I'm using my own class to source PCM data instead of AudioFileReadBytes()
I know the data that comes out of my source is good, because it plays right in the sim, and it writes to disk as a correct audio file
I've played around with sample rates a bit; I'm normally using 11025Hz sampling to cut down on file size (it's all voice, so it sounds fine). at 44100Hz, but with the same size of buffers, I get the same stuttering problem, but the audio segments come a lot faster, about 4 times faster. That's why I think it's only playing part of each buffer.
The only reason I can conceive that it would only play part of each buffer is a latency problem... like the audio toolbox code is running out of full buffers while I'm still filling an empty one. But that would cause it to play the preloaded buffers correctly, and then start stuttering, and that doesn't happen, it stutters the whole way through
I've tried humongous buffers, like 10MB buffers, and I just get silence and a single stutter of audio at the end of playback. I've also tried preloading more buffers than normal, like 10 seconds worth of audio, and it behaves the same.
The audio session is being set with AVAudioSession, not the Audio Toolbox calls, and it's being set to the Playback category for playback
I have no idea how to try and attack this problem, it makes no sense to me that it works fine on the simulator but not the device.
Code for the playing callback and the set up for the audio queue services: http://pastebin.com/mfaa546c
It turns out that the use of NSData's GetBytes:length: was causing the problem. The buffer filled with that method was playing incorrectly. However, doing a memcpy from that buffer to another buffer would prevent the problem.