I'm developing an interactive storybook type application for the iPhone and I've recently encountered a frustrating bug concerning audio mixing on the device.
Firstly, I setup an audio session. I set the category to AVAudioSessionCategoryAmbient and then init and play my AVAudioPlayer instance. Now, in the background whilst the audio is playing I'm pre-loading a video to play using an MPMoviePlayerController followed by a call to prepareToPlay. The reason I pre-load the video this way is because I need it to play instantly later on cue with fairly strict timing.
In this configuration, the audio/movie works fine and they mix and do not interrupt each other. However, this particular audio session category does not permit audio to continue playing while the device is locked, a feature I really need. As a result I'm forced to consider a different category: AVAudioSessionCategoryPlayback.
By default this category does not permit mixing with other audio, according to the Apple docs. To enable mixing with other audio I am overriding the relevant category:
OSStatus propertySetError = 0;
UInt32 setProperty = 1;
propertySetError = AudioSessionSetProperty(kAudioSessionProperty_OverrideCategoryMixWithOthers, sizeof(setProperty), &setProperty);
assert(propertySetError == 0);
Unfortunately, this solves my playing whilst locked issue but introduces another issue: the AVAudioPlayer audio is interrupted briefly as the video loads with a minor stutter. The stutter is small, perhaps less than a second but is enough to disrupt the user experience. I've read this related post which enabled me to pre-load the video with the AVAudioSessionCategoryAmbient, but unfortunately the same approach doesn't seem to work with the new category.
The audio session category is applied successfully, according to the return code. Does anyone know why enabling audio mixing with this category is not the same as the mixing facility provided by ambient category?
The best way I've found working a similar problem is to use the newer AVPlayer (+1 #adam) and set your app to enable background audio and receive remote control notifications. I was tipped off to this by #MarquelV following How can you play music from the iPod app while still receiving remote control events in your app?
If you can get backgrounding working properly, that should enable you to continue playing while the device is locked. Oh, and don't forget to add keys to info.plist, its easy to do and then have no idea why it isn't working.
Related
We have a VOIP application, that records and plays audio. As such, we are using PlayAndRecord (kAudioSessionCategory_PlayAndRecord) audio session category.
So far, we have used it successfully with iPhone 4/4s/5 with both iOS 6 and iOS 7 where call audio and tones played clearly and were audible.
However, with iPhone 5s, we observed that both the call audio and tones sound robotic/garbled in speaker mode. When using earpiece/bluetooth/headset, sound is clear and audible.
iOS Version used with iPhone 5s: 7.0.4
We are using audiounits for recording/playing of call audio.
When setting audio properties like session category, audio route, session mode etc., we tried both the older (deprecated) AudioSessionSetProperty() and AVAudioSession APIs.
For playing tones, we are using AVAudioPlayer. Playing of tones during the VOIP call and also when pressing keypad controller within the app produces robotic sound.
When instantiating the audio component using AudioComponentInstanceNew, we set componentSubType to kAudioUnitSubType_VoiceProcessingIO.
When replacing kAudioUnitSubType_VoiceProcessingIO with kAudioUnitSubType_RemoteIO, we noticed that the sound of call audio and tones was no longer robotic, it was quite clear, but the volume level was very low when using speaker mode.
In summary, keeping all the other audio APIs the same:
kAudioUnitSubType_VoiceProcessingIO: Volume is high (desirable) but sound of tones and call audio was robotic in speaker mode.
kAudioUnitSubType_RemoteIO: Sound of tones and call audio was clear but it is not audible.
STEPS TO REPRODUCE
- Set audio session category to playAndRecord.
- Set audio route to speaker
- Set all the other audio properties like starting audio unit, activating the audio session, instantiating the audio components.
- Set the input and render callbacks
- Try both options
1. Play tones using AVAudioPlayer
2. Play call audio
Any suggestions on how to get over this issue. Raised as an issue with Apple but no response yet from them.
i have shared the code here github link
The only difference between kAudioUnitSubType_VoiceProcessingIO and kAudioUnitSubType_RemoteIO is that voiceProcessing includes code to tune out acoustic echo i.e. tunes out the noise from the speaker so the microphone doesn't pick it up. Its been a long time since I've played with the audio framework but I remember that to sound off there could be any number of things,
Are you doing any work in the audio callbacks that could be taking a long time?
The callbacks run on realtime threads. if your processing takes too long you can miss data. Would be helpful to track the data over a fixed period of time to see are you capturing it all. Use something like wireShark to sniff the network. Record the number of packets and see did the phone capture the same.
Are you modifying any of the audio?
Do you have a circular buffer that might be causing an issue?
I've had several issues doing this and one was using a third party circular buffer that was described as low latency and efficient ... it wasn't. I answered my own question here and included my circular buffer implementation that greatly improved my audio as the issue was I was skipping data.
Give this a go and let me know:
iOS UI are causing a glitch in my audio stream
Please be aware that some of this code is unique to the audio format ALaw, 0xD5 is a byte of silence in ALaw, if you are using linear PCM or any other that will probably be a noise of some kind.
Is there a way to determine whether VoiceOver is currently announcing and when it stops. I've tried UIAccessibilityVoiceOverStatusChanged but my understanding is that this is only if you switch VoiceOver on or off. Any help would be greatly appreciated. thanks.
We use otherAudioIsPlaying, the problem is some app's running in the background like some pedometer monitors etc. turn on the audio it seems and never release it so even though nothing is actually being spoken or played otherAudioIsPlaying always returns 1 until you remove the other application from the background. So now not only can you not play music but you have no idea that another application in the background will mess up this test. Apple really needs to put in an API to determine if Voice Over is currently speaking or not.
These are all the Accessibility booleans that I found in the documentation:
UIAccessibilityPostNotification
UIAccessibilityIsVoiceOverRunning
UIAccessibilityIsMonoAudioEnabled
UIAccessibilityIsClosedCaptioningEnabled
UIAccessibilityRegisterGestureConflictWithZoom
I don't think that there are any booleans to do what you are talking about.
You could use the audio session's "OtherAudioIsPlaying" property to check if another system process is using the audio hardware at the moment. It should be "true" if VoiceOver is speaking and "false" if not.
Actually this might not work properly if the user is playing music in the background. But most users running VoiceOver will usually not have any other audio enabled permanently, since it makes it harder to understand what VoiceOver is saying.
Here is an example for usage:
UInt32 otherAudioIsPlaying;
UInt32 propertySize = sizeof(otherAudioIsPlaying);
AudioSessionGetProperty(kAudioSessionProperty_OtherAudioIsPlaying,&propertySize, &otherAudioIsPlaying);
if(otherAudioIsPlaying) {
// other application is generating sound output (including VoiceOver)
// but might also be any other app (like iPod App)
}
I have an audio app that is having some problems with the way iOS 5 has changed audio behaviors. When my app's audio is playing (AVAudioSessionCategoryPlayback), and a Clock.app alarm or timer is fired from the OS, the UIAlertView notification pops up, but without the audio alert. My application sound ducks fine to get out of the way of the audio alert, but the alarm app's audio alert does not sound.
Naturally, tons of support requests poured in over the iOS 5 change. I have solved this temporarily by setting kAudioSessionProperty_OverrideCategoryMixWithOthers which lets the alarm audio come through, but there are a few very undesirable side-effects when doing this:
Other app's audio can play with/over mine.
The remote control events are not routed to my app, but to iPod.app.
None of the above drawbacks are acceptable for my app's requirements. I have been hacking away at this for some time now but haven't been able to crack it. How can I setup my audio such that:
My app's audio still uses the AVAudioSessionCategoryPlayback category for background audio.
The Clock.app alarms still have their audio alerts make sound
The app still responds to remote control notifications
After writing this question I went to file a bug report on this. I created a small sample project that I thought would replicate the issue, but I could not replicate it! This caused me to dig in deep once again and try to figure out what was up here…
I fired up an iOS alarm, then I placed a break point in audioPlayerBeginInterruption: and traced through my code line by line in the debugger. I noticed that before my code ran (while I was paused in the debugger), the iOS 5 alarm was sounding! Luckily it still sounded even as I was stepping through my app, so I was able to figure out which pieces of code specifically caused it to stop sounding.
Part of my interruptionHandler is to (obviously) stop the internal audio of my app to let the interruption come through. I never thought to inspect this method before, but turns out the problem existed in there. My stop method would call prepareToPlay immediately after stopping to make resuming faster the next time.
[self.player stop];
[self.player prepareToPlay]; // <- iOS 5 alarm sound stopped here.
The docs state the prepareToPlay method
preloads buffers and acquires the audio hardware needed for playback, which minimizes the lag between calling the play method and the start of sound output.
Sounds reasonable, and this worked for lesser iOS versions. My hypothesis is that must have made a change to the Clock.app alarm system such that the new alarm sounds use the hardware, whereas before it used the software. This is what I think is causing the iOS 5 alarms to be silent in some apps.
Removing the prepareToPlay lines caused the alarm to sound without using kAudioSessionProperty_OverrideCategoryMixWithOthers, thus solving all my issues laid out in this question.
TL;DR
Remove the prepareToPlay calls from your stop sound code logic. It will take a microsecond longer to start later, but will allow interruptions to sound.
In my app i record and play audio at the same time. The app is almost finished. But there is one thing, that annoying me. When audio session is set to PlayAndRecord, sounds become quiet in comparison with the same sounds with the SoloAmbient category. Is there any way to make sound louder using PlayAndRecord?
when you use the session for play and record, the playback comes out of the speaker used for the phone, otherwise it comes out the speaker located at the bottom of the phone. this is to prevent feedback. you can override this like so (but watch out for feedback, not an issue if you aren't doing both at once)
//when the category is play and record the playback comes out of the speaker used for phone conversation to avoid feedback
//change this to the normal or default speaker
UInt32 doChangeDefaultRoute = 1;
AudioSessionSetProperty (kAudioSessionProperty_OverrideCategoryDefaultToSpeaker, sizeof (doChangeDefaultRoute), &doChangeDefaultRoute);
this code works on 3.1.2, earlier sdk's you have to do differently.
UInt32 audioRouteOverride = kAudioSessionOverrideAudioRoute_Speaker;
status = AudioSessionSetProperty (kAudioSessionProperty_OverrideAudioRoute, sizeof (audioRouteOverride), &audioRouteOverride);
you have to be careful with this method, it will override even if you have headphones plugged in, you have to monitor interruptions and change the routes accordingly. much better now using 3.1.2
Ask the user to plug in headphones?
The headphone + mic combination doesn't suffer from this problem.
I don't know if it's a bug, a consequence of the audio hardware,
or if the quiet playback is just an intentional and hamfisted
way of getting cleaner recordings.
UPDATE
I found out that setting the PlayAndRecord session changes your audio route to the receiver.
Apparently the use case is for telephony applications where the user holds the device up to his ear.
If that doesn't violate the Principle of Least Surprise, I don't know what does.
I need to use AVAudioPlayer and MPMoviePlayerController simultaneously i.e play a movie while the background loop is playing is it possible.
My bg music loop stops working when a movie starts playing , so I tried to stop the bg music loop and start the movie and when movie stops, start playing the bg loop again but this is also is not working.
In fact it is achievable from 3.x onwards. I had once done this in one of my applications and its got nothing to do with any backgrounding or any other new iOS features. You just need to configure the audio sessions properly. If you make sure that both your audio player as well as the video player are using the ambient audio session it is possible.
Prior to starting your AVAudioPlayer to play, set the category of audio session to AVAudioSessionCategoryAmbient. This is the system wide common audio session. This allows other sounds to play along such as the sound from the iPod application.
In case you are using the MPMoviePlayerController from iOS 3.2 or above make sure you set the useApplicationAudioSession to NO. This ensures other sounds to play along. It is the default in earlier versions of iOS.
There is a limitation in the current iPhone SDK (2.x and 3.x) that does not allow any background audio playing while a movie is playing in the MPMoviePlayerController API. Unfortunately, it is not possible to do what you are asking at this time within the iPhone SDK.
I would suggest logging an enhancement request to Apple to offer this in a future version of the SDK.