TLDR: How to jumpstart a webviews audioSession without reloading its content?
Hello everyone. I am having a very specific issue with the wkwebview on iOS in regards to the AudioSession.Category management. I guess I start with that it is well understood that the wkwebview operates on its own process and therefor owns its own AudioSession. That means that any changes on the APPs AudioSession won't reflect on the webviews AudioSession.
We build an app, which is entirely web based and has to be for reasons I don't want to go into, so our IOS app is essentially a wrapper for the web content. Recent web content required us to use webrtc functionality do do audio recordings which is only available from iOS 14.3 upwards, which required us to work out a solution, which is backwards compatible down to iOS 12. The idea was to create a native audio recorder, which then interfaces with the webview to transfer audio recordings to the web content.
In order to make recordings within the app the AudioSession.Category needs to change to playAndRecord and is triggered by the webviews content through the messageHandler interface. A change of the APP category will trigger an interruption of the webview audioSession, which from then on does not play any media anymore (until a new url is loaded(single page apps do not make new requests in general)). To fix this a reload of the current url is needed to un-interrupted the sound, which can be somewhat masked behind loading screens but isn't a really good solution.
I am looking for ways to start the audio again without reloading the webview.
Thanks for any insights on this.
I've recently stuck with similar problem. Here is what I've found.
Whenever background music like YouTube in WKWebView or Music app is playing and your app audio session category or mode (but not options [1]) is changed while app's audio session is active background audio is interrupted. Even when re-configuration of audio session is happen between mixable configurations. The solution I've found is to deactivate app audio session prior applying new configuration and activating it again. And in case of playAndRecord it is important not to forget to add [.mixWithOthers]/[.mixWithOthers, .duckOthers] option. For some reason I can't find formal documentation on this behavior but confirmed it with simple demo app which provide manual way to activate/deactivate audio session and configure it to variety of category/mode/options.
[1] It is possible to switch between [.mixWithOthers] and [.mixWithOthers, .duckOthers] on active audio session and not to interrupt background audio, if category and mode are not changed during reconfiguration.
Related
I'm doing some audio streaming with iOS 7's Multipeer Connectivity framework. The streaming works well, but when I put the app on background it stops working.
Someone can tell me if this is a framework limitation, or I'm doing something wrong?
And, if it is a framework limitation, is it possible to do something to avoid this?
Can I use Background Tasks, to keep streaming and music working on background?
Is possible do this? If is not possible, do any alternatives exist for MultiPeer audio streaming between iOS devices?.
I´m using this example: https://github.com/tonyd256/TDAudioStreamer.
Explained on this page: http://robots.thoughtbot.com/streaming-audio-to-multiple-listeners-via-ios-multipeer-connectivity.
Thanks a lot!
On the Apple documentation for playing audio in the background (scroll down a bit). Some relevant paragraphs:
When the UIBackgroundModes key contains the audio value, the system’s media frameworks automatically prevent the corresponding app from being suspended when it moves to the background. As long as it is playing audio or video content or recording audio content, the app continues to run in the background. However, if recording or playback stops, the system suspends the app.
You can use any of the system audio frameworks to work with background audio content, and the process for using those frameworks is unchanged.
This means that iOS should recognize that you're playing audio through Core Audio, and keep your app unsuspended, as long as you've correctly configured your app for playing audio in the background.
Because your app is not suspended while playing media files, callbacks operate normally while your app is in the background. In your callbacks, though, you should do only the work necessary to provide data for playback. For example, a streaming audio app would need to download the music stream data from its server and push the current audio samples out for playback. Apps should not perform any extraneous tasks that are unrelated to playback.
You should be able to operate normally as long as your app is still playing audio, and is allowed to do what it needs to in order to continue playing audio. This means that you should be able to continue to use MPC in the background to receive the audio data and play it.
Be sure to read the entire documentation on the subject, especially regarding Audio Sessions.
iOS devices get limited cpu cycles for explicit purposes when they have been backgrounded by the user.
According to Apple's documentation on multitasking and execution in the background, the following types of apps are supported, but have to be explicitly declared:
Apps that play audible content to the user while in the background, such as a music player app
Apps that record audio content while in the background.
Apps that keep users informed of their location at all times, such as a navigation app
Apps that support Voice over Internet Protocol (VoIP)
Apps that need to download and process new content regularly
Apps that receive regular updates from external accessories
Your case falls under Apps that play audible content to the user while in the background, such as a music player app. You can find more information from the link provided above.
I have my v 1.1 of my app (iphone) which has been rejected from Apple.
Reasons for rejection:
2.16
We found that your app uses a background mode but does not include
functionality that requires that mode to run persistently. This
behavior is not in compliance with the App Store Review Guidelines.
We noticed your app declares support for audio in the
UIBackgroundModes key in your Info.plist, but no audible content is
played when the application is in the background. While your intention
may have been to provide this functionality, at the time of review, we
were not able to play background audio for your app.
As indicated in the iOS Application Programming Guide: "This key is
intended for use by applications that provide audible content to the
user while in the background, such as music-player or streaming-audio
applications." Therefore, it would be appropriate to provide audible
content to the user while the app is in the background or remove the
"audio" setting from the UIBackgroundModes key.
In the v 1.0 which has been accepted the required background mode "App audio plays" was already enable.
When I launch one sound of my app the sound is played and I when click on the lock screen the sound continues to be played in background.
In the new version (v1.1) I added the function which detects when I push the button Home of the iPhone. If the button home is pressed so the sound is paused.
When i try on my device it's run correctly and without bug.
So I don't understand exactly what is the problem with my app?
If you're pausing the sound when you hit the home button, then you aren't playing a sound in the background. Either change your app so that it plays sound in the background all the time, or email Apple and explain that your app needs the background functionality so that it can play while the phone is locked.
I'm not sure but you might be able to play the sound during the lock screen without audio background mode. Try removing it and see if your app behaves the same. If it does, then you should remove the key (no reason to have it).
If you use the audio background mode, your app has to play continuous audio (like the Music app, or Downcast for example). Is that what yours does? Or does it just play a sound briefly?
You should be able to play a short sound (< 10 mins) by using beginBackgroundTaskWithExpirationHandler: (I think!)
See below the answer to my Question from Apple :
Jul 31, 2012 10:21 AM. From Apple.
Hello,
Thank you for providing this information. However, in order to ensure that the audio background feature is properly implemented, it would be appropriate to press on the home button once the audio is launched.
For more information on how to implement this feature, please refer to iOS Application Programming Guide
Please make necessary change to resolve this issue.
So my application for iphone4 reads data from the accelerometer and sends it to another application via tcp sockets. I need my app to work in background mode, so what I did was:
I put an mp3 file in the application's Documents folder
I used AVAudioPlayer library to play the file in a loop. It works.
I edited Info.plist and added option "required background
modes" with "audio" on.
Still, the scheduler suspends the application whenever I press the iphone's home button. Is there anything I missed?
I read apple's documentation, but I didn't find a solution. A few thoughts on this:
do I have to edit appDelegate.m?
is it because I use AVAudioPlayer instead of the iPod?
is it because I play an audio file from the application documents
folder?
I read about one person changing iOS Development Target from 4.0 to
3.2.1, but that didn't work for me.
And finally, say I get this to work, would the application still be getting data from the accelerometer?
On a side note, I don't want to submit the application to the App Store.
No, you will not receive accelerometer notifications in background mode. As far as I know, it is not possible. Check Executing Code in Background.
If you read the docs carefully, you will know that the whole background code model is based on responding to specific events (location and voip modes).
As for the audio mode here is an extract from Apple:
Your application should limit itself to doing only the work necessary
to provide data for playback while in the background. For example, a
streaming audio application would download any new data from its
server and push the current audio samples out for playback. You should
not perform any extraneous tasks that are unrelated to playing the
content.
Not sure whether you have solved your issue or not since this question was posted more than one year ago. Also, not sure whether playing audio is a must in your app or not. If both answers are no, my recent investigation may help a bit.
Here are how I get my app getting accelerometer data at the background
1. Follow this tutorial http://mobile.tutsplus.com/tutorials/iphone/ios-multitasking-background-location/ to get the background location working.
2. Follow this tutorial http://jonathanhui.com/ios-motion to get the accelerometer working.
Then you can get an app collecting accelerometer data at the background. Hope this helps.
Ok, I'm trying to let a user choose songs from their iPod library to listen to, but I still want to receive remote control notifications (headphones, lock screen osd, etc.) in my app so I can do some extra things. So far I can get either iPod music playing, or headphone events, but not both simultaneously.
Here's what I know so far...
If you use the MPMusicPlayer, you can easily have programmatic access to the entire music library. However, it, not your app, receives the remote notifications regardless if you use applicationMusicPlayer or ipodMusicPlayer.
If you use AVAudioPlayer (Apple's recommended player for most sounds in your app), you can easily get remote notifications, but it doesn't natively have access to the iPod library.
AVAudioPlayer can be initialized with an asset URL, and tracks in the iPod library (type MPMediaItem) do have a URL property that returns a NSURL instance which the documentation says its explicitly for use with AVAsset objects, but when you try initializing the AVAudioPlayer with that NSURL, it fails. (I used the 'now playing' track in iPod which was a MP3 and it did return a valid NSURL object but initialization failed. Even worse, when it was an Audible.com file, the NSURL property flat-out returned nil.)
If you try using an instance of the AVAudioPlayer to get remote events (say, with a blank sound file), then simultaneously use the MPMusicPlayer class to play iPod music, you have remote control access until you actually start iPod playback at which time you lose it since your audio session gets deactivated and the system audio session becomes active.
If you try the same as #4 but you instead set the audio session's category to a mixable variant, your session doesn't get deactivated, but you still lose remote control capability once the iPod starts playing.
In short, whenever MPMusicPlayer is playing, I can't seem to get remote events, and I don't know of any other way to play content from the iPod's library other than by using MPMusicPlayer.
ANY suggestions on how to get around this would be welcome. Creative or flat-out crazy. Don't care so long as it works.
Anyone? Anyone? Bueller? Bueller?
M
HA! Solved! I knew it was possible! (Thanks Async-games.com support!)
Here's how to play iPod music in your app, with background support, and with your app receiving the remote control notifications.
You have to use AVPlayer (but not AVAudioPlayer. No idea why that is!) initialized with the asset URL from the MPMediaItem you got from the library picker (or current item in the MPMusicPlayerController or wherever), then set the audio session's category to Playable (do NOT enable the mixing override or you'll lose the remote events!) and add the appropriate keys to your info.plist file telling the OS your app wants to support background audio.
Done and done!
This lets you play items from your iPod library (except Audible.com files for some reason!) in the background and still get remote events. Granted since this is your audio player which is separate from, and will interrupt the iPod app, you have to do more work, but those are the breaks!
Damn though... I just wished it worked with Audible.com files. (For those interested, the reason it doesn't is the asset URL for an audible file returns nil. Stinks! But what can ya do!)
This is probably not going to be of any use anymore for the OP, but as it may be for people finding this page through googling, I will post it anyway.
An alternative (but rather ugly) approach, if you are only interested in the music remote control events and still want to be able to play the audible.com files...
Just keep using the MPMusicPlayer and track its notifications (now playing and state changed). To keep receiving these notifications in the background, you can do the "background thread magic" described in various places to keep your app from being suspended. You are not going to receive the remote controls directly (as the iPod player is receiving them), but by tracking the changes in "now playing" you can infer the ControlPreviousTrack and ControlNextTrack events, and by tracking the playbackState, you can infer the TogglePlayPause command.
The downside is that you are app is going to be running at all times for no good reason (although, to be fair, if iOS is programmed correctly, a background thread doing nothing should consume almost no battery).
Another alternative: use a MPMoviePlayer? I have checked that it works fine in the background, and should receive remote control events as well. It can play MPMediaItem natively, so hopefully the Audible.com files as well...
There is no way around this. If the users iPod app is playing an iPod selection, then all remote events are going to go to the iPod, not your app.
One think I noticed about MPMediaItemPropertyAssetURL is that, although the object returned is in NSURL but the absoluteString is something like this:
ipod-library://item/item.mp3?id=580807975475997900
Which is not what AVAudioPlayer want. What AVAudioPlayer want is NSURL object that is created from a file with a valid file path.
And I have no idea how to get file path from MPMediaItem. So I guess maybe AVPlayer is the way to go if you want to play iPod track without using MPMusicPlayer.
I'm a newbie to cocoa programing on iPhone.
My client has a website that plays YouTube videos. Once a video is finished playing, it will automatically play the next one. This is done by using the YouTube API and swfobject.
After some research, I was told that Safari on iPhone does not support flash. This make the current swfobject code not working on the iPhone browser.
As workaround, when the user clicked on an embedded player, iPhone will launch the YouTube app.
Is to possible to determine when the YouTube app has finished playing and has returned control back to browser?
You can use UIWebView to play videos from YouTube. You'll have to control those from your application though (start, stop, play next, etc).
Also In iOS you can register custom URI schemes and then redirect the browser (or UIWebView) back to your application. This is how many application do 3-legged OAuth for example (which requires a browser interaction). Which might require having a control over the server.
No, it is not possible to jump out of your App and then come back to it again. Once you leave your App, you are finished.
You'll need to stay in a UIWebView, or come up with another method of playing the YouTube videos from within your App.
-t