Media handling with media response and exit handling with exit intents conflict somehow with users expectations. Links to the docs supplied below. Expected/actual behaviour also described.
Is there any chance to get that handled (at least in the near future) by defining custom utterances for media handling? As far as I know there is no possibility to define custom utterances/intents for "play" / "pause" / "stop" / "start over".
I set up a AoG to play streams through the media response.
When I want to completely end the conversation with an exit intent from within media response it doesn't really stop the conversation, but just pauses the player with no voice output whereas on any visual device it shows a play button instead of the pause button. On second "stop" utterance (or whatever calls the exit intent) the action finally finishes and plays the desired exit audio/shows desired text making clear that the action really ended.
Whereas this is somehow the expected behaviour this is still pretty annoying according user expectations. When a user says "STOP" s/he probably wants to end the conversation and not pause a stream, or am I wrong here? User case studies in our company showed that at least.
Solution would be to be able to add custom voice output when stopping media playback.
This is a known bug in the Media response.
Ending a conversation with a MediaResponse will open up player features like pause, forward, backwards and raising / lowering the volume. This is however currently broken on a Google Home device as the audio will never play. We had it working for several months until it broke and we had to switch to keeping the conversation open in order to make the audio play.
We have noticed that there are several undocumented commands which does strange things... For instance, when playing an audio with the conversation open and saying "Lämna" which is the Swedish equivalent to "Leave", the conversation will end and the audio will keep playing. The command "Stopp" will stop the audio but keep the conversation open. "Avsluta" = "End" will stop the audio and close the conversation.
None of these actions will callback to our backend and things seems to change on a weekly basis.
I submitted a background audio app for certification and has failed with two reasons in which I could not figure out why.
Reason 1:
This app failed to correctly respond to at least one of the play,
pause, or play/pause events.
I understand that the MediaControl events for Play, Pause, Stop and PlayPause need to be catered, and have done so (and tested on both tablets and local devices that they are working) in the code. However, due to the reason that stopping a media stream and restarting it requires a longer-than-expected time, I used MediaElement.Pause() for both "Pause" and "Stop".
I read another post who had similar problem at the certification phase. Somebody recommended to use MediaElement.PlaybackRate = 0; instead. However, this is not ideal for long pauses as the stream will not move on.
What I wish to know is am I doing this the right way? For all my MediaControl events I have made sure that the MediaControl.IsPlaying property is correctly set as well.
Also, another reason it failed was this:
App failed the Perf test in the Windows ACK. See the following links
for more information: Test cases ran:
http://msdn.microsoft.com/en-us/library/windows/apps/hh920274.aspx
I have ran my app against the ACK and it all passed. The only thing I can think of is that the app does not enter suspend mode when the hardware (or on-screen) media control pause button is pressed. I have placed a debugger in the App_Suspending event but it never hits there.
As the description is too vague I am not sure if this is the problem. But if it's the case, can I know how do I force the app to enter suspended mode? I tried looking in the Window.Current class and Application.Current class, but to no avail.
Thanks!
For your first issue be sure that your media element is ready to play using :
while (CurrentTrack.CurrentState == MediaElementState.Opening || CurrentTrack.CurrentState == MediaElementState.Buffering)
{
await Task.Delay(100);
}
CurrentTrack.Play();
Also you have to stop your media element when the view is unload.
Regards.
After nearly 10 attempts in releasing the app, I finally got to the root of the problem, thanks to some guessing work by the folks at Microsoft too.
My app will automatically start the MediaElement streaming after the app is started. The background-capable audio will prevent the app from passing WACK because it will never enter suspended mode!
So, in order to get pass the store's WACK I had to remove the auto-starting feature, and now the app is in the store! (Phew).
I know this question has been asked many times and we have gotten this link as an answer from Apple:
http://developer.apple.com/library/ios/#qa/qa1561/
However, recently, I met with this case below. I don't see any other options except using 'exit(0)' directly.
I have an app which can collect digital data from camera or microphone of my iPhone.
Then I designed a button which reads 'Save and Exit', because this is a very common scenario of my app.
Without the capability to exit my app programatically, I can only have a stupid 'Save' button. Then my users have to click twice ( 1. the 'Save' button, and then 2. the 'Home' button ) to get the same result - a.k.a these two actions in sequence together.
Then I realized, maybe, this is a flaw of my design. Hence I changed it this way.
I removed the 'Save the Exit' button and, instead, added a note in the bottom says 'Use HOME button to SAVE & EXIT'. I tried to catch the quitting signal from users' action and then do the SAVE in prior.
By doing this, user can simply use the native HOME button to do the 'SAVE and EXIT'.
However, this solution is not perfect.
If my app takes too long to save the data, there's still a chance for iOS to kill my app after user hits the 'HOME' button. Then user can lost their data without notice!
If I can exit programmatically, I can try to 'SAVE' first. If 'SAVE' fails, I have plenty of time to inform my user or do all the remediation work needed.
Do you think this is a valid justification to use 'exit(0)' in my app without being rejected?
No, I very much doubt that Apple will let you have a "save and exit" button. It goes completely against the platform conventions and directly contradicts Apple's HIG.
Under which circumstances do you think you'll have data that can't be saved in the amount of time iOS gives an app to clean up? Have you actually measured this and found it to be a problem?
Why don't you try just saving automatically all the time? You could save to a temp directory and then at exit do a file move operation. This would probably be much faster than your current save operation which sometimes fails.
When my app moves to the background while either playing audio or recording audio, I would like to provide something like the the green "In Call" status bar that appears when you are in a call and swap out to another app, giving you a quick "return to app" capability.
I thought this might be a private API, but I noticed the Griffin iTalk app does this when it is recording audio (see picture) - so I know it can be done, but I have not been able to figure out what the API is (the Info.plist setting, the AVAudioSession, the UIApplication/Delegate or the whatever) to make this magic happen. My app is currently working and recording audio in the background and works just fine, I assume there is some bit somewhere I'm not setting to get this behavior.
Can someone either point me to the right docs or sample code that exhibits this behavior? (I've scoured the Audio docs and haven't been able to find it).
Thanks!
There may be a private API to do this, but if you want to get into the App Store, the thing to do is to register as a VOIP app, which gets you the in-call status bar, like Skype or (I imagine) iTalk.
Actually the red-bar actually only appears when the application is recording audio in the background. I've only ever done this when marking the application VOIP and Audio.
You may be able to do this by only marking is as background Audio application only, but I haven't tried this.
I'm trying to get an audio file to autoplay in Safari on an iPad. If I go to the page using Safari on my Mac, it's fine. On the iPad, autoplay does not work.
I would like to also emphasize that the reason you cannot do this is a business decision that Apple made, not a technical decision. To wit, there was no rational technical justification for Apple to disable sound from web apps on the iPod Touch. That is a WiFi device only, not a phone that may incur expensive bandwidth charges, so that argument has zero merit for that device. They may say they are helping with WiFi network management, but any issues with bandwidth on my WiFi network is my concern, not Apple's.
Also, if what they really cared about was preventing unwanted, excessive bandwidth consumption, they would provide a setting to allow users to opt-in to web apps sounds. Also, they would do something to restrict web sites from consuming equivalent bandwidth by other means. But there are no such restrictions. A web site can download huge files in the background over and over and Apple could care less about that. And finally, I believe the sounds can be downloaded anyway, so NO BANDWIDTH IS ACTUALLY SAVED! Apple just does not allow them to play automatically without user interaction, making them unusable for games, which of course is their real intent.
Apple blocked sound because they started to notice that HTML5 apps can be just as capable as native apps, if not more so. If you want sound in Web Apps you need to lobby Apple to stop being anti-competitive like Microsoft. There is no technical problem that can be fixed here.
UPDATE: This is a hack and it's not working anymore on IOS 4.X and above. This one worked on IOS 3.2.X.
It's not true. Apple doesn't want to autoplay video and audio on IPad because of the high amout of traffic you can use on mobile networks. I wouldn't use autoplay for online content. For Offline HTML sites it's a great feature and thats what I've used it for.
Here is a "javascript fake click" solution: http://www.roblaplaca.com/examples/html5AutoPlay/
Copy & Pasted Code from the site:
<script type="text/javascript">
function fakeClick(fn) {
var $a = $('');
$a.bind("click", function(e) {
e.preventDefault();
fn();
});
$("body").append($a);
var evt,
el = $("#fakeClick").get(0);
if (document.createEvent) {
evt = document.createEvent("MouseEvents");
if (evt.initMouseEvent) {
evt.initMouseEvent("click", true, true, window, 0, 0, 0, 0, 0, false, false, false, false, 0, null);
el.dispatchEvent(evt);
}
}
$(el).remove();
}
$(function() {
var video = $("#someVideo").get(0);
fakeClick(function() {
video.play();
});
});
</script>
This is not my source. I've found this some time ago and tested the code on an IPad and IPhone with IOS 3.2.X.
Apple annoyingly rejects many HTML5 web standards on their iOS devices, for a variety of business reasons. Ultimately, you can't pre-load or auto-play sound files before a touch event, it only plays one sound at a time, and it doesn't support Ogg/Vorbis. However, I did find one nifty workaround to let you have a little more control over the audio.
<audio id="soundHandle" style="display:none;"></audio>
<script type="text/javascript">
var soundHandle = document.getElementById('soundHandle');
$(document).ready(function() {
addEventListener('touchstart', function (e) {
soundHandle.src = 'audio.mp3';
soundHandle.loop = true;
soundHandle.play();
soundHandle.pause();
});
});
</script>
Basically, you start playing the audio file on the very first touch event, then you pause it immediately. Now, you can use the soundHandle.play() and soundHandle.pause() commands throughout your Javascript and control the audio without touch events. If you can time it perfectly, just have the pause come in right after the sound is over. Then next time you play it again, it will loop back to the beginning. This won't resolve all the mess Apple has made here, but it's one solution.
As of iOS 4.2.1, neither the fake click nor the .load() trick will work to autoplay audio on any iOS device. The only option I know of is to have the user actually perform a click action and begin playback in the click event handler without wrapping the .play() call in anything asynchronous (ajax, setTimeout, etc).
Someone please tell me if I'm mistaken. I had working loopholes for both iPad and iPhone before the most recent update, and all of them look like they've been closed.
I confirm that the audio isn't working as described (at least on iPad running 4.3.5). The specific issue is the audio won't load in an asynchronous method (ajax, timer event, etc) but it will play if it was preloaded. The problem is the load has to be on a user-triggered event. So if you can have a button for the user to initiate the playing you can do something like:
function initSounds() {
window.sounds = new Object();
var sound = new Audio('assets/sounds/clap.mp3');
sound.load();
window.sounds['clap.mp3'] = sound;
}
Then to play it, eg in an ajax request, you can do
function doSomething() {
$.post('testReply.php',function(data){
window.sounds['clap.mp3'].play();
});
}
Not the greatest solution, but it may help, especially knowing the culprit is the load function in a non-user-triggered event.
Edit: I found Apple's explanation, and it affects iOS 4+:
http://developer.apple.com/library/safari/#documentation/AudioVideo/Conceptual/Using_HTML5_Audio_Video/Device-SpecificConsiderations/Device-SpecificConsiderations.html
Try calling the .load() method before the .play() method on the audio or video tag... which will be HTMLAudioElement or HTMLVideoElement respectively. That was the only way it would work on the iPad for me!
Autoplay doesn't work on the iPad or iPhone for either the video or the audio tags. This is a restriction set in place by Apple to save the users bandwidth.
It seems to me that the answer to this question is (at least now) clearly documented on the Safari HTML5 docs:
User Control of Downloads Over Cellular Networks
In Safari on iOS (for all devices, including iPad), where the user may be on a cellular network and be charged per data unit, preload and autoplay are disabled. No data is loaded until the user initiates it. This means the JavaScript play() and load() methods are also inactive until the user initiates playback, unless the play() or load() method is triggered by user action. In other words, a user-initiated Play button works, but an onLoad="play()" event does not.
This plays the movie: <input type="button" value="Play" onClick="document.myMovie.play()">
This does nothing on iOS: <body onLoad="document.myMovie.play()">
Have puzzled over this whilst developing a quick prototype web app under iOS4.2 (dev build). Solution is actually quite simple. Note that you can't seem to already have the tag in your HTML page, you actually need to insert the tag into the document dynamically (note this example uses Prototype 1.7 for some extra Javascript snazziness):
this.soundFile = new Element("audio", {id: "audio-1", src: "audio-1.mp3", controls: ""});
document.body.appendChild(this.soundFile);
this.soundFile.load();
this.soundFile.play();
As you've not set a controller, there shouldn't be any need to do anything tricksy with CSS to hide it / position it off-screen.
This seems to work perfectly.
Apple does not support the standard completely, but there is a workaround. Their justification is cellular network congestion, maybe because you'll land on pages that play random audio. This behavior doesn't change when a device goes on wifi, maybe for consistency. By the way, those pages are usually a bad idea anyway. You should not be trying to put a soundtrack on every web page. That's just wrong. :)
html5 audio will play if it is started as the result of a user action. Loading a page does not count. So you need to restructure your web app to be an all-in-one-page app. Instead of a link that opens a page that plays audio, you need that link to play it on the current page, without a page change. This "user interaction" rule applies to the html5 methods you can call on an audio or video element. The calls return without any effect if they are fired automatically on page load, but they work when called from event handlers.
If you create an Audio element using:
var a = new Audio("my_audio_file.wav");
And add a suspend event listener via:
a.addEventListener("suspend", function () {console.log('suspended')}, false);
And then load the file into mobile Safari (iPad or iPhone), you'll see the 'suspended' get logged in the developer console. According to the HTML5 spec, this means, "The user agent is intentionally not currently fetching media data, but does not have the entire media resource downloaded."
Calling a subsequent a.load(), testing for the "canplay" event and then using a.play() seems like a suitable method for auto triggering the sound.
My solution is trigger from a real touch event in a prior menu. They don't force you to use a specific player interface of course. For my purposes, this works well. If you don't have an existing interface to use, afaik you're buggered. Maybe you could try tilt events or something...
Works with jQuery, tested on Ipad v.5.1.1
$('video').get(0).play();
You have to append/remove the video element from the page.
I handled this by attaching an event handler for all the events for which you are allowed to trigger audio to the body element which triggers any html audio elements with autoplay to play once.
var mobile = /iPad|iPhone|iPod|android/.test(navigator.userAgent) && !window.MSStream;
if (mobile) {
$('body').on('touchstart click doubleclick keydown', function() {
$("audio[autoplay='autoplay']").each(
function(){
this.play();
this.removeAttribute('autoplay');
});
});
}
U can use howlerJS with the html5 option set to true
This seems to work:
<html>
<title>
iPad Sound Test - Auto Play
</title>
</head>
<body>
<audio id="audio" src="mp3test.mp3" controls="controls" loop="loop">
</audio>
<script type="text/javascript">
window.onload = function() {
var audioPlayer = document.getElementById("audio");
audioPlayer.load();
audioPlayer.play();
};
</script>
</body>
</html>
See it in action here: http://www.johncoles.co.uk/ipad/test/1.html (Archived)
As of iOS 4.2 this no-longer works. Sorry.