Google assistant media player goes away on pause - actions-on-google

BRIEF :
I have created Google assistant application that plays music using Google Action Builder. On specific command, it triggers a webhook. Webhook contains MediaResponse
OR Media from '#assistant/conversation' Library and the code is following
conv.add(new Media({
mediaType: 'AUDIO',
start_offset: `3.000000001s`,
mediaObjects: [{
name: music,
description: 'This is example of code ',
url: `https://example.com`,
image: {
large: {
url: 'https://example.com'
},
}
}]
}));
It is running well on android and the emulator .
ISSUE :
When I pause the music (USING PAUSE BUTTON), the Media player goes away.
What should I do to keep the media player so that I can resume the music?
Any information regarding this would be appreciated & Thanks in advance.
EDITED: It works well for showing media player and plays music but if you click pause button it goes away for both above devices(Android/Test Emulator).

Just adding acknowledgment to it fixed the issue.
app.handle('media_status', (conv) => {
const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
switch (mediaStatus) {
case 'FINISHED':
conv.add('Media has finished playing.');
break;
case 'FAILED':
conv.add('Media has failed.');
break;
case 'PAUSED' || 'STOPPED':
if (conv.request.context) {
// Persist the media progress value
const progress = conv.request.context.media.progress;
}
// Acknowledge pause/stop
conv.add(new Media({
mediaType: 'MEDIA_STATUS_ACK'
}));
break;
default:
conv.add('Unknown media status received.');
}
});

Related

In my Google Action, how do I efficiently conclude play of one MP3 and "skip" to another MP3 using a single user utterance?

I have a one-scene Action that calls my webhook 'randomSpeech' (mentioned below) upon invocation, which plays an MP3. I added a "skip" intent to skip to the next MP3. When I say "skip", the Action should transition (loop) back into the webhook 'randomSpeech', and since there is a counter, x, the Action should begin playing the 2nd MP3 in the switch statement.
However, I have to say the word "skip" twice in order for it to work.
The 1st time I say "skip", the system intent, MEDIA_STATUS_FINISHED automatically calls the 'mediaStatus' handler and the text 'Media has finished.' is added to the conversation. Even though I've configured the "skip" intent to call the handler, 'randomSpeech', it doesn't appear to happen as no new Media is added to the conversation. It's almost like 'randomSpeech', is completely ignored!
The 2nd time I say "skip", the second MP3 finally begins playing.
My main question is, how can I make it so the user only has to say "skip" one time?
let x = 1;
app.handle('randomSpeech', (conv) => {
switch(x) {
case(1):
conv.add(new Media({
mediaObjects: [
{
name: 'NEVER GIVE UP',
description: 'some athlete',
url: 'http://zetapad.com/speeches/nevergiveup.mp3',
image: {
large: {
url: 'https://www.keepinspiring.me/wp-content/uploads/2020/02/motivation-gets-you-started-jim-ryun-quote-min.jpg'
}
}
}
],
mediaType: 'AUDIO',
optionalMediaControls: ['PAUSED', 'STOPPED'],
startOffset: '5s'
}));
x++;
break;
case(2):
conv.add(new Media({
mediaObjects: [
{
name: 'SPEECHLESS',
description: 'Denzel Washington (feat Will Smith)',
url: 'http://zetapad.com/speeches/denzel.mp3',
image: {
large: {
url: 'https://www.keepinspiring.me/wp-content/uploads/2020/02/motivational-quotes-2-min.jpg'
}
}
}
],
mediaType: 'AUDIO',
optionalMediaControls: ['PAUSED', 'STOPPED']
}));
break;
}
});
app.handle('media_status', (conv) => {
const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
switch(mediaStatus) {
case 'FINISHED':
conv.add('Media has finished.');
break;
case 'FAILED':
conv.add('Media has failed.');
break;
case 'PAUSED' || 'STOPPED':
if (conv.request.context) {
// Persist the media progress value
const progress = conv.request.context.media.progress;
}
conv.add(new Media({
mediaType: 'MEDIA_STATUS_ACK'
}));
break;
default:
conv.add('Unknown media status received.');
}
});
Images from the only scene, "Motivation":
Scene
On enter
Intent handling
Further notes:
MEDIA_STATUS_PAUSED / MEDIA_STATUS_FINISHED / MEDIA_STATUS_STOPPED all only call the 'media_status' wehbook
The issue at the heart of your question is that "skip" is a built-in Media Player command (although this is not clearly documented), so when the user says "skip", the player treats this as the audio being completed, so it sends the MEDIA_STATUS_FINISHED Intent, just as it the user listened to it all the way through.
The good news is - you actually want to handle both these cases the same way! So if the user skips to the next audio, or finishes the first and it should advance to the next audio - you want to play the next audio.
In your code, "playing the next audio" is all done as part of your switch statement. So you should probably put that into a regular JavaScript function by itself. You can then call that function from the different handlers that you have setup.
It might look something like this (without some of the code details):
function nextAudio( conv ){
// Code goes here to figure out the next audio to play and send it back
}
app.handle('randomSpeech', (conv) => {
nextAudio( conv );
}
app.handle('media_status', (conv) => {
const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
switch(mediaStatus) {
case 'FINISHED':
nextAudio( conv );
break;
// Other media status cases can go here
}
});

How to close conversation from the webhook in #assistant/conversation

I want to close the conversation after the media started playing in #assistant/conversation. As I am doing here
app.intent("media", conv => {
conv.ask(`Playing your Radio`);
conv.ask(
new MediaObject({
url: ""
})
);
return conv.close(new Suggestions(`exit`));
});
As Jordi had mentioned, suggestion chips cannot be used to close a conversation. Additionally, the syntax of the #assistant/conversation is different from actions-on-google. As you're using the tag dialogflow-es-fulfillment but also actions-builder, I really don't know which answer you want. As such, I'm going to put two answers depending on which you're using.
Dialogflow
If you are using Dialogflow, you are pretty much set. You should switch to using actions-on-google and instantiate the dialogflow constant.
const {dialogflow} = require('actions-on-google')
const app = dialogflow()
Actions Builder
The syntax of the #assistant/conversation lib is different. Some method names are different. Additionally, you will need to go through Actions Builder to canonically close the conversation.
In your scene, you will need to transition the scene to End Conversation to close, rather than specifying it as part of your response. Still, your end transition should not have suggestion chips.
You will need to refactor your webhook:
const {conversation} = require('#assistant/conversation')
const app = conversation()
app.handle("media", conv => {
conv.add(`Playing your Radio`);
conv.add(
new MediaObject({
url: ""
})
);
conv.add(new Suggestions(`exit`));
});
As it seems you are trying to have a media control and after that to end the conversation, you should refer to the doc (https://developers.google.com/assistant/conversational/prompts-media) to check the available events as you have the chance to control each one for the media playback.
For example
// Media status
app.handle('media_status', (conv) => {
const mediaStatus = conv.intent.params.MEDIA_STATUS.resolved;
switch(mediaStatus) {
case 'FINISHED':
conv.add('Media has finished playing.');
break;
case 'FAILED':
conv.add('Media has failed.');
break;
case 'PAUSED' || 'STOPPED':
if (conv.request.context) {
// Persist the media progress value
const progress = conv.request.context.media.progress;
}
// Acknowledge pause/stop
conv.add(new Media({
mediaType: 'MEDIA_STATUS_ACK'
}));
break;
default:
conv.add('Unknown media status received.');
}
});
Once you get the FINISHED status you can offer the suggestion chip to exit the conversation.

Youtube iframe on ionic app generate play icon background

I'm developing an app, using Ionic 3, thats reproduce a youtube video. As I want an embedded video I use an iframe where the src is the video's url.
When I test on an Android device, i get this before the video starts playing.
Is there a way to avoid that background? or make it personalized?
Testing it using "ionic serve" makes the background completely black, so it only happens running on an android device.
Why not use a temporary <img>, as soon as the user clicks on the img, the <iframe> tag is toggled on, with Autoplay = true.
I recommend using angular youtube-player. You can detect when the video is ready to play. If the video is not ready yet, just display an image or a spinner.
Here is an example:
HTML:
<img class="video-loading-cover" src="assets/images/home/ytcover.png" height="550" width="1400" [hidden]="isVideoLoaded" alt="">
<youtube-player #youTubePlayer (ready)="playVideo($event)" (stateChange)="onPlayerStateChange($event)" (error)="hideVideo()"
width="100%" height="540px" [playerVars]="playerVars" [videoId]="videoId"></youtube-player>
TS:
#ViewChild('youTubePlayer') youTubePlayer: YT.Player;
isVideoLoaded: boolean;
videoId = 'your video id';
// optional
playerVars: YT.PlayerVars = {
autoplay: AutoPlay.AutoPlay,
loop: Loop.Loop,
playlist: 'yourPlaylist',
controls: Controls.Hide,
enablejsapi: JsApi.Enable,
origin: window.location.origin,
rel: RelatedVideos.Hide,
iv_load_policy: IvLoadPolicy.Hide,
autohide: AutoHide.HideAllControls,
showinfo: ShowInfo.Hide
};
ngOnInit() {
const tag = document.createElement('script');
tag.src = 'https://www.youtube.com/iframe_api';
document.body.appendChild(tag);
}
onPlayerStateChange(el) {
this.youTubePlayer.mute();
switch (el.data) {
case -1:
case 2:
this.youTubePlayer.playVideo();
break;
case 1:
this.isVideoLoaded = true;
break;
}
}
hideVideo(): void {
this.isVideoLoaded = false;
}
playVideo(event): void {
this.youTubePlayer.mute();
setTimeout(() => {
this.youTubePlayer.playVideo();
}, 10);
}
Note that in my example when the youtube video is loading it will call the (error)="hideVideo()" function and an image will be displayed. When it is available it will automatically hide the image and play the video.

How to link out to a YouTube Playlist in the official app from my own mobile app

I am working on an ionic application. I've successfully been able to use my app to open the official youtube application to play a single video using the following:
youtube://kVHB26MAh2E
However, I would like to open a playlist page in the official app, so I tried this:
youtube://playlist?list={my playlist ID}
but that did not work. The syntax is a little different for these "deeplinks" so what exactly am I missing?
Thank you very much!
If it helps, this is how I manage fallback urls. I trigger the initial call with openYouTube():
launchExternalApp(iosSchemaName: string, androidPackageName: string, appUrl: string, httpUrl: string, id: string) {
let app: string;
if (this.platform.is('android')) {
app = androidPackageName;
} else if (this.platform.is('ios')) {
app = iosSchemaName;
} else {
this.iab.create(httpUrl + id);
return;
}
this.appAvailability.check(app).then(
() => { // success callback
this.iab.create(appUrl + id);
},
() => { // error callback
this.iab.create(httpUrl + id);
}
);
}
openYouTube() {
this.launchExternalApp('youtube://', 'com.youtube.android', 'youtube://playlist?list=', 'https://youtube.com/playlist?list=', 'PLTzXf6BfROEktE823y4AuhQrvJP2ZuX6q');
}
I figured out how to do it:
youtube://youtube.com/playlist?list={id}
This at least works for iOS, I have not tested android yet.

reading videos using cordova-plugin-media-streaming close the window automatically

I'm working on an ionic mobile application where I needed to read videos on streaming by providing the URI of the video online. So I used cordova-plugin-media-streamingplugin offered by cordova.
My problem is that: the window reading the video closes automatically after the video finishes, the user won't be able to play the video again in this window.
In the official documentation of the plugin [that i found here], there is an attribute called shouldAutoClosethat should be set to false to avoid that problem. But this didn't work for me.
Here is the code I used to play a video on streaming :
startVideo(item : Multimediasendtrust) {
let options = {
successCallback: () => { console.log('Finished Video') },
errorCallback: (e) => { console.log('Error: ', e) },
orientation: 'portrait',
controls: true,
shouldAutoClose: false
};
console.log('those are option ',options );
console.log('the link of the video ', item.url_media);
this.streamingMedia.playVideo(item.url_media, options); }
Can anyone help please. Thanks in advance.