Is there a way to capture & handle "next" and "previous” using Google Actions with Actions SDK as the fulfillment tool When Media Response was sent? - actions-on-google

I'm using the Media Response to play audio in my Actions SDK (Action SDK is being used as the Fulfillment tool in my Actions Console) driven Action. At the end of each audio clip, I'm using the MEDIA_STATUS callback to advance to another mp3 file in a predefined playlist. As a result, users should be able to navigate forwards / backwards.
When testing on my Google Home mini, Google Assistant on Android and Smart Display, I can intercept "next" and advance to the next audio clip (it sends a request with intent of type MEDIA_STATUS). However, I can't properly intercept "previous" Whenever I try, the audio will restart. Real devices seems to be handling this intent on it's own and does not throw any console output (as my webhook is not accessed at all).
Dialog Flow seem to handle "next" and "previous" as follow up intents, but I need to do the same without using Dialog Flow as the Fulfillment tool.
Can anyone please help with this particular problem?

Currently only a FINISHED event is supported for the Media Response. So, you would not be able to handle distinguish between next and previous.

Referring to #Leon Nicholls answer, I am really confused. I followed his Google Audioplayer tutorial/guide which links to an audioplayer template action
You can still download it, and if you unzip it, you will see code that is put there specifically to handle previous and next tracks.
Now it appears Leon is saying that it can't handle previous and next by voice (but buttons appear to work).
Here is the sample code below. There are links to tutorial actions in it, but now all the content has been removed. I simply need blind people to be able to select and play various bit of audio, too long for SSML, which must be navigable (at least next and pause/resume later) may be part of a playlist. How do I achieve this? Suggesting "Media Partner" or "Podcast" is not a solution as neither available in the UK.
// Handle the More/Yes/Next intents
app.intent(['More', 'Yes', 'Next'], (conv) => {
console.log(`More: fallbackCount=${conv.data.fallbackCount}`);
nextTrack(conv, false);
});
// Handle the Repeat/Previous intent
app.intent(['Repeat', 'Previous'], (conv) => {
console.log(`Repeat: ${conv.user.storage.track}`);
nextTrack(conv, false, true);
});
// Select the next track to play from the data
const nextTrack = (conv, intro, backwards) => {
console.log(`nextTrack: ${conv.user.storage.track}`);
let track = data[0];
// Persist the selected track in user storage
// https://developers.google.com/actions/assistant/save-data#save_data_across_conversations
if (conv.user.storage.track) {
conv.user.storage.track = parseInt(conv.user.storage.track, 10);
if (backwards) {

Related

How to prevent google action from closing the conversation?

I'm developing a Google Action through DialogFlow and a webhook (that will run on a Nest Hub) that I want to act like this:
the user invokes the action "Hey Google, talk to ACTIONAME"
through the Default Welcome Intent ("hooked" to my web service) the Action replies to the user and open a website
app.intent('Default Welcome Intent', conv => {
conv.ask('Hi! I'm opening your site')
conv.ask(new HtmlResponse({
url: 'https://MY_IOT_SITE'
}))
})
now, the user could be "silent" for mins or hours, but I'd like to prevent Google Actions to close the ACTIONNAME and return to the clock, while until now the action closes after a couple of minutes
Is it somehow possible?
Thank you.
This is not possible. The platform intentionally places an upper bound on how long an action can run without any user input. This is done so that an action cannot occupy the device longer than expected and prevent future inputs from unintentionally getting routed to your action rather than the Google Assistant.
You can take a look at additional guidelines when developing your web app.
Since your question refers to an IoT-related website, you may want to take a look at the Smart Home reference, which provides an alternative way to let users control smart home devices with their voice or built-in graphical widgets.

is there a way to capture & handle "Previous" using Google Actions & Media Player

I'm using the MediaObject responses to play audio in my Dialogflow driven Action. At the end of each audio clip, I'm using the mediaStatus callback to advance to another episode in a predefined playlist. As a result, users should be able to navigate forwards / backwards.
On Alexa, Next/Previous are built in intents, but my understanding is that with Google Actions, you have to handle these manually.
When testing on my Google Home, I can intercept "Hey Google, Next" and advance to the next audio clip. However, I can't properly intercept "Hey Google, Previous." Whenever I try, the audio will restart / break the conversation. Google Home seems to be handling this intent on it's own & does not throw any console output (so it's happening on device perhaps?)
Does anyone have a reliable solution for this?
If your action is still running, then the user just has to say "next" or "previous". Saying, "OK, Google..." will take it out of the context of your action and then it is up to the assistant to decide if it wants to handle the user request. Dialogflow provides follow-up intents for "next" and "previous" that you can add to your agent.
I'm working with MediaResponse as you too,
Actually I can implemented features : Next|Previous, manually.
The logic is so simple, so you don't need to get help from Google.
The logic should be :
1 - Had song list
2 - Define the selected song index
3 - Play the first song with selected song index = 0
4 - Next|Previous feature just be increase|decrease selected song index (selectedSongIndex++|selectedSongIndex--)
p/s : You can define phrase like "please next" "next" "next song" ... with your Custom Intents, to catch it

How do I play an audio file when using Fulfillment with Dialogflow

I'm making a Google Assistant action, similar to what Google does when you say "Play an E note".
I've managed to get my nodejs app to reply back the parameter, but now I need to pass an audio file. How do I do that?
The typical way to do this is to place the audio file on a hosting service somewhere (Firebase Hosting is a good choice, particularly if you're also using Firebase Cloud Functions for your Action, but any place that can serve a file via HTTPS works) and then send back SSML as your response that includes the audio.
This might look something like this:
var audioUrl = 'https://example.com/audiofile.mp3';
var msg = `<speak><audio src="${audioUrl}"></audio></speak>`
app.tell( msg );
Adjust this for your own audio file, and you might want to use app.ask() instead of tell if you are prompting the user to reply to your audio.

How to use standard events in the react native Facebook SDK

The Facebook SDK has the ability to record both custom events and standard events. The standard events are things like "Purchases" "Add to cart" "Completed Registration" etc...
Recording these standard events gives you access to specific bidding features on Facebook ads that you don't get without the events.
I have an app that has the React Native FBSDK
There are two methods for defining an event - one for purchases and one for everything else as seen here
There is zero documentation for standard events on react within the SDK on Github or on the event tracking docs on Facebook's developer platform.
Right now I'm trying to track the standard events by using their various names, as recorded across FB's documentation. I've tried the following:
AppEventsLogger.logEvent('FBSDKAppEventNameCompletedRegistration');
AppEventsLogger.logEvent('CompletedRegistration');
AppEventsLogger.logEvent('Completed Registration');
All of these just create custom events with those names, but aren't recognized as standard events.
Has anyone gotten standard events to work using the React Native wrapper for the FB SDK? If so how do you name the events to get FB to recognize them?
Update: As the comment below highlights, the more recent link is https://developers.facebook.com/docs/marketing-api/app-event-api/
It looks like you'll have to pass the strings that those standard events get evaluated to, i.e. instead of 'FBSDKAppEventNameCompletedRegistration', you'll have to use: 'fb_mobile_complete_registration'.
Here's the source:
Sorry if this is a bit late. Hope this helps.
I managed to find the actual event name by generating standard event code using tool on Facebook's documentation, run the code in AppDelegate.m, and get the exact key-values from Events Manager. With this roundabout way, I realized the actual name of Add to Cart event was fb_mobile_add_to_cart. From there I googled for the name and found the list documented in Marketing API (why not App Events?).
I don't know if it is the right place, but you can refer to https://developers.facebook.com/docs/marketing-api/app-event-api/ for actual standard event names and parameter names. At least it worked in my case. Here's my Add to Cart event code:
function logAddToCart(totalPrice, contentType, contentId, currency) {
const params = {
'fb_content_type': contentType,
'fb_content_id': contentId,
'fb_currency': currency
};
AppEventsLogger.logEvent('fb_mobile_add_to_cart', totalPrice, params);
}
I made a simple package with all events. Just import like
import FBEvents from "react-native-fbsdk-events";
// ...
AppEventsLogger(FBEvents.COMPLETE_REGISTRATITON, params);

What is the name of the App Launch event for Facebook Analytics?

I'm trying to implement the new analytics for a Facebook game (using HTML/Javascript and Flash on Canvas, so there is no mobile version), but it seems that the documentation is incomplete. It says that there are 14 predefined events:
"Events are one of 14 predefined events such as 'added to cart' in a
commerce app or 'level achieved' in a game"
Source: https://developers.facebook.com/docs/reference/javascript/FB.AppEvents.LogEvent
"The fourteen pre-defined events are: App Launch, Complete
Registration, Content View, Search, Rating, Tutorial Completed, Add to
Cart, Add to Wishlist, Initiated Checkout, Add Payment Info, Purchase,
Level Achieved, Achievement Unlocked, Spent Credits."
Source: https://developers.facebook.com/docs/app-events/faq
However, on the reference page where all the events should be listed, the list is only 12 items long, and there is no "App launch" event:
https://developers.facebook.com/docs/reference/javascript/FB.AppEvents.LogEvent#events
Now, there are some sample event lists for some games, but they are very basic and they don't include the actual code: https://developers.facebook.com/docs/app-events/best-practices#casual
which recommends to use these events:
App Install
App Launch
Completed Registration
Completed Tutorial
Level Achieved
Achievement Unlocked
(...)
Here is what I have so far:
FB.AppEvents.activateApp()
But is this event the equivalent of App Install or App Launch?
Also, should I send this before the user accepts to share his basic info or after? I'm having so many questions because it's not clear what activateApp() does...
Here is some code for sending some other events that could be useful:
FB.AppEvents.logEvent(FB.AppEvents.EventNames.COMPLETED_REGISTRATION);
FB.AppEvents.logEvent(FB.AppEvents.EventNames.COMPLETED_TUTORIAL);
var params = {};
params[FB.AppEvents.ParameterNames.LEVEL] = '12'; //player level
FB.AppEvents.logEvent(
FB.AppEvents.EventNames.ACHIEVED_LEVEL,
null, // numeric value for this event - in this case, none
params
);
I still have more questions: how can I properly send the game version number (maybe with activateApp?) so I can create segments and cohorts later? Some example codes would be really appreciated!
Thanks in advance!
FB.AppEvents.activateApp() provides install and launch event functionality, which is why those two events are not enumerated as options in https://developers.facebook.com/docs/reference/javascript/FB.AppEvents.LogEvent#events. Activate app doesn't take a parameter. You might want to look at using a custom event to satisfy your use case.