I am trying to send a payload from my service with a Basic Card and the payload I am sending looks like such with the following fulfillmentMessages:
{
"fulfillmentMessages": [
{
"simpleResponses": {
"simpleResponses": [
{
"displayText": "A 13-inch laptop with advanced color, sound and streaming for an immersive viewing experience. Featuring Dell Cinema and next-generation InfinityEdge.",
"ssml": "<speak>Here's the first item I found. XPS 13 on sale for $849.99. To navigate you can say Next, Previous, or First. You can also say 'More Details' or 'Text me the product link'.</speak>"
}
]
}
},
{
"basicCard": {
"buttons": [
{
"openUriAction": {
"uri": "https://www.dell.com/en-us/shop/productdetails/xps-13-9370-laptop/dycwi622h"
},
"title": "See on Dell.com"
}
],
"formattedText": "A 13-inch laptop with advanced color, sound and streaming for an immersive viewing experience. Featuring Dell Cinema and next-generation InfinityEdge.",
"image": {
"accessibilityText": "product image",
"imageUri": "https://i.dell.com/is/image/DellContent//content/dam/global-site-design/product_images/dell_client_products/notebooks/xps_notebooks/13_9370/global_spi/rose_gold/notebook-xps-13-9370-best-of-500-rosegold-ng.psd?fmt=png-alpha"
},
"subtitle": "$849.99",
"title": "XPS 13"
}
}
]
}
I am running into issues with Actions On Google throwing errors with my payload defined this way:
MalformedResponse
Failed to parse Dialogflow response into AppResponse because of empty speech response.
I am unsure what the issue is here, or why Actions on Google throws this exception when the Dialogflow tester shows our card as expected.
How do I correctly format my payload so that both Actions on Google and Dialogflow display as expected?
Figured it out.
Needed to assign the platform value to each message object as "ACTIONS_ON_GOOGLE".
Related
Trying to implement the action.devices.commands.mediaClosedCaptioningOn command for the TransportControl trait.. I have included 'CAPTION_CONTROL' in the transportControlSupportedCommands array I send on the SYNC intent for the device, but when I actually ask Assistant to turn on subtitles on the device it replies "That device cannot play video" and my code never actually gets an EXECUTE Intent from Assistant.
I've also successfully implemented the mediaShuffle command but noticed that when I issue it, Assistant responds with "Music will shuffle after the current item" when my device actually plays videos.
Anyone have any clue how I tell Assistant that my device plays video? I can't seem to find it in the documentation for TransportControl -- is it under a different trait maybe?
Edit 10/13/2020:
As requested, the complete SYNC response I'm currently sending for this device:
{
"requestId": "11724082196092058876",
"payload": {
"agentUserId": "xxxxxx",
"devices": [
{
"id": "10.0.6.252",
"type": "action.devices.types.STREAMING_STICK",
"traits": [
"action.devices.traits.TransportControl",
"action.devices.traits.Volume",
"action.devices.traits.MediaState"
],
"name": {
"name": "Bedroom Stick"
},
"willReportState": true,
"roomHint": "bedroom",
"attributes": {
"transportControlSupportedCommands": [
"NEXT",
"PREVIOUS",
"PAUSE",
"STOP",
"RESUME",
"CAPTION_CONTROL",
"SEEK_TO_POSITION",
"SEEK_RELATIVE",
"SET_REPEAT",
"SHUFFLE"
],
"volumeMaxLevel": 100,
"volumeCanMuteAndUnmute": true,
"levelStepSize": 10,
"commandOnlyVolume": false,
"volumeDefaultPercentage": 100,
"supportActivityState": false,
"supportPlaybackState": true
}
}
]
}
}
Exact assistant queries that produce this issue include "OK Google, turn on subtitles on Bedroom Stick" (Response: "That device can't play videos.") and "OK Google, shuffle on Bedroom Stick." (Response: "Alright, music will start shuffling after this track.")
There are certain changes you need to make to your sync response that you have posted which might make your device function properly .
In the sync response I do see that you are making use of a STREAMING_STICK as your device type. If you go through the documentation you will see that this device has the following required traits :
action.devices.traits.AppSelector
action.devices.traits.MediaState
action.devices.traits.TransportControl
action.devices.traits.Volume
Your sync response is missing the AppSelector trait.
It would be good to add the deviceInfo attribute at the end of the sync response.
More information about how to work around with Streaming stick can be found here :
https://developers.google.com/assistant/smarthome/guides/streamingstick#response
When calling the webhook multiple times in one scene and sending simple responses there is a bug at merging the simple responses.
prompt from the first webhook call
{
"override": false,
"firstSimple": {
"speech": "<speak><audio src=\"https://www.example.com/audio/file1.mp3\"></speak>",
"text": "Text 1"
}
}
prompt from the second webhook call
{
"override": false,
"firstSimple": {
"speech": "<speak><audio src=\"https://www.example.com/audio/file2.mp3\"></audio> <audio src=\"https://www.example.com/audio/file3.mp3\"></audio></speak>",
"text": " Text 2"
}
}
merged prompt in the response send to the user
{
"firstSimple": {
"speech": "<speak><speak><audio src=\"https://www.example.com/audio/file1.mp3\"></speak> <audio src=\"https://www.example.com/audio/file2.mp3\"/> <audio src=\"https://www.example.com/audio/file3.mp3\"/></speak>",
"text": "Text 1 Text2"
}
}
So with the two speak tags the SSML is invalide and is not spoken out.
Sometimes the speech object is completely missing.
I already created an Github issue for that.
So found out that the merging Bug is related to invalid SSML. Unfortunately there is no error message from Google for SSML errors.
And as a workaround for the problem that the speech object is completely missing I changed conv.add(new Simple('Text')) to conv.prompt.firstSimple = new Simple('Text') or conv.prompt.lastSimple = new Simple('Text').
The Problem
I have a Google Actions SDK project and I am using the nodejs client library for building fulfillment. I am facing some problem trying to use the Confirmation Helper intent. It is overriding previous responses given before it. Let me give you a simplified example of the problem:
First, Action says, "Hi, hope you are having a great day"
Then, Action asks for a Confirmation, "Do you want today's weather report?"
But in the simulator, all I hear is the second question. I am targeting a Voice Only situation, so I really need all the responses. Interestingly, I can see all the responses in the AUDIO tab of simulator. How can I hear both of these phrases?
I am copy/pasting the response JSON as shown in the RESPONSE tab.
{
"expectUserResponse": true,
"expectedInputs": [
{
"possibleIntents": [
{
"intent": "actions.intent.CONFIRMATION",
"inputValueData": {
"#type": "type.googleapis.com/google.actions.v2.ConfirmationValueSpec",
"dialogSpec": {
"requestConfirmationText": "Do you want today's weather report?"
}
}
}
],
"inputPrompt": {
"richInitialPrompt": {
"items": [
{
"simpleResponse": {
"textToSpeech": "Hope you are doing well!"
}
}
]
}
}
}
],
"conversationToken": ""
}
How can I fix this situation. Thanks!
This is intentional behavior as using one of the intents like Confirmation will be the only response, and other responses will be ignored.
There are two potential ways to get around this.
Combine all of your responses to be in the Confirmation
Create your own Yes/No intent and use simple responses for everything.
Problem:
In my case what happening is, it is always saying Account Linking Failed
What I am doing
following this doc word by word: https://developers.facebook.com/docs/messenger-platform/identity/account-linking/
when I need user to be linked to my app, I send a login button to the user like this:
http.post(`https://graph.facebook.com/v2.6/me/messages?access_token=${PAGE_ACCESS_TOKEN}`, {
"recipient": {
"id": "2829343507138856"
},
"message": {
"attachment": {
"type": "template",
"payload": {
"template_type": "button",
"text": "Try the log in button!",
"buttons": [
{
"type": "account_link",
"url": "https://dev-sysborg.auth0.com/authorize?response_type=code&client_id=VYierFrTxMb1Kky3DfPEqUHXaShdmH43"
}
]
}
}
}
}).then(() => {
response.send("ok")
})
user see this login button like this on their messenger screen:
when user try to login by pressing the button, it pops up a new window and open login screen in that window:
and after entering credentials it redirects to a screen which says Account linking failed,
something like this:
I am sure there is nothing wrong with the Auth0 configuration because it is working fine with postman and connected very well with some other platforms
someone said I must have to subscribe to account linking event so I have subscribed to these three events: messages, messaging_postbacks, messaging_account_linking
I am Stuck at the end of the story,
Please help me out
I have a messenger bot webview with a form with option to select, I am having an issue with message back to the thread using beginShareFlow. I have added Messenger Extensions JS SDK for the Webview. How do i implement this? Any link real world example is helpful.
Compose a JSON message to send back to the thread, and then on user share button click or whatever event you wish, run MessengerExtensions.startShareFlow, passing in functions to handle success and error events, the message you created, and the message share type. Find more at the docs
<script>
var messageToShare = {
"attachment":{
"type":"template",
"payload":{
"template_type":"generic",
"elements": [{
"title":"I took Peter's 'Which Hat Are You?' Quiz",
"image_url": "https://bot.peters-hats.com/img/hats/fez.jpg",
"subtitle": "My result: Fez",
"default_action":{
"type":"web_url",
"url": "https://bot.peters-hats.com/view_quiz_results.php?user=24601"
},
"buttons":[{
"type":"web_url",
"url":"https://bot.peters-hats.com/hatquiz.php?referer=24601",
"title":"Take the Quiz"
}]
}]
}
}
};
MessengerExtensions.beginShareFlow(function success(response) {
// Share successful
}, function error(errorCode, errorMessage) {
// The user was not able to share
},
messageToShare,
"broadcast");
</script>