Google Assistant flow with multiple actions_intent_OPTION handlers - actions-on-google

Question

I have webhooks configured through Dialogflow for a template chatbot UI starter project I'm making on Github. I have a bot integrated through Facebook Messenger and Google Assistant. All of Facebook works fine because the actions send back strings and it's easy to handle. But when Google Assistant tries to handle items of "#type": "type.googleapis.com/google.actions.v2.OptionValueSpec", actions_intent_OPTION is needed on the event in Dialogflow to handle the response. If I have just one in my app, it works fine, but when I add a second list item / carousel item of type OptionValueSpec, the flow chokes. I have details on the attached image. My guess is actions_intent_OPTION is needed to handle the list, but when I put that on multiple intents on the events section, the flow doesn't know how to handle it.
Comparison of Facebook Messenger (working) to Google Assistant (with bug)
Detailed full flow view of Google Assistant
Responses sent to Dialogflow that subsequently get sent to Google Actions...
Exact responses related to the UI pics above.
// working as expected
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "Hey there! This is a guided tour of common components between Facebook Messenger and Google Assistant."
}
},
{
"simpleResponse": {
"textToSpeech": "You can start coding the sample project at github.com/ianrichard."
}
}
],
"suggestions": [{
"title": "Show me demos!"
},
{
"title": "Show code & docs"
}
]
}
}
// working as expected
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "Animated GIFs are always fun to add to the mix!"
}
},
{
"basicCard": {
"image": {
"url": "https://somewebsite.com/colbert.gif",
"accessibilityText": "Stephen Colbert at the beginning of the show being happy."
}
}
}
],
"suggestions": [{
"title": "What about a card?"
}]
}
}
// working as expected
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "Absolutely!"
}
},
{
"simpleResponse": {
"textToSpeech": "Named for a winding stretch of Hill Country highway, Devil’s Backbone is a Belgian-style tripel. Featuring a beautiful pale-golden color, this ale’s spicy hops and Belgian yeast work together to create a distinctive flavor and aroma. Don’t let the light color fool you, this one has a dark side too. Traditional Belgian brewing techniques add strength without increasing heaviness."
}
},
{
"basicCard": {
"image": {
"url": "https://somewebsite.com/devils-backbone.jpg",
"accessibilityText": "Devil’s Backbone"
},
"title": "Devil’s Backbone",
"subtitle": "Belgian-Style Tripel",
"buttons": [{
"title": "Read More",
"openUrlAction": {
"url": "https://realalebrewing.com/beers/devils-backbone/"
}
}]
}
}
],
"suggestions": [{
"title": "How about a list?"
}]
}
}
// working as expected
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "Absolutely!"
}
},
{
"simpleResponse": {
"textToSpeech": "Who’s your favorite GOT character!?"
}
}
]
},
"systemIntent": {
"intent": "actions.intent.OPTION",
"data": {
"#type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
"listSelect": {
"items": [{
"optionInfo": {
"key": "tyrion"
},
"title": "Tyrion Lannister",
"description": "Peter Dinklage",
"image": {
"url": "https://somewebsite.com/got-tyrion.jpg",
"accessibilityText": "Tyrion Lannister"
}
},
{
"optionInfo": {
"key": "daene"
},
"title": "Daenerys Targaryen",
"description": "Emilia Clarke",
"image": {
"url": "https://somewebsite.com/got-daenerys.jpg",
"accessibilityText": "Daenerys Targaryen"
}
},
{
"optionInfo": {
"key": "jon"
},
"title": "Jon Snow",
"description": "Kit Harington",
"image": {
"url": "https://somewebsite.com/got-jon.jpg",
"accessibilityText": "Jon Snow"
}
}
]
}
}
}
}
// if two events with the same actions_intent_OPTION are defined, it goes straight to the end and the list option handler is never invoked
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "The end"
}
},
{
"simpleResponse": {
"textToSpeech": "Well, that’s the end of the demo. Hope you enjoyed!"
}
}
],
"suggestions": [{
"title": "Start over"
}]
}
}
// otherwise, it will show the last carousel
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "I drink and I know things!"
}
},
{
"simpleResponse": {
"textToSpeech": "What are you going to buy your wife from Tiffany?"
}
}
]
},
"systemIntent": {
"intent": "actions.intent.OPTION",
"data": {
"#type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
"carouselSelect": {
"items": [{
"optionInfo": {
"key": "sunglasses"
},
"title": "Aviator Sunglasses",
"description": "$360",
"image": {
"url": "https://somewebsite.com/tiffany-glasses.jpg",
"accessibilityText": "Aviator Sunglasses"
}
},
{
"optionInfo": {
"key": "ring"
},
"title": "Infinity Ring",
"description": "$200",
"image": {
"url": "https://somewebsite.com/tiffany-ring.jpg",
"accessibilityText": "Infinity Ring"
}
},
{
"optionInfo": {
"key": "earrings"
},
"title": "Soleste Earrings",
"description": "$5,600",
"image": {
"url": "https://somewebsite.com/tiffany-earrings.jpg",
"accessibilityText": "Soleste Earrings"
}
},
{
"optionInfo": {
"key": "pendant"
},
"title": "Infinity Pendant",
"description": "$250",
"image": {
"url": "https://somewebsite.com/tiffany-necklace.jpg",
"accessibilityText": "Infinity Pendant"
}
},
{
"optionInfo": {
"key": "watch"
},
"title": "East West Mini",
"description": "$7,500",
"image": {
"url": "https://somewebsite.com/tiffany-watch.jpg",
"accessibilityText": "East West Mini"
}
}
]
}
}
}
}
// but the carousel option handler isn't processed correctly :( - keeps repeating this same thing.
{
"richResponse": {
"items": [{
"simpleResponse": {
"textToSpeech": "What!? None of them?"
}
},
{
"simpleResponse": {
"textToSpeech": "What are you going to buy your wife from Tiffany?"
}
}
]
},
"systemIntent": {
"intent": "actions.intent.OPTION",
"data": {
"#type": "type.googleapis.com/google.actions.v2.OptionValueSpec",
"carouselSelect": {
"items": [{
"optionInfo": {
"key": "sunglasses"
},
"title": "Aviator Sunglasses",
"description": "$360",
"image": {
"url": "https://somewebsite.com/tiffany-glasses.jpg",
"accessibilityText": "Aviator Sunglasses"
}
},
{
"optionInfo": {
"key": "ring"
},
"title": "Infinity Ring",
"description": "$200",
"image": {
"url": "https://somewebsite.com/tiffany-ring.jpg",
"accessibilityText": "Infinity Ring"
}
},
{
"optionInfo": {
"key": "earrings"
},
"title": "Soleste Earrings",
"description": "$5,600",
"image": {
"url": "https://somewebsite.com/tiffany-earrings.jpg",
"accessibilityText": "Soleste Earrings"
}
},
{
"optionInfo": {
"key": "pendant"
},
"title": "Infinity Pendant",
"description": "$250",
"image": {
"url": "https://somewebsite.com/tiffany-necklace.jpg",
"accessibilityText": "Infinity Pendant"
}
},
{
"optionInfo": {
"key": "watch"
},
"title": "East West Mini",
"description": "$7,500",
"image": {
"url": "https://somewebsite.com/tiffany-watch.jpg",
"accessibilityText": "East West Mini"
}
}
]
}
}
}
}
Intent References

Answer 1

FINALLY works! Thanks #Prisoner!
In Dialogflow...
First list, define the output context
Second list, define the input context and output context
Question after the second list, define the input context
In your webhook...
(The missing piece of the puzzle that made it work)
Set the output context for the list options
{
"speech": "",
"displayText": "",
"data": { "google": { ... } },
"contextOut": [
{
"name": "carouselExample",
"lifespan": 0,
"parameters": null
}
]
}

Answer 2

Broadly speaking, the problem is that Dialogflow doesn't know where in the conversation you are when it gets the action_intent_OPTION event. For that event, it doesn't try to do Entity matching, but the issue of conversational context is a problem in general (what happens, for example, if you have two different option carousels which have overlapping answers?).
The solution is twofold:
When you send back the response that includes the option information, you should also set an outgoing Context. You can include other information in this Context, but in your case it sounds mostly like you just want to keep track of where you are in the conversation.
You can then differentiate the two Intents with the option event by specifying which context each should be triggered for. Dialogflow will match both the event and the context to determine the best Intent to use.