We are developing interactive audiobooks for voice and have problems with some of our continuations with google assistant.
Example: In our story "Das tapfere Schneiderlein", the user hast to decide if he wants "Pflaumenmus" (plum butter) or "Apfelmus" (apple puree).
In the Test-console, everything works fine, both answers lead to the correct audio.
BUT with Google Assistant on Mobile device, only Pflaumenmus works. If I answer "Apfelmus", the action leaves conversations and opens Apple puree recipes with Google search. (see example image below, it's German, but still understandable I guess)
As we can never now, what our customers might answer, how can we prevent this from happening? (We are using Actions Builder.)
Example
This might be a result of an update regarding the Google Assistant Actions fallback intent behavior change that we announced on October 15/2020.
Follow the message from Google to make it work as you expect:
In order to provide a better experience, we now allow users to ask for some Assistant features, such as the weather or time, from within your Action. To perform this function, the Assistant detects if your Action matched a user's query with a fallback intent or NO_MATCH intent. If that is the case, and an appropriate response is available, Assistant responds to the user's request. If no response is available, or Assistant doesn't understand the query, the conversation continues within your Action.
As of October 15, 2020, this new behavior applies only if the fallback does not use a webhook. Starting January 15th 2021, we'll start enabling this feature for any Dialogflow fallback intent or Actions Builder NO_MATCH intent whether or not they use a webhook.
This change should not impact the operation of your Actions, unless you are using fallbacks as a way to collect input from your users. Going forward, you should only use fallback intents or NO_MATCH intents as a way to reprompt the user in the context of your Action. If you want your Actions to attempt to capture data from a wider range of user responses, create an intent that uses a Free form text type if you use Actions Builder. If you use Dialogflow, add an intent with a #sys.any type as the training phrase.
Related
We have an action where the user can say the name of a company, and the action will return a number of scores for said company. During testing, we have found that the action leaves the conversation and a google search for the queried company is returned instead.
This happens intermittently. More often than not, the action will return the score. Has anyone ever come across this behaviour before?
Behaviour observed on an Android phone and a Google Nest Hub.
This is an example of No-Match yielding, where the Assistant will answer a question that a user asked which your conversation is unable to match. This may because you're using a fallback intent to capture this query, which implicitly signals that your action cannot handle this prompt.
To get around it, you should move to a pattern where you have an intent that captures free-form text and ensure that intent gets matched.
I'm building an action for AoG using Dialogflow, using node.js as the fulfillment webhook.
I use suggestions and carousels in my app and want to respond to when a user clicks. My current implementation uses dialoglow's fallback, and then I check the payload manually for rawInputs.inputType="TOUCH" for suggestions and intputs.intent=actions.intent.OPTION for carousels. I'm looking for a more elegant way to do so with dialogflow and the webhook.
Does anyone know if there's a way to either:
Detect a carousel selection event in Dialogflow that can later be used as intent
A built-in method in the node.js webhook to catch this event.
Suggestion chips cannot be detected through Dialogflow directly - they are treated just like the user said or typed the chip that was selected.
However, you can detect that a carousel option has been selected. You can't determine which carousel item is selected in Dialogflow, you need to do that in your fulfillment webhook.
You can create an Intent which does not have any training phrases set, but which will trigger on an Event named actions_intent_OPTION (based on the native Intent name, but with the periods replaced by an underscore).
I'm just trying to figure out the main differences between these two types of actions. I mean a action that's use DialogFlow seems to be more conversational and more customizable. How does the Smart Home action handle the conversation? Is that a standard conversation based on the target device type?
When to create a Smart Home action and when to use DialogFlow?
To understand the difference, you need to understand the difference between a smart home action and a conversational action.
Conversational Actions
This is where the user initiates a conversation with "talk to X". Your action gets a WELCOME event. Then the user says more things and your action needs to process the user query and provide a text-based response.
Smart Home Actions
With the smart home integration, the user just gives a command directly. "Turn on my lights," for example, without precluding that with a "talk to" statement. Another big difference is that Google processes the user's query directly. Your smart home action does not get the user's text. Instead, there's a JSON request that specifies the user's intent.
The text that comes back is generated from Google as well, with parameters from your integration. Saying "turn on my lights" will result in "Ok, turning on lamp" or "Sorry, your lamp is offline" depending on what response your fulfillment sends.
There are a number of device types supported out of the box, as well as many traits. Traits specify the types of things that a device can do, such as turning on/off, or changing color. The traits are not explicitly tied to type, eg. you could change the color of a vacuum.
When to use each
If you're building or integrating a device that is meant to work with the Google Assistant, I'd suggest you look at smart home first. It will give users a better experience in being able to directly send queries, and it will make it easier for you to build fulfillment as the requests are structured already.
However, if it will not work for your application, you would want to look at Dialogflow, which will give you a greater level of flexibility in what the user can say.
We are working on project using Google Home.
Details:
We have built certain intents in Dialog Flow. It has certain follow-up questions to get the parameter values, as a multi-turn dialogue. When testing using Dialog Flow, test console, I am asking
Can you help in booking a table: It prompts back with right question (Where do you want to book a table) as configured in Dialog Flow
Where do you want to book a table : I answer - "Some Restaurant". It prompts back with right question (When do you want to book a table) as configured in Dialog Flow
When do you want to book a table: I answer: "Today" . It prompts back with right question (For how many guests) as configured in Dialog Flow.
For how many guests? - I answer: "4 people." It ends the conversation, as configured in Dialog Flow.
The above conversation works perfectly fine as expected.
When I test using the integration for Google Home (using simulator with action SDK) [See how it works in Google Assistant]
Invoke the app (by using the explicit invocation - Talk to [APP NAME]) - App gets invoked with the right greeting message as configured
After that when I ask the questions as mentioned - above - app leaves the conversation? Nothing is answered back.
Not sure why this issue is happening - anything I am missing in the configuration?
Walk through your intents and make sure the 'set this intent to end conversation' is not set to enabled in Dialogflow (and if you're using a webhook not ending there). Look down at the Responses section in DF.
Start with Welcome Default intent, and then check each intent, all follow-up prompts.
For personal gmail accounts, the Web&App activity, when turned on, it automatically gets enabled.
For gsuite accounts, even when the Web&App activity turned on, it needs to be enabled by the admin of that organization. Only after when it is enabled, the simulator will behave as expected.
I think Actions straight up doesn't work for some (all?) Gsuite accounts, regardless of what permissions you set. Google knows but doesn't care. I spent weeks in an Actions support conversation on this topic and they ultimately punted me to the Gsuite team, who couldn't help. See also:
Sorry, this action is not available in simulation
Actions on Google won't respond to explicit invocations
Given I have a published google assistant voice app with dialogflow. If I change fulfillment parameters (e. g. sent headers or URL), will this require a new review of the voice app?
If so, how can I smoothly change these parameters without breaking my app? I never really know when the review is finished, but I have to since I would need to change my deployment configuration to fit the new fulfillment properties in the right moment.
Fulfillment changes do not require a new review. Once you deploy to your server, it is effectively live and new users will use the latest webhook. Things that do require a new review are:
Actions on Google console changes such as a thumbnail or description ...etc.
Changes made to the action package (if you are using Actions SDK).
Changes made to Dialogflow that involve adding or removing account linking and/or involve things like adding intents or user expressions for implicit invocation purposes.
Once you pass the first review, it will frequently be faster for simple changes like these.