I am trying to integrate with google assistant/google home using Dialogflow and we are hitting some issues.
The issue is the following.
I have a google actions project that I have created which is linked to a corresponding dialogflow project. I have enabled the v2 APIs on both the dialogflow project and the integration of the dialogflow project and the google actions project.
The dialogflow project has intents that call a webhook for fulfillment. We have a service setup that responds to the webhook APIs. This service returns responses in V2 format including rich messages (like card, quick_replies, and carousel_select) under the key fulfillmentMessages:
However it appears that when dialogflow forwards this information to the google actions, it is not passing any of this information. On the other hand if I include a key called fulfillMentText in my service response, then dialogflow will forward that information to google actions as
[{"simpleResponse":{"textToSpeech":"Nice to meet you, Bob!"}}]}
It is not clear to me based on reading available docs, what I need to do so that dialogflow will also propagate the contents of fulfillmentMessages to google actions.
Thank you in advance.
The message objects for Actions on Google are different than the generic ones available for the other platforms Dialogflow supports. You need to also send the Actions on Google compatible messages in order to have them visible for the Assistant.
If you're using the dialogflow-fulfillment library, you can import the actions-on-google objects and add them to to the response and the library will handle them.
Don't worry about mixing the rich media types - only those that are appropriate for each platform will show.
Related
I'm creating a bot using IBM Watson Assistant. I am trying to use a webhook, but don't know the format of the POST request JSON/HTML which is sent to the webhook.
My case study is a shop where user can pre-order. I want to send the order details to my back-end server and give the user a reference number for the pre-order. I found nothing in the documentation about what POST request format is sent from IBM Watson Assistant and in what format response should be returned.
I know IBM Watson Assistant does not require a particular response format. It allows the developer to manipulate the response as the developer wants.
IBM Watson Assistant has a documented API. There are the recommended V2 Assistant API which can be used to create session and then send messages. The older V1 Assistant API has more functions and is deeper into the system. Both APIs can be used to write chatbots.
If you mean a Webhook as the Watson Assistant feature to reach out from a dialog node to an external service, the process is the following:
in the global configuration, you define the URL and the header
for a dialog node, you enable webhooks, then define the key / value pairs that are sent as payload. They can differ by dialog node.
Typically, the expected result is JSON data because it is the easiest to process.
This IBM Cloud Solution tutorial on building a Slack bot with Watson Assistant uses webhooks to call out to a Db2 database. The code is available in a GitHub repo.
Is there an API exposed for Actions on Google, similar to what Dialogflow offers with their API? The only API-like flow I have found through my research is this webhook flow API, but that only deals with conversation requests, prompts, and responses, which I have already handled.
Ideally I'd like to be able to dynamically create "agents" and their conversation flows without having to use the AoG console, similar to what Amazon offers with Alexa SMAPI.
There's not a full API to do everything that you want end-to-end. Some parts, like Dialogflow and fulfillment, can be automated, but it will still require some manual work in the Actions Console.
I had a conversation with another developer on this subject once. As a workaround, which is admittedly hacky, they decided to use the Puppeteer library to programmatically control a browser instance to fill in fields and click buttons.
That may not necessarily work when the console changes, and isn't a good substitute for an API, but it may work for you.
Yes you can do it using Google Dialogflow REST API
Here are APIs for the agent :
There are many more APIs available for different operations.
We have integrated IBM Watson Assistant skill/workspace with a Facebook page using the Watson features. We did this using an integrated approach from Virtual Assistants tab.
We are able to get the response in Facebook Messenger from Watson skill/workspace FAQS. Now we want to add a few more questions to skill/workspace and get the response from a database.
We know that we can use IBM Cloud Functions to get DB data and respond back with the data, but Cloud Functions action types (web_action and cloud_function or server) incur a cost, hence we are looking for another approach.
We have our own APIs developed for the DB and want use those in Watson Assistant dialogue node actions. Please let us know how we can add it in actions and get a response from the API without using client application/cloud functions.
Note: we haven't developed any application for this chatbot, we directly integrated Watson skill/workspace with the Facebook page and trying to call API calls wherever we require them from the dialogue nodes.
As you can see, IBM Watson Assistant allows to invoke three different types of actions from a dialog node.
client,
server (cloud_function),
web_action.
Because for cloud_function and web_action the action is hosted as Cloud Function on IBM Cloud, the computing resources are charged. For type client, your app would handle the API call and the charges depend on where your app is hosted. Thus, there are always costs.
What you could do is to write a wrapper function that is deployed as web_action or cloud_function. Thus, there isn't much of computing resource needed and the charges would be minimal. But again, independent of the action type, there are always costs (maybe not charges) - one way or another...
I am trying to forward Google Smart Home events to my Dialogflow fulfillment service. I am creating 3 intents with no input or output contexts set, no training phases and with the following events:
action_devices_SYNC
action_devices_EXECUTE
action_devices_QUERY
See also https://imgur.com/a/4eN9S.
Is that correct? I can't find confirmation in the docs, so that's why I am asking it here.
reasoning
The reason why I asked about connecting Google Smart Home with my Dialogflow endpoint is that I already have that endpoint in place. I hoped I could do something similar as in https://stackoverflow.com/a/49119822/9038652, where I bound a Dialogflow intent to the actions_intent_OPTION event.
There isn't a reason to use Dialogflow to do smart home fulfillment, and it's actually not possible.
Dialogflow is great for taking unstructured user utterances and making sense of them. However, with smart home, Google handles all of the NLU and parsing. You, as the integration, will just receive a JSON request and will be expected to provide a JSON response.
So you will skip using Dialogflow and instead just build your webhook to parse the intents and give a valid response.
Dialogflow's service does not have a way to take in an intent name and expose a single endpoint URL that can be called by the Google Assistant. It also does not have integration with an OAuth server to do the account linking step.
I started writing a dialogflow fulfillment extending https://dialogflow.com/docs/getting-started/basic-fulfillment-conversation. They don't use DialogflowApp and just call response.json().
However, I also found https://developers.google.com/actions/reference/nodejs/DialogflowApp where fulfillment is done with the DialogflowApp.
When do you use each? and why does the Dialogflow Key Concepts and Basic Fulfillment document not mention to DialogflowApp at all?
The first route (sending response directly) doesn't seem to have a way to deal with permissions and push/daily notifications, while DialogflowApp does.
Will the documentation for sending responses directly be extended to handle permissions? What is the best practice?
Thanks.
If you're using Actions on Google, go with the Actions on Google client library and DialogflowApp. This only supports Actions on Google.
If you are using other integrations that work with Dialogflow, such as Slack, then you should use the fulfillment library and documentation provided by Dialogflow.