We have a chatbot for our website today, that is not build using Google technology. The bot has a JSON REST API where you can send the question to and which replies with the corresponding answers. So all the intents and entities are being resolved by the existing chatbot.
What is the best way to wrap this functionality in Google Assistant / for Google Home?
To me it seems I need to extract the "original" question from the JSON that is send to our webservice (when I enable fullfilment).
But since context is used to exchange "state" I have to find a way to exchange the context between the dialogflow and our own chatbot (see above).
But maybe there are other ways ? Can it (invoke our chatbot) be done directly (without DialogFlow as man in the middle) ?
This is one of the those responses that may not be enough for someone who doesn't know what I am talking about and too much for someone who does. Here goes:
It sounds to me as if you need to build an Action with the Actions SDK rather than with Dialog flow. Then you implement a text "intent" in your Action - i.e. one that runs every time the user speaks something. In that text intent you ask the AoG platform for the text - see getRawInput(). Now you do two things. One, you take that raw input and pass it to your bot. Two, you return a promise to tell AoG that you are working on a reply but you don't have it yet. Once the promise is fulfilled - i.e. when your bot replies - you reply with the text you got from your bot.
I have a sample Action called the French Parrot here https://github.com/unclewill/french_parrot. As far as speech goes it simply speaks back whatever it hears as a parrot would. It also goes to a translation service to translate the text and return the (loose) French equivalent.
Your mission, should you choose to accept it, is to take the sample, rip out the code that goes to the translation service and insert the code that goes to your bot. :-)
Two things I should mention. One, it is not "idiomatic" Node or JavaScript you'll find in my sample. What can I say - I think the rest of the world is confused. Really. Two, I have a minimal sample of about 50 lines that eschews the translation here https://github.com/unclewill/parrot. Another option is to use that as a base and add code to call your bot and the Promise-y code to wait on it to it.
If you go the latter route remove the trigger phrases from the action package (action.json).
So you already have a Backend that process user inputs and sends responses back and you want to use it to process a new input flow (coming from Google Assistant)?
That actually my case, I've a service as a Facebook Messenger ChatBot and recently started developing a Google Home Action for it.
It's quite simple. You just need to:
Create an action here https://console.actions.google.com
Download GActions-Cli from here https://developers.google.com/actions/tools/gactions-cli
Create a JSON file action.[fr/en/de/it].json (choose a language). The file is your mean to define your intents and the URL to your webhook (a middleware between your backend and google assistant). It may look like this:
{
"locale": "en",
"actions": [
{
"name": "MAIN",
"description": "Default Welcome Intent",
"fulfillment": {
"conversationName": "app name"
},
"intent": {
"name": "actions.intent.MAIN",
"trigger": {
"queryPatterns": [
"Talk to app name"
]
}
}
}
],
"conversations": {
"app name": {
"name": "app name",
"url": "https://your_nodejs_middleware.com/"
}
}
}
Upload the JSON file using gactions update --action_package action.en.json --project PROJECT_ID
AFAIK, there only a Node.js client library for Actions-on-google https://github.com/actions-on-google/actions-on-google-nodejs that why you need a Node.js middleware before hitting your backend
Now, user inputs will be sent to your Node.js middleware (app.js) hosted at https://your_nodejs_middleware.com/ which may look like:
//require express and all required staff to build a Node.js server,
//look on internet how to build a simple web server in Node.js
//if you a new to this domain. const {
ActionsSdkApp } = require('actions-on-google');
app.post('/', (req, res) => {
req.body = JSON.parse(req.body);
const app = new ActionsSdkApp({
request: req,
response: res
});
// Create functions to handle requests here
function mainIntent(app) {
let inputPrompt = app.buildInputPrompt(false,
'Hey! Welcome to app name!');
app.ask(inputPrompt);
}
function respond(app) {
let userInput = app.getRawInput();
//HERE you get what user typed/said to Google Assistant.
//NOW you can send the input to your BACKEND, process it, get the response_from_your_backend and send it back
app.ask(response_from_your_backend);
}
let actionMap = new Map();
actionMap.set('actions.intent.MAIN', mainIntent);
actionMap.set('actions.intent.TEXT', respond);
app.handleRequest(actionMap); });
Hope that helped!
Thanks for all the help, the main parts of the solution are already given, but I summarize them here
action.json that passes on everything to fullfilment service
man in the middle (in my case IBM Cloud Function) to map JSON between services
Share context/state through the conversationToken property
You can find the demo here: Hey Google talk to Watson
Related
I have created a Webhook (https://moviebotdf.herokuapp.com/get-movie-details), it is tested with postman and dialogflow and working properly.
I want to integrate it with IBM Watson Assistant via programmatic call, but this is not returning anything (i.e. the output is "").
I checked the IBM support (https://cloud.ibm.com/docs/services/assistant?topic=assistant-dialog-actions&locale=en) and also other solutions as calling a function that could call the webhook but I am having even less succcess there. As I understand from the support, a direct call from the Assistant to the Webhook should be possible (and easier for newbies like me), hence is the solution I seek. Code in the Assistant is as follows:
{
"context": {
"skip_user_input": true,
"prodname": "$prodname"
},
"output": {
"text": {
"values": [
"$dataToSend"
],
"selection_policy": "sequential"
}
},
"actions": [
{
"name": "https://moviebotdf.herokuapp.com/get-movie-details",
"type": "client",
"parameters": {
"prodname": "$prodname"
},
"result_variable": "context.dataToSend"
}
]
}
So "prodname" is captured by Watson Assistant in the previous node (I checked that and it is working correctly) and sent to the Webhook. The variable used in the Webhook is also called "prodname". The expected output from the Webhook is stored in the variable "dataToSend", but as said above the answer in Watson is only "" as "$dataToSend" is "".
I tried also with "result_variable": "dataToSend" and "result_variable": "$dataToSend" without success, so what I guess is that the webhook is not being called...
I am new in the topic, so please do not hesitate to correct any problems in my post.
Thanks in any case in advance!
AdriĆ
IBM Watson Assistant lists three different options of making a programmatic call from within a dialog node:
client: your app is in charge of calling out to the action
server or cloud_function: IBM Cloud Functions action is invoked from Watson Assistant
web_action: The web action of an IBM Cloud Functions action is invoked from Watson Assistant
If you host your webhook on IBM Cloud Functions, then Watson Assistant can directly call it. With your current hosting and client specified, your app is in charge. In that case your app needs to check that the context includes the information about a client action, extract that related metadata, invoke the webhook and send the data back to Watson Assistant.
I have written an example for such a client action for my Watson conversation tool. See that repo for instructions.
I want to know whether IBM Watson Assistant has the feature of calling webhook.
It means when an intent of workspace is matched Watson Assistant need to send a post request to an external server including the intent in the request body.
Example for a webhook:
https://123.456.789.12:7788/myWebhook/testMethod
IBM Watson Assistant need to send a Post request to this service and that service will return a text string. Watson Assistant should get that text and show it to the user.
That is my usecase. Will it work with Watson Assistant?
i found the below documentation from IBM site.
https://console.bluemix.net/docs/services/conversation/dialog-actions.html
It says to update the json response. i.e. add another key value pair to json object as "action". in the action json array there is "name" parameter. I added above url to name parameter and checked by calling the intent whether a request comes to my web service but it didn't.
following is my json response. i assumed once the intent is matched a post request should go to my web service. but i checked my web service by printing the request body on the console. but no any request came to it. could you please tell me where did i miss?
{
"context": {
"skip_user_input": true
},
"output": {
"text": {
"values": [
"your current outstanding balance is $my_balance ."
],
"selection_policy": "sequential"
}
},
"actions": [
{
"name": "https://123.456.789.12:7788/myWebhook/testMethod",
"type": "client",
"parameters": {
"body": "$body"
},
"result_variable": "context.my_balance"
}
]
}
You found the correct method, i.e. dialog actions, to implement webhooks.
Watson Assistant supports server- or client-side actions:
For the server-side action you would set up an action with IBM Cloud Functions. That action would call the webhook.
For client side, you would pass the information similar to what you show in the question to the app (client). Your app would need to react and to call the webhook.
From what I read about your case I recommend checking out the server-side action. This tutorial about a database-driven bot implements a couple of those actions. Instead of calling the database, you would call out the webhook.
As of August 2019, there is now an integrated webhook feature in Watson Assistant.
Go to "Options" in the Assistant dialog and enable webhook. Paste the url you got after creating an action from cloud functions. Don't forget to add ".json" to the url you paste in the assistant webhook page. See more information here:
https://cloud.ibm.com/docs/assistant?topic=assistant-dialog-webhooks
I am making a google action.I have one scenario where calculation requires some time in cloud fulfillment but i don't want to keep user waiting for answer.
I want to respond to user whenever my answer is ready even when conversation with user is ended i want to send my answer in notification or something like that.
I just found this on google actions documents.
https://developers.google.com/actions/assistant/updates
Is this possible in google actions and how?
What you mean here is notifications. You can use it but please pay attention to the warning at the top of the link you provided: "Updates and notifications are currently in Developer Preview. You can build apps using the features described in this article, but you can't currently publish them".
As for the steps to crated a daily notification:
Navigate to your actions.intent.CONFIGURE_UPDATES intent.
Under Responses, go to the Google Assistant tab, click Add Message Content, and select Custom Payload.
In the "Custom Payload" box, add the following code to call the AskToRegisterDailyUpdate helper. Swap INTENT_NAME to the intent that you want to be invoked when the user interacts with your notification.
{
"google": {
"system_intent": {
"intent": "actions.intent.REGISTER_UPDATE",
"data": {"#type": "type.googleapis.com/google.actions.v2.RegisterUpdateValueSpec",
"intent": "INTENT_NAME",
"triggerContext": {
"timeContext": { "frequency": "DAILY" }
}
}
}
}
}
If using a webhook, you can call this API directly via the client library:
appMap.set('setup_update', function(app) {
app.askToRegisterDailyUpdate('INTENT_NAME');
})
})
Add one more intent called "finish_update_setup" and enter actions_intent_REGISTER_UPDATE as its Event.
Set the intent's Action to "finish_update_setup" as well.
In your webhook, open index.js and add the following. Replace Ok, I'll start giving you daily updates and Ok, I won't give you daily updates. with whatever response you want to give the user:
appMap.set('finish_update_setup', function(app)) {
if (app.isUpdateRegistered()) {
app.tell("Ok, I'll start giving you daily updates.");
} else {
app.tell("Ok, I won't give you daily updates.");
}
}
Deploy the webhook to Firebase Functions and enable webhook fulfillment in Dialogflow.
If you want to see how to create a simple notification (not daily one) - please check this doc on push notifications.
If you don't have an immediate reply to send but expect one soon-ish what you do is return a "promise". When you are in a position to reply, "fulfilling" the promise causes your reply to be delivered. I don't know what the actual timeout is but in my case I'm pretty sure at least a few second delay is allowed.
As for the updates or notifications, the API is there but the docs say you can't deploy an Action to production using them. There is a slightly cryptic comment to "contact support" if you need them.
One of these days I might try.
So, I am in the process of writing some Google Actions code against the Actions SDK. I am able to receive the requests and process them fine. I am also able to send responses that do not expect user response fine. However, when I send a response that expects a user to make a choice, the google assistant just keeps asking the question over and over no matter what the user says (except stop of course). Here's my response json. Can anyone help?
I should note that ActionOneIntent, ActionTwoIntent, and ActionThreeIntent are all configured properly in the action.json file, and work properly when invoked via a deep command to my service (ok google, ask my service to open action two).
I just can't get a response to this packet to work:
{
"conversation_token":"{REMOVED}",
"expect_user_response":true,
"expected_inputs":[
{
"input_prompt":{
"initial_prompts":[
{
"ssml":"<speak><p>Hello, would you like choice one, two, or three?</p></speak>"
}
]
},
"possible_intents":[
{
"intent":"AnswerOneIntent"
},
{
"intent":"AnswerTwoIntent"
},
{
"intent":"AnswerThreeIntent"
}
]
}
]
}
In the earliest version of the Actions SDK you could define "inDialogTriggers" as part of your Actions package. However, this was removed some time in December and the idea now is that developers process "assistant.intent.action.TEXT" which is the raw transcription of the user's input.
As such your only "possible intent" in the response should be "assistant.intent.action.TEXT" at this point. See here for reference: https://developers.google.com/actions/reference/conversation#http-response
my boss give me the task to creata a chatbot, not made with Telegram or Slack, in which use Watson Conversation service.
More, the chat bot has to be inserted inside a web page, then it has to be embeddable in html as javascript.
Are there anyone who knows other good platforms to performe these tasks?
Thanks for any help.
After replying in comments and I had another look and realised Microsoft Bot Framework could work with minimal dev investment (in the beginning).
https://docs.botframework.com/en-us/support/embed-chat-control2/
This little guy is fun. You should give him a try.
http://www.program-o.com/
I strongly suggest you to build more an assistant than a simple bot, using a language understanding service tool like Microsoft LUIS, that is part of the Microsoft Cognitive Services.
You can then combine this natural language processing tool with bot SDK like MicroSoft Botframework mentioned above, so that you could easily run queries in natural language, parse the response in a dialog in entities and intents, and provide a response in natural language.
By an example, a parsed dialog response will have something like this json
{
"intent": "MusicIntent",
"score": 0.0006564476,
"actions": [
{
"triggered": false,
"name": "MusicIntent",
"parameters": [
{
"name": "ArtistName",
"required": false,
"value": [
{
"entity": "queen",
"type": "ArtistName",
"score": 0.9402311
}
]
}
]
}
]
}
where you can see that this MusicIntent has a entity queen of type ArtistName that has been recognized by the language understanding system.
that is, using the BotFramework like doing
var artistName=BotBuilder.EntityRecognizer.findEntity(args.entities, Entity.Type.ArtistName);
A good modern bot assistant framework should support at least a multi-turn dialog mode that is a dialog where there is an interaction between the two party like
>User:Which artist plays Stand By Me?
(intents=SongIntent, songEntity=`Stand By Me`)
>Assistant:The song `Stand by Me` was played by several artists. Do you mean the first recording?
>User:Yes, that one!
(intents=YesIntent)
>Assistant: The first recording was by `Ben E. King` in 1962. Do you want to play it?
>(User)Which is the first album composed by Ben E.King?
(intents=MusicIntent, entity:ArtistName)
>(Assistant) The first album by Ben E.King was "Double Decker" in 1960.
>(User) Thank you!
(intents=Thankyou)
>(Assistant)
You are welcome!
Some bot frameworks use then a WaterFall model to handle this kind of language models interactions:
self.dialog.on(Intent.Type.MusicIntent,
[
// Waterfall step 1
function (session, args, next)
{
// prompts something to the user...
BotBuilder.Prompts.text(session, msg);
},
// waterfall step 2
function (session, args, next)
{
// get the response
var response=args.response;
// do something...
next();//trigger next interaction
},
// waterfall step 3 (last)
function (session, args)
{
}
]);
Other features to consider are:
support for multi-languages and automatic translations;
3rd party services integration (Slack, Messenger, Telegram, Skype, etc);
rich media (images, audio, video playback, etc);
security (cryptography);
cross-platforms sdk;
I've started to do some work in this space using this open source project called Talkify:
https://github.com/manthanhd/talkify
It is a bot framework intended to help orchestrate flow of information between bot providers like Microsoft (Skype), Facebook (Messenger) etc and your backend services.
I'd really like people's input to see if how it can be improved.