I integrated my Dialogflow chatbot with Facebook Messenger and I was so far able to successfully send custom text and audio files from my node.js webhook.
The problem is that in order to send audio it must be online and I have to provide its URL. Something like this:
{
"payload": {
"facebook": {
"attachment": {
"payload": {
"url": "https://actions.google.com/sounds/v1/alarms/bugle_tune.ogg"
},
"type": "audio"
}
}
},
"platform": "FACEBOOK"
},
My main goal is to be able, based on a decision that I make, to choose whether to send the response back to Messenger as text or as text-to-speech of the usual text response.
For Example: if the user says hi and the chatbot should answer by "hello", I want based on certain conditions to be able to either forward the "hello" as text or to send it as a vocal message (ssml or text to speech).
Based on my search I'm kind of convinced that fulfillment payloads for facebook do not include the SSML type. If true, is there a turnaround solution for my problem? Is there another way to send text-to-speech?
(The platform doesn't necessarily have to be Facebook, but should be feasible to integrate with Dialogflow)
Related
I have been trying to make my own chatbot with dialogflow CX,
I cant see to find enough DOC about this tool.
I am trying to make the bot start the conversation when i join the session but i cant find a way to do it.
Right now my chatbot needs a "hello" or some training word to start the dialog, but i want the chatbot to start this.
I think you can do it with "Custom payload" but i cant find an example of how to do it.
Also i know in DialogFlow ES you had a "Suggestion Chip" option where you could put in a button the answer options,
but i cant find it on CX, do i have to code it now?, can any kind hearth give me an example or extra documentation about how to code this bot?
Pd: I am new and learning how to do this chatbot, google cloud and object programming need advice in general, thanks!
Right now i am using https://cloud.google.com/dialogflow/cx/docs official doc
custom buttons with hints/suggestions as you outline in your question are only available in Dialogflow CX for integrated services. You can find information about which service is supported on this page. Otherwise, if you are able, you can develop your own integration through their API, i'm using the Python one.
If you decide, for example, to activate the Messenger integration to make your bot available via FB Messenger, you can visit the specific page and find, for example, that buttons can be set up this way.
There are many other response types, you can browse them in the same page (list, button, description, image, card): for each of them google provides a sample code to put in the "Custom Payload" box for the fulfillment. For example a box to www.yoursite.org would work like this:
{
"richContent": [
[
{
"type": "button",
"icon": {
"type": "chevron_right",
"color": "#FF9800"
},
"text": "Button text",
"link": "https://yoursite.org",
"event": {
"name": "",
"languageCode": "en",
"parameters": {}
}
}
]
]
}
by specifying "parameters" or "event" you can trigger Dialogflow events to manage the conversation flow.
Hope this made things clearer for you!
I have an application that communicates with Google Assistant via Webhook. When a user asks for something, my app sends the question to AI (Watson IBM). After it gets the response, I want to show it to the user and end the conversation. So I send a text from Watson and nextSecene =actions.scene.END_CONVERSATION. But Google Assistant just ends the conversation without showing the response to user. So is it possible to show the response message to the user and than end the conversation?
Example of my app JSON format response:
GAResponse(prompt=GAPrompt(override=false, firstSimple=GAFirstSimple(speech=<speak>You are very smart bro,y<break time="100ms"/> and i love monsters like you.</speak>, text=You are very smart bro and i love monsters like you), content=null, lastSimple=null, link=null, canvas=null, orderUpdate=null), scene=GAScene(name=null, slotFillingStatus=null, slots=null, next=actions.scene.END_CONVERSATION) ...)
Yes this is possible.
I'm not sure which library your using to generate your response json, but below is an example that provides the speech and text data and ends the conversation. You can learn more on the fulfillment (aka webhook) on the reference documentation
{
"session": {
"id": "example_session_id",
"params": {}
},
"prompt": {
"override": false,
"firstSimple": {
"speech": "<speak>You are very smart bro, <break time="100ms"/> and i love monsters like you.</speak>",
"text": "You are very smart bro and i love monsters like you"
}
},
"scene": {
"name": "SceneName",
"slots": {},
"next": {
"name": "actions.scene.END_CONVERSATION"
}
}
}
If you're interested in using Assistant Conversation libraray, check out this link to see an example response
I created a chatbot on Dialogflow which informs the user about the names of the members of my (extended) family and about where they are the living. I have created a small database with MySQL which has these data stored and I fetch them with a PHP script whenever this is appropriate depending on the interaction of the user with the chatbot.
I have integrated this chatbot to Facebook Messenger. My question is the following:
Can I directly trigger the Facebook messenger camera to be turned on through Dialogflow (and without using any other front-end camera)?
The reason I want to turn on the camera is to allow the user to take a photo of himself/herself and then I will process the photo with some computer vision libraries to infer if this person at the photo is a family member of mine. Obviously, I can simply create another basic front-end where I will turn on a camera whenever e.g. an intent called 'Camera' is triggered but I was wondering if can do this directly on facebook messenger.
The json response that I am receiving at my back-end from Dialogflow contains only the following UI capabilities:
"surface": {
"capabilities": [
{
"name": "actions.capability.MEDIA_RESPONSE_AUDIO"
},
{
"name": "actions.capability.SCREEN_OUTPUT"
},
{
"name": "actions.capability.AUDIO_OUTPUT"
},
{
"name": "actions.capability.WEB_BROWSER"
}
]
}
therefore my first impression is that turning on a camera directly through Dialoglfow (and Facebook Messenger) is not possible.
Am I right?
Well, you're right... but probably not for the reasons you think.
First, you're not going to be able to turn on the Facebook Messenger camera because you're not using Facebook Messenger. You're using the Google Assistant. Right now, the Google Assistant doesn't define a way to send an image to the Action you're interacting with. (It does work with Google Lens, but at this point, there is no way to get that to you.)
Second, you wouldn't actually "turn on" the camera. If the user sends you an image through the Messenger camera, you can process this by looking at the originalRequest field in the JSON you get on your fulfillment which should contain the message from Facebook which contains the image.
I'm trying to create a chatbot which once "greetings" process is done goes on and initiate a new topic without any user query. It has to be something like the following:
bot : hello
user : hello
bot : how old are you?
user : 35
bot : Great.
bot : Let's talk about politics. Are you american?
Until the "great" line everything works but then I cannot trigger the event that will prompt the line "Let's talk about politics...."
The doc is vague, can I do this without webhooks? And if not, how would a webhook like this look like?
You can define multiple responses in Dialogflow's console as seen in the screenshots below by clicking the Add Message Content button in the response section of the intent you'd like to add the response to. You can also send multiple messages for some platforms (depending on platform feature availability) with webhook fulfillment using rich messaging responses documented here: https://dialogflow.com/docs/rich-messages
Go to the response section of the intent you'd like to add a 2nd response to:
Click ADD MESSAGE CONTENT and select Text response:
Enter you second message in the second text box provided:
Yes, you can define multiple responses. If you are planning to use Facebook Messenger platform to show the responses you can use the code below. Change "Response 1" and "Response 2" to your desired text and dump the my_result object as json and return it back. You need to change the "platform" if you want to use any other platforms than messenger.
my_result = {
"fulfillmentMessages": [
{
"text": {
"text": [
"Response 1"
]
},
"platform": "FACEBOOK"
},
{
"text": {
"text": [
"Response 2"
]
},
"platform": "FACEBOOK"
}
]
}
In api.ai,
Rich formatting is supported for skype pretty fine. Here's an example:-
Here's the code of custom payload of api.ai:-
{
"skype": {
"text": "OCAS means Online Credit Approval System.\n Click [here] (http://www.erainfotechbd.com/product/ocas-online-credit-approval-system/) for more information. ",
"attachments": [
{
"contentType": "image/png",
"contentUrl": "http://weknowyourdreams.com/images/smile/smile-07.jpg",
"name": "Profile-picture.png"
}
]
}
}
Now, I wanna do something like this in facebook messenger also. But in the api.ai doc, only audio, video and file is supported.
Is there any other way to do so for url also in messenger ?
Generally webhook responses can deliver any custom payload to most platforms that API.AI supports, so any feature supported by Facebook Messenger, Skype, Google Assistant, etc. can be utilized through API.AI. Unfortunately Facebook doesn't seem to support hyperlinked text. The closest analog I could find is a what Facebook calls a URL Button. Below is an example of how you could create a API.AI response that has a Facebook URL Button with Facebook's Button template:
{
"speech": "OCAS means Online Credit Approval System. Check your phone for more information.",
"displayText": "OCAS means Online Credit Approval System. Click here for more information: http://www.erainfotechbd.com/product/ocas-online-credit-approval-system/",
"data": {
"facebook": {
"attachment": {
"type": "template",
"payload": {
"template_type": "button",
"text": "OCAS means Online Credit Approval System",
"buttons": [
{
"type": "web_url",
"url": "http://www.erainfotechbd.com/product/ocas-online-credit-approval-system/",
"title": "Click here for more information"
}
]
}
}
}
}
}
You can use also use any other Facebook Messenger APIs by including any JSON inside the "message" attribute that you see from Facebook's Messenger documentation and it will be passed along to Facebook by API.AI
PS - the code above should produce a message that looks something like this: