Hello I'm developing a conversational program for google nest 2 with google actions sdk.
(using google cloud function as a webhook for the google action and using google interactive canvas API. )
This is a brief description for my conversation.
I wake up the my conversational program with wake up keyword "Ok google talk to my manager"
During conversation, the google nest hub delivers a set of questions that can be answered in voice or by touching a button in the display. (each question is answered in each scene)
I have trouble that google nest agent doesn't listen to user at the start of the conversation.
the agent delivers the first question scene in verbal but don't listen to user(no 4 dots icon that expresses the speaker is listening to the user).
(for later scenes, it listens to user well!)
It happens when user stops for a moment and restarts the conversation program too.
I'm wondering why this happens cause before adapting google interactive canvas API to google action, the agent worked well.
Are there any error in my webhook handlers?
Is there anyway to see related logs in the google cloud function?
Related
I have created an agent in DialogFlow that interacts with the user giving her the ability to check for details like demographic from a back-end database. I have achieved this using Fulfillment/Webhook calls and have successfully parsed/returned different types of Responses to the user to play with. I have tested this on both an Android device and the Google Actions Simulator.
Now, what I want to achieve is to give user the ability to "Set a Reminder" on a platform like Google Calendar or similar through my App. By default, Google Assistant does this by simply telling it to "remind me..." and Google creates a reminder in the calendar. But as soon as I invoke my app, the Assistant is out of scope so using the "remind me" command won't work. I want to use the Google Calendar for example to create a reminder through my agent so that the user is notified at the due time.
I understand that when the user invokes an App in Google Assistant, she is using her Google credentials so I guess if I want to use the App to create a reminder, I should have the option. I have had a look at the Calendar API but I am not sure if I should proceed that way. Is there a better way of achieving this?
Please note, I want to use a reminder service that is available to the user without subscribing to a third-party service.
The Calendar API is the way to go. Details here.
In those docs you'll see reference to the access token, which you can get by using the helper intent askForSignIn (example of use here in one of Google's example projects).
Hope this helps!
So for all those who are facing the same issue, here is what I did:
First created an account on https://auth0.com
Followed this guide to configure the necessary settings to link to Google Assistant (ignored the Jovo part)
Used Auth0 guide to configure and link to my Google project credentials (ignored the Admins SDK Service part)
Now when I run my Google Actions App on the device, a button "Link [App Name] to Google" showed up. Clicking on that "signed me in".
Parsed the response in my webhook to grab originalRequest.data.user.accessToken
This is the token mentioned in #lukedavis answer. The next step was to use the token to execute a call to the Calendar API. Upon trying that, I am facing a different problem which I am trying to find the solution for. But the above should assist those who wish get the accessToken.
Refer me to some ready made example of google assistant notification, I'm a tech guy I have already developed app with dialogflow(donot support notification yet), I have both google home and mobile assistant, just want to see google action notification in action.
There's this very detailed guide by Google with ready made code that you can easily adapt to your needs.
It offers code for Dialogflow as well as for the plain ActionsSDK.
Basically the steps are:
Ask for permission
React to it in your fulfillment code (confirming the user's decision)
Notifying Google's backend when it should send a notification to the user (you need a JSON web token). This last step has to be done on some backend which knows about when to send notifications
Those steps are outlined in more detail on the linked site and there's also described how to obtain the token.
I am developing a Google Assistant app for Google Home, and about to launch the app. I am wondering, once my app is live on the Google Assistant platform, whether I (as a developer) would have access to all of the conversation histories as users interact with my app.
If so, does Actions on Google / Google have some sort of interface to view / download the history? Or, do I have to log and capture the history myself? I thought the history will be really helpful for me to improve my app.
Many thanks!
It depends a bit on how you have built your Action.
If you have built it with one of the templates - then no, you don't have access to the conversations.
If you have built it with the Actions SDK or with Dialogflow, then you will have access to quite a bit of information that is delivered to your fulfillment webhook. If you have intents that do not send anything to your webhook - you will not get that information.
There are tools that help you examine conversation flow, see where users get stuck or fall out of the conversation, or how they're using your Action. Most of them have good integration with the Actions on Google libraries. I use a combination of Chatbase, Dashbot.io, and Google Analytics.
Would someone please explain or point me to a tutorial to develop a random question testbank chatbot in API.ai.
I'd recommend starting with https://triviatemplate.com/ which creates an entire Google Assistant app for you, you just provide the questions. In the background Trivia Templates creates an API.AI agent in the background for you in the background and deploys it to the Google Assistant.
After you've created your Trivia template assistant app, you can login to API.AI with the same account and see the agent that was created for you, tweak the responses or edit the functionality and deploy to other platforms (Facebook Messenger, Slack, Alexa, etc.)
For my conversation actions, would like to authenticate the user before he can utilize my services to ensure he is the true user and not anyone else. Can Google provide voice prints to recognize the person whose speaking to Google Home?
Am confident that Google must be recording voice. So it would be really helpful for authentication if its shared.
While they do record your voice (you can see and hear a copy of all requests done via voice), they do not pass this on to services. They also are not (currently, anyway) trying to do user identification based on voice.