How to get user's unique identity from google home's voice match profile? - actions-on-google

I want to develop an application(action) on Google Home, which will return some confidential information to user. So, I can't authenticate the user based on account linking as that device can be used by any person in the room. How can I get user's unique identity(may be email id) from voice match profile?

Account Linking is tied to the specific user that says "Ok Google" or "Hey Google" to invoke your Action. If the user that says it does not have a registered voice on the device they use (they are an "anonymous" user), then they cannot do Account Linking. This sounds like the level of security you're aiming for and it isn't clear why you think otherwise.
You can get a user's unique identity (with or without Account Linking) by using app.getUser().userId if you're using the node.js library. However, if the user is "anonymously" on a device (no voice registration has been done, or they do not match a registered voice), this identity will change each session because there is no way to determine who the user is.
You cannot get the user's email ID. (And, even if you could, one would hope that it follows similar rules to handling anonymous users above.)
Update to clarify what I mean by an "anonymous" request:
When a user first sets up a Google Home device, they're prompted to register for Voice Match on that device. Registering their voice requires them to say "Ok Google" and "Hey Google" a few times so a voice pattern can be established. Afterwards, these trigger words spoken by this person will have the rest of the statement associated with their Google Assistant account. If they have done Account Linking to your service, their Google Assistant account (which is matched to their voice for the invocation phrase) is linked to your service.
Other users can be permitted to register for Voice Match for a device. This will associate their voice for the invocation phrases with their Google Assistant account. If that user has done Account Linking with your service, then their voice triggering the invocation phrase will have things associated with their linked account.
But what about someone who uses the Home without having setup Voice Match for that device? The invocation phrases still work. They can still invoke your Action. But what Assistant account is associated with it?
Since the specific Home device doesn't know the person invoking it and making the request, this is an "anonymous" user. Anonymous users can't do Account Linking, since there is no Assistant account that is associated with the request. Although there is a userID provided - this ID will change for every conversation since the device has no way to know that the anonymous person making the request this time was the same anonymous person making it last time.
But what happens if the user has setup a Home device, but has not setup Voice Match on it at all? For security reasons, the system has to assume that this is really a shared device and that all requests have to be handled anonymously.
Additional Update
The "anonymous UserID" that is talked about above has been deprecated and will be removed in May 2019.
It is now possible to get the user's email address, assuming they have registered it as part of their Google profile, by using Account Linking with Google Sign-In.
Conceptually, may other parts of the answer remain valid, although some details may have changed.

You can use Account Linking for that purpose. If your phone is in locked state, then only the person who's voice will be registered in phone will be able to invoke the google assistant using OK Google.
If this type of invocation works for you, then go ahead and read the rest of the post. I've already posted a similar answer here
We have to enable the webhook first and we can see how to enable the webhook in the dialog flow fulfillment docs
If we are going to use Google Assistant, then we have to enable the Google Assistant Integration in the integrations first.
Then follow the steps mentioned below for the Account Linking in actions on google:-
Go to google cloud console -> APIsand Services -> Credentials -> OAuth 2.0 client IDs -> Web client -> Note the client ID, client secret from there
-> Download JSON - from json note down the project id, auth_uri, token_uri
-> Authorised Redirect URIs -> White list our app's URL -> in this URL fixed part is https://oauth-redirect.googleusercontent.com/r/ and append the project id in the URL
-> Save the changes
Actions on Google -> Account linking setup
1. Grant type = Authorisation code
2. Client info
1. Fill up client id,client secrtet, auth_uri, token_uri
2. Enter the auth uri as https://www.googleapis.com/auth and token_uri as https://www.googleapis.com/token
3. Save and run
4. It will show an error while running on the google assistant, but dont worry
5. Come back to the account linking section in the assistant settings and enter auth_uri as https://accounts.google.com/o/oauth2/auth
and token_uri as https://accounts.google.com/o/oauth2/token
6. Put the scopes as https://www.googleapis.com/auth/userinfo.profile and https://www.googleapis.com/auth/userinfo.email
and weare good to go.
7. Save the changes.
In the hosting server logs, we can see the access token value and through access token, we can get the details regarding the email address.
Append the access token to this link "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" and we can get the required details in the resulting json page.
accessToken = req.get("originalRequest").get("data").get("user").get("accessToken")
r = requests.get(link)
print("Email Id= " + r.json()["email"])
print("Name= " + r.json()["name"])

Related

Why does it appear I am signing into googleusercontent.com when doing gapi.auth2.getAuthInstance().signIn() from a GSuite Add On?

I have written a GSuite Add On that makes Google Rest API calls using https://developers.google.com/docs/api/reference/rest/v1/documents. To make those calls my Add On needs to be authenticated to Google using the right scopes. So I took the usual web app code that I am used to:
gapi.auth2.getAuthInstance().signIn()
Before that
gapi.client.init()
is called with an API Key and Client ID. The Client ID is associated with a project in https://console.developers.google.com/ - let's call it Mike Add On. I had to configure the uri that is hosting my add on as an authorised origin - it was something like
https://n-5eu5f2kfjyztq5mypu757655tecik5nd52mv7sztq-0lu-script.googleusercontent.com
I would expect the project name (Mike Add On) to come up in the OAuth Consent screen when a user of my add on logs in. But instead I get
Choose an account to continue to googleusercontent.com
Why is googleusercontent.com coming up as the app name, instead of the one associated with the Client ID?

Authentication needed when chat bot conversing with user

This got stuck in my head from many days, can anyone help or say at-least this is not at all possible?
I'm working on developing a chat bot using dialogflow which integrates multiple applications along with google home assistant, dialogflow, actions on google and an application which i want to manage using chat or voice commands. Until now its good and got amazed of features providing by google.
But i'm expecting one more feature. Don't know whether any alternatives available for this or not, but i tried exploring and reached to desert. Below are my requirements, if others think this is really unique and useful to them as well then i can say they are improvements or add-ons i'm expecting from DialogFlow.
Let's take an example of a chat bot which is serving users through google assistant and as a web bot as well. Now while conversing, intents may trigger web-hook in fulfillments which may require an authentication like OTP(Nope if anyone thought it for payments) which means registered users or limited users only can perform actions. This is same as we use roles and groups in all the applications.
The way google is sending google prompt to the user for logging into gmail, is there any way that we can collect PIN or OTP or PASSWORD through some notification sent to the users phone as some card's or input box like and html while conversing with chatbot through web or home assistant etc..., so that it helps in adding more security.
I recently worked in a chatbot project where I had to authenticate my users. I'm writing an article about it, but I'll tell you what I did:
First of all, I'm using OAuth 2.0 protocol to authenticate my users, but if you doesn't use OAuth, there's no problem, you could do something equivalent.
I'm using Authorization Code Grand flow.
Let's see the steps:
Step 1 - Authorization Url:
My bot generates an authorization url which contains all needed data to identify the conversation in callback moment. Like this one:
https://authorization-server.com/oauth/authorize
?client_id={your-client}
&response_type=code
&state={conversation_id: 123456789}
&redirect_uri={your-callback-url}
Notice that the state parameter contains the conversation_id which identifies your conversation, this state parameter will be back when users return to your handler.
Step 2 - User Authentication
When users click in this link, they'll be redirected to your login page at your authorization server.
Step 3 - Callback
After users get authenticated, they'll be redirect back to your handler (an endpoint which will receive the authorization code from authentication server and the state parameter).
When it received this authorization code, it'll be exchanged by an access token in authorization server.
Step 4 - Store token
In the final step, you already has an access token and the conversation_id parameter, you can store it in a database, in a cache or be stateless. Your rules!
In my case, I'm using Watson Assistant with Cloudant database, and I store those access tokens in my database. So, when users request something to my bot, it could get this token from database and pass to my back-end servers.
This kind of approach, I call "magic link". And you could improve it by shortening the url as bit.ly does.
I hope it could help you, feel free to ask me if you need.
Best
You probably don't want to implement the OTP scheme yourself. While you could do this, there are other systems already in place that will do this for you.
The best is the one that you reference - Google Sign In.
Fortunately, you can leverage Google Sign In for both your website (where you would get the user to sign in and then pass this information along as you do the Dialogflow calls) and for the Assistant (where Google will pass along an ID token, indicating it has authenticated the user).

How to map social credentials with custom ones

My company has userbase of course, but I want to allow users to login and use my applications with their social accounts e.g. Outlook, Facebook, Gmail. Something that is usually not clear to me when I read resources on the Internet on the topic is how to map the social credentials with ones in our database? I know we should use an API platform or something like that, but the user identity part is not clear to me.
You basically need to, as you noted, tap into the provided response and transform or link or provision it to existing identities in your own userbase. A lot of this depends on your method of delegating authentication to external provides and things they expose back to you as part of the user profile. You basically need to grab the user profile, parse it and then determine which field can be used to link that profile to an existing account, and then establish the authentication session based on the final result.
Here is a link to a technical walkthrough that describes the same process with an SSO solution: https://apereo.github.io/2018/04/20/cas-delegated-authn-account-linking/

How to implement Login in Dialogflow chatbot

i want to add login to my chatbot with webhook.I want to take username and password from user and send it to the webhook.
I know you didn't get much search results for that
there are following approaches
If you use only Google Assistant for chatbot then use built in "Sign-in" required option and then open link https://console.actions.google.com/project/your-project-name/accountlinking/ and in that provide your required settings like
"Do you want signup ?" or "Login Only"
select type of Oauth and Implicit
Enter the endpoint url
and bingo you will get button in Google Assistant for login
If you wanna add login for web-demo use Detect Intent API
then go for Account linking OAuth2 using either code flow or implicit flow but I would suggest to go for implicit flow first.
there are various resourses out here for each language.
Integrating API you need a service account keys add that to your enviornment for Authentication to google apis
then use detectIntent function to get response and IMPORTANT use Unique session id
hope you got it ;-)

How will my .Net app acquire the necessary tokens from a user who will subscribe to my app?

I am making a .NET application that will allow potential users to upload invoices they make on my app to their quickbooks account that they have set up and synchronized with their QBD version.
what steps will they/I need to take so that when they use my app, it will upload invoices to their account? Is it:
a) when they set up their account with the Intuit AppCenter, they will pick my app from the app center services (in doing so, it will generate a set of oAuth connection parameters for me to put into my app to push invoices to their account, if so how will I get these)?
b) Will I set up an account via the Intuit Platform Partners, and ask the user to use my account ID and Password by which they will "create new app" under my account for the purposes of generating oAuth connection information for my app to use?
c) Something different from the a) and b)?
Thanks!
The OAuth stuff is specifically designed to allow end-users to push a set of OAuth tokens/credentials from QuickBooks to your app. You then store the OAuth tokens, and use those to communicate with QuickBooks.
This is an accurate description of what happens, yes:
a) when they set up their account with the Intuit AppCenter, they will
pick my app from the app center services (in doing so, it will
generate a set of oAuth connection parameters for me to put into my
app to push invoices to their account, ...
You can read more about that process in the docs:
https://developer.intuit.com/docs/0025_quickbooksapi/0010_getting_started/0020_connect/0011_from_the_intuit_app_center
The other scenario is that, if you allow it, the user may choose to connect to QuickBooks from within your app. In this case, they'll click the "Connect to QuickBooks" button (see the docs here: https://developer.intuit.com/docs/0025_quickbooksapi/0010_getting_started/0020_connect/0010_from_within_your_app) which forwards them to Intuit's site, generates the OAuth tokens, and then sends them back to your site with the OAuth tokens.
In either case, the OAuth tokens will be sent to you so that you can store them and use them to communicate with the QuickBooks APIs.
This:
... and ask
the user to use my account ID and Password by which they will "create
new app" under my account for the purposes of generating oAuth
connection information for my app to use?
Is absolutely not what you want them to do. The whole point of OAuth is that you don't need to share any usernames/passwords of any time with any one.
Here are some additional answers to your other comments:
The part I am trying to figure out is what my customers will need to
do to be able to benefit from my app and its QuickBooks Integration
functionality.
They will need to log in to their Intuit.com account, and follow the prompts to connect their QuickBooks company to your app (i.e. follow the prompts to forward the OAuth credentials to your app).
Sounds like he/she will need to set up some account with ID and
password.
If they don't already have an Intuit.com account, they will be prompted to create one during the OAuth process. In the case of QuickBooks Online, they will ALWAYS already have an Intuit.com account (it's the same thing they use to log in to QuickBooks Online).
Now it seems to me that there are two ways that the customer can set
up accounts with Intuit.
They can set up an account through AppCenter, or within QuickBooks desktop directly (when you install the software, it prompts you) or if you're using any Intuit service already (e.g. QuickBooks Online) then you use your existing QuickBooks Online account.
Developer.Intuit.com accounts are only for developers (e.g. for YOU). Your end-users will not have a developer.intuit.com account. Nor do they need one.
a) what are the differences between the two?
Developer.Intuit.com accounts are only for developers. Your end-user will never see the developer.intuit.com website, nor will they see the option to "Create an App".
b) which should be used if the customer wants to set up syncing their
QBD? c)Which should be used if the customer wants to use my .net App.
d)Which should the customer use if they want both (b) and (c)? e) If
the customer creates an account the 2nd way, won't they get confused
by all the "developer" lingo?
None of those questions are applicable - end-users will never see or be prompted about any of the developer stuff. The only reason you see that is because you're registered as a developer. Normal end-users will not be, and thus won't have any of these options.
f)What if the customer has already synced their QBD with intuit?
Then they can just log in to their existing account to get connected (i.e. to send the OAuth credentials over to your app).
With regards to your questions:
A developer can choose to get his app listed on the appcenter or not depending on mkting requirements.
To understand the difference in the process- you can create a sample and then on developer.intuit.com, go to My Apps-> manage my app-> select your app. Then test connections.
You can use any of the above to use QBD data. Please see the additional info too that I have provided.
Also go through the link which consolibyte has mentioned.
If you are logging in from appcenter app, then since you are already logged in, then only OAUTH will be required.
In logging from within your app, you do not need to provide your user Id and password. The user needs to register for the first time with Intuit.com and then OAUTH process follows for him. These open id/email details can be saved in your db so that user need to enter them a next time.
The customer is never about the details of the developer except for the app which will access their company data.
The sync manager for QBD
OAUTH authorizes your app to connect to their company files.
Any of the above can be used to access QBD data.
The sync manager runs manually or automatically. So, customer can choose to sync/not sync his data.
---------------Addtional info----------
There are two ways to integrate with QuickBooks, the first is with REST APIs and the second is with the downloadable QBXML SDK v12. The REST APIs and that integration model is for applications that are to be sold to customers via our Appcenter. They are generally Saas applications.
The second model is the QBXML sdk which is available for anyone to use, especially for custom integrations.
The reference for the QBXML SDK is here
http://member.developer.intuit.com/qbSDK-current/Common/newOSR/index.html
Please go through:
https://developer.intuit.com/docs/0025_quickbooksapi/0055_devkits ->QBXML SDK
Lastly through the QBXML SDK you can add a custom field to any entity you want. See the reference above and take a look at DataExt Add Request. and once added Query Request. I believe this will give you what you are looking for.
Keep in mind this is for QuickBooks for Windows not QuickBooks Online.
You can go through FAQ page too-
https://developer.intuit.com/docs/0025_quickbooksapi/0058_faq
I am settled at this point from what Consolibyte has shared and what I have learned also. The whole picture (for me at least) is that both the Developer Account AND the App Center Account is needed. The Developer account provides the consumer key, consumer secret and application token that the .Net app will use to authenticate with Intuit. The App Center Account is needed to allow the user to sync their QuickBooks Desktop data to a location where the app center (and other 3rd party apps) can have access to it (after user authorizes of course), and to give it's user an ID and Password needed to authorize the .net app ( or any app for that matter) to access his/her Intuit data. It is the combination of the Developer Account's tokens and the App center Account's ID and password that allow the generation of an authentication/access token by which the .Net app can instantiate a Dataservices object so it can read/write to the user's instance of QB data in the App Center. Apart from the user creating an account in the App Center and synching their QB Desktop data with it, There should be little or nothing else to do except provide the ID and password, providing the .Net app is coded right. I am not talking about OpenID in this case, but do know that it uses the same ID and password.
...so I'm good. Thx.