Related
Here is my scenario. Imagine there is a Yoga studio that uses a professional booking and reservation system that exposes an API. Through this API an application can make a reservation for a client. The API takes the client's userid and password to make the reservation. The booking API doesn't use OAuth or any social media sign-ins.
My desire is to create an Assistant Action that would retrieve the list of classes and allow the client to make a booking.
My puzzle is what design/architecture to look towards to supply the userid/password pair required by the booking API.
How have others solved this puzzle?
Should I store the userid/password as "user state" associated with the action?
First, you should have a conversation with the API provider about why they don't provide an OAuth-based solution. This is a security vulnerability waiting to happen, if it hasn't already.
Second, you need to think very carefully about your own risk profile in this case:
Google does not allow you to collect credential information (ie - passwords) through your Action.
Because of this, you must use Account Linking to authenticate them.
This means that you will need something (ie - a database or data store) to manage their account on your side.
This database would be a good place to keep the username/password you need to use for them for the API...
...but it now means that you need to take extreme care about protecting this database.
You don't really say how this API allows for accounts to be created and managed. If these accounts are just used for you (ie - the user doesn't necessarily see them), then you can mitigate some of that risk by treating the username/password as an opaque token that you manage and generate and that the user never sees.
If this is something that the user is aware of, then you'll need to approach the account linking in one of two ways:
Have them log into your service via an app or webapp using this credential info that you will need to save (ack!) and then link to the Assistant using OAuth.
Have them log into your service via an app or webapp using Google Sign-In, which will carry over to your Action. Then have them provide the credential info for the API, which you will need to save (ack!).
This got stuck in my head from many days, can anyone help or say at-least this is not at all possible?
I'm working on developing a chat bot using dialogflow which integrates multiple applications along with google home assistant, dialogflow, actions on google and an application which i want to manage using chat or voice commands. Until now its good and got amazed of features providing by google.
But i'm expecting one more feature. Don't know whether any alternatives available for this or not, but i tried exploring and reached to desert. Below are my requirements, if others think this is really unique and useful to them as well then i can say they are improvements or add-ons i'm expecting from DialogFlow.
Let's take an example of a chat bot which is serving users through google assistant and as a web bot as well. Now while conversing, intents may trigger web-hook in fulfillments which may require an authentication like OTP(Nope if anyone thought it for payments) which means registered users or limited users only can perform actions. This is same as we use roles and groups in all the applications.
The way google is sending google prompt to the user for logging into gmail, is there any way that we can collect PIN or OTP or PASSWORD through some notification sent to the users phone as some card's or input box like and html while conversing with chatbot through web or home assistant etc..., so that it helps in adding more security.
I recently worked in a chatbot project where I had to authenticate my users. I'm writing an article about it, but I'll tell you what I did:
First of all, I'm using OAuth 2.0 protocol to authenticate my users, but if you doesn't use OAuth, there's no problem, you could do something equivalent.
I'm using Authorization Code Grand flow.
Let's see the steps:
Step 1 - Authorization Url:
My bot generates an authorization url which contains all needed data to identify the conversation in callback moment. Like this one:
https://authorization-server.com/oauth/authorize
?client_id={your-client}
&response_type=code
&state={conversation_id: 123456789}
&redirect_uri={your-callback-url}
Notice that the state parameter contains the conversation_id which identifies your conversation, this state parameter will be back when users return to your handler.
Step 2 - User Authentication
When users click in this link, they'll be redirected to your login page at your authorization server.
Step 3 - Callback
After users get authenticated, they'll be redirect back to your handler (an endpoint which will receive the authorization code from authentication server and the state parameter).
When it received this authorization code, it'll be exchanged by an access token in authorization server.
Step 4 - Store token
In the final step, you already has an access token and the conversation_id parameter, you can store it in a database, in a cache or be stateless. Your rules!
In my case, I'm using Watson Assistant with Cloudant database, and I store those access tokens in my database. So, when users request something to my bot, it could get this token from database and pass to my back-end servers.
This kind of approach, I call "magic link". And you could improve it by shortening the url as bit.ly does.
I hope it could help you, feel free to ask me if you need.
Best
You probably don't want to implement the OTP scheme yourself. While you could do this, there are other systems already in place that will do this for you.
The best is the one that you reference - Google Sign In.
Fortunately, you can leverage Google Sign In for both your website (where you would get the user to sign in and then pass this information along as you do the Dialogflow calls) and for the Assistant (where Google will pass along an ID token, indicating it has authenticated the user).
I want to develop an application(action) on Google Home, which will return some confidential information to user. So, I can't authenticate the user based on account linking as that device can be used by any person in the room. How can I get user's unique identity(may be email id) from voice match profile?
Account Linking is tied to the specific user that says "Ok Google" or "Hey Google" to invoke your Action. If the user that says it does not have a registered voice on the device they use (they are an "anonymous" user), then they cannot do Account Linking. This sounds like the level of security you're aiming for and it isn't clear why you think otherwise.
You can get a user's unique identity (with or without Account Linking) by using app.getUser().userId if you're using the node.js library. However, if the user is "anonymously" on a device (no voice registration has been done, or they do not match a registered voice), this identity will change each session because there is no way to determine who the user is.
You cannot get the user's email ID. (And, even if you could, one would hope that it follows similar rules to handling anonymous users above.)
Update to clarify what I mean by an "anonymous" request:
When a user first sets up a Google Home device, they're prompted to register for Voice Match on that device. Registering their voice requires them to say "Ok Google" and "Hey Google" a few times so a voice pattern can be established. Afterwards, these trigger words spoken by this person will have the rest of the statement associated with their Google Assistant account. If they have done Account Linking to your service, their Google Assistant account (which is matched to their voice for the invocation phrase) is linked to your service.
Other users can be permitted to register for Voice Match for a device. This will associate their voice for the invocation phrases with their Google Assistant account. If that user has done Account Linking with your service, then their voice triggering the invocation phrase will have things associated with their linked account.
But what about someone who uses the Home without having setup Voice Match for that device? The invocation phrases still work. They can still invoke your Action. But what Assistant account is associated with it?
Since the specific Home device doesn't know the person invoking it and making the request, this is an "anonymous" user. Anonymous users can't do Account Linking, since there is no Assistant account that is associated with the request. Although there is a userID provided - this ID will change for every conversation since the device has no way to know that the anonymous person making the request this time was the same anonymous person making it last time.
But what happens if the user has setup a Home device, but has not setup Voice Match on it at all? For security reasons, the system has to assume that this is really a shared device and that all requests have to be handled anonymously.
Additional Update
The "anonymous UserID" that is talked about above has been deprecated and will be removed in May 2019.
It is now possible to get the user's email address, assuming they have registered it as part of their Google profile, by using Account Linking with Google Sign-In.
Conceptually, may other parts of the answer remain valid, although some details may have changed.
You can use Account Linking for that purpose. If your phone is in locked state, then only the person who's voice will be registered in phone will be able to invoke the google assistant using OK Google.
If this type of invocation works for you, then go ahead and read the rest of the post. I've already posted a similar answer here
We have to enable the webhook first and we can see how to enable the webhook in the dialog flow fulfillment docs
If we are going to use Google Assistant, then we have to enable the Google Assistant Integration in the integrations first.
Then follow the steps mentioned below for the Account Linking in actions on google:-
Go to google cloud console -> APIsand Services -> Credentials -> OAuth 2.0 client IDs -> Web client -> Note the client ID, client secret from there
-> Download JSON - from json note down the project id, auth_uri, token_uri
-> Authorised Redirect URIs -> White list our app's URL -> in this URL fixed part is https://oauth-redirect.googleusercontent.com/r/ and append the project id in the URL
-> Save the changes
Actions on Google -> Account linking setup
1. Grant type = Authorisation code
2. Client info
1. Fill up client id,client secrtet, auth_uri, token_uri
2. Enter the auth uri as https://www.googleapis.com/auth and token_uri as https://www.googleapis.com/token
3. Save and run
4. It will show an error while running on the google assistant, but dont worry
5. Come back to the account linking section in the assistant settings and enter auth_uri as https://accounts.google.com/o/oauth2/auth
and token_uri as https://accounts.google.com/o/oauth2/token
6. Put the scopes as https://www.googleapis.com/auth/userinfo.profile and https://www.googleapis.com/auth/userinfo.email
and weare good to go.
7. Save the changes.
In the hosting server logs, we can see the access token value and through access token, we can get the details regarding the email address.
Append the access token to this link "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" and we can get the required details in the resulting json page.
accessToken = req.get("originalRequest").get("data").get("user").get("accessToken")
r = requests.get(link)
print("Email Id= " + r.json()["email"])
print("Name= " + r.json()["name"])
The Problem
MS Azure provides an extensive REST API. However, there is a significant amount of complexity when trying to get that API to work. From outdated and incomplete documentation to simple examples not working, performing what should be an easy task is instead nightmarish.
The Task
What are the exact, precise, detailed steps necessary to list the available VMs for someone who has logged in using Oauth2? For instance, this can be done using the azure-cli.
azure vm list
What are the steps to accomplish the same thing using REST and Oauth2?
The Requirements
The answer must not use Visual Studio, PowerShell, C#, an SDK, or any other such tool to accomplish this task. Only the portal is allowed for setup, and only Oauth2 is allowed for authorization, and only the REST API is allowed for actual information retrieval.
The answer must not simply link to external sources, although external references are encouraged for completeness.
The answer may assume the user has an Azure account. It must include information about creating the Oauth2 client, credentials, and any step necessary to get the appropriate token.
The answer must be detailed.
The answer must provide examples. Images, too are strongly encouraged.
The answer should include information about possible errors, their meanings, and what too look for to fix them.
First, we can find this rest API in azure resource portal. It is the same with Azure CLI command azure vm list.
I have tested it on my local with http request. here is my tested screenshot:
Request URL: https://management.azure.com/subscriptions/<subscription ID>/providers/Microsoft.Compute/virtualMachines?api-version=2016-03-30
Header:Authorization: bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI.....
So It is very import if we get the access token. The following demonstrate us how to get the access token.
Get Token(POST):
Request URL: https://login.microsoftonline.com/<tenant id>/oauth2/token
Body: grant_type=client_credentials&client_id=<client id>&resource=https%3A%2F%2Fmanagement.core.windows.net%2F&client_secret=<client secret>
Here is my screenshot in fildder:
We need to get client id and client secret in azure ad application. For how to regist an application in Azure AD. Please have a look at this article: https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-create-service-principal-portal.
Please note:
1) we need to add "Windows Azure Service Management API" in portal "Required permissions" like the following screenshot:
2) We need assign "Contributor" for this service principal. click subscription-> Access Control-> click Add -> click "Select a Role" -> click Contributor->click Add User-> Find the application you created above-> click OK.
Overview
Making requests against the Azure Rest API is a bit more complicated than perhaps you would think at first glance. In particular, there are a number of esoteric and not-so-helpful error messages you may run into while getting the nobs tweaked just right.
Introduction and Terms
Setting up The Application
Getting the access_token
Making the API request
Common errors thus far
Introduction and Terms
One of the pieces of this process that can make it so confusing and difficult is the terminology. Until you understand that, knowing how to deal with errors is very difficult. We'll go over some of the more common ones here.
Subscription - This is basically what you'd expect. It refers to the Microsoft Azure Services subscription. It basically acts as the top-level umbrella for the organization.
Tenant - This is like a sub-organization, maybe a department or group. There can be multiple tenants under a single subscription.
User - As expected, a user is a single individual. Users are scoped to tenants.
Application - The Application is the software program trying to use the API. It must be registered and configured to do so.
Service Principal - This is essentially The Application. It is the entity making API Service requests.
Setting Up the Application
Although you might not guess it, this is probably the most complicated part of the process. Let's start by creating The Application in the portal.
Create The Application
Follow this click path: Azure Active Directory -> App Registrations -> New
There should be a form for application creation, with the following fields:
Name
This is simple the name of The Application. When authenticating, it will be shown to users. For the purposes of this "tutorial", we'll call it API Tutorial. This can be edited after creation.
Application Type
The type of The Application. For our purposes, we should choose "Web app/API". This cannot be edited after creation.
Sign-on URL
The is the redirect that will be used if we go the "authorization_code" route for sign on. This can be useful as the response will include an "id_token". We'll get into that a bit later. For now, let's make this http://123AzureApiTutorial.com/code. This can be edited after creation.
Once the Application has been created, you should see a property, Application ID. This is the client_id used in the OAuth2 flow. Take a note of its value.
Create the Client Secret
The OAuth2 flow requires a client secret value for authentication.
To generate it, follow this click path: Azure Active Directory -> App Registrations -> API Tutorial -> All Settings -> Keys
Enter the key description: API Tutorial Key, and the Duration: In 1 year.
Click Save. This will generate the Key Value. You must copy the value here and save it somewhere. You will not have another opportunity to do so.
This value is the client_secret in the OAuth2 flow.
Add the correct permissions
To get to the permissions, follow this click path: Azure Active Directory -> App Registrations -> API Tutorial -> All Settings -> Required Permissions -> Add
Here you will see the list of possible APIs. The one we care about for Azure is Windows Azure Service Management API. There is currently only one permission: Access Azure Service Management as organization users (preview). Select it, click Select, and then Done.
Getting the access_token
The access_token is what allows us to make requests against the API. There are two primary ways to do this. I suggest reading about both before trying to implement them.
Authorization Code
The authorization code is a two-step process. First we obtain the authorization code, and then we use that to get the access_token. A benefit of this route is that we get back an id_token as well, with a variety of useful claims like the user's name, email address, etc.
The request format is as follows: (GET) https://login.microsoftonline.com/<tenant-id>/oauth2/authorize?client_id=<client-id>&scope=api&redirect_uri=<redirect-uri>&response_type=code&prompt=consent. Let's go over the parameters here really quick.
Tenant ID
This can be obtained be using the click path Azure Active Directory -> Properties and copy the Directory ID. This is, in fact the Tenant ID value. It just has a different name to help with the overall confusion.
Client ID
This is the Application ID we retrieved previously.
Scope
This is the scope of the code. We just want to use the API.
Redirect URI
This is the sign-on URL you specified when creating API Tutorial. After the user logs in, they will be redirected to this URL with a "code" parameter in the query string.
Response Type
This is what we want the response to be. We want an authorization code, so we just use the value code.
Prompt
This specifies whether or not to prompt the user to consent to the permissions. If we did not have this, and changed permissions, the request would just unexpectedly fail. Very frustrating. But it can be removed once permission has been granted as long as you don't change the permissions. If the application is accessing an API that requires admin permission, this value should be admin_consent.
Alright, so once we shoot off this request we will be redirected to the login page. We login, accept the permissions, and then we should be redirected to here: http://123AzureApiTutorial.com/code?code=SOME_REALLY_LONG_STRING_OF_CHARACTERS. That string of characters is the code.
Getting the Access Token
Next, we take the code and use it to get the access_token. To do so, we need to make another request.
(POST) https://login.microsoftonline.com/<tenant-id>/oauth2/token
In addition to the url, we need to add parameters. These should be consistent with the content type application/x-www-form-urlencoded. This means they are submitted as form parameters. They are as follows:
client_id
This is again the client id (Application ID) we already have.
client_secret
This is the Application Key we generated earlier. I hope you saved it! If not, go back to that step and generate another one.
code
This is the value of the code we just received: SOME_REALLY_LONG_STRING_OF_CHARACTERS.
`grant_type
Because we're going the authorization code route, this value should be authorization_code
redirect_uri
This is the redirect uri we specified for the API Tutorial. The value from our example should be http://123AzureApiTutorial.com/code.
resource
This is very important. It is the resource API we want to access. For the Azure API, this value is https://management.azure.com.
Our response will be a json object with a variety of fields. Of these, the one we care about is access_token. Yay!
Client Credentials
This methodology skips getting the code (and thus needing the redirect_uri) at the expense of not getting an id_token.
The request is the same as in the Getting the Access Token section, with a few small differences.
We do not need to specify redirect_uri.
The value of grant_type should change to client_credentials.
Alright, we have an access_token! Now we're cooking!
Making the API request
With all the prep work thus far, this is the easiest part of the process.
The API URL we are requesting against is:
https://management.azure.com/subscriptions/<subscription-id>/providers/Microsoft.Compute/virtualMachines?api-version=2016-03-30
Add the following header to the request:
Authorization: Bearer <access-token>. Yes, the access_token value must be prefaced with the word "Bearer" in the header value.
"But wait!" You exclaim. "How do I get the subscription id?"
Excellent question! To find it through the portal, click Subscriptions -> -> Overview and copy the Subscription Id value.
Use that value, and run the request. You should see all the vms listed!
Common errors thus far
InvalidAuthenticationToken
When making the API request, you get an error response that says something like this: InvalidAuthenticationToken: The access token is invalid. This means you haven't added the API permissions to the API Tutorial. Go back to the Add the correct permissions step and double-check you have the right permissions. Also, when requesting the token make sure you use the prompt=consent, otherwise the you will not be prompted with the new permissions and the token will fail.
InvalidAuthenticationTokenTenant
Make sure that the tenant-id used when requesting the token belongs to the subscriber used when making the API call.
I am making a .NET application that will allow potential users to upload invoices they make on my app to their quickbooks account that they have set up and synchronized with their QBD version.
what steps will they/I need to take so that when they use my app, it will upload invoices to their account? Is it:
a) when they set up their account with the Intuit AppCenter, they will pick my app from the app center services (in doing so, it will generate a set of oAuth connection parameters for me to put into my app to push invoices to their account, if so how will I get these)?
b) Will I set up an account via the Intuit Platform Partners, and ask the user to use my account ID and Password by which they will "create new app" under my account for the purposes of generating oAuth connection information for my app to use?
c) Something different from the a) and b)?
Thanks!
The OAuth stuff is specifically designed to allow end-users to push a set of OAuth tokens/credentials from QuickBooks to your app. You then store the OAuth tokens, and use those to communicate with QuickBooks.
This is an accurate description of what happens, yes:
a) when they set up their account with the Intuit AppCenter, they will
pick my app from the app center services (in doing so, it will
generate a set of oAuth connection parameters for me to put into my
app to push invoices to their account, ...
You can read more about that process in the docs:
https://developer.intuit.com/docs/0025_quickbooksapi/0010_getting_started/0020_connect/0011_from_the_intuit_app_center
The other scenario is that, if you allow it, the user may choose to connect to QuickBooks from within your app. In this case, they'll click the "Connect to QuickBooks" button (see the docs here: https://developer.intuit.com/docs/0025_quickbooksapi/0010_getting_started/0020_connect/0010_from_within_your_app) which forwards them to Intuit's site, generates the OAuth tokens, and then sends them back to your site with the OAuth tokens.
In either case, the OAuth tokens will be sent to you so that you can store them and use them to communicate with the QuickBooks APIs.
This:
... and ask
the user to use my account ID and Password by which they will "create
new app" under my account for the purposes of generating oAuth
connection information for my app to use?
Is absolutely not what you want them to do. The whole point of OAuth is that you don't need to share any usernames/passwords of any time with any one.
Here are some additional answers to your other comments:
The part I am trying to figure out is what my customers will need to
do to be able to benefit from my app and its QuickBooks Integration
functionality.
They will need to log in to their Intuit.com account, and follow the prompts to connect their QuickBooks company to your app (i.e. follow the prompts to forward the OAuth credentials to your app).
Sounds like he/she will need to set up some account with ID and
password.
If they don't already have an Intuit.com account, they will be prompted to create one during the OAuth process. In the case of QuickBooks Online, they will ALWAYS already have an Intuit.com account (it's the same thing they use to log in to QuickBooks Online).
Now it seems to me that there are two ways that the customer can set
up accounts with Intuit.
They can set up an account through AppCenter, or within QuickBooks desktop directly (when you install the software, it prompts you) or if you're using any Intuit service already (e.g. QuickBooks Online) then you use your existing QuickBooks Online account.
Developer.Intuit.com accounts are only for developers (e.g. for YOU). Your end-users will not have a developer.intuit.com account. Nor do they need one.
a) what are the differences between the two?
Developer.Intuit.com accounts are only for developers. Your end-user will never see the developer.intuit.com website, nor will they see the option to "Create an App".
b) which should be used if the customer wants to set up syncing their
QBD? c)Which should be used if the customer wants to use my .net App.
d)Which should the customer use if they want both (b) and (c)? e) If
the customer creates an account the 2nd way, won't they get confused
by all the "developer" lingo?
None of those questions are applicable - end-users will never see or be prompted about any of the developer stuff. The only reason you see that is because you're registered as a developer. Normal end-users will not be, and thus won't have any of these options.
f)What if the customer has already synced their QBD with intuit?
Then they can just log in to their existing account to get connected (i.e. to send the OAuth credentials over to your app).
With regards to your questions:
A developer can choose to get his app listed on the appcenter or not depending on mkting requirements.
To understand the difference in the process- you can create a sample and then on developer.intuit.com, go to My Apps-> manage my app-> select your app. Then test connections.
You can use any of the above to use QBD data. Please see the additional info too that I have provided.
Also go through the link which consolibyte has mentioned.
If you are logging in from appcenter app, then since you are already logged in, then only OAUTH will be required.
In logging from within your app, you do not need to provide your user Id and password. The user needs to register for the first time with Intuit.com and then OAUTH process follows for him. These open id/email details can be saved in your db so that user need to enter them a next time.
The customer is never about the details of the developer except for the app which will access their company data.
The sync manager for QBD
OAUTH authorizes your app to connect to their company files.
Any of the above can be used to access QBD data.
The sync manager runs manually or automatically. So, customer can choose to sync/not sync his data.
---------------Addtional info----------
There are two ways to integrate with QuickBooks, the first is with REST APIs and the second is with the downloadable QBXML SDK v12. The REST APIs and that integration model is for applications that are to be sold to customers via our Appcenter. They are generally Saas applications.
The second model is the QBXML sdk which is available for anyone to use, especially for custom integrations.
The reference for the QBXML SDK is here
http://member.developer.intuit.com/qbSDK-current/Common/newOSR/index.html
Please go through:
https://developer.intuit.com/docs/0025_quickbooksapi/0055_devkits ->QBXML SDK
Lastly through the QBXML SDK you can add a custom field to any entity you want. See the reference above and take a look at DataExt Add Request. and once added Query Request. I believe this will give you what you are looking for.
Keep in mind this is for QuickBooks for Windows not QuickBooks Online.
You can go through FAQ page too-
https://developer.intuit.com/docs/0025_quickbooksapi/0058_faq
I am settled at this point from what Consolibyte has shared and what I have learned also. The whole picture (for me at least) is that both the Developer Account AND the App Center Account is needed. The Developer account provides the consumer key, consumer secret and application token that the .Net app will use to authenticate with Intuit. The App Center Account is needed to allow the user to sync their QuickBooks Desktop data to a location where the app center (and other 3rd party apps) can have access to it (after user authorizes of course), and to give it's user an ID and Password needed to authorize the .net app ( or any app for that matter) to access his/her Intuit data. It is the combination of the Developer Account's tokens and the App center Account's ID and password that allow the generation of an authentication/access token by which the .Net app can instantiate a Dataservices object so it can read/write to the user's instance of QB data in the App Center. Apart from the user creating an account in the App Center and synching their QB Desktop data with it, There should be little or nothing else to do except provide the ID and password, providing the .Net app is coded right. I am not talking about OpenID in this case, but do know that it uses the same ID and password.
...so I'm good. Thx.