I am studying the smarthome program belongs to Actions on Google(Official Overview)
Figure 1
In the figure, google said that it need oauth2 authentication, but I have a question about what's the role of "Assistant Client App" in this oauth2 authentication procedure, is "Assistant Client App" the Third-party application so called in the OAuth2 model or other role?
Another question is that if the "Assistant Client App" is the Third-party application, does it mean that I have to build a Oauth server so the "Assistant Client App" can come and ask for the access grant?
Thank You for help!!!
The Assistant Client app is the application on one's phone for the Google Assistant. In the settings one is able to set up a link to a smart home integration.
Related
We are using SignIn with Google on our Flutter App that uses AWS cognito. Recently it started giving error:
"Error 403: access_denied The developer hasn’t given you access to this app. It’s currently being tested and it hasn’t been verified by Google. If you think you should have access, contact the developer"
The app was in "Testing" and seems to have reached the limit of 100 users. In order to have more than 100 users use the SignIn with Google option, we need to put the Google Authentication App to "Production". In order to do that Google is asking for domain verification of the amazoncognito.com used in "Authorized redirect URIs". Since we don't own this domain we can't verify it.
Can someone please help with how can we get the verification of the GCP "oauth consent screen" done with other domains like amazoncognito.com?
Thanks in advance.
I don't think this is currently possible.
Your best bet would be to file a feature request with Google for a supported verification progress for third-party user identity services.
I would suggest using the Google Workspace add-ons component for this type of request.
I'm trying to make a chatbot with Dialogflow for Google Home. It requires the user to input a URL. Now it will definitely be a long and complicated URL which I can't recreate and I can't have the user speak into the google home.
The idea I had was that the user would input the URL on an agent on messenger. I store this on a Firebase database and then access it with a second agent.
Now the issue I have is authentication, I was hoping to use account linking on my google action with facebook. But I can't login to Facebook with google home. Or if I can, I can't find any documentation specific to that case. Facebook doesn't provide the necessary client ID and secret(as far as I can see).
I managed amazon and Gmail account linking with Alexa and an Amazon Echo. In those cases, you would have to login to google or amazon on the Alexa app or webpage. Then this will be integrated with your Echo and the skill will become usable.
Anyone have an idea of how I can make the link happen, if not then anyone have an idea as to how I can solve the overall problem?
This question has been left unanswered on other forums, but I was hoping to either get it solved or find an alternative.
There are three approaches to solving your overall problem - getting the URL manually entered and available to your Action. Two of them tackle it the way you've suggested - involving authenticating to Facebook and tying that to the Assistant account somehow. One solves it entirely inside the Assistant.
Account linking to the Facebook account
You've tagged firebase-authentication, so I'm going to assume that you're using it to do the auth and you've enabled Facebook login through it. This means your user has a "Firebase Account", but they log into that account using Facebook.
I will assume you have a way to get the URL from messenger once they're logged in.
The trick in this case is to setup Account Linking between their Firebase account and their Assistant account. This is done by setting up an OAuth2 server that has access to the Firebase accounts and will create authorization and refresh tokens that are given to the Assistant.
In the Action, you'll send the user to the Sign In helper, which will redirect them to your login page and send back a one-time auth code to the Assistant. The assistant will then use your OAuth2 server to exchange this code for auth and refresh tokens. Periodically it will use the refresh token to get new auth tokens.
When the user returns to the conversation through the Assistant, you'll be handed an auth token and you can use this to lookup the user. Since you also know their Facebook account, you can get the URL via however you planned to do so.
There are drawbacks to this method - it is very complicated, and setting up your own OAuth2 server is not for the faint of heart. You may be able to use something like Auth0 instead of Firebase Authentication to accomplish the same thing, but then you don't have the ease of access to the Firebase database.
Account linking to both Facebook and Google
In your Firebase account, however, you don't need to limit them to just logging into Facebook. You can have them use Firebase to record both the Facebook and Google accounts that they're using. This would "link" the two accounts together in your system.
With this, you don't need to setup an OAuth2 server. Instead, you can have the Assistant use Google Sign In for authentication. If the Google Cloud Project that Firebase is using and the Assistant are using are the same project, then once the user has logged in to your project's web page with their Google account, you'll get an identity token on the Assistant which will contain their Google ID. You can use this to match up with their Firebase account and get the Facebook ID and proceed from there.
But this is still a lot of work and kinda messy, jumping between systems.
Using just the Google Assistant (and maybe a web page)
If you're willing to make some assumptions about the devices your users are using, then you may be able to do it all just using the Assistant. The Assistant doesn't just run on the Google Home and other smart speakers, it also works on most current Android and iOS devices.
So you can detect if they have such a device available and, if they do and they're not currently on it, direct them to switch to that device when you need the URL.
If they don't have such a device available (perhaps because their version of Android is older), and you think this may be a common scenario, you may need to make another entry source available. This could be one of the solutions above, or you may want to just have a simple web page (done via Firebase Hosting and Firebase Functions, perhaps) where they log in using their Google account (so you get their ID) and you let them enter the URL. If you just need a URL - going through Dialogflow may be more complexity than you need.
I have been making Google Home apps, but now I want to make apps that are connected to Firebase server and DialogFlow API. How is it possible to integrate user authentication in an ios app to identify that user who is using Google Home with that login state?
For example, if I make a restaurant ordering app, I want any user to make a request through Google Home and the user can check his/her ordering history on an ios app.
A user logs in through an app and then how does google home detect that user as a logged in user? I have been searching how to implement this, but nothing hits.
You will need to implement account linking for Actions on Google. This will require two server-side components that you'll need to build:
You'll need to create an OAuth server that will allow the user to authenticate to your service (and use Firebase auth as a component of this) and authorize the Google Assistant to access your service. It does this by generating tokens that it hands to the Assistant.
Your Dialogflow fulfillment webhook will get this token passed to it as part of the conversation. It uses this token to determine which user is making the request and confirm it is valid.
There are OAuth services that are available that work with Google's auth that you should be able to fit in here, or you can write your own using Firebase Hosting and Firebase Cloud Functions. The Dialogflow webhook can, of course, be done using Firebase Cloud Functions.
I can already connect my conversation to my Facebook page and the bot is working.
My Problem is that i couldn't find a single tutorial on how to make calls to a server that is owned by a company and has information about accounts and payments, for example when i type to the Chat-bot that i want to see a list of accounts, the chat-bot would reply with a list from that API; Or i can login to that server with my account by typing the account's name and password in the chat-bot. I just wanted to know if anyone knows how to do that or if its even possible or knows a tutorial that explains it. I also would prefer if the tutorial was detailed because i'm new to this field.
As #Felipe Paixao said, you need to develop an orchestrating application.
I've found one example that explains step-by-step how to use Watson Conversation with Facebook Messenger.
Requirements:
Facebook page
IBM Bluemix
A Facebook Developer Account
Source code.
Step - IBM Bluemix:
Create a SDK for Node.js application
Create a Watson Conversation service
Create your Workspace for Watson Conversation and build your Dialog
Configure the JS app and integrate it with the Watson Service Connections -> Connect Existent:
In your App created in IBM Bluemix, access the Tools and go to IDE Eclipse, and create a new file: app.js with this code.
Replace with your credentials the username and password
You can see your credentials inside Service Credentials, or going to your workspace and access:
Step - Facebook Developers:
Enter to facebook developers with your Facebook account
In the Dashboard select Create a New App and Select Messenger
Put the URL from your Bluemix (Now IBM Cloud) app and select the options marked: messages, message_deliveries, messaging_options e messaging_postbacks.
Now, return to our app to configure the app.js file with our new FB token autogenerated and deploy it.
Step Final
Test your chatbot: If you made everything OK, we will have your chatbot available on your Facebook page.
Obs.: For built your example, you can add the lines of code for call your API in the app.js using the http module, using the Advanced JSON in Watson Conversation and create one "action": "callApi", and recognize with your code for do something, like this example.
See the Official Reference that I used to build Facebook Messenger with Node.js SDK (Back-end) source code based on Priscila Parodi codes.
You need to develop an orchestrating application, that would be responsible to connect the IBM Watson Conversation with your desired Fronted, Facebook in this case, and the Backend systems. I imagine that you are using the automatic deploy to Facebook from the Conversation Tooling, this option would not resolve your need to access a Backend.
The image passed by #data_henrik shows exactly that.
In conversation, you can create context variables that will be used as triggers in the Orchestrator to let him know when you need to call a Backend service to provide an information and then write the response into the Conversation context.
About using the Chatbot to login into a system, passing a password, this isn't a safe option since all information typed would go to Watson Conversation log and any person with access to the tooling/api could see the passwords.
I'm creating the flow in my application to sign up new users who come from the Intuit App Store. In a presentation (and other places in the documentation) it talks about getting OpenID information after the user completes the Try It process (simulated with Test subscribe to app from Intuit App Center (OpenID)).
When I do "Test subscribe to app from Intuit App Center (OpenID)" I don't get any OpenID info. And after going throught he Direct Connect javascript/oauth process I only get back OAuth credentials, no OpenID.
How are we suppossed to connect the OpenID to the account we create using the OAuth info gathered from the "Test subscribe to app from Intuit App Center (OpenID)" and Direct Connect process?
The IPP documentation for this part of the process needs major help.
Ok, figured it out:
You need to do the whole OpenID process on the page that is hit from Intuit Sign In, Try It, or Test subscribe to app from Intuit App Center (OpenID) links.
From the documentation is sounds like OpenID info is passed to you from the process on the Intuit side before it hits your app. But this is not true. You have to go explicitly do OpenID after your app is hit.
Intuit: Please improve your documentation of this, it's not clear at all.