I have created a conversational action in Actions Builder that uses account linking via OAuth (not Google sign in). Functionally, everything is working well.
The only issue I noticed is that when I try to link accounts using anything except a phone in the simulator, I get a response telling me to link accounts using the Google Home app. Since my action is a conversational action and not a smart home action, my action will never be found in the Google Home app.
I have confirmed that this kind of account linking is supported with conversational actions (via docs as well as Actions on Google support). What support could not answer for me is what I can do about this, they directed me here for help.
So my question is, how is account linking handled for conversational actions on non-phone devices (e.g. smart speaker)? Does it simply not run at all, or do I have to do something to remove these Google Home prompts?
Thanks
If the Action is running on a device type that does not support a web browser then the authentication step is not going to work. If you have not signed in before, the operation will fail. Though if you have signed in already, any device will be able to retrieve your access token.
The way to do this would be to handle cases for a non-authenticated device that cannot be authenticated. This may mean providing some part of your Action that can run in these conditions. Alternately, you'll need to inform people that they'll first need to use their phone to perform account linking.
Related
So I'm building this app for a client (It's a basic scheduling app like HootSuite but simpler) and he wants the ability to add multiple Facebook accounts with an IAP (aka In-App Purchase) after said IAP is purchased. The IAP is all done and works great but I cannot figure out how to link multiple Facebook accounts. I tried to use the same login method that a previous developer used (It's an ongoing freelance project and 2 other devs worked on it before I joined) to no avail since the login process stops once the app realizes that the user has already signed in...
So I guess that my question is this: Is there a way to link multiple Facebook accounts at once in a single iOS app? If so, how exactly? Forget the IAPs if that's too confusing, everything is fine there anyways. The goal is to be as seamless as possible, so ideally there would be no need for the user to log out and log back into the other account (Something I've seen a lot in other Questions/Answers).
I don't think I am authorized to show the code to you guys (that would also require sharing the entire project, it's a real spider-web!), but could someone please lead me in the right direction?
Please let me know if you have any questions!
Thank you in advance.
Notes:
Just like the title says, we use Firebase for our storage, Cloud Firestore for our database and Facebook login (through Firebase) for login since the purpose of the app is to schedule posts on IG.
I use a basic variable called "activeUsers" to track the number of accounts "linked" to the app so we can know whether or not to show the "PurchaseModel" and, obviously, receipt verification and all that for IAPs. Thought it could be useful!
3. When the app starts, you are prompted to link a Facebook account to enable the scheduler. THIS WORKS GREAT! What I am trying to achieve is to add other accounts AFTER the first one has been "enabled"/logged into.
It's not possible to link multiple accounts from the same provider to a single Firebase Auth account.
You can only link multiple accounts of different types (Facebook, Google, GitHub, etc) to a single Firebase account as described in the documentation.
I'm trying to make a chatbot with Dialogflow for Google Home. It requires the user to input a URL. Now it will definitely be a long and complicated URL which I can't recreate and I can't have the user speak into the google home.
The idea I had was that the user would input the URL on an agent on messenger. I store this on a Firebase database and then access it with a second agent.
Now the issue I have is authentication, I was hoping to use account linking on my google action with facebook. But I can't login to Facebook with google home. Or if I can, I can't find any documentation specific to that case. Facebook doesn't provide the necessary client ID and secret(as far as I can see).
I managed amazon and Gmail account linking with Alexa and an Amazon Echo. In those cases, you would have to login to google or amazon on the Alexa app or webpage. Then this will be integrated with your Echo and the skill will become usable.
Anyone have an idea of how I can make the link happen, if not then anyone have an idea as to how I can solve the overall problem?
This question has been left unanswered on other forums, but I was hoping to either get it solved or find an alternative.
There are three approaches to solving your overall problem - getting the URL manually entered and available to your Action. Two of them tackle it the way you've suggested - involving authenticating to Facebook and tying that to the Assistant account somehow. One solves it entirely inside the Assistant.
Account linking to the Facebook account
You've tagged firebase-authentication, so I'm going to assume that you're using it to do the auth and you've enabled Facebook login through it. This means your user has a "Firebase Account", but they log into that account using Facebook.
I will assume you have a way to get the URL from messenger once they're logged in.
The trick in this case is to setup Account Linking between their Firebase account and their Assistant account. This is done by setting up an OAuth2 server that has access to the Firebase accounts and will create authorization and refresh tokens that are given to the Assistant.
In the Action, you'll send the user to the Sign In helper, which will redirect them to your login page and send back a one-time auth code to the Assistant. The assistant will then use your OAuth2 server to exchange this code for auth and refresh tokens. Periodically it will use the refresh token to get new auth tokens.
When the user returns to the conversation through the Assistant, you'll be handed an auth token and you can use this to lookup the user. Since you also know their Facebook account, you can get the URL via however you planned to do so.
There are drawbacks to this method - it is very complicated, and setting up your own OAuth2 server is not for the faint of heart. You may be able to use something like Auth0 instead of Firebase Authentication to accomplish the same thing, but then you don't have the ease of access to the Firebase database.
Account linking to both Facebook and Google
In your Firebase account, however, you don't need to limit them to just logging into Facebook. You can have them use Firebase to record both the Facebook and Google accounts that they're using. This would "link" the two accounts together in your system.
With this, you don't need to setup an OAuth2 server. Instead, you can have the Assistant use Google Sign In for authentication. If the Google Cloud Project that Firebase is using and the Assistant are using are the same project, then once the user has logged in to your project's web page with their Google account, you'll get an identity token on the Assistant which will contain their Google ID. You can use this to match up with their Firebase account and get the Facebook ID and proceed from there.
But this is still a lot of work and kinda messy, jumping between systems.
Using just the Google Assistant (and maybe a web page)
If you're willing to make some assumptions about the devices your users are using, then you may be able to do it all just using the Assistant. The Assistant doesn't just run on the Google Home and other smart speakers, it also works on most current Android and iOS devices.
So you can detect if they have such a device available and, if they do and they're not currently on it, direct them to switch to that device when you need the URL.
If they don't have such a device available (perhaps because their version of Android is older), and you think this may be a common scenario, you may need to make another entry source available. This could be one of the solutions above, or you may want to just have a simple web page (done via Firebase Hosting and Firebase Functions, perhaps) where they log in using their Google account (so you get their ID) and you let them enter the URL. If you just need a URL - going through Dialogflow may be more complexity than you need.
I created a DialogFlow app that works perfectly in my developer account.
But I need to test it as another user, so in my Google Action simulator, I added another test account as a OWNER to the project.
I verified this user has OWNER status in both the DialogFlow and Google Action UI.
They are able to edit both projects with full access.
However, when the test account tries to use the simulator it says:
Sorry, this action is not available for your app. Sorry, this action
is not available for your app. In the simulator, you are only able to
invoke actions that you have added to your app. For more information
on how to invoke your app, click here
https://developers.google.com/actions/discovery/explicit
This makes no sense to me:
I verified the test user has OWNER/ADMIN status on both DialogFlow and Google Actions for the project.
In DialogFlow, I verified the Google Assist is pointing to Default Welcome Intent as the default intent.
In Google Actions, I verified that the Default Welcome Intent is visible on the BUILD > ACTIONS UI.
As others suggested, I tried renaming the app several times with no luck.
I tried publishing an Alpha version, with still no luck.
My gut tells me it is some sort of permission/visibility problem, but I am completely stuck. Any ideas would be of great help.
EDIT:
THIS ISSUE HAS BEEN SOLVED. SEE COMMENT BELOW.
"I revisited my activity controls and noticed my "Web and App Activity" had a special clause due to organization permissions. I completely missed it due to the confusing UI. Posted a picture in the main thread. Thanks for reminding me to double check that area."
After you asked your company admin to enable this feature, it should be same as the image in below.
This error message is quite misleading. I have encountered this error a couple of times, and generally the problem was with one of the privacy settings in my Google account.
In this case you should go to https://myaccount.google.com/activitycontrols, confirm that you are logged in using your test account, and make sure the following are enabled:
Web & App Activity
Device Information
Voice & Audio Activity
I have created an agent in DialogFlow that interacts with the user giving her the ability to check for details like demographic from a back-end database. I have achieved this using Fulfillment/Webhook calls and have successfully parsed/returned different types of Responses to the user to play with. I have tested this on both an Android device and the Google Actions Simulator.
Now, what I want to achieve is to give user the ability to "Set a Reminder" on a platform like Google Calendar or similar through my App. By default, Google Assistant does this by simply telling it to "remind me..." and Google creates a reminder in the calendar. But as soon as I invoke my app, the Assistant is out of scope so using the "remind me" command won't work. I want to use the Google Calendar for example to create a reminder through my agent so that the user is notified at the due time.
I understand that when the user invokes an App in Google Assistant, she is using her Google credentials so I guess if I want to use the App to create a reminder, I should have the option. I have had a look at the Calendar API but I am not sure if I should proceed that way. Is there a better way of achieving this?
Please note, I want to use a reminder service that is available to the user without subscribing to a third-party service.
The Calendar API is the way to go. Details here.
In those docs you'll see reference to the access token, which you can get by using the helper intent askForSignIn (example of use here in one of Google's example projects).
Hope this helps!
So for all those who are facing the same issue, here is what I did:
First created an account on https://auth0.com
Followed this guide to configure the necessary settings to link to Google Assistant (ignored the Jovo part)
Used Auth0 guide to configure and link to my Google project credentials (ignored the Admins SDK Service part)
Now when I run my Google Actions App on the device, a button "Link [App Name] to Google" showed up. Clicking on that "signed me in".
Parsed the response in my webhook to grab originalRequest.data.user.accessToken
This is the token mentioned in #lukedavis answer. The next step was to use the token to execute a call to the Calendar API. Upon trying that, I am facing a different problem which I am trying to find the solution for. But the above should assist those who wish get the accessToken.
I'm having a lot of trouble testing my google action's account linking. It uses the authorization oauth2 flow for account linking. I do not have transactions enabled but instead have required sign-in on the welcome intent and both of my other (custom) intents.
Immediately after implementing account linking, I tested the flow via the simulator and it worked perfectly. I then wanted to link my google account to a different account on my server, which means I needed to unlink and then link again. This is where things get fuuuuunky.
It looks like the docs are a bit outdated, but I eventually found a way to unlink, at least partially. The note in the blue box at the bottom of this page notes that you can unlink by disabling testing. The UI looks different from the screenshot, but I disabled testing within the Action Simulator UI by toggling 'Testing on Device'. I then re-enabled it and indeed my requests were no longer getting access tokens!
However, at this point, the linking flow was no longer being invoked when I ran commands via the simulator. I waited for a few hours and tried again - no dice. My dialogflow webhook simply isn't getting an access_token in the request body (property no longer exists), which is good, but I'm never prompted to re-link the account.
I tried the same using the Google Assistant app on a phone registered to my google account (same Google account being used in the simulator). Again, it worked the first time but after I toggled testing on my device, it started to fail permanently. So it seems device-specific? I did try an incognito chrome session as well as deleting cookies, neither of which worked.
Any ideas how I can properly unlink and then re-link my account during testing of a google action?
Thanks!