What I'm trying to achieve is user authentication (account linking) before the default welcome intent is called.
(I'm aware that in the design documents it is recommended to not require users to authenticate, but it is required for our action)
We have an undeployed action that I have tested in the simulator. The conversation flow is
The user is prompted to signin ie, new SignIn()
They are prompted to create an new item
The item is saved and the user receives a success/fail response and continues
In DialogFlow I have a Default Welcome Intent (prompts the user to create an new item), an intent to capture the item.
Our fulfillment intents
app.intent('ask_for_sign_in_detail', (conv) => {
conv.ask(new SignIn());
});
app.intent('ask_for_sign_in_confirmation', (conv, params, signin) => {
if (signin.status !== 'OK') {
return conv.ask('You need to sign in before using the app.');
}
// const access = conv.user.access.token;
// exchange access.token for jwt from backend
return conv.ask('Great! Thanks for signing in.');
});
app.intent('Default Welcome Intent - fallback', (conv) => {
createItem(conv);
});
In DialogFlow under integrations for the Google Assistant I have required that Sign In is required before the Default Welcome Intent.
When I go to test in the simulator I can see that under account linking there is no information (ie, no accounts linked) and there is a message stating 'No account is linked to Google. Start a test conversation for account linking.'
Starting a conversation I get the following error:
Your voice wasn't recognized, so I can't connect you to Talkatoo.
Check the Voice Match settings in the Google Home app.
I am not sure where I have strayed trying to authenticate users with their Google accounts.
This has been fixed by Google as of 2019-07-19 02:57 PDT. Received a response from the AoG team, and tested it in the simulator. It was a Google bug.
Related
I am trying to implement stripe connect in my flutter app. Here are the steps I need to implement. Can anyone please navigate me on how I could achieve this in Flutter?
I am able to create a button with the endpointUrl but that's all..
Thanks
I found out this myself using firebase cloud functions:
first you create an https function in the firebase cloud function
then you add the link created by the function to your stripe dashboard
then you write the following logic to your function
obtain the the authorisation code
fetch data from stripe
save the response somewhere (in my case in realtime database)
Here is the function
exports.connectStripeStandardAccount = functions.https.onRequest((req, res) => {
let authCode = req.query.code;
return stripe.oauth.token({
grant_type: 'authorization_code',
code: authCode,
}).then(async response => {
await admin.database()
.ref(`/accounts/${authCode}`)
.set(response);
return res.send("Well done, account integration is completed. You can now close the window and go back to the app");
});
});
The answer selected is not completely correct:
If you dont assign the account_id to a user then it's of no use.
The only way to pass the user_id (fUser.uid) is to pass it using the state parameter.
exports.StripePI = functions.https.onRequest(async (req, res) => {
// console.log('accountIdq ' + req.query.error);
// console.log('accountIdq ' + req.query.state);
// return;
// if(!req.query.code)
// return res.send("An Error has occured please try again");
const response = await stripe.oauth.token({
grant_type: 'authorization_code',
code: req.query.code,
}).then(async response => {
var connected_account_id = response.stripe_user_id;
await admin.firestore().collection('Registration').doc(req.query.state)
.update({customer_id : connected_account_id});
return res.send("Well done, account integration is completed. You can now close the window and go back to the app");
});
});
If you want to create an in-app stripe connect account registration with flutter you will need these:
A server or service to complete the OAuth like Firebase Functions or Integromat (I used Integromat)
A link that will redirect to your app (I used Firebase Dynamic Link)
STEPS TO CREATE THE REGISTRATION FLOW
INTEGROMAT/FIREBASE FUNCTIONS SETUP
I decided to use Integromat instead of Firebase Functions because is easier to set up, doesn't need any code, and decreases my server load.
If you want to create it on Firebase Functions you will need to have a Blaze Plan
If you don't know it, Integromat will automate processes that you currently handle manually, via webhooks. It is not only capable of connecting apps (like GoogleCloud, Facebook, AWS...) but can also transfer and transform data.
Create a new scenario and add a Custom Webhook. Click on it and click on add, name it, and save it. It will now create a custom link to your webhook.
Close and click on the semi-sphere next to the webhook, to add the new module.
Select HTTP and Make a Request.
In the URL section insert https://connect.stripe.com/oauth/token.
Method POST.
Body Type Application/x-www-form-urlencoded.
Create now those fields :
Key client_secret - value your stripe client secret You can find it on your stripe dashboard. I advise you to first use the test mode and after that, change the value to the live key.
Key grant_type - value authorization_code
Key code - leave the value blank. We will add it later.
Save and close
For Firebase Functions you can create a new HTTPS function (I didn't test this)
var stripe = require("stripe")(*your stripe client secret*);
exports.connectStripeStandardAccount = functions.https.onRequest((req, res) =>{
let authCode = req.query.code;
return stripe.oauth.token({
grant_type: 'authorization_code',
code: authCode,
});
});
Remember to install stripe package npm install stripe
STRIPE SETUP
If you are in the test mode go to this link
If you are in the live mode go to this link
Go on the bottom and activate oAuth for standard accounts or for Express Account.
Click on Add URI and add the webhook link of Integromat that you created or the link related to your Firebase function.
If you used Firebase add this link https://us-central1-<project-id>.cloudfunctions.net/connectStripeStandardAccount
For Integromat you will need to create the structure. To do this click on Test OAuth, copy the link, and open it in incognito mode. Open your Integromat scenario and click on your webhook. Now click on Re-determine data structure.
Return to your stripe registration page and click on Ignore account form at the top.
Return on Integromat and select the HTTPS request, modify the field code, and insert the variable code (will open a dialog with all queries from the webhook). Confirm and save.
Now click on the play button and reopen the stripe registration link in incognito mode and click on Ignore account form. Return in Integromat and add a JSON module after the HTTPS request. In the JSON string insert the Data variable and save. Create a Webhook Response module after the JSON module.
In the status put 301, then click on Ok.
DEEP LINK SETUP
It's time to set up the redirect link that will return the user to our flutter app or on our website if the user hasn't it installed.
I used Firebase Dynamic Link You can follow this tutorial for set up.
Go to the dashboard and create a new Link prefix and a new dynamic link, remember to select to redirect your users to the right app.
Click on the three dots in your dynamic link row and click on Link Details. Copy the extended link.
Open Integromat and select the last module you created (Webhook Response). Click on Show advanced settings and on the Header add :
Key Location - value the extended dynamic link that you copied.
If you want your app to elaborate data from the stripe OAuth response you can modify the extended dynamic link by adding ? on the link parameter: link=https://test.page.link?stripe_user_id={{14.stripe_user_id}}
And select the variable parsed from the JSON module. Remember to click on the save icon to save your scenario.
On Firebase Functions you can do this when the function stripe.oauth.token finish (I didn't test it):
res.setHeader('Location', your dynamic link);
res.status(301).send();
Remember to deploy it.
FLUTTER APP SETUP
The code here is very simple. To initialize the connect account registration you only need to set up a button that will launch the stripe connect URL. You can use launch(url);
You can find that URL here. Remember to be logged in to your stripe account to get the right stripe client id. You can easily get it in the same section you added the webhook link in your stripe connect settings.
Delete &redirect_uri=https://sub2.example.com on the URL.
Now you can test your app and will see that when you complete your stripe connect registration/login you will be redirected to your app.
If you want to have an in-app web view you can use this package
To handle the response, you need to have installed the package firebase_dynamic_links
Set your Main widget Stateful and on the initState run the method getDynamic() :
void getDynamic() {
FirebaseDynamicLinks.instance.getInitialLink().then((value) {
if (value != null) {
_connect(value);
}
});
FirebaseDynamicLinks.instance.onLink(onSuccess: (value) async {
if (value != null) {
_connect(value);
}
}, onError: (error) async {
debugPrint('DynamicLinks onError $error');
});
}
void _connect(value) {
Uri deepLink = value.link;
print("Link :" + deepLink.path);
print("Query :" + deepLink.queryParameters.toString());
String stripeUserId = deepLink.queryParameters["stripe_user_id"];
}
You need to have both of them to handle dynamic links when your app is running and when it's closed.
We are creating an Action on Google using Dialogflow V2 API. We are using firebase cloud functions for our fulfillment and we are using an external rest api for our crud operations.
We have an undeployed action that I have tested in the simulator. The conversation flow is
The user is prompted to signin ie, new SignIn()
They are asked what they want to do and they respond
Their response is saved in our backend using their credentials
new SignIn() works in the simulator, but when I test on the Google Home Mini the SignIn() responds as if the user has rejected the propmpt to signin before they have time to respond
Is there some restriction regarding testing an Action that include Account Linking on the Google Home Mini?
The intent containing the new SignIn() is my Welcome intent, could that be causing the issue?
Here are the two intents handling SignIn(). Start Signin Intent is triggered by the Welcome Event
app.intent("Start Signin", (conv) => {
conv.ask(new SignIn());
});
app.intent("Get Signin", (conv, params, signin) => {
if (signin.status === "OK") {
const payload = conv.user.profile.payload;
conv.ask(`I got your account details, ${payload.name}. What do you want to do?`);
} else {
conv.ask(`I won't be able to save your data, please login`);
}
});
so I've beentrying to retrieve the email associated with the current user using the google home.
Documentation is kind of hard to find on the subject and from what I could gather, I should be able to use the SignIn class from actions-on-google. So here is my setup.
DialogFlow -> Created two intent, one to start the sign in process, the other to follow up on the process. (The second one has the event 'actions_intent_SIGN_IN' to it.)
Actions on google config : Account Linking.
Selected - Yes, allow users to sign up for new accounts via voice
Selected - Linking Type : Google Sign In
And added the client id to my fulfillment layer by adding the clientId to my dialogflow config.
dialogflow({clientId})
So, when I run this in the emulator (I get the same exact thing on my google home device) I get an error as soon as my sign in goes to the followup intent (actions_intent_SIGN_IN), which is that my signin.status is Error. From there, I don't know what I can do to get more information on what this error is and how to fix it.
Any idea ? Thanks !
PS : It might not even be something that can be done ? Is there any other way to retrieve the email of the user ? I was able to retrieve it's name using Permission, but there's nothing more that SignIn for email as far as I know.
I can show you how I get my email address with the Google Account linking:
You need to have your accessToken available then you could use what is shown in this answer. The accessToken is in conv.user.access.token when the SignIn is completed.
In node this looks like that:
let link = "https://www.googleapis.com/oauth2/v1/userinfo?access_token="+accessToken;
return new Promise(resolve => {
request(link,(error, response, body) => {
if (!error && response.statusCode === 200) {
let data = JSON.parse(body);
let name = data.given_name ? data.given_name : '';
conv.ask(new SimpleResponse({
speech: "Hello "+ name + "!",
text: "Hello "+ name + "!"
}));
resolve();
} else {
console.log("Error in request promise: "+error);
resolve();
}
})
})
Everything you need should be in the data object.
Hope it helps.
About your error, I am not sure but try doing all step in the link above. I'm using the Sign-In required box with Dialogflow and not a new SignIn but it should work the same.
I've implemented the implicit flow for Actions on Google account linking, and am using Dialogflow (previously API.AI) to define intents.
The full flow works in the device simulator (from AOG). The first intent gets a "It looks like your account isn't linked yet..." response, and the debug pane includes a URL to initiate linking:
https://assistant.google.com/services/auth/handoffs/auth/start?account_name=[account]#gmail.com&provider=[project_id]_dev&scopes=email&return_url=https://www.google.com/
If I follow this URI in a cache-less window:
I'm redirected to my app's authentication page
I choose to sign in with my Google account (same as [account] above)
I'm redirected to google.com with a success message in the URI bar
The simulator now accepts actions via my app and responds correctly
However, if I follow the same flow using a physical Google Home & the gH app for Android.
Device tells me account not yet linked
Open Google home and follow 'Link to [my app]' link
Browser opens to authentication page
Sign in as user
Redirected to a white page with a single link "Return to app", which has an href: about:invalid#zClosurez
Linking was unsuccessful, so additional attempts to run intents on the Google Home get the same "Account not yet linked" response.
I've inspected the intermediate access_token and state variables at length, and they all match and look to be correctly formatted:
Authentication URL (app sign in page): https://flowdash.co/auth/google?response_type=token&client_id=[client_id]&redirect_uri=https://oauth-redirect.googleusercontent.com/r/[project_id]&scope=email&state=[state]
After authenticating, redirected to (this is the white screen with 'return to app' broken link): https://oauth-redirect.googleusercontent.com/r/genzai-app#access_token=[token]&token_type=bearer&state=[state]
So, it seems there's something non-parallel about the way the simulator and physical devices work in terms of implicit flow account linking.
I've been struggling with this, and with the AOG support team for a very long time to no avail. Anyone else see a similar issue?
Updated with response redirect code:
Login handled by react-google-login component with profile & email scopes. On success we call:
finish_auth(id_token) {
let provider = {
uri: '/api/auth/google_auth',
params: ['client_id', 'redirect_uri', 'state', 'response_type'],
name: "Google Assistant"
}
if (provider) {
let data = {};
provider.params.forEach((p) => {
data[p] = this.props.location.query[p];
});
if (id_token) data.id_token = id_token;
api.post(provider.uri, data, (res) => {
if (res.redirect) window.location = res.redirect;
else if (res.error) toastr.error(res.error);
});
} else {
toastr.error("Provider not found");
}
}
provider.uri hits this API endpoint:
def google_auth(self):
client_id = self.request.get('client_id')
redirect_uri = self.request.get('redirect_uri')
state = self.request.get('state')
id_token = self.request.get('id_token')
redir_url = user = None
if client_id == DF_CLIENT_ID:
# Part of Google Home / API.AI auth flow
if redirect_uri == "https://oauth-redirect.googleusercontent.com/r/%s" % secrets.GOOGLE_PROJECT_ID:
if not user:
ok, _email, name = self.validate_google_id_token(id_token)
if ok:
user = User.GetByEmail(_email, create_if_missing=True, name=name)
if user:
access_token = user.aes_access_token(client_id=DF_CLIENT_ID)
redir_url = 'https://oauth-redirect.googleusercontent.com/r/%s#' % secrets.GOOGLE_PROJECT_ID
redir_url += urllib.urlencode({
'access_token': access_token,
'token_type': 'bearer',
'state': state
})
self.success = True
else:
self.message = "Malformed"
else:
self.message = "Malformed"
self.set_response({'redirect': redir_url}, debug=True)
I am able to make it work after a long time. We have to enable the webhook first and we can see how to enable the webhook in the dialog flow fulfillment docs If we are going to use Google Assistant, then we have to enable the Google Assistant Integration in the integrations first. Then follow the steps mentioned below for the Account Linking in actions on google:-
Go to google cloud console -> APIsand Services -> Credentials -> OAuth 2.0 client IDs -> Web client -> Note the client ID, client secret from there -> Download JSON - from json note down the project id, auth_uri, token_uri -> Authorised Redirect URIs -> White list our app's URL -> in this URL fixed part is https://oauth-redirect.googleusercontent.com/r/ and append the project id in the URL -> Save the changes
Actions on Google -> Account linking setup 1. Grant type = Authorisation code 2. Client info 1. Fill up client id,client secrtet, auth_uri, token_uri 2. Enter the auth uri as https://www.googleapis.com/auth and token_uri as https://www.googleapis.com/token 3. Save and run 4. It will show an error while running on the google assistant, but dont worry 5. Come back to the account linking section in the assistant settings and enter auth_uri as https://accounts.google.com/o/oauth2/auth and token_uri as https://accounts.google.com/o/oauth2/token 6. Put the scopes as https://www.googleapis.com/auth/userinfo.profile and https://www.googleapis.com/auth/userinfo.email and weare good to go. 7. Save the changes.
In the hosting server(heroku)logs, we can see the access token value and through access token, we can get the details regarding the email address.
Append the access token to this link "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" and we can get the required details in the resulting json page.
`accessToken = req.get("originalRequest").get("data").get("user").get("accessToken")
r = requests.get(link)
print("Email Id= " + r.json()["email"])
print("Name= " + r.json()["name"])`
Not sure which python middleware or modules you are using but
self.set_response({'redirect': redir_url}, debug=True)
seems to be setting parameters for a returning a response which isn't correct. Instead you should redirect your response to the redirect_url. For example importing the redirect module in Flask or Django like:
from flask import redirect or from django.shortcuts import redirect
then redirect like:
return redirect(redirect_url)
It appears Google has made a change that has partially solved this problem in that it is now possible to complete the implicit account linking flow outside of the simulator, in the way outlined in my question.
It seems the problem stemmed from an odd handling (on the AOG side) of the client-side redirect case used after sign in with the Google sign-in button.
From Jeff Craig in this thread:
The current workaround, where we provide the "Return to app" link
currently what we're able to provide. The issue is with the way that
redirecting to custom-scheme URIs is handled in Chrome, specifically,
with regard to the redirect happening in the context of a user action.
XHR will break that context, so what is happening is that you click
the Google Sign-In Button, which triggers an XHR to Google's servers,
and then you (most likely) do a client-side redirect back to the
redirect_url we supply, our handler executes, and isn't able to do a
JS redirect to the custom scheme URI of the app, because were outside
of the context of a direct user click.
This is more of a problem with the Implicit (response_type=token) flow
than with the authorization code (response_type=code) flow, and the
"Return to app" link is the best fallback case we currently have,
though we are always looking for better solutions here as well.
The current behavior shows the 'Return to app' link, but as of last week, this link's href is no longer about:invalid#zClosurez, but instead successfully completes the sign-in and linking process. It's an odd and confusing UX that I hope Google will improve in the future, but it was sufficient to get my app approved by the AOG team without any changes to my flow.
I have an inherited Ionic framework app that is using Ionic.io
The app authorises against our API, and is given an API token to use in future requests.
I'm trying to work on push notifications - I've set up ionic push, and can trigger push notifications out to all users with no problems.
I'd like the ability to target specific users / devices to send notifications, and I understand that to do this, I have to register the device to generate a token.
Within my $ionicPlatform.ready function, I have:
$ionicPush.register().then(function(t) {
return $ionicPush.saveToken(t);
}).then(function(t) {
console.log('Token saved:', t.token);
});
This however does not seem to be returning a token, and calling
console.log($ionicPush);
Shows that the token is not set.
Any ideas here? What am I missing?
So after digging though some documentation, I found that the issue was linked with the ionic user.
In the main run function, I fire off a
if ($ionicAuth.isAuthenticated()) {
If this fails, I try a login of a user, and a register of the user if appropriate. (user is already logged in using a custom auth token against our api)
Before I attempt to register the token, I then have to reload the user in order to have the app save off and push back up to ionic.io.
$ionicUser.load().then(function() {
$ionicPush.register().then(function(t) {
console.log('Token sent:', t.token);
return $ionicPush.saveToken(t);
}).then(function(t) {
console.log('Token saved:', t.token);
});
});