Google Actions SDK Sign-In implicit flow - actions-on-google

EDIT: On phone assistant its working now problem just exist in google action simulator
I just try to setup Google Actions SDK account Linking with implicit grant and try to test it in Simulator.
First question is this even possible in Simulator?
To Do so I added at the action console account linking with the type
implicit grant to my action.
The url I used is working.
Now I added a signup request to my action. For testing so if I write signup in simulator the server response with:
{
conversationToken: JSON.stringify(state),
expectUserResponse: true,
expectedInputs: [
{
inputPrompt: {
initialPrompts: [
{
textToSpeech: "PLACEHOLDER_FOR_SIGN_IN"
}
],
noInputPrompts: []
},
possibleIntents: [
{
"intent": "actions.intent.SIGN_IN",
"inputValueData": {}
}
],
speechBiasingHints: []
}
]
}
After this the server didn't request the sign in page route (the address is correct!). It just responds with SignIN intent ERROR :
{
"isInSandbox'": false,
"surface": {
"capabilities": [
{
"name": "actions.capability.AUDIO_OUTPUT"
},
{
"name": "actions.capability.SCREEN_OUTPUT"
}
]
},
"inputs": [
{
"rawInputs": [
{
"query": "i think so",
"inputType": "VOICE"
}
],
"arguments": [
{
"name": "SIGN_IN",
'extension': {
"#type": "type.googleapis.com/google.actions.v2.SignInValue",
"status": "Error"
}
}
],
"intent': "actions.intent.SIGN_IN"
}
],
"device": {
"locale": "en-US"
},
"conversation": {
"conversationId": "1494606917128",
"type": "ACTIVE",
"conversationToken": "[\"_actions_on_google_\"]"
}
}
Why? Where is the problem? Can I see a error message somewhere?
Here is what happen in the simulator between 3 and 4:

Is it same when you use the phone app? For me it opens an embedded browser with my /auth endpoint, which the simulator doesn’t do.

I am able to make it WORKING after a long time.
We have to enable the webhook first and we can see how to enable the webhook in the dialog flow fulfillment docs
If we are going to use Google Assistant, then we have to enable the Google Assistant Integration in the integrations first.
Then follow the steps mentioned below for the Account Linking in actions on google:-
Go to google cloud console -> APIsand Services -> Credentials -> OAuth 2.0 client IDs -> Web client -> Note the client ID, client secret from there
-> Download JSON - from json note down the project id, auth_uri, token_uri
-> Authorised Redirect URIs -> White list our app's URL -> in this URL fixed part is https://oauth-redirect.googleusercontent.com/r/ and append the project id in the URL
-> Save the changes
Actions on Google -> Account linking setup
1. Grant type = Authorisation code
2. Client info
1. Fill up client id,client secrtet, auth_uri, token_uri
2. Enter the auth uri as https://www.googleapis.com/auth and token_uri as https://www.googleapis.com/token
3. Save and run
4. It will show an error while running on the google assistant, but dont worry
5. Come back to the account linking section in the assistant settings and enter auth_uri as https://accounts.google.com/o/oauth2/auth
and token_uri as https://accounts.google.com/o/oauth2/token
6. Put the scopes as https://www.googleapis.com/auth/userinfo.profile and https://www.googleapis.com/auth/userinfo.email
and weare good to go.
7. Save the changes.
In the hosting server logs, we can see the access token value and through access token, we can get the details regarding the email address.
Append the access token to this link "https://www.googleapis.com/oauth2/v1/userinfo?access_token=" and we can get the required details in the resulting json page.
accessToken = req.get("originalRequest").get("data").get("user").get("accessToken")
r = requests.get(link)
print("Email Id= " + r.json()["email"])
print("Name= " + r.json()["name"])
P.S. You can use the Grant Type as Implicit also instead of Authorisation code.

Related

Dialogflow - Firestore - Webhook call failed. Error: UNAVAILABLE, State: URL_UNREACHABLE, Reason: UNREACHABLE_5xx, HTTP status code: 500

I am trying for a fulfillment in Dialogflow using Firebase. There is data in Firestore and the intent is expected to retrieve the results based on the given parameter in the related intent. The intent is enables for webhook call. But the intent returns with one of the predefined responses.
I always get the error "message": "Webhook call failed. Error: UNAVAILABLE, State: URL_UNREACHABLE, Reason: UNREACHABLE_5xx, HTTP status code: 500."
Following is the Diagnostic Info from the test console
{
"responseId": "e27d24ba-cb14-4170-a7d8-a97314aee001-cad07fe1",
"queryResult": {
"queryText": "novaluron",
"parameters": {
"chemical": "novaluron"
},
"allRequiredParamsPresent": true,
"fulfillmentText": "This molecule is useful",
"fulfillmentMessages": [
{
"text": {
"text": [
"This molecule is useful"
]
}
}
],
"intent": {
"name": "projects/ppcagent-ahe9/agent/intents/6d9df198-9517-4d61-a480-87c158accdc5",
"displayName": "ChemicalDetails"
},
"intentDetectionConfidence": 0.3,
"diagnosticInfo": {
"webhook_latency_ms": 106
},
"languageCode": "en",
"sentimentAnalysisResult": {
"queryTextSentiment": {
"score": 0.3,
"magnitude": 0.3
}
}
},
"webhookStatus": {
"code": 14,
"message": "Webhook call failed. Error: UNAVAILABLE, State: URL_UNREACHABLE, Reason: UNREACHABLE_5xx, HTTP status code: 500."
}
}
Any help will be greatly appreciated.
This error could be due to different reasons, you can check these points:
Use of insecure connection.
You need to use HTTP requests. The service must use HTTPS and the URL must be publicly accessible in order for the fulfillment to function. DialogFlow does not support self-signed SSL certs. For information on SSL setup:
Permission errors.
Server files and folders need correct permissions and ownerships set in order to work properly. If Websites request files for which the server has no access then it will lead to an error.You can see more documentation about permission.
Bad scripting.
You need to check the code and check if there are any errors and modify them.

clientId missing from resource_access field in jwt token when using impersonation

I'm using Keycloak 14.0.0 and enabled the feature preview of token_exchange in order to do impersonation. After configuring my user in Keycloak to take on the impersonation role on the client "realm-management" (as according to the [documentation][1]), the actual request to do the token exchange fails as the token is not valid.
After some debugging it turns out that the jwt token is indeed malformed:
...
"session_state": "a03aeg0e-b5ce-4a50-9038-c339e50338c4",
"acr": "1",
"allowed-origins": [
"http://0.0.0.0:9180"
],
"scope": "openid identity_provider email admin profile company",
"permissions": [
"consented-readonly",
"readonly",
"trackingdisabled"
],
"resource_access": {
".roles": [
"impersonation"
]
},
"email_verified": false,
"idp": "myidp",
...
In the above, please notice the ".roles". I assume this is incorrect. It should be something like:
"resource_access": {
"myclient": {
"roles": [
"impersonation"
]
}
How can this be fixed?
[1]: https://www.keycloak.org/docs/latest/securing_apps/index.html#impersonation
It turns out that the configuration of a mapper was incorrect. In this case it was the "client roles" mapper (client scopes -> roles -> mapper -> client roles in keycloak ui) which, in my keycloak setup, had the value of:
resource_access..roles
This is incorrect as it should contain a clientId placeholder as shown below:
resource_access.${client_id}.roles
after this change the accessToken includes the actual client resulting in a valid json in the accessToken

"The agent returned an empty TTS" when action is not opened separately to asking an intent

When I invoke the skill with 'okay Google, ask {skillname} to {utterance}' I get the response 'The agent returned an empty tts" and the conversation closes. It doesn't even seem to hit my backend - I've tried hosting the backend code on both a local server and on AWS and the same issue happens in both cases. There's no log of the request being made either on Stackdriver on Google or on CloudWatch on AWS. The only response I get is the following in the debug tab:
{
"response": "The agent returned an empty TTS.",
"expectUserResponse": false,
"conversationToken": "EosDS2o4d0...",
"audioResponse": "",
"ssmlMarkList": [],
"debugInfo": {
"sharedDebugInfoList": [
{
"name": "Response Validation",
"debugInfo": "The agent returned an empty TTS.",
"subDebugEntryList": []
}
],
"conversationBuilderExecutionEventsList": []
},
"visualResponse": {
"visualElementsList": [
{
"displayText": {
"content": "The agent returned an empty TTS."
}
}
],
"suggestionsList": [],
"agentLogoUrl": ""
},
"clientError": 0,
"is3pResponse": true,
"clientOperationList": [],
"projectName": "",
"renderedHtml": ""
}
Nothing in any of the other tabs. When I try invoking the phrases in the same way on my phone, I just see a loading symbol and again nothing in the logs to say it's even hitting the backend.
Weirdly, this doesn't happen when I say 'okay Google, talk to {skillname} {utterance}' - this works absolutely fine.
All the intents work perfectly well when I've opened my action first, and then invoke them. I've built this using Jovo and published on Alexa also and the problem is only happening on Google.

Login if "Required User Actions" is existing

i have the following problem if an existing keycloak user has one or more required user action.
We use the keycloak (3.4.2.Final) rest api to login into keycloak. But keycloak returns always code 401 (invalid_grant) if user locked out or any required user action exists. So i can not identify what the main problem is.
For lockout i don't need to verify user's password, with any password (calling attackdetection rest api) can i tell the end user that the account is locked BUT for the required user actions not the same, user has to login successfully then can i handle next steps.
For example UPDATE_PASSWORD:
Login with username/password -> Login successfully OR username/password validated -> handle user action -> update password -> user logged in
I would appreciate any ideas.
Just in case someone still cares about an answer:
You can update the User using keycloak's admin REST api.
PUT /auth/admin/realms/myrealm/users/myKeycloakUserId
{
"id": "myKeycloakUserId",
"createdTimestamp": 1547730135139,
"username": "myusername",
"enabled": true,
"totp": false,
"emailVerified": true,
"firstName": "myfirstname",
"lastName": "mylastname",
"email": "myemail",
"attributes": {
"privacyStatementAcceptedDate": [
"2019-01-17T13:02:15.189Z"
],
"welcomeEmailDate": [
"2019-01-17T13:02:30.370Z"
],
"gender": [
"female"
],
"legalAgeConfirmedDate": [
"2019-01-17T13:02:15.189Z"
],
"termsAcceptedDate": [
"2019-01-17T13:02:15.189Z"
]
},
"disableableCredentialTypes": [
"password"
],
"requiredActions": [],
"notBefore": 0,
"access": {
"manageGroupMembership": true,
"view": true,
"mapRoles": true,
"impersonate": true,
"manage": true
}
}
Someone might want to try to send only the requiredActions-property instead of all other values (since these should be retrieved first), but I don't know if this would actually work. I thought: Better safe than sorry, so I retrieve the data first and then I only set the requiredActions-property to an empty array and send it back.
If you are using the Java client library (keycloak-admin-client), you might do it like this:
org.keycloak.admin.client.Keycloak keycloak = KeycloakBuilder.builder()
.serverUrl(KEYCLOAK_ADMIN_BASEURL))
.realm(KEYCLOAK_ADMIN_REALM))
.grantType(OAuth2Constants.CLIENT_CREDENTIALS)
.clientId(KEYCLOAK_ADMIN_CLIENTID))
.clientSecret(KEYCLOAK_ADMIN_CLIENTSECRET))
.build();
final UserResource keycloakUserResource = keycloak.realm(
configurationService.getValue(KEYCLOAK_ADMIN_REALM)).users().get(
keycloakUserId);
//load the users user-representation from Keycloak
final UserRepresentation userRepresentation = keycloakUserResource.toRepresentation();
if (userRepresentation == null) {
throw new Exception("Failed to load the UserRepresentation for the current User");
}
//adapt the user-representation
userRepresentation.setRequiredActions(Collections.emptyList());
//update user in Keycloak
keycloakUserResource.update(userRepresentation);

Paypal REST API: How to retrieve payment ID after user has approved the payment.

By following the guide on https://developer.paypal.com/webapps/developer/docs/integration/web/accept-paypal-payment/ , I have successfully created a payment and redirect the user to approve it.
The created payment is something look like bellow, and I save it in user's session for further reference.
{
"id": "PAY-6RV70583SB702805EKEYSZ6Y",
"create_time": "2013-03-01T22:34:35Z",
"update_time": "2013-03-01T22:34:36Z",
"state": "created",
"intent": "sale",
"payer": {
"payment_method": "paypal"
},
"transactions": [
{
"amount": {
"total": "7.47",
"currency": "USD",
"details": {
"subtotal": "7.47"
}
},
"description": "This is the payment transaction description."
}
],
"links": [
{
"href": "https://api.sandbox.paypal.com/v1/payments/payment/PAY-6RV70583SB702805EKEYSZ6Y",
"rel": "self",
"method": "GET"
},
{
"href": "https://www.sandbox.paypal.com/webscr?cmd=_express-checkout&token=EC-60U79048BN7719609",
"rel": "approval_url",
"method": "REDIRECT"
},
{
"href": "https://api.sandbox.paypal.com/v1/payments/payment/PAY-6RV70583SB702805EKEYSZ6Y/execute",
"rel": "execute",
"method": "POST"
}
]
}
After user approved the payment, Paypal will redirect the user to the return_url. For example, http://<return_url>?token=EC-60U79048BN7719609&PayerID=7E7MGXCWTTKK2.
In order to execute the payment, a POST request has to made to https://api.sandbox.paypal.com/v1/payments/payment/{payment_id}/execute/.
Question
The only two pieces of information provided from Paypal in the URL is token and PayerID. How can I find the corresponding payment_id?
Possible Solution
The token is part of the approval_url, parse the URL and store the token -> payment relationship can solve the problem. But I'm looking for a better solution that doesn't require parsing.
I think the paypal documentation isn't clear about this. But you can do something simple to resolve your problem passing de PaymentID through a parameter in your return url.
Like this:
return_url = 'http://www.yourdomain.com/paypal/success/?paymentID=PAY-1234567'
When the Paypal redirect to your site, then, it will return the paymentID together with the other parameters.
You would have to remember the Payment ID on your side (typically attached with your user session - shopping cart or order or as a session cookie) before redirecting the user to PayPal approval url. Once the is redirected back to your return Url along with the PayerID - you would need to extract the PaymentID from your user session and execute the Payment.
The Payment Id can be obtained in PHP by using the following method after the first API request has returned a successful response:
$payment->getId();
The online code sample (http://paypal.github.io/PayPal-PHP-SDK/sample/doc/payments/CreatePaymentUsingPayPal.html) shows how to send the request, however it does not include the getId() method.
To find this out I had to look in the downloaded SDK files at the file sample\payments\CreatePayment.php which has the following example code showing the use of this method:
ResultPrinter::printResult('Create Payment Using Credit Card', 'Payment', $payment->getId(), $request, $payment);
I found this link to be extremely helpful, in case anyone wants to check it out: https://github.com/paypal/PayPal-NET-SDK/issues/79
Since the v1/payments API has been deprecated for some time now, the best solution is to use the current v2/checkout/orders API for all new integrations. The order ID is returned in the URL.
However, redirects are an old integration method, for old websites. It is preferred to not use any redirects, at all, and keep your site loaded at ALL times. Instead use this approval flow: https://developer.paypal.com/demo/checkout/#/pattern/server
paymentid:
After you created the payment, in return json, you can get paymentid.
like this: "id":"PAY-01K00482KX842131HKORKVKY"
payerid:
you can use API:GET /v1/payments/payment/{paymentId} to get payer id after user approved the payment,and you will find payerid in return json,like this:
{
"id":"PAY-01K00482KX842131HKORKVKY",
"create_time":"2014-06-19T09:17:31Z",
"update_time":"2014-06-19T09:17:31Z",
"state":"created",
"intent":"sale",
"payer":{
"payment_method":"paypal",
"payer_info":{
"email":"buyer#samsung.com",
"first_name":"buyer",
"last_name":"samsung",
"payer_id":"H2DKRTTYWW8HS",
"shipping_address":{ "line1":"Lushan Road Num.188", "line2":"JianYe",
"city":"Tucson",
"state":"AZ",
"postal_code":"85715",
"country_code":"US",
"recipientName":"buyer samsung"}}},
"transactions":[{
"amount":{
"total":"12.00",
"currency":"USD",
"details"{"subtotal":"12.00"}},
"description":"creating a payment"}],
"links":[
{"href":"xxxxxxx","rel":"self","method":"GET"},
{"href":"xxxxxxx","rel":"approval_url","method":"REDIRECT"},
{"href":"xxxxxxx","rel":"execute","method":"POST"}]}