Our project was created after May 2017, therefore we believe it is using the v2 Google Assistant API, in accordance with what the documentation says here.
However, the 'Google-Assistant-API-Version' header in HTTP requests sent to our webhook contains the value 'v1'.
Google-Assistant-API-Version: v1
All the content is using camelCase, rather than snake_case, however, so it does appear to be using v2. As far as we can tell, we are also responding using the v2 format, and this works fine.
We also include the 'Google-Assistant-API-Version' header in our responses with the value 'v2'. For testing purposes we have both removed this header entirely and changed its value to 'v1'. In both cases, the skill continues to function normally.
I have also attempted to change my action package to include the 'fulfillmentApiVersion' field (as described here), but it doesn't appear to have any effect.
{
"actions": [{
"name": "actions.devices",
"deviceControl": {},
"fulfillment": {
"conversationName": "automation"
}
}
],
"conversations": {
"automation": {
"name": "automation",
"url": "https://**************************",
"fulfillmentApiVersion": 2
}
}
}
While we are not experiencing any problems at this time, we wonder if this is a sign of some dormant error, and were wondering if anyone could offer some clarification?
The Google Assistant API version may be incorrect, or it may be referring to the API version for Smart Home. The Smart Home APIs launched publicly around May and use a different protocol for the data returned from the webhook.
If you are building a Smart Home application, please refer to the documentation above on how to write your webhook responses.
Related
I follow the instructions on the official documentation for the trait channels, my sync JSON is very similar to the example. The attribute section is like this.
"availableChannels": [{
"key": "5f0e8de23cd9b23a5198ca6b",
"names": ["Fox", "KTVU"],
"number": "100"
}, {
"key": "5f0e8df03cd9b23a5198ca6c",
"names": ["ABC"],
"number": "101"
}, {
"key": "5f0e8dfa3cd9b23a5198ca6d",
"names": ["Univision"],
"number": "102"
}],
The sync JSON is valid, but the problem is when I said "Hey google change channel to FOX on TV", the google assistant response
"OK, playing American Broadcasting Company on The TV"
then error for
"Sorry, something went wrong and I am unable to control your device"
The problem is that I don't get any HTTP request on my webhook URL, so something on the google side is broken or maybe my sync is broken but I follow the documentation, this trait is "new" so maybe is something undocumented or is just broken. Any ideas?
Also, there is no error or entries on the Google Cloud Logs
The reason you are not getting any smart home request on your fulfillment endpoint can be due to not being able to deliver SYNC for this device properly. Please make sure to reissue your Sync response using Request Sync (or unlinking and relinking for testing), every time you make an update on your channel catalog.
The screenshot that has been posted shows a partial SYNC response. There is a high possibility that the code causing the error comes after the code presented in the screenshot. As already mentioned in the comments further analysis of the SYNC response needs to be done for pointing out the exact root cause.
If you’re still facing issues after that, please report to our public bug tracker (https://issuetracker.google.com/issues/new?component=655104&template=1284148).
Introduction:
I feel like I'm missing something terribly obvious about how Watson Assistant should be designed at an architecture level, but maybe I'm not.
The specific problem I'm having is that I can't seem to get API calls for information back into the conversation.
The bigger issue is that I'm not sure I setting this all up correctly for the long-haul of what I'm trying to accomplish.
Purpose:
I am building a 24/7 customer-service Tier 1 helpdesk for our managed networks. A user of one of our networks should, via SMS, Web chat, Facebook messenger, and eventually phone call, be able to ask for:
Instructions on how to connect their specific device (PC, Mac, Chromebook, Xbox, Apple TV, etc.)
Ask for help troubleshooting if the instructions don't result in a successful connection. (Step by step instructions for deleting the saved network, restarting the wireless card, etc.)
Help creating a case - at which point the conversation becomes Watson asking for a bunch of information, like what time and date they first experienced the problem, any other times/dates they experienced the problem, their MAC address, etc. etc.
Problems:
I have most of the dialog built and working well. Getting information via Entities, saving to context variables, spitting them back out to make the conversation work, digressions, etc. all working.
I cannot, for the life of me, figure out what I am doing wrong when trying to GET information from an external API.
We have a 'daily password' for our guest networks, and we would like a user who asks for the daily password to receive it. This involves a very simple GET request to a publicly accessible server.
I have built a BlueMix/IBM Cloud function that works perfectly, but I can't seem to successfully call or receive information back from it.
Watson Error:
Error when updating output with output of dialog node id [node_66_xxxxxxxxxx]. Node output is [{"text":{"values":["Today's password for <? $guestNetwork.ssid ?> is <? $guestNetwork.password ?>"],"selection_policy":"sequential"}}] SpEL evaluation error: Expression [ $guestNetwork.ssid ] converted to [ context['guestNetwork'].ssid ] at position 0: EL1007E: Property or field 'ssid' cannot be found on null (and there is 1 more error in the log)
This error leads me to believe I am not properly defining the result variable in Watson, or improperly trying to retrieve it in conversation - because I do know my code returns the SSID and Password when I run it in BlueMix Console.
JSON for the action itself (and yes, I am setting the credentials in the previous node):
{
"output": {
"text": {
"values": [
"Today's password for <? $guestNetwork.ssid ?> is <? $guestNetwork.psk ?>"
],
"selection_policy": "sequential"
}
},
"actions": [
{
"name": "get-http-resource/getGuestNetworkPassword",
"type": "server",
"credentials": "$private.myCredentials",
"result_variable": "$guestNetwork"
}
]
}
Question:
Is my idea of a 'serverless' Watson possible by using Watson <-> IBM Cloud <-> external services? We don't currently have an 'application' or a server, it's all integrations between existing services.
Can anyone help me understand what I'm doing wrong when trying to access that variable?
Bonus points: How do I know to access the variable only after the action has completed successfully in IBM cloud? Basically, if accessing the information via the IBM Cloud function I wrote takes 1.5 seconds, do I need to pause the dialog for 1.5 seconds? Or am I completely missing the point for how to get external info in and out of Watson?
EDIT:
After watching Mitch's video, I have changed a couple things around, and the error message has evolved to this:
"Error when updating output with output of dialog node id
[node_66_1533646714776]. Node output is [{"text":{"values":["Today's
password for is "],"selection_policy":"sequential"}}] SpEL evaluation error:
Expression [ $guestNetwork['ssid'] ] converted to [
context['guestNetwork']['ssid'] ] at position 24: EL1012E: Cannot
index into a null value" error.
Without seeing your dialog, its a guess, but most common error I see is that you just need to jump to a child dialog node after doing the action call. You cannot do the action call and show the response in the same dialog node, as dialog needs a chance to run the action.
Its outlined in my video here:
https://ibm-dte.mybluemix.net/ibm-watson-assistant?refresh
see the video on dialog callouts. Its 13 minutes long I'm sure you only need about 2 of them, but still, should help.
What you are trying to do is definitely possible, especially if it works from within the cloud function environment.
We re-created the action using the default package (not having it in a sub-package), and it started working immediately.
Things to note: Watson dialog editor does not like dashes in the package name.
Thanks Mitch!
I'm using Dialogflow (previously API.ai) to create an Actions on Google app.
Using Dialogflow I have set up a custom Entity to highlight single words from multiple Intents.
ie. mashable, recode, bbc sport are all words picked up as Entities.
The fulfilment sends a post webhook to an api I created.
When using Dialogflow it sends off
"parameters": {
"news-agent": "BBC Sport"
},
Which is fine, I set up API to detect the parameter and when using Actions on Google simulator
"parameters": {
"news-agent": ""
},
The parameter is blank, I don't see anything in the documentation about why this is happening.
Could someone help?
It's possible that Actions on Google is not picking up a value for the "news-agent" slot.
Try making the "news-agent" slot required in Dialogflow and define a re-prompt question for it.
That way, the event won't be sent to your app unless the "news-agent" slot is filled.
We've written a set of Powershell scripts to automate the process of creating new instances of our web app in our Azure environment. One part of these scripts uses the Graph API to create a few Azure AD objects, as well as corresponding objects in Auth0, which we use for single sign-on. So far, we haven't been successful in programmatically creating a new Key for the AAD Application. We've tried a number of methods, and while we can see that the resulting Application does have a Key in place (and the connection object in Auth0 has that key in its Client Secret field), we always receive this error when trying to authenticate:
By this point, we've already granted access to the app to single sign-on and to read directory data in AAD. (edit: We do this through manually following the provisioning_ticket_URL that is part of the object returned by the Auth0 API call to create the Connection object. This URL prompts us for Azure credentials and then displays this page:
)
This error persists until we manually create a new Key for the application through the Azure portal, save the application, and copy the newly created key into the Auth0 connection. Doing so always resolves the problem, but we'd like to avoid this extra step.
Within the body of the AAD application that we're creating in the API call, we have this section that defines the key:
"passwordCredentials":
[
{
"startDate": "2016-10-28T20:40:32Z",
"endDate": "2017-10-29T20:40:32Z",
"keyId": "(a GUID)",
"value": "(a Base64 string)"
}
],
As for those values, we've tried to generate them in quite a few different ways, and it accepts the values on the API PUT call, but they all still give us that same error when we try to log in. As an example, one way we tried was to plug in the values from $appKeyGUID and $appKeyValue from this:
$appKeyGUID = [guid]::NewGuid()
$guidBytes = [System.Text.Encoding]::UTF8.GetBytes($appKeyGUID)
$appKeyValue = [System.Convert]::ToBase64String($guidBytes);
into the keyID and value, respectively. I've read elsewhere that the value should be 44 characters long, though, which this one will not be.
But it seems like the value itself might not be the problem. I've tried generating a key through the Azure portal, using a Graph API GET call to retrieve the keyId, and then hardcoding those two exact values into the application body, but logging in still yields the same error.
Any idea where I'm going wrong?
EDIT: Per Philippe's suggestion, I tried changing only the Display Name of the AD application through the portal, and that did indeed resolve the problem. This led me to think that maybe something was wrong elsewhere within the application body that was being fixed when saving through the portal. I checked the manifest before and after doing that manual save, and there was indeed one small difference: within the RequiredResourceAccess section (of which I learned from here http://www.cloudidentity.com/blog/2015/09/01/azure-ad-permissions-summary-table/ and here https://www.microsoftpressstore.com/articles/article.aspx?p=2473127&seqNum=2), I had this:
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Role"
},
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Scope"
}
Instead of this, which the portal changes it to
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Role,Scope"
},
So I changed the body we're sending to match the second format. Unfortunately, we still are getting the same error with this change in place. Further, I verified that the manifest is now identical before and after making the save on the portal, as is the body returned by an API GET call on the application. There must be something else that the portal save is changing other than the application's definition.
After that, I tried using the Graph API to perform two PATCH calls to update the display name to something and then change it back, hoping that it would behave similarly to doing it through the portal and fix the problem. I verified through the portal that the PATCH calls were indeed changing the display name of the app. Sadly, it seems those edits didn't fix the issue, and I'm still getting the original error.
We create the application with a Graph API call like this:
$uri = "https://graph.windows.net/$waadTenant/applications?api-version=1.6"
$newADApp = (Invoke-RestMethod –Uri $uri –Headers $authHeader –Method POST -Body $newappbody –Verbose)
And here is the $newappbody that we end up using to define the application. I've left some things hard coded for troubleshooting purposes:
{
"odata.type": "Microsoft.DirectoryServices.Application",
"displayName": "customer1",
"homepage": "https://customer1.(our tenant).com",
"identifierUris":
[
"https://customer1.(our tenant).com"
],
"replyUrls":
[
"https://(our tenant).auth0.com/login/callback"
],
"passwordCredentials":
[
{
"startDate": "(a hardcoded date)",
"endDate": "(a hardcoded date)",
"keyId": "(hardcoded GUID that was previously generated by the portal and extracted through an API GET)",
"value": "(hardcoded Base64 like above)"
}
],
"requiredResourceAccess":
[
{
"resourceAppId": "00000002-0000-0000-c000-000000000000",
"resourceAccess":
[
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Role,Scope"
},
{
"id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
"type": "Scope"
},
{
"id": "6234d376-f627-4f0f-90e0-dff25c5211a3",
"type": "Scope"
}
]
}
]
}
It seems like the ultimate issue here is related to Application Consent.
I will go into more details here, but feel free to quickly take a read through this document which describes our Consent Framework:
https://azure.microsoft.com/en-us/documentation/articles/active-directory-integrating-applications/
The stuff you are showing in your Application manifest are related to your application configuration, however, the consent record for your app will not be found there, thus I think your 'a/b' testing will not be fruitful.
I think what you are really finding here is that the Azure Portal has some "magic" in the background which will consent to your application if you are an administrator and you save the application. When you register an application from start to finish in the portal (again, as an admin), we record consent for you automatically once you save the app. This happens in the background, and the resulting 'objects' that get created are: A service principal for your app in your tenant, and consent links between you as the user and your service principal. The link that gets created is specifically an 'admin tenant-wide consent' object, which is the same thing you would get if you added "prompt=admin_consent" to your login url as a query string.
When you create an application using other methods, like the Graph API or AAD PowerShell, these objects are not created, which will result in the kind of error you are seeing when you try and authenticate.
The solution here is that you will need to have an interactive login experience one time with an Admin of the tenant before you are able to get other people to sign into the application. During this login experience, you must have the admin consent to your app and the permissions it requires. After this one time consent experience, subsequent calls to/from your application should not require consent, and you will not run into the issue you are having.
So in summary:
Automatically create an App using whatever method you want
Then use your application information, and create an interactive login with the administrator, using the query string "prompt=admin_consent"
Then use your app however you expect it to work
Thanks!
Shawn Tabrizi
I've been using Parse REST API for push notifications to targeted users (using "where"), it works well . I am now trying to add a custom data field so the devices can handle a tapped notification (i.e. redirect to specific object id etc). From what I have read, I should add the custom properties in the data json node, at the same level as alert, like the documented example below. But every time I make the request I get a 400 Bad Request. But, when I use the Parse.com web console with json: { "alert":"Notification Message","CustomObjectId":"12345" } ... it works - I see the CustomObjectId in the push notification (on the parse.com push tab).
Why is my REST API request failing, but the seemingly same request on the Parse.com website console is working? What am I missing?
My Parse REST Request:
{"where":{"UserId":"MyUserId"},"data":{"alert":"Notification Message","CustomObjectId":"12345"}}
Example from documentation (note this is using channels, not a where though) https://parse.com/docs/rest/guide/#push-notifications :
{
"channels": [
"Indians"
],
"data": {
"action": "com.example.UPDATE_STATUS",
"alert": "Ricky Vaughn was injured during the game last night!",
"name": "Vaughn",
"newsItem": "Man bites dog"
}
}
Thank you for the help,
Tim
For anyone else running into this, make sure you have "REST push enabled?" set to true in your Parse settings on the website. It's defaulted to false. Parse seems to allow 5 or so requests through per day with "REST push enabled" set to false. So after you wire this up, you may think you've successfully integrated with Parse, but then you start getting 400 Bad Request errors. This would lead many to think the issue is with their code possibly malforming requests or missing required parameters, not an authorization level issue like "REST push enabled" is not allowed.