Serverless Watson Deployment? (Questions about integrations with other RESTful services) - rest

Introduction:
I feel like I'm missing something terribly obvious about how Watson Assistant should be designed at an architecture level, but maybe I'm not.
The specific problem I'm having is that I can't seem to get API calls for information back into the conversation.
The bigger issue is that I'm not sure I setting this all up correctly for the long-haul of what I'm trying to accomplish.
Purpose:
I am building a 24/7 customer-service Tier 1 helpdesk for our managed networks. A user of one of our networks should, via SMS, Web chat, Facebook messenger, and eventually phone call, be able to ask for:
Instructions on how to connect their specific device (PC, Mac, Chromebook, Xbox, Apple TV, etc.)
Ask for help troubleshooting if the instructions don't result in a successful connection. (Step by step instructions for deleting the saved network, restarting the wireless card, etc.)
Help creating a case - at which point the conversation becomes Watson asking for a bunch of information, like what time and date they first experienced the problem, any other times/dates they experienced the problem, their MAC address, etc. etc.
Problems:
I have most of the dialog built and working well. Getting information via Entities, saving to context variables, spitting them back out to make the conversation work, digressions, etc. all working.
I cannot, for the life of me, figure out what I am doing wrong when trying to GET information from an external API.
We have a 'daily password' for our guest networks, and we would like a user who asks for the daily password to receive it. This involves a very simple GET request to a publicly accessible server.
I have built a BlueMix/IBM Cloud function that works perfectly, but I can't seem to successfully call or receive information back from it.
Watson Error:
Error when updating output with output of dialog node id [node_66_xxxxxxxxxx]. Node output is [{"text":{"values":["Today's password for <? $guestNetwork.ssid ?> is <? $guestNetwork.password ?>"],"selection_policy":"sequential"}}] SpEL evaluation error: Expression [ $guestNetwork.ssid ] converted to [ context['guestNetwork'].ssid ] at position 0: EL1007E: Property or field 'ssid' cannot be found on null (and there is 1 more error in the log)
This error leads me to believe I am not properly defining the result variable in Watson, or improperly trying to retrieve it in conversation - because I do know my code returns the SSID and Password when I run it in BlueMix Console.
JSON for the action itself (and yes, I am setting the credentials in the previous node):
{
"output": {
"text": {
"values": [
"Today's password for <? $guestNetwork.ssid ?> is <? $guestNetwork.psk ?>"
],
"selection_policy": "sequential"
}
},
"actions": [
{
"name": "get-http-resource/getGuestNetworkPassword",
"type": "server",
"credentials": "$private.myCredentials",
"result_variable": "$guestNetwork"
}
]
}
Question:
Is my idea of a 'serverless' Watson possible by using Watson <-> IBM Cloud <-> external services? We don't currently have an 'application' or a server, it's all integrations between existing services.
Can anyone help me understand what I'm doing wrong when trying to access that variable?
Bonus points: How do I know to access the variable only after the action has completed successfully in IBM cloud? Basically, if accessing the information via the IBM Cloud function I wrote takes 1.5 seconds, do I need to pause the dialog for 1.5 seconds? Or am I completely missing the point for how to get external info in and out of Watson?
EDIT:
After watching Mitch's video, I have changed a couple things around, and the error message has evolved to this:
"Error when updating output with output of dialog node id
[node_66_1533646714776]. Node output is [{"text":{"values":["Today's
password for is "],"selection_policy":"sequential"}}] SpEL evaluation error:
Expression [ $guestNetwork['ssid'] ] converted to [
context['guestNetwork']['ssid'] ] at position 24: EL1012E: Cannot
index into a null value" error.

Without seeing your dialog, its a guess, but most common error I see is that you just need to jump to a child dialog node after doing the action call. You cannot do the action call and show the response in the same dialog node, as dialog needs a chance to run the action.
Its outlined in my video here:
https://ibm-dte.mybluemix.net/ibm-watson-assistant?refresh
see the video on dialog callouts. Its 13 minutes long I'm sure you only need about 2 of them, but still, should help.
What you are trying to do is definitely possible, especially if it works from within the cloud function environment.

We re-created the action using the default package (not having it in a sub-package), and it started working immediately.
Things to note: Watson dialog editor does not like dashes in the package name.
Thanks Mitch!

Related

Manually Setting a firebaseStorageDownloadTokens on image Upload to Firebase Storage with Flutter

I am trying to put a file to Firebase Storage with the following:
final metadata = firebase_storage.SettableMetadata(
customMetadata: {'firebaseStorageDownloadTokens': customToken});
_uploadTask =
_storage.ref().child(filePath).putFile(widget.file!, metadata);
I get the following error in my console:
E/StorageException( 6906): Caused by: java.io.IOException: { "error": { "code": 400, "message": "Not allowed to set custom metadata for firebaseStorageDownloadTokens" }}
Submitting this as both a bug & feature request on the plugin site, but in the meantime, in the off-chance I'm just writing this wrong, as documentation on this isn't thorough, I thought I'd submit here to see if anybody has done it successfully from the client.
I'll be writing a cloud function later if I can't do it from client, but since this isn't a high security thing I'm doing...I have many reasons to avoid forcing my app to get the downloadUrl....my aim is to create a predictable downloadUrl...readily done in the cloud, I know, just looking to do it from the client here.
After talking to some folks, this is not allowed from the client. It has to be done with a Cloud Function. It's highly discouraged. Here is why:
Anyone can access images if they have the image ID and the downloadToken. The idea is that both are hard to guess...hence an modicum of security. You can set security rules to getDownloadUrl for authenticated users, hence "restricting" the viewability through the obscurity of the image name, downloadToken and location.
By manually setting a downloadToken, you erode that "security."
It is allowed, however, you have to do it with a cloud function, triggering on create of the image, over-writing the auto-generated downloadToken. Send me a DM for help in that if you need it.

Using slot in Watson Assistant and Node RED

I'm building a chatbot on Telegram with Watson Assistant and Node RED. I need to take a date and time from the user for booking an appointment, so I used a slot that require the insertion of the two information. Using the trial chatbot offered by Watson, I have no problem with slot; but using Node RED, I can't go beyond entering the date. Through the debug, I saw that after entering the date, then after running the first slot, this error is returned "msg.payload.content is empty". Moreover, going to see the body of the output message returned by the assistant, the msg.payload.output.generic field is empty. On the other hand, it should contain the response of the assistant who requests, after having entered the date, also to insert the time. It seems that the bot is stuck on entering the date, but in reality I don't think so, because in the trial chatbot, it works perfectly.
What could be the problem?
Neither the Assistant V1 nor the V2 set or look at msg.payload.content. On input they look at msg.payload and on exit they assign the response from Watson Assistant to msg.payload.
If you are getting a "msg.payload.content" is empty error, then that will be happening somewhere in your flow. Most likely at the end where you are trying to process the response. If msg.payload.content is empty then the assistant dialog is not returning any output. This is strange as it should be returning the prompt for the currently empty slot.
What does msg.payload look like?
Are you using the V1 or the V2 node, and which version of node-red-node-watson nodes are you using? You can tell, by going to the palette.
Both V1 and V2 nodes, however, have been tested with slots, and the response does end up on msg.payload.content. Current released version of node-red-node-watson is 0.9.0.

Google Actions no execute

I have written a simple Java application based on Google example code:
https://github.com/actions-on-google/smart-home-nodejs
and watched a lot of movies, took Google codelabs, but I'm stuck.
My application responses to required actions (intents: action.devices.SYNC, action.devices.QUERY, action.devices.EXECUTE) - chcecked with Google validator, JSON validators. I have also implemeted simple OAuth authorization, created Google Actions projects for Smart Home.
After few hours of debugging I have got:
my test application is listed in Add devices menu,
I can select my application from list and connect it,
Google performs OAuth checking and everythings is OK (I see this in Nginx logs and my application logs),
then Google Actions sends action.devices.SYNC request to my server,
I responde with a test device:
{
"requestId":"14182994871341693427",
"payload":{
"agentUserId":"rysiek",
"devices":[
{
"id":"dev1",
"type":"action.devices.types.LIGHT",
"traits":[
"action.devices.traits.OnOff"
],
"name":{
"name":"główne"
},
"willReportState":false,
"roomHint":"pokój dzienny"
}
]
}
}
I have also tryied with willReportState: true,
then it (Google Assistant) asks me to assign devices to room which I'm doing.
After this nothing more is working.
I'm trying to say to my phone Turn on light główne (in polish "Włącz światło główne", "Włącz światło główne w pokoju dziennym" etc - tried a lot of sentences), but it just shows me web search results. No request is send to my server (action.devices.EXECUTE) - I have everything behind a Nginx proxy, and no requests are done to my server (not request is visible in logs).
I have no idea, what to do to make it work. Any help will be appreciated.
BTW1. Google Assistant is available in Poland from this year (16.01.2019) from few days and I'm wondering if this is my problem or Google problem? Is this even possible?
For example I'm saying Ok Google, synchronize my devices (in polish: Ok Google, synchronizuj moje urządzenia), and I get response in english:
"Sorry, I didn't get any response.".
But I see that there was a request for SYNC, and my server responed.
BTW2. I have connected Google Chromecast to Google Assistant, and it works without a problem - I'm mean I saying Turn on TV (in polish: Włącz telewizor) and TV is started like expected.
Checking the list of supported languages for smart home traits, Polish is currently not included. While it does seem like your SYNC response does work, you won't be able to interact with devices in Polish right now. While this will likely happen in the future, you'll only be able to use one of the languages mentioned above for the moment.
Once Polish support for smart home actions are supported, you shouldn't need to do any extra work for commands to succeed.

Actions on Google not passing parameters

I'm using Dialogflow (previously API.ai) to create an Actions on Google app.
Using Dialogflow I have set up a custom Entity to highlight single words from multiple Intents.
ie. mashable, recode, bbc sport are all words picked up as Entities.
The fulfilment sends a post webhook to an api I created.
When using Dialogflow it sends off
"parameters": {
"news-agent": "BBC Sport"
},
Which is fine, I set up API to detect the parameter and when using Actions on Google simulator
"parameters": {
"news-agent": ""
},
The parameter is blank, I don't see anything in the documentation about why this is happening.
Could someone help?
It's possible that Actions on Google is not picking up a value for the "news-agent" slot.
Try making the "news-agent" slot required in Dialogflow and define a re-prompt question for it.
That way, the event won't be sent to your app unless the "news-agent" slot is filled.

Automating creation of an Azure AD Web App Key

We've written a set of Powershell scripts to automate the process of creating new instances of our web app in our Azure environment. One part of these scripts uses the Graph API to create a few Azure AD objects, as well as corresponding objects in Auth0, which we use for single sign-on. So far, we haven't been successful in programmatically creating a new Key for the AAD Application. We've tried a number of methods, and while we can see that the resulting Application does have a Key in place (and the connection object in Auth0 has that key in its Client Secret field), we always receive this error when trying to authenticate:
By this point, we've already granted access to the app to single sign-on and to read directory data in AAD. (edit: We do this through manually following the provisioning_ticket_URL that is part of the object returned by the Auth0 API call to create the Connection object. This URL prompts us for Azure credentials and then displays this page:
)
This error persists until we manually create a new Key for the application through the Azure portal, save the application, and copy the newly created key into the Auth0 connection. Doing so always resolves the problem, but we'd like to avoid this extra step.
Within the body of the AAD application that we're creating in the API call, we have this section that defines the key:
"passwordCredentials":
[
{
"startDate": "2016-10-28T20:40:32Z",
"endDate": "2017-10-29T20:40:32Z",
"keyId": "(a GUID)",
"value": "(a Base64 string)"
}
],
As for those values, we've tried to generate them in quite a few different ways, and it accepts the values on the API PUT call, but they all still give us that same error when we try to log in. As an example, one way we tried was to plug in the values from $appKeyGUID and $appKeyValue from this:
$appKeyGUID = [guid]::NewGuid()
$guidBytes = [System.Text.Encoding]::UTF8.GetBytes($appKeyGUID)
$appKeyValue = [System.Convert]::ToBase64String($guidBytes);
into the keyID and value, respectively. I've read elsewhere that the value should be 44 characters long, though, which this one will not be.
But it seems like the value itself might not be the problem. I've tried generating a key through the Azure portal, using a Graph API GET call to retrieve the keyId, and then hardcoding those two exact values into the application body, but logging in still yields the same error.
Any idea where I'm going wrong?
EDIT: Per Philippe's suggestion, I tried changing only the Display Name of the AD application through the portal, and that did indeed resolve the problem. This led me to think that maybe something was wrong elsewhere within the application body that was being fixed when saving through the portal. I checked the manifest before and after doing that manual save, and there was indeed one small difference: within the RequiredResourceAccess section (of which I learned from here http://www.cloudidentity.com/blog/2015/09/01/azure-ad-permissions-summary-table/ and here https://www.microsoftpressstore.com/articles/article.aspx?p=2473127&seqNum=2), I had this:
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Role"
},
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Scope"
}
Instead of this, which the portal changes it to
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Role,Scope"
},
So I changed the body we're sending to match the second format. Unfortunately, we still are getting the same error with this change in place. Further, I verified that the manifest is now identical before and after making the save on the portal, as is the body returned by an API GET call on the application. There must be something else that the portal save is changing other than the application's definition.
After that, I tried using the Graph API to perform two PATCH calls to update the display name to something and then change it back, hoping that it would behave similarly to doing it through the portal and fix the problem. I verified through the portal that the PATCH calls were indeed changing the display name of the app. Sadly, it seems those edits didn't fix the issue, and I'm still getting the original error.
We create the application with a Graph API call like this:
$uri = "https://graph.windows.net/$waadTenant/applications?api-version=1.6"
$newADApp = (Invoke-RestMethod –Uri $uri –Headers $authHeader –Method POST -Body $newappbody –Verbose)
And here is the $newappbody that we end up using to define the application. I've left some things hard coded for troubleshooting purposes:
{
"odata.type": "Microsoft.DirectoryServices.Application",
"displayName": "customer1",
"homepage": "https://customer1.(our tenant).com",
"identifierUris":
[
"https://customer1.(our tenant).com"
],
"replyUrls":
[
"https://(our tenant).auth0.com/login/callback"
],
"passwordCredentials":
[
{
"startDate": "(a hardcoded date)",
"endDate": "(a hardcoded date)",
"keyId": "(hardcoded GUID that was previously generated by the portal and extracted through an API GET)",
"value": "(hardcoded Base64 like above)"
}
],
"requiredResourceAccess":
[
{
"resourceAppId": "00000002-0000-0000-c000-000000000000",
"resourceAccess":
[
{
"id": "5778995a-e1bf-45b8-affa-663a9f3f4d04",
"type": "Role,Scope"
},
{
"id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6",
"type": "Scope"
},
{
"id": "6234d376-f627-4f0f-90e0-dff25c5211a3",
"type": "Scope"
}
]
}
]
}
It seems like the ultimate issue here is related to Application Consent.
I will go into more details here, but feel free to quickly take a read through this document which describes our Consent Framework:
https://azure.microsoft.com/en-us/documentation/articles/active-directory-integrating-applications/
The stuff you are showing in your Application manifest are related to your application configuration, however, the consent record for your app will not be found there, thus I think your 'a/b' testing will not be fruitful.
I think what you are really finding here is that the Azure Portal has some "magic" in the background which will consent to your application if you are an administrator and you save the application. When you register an application from start to finish in the portal (again, as an admin), we record consent for you automatically once you save the app. This happens in the background, and the resulting 'objects' that get created are: A service principal for your app in your tenant, and consent links between you as the user and your service principal. The link that gets created is specifically an 'admin tenant-wide consent' object, which is the same thing you would get if you added "prompt=admin_consent" to your login url as a query string.
When you create an application using other methods, like the Graph API or AAD PowerShell, these objects are not created, which will result in the kind of error you are seeing when you try and authenticate.
The solution here is that you will need to have an interactive login experience one time with an Admin of the tenant before you are able to get other people to sign into the application. During this login experience, you must have the admin consent to your app and the permissions it requires. After this one time consent experience, subsequent calls to/from your application should not require consent, and you will not run into the issue you are having.
So in summary:
Automatically create an App using whatever method you want
Then use your application information, and create an interactive login with the administrator, using the query string "prompt=admin_consent"
Then use your app however you expect it to work
Thanks!
Shawn Tabrizi