Actions on Google Smart Home Skill: Room Hint / Home Graph - actions-on-google

The documentation at https://developers.google.com/actions/smarthome/create-app#actiondevicessync mentions that the roomHint field of the JSON response to the sync request can be used to have Google automatically assign devices to correct rooms.
However, no matter what I return in that field, the user still has to manually assign every device to a room and I cannot get Google to automatically recognize the correct room using this roomHint field
Here's an example response:
{
"requestId": "500166151965294748",
"payload": {
"devices": [
{
"id": "9",
"type": "action.devices.types.LIGHT",
"traits": [
"action.devices.traits.OnOff"
],
"name": {
"name": "Light"
},
"willReportState": false,
"roomHint": "Attic"
}
]
}
}

Right now supplying a value for the roomHint is not used by the HomeGraph to determine which room this device is in.

Related

Purchase event has been uploaded via Offline Conversions API, but the Event Set does not show any data

I followed the guide here: https://developers.facebook.com/docs/marketing-api/offline-conversions
Unlike "regular" Event Sets, which includes a "Test Events" tab in its dashboard, offline event sets don't seem to have this feature. You must either upload a CSV or call the API.
However, the offline event set shows no data coming from the API at all; the history tab only shows the CSV uploads, which were "last received 10 days ago". It doesn't even include the test upload I made today.
Is this a bug? How long should I wait for the data to appear in the events manager for my offline event?
Sample call
POST https://graph.facebook.com/v15.0/<offline_event_set_id>/events?access_token=<system_user_access_token>
{
"upload_tag": "store_data",
"data": [
{
"match_keys": {
"em": "<hashed>",
"ph": "<hashed>"
},
"currency": "PHP",
"value": 100,
"event_id": "test",
"event_name": "Purchase",
"event_time": "1669633380",
"custom_data": {
"event_source": "in_store"
},
"action_source": "physical_store",
"order_id": "test",
"data_processing_options": []
}
]
}
The response is as follows:
{"id":"<offline_event_set_id>","num_processed_entries":1}
Which seems to indicate that the event was uploaded successfully. But that event never shows up in the Overview tab of that offline event set.
Would appreciate any insights/guides elsewhere/answers, I've spent a few days on this with no success.
The "error": I was encoding the event_time as a string, whereas Facebook expects this value to be an integer. After updating my POST body to correct that, the events started showing up within minutes in the Overview tab.
{
"upload_tag": "store_data",
"data": [
{
"match_keys": {
"em": "<hashed>",
"ph": "<hashed>"
},
"currency": "PHP",
"value": 100,
"event_id": "test",
"event_name": "Purchase",
"event_time": 1669633380, // <-- The only change was removing the quotes
"custom_data": {
"event_source": "in_store"
},
"action_source": "physical_store",
"order_id": "test",
"data_processing_options": []
}
]
}
I really wish Facebook had returned some kind of error or warning, but at least I found the issue. Be careful with your data types, people!

Add files to Salesforce CMS channel folder via Connect API?

I'm developing an integration that will programmatically create product entries in Salesforce, and part of that process needs to be the addition of product images. I'm using the Connect API and am able to make a GET call to the right folder like this (I've scrambled the IDs and what not for this example):
https://example.salesforce.com/services/data/v52.0/connect/cms/delivery/channels/0591G0000000006/contents/query?folderId=9Pu1M000000fxUMSYI
That returns a payload like this:
{
"currentPageUrl": "/services/data/v52.0/connect/cms/delivery/channels/0ap1G0000000006/contents/query?page=0&pageSize=250",
"items": [
{
"contentKey": "MCZ2YVCGLNSBETNIG5P5QMIS4KNA",
"contentNodes": {
"source": {
"fileName": "PET Round.jpg",
"isExternal": false,
"mediaType": "Image",
"mimeType": "image/jpeg",
"nodeType": "MediaSource",
"referenceId": "05T0R000005MthL",
"resourceUrl": "/services/data/v52.0/connect/cms/delivery/channels/0ap1G0000000007/media/MCY2YVCGLNSBETNIG5P4QMIS4KNA/content",
"unauthenticatedUrl": "/cms/delivery/media/MCZ2YVCGLNSBETNIG5P4QMIS4KNA",
"url": "/cms/delivery/media/MCY2YVCGLNSBETNIG5P4QMIS4KNA"
},
"title": {
"nodeType": "NameField",
"value": "844333"
}
},
"contentUrlName": "844333",
"language": "en_US",
"managedContentId": "20T0R0000008U9qUAE",
"publishedDate": "2021-08-18T16:20:57.000Z",
"title": "844333",
"type": "cms_image",
"typeLabel": "Image",
"unauthenticatedUrl": "/cms/delivery/v52.0/0DB1G0000008tfOWAU/contents/20Y0R0000008y9qUAE?oid=00D0R000000OI7GUAW"
}
]
}
I am also able to retrieve images by contentKey with a GET call like this:
https://example.salesforce.com/services/data/v52.0/connect/cms/delivery/channels/0ap1G0000000007/media/MCZ2ZVCGLNSBETMIG5P4QMIS4KNA/content
Anyone know what the endpoint should look like and what parameters etc it should have? I'm having trouble finding anything for this specific scenario in the docs but surely there's a way.
Thanks!

How to use roomHint and structureHint with smarthome actions on Google

We are currently setting un a Smarthome action, and we would like to provide roomHint on the first sync (not on request sync) as it's really tedious to set up rooms on the first sync, but it does not work.
We tried to name rooms in english and also in italian, (as it's not really clear from the documentation if there is a list on room names that we can use?) but no way.
So can you please give us a hint how to use the roomHint field?
Also in the API doc we've found structureHint, does it work? The documentation for SYNC intent does not mention this field.
Here is our SYNC intent with one device and room, we took office from the example JSON:
{
"requestId": "3582198904737125163",
"payload": {
"agentUserId": "xyz#qwertyz.com",
"devices": [
{
"id": "deviceID",
"type": "action.devices.types.LIGHT",
"traits": [
"action.devices.traits.OnOff"
],
"name": {
"name": "Lampadina",
"defaultNames": [
"Lampadina_XYZ"
],
"nicknames": [
"Lampadina"
]
},
"willReportState": false,
"customData": {
"modelType": "DEVICE"
},
"roomHint": "office"
}
]
}
}
Thanks
Unfortunately, I believe the structureHint is only in the HomeGraph API sync response.
It cannot be used in the Sync intent.
If someone can tell me I'm wrong and how to use it, you'd be a hero.

Google Home dialogFlow V2 API mediaResponse not working

I decided to upgrade my Google Assistant action to use "dialogFlow V2 API" and my webhook returns an object like this
{
"fulfillmentText": "Testing",
"fulfillmentMessages": [
{
"text": {
"text": [
"fulfillmentMessages text attribute"
]
}
}
],
"payload": {
"google": {
"richResponse": {
"items": [
{
"mediaResponse": {
"mediaType": "AUDIO",
"mediaObjects": [
{
"name": "mediaResponse name",
"description": "mediaResponse description",
"largeImage": {
"url": "https://.../640x480.jpg"
},
"contentUrl": "https://.../20183832714.mp3"
}
]
},
"simpleResponse": {
"textToSpeech": "simpleResponse: testing",
"ssml": "simpleResponse: ssml",
"displayText": "simpleResponse displayText"
}
}
]
}
}
},
"source": "webhook-play-sample"
}
But I get an error message saying my action it is not available, is mediaResponse supported by V2?, should I format my object differently?, also, when I remove "mediaResponse" object works just fine and the assistant will speak the simpleResponse part.
This action was re-created this Mid March 2018 and I read about May deadline and that is why I decide to upgrade to V2, do you think I should go back to V1, I know I will have to delete it and re-created but that is fine. This is a link to the JSON object I see in the debug tab. Thanks once again
I set "API V2" in my action dialogFlow console, this is a screenshot of that setting
Here is an screenshoot of my action's integration -> Google Assistant
Thanks Allen, Yes I do have "expectUserResponse": false, I added the suggestion object you recommended but, unfortunately nothing changed, I am still getting this error
Simulator debug tag details
First of all - this is not a problem with Dialogflow V2. You also seem to be confusing the sunset of Actions on Google V1 with the release of Dialogflow V2 - they are two different creatures completely. If your project was using AoG V1, there would be a setting on the Actions integration screen, and thee isn't.
It is fine if you want to move to Dialogflow V2, but it isn't required. Media definitely works under Dialogflow V2.
The array of items must include a simpleResponse item first, before any of the other items in the RichResponse. (You also shouldn't include both ssml and textToSpeech - just one of them.) You also don't need the fulfillmentText and fulfillmentMessages components, since those are provided by the richResponse.
You also need to include suggestions chips unless you have set expectUserResponse to false. Somewhere in the simulator debug is probably a block that says
{
"name": "MalformedResponse",
"debugInfo": "expected_inputs[0].input_prompt.rich_initial_prompt: Suggestions must be provided if media_response is used..",
"subDebugEntryList": []
}
So something more like this should work:
{
"payload": {
"google": {
"richResponse": {
"items": [
{
"simpleResponse": {
"textToSpeech": "simpleResponse: testing",
"displayText": "simpleResponse displayText"
},
"mediaResponse": {
"mediaType": "AUDIO",
"mediaObjects": [
{
"name": "mediaResponse name",
"description": "mediaResponse description",
"largeImage": {
"url": "https://.../640x480.jpg"
},
"contentUrl": "https://.../20183832714.mp3"
}
]
}
}
]
"suggestions": [
{
"title": "This"
},
{
"title": "That"
}
]
}
}
},
"source": "webhook-play-sample"
}

Persistent menu not showing in Facebook Messenger chat bot

As I don't know why suggested, using Postman.
Per docs, have succesfully POSTed the configuration to facebook APIs:
which is not supposed to be anyways locale specific. Even I don't see here
Localization: Developers can now provide text in multiple languages (or entirely different menus) for each local your bot's users may come from.
Like my brother, I have tried almost everything so far
This looks like some crazy bug. Is there some work around to add a simplest persistent menu?
Wasted 2 hours on this issue. Until I realised you have to delete the conversation then refresh facebook with ignore cache (ctrl+shift+r in chrome) and then it will show.
The FB API document states that the API link to hit for applying persistent menu to the page specific bot is:
https://graph.facebook.com/v2.6/me/messenger_profile?access_token=<PAGE_ACCESS_TOKEN>
Notice the me after version number i.e v2.6 in this specific case. However, this did not worked for a lot of people
There is small change in the API link to hit:
graph.facebook.com/v2.6/Page ID/messenger_profile?access_token=PAGE ACCESS TOKEN
Notice that me is replaced with the fb Page Id.
And the sample payload can still be the same:
{
"get_started": {
"payload": "Get started"
},
"persistent_menu": [
{
"locale": "default",
"composer_input_disabled": false,
"call_to_actions": [
{
"title": "Stop notifications",
"type": "nested",
"call_to_actions": [
{
"title": "For 1 week",
"type": "postback",
"payload": "For_1_week"
},
{
"title": "For 1 month",
"type": "postback",
"payload": "For_1_month"
},
{
"title": "For 1 year",
"type": "postback",
"payload": "For_1_year"
}
]
},
{
"title": "fresh jobs",
"type": "postback",
"payload": "fresh jobs"
},
{
"title": "More",
"type": "nested",
"call_to_actions": [
{
"title": "like us",
"type": "web_url",
"url": "https://www.facebook.com/nordible/"
},
{
"title": "blog",
"type": "web_url",
"url": "http://xameeramir.github.io/"
}
]
}
]
}
]
}
Notice that it is mandatory to configure get_started button before setting up the persistent_menu.