I want to create a smart home app Google Home using Actions SDK. As it is now, I have a cloud server and its OAuth 2.0 API and some real IOT devices, all the cloud environment is prepared. When I 've browsed the document of Smart Home, I feel confused, I edit a json file and upload it to my google project with gaction, and isn't done? If it is, How to handle the response json of SNYC,QUERY and EXECUTE? Thanks a lot.
In the Actions on Google console for your project, there should be a webhook field. You put the URL that the HomeGraph will call. In your webhook, you'll receive a JSON payload that contains the intent and other parameters for you to handle.
let reqdata = request.body;
let input = reqdata.inputs[0];
let intent = input.intent;
switch (intent) {
case "action.devices.SYNC":
console.log('post /ha SYNC');
// Do sync
break;
case "action.devices.QUERY":
console.log('post /ha QUERY');
// Do query
break;
case "action.devices.EXECUTE":
console.log('post /ha EXECUTE');
// Do execute
break;
default:
response.status(401).set({
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Headers': 'Content-Type, Authorization'
}).json({error: "missing intent"});
break;
}
You should return a JSON payload as a response.
The actual way to adjust your IoT device is entirely dependent on your server and device implementation.
You can check out the sample project to figure out a bit more.
Related
I have a Flutter app where I am running a Google Apps Script through an http request. The purpose of the script is to create a Form and link the responses to a spreadsheetID that is passed in. The script is configured to only allow Google accounts access it and I've set up the flutter app to use a Service Account to access the script using the format:
getCredentials().then( (AuthClient client){
response = client.get(url, headers{"Authorization": "Bearer ${client.access_token}");
});
Issue: The issue is that the first time that the Service Account makes a request it will get an HTML response saying that it the account needs to give permission to the script to access its data and I'm not sure how to do that.
I'm fairly new to making http requests and using it with the GoogleAPI so I'm stuck. Any advice?
Goal
Create a web page which anyone can use to submit a Google sheet link and for the app to create a form and link the sheet to that.
Authorization
For this users will require a google account and they will be required to go through the OAuth process to authorize your app.
To create the form and link it from client-side JavaScript you would indeed need to call the Apps Script API, though you cannot do this with a service account.
From: https://developers.google.com/apps-script/api/how-tos/execute
Warning: The Apps Script API doesn't work with service accounts.
Luckily, you don't need a service account to do this.
Instructions
Create an Apps Script project with a function something like:
function createForm(ssID){
form = FormApp.create("Your New Form");
form.setDestination(FormApp.DestinationType.SPREADSHEET, ssID);
let formLink = form.getPublishedUrl();
return formLink;
}
Save and take a note of the ID of the script project.
Set up a GCP project (sounds like you already have one).
Make sure the Apps Script API is enabled in your GCP.
Configure the OAuth consent screen and add the scope - https://www.googleapis.com/auth/forms.
Create an API key and a Client ID - add http://localhost:8000 or whatever port you are testing on to the "Authorized JavaScript Origins"
Create OAuth credentials "web browser (JavaScript)".
Link your Apps Script project to the same GCP project - Instructions
Deploy the Apps Script project as an API executable - take not of the deployment ID, although the documentation says that you need the script ID, it is wrong, at least with the new Apps Script IDE.
Write the client-side JavaScript in your app like what is found in the quickstart. Which will enable users to authorize the app. You need to add in the scopes and keys there too. I recommend just following the quick start steps first to get a feel for it. You can use the authorization parts without modification.
Then add in the function that will call your Apps Script, something like this:
function appsScriptCreateForm(ssId) {
var scriptId = "<DEPLOYMENT_ID>";
// Run your Apps Script function
gapi.client.script.scripts
.run({
scriptId: scriptId,
resource: {
function: "createForm",
parameters: [ssId],
},
})
.then(function (resp) {
var result = resp.result;
// ERROR HANDLING
if (result.error && result.error.status) {
appendPre("Error calling API:");
appendPre(JSON.stringify(result, null, 2));
} else if (result.error) {
var error = result.error.details[0];
appendPre("Script error message: " + error.errorMessage);
if (error.scriptStackTraceElements) {
appendPre("Script error stacktrace:");
for (var i = 0; i < error.scriptStackTraceElements.length; i++) {
var trace = error.scriptStackTraceElements[i];
appendPre("\t" + trace.function + ":" + trace.lineNumber);
}
}
// IF SUCCESSFUL
} else {
console.log("success", resp);
}
});
}
Write your HTML with the buttons and inputs necessary.
Add event listeners where appropriate.
Profit!
Please note
This set up is your project running with the authorization of other accounts.
The API requests count against your quota.
You can see details of all the executions in your GCP Project Dashboard.
Users require a Google account and need to authorize the app.
In the Apps Script function above, you just need to pass in the Spreadsheet ID. Not the whole link. You could ask for the whole link and then use Regex to extract the ID if you wanted.
This can be quite tricky and easy to miss a step or make a mistake, so double check your work.
If, after successful authorization, when trying to run the script you get a 404 error, the request has been built wrong, check your IDs. If you get a 500 error, that can mean that the Apps Script function has successfully been called, but, there was an error within Apps Script and failed, check the executions page of the Apps Script editor.
References
Apps Script How to Execute
Apps Script JS Quickstart - Highly recommended you follow these steps first and get that working!
How to link your Apps Script to GCP
Understand that we can use document.referrer in JavaScript to detect where a visitor came from on the web browser.
Wondering if there's any mechanism using JavaScript to detect if the visitor arrived at the website by clicking on the link of a mobile app (e.g. Facebook)?
Mobile apps won’t send referrers. Smartphone apps are unable to pass “referring page” information to your site analytics. So direct traffic stats may be inflated.
Your best bet is to add a querystring in the links and then check that.
Ie.
https://www.yoursite.com?ref=facebook
index.html
<script>
const urlParams = new URLSearchParams(window.location.search);
const ref = urlParams.get('ref');
switch(ref) {
case "facebook":
// Count Facebook
break;
case "snapchat":
// Count Snapchat
break;
default:
// Count misc
break;
}
</script>
I am trying to implement stripe connect in my flutter app. Here are the steps I need to implement. Can anyone please navigate me on how I could achieve this in Flutter?
I am able to create a button with the endpointUrl but that's all..
Thanks
I found out this myself using firebase cloud functions:
first you create an https function in the firebase cloud function
then you add the link created by the function to your stripe dashboard
then you write the following logic to your function
obtain the the authorisation code
fetch data from stripe
save the response somewhere (in my case in realtime database)
Here is the function
exports.connectStripeStandardAccount = functions.https.onRequest((req, res) => {
let authCode = req.query.code;
return stripe.oauth.token({
grant_type: 'authorization_code',
code: authCode,
}).then(async response => {
await admin.database()
.ref(`/accounts/${authCode}`)
.set(response);
return res.send("Well done, account integration is completed. You can now close the window and go back to the app");
});
});
The answer selected is not completely correct:
If you dont assign the account_id to a user then it's of no use.
The only way to pass the user_id (fUser.uid) is to pass it using the state parameter.
exports.StripePI = functions.https.onRequest(async (req, res) => {
// console.log('accountIdq ' + req.query.error);
// console.log('accountIdq ' + req.query.state);
// return;
// if(!req.query.code)
// return res.send("An Error has occured please try again");
const response = await stripe.oauth.token({
grant_type: 'authorization_code',
code: req.query.code,
}).then(async response => {
var connected_account_id = response.stripe_user_id;
await admin.firestore().collection('Registration').doc(req.query.state)
.update({customer_id : connected_account_id});
return res.send("Well done, account integration is completed. You can now close the window and go back to the app");
});
});
If you want to create an in-app stripe connect account registration with flutter you will need these:
A server or service to complete the OAuth like Firebase Functions or Integromat (I used Integromat)
A link that will redirect to your app (I used Firebase Dynamic Link)
STEPS TO CREATE THE REGISTRATION FLOW
INTEGROMAT/FIREBASE FUNCTIONS SETUP
I decided to use Integromat instead of Firebase Functions because is easier to set up, doesn't need any code, and decreases my server load.
If you want to create it on Firebase Functions you will need to have a Blaze Plan
If you don't know it, Integromat will automate processes that you currently handle manually, via webhooks. It is not only capable of connecting apps (like GoogleCloud, Facebook, AWS...) but can also transfer and transform data.
Create a new scenario and add a Custom Webhook. Click on it and click on add, name it, and save it. It will now create a custom link to your webhook.
Close and click on the semi-sphere next to the webhook, to add the new module.
Select HTTP and Make a Request.
In the URL section insert https://connect.stripe.com/oauth/token.
Method POST.
Body Type Application/x-www-form-urlencoded.
Create now those fields :
Key client_secret - value your stripe client secret You can find it on your stripe dashboard. I advise you to first use the test mode and after that, change the value to the live key.
Key grant_type - value authorization_code
Key code - leave the value blank. We will add it later.
Save and close
For Firebase Functions you can create a new HTTPS function (I didn't test this)
var stripe = require("stripe")(*your stripe client secret*);
exports.connectStripeStandardAccount = functions.https.onRequest((req, res) =>{
let authCode = req.query.code;
return stripe.oauth.token({
grant_type: 'authorization_code',
code: authCode,
});
});
Remember to install stripe package npm install stripe
STRIPE SETUP
If you are in the test mode go to this link
If you are in the live mode go to this link
Go on the bottom and activate oAuth for standard accounts or for Express Account.
Click on Add URI and add the webhook link of Integromat that you created or the link related to your Firebase function.
If you used Firebase add this link https://us-central1-<project-id>.cloudfunctions.net/connectStripeStandardAccount
For Integromat you will need to create the structure. To do this click on Test OAuth, copy the link, and open it in incognito mode. Open your Integromat scenario and click on your webhook. Now click on Re-determine data structure.
Return to your stripe registration page and click on Ignore account form at the top.
Return on Integromat and select the HTTPS request, modify the field code, and insert the variable code (will open a dialog with all queries from the webhook). Confirm and save.
Now click on the play button and reopen the stripe registration link in incognito mode and click on Ignore account form. Return in Integromat and add a JSON module after the HTTPS request. In the JSON string insert the Data variable and save. Create a Webhook Response module after the JSON module.
In the status put 301, then click on Ok.
DEEP LINK SETUP
It's time to set up the redirect link that will return the user to our flutter app or on our website if the user hasn't it installed.
I used Firebase Dynamic Link You can follow this tutorial for set up.
Go to the dashboard and create a new Link prefix and a new dynamic link, remember to select to redirect your users to the right app.
Click on the three dots in your dynamic link row and click on Link Details. Copy the extended link.
Open Integromat and select the last module you created (Webhook Response). Click on Show advanced settings and on the Header add :
Key Location - value the extended dynamic link that you copied.
If you want your app to elaborate data from the stripe OAuth response you can modify the extended dynamic link by adding ? on the link parameter: link=https://test.page.link?stripe_user_id={{14.stripe_user_id}}
And select the variable parsed from the JSON module. Remember to click on the save icon to save your scenario.
On Firebase Functions you can do this when the function stripe.oauth.token finish (I didn't test it):
res.setHeader('Location', your dynamic link);
res.status(301).send();
Remember to deploy it.
FLUTTER APP SETUP
The code here is very simple. To initialize the connect account registration you only need to set up a button that will launch the stripe connect URL. You can use launch(url);
You can find that URL here. Remember to be logged in to your stripe account to get the right stripe client id. You can easily get it in the same section you added the webhook link in your stripe connect settings.
Delete &redirect_uri=https://sub2.example.com on the URL.
Now you can test your app and will see that when you complete your stripe connect registration/login you will be redirected to your app.
If you want to have an in-app web view you can use this package
To handle the response, you need to have installed the package firebase_dynamic_links
Set your Main widget Stateful and on the initState run the method getDynamic() :
void getDynamic() {
FirebaseDynamicLinks.instance.getInitialLink().then((value) {
if (value != null) {
_connect(value);
}
});
FirebaseDynamicLinks.instance.onLink(onSuccess: (value) async {
if (value != null) {
_connect(value);
}
}, onError: (error) async {
debugPrint('DynamicLinks onError $error');
});
}
void _connect(value) {
Uri deepLink = value.link;
print("Link :" + deepLink.path);
print("Query :" + deepLink.queryParameters.toString());
String stripeUserId = deepLink.queryParameters["stripe_user_id"];
}
You need to have both of them to handle dynamic links when your app is running and when it's closed.
so I've beentrying to retrieve the email associated with the current user using the google home.
Documentation is kind of hard to find on the subject and from what I could gather, I should be able to use the SignIn class from actions-on-google. So here is my setup.
DialogFlow -> Created two intent, one to start the sign in process, the other to follow up on the process. (The second one has the event 'actions_intent_SIGN_IN' to it.)
Actions on google config : Account Linking.
Selected - Yes, allow users to sign up for new accounts via voice
Selected - Linking Type : Google Sign In
And added the client id to my fulfillment layer by adding the clientId to my dialogflow config.
dialogflow({clientId})
So, when I run this in the emulator (I get the same exact thing on my google home device) I get an error as soon as my sign in goes to the followup intent (actions_intent_SIGN_IN), which is that my signin.status is Error. From there, I don't know what I can do to get more information on what this error is and how to fix it.
Any idea ? Thanks !
PS : It might not even be something that can be done ? Is there any other way to retrieve the email of the user ? I was able to retrieve it's name using Permission, but there's nothing more that SignIn for email as far as I know.
I can show you how I get my email address with the Google Account linking:
You need to have your accessToken available then you could use what is shown in this answer. The accessToken is in conv.user.access.token when the SignIn is completed.
In node this looks like that:
let link = "https://www.googleapis.com/oauth2/v1/userinfo?access_token="+accessToken;
return new Promise(resolve => {
request(link,(error, response, body) => {
if (!error && response.statusCode === 200) {
let data = JSON.parse(body);
let name = data.given_name ? data.given_name : '';
conv.ask(new SimpleResponse({
speech: "Hello "+ name + "!",
text: "Hello "+ name + "!"
}));
resolve();
} else {
console.log("Error in request promise: "+error);
resolve();
}
})
})
Everything you need should be in the data object.
Hope it helps.
About your error, I am not sure but try doing all step in the link above. I'm using the Sign-In required box with Dialogflow and not a new SignIn but it should work the same.
We have a chatbot for our website today, that is not build using Google technology. The bot has a JSON REST API where you can send the question to and which replies with the corresponding answers. So all the intents and entities are being resolved by the existing chatbot.
What is the best way to wrap this functionality in Google Assistant / for Google Home?
To me it seems I need to extract the "original" question from the JSON that is send to our webservice (when I enable fullfilment).
But since context is used to exchange "state" I have to find a way to exchange the context between the dialogflow and our own chatbot (see above).
But maybe there are other ways ? Can it (invoke our chatbot) be done directly (without DialogFlow as man in the middle) ?
This is one of the those responses that may not be enough for someone who doesn't know what I am talking about and too much for someone who does. Here goes:
It sounds to me as if you need to build an Action with the Actions SDK rather than with Dialog flow. Then you implement a text "intent" in your Action - i.e. one that runs every time the user speaks something. In that text intent you ask the AoG platform for the text - see getRawInput(). Now you do two things. One, you take that raw input and pass it to your bot. Two, you return a promise to tell AoG that you are working on a reply but you don't have it yet. Once the promise is fulfilled - i.e. when your bot replies - you reply with the text you got from your bot.
I have a sample Action called the French Parrot here https://github.com/unclewill/french_parrot. As far as speech goes it simply speaks back whatever it hears as a parrot would. It also goes to a translation service to translate the text and return the (loose) French equivalent.
Your mission, should you choose to accept it, is to take the sample, rip out the code that goes to the translation service and insert the code that goes to your bot. :-)
Two things I should mention. One, it is not "idiomatic" Node or JavaScript you'll find in my sample. What can I say - I think the rest of the world is confused. Really. Two, I have a minimal sample of about 50 lines that eschews the translation here https://github.com/unclewill/parrot. Another option is to use that as a base and add code to call your bot and the Promise-y code to wait on it to it.
If you go the latter route remove the trigger phrases from the action package (action.json).
So you already have a Backend that process user inputs and sends responses back and you want to use it to process a new input flow (coming from Google Assistant)?
That actually my case, I've a service as a Facebook Messenger ChatBot and recently started developing a Google Home Action for it.
It's quite simple. You just need to:
Create an action here https://console.actions.google.com
Download GActions-Cli from here https://developers.google.com/actions/tools/gactions-cli
Create a JSON file action.[fr/en/de/it].json (choose a language). The file is your mean to define your intents and the URL to your webhook (a middleware between your backend and google assistant). It may look like this:
{
"locale": "en",
"actions": [
{
"name": "MAIN",
"description": "Default Welcome Intent",
"fulfillment": {
"conversationName": "app name"
},
"intent": {
"name": "actions.intent.MAIN",
"trigger": {
"queryPatterns": [
"Talk to app name"
]
}
}
}
],
"conversations": {
"app name": {
"name": "app name",
"url": "https://your_nodejs_middleware.com/"
}
}
}
Upload the JSON file using gactions update --action_package action.en.json --project PROJECT_ID
AFAIK, there only a Node.js client library for Actions-on-google https://github.com/actions-on-google/actions-on-google-nodejs that why you need a Node.js middleware before hitting your backend
Now, user inputs will be sent to your Node.js middleware (app.js) hosted at https://your_nodejs_middleware.com/ which may look like:
//require express and all required staff to build a Node.js server,
//look on internet how to build a simple web server in Node.js
//if you a new to this domain. const {
ActionsSdkApp } = require('actions-on-google');
app.post('/', (req, res) => {
req.body = JSON.parse(req.body);
const app = new ActionsSdkApp({
request: req,
response: res
});
// Create functions to handle requests here
function mainIntent(app) {
let inputPrompt = app.buildInputPrompt(false,
'Hey! Welcome to app name!');
app.ask(inputPrompt);
}
function respond(app) {
let userInput = app.getRawInput();
//HERE you get what user typed/said to Google Assistant.
//NOW you can send the input to your BACKEND, process it, get the response_from_your_backend and send it back
app.ask(response_from_your_backend);
}
let actionMap = new Map();
actionMap.set('actions.intent.MAIN', mainIntent);
actionMap.set('actions.intent.TEXT', respond);
app.handleRequest(actionMap); });
Hope that helped!
Thanks for all the help, the main parts of the solution are already given, but I summarize them here
action.json that passes on everything to fullfilment service
man in the middle (in my case IBM Cloud Function) to map JSON between services
Share context/state through the conversationToken property
You can find the demo here: Hey Google talk to Watson