I am trying to use the results of an api call in conversation but haven't been able to pass the results so that I can use them in conv.ask. In the example here, I am able to log the "wind inner" but when I try to use it in conv.ask, I get "undefined." I know it is a scoping issue, but I haven't been able to solve it. Thanks!
app.intent('weather', (conv) => {
var url = "http://api.wunderground.com/api/"+apiKey+"/yesterday/q/55417.json";
var request = http.get(url, function (response) {
var buffer = "",
data,
history;
response.on("data", function (chunk) {
buffer += chunk;
});
response.on("end", function (err) {
console.log(buffer);
console.log("\n");
data = JSON.parse(buffer);
history = data.history;
var wind = (history.dailysummary[0].maxwspdi);
console.log("wind inner: ", wind);//this works
});
});
conv.ask("the wind speed is" + wind + "miles per hour");
//unable to get the wind variable to be defined ouside the api call
});
You should be using a Promise to return asynchronous responses. Additionally, make sure you wait until you get the response data to generate your text-to-speech. Here's a snippet that should work.
app.intent('weather', (conv) => {
return new Promise((resolve, reject) => {
const url = "http://api.wunderground.com/api/"+apiKey+"/yesterday/q/55417.json";
const request = http.get(url, function (response) {
var buffer = "",
data,
history;
response.on("data", function (chunk) {
buffer += chunk;
});
response.on("end", function (err) {
console.log(buffer);
console.log("\n");
data = JSON.parse(buffer);
history = data.history;
const wind = (history.dailysummary[0].maxwspdi);
console.log("wind inner: ", wind);//this works
conv.ask("the wind speed is" + wind + "miles per hour");
resolve();
});
});
});
});
Related
It's fairly simple to make one call to Adobe PDF Services, get the result, and save it, for example:
// more stuff above
exportPdfOperation.execute(executionContext)
.then(result => result.saveAsFile(output))
But if I want to do two, or more, operations, do I need to keep saving the result to the file system and re-providing it (is that even a word ;) to the API?
So this tripped me up as well. In most demos, you'll see:
result => result.saveAsFile()
towards the end. However, the object passes to the completed promise, result, is a FileRef object that can then be used as the input to another call.
Here's a sample that takes an input Word doc and calls the API method to create a PDF. It then takes that and runs OCR on it. Both methods that wrap the API calls return FileRefs, so at the end I saveAsFile on it. (Note, this demo is using v1 of the SDK, it would work the same w/ v2.)
const PDFToolsSdk = require('#adobe/documentservices-pdftools-node-sdk');
const fs = require('fs');
//clean up previous
(async ()=> {
// hamlet.docx was too big for conversion
const input = './hamlet2.docx';
const output = './multi.pdf';
const creds = './pdftools-api-credentials.json';
if(fs.existsSync(output)) fs.unlinkSync(output);
let result = await createPDF(input, creds);
console.log('got a result');
result = await ocrPDF(result, creds);
console.log('got second result');
await result.saveAsFile(output);
})();
async function createPDF(source, creds) {
return new Promise((resolve, reject) => {
const credentials = PDFToolsSdk.Credentials
.serviceAccountCredentialsBuilder()
.fromFile(creds)
.build();
const executionContext = PDFToolsSdk.ExecutionContext.create(credentials),
createPdfOperation = PDFToolsSdk.CreatePDF.Operation.createNew();
// Set operation input from a source file
const input = PDFToolsSdk.FileRef.createFromLocalFile(source);
createPdfOperation.setInput(input);
let stream = new Stream.Writable();
stream.write = function() {
}
stream.end = function() {
console.log('end called');
resolve(stream);
}
// Execute the operation and Save the result to the specified location.
createPdfOperation.execute(executionContext)
.then(result => resolve(result))
.catch(err => {
if(err instanceof PDFToolsSdk.Error.ServiceApiError
|| err instanceof PDFToolsSdk.Error.ServiceUsageError) {
reject(err);
} else {
reject(err);
}
});
});
}
async function ocrPDF(source, creds) {
return new Promise((resolve, reject) => {
const credentials = PDFToolsSdk.Credentials
.serviceAccountCredentialsBuilder()
.fromFile(creds)
.build();
const executionContext = PDFToolsSdk.ExecutionContext.create(credentials),
ocrOperation = PDFToolsSdk.OCR.Operation.createNew();
// Set operation input from a source file.
//const input = PDFToolsSdk.FileRef.createFromStream(source);
ocrOperation.setInput(source);
let stream = new Stream.Writable();
stream.end = function() {
console.log('end called');
resolve(stream);
}
// Execute the operation and Save the result to the specified location.
ocrOperation.execute(executionContext)
.then(result => resolve(result))
.catch(err => reject(err));
});
}
I'm using puppeteer to scrape page that has contents that change periodically and use express to present data in rest api.
If I turn on headless chrome to see what is being shown in the browser, the new data is there, but the data is not showing up in get() and http://localhost:3005/api-weather. The normal browser only shows the original data.
const express = require('express');
const server = new express();
const cors = require('cors');
const morgan = require('morgan');
const puppeteer = require('puppeteer');
server.use(morgan('combined'));
server.use(
cors({
allowHeaders: ['sessionId', 'Content-Type'],
exposedHeaders: ['sessionId'],
origin: '*',
methods: 'GET, HEAD, PUT, PATCH, POST, DELETE',
preflightContinue: false
})
);
const WEATHER_URL = 'https://forecast.weather.gov/MapClick.php?lat=40.793588904953985&lon=-73.95738513173298';
const hazard_url2 = `file://C:/Users/xdevtran/Documents/vshome/wc_api/weather-forecast-nohazard.html`;
(async () => {
try {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.setRequestInterception(true);
page.on("request", request => {
console.log(request.url());
request.continue();
});
await page.goto(hazard_url2, { timeout: 0, waitUntil: 'networkidle0' });
hazard = {
"HazardTitle": "stub",
"Hazardhref": "stub"
}
let forecast = await page.evaluate(() => {
try {
let forecasts = document.querySelectorAll("#detailed-forecast-body.panel-body")[0].children;
let weather = [];
for (var i = 0, element; element = forecasts[i]; i++) {
period = element.querySelector("div.forecast-label").textContent;
forecast = element.querySelector("div.forecast-text").textContent;
weather.push(
{
period,
forecast
}
)
}
return weather;
} catch (err) {
console.log('error in evaluate: ', err);
res.end();
}
}).catch(err => {
console.log('err.message :', err.message);
});
weather = forecast;
server.get('/api-weather', (req, res) => {
try {
res.end(JSON.stringify(weather, null, ' '));
console.log(weather);
} catch (err) {
console.log('failure: ', err);
res.sendStatus(500);
res.end();
return;
}
});
} catch (err) {
console.log('caught error :', err);
}
browser.close();
})();
server.listen(3005, () => {
console.log('http://localhost:3005/api-weather');
});
I've tried several solutions WaitUntil, WaitFor, .then and sleep but nothing seems to work.
I wonder if it has something to do with express get()? I'm using res.end() instead of res.send() is because the json looks better when I use res.end(). I don't really know the distinction.
I'm also open to using this reload solution. But I received errors and didn't use it.
I also tried waitForNavigation(), but I don't know how it works, either.
Maybe I'm using the wrong search term to find the solution. Could anyone point me in the right direction? Thank you.
I have a route set up to receive a webhook from SendGrid, which sends MultiPart/form data. I can get the various fields to output in the console with busboy, but I'm struggling to fill in the final piece of the puzzle: getting this parsed data into a Collection object (or just into MongoDB if not familiar with meteor).
I thought something like the following would work, but the data arrays in the db are always blank, i'm guessing i'm missing a crucial step in knowing when the stream has finished?
WebApp.connectHandlers.use('/applicants', (req, res, next) => {
let body = '';
req.on('data', Meteor.bindEnvironment(function (data) {
body += data;
let bb = new Busboy({ headers: req.headers });
let theEmail = [];
bb.on('field', function(fieldname, val) {
console.log('Field [%s]: value: %j', fieldname, val);
let theObject = [];
theObject[fieldname] = val;
theEmail.push(theObject);
}).on('error', function(err) {
console.error('oops', err);
}).on('finish', Meteor.bindEnvironment(function() {
console.log('Done parsing form!');
// try to add data to database....
Meteor.call('applicants.add', theEmail);
}));
return req.pipe(bb);
}));
req.on('end', Meteor.bindEnvironment(function () {
res.writeHead(200);
res.end();
}));
I'm building facebook chatbot using AWS Lambda and MongoDB. At the moment, my application is pretty simple but I'm trying to nail down the basics before I move onto the complex stuff.
I understand AWS Lambda is stateless but I've read adding below line in handler along with variables initialized outside handler, I don't have to establish DB connection on every request.
context.callbackWaitsForEmptyEventLoop = false;
(I've read this from this article; https://www.mongodb.com/blog/post/optimizing-aws-lambda-performance-with-mongodb-atlas-and-nodejs)
I'm adding my entire code below
'use strict'
const
axios = require('axios'),
mongo = require('mongodb'),
MongoClient = mongo.MongoClient,
assert = require('assert');
var VERIFY_TOKEN = process.env.VERIFY_TOKEN;
var PAGE_ACCESS_TOKEN = process.env.PAGE_ACCESS_TOKEN;
var MONGO_DB_URI = process.env.MONGO_DB_URI;
let cachedDb = null;
let test = null;
exports.handler = (event, context, callback) => {
var method = event.context["http-method"];
context.callbackWaitsForEmptyEventLoop = false;
console.log("test :: " + test);
if (!test) {
test = "1";
}
// process GET request --> verify facebook webhook
if (method === "GET") {
var queryParams = event.params.querystring;
var rVerifyToken = queryParams['hub.verify_token']
if (rVerifyToken === VERIFY_TOKEN) {
var challenge = queryParams['hub.challenge'];
callback(null, parseInt(challenge))
} else {
var response = {
'body': 'Error, wrong validation token',
'statusCode': 403
};
callback(null, response);
}
// process POST request --> handle message
} else if (method === "POST") {
let body = event['body-json'];
body.entry.map((entry) => {
entry.messaging.map((event) => {
if (event.message) {
if (!event.message.is_echo && event.message.text) {
console.log("BODY\n" + JSON.stringify(body));
console.log("<<MESSAGE EVENT>>");
// retrieve message
let response = {
"text": "This is from webhook response for \'" + event.message.text + "\'"
}
// facebook call
callSendAPI(event.sender.id, response);
// store in DB
console.time("dbsave");
storeInMongoDB(event, callback);
}
} else if (event.postback) {
console.log("<<POSTBACK EVENT>>");
} else {
console.log("UNHANDLED EVENT; " + JSON.stringify(event));
}
})
})
}
}
function callSendAPI(senderPsid, response) {
console.log("call to FB");
let payload = {
recipient: {
id: senderPsid
},
message: response
};
let url = `https://graph.facebook.com/v2.6/me/messages?access_token=${PAGE_ACCESS_TOKEN}`;
axios.post(url, payload)
.then((response) => {
console.log("response ::: " + response);
}).catch(function(error) {
console.log(error);
});
}
function storeInMongoDB(messageEnvelope, callback) {
console.log("cachedDB :: " + cachedDb);
if (cachedDb && cachedDb.serverConfig.isConnected()) {
sendToAtlas(cachedDb.db("test"), messageEnvelope, callback);
} else {
console.log(`=> connecting to database ${MONGO_DB_URI}`);
MongoClient.connect(MONGO_DB_URI, function(err, db) {
assert.equal(null, err);
cachedDb = db;
sendToAtlas(db.db("test"), messageEnvelope, callback);
});
}
}
function sendToAtlas(db, message, callback) {
console.log("send to Mongo");
db.collection("chat_records").insertOne({
facebook: {
messageEnvelope: message
}
}, function(err, result) {
if (err != null) {
console.error("an error occurred in sendToAtlas", err);
callback(null, JSON.stringify(err));
} else {
console.timeEnd("dbsave");
var message = `Inserted a message into Atlas with id: ${result.insertedId}`;
console.log(message);
callback(null, message);
}
});
}
I did everything as instructed and referenced a few more similar cases but somehow on every request, "cachedDb" value is not saved from previous request and the app is establishing the connection all over again.
Then I also read that there is no guarantee the Lambda function is using the same container on multiple requests so I made another global variable "test". "test" variable value is logged "1" from the second request which means it's using the same container but again, "cachedDb" value is not saved.
What am I missing here?
Thanks in advance!
In short AWS Lambda function is not a permanently running service of any kind.
So, far I know AWS Lambda works on idea - "one container processes one request at a time".
It means when request comes and there is available running container for the Lambda function AWS uses it, else it starts new container.
If second request comes when first container executes Lambda function for first request AWS starts new container.
and so on...
Then there is no guarantee in what container (already running or new one) Lambda function will be executed, so... new container opens new DB connection.
Of course, there is an inactivity period and no running containers will be there after that. All will start over again by next request.
I am using Spotify's Web Api.
I have an array of songSearches. And for each songSearch within songSearches, a request is made to search for these songs and add it to an array of Songs that I have which hold the track's spotify ID. I need all of these IDs at once so I can execute one POST request to add all of these tracks to a playlist. Therefore, all of the search requests must be completed before the POST request is made.
All of this needs to happen at the click of a button.
Code executed when button is clicked:
onSubmit(){
while (this.songsService.songSearches.length){
this.spotifyserv.searchTrack(this.songsService.songSearches[0]);
this.songsService.songSearches.shift();
}
this.spotifyserv.add_tracks_to_playlist(); //Needs to wait until all requests ^ have been completed
}
Search Track function:
searchTrack(searchParams: SongSearchParams, type='track'){
var headers = new Headers({'Authorization': 'Bearer ' + this.hash_params.access_token});
this.user_url = "https://api.spotify.com/v1/search?query="+searchParams.artist+' '+searchParams.title+"&offset=0&limit=1&type="+type+"&market=US";
return this.http.get(this.user_url, {headers : headers})
.subscribe(
response => {
var res = response.json();
var searched_song = {artist : null, title : null, imagePath : null, spotifyID : null}
searched_song.artist = res.tracks.items[0].artists[0].name;
searched_song.title = res.tracks.items[0].name;
searched_song.imagePath = res.tracks.items[0].album.images[0].url;
searched_song.spotifyID = res.tracks.items[0].id;
this.songsService.addSong(searched_song);
}
);
}
Add Tracks to Playlist function
add_tracks_to_playlist(){
var headers = new Headers({'Authorization': 'Bearer ' + this.hash_params.access_token});
headers.append('Accept', 'application/json');
this.user_url = "https://api.spotify.com/v1/users/" + this.user_id + "/playlists/" + this.playlist_id + "/tracks";
let songs: Song[] = this.songsService.getSongs(); //playlist_id is hardcoded rn. needs to be added dynamically
let songIDs : String [] = [];
for (var i = 0; i < songs.length; i++){
songIDs.push("spotify:track:" + songs[i].spotifyID);
}
let body = {"uris": songIDs};
this.http
.post(this.user_url, JSON.stringify(body), {headers : headers})
.toPromise()
.then(response => console.log(response))
.catch(this.handleError);
}
I understand that I probably want to be using Promise.all() somewhere but I am not sure how/where to use it
As you mention, Promise.all() could help you if searchTrack() returns a Promise. So something like this:
searchTrack(searchParams: SongSearchParams, type='track') {
// ...
return this.http.get(this.user_url, {headers : headers})
.toPromise()
.then(response => {
// Code from your subscribe
});
}
and change the onSubmit
onSubmit(){
const searchPromises: Promise<void>[] = [];
while (this.songsService.songSearches.length){
searchPromises.push(this.spotifyserv.searchTrack(this.songsService.songSearches[0]));
this.songsService.songSearches.shift();
}
//Needs to wait until all requests ^ have been completed
Promise.all(searchPromises)
.then(() => this.spotifyserv.add_tracks_to_playlist());
}