I am testing with a payment processing system and every time a transaction is completed, the payment processor should hit my endpoint with a POST request with payment details so I can save it to my database (Firestore).
Only thing is the function fails on the first try. What I mean is, say a customer pays, the payment processor hits my cloud function, it fails to save to my database. When a second customer makes the transaction a minute, 5 minutes or even 18 minutes later according to my observation, everything works as expected.
Am I facing a cold start problem or what is happening. And how do I solve this.
Here is my function
exports.stkCallback = functions.https.onRequest(async (request, response) => {
if (request.method === 'POST') {
if (request.body.Body.stkCallback.ResultCode === 0) {
const jsonData = request.body.Body.CallbackMetadata;
console.log("USER HAS COMPLETED THE TRANSACTION");
var transactionID;
///This below line logs successfully everytime meaning my payment processor has sent the POST
/// request
console.log("checkoutid:", request.body.Body.CheckoutRequestID)
///I have saved the CheckoutRequestID previously to Firestore so I first query the document
//// with that ID (CheckoutRequestID) and get its data so I can update the transaction as
//// complete
var docRef=db.collection("Transactions").doc(request.body.Body.CheckoutRequestID);
await docRef.get().then((doc) =>{
// eslint-disable-next-line promise/always-return
if (doc.exists) {
//console.log("Document data:", doc.data());
transactionID=doc.id;
transactionData.push(doc.data());
} else {
// doc.data() will be undefined in this case
console.log("No such document!");
}
}).catch((error)=> {
console.log("Error getting document:", error);
});
///Once I get the data I can then go ahead and do other operations.
///Only the above query fails the first time which I don't know why
///By failing Saying No such Document. Which the document does exist
***carrying out other operations using the fetched transactionID and transactionData***
response.sendStatus(200);
} else {
console.log("USER HAS CANCELLED THE TRANSACTION");
response.sendStatus(200);
}
I have refactored my code and reproduced it to the below
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();
exports.stkCallback = functions.https.onRequest(async (request, response) => {
const accountSid = "#";
const authToken = "#";
const client = require("twilio")(accountSid, authToken);
if (request.method === 'POST') {
if (request.body.Body.ResultCode === 0) {
const jsonData = request.body.Body.CallbackMetadata;
console.log("USER HAS COMPLETED THE TRANSACTION");
var transactionID;
///This below line logs successfully everytime meaning my
/////payment processor has sent the POST request
console.log("checkoutid:",
request.body.Body.CheckoutRequestID)
////The below function is critical to all the other below functions below it as
///it supplies the necessary data all the way down
///It is also the function that fails on the first run
var docRef= await db.collection("Transactions").doc(request.body.Body.CheckoutRequestID).get()
.catch((error)=> {
console.log("Error getting document:", error);
});
//// a log of some data from the above function
//// when it fails, the below log is undefined,
console.log("tyyy",docRef.data().Home)
transactionID=docRef.id;
////the data returned from above function is used to perform other operations.
////Below is just one of them
////consequently, it will fail as some values like doc.data().Uid will be ////undefined
await db.collection("Users").doc(doc.data().Uid).collection("Transactions").doc(transactionID).update({
TransactionComplete: true,
transactionCompletedTimeDb: admin.firestore.FieldValue.serverTimestamp(),
Amount: jsonData.Item[0].Value,
ReceiptNO: jsonData.Item[1].Value,
TransactionDate: jsonData.Item[3].Value,
PhoneNumber: jsonData.Item[4].Value,
UserId: doc.data().Uid
})
// eslint-disable-next-line promise/always-return
.catch((error)=> {
// The document probably doesn't exist.
console.error("Error updating document: ", error);
});
response.sendStatus(200);
} else {
console.log("USER HAS CANCELLED THE TRANSACTION");
response.sendStatus(200);
}
});
Attaching an image of a failed function, do note the time
An image of logs of the same triggered function right after (3 minutes later). As you can see the function completes successfully
This seems like a Cold Start Issue
The mitigation of this issue will depend on many information that you are not sharing with us like the complete function, dependencies that you are using, and instance size.
Spreading a loaded function into multiple small functions will help with the cold start time, also using smaller, updated, and cloud oriented libraries will also help.
Also, the size of the payload could be an important factor here, how big is the size of the payload sent to the function and how big is the size of the info that you are writing into the logs? All these small pieces have an important influence on the performance of a cold start.
As a quick solution for your Issue, I can safely say that creating a Scheduled task that triggers your functions every 30 minutes, for example, would be enough to mitigate your issue in the short term.
Related
I am trying to understand the cost of Datastore. It seems that it subscribes to all Mutations. So if there are 50 users, then each message will be send 50 times, even if it not required.
As each real time mutation costs money, we will be paying unnecessary 49 times for this real time message mutation.
Also , it seems to me SyncExpression doesn't have any effect on this Subscription.
I am really stuck here. It will be great of someone can clarify
Amplify generates the datastore boilerplate code for you, but you still need to call it. You won't pay for every user and every mutation.
You will only subscribe to a mutation (explicitly call the code to listen for changes) on a per-user basis for things that user is interested in. e.g. if you are viewing a TODO item, you'd subscribe the user to that item and they'll immediately see if someone else modify it on another device.
UPDATE
Long story... I was triggering back-end computation via GraphQL by making a lambda resolver. The computation took too long and the GQL call would timeout. I updated the code so the GQL call called itself asynchronously (re-trigger the lambda), and returned immediately. Then when the long-running task completed in the spun-up lambda, I updated the a record in the database.
I update the record using AppSync instead of direct GQL so it would trigger mutations, and in the react client, I listen to a mutation for the specific record that will be updated. This way, there is just 1 user listening (if they've triggered the long running action) and that user is only notified about changes to the single DB record they're interested in, and not receiving other user's updates.
I don't know if all this is applicable to your situation. The code snippets below may help you, but they're somewhat out of context.
// In amplify/backend/api/projectname/schema.graphql
type Subscription {
onCouponWithIdUpdated(id: ID!): Coupon #aws_subscribe(mutations: ["updateCoupon"])
}
// In my useSendCoupon hook...
// Subscribe to coupon updates
useEffect(() => {
if (0 === couponId) {
return
}
console.log(`subscribe to coupon updates for couponId:`, couponId)
const onCouponWithIdUpdated = /* GraphQL */ `
subscription OnCouponWithIdUpdated($id: ID!) {
onCouponWithIdUpdated(id: $id) {
id
proofLink
owner
}
}
`
const subscription = API
.graphql(graphqlOperation(onCouponWithIdUpdated, { id: couponId }))
.subscribe({
next: ({ provider, value }) => {
const coupon = value.data.onCouponWithIdUpdated
//console.log(`Proof Link:`, coupon.proofLink)
setProofLinks([coupon.proofLink])
setSendCouponState(COUPON_STATE_PREVIEW_SUCCESS)
},
error: error => console.warn(error)
})
console.log('subscribed: ', subscription)
return () => {
console.log(`unsubscribe to coupon updates`)
subscription.unsubscribe()
}
}, [couponId])
// inside a lambda...
const updateCouponWithProof = async (authorization, couponId, proofLink) => {
const initializeClient = () => new AWSAppSyncClient({
url: process.env.API_XXXX_GRAPHQLAPIENDPOINTOUTPUT,
region: process.env.REGION,
auth: {
type: AUTH_TYPE.AMAZON_COGNITO_USER_POOLS,
jwtToken: authorization
},
disableOffline: true,
})
const executeMutation = async (mutation, operationName, variables) => {
const client = initializeClient()
try {
const response = await client.mutate({
mutation: gql(mutation),
variables,
fetchPolicy: "no-cache",
})
return response.data[operationName]
} catch (err) {
console.log("Error while trying to mutate data", err)
throw JSON.stringify(err)
}
}
const updateCoupon = /* GraphQL */ `
mutation UpdateCoupon(
$input: UpdateCouponInput!
$condition: ModelCouponConditionInput
) {
updateCoupon(input: $input, condition: $condition) {
id
proofLink
owner
}
}
`
const variables = { input: { id: couponId, proofLink } }
try {
return await executeMutation(updateCoupon, 'updateCoupon', variables)
} catch (error) {
console.log(`executeMutation error`, error)
}
}
This is my first Google Action project. I have a simple slot after the invocation. User enters the value on prompt and slot invokes the webhook and make a call to API using the user input. All works fine. However the webhook returns to users even before the API call finish processing and returns the value (line 1 conv.add). I do see in the logs that everything from API is logged fine after the webhook returns to user. Below is the code I am using. I am using inline editor. What am I missing? Thanks for help in advance.
const { conversation } = require('#assistant/conversation');
const functions = require('firebase-functions');
var https = require('https');
const fetch = require('node-fetch');
const app = conversation({debug: true});
app.handle('SearchData', conv => {
const body = JSON.stringify({
val: "this is my body"
});
// prepare the header
var postheaders = {
'Content-Type' : 'application/json',
'Auth' : 'MyAuthCreds'
};
fetch('https://host.domain.com/data', {
method: 'post',
body: body,
headers: postheaders,
})
.then(res => res.json())
.then(d => {
console.log(d);
var profile = d;//JSON.parse(d);
console.log(d.entries);
console.log("Length: "+ d.entries.length);
if(d.entries.length > 0)
{
console.log("Data found");
conv.add("Data found"); //line 1
}
else
{
console.log("no data found");
conv.add("no data found"); //line 1
}
})
.catch(function (err) {
// POST failed...
console.log(err);
});
});
exports.ActionsOnGoogleFulfillment = functions.https.onRequest(app);
Your issue is that your handler is making API calls which are asynchronous, but the Assistant Conversation library doesn't know that you're doing so. So as soon as the handler finishes, it tries to send back a response, but your asynchronous responses (the stuff in the then() blocks) haven't executed yet.
To address this, you need to return a Promise object so the library knows to wait till the Promise is fulfilled before it returns.
Fortunately, in your case, this should be pretty straightforward. fetch and all the .then() blocks return a Promise. So all you need to do is add a return statement in front of the call to fetch. So something like this:
return fetch('https://host.domain.com/data', {
userSchema.pre('save',async function(next){
//hash the password before saving user to database
next()
})
Hey guys I'm trying to understand the concept of middleware in mongoose. Assuming that I have an userSchema that I run the pre hook on to hash the password before saving the user to the database. On the surface, as far as I understand, the above code that I have will hash the password (not the important part for this question) and then call next() to signal that the function is done. However, I am trying to understand how things work under the hood. I want to know what is next() ? Can you guys walk me through an example of how everything works together under the hood from start to end once the code get executed or somehow help me to have a better understanding of this concept? Thanks
Short : with the pre method you can register listeners for certain events of your Schemas. So pre('save', callback) will fire whenever you save a document of said Model. pre means it will execute before the event, so it can be used (for example) to hash a password before saving it to the document.
However, you have several options to define them, see below :
The combination of using an async callback function and providing the next parameter is not necessary, you can either :
use normal callback with next parameter
the next parameter is a function provided to you by mongoose to have a way out, or to tell mongoose you are done and to continue with the next step in the execution chain. Also it is possible to pass an Error to next it will stop the execution chain.
schema.pre('save', function(next) {
// do stuff
if (error) { return next(new Error("something went wrong"); }
return next(null);
});
use async callback
Here the execution chain will continue once your async callback has finished. If there is an error and you want to break/stop execution chain you just throw it
schema.pre('save', async function() {
// do stuff
await doStuff()
await doMoreStuff()
if (error) { throw new Error("something went wrong"); }
return;
});
Straight from the docs : https://mongoosejs.com/docs/middleware.html#pre
Example
const { Schema, model } = require('mongoose');
const SomeSchema = new Schema ({
name : { type : String }
});
SomeSchema.pre('save', function (next) {
console.log('pre save fired!');
return next();
});
const Some = model('Some', SomeSchema);
console.log('before create');
const doc = new Some({ name : 'test' });
doc.save((err) => {
console.log('after saved');
});
This will output
before create
pre save fired!
after saved
I am using the inline editor within Dialogflow with the aim of making queries to the database I have created within Firestore.
In short, the user requests a list of courses, I'd like the chatbot to then grab that information form the db and display that back to the user.
Below I have tried to create a function that will do this, I want to take the user input, say "Art Courses" and have my db return those results.
So far, I have created a function that is triggered when the intent is matched, like so;
function getCourses(agent){
let courseRequest = agent.parameters.courseRequest;
if (getCourses){
console.log('Here is the list you requested for ${getCourses}' + parameters.courseRequest);
return admin.firestore().collection('Course_Information').doc.where('CoureTypes').get();
}
}
Are there any notable things I need to add to my function to perform what I wish to achieve?
Thank you.
UPDATE
This code deploys fine, but when I communicate with my bot and trigger the CourseEnquiry intent, cloud Functions shows this error:
admin.collection is not a function
Whilst this seems self explanatory I can't make sure of what it means, I thought declaring const admin = require('firebase-admin');enables me to use admin.collection
// See https://github.com/dialogflow/dialogflow-fulfillment-nodejs
// for Dialogflow fulfillment library docs, samples, and to report issues
'use strict';
const functions = require('firebase-functions');
const {WebhookClient} = require('dialogflow-fulfillment');
const {Card, Suggestion} = require('dialogflow-fulfillment');
const admin = require('firebase-admin');
process.env.DEBUG = 'dialogflow:debug'; // enables lib debugging statements
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
console.log('Dialogflow Request headers: ' + JSON.stringify(request.headers));
console.log('Dialogflow Request body: ' + JSON.stringify(request.body));
function getDate(agent){
var today = new Date();
}
function welcome(agent) {
agent.add(`Welcome to my agent!`);
}
function test(agent){
agent.add("The test is successful");
}
function getCourses(agent){
// Get the database collection and document
const getCourseDoc = admin.collection('Course_Information').doc('Course_Types');
return getCourseDoc.get()
.then(doc => {
if (!doc.exists) {
agent.add('No data found in the database!');
} else {
agent.add(doc.data().entry);
}
return Promise.resolve('Here is the information you wanted');
}).catch(() => {
agent.add('Error reading entry from the Firestore database.');
});
}
function getSubmissionDateSep(agent){
agent.add('Your next submission date is for coursework 1 is');
}
let intentMap = new Map();
intentMap.set('Default Welcome Intent', welcome);
intentMap.set('Test_Test', test);
intentMap.set('CourseEnquiry', getCourses);
intentMap.set('Submission_Dates - sept', getSubmissionDateSep);
agent.handleRequest(intentMap);
});
UPDATE #2
Hey guys, still not got anywhere with this, I have tried adding:
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
According to this document but I get this error when deploying:
The deployment of your Cloud Function failed:
Function load error: Code in file index.js can't be loaded.
Is there a syntax error in your code?
Detailed stack trace: Error: Firebase config variables are not available. Please use the latest version of the Firebase CLI to deploy this function.
You don't show how you're responding to the user with your results, but you'll want to make sure you handle that as part of the then() clause in a Promise. Since the get() in the firestore collection returns a Promise, and you are returning it from your function, you need to make sure that the calling function treats it as a Promise, has a then() clause, and sends back the result as part of something inside this clause.
Im using angularfire to save data into my firebase. Here is a quick code.
$scope.users.$add({
Name:$scope.username,
Age:$scope.newage,
Contact: $scope.newcontact,
});
alert('Saved to firebase');
I am successful in sending these data to my firebase however how can I catch an error if these data are not saved successfully? Any ideas?
EDIT
So after implementing then() function.
$scope.users.$add({
Name:'Frank',
Age:'20',
Contact: $scope.newcontact,
}).then(function(ref) {
alert('Saved.');
}).catch(function(error) {
console.error(error); //or
console.log(error);
alert('Not Saved.');
});
When connected to the internet. The then() function is fine. It waits for those data to be saved in firebase before showing the alert.
What I want is for it to tell me that data is not saved. catch error function is not firing when i am turning off my internet connection and submitting those data.
When you call $add() it returns a promise. To detect when the data was saved, implement then(). To detect when saving failed, implement catch():
var list = $firebaseArray(ref);
list.$add({ foo: "bar" }).then(function(ref) {
var id = ref.key;
console.log("added record with id " + id);
list.$indexFor(id); // returns location in the array
}).catch(function(error) {
console.error(error);
});
See the documentation for add().
Update
To detect when the data cannot be saved due to not having a network connection is a very different problem when it comes to the Firebase Database. Not being able to save in this case is not an error, but merely a temporary condition. This condition doesn't apply just to this set() operation, but to all read/write operation. For this reason, you should handle it more globally, by detecting connection state:
var connectedRef = firebase.database().ref(".info/connected");
connectedRef.on("value", function(snap) {
if (snap.val() === true) {
alert("connected");
} else {
alert("not connected");
}
});
By listening to .info/connected your code can know that the user is not connected to the Firebase Database and handle it according to your app's needs.