This is the method used for database connection and obtain results:
public Future<String> getDatabaseUsers(JDBCClient client) {
return Future.future(pHandler -> {
client.getConnection(res -> {
if (res.succeeded()) {
SQLConnection con = res.result();
con.query("select u.id, u.name from users u", rHandler -> {
String data;
if (rHandler.succeeded()) {
data = Json.encode(rHandler.result().getResults());
} else {
data = "Database execution error";
}
con.close();
pHandler.complete(data);
});
} else {
pHandler.complete("Cannot connect to database");
}
});
});
}
This is the caller method:
private void handleRequest(RoutingContext routingContext, JDBCClient client,
Handler<List<String>> resultHandler) {
routingContext.vertx().<String>executeBlocking(pHandler -> {
pHandler.complete(getDatabaseUsers(client).result());
}, ar -> {
List<String> responses = new ArrayList<>();
if (ar.succeeded()) {
responses.add(ar.result());
System.out.println(Json.encode(responses)); // Null here
}
resultHandler.handle(responses);
});
}
The resultHandler object is used to add more responses from other processes; but this is not the problem actually.
And this is the endpoint:
router.get("/db").handler(ctx -> handleRequest(
ctx, client, (list) -> ctx.response()
.putHeader("Content-Type", "text/plain")
.end(Json.encode(list))));
The problem with this code, is that service response is [null] and the database method is not accomplished yet.
So, what should I do to wait for database response and then send the response to the client?
There are a number of issues with this code:
getDatabaseUsers swallows errors. You probably want to do pHandler.fail(res.failure()); or something similar.
This code: pHandler.complete(getDatabaseUsers(client).result()); is not doing what you think it's doing. result() doesn't block, so you complete your future with another unfishined future.
What you actually want is the reverse:
getDatabaseUsers(client).handle((r) -> { // This is called when the futures actually completes
if (r.succeeded()) {
pHandler.complete(r.result());
}
...
});
I would really recommend going over this guide, as it covers exactly your case.
Related
Taking a piece from Vert.x website example:
private Future<Void> prepareDatabase() {
Promise<Void> promise = Promise.promise();
dbClient = JDBCClient.createShared(vertx, new JsonObject() //(1)
.put("url", "jdbc:hsqldb:file:db/wiki") //(2)
.put("driver_class", "org.hsqldb.jdbcDriver") //(3)
.put("max_pool_size", 30)); //(4)
dbClient.getConnection(ar -> { //(5)
if (ar.failed()) {
LOGGER.error("Could not open a database connection", ar.cause());
promise.fail(ar.cause()); //(6)
} else {
SQLConnection connection = ar.result(); //(7)
connection.execute(SQL_CREATE_PAGES_TABLE, create -> {
connection.close(); //(8)
if (create.failed()) {
LOGGER.error("Database preparation error", create.cause());
promise.fail(create.cause());
} else {
promise.complete(); //(9)
}
});
}
});
return promise.future();
}
In (8), the connection is closed at the very beginning of the handler. What if we execute a query and then iterate the result in the handler:
private fun jdbcQuery(sql: String, params: JsonArray): Future<ResultSet> {
val promise: Promise<ResultSet> = Promise.promise()
getJDBCClient().getConnection { ar ->
if (ar.succeeded()) {
val connection = ar.result()
connection.queryWithParams(sql, params) { res ->
connection.close() //(10) release the connection
if (res.succeeded()) {
val result = res.result()
promise.complete(result)
} else {
promise.fail(res.cause())
}
}
} else {
promise.fail(ar.cause())
}
}
return promise.future()
}
I can fetch the data inside if (res.succeeded()).
My question is: why we can close and release the connection before iterating to fetch data? How does it work?
The queryWithParams API fetches the entire response from the DB when it is executed. Results are not fetched lazily. For this reason, it is safe to close the connection at the beginning of your response handler callback, because by that time the entire result set has already been received by the client. Results are only fetched lazily when you use the queryStream API. If you were using that API, you would want to wait to close the connection until all the results were received.
I'm building facebook chatbot using AWS Lambda and MongoDB. At the moment, my application is pretty simple but I'm trying to nail down the basics before I move onto the complex stuff.
I understand AWS Lambda is stateless but I've read adding below line in handler along with variables initialized outside handler, I don't have to establish DB connection on every request.
context.callbackWaitsForEmptyEventLoop = false;
(I've read this from this article; https://www.mongodb.com/blog/post/optimizing-aws-lambda-performance-with-mongodb-atlas-and-nodejs)
I'm adding my entire code below
'use strict'
const
axios = require('axios'),
mongo = require('mongodb'),
MongoClient = mongo.MongoClient,
assert = require('assert');
var VERIFY_TOKEN = process.env.VERIFY_TOKEN;
var PAGE_ACCESS_TOKEN = process.env.PAGE_ACCESS_TOKEN;
var MONGO_DB_URI = process.env.MONGO_DB_URI;
let cachedDb = null;
let test = null;
exports.handler = (event, context, callback) => {
var method = event.context["http-method"];
context.callbackWaitsForEmptyEventLoop = false;
console.log("test :: " + test);
if (!test) {
test = "1";
}
// process GET request --> verify facebook webhook
if (method === "GET") {
var queryParams = event.params.querystring;
var rVerifyToken = queryParams['hub.verify_token']
if (rVerifyToken === VERIFY_TOKEN) {
var challenge = queryParams['hub.challenge'];
callback(null, parseInt(challenge))
} else {
var response = {
'body': 'Error, wrong validation token',
'statusCode': 403
};
callback(null, response);
}
// process POST request --> handle message
} else if (method === "POST") {
let body = event['body-json'];
body.entry.map((entry) => {
entry.messaging.map((event) => {
if (event.message) {
if (!event.message.is_echo && event.message.text) {
console.log("BODY\n" + JSON.stringify(body));
console.log("<<MESSAGE EVENT>>");
// retrieve message
let response = {
"text": "This is from webhook response for \'" + event.message.text + "\'"
}
// facebook call
callSendAPI(event.sender.id, response);
// store in DB
console.time("dbsave");
storeInMongoDB(event, callback);
}
} else if (event.postback) {
console.log("<<POSTBACK EVENT>>");
} else {
console.log("UNHANDLED EVENT; " + JSON.stringify(event));
}
})
})
}
}
function callSendAPI(senderPsid, response) {
console.log("call to FB");
let payload = {
recipient: {
id: senderPsid
},
message: response
};
let url = `https://graph.facebook.com/v2.6/me/messages?access_token=${PAGE_ACCESS_TOKEN}`;
axios.post(url, payload)
.then((response) => {
console.log("response ::: " + response);
}).catch(function(error) {
console.log(error);
});
}
function storeInMongoDB(messageEnvelope, callback) {
console.log("cachedDB :: " + cachedDb);
if (cachedDb && cachedDb.serverConfig.isConnected()) {
sendToAtlas(cachedDb.db("test"), messageEnvelope, callback);
} else {
console.log(`=> connecting to database ${MONGO_DB_URI}`);
MongoClient.connect(MONGO_DB_URI, function(err, db) {
assert.equal(null, err);
cachedDb = db;
sendToAtlas(db.db("test"), messageEnvelope, callback);
});
}
}
function sendToAtlas(db, message, callback) {
console.log("send to Mongo");
db.collection("chat_records").insertOne({
facebook: {
messageEnvelope: message
}
}, function(err, result) {
if (err != null) {
console.error("an error occurred in sendToAtlas", err);
callback(null, JSON.stringify(err));
} else {
console.timeEnd("dbsave");
var message = `Inserted a message into Atlas with id: ${result.insertedId}`;
console.log(message);
callback(null, message);
}
});
}
I did everything as instructed and referenced a few more similar cases but somehow on every request, "cachedDb" value is not saved from previous request and the app is establishing the connection all over again.
Then I also read that there is no guarantee the Lambda function is using the same container on multiple requests so I made another global variable "test". "test" variable value is logged "1" from the second request which means it's using the same container but again, "cachedDb" value is not saved.
What am I missing here?
Thanks in advance!
In short AWS Lambda function is not a permanently running service of any kind.
So, far I know AWS Lambda works on idea - "one container processes one request at a time".
It means when request comes and there is available running container for the Lambda function AWS uses it, else it starts new container.
If second request comes when first container executes Lambda function for first request AWS starts new container.
and so on...
Then there is no guarantee in what container (already running or new one) Lambda function will be executed, so... new container opens new DB connection.
Of course, there is an inactivity period and no running containers will be there after that. All will start over again by next request.
I have two reactive methods in "Layer A" that's exposed to the app:
LAYER A - publicly accessible methods to the rest of the app
public Single<ResponseData> postMyData(ReqData data, Long id) {
if (!isUserLoggedIn()) return postLogin().andThen(postData(data, id));
return postData(data, id);
}
public Completable postLogin() {
Account account = getAccountData();
ReqLoginData loginData = new ReqLoginData(account.email, account.pass, OAUTH_CLIENT_ID, OAUTH_CLIENT_SECRET);
Single<ResponseLogin> singlePostLogin = postLogin(loginData);
return Completable.create(subscriber -> singlePostLogin.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(loginResponse -> {
// Success
storeAccessToken(loginResponse);// *** this will recreate apiClient with new access token
if (!subscriber.isDisposed()) subscriber.onComplete();
}, throwable -> { if (!subscriber.isDisposed()) subscriber.onError(throwable);}
));
}
LAYER B - only accessible by Layer A
public Single<ResponseLogin> postLogin(ReqLoginData loginData) {
Log.d(TAG, "using apiClient "+apiClient.toString());
Single<Response<ResponseLogin>> postLogin = apiClient.postLogin(loginData);
return Single.create(subscriber -> postLogin.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(response -> {
if(response.body() != null) {
if (!subscriber.isDisposed()) subscriber.onSuccess(response.body());
} else {
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.message()));
}
}, throwable -> {
throwable.printStackTrace();
if(!subscriber.isDisposed()) subscriber.onError(throwable);
}));
}
public Single<ResponseData> postData(ReqData data, Long id) {
Log.d(TAG, "using apiClient "+apiClient.toString());
Single<Response<ResponseData>> postData = apiClient.postData(id, data);
return Single.create(subscriber -> postData.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(response -> {
if(response.body() != null) {
if (!subscriber.isDisposed()) subscriber.onSuccess(response.body());
} else {
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.message()));
}
}, throwable -> {
throwable.printStackTrace();
if(!subscriber.isDisposed()) subscriber.onError(throwable);
}));
}
LAYER C - API Interface
#Headers({"Content-Type: application/json"})
#POST(Constants.API_PREFIX + "/auth/token/")
Single<Response<ResponseLogin>> postLogin(#Body ReqLoginData reqLoginData);
#Headers({"Content-Type: application/json"})
#POST(Constants.API_PREFIX + "/data/{id}/")
Single<Response<ResponseData>> postData(#Path("id") Long id, #Body ReqData reqData);
A huge challenge for me at the moment is to make sure the user is logged-in before attempting to POST data. So, using postLogin().andThen(postData(data, id)) made sense to me :
At the time of my late-night development, it seemed that both methods postLogin() and postData() are called in parallel - with postLogin() first.
How can I make postMyData() execute postLogin() first, wait for success or failure, followed by postData() on success ?
Note: postLogin() will update apiClient with the new token inside storeAccessToken(). I now see that postData() doesn't use a reference to the new apiClient. It's important that postData() will use the new apiClient that's instantiated within storeAccessToken()
Sorry for the newbie question :-(
it turned out that Completable.andThen(Single) works very well to serialize code execution. Unfortunately, I didn't (half asleep) realize that only code within the create() method was actually scheduled.
my code below answers the problem I had (these are both in LAYER-B):
public Single<ResponseLogin> postLogin(ReqLoginData loginData) {
return Single.create(subscriber -> {
Log.d(TAG, "using apiClient "+apiClient.toString());
apiClient.postLogin(loginData)
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(response -> {
if(response.isSuccessful() && response.body() != null) {
if (!subscriber.isDisposed()) subscriber.onSuccess(response.body());
} else {
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.message()));
}
}, throwable -> {
throwable.printStackTrace();
if(!subscriber.isDisposed()) subscriber.onError(throwable);
});
});
}
public Single<ResponseData> postData(ReqData data, Long id) {
return Single.create(subscriber -> {
Log.d(TAG, "using apiClient "+apiClient.toString());
apiClient.postData(id, data)
.subscribeOn(Schedulers.io())
.observeOn(Schedulers.io())
.subscribe(response -> {
if(response.body() != null) {
if (!subscriber.isDisposed()) subscriber.onSuccess(response.body());
} else {
if (!subscriber.isDisposed()) subscriber.onError(new Throwable(response.message()));
}
}, throwable -> {
throwable.printStackTrace();
if(!subscriber.isDisposed()) subscriber.onError(throwable);
});
});
}
I've implemented a REST endpoint in ballerinalang called https://localhost:9090/isValidUser. And here is my code below
import ballerina.net.http;
#http:configuration {
basePath:"/",
httpsPort:9090,
keyStoreFile:"${ballerina.home}/bre/security/wso2carbon.jks",
keyStorePassword:"wso2carbon",
certPassword:"wso2carbon",
trustStoreFile:"${ballerina.home}/bre/security/client-truststore.jks",
trustStorePassword:"wso2carbon"
}
service<http> authentication {
#http:resourceConfig {
methods:["POST"],
path:"/isValidUser"
}
resource isValidUser (http:Request req, http:Response res) {
println(req.getHeaders());
res.send();
}
}
Now I need to do is when I invoke that URL from the browser, I need to redirect the user to another URL called https://localhost:3000 after some validations happen within my service.
So how can I do this redirection from ballerinalang?
Ballerina has provided smooth API to do the redirection. Please check following code which elaborates the Listener endpoint redirection.
service<http:Service> redirect1 bind {port:9090} {
#http:ResourceConfig {
methods:["GET"],
path:"/"
}
redirect1 (endpoint client, http:Request req) {
http:Response res = new;
_ = client -> redirect(res, http:REDIRECT_TEMPORARY_REDIRECT_307,
["http://localhost:9093/redirect2"]);
}
}
The complete example is available in
Ballerina Redirects
In Ballerina, you need to handle the redirects yourself by setting the necessary headers and status code. The following example is a simple demo of how you can redirect in Ballerina. (note: I tried this in Ballerina 0.95.2)
import ballerina.net.http;
#http:configuration {basePath:"/hello"}
service<http> helloWorld {
#http:resourceConfig {
methods:["GET"],
path:"/"
}
resource sayHello (http:Request request, http:Response response) {
map qParams = request.getQueryParams();
var name, _ = (string)qParams.name;
if (isExistingUser(name)) {
response.setStringPayload("Hello Ballerina!");
} else {
response.setHeader("Location", "http://localhost:9090/hello/newUser");
response.setStatusCode(302);
}
_ = response.send();
}
#http:resourceConfig {path:"/newUser"}
resource newUser (http:Request request, http:Response response) {
string msg = "New to Ballerina? Welcome!";
response.setStringPayload(msg);
_ = response.send();
}
}
function isExistingUser (string name) (boolean) {
return name == "Ballerina";
}
There is a parse CloudCode function created as such:
Parse.Cloud.define("getCurrentEvents", function(request, response) {
var TimedEvent = Parse.Object.extend("TimedEvent");
var query = new Parse.Query(TimedEvent);
query.greaterThan("expiresOn", new Date());
query.find({
success: function(results) {
response.success(results);
},
error: function(error) {
response.error("There was an error while looking for TimedEvents");
}
});
});
It returns an array of TimedEvent, as shown in the curl test here:
{"result":[{"expiresOn":{"__type":"Date","iso":"2014-07-31T22:31:00.000Z"},"playMode":"Normal","tableId":"Carnival","objectId":"J1LSO3EnKi","createdAt":"2014-07-28T21:48:22.983Z","updatedAt":"2014-07-28T22:32:14.304Z","__type":"Object","className":"TimedEvent"}]}
When trying to access it from Unity SDK however, I get a "cannot convert to destination type" exception with the following line:
System.Threading.Tasks.Task<Parse.ParseObject[]> task =
Parse.ParseCloud.CallFunctionAsync<Parse.ParseObject[]> ("getCurrentEvents", parameters);
I also tried
System.Threading.Tasks.Task<IEnumerable<Parse.ParseObject>> task =
Parse.ParseCloud.CallFunctionAsync<IEnumerable<Parse.ParseObject[]>> ("getCurrentEvents", parameters);
with the same (lack of) results. What kind of signature is the SDK expecting?
Have you tried something like this (without IEnumerable?):
Threading.Tasks.Task<Parse.ParseObject> task = Parse.ParseCloud.CallFunctionAsync<Parse.ParseObject>("getCurrentEvents", parameters);
But better yet, you could extend ParseObject to create your own TimedEvent class in Unity, like this:
[ParseClassName("TimeEvent")]
public class TimeEvent : ParseObject
{
[ParseFieldName("expiresOn")]
public DateTime expiresOn
{
get { return GetProperty<DateTime>("expiresOn"); }
set { SetProperty(value, "expiresOn"); }
}
[ParseFieldName("playMode")]
public string playMode
{
get { return GetProperty<string>("playMode"); }
set { SetProperty(value, "playMode"); }
}
[ParseFieldName("tableId")]
public string tableId
{
get { return GetProperty<string>("tableId"); }
set { SetProperty(value, "tableId"); }
}
// any other fields you want to access
}
Then you can query your data like this:
IEnumerator getTimedEvents(Dictionary<string, object> parameters)
{
var cloudTask = Parse.ParseCloud.CallFunctionAsync<TimeEvent>("getCurrentEvents", parameters);
while (!cloudTask.IsCompleted)
yield return null;
if (cloudTask.IsCanceled || cloudTask.IsFaulted)
{
// handle error
}
else
{
TimeEvent t = cloudTask.Result;
// do stuff with t
}
}
P.S. Don't forget to register your Parse class somewhere (I usually do it in the Awake() of an early GameObject). In your case, you would do it like this:
Parse.ParseObject.RegisterSubclass<TimedEvent>();