How to use Java libraries asynchronously in a Scala Play 2.0 application? - scala

Question

I see in the Play 2.0 Scala doc for calling web services that the idiomatic approach is to use Scala's asynchronous mechanisms to call web services. So if I'm using Java libraries for, say, downloading images from S3 and uploading to Facebook and Twitter (restfb and twitter4j), does this make for a highly inefficient use of resources (what resources?) or does it not make much difference (or no difference at all)?
If it makes a difference, how would I go about making something like the following asynchronous? Is there a quick way, or would I have to write libraries from scratch?
Note this will be running on heroku, if that matters in this discussion.
def tweetJpeg = Action(parse.urlFormEncoded) { request =>
val form = request.body
val folder = form("folder").head
val mediaType = form("type").head
val photo = form("photo").head
val path = folder + "/" + mediaType + "/" + photo
val config = Play.current.configuration;
val awsAccessKey = config.getString("awsAccessKey").get
val awsSecretKey = config.getString("awsSecretKey").get
val awsBucket = config.getString("awsBucket").get
val awsCred = new BasicAWSCredentials(awsAccessKey, awsSecretKey)
val amazonS3Client = new AmazonS3Client(awsCred)
val obj = amazonS3Client.getObject(awsBucket, path)
val stream = obj.getObjectContent()
val twitterKey = config.getString("twitterKey").get
val twitterSecret = config.getString("twitterSecret").get
val token = form("token").head
val secret = form("secret").head
val tweet = form("tweet").head
val cb = new ConfigurationBuilder();
cb.setDebugEnabled(true)
.setOAuthConsumerKey(twitterKey)
.setOAuthConsumerSecret(twitterSecret)
.setOAuthAccessToken(token)
.setOAuthAccessTokenSecret(secret)
val tf = new TwitterFactory(cb.build())
val twitter = tf.getInstance()
val status = new StatusUpdate(tweet)
status.media(photo, stream)
val twitResp = twitter.updateStatus(status)
Logger.info("Tweeted " + twitResp.getText())
Ok("Tweeted " + twitResp.getText())
}
def facebookJpeg = Action(parse.urlFormEncoded) { request =>
val form = request.body
val folder = form("folder").head
val mediaType = form("type").head
val photo = form("photo").head
val path = folder + "/" + mediaType + "/" + photo
val config = Play.current.configuration;
val awsAccessKey = config.getString("awsAccessKey").get
val awsSecretKey = config.getString("awsSecretKey").get
val awsBucket = config.getString("awsBucket").get
val awsCred = new BasicAWSCredentials(awsAccessKey, awsSecretKey)
val amazonS3Client = new AmazonS3Client(awsCred)
val obj = amazonS3Client.getObject(awsBucket, path)
val stream = obj.getObjectContent()
val token = form("token").head
val msg = form("msg").head
val facebookClient = new DefaultFacebookClient(token)
val fbClass = classOf[FacebookType]
val param = com.restfb.Parameter.`with`("message", msg)
val attachment = com.restfb.BinaryAttachment`with`(photo + ".png", stream)
val fbResp = facebookClient.publish("me/photos", fbClass, attachment, param)
Logger.info("Posted " + fbResp.toString())
Ok("Posted " + fbResp.toString())
}
My attempt at a guess:
Yes it's better to do things asynchronous; you're tying up threads if you do everything synchronously. Threads are memory hogs, so your server can only use so many; the more that are tied up waiting, the fewer requests your server can respond to.
No it's not a huge issue. With node.js (and Rails? Django?) it is a huge issue because there's only one thread and so it blocks your whole web server. A JVM server is multithreaded so you can still service new requests.
You can easily wrap the whole thing in a future, or do it more granularly, but that doesn't really buy you anything because you're calling the same methods, so you're just shifting the wait from one thread do another.
If those Java libraries offer asynchronous methods, you can wrap those in a future to get the real benefits of asynchrony <-how to do?. Otherwise yes you're looking at writing from the ground up.
Don't really know if running on heroku matters. Is one dyno == one simultaneous request?

Answer 1

I think it's best to do these requests asynchronously for two main reasons:
high latency (network calls)
failures
With Play, you should use the Akka actors to make your actions it provides great ways to deal with these two concerns.
The problem synchronous code is that it will block the web server. So it won't be available to other requests. Here we will make the wait in other threads unrelated to the web server.
You could do something like:
// you will have to write the TwitterActor
val twitterActor = Akka.system.actorOf(Props[TwitterActor], name = "twitter-actor")
def tweetJpeg = Action(parse.urlFormEncoded) { request =>
val futureMessage = (twitterActor ? request.body).map {
// Do something with the response from the actor
case ... => ...
}
async {
futureMessage.map( message =>
ok("Tweeted " + message)
)
}
}
Your actor would receive the body and send back the response of the service.
Moreover with Akka, you can tune your process to have several actors available, have a circuit breaker ...
To go further: http://doc.akka.io/docs/akka/2.1.2/scala/actors.html
Ps: I never tried play on Heroku so I don't know the impact of a single dynamo.