I want to use AWS SQS in my Play 2.4 project.
There are two options at moment:
There is a SQS wrapper called https://github.com/kifi/franz which supports reactive way of using that. But it seems not quite popular on Github not sure how mature it is? Whether the developers will continue to maintain it.
There is Java SQS SDK, but it doesn't support Scala Future (reactive way). If I want to make it non-blocking, could I use Akka or?
I haven't used SQS yet but I did have a similar need for DynamoDB. I tried using a third party Scala specific library -- bad idea (buggy, bad api, missing features). In the end I wrapped the AWS Java SDK. Making it 'reactive' is pretty easy and is similar to most things you would do in Scala. Call the AWS functions within a Future body (Future {}) and use functional constructs to process the result (map, reduce etc..).
Related
In our Scala/Play application we use activiti. (also experimenting with camunda) users can create workflows (shown in this picture http://camunda.com/ ). All calls to these external workflow engines are wrapped in Scala Future (activiti and camunda APIs are all Java blocking APIs).
is there any library to implement workflows totally using Akka/Actors avoiding heavy toolkits like activiti/camunda? Or ideas how to best use Akka with activiti/camunda ?
You could try and use the Akka FSM dsl to do the same bypassing activity and also blocking apis. see http://doc.akka.io/docs/akka/snapshot/scala/fsm.html
Note that camunda has very powerful asynchronous continuation features which allow you to delegate any long-running processing to background threads. This allows very flexible configuration of "how much work" is done synchronously in the client (possibly HTTP) thread. This can give you a good balance between performance and fault tolerance.
I know of the existence of the Catify BPMN Engine, built using Akka (Java). I do not have any experience with it, nor do I know for sure whether API calls are asynchronous, but I would expect so. Since it is written in Akka it should combine well with Play!.
I have not used Finagle nor Akka in practice, but I have been reading a lot of about them.
Finagle being a RPC system and Akka a toolkit for highly concurrent applications, why all the people compare them as two possible solutions which cannot be used together? All searches I've done propose to use one or the other, no one proposes to use them together.
Finagle, for example, has a very interesting way of defining endpoints via thrift and its IDL. With this IDL we could define a custom endpoint and through scooge or whatever code generation tool, it would be possible to have a service with no effort. Also a client to connect to this service is created with a lot of common client issues automatically resolved (reconnection, timeout, retries, load-balancing, connection-pooling, ...).
Akka instead, solves a lot of concurrency headaches and it scales extremely well without all the complexities of hand controlled threading.
As a summary, why not use them together?:
Finagle + Thrift (with its IDL): It facilitates service design and development as well as deployment (which includes ease of scaling-out).
Akka: It uses all the server power through its Actor system and it scales extremely well if I change server properties (for example if it's deployed on EC2 and I convert my node from m1.small to m1.large).
What do you think?
NOTE: Asume that the issue of mapping Futures and Promises is resolved, as well as a mismatch between FuturePools and ExecutionContexts. The pattern would be to convert Finagle to the scala way of using Futures.
You are right in that service discovery and service implementation are orthogonal concerns, and I can follow your argument about using Finagle for the former and Akka for the latter. You could in principle use the two together without seeking a grand unification of futures, since you only need to send the service’s reply back to the requesting Actor in a message, i.e. you would need to add your own little “pipeTo” pattern on top of Twitter futures.
As a Java developer, I'm used to use Spring Batch for batch processing, generally using a streaming library to export large XML files with StAX for exemple.
I'm now developping a Scala application, and wonder if there's any framework, tool or guideline to achieve batch processing.
My Scala application uses the Cake Pattern and I'm not sure how I could integrate this with SpringBatch. Also, I'd like to follow the guidelines described in Functional programming in Scala and try to keep functional purity, using stuff like the IO monad...
I know this is kind of an open question, but I never read anything about this...
Has anyone already achieved functional batch processing here? How was it working? Am I supposed to have a main that creates a batch processing operation in an IO monad and run it? Is there any tool or guideline to help, monitor or handle restartability, like we use Spring Batch in Java.
Do you use Spring Batch in Scala?
How do you handle the integration part, for exemple waiting for a JMS/AMQP message to start the treatment that produces an XML?
Any feedback on the subjet is welcome
You don't mention what kind of app you are developing with Scala, so I'm going to wild guess here and suppose you are doing a server side one. Going further with wild guessing let's say you are using Akka... because you are using it, aren't you? :)
In that case, I guess what you are looking for is Akka Quartz Scheduler, the official Quartz Extension and utilities for cron-style scheduling in Akka. I haven't tried it myself, but from your requirements it seems that Akka + this module would be a good fit. Take into account that Akka already provides hooks to handle restartability of failed actors, and I don't think that it would be that difficult to add monitoring of batch processes leveraging the lifecycle callbacks built into actors.
Regarding interaction with JMS/AMQP messaging, you could use the Akka Camel module, that provides support for sending and receiving messages through a lot of protocols, including JMS. Using this module you could have a consumer actor receiving messages from some JMS endpoint, and fire whatever process you want from there, probably forwarding or sending a new message to the actor responsible for that process. If the process is fired either by a cron style timer or an incoming message you can reuse the same actor to accomplish the task.
Hey I love Play Framework Scala and I also am falling in love with Firebase. I was wondering though, I'm planning on building an app using AngularFire and I'm going to need to do some server-side logic/computation and make some server-side queries to Firebase. Is this possible to do with a Play Framework Scala setup? If so what is the recommended approach? If not, is it coming? If so when? I think it's so cool that the Firebase guys used Scala to build Firebase, but I'm bummed there is no Scala API to work with (that I can see). Maybe I could use the Java API somehow, but write still write the app in Scala? Any help would be great. Thanks!
Scala is highly interoperable with Java (compiles to the same bytecode) so you should be able to use the Java API straight-up without any issues.
While some libraries add Scala-specific wrappers to make an API more idiomatic and pleasant for a functional programming style and to smooth some rough edges, it's often not strictly necessary.
If for some reason you didn't want to use the Java client libs, you could also interface with the Firebase REST API via Play 2's very convenience and succinct web services library.
Should be no problem with either their Java SDK (when using Firebase on the backend) or the JavaScript SDK (when using it on the client). But you won't have native support for Scala or Play, especially no support for Iteratee/Enumeratee in Play.
The Java API looks quite good and seems to be event driven. So it should be no problem to integrate it in Play in a scalable way.
I need to build very concurrent web service which will expose REST based API for JavaScript (front end) and Rails (back end). Web service will be suiting data access API to MongoDB.
I already wrote an initial implementation using NodeJS and would like to try Scala based solution. I'm also considering Erlang, for which every web framework is actor based.
So I'm looking for web framework explicitly build using Actors in order to support massive load of requests I'm very new to Scala and I don't quite understand how Actor might work if almost all frameworks for Scala are based on Java servlets which creates a thread on each request which will just exhaust all resources in my scenario.
If you're really going to have 10k+ long active connections at a time, then any standard Java application server/framework (maybe, except for Netty) will not work for you - all of them are consuming lots of memory (even if any kind of smart NIO is used). You'd better stick to a clustered event-loop based solution (like node.js that you've already tried), mongrel backed with zeroMQ, nginx with the mode for writing into MQ polled by Scala Actors, etc.
Among the Scala/Java frameworks, Lift has a good async support for REST (though it's not directly tied to actors). OTOH, LinkedIn uses Scalatra + stdlib actors for their REST services behind Signal ,and feels just fine.
Another option is Play framework. The latest 1.1 release supports Scala. It also supports akka as a module.
As far as Scalatra itself, they have been working on a new request
abstraction called SSGI (akin to the Servlet/Rack/WSGI/WAI layer),
that they said should ennable them to break from solely running as a
Servlet and also run on top of something built with Netty. See thread here.
http://github.com/scalatra/ssgi
There's some other interesting frameworks at the Scalatra level of simplicity since designed from the ground up to support asynchronous web services (won't tie up a thread per request):
https://github.com/jdegoes/blueeyes - Not a servlet; built on Netty.
("loosely inspired by ... Scalatra")
http://spray.cc/ - Built on Akka actors, Akka Mist. Servlet 3.0 or Jetty continuations
("spray was heavily inspired by BlueEyes and Scalatra.")
And at a lower level:
https://github.com/rschildmeijer/loft - "Continuation based non-blocking, asynchronous, single threaded web
server."
Not production-ready, but rather interesting-looking. Continuations require the compiler plugin.
http://liftweb.net/ Indeed, a request starts off as a servlet, but then lift uses comet support found in many servlet containers to break away from the thread, keeping the request context (which the container then doesn't destroy) which then can be used to output data in actors.
http://akkasource.org also has support for rest, but it will block the thread until the actor finishes with its work