Is there any distributed middleware, like JXTA or JMS, for Scala?
I'm looking for a middleware that provides discovery, name service, service publications, availability verification, groups and so on, for Scala language.
Akka stack has many features , if you want AMQP reference then http://doc.akka.io/docs/akka-modules/1.3.1/modules/amqp.html
I just wrote this to help you use JMS in Scala, should you so wish
https://github.com/fancellu/jmsScala
I'm not sure what you mean by "for Scala". Since it runs within the JVM, with very good interoperability with Java, you can just use whatever Java-based facilities float your boat. In some cases there may be Scala "skins" available, but it doesn't matter a lot. For instance, here's an article on using ActiveMQ from Scala, with no Scala skin.
I think I would go further than Viral and say that Akka is the answer you're looking for. It doesn't provide all the features you mention (or at least takes a different approach to some of them) but it's a very powerful middleware suite that distributes well and is getting a lot of use and attention.
Related
What I want to do: I want to add communication capabilities to a couple of applications (soon to be jar libraries for Java) in Scala, and I want to do it in the most painless way, with no Tomcat, wars, paths for GET requests, RPC servers, etc.
What I have done: I've been checking a number of libraries, like Jetty, JAX-RS, Jackson, etc. But then I see the examples and they usually involve many different folders for configuration, WSDL files, etc. Most of the examples lack a main method and I don't have a clear picture about how many additional requirements may they have (e.g. Tomcat).
What I am planning to do: I'm considering to simply open a socket on the "server" to listen, then connect with the "client" and transfer some JSON, in both directions. This should be fairly standard so that I can use other programming languages in compatible ways (e.g. Python).
What I am asking: I would like to know whether there is some library that makes this easier. Not necessarily using raw sockets, but setting up some process communication in just a few lines, maybe not as simple as Node.js, but something similar.
Bonus: It would be cool to
be able to use other programming languages (e.g. Python) by using open standards
have authentication
But I don't really need any of those at this point.
I think you need RPC client/server system, I would suggest to take one of these two:
Finagle - super flexible and powerful RPC client/server from Finagle. You can define your service with Thrift, and it will generate stubs for client/server in scala. With Thrift it should be straightforward to add Python support.
Spray - much smaller library, focused on creating REST services. It's not so powerful as Finagle, however much easier. And REST allows you to use any other clients
Remotely - an elegant RPC system for reasonable people. Interesting and very promising project, however maybe difficult to start with because of extensive Scalaz+Shapeless+Macro usage
Honestly if you want something that is cross-language compatible, simple, straightforward, and concise then you do not want to use plain old sockets!
Check out dropwizard. It is amazing and I use it for small and large projects alike! It is usually configured by no more than a single configuration file. It supports authentication too!
Out of the box it gives you really great inter-process communication over JSON (using Jackson) and much much more. There is also pretty decent Scala support for dropwizard.
If you must roll your own then I'd recommend using Jackson for JSON parsing. It's super simple to use and also has great scala support.
If you've got a "controlled" use case where the client and server are on the same LAN and deployed in tandem, I'd (controversially) recommend Java RMI; it's dumb and JVM-specific (and uses a Java-specific protocol), but it's very simple to use.
If you need something more robust and cross-language, I'd recommend Apache Thrift. You write your interfaces in a platform-independent interface definition language, and it's very clear which changes are compatible and which are not; the thrift compiler generates skeleton interfaces for you to use, and then you just write an implementation of that interface and a couple of lines to start the server (as you can see from the example on the homepage). It's also got good support for async implementations if you need the performance. Thrift itself is reasonably standard and cross-platform, with its own binary protocol, or you can use JSON as a transport if you really want to (I'd recommend against that though).
RabbitMQ provides one easy way to do what you want without writing a server and implementing your own persistence, flow control, authentication, etc. You can brew or apt-get install it.
You start up a broker daemon process (i.e. manages message queues)
In the Scala producer, you can use Maven-provided Java API to send JSON strings without any fuss (e.g. no definition languages) to specified queues
Then in your other Scala program, connect to the broker, and listen for messages on the queue, and parse the incoming JSON
Because it is so popular, there are many tutorials online for different patterns you may want to use to distribute the messages, e.g. pub/sub, one-to-one, exactly-once delivery, etc.
I'm developing a gateway that will be located between mobile web apps and web services on backend systems. The purpose of this gateway is to protect web apps against changes to backend web service api's, to introduce concurrency, transform messages, buffering, etc.
My proposed architecture is as follows:
Platform independent mobile web apps using PhoneGap (done)
Gateway is a web service using Scala for business logic and ZeroMQ for message passing (new)
Backends are existing web services (existing)
The gateway is purely responsible passing, translation, aggregation, etc. of messages and does not need to keep state or do user authentication at this point - it's simply responsible for being a single interface that knows how to talk to mobile apps on the one side and one or more services on the other side.
I'm strongly considering using Scala as the development language because it seems well suited for this type of application, but what would be the correct architecture for such a Scala service? I looked at frameworks such as Lift and Play and also considered doing a simple "java" based web service and just use Scala as to implement my business logic. I believe strongly in keeping things as simple as possible. I'm wary of complicated setups and thousands of lines of dead code in frameworks that may never be used. On the other side, limiting oneself to a 'role your own' solution and creating a lot of work and having to maintain code that may have been part of existing solutions is also not ideal.
Some things to consider: I'm the architect and developer, but my knowledge of Scala is limited to the first half of "Programming in Scala, Second Edition". Also, my time is very limited. Still, I want to get this right the first time.
I'm hoping that some clever gent or dame will provide me with insights to this type of solution and maybe a link or two to get started quick. I really need to get going FAST, but hope that the experience or insights of another professional could help me avoid pitfalls along the way. Any insights to development environments and tools will also be helpful. I have to develop on Mac (company rules) but will be deploying on Ubuntu server. I'm currently juggling between installs of Eclipse or Idea as IDE's and the scala compiler or sbt for building.
UPDATE
Thanks for all the potential answers below. I had a look at each and every suggestion and all of them have merit. The problem is now to bet on the right horse. Spray is possibly the simplest solution to the problem, but I also found Finagle. It seams like a stunning solution to my problem. I'm a bit concerned that it is built on top of Netty instead of Akka. Does anyone see any problem with that. I was hoping to keep my solution as purely Scala as possible, but Finagle appears to be the most mature of the lot. Any ideas?
It might be worth taking a look at Akka, which provides a lightweight framework for writing concurrent, fault-tolerant applications on the JVM, as well as useful built-in abstractions for writing web services. In addition, the Spray project looks like it could be useful for your purposes, providing HTTP server and client libraries on top of Akka (Although, unlike Akka itself, I haven't used this so myself so can't vouch for it).
If you're using a web service that is purely for other services, then an Akka based service like Spray or Blueeyes is probably your best bet. You can use Jerkson to do the JSON mapping to and from your case classes and use Akka-Camel to link to ZeroMQ.
As mistertim says, Akka is a good option. I'm biased, but I think Lift and its RestHelper is a fantastic way to go also (I've built several APIs for iPhone apps using Lift). Beyond those, I know several people who are very happy with Unfiltered.
Unfiltered is very interesting option if you want to keep it lightweight, as is not a full blown web framework.
To be honest, the documentation can use more detail, but to provide a web service you just write:
object MyService extends unfiltered.filter.Plan {
def intent = {
case req # GET(Path ("/myendpoint")& Params(params)) =>
for {
param1<-params("param1")
param2<-params("param2")
} yield ResponseString(yourMethod(param1,param2))
}
override def main(args:Array[String]) = unfiltered.jetty.Http.anylocal.filter(MyService).run();
}
Very useful if you don't want to use a framework and don't want to mix java/java annotations with Scala (otherwise, you have the usual java solutions)
Still, I would recommend to look into Akka if you think you think actors will fit your problem
If your system is middle-ware and get the things done quickly, then Spray will be a best. Spray was developed on top of Akka. Akka has ZeroMQ support.
in future if you are going to add some other web like module along with your middle-ware, then choosing Lift + Akka will be better because, Lift also provides Spray like web service stuff + it will be easy to start developing other modules.
You can choose SBT as your build, there are some project templates, template generation tools available, so you can get the project build very quickly.
In my experience , SBT + InteliJ Idea works well, have a look on sbt-idea plugin
Spray Project Template
Lift Project Template Generator
For a project I'm using both Scala and Riak (two things I have never worked with before ;) ).
Google searches seem to suggest using Riakki. However, it seems like that particular library hasn't been maintained since 2009 and doesn't even compile on my system. There is a more up-to-date fork on GitHub that does seem to work with more recent Scala versions. But Riakki seems to depend on Jiak, which has been deprecated since february of last year.
Seems like the only reasonable choice would be to use the official Riak Java-library from Scala. That's certainly possible, but I'd like to do things the Scala-way as I'm trying to learn the language. Having to interface with a Java-style API might ruin a bit of the fun. Writing my own wrapper sounds like it will be too much work.
tl;dr: I want to use Riak from Scala. What are other people using?
edit: just found Ryu (can't link to it - annoying limit on amount of hyperlinks per question for new users). Doesn't seem all that mature though.
Stackmob recently opensourced Scalariak.
Scaliak is a scala-ified version of the High-Level Riak Java Client w/
a Functional Twist. It is currently being used in production at
StackMob.
Scaliak is currently feature incomplete vs. the original High-Level
Riak Java Client. What is currently supported are mostly features
being used in production (there have been a few features implemented
and subsequently not used).
There is also Raiku which states that it is async.
I'm in the same bucket - excuse the bad pun - although I have some experience with Scala. I'm thinking of using the official Java client.
When you are toiling up a steep learning curve, you don't need to be dealing with incomplete and potentially wobbly API's. In my experience, using Java API's from Scala is minimally painful.
I think there'll be enough delight in playing with our new Raik toy that we'll forget about whatever un-Scala-ish foibles the Java API inflicts upon us. All the best.
I'm the author of yet another Scala Riak client, simply called riak-scala-client. It is based on Akka and Spray, it is not built on top of the existing Java client, and most importantly it is completely non-blocking.
Check it out at http://riak.scalapenos.com and let me know what you think.
Say the use scala to process incoming email etc.
In what context is (or could) they be running scala?
Can it run inside its own daemon?
Can it run inside of tomcat?
Or would you use it in a cron job?
or is it all of the above? :)
Sorry this is an open question, and I don't know much about scala, but I just want an idea of how one could utilize scala and under what context it can run in.
Scala is a general purpose language, and can be used pretty much everywhere that doesn't have restrictions of its own. One limitation it has is that it must be backed by a Java VM (or .Net if you go with the experimental stuff), which can bring limitations of its own.
When people said "backend", they usually mean using Scala to provide services to other software, leaving user-facing layers to other languages. This combination leverages the speed advantages of static typing in Scala, to benefits in the speed of development and interaction other languages -- PHP or Ruby, for example -- might provide for UI front ends.
I am looking into something lightweight, that, at a minimum should support the following features:
Support for easy definition of actions through metadata
Wrapper that extracts parameters from request into clojure map, or as function parameters
Support for multiple forms of authentication (basic, form, cookie)
basic authorization based of api method metadata
session object wrapped in clojure map
live coding from REPL (no need to restart server)
automatic serialization of return value to json and xml
have nice (pluggable) url parameter handling (eg /action/par1/par2 instead of /action?par1=val1&par2=val2)
I know it is relatively easy to roll my own micro-framework for each one of these options, but why reinvent the wheel if something like that already exists? Especially if it is:
Active project with rising number of contributors/users
Have at least basic documentation and tutorial online.
First of all, I think that you are unlikely to find a single shrinkwrapped solution to do all this in Clojure (except in the form of a Java library to be used through interop). What is becoming Clojure's standard Web stack comprises a number of libraries which people mix and match in all sorts of ways (since they happily tend to be perfectly compatible).1
Here's a list of some building blocks which you might find useful:
Ring -- Clojure's basic HTTP request handling library; all the other webby libraries (for writing routes &c.) that I know of are compatible with Ring. Ring is being actively developed, has a robust community, is very well-written and has a nice SPEC document detailing its design philosophy. This blog post provides a nice example of how it might be used (reacting to GitHub commits).
Sandbar -- currently an authentication library, more types of functionality planned; under development.
Compojure -- a mature and robust library which provides a nice DSL for writing routes to be used on top of Ring. This will give you the nice URL parameter handling.
Compojure-rest -- "a library for building RESTful applications on top of Compojure". Compojure-rest is, as far as I can tell, in its early stages of development; perhaps you might see this as an opportunity to influence its design. :-)
For dealing with XML, there's clojure.contrib.lazy-xml (and the helper library clojure.contrib.zip-filter.xml) and Enlive (the built-in clojure.xml namespace is currently not very usable); these would be used in tandem (though for your purposes the former might suffice).
For JSON, there a library in contrib and clojure-json (and I think there was at least one other lib I seem to be forgetting now...); pick the one you like best.
All of will be perfectly happy with a REPL-driven development style (see the accepted answer to this SO question for a Ring trick which is very much to the purpose here). I suppose the above collection of links does leave a few blind spots (in particular, the authentication story is still being ironed out, as far as I can tell), but hopefully it's a good start.
1The only single-package solution for building webapps in Clojure that I know of is Conjure, inspired by Rails; unfortunately I have to admit that I don't know much about it, so if you feel interested, follow the link and look around the sources, wiki &c.
While building my first Clojure rest service I found myself asking often the same question. The Clojure Toolbox helped me a lot: http://www.clojure-toolbox.com/
If you are looking for some sample, real-world, illustrative code to get you started, then you could study this clojure-news-feed on github project which demonstrates how to implement a non-trivial RESTful web service with compojure/ring that wraps both SQL (postgresql or mysql) and NoSQL (cassandra), search (solr), caching (redis), event logging (kafka), connection pooling (c3po), and real-time metrics via JMX.
This blog about Building a Scalable News Feed Web Service in Clojure provides a good introduction. I ran some load tests against this service on a humble AWS deployment and got about eighty transactions per second with less than a half second average latency per transaction.
Take a look at liberator library http://clojure-liberator.github.io/liberator/ It's noy a standalone solution, buy very good for rest service definition.
Just to provide an updated answer to this old question, currently (in 2018) I think Luminus provides an excellent starting point. It's using many of the libraries (ring, compojure, etc.) mentioned in previous answers, is modular and as close to "single package" as you can get with Clojure. Specifically for REST, take a look at compojure-api. Luminus recommends buddy for authentication, I've had good success using it both for traditional session-based auth as well as Oauth and stateless JWTs.