Related
Note: Unfortunately this question was closed, but I'm trying to maintain it for if someone else comes along with the same question.
I've been looking for a good solution to developing a service in Scala that will sit between mobile devices and existing web services.
The current list of viable options are:
Finagle
Spray
BlueEyes
Akka
Play2 Mini
Unfiltered
Lift
Smoke
Scalatra
There are probably more options out there. How does one decide which one to use? What are the traits (excuse the pun ;-) of a good Scala middleware choice. On the one side, I would like to go for Akka, because it is part of the TypeSafe Scala stack, but on the other, something like Finagle has a rich set of libraries and makes plumbing so easy. Spray looks nice and simple to use.
Any advice, insights or experience would be greatly appreciated. I'm sure someone out there must have some experience with some of these that they won't mind sharing.
UPDATE:
I would love for this question to be reopened. A good answer to this question will help new Scalateers to avoid related pitfalls.
UPDATE 2:
These are my own experiences since asking this question:
Finagle - I used Finagle for a project and it's rock solid.
Spray - In my latest project I'm using Spray and I'm extremely happy. The latest releases are built on Akka 2 and you can run it directly with the Spray-can library which removes the need for a web server. Spray is a set of libraries, rather than a framework and is very modular. The Learn about Spray: REST on Akka video gives a great overview, and this blog at Cakesolutions shows a really nice development approach and architecture.
UPDATE 3:
Life moves pretty fast. If you don't stop and look around once in a while, you could miss it. - Ferris Bueller
These days the choice has become simpler. In my humble opinion Spray has won the battle. It is being integrated into Akka to become the next Akka HTTP. I have been using Spray now on multiple projects and can honestly say that it's fantastic and best supported software I have ever encountered.
This does not answer the initial question, but at least gives some indication on why Spray seems like the best choice in most cases. It is extremely flexible, non-blocking and very stable. It has both client-side and server-side libraries and a great testkit. Also, have a look at these stats to get an idea on performance: Web Framework Benchmarks
I personally started with spray a long time ago and tried everything else there was out there for Scala. While Scala, spray, akka, shapeless, and scalaz certainly have a bit of a learning curve, once you start digging in and really learning how you are supposed to use the technologies, they make sense and I immediately saw the benefits especially for the kind of work I'm doing right now.
Personally I think nothing really stands up to spray for building both servers, rest apis, http clients, and whatever else you want. What I love about spray is that they built with akka in mind. It may have been a really early project when I first started using it, but the architecture made sense. Those guys knew what they were doing in terms of exploiting the benefits of using an actor model and not having any blocking operations.
While actors might take a bit getting used to, I do like them. They have made my systems very scalable and cheap to run because I don't need as beefy hardware as in the past. Plus, spray has that spray-routing DSL so making a rest api is relatively simple as long as you follow the rules ... don't block. That of course means don't go and pull in apache commons http client to make client requests from the api or actors because you will be going back to blocking models.
So far I am very happy with spray, typesafe, and akka. Their models just naturally lend themselves to building very resilient systems that come back up on their own if anything should happen and you take a fail-fast approach. The one beef that I have with spray (and it's not spray's fault) is the damn IDE support for the routing DSL. I absolutely despise Eclipse and have always been an IDEA user. When I started using the Scala plugin, everything seemed ok. Then my routing dsl naturally evolved into way bigger beasts. Something about the way IDEA parses that code makes it shit its pants anytime it encounters anything with spray-routing or shapeless. It's to the point where it's unusable (I type 2-3 letters and have to wait 5 minutes to regain control).
So, for any spray-routing or heavy shapeless code, I fire up emacs with ensime, ensime-sbt, and scala-mode2. Now if I could only get a Cassandra library with the quality of astyanax and built using a more non-blocking architecture.
Here you can find a great list of scala resources with a brief description of all the alternatives you listed.
From my own experience, I use Scalatra and it is tiny, simple and effective for things like uri mapping and calling web services.
I know Scala. I've used mongoDB, redis, sbt, ... backend stuff.
I know basic HTML, CSS and JavaScript but have never done real web development.
I don't know what AJAX is. I don't really know any frontend stuff.
What should I learn before I start trying to tackle Lift?
If you know Scala, you are pretty much set on the programming side.
On the other hand, Lift requires HTML/CSS templates. In fact, it is completely separate from programming, so that a professional web designer can work on them, while the programmer works on the code. If you can revise your HTML/CSS stuff, it will help.
You don't need to know AJAX -- that part Lift takes care of for you.
So, it seems you are mostly set. I strongly recommend Timothy Perret's Lift in Action, from Manning. Though the book hasn't been released yet, you can get the Manning Early Access Program (MEAP) for it and, as it happens, the whole book has been written already.
If you already know Scala, learn some basics of web development and how Java handles it:
HTTP protocol
Servlets (also know what JSP is)
AJAX
Now i know that this one is actually not a very technical question but one that has been bothering me for some time. Actually we are using a lot of C++ and PHP at our company and some of our developers are really hoping for a new and modern language to come by to help us getting more productive. I have been talking about what scala can do and the other coders seem to gain some interest in the language. The tough job is, how do you convince your boss to consider scala as a language for the company. I saw the presentation "Sneaking Scala into your company", but it deals with the situation that you are using Java at your company which we don't.
How do you fight of the usual "that is just esoteric stuff" and "we can already do that in $LANGUAGE" arguments. I was planing to give a talk about Scala, and since I don't have much time I need ideas how to get people interested in the language rather then setting of reactions like "currying? we can already do something like this with boost::bind".
How did you guys do it?
Regards,
raichoo
EDIT: Gave my talk yesterday, people were very excited. My company is going to give it a try! Thanks for all your suggestions.
If you don't already have killer arguments, what are you basing your reasoning on that Scala will make your company more productive?
Don't like something then hunt for reasons to use it at work. Let the reasons speak for themselves..
"A hammer looking for nails"
Using it to do some stuff around the side, as datamigrations, testing and similar things will make sure the necessary experience is built and can give it some exposure.
ScalaTest is really nice to help with acceptance/integration testing. (Yes, I know it is nice for unit testing, but I do not see that immediately happening with C++/PHP target code, and it would probably be unwise).
Proof of Concept and other Prototypes are great for 2 reasons
1) It showcases the capabilities
2) You are certain they will be thrown away if you have to reimplement them in C++/PHP
Now a bad time to introduce Scala would be when you REALLY need it : hopes will be high, it will not immediately work as intended, hopes are dashed and everybody will blame Scala. As a result it will be burnt for a long time in the organisation.
Sooner or later some suit will think it was his idea to introduce Scala and use it on a formal project. If that project is moderately successful, then it is sold.
These kind of changes are complicated people issues, and the harder you push, the harder you will face push-back. On the other hand the persistent mind can move mountains.
Redo some of your work related code in Scala and compare KLOC, code structure and performence, if it looks and works better, show it to your peers and your managers.
In other words:
Talk is cheap. Show me the code.
-- Torvalds, Linus (2000-08-25)
In case of our company (and I assume, many companies share the same scenario), move to Scala (from Java) was initiated by tech people, who 1. wanted to work more productive writing code (living in the 21st century utilize modern approaches), 2. have less troubles building concurrent applications (Actors concept promoted by Scala is a way simpler than Java thread-based concurrency) 2.1 have a simpler way of building scalable staged event driven architectures.
In our company, transition to Scala was more or less simple, because Scala was literlly sold to business people as a library to Java :) -> from their POV, we're still using the same platform (JVM), application servers, etc., but developers are having more fun from their work, and therefore, are more inspired and work more efficiently.
Maybe you could pitch Scala by showing off the suite of tools that is used for development? For example, if you are not already using Eclipse in your company, show your execs a demo of what a modern IDE can do for your productivity.
There is a book called "Fearless Change" (Linda Rising) that describes a pattern language for "powerless leaders" (I LOVE that role title!). SE-radio had a really motivating interview with the author: http://www.se-radio.net/podcast/2009-06/episode-139-fearless-change-linda-rising. Listen up on that interview to collect a few non-technical strategies that can help you in this struggle!
I haven't used Scala yet for any real business code, but I know people who have.
One group used it to write a tool to analyze log files. So they didn't use it for mission-critical business code, but for a non-critical tool to support the project.
Another person I know is an architect and he just went and wrote some Scala code on his own for some production code without telling his manager. After the code was deployed successfully he did tell it. One of the things he mentioned is that because Scala runs on the JVM, the people who support the application don't even notice - to them, Scala is just another library that's included with the application (they were already used to the JVM). Ofcourse this approach is risky and not everybody will be in the position or be willing to do this.
You could start small - use it as your personal preferred scripting language for small things that you need yourself. Tell your fellow developers about it and make them enthusiasts too. If they also start using it then you can step it up to make some side code for your project (such as for example that log analyser tool).
This isn't a really easy task. I would concentrate on the fact that you will be able to produce code and therefore products faster and with a higher quality. That's always the two reasons, business wants to hear from you and will listen to.
Maybe you can show an example of 1-2 very small projects you did in your company with C++/PHP and compare the effort, quality etc. with a similar/the same implemenation in Scala? This would be very impressive and should also convince people who are not on the coding side.
There was a very good talk at Scala Days 2010 by David Copeland:
Sneaking Scala into your organisation
The executive summary: Testing. You can use Scala for testing without affecting release code.
we are considering Scala for a new Project within our company. We have some Junior Programmers with only PHP knowledge, and we are in doubt that they can handle Scala. What are your opinions? Some say: "Scala is a complicated beast!", some say: "It's easy once you got it." Maybe someone has real-world experience?
"My coworkers will not understand Scala" is simultaneously overstating its difficulty and insulting your coworkers.
Scala is not that difficult. It's just another programming language. Any trouble that junior programmers have with Scala is going to be more or less the same trouble they would have with any other language.
Your coworkers are smart. Of course, I don't know them, but it's a pretty safe bet unless your company is the kind of organisation that hires stupid people, in which case, you have bigger problems.
That said, at my company we have some core products developed in Scala, and we don't find that people have any more trouble with it than Java. The code is generally more clear and concise, easier to generalise and reuse, etc.
I guess Scala could be used as a "beginners" language. Even though there are tricky ideas behind it, you dont have to use/explain them in the beginning. If you explain pure OO with Scala, I would say it is straight forward and easily understandable. As Scala reduces a lot of code overhead from other languages it might even be easier to learn concepts with Scala than with Java/C++.
A major drawback I see with Scala as a beginner language is the lack of documentation. Don't get this wrong, the official Scala doc is very good and also the few books that are available are quite useful to get the details of the language, but those have not been written for beginners. For example in Java you find hundreds of books titled something like "Learning OO with Java" you wont find that for Scala which may be a show stopper.
As Hannes mentioned, only do new language introductions within research projects and not productive or even flagship projects. If you have some juniors, that makes the situation even better, take some internal tool, you always wanted and needed and let them create it during a research project. This is also a nice opportunity to experiment with different development-processes. And your juniors most probably like to be challenged and will deliver a good prototype and a very well proofed opinion if Scala can be used as a beginner language.
I believe that most people moving to Scala are experienced and enthusiastic coders. I'd suggest that you get in-house experience with a Scala project with your senior programmers first before forming a strategy for mentoring your junior coders. I'd also suggest that you only involve people who are eager to join in.
I would advocate it. But with the proviso that you have clear guidelines on what language features are acceptable for your team. For example, coding primarily in an imperative style (which is familiar for Java-trained people), or perhaps limiting the employment of recursion or closures.
Also plan for seniors to mentor the juniors. This may take the form of any combination of: pair programming, code reviews, info sessions, regular discussion forums, etc.
The opportunity that scala presents for vastly improved coding on the JVM is too great to pass up. When your seniors get into it I would not be surprised if they find renewed passion for development. When your juniors get into it they will be learning best practice JVM development from the start.
If you choose to go this route, perhaps they'll find easier to use the Scalate framework than a more traditional one like Lift, since it allows mixing HTML with Scala, much in the same way as PHP works.
Scala is a very 'normal' programming language. Any programmer should be able to learn this language. The people that have difficulties learning Scala mostly are experienced with imperative languages and are surprised by the functional concepts. So unexperienced programmers may learn it even faster. In my opinion should be no problem, to assign it to juniors. From a management point of view, I would assign a junior and a senior developer as a team (or more of both, depending on the size of the project).
I think it depends whats more importent for you. If you want to learn as possible about OO Programming and the standard stuff its a bad idea.
But what you really give them is a opportunity to learn something really cool and unique. Witch can be good motivation. Scala has many cool stuff in it. If you can handle Scala you can handle a lot of otherstuff as well.
Talk to the Programmers (all of them) and tell them why you wanne to use scala. Ask if the have to motivation to make and learn something not everybody can do and go the extra mile?
If the are go with it!
My initial thought would be that Scala will be too heavy for them but then I guess because Scala is an OO/Imperitave/Functional hybrid, one could introduce them to the OO/Imperative part of scala until their comfortable, but then again they will probably have bad PHP habits in Scala since scala authors mostly prefer the functional style over the imperative one.
So, it could work, but I would do it for a research project, and definately not for a flagship one.
Edit: Perhaps this should be said also: It seems that functional/OO hybrids like Scala is becoming more popular especially because of how functional languages handles parallel processing as opposed to how we know it in languages like in Java. The amount of cores found in a chip is increasing rapidly, so this is important. However, mentioning PHP, it seems that you are developing web server scripts where threading is less important. PHP doesn't even have threads.
This raises another point. Do you want to develop Scala Web applications i.e. Lift. If so then you have a doubled up learning curve which should also be considered.
Imagine that you would have picked Java and asked whether they could handle Java. If your answer is they could, then they can probably handle Scala.
Scala is only marginally more difficult due to:
No great IDE support. The support ranges from poor to good. Not necessarily an issue for a PHP programmer.
Documentation not as rich as Java
Both Java and Scala have new challenges for a PHP programmer (JVM, new libraries, compiled language, statically typed).
I don't think Scala is a complicated beast, but you do need to understand some of the syntactic sugar and design principles, which would be true of learning Java as well.
Yes, if...
Strategic decision has been made to go with Scala
Company can handle the hit (financial and time) that will come from the steep learning curve.
No, if...
No senior Java, C# or C++ programmers can be put on the project too
Can't find a Scala programmer to act as a lead
Programmers don't have the patience to learn Scala or deal with a language where Jars (libraries) are scattered all over the place, rather than in one or two neat packages like PHP.
*Note: if the junior programmers were C++, C#, or Java Software Engineers rather than PHP, then my answer would be different like, Go for it!
I would not recommend it. My experience of Scala is only from homebrew projects, but I would imagine the currently lousy IDE support, quite frequent API changes and a very flexible syntax (that allows one to hang himself and everybody else participating in the project) would cause a lot of problems in a bigger, more official project.
Give them IntelliJ and throw 'em in the deep end.
Here is a blog post I recently stumbled upon:
http://james-iry.blogspot.com/2008/07/java-is-too-academic.html
It shows that even Java can be too academic to be understood by programmers which have no experience in functional programming. On the other hand, Scala allows to write code the "imperative way", so you can avoid all the FP stuff if you do not understand it. In my opinion, Scala is much more concise than Java, so I guess a "junior programmer" should be able to handle it.
I realize that this may be subjective but I truly need an answer to this and I can't seem to find anything close enough to it in the rest of the Forum. I have read some folks say that the framework (any MVC framework) can obscure too many things while others say that it can promote good practices. I realize that frameworks are great for a certain level of programmer but what about individuals starting out? Should one just focus on the language or learn them together?
I think web development is way more than anyone grasps when they first start getting into it! Read this and know that it is all optional...but required to be really good at what you do.
I suggest that you spend time learning your language first. I would suggest learning C# simply because it is vastly more marketable and it is usually directly supported in most of MS products. By learning C# - programming in ASP.NET, console apps, servers, services, desktop apps, etc. will all be within your reach. You can program for most of the MS products as well as on many Linux type platforms.
Once you have this down then you can move to programming for the web as programming for the web has some intricacies that most other environments don't have. Concepts such as sessions, caching, state management, cross site scripting, styling, client side vs server side programming, browser support, how HTTP works, get vs post, how a form works, cookies, etc. are all at the top of the list of things to learn separately not to mention learning the ASP.NET base frameworks and namespaces.
Once you have the programming language down and then the concepts of web programming I suggest that you pause and learn database design. Don't worry about performance just yet...try to first learn good design. Performance will come next. A good start for you is Access (blasphemy I know). It is easy for a beginner to work with. And it translates into a more robust platform such as SQL Server easily. Learn at the very least some SQL...but I suggest that you learn as much as your stomach can handle. I heard someone say that SQL is like the assembly language of the database. The number one thing that slows an application to a halt is piss poor database design and poor queries. Once you have this knowledge - stuff it away in the back of your mind and take a look at a good ORM. NHybernate is probably best at the moment but is more complex that the basic learner needs. For that reason I currently suggest getting LINQ to SQL up and running as it is SUPER EASY to work with. Then look at Entity Framework (although I still think it sucks...and you should wait till EF 2.0...ERRRRR...now 4.0 released with .net 4.0). Then NHybernate.
Now is the time to start to understand the infrastructure that is required by web development. You may bump your head against this as you learn some of the web programming stuff. But you need to understand the basics of DNS, IIS, load balancers, sticky routing, round robin, clustering, fault tolerance, server hardware setup, web farms, cache farms (MemCached Win32, Velocity), SMTP, MSMQ, database mail queuing, etc. Many people may say you don't need this. That there will be some knowledgeable network admin to help you out here. However they generally know things that impact them...not you. The more you know here the more valuable you will be to the company that hires you.
Now you can get into the details of best practices and design patterns. Learn about the basics such as repository pattern, factory pattern, facade pattern, model view presenter pattern, model view controller pattern, observer pattern, and various other things. Follow Martin Fowler and others for suggestions here. Take a look at concepts such as inversion of control, dependency injection, SOLID principle, DRY, FIT, test driven design, and domain driven design, etc. Learn as much as you can here before moving to the next step.
NOW you can think about frameworks! Start by creating a basic application with ASP Classic (comes with IIS for free!). This will give you a flavor of a no frills web development environment. Take a look at ASP.NET web forms (briefly) to see how MS attempted to make things easier by hiding all the complex stuff (which you now know how to manage on your own from your readings of the above materials!!!). Now you no longer need ASP.NET Web Forms. Move immediately to ASP.NET MVC. The MVC framwork gives you all the power you need to create a good easily manageable web application. If you build something really big no framework for pure web development may be able to deal with what you need. However MVC is way more extensible for such UBER custom scenarios.
Now that you have made it through the journey to ASP.NET MVC you can take a look at things such as Microsofts Enterprise Application Blocks (such as they use at MySpace). Take a look at Elmah error logging (a must have). Look at how to build a custom SiteMapProvider for your MVC site. If you need to get into searching stuff understand Lucene.NET.
And if you made it this far...you are ready to figure out the rest on your own as it comes up! Have fun. There is a lot of room in this space for a person with some understanding of all of the above concepts.
You'll be using SOME sort of framework. The question is, what level do you want to learn at?
You'll probably not care to learn about asynchronous I/O and mutlithreaded vs. select/poll styles of web servers.
So then, your language of choice is going to provide a layer atop this, the languages preferred "web interface" API. For Java it's Servlets, the lowest level you'd typically code at for server side web applications.
You should find what this "lower level" layer is in your language and learn the API at least. You should know basic HTTP like status codes, cookies, redirects, POST vs GET, URL encoding, and possibly what some of the more important headers do.
You'll then come to appreciate what these higher level frameworks bring to the table, and be better able to evaluate what is the appropriate level of abstraction for your needs/project.
Web development requires a certain degree of organization, since it relies so much on separation of concerns. The browser, for example, is designed to display data and interact with the user. It is not designed to lookup data from a database, or perform analysis. Consequently, a web development framework can help provide services that are needed to make the browser experience a practical one.
The nice thing about employing a platform is that it will provide core components essential to the making of any web application that you won't (and shouldn't) have to think about, such as user membership, for example. Many of the design decisions and deep thinking about how to implement these services has already been done for you, freeing you to focus on what you actually want you application to do.
Of the available frameworks, I find that frameworks that implement the MVC (model-view-controller) pattern are very practical. They clearly organize different functions of web development, while giving you full control over the markup presented to the browser.
All that said, you will need some fundamental skills to fully realize web development, such as HTML, CSS, and a core programming language for the actual underlying program, whether you use a platform or not.
I don't think I agree with the Andrew. I don't think learning C is a pre requisite for web development. In fact, learning something like Javascript, Action-script or PHP is often easier due in large part to the vast numbers of sites and tutorials available, and are enough to expose you to the fundamentals of pretty much every programing language. Variable, Conditions, Loops and OOP. I just think learning C# introduces a lot of learning that isn't really relevant to web development such as pointers and memory management.
As for wether you should learn a framework first? Definitely not. Never ever. You need to be able to stand on your own two feet first and be comfortable with HTML/CSS, Server Side Scripting (PHP/ASP/Python/Ruby whatever) and love it or loathe it, but you're going to have to have a decent understanding of Flash and Action-script.
The order in which you learn these is entirely up to you. But my learning plan would go like this...
Start with HTML. It takes about half an hour to get the basics (it's made up of tags with attributes, end of lesson 1) and it's good to get it out of the way first.
Then start leaning CSS. You'll get the basics again, very quickly. But CSS is a minefield so expect to spend the rest of your life figuring it out.
Next up Action-script. Most people wouldn't agree with me, but bear with me. HTML and CSS aren't programming languages. Action-script is. And learning a programing language for the first time is difficult and tedious. The advantage Action-script has over most other languages is that the results are very visual. It's enjoyable to work with and you can sit back and take pride in your accomplishments at regular intervals. This isn't possible with server-side scripting languages or Javascript and there's a whole host of stuff you need to learn to get server side scripting up and running. You can't build space invaders in with PHP for example.
I've changed my thinking here. I would encourage beginners to ignore ActionScript and focus on Javascript. I still believe that being able to see stuff on screen quickly is a good motivator, but I would encourage people to look at canvas tag tutorials and frameworks. Javascript has come a long way since 2009, and is now the lingua franca of programming, so it's incredibly useful. My initial point about HTML and CSS not being programming languages still stands.
Then, you can start with your server side language. At the same time, you're going to have to figure out the database stuff. I recommend PHP and MySQL because it's free.
Again, I've changed my thinking here. I would encourage beginners to use Javascript on the backend (Node.js), and split their database learning between relational databases and noSql solutions such as Mongo.
Then.... learn your framework. Or better yet, roll your own. That's what I've been doing and it's supercharged my learning.
If you're getting into web development, You HAVE to know how those building blocks work. You don't have to be an expert in all the areas, but you should try to become an expert in at least one of them. If you start learning a framework before you get the fundamentals you'll be in a sticky middle ground where you don't understand why things don't work which will infuriate you, and anyone who has to work with you.
you should learn how to use framework because it would be helpfull for u in the future also it is easier to learn.
MVC will help you a lot .. trust me ... i was developing web project not using mvc and it is like mess ... (in the past there are no well know mvc and i never heard about it)
Short version: yes, and then some.
FWIW : This more generic answer may be of use to someone out there.
What: Frameworks take out tedium of using boiler-plate code again and again. They hide complexity and design issues under wizards and conventions. They also use special libraries, design patterns etc. in ways that are far from obvious to a beginner.
So using a framework is good for getting things done without knowing exactly how - like using an ATM without knowing the internals. You just add your code bits in certain places and things 'just work'.
HTML > CSS > Ruby > SQL > Rails/Javascript framework > Libraries would make for a good learning track. Rest you learn as you go along by being curious, hanging out on forums or as extended learning as need arises.
HOW: The problem starts the minute you step outside simple text-book examples (i.e. when you try to get it to do something even a bit different).
Decoding cryptic error messages when it seems like you've done everything right but things still don't work. Searching on error strings in forums may help out. Or just re-starting from scratch.
Reading up articles and books, videos, trial-and-error, hard-work, search-engines, stackoverflow/forums, local gurus, design articles, using libraries, source-code browsing are a good way to climb the learning curve gently and on a requirement basis.
Working-against-the-framework is the number one problem for beginners. Understanding what the framework expects is key to avoiding white-hair in this phase. Having enough insight to manually do what the framework automates may help reduce this second-guessing effort.
WHY: For more advanced debugging/design, it's good to know what the framework is doing under the hood esp. when things don't work as you planned. Initially you can take the help of local-gurus or forum gurus who've already done the hard work. Later as you go deeper you can take on more of that role. For example there's a "rebuilding rails" book which looks under the hood of Ruby on Rails.
Note: Some of the tips are oriented towards Ruby/Rails but you can easily substitute your favourite language/framework instead.