Can monads replace macros when writing a redis json driver? - macros

In this blog entry by Jim Duey - he provides a list of reasons that you'd want to use monads. One his reasons is this:
So what are some clues that a monadic solution is possible? It seems to me that anytime you're copying and pasting code to define a new function that's similar to an existing one, there might be a monad lurking.
This is actually quite similar to the justification for using a Clojure style macro.
In this presentation by Aaron Bedra - he talks about a use case for Macros when writing a redis driver to generate json. On slide 66 - he shows an example of this.
(defmacro defcommand
[name params]
(let [p (parameters params)]
`(defn ~name ~params
My question is - can Monads be used to solve this problem of duplicate code when generating json for redis calls instead of a macro?
Assumptions
I understand it is more idiomatic to choose a macro over a monad in Clojure. For the purpose of this question I'm choosing to ignore what is idiomatic and just look at what is possible.

anytime you're copying and pasting code to define a new function
that's similar to an existing one
I would create a higher order function to get rid of copy paste, as simple as that.
Now, Macro or Monad or even a simple higher order function solves a single purpose but at different conceptual levels. The purpose is to "abstract away a common pattern in a system".
The different conceptual levels at which you can see a pattern appearing in a system are:
Same chunk of code - Use function / higher order function
Same chunk of code but that require some sort of "compile time" preprocessing to make it abstract - Macro
Pattern of "A value inside a box with some context" and it fits with the monad laws - Monads. (Maybe, IO, List - all these are API patterns where we see a value inside a box with some context)
I know it sounds very "abstract", but once you practice enough thinking about how to abstract general patterns in your system, you will eventually get the intuition about which tool to use for which pattern.

Monads and Macros operate at different levels of abstraction.
I think of macros as "code monkeys": anytime I find myself writing a lot of boilerplate, I take a moment and think whether macros can help.
Monads on the other hand are very powerful abstractions generally used in order to isolate and compose side effects in a purely functional setting. (I'm not an expert on Monads so take this with a grain of salt).
The bottom line is that I think they are solving fundamentally different problems.
If we take the example you provided, writing a Redis driver, any function accessing the network is a candidate for an abstraction through monads - in this case, both the IO and Maybe (or Either) monads. But not because of repetition, but rather because your functions have side effects.
Now this is where Monads in clojure can get harder than it's worth: If you want to compose both monads, you need to use monad transformers. Obviously this is not enforced by clojure since it's a dynamic language so one can argue debugging composed monadic code in a dynamic language can be very hard. A static type system such as Haskell's would be of great help here.
So while a redis driver can definitely be written in a monadic style, I believe the point Aaron is making in the presentation is avoiding repetition, and for that, I believe macros are a better fit in Clojure.
I hope this is helpful.
EDIT:
I should also node that monads in Clojure would be hardly useful without the Haskell inspired do notation which is essentially made possible by macros, providing further proof that macros are at a different level of abstraction.

Related

How to enforce Functional Programming on Scala

I'm starting to learn Functional Programming and would like to do so with Scala, not Haskell or Lisp.
But some people claim that learning Scala as the first functional language slows down your learning of Functional Programming, because Scala allows you to program both ways, and one tends to program the procedural way when confronted with a hard problem.
How can i make sure that 'm programming in a purely functional way? Maybe, due to not being able to properly distinguish both styles, I'll inadvertently program procedurally).
I know, for example, that I should only use vals and not vars.
The other answers have made some good points, but for an attempt to quickly get down some guidelines, here's how I'd start:
Firstly, some things to completely avoid:
Don't use the var keyword.
Don't use the while keyword.
Don't use anything in the scala.collection.mutable package.
Don't use the asInstanceOf method.
Don't use null. If you ever come across null (in someone else's code), immediately wrap it in a more appropriate datatype (usually Option will do nicely).
Then, a couple of things to generally avoid:
Be wary of calling anything with a return type of Unit. A function with a return type of Unit is either doing nothing, or acting only by side-effects. In some cases you won't be able to avoid this (IO being the obvious one), but where you see it elsewhere it's probably a sign of impurity.
Be wary of calling into Java libraries - they are typically not designed with functional programming in mind, and will often require you to abandon the functional approach.
Once you've avoided these things, what can you do to move your code to being more functional?
When you're performing direct recursion, look for opportunities to generalise it through the use of higher order combinators. fold is likely your biggest candidate here - most operations on lists can be implemented in terms of a suitable fold.
When you see destructuring operations on a data structure (typically through pattern matching), consider whether instead you can lift the computation into the structure and avoid destructuring it. An obvious example is the following code snippet:
foo match {
case Some(x) => Some(x + 2)
case None => None
}
can be replaced with:
foo map ( _ + 2 )
I dare say that your goal is already misleading:
I'm starting to learn Function Programming, and I really wanna learn
Scala, not Haskell or Lisp.
If you are really interested in learning concepts of function programming, then why not use a language such as Haskell that (more or less) does not allow you to use procedural or object-oriented concepts? In the end, the language is "just" a tool that helps you learning concepts of FP, you could just as well read loads of papers about FP. At least theoretically, I think it is usually easier to learn concepts of computer science with concrete tools at hand.
On the other hand, if you are interested in learning the language Scala, then why not use all features that it offers, regardless of whether they stem from the FP or the OO world?
In order to conclude with a somewhat practical advise: You could search for FP tutorials that use Scala or for blog entries etc. that describe how to realise certain FP-concepts in Scala and try to follow them. This way, it is less likely that you make use of non-FP concepts.
You don't buy a Ferarri to deliver furniture. Scala's fundamental strength is the fact that in your words, it goes both ways:). Whether or not you are programming in a functional style is decided by the techniques you use.
The best thing you can do is thoroughly review fundamental concepts of functional programming and seek the appropriate Scala implementation of the respective concepts. But if you want to program purely functional style, then go for Haskell, Lisp, Erlang, OCaml, or whatever other purely functional dialect.
Functional programming
Introduction
Functional thinking
Scala
If you want to learn Scala, then make sure to include both OO and FP in your learning curve. Lambda expressions + OO concepts + syntactic sugar made possible by IMHO the most advanced compiler on the face of the planet lead to something quite amazing. Take advantage of it!
I think learning is non linear process, it helps to see lots of ways of doing the same thing, also be opportunistic and use any learning resources that are available for you. For example Martin Odersky the creator of Scala offers a free course called "Functional Programming Principles in Scala" https://class.coursera.org/progfun-002/class/index there are some very high quality video lectures, and some really good assignments where the automated grader will tell you that your code is not functional enough and you loose style points because you are using var instead of val
I think the thing you want to focus on is learning the Functional Programming Paradigm and for me learning a paradigm is about learning what types of problems are easy to solve in one paradigm and are hard to solve in another paradigm. Focus on the paradigm and I think you will find that learning both about Haskell and Scala will teach you the functional paradigm faster, because you will be able to ask the question what are the common features between Scala and Haskell, what are the differences .... etc
I know, for example, that I should only use vals and not vars.
That's already a good start, other non-so-functional things to avoid are mutable collections and loops.
Have a look at immutable collections and recursion instead.
Of course, once you are familiar with the functional concepts, there might also be good reasons to use scala's non-functional features.

Elegant AST model

I am in the process of writing a toy compiler in scala. The target language itself looks like scala but is an open field for experiment.
After several large refactorings I can't find a good way to model my abstract syntax tree. I would like to use the facilities of scala's pattern matching, the problem is that the tree carries moving information (like types, symbols) along the compilation process.
I can see a couple of solutions, none of which I like :
case classes with mutable fields (I believe the scala compiler does this) : the problem is that those fields are not present a each stage of the compilation and thus have to be nulled (or Option'd) and it becomes really heavy to debug/write code. Moreover, if for exemple, I find a node with null type after the typing phase I have a really hard time finding the cause of the bug.
huge trait/case class hierarchy : something like Node, NodeWithSymbol, NodeWithType, ... Seems like a pain to write AND work with
something completly hand crafted with extractors
I'm also not sure if it is good practice to go with a fully immutable AST, especially in scala where there is no implicit sharing (because the compiler is not aware of immutability) and it could hurt performances to copy the tree all the time.
Can you think of an elegant pattern to model my tree using scala's powerful type system ?
TL;DR I prefer to keep the AST immutable and carry things like type information in a separate structure, e.g. a Map, that can be referred by IDs stored in the AST. But there is no perfect answer.
You're by no means the first to struggle with this question. Let me list some options:
1) Mutable structures that get updated at each phase. All the up and downsides you mention.
2) Traits/cake pattern. Feasible, but expensive (there's no sharing) and kinda ugly.
3) A new tree type at each phase. In some ways this is the theoretically cleanest. Each phase can deal only with a structure produced for it by the previous phase. Plus the same approach carries all the way from front end to back end. For instance, you may "desugar" at some point and having a new tree type means that downstream phase(s) don't have to even consider the possibility of node types that are eliminated by desugaring. Also, low level optimizations usually need IRs that are significantly lower level than the original AST. But this is also a lot of code since almost everything has to be recreated at each step. This approach can also be slow since there can be almost no data sharing between phases.
4) Label every node in the AST with an ID and use that ID to reference information in other data structures (maps and vectors and such) that hold information computed for each phase. In many ways this is my favorite. It retains immutability, maximizes sharing and minimizes the "excess" code you have to write. But you still have to deal with the potential for "missing" information that can be tricky to debug. It's also not as fast as the mutable option, though faster than any option that requires producing a new tree at each phase.
I recently started writing a toy verifier for a small language, and I am using the Kiama library for the parser, resolver and type checker phases.
Kiama is a Scala library for language processing. It enables convenient analysis and transformation of structured data. The programming styles supported by the library are based on well-known formal language processing paradigms, including attribute grammars, tree rewriting, abstract state machines, and pretty printing.
I'll try to summarise my (fairly limited) experience:
[+] Kiama comes with several examples, and the main contributor usually responds quickly to questions asked on the mailing list
[+] The attribute grammar paradigm allows for a nice separation into "immutable components" of the nodes, e.g., names and subnodes, and "mutable components", e.g., type information
[+] The library comes with a versatile rewriting system which - so far - covered all my use cases
[+] The library, e.g., the pretty printer, make nice examples of DSLs and of various functional patterns/approaches/ideas
[-] The learning curve it definitely steep, even with examples and the mailing list at hand
[-] Implementing the resolving phase in a "purely function" style (cf. my question) seems tricky, but a hybrid approach (which I haven't tried yet) seems to be possible
[-] The attribute grammar paradigm and the resulting separation of concerns doesn't make it obvious how to document the properties nodes have in the end (cf. my question)
[-] Rumour has it, that the attribute grammar paradigm does not yield the fastest implementations
Summarising my summary, I enjoy using Kiama a lot and I strongly recommend that you give it a try, or at least have a look at the examples.
(PS. I am not affiliated with Kiama)

Real World Functional Programming in Scala

Soooo...
Semigroups, Monoids, Monads, Functors, Lenses, Catamorphisms, Anamorphisms, Arrows... These all sound good, and after an exercise or two (or ten), you can grasp their essence. And with Scalaz, you get them for free...
However, in terms of real-world programming, I find myself struggling to find usages to these notions. Yes, of course I always find someone on the web using Monads for IO or Lenses in Scala, but... still...
What I am trying to find is something along the "prescriptive" lines of a pattern. Something like: "here, you are trying to solves this, and one good way to solve it is by using lenses this way!"
Suggestions?
Update: Something along these lines, with a book or two, would be great (thanks Paul): Examples of GoF Design Patterns in Java's core libraries
The key to functional programming is abstraction, and composability of abstractions. Monads, Arrows, Lenses, these are all abstractions which have proven themselves useful, mostly because they are composable. You've asked for a "prescriptive" answer, but I'm going to say no. Perhaps you're not convinced that functional programming matters?
I'm sure plenty of people on StackOverflow would be more than happy to try and help you solve a specific problem the FP way. Have a list of stuff and you want to traverse the list and build up some result? Use a fold. Want to parse XML? hxt uses arrows for that. And monads? Well, tons of data types turn out to be Monads, so learn about them and you'll discover a wealth of ways you can manipulate these data types. But its kind of hard to just pull examples out of thin air and say "lenses are the Right Way to do this", "monoids are the best way to do that", etc. How would you explain to a newbie what the use of a for loop is? If you want to [blank], then use a for loop [in this way]. It's so general; there are tons of ways to use a for loop. The same goes for these FP abstractions.
If you have many years of OOP experience, then don't forget you were once a newbie at OOP. It takes time to learn the FP way, and even more time to unlearn some OOP tendencies. Give it time and you will find plenty of uses for a Functional approach.
I gave a talk back in September focused on the practical application of monoids and applicative functors/monads via scalaz.Validation. I gave another version of the same talk at the scala Lift Off, where the emphasis was more on the validation. I would watch the first talk until I start on validations and then skip to the second talk (27 minutes in).
There's also a gist I wrote which shows how you might use Validation in a "practical" application. That is, if you are designing software for nightclub bouncers.
I think you can take the reverse approach and instead when writing a small piece of functionality, ask yourself whether any of those would apply: Semigroups, Monoids, Monads, Functors, Lenses, Catamorphisms, Anamorphisms, Arrows... A lots of those concepts can be used in a local way.
Once you start down that route, you may see usage everywhere. For me, I sort of get Semigroups, Monoids, Monads, Functors. So take the example of answering this question How do I populate a list of objects with new values. It's a real usage for the person asking the question (a self described noob). I am trying to answer in a simple way but I have to refrain myself from scratching the itch "there are monoids in here".
Scratching it now: using foldMap and the fact that Int and List are monoids and that the monoid property is preserved when dealing with tuple, maps and options:
// using scalaz
listVar.sliding(2).toList.foldMap{
case List(prev, i) => Some(Map(i -> (1, Some(List(math.abs(i - prev))))))
case List(i) => Some(Map(i -> (1, None)))
case _ => None
}.map(_.mapValues{ case (count, gaps) => (count, gaps.map(_.min)) })
But I don't come to that result by thinking I will use hard core functional programming. It comes more naturally by thinking this seems simpler if I compose those monoids combined with the fact that scalaz has utility methods like foldMap. Interestingly when looking at the resulting code it's not obvious that I'm totally thinking in terms of monoid.
You might like this talk by Chris Marshall. He covers a couple of Scalaz goodies - namely Monoid and Validation - with many practical examples. Ittay Dror has written a very accessible post on how Functor, Applicative Functor, and Monad can be useful in practice. Eric Torreborre and Debasish Gosh's blogs also have a bunch of posts covering use cases for categorical constructs.
This answer just lists a few links instead of providing some real substance here. (Too lazy to write.) Hope you find it helpful anyway.
I understand your situation, but you will find that to learn functional programming you will need to adjust your point of view to the documentation you find, instead of the other way around. Luckily in Scala you have the possibility of becoming a functional programmer gradually.
To answer your questions and explain the point-of-view difference, I need to distinguish between "type classes" (monoids, functors, arrows), mathematically called "structures", and generic operations or algorithms (catamorphisms or folds, anamorphisms or unfolds, etc.). These two often interact, since many generic operations are defined for specific classes of data types.
You look for prescriptive answers similar to design patterns: when does this concept apply? The truth is that you have surely seen the prescriptive answers, and they are simply the definitions of the different concepts. The problem (for you) is that those answers are intrinsically different from design patterns, but it is so for good reasons.
On the one hand, generic algorithms are not design patterns, which suggest a structure for the code you write; they are abstractions defined in the language which you can directly apply. They are general descriptions for common algorithms which you already implement today, but by hand. For instance, whenever you are computing the maximum element of a list by scanning it, you are hardcoding a fold; when you sum elements, you are doing the same; and so on. When you recognize that, you can declare the essence of the operation you are performing by calling the appropriate fold function. This way, you save code and bugs (no opportunity for off-by-one errors), and you save the reader the effort to read all the needed code.
On the other hand, structures concern not the goal you have in mind but properties of the entities you are modeling. They are more useful for bottom-up software construction, rather than top-down: when defining your data, you can declare that it is a e.g. a monoid. Later, when processing your data, you have the opportunity to use operations on e.g. monoids to implement your processing. In some cases it is useful to strive to express your algorithm in terms of the predefined ones. For instance, very often if you need to reduce a tree to a single value, a fold can do most or all of what you need. Of course, you can also declare that your data type is a monoid when you need a generic algorithm on monoids; but the earlier you notice that, the earlier you can start reusing generic algorithms for monoids.
Last advice is that probably most of the documentation you will find about these concepts concerns Haskell, because this language has been around for much more time and supports them in a quite elegant way. Quite recommended here are Learn you a Haskell for Great Good, a Haskell course for beginners, where among others chapters 11 to 14 focus on some type classes, and Typeclassopedia (which contains links to various articles with specific examples). EDIT: Finally, an example of applications of Monoids, taken from Typeclassopedia, is here: http://apfelmus.nfshost.com/articles/monoid-fingertree.html. I'm not saying there is little documentation for Scala, just that there is more in Haskell, and Haskell is where the application of these concepts to programming was born.

Use of Clojure macros for DSLs

I am working on a Clojure project and I often find myself writing Clojure macros for DSLs, but I was watching a Clojure video of how a company uses Clojure in their real work and the speaker said that in practical use they do not use macros for their DSLs, they only use macros to add a little syntactic sugar. Does this mean I should write my DSL in using standard functions and then add a few macros at the end?
Update:
After reading the many varied (and entertaining) responses to this question I have realized that the answer is not as clear cut as I first thought, for many reasons:
There are many different types of API in an application (internal, external)
There are many types of user of the API (business user who just wants to get something done fast, Clojure expert)
Is there macro there to hide boiler plate code?
I will go away and think about the question more deeply, but thanks for your answers as they have given me lots to think about. Also I noticed that Paul Graham thinks the opposite of the Christophe video and thinks macros should be a large part of the codebase (25%):
http://www.paulgraham.com/avg.html
To some extent I believe this depends on the use / purpose of your DSL.
If you are writing a library-like DSL to be used in Clojure code and want it to be used in a functional way, then I would prefer functions over macros. Functions are "nice" for Clojure users because they can be composed dynamically into higher order functions etc. For example, you are writing a functional web framework like Ring.
If you are writing a imperative DSL that will be used pretty independently of other Clojure code and you have decided that you definitely don't need higher order functions, then the usage will be pretty similar and you can chose whichever makes most sense. For example, you might be creating some kind of business rules engine.
If you are writing a specialised DSL that needs to produce highly performant code, then you will probably want to use macros most of the time since they will be expanded at compile time for maximum efficiency. For example, you're writing some graphics code that needs to expand to exactly the right sequence of OpenGL calls......
Yes!
Write functions whenever possible. Never write a macro when a function will do. If you write to many macros you end up with somthing that is much harder to extend. Macros for example cant be applied or passed around.
Christophe Grand: (not= DSL macros)
http://clojure.blip.tv/file/4522250/
No!
Don't be afraid of using macros extensively. Always write a macro when in doubt. Functions are inferior for implementing DSLs - they're taking the burden onto the runtime, whereas macros allows to do many heavyweight computations in a compilation time. Just think of a difference of implementing, say, an embedded Prolog as an interpreter function and as a macro which compiles Prolog into some form of a WAM.
And do not listen to those who say that "macros cant be applied or passed around", this argument is entirely a strawman. Those people are advocating interpreters over compilers, which is simply ridiculous.
A couple of tips on how to implement DSLs using macros:
Do it in stages. Define a long chain of languages from your DSL to the underlying Clojure. Keep each transform as simple as possible - this way you'd be able to easily maintain and debug your DSL compiler.
Prepare a toolbox of DSL components that you will reuse when implementing your DSLs. It should include target languages of different semantics (e.g., untyped eager functional - it is Clojure itself, untyped lazy functional, first order logic, typed imperative, Hindley-Millner typed eager functional, dataflow, etc.). With macros it is trivial to combine properties of all that target semantics seamlessly.
Maintain a set of compiler-building tools. It should include parser generators (useful even if your DSLs are entirely in S-expressions), term rewriting engines, pattern matching engines, implementations for some common algorithms on graphs (e.g., graph colouring), etc.
Here's an example of a DSL in Haskell that uses functions rather than macros:
http://contracts.scheming.org/
Here is a video of Simon Peyton Jones giving a talk about this implementation:
http://ulf.wiger.net/weblog/2008/02/29/simon-peyton-jones-composing-contracts-an-adventure-in-financial-engineering/
Leverage the characteristics of Clojure and FP before going down the path of implementing your own language. I think SK-logic's tips give you a good indication of what is needed to implement a full blown language. There are times when it's worth the effort, but those are rare.

Debunking Scala myths [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
What are the most commonly held misconceptions about the Scala language, and what counter-examples exist to these?
UPDATE
I was thinking more about various claims I've seen, such as "Scala is dynamically typed" and "Scala is a scripting language".
I accept that "Scala is [Simple/Complex]" might be considered a myth, but it's also a viewpoint that's very dependent on context. My personal belief is that it's the very same features that can make Scala appear either simple or complex depending oh who's using them. Ultimately, the language just offers abstractions, and it's the way that these are used that shapes perceptions.
Not only that, but it has a certain tendency to inflame arguments, and I've not yet seen anyone change a strongly-held viewpoint on the topic...
Myth: That Scala’s “Option” and Haskell’s “Maybe” types won’t save you from null. :-)
Debunked: Why Scala's "Option" and Haskell's "Maybe" types will save you from null by James Iry.
Myth: Scala supports operator overloading.
Actually, Scala just has very flexible method naming rules and infix syntax for method invocation, with special rules for determining method precedence when the infix syntax is used with 'operators'. This subtle distinction has critical implications for the utility and potential for abuse of this language feature compared to true operator overloading (a la C++), as explained more thoroughly in James Iry's answer to this question.
Myth: methods and functions are the same thing.
In fact, a function is a value (an instance of one of the FunctionN classes), while a method is not. Jim McBeath explains the differences in greater detail. The most important practical distinctions are:
Only methods can have type parameters
Only methods can take implicit arguments
Only methods can have named and default parameters
When referring to a method, an underscore is often necessary to distinguish method invocation from partial function application (e.g. str.length evaluates to a number, while str.length _ evaluates to a zero-argument function).
I disagree with the argument that Scala is hard because you can use very advanced features to do hard stuff with it. The scalability of Scala means that you can write DSL abstractions and high-level APIs in Scala itself that otherwise would need a language extension. So to be fair you need to compare Scala libraries to other languages compilers. People don't say that C# is hard because (I assume, don't have first hand knowledge on this) the C# compiler is pretty impenetrable. For Scala it's all out in the open. But we need to get to a point where we make clear that most people don't need to write code on this level, nor should they do it.
I think a common misconception amongst many scala developers, those at EPFL (and yourself, Kevin) is that "scala is a simple language". The argument usually goes something like this:
scala has few keywords
scala reuses the same few constructs (e.g. PartialFunction syntax is used as the body of a catch block)
scala has a few simple rules which allow you to create library code (which may appear as if the language has special keywords/constructs). I'm thinking here of implicits; methods containing colons; allowed identifier symbols; the equivalence of X(a, b) and a X b with extractors. And so on
scala's declaration-site variance means that the type system just gets out of your way. No more wildcards and ? super T
My personal opinion is that this argument is completely and utterly bogus. Scala's type system taken together with implicits allows one to write frankly impenetrable code for the average developer. Any suggestion otherwise is just preposterous, regardless of what the above "metrics" might lead you to think. (Note here that those who I've seen scoffing at the non-complexity of Java on Twitter and elsewhere happen to be uber-clever types who, it sometimes seems, had a grasp of monads, functors and arrows before they were out of short pants).
The obvious arguments against this are (of course):
you don't have to write code like this
you don't have to pander to the average developer
Of these, it seems to me that only #2 is valid. Whether or not you write code quite as complex as scalaz, I think it's just silly to use the language (and continue to use it) with no real understanding of the type system. How else can one get the best out of the language?
There is a myth that Scala is difficult because Scala is a complex language.
This is false--by a variety of metrics, Scala is no more complex than Java. (Size of grammar, lines of code or number of classes or number of methods in the standard API, etc..)
But it is undeniably the case that Scala code can be ferociously difficult to understand. How can this be, if Scala is not a complex language?
The answer is that Scala is a powerful language. Unlike Java, which has many special constructs (like enums) that accomplish one particular thing--and requires you to learn specialized syntax that applies just to that one thing, Scala has a variety of very general constructs. By mixing and matching these constructs, one can express very complex ideas with very little code. And, unsurprisingly, if someone comes along who has not had the same complex idea and tries to figure out what you're doing with this very compact code, they may find it daunting--more daunting, even, than if they saw a couple of pages of code to do the same thing, since then at least they'd realize how much conceptual stuff there was to understand!
There is also an issue of whether things are more complex than they really need to be. For example, some of the type gymnastics present in the collections library make the collections a joy to use but perplexing to implement or extend. The goals here are not particularly complicated (e.g. subclasses should return their own types), but the methods required (higher-kinded types, implicit builders, etc.) are complex. (So complex, in fact, that Java just gives up and doesn't try, rather than doing it "properly" as in Scala. Also, in principle, there is hope that this will improve in the future, since the method can evolve to more closely match the goal.) In other cases, the goals are complex; list.filter(_<5).sorted.grouped(10).flatMap(_.tail.headOption) is a bit of a mess, but if you really want to take all numbers less than 5, and then take every 2nd number out of 10 in the remaining list, well, that's just a somewhat complicated idea, and the code pretty much says what it does if you know the basic collections operations.
Summary: Scala is not complex, but it allows you to compactly express complex ideas. Compact expression of complex ideas can be daunting.
There is a myth that Scala is non-deployable, whereas a wide range of third-party Java libraries can be deployed without a second thought.
To the extent that this myth exists, I suspect it exists among people who are not accustomed to separating a virtual machine and API from a language and compiler. If java == javac == Java API in your mind, you might get a little nervous if someone suggests using scalac instead of javac, because you see how nicely your JVM runs.
Scala ends up as JVM bytecode, plus its own custom library. There's no reason to be any more worried about deploying Scala on a small scale or as part of some other large project as there is in deploying any other library that may or may not stay compatible with whichever JVM you prefer. Granted, the Scala development team is not backed by quite as much force as the Google collections, or Apache Commons, but its got at least as much weight behind it as things like the Java Advanced Imaging project.
Myth:
def foo() = "something"
and
def bar = "something"
is the same.
It is not; you can call foo(), but bar() tries to call the apply method of StringLike with no arguments (results in an error).
Some common misconceptions related to Actors library:
Actors handle incoming messages in a parallel, in multiple threads / against a thread pool (in fact, handling messages in multiple threads is contrary to the actors concept and may lead to racing conditions - all messages are sequentially handled in one thread (thread-based actors use one thread both for mailbox processing and execution; event-based actors may share one VM thread for execution, using multi-threaded executor to schedule mailbox processing))
Uncaught exceptions don't change actor's behavior/state (in fact, all uncaught exceptions terminate the actor)
Myth: You can replace a fold with a reduce when computing something like a sum from zero.
This is a common mistake/misconception among new users of Scala, particularly those without prior functional programming experience. The following expressions are not equivalent:
seq.foldLeft(0)(_+_)
seq.reduceLeft(_+_)
The two expressions differ in how they handle the empty sequence: the fold produces a valid result (0), while the reduce throws an exception.
Myth: Pattern matching doesn't fit well with the OO paradigm.
Debunked here by Martin Odersky himself. (Also see this paper - Matching Objects with Patterns - by Odersky et al.)
Myth: this.type refers to the same type represented by this.getClass.
As an example of this misconception, one might assume that in the following code the type of v.me is B:
trait A { val me: this.type = this }
class B extends A
val v = new B
In reality, this.type refers to the type whose only instance is this. In general, x.type is the singleton type whose only instance is x. So in the example above, the type of v.me is v.type. The following session demonstrates the principle:
scala> val s = "a string"
s: java.lang.String = a string
scala> var v: s.type = s
v: s.type = a string
scala> v = "another string"
<console>:7: error: type mismatch;
found : java.lang.String("another string")
required: s.type
v = "another string"
Scala has type inference and refinement types (structural types), whereas Java does not.
The myth is busted by James Iry.
Myth: that Scala is highly scalable, without qualifying what forms of scalability.
Scala may indeed be highly scalable in terms of the ability to express higher-level denotational semantics, and this makes it a very good language for experimentation and even for scaling production at the project-level scale of top-down coordinated compositionality.
However, every referentially opaque language (i.e. allows mutable data structures), is imperative (and not declarative) and will not scale to WAN bottom-up, uncoordinated compositionality and security. In other words, imperative languages are compositional (and security) spaghetti w.r.t. uncoordinated development of modules. I realize such uncoordinated development is perhaps currently considered by most to be a "pipe dream" and thus perhaps not a high priority. And this is not to disparage the benefit to compositionality (i.e. eliminating corner cases) that higher-level semantic unification can provide, e.g. a category theory model for standard library.
There will possibly be significant cognitive dissonance for many readers, especially since there are popular misconceptions about imperative vs. declarative (i.e. mutable vs. immutable), (and eager vs. lazy,) e.g. the monadic semantic is never inherently imperative yet there is a lie that it is. Yes in Haskell the IO monad is imperative, but it being imperative has nothing to with it being a monad.
I explained this in more detail in the "Copute Tutorial" and "Purity" sections, which is either at the home page or temporarily at this link.
My point is I am very grateful Scala exists, but I want to clarify what Scala scales and what is does not. I need Scala for what it does well, i.e. for me it is the ideal platform to prototype a new declarative language, but Scala itself is not exclusively declarative and afaik referential transparency can't be enforced by the Scala compiler, other than remembering to use val everywhere.
I think my point applies to the complexity debate about Scala. I have found (so far and mostly conceptually, since so far limited in actual experience with my new language) that removing mutability and loops, while retaining diamond multiple inheritance subtyping (which Haskell doesn't have), radically simplifies the language. For example, the Unit fiction disappears, and afaics, a slew of other issues and constructs become unnecessary, e.g. non-category theory standard library, for comprehensions, etc..