After reading Solving Problems in a Generic Way using Shapeless's first sentence of the conclusion:
In this article, I've demonstrated how generic solutions can be created for ADTs without relying on an expensive runtime feature such as reflection
Does that mean that Shapeless's coproducts do not use run-time reflection or casting, contrary to ADTs in Scala?
I'm the author of the blog post. I think there's been a misunderstanding: I didn't mean to imply that the ADTs rely on runtime reflection. What I was referring to was this sentence from the introduction:
Traditionally, generic programs have been written with the help of reflection APIs.
As far as I know, ADTs don't use runtime reflection, but without shapeless, there's not a lot of choices for traversing an ADT in a generic way. One way to achieve this is to use reflection for looking up object fields at runtime and iterating through the fields. You could also write code that traverses through your ADT and pattern matches on every ADT node, but that solution will only work for your ADT and not for all of the other ADTs, i.e. the solutions is would not be generic.
Related
In Scala 3 it is possible to get the compiler to derive instances of certain typeclasses using the derives TC syntax.
The documentation page here https://dotty.epfl.ch/docs/reference/contextual/derivation.html gives an example of deriving from typeclasses Eq Ordering and Show, saying that these typeclasses are "common examples".
Is Show in the library somewhere?
If not, is there a typeclass for pretty printing in the standard library?
Have I been spoiled by Haskell?
I guess no. At least not so far. It's easy to define it yourself. And it's easy to define many custom flavors of it (for any necessary use case).
I guess no.
Probably you are :) As well as by cats, scalaz etc.
https://typelevel.org/cats/typeclasses/show.html
https://github.com/scalaz/scalaz/blob/series/7.3.x/core/src/main/scala/scalaz/Show.scala
I've heard people claim that:
Scala's type system is amazing (existential types, variant, co-variant)
Because of the power of macros, everything is a library in Clojure: (pattern matching, logic programming, non-determinism, ..)
Question:
If both assertions are true, why is Scala's type system not a library in Clojure? Is it because:
types are one of these things that do not work well as a library? [i.e. the changes would somehow have to threaded through every existing clojure library, including clojure.core?]
is Scala's notion of types fundamentally incompatible with clojure protocol / records?
... ?
It's an interesting question.
You are certainly right about Scala having an amazing type system, and about Clojure being phenomenal for meta-programming and extension of the language (although that is about more than just macros....).
A few reasons I can think of:
Clojure is a dynamically typed language while Scala is a statically typed language. Having powerful type inference isn't so much use in a language where you can assume relatively little about the types of your inputs.
Clojure already has a very interesting project to add typing as a library (Typed Clojure) which looks very promising - however it's very different in approach to Scala as it is designed for a dynamic language from the start (inspired more by Typed Racket, I believe).
Clojure philosophy actually discourages certain OOP concepts (particularly implementation inheritance, mutable objects, and data encapsulation). A type system that supports these things (as Scala does) wouldn't be a good fit for Clojure idioms - at best they would be ignored, but they could easily encourage a style of development that would cause people to run into severe problems later.
Clojure already provides tools that solve many of the problems you would typically solve with types in other languages - e.g. the use of protocols for polymorphism.
There's a strong focus in the Clojure community on simplicity (in the sense of the excellent video "Simple Made Easy" - see particularly the slide at 39:30). While Scala's type system is certainly amazing, I think it's a stretch to describe it as "Simple"
Putting in a Scala-style type system would probably require a complete rewrite of the Clojure compiler and make it substantially more complex. Nobody seems to have signed up so far to take on that particular challenge... and there's a risk that even if someone were willing and able to do this then the changes could be rejected for the various cultural / technical reasons covered above.
In the absence of a major change to Clojure itself (which I think would be unlikely) then one interesting possibility would be to create a DSL within Clojure that provided Scala-style type inference for a specific domain and compiled this DSL direct to optimised Java bytecode. I could see that being a useful approach for specific problem domains (large scale numerical data crunching with big matrices, for example).
To simply answer your question "... why is Scala's type system not a library in Clojure?":
Because the type system is part of the scala compiler and not of the scala library. The whole power of scalas type system only exists at compile time. The JVM has no support for things like that, because of type erasure and also, because it would simply slow down execution. And also there is no need for it. If you have a statically typed language, you don't need type information at runtime, unless you want to do dirty stuff.
edit:
#mikera the jvm is sure capable of running the scala compiler, I did not say anything like that. I just said, that the jvm has no support for type systems like that. It does not even support generics. At runtime all these types are gone. The compiler checks for the correctness of a program and removes all the higher kinded types / generics.
example:
val xs: List[Int] = List(1,2,3,4)
val x1: Int = xs.head
will at runtime look like this:
val xs: List = List.apply(1,2,3,4)
val x1: Int = xs.head.asInstanceOf[Int]
But it doesn't matter, because the compiler checked it before. You can only get in trouble here, when you use reflection, because you could put any value in the list and it would break at runtime exactly where the value is casted to Int.
And this is one of the reasons, why the scala type system is not part of the scala library, but built into the compiler.
And also the question of the OP was "... why is Scala's type system not a library in Clojure?" and not "Is it possible to create a type system such as scalas for clojure?" and I perfectly answered that question.
After watching Martin's keynote on Reflection and Compilers I can't seem to get this crazy question out of my head. Martin talks among other things about the "(Wedding) Cake Pattern", where traits play the central part. I'm wondering, why in the world do we need packages when we already have traits? Is there anything a package can do, what a trait (at least theoretically) cannot?
I'm not talking about the current implementation, I'm just trying to imagine what programming would be like if we replace packages with traits. In my head it would be like this:
one keyword less (package is unneeded)
no need for package objects
To summarize all my questions:
Is it theoretically possible to remove packages from the language and use traits instead.
What other benefits would we gain from this change? (I was thinking about first class packages and first class imports, but mixin composition is a compile time thing, although the super calls are dynamically bound)
Is Java/JVM compatibility the only thing, which would stand in the way?
Update
Daniel Spiewak talks in this keynote about the Dependency Injection being just the top of the iceberg of all the stuff you can do with the Cake Pattern.
Martin Odersky has said that Scala could get by with just traits, objects, methods and paths (I hope I didn't forget something).
Both classes and packages are just there because Scala is intended to be a hosted language, i.e. a language which runs on (this is actually not the interesting bit) and interoperates with (this is the important point) a host platform. Some of the host platforms that Scala is intended to interoperate with are the Java platform and the CLI, both which have a concept of classes and packages (namespaces in the case of the CLI) that is significantly distinct enough that it cannot be easily expressed as traits or objects. This is unlike interfaces, which can be trivially mapped to and from purely abstract traits.
The above statement was made in a discussion about potentially removing generics from Scala, because everything generics can do can also be achieved by abstract types.
In scala the object and package serve almost the same purpose and objects are also called modules. Objects deserve to be thought of as modules because they can contain any definition including other objects of course and, significantly, types.
A trait can be thought of as an abstract module. It can contain any definition and any member can be abstract including, again significantly, type members. I am reciting all this just to highlight the symmetry. Perhaps OT but to me traits seem to be as big an innovation in scala as the merging of object and functional ideas.
To finally give an answer:
I think packages could be removed in favour of objects (not traits).
The benefit would be a simplification - package objects would not need to be explicitly defined.
I think packages are distinct from objects for Java/JVM compatibility.
Some more commentary: in the video Martin talks of traits (abstract modules) more than concrete modules because the latter only appear at the last moment to assemble and reify some combination of abstract modules.
It is good to use abstract modules even when not "mixing a cake". e.g. when sketching out some code you might define a module to contain definitions. But as soon as you come to a type or value you are not ready to fill in, don't supply a dummy such as null. Instead switch the object to a trait and leave the member abstract.
I've been hearing a lot about different JVM languages, still in vaporware mode, that propose to implement reification somehow. I have this nagging half-remembered (or wholly imagined, don't know which) thought that somewhere I read that Scala somehow took advantage of the JVM's type erasure to do things that it wouldn't be able to do with reification. Which doesn't really make sense to me since Scala is implemented on the CLR as well as on the JVM, so if reification caused some kind of limitation it would show up in the CLR implementation (unless Scala on the CLR is just ignoring reification).
So, is there a good side to type erasure for Scala, or is reification an unmitigated good thing?
See Ola Bini's blog. As we all know, Java has use-site covariance, implemented by having little question marks wherever you think variance is appropriate. Scala has definition-site covariance, implemented by the class designer. He says:
Generics is a complicated language feature. It becomes even more
complicated when added to an existing language that already has
subtyping. These two features don’t play very well together in the
general case, and great care has to be taken when adding them to a
language. Adding them to a virtual machine is simple if that machine
only has to serve one language - and that language uses the same
generics. But generics isn’t done. It isn’t completely understood how
to handle correctly and new breakthroughs are happening (Scala is a
good example of this). At this point, generics can’t be considered
“done right”. There isn’t only one type of generics - they vary in
implementation strategies, feature and corner cases.
...
What this all means is that if you want to add reified generics to the
JVM, you should be very certain that that implementation can encompass
both all static languages that want to do innovation in their own
version of generics, and all dynamic languages that want to create a
good implementation and a nice interfacing facility with Java
libraries. Because if you add reified generics that doesn’t fulfill
these criteria, you will stifle innovation and make it that much
harder to use the JVM as a multi language VM.
i.e. If we had reified generics in the JVM, most likely those reified generics wouldn't be suitable for the features we really like about Scala, and we'd be stuck with something suboptimal.
In Java and C++ designing program's objects hierarchy is pretty obvious. But beginning Scala I found myself difficult to decide what classes to define to better employ Scala's syntactic sugar facilities (an even idealess about how should I design for better performance). Any good readings on this question?
I have read 4 books on Scala, but I have not found what you are asking for. I guess you have read "Programming in Scala" by Odersky (Artima) already. If not, this is a link to the on-line version:
http://www.docstoc.com/docs/8692868/Programming-In-Scala
This book gives many examples how to construct object-oriented models in Scala, but all examples are very small in number of classes. I do not know of any book that will teach you how to structure large scale systems using Scala.
Imperative object-orientation has
been around since Smalltalk, so we
know a lot about this paradigm.
Functional object-orientation on the
other hand, is a rather new concept,
so in a few years I expect books
describing large scale FOO systems to
appear. Anyway, I think that the PiS
book gives you a pretty good picture
how you can put together the basic
building blocks of a system, like
Factory pattern, how to replace the
Strategy pattern with function
literals and so on.
One thing that Viktor Klang once told me (and something I really agree upon) is that one difference between C++/Java and Scala OO is that you define a lot more (smaller) classes when you use Scala. Why? Because you can! The syntactic sugar for the case class result in a very small penalty for defining a class, both in typing and in readability of the code. And as you know, many small classes usually means better OO (fewer bugs) but worse performance.
One other thing I have noticed is that I use the factory pattern a lot more when dealing with immutable objects, since all "changes" of an instance results in creating a new instance. Thank God for the copy() method on the case class. This method makes the factory methods a lot shorter.
I do not know if this helped you at all, but I think this subject is very interesting myself, and I too await more literature on this subject.
Cheers!
This is still an evolving matter. For instance, the just released Scala 2.8.0 brought support of type constructor inference, which enabled a pattern of type classes in Scala. The Scala library itself has just began using this pattern. Just yesterday I heard of a new Lift module in which they are going to try to avoid inheritance in favor of type classes.
Scala 2.8.0 also introduced lower priority implicits, plus default and named parameters, both of which can be used, separately or together, to produce very different designs than what was possible before.
And if we go back in time, we note that other important features are not that old either:
Extractor methods on case classes object companions where introduced February 2008 (before that, the only way to do extraction on case classes was through pattern matching).
Lazy values and Structural types where introduced July 2007.
Abstract types support for type constructors was introduced in May 2007.
Extractors for non-case classes was introduced in January 2007.
It seems that implicit parameters were only introduced in March 2006, when they replaced the way views were implemented.
All that means we are all learning how to design Scala software. Be sure to rely on tested designs of functional and object oriented paradigms, to see how new features in Scala are used in other languages, like Haskell and type classes or Python and default (optional) and named parameters.
Some people dislike this aspect of Scala, others love it. But other languages share it. C# is adding features as fast as Scala. Java is slower, but it goes through changes too. It added generics in 2004, and the next version should bring some changes to better support concurrent and parallel programming.
I don't think that there are much tutorials for this. I'd suggest to stay with the way you do it now, but to look through "idiomatic" Scala code as well and to pay special attention in the following cases:
use case classes or case objects instead of enums or "value objects"
use objects for singletons
if you need behavior "depending on the context" or dependency-injection-like functionality, use implicits
when designing a type hierarchy or if you can factor things out of a concrete class, use traits when possible
Fine grained inheritance hierarchies are OK. Keep in mind that you have pattern matching
Know the "pimp my library" pattern
And ask as many questions as you feel you need to understand a certain point. The Scala community is very friendly and helpful. I'd suggest the Scala mailing list, Scala IRC or scala-forum.org
I've just accidentally googled to a file called "ScalaStyleGuide.pdf". Going to read...