Scala 2.10 brings reflection other than that provided the JVM (or I guess CLR).
What in particular do we have to look forward to, and how will it improve on the platform?
For example, will there be a class that reflects the language's convertibility between fields and accessor methods, so that I can iterate over the properties of an object?
update 2012-07-04:
Daniel SOBRAL (also on SO) details in his blog post "JSON serialization with reflection in Scala! Part 1 - So you want to do reflection?" some of the features coming with reflection:
To recapitulate, Scala 2.10 will come with a Scala reflection library.
That library is used by the compiler itself, but divided into layers through the cake pattern, so different users see different levels of detail, keeping jar sizes adequate to each one's use, and hopefully hiding unwanted detail.
The reflection library also integrates with the upcoming macro facilities, enabling enterprising coders to manipulate code at compile time.
update 2012-06-14. (from Eugene Burmako):
In Scala 2.10.0-M4, we have released the new reflection API that will most likely make it into 2.10.0-final without significant changes.
More details about the API can be found:
SO answer Get companion object instance with new Scala reflection API
Scala Reflection SIP, June 2012 by Martin Odersky (SIP, actually "Scala Improvement Process")
summary and migration route from M3
Extracts:
Universes and mirrors are now separate entities:
universes host reflection artifacts (trees, symbols, types, etc),
mirrors abstract loading of those artifacts (e.g. JavaMirror loads stuff
using a classloader and annotation unpickler, while GlobalMirror uses internal compiler classreader to achieve the same goal).
Public reflection API is split into scala.reflect.base and scala.reflect.api.
The former represents a minimalistic snapshot that is exactly enough to
build reified trees and types.
To build, but not to analyze - everything smart (for example, getting a type signature) is implemented in scala.reflect.api.
Both reflection domains have their own universe: scala.reflect.basis and
scala.reflect.runtime.universe.
The former is super lightweight and doesn't involve any classloaders,
while the latter represents a stripped down compiler.
Initial answer, Sept. 2011:
You can see evolutions of the reflect package in the Scala GitHub repo, with this two very recent commits:
Changes to Liftcode to use new reflection semantics, where a compiler uses type checking.
Started work on compiler toolbox that can compile reflect trees at runtime.
(Liftcode being, according to this thread, aims at simplifying "writing code that writes code")
The class scala/reflect/internal/Importers.scala (created yesterday!) is a good example of using those latest reflection feature.
Two links which should be of interest:
The scala-internals mailing list discussion on the reflection api.
The nightly build api doc for 2.10-SNAPSHOT.
Personally I am hoping to use this for doing runtime discovery of extensions (i.e. a type that extends a known trait), and generating UI forms and a few other things from those.
With current 2.10M4 you already can iterate over members of a class:
reflect.runtime.universe.typeOf[MyClass].members.filter(!_.isMethod)
The above code lists Symbol objects representing members of a class MyClass which are not methods. There are tons of ways you can fine-tune this.
Related
I'm building a tool that will receive unpredictable data structure, and I want to generate case class to accomplish the structure of the received data.
I'm trying to figure out if it's possible to generate case class at runtime? This structure will be know only at runtime.
It's something similar to what macro does, but in runtime.
I've found this project on the internet
mars
Which is very close to what I want to do ,but I couldn't find if it was successful of not.
Another way of doing it is generate the code, compile and put the result in the classpath, like IScala is doing to use the code in an iterative way. But I don't think that this will scale.
Does anybody has already done something like runtime code generation?
This question was also posted in scala-user mailing list
UPDATE: (as per the comments)
If all you want is throw-away code generated at runtime to be fed into to a library that cannot work with just lists and maps, and not code to be stored and used later, it would make sense to look for solutions to this problem for Java or JVM. That is, unless the library requires some Scala specific features not available to vanilla JVM bytecode (Scala adds some extras to the bytecode, which Java code doesn't need/have).
what is the benefit of generating statically typed code dynamically? as opposed to using a dynamic data structure.
I would not attempt that at all. Just use a structure such as nested lists and maps.
Runtime code generation is one of the purposes of the Mars Project. Mars is under development, at the moment there is no release version. Mars requires its own toolchain to expand macros at runtime and should use several features unique to scala.meta (http://scalameta.org/), for example, AST interpretation and AST persistence. Currently we are working on ASTs typechecking in scala-reflect, required for runtime macros expansion.
I just realized I cannot have annotations in scala, that are preserved and analyzed at runtime. I also checked this question, but I didn't quite get it what are the alternatives.
DI - an answer mentions that there is no need for DI framework in scala. While that might be the case on a basic level (although I didn't quite like that example; what's the idiomatic way of handling DI?), Java DI frameworks like spring are pretty advanced and handle many things like scheduled jobs, caching, managed persistence, etc, all through annotations, and sometimes - custom ones.
ORM - I'll admit I haven't tried any native scala ORM, but from what I see in squeryl, it also makes some use of annotations, meaning they are unavoidable?
any serialization tool - how do you idiomatically customize serialization output to JSON/XML/...?
Web service frameworks - how do you define (in code) the mappings, headers, etc. for RESTful or SOAP services?
Scala users need to have a hybrid scala/java (for the annotations) project in order to use these facilities that are coming from Java?
And are the native scala alternatives for meta-data nicer than annotations? I'm not yet fully into the scala mode of thinking, and therefore most of the examples look ugly to me, compared to using annotations, so please try to be extra convincing :)
Actually, Scala does have runtime-retained annotations. The difference is that they are not stored as Java annotations but are instead encoded inside the contents of binary ScalaSignature annotation (which, by itself, is a runtime-retained Java annotation).
So, Scala annotations can be retrieved at runtime, but instead of using Java reflection, one must use Scala reflection:
class Awesome extends StaticAnnotation
#Awesome
class AwesomeClass
import scala.reflect.runtime.universe._
val clazz = classOf[AwesomeClass]
val mirror = runtimeMirror(clazz.getClassLoader)
val symbol = mirror.classSymbol(clazz)
println(symbol.annotations) // prints 'List(Awesome)'
Unfortunately, Scala reflection is still marked as experimental and is actually unstable at this point (SI-6240 or SI-6826 are examples of quite serious issues). Nevertheless, it seems like the most straightforward replacement for Java reflection and annotations in the long term.
As for now, one has to use Java annotations which I think is still a nice solution.
Regarding frameworks and libraries for DI/ORM/WS/serialization - Scala still doesn't seem to be mature in this area, at least not as Java is. There are plenty of ongoing projects targeting these problems, some of them are already really nice, others are still in development. To name a few that come to my mind: Squeryl, Slick, Spray, Pickling.
Also, Scala has some advanced features that often make annotations unneccessary. Typeclasses (implemented with implicit parameters) are probably good example of that.
After watching Martin's keynote on Reflection and Compilers I can't seem to get this crazy question out of my head. Martin talks among other things about the "(Wedding) Cake Pattern", where traits play the central part. I'm wondering, why in the world do we need packages when we already have traits? Is there anything a package can do, what a trait (at least theoretically) cannot?
I'm not talking about the current implementation, I'm just trying to imagine what programming would be like if we replace packages with traits. In my head it would be like this:
one keyword less (package is unneeded)
no need for package objects
To summarize all my questions:
Is it theoretically possible to remove packages from the language and use traits instead.
What other benefits would we gain from this change? (I was thinking about first class packages and first class imports, but mixin composition is a compile time thing, although the super calls are dynamically bound)
Is Java/JVM compatibility the only thing, which would stand in the way?
Update
Daniel Spiewak talks in this keynote about the Dependency Injection being just the top of the iceberg of all the stuff you can do with the Cake Pattern.
Martin Odersky has said that Scala could get by with just traits, objects, methods and paths (I hope I didn't forget something).
Both classes and packages are just there because Scala is intended to be a hosted language, i.e. a language which runs on (this is actually not the interesting bit) and interoperates with (this is the important point) a host platform. Some of the host platforms that Scala is intended to interoperate with are the Java platform and the CLI, both which have a concept of classes and packages (namespaces in the case of the CLI) that is significantly distinct enough that it cannot be easily expressed as traits or objects. This is unlike interfaces, which can be trivially mapped to and from purely abstract traits.
The above statement was made in a discussion about potentially removing generics from Scala, because everything generics can do can also be achieved by abstract types.
In scala the object and package serve almost the same purpose and objects are also called modules. Objects deserve to be thought of as modules because they can contain any definition including other objects of course and, significantly, types.
A trait can be thought of as an abstract module. It can contain any definition and any member can be abstract including, again significantly, type members. I am reciting all this just to highlight the symmetry. Perhaps OT but to me traits seem to be as big an innovation in scala as the merging of object and functional ideas.
To finally give an answer:
I think packages could be removed in favour of objects (not traits).
The benefit would be a simplification - package objects would not need to be explicitly defined.
I think packages are distinct from objects for Java/JVM compatibility.
Some more commentary: in the video Martin talks of traits (abstract modules) more than concrete modules because the latter only appear at the last moment to assemble and reify some combination of abstract modules.
It is good to use abstract modules even when not "mixing a cake". e.g. when sketching out some code you might define a module to contain definitions. But as soon as you come to a type or value you are not ready to fill in, don't supply a dummy such as null. Instead switch the object to a trait and leave the member abstract.
I was a little surprised when I started using Lift how heavily it uses reflection (or appears to), it was a little unexpected in a statically-typed functional language. My experience with JSP was similar.
I'm pretty new to web development, so I don't really know how these tools work, but I'm wondering,
What aspects of web development encourage using reflection?
Are there any tools (in statically typed languages) that handle (1) referring to code from a template page (2) object-relational mapping, in a way that does not use reflection?
Please see lift source. It doesn't use reflection for most of the code that I have studied. Almost everything is statically typed. If you are referring to lift views they are processed as Xml nodes, that too is not reflection.
Specifically referring to the <lift:Foo.bar/> issue:
When <lift:Foo.bar/> is encountered in the code, Lift makes a few guesses, how the original name should have been (different naming conventions) and then calls java.lang.Class.forName to get the class. (Relevant code in LiftSession.scala and ClassHelpers.scala.) It will only find classes registered with addToPackages during boot.
Note that it is also possible (and common) to register classes and methods manually. Convention is still that all transformations must be of the form NodeSeq => NodeSeq because that is the only thing which makes sense for an untyped HTML/XHTML output.
So, what you have is Lift‘s internal registry of node transformations on one side, and on the other side the implicit registry of the module. Both types use a simple string lookup to execute a method. I guess it is arguable if one is more reflection based than the other.
I'm still at the beginning in learning scala in addition to java and i didn't get it how is one supposed to do DI there? can or should i use an existing DI library, should it be done manually or is there another way?
Standard Java DI frameworks will usually work with Scala, but you can also use language constructs to achieve the same effect without external dependencies.
A new dependency injection library specifically for Scala is Dick Wall's SubCut.
Whereas the Jonas Bonér article referenced in Dan Story's answer emphasizes compile-time bound instances and static injection (via mix-ins), SubCut is based on runtime initialization of immutable modules, and dynamic injection by querying the bound modules by type, string names, or scala.Symbol names.
You can read more about the comparison with the Cake pattern in the GettingStarted document.
Dependency Injection itself can be done without any tool, framework or container support. You only need to remove news from your code and move them to constructors. The one tedious part that remains is wiring the objects at "the end of the world", where containers help a lot.
Though with Scala's 2.10 macros, you can generate the wiring code at compile-time and have auto-wiring and type-safety.
See the Dependency Injection in Scala Guide
A recent project illustrates a DI based purely on constructor injection: zalando/grafter
What's wrong with constructor injection again?
There are many libraries or approaches for doing dependency injection in Scala. Grafter goes back to the fundamentals of dependency injection by just using constructor injection: no reflection, no xml, no annotations, no inheritance or self-types.
Then, Grafter add to constructor injection just the necessary support to:
instantiate a component-based application from a configuration
fine-tune the wiring (create singletons)
test the application by replacing components
start / stop the application
Grafter is targeting every possible application because it focuses on associating just 3 ideas:
case classes and interfaces for components
Reader instances and shapeless for the configuration
tree rewriting and kiama for everything else!
I haven't done so myself, but most DI frameworks work at the bytecode level (AFAIK), so it should be possible to use them with any JVM language.
Previous posts covered the techniques. I wanted to add a link to Martin Odersky's May 2014 talk on the Scala language objectives. He identifies languages that "require" a DI container to inject dependencies as poorly implemented. I agree with this personally, but it is only an opinion. It does seem to indicate that including a DI dependency in your Scala project is non-idiomatic, but again this is opinion. Practically speaking, even with a language designed to inject dependencies natively, there is a certain amount of consistency gained by using a container. It is worth considering both points of view for your purposes.
https://youtu.be/ecekSCX3B4Q?t=1154
I would suggest you to try distage (disclaimer: I'm the author).
It allows you to do much more than a typical DI does and has many unique traits:
distage supports multiple configurations (e.g. you may run your app
with different sets of component implementations),
distage allows you to correctly share dependencies across your tests
and easily run same tests for different implementations of your
components,
distage supports roles so you may run multiple services within the same process sharing dependencies between them,
distage does not depend on scala-reflect
(but supports all the necessary features of Scala typesystem, like
higher-kinded types).
You may also watch our talk at Functional Scala 2019 where we've discussed and demonstrated some important capabiliteis of distage.
I have shown how I created a very simple functional DI container in scala using 2.10 here.
In addition to the answer of Dan Story, I blogged about a DI variant that also uses language constructs only but is not mentioned in Jonas's post: Value Injection on Traits (linking to web.archive.org now).
This pattern is working very well for me.