I would like to develop a tool that post-processes a scala program once all the heavy lifting has been completed by the Scala compiler. From what I understand the different phases of the Scala compiler incrementally simplify the program in terms of its syntactic sugars and advanced features like lambdas, closures, pattern-matching etc. However, I notice that what comes out of the so-called cleanup phase - which is the last phase before code-generation - looks like scala but it is not really scala.
Does anyone know personally or can point me to a resource that can help me understand the language that comes out of the cleanup phase ?
To give you an example, in the output of the cleanup phase I see things like:
case <synthetic> val x1: Foo$Bar = l;
case9(){
if (...some condition...)
matchEnd8(scala.Predef.Set().empty())
else
case10()
};
My hypothesis is that this is the result of translating pattern matching but it does not look like valid scala syntax as far as I understand (I am not an experienced Scala developer at all!).
I guess it all comes down to this: is it possible to convert the output of the cleanup phase to valid - compilable - scala code in general ?
In general, at any stage in the scalac compiler (even right after parsing), the internal representation used by the compiler is not valid Scala code anymore. That is essentially because of the existence of labels and gotos, which you discovered.
A structure of the form
labelName(...params){
...
}
is a label definition, and a call of the form
labelName(...args)
is a jump to that label, assigning the ...args to the ...params.
Labels and gotos are used by scalac (and dotc, but with a different representation) to represent while and do..while loops (immediately after parsing), the translation of matches and the tail-recursive-optimized functions.
In general, there is no way to go back from the internal representation to valid Scala code, especially so far in the pipeline as after cleanup.
Related
Is the reason a val variable can be used to contain a function definition is because functions are first class citizens where they can be contained in variables?
In Scala damn near everything is an expression. From a practical perspective what that means is pretty much every bit of syntactically correct Scala code that you can write evaluates to an object that can you can do more Scala on. Examples of things you can do to these objects are: call a method on it, pass it to a function, or store it in a val. Expressions can be thought of in contrast to statements, which are just instructions to the computer to do something. An example of the use of statements in Scala are import commands. The heavy prevalence of expressions in Scala are a deliberate design choice intended to make the language more flexible and extensible.
Both F# and Scala act as a hybrid language that is often used to bridge the words of tradional object oriented code to functional code.
A concept that belongs more to the OO world are exceptions, whereas the functional world favors Option types in many cases.
To wrap existing library code that relies on exceptions - and make it more functional - I would thus like to "lift" exception-throwing code to instead return an option type.
In Scala, there is a nice library function to "catch all" and convert to option. It can be used like this:
import scala.util.control.Exception._
val functionalVersion = allCatch opt myFunction
see In Scala, is there a pre-existing library function for converting exceptions to Options?
Now that I'm moving to F# I have the same requirement, but I can't seem to find an existing utility function for this - and also struggle to implement one myself.
I can create such a wrapper for a unit function, aka an action
let catchAll f = try Some (f()) with | _ -> None
But the problem here is that I don't want to first wrap all the exeption throwing code into an action.
For example I would like to wrap the array-indexing operator, so that it doesn't throw.
// Wrap out-of-bounds exception with option type
let maybeGetIndex (array: int[]) (index: int) = catchAll (fun () -> array.[index])
maybeGetIndex [| 1; 2; 3 |] 10 // -> None
However, it would be much nicer if one could simple write
(catchAll a.[index])
i.e. apply catchAll to a whole expression before it is evaluated.
(Scala can achieve this through call-by-name parameters which seem to be missing from F#)
So this question is twofold:
Is there an existing library function to wrap exceptions into option
types?
Is there a language feature that would allow me to implement
it?
First of all, I think that it's not true that a "concept that belongs more to the object-oriented world are exceptions". Exceptions exist in many functional languages of the ML family and, for example, OCaml relies on them quite heavily and uses them even for certain control flow structures. In F#, this is not so much the case, because .NET exceptions are somewhat slower, but I see exceptions very much orthogonal to the object-oriented/functional issue.
For this reason, I actually find exceptions often preferable to option types in F#. The downside is that there is less type-checking (you do not know what might throw), but the upside is that the language provides a nice integrated langauge support for exceptions. If you need to handle exceptional situations then exceptions are a good way of doing that!
To answer your original question about syntactic tricks you can make - I would probably just use a function as your existing code does, because that's explicit and easy to understand (and presumably, you'll only need exception wrapping in some core functions that you implement).
That said, you can define a computation expression builder that wraps the code in the body and serves as your catchAll function with a somewhat neater syntax:
type CatchAllBuilder() =
member x.Delay(f) = try Some(f()) with _ -> None
member x.Return(v) = v
let catchAll = CatchAllBuilder()
This lets you write something like:
catchAll { return Array.empty.[0] }
As mentioned earlier, I wouldn't do this because (i) I don't think all exceptions need to be converted to options in F# and (ii) it introduces unfamiliar syntax that new team members (and future you) might be confused by, but it is probably the nicest syntax you can get.
[EDIT: Now a working version with return - this is somewhat less pretty, but perhaps still useful!]
I have a factory that should return an implementation depending on the name.
val moduleMap = Map(Modules.moduleName -> new ModuleImpl)
def getModule(moduleName: String): Module =
moduleMap.get(moduleName) match {
case Some(m) => m
case _ =>
throw new ModuleNotFoundException(
s"$moduleName - Module could not be found.")
}
In order for each call to the "getModule" method not to create an instance, there is a map in which all the modules must be initialized in bootstrap class.
I would like to get rid of the need to do this manually(also all classes have a distinctive feature).
List of options that came to my mind:
Reflection(we can use Scala Reflection API or any thrid-party
library)
Automated process.
Need to initialize immediately at startup.
Reflection is a pain.
Metaprogramming(ScalaMeta) + Reflection
Macros only change the code, the execution happens later.
Can we move initialization process to compile time?
I know that compiler can optimize and replace code, next fragment before compilation
val a = 5 + 5
after compilation compiler change that piece to 10, can we use some directives or another tools to evaluate and execute some code at compile time and use only final value?
Do you use any framework or you write your own? I answered similar question about Guice here. You can use it without Guice as well: instead of Module you will have your Factory, which you need to initialize from somewhere, and during initialization, you will fill your map using reflection
In general I think it is the easiest approach. Alternatively, you can write macros, which just replaces part of reflective initialization, but not sure that it will give you some profit (if I understand your question right, this initialization will happen just once at startup).
I do not see how scalameta can help you? Probably, only in case if all your implementations are in source tree available to you, so you can analyze it and generate initialization (similar to macros)? Probably, this would add such plus as easier search for implementation, but will add minus: will work only on implementations in your sources.
Your example of compile-time optimization is not applicable. In your example, you talk about compile-time constant (even with arithmetic it could be a problem, see this comment), but in your question you need specific run-time behavior. So compile time could be only code generation from macros or based on scalameta from my point of view.
In groovy one can do:
class Foo {
Integer a,b
}
Map map = [a:1,b:2]
def foo = new Foo(map) // map expanded, object created
I understand that Scala is not in any sense of the word, Groovy, but am wondering if map expansion in this context is supported
Simplistically, I tried and failed with:
case class Foo(a:Int, b:Int)
val map = Map("a"-> 1, "b"-> 2)
Foo(map: _*) // no dice, always applied to first property
A related thread that shows possible solutions to the problem.
Now, from what I've been able to dig up, as of Scala 2.9.1 at least, reflection in regard to case classes is basically a no-op. The net effect then appears to be that one is forced into some form of manual object creation, which, given the power of Scala, is somewhat ironic.
I should mention that the use case involves the servlet request parameters map. Specifically, using Lift, Play, Spray, Scalatra, etc., I would like to take the sanitized params map (filtered via routing layer) and bind it to a target case class instance without needing to manually create the object, nor specify its types. This would require "reliable" reflection and implicits like "str2Date" to handle type conversion errors.
Perhaps in 2.10 with the new reflection library, implementing the above will be cake. Only 2 months into Scala, so just scratching the surface; I do not see any straightforward way to pull this off right now (for seasoned Scala developers, maybe doable)
Well, the good news is that Scala's Product interface, implemented by all case classes, actually doesn't make this very hard to do. I'm the author of a Scala serialization library called Salat that supplies some utilities for using pickled Scala signatures to get typed field information
https://github.com/novus/salat - check out some of the utilities in the salat-util package.
Actually, I think this is something that Salat should do - what a good idea.
Re: D.C. Sobral's point about the impossibility of verifying params at compile time - point taken, but in practice this should work at runtime just like deserializing anything else with no guarantees about structure, like JSON or a Mongo DBObject. Also, Salat has utilities to leverage default args where supplied.
This is not possible, because it is impossible to verify at compile time that all parameters were passed in that map.
This is just one of those "I was wondering..." questions.
Scala has immutable data structures and (optional) lazy vals etc.
How close can a Scala program be to one that is fully pure (in a functional programming sense) and fully lazy (or as Ingo points out, can it be sufficiently non-strict)? What values are unavoidably mutable and what evaluation unavoidably greedy?
Regarding lazyness - currently, passing a parameter to a method is by default strict:
def square(a: Int) = a * a
but you use call-by-name parameters:
def square(a: =>Int) = a * a
but this is not lazy in the sense that it computes the value only once when needed:
scala> square({println("calculating");5})
calculating
calculating
res0: Int = 25
There's been some work into adding lazy method parameters, but it hasn't been integrated yet (the below declaration should print "calculating" from above only once):
def square(lazy a: Int) = a * a
This is one piece that is missing, although you could simulate it with a local lazy val:
def square(ap: =>Int) = {
lazy val a = ap
a * a
}
Regarding mutability - there is nothing holding you back from writing immutable data structures and avoid mutation. You can do this in Java or C as well. In fact, some immutable data structures rely on the lazy primitive to achieve better complexity bounds, but the lazy primitive can be simulated in other languages as well - at the cost of extra syntax and boilerplate.
You can always write immutable data structures, lazy computations and fully pure programs in Scala. The problem is that the Scala programming model allows writing non pure programs as well, so the type checker can't always infer some properties of the program (such as purity) which it could infer given that the programming model was more restrictive.
For example, in a language with pure expressions the a * a in the call-by-name definition above (a: =>Int) could be optimized to evaluate a only once, regardless of the call-by-name semantics. If the language allows side-effects, then such an optimization is not always applicable.
Scala can be as pure and lazy as you like, but a) the compiler won't keep you honest with regards to purity and b) it will take a little extra work to make it lazy. There's nothing too profound about this; you can even write lazy and pure Java code if you really want to (see here if you dare; achieving laziness in Java requires eye-bleeding amounts of nested anonymous inner classes).
Purity
Whereas Haskell tracks impurities via the type system, Scala has chosen not to go that route, and it's difficult to tack that sort of thing on when you haven't made it a goal from the beginning (and also when interoperability with a thoroughly impure language like Java is a major goal of the language).
That said, some believe it's possible and worthwhile to make the effort to document effects in Scala's type system. But I think purity in Scala is best treated as a matter of self-discipline, and you must be perpetually skeptical about the supposed purity of third-party code.
Laziness
Haskell is lazy by default but can be made stricter with some annotations sprinkled in your code... Scala is the opposite: strict by default but with the lazy keyword and by-name parameters you can make it as lazy as you like.
Feel free to keep things immutable. On the other hand, there's no side effect tracking, so you can't enforce or verify it.
As for non-strictness, here's the deal... First, if you choose to go completely non-strict, you'll be forsaking all of Scala's classes. Even Scalaz is not non-strict for the most part. If you are willing to build everything yourself, you can make your methods non-strict and your values lazy.
Next, I wonder if implicit parameters can be non-strict or not, or what would be the consequences of making them non-strict. I don't see a problem, but I could be wrong.
But, most problematic of all, function parameters are strict, and so are closures parameters.
So, while it is theoretically possible to go fully non-strict, it will be incredibly inconvenient.