I need to get the AST of the implementation of a lambda function. And I get stuck because reify works on the argument and not on it's value.
val x = (a: Int) => println("a")
val t = showRaw(reify(thevalueof x)) //something here should change
println(t)
should print:
Expr(Function(List(ValDef(Modifiers(PARAM), newTermName("a"), Ident(scala.Int), EmptyTree)), Apply(Select(Select(This(newTypeName("scala")), newTermName("Predef")), newTermName("println")), List(Literal(Constant("a"))))))
I supposed there should be a trick with splice or so, but I just can't manage to do this.
Currently there is no robust way of getting ASTs of the program outside of a macro application (and reify is a macro, so it abides by the same rules). However, we are experimenting with a next-gen macro engine that might fix this problem: http://scalamacros.org/news/2014/03/02/project-palladium.html.
Related
I have a function that expects a variable number of parameters of the same type, which sounds like the textbook use case for varargs:
def myFunc[A](as: A*) = ???
The problem I have is that myFunc cannot accept empty parameter lists. There's a trivial way of enforcing that at runtime:
def myFunc[A](as: A*) = {
require(as.nonEmpty)
???
}
The problem with that is that it happens at runtime, as opposed to compile time. I would like the compiler to reject myFunc().
One possible solution would be:
def myFunc[A](head: A, tail: A*) = ???
And this works when myFunc is called with inline arguments, but I'd like users of my library to be able to pass in a List[A], which this syntax makes very awkward.
I could try to have both:
def myFunc[A](head: A, tail: A*) = myFunc(head +: tail)
def myFunc[A](as: A*) = ???
But we're right back where we started: there's now a way of calling myFunc with an empty parameter list.
I'm aware of scalaz's NonEmptyList, but in as much as possible, I'd like to stay with stlib types.
Is there a way to achieve what I have in mind with just the standard library, or do I need to accept some runtime error handling for something that really feels like the compiler should be able to deal with?
What about something like this?
scala> :paste
// Entering paste mode (ctrl-D to finish)
def myFunc()(implicit ev: Nothing) = ???
def myFunc[A](as: A*) = println(as)
// Exiting paste mode, now interpreting.
myFunc: ()(implicit ev: Nothing)Nothing <and> [A](as: A*)Unit
myFunc: ()(implicit ev: Nothing)Nothing <and> [A](as: A*)Unit
scala> myFunc(3)
WrappedArray(3)
scala> myFunc(List(3): _*)
List(3)
scala> myFunc()
<console>:13: error: could not find implicit value for parameter ev: Nothing
myFunc()
^
scala>
Replacing Nothing with a class that has an appropriate implicitNotFound annotation should allow for a sensible error message.
Let's start out with what I think is your base requirement: the ability to define myFunc in some way such that the following occurs at the Scala console when a user provides literals. Then maybe if we can achieve that, we can try to go for varargs.
myFunc(List(1)) // no problem
myFunc(List[Int]()) // compile error!
Moreover, we don't want to have to force users either to split a list into a head and tail or have them convert to a ::.
Well when we're given literals, since we have access to the syntax used to construct the value, we can use macros to verify that a list is non-empty. Moreover, there's already a library that'll do it for us, namely refined!
scala> refineMV[NonEmpty]("Hello")
res2: String Refined NonEmpty = Hello
scala> refineMV[NonEmpty]("")
<console>:39: error: Predicate isEmpty() did not fail.
refineMV[NonEmpty]("")
^
Unfortunately this is still problematic in your case, because you'll need to put refineMV into the body of your function at which point the literal syntactically disappears and macro magic fails.
Okay what about the general case that doesn't rely on syntax?
// Can we do this?
val xs = getListOfIntsFromStdin() // Pretend this function exists
myFunc(xs) // compile error if xs is empty
Well now we're up against a wall; there's no way a compile time error can happen here since the code has already been compiled and yet clearly xs could be empty. We'll have to deal with this case at runtime, either in a type-safe manner with Option and the like or with something like runtime exceptions. But maybe we can do a little better than just throw our hands up in the air. There's two possible paths of improvement.
Somehow provide implicit evidence that xs is nonempty. If the compiler can find that evidence, then great! If not, it's on the user to provide it somehow at runtime.
Track the provenance of xs through your program and statically prove that it must be non-empty. If this cannot be proved, either error out at compile time or somehow force the user to handle the empty case.
Once again, unfortunately this is problematic.
I strongly suspect this is not possible (but this is still only a suspicion and I would be happy to be proved wrong). The reason is that ultimately implicit resolution is type-directed which means that Scala gets the ability to do type-level computation on types, but Scala has no mechanism that I know of to do type-level computation on values (i.e. dependent typing). We require the latter here because List(1, 2, 3) and List[Int]() are indistinguishable at the type level.
Now you're in SMT solver land, which does have some efforts in other languages (hello Liquid Haskell!). Sadly I don't know of any such efforts in Scala (and I imagine it would be a harder task to do in Scala).
The bottom line is that when it comes to error checking there is no free lunch. A compiler can't magically make error handling go away (although it can tell you when you don't strictly need it), the best it can do is yell at you when you forget to handle certain classes of errors, which is itself very valuable. To underscore the no free lunch point, let's return to a language that does have dependent types (Idris) and see how it handles non-empty values of List and the prototypical function that breaks on empty lists, List.head.
First we get a compile error on empty lists
Idris> List.head []
(input):1:11:When checking argument ok to function Prelude.List.head:
Can't find a value of type
NonEmpty []
Good, what about non-empty lists, even if they're obfuscated by a couple of leaps?
Idris> :let x = 5
-- Below is equivalent to
-- val y = identity(Some(x).getOrElse(3))
Idris> :let y = maybe 3 id (Just x)
-- Idris makes a distinction between Natural numbers and Integers
-- Disregarding the Integer to Nat conversion, this is
-- val z = Stream.continually(2).take(y)
Idris> :let z = Stream.take (fromIntegerNat y) (Stream.repeat 2)
Idris> List.head z
2 : Integer
It somehow works! What if we really don't let the Idris compiler know anything about the number we pass along and instead get one at runtime from the user? We blow up with a truly gargantuan error message that starts with When checking argument ok to function Prelude.List.head: Can't find a value of type NonEmpty...
import Data.String
generateN1s : Nat -> List Int
generateN1s x = Stream.take x (Stream.repeat 1)
parseOr0 : String -> Nat
parseOr0 str = case parseInteger str of
Nothing => 0
Just x => fromIntegerNat x
z : IO Int
z = do
x <- getLine
let someNum = parseOr0 x
let firstElem = List.head $ generateN1s someNum -- Compile error here
pure firstElem
Hmmm... well what's the type signature of List.head?
Idris> :t List.head
-- {auto ...} is roughly the same as Scala's implicit
head : (l : List a) -> {auto ok : NonEmpty l} -> a
Ah so we just need to provide a NonEmpty.
data NonEmpty : (xs : List a) -> Type where
IsNonEmpty : NonEmpty (x :: xs)
Oh a ::. And we're back at square one.
Use scala.collection.immutable.::
:: is the cons of the list
defined in std lib
::[A](head: A, tail: List[A])
use :: to define myFunc
def myFunc[A](list: ::[A]): Int = 1
def myFunc[A](head: A, tail: A*): Int = myFunc(::(head, tail.toList))
Scala REPL
scala> def myFunc[A](list: ::[A]): Int = 1
myFunc: [A](list: scala.collection.immutable.::[A])Int
scala> def myFunc[A](head: A, tail: A*): Int = myFunc(::(head, tail.toList))
myFunc: [A](head: A, tail: A*)Int
I am scratching my head on an example I've seen in breeze's documentation about distributions.
After creating a Rand instance, they show that you can do the following:
import breeze.stats.distributions._
val pois = new Poisson(3.0);
val doublePoi: Rand[Double] = for(x <- pois) yield x.toDouble
Now, this is very cool, I can get a Rand object that I can get Double instead of Int when I call the samples method. Another example might be:
val abc = ('a' to 'z').map(_.toString).toArray
val letterDist: Rand[String] = for(x <- pois) yield {
val i = if (x > 26) x % 26 else x
abc(i)
}
val lettersSamp = letterDist.samples.take(20)
println(letterSamp)
The question is, what is going on here? Rand[T] is not a collection, and all the for/yield examples I've seen so far work on collections. The scala docs don't mention much, the only thing I found is translating for-comprehensions in here.
What is the underlying rule here? How else can this be used (doesn't have to be a breeze related answer)
Scala has rules for translating for and for-yield expressions to the equivalent flatMap and map calls, optionally also applying filters using withFilter and such. The actual specification for how you translate each for comprehension expression into the equivalent method calls can be found in this section of the Scala Specification.
If we take your example and compile it we'll see the underlying transformation happen to the for-yield expression. This is done using scalac -Xprint:typer command to print out the type trees:
val letterDist: breeze.stats.distributions.Rand[String] =
pois.map[String](((x: Int) => {
val i: Int = if (x.>(26))
x.%(26)
else
x;
abc.apply(i)
}));
Here you can see that for-yield turns into a single map passing in an Int and applying the if-else inside the expression. This works because Rand[T] has a map method defined:
def map[E](f: T => E): Rand[E] = MappedRand(outer, f)
For comprehensions are just syntactic sugar for flatMap, map and withFilter. The main requirement for use in a for comprehension is that those methods are implemented. They are therefore not limited to collections E.g. some common non-collections used in for comprehensions are Option, Try and Future.
In your case, Poisson seems to inherit from a trait called Rand
https://github.com/scalanlp/breeze/blob/master/math/src/main/scala/breeze/stats/distributions/Rand.scala
This trait has map, flatmap, and withFilter defined.
Tip: If you use an IDE like IntelliJ - you can press alt+enter on your for comprehension, and chose convert to desugared expression and you will see how it expands.
How can you define a function myEval(f, args) in Scala which takes as input another function f and arguments args and whose output is f(args)?
I don't want myEval to have any prior knowledge about the arity or argument types of f.
Why is this useful? It is one way to solve the problem of implementing a generic timeMyFunction(f, args) method. If there's a way to do this by some sort of lazy val construction, that would also be interesting.
Edit: The better way to implement a timing method is explained in this question. By calling timeMyFunction( { f(args) } ), the function call is wrapped in an anonymous function Unit => Unit. So timeMyFunction only needs to take 0-arity functions.
Edit 2: See Dirk's answer for what is perhaps a more efficient way which avoids an anonymous function by passing f by reference.
So my justification for the question is now purely my Scala education.
The Scala standard library isn't going to help you generalize over arity in most cases, but Shapeless is perfect for this. Here's how you could write your function in Shapeless 1.2.4:
import shapeless._
def foo[F, P <: Product, A <: HList, R](f: F, p: P)(implicit
fl: FnHListerAux[F, A => R],
pl: HListerAux[P, A]
): R = fl(f)(pl(p))
And then:
scala> foo((i: Int, s: String) => s * i, (3, "a"))
res0: String = aaa
It looks complicated, but essentially you're just saying that you need evidence that a function f of some arbitrary arity can be converted to a single-argument function from a heterogeneous list A to the result R, and that the tuple P can be converted to a heterogeneous list of the same type.
An alternative using pass-by-name would be:
def timeIt[T](thunk: =>T): T = {
// ... set-up the timer
val answer: T = thunk
// ... evaluate the result of the timer
answer // in case, you need the result and want the timing to
} // happen just as a side-effect
timeIt(someFunction(someArg1, ...))
Though this looks as if it would call someFunction directly, it does not, since timeIt takes the argument "by name", i.e., the scala compiler generates a hidden closure, which performs the call, when the actual value is needed in timeIt itself.
This variant may introduce some timing noise due to the overhead of the "pass-by-name" convention.
I have the following method:
def lift[P <: Product, L <: HList](params: P)(implicit hl: Generic.Aux[P, L]) = {
directive[L](_(hl to params))
}
and it perfectly works if i pass more then two arguments:
val result = lift("string", 'a', 10) // compiles
val result2 = list(true, 5) // compiles
But when i'm passing a single argument it can't resolve implicit:
val failes = lift("string")
It can't find Generic implicit for [String, Nothing], why does it work in other cases?
You're seeing the result of auto-tupling, which is a Scala (mis-)feature that causes lift(true, 5) to be parsed as lift((true, 5)) when there's no lift method with the appropriate number of values (two, in this case). The compiler won't automatically wrap a single value in a Tuple1, however—you just get a compiler error.
See for example this answer for more details about auto-tupling, and this thread for some reasons auto-tupling is a terrible thing to include in your language.
There are a couple of possible workarounds. The first would be to create an implicit conversion from values to Tuple1, as suggested in this answer. I wouldn't personally recommend this approach—every implicit conversion you introduce into your code is another mine in the minefield.
Instead I'd suggest avoiding autotupling altogether. Write out list((true, 5)) explicitly—you get a lot of extra clarity at the cost of only a couple of extra characters. Unfortunately there's no comparable literal support for Tuple1, so you have to write out lift(Tuple1("string")), but even that's not too bad, and if you really wanted you could define a new liftOne method that would do it for you.
In languages like SML, Erlang and in buch of others we may define functions like this:
fun reverse [] = []
| reverse x :: xs = reverse xs # [x];
I know we can write analog in Scala like this (and I know, there are many flaws in the code below):
def reverse[T](lst: List[T]): List[T] = lst match {
case Nil => Nil
case x :: xs => reverse(xs) ++ List(x)
}
But I wonder, if we could write former code in Scala, perhaps with desugaring to the latter.
Is there any fundamental limitations for such syntax being implemented in the future (I mean, really fundamental -- e.g. the way type inference works in scala, or something else, except parser obviously)?
UPD
Here is a snippet of how it could look like:
type T
def reverse(Nil: List[T]) = Nil
def reverse(x :: xs: List[T]): List[T] = reverse(xs) ++ List(x)
It really depends on what you mean by fundamental.
If you are really asking "if there is a technical showstopper that would prevent to implement this feature", then I would say the answer is no. You are talking about desugaring, and you are on the right track here. All there is to do is to basically stitch several separates cases into one single function, and this can be done as a mere preprocessing step (this only requires syntactic knowledge, no need for semantic knowledge). But for this to even make sense, I would define a few rules:
The function signature is mandatory (in Haskell by example, this would be optional, but it is always optional whether you are defining the function at once or in several parts). We could try to arrange to live without the signature and attempt to extract it from the different parts, but lack of type information would quickly come to byte us. A simpler argument is that if we are to try to infer an implicit signature, we might as well do it for all the methods. But the truth is that there are very good reasons to have explicit singatures in scala and I can't imagine to change that.
All the parts must be defined within the same scope. To start with, they must be declared in the same file because each source file is compiled separately, and thus a simple preprocessor would not be enough to implement the feature. Second, we still end up with a single method in the end, so it's only natural to have all the parts in the same scope.
Overloading is not possible for such methods (otherwise we would need to repeat the signature for each part just so the preprocessor knows which part belongs to which overload)
Parts are added (stitched) to the generated match in the order they are declared
So here is how it could look like:
def reverse[T](lst: List[T]): List[T] // Exactly like an abstract def (provides the signature)
// .... some unrelated code here...
def reverse(Nil) = Nil
// .... another bit of unrelated code here...
def reverse(x :: xs ) = reverse(xs) ++ List(x)
Which could be trivially transformed into:
def reverse[T](list: List[T]): List[T] = lst match {
case Nil => Nil
case x :: xs => reverse(xs) ++ List(x)
}
// .... some unrelated code here...
// .... another bit of unrelated code here...
It is easy to see that the above transformation is very mechanical and can be done by just manipulating a source AST (the AST produced by the slightly modified grammar that accepts this new constructs), and transforming it into the target AST (the AST produced by the standard scala grammar).
Then we can compile the result as usual.
So there you go, with a few simple rules we are able to implement a preprocessor that does all the work to implement this new feature.
If by fundamental you are asking "is there anything that would make this feature out of place" then it can be argued that this does not feel very scala. But more to the point, it does not bring that much to the table. Scala author(s) actually tend toward making the language simpler (as in less built-in features, trying to move some built-in features into libraries) and adding a new syntax that is not really more readable goes against the goal of simplification.
In SML, your code snippet is literally just syntactic sugar (a "derived form" in the terminology of the language spec) for
val rec reverse = fn x =>
case x of [] => []
| x::xs = reverse xs # [x]
which is very close to the Scala code you show. So, no there is no "fundamental" reason that Scala couldn't provide the same kind of syntax. The main problem is Scala's need for more type annotations, which makes this shorthand syntax far less attractive in general, and probably not worth the while.
Note also that the specific syntax you suggest would not fly well, because there is no way to distinguish one case-by-case function definition from two overloaded functions syntactically. You probably would need some alternative syntax, similar to SML using "|".
I don't know SML or Erlang, but I know Haskell. It is a language without method overloading. Method overloading combined with such pattern matching could lead to ambiguities. Imagine following code:
def f(x: String) = "String "+x
def f(x: List[_]) = "List "+x
What should it mean? It can mean method overloading, i.e. the method is determined in compile time. It can also mean pattern matching. There would be just a f(x: AnyRef) method that would do the matching.
Scala also has named parameters, which would be probably also broken.
I don't think that Scala is able to offer more simple syntax than you have shown in general. A simpler syntax may IMHO work in some special cases only.
There are at least two problems:
[ and ] are reserved characters because they are used for type arguments. The compiler allows spaces around them, so that would not be an option.
The other problem is that = returns Unit. So the expression after the | would not return any result
The closest I could come up with is this (note that is very specialized towards your example):
// Define a class to hold the values left and right of the | sign
class |[T, S](val left: T, val right: PartialFunction[T, T])
// Create a class that contains the | operator
class OrAssoc[T](left: T) {
def |(right: PartialFunction[T, T]): T | T = new |(left, right)
}
// Add the | to any potential target
implicit def anyToOrAssoc[S](left: S): OrAssoc[S] = new OrAssoc(left)
object fun {
// Use the magic of the update method
def update[T, S](choice: T | S): T => T = { arg =>
if (choice.right.isDefinedAt(arg)) choice.right(arg)
else choice.left
}
}
// Use the above construction to define a new method
val reverse: List[Int] => List[Int] =
fun() = List.empty[Int] | {
case x :: xs => reverse(xs) ++ List(x)
}
// Call the method
reverse(List(3, 2, 1))