Methods are often declared with obvious parameter names, e.g.
def myMethod(s: String, image: BufferedImage, mesh: Mesh) { ... }
Parameter names correspond to parameter types.
1) "s" is often used for String
2) "i" for Int
3) lowercased class name for one word named classes (Mesh -> mesh)
4) lowercased last word from class name for long class names (BufferedImage -> image)
(Of course, it would not be convenient for ALL methods and arguments. Of course, somebody would prefer other rules…)
Scala macros are intended to generate some expressions in code. I would like to write some specific macros to convert to correct Scala expressions something like this:
// "arguments interpolation" style
// like string interpolation
def myMethod s[String, BufferedImage, Mesh]
{ /* code using vars "s", "image", "mesh" */ }
// or even better:
mydef myMethod[String, BufferedImage, Mesh]
{ /* code using vars "s", "image", "mesh" */ }
Is it possible?
Currently it is not possible and probably it will never be. Macros can not introduce their own syntax - they must be represented through valid Scala code (which can be executed at compile time) and, too, they must generate valid Scala code (better say a valid Scala AST).
Both of your shown examples are not valid Scala code, thus Macros can not handle them. Nevertheless, the current nightly build of Macro Paradise includes untyped macros. They allow to write Scala code which is typechecked after they are expanded, this means it is possible to write:
forM({i = 0; i < 10; i += 1}) {
println(i)
}
Notice, that the curly braces inside of the first parameter list are needed because, although the code is not typechecked when one writes it, it must represent a valid Scala AST.
The implementation of this macro looks like this:
def forM(header: _)(body: _) = macro __forM
def __forM(c: Context)(header: c.Tree)(body: c.Tree): c.Tree = {
import c.universe._
header match {
case Block(
List(
Assign(Ident(TermName(name)), Literal(Constant(start))),
Apply(Select(Ident(TermName(name2)), TermName(comparison)), List(Literal(Constant(end))))
),
Apply(Select(Ident(TermName(name3)), TermName(incrementation)), List(Literal(Constant(inc))))
) =>
// here one can generate the behavior of the loop
// but omit full implementation for clarity now ...
}
}
Instead of an already typechecked expression, the macro expects only a tree, that is typechecked after the expansion. The method call itself expects two parameter lists, whose parameter types can be delayed after the expansion phase if one uses an underscore.
Currently there is a little bit of documentation available but because it is extremely beta a lot of things will probably change in future.
With type macros it is possible to write something like this:
object O extends M {
// traverse the body of O to find what you want
}
type M(x: _) = macro __M
def __M(c: Context)(x: c.Tree): c.Tree = {
// omit full implementation for clarity ...
}
This is nice in order to delay the typechecking of the whole body because it allows to to cool things...
Macros that can change Scalas syntax are not planned at the moment and are probably not a good idea. I can't say if they will happen one day only future can tell us this.
Aside from the "why" (no really, why do you want to do that?), the answer is no, because as far as I know macros cannot (in their current state) generate methods or types, only expressions.
Related
I'm reading some code that looks like this:
trait Thingy {
def apply(d: => Unit): Unit
...
}
object Thingy {
def apply(d: => Unit) = {
...
}
}
Where does Odersky in his book edition 3 write about traits and traits having companion objects? His book mostly talks about how classes can have companion objects but not traits. Where is this written?
Also, how would you use thingy? I see that it's use like this:
Thingy {
implicit variable =>
val something = ...
}
What is going on in the above? Odersky doesn't seem to write about how curly braces can be used to call the apply method. Is this what is going on? Where is this written?
In Chapter 9, section 4 "Writing new control structures", Odersky says (all typos are mine):
One way in which you can make the client code look a bit more like a built-in control structure is to use curly braces instead of parentheses to surround the argument list. In any method invocation in Scala in which you're passing in exactly one argument, you can opt to use curly braces to surround the argument instead of parentheses.
In Chapter 3, "Step 7" he also talks about apply:
When you apply parentheses surrounding one or more values to a variable, Scala will transform the code into an invocation of a method named apply on that variable.
Applying these two rules together results in
Thingy { ... }
being rewritten into
Thingy( ... )
and then into
Thingy.apply(...)
It gets even funnier with implicit parameters. For example, for typeclasses it is common to define an apply that looks as follows:
trait MyTypeClass[A]
object MyTypeClass {
def apply[A](implicit inst: MyTypeClass[A]) = inst
}
so that the value-expression MyTypeClass[Int] desugars into MyTypeClass.apply[Int](someImplicitlyInjectedInstance), and thus is a value of type MyTypeClass[Int]. So, both the value MyTypeClass[Int] and the type MyTypeClass[Int] look exactly the same.
On the fact that it is not told explicitly that traits can also have companion objects: there are a lot of things that aren't told explicitly. For example, it was not told explicitly that you can write down type-lambdas in Scala, but it turned out that you actually can.
I wonder why Arbitrary is needed because automated property testing requires property definition, like
val prop = forAll(v: T => check that property holds for v)
and value v generator. The user guide says that you can create custom generators for custom types (a generator for trees is exemplified). Yet, it does not explain why do you need arbitraries on top of that.
Here is a piece of manual
implicit lazy val arbBool: Arbitrary[Boolean] = Arbitrary(oneOf(true, false))
To get support for your own type T you need to define an implicit def
or val of type Arbitrary[T]. Use the factory method Arbitrary(...) to
create the Arbitrary instance. This method takes one parameter of type
Gen[T] and returns an instance of Arbitrary[T].
It clearly says that we need Arbitrary on top of Gen. Justification for arbitrary is not satisfactory, though
The arbitrary generator is the generator used by ScalaCheck when it
generates values for property parameters.
IMO, to use the generators, you need to import them rather than wrapping them into arbitraries! Otherwise, one can argue that we need to wrap arbitraries also into something else to make them usable (and so on ad infinitum wrapping the wrappers endlessly).
You can also explain how does arbitrary[Int] convert argument type into generator. It is very curious and I feel that these are related questions.
forAll { v: T => ... } is implemented with the help of Scala implicits. That means that the generator for the type T is found implicitly instead of being explicitly specified by the caller.
Scala implicits are convenient, but they can also be troublesome if you're not sure what implicit values or conversions currently are in scope. By using a specific type (Arbitrary) for doing implicit lookups, ScalaCheck tries to constrain the negative impacts of using implicits (this use also makes it similar to Haskell typeclasses that are familiar for some users).
So, you are entirely correct that Arbitrary is not really needed. The same effect could have been achieved through implicit Gen[T] values, arguably with a bit more implicit scoping confusion.
As an end-user, you should think of Arbitrary[T] as the default generator for the type T. You can (through scoping) define and use multiple Arbitrary[T] instances, but I wouldn't recommend it. Instead, just skip Arbitrary and specify your generators explicitly:
val myGen1: Gen[T] = ...
val mygen2: Gen[T] = ...
val prop1 = forAll(myGen1) { t => ... }
val prop2 = forAll(myGen2) { t => ... }
arbitrary[Int] works just like forAll { n: Int => ... }, it just looks up the implicit Arbitrary[Int] instance and uses its generator. The implementation is simple:
def arbitrary[T](implicit a: Arbitrary[T]): Gen[T] = a.arbitrary
The implementation of Arbitrary might also be helpful here:
sealed abstract class Arbitrary[T] {
val arbitrary: Gen[T]
}
ScalaCheck has been ported from the Haskell QuickCheck library. In Haskell type-classes only allow one instance for a given type, forcing you into this sort of separation.
In Scala though, there isn't such a constraint and it would be possible to simplify the library. My guess is that, ScalaCheck being (initially written as) a 1-1 mapping of QuickCheck, makes it easier for Haskellers to jump into Scala :)
Here is the Haskell definition of Arbitrary
class Arbitrary a where
-- | A generator for values of the given type.
arbitrary :: Gen a
And Gen
newtype Gen a
As you can see they have a very different semantic, Arbitrary being a type class, and Gen a wrapper with a bunch of combinators to build them.
I agree that the argument of "limiting the scope through semantic" is a bit vague and does not seem to be taken seriously when it comes to organizing the code: the Arbitrary class sometimes simply delegates to Gen instances as in
/** Arbirtrary instance of Calendar */
implicit lazy val arbCalendar: Arbitrary[java.util.Calendar] =
Arbitrary(Gen.calendar)
and sometimes defines its own generator
/** Arbitrary BigInt */
implicit lazy val arbBigInt: Arbitrary[BigInt] = {
val long: Gen[Long] =
Gen.choose(Long.MinValue, Long.MaxValue).map(x => if (x == 0) 1L else x)
val gen1: Gen[BigInt] = for { x <- long } yield BigInt(x)
/* ... */
Arbitrary(frequency((5, gen0), (5, gen1), (4, gen2), (3, gen3), (2, gen4)))
}
So in effect this leads to code duplication (each default Gen being mirrored by an Arbitrary) and some confusion (why isn't Arbitrary[BigInt] not wrapping a default Gen[BigInt]?).
My reading of that is that you might need to have multiple instances of Gen, so Arbitrary is used to "flag" the one that you want ScalaCheck to use?
I have a following function:
def getIntValue(x: Int)(implicit y: Int ) : Int = {x + y}
I see above declaration everywhere. I understand what above function is doing. It is a currying function which takes two arguments. If you omit the second argument, it will invoke implicit definition which returns int instead. So I think it is something very similar to defining a default value for the argument.
implicit val temp = 3
scala> getIntValue(3)
res8: Int = 6
I was wondering what are the benefits of above declaration?
Here's my "pragmatic" answer: you typically use currying as more of a "convention" than anything else meaningful. It comes in really handy when your last parameter happens to be a "call by name" parameter (for example: : => Boolean):
def transaction(conn: Connection)(codeToExecuteInTransaction : => Boolean) = {
conn.startTransaction // start transaction
val booleanResult = codeToExecuteInTransaction //invoke the code block they passed in
//deal with errors and rollback if necessary, or commit
//return connection to connection pool
}
What this is saying is "I have a function called transaction, its first parameter is a Connection and its second parameter will be a code-block".
This allows us to use this method like so (using the "I can use curly brace instead of parenthesis rule"):
transaction(myConn) {
//code to execute in a transaction
//the code block's last executable statement must be a Boolean as per the second
//parameter of the transaction method
}
If you didn't curry that transaction method, it would look pretty unnatural doing this:
transaction(myConn, {
//code block
})
How about implicit? Yes it can seem like a very ambiguous construct, but you get used to it after a while, and the nice thing about implicit functions is they have scoping rules. So this means for production, you might define an implicit function for getting that database connection from the PROD database, but in your integration test you'll define an implicit function that will superscede the PROD version, and it will be used to get a connection from a DEV database instead for use in your test.
As an example, how about we add an implicit parameter to the transaction method?
def transaction(implicit conn: Connection)(codeToExecuteInTransaction : => Boolean) = {
}
Now, assuming I have an implicit function somewhere in my code base that returns a Connection, like so:
def implicit getConnectionFromPool() : Connection = { ...}
I can execute the transaction method like so:
transaction {
//code to execute in transaction
}
and Scala will translate that to:
transaction(getConnectionFromPool) {
//code to execute in transaction
}
In summary, Implicits are a pretty nice way to not have to make the developer provide a value for a required parameter when that parameter is 99% of the time going to be the same everywhere you use the function. In that 1% of the time you need a different Connection, you can provide your own connection by passing in a value instead of letting Scala figure out which implicit function provides the value.
In your specific example there are no practical benefits. In fact using implicits for this task will only obfuscate your code.
The standard use case of implicits is the Type Class Pattern. I'd say that it is the only use case that is practically useful. In all other cases it's better to have things explicit.
Here is an example of a typeclass:
// A typeclass
trait Show[a] {
def show(a: a): String
}
// Some data type
case class Artist(name: String)
// An instance of the `Show` typeclass for that data type
implicit val artistShowInstance =
new Show[Artist] {
def show(a: Artist) = a.name
}
// A function that works for any type `a`, which has an instance of a class `Show`
def showAListOfShowables[a](list: List[a])(implicit showInstance: Show[a]): String =
list.view.map(showInstance.show).mkString(", ")
// The following code outputs `Beatles, Michael Jackson, Rolling Stones`
val list = List(Artist("Beatles"), Artist("Michael Jackson"), Artist("Rolling Stones"))
println(showAListOfShowables(list))
This pattern originates from a functional programming language named Haskell and turned out to be more practical than the standard OO practices for writing a modular and decoupled software. The main benefit of it is it allows you to extend the already existing types with new functionality without changing them.
There's plenty of details unmentioned, like syntactic sugar, def instances and etc. It is a huge subject and fortunately it has a great coverage throughout the web. Just google for "scala type class".
There are many benefits, outside of your example.
I'll give just one; at the same time, this is also a trick that you can use on certain occasions.
Imagine you create a trait that is a generic container for other values, like a list, a set, a tree or something like that.
trait MyContainer[A] {
def containedValue:A
}
Now, at some point, you find it useful to iterate over all elements of the contained value.
Of course, this only makes sense if the contained value is of an iterable type.
But because you want your class to be useful for all types, you don't want to restrict A to be of a Seq type, or Traversable, or anything like that.
Basically, you want a method that says: "I can only be called if A is of a Seq type."
And if someone calls it on, say, MyContainer[Int], that should result in a compile error.
That's possible.
What you need is some evidence that A is of a sequence type.
And you can do that with Scala and implicit arguments:
trait MyContainer[A] {
def containedValue:A
def aggregate[B](f:B=>B)(implicit ev:A=>Seq[B]):B =
ev(containedValue) reduce f
}
So, if you call this method on a MyContainer[Seq[Int]], the compiler will look for an implicit Seq[Int]=>Seq[B].
That's really simple to resolve for the compiler.
Because there is a global implicit function that's called identity, and it is always in scope.
Its type signature is something like: A=>A
It simply returns whatever argument is passed to it.
I don't know how this pattern is called. (Can anyone help out?)
But I think it's a neat trick that comes in handy sometimes.
You can see a good example of that in the Scala library if you look at the method signature of Seq.sum.
In the case of sum, another implicit parameter type is used; in that case, the implicit parameter is evidence that the contained type is numeric, and therefore, a sum can be built out of all contained values.
That's not the only use of implicits, and certainly not the most prominent, but I'd say it's an honorable mention. :-)
I would like to implement an external DSL such as SQL in Scala using Macros. I have already seen papers on how to implement internal DSLs with Scala. Also, I've recently written an article about how this can be done in Java, myself.
Now, internal DSLs always feel a bit clumsy as they have to be implemented and used in the host language (e.g. Scala) and adhere to the host language's syntax constraints. That's why I'm hoping that Scala Macros will allow to internalise an external DSL without any such constraints. However, I don't fully understand Scala Macros and how far I can go with them. I've seen that SLICK and also a much less-known library called sqltyped have started using Macros, but SLICK uses a "Scalaesque" syntax for querying, which isn't really SQL, whereas sqltyped uses Macros to parse SQL strings (which can be done without Macros, too). Also, the various examples given on the Scala website are too trivial for what I'm trying to do
My question is:
Given an example external DSL defined as some BNF grammar like this:
MyGrammar ::= (
'SOME-KEYWORD' 'OPTION'?
(
( 'CHOICE-1' 'ARG-1'+ )
| ( 'CHOICE-2' 'ARG-2' )
)
)
Can I implement the above grammar using Scala Macros to allow for client programs like this? Or are Scala Macros not powerful enough to implement such a DSL?
// This function would take a Scala compile-checked argument and produce an AST
// of some sort, that I can further process
def evaluate(args: MyGrammar): MyGrammarEvaluated = ...
// These expressions produce a valid result, as the argument is valid according
// to my grammar
val result1 = evaluate(SOME-KEYWORD CHOICE-1 ARG-1 ARG-1)
val result2 = evaluate(SOME-KEYWORD CHOICE-2 ARG-2)
val result3 = evaluate(SOME-KEYWORD OPTION CHOICE-1 ARG-1 ARG-1)
val result4 = evaluate(SOME-KEYWORD OPTION CHOICE-2 ARG-2)
// These expressions produce a compilation error, as the argument is invalid
// according to my grammar
val result5 = evaluate(SOME-KEYWORD CHOICE-1)
val result6 = evaluate(SOME-KEYWORD CHOICE-2 ARG-2 ARG-2)
Note, I'm not interested in solutions that parse strings, the way sqltyped does
It's been some time since this question was answered by paradigmatic, but I've just stumbled upon it and thought it's worth extending.
An internalized DSL must indeed be valid Scala code with all the names defined before macro expansion, however one can overcome this restriction with a carefully designed syntax and Dynamics.
Let's say we wanted to create a simple, silly DSL allowing us to introduce people in a classy way. It might look like this:
people {
introduce John please
introduce Frank and Lilly please
}
We would like to translate (as part of compilation) the above code to an object (of a class derived for example from class People) containing definitions of fields of type Person for every introduced person - something like this:
new People {
val john: Person = new Person("John")
val frank: Person = new Person("Frank")
val lilly: Person = new Person("Lilly")
}
To make it possible we need to define some artificial objects and classes having two purposes: defining grammar (somewhat...) and tricking the compiler into accepting undefined names (like John or Lilly).
import scala.language.dynamics
trait AllowedAfterName
object and extends Dynamic with AllowedAfterName {
def applyDynamic(personName: String)(arg: AllowedAfterName): AllowedAfterName = this
}
object please extends AllowedAfterName
object introduce extends Dynamic {
def applyDynamic(personName: String)(arg: AllowedAfterName): and.type = and
}
These dummy definitions make our DSL code legal - the compiler translates it to the below code before proceeding to macro expansion:
people {
introduce.applyDynamic("John")(please)
introduce.applyDynamic("Frank")(and).applyDynamic("Lilly")(please)
}
Do we need this ugly and seemingly redundant please? One could probably come up with a nicer syntax, for example using Scala's postfix operator notation (language.postfixOps), but that gets tricky due to semicolon inference (you can try it yourself in the REPL console or IntelliJ's Scala Worksheet). It's easiest to just interlace keywords with undefined names.
As we've made the syntax legal, we can process the block with a macro:
def people[A](block: A): People = macro Macros.impl[A]
class Macros(val c: whitebox.Context) {
import c.universe._
def impl[A](block: c.Tree) = {
val introductions = block.children
def getNames(t: c.Tree): List[String] = t match {
case q"applyDynamic($name)(and).$rest" =>
name :: getNames(q"$rest")
case q"applyDynamic($name)(please)" =>
List(name)
}
val names = introductions flatMap getNames
val defs = names map { n =>
val varName = TermName(n.toLowerCase())
q"val $varName: Person = new Person($n)"
}
c.Expr[People](q"new People { ..$defs }")
}
}
The macro finds all the introduced names by pattern matching against the expanded dynamic calls and generates the desired output code. Notice that the macro must be whitebox in order to be allowed to return an expression of a type derived from the one declared in the signature.
I don't think so. The expression you pass to a macro must be a valid Scala expression and identifiers should be defined.
I've been working with Scala Macros and have the following code in the macro:
val fieldMemberType = fieldMember.typeSignatureIn(objectType) match {
case NullaryMethodType(tpe) => tpe
case _ => doesntCompile(s"$propertyName isn't a field, it must be another thing")
}
reify{
new TypeBuilder() {
type fieldType = fieldMemberType.type
}
}
As you can see, I've managed to get a c.universe.Type fieldMemberType. This represents the type of certain field in the object. Once I get that, I want to create a new TypeBuilder object in the reify. TypeBuilder is an abstract class with an abstract parameter. This abstract parameter is fieldType. I want this fieldType to be the type that I've found before.
Running the code shown here returns me a fieldMemberType not found. Is there any way that I can get the fieldMemberType to work inside the reify clause?
The problem is that the code you pass to reify is essentially going to be placed verbatim at the point where the macro is being expanded, and fieldMemberType isn't going to mean anything there.
In some cases you can use splice to sneak an expression that you have at macro-expansion time into the code you're reifying. For example, if we were trying to create an instance of this trait:
trait Foo { def i: Int }
And had this variable at macro-expansion time:
val myInt = 10
We could write the following:
reify { new Foo { def i = c.literal(myInt).splice } }
That's not going to work here, which means you're going to have to forget about nice little reify and write out the AST by hand. You'll find this happens a lot, unfortunately. My standard approach is to start a new REPL and type something like this:
import scala.reflect.runtime.universe._
trait TypeBuilder { type fieldType }
showRaw(reify(new TypeBuilder { type fieldType = String }))
This will spit out several lines of AST, which you can then cut and paste into your macro definition as a starting point. Then you fiddle with it, replacing things like this:
Ident(TypeBuilder)
With this:
Ident(newTypeName("TypeBuilder"))
And FINAL with Flag.FINAL, and so on. I wish the toString methods for the AST types corresponded more exactly to the code it takes to build them, but you'll pretty quickly get a sense of what you need to change. You'll end up with something like this:
c.Expr(
Block(
ClassDef(
Modifiers(Flag.FINAL),
anon,
Nil,
Template(
Ident(newTypeName("TypeBuilder")) :: Nil,
emptyValDef,
List(
constructor(c),
TypeDef(
Modifiers(),
newTypeName("fieldType"),
Nil,
TypeTree(fieldMemberType)
)
)
)
),
Apply(Select(New(Ident(anon)), nme.CONSTRUCTOR), Nil)
)
)
Where anon is a type name you've created in advance for your anonymous class, and constructor is a convenience method I use to make this kind of thing a little less hideous (you can find its definition at the end of this complete working example).
Now if we wrap this expression up in something like this, we can write the following:
scala> TypeMemberExample.builderWithType[String]
res0: TypeBuilder{type fieldType = String} = $1$$1#fb3f1f3
So it works. We've taken a c.universe.Type (which I get here from the WeakTypeTag of the type parameter on builderWithType, but it will work in exactly the same way with any old Type) and used it to define the type member of our TypeBuilder trait.
There is a simpler approach than tree writing for your use case. Indeed I use it all the time to keep trees at bay, as it can be really difficult to program with trees. I prefer to compute types and use reify to generate the trees. This makes much more robust and "hygienic" macros and less compile time errors. IMO using trees must be a last resort, only for a few cases, such as tree transforms or generic programming for a family of types such as tuples.
The tip here is to define a function taking as type parameters, the types you want to use in the reify body, with a context bound on a WeakTypeTag. Then you call this function by passing explicitly the WeakTypeTags you can build from universe Types thanks to the context WeakTypeTag method.
So in your case, that would give the following.
val fieldMemberType: Type = fieldMember.typeSignatureIn(objectType) match {
case NullaryMethodType(tpe) => tpe
case _ => doesntCompile(s"$propertyName isn't a field, it must be another thing")
}
def genRes[T: WeakTypeTag] = reify{
new TypeBuilder() {
type fieldType = T
}
}
genRes(c.WeakTypeTag(fieldMemberType))