I'm trying to port the Scala track of Exercism to Scala 3. Currently stuck on the "testgen" subproject, as I have next to no experience of reflections and macros. This Scala 2 method seems to ensure that a String can be coerced to a literal constant before stringifying back the result?
It is used in the KinderGardenTestGenerator and WordCountTestGenerator, presumably to sanitize student input?
So I want to replace it in Scala 3 with something like
def escape(raw: String): String = {
Literal(StringConstant(Expr(e))).toString
}
It seems to get access to the reflect methods you need a (using Quotes) and to do that you need to use inline.
The closest I've gotten to a solution is this, splitting out the methods into their own object:
import scala.quoted.*
object Escaper {
inline def escape(inline raw: String): String = ${literalize('{raw})}
def literalize(e: Expr[String])(using Quotes): Expr[String] = {
import quotes.reflect.*
Expr(Literal(StringConstant(e.valueOrAbort)).toString)
}
}
It seem to compile, but fails once it reaches compiling KinderGardenTestGenerator, where I get the following error:
[error] -- Error: /home/larsw/projects/scala/testgen/src/main/scala/KindergartenGardenTestGenerator.scala:33:47
[error] 33 | s"""Garden.defaultGarden(${escape(diagram.toString)}).$property("$student")"""
[error] | ^^^^^^^^^^^^^^^^^^^^^^^^
[error] | Could not find class testgen.Escaper$ in classpath
[error] |----------------------------------------------------------------------------
[error] |Inline stack trace
[error] |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error] |This location contains code that was inlined from Escaper.scala:8
[error] 8 | inline def escape(inline raw: String): String = ${literalize('{raw})}
[error] | ^^^^^^^^^^^^^^^^^^^^^
[error] ----------------------------------------------------------------------------
[error] -- Error: /home/larsw/projects/scala/testgen/src/main/scala/WordCountTestGenerator.scala:25:37
[error] 25 | val sentence = Escaper.escape(args("sentence").toString)
[error] | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error] | Could not find class testgen.Escaper$ in classpath
[error] |----------------------------------------------------------------------------
[error] |Inline stack trace
[error] |- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
[error] |This location contains code that was inlined from Escaper.scala:8
[error] 8 | inline def escape(inline raw: String): String = ${literalize('{raw})}
[error] | ^^^^^^^^^^^^^^^^^^^^^
[error] ----------------------------------------------------------------------------
And it feels like overkill to inline, my use case isn't that advanced, I don't need to generate code. My questions are:
Is there a way to Literalize and sanitize the strings without macros or reflection?
Or is there some way to access the reflection methods without inline and (using Quotes)
Or if my proposed solution is the only way - Why does it not find it on the classpath, even though import seems to work?
(Also interested in links to good videos or courses teaching Scala 3 macros. I'm eager to learn more and is very excited about the possibilities here, especially since Exercism are adding "representers" that can give students and mentors feedback and improvement suggestions for solutions, which looks like a great fit for macros.)
This Scala 2 method seems to ensure that a String can be coerced to a literal constant before stringifying back the result?
No, this is just an exotic way to transform \n into \\n etc.
Scala: How can I get an escaped representation of a string?
Your Scala 3 macro now does the wrong job. Try Simão Martins's answer from there
import scala.quoted.*
inline def escape(inline raw: String): String = ${escapeImpl('{raw})}
def escapeImpl(raw: Expr[String])(using Quotes): Expr[String] =
import quotes.reflect.*
Literal(StringConstant(raw.show)).asExprOf[String]
I guess a Scala 3 macro is now an overkill. Just try to escape with a different implemenation from there, e.g. 0__'s
def escape (s: String): String = "\"" + escape0(s) + "\""
def escape0(s: String): String = s.flatMap(escapedChar)
def escapedChar(ch: Char): String = ch match {
case '\b' => "\\b"
case '\t' => "\\t"
case '\n' => "\\n"
case '\f' => "\\f"
case '\r' => "\\r"
case '"' => "\\\""
case '\'' => "\\\'"
case '\\' => "\\\\"
case _ => if (ch.isControl) "\\0" + Integer.toOctalString(ch.toInt)
else String.valueOf(ch)
}
Related
In a Scala 3 macro that takes a type parameter T, you can use TypeRepr.of[T] and the new Scala 3 reflection API to explore the companionClass of T, and find the Symbol for an arbitrary method on that companion class (eg companionClass.declarations.find(_.name == "list") to find a list() method).
Given the Symbol for a companion object method, how would you then invoke that method within a quoted code block?
I'm guessing I would need to convert that Symbol to a Expr[T], but I don't know how to do that!
In a Scala 2 macro, the invocation of a listMethod of type c.universe.Symbol in a q"..." quasiquote seems pretty simple - just say $listMethod, and then you can start mapping on the resulting list, eg:
q"""
$listMethod.map(_.toString)
"""
Trying to do a similar thing in a Scala 3 macro gets an error like this:
[error] 27 | ${listMethod}.map(_.toString)
[error] | ^^^^^^^^^^
[error] | Found: (listMethod : x$1.reflect.Symbol)
[error] | Required: quoted.Expr[Any]
What is the correct code to get this working in Scala 3?
You can see more code context in the AvroSerialisableMacro classes (Scala 2 compiles, Scala 3 currently nowhere near!) here: https://github.com/guardian/marley/pull/77/files
First, let's talk how to call a method using symbol name in general.
You might need Select. You can call obtain it in a a few different ways, e.g.:
New(TypeTree.of[YourType]).select(primaryConstructor) // when you want to create something
expression.asTerm.select(method) // when you want to call it on something
Once you selected method you can provide arguments:
select.appliedToArgs(args) // if there is only 1 argument list
select.appliedToArgss(args) // if there is more than one argument list
// (type parameter list is listed in paramSymss
// but shouldn't be used here, so filter it out!)
select.appliedToNone // if this is a method like "def method(): T"
// (single, but empty, parameter list)
select.appliedToArgss(Nil) // is this is a method like "def method: T"
// (with not even empty parameter list)
There are also other methods like appliedToType, appliedToTypeTrees, but if you have a method name as a Symbol and want to use it to call something this should be a good starting point.
And remember that source code of Quotes is your friend, so even when your IDE doesn't give you any suggestions, it can point you towards some solution.
In theory these methods are defined on Term rather than Select (<: Term) but your use case will be most likely picking an expression and calling a method on it with some parameters. So a full example could be e.g.
val expression: Expr[Input]
val method: Symbol
val args: List[Term]
// (input: Input).method(args) : Output
expression // Expr[Input]
.asTerm // Term
.select(method) // Select
.appliedToArgs(args) // Term
.asExpr // Expr[?]
.asExprOf[Output] // Expr[Output]
Obviously, proving that the expression can call method and making sure that types of Terms in args match allowed types of values that you pass to the method, is on you. It is a bit more hassle than it was in Scala 2 since quotes allow you to work with Type[T] and Expr[T] only, so anything that doesn't fall under that category has to be implemented with macros/Tasty ADT until you get to the point that you can return Expr inside ${}.
That said, the example you linked shows that these calls are rather hardcoded, so you don't have to look up Symbols and call them. Your code will most likely do away with:
// T <: ThriftEnum
// Creating companion's Expr can be done with asExprOf called on
// Ref from Dmytro Mitin's answer
def findCompanionOfThisOrParent(): Expr[ThriftEnumObject[T]] = ...
// _Expr_ with the List value you constructed instead of Symbol!
val listOfValues: Expr[List[T]] = '{
${ findCompanionOfThisOrParent() }.list
}
// once you have an Expr you don't have to do any magic
// to call a method on it, Quotes works nice
'{
...
val valueMap = Map(${ listOfValues }.map(x => x ->
org.apache.avro.generic.GenericData.get.createEnum(
com.gu.marley.enumsymbols.SnakesOnACamel.toSnake(x.name), schemaInstance)
): _*)
...
}
Difference between Scala 2 quasiquotes and Scala 3 quotations is that the former must compile during compile time of the main code using macros (i.e. during macro expansion, macro runtime) while the latter must compile earlier, at macro compile time. So Scala 3 quotations '{...}/${...} are more like Scala 2 reify{...}/.splice than Scala 2 quasiquotes q"..."/${...}.
`tq` equivalent in Scala 3 macros
You have to re-create AST. Let's see what shape AST should have:
object B:
def fff(): Unit = ()
import scala.quoted.*
inline def foo(): Unit = ${fooImpl}
def fooImpl(using Quotes): Expr[Unit] =
import quotes.reflect.*
println('{B.fff()}.asTerm.show(using Printer.TreeStructure))
'{()}
foo() // ... Apply(Select(Ident("B"), "fff"), Nil)
So in order to re-create AST try to use Apply(...) and Select.unique(..., "list"):
import scala.quoted.*
inline def foo[T](): Unit = ${fooImpl[T]}
def fooImpl[T: Type](using Quotes): Expr[Unit] =
import quotes.reflect.*
val sym = TypeRepr.of[T].typeSymbol
'{
println("aaa")
${
Apply(
Select.unique(
Ref(sym.companionModule),
"list"
),
Nil
).asExprOf[Unit]
}
}
Testing (in a different file):
class A
object A {
def list(): Unit = println("list")
}
foo[A]()
//scalac: {
// scala.Predef.println("aaa")
// A.list()
//}
// prints at runtime:
// aaa
// list
Using method symbol rather than its name and using convenience methods rather than AST nodes directly, you can rewrite fooImpl as
def fooImpl[T: Type](using Quotes): Expr[Unit] =
import quotes.reflect.*
val sym = TypeRepr.of[T].typeSymbol
val listMethod = sym.companionClass.declarations.find(_.name == "list").get
'{
println("aaa")
${
Ref(sym.companionModule)
.select(listMethod)
.appliedToArgs(Nil)
.asExprOf[Unit]
}
}
This is just an example how to create an AST. You should use your actual return type of def list() instead of Unit in .asExprOf[Unit].
How to get the list of default fields values for typed case class?
scala 3 macro how to implement generic trait
I was running the following code on Scala 2.12 fine:
private[this] var requestByTimeoutById: SortedMap[Long, Map[String, Request]] = SortedMap.empty
def drop(request: Request): Unit = {
requestById = requestById - request.command.msgUuid
// more logic
}
After upgrading to Scala 2.13.7 I get a compile error:
[error] /Users/Shared/trex/core/src/main/scala/com/github/trex_paxos/Driver.scala:90:31: type mismatch;
[error] found : scala.collection.Map[String,BaseDriver.this.Request]
[error] required: scala.collection.SortedMap[String,BaseDriver.this.Request]
[error] requestById = requestById - request.command.msgUuid
Asking IntellJ to open the definition of the function I get to:
#deprecated("Use - or removed on an immutable Map", "2.13.0")
def - (key: K): Map[K, V]
Which is clearly returning the wrong type. How do I remove an element in the scala.collection.SortedMap?
It looks like I can do this:
requestById = requestById.filterNot(key => key.equals(request.command.msgUuid))
which seems really clunky. Is there a more idiomatic way?
Say I have something like this (VERY OVER-SIMPLIFIED):
case class Foo(bar: String)
val mockFnThatTakesFoo = mock[Foo => Unit]
def fooHasBarSetTo(expectedBar: String, foo: Foo): Boolean = {
val actualBar = foo.bar
actualBar shouldEqual expectedBar
true
}
mockFnThatTakesFoo(argThat(fooHasBarSetTo("blah", _))) wasCalled once
This works. However, the assertion itself is a little bit convoluted and it could be made more readable.
I tried this:
val withFooHavingBarSetTo = (expectedBar: String) => argThat(fooHasBarSetTo(expectedBar, _))
//and then
mockFnThatTakesFoo(withFooHavingBarSetTo("blah")) wasCalled once
Much neater! but doesn't work :/
> [info] FooSpec:
[info] - should do stuff *** FAILED ***
[info] org.mockito.exceptions.misusing.InvalidUseOfMatchersException: Invalid use of argument matchers!
[info] 1 matchers expected, 2 recorded:
[info] -> at com.foo.FooSpec.argThat(FooSpec.scala:28)
[info] -> at com.foo.FooSpec.$anonfun$new$5(FooSpec.scala:204)
Any idea how can this be done?
I believe Mockito is implemented using macro and it tracks positions where argThat and other Mockito methods are placed. For example if you try to create a variable like
val blahBar = argThat(fooHasBarSetTo("blah", _))
You will get an exception "Misplaced or misused argument matcher detected here". So it is impossible to return argThat matcher from another method.
If you need to reuse mock matcher with different argument value I see only one possible solution like
def mockCalledOnceWithFooHavingBar(expectedBar: String) {
mockFnThatTakesFoo(argThat(fooHasBarSetTo(expectedBar, _))) wasCalled once
}
mockCalledOnceWithFooHavingBar("blah")
mockCalledOnceWithFooHavingBar("bar")
I use slick 2.0.2 and I just want to do a simple filter or use a where sub-statement, I just want to do the logical operations like "and", "or" and "not" inside the filter :
val subjectdata = TableQuery[SubjectTable]
...
subjectdata.where(i=>(i.id===id && i.userId===rs.user.get.identityId.userId)).list()
and get error:
[error] G:\testprojects\slickplay\app\controllers\ShopController.scala:89: Cannot perform option-mapped operation
[error] with type: (Long, String) => R
[error] for base type: (Long, Long) => Boolean
[error] subjectdata.where(i=>(i.id===id && i.userId===rs.user.get.identityId
.userId)).list()
[error]
^
In slick 1.0.1 I can do:
val results = Query(TableClass)
.filter(r => r.isNull || r.expires > new Timestamp(DateTime.now().getMillis()))
.list
I want to do something similar on TableQuery in Slick2. How to do it?
One thing to know is that Slick's operations are more strict about types than Scala's. Both operands have to have the same base type, optionally wrapped in an Options. So comparing a Double to Double or an Option[Double] is ok, but comparing it to an Int will give you such a compile time warning. The error message hints a bit you towards the problem
[error] G:\testprojects\slickplay\app\controllers\ShopController.scala:89: Cannot perform option-mapped operation
[error] with type: (Long, String) => R
[error] for base type: (Long, Long) => Boolean
[error] subjectdata.where(i=>(i.id===id && i.userId===rs.user.get.identityId
.userId)).list()
In (Long, String) => R you see that the arguments do not have matching types and that the return type cannot be determined. So I assume either id or rs.user.get.identityId is a String. Turn is into an Int using .toInt. Alternatively you can convert the db-side value using .asColumnOf[String].
Is it possible to invert matches with Scala parser combinators? I am trying to match lines with a parser that do not start with a set of keywords. I could do this with an annoying zero width negative lookahead regular expression (e.g. "(?!h1|h2).*"), but I'd rather do it with a Scala parser. The best I've been able to come up with is this:
def keyword = "h1." | "h2."
def alwaysfails = "(?=a)b".r
def linenotstartingwithkeyword = keyword ~! alwaysfails | ".*".r
The idea is here that I use ~! to forbid backtracking to the all-matching regexp, and then continue with a regex "(?=a)b".r that matches nothing. (By the way, is there a predefined parser that always fails?) That way the line would not be matched if a keyword is found but would be matched if keyword does not match.
I am wondering if there is a better way to do this. Is there?
You can use not here:
import scala.util.parsing.combinator._
object MyParser extends RegexParsers {
val keyword = "h1." | "h2."
val lineNotStartingWithKeyword = not(keyword) ~> ".*".r
def apply(s: String) = parseAll(lineNotStartingWithKeyword, s)
}
Now:
scala> MyParser("h1. test")
res0: MyParser.ParseResult[String] =
[1.1] failure: Expected failure
h1. test
^
scala> MyParser("h1 test")
res1: MyParser.ParseResult[String] = [1.8] parsed: h1 test
Note that there is also a failure method on Parsers, so you could just as well have written your version with keyword ~! failure("keyword!"). But not's a lot nicer, anyway.