I am learning Scala with Scala-IDE in Eclipse. While following the Chapter 9: Control Abstraction, part 1: Reducing Code Duplication in the Programming in Scala book, I have written the code from the book (fully represented below), it worked fine! When I started removing the unnecessary blank lines, a strange thing happened. Here is the entire code before the removal of the blank lines in the filterFiles() method:
object Code_c09s01_ControlAbstraction extends App{
object FilesFilter {
private def filterFiles(path: String, pattern: String, matcher: (String, String) => Boolean) = {
val files = (new java.io.File(path)) listFiles
for(file <- files if matcher(file.getName, pattern)) yield file
}
def filterExtension(path: String, pattern: String) = filterFiles(path, pattern, _.endsWith(_))
def filterName(path: String, pattern: String) = filterFiles(path, pattern, _.contains(_))
def filterRegex(path: String, pattern: String) = filterFiles(path, pattern, _.matches(_))
}
def printArray[A](message: String, arr: Array[A]) {
println (message)
println (arr mkString("\n"))
}
def test() {
val path = "C:\\";
printArray("--- filtering by ext: ---", FilesFilter.filterExtension(path, ".txt"))
printArray("--- filtering by containment: ---", FilesFilter.filterName(path, "1"))
printArray("--- filtering by regex: ---", FilesFilter.filterRegex(path, "."))
}
test
}
which works just fine! However, after removing the blank lines from the filterFiles() method, the method now looks like this:
private def filterFiles(path: String, pattern: String, matcher: (String, String) => Boolean) = {
val files = (new java.io.File(path)) listFiles
for(file <- files if matcher(file.getName, pattern)) yield file
}
And the IDE gives me errors on both lines of the body. The error of the first line says:
ambiguous reference to overloaded definition, both method listFiles in class File of type (x$1: java.io.FileFilter)Array[java.io.File] and method listFiles in class File of type
(x$1: java.io.FilenameFilter)Array[java.io.File] match argument types (Null)
the error on the second line says:
illegal start of simple expression
and all the three calls to the printArray() in the test() method now also tell this:
type mismatch; found : Unit required: Array[?]
What does it all mean? Scala is not supposed to behave like Python when code alignment can ruin the code flow... so how come that removing the blank line between the first and the second line of the body of the filterFiles() method puts up such a serious error? Is it a bug somewhere or does it follow directly from the rules of Scala? Note: if I add ; between the line, it sorts everything out. Is it just semicolon inference bug?
When object's method can none or single argument you can call it like
val files = object method arg
or, in your snippet
val files = object method
arg
In your code compiler wants to call listfiles with an for expression as an agument which returns Array[File]. And complains that listFiles hasn't implementation with this type of argument. Empty line prevents treating for expression as a parameter for listFiles function
Looks like semicolon inference problem as you suggest, I guess the compiler expects method arguments if you leave parenthesis out. Instead of semicolon, you can add parenthesis after listFiles:
val files = (new java.io.File(path)) listFiles ()
Related
In a Scala 3 macro that takes a type parameter T, you can use TypeRepr.of[T] and the new Scala 3 reflection API to explore the companionClass of T, and find the Symbol for an arbitrary method on that companion class (eg companionClass.declarations.find(_.name == "list") to find a list() method).
Given the Symbol for a companion object method, how would you then invoke that method within a quoted code block?
I'm guessing I would need to convert that Symbol to a Expr[T], but I don't know how to do that!
In a Scala 2 macro, the invocation of a listMethod of type c.universe.Symbol in a q"..." quasiquote seems pretty simple - just say $listMethod, and then you can start mapping on the resulting list, eg:
q"""
$listMethod.map(_.toString)
"""
Trying to do a similar thing in a Scala 3 macro gets an error like this:
[error] 27 | ${listMethod}.map(_.toString)
[error] | ^^^^^^^^^^
[error] | Found: (listMethod : x$1.reflect.Symbol)
[error] | Required: quoted.Expr[Any]
What is the correct code to get this working in Scala 3?
You can see more code context in the AvroSerialisableMacro classes (Scala 2 compiles, Scala 3 currently nowhere near!) here: https://github.com/guardian/marley/pull/77/files
First, let's talk how to call a method using symbol name in general.
You might need Select. You can call obtain it in a a few different ways, e.g.:
New(TypeTree.of[YourType]).select(primaryConstructor) // when you want to create something
expression.asTerm.select(method) // when you want to call it on something
Once you selected method you can provide arguments:
select.appliedToArgs(args) // if there is only 1 argument list
select.appliedToArgss(args) // if there is more than one argument list
// (type parameter list is listed in paramSymss
// but shouldn't be used here, so filter it out!)
select.appliedToNone // if this is a method like "def method(): T"
// (single, but empty, parameter list)
select.appliedToArgss(Nil) // is this is a method like "def method: T"
// (with not even empty parameter list)
There are also other methods like appliedToType, appliedToTypeTrees, but if you have a method name as a Symbol and want to use it to call something this should be a good starting point.
And remember that source code of Quotes is your friend, so even when your IDE doesn't give you any suggestions, it can point you towards some solution.
In theory these methods are defined on Term rather than Select (<: Term) but your use case will be most likely picking an expression and calling a method on it with some parameters. So a full example could be e.g.
val expression: Expr[Input]
val method: Symbol
val args: List[Term]
// (input: Input).method(args) : Output
expression // Expr[Input]
.asTerm // Term
.select(method) // Select
.appliedToArgs(args) // Term
.asExpr // Expr[?]
.asExprOf[Output] // Expr[Output]
Obviously, proving that the expression can call method and making sure that types of Terms in args match allowed types of values that you pass to the method, is on you. It is a bit more hassle than it was in Scala 2 since quotes allow you to work with Type[T] and Expr[T] only, so anything that doesn't fall under that category has to be implemented with macros/Tasty ADT until you get to the point that you can return Expr inside ${}.
That said, the example you linked shows that these calls are rather hardcoded, so you don't have to look up Symbols and call them. Your code will most likely do away with:
// T <: ThriftEnum
// Creating companion's Expr can be done with asExprOf called on
// Ref from Dmytro Mitin's answer
def findCompanionOfThisOrParent(): Expr[ThriftEnumObject[T]] = ...
// _Expr_ with the List value you constructed instead of Symbol!
val listOfValues: Expr[List[T]] = '{
${ findCompanionOfThisOrParent() }.list
}
// once you have an Expr you don't have to do any magic
// to call a method on it, Quotes works nice
'{
...
val valueMap = Map(${ listOfValues }.map(x => x ->
org.apache.avro.generic.GenericData.get.createEnum(
com.gu.marley.enumsymbols.SnakesOnACamel.toSnake(x.name), schemaInstance)
): _*)
...
}
Difference between Scala 2 quasiquotes and Scala 3 quotations is that the former must compile during compile time of the main code using macros (i.e. during macro expansion, macro runtime) while the latter must compile earlier, at macro compile time. So Scala 3 quotations '{...}/${...} are more like Scala 2 reify{...}/.splice than Scala 2 quasiquotes q"..."/${...}.
`tq` equivalent in Scala 3 macros
You have to re-create AST. Let's see what shape AST should have:
object B:
def fff(): Unit = ()
import scala.quoted.*
inline def foo(): Unit = ${fooImpl}
def fooImpl(using Quotes): Expr[Unit] =
import quotes.reflect.*
println('{B.fff()}.asTerm.show(using Printer.TreeStructure))
'{()}
foo() // ... Apply(Select(Ident("B"), "fff"), Nil)
So in order to re-create AST try to use Apply(...) and Select.unique(..., "list"):
import scala.quoted.*
inline def foo[T](): Unit = ${fooImpl[T]}
def fooImpl[T: Type](using Quotes): Expr[Unit] =
import quotes.reflect.*
val sym = TypeRepr.of[T].typeSymbol
'{
println("aaa")
${
Apply(
Select.unique(
Ref(sym.companionModule),
"list"
),
Nil
).asExprOf[Unit]
}
}
Testing (in a different file):
class A
object A {
def list(): Unit = println("list")
}
foo[A]()
//scalac: {
// scala.Predef.println("aaa")
// A.list()
//}
// prints at runtime:
// aaa
// list
Using method symbol rather than its name and using convenience methods rather than AST nodes directly, you can rewrite fooImpl as
def fooImpl[T: Type](using Quotes): Expr[Unit] =
import quotes.reflect.*
val sym = TypeRepr.of[T].typeSymbol
val listMethod = sym.companionClass.declarations.find(_.name == "list").get
'{
println("aaa")
${
Ref(sym.companionModule)
.select(listMethod)
.appliedToArgs(Nil)
.asExprOf[Unit]
}
}
This is just an example how to create an AST. You should use your actual return type of def list() instead of Unit in .asExprOf[Unit].
How to get the list of default fields values for typed case class?
scala 3 macro how to implement generic trait
Following code does not work if I don't uncomment the commented line.It says
type mismatch; found : Unit required: Option[String]
import java.io._
import org.apache.pdfbox.pdmodel.PDDocument
import org.apache.pdfbox.text.PDFTextStripper
object PdfToText {
def main(args: Array[String]) {
val filename = "D:\\Scala\\DATA\\HelloWorld.pdf"
getTextFromPdf(filename)
}
def getTextFromPdf(filename: String): Option[String] = {
val pdf = PDDocument.load(new File(filename))
println(new PDFTextStripper().getText(pdf))
// Some(new PDFTextStripper().getText(pdf))
}
}
Code executes fine if I keep the line -
Some(new PDFTextStripper().getText(pdf))
Output:
Welcome
to
The World of Scala
Could anyone please explain the behaviour of the line
" Some(new PDFTextStripper().getText(pdf)"
The result of a method is the result of the final line of code. println returns Unit. If that's the last line then that's what the method returns, which conflicts with the stated Option[String] return type.
The code new PDFTextStripper().getText(pdf) returns a String and wrapping it in Some() makes it an Option[String] which does match the stated method return type.
explanation
Here is a method that compiles.
def six(): Int = { // takes no arguments and returns an Int
println("top") // string sent to stdout
6 // this is the Int it returns
}
Here is a method that does not compile.
def six(): Int = { // takes no arguments and returns an Int
6 // this is the Int it should return
println("end") // Error: type mismatch
}
This method is supposed to return an Int (that's what the : Int means) but the last line is a println() statement and println returns a Unit, not an Int, so that causes the error. This method is trying to return Unit when it is supposed to return Int.
This is a very basic concept in the Scala language.
Option is a Scala type that represents the presence or absence of a value. Instead of runtime constructs like null and exceptions which have significant downsides, Option (and equivalent constructs in other languages) allows for compile-time checking that you are handling both possibilities.
A common use of Option is a database lookup by id. It is quite possible there is no entity with that id. The return type of the function in Scala would be Option[Employee]. If you find one, it would return Some(employee); otherwise simply None. Both subclass Option. Note that you can think of Option like a container of zero or one element.
In your case, you have defined your function to return Option[String], which is why returning Some(String) containing the text of the file makes the compiler happy. I hope that answers your question.
Please note though that your function isn't really designed in a way that uses Option effectively since there is no issue of presence/absence--just always present. In other words, returning an Option isn't useful here. However, it would be perfectly appropriate to use Option to represent the cases where the file is or isn't available on the file system to be read.
For example:
object PdfToText {
def main(args: Array[String]) {
val filename = "D:\\Scala\\DATA\\HelloWorld.pdf"
val text = getTextFromPdf(filename)
val result = text.map(t => s"File contents: ${t}").getOrElse("Empty file")
println(result)
}
def getTextFromPdf(filename: String): Option[String] = {
val file = new File(filename)
if (file.exists() && !file.isDirectory) {
val pdf = PDDocument.load(new File(filename))
Some(new PDFTextStripper().getText(pdf))
} else {
None
}
}
}
Here presence is defined by the existence of a readable file, which allows me to return its contents in a Some, and absence is defined by the nonexistence of the file or the file being a directory, which manifests as a None
I then account for both possibilities in main. If there's text to be read because the function gave me back a Some, I call map (which only fires on Somes) to transform the text into a different string. If I get None, we skip over to the getOrElse and produce a default string.
Either way, we print out whatever we got, which is guaranteed to be a String no matter what happened with the original File.
As a shameless plug, you can learn more about Option in our tutorial Nine Reasons to Try Scala. Just fast forward to 8:36.
Trying to wrap my head around the varying uses of the _. Right now I'm struggling with this example:
object Chapter9 extends App {
FileMatcher.filesEnding(".scala").foreach(println)
}
object FileMatcher {
private def filesHere = (new java.io.File(".")).listFiles
private def filesMatching(matcher: String => Boolean) = {
for (file <- filesHere; if matcher(file.getName))
yield file
}
def filesEnding(query: String) =
filesMatching(_.endsWith(query))
def filesContaining(query: String) =
filesMatching(_.contains(query))
def filesRegex(query: String) =
filesMatching(_.matches(query))
}
So clearly we want to abstract away the common work of looping/filtering/yielding for the varying types of matchers, makes sense to put it in a helper function.
I'm getting hung up on the _.endsWith part. My understanding is that this underscore (being the first and only one used in the method body) will be filled in by the first parameter, which in this case is query. I tried to test this theory by doing:
def filesEnding(query: String) = {
println(_: String)
}
But the program doesn't print anything. So what is _ here? How does Scala know what object to to search for an endsWith method on?
It looks like from output of the program that somehow file gets filled in for this underscore but have no idea how. Maybe the underscore remains a "wildcard" until it is used inside filesMatching's body and by that point the nearest enclosing scope is the for and the first "parameterisfile`?
Look at the signature for filesMatching(). Notice that it takes one argument of type String => Boolean. So its argument is a function that itself take a String argument and turns it into a Boolean.
Now remember that an anonymous function often looks something like this:
{ x => /* do something with x */ }
And in cases where x is used only once, then that can be abbreviated to a single _. So, working backwards, this
filesMatching(_.endsWith(query))
can be rewritten as this
filesMatching(x => x.endsWith(query))
So the filesMatching() code has its argument, a function that takes a string (which in the anonymous function I've called x). That function, matcher, is invoked with the string file.getName to get a Boolean. That boolean value is tested in an if clause:
if matcher(file.getName)
TL;DR: The underscore is shorthand for the file.getName string.
The canonical answer is What are all the uses of an underscore in Scala?
But -Xprint:parser shows
((x$1: String) => println((x$1: String)))
which is uninteresting except for the redundantly typed expression in the body of the function.
It doesn't seem to generate any extra code. The param is already a String.
I don't think your example compiles? Or I don't know what you're asking.
Explicit types can help debug when type of an anonymous function aren't inferred as you wish.
Edit: I gave this a try:
object Chapter9 extends App {
FileMatcher.filesEnding(".scala").foreach(println)
}
object FileMatcher {
private def filesHere = (new java.io.File(".")).listFiles
private def filesMatching(matcher: String => Boolean) = {
for (file <- filesHere; if matcher(file.getName))
yield file
}
def filesEnding(query: String) = {
println(_: String)
}
}
An expression with an underscore as an anonymous function needs its expected type to tell it what type the underscore is, unless explicitly annotated as you did. But that is not common usage.
Instead of (_: Int) * 2, (i: Int) => i * 2, but that's a style question.
I've written a simple code in Scala with implicit conversion of Function1 to some case class.
object MyApp extends App{
case class FunctionContainer(val function:AnyRef)
implicit def cast(function1: Int => String):FunctionContainer = new FunctionContainer(function1)
def someFunction(i:Int):String = "someString"
def abc(f : FunctionContainer):String = "abc"
println(abc(someFunction))
}
But it doesn't work. Compiler doesn't want to pass someFunction as an argument to abc. I can guess its reasons but don't know exactly why it doesn't work.
When you use a method name as you have, the compiler has to pick how to convert the method type to a value. If the expected type is a function, then it eta-expands; otherwise it supplies empty parens to invoke the method. That is described here in the spec.
But it wasn't always that way. Ten years ago, you would have got your function value just by using the method name.
The new online spec omits the "Change Log" appendix, so for the record, here is the moment when someone got frustrated with parens and introduced the current rules. (See Scala Reference 2.9, page 181.)
This has not eliminated all irksome anomalies.
Conversions
The rules for implicit conversions of methods to functions (§6.26) have been tightened. Previously, a parameterized method used as a value was always implicitly converted to a function. This could lead to unexpected results when method arguments were forgotten. Consider for instance the statement below:
show(x.toString)
where show is defined as follows:
def show(x: String) = Console.println(x)
Most likely, the programmer forgot to supply an empty argument list () to toString. The previous Scala version would treat this code as a partially applied method, and expand it to:
show(() => x.toString())
As a result, the address of a closure would be printed instead of the value of s. Scala version 2.0 will apply a conversion from partially applied method to function value only if the expected type of the expression is indeed a function type. For instance, the conversion would not be applied in the code above because the expected type of show’s parameter is String, not a function type. The new convention disallows some previously legal code. Example:
def sum(f: int => double)(a: int, b: int): double =
if (a > b) 0 else f(a) + sum(f)(a + 1, b)
val sumInts = sum(x => x) // error: missing arguments
The partial application of sum in the last line of the code above will not be converted to a function type. Instead, the compiler will produce an error message which states that arguments for method sum are missing. The problem can be fixed by providing an expected type for the partial application, for instance by annotating the definition of sumInts with its type:
val sumInts: (int, int) => double = sum(x => x) // OK
On the other hand, Scala version 2.0 now automatically applies methods with empty parameter lists to () argument lists when necessary. For instance, the show expression above will now be expanded to
show(x.toString())
Your someFunction appears as a method here.
You could try either
object MyApp extends App{
case class FunctionContainer(val function:AnyRef)
implicit def cast(function1: Int => String):FunctionContainer = new FunctionContainer(function1)
val someFunction = (i:Int) => "someString"
def abc(f : FunctionContainer):String = "abc"
println(abc(someFunction))
}
or
object MyApp extends App{
case class FunctionContainer(val function:AnyRef)
implicit def cast(function1: Int => String):FunctionContainer = new FunctionContainer(function1)
def someFunction(i:Int): String = "someString"
def abc(f : FunctionContainer):String = "abc"
println(abc(someFunction(_: Int)))
}
By the way: implicitly casting such common functions to something else can quickly lead to problems. Are you absolutely sure that you need this? Wouldn't it be easier to overload abc?
You should use eta-expansion
println(abc(someFunction _))
Excuse the long set-up. This question relates to, but is not answered by, Scala: ambiguous reference to overloaded definition - best disambiguation? .
I'm pretty new to Scala, and one thing that's throwing me off is that Scala both:
Has first-class functions
Calls functions when using object-dot notation without any parenthetical argument lists (as if the function were a property)
These two language features are confusing me. Look at the below code:
class MyClass {
def something(in: String): String = {
in + "_X"
}
def something: String => String = {
case _ => "Fixed"
}
}
val my = new MyClass()
println(List("foo", "bar").map(my.something))
I would expect this to print List("foo_X", "bar_X") by calling the something prototype that matches the map's required String => ? argument. Instead, the output is List("Fixed", "Fixed") - Scala 2.11 is invoking the no-argument something() and then passing its return value to the map.
If we comment out the second no-argument prototype of something, the output changes to be the expected result, demonstrating that the other prototype is valid in context.
Adding an empty argument list to the second prototype (making it def something()) also changes the behavior.
Changing the my.something to my.something(_) wakes Scala up to the ambiguity it was silently ignoring before:
error: ambiguous reference to overloaded definition,
both method something in class MyClass of type => String => String
and method something in class MyClass of type (in: String)String
match argument types (String)
println(List("foo", "bar").map(my.something(_)))
Even using the supposedly-for-this-purpose magic trailing underscore doesn't work:
val myFun: (String) => String = my.something _
This results in:
error: type mismatch;
found : () => String => String
required: String => String
val myFun: (String) => String = my.something _
My questions:
If I have MyClass exactly as written (no changes to the prototypes, especially not adding an empty parameter list to one of the prototypes), how do I tell Scala, unambiguously, that I want the first one-argument version of something to pass as an argument to another call?
Since there are clearly two satisfying arguments that could be passed to map, why did the Scala compiler not report the ambiguity as an error?
Is there a way to disable Scala's behavior of (sometimes, not always) treating foo.bar as equivalent to foo.bar()?
I have filed a bug on the Scala issue tracker and the consensus seems to be that this behaviour is a bug. The compiler should have thrown an error about the ambiguous reference to "my.something".