I am playing around with calling external command from Scala. Here is a stripped out example of what I am working on:
import scala.sys.process._
object Testing {
def main(args: Array[String]) {
val command = "ls"
val result = command!
if (result != 0) { // <---- illegal start of simple expression
println("Error")
return
}
}
}
I am getting a compile error: illegal start of simple expression for the line with the if statement. I can fix it with a new line:
val result = command!
// Add a line
if (result != 0) {
My suspicion is that it has something to do with the ! postfix function, but it was my understanding that superfluous lines/whitespaces shouldn't make a difference to the compiler.
You need to explicitly enable postfix expressions:
1) Importing the flag locally: import scala.language.postfixOps
2) or adding the flag to the project itself: scalacOptions += "-language:postfixOps"
The above link in the comment from #Łukasz contains lots of info about this feature. Also, see http://docs.scala-lang.org/style/method-invocation.html in the "Suffix Notation" section for your exact use case.
EDIT: maybe it was not clear enough, but as #Łukasz pointed in comments, importing/enabling postfix expressions doesn't make your code compile. It just avoids the compiler warning. Your code won't compile because the semicolons are optional and the compiler is treating the ! operator as infix, and thus taking elements from the next line for the expressions. This is exactly what the documentation in the link above states with exactly this same example:
This style is unsafe, and should not be used. Since semicolons are
optional, the compiler will attempt to treat it as an infix method if
it can, potentially taking a term from the next line.
names toList
val answer = 42 // will not compile!
This may result in unexpected compile errors at best, and happily
compiled faulty code at worst. Although the syntax is used by some
DSLs, it should be considered deprecated, and avoided.
Related
I'm trying to perform the equivalent of the eval method from Python in nim.
I was under the impression that parseStmt from the macros package should help me with this, but I'm facing a compilation issue that I don't understand.
import macros
echo parseStmt("1 + 2")
I would have expected this to print 3 when executed, but instead the compilation complains that
Error: request to generate code for .compileTime proc: $
I found this thread, and the examples there work, and following this, I was able to make the following program that works as I would expect:
import macros
import strformat
macro eval(value: string): untyped =
result = parseStmt fmt"{value}"
echo eval("1+2")
But I don't undertand why it needs to be written in this way at all. If I inline the statement, let value = "1 + 2"; echo parseStmt fmt"{value}", I get the same compile error as above.
Also, why is parseStmt value different from parseStmt fmt"{value}", in the context of the eval macro above?
What am I missing here?
Thank you in advance for any clarifications!
Unlike Python which is an interpreted language, Nim is compiled. This means that all code is parsed and turned into machine code on compile-time and the program that you end up with doesn't really know anything about Nim at all (at least as long as you don't import the Nim compiler as a module, which is possible). So parseStmt and all macro/template expansion stuff in Nim is done completely during compilation. The error, although maybe a bit hard to read, is trying to tell you that what was passed to $ (which is the convert-to-string operator in Nim, and called by echo on all its arguments) is a compile-time thing that can't be used on runtime. In this case it's because parseStmt doesn't return "3", it returns something like NimNode(kind: nnkIntLit, intVal: 3), and the NimNode type is only available during compile-time. Nim however allows you to run code on compile-time to return other code, this is what a macro does. The eval macro you wrote there takes value which is any statement that resolves to a string during runtime, passed as a NimNode. This is also why result = parseStmt value won't work in your case, because value is not yet a string, but could be something like reading a string from standard input which would result in a string during runtime. However the use of strformat here is a bit confusing and overkill. If you change your code to:
import macros
macro eval(value: static[string]): untyped =
result = parseStmt value
echo eval("1+2")
It will work just fine. This is because we have now told Nim that value needs to be a static i.e. known during compile-time. In this case the string literal "1+2" is obviously known at compile-time, but this could also be a call to a compile-time procedure, or even the output of staticRead which reads a file during compilation.
As you can see Nim is very powerful, but the barrier between compile-time and run-time can sometimes be a bit confusing. Note also that your eval procedure doesn't actually evaluate anything at compile-time, it simply returns the Nim code 1 + 2 so your code ends up being echo 1 + 2. If you want to actually run the code at compile-time you might want to look into compile-time procedures.
Hope this helps shed some light on your issue.
Note: while this answer outlines why this happens, keep in mind that what you're trying to do probably won't result in what you want (which I assumed to be runtime evaluation of expressions).
You're trying to pass a NimNode to parseStmt which expects a string. The fmt macro automatically stringifies anything in the {}, you can omit the fmt by doing $value to turn the node into a string.
As I already noted, this will not work as it does in Python: Nim does not have runtime evaluation. The expression in the string is going to be evaluated at compile time, so a simple example like this will not do what you want:
import std/rdstdin
let x = readLineFromStdin(">")
echo eval(x)
Why?
First of all, because you're stringifying the AST you pass to the eval, it's not the string behind the x variable that's going to get passed to the macro - it's going to be the symbol that denotes the x variable. If you stringify a symbol, you get the underlying identifier, which means that parseStmt will receive "x" as its parameter. This will effect in the string stored in x being printed out, which is wrong.
What you want instead is the following:
import std/rdstdin
import std/macros
macro eval(value: static string): untyped =
result = parseStmt(value)
echo eval("1 + 2")
This prevents runtime-known values from being passed to the macro. You can only pass consts and literals to it now, which is the correct behavior.
I am working on a project which reads source code of various languages. The project itself is written in scala but what I'm doing should be familiar if you know antlr. I have used the scala.g4 grammar on github to generate the Parser, Lexer etc for antlr4. I have written a subclass of ScalaBaseListener which simply prints on overridden the Enter methods
eg
override def enterClassDef(ctx: ScalaParser.ClassDefContext): Unit = {
println(ctx.getText)
}
In my application's main I am attempting to walk the whole tree from a file source like so:
import ScalaLexer._
import org.antlr.v4.runtime._
import org.antlr.v4.runtime.tree._
import scala.io.Source
object Main extends App {
val fileContents = Source.fromFile(args(0)).getLines.mkString
val charStream = new ANTLRInputStream(fileContents)
val lexer = new ScalaLexer(charStream)
val tokens = new CommonTokenStream(lexer)
val parser = new ScalaParser(tokens)
val tree = parser.compilationUnit
ParseTreeWalker.DEFAULT
.walk(new ScalaMySubclassListener(), tree)
}
I have found that if the source file is say, just a couple of classes:
class Foo {
def bar = {
1
}
def baz = 1
}
class Foo1 {
def bar = {
1
}
def baz = 1
}
I can see from the output of my program that every leaf in the tree is walked.
However, if I were to add an import statement at the top of the file (as there will often be in a scala source file)
import Thing._
class Foo {
def bar = {
1
}
def baz = 1
}
class Foo1 {
def bar = {
1
}
def baz = 1
}
only the leaves in the import statement get walked. The rest of the file gets ignored.
When I parse the source file using the antlr4 GUI, the whole tree is visible.
The first thing to do when the parse tree seems cut off is to check whether there are any syntax errors as that would be the most common cause. Since you didn't mess with error handling at all in your code, that means that any syntax errors should be printed to stderr. Since none are, there apparently weren't any syntax errors.
But let's not give up on the idea of there being a syntax error just yet. One common pitfall when it comes to syntax errors in ANTLR is if your start rule does not end with an EOF. If that's the case, ANTLR will simply try to find a prefix of the input that's syntactically valid and ignore the rest. That is, it will stop at the first syntax error without actually producing an error message (as long as there's a valid program leading up to that error - since many grammars accept empty programs, that is very often the case). And sure enough: if we look at Scala.g4 there's no EOF anywhere in the grammar (at the time of this writing anyway). So let's add EOF at the end of the compilationUnit rule. Now if we recompile everything and run your code again, we finally get a syntax error:
line 1:20 mismatched input 'Foo' expecting {<EOF>, '.', ',', 'implicit', 'lazy', 'case', '#', 'override', 'abstract', 'final', 'sealed', 'private', 'protected', 'import', 'class', 'object', 'trait', 'package'}
Now there's two things that might strike you as curious:
Why does ANTLR detect a syntax error when run from your code, but not from the TestRig GUI (even after adding the EOF, the GUI will still show a correct tree).
Why does the error message claim that Foo appears on column 20 of line 1 when it's actually on line 3?
The answer to both of these questions is the same: The input that you're feeding ANTLR is not what's in your test file. To verify this, try printing fileContents after you read it in. You'll see that the all input is on a single line, starting with import Thing._class Foo, which clearly isn't correct syntax.
The reason that happens is that getLines gives you a list of lines without line endings and mkString joins them together without any separator. The quick fix would be to simply pass "\n" as the separator to mkString, but the better solution is to not read the file yourself at all.
Instead you can make ANTLR do it by creating your input stream using CharStreams.fromFileName. This will also get rid of the warning about ANTLRInputStream being deprecated.
I'm working on a mapper and wanted a typesafe way to capture class fieldnames for mapping and went with a syntax I'd used in C#:
case class Person(name: String, age: Int)
new Mapping[Person]() {
field(_.age).name("person_age").colType[java.lang.Integer]
field(_.name).name("person_name")
}
where def field(m: T => Unit): FieldMap
This triggers the following warnings:
Warning:(97, 13) a pure expression does nothing in statement position; you may be omitting necessary parentheses
field(_.age).name("person_age").colType[java.lang.Integer]
^
Warning:(98, 13) a pure expression does nothing in statement position; you may be omitting necessary parentheses
field(_.name).name("person_name")
^
So clearly that's not a desirable syntax. Any way I can tweak the signature of field to avoid the warning or is there a more idiomatic scala way of mapping fields in a typesafe manner?
Note: #sjrd's answer indeed gets rid of the warning, but the attempted feature doesn't seem feasible with scala reflection after all. My end goal is a Mapper that allows the specifying of T members in a compile time checked mannner, rather than strings, so it's less vulnerable to typo's and refactoring issues.
The field method takes a T => Unit function as parameter. Hence, the lambda _.age, which is equivalent to x => x.age, is typechecked as returning Unit. The compiler warns that you are using a pure expression (x.age) in statement position (expected type Unit), which basically means that the expression is useless, and might as well be removed.
There is a very simple symptomatic solution to your problem: replace m: T => Unit by m: T => Any. Now your expression x.age is not in statement position anymore, and the compiler is happy.
But your code suggests that there is something wrong a little bit deeper, since you obviously don't use the result of m anywhere. Why is m for anyway?
I wonder if this:
object Foo {
val regex = "some complex regex".r
def foo() {
// use regex
}
}
and this:
object Foo {
def foo() {
val regex = "some complex regex".r
// use regex
}
}
will have any performance difference. i.e., will scala compiler recognize that "some complex regex".r is a constant and cache it, so that it will not recompile every time?
It will have a difference in runtime. Expression from first example will be calculated only once. Expression from second - every time you call Foo.foo(). Calculation here means applying implicitly added function "r" (from scala-library) to the string:
scala> ".*".r
res40: scala.util.matching.Regex = .*
This function actually compiles the regular expression every time you call it (no caching).
Btw, any naive caching of regexps in runtime is vulnerable to OutOfMemory - however, I believe it's possible to implement it safely with WeakHashMap, but current Java's Pattern implementation (which is underlying to scala's Regex) doesn't implement it actually, probably because such implementation may not have predictable effect on performance (GC may have to remove most of cached values every time it's running). Cache with eviction is more predictable, but still not so easy way (who's gonna choose timeout/size for it?). Talking about scala-way, some smart macro could do optimization in compile-time (do caching only for 'string constant'-based regexps), but by default:
Scala compiler also doesn't have any optimizations about regexps because regexp is not a part of the scala language.
So it's better to move static "".r constructions out of the function.
When I run an external process, looks like we need to add a blank line after the calling. Why it throws that error?
class JobActor extends Actor {
def receive = {
case msg:String =>
Process(msg)!
// here need a blank line, otherwise it throws error
sender ! "complete"
}
}
the error is
[error] found : akka.actor.ActorRef
[error] required: scala.sys.process.ProcessLogger
[error] sender ! "complete"
[error] ^
[error] one error found
You hit the exact reason of why postfix operators is a feature that generates a warning when you compile with the -feature flag.
Here's an extract that explains why such feature is still under discussion (emphasis added):
postfixOps. Only where enabled, postfix operator notation (expr op)
will be allowed. Why keep the feature? Several DSLs written in Scala
need the notation. Why control it? Postfix operators interact poorly
with semicolon inference. Most programmers avoid them for this reason.
(source)
Process defines two methods
abstract def !: Int
abstract def !(log: ProcessLogger): Int
When you do
Process(msg)!
you mean to call the former, but since there's no unambiguous indication that the line should end (i.e. that a semicolon should be inferred), the parser starts reading the next line, it finds something can can syntactically be an argument (sender) and you end up calling the second version of ! instead.
The resulting code is actually:
Process(msg)! sender ! "complete"
i.e.
(Process(msg).!(sender)).!("complete")
hence the error: sender is not an instance of ProcessLogger.
In order to fix it, you have to untangle the ambiguity yourself. There's many ways of doing it, the simplest one being to avoid the postfix operator altogether:
Process(msg).!
sender ! "complete"
Actually, given this other question of yours, you can even just do
msg.!
sender ! "complete"
This could be another case where Scala's grammar does not allow for postfix operators in any other place than the end of an expression.
The first one was described in "Why does this code need an empty line or a semicolon?"
Adding a semi-colon might also make the code compile, because of the semicolon inference.
Process(msg)!;
sender ! "complete"