I've got very strange behaviour when I run playframework in scala. I used anorm as database access layer thus I've started doing some code and I saw very strange scala compiler behavoiur.
This is code which is working:
case class P_Page_Control(Control_ID:Int,
Client_ID:String,
cContent: String,
Page_ID: Int,
Language_ID: Int,
InsertTime: Date,
ChangeTime: Option[Date],
IsDeleted: Boolean)
and:
object P_Page_Control { val parser = {
get[Int]("Control_ID") ~
get[String]("Client_ID") ~
get[String]("Content") ~
get[Int]("Page_ID") ~
get[Int]("Language_ID") ~
get[Date]("InsertTime") ~
get[Option[Date]]("ChangeTime") ~
get[Boolean]("IsDeleted") map {
case a ~ b ~ c ~ d ~ e ~ f ~ g ~ h =>
P_Page_Control(a, b, c, d, e, f, g, h)
}}}
For this moment no compilation error. Works fine.
But when I change property name I got errors:
object P_Page_Control { val parser = {
get[Int]("Control_ID") ~
get[String]("Client_ID") ~
get[String]("Content") ~
get[Int]("Page_ID") ~
get[Int]("Language_ID") ~
get[Date]("InsertTime") ~
get[Option[Date]]("ChangeTime") ~
get[Boolean]("IsDeleted") map {
case A_B ~ b ~ c ~ d ~ e ~ f ~ g ~ h =>
P_Page_Control(A_B, b, c, d, e, f, g, h)
}}}
As I'm a totally new to Scala I thought _ is some magic keyword or other magic stuff.
So I changed property name to aBB_AccAd but there was no compilation errors.
ooops...
Next funny thing: I renamed this to AAbbdddsadasdasAAFFFFeeee and I saw again compilation errors.
So what motivates Scala to throw compilation error for some set of literals?
Is this a bug or feature ? :-)
Names in patterns, which start with a capital letter, are interpreted as variable or object names that refer to an extractor (an object with an unapply or unapplySeq method). Since you haven't declared such a variable or object, Scala can't find it and throws an error.
Related
Consider
import util.parsing.combinator._
object TreeParser extends JavaTokenParsers {
lazy val expr: Parser[String] = decimalNumber | sum
//> expr: => TreeParser.Parser[String]
lazy val sum: Parser[String] = expr ~ "+" ~ expr ^^ {case a ~ plus ~ b => s"($a)+($b)"}
//> sum: => TreeParser.Parser[String]
println(parseAll(expr, "1 + 1")) //> TreeParser.ParseResult[String] = [1.3] failure: string matching regex
//| `\z' expected but `+' found
}
The same story with fastparse
import fastparse.all._
val expr: P[Any] = P("1" | sum)
val sum: P[Any] = expr ~ "+" ~ expr
val top = expr ~ End
println(top.parse("1+1")) // Failure(End:1:2 ..."+1")
Parsers are great to discover that taking the first literal is a bad idea but do not try to fall back to the sum production. Why?
I understand that parser takes the first branch that can successfully eat up a part of input string and exits. Here, "1" of expression matches the first input char and parsing completes. In order to grab more, we need to make sum the first alternative. However, plain stupid
lazy val expr: Parser[String] = sum | "1"
endы up with stack overflow. The library authors therefore approach it from another side
val sum: P[Any] = P( num ~ ("+".! ~/ num).rep )
val top: P[Any] = P( sum ~ End )
Here, we start sum with terminal, which is fine but this syntax is more verbose and, furthermore, it produces a terminal, followed by a list, which is good for a reduction operator, like sum, but is difficult to map to a series of binary operators.
What if your language defines expression, which admits a binary operator? You want to match every occurrence of expr op expr and map it to a corresponding tree node
expr ~ "op" ~ expr ^^ {case a ~ _ ~ b => BinOp(a,b)"}
How do you do that? In short, I want a greedy parser, that consumes the whole string. This is what I mean by 'greedy' rather than greedy algorigthm that jumps into the first wagon and ends up in a dead end.
As I have found here, we need to replace | alternative operator with secret |||
//lazy val expr: Parser[String] = decimalNumber | sum
lazy val backtrackGreedy: Parser[String] = decimalNumber ||| sum
lazy val sum: Parser[String] = decimalNumber ~ "+" ~ backtrackGreedy ^^ {case a ~ plus ~ b => s"($a)+($b)"}
println(parseAll(backtrackGreedy, "1 + 1")) // [1.6] parsed: (1)+(1)
The order of alternatives does not matter with this operator. To stop stack overflow, we need to eliminate the left recursion, sum = expr + expr => sum = number + expr.
Another answer says that we need to normalize, that is instead of
def foo = "foo" | "fo"
def obar = "obar"
def foobar = foo ~ obar
we need to use
def workingFooBar = ("foo" ~ obar) | ("fo" ~ obar)
But first solution is more striking.
The parser does backtrack. Try val expr: P[String] = P(("1" | "1" ~ "+" ~ "1").!) and expr.parse("1+1") for example.
The problem is in your grammar. expr parses 1 and it is a successful parsing by your definition. Then sum fails and now you want to blame the dutiful expr for what happened?
There are plenty of examples on how to deal with binary operators. For example, the first example here: http://lihaoyi.github.io/fastparse/
What is the meaning and purpose of the syntactic sugar in this code?
def exp: Parser[Expr] = operands ~ binOp ~ exp ^^ {case e1~o~e2=>BinaryOp(o,e1,e2)}
In particular, what do each of these expressions mean?
operands ~ binOp ~ exp ^^
e1~o~e2
operands ~ binOp ~ exp ^^ ...
Operators in scala are just ordinary method calls:
operands ~ binOp ~ exp ^^ ...
is the same as
operands.~(binOp).~(exp).^^(...)
You can see the documentation for the ~ and ^^ methods here, or you should be able to click through to them in your IDE.
case e1~o~e2
This is matching a case class called ~ - lots of two-parameter things can be written in this "infix notation" in scala. It's equivalent to:
case ~(e1, ~(o, e2))
(see the documentation on case classes)
Those don't have any special meaning - they're just methods that are called ~ and ^^. You'll need to look at the documentation/implementation of whichever library you're using that defined them to figure out what they do.
I want to parse a part of my DSL formed like this:
configSignal: sticky Config
Semantically this is:
argument_name: 0_or_more_modifiers argument_type
I tried implementing the following parser:
def parser = ident ~ ":" ~ rep(ident) ~ ident ^^ {
case name ~ ":" ~ modifiers ~ returnType => Arg(name, returnType, modifiers)
}
Thing is, the rep(ident) part is applied until there are no more tokens and the parser fails, because the last ~ ident doesn't match. How should I do this properly?
Edit
In the meantime I realized, that the modifiers will be reserved words (keywords), so now I have:
def parser = ident ~ ":" ~ rep(modifier) ~ ident ^^ {
case name ~ ":" ~ modifiers ~ returnType => Arg(name, returnType, modifiers)
}
def modifier = "sticky" | "control" | "count"
Nevertheless, I'm curious if it would be possible to write a parser if the modifiers weren't defined up front.
"0 or more idents followed by ident" is equivalent to "1 or more idents", so just use rep1
Its docs:
def rep1[T](p: ⇒ Parser[T]): Parser[List[T]]
A parser generator for non-empty repetitions.
rep1(p) repeatedly uses p to parse the input until p fails -- p must succeed at least once (the result is a List of the consecutive results of p)
p a Parser that is to be applied successively to the input
returns A parser that returns a list of results produced by repeatedly applying p to the input (and that only succeeds if p matches at least once).
edit in response to OP's comment:
I don't think there's a built-in way to do what you described, but it would still be relatively easy to map to your custom data types by using regular List methods:
def parser = ident ~ ":" ~ rep1(ident) ^^ {
case name ~ ":" ~ idents => Arg(name, idents.last, idents.dropRight(1))
}
In this particular case, you wouldn't have to worry about idents being Nil, since the rep1 parser only succeeds with a non-empty list.
Not digging in details of DSL, I could write (based on that example):
def InputLine = rule { Number ~ zeroOrMore("+" ~ Number ~> ((x: Int, y: Int) => x + y)) ~ EOI }
I need to call lambda function from macro. Scala AST representation of it is as follows:
List(
ValDef(Modifiers(PARAM), newTermName("x"), TypeTree().setOriginal(Select(Ident(scala), scala.Int)), EmptyTree),
ValDef(Modifiers(PARAM), newTermName("y"), TypeTree().setOriginal(Select(Ident(scala), scala.Int)), EmptyTree))
Important to note is that both x and y has types in AST. This works fine.
Next step is to use type inference, and simplify lambda as follows:
def InputLine = rule { Number ~ zeroOrMore("+" ~ Number ~> ((_:Int) + _)) ~ EOI }
Scala AST for that lambda:
List(
ValDef(Modifiers(PARAM | SYNTHETIC), newTermName("x$1"), TypeTree().setOriginal(Select(Ident(scala), scala.Int)), EmptyTree),
ValDef(Modifiers(PARAM | SYNTHETIC), newTermName("x$2"), TypeTree(), EmptyTree))
Note that x$2 param has no explicit type. I need that to call the function. Where should I get it? The solution "just pass parameter of type Any to a function" will not satisfy scalac as it requires Int.
This question is logically followed by that one.
The code is available at parboled2#gihub.
Number ~> ((_:Int) + _)
is transformed to something like this
pimpActionOp[...](rule).~>.apply[...](function)
type classes of pimpActionOp and apply contain all necessary information to restore types.
I am working on a Parsing logic that needs to take operator precedence into consideration. My needs are not too complex. To start with I need multiplication and division to take higher precedence than addition and subtraction.
For example: 1 + 2 * 3 should be treated as 1 + (2 * 3). This is a simple example but you get the point!
[There are couple more custom tokens that I need to add to the precedence logic, which I may be able to add based on the suggestions I receive here.]
Here is one example of dealing with operator precedence: http://jim-mcbeath.blogspot.com/2008/09/scala-parser-combinators.html#precedencerevisited.
Are there any other ideas?
This is a bit simpler that Jim McBeath's example, but it does what you say you need, i.e. correct arithmetic precdedence, and also allows for parentheses. I adapted the example from Programming in Scala to get it to actually do the calculation and provide the answer.
It should be quite self-explanatory. There is a heirarchy formed by saying an expr consists of terms interspersed with operators, terms consist of factors with operators, and factors are floating point numbers or expressions in parentheses.
import scala.util.parsing.combinator.JavaTokenParsers
class Arith extends JavaTokenParsers {
type D = Double
def expr: Parser[D] = term ~ rep(plus | minus) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def plus: Parser[D=>D] = "+" ~ term ^^ {case "+"~b => _ + b}
def minus: Parser[D=>D] = "-" ~ term ^^ {case "-"~b => _ - b}
def term: Parser[D] = factor ~ rep(times | divide) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def times: Parser[D=>D] = "*" ~ factor ^^ {case "*"~b => _ * b }
def divide: Parser[D=>D] = "/" ~ factor ^^ {case "/"~b => _ / b}
def factor: Parser[D] = fpn | "(" ~> expr <~ ")"
def fpn: Parser[D] = floatingPointNumber ^^ (_.toDouble)
}
object Main extends Arith with App {
val input = "(1 + 2 * 3 + 9) * 2 + 1"
println(parseAll(expr, input).get) // prints 33.0
}