I want to express this grammar in scala StdTokenParsers:
expr -> expr ("+"|"-") ~ muldivexpr | muldivexpr
"+" and "-" is left associative.
The grammar is left recursive so it caused infinite recursions. I can rewrite to remove left recursion, but it will change to right associativity.
Now I am planning to use scala rep() to rewrite it as:
expr -> rep(muldivexpr ("+"|"-")) ~ muldivexpr
but will rep() that change the associativity? How does rep() work in this case?
I am asking this question because I have to output the AST in the future.
You are most likely looking for:
chainl1[T](p: => Parser[T], q: => Parser[(T, T) => T]): Parser[T]
The general idea is p is an operand, and q is a separator yielding a function which can combine two operands, e.g.
chainl1(muldivexpr,
"+" ^^^ { (l: Expr, r: Expr) => Addition(l, r) }
| "-" ^^^ { (l: Expr, r: Expr) => Subtraction(l, r) }
)
Related
Why does foldLeft syntax operator works, for example i would expect this code
(10 /: (1 to 5))(_ + _)
To give me an error "value /: is not a member of Int". How does it expands method /: on all types in type system?
Here is the definition of the "shortcut" operator:
def /:[B](z: B)(op: (B, A) => B): B = foldLeft(z)(op)
If operator ends with a colon, it is a right-associative. 1 :: Nil is another example, there is no method :: on Int
this all works:
(1 to 5)./:(10)(_ + _)
((1 to 5) foldLeft 10)(_ + _) (almost the same as your example,
but here it's more obvious that foldLeft is actually a method on the
range object)
(1 to 5).foldLeft(10)(_ + _)
Your question is not entirely clear (there's no n mentioned in your expression), but: Operators that end with a colon are interpreted as methods on the right-hand argument, not the left. Your expression is equivalent to
(1 to 5)./:(10)(_ + _)
in which /: is more clearly seen to be a method of the Range object on the left.
Using an extension of the Scala parser, I wish to parse two different types of information – patterns and filters. These patterns and filters may appear in any order.
With the format for patterns and filters defined in the variables pattern and filter, respectively, I want to unite them so that filters and patterns may be supplied in any order:
val patternRepetition: Parser[List[ParsedPattern]] = {
rep1sep(pattern, ".") <~ opt(".")
}
val patternOrFilter: Parser[List[ParsedPattern]] = {
patternRepetition | rep1(filterSpec)
}
val patternList: Parser[List[ParsedPattern]] = {
rep1(patternOrFilter) ^^ {
_.flatten
}
}
For reference, the code for pattern and filter are shown at the end of this post.
Some examples of what patternList is supposed to match:
?A ?C ?B .
?A ?C ?B .
FILTER(?A > ?B)
?A ?C ?B .
?A ?C ?B
FILTER(?A > ?B)
?A ?C ?B
FILTER(?A > ?B)
?A ?C ?B
As soon as a filter appears, the parser complains about an unexpected ( character. However, changing | to ~ in patternOrFilter will successfully parse a list of patterns followed by a filter (such as in the second and third example), so I believe there is an issue with my use of the alternative operator |.
Therefore, my question is: why does the | operator fail to recognize filters?
A pattern is currently defined as the following:
val pattern: Parser[ParsedPattern] = {
"?A" ~! "?C" ~! "?B" ^^ {
case s ~ p ~ o =>
ParsedPattern(s, p, o)
}
}
This obviously matches:
?A ?C ?B
A filter is defined with the following code:
val filter: Parser[ParsedPattern] = {
"FILTER(" ~> "?A" ~ ">" ~ "?B") <~ ")" ^^ {
case lhs ~ comp ~ rhs =>
ParsedPattern(lhs, comp, rhs)
}
}
This matches the following line:
FILTER(?A > ?B)
| parses ONE side only (patterns OR filter but not both).
Try a new rule like (change types, I used _):
def patternOrFilterAll: Parser[_] = {
rep1(patternOrFilter)
}
You may need to change patternRepetition and patternOrFilter to get the return value you want.
edit
Your Parser does not work because of the usage of ~!. If you use ~ instead the parser then your example will work (e.g. http://scalafiddle.net/console/97ea3cfb64eeaa1edba65501d0bb3c86 ).
The reason: The parser uses backtracking and ~! will disable that. The parser needs backtracking because the url regex may fail.
I am working on a simple expression parser, however given the following parser combinator declarations below, I can't seem to pass my tests and a right associative tree keeps on popping up.
def EXPR:Parser[E] = FACTOR ~ rep(SUM|MINUS) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def SUM:Parser[E => E] = "+" ~ EXPR ^^ {case "+" ~ b => Sum(_, b)}
def MINUS:Parser[E => E] = "-" ~ EXPR ^^ {case "-" ~ b => Diff(_, b)}
I've been debugging hours for this. I hope someone can help me figure it out it's not coming out right.
"5-4-3" would yield a tree that evaluates to 4 instead of the expected -2.
What is wrong with the grammar above?
I don't work with Scala but do work with F# parser combinators and also needed associativity with infix operators. While I am sure you can do 5-4 or 2+3, the problem comes in with a sequence of two or more such operators of the same precedence and operator, i.e. 5-4-2 or 2+3+5. The problem won't show up with addition as (2+3)+5 = 2+(3+5) but (5-4)-2 <> 5-(4-2) as you know.
See: Monadic Parser Combinators 4.3 Repetition with meaningful separators. Note: The separators are the operators such as "+" and "*" and not whitespace or commas.
See: Functional Parsers Look for the chainl and chainr parsers in section 7. More parser combinators.
For example, an arithmetical expressions, where the operators that
separate the subexpressions have to be part of the parse tree. For
this case we will develop the functions chainr and chainl. These
functions expect that the parser for the separators yields a function
(!);
The function f should operate on an element and a list of tuples, each
containing an operator and an element. For example, f(e0; [(1; e1);
(2; e2); (3; e3)]) should return ((eo 1 e1) 2 e2) 3 e3. You may
recognize a version of foldl in this (albeit an uncurried one), where
a tuple (; y) from the list and intermediate result x are combined
applying x y.
You need a fold function in the semantic parser, i.e. the part that converts the tokens from the syntactic parser into the output of the parser. In your code I believe it is this part.
{case a~b => (a /: b)((acc,f) => f(acc))}
Sorry I can't do better as I don't use Scala.
"-" ~ EXPR ^^ {case "-" ~ b => Diff(_, b)}
for 5-4-3, it expands to
Diff(5, 4-3)
which is
Diff(5, Diff(4, 3))
however, what you need is:
Diff(Diff(5, 4), 3))
// for 5 + 4 - 3 it should be
Diff(Sum(5, 4), 3)
you need to involve stack.
It seems using "+" ~ EXPR made the answer incorrect. It should have been FACTOR instead.
I am working on a Parsing logic that needs to take operator precedence into consideration. My needs are not too complex. To start with I need multiplication and division to take higher precedence than addition and subtraction.
For example: 1 + 2 * 3 should be treated as 1 + (2 * 3). This is a simple example but you get the point!
[There are couple more custom tokens that I need to add to the precedence logic, which I may be able to add based on the suggestions I receive here.]
Here is one example of dealing with operator precedence: http://jim-mcbeath.blogspot.com/2008/09/scala-parser-combinators.html#precedencerevisited.
Are there any other ideas?
This is a bit simpler that Jim McBeath's example, but it does what you say you need, i.e. correct arithmetic precdedence, and also allows for parentheses. I adapted the example from Programming in Scala to get it to actually do the calculation and provide the answer.
It should be quite self-explanatory. There is a heirarchy formed by saying an expr consists of terms interspersed with operators, terms consist of factors with operators, and factors are floating point numbers or expressions in parentheses.
import scala.util.parsing.combinator.JavaTokenParsers
class Arith extends JavaTokenParsers {
type D = Double
def expr: Parser[D] = term ~ rep(plus | minus) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def plus: Parser[D=>D] = "+" ~ term ^^ {case "+"~b => _ + b}
def minus: Parser[D=>D] = "-" ~ term ^^ {case "-"~b => _ - b}
def term: Parser[D] = factor ~ rep(times | divide) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def times: Parser[D=>D] = "*" ~ factor ^^ {case "*"~b => _ * b }
def divide: Parser[D=>D] = "/" ~ factor ^^ {case "/"~b => _ / b}
def factor: Parser[D] = fpn | "(" ~> expr <~ ")"
def fpn: Parser[D] = floatingPointNumber ^^ (_.toDouble)
}
object Main extends Arith with App {
val input = "(1 + 2 * 3 + 9) * 2 + 1"
println(parseAll(expr, input).get) // prints 33.0
}
How do you use Map.foldLeft? According to the docs it looks like
foldLeft [B] (z: B)(op: (B, (A, B)) ⇒ B) : B
But I'm having difficulty:
Map("first"->1,"second"->2).foldLeft(0)((a,(k,v)) => a+v )
error: not a legal formal parameter
The error points to the open bracket in front of k.
If you want to use the (a, (k, v)) syntax, you need to advise the compiler to use pattern matching.
Map("first"->1, "second"->2).foldLeft(0){ case (a, (k, v)) => a+v }
Note that a case statement requires curly braces.
I think, you can't do the pattern match on tuples as you expect:
Map("first"->1,"second"->2).foldLeft(0)((a, t) => a + t._2)
Actually, using values and sum is simpler.
Map("first"->1,"second"->2).values.sum
The trick is to use a partial function as the code block, in other words you add a case statement that matches on the arguments:
Map("first" -> 1, "second" -> 2).foldLeft(0) { case (a, (k, v)) => a + v }
This is not really an answer to your question but I found it useful when starting out with folds, so I'll say it anyway! Note that the /: method "alias" for foldLeft can be clearer for two reasons:
xs.foldLeft(y) { (yy, x) => /* ... */ }
(y /: xs) { (yy, x) => /* ... */ }
Note that in the second line:
it's more clear that the value y is being folded into the collection xs
you can easily remember the ordering of the Tuple2 argument is the same as the ordering of the method "call"