class ExprParser extends RegexParsers {
val number = "[0-9]+".r
def expr: Parser[Int] = term ~ rep(
("+" | "-") ~ term ^^ {
case "+" ~ t => t
case "-" ~ t => -t
}) ^^ { case t ~ r => t + r.sum }
def term: Parser[Int] = factor ~ (("*" ~ factor)*) ^^ {
case f ~ r => f * r.map(_._2).product
}
def factor: Parser[Int] = number ^^ { _.toInt } | "(" ~> expr <~ ")"
}
I get the following warning when compiling
warning: match may not be exhaustive.
It would fail on the following input: ~((x: String forSome x not in ("+", "-")), _)
("+" | "-") ~ term ^^ {
^
one warning found
I heard that #unchecked annotation can help. But in this case where should I put it?
The issue here is that with ("+" | "-") you are creating a parser that accepts only two possible strings. However when you map on the resulting parser to extract the value, the result you're going to extract will just be String.
In your pattern matching you only have cases for the strings "+" and "-", but the compiler has no way of knowing that those are the only possible strings that will show up, so it's telling you here that your match may not be exhaustive since it can't know any better.
You could use an unchecked annotation to suppress the warning, but there are much better, more idiomatic ways, to eliminate the issue. One way to solve this is to replace those strings with some kind of structured type as soon as possible. For example, create an ADT
sealed trait Operation
case object Plus extends Operation
case object Minus extends Operation
//then in your parser
("+" ^^^ Plus | "-" ^^^ Minus) ~ term ^^ {
case PLus ~ t => t
case Minus ~ t => -t
}
Now it should be able to realize that the only possible cases are Plus and Minus
Add a case to remove the warning
class ExprParser extends RegexParsers {
val number = "[0-9]+".r
def expr: Parser[Int] = term ~ rep(
("+" | "-") ~ term ^^ {
case "+" ~ t => t
case "-" ~ t => -t
case _ ~ t => t
}) ^^ { case t ~ r => t + r.sum }
def term: Parser[Int] = factor ~ (("*" ~ factor)*) ^^ {
case f ~ r => f * r.map(_._2).product
}
def factor: Parser[Int] = number ^^ { _.toInt } | "(" ~> expr <~ ")"
}
Related
I am trying to write a Scala Parser combinator for the following input.
The input can be
10
(10)
((10)))
(((10)))
Here the number of brackets can keep on growing. but they should always match. So parsing should fail for ((((10)))
The result of parsing should always be the number at the center
I wrote the following parser
import scala.util.parsing.combinator._
class MyParser extends RegexParsers {
def i = "[0-9]+".r ^^ (_.toInt)
def n = "(" ~ i ~ ")" ^^ {case _ ~ b ~ _ => b.toInt}
def expr = i | n
}
val parser = new MyParser
parser.parseAll(parser.expr, "10")
parser.parseAll(parser.expr, "(10)")
but now how do I handle the case where the number of brackets keep growing but matched?
Easy, just make the parser recursive:
class MyParser extends RegexParsers {
def i = "[0-9]+".r ^^ (_.toInt)
def expr: Parser[Int] = i | "(" ~ expr ~ ")" ^^ {case _ ~ b ~ _ => b.toInt}
}
(but note that scala-parser-combinators has trouble with left-recursive definitions: Recursive definitions with scala-parser-combinators)
I am writing a Parser in scala and got stuck at this point:
private def expression : Parser[Expression] = cond | variable | integer | liste | function
private def cond : Parser[Expression] = "if" ~ predicate ~ "then" ~ expression ~ "else" ~ expression ^^ {case _~i~_~t~_~el => Cond(i,t,el)}
private def predicate: Parser[Predicate] = identifier ~ "?" ~ "(" ~ repsep(expression, ",") ~ ")" ^^{case n~_~_~el~_ => Predicate(n,el)}
private def function: Parser[Expression] = identifier ~ "(" ~ repsep(expression, ",") ~ ")" ^^{case n~_~el~_ => Function(n,el)}
private def liste: Parser[Expression] = "[" ~ repsep(expression, ",") ~ "]" ^^ {case _~ls~_ => Liste(ls)}
private def variable: Parser[Expression] = identifier ^^ {case v => Variable(v)}
def identifier: Parser[String] = """[a-zA-Z0-9]+""".r ^^ { _.toString }
def integer: Parser[Integer] = num ^^ { case i => Integer(i)}
def num: Parser[String] = """(-?\d*)""".r ^^ {_.toString}
My problem is that when it comes to an "expression" the Parser does not always takes the right way. Like if its funk(x,y) it tries to parse it like a variable ant not like a function.
Any idea?
Change order of parsers in your expression parser - put function before variable and after cond. In general, when you compose parsers using alternative A | B, then parser A shouldn't be able to parse input that is prefix of input parsable by parser B.
In earlier versions of scala the discard method existed to throw away the results of a parser:
lazy val throwThisAway: Parser[String] = (ows ~> discard(comma | EOF | EOL)) <~ ows
How may that be achieved in current versions of the library .. i.e while simply doing
otherParser ~ throwThisAway ~ anotherParser ^^ { case a ~ b // only 2, not 3, parser results
You can simply reference the extra parser result but not use it:
otherParser ~ throwThisAway ~ anotherParser ^^ { case a ~ x ~ b => // eg.: SomeCaseClass(a,b)
or even:
otherParser ~ throwThisAway ~ anotherParser ^^ { case a ~ _ ~ b => ...
You can use <~ (sequential composition that keeps only the left) or ~> (sequential composition that keeps only the right) instead. For example:
(otherParser <~ throwThisAway) ~ anotherParser ^^ { case a ~ b => ... }
Or
otherParser ~ (throwThisAway ~> anotherParser) ^^ { case a ~ b => ... }
Programming in Scala's Chapter 33 explains Combinator Parsing:
It provides this example:
import scala.util.parsing.combinator._
class Arith extends JavaTokenParsers {
def expr: Parser[Any] = term~rep("+"~term | "-"~term)
def term: Parser[Any] = factor~rep("*"~factor | "/"~factor)
def factor: Parser[Any] = floatingPointNumber | "("~expr~")"
}
How can I map expr to a narrower type than Parser[Any]? In other words,
I'd like to take def expr: Parser[Any] and map that via ^^ into a stricter type.
Note - I asked this question in Scala Google Groups - https://groups.google.com/forum/#!forum/scala-user, but haven't received a complete answer that helped me out.
As already stated in the comments, you can narrow down the type to anything you like. You just have to specify it after the ^^.
Here is a complete example with a data structure from your given code.
object Arith extends JavaTokenParsers {
trait Expression //The data structure
case class FNumber(value: Float) extends Expression
case class Plus(e1: Expression, e2: Expression) extends Expression
case class Minus(e1: Expression, e2: Expression) extends Expression
case class Mult(e1: Expression, e2: Expression) extends Expression
case class Div(e1: Expression, e2: Expression) extends Expression
def expr: Parser[Expression] = term ~ rep("+" ~ term | "-" ~ term) ^^ {
case term ~ rest => rest.foldLeft(term)((result, elem) => elem match {
case "+" ~ e => Plus(result, e)
case "-" ~ e => Minus(result, e)
})
}
def term: Parser[Expression] = factor ~ rep("*" ~ factor | "/" ~ factor) ^^ {
case factor ~ rest => rest.foldLeft(factor)((result, elem) => elem match {
case "*" ~ e => Mult(result, e)
case "/" ~ e => Div(result, e)
})
}
def factor: Parser[Expression] = floatingPointNumber ^^ (f => FNumber(f.toFloat)) | "(" ~> expr <~ ")"
def parseInput(input: String): Expression = parse(expr, input) match {
case Success(ex, _) => ex
case _ => throw new IllegalArgumentException //or change the result to Try[Expression]
}
}
Now we can start to parse something.
Arith.parseInput("(1.3 + 2.0) * 2")
//yields: Mult(Plus(FNumber(1.3),FNumber(2.0)),FNumber(2.0))
Of course you can also have a Parser[String] or a Parser[Float], where you directly transform or evaluate the input String. It is as I said up to you.
I have created a Scala parser combinator to filter data records based on the answer I got to an previous question How to parse a string with filter citeria in scala and use it to filter objects
I would like to add the calculator parser combinator from the answer to this question
Operator Precedence with Scala Parser Combinators
to the bottom of the parser combinator that I created based on the first question. The calculator parser combinator therefore needs to accept a dataRecord so that an expression like "( doubleValue1 / 10 ) * 2 + doubleValue2" can be parsed to an function that subsequently can take a dataRecord.
This is what I came up with but the plus, minus, times and divide parser combinators are now broken because the + - * / operators are members of Double and not the function DataRecord => Double. How can I fix these parser combinators so that an expression like "( doubleValue1 / 10 ) * 2 + doubleValue2" can be succesfully parsed and results in an function that can take a dataRecord?
import scala.util.parsing.combinator._
import scala.util.parsing.combinator.JavaTokenParsers
object Main extends Arith with App {
val dataRecord = new DataRecord(100, 75 )
val input = "( doubleValue1 / 10 ) * 2 + doubleValue2"
println(parseAll(arithmicExpr, input).get(dataRecord)) // prints 95
}
class DataRecord( val doubleValue1 : Double, val doubleValue2 : Double )
class Arith extends JavaTokenParsers {
type D = Double
type Extractor[Double] = DataRecord => Double
//arithmic expression
def arithmicExpr: Parser[Extractor[D]] = term ~ rep(plus | minus) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def plus: Parser[Extractor[D]=>Extractor[D]] = "+" ~ term ^^ {case "+"~b => _ + b}
def minus: Parser[Extractor[D]=>Extractor[D]] = "-" ~ term ^^ {case "-"~b => _ - b}
def term: Parser[Extractor[D]] = factor ~ rep(times | divide) ^^ {case a~b => (a /: b)((acc,f) => f(acc))}
def times: Parser[Extractor[D]=>Extractor[D]] = "*" ~ factor ^^ {case "*"~b => _ * (b) }
def divide: Parser[Extractor[D]=>Extractor[D]] = "/" ~ factor ^^ {case "/"~b => _ / b}
def factor: Parser[Extractor[D]] = fpn | "(" ~> arithmicExpr <~ ")" | intExtractor
def fpn: Parser[Extractor[D]] = floatingPointNumber ^^ (s => Function.const(s.toDouble)_)
def intExtractor: Parser[Extractor[D]] = ("doubleValue1" | "doubleValue2") ^^ {
case "doubleValue1" => _.doubleValue1
case "doubleValue2" => _.doubleValue2
}
}
Your approach to avoid a left recursive grammar is nice, but makes the types really complex. I prefer a different approach:
object ArithParser extends JavaTokenParsers {
//arithmic expression
def arithmicExpr: Parser[Extractor[D]] = plus
def plus: Parser[Extractor[D]] = repsep(times, "+") ^^ { summands : List[Extractor[D]] =>
(in : DataRecord) => summands.map((e : Extractor[D]) => e(in)).foldLeft(0d)(_ + _)
}
def times: Parser[Extractor[D]] = repsep(division, "*") ^^ { factors : List[Extractor[D]] =>
(in : DataRecord) => factors.map((e : Extractor[D]) => e(in)).foldLeft(1d)(_ * _)
}
def division : Parser[Extractor[D]] = rep1sep(number, "/") ^^ {divisons : List[Extractor[D]] =>
(in : DataRecord) => divisons.map((e : Extractor[D]) => e(in)).reduce(_ / _)
} | number
def number : Parser[Extractor[D]] = fpn | intExtractor
def fpn: Parser[Extractor[D]] = floatingPointNumber ^^ (s => Function.const(s.toDouble)_)
def intExtractor: Parser[Extractor[D]] = ("doubleValue1" | "doubleValue2") ^^ {
case "doubleValue1" => _.doubleValue1
case "doubleValue2" => _.doubleValue2
}
}
You can find a live demo here.
This code can be further improved: It contains lots of repeating structures. Perhaps this is a good case for Stack exchange's code review site.
Enhancements for other arithmetic operators, for mathematical functions and especially for braces are straight forward.