parboiled2 parser to extract token and fixed text - scala

Question

Need to extract tokens and fixed text. Example:
"Hello {token1} today's date is {token2} would you like to call {token3}"
would return
FixedPart("Hello ")
TokenPart(token1)
FixedPart(" today's date is ")
TokenPart(token2)
FixedPart(" would you like to call ")
TokenPart(token3)
Here is naive implementation
import org.parboiled2.ParserInput
import org.parboiled2.Parser
import org.parboiled2.CharPredicate
sealed trait Part
case class TokenPart(tokenName : String ) extends Part
case class FixedPart( text : String ) extends Part
class MyParser(val input: ParserInput) extends Parser {
def Token = rule { '{' ~ capture(TokenName) ~> (TokenPart(_)) ~'}' }
//how this should be implemented??
def NotToken = rule { capture (!Token) ~>(FixedPart(_) )}
def TokenName = rule { CharPredicate.Alpha ~ oneOrMore (CharPredicate.AlphaNum) }
// This would not work
def TokenNotToken = rule { (Token|NotToken) }
def InputLine = rule { zeroOrMore (TokenNotToken) }
}
object MyParser {
def main(args: Array[String]) {
val res = new MyParser("Hello {token1} today's date is {token2} would you like to call {token3}").InputLine.run() // Success
println( res )
}
}
Any other to implement this ??

Answer 1

Hi I modified your code and added some comments (I hope they will be helpful), so it works, and (I guess) does what you wanted it to do:
import org.parboiled2.ParserInput
import org.parboiled2.Parser
import org.parboiled2.CharPredicate
sealed trait Token
case class TokenPart(tokenName : String) extends Token
case class StringToken(text: String) extends Token
// I moved pre-evaluated char predicates to the companion
// you may leave them inside the class if you want.
// I also moved literals like startToken and endToken here
object TokenExtractor {
val AlphaChar = CharPredicate.Alpha
val AlphaNum = CharPredicate.AlphaNum
val startToken = "{"
val endToken = "}"
}
class TokenExtractor(val input: ParserInput) extends Parser {
import TokenExtractor._
// may be you wanted zero or more? Anyway in this case
// shortcut can play nice here. In fact, if you want to stick
// with oneOrMore you can user AlphaNum.+ instead
def TokenName = rule {
AlphaChar ~ AlphaNum.*
}
// There's a shortcut for Extraction syntax. If you are extracting
// data to the case class and Rule arguments match the number of
// items in the case class's apply method
// you can simply give a name of this case class:
// the extraction operator '~>' should be located at the end of the
// from the official documtation:
// https://github.com/sirthias/parboiled2
// One more very useful feature is special support for
// case class instance creation:
//
// case class Person(name: String, age: Int)
// (foo: Rule2[String, Int]) ~> Person
//
def Token = rule {
startToken ~ capture(TokenName) ~ endToken ~> TokenPart
}
// the text should follow until the parser will meet the
// enclosing '{' character. Disclosing is not mandatory :)
def Text = rule {
oneOrMore(noneOf(startToken))
}
// Here we're capturing a data that matches
// pre-defined rule (in our case Text)
def TextString = rule {
capture(Text) ~> StringToken
}
def TextPart = rule {
TextString | Token
}
// EOI is mandatory. Parser is greedy, so it tells the parser
// where parsing procedure must end, so please, add it at the
// end of the input
def InputLine = rule {
zeroOrMore(TextPart) ~ EOI
}
}
object Main {
def main(args: Array[String]) {
val example =
"Hello {token1} today's date is {token2} would you like to call {token3}"
// parser input can be string, so put it inside the constructor
val result = new TokenExtractor(example).InputLine.run()
println(result)
}
}