I made a lexer based on Parsing M-Expressions in Scala with combinators.
import scala.util.parsing.combinator.lexical._
import scala.util.parsing.combinator.token._
trait MyTokens extends Tokens {
case class Id(chars: String) extends Token
case class Num(chars: String) extends Token
}
class MyLexical extends Lexical with MyTokens {
def whitespace = rep(whitespaceChar)
def token: Parser[Token] = (
rep1(letter) ^^ { case charList => Id(charList mkString "") }
| rep1(digit) ^^ { case charList => Num(charList mkString "") }
)
}
object Main extends App {
val lexical = new MyLexical
val scanner = new lexical.Scanner("abc012def345")
def tokenList(s: lexical.Scanner): List[lexical.Token] =
if (s.atEnd) Nil
else s.first :: tokenList(s.rest)
println(tokenList(scanner))
}
It worked well resulting in List(Id(abc), Num(012), Id(def), Num(345)). However, the function tokenList should be a method of Scanner to call it as scanner.tokenList. In order to achieve it, it seems that the inner class MyLexical.Scanner should be overriden. Is it possible? If you can, how to do it?
I don't think you can add a method to inner class, but you can use implicit conversion to achieve similar result. It would look like following:
object ScannerImplicits {
implicit class BetterScanner(scanner: MyLexical.Scanner) {
def tokenList(s: MyLexical.Scanner): List[MyLexical.Token] = {
if (s.atEnd) Nil
else s.first :: tokenList(s.rest)
}
}
}
Than you'll be able to write
import ScannerImplicits ._
new lexical.Scanner("abc012def345").tokenList
Related
I need to write two functions to get the output format and the output index for file conversion. As part of this, I wrote a TransformSettings class for these methods and set the default value. And in the transformer class, I created a new object of TransformSettings class to get the default values for each job run. Also, I have another class called ParquetTransformer that extends Transformer where I want to change these default values. So I implemented like below.
class TransformSettings{
def getOuputFormat: String = {
"orc"
}
def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("orc.column.index.access")
}
}
class Transformer{
def getTransformSettings: TransformSettings = {
new TransformSettings
}
def posttransform(table: AWSGlueDDL.Table):Dateframe ={
val indexAccess = getTransformSettings.getOuputIndex(table: AWSGlueDDL.Table)
........
}
}
class ParquetTransformer extends Transformer{
override def getTransformSettings: TransformSettings = {
val transformSettings = new TransformSettings {
override def getOuputFormat: String = {
"parquet"
}
override def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("parquet.column.index.access")
}
}
}
}
Is there a way to avoid creating a brand new object of TransformSettings in Transfomer class every time this is called?
Also is there a way to rewrite the code using Scala value class?
As #Dima proposed in the comments try to make TransformSettings a field / constructor parameter (a val) in the class Transformer and instantiate them outside
class TransformSettings{
def getOuputFormat: String = {
"orc"
}
def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("orc.column.index.access")
}
}
class Transformer(val transformSettings: TransformSettings) {
def posttransform(table: AWSGlueDDL.Table): DataFrame ={
val indexAccess = transformSettings.getOuputIndex(table: AWSGlueDDL.Table)
???
}
}
val parquetTransformSettings = new TransformSettings {
override def getOuputFormat: String = {
"parquet"
}
override def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("parquet.column.index.access")
}
}
class ParquetTransformer extends Transformer(parquetTransformSettings)
You don't seem to need value classes (... extends AnyVal) now. They are more about unboxing, not about life-cycle management. TransformSettings and Transformer can't be value classes because they are not final (you're extending them in class ParquetTransformer extends Transformer... and new TransformSettings { ... }). By the way, value classes have many limatations
https://failex.blogspot.com/2017/04/the-high-cost-of-anyval-subclasses.html
https://github.com/scala/bug/issues/12271
Besides value classes, there are scala-newtype library in Scala 2 and opaque types in Scala 3.
Need to extract tokens and fixed text. Example:
"Hello {token1} today's date is {token2} would you like to call {token3}"
would return
FixedPart("Hello ")
TokenPart(token1)
FixedPart(" today's date is ")
TokenPart(token2)
FixedPart(" would you like to call ")
TokenPart(token3)
Here is naive implementation
import org.parboiled2.ParserInput
import org.parboiled2.Parser
import org.parboiled2.CharPredicate
sealed trait Part
case class TokenPart(tokenName : String ) extends Part
case class FixedPart( text : String ) extends Part
class MyParser(val input: ParserInput) extends Parser {
def Token = rule { '{' ~ capture(TokenName) ~> (TokenPart(_)) ~'}' }
//how this should be implemented??
def NotToken = rule { capture (!Token) ~>(FixedPart(_) )}
def TokenName = rule { CharPredicate.Alpha ~ oneOrMore (CharPredicate.AlphaNum) }
// This would not work
def TokenNotToken = rule { (Token|NotToken) }
def InputLine = rule { zeroOrMore (TokenNotToken) }
}
object MyParser {
def main(args: Array[String]) {
val res = new MyParser("Hello {token1} today's date is {token2} would you like to call {token3}").InputLine.run() // Success
println( res )
}
}
Any other to implement this ??
Hi I modified your code and added some comments (I hope they will be helpful), so it works, and (I guess) does what you wanted it to do:
import org.parboiled2.ParserInput
import org.parboiled2.Parser
import org.parboiled2.CharPredicate
sealed trait Token
case class TokenPart(tokenName : String) extends Token
case class StringToken(text: String) extends Token
// I moved pre-evaluated char predicates to the companion
// you may leave them inside the class if you want.
// I also moved literals like startToken and endToken here
object TokenExtractor {
val AlphaChar = CharPredicate.Alpha
val AlphaNum = CharPredicate.AlphaNum
val startToken = "{"
val endToken = "}"
}
class TokenExtractor(val input: ParserInput) extends Parser {
import TokenExtractor._
// may be you wanted zero or more? Anyway in this case
// shortcut can play nice here. In fact, if you want to stick
// with oneOrMore you can user AlphaNum.+ instead
def TokenName = rule {
AlphaChar ~ AlphaNum.*
}
// There's a shortcut for Extraction syntax. If you are extracting
// data to the case class and Rule arguments match the number of
// items in the case class's apply method
// you can simply give a name of this case class:
// the extraction operator '~>' should be located at the end of the
// from the official documtation:
// https://github.com/sirthias/parboiled2
// One more very useful feature is special support for
// case class instance creation:
//
// case class Person(name: String, age: Int)
// (foo: Rule2[String, Int]) ~> Person
//
def Token = rule {
startToken ~ capture(TokenName) ~ endToken ~> TokenPart
}
// the text should follow until the parser will meet the
// enclosing '{' character. Disclosing is not mandatory :)
def Text = rule {
oneOrMore(noneOf(startToken))
}
// Here we're capturing a data that matches
// pre-defined rule (in our case Text)
def TextString = rule {
capture(Text) ~> StringToken
}
def TextPart = rule {
TextString | Token
}
// EOI is mandatory. Parser is greedy, so it tells the parser
// where parsing procedure must end, so please, add it at the
// end of the input
def InputLine = rule {
zeroOrMore(TextPart) ~ EOI
}
}
object Main {
def main(args: Array[String]) {
val example =
"Hello {token1} today's date is {token2} would you like to call {token3}"
// parser input can be string, so put it inside the constructor
val result = new TokenExtractor(example).InputLine.run()
println(result)
}
}
I have some (Akka) actor code that is using a case class + the copy constructor to update state:
def foo(state:StateCaseClass) : Receive = {
import state._
{
case Bar(updates) =>
context become foo(copy(/* change a limited number of things */))
// ... other message processing w/ lots of context become foo(copy(...))
}
}
I'd like to add below the import
def update = context become foo(copy(_))
so that the code can be
def foo(state:StateCaseClass) : Receive = {
import state._
def update = context become foo(copy(_))
{
case Bar(updates) =>
update(/* change a limited number of things */)
// ... etc
}
}
but that doesn't compile. I can of course tweak the def update a bit to get rid of most of boilerplate, but the copy still sticks around:
def foo(state:StateCaseClass) : Receive = {
import state._
def update(newState:StateCaseClass) = context become foo(newState)
{
case Bar(updates) =>
update(copy(/* change a limited number of things */))
// ... etc
}
}
Is there comparable syntax that will let me pass through the args to the case class copy constructor and dry out that last bit?
Disclaimer: I guess the best solution is to use context become explicitly. And I don't recommend you to use the code below.
I guess it's impossible without metaprogramming (macros). You have to create a method with default values for named parameters.
You could always create such method manually like this:
def update(filed1: Int = state.field1, field2: String = state.field2) =
context become foo(StateCaseClass(filed1, filed2))
...
update(field1 = 0)
...
update(field2 = "str")
But I guess it's not what you want.
The only way to get such method without metaprogramming is... to use method copy itself. Method copy calls constructor and you could call become in constructor.
The code below works, but I strongly don't recommend you to use it! It's a cryptocode and it will confuse all other developers.
import akka.actor._
trait ReceiveHelper extends PartialFunction[Any, Unit] {
def receive: PartialFunction[Any, Unit]
override def apply(v: Any) = receive(v)
override def isDefinedAt(v: Any) = receive isDefinedAt v
}
sealed trait TestActorMessage
case object Get extends TestActorMessage
case class SetInt(i: Int) extends TestActorMessage
case class SetString(s: String) extends TestActorMessage
class TestActor extends Actor {
case class Behaviour(intField: Int, strField: String) extends ReceiveHelper {
context become this
val receive: Receive = {
case Get => sender ! (intField -> strField)
case SetInt(i) => copy(intField = i)
case SetString(s) => copy(strField = s)
}
}
def receive = Behaviour(0, "init")
}
Usage:
val system = ActorSystem("testSystem")
val testActor = system.actorOf(Props[TestActor], "testActor")
import akka.pattern.ask
import akka.util.Timeout
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext.Implicits.global
implicit val timeout = Timeout(5 seconds)
testActor ? Get foreach println
// (0,init)
testActor ! SetInt(666)
testActor ? Get foreach println
// (666,init)
testActor ! SetString("next")
testActor ? Get foreach println
// (666,next)
I have an optional field on my requests:
case class SearchRequest(url: String, nextAt: Option[Date])
My protocol is:
object SearchRequestJsonProtocol extends DefaultJsonProtocol {
implicit val searchRequestFormat = jsonFormat(SearchRequest, "url", "nextAt")
}
How do I mark the nextAt field optional, such that the following JSON objects will be correctly read and accepted:
{"url":"..."}
{"url":"...", "nextAt":null}
{"url":"...", "nextAt":"2012-05-30T15:23Z"}
I actually don't really care about the null case, but if you have details, it would be nice. I'm using spray-json, and was under the impression that using an Option would skip the field if it was absent on the original JSON object.
Works for me (spray-json 1.1.1 scala 2.9.1 build)
import cc.spray.json._
import cc.spray.json.DefaultJsonProtocol._
// string instead of date for simplicity
case class SearchRequest(url: String, nextAt: Option[String])
// btw, you could use jsonFormat2 method here
implicit val searchRequestFormat = jsonFormat(SearchRequest, "url", "nextAt")
assert {
List(
"""{"url":"..."}""",
"""{"url":"...", "nextAt":null}""",
"""{"url":"...", "nextAt":"2012-05-30T15:23Z"}""")
.map(_.asJson.convertTo[SearchRequest]) == List(
SearchRequest("...", None),
SearchRequest("...", None),
SearchRequest("...", Some("2012-05-30T15:23Z")))
}
You might have to create an explicit format (warning: psuedocodish):
object SearchRequestJsonProtocol extends DefaultJsonProtocol {
implicit object SearchRequestJsonFormat extends JsonFormat[SearchRequest] {
def read(value: JsValue) = value match {
case JsObject(List(
JsField("url", JsString(url)),
JsField("nextAt", JsString(nextAt)))) =>
SearchRequest(url, Some(new Instant(nextAt)))
case JsObject(List(JsField("url", JsString(url)))) =>
SearchRequest(url, None)
case _ =>
throw new DeserializationException("SearchRequest expected")
}
def write(obj: SearchRequest) = obj.nextAt match {
case Some(nextAt) =>
JsObject(JsField("url", JsString(obj.url)),
JsField("nextAt", JsString(nextAt.toString)))
case None => JsObject(JsField("url", JsString(obj.url)))
}
}
}
Use NullOptions trait to disable skipping nulls:
https://github.com/spray/spray-json#nulloptions
Example:
https://github.com/spray/spray-json/blob/master/src/test/scala/spray/json/ProductFormatsSpec.scala
Don't know if this will help you but you can give that field a default value in the case class definition, so if the field is not in the json, it will assign the default value to it.
Easy.
import cc.spray.json._
trait MyJsonProtocol extends DefaultJsonProtocol {
implicit val searchFormat = new JsonWriter[SearchRequest] {
def write(r: SearchRequest): JsValue = {
JsObject(
"url" -> JsString(r.url),
"next_at" -> r.nextAt.toJson,
)
}
}
}
class JsonTest extends FunSuite with MyJsonProtocol {
test("JSON") {
val search = new SearchRequest("www.site.ru", None)
val marshalled = search.toJson
println(marshalled)
}
}
For anyone who is chancing upon this post and wants an update to François Beausoleil's answer for newer versions of Spray (circa 2015+?), JsField is deprecated as a public member of JsValue; you should simply supply a list of tuples instead of JsFields. Their answer is spot-on, though.
As far as I understand it, Scala creates an anonymous class if I create a class using the new keyword and follow the class name with a constructor:
class MyClass {
def doStuff() {
// ...
}
}
val mc = new MyClass {
doStuff()
}
The nice thing being that all the code in the constructor is in the scope of the new object.
Is there a way I can reproduce this syntax where the class is created by a factory method rather than the new keyword? i.e. make the following code work:
val mf = new MyFactory
val mc = mf.MyClass {
doStuff()
}
I can't find a way to do it but Scala has so much to it that this might be pretty easy!
Using an import as suggested by #Ricky below I can get:
val mf = MyFactory;
val mc = mf.MyClass
{
import mc._
doStuff()
}
(Where the blank line before the block is needed) but that code block is not a constructor.
You can do this, but you still have to keep the new keyword, and create the nested class as a path-dependent type:
class Bippy(x: Int) {
class Bop {
def getIt = x
}
}
val bip = new Bippy(7)
val bop = new bip.Bop
bop.getIt // yields 7
val bop2 = new bip.Bop{ override def getIt = 42 }
bop2.getIt // yields 42
I don't think it's possible. However, a common pattern is to add a parameter to factory methods which takes a function modifying the created object:
trait MyClass {
var name = ""
def doStuff():Unit
}
class Foo extends MyClass {
def doStuff() { println("FOO: " + name) }
}
trait MyClassFactory {
def make: MyClass
def apply( body: MyClass => Unit ) = {
val mc = make
body(mc)
mc
}
}
object FooFactory extends MyClassFactory {
def make = new Foo
}
You can then create and modify instance with a syntax close to your example:
val foo = FooFactory { f=>
f.name = "Joe"
f.doStuff
}
It sounds like you're just looking to mix in a trait. Instead of calling myFactoryMethod(classOf[Foo]] which ideally would do (if Scala permitted it):
new T {
override def toString = "My implementation here."
}
you can instead write
trait MyImplementation {
override def toString = "My implementation here."
}
new Foo with MyImplementation
However, if you are just looking to get the members of the new object accessible without qualification, remember you can import from any stable identifier:
val foo = new Bar
import foo._
println(baz) //where baz is a member of foo.