I have scala expression stored in String variable:
val myExpr = "(xml \ \"node\")"
How do I execute this?
s"${myExpr}"
Right now it only gives me the string contents
What I'm trying to achieve is parsing user string input in the form:
"/some/node/in/xml"
and get that corresponding node in Scala:
(xml \ "node" \ "in" \ "xml")
For the REPL, my init includes:
implicit class interpoleter(val sc: StringContext) {def i(args: Any*) = $intp interpret sc.s(args: _*) }
with which
scala> val myExpr = "(xml \\ \"node\")"
myExpr: String = (xml \ "node")
scala> val xml = <x><node/></x>
xml: scala.xml.Elem = <x><node/></x>
scala> i"${myExpr}"
res3: scala.xml.NodeSeq = NodeSeq(<node/>)
res2: scala.tools.nsc.interpreter.IR.Result = Success
because isn't code really just a string, like everything else?
Probably, there is some more idiomatic way in recent scala versions, but you can use Twitter's Eval for that:
val i: Int = new Eval()("1 + 1") // => 2
Related
In the following example (scala 2.11 and play-json 2.13)
val j ="""{"t":2.2599999999999997868371792719699442386627197265625}"""
println((Json.parse(j) \ "t").as[BigDecimal].compare(BigDecimal("2.2599999999999997868371792719699442386627197265625")))
The output is -1. Shouldn't they be equal ? On printing the parsed value, it prints rounded off value:
println((Json.parse(j) \ "t").as[BigDecimal]) gives 259999999999999786837179271969944
The problem is that by default play-json configures the Jackson parser with the MathContext set to DECIMAL128. You can fix this by setting the play.json.parser.mathContext system property to unlimited. For example, in a Scala REPL that would look like this:
scala> System.setProperty("play.json.parser.mathContext", "unlimited")
res0: String = null
scala> val j ="""{"t":2.2599999999999997868371792719699442386627197265625}"""
j: String = {"t":2.2599999999999997868371792719699442386627197265625}
scala> import play.api.libs.json.Json
import play.api.libs.json.Json
scala> val res = (Json.parse(j) \ "t").as[BigDecimal]
res: BigDecimal = 2.2599999999999997868371792719699442386627197265625
scala> val expected = BigDecimal("2.2599999999999997868371792719699442386627197265625")
expected: scala.math.BigDecimal = 2.2599999999999997868371792719699442386627197265625
scala> res.compare(expected)
res1: Int = 0
Note that setProperty should happen first, before any reference to Json. In normal (non-REPL) use you'd set the property via -D on the command line or whatever.
Alternatively you could use Jawn's play-json parsing support, which just works as expected off the shelf:
scala> val j ="""{"t":2.2599999999999997868371792719699442386627197265625}"""
j: String = {"t":2.2599999999999997868371792719699442386627197265625}
scala> import org.typelevel.jawn.support.play.Parser
import org.typelevel.jawn.support.play.Parser
scala> val res = (Parser.parseFromString(j).get \ "t").as[BigDecimal]
res: BigDecimal = 2.2599999999999997868371792719699442386627197265625
Or for that matter you could switch to circe:
scala> import io.circe.Decoder, io.circe.jawn.decode
import io.circe.Decoder
import io.circe.jawn.decode
scala> decode(j)(Decoder[BigDecimal].prepare(_.downField("t")))
res0: Either[io.circe.Error,BigDecimal] = Right(2.2599999999999997868371792719699442386627197265625)
…which handles a range of number-related corner cases more responsibly than play-json in my view. For example:
scala> val big = "1e2147483648"
big: String = 1e2147483648
scala> io.circe.jawn.parse(big)
res0: Either[io.circe.ParsingFailure,io.circe.Json] = Right(1e2147483648)
scala> play.api.libs.json.Json.parse(big)
java.lang.NumberFormatException
at java.math.BigDecimal.<init>(BigDecimal.java:491)
at java.math.BigDecimal.<init>(BigDecimal.java:824)
at scala.math.BigDecimal$.apply(BigDecimal.scala:287)
at play.api.libs.json.jackson.JsValueDeserializer.parseBigDecimal(JacksonJson.scala:146)
...
But that's out of scope for this question.
To be honest I'm not sure why play-json defaults to DECIMAL128 for the MathContext, but that's a question for the play-json maintainers, and is also out of scope here.
Can someone tell me why do we have two separate ways of representing pipe(|) and comma(,). Like
sc.textFile(file).map( x => x.split(","))
for comma, and
sc.textFile(file).map( x => x.split('|'))
for pipe.
Keeping both in double quotes, its failing with pipe and comma is giving me correct result.
Below is the full code which I am running
package com.rakesh.singh
import org.apache.spark._
import org.apache.spark.SparkContext._
import org.apache.log4j._
object MPMovie {
def namex ( x : String) = {
val fields = x.split('|')
val id = fields(0).toInt
val name = fields(1).toString
(id , name)
}
def main(rakesh : Array[String]) = {
Logger.getLogger("yoyo").setLevel(Level.ERROR)
val conf = new SparkConf().setAppName("Movies").setMaster("local[2]")
val sc = new SparkContext(conf)
val rdd = sc.textFile("F:/Raakesh/ml-100k/movies.data")
val names = sc.textFile("F:/Raakesh/ml-100k/names.data")
val mappednames = names.map(namex)
val splited = rdd.map(x => (x.split("\t")(1).toInt,1))
//.map(x => (x,1))
val counteachmovie = splited.reduceByKey( (a ,b )=> a + b).map( x => (x._2 , x._1))
val mpm = counteachmovie.max()
println(s"the final value of mpm is $mpm")
mappednames.foreach(println)
val finalname = mappednames.lookup(mpm._2)(0)
println(s"the final value of mpm is $finalname")
}
}
and data files are
movies.data
196 101 3 881250949
186 101 3 891717742
22 103 1 878887116
244 102 2 880606923
names:Data
101|Sajan
102|Mela
103|Hum
There are two different split methods:
The split(",") method comes originally from String.split(regex: String), it works with arbitrary regexes as separators, e.g.
scala> "helloABCworldCABfooBBACCAbar".split("[ABC]+")
res0: Array[String] = Array(hello, world, foo, bar)
The other split('|') comes from StringOps.split(separator: Char), and is rather like a generic Scala-collection operation. It doesn't work with regex, but it works on all StringLike collections, for example on StringBuilders:
scala> val b = new StringBuilder
b: StringBuilder =
scala> b ++= "hello|"
res2: b.type = hello|
scala> b ++= "world"
res3: b.type = hello|world
scala> b.split('|')
res4: Array[String] = Array(hello, world)
The "|" doesn't work with the first method, because it's a nonsensical "OR"-regex. In order to use the pipe | with the split(regex: String) version, you either have to escape it like this "\\|" or (often easier) to enclose it into "[|]"-character class.
I'm trying to parse a command line argument for an sbt InputTask using SBT Parsers (http://www.scala-sbt.org/0.13/docs/Parsing-Input.html) but I'm failing to write a parser to match the following pseudo-regex:
\w+(-n|--dry-run)\w+
Here's the most sensible way of expressing this that I can think of. The results here should be Some(true) if the input string matches.
import sbt.complete.Parser
import sbt.complete.DefaultParsers._
val dryRunOptions: Parser[String] = OptSpace ~> ("-n" | "--dry-run") <~ OptSpace
val dryRunParser: Parser[Boolean] = flag(dryRunOptions)
Parser(dryRunParser)("-n").result
Parser(dryRunParser)(" -n").result
Parser(dryRunParser)("-n ").result
Parser(dryRunParser)(" -n ").result
Parser(dryRunParser)("--dry-run").result
Parser(dryRunParser)(" --dry-run").result
Parser(dryRunParser)("--dry-run ").result
Parser(dryRunParser)(" --dry-run ").result
Unfortunately, this does not match any of these cases!
res0: Option[Boolean] = None
res1: Option[Boolean] = None
res2: Option[Boolean] = None
res3: Option[Boolean] = None
res4: Option[Boolean] = None
res5: Option[Boolean] = None
res6: Option[Boolean] = None
res7: Option[Boolean] = None
I can get this to match several of the cases with a couple of variations on this but never all of them. Any help appreciated!
You are checking correctness of your parser in the wrong way. It looks like that in this case you should use .resultEmpty.isValid instead of .result, like in tests here. Then it occurs that your code works fine:
import sbt.complete.Parser
import sbt.complete.DefaultParsers._
val dryRunOptions: Parser[String] = OptSpace ~> ("-n" | "--dry-run") <~ OptSpace
val dryRunParser: Parser[Boolean] = flag(dryRunOptions)
val test = Seq("-n", " -n", "-n ", " -n ",
"--dry-run", " --dry-run", "--dry-run ", " --dry-run ")
test.foldLeft(true)((b:Boolean, input:String) =>
b && Parser(dryRunParser)(input).resultEmpty.isValid)
And the result:
res0: Boolean = true
I could not find how to convert a String into runnable code, for instance:
val i = "new String('Yo')"
// conversion
println(i)
should print
Yo
after the conversion.
I found the following example in another post:
import scala.tools.nsc.interpreter.ILoop
import java.io.StringReader
import java.io.StringWriter
import java.io.PrintWriter
import java.io.BufferedReader
import scala.tools.nsc.Settings
object FuncRunner extends App {
val line = "sin(2 * Pi * 400 * t)"
val lines = """import scala.math._
|var t = 1""".stripMargin
val in = new StringReader(lines + "\n" + line + "\nval f = (t: Int) => " + line)
val out = new StringWriter
val settings = new Settings
val looper = new ILoop(new BufferedReader(in), new PrintWriter(out))
val res = looper process settings
Console println s"[$res] $out"
}
link: How to convert a string from a text input into a function in a Scala
But it seems like scala.tools is not available anymore, and I'm a newbie in Scala so i could not figure out how to replace it.
And may be there are just other ways to do it now.
Thanks !
You can simple execute your code contained inside String using Quasiquotes(Experimental Module).
import scala.reflect.runtime.universe._
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
// TO compile and run code we will use a ToolBox api.
val toolbox = currentMirror.mkToolBox()
// write your code starting with q and put it inside double quotes.
// NOTE : you will have to use triple quotes if you have any double quotes usage in your code.
val code1 = q"""new String("hello")"""
//compile and run your code.
val result1 = toolbox.compile(code1)()
// another example
val code2 = q"""
case class A(name:String,age:Int){
def f = (name,age)
}
val a = new A("Your Name",22)
a.f
"""
val result2 = toolbox.compile(code2)()
Output in REPL :
// Exiting paste mode, now interpreting.
import scala.reflect.runtime.universe._
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
toolbox: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = scala.tools.reflect.ToolBoxFactory$ToolBoxImpl#69b34f89
code1: reflect.runtime.universe.Tree = new String("hello")
result1: Any = hello
code2: reflect.runtime.universe.Tree =
{
case class A extends scala.Product with scala.Serializable {
<caseaccessor> <paramaccessor> val name: String = _;
<caseaccessor> <paramaccessor> val age: Int = _;
def <init>(name: String, age: Int) = {
super.<init>();
()
};
def f = scala.Tuple2(name, age)
};
val a = new A("Your Name", 22);
a.f
}
result2: Any = (Your Name,22)
scala>
To learn more about Quasiquotes :
http://docs.scala-lang.org/overviews/quasiquotes/setup.html
I found a simple solution using the ToolBox tool :
val cm = universe.runtimeMirror(getClass.getClassLoader)
val tb = cm.mkToolBox()
val str = tb.eval(tb.parse("new String(\"Yo\")"))
println(str)
This is printing:
Yo
The scala compiler (and the "interpreter loop") are available here. For example:
https://github.com/scala/scala/blob/v2.11.7/src/repl/scala/tools/nsc/interpreter/ILoop.scala
A compiled jar that has that class will be in your scala distribution under lib\scala-compiler.jar.
One way could be to take/parse that string code and then write it yourself in some file as Scala code. This way it would be executed by Scala compiler when you will call it.
An example: just like Scala is doing with Java. It takes this code and then convert it into Java by making use of main method.
object Xyz extends App {
println ("Hello World!")
}
I'm using https://github.com/json4s/json4s. How do I convert its values such as JDouble, JBool to corresponding Scala's data types -- Double and Boolean?
UPDATE:
scala> (json \ "status")
res8: org.json4s.JValue = JBool(false)
scala> (json \ "status").extract[Boolean]
<console>:16: error: No org.json4s.Formats found. Try to bring an instance of org.json4s.Formats in scope or use the org.json4s.DefaultFormats.
(json \ "status").extract[Boolean]
As mentioned in Read Me, heres how you do it.. :)
import org.json4s._
import org.json4s.native.JsonMethods._
implicit val formats = DefaultFormats
val json = parse("""
{
"mydouble" : 3.14,
"isPie" : true
}
""")
val dbl = (json \ "mydouble").extractOpt[Double]
//> dbl : Option[Double] = Some(3.14)
val bool = (json \ "isPie").extractOpt[Boolean]
//> bool : Option[Boolean] = Some(true)
Looking at the code (https://github.com/json4s/json4s/blob/7c70e9664232ffee4acf24c8969424cd37957736/ast/src/main/scala/org/json4s/JsonAST.scala) shows that you just need to call the JValue.values method.
UPDATE: From your comment it seems that it's not so much that you have a JDouble and want to extract its Double value (and similarly extract a Boolean from a JDouble. Instead, you have a JValue
and want to extract its value as a Double or Boolean (knowing in advance the expected type).
This can be done with extract, as explained in the README that you linked to:
(json \ "status").extract[Double]
or:
(json \ "status").extract[Boolean]
See also this test file for more examples:
https://github.com/json4s/json4s/blob/master/tests/src/test/scala/org/json4s/ExtractionExamplesSpec.scala