I would like to traverse a collection resulting from the Scala JSON toolkit at github.
The problem is that the JsonParser returns "Any" so I am wondering how I can avoid the following error:
"Value foreach is not a member of Any".
val json = Json.parse(urls)
for(l <- json) {...}
object Json {
def parse(s: String): Any = (new JsonParser).parse(s)
}
You will have to do pattern matching to traverse the structures returned from the parser.
/*
* (untested)
*/
def printThem(a: Any) {
a match {
case l:List[_] =>
println("List:")
l foreach printThem
case m:Map[_, _] =>
for ( (k,v) <- m ) {
print("%s -> " format k)
printThem(v)
}
case x =>
println(x)
}
val json = Json.parse(urls)
printThem(json)
You might have more luck using the lift-json parser, available at: http://github.com/lift/lift/tree/master/framework/lift-base/lift-json/
It has a much richer type-safe DSL available, and (despite the name) can be used completely standalone outside of the Lift framework.
If you are sure that in all cases there will be only one type you can come up with the following cast:
for (l <- json.asInstanceOf[List[List[String]]]) {...}
Otherwise do a Pattern-Match for all expected cases.
Related
Learning Scala and having some troubles with streams.
I'm trying to filter a collection of "Element" (from scala-parser library, kind of all the Soup objects) based on the fact that it contains a "%" and extract the value.
override def extractRoi(line: Element): Double = {
line.asInstanceOf[JsoupElement].underlying
.select("td")
.stream()
.map(e => e.text().toString)
.filter(e => e.contains("%"))
.findFirst()
.orElse("")
.replace("%", "").toDouble
}
when I map by doing "e.text()" I should have a Stream[String] but it is a Stream[_$2] and I don't understand why. So the code doesn't when at the orElse.
What transformation do I need to do to end up with a Stream[String] ?
It seems that using Java streams in Scala does something weird. In my code above it couldn't infer the right type. So this is how I fixed it:
override def extractRoi(line: Element): Double = {
line.select("td")
.iterator()
.asScala
.map(e => e.text().toString)
.filter(e => e.contains("%"))
.findFirst()
.orElse("")
.replace("%", "").toDouble
}
I have a task to transform the following code-block:
val instance = instanceFactory.create
val result = instance.ackForResult
to for-comprehension expression.
As for-comprehension leans on enumeration of elements, I tried to get around it with wrapper class:
case class InstanceFactoryWrapper(value:InstanceFactory) {
def map(f: InstanceFactory => Instance): Instance
= value.create()
}
where map-method must handle only one element and return a single result: Instance
I tested this approach with this expression:
for {
mediationApi <- InstanceFactoryWrapper(instanceFactoryWrapper)
}
But it does't work: IDEA recommends me to use foreach in this part. But "foreach" doesn't return anything, as opposed to map.
What am I doing wrong?
Simply put when working with List\Option\Either or other lang types comprehensions are useful to transform nested map\flatMap\withFilter into sequences.
Use custom classes in for-comprehension
But what about your own classes or other 3rd party ones?
You need to implement monadic operations in order to use them in for-comprehensions.
The bare minimum: map and flatMap.
Take the following example with a custom Config class:
case class Config[T](content: T) {
def flatMap[S](f: T => Config[S]): Config[S] =
f(content)
def map[S](f: T => S): Config[S] =
this.copy(content = f(content))
}
for {
first <- Config("..")
_ = println("Going through a test")
second <- Config(first + "..")
third <- Config(second + "..")
} yield third
This is how you enable for-comprehension.
How to conveniently convert Seq[Try[Option[String, Any]]] into Try[Option[Map[String, Any]]].
If any Try before convert throws an exception, the converted Try should throw as well.
Assuming that the input type has a tuple inside the Option then this should give you the result you want:
val in: Seq[Try[Option[(String, Any)]]] = ???
val out: Try[Option[Map[String,Any]]] = Try(Some(in.flatMap(_.get).toMap))
If any of the Trys is Failure then the outer Try will catch the exception raised by the get and return Failure
The Some is there to give the correct return type
The get extracts the Option from the Try (or raises an exception)
Using flatMap rather than map removes the Option wrapper, keeping all Some values and discaring None values, giving Seq[(String, Any)]
The toMap call converts the Seq to a Map
Here is something that's not very clean but may help get you started. It assumes Option[(String,Any)], returns the first Failure if there are any in the input Seq and just drops None elements.
foo.scala
package foo
import scala.util.{Try,Success,Failure}
object foo {
val x0 = Seq[Try[Option[(String, Any)]]]()
val x1 = Seq[Try[Option[(String, Any)]]](Success(Some(("A",1))), Success(None))
val x2 = Seq[Try[Option[(String, Any)]]](Success(Some(("A",1))), Success(Some(("B","two"))))
val x3 = Seq[Try[Option[(String, Any)]]](Success(Some(("A",1))), Success(Some(("B","two"))), Failure(new Exception("bad")))
def f(x: Seq[Try[Option[(String, Any)]]]) =
x.find( _.isFailure ).getOrElse( Success(Some(x.map( _.get ).filterNot( _.isEmpty ).map( _.get ).toMap)) )
}
Example session
bash-3.2$ scalac foo.scala
bash-3.2$ scala -classpath .
Welcome to Scala 2.13.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_66).
Type in expressions for evaluation. Or try :help.
scala> import foo.foo._
import foo.foo._
scala> f(x0)
res0: scala.util.Try[Option[Equals]] = Success(Some(Map()))
scala> f(x1)
res1: scala.util.Try[Option[Equals]] = Success(Some(Map(A -> 1)))
scala> f(x2)
res2: scala.util.Try[Option[Equals]] = Success(Some(Map(A -> 1, B -> two)))
scala> f(x3)
res3: scala.util.Try[Option[Equals]] = Failure(java.lang.Exception: bad)
scala> :quit
If you're willing to use a functional support library like Cats then there are two tricks that can help this along:
Many things like List and Try are traversable, which means that (if Cats's implicits are in scope) they have a sequence method that can swap two types, for example converting List[Try[T]] to Try[List[T]] (failing if any of the items in the list are failure).
Almost all of the container types support a map method that can operate on the contents of a container, so if you have a function from A to B then map can convert a Try[A] to a Try[B]. (In Cats language they are functors but the container-like types in the standard library generally have map already.)
Cats doesn't directly support Seq, so this answer is mostly in terms of List instead.
Given that type signature, you can iteratively sequence the item you have to in effect push the list type down one level in the type chain, then map over that container to work on its contents. That can look like:
import cats.implicits._
import scala.util._
def convert(listTryOptionPair: List[Try[Option[(String, Any)]]]): Try[
Option[Map[String, Any]]
] = {
val tryListOptionPair = listTryOptionPair.sequence
tryListOptionPair.map { listOptionPair =>
val optionListPair = listOptionPair.sequence
optionListPair.map { listPair =>
Map.from(listPair)
}
}
}
https://scastie.scala-lang.org/xbQ8ZbkoRSCXGDJX0PgJAQ has a slightly more complete example.
One way to approach this is by using a foldLeft:
// Let's say this is the object you're trying to convert
val seq: Seq[Try[Option[(String, Any)]]] = ???
seq.foldLeft(Try(Option(Map.empty[String, Any]))) {
case (acc, e) =>
for {
accOption <- acc
elemOption <- e
} yield elemOption match {
case Some(value) => accOption.map(_ + value)
case None => accOption
}
}
You start off with en empty Map. You then use a for comprehension to go through the current map and element and finally you add a new tuple in the map if present.
The following solutions is based on this answer to the point that almost makes the question a duplicate.
Method 1: Using recursion
def trySeqToMap1[X,Y](trySeq : Seq[Try[Option[(X, Y)]]]) : Try[Option[Map[X,Y]]] = {
def helper(it : Iterator[Try[Option[(X,Y)]]], m : Map[X,Y] = Map()) : Try[Option[Map[X,Y]]] = {
if(it.hasNext) {
val x = it.next()
if(x.isFailure)
Failure(x.failed.get)
else if(x.get.isDefined)
helper(it, m + (x.get.get._1-> x.get.get._2))
else
helper(it, m)
} else Success(Some(m))
}
helper(trySeq.iterator)
}
Method 2: directly pattern matching in case you are able to get a stream or a List instead:
def trySeqToMap2[X,Y](trySeq : LazyList[Try[Option[(X, Y)]]], m : Map[X,Y]= Map.empty[X,Y]) : Try[Option[Map[X,Y]]] =
trySeq match {
case Success(Some(h)) #:: tail => trySeqToMap2(tail, m + (h._1 -> h._2))
case Success(None) #:: tail => tail => trySeqToMap2(tail, m)
case Failure(f) #:: _ => Failure(f)
case _ => Success(Some(m))
}
note: this answer was previously using different method signatures. It has been updated to conform to the signature given in the question.
Trying to emit a for yield block from a blackbox macro, but I'm failing to understand how you can create the block with valid syntax.
So below source is a hardcoded param name as this block is later inserted inside a method that will have the matching param name. params is just params: Seq[c.universe.ValDef], enclosing the case class fields.
def extract(source: Source): Option[CaseClass] = { ... }
val extractors = accessors(c)(params) map {
case (nm, tpe) => {
val newTerm = TermName(nm.toString + "Opt")
q"""$newTerm <- DoStuff[$tpe].apply("$nm", source)"""
}
}
val extractorNames = accessors(c)(params) map {
case (nm, tpe) => TermName(nm.toString + "Opt")
}
This is basically taking a case class, and outputting a for yield black to basically recreate the case class from a comprehension.
Every field in the case class of the form name: Type is transformed to a set of extractors that yield the same case class instance back if the for comprehension is successful.
case class Test(id: Int, text: String)
Will be macro transformed to the following, where Extract is just a type class and Extract.apply[T : Extract] is just materialising the context bound with implicitly[Extract[T]]:
for {
idOpt <- Extract[Int].apply("id", source): Option[Int]
textOpt <- Extract[String].apply("text", source): Option[String]
} yield Test(idOpt, textOpt)
The problem comes in having to quote the inner for yield expressions with and output a <- b blocks.
def extract(source: Source): Option[$typeName] = {
for {(..$extractors)} yield $companion.apply(..$extractorNames)
}
The error is ';' expected but '<-' found, which is pretty obvious as a <- b is invalid Scala by itself. What is the correct way to generate and quasiquote the expression block such that the above would work?
Here is a list of all the different kinds of quasiquotes.
There you can see that to express the a <- b syntax you need the fq interpolator.
So that code will probably become:
val extractors = accessors(c)(params) map {
case (nm, tpe) => {
val newTerm = TermName(nm.toString + "Opt")
fq"""$newTerm <- DoStuff[$tpe].apply("$nm", source)"""
}
}
And then with the normal interpolator:
q"for (..$extractors) yield $companion.apply(..$extractorNames)"
Option monad is a great expressive way to deal with something-or-nothing things in Scala. But what if one needs to log a message when "nothing" occurs? According to the Scala API documentation,
The Either type is often used as an
alternative to scala.Option where Left
represents failure (by convention) and
Right is akin to Some.
However, I had no luck to find best practices using Either or good real-world examples involving Either for processing failures. Finally I've come up with the following code for my own project:
def logs: Array[String] = {
def props: Option[Map[String, Any]] = configAdmin.map{ ca =>
val config = ca.getConfiguration(PID, null)
config.properties getOrElse immutable.Map.empty
}
def checkType(any: Any): Option[Array[String]] = any match {
case a: Array[String] => Some(a)
case _ => None
}
def lookup: Either[(Symbol, String), Array[String]] =
for {val properties <- props.toRight('warning -> "ConfigurationAdmin service not bound").right
val logsParam <- properties.get("logs").toRight('debug -> "'logs' not defined in the configuration").right
val array <- checkType(logsParam).toRight('warning -> "unknown type of 'logs' confguration parameter").right}
yield array
lookup.fold(failure => { failure match {
case ('warning, msg) => log(LogService.WARNING, msg)
case ('debug, msg) => log(LogService.DEBUG, msg)
case _ =>
}; new Array[String](0) }, success => success)
}
(Please note this is a snippet from a real project, so it will not compile on its own)
I'd be grateful to know how you are using Either in your code and/or better ideas on refactoring the above code.
Either is used to return one of possible two meaningful results, unlike Option which is used to return a single meaningful result or nothing.
An easy to understand example is given below (circulated on the Scala mailing list a while back):
def throwableToLeft[T](block: => T): Either[java.lang.Throwable, T] =
try {
Right(block)
} catch {
case ex => Left(ex)
}
As the function name implies, if the execution of "block" is successful, it will return "Right(<result>)". Otherwise, if a Throwable is thrown, it will return "Left(<throwable>)". Use pattern matching to process the result:
var s = "hello"
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints "HELLO"
s = null
throwableToLeft { s.toUpperCase } match {
case Right(s) => println(s)
case Left(e) => e.printStackTrace
}
// prints NullPointerException stack trace
Hope that helps.
Scalaz library has something alike Either named Validation. It is more idiomatic than Either for use as "get either a valid result or a failure".
Validation also allows to accumulate errors.
Edit: "alike" Either is complettly false, because Validation is an applicative functor, and scalaz Either, named \/ (pronounced "disjonction" or "either"), is a monad.
The fact that Validation can accumalate errors is because of that nature. On the other hand, / has a "stop early" nature, stopping at the first -\/ (read it "left", or "error") it encounters. There is a perfect explanation here: http://typelevel.org/blog/2014/02/21/error-handling.html
See: http://scalaz.googlecode.com/svn/continuous/latest/browse.sxr/scalaz/example/ExampleValidation.scala.html
As requested by the comment, copy/paste of the above link (some lines removed):
// Extracting success or failure values
val s: Validation[String, Int] = 1.success
val f: Validation[String, Int] = "error".fail
// It is recommended to use fold rather than pattern matching:
val result: String = s.fold(e => "got error: " + e, s => "got success: " + s.toString)
s match {
case Success(a) => "success"
case Failure(e) => "fail"
}
// Validation is a Monad, and can be used in for comprehensions.
val k1 = for {
i <- s
j <- s
} yield i + j
k1.toOption assert_≟ Some(2)
// The first failing sub-computation fails the entire computation.
val k2 = for {
i <- f
j <- f
} yield i + j
k2.fail.toOption assert_≟ Some("error")
// Validation is also an Applicative Functor, if the type of the error side of the validation is a Semigroup.
// A number of computations are tried. If the all success, a function can combine them into a Success. If any
// of them fails, the individual errors are accumulated.
// Use the NonEmptyList semigroup to accumulate errors using the Validation Applicative Functor.
val k4 = (fNel <**> fNel){ _ + _ }
k4.fail.toOption assert_≟ some(nel1("error", "error"))
The snippet you posted seems very contrived. You use Either in a situation where:
It's not enough to just know the data isn't available.
You need to return one of two distinct types.
Turning an exception into a Left is, indeed, a common use case. Over try/catch, it has the advantage of keeping the code together, which makes sense if the exception is an expected result. The most common way of handling Either is pattern matching:
result match {
case Right(res) => ...
case Left(res) => ...
}
Another interesting way of handling Either is when it appears in a collection. When doing a map over a collection, throwing an exception might not be viable, and you may want to return some information other than "not possible". Using an Either enables you to do that without overburdening the algorithm:
val list = (
library
\\ "books"
map (book =>
if (book \ "author" isEmpty)
Left(book)
else
Right((book \ "author" toList) map (_ text))
)
)
Here we get a list of all authors in the library, plus a list of books without an author. So we can then further process it accordingly:
val authorCount = (
(Map[String,Int]() /: (list filter (_ isRight) map (_.right.get)))
((map, author) => map + (author -> (map.getOrElse(author, 0) + 1)))
toList
)
val problemBooks = list flatMap (_.left.toSeq) // thanks to Azarov for this variation
So, basic Either usage goes like that. It's not a particularly useful class, but if it were you'd have seen it before. On the other hand, it's not useless either.
Cats has a nice way to create an Either from exception-throwing code:
val either: Either[NumberFormatException, Int] =
Either.catchOnly[NumberFormatException]("abc".toInt)
// either: Either[NumberFormatException,Int] = Left(java.lang.NumberFormatException: For input string: "abc")
in https://typelevel.org/cats/datatypes/either.html#working-with-exception-y-code