I have a
object radExtractor{
def unapplySeq(row:HtmlTableRow):Option[List[String]]={
val lista = (for{
a<-row.getByXPath("td/span/a")
ah= a.asInstanceOf[DomNode]
if(ah.getFirstChild!=null)
} yield a.asInstanceOf[DomNode].getFirstChild.toString).toList
lista match{
case Nil=>None
case l # List(duns,companyname,address,city,postal,_bs,orgnummer, _*) =>Some(l)
case _ =>println("WTF");None
}
}
}
and I want to use it in a list comprehension like:
val toReturn = for{
rad<-rader
val radExtractor(duns,companyname,address,city,postal,_,orgnummer,_*)=rad
} yield Something(duns,companyname,address,city,postal,orgnummer)
But when a "rad" in "rader" fails because the extractor returns None I get a MatchError.
Isn't the extractor for comprehension supposed to handle/ignore the None cases or did I just miss something?
I could do
val toReturn = rader.collect{case radExtractor(duns,companyname,address,city,postal,_,orgnummer, _*)=>
Something(companyname=companyname,address=address,city=city,postalcode=postal,orgnummer=orgnummer,duns=duns.toInt)
}
But that would not be as sexy ;)
Thank you
Because you are performing the pattern match in an assignment to a val:
val radExtractor(duns,companyname,address,city,postal,_,orgnummer,_*)=rad
... the match must succeed, or you will encounter an error. The above syntax is valid outside a for-comprehension and Scala does not provide any special behaviour for non-matching cases.
To filter out non-matching values in a for-comprehension, use the pattern directly to the left of the <-:
val toReturn = for {
radExtractor(duns,companyname,address,city,postal,_,orgnummer,_*) <- rader
} yield Something(duns,companyname,address,city,postal,orgnummer)
Related
How to conveniently convert Seq[Try[Option[String, Any]]] into Try[Option[Map[String, Any]]].
If any Try before convert throws an exception, the converted Try should throw as well.
Assuming that the input type has a tuple inside the Option then this should give you the result you want:
val in: Seq[Try[Option[(String, Any)]]] = ???
val out: Try[Option[Map[String,Any]]] = Try(Some(in.flatMap(_.get).toMap))
If any of the Trys is Failure then the outer Try will catch the exception raised by the get and return Failure
The Some is there to give the correct return type
The get extracts the Option from the Try (or raises an exception)
Using flatMap rather than map removes the Option wrapper, keeping all Some values and discaring None values, giving Seq[(String, Any)]
The toMap call converts the Seq to a Map
Here is something that's not very clean but may help get you started. It assumes Option[(String,Any)], returns the first Failure if there are any in the input Seq and just drops None elements.
foo.scala
package foo
import scala.util.{Try,Success,Failure}
object foo {
val x0 = Seq[Try[Option[(String, Any)]]]()
val x1 = Seq[Try[Option[(String, Any)]]](Success(Some(("A",1))), Success(None))
val x2 = Seq[Try[Option[(String, Any)]]](Success(Some(("A",1))), Success(Some(("B","two"))))
val x3 = Seq[Try[Option[(String, Any)]]](Success(Some(("A",1))), Success(Some(("B","two"))), Failure(new Exception("bad")))
def f(x: Seq[Try[Option[(String, Any)]]]) =
x.find( _.isFailure ).getOrElse( Success(Some(x.map( _.get ).filterNot( _.isEmpty ).map( _.get ).toMap)) )
}
Example session
bash-3.2$ scalac foo.scala
bash-3.2$ scala -classpath .
Welcome to Scala 2.13.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_66).
Type in expressions for evaluation. Or try :help.
scala> import foo.foo._
import foo.foo._
scala> f(x0)
res0: scala.util.Try[Option[Equals]] = Success(Some(Map()))
scala> f(x1)
res1: scala.util.Try[Option[Equals]] = Success(Some(Map(A -> 1)))
scala> f(x2)
res2: scala.util.Try[Option[Equals]] = Success(Some(Map(A -> 1, B -> two)))
scala> f(x3)
res3: scala.util.Try[Option[Equals]] = Failure(java.lang.Exception: bad)
scala> :quit
If you're willing to use a functional support library like Cats then there are two tricks that can help this along:
Many things like List and Try are traversable, which means that (if Cats's implicits are in scope) they have a sequence method that can swap two types, for example converting List[Try[T]] to Try[List[T]] (failing if any of the items in the list are failure).
Almost all of the container types support a map method that can operate on the contents of a container, so if you have a function from A to B then map can convert a Try[A] to a Try[B]. (In Cats language they are functors but the container-like types in the standard library generally have map already.)
Cats doesn't directly support Seq, so this answer is mostly in terms of List instead.
Given that type signature, you can iteratively sequence the item you have to in effect push the list type down one level in the type chain, then map over that container to work on its contents. That can look like:
import cats.implicits._
import scala.util._
def convert(listTryOptionPair: List[Try[Option[(String, Any)]]]): Try[
Option[Map[String, Any]]
] = {
val tryListOptionPair = listTryOptionPair.sequence
tryListOptionPair.map { listOptionPair =>
val optionListPair = listOptionPair.sequence
optionListPair.map { listPair =>
Map.from(listPair)
}
}
}
https://scastie.scala-lang.org/xbQ8ZbkoRSCXGDJX0PgJAQ has a slightly more complete example.
One way to approach this is by using a foldLeft:
// Let's say this is the object you're trying to convert
val seq: Seq[Try[Option[(String, Any)]]] = ???
seq.foldLeft(Try(Option(Map.empty[String, Any]))) {
case (acc, e) =>
for {
accOption <- acc
elemOption <- e
} yield elemOption match {
case Some(value) => accOption.map(_ + value)
case None => accOption
}
}
You start off with en empty Map. You then use a for comprehension to go through the current map and element and finally you add a new tuple in the map if present.
The following solutions is based on this answer to the point that almost makes the question a duplicate.
Method 1: Using recursion
def trySeqToMap1[X,Y](trySeq : Seq[Try[Option[(X, Y)]]]) : Try[Option[Map[X,Y]]] = {
def helper(it : Iterator[Try[Option[(X,Y)]]], m : Map[X,Y] = Map()) : Try[Option[Map[X,Y]]] = {
if(it.hasNext) {
val x = it.next()
if(x.isFailure)
Failure(x.failed.get)
else if(x.get.isDefined)
helper(it, m + (x.get.get._1-> x.get.get._2))
else
helper(it, m)
} else Success(Some(m))
}
helper(trySeq.iterator)
}
Method 2: directly pattern matching in case you are able to get a stream or a List instead:
def trySeqToMap2[X,Y](trySeq : LazyList[Try[Option[(X, Y)]]], m : Map[X,Y]= Map.empty[X,Y]) : Try[Option[Map[X,Y]]] =
trySeq match {
case Success(Some(h)) #:: tail => trySeqToMap2(tail, m + (h._1 -> h._2))
case Success(None) #:: tail => tail => trySeqToMap2(tail, m)
case Failure(f) #:: _ => Failure(f)
case _ => Success(Some(m))
}
note: this answer was previously using different method signatures. It has been updated to conform to the signature given in the question.
When I am coding with options I find the fold method very useful. Instead of writing if defined statements I can do
opt.fold(<not_defined>){ defined => }
this is good. but what to do if we are working with multiple options. or multiple eithers. Now I have to resort to writing code like
if (x.isDefined && y.isRight) {
val z = getSomething(x.get)
if (z.isDefined) {
....
Depending on the number of things involved, this code becomes very nested.
is there a functional trick to make this code a little un-nested and concise.... like the fold operation above?
have you tried for comprehension? Assuming you don't want to treat individual errors or empty optionals:
import scala.util._
val opt1 = Some("opt1")
val either2: Either[Error, String] = Right("either2")
val try3: Try[String] = Success("try3")
for {
v1 <- opt1
v2 <- either2.right.toOption
v3 <- try3.toOption
} yield {
println(s"$v1 $v2 $v3")
}
Note that Either is not right biased, so you need to call the .right method on the for comprehension (I think cats or scalaz have a right biased Either). Also, we are converting the Either and the Try to optionals, discarding errors
Cases when .isDefined is followed by .get call can be refactored using custom extractors for pattern matching:
def getSomething(s: String): Option[String] = if (s.isEmpty) None else Some(s.toUpperCase)
object MyExtractor {
def unapply(t: (Option[String], Either[Int, String])): Option[String] =
t match {
case (Some(x), Right(y)) => getSomething(x)
case _ => None
}
}
val x: Option[String] = Some("hello world")
val y: Either[Int, String] = Right("ok")
(x, y) match {
case MyExtractor(z) => z // let's do something with z
case _ => "world"
}
// HELLO WORLD
We managed to get rid of all .isDefined, .get and even .right calls by replacing them by explicit pattern matching thanks to our custom extractor MyExtractor.
Trying to emit a for yield block from a blackbox macro, but I'm failing to understand how you can create the block with valid syntax.
So below source is a hardcoded param name as this block is later inserted inside a method that will have the matching param name. params is just params: Seq[c.universe.ValDef], enclosing the case class fields.
def extract(source: Source): Option[CaseClass] = { ... }
val extractors = accessors(c)(params) map {
case (nm, tpe) => {
val newTerm = TermName(nm.toString + "Opt")
q"""$newTerm <- DoStuff[$tpe].apply("$nm", source)"""
}
}
val extractorNames = accessors(c)(params) map {
case (nm, tpe) => TermName(nm.toString + "Opt")
}
This is basically taking a case class, and outputting a for yield black to basically recreate the case class from a comprehension.
Every field in the case class of the form name: Type is transformed to a set of extractors that yield the same case class instance back if the for comprehension is successful.
case class Test(id: Int, text: String)
Will be macro transformed to the following, where Extract is just a type class and Extract.apply[T : Extract] is just materialising the context bound with implicitly[Extract[T]]:
for {
idOpt <- Extract[Int].apply("id", source): Option[Int]
textOpt <- Extract[String].apply("text", source): Option[String]
} yield Test(idOpt, textOpt)
The problem comes in having to quote the inner for yield expressions with and output a <- b blocks.
def extract(source: Source): Option[$typeName] = {
for {(..$extractors)} yield $companion.apply(..$extractorNames)
}
The error is ';' expected but '<-' found, which is pretty obvious as a <- b is invalid Scala by itself. What is the correct way to generate and quasiquote the expression block such that the above would work?
Here is a list of all the different kinds of quasiquotes.
There you can see that to express the a <- b syntax you need the fq interpolator.
So that code will probably become:
val extractors = accessors(c)(params) map {
case (nm, tpe) => {
val newTerm = TermName(nm.toString + "Opt")
fq"""$newTerm <- DoStuff[$tpe].apply("$nm", source)"""
}
}
And then with the normal interpolator:
q"for (..$extractors) yield $companion.apply(..$extractorNames)"
I have the following for-yield loop which takes in a boolean and should either yield Some(string) or None, depending on the boolean:
val theBoolean = false
val x: Option[String] =
for {
theArg <- theBoolean
} yield {
if (theArg) {
"abc"
} else {
None
}
}
This works great if theBoolean is actually a Boolean like false. However if I wanted to pass in an Option[Boolean]:
val theBoolean = Some(false)
it seems like Scala automatically applies a Some() wrapper to the None return - I get a complaint that "Expression of type Option[Serializable] doesn't conform to expected type Option[String]" (with None being the Serializable). The yield is perfectly happy with the same string return though (it doesn't become an Option[Option[String]]
How would I return a None in this case?
A for-comprehension is just syntactic sugar for a series of flatMap, map and filter.
Let's desugar your code then:
val theBoolean = Some(false)
val x = theBoolean.map { theArg =>
if (theArg) {
"abc"
} else {
None
}
}
As you can see, you're just mapping over the value of the Option, so you'll either return Some(abc), Some(None) or None (in case theBoolean is already None).
The lowest common type of None and "abc" is java.Serializable, so that's why the type of x is inferred as Option[Serializable], which as meaningless as Option[Any].
Possible solutions are:
using a flatMap
theBoolean.flatMap(theArg => if (theArg) Some("abc") else None)
or even shorter
theBoolean.flatMap(if (_) Some("abc") else None)
filtering and mapping
theBoolean.withFilter(identity).map(_ => "abc")
Where I used identity since you're testing the value itself.
Clearly you can always leverage the syntactic sugar provided by a for-comprehension, although it doesn't really make a difference in this case
for {
theArg <- theBoolean
if theArg
} yield "abc"
You are right, everything within the yield block is wrapped into an Option (that's how for-comprehensions for options work). In general, for-comprehensions describe what should be done with the content that can be found inside the monad on which for-comprehension is invoked, but the end result (for the world outside of the yield block) is still a monad of the same type (e.g. Option, Try or List).
On yet more general note: there are a lot of descriptions about what a monad is. You can assume that a monad is that infamous Schrödinger box and you are pondering what could happen with that cat hidden there, but all this still remains a possibility because the box isn't open yet.
A possible way to do what you want:
val theBoolean = false
val x: Option[String] =
for {
theArg <- theBoolean if theArg
} yield {
"abc"
}
Instead of a for comprehension it sounds like you want a flatMap instead of a for comprehension.
scala> Some(false).flatMap(if (_) Some("abc") else None)
res4: Option[String] = None
LINQ-style queries in Scala with json4s look as follows:
val jvalue = parse(text) // (1)
val jobject = for(JObject(o) <- jvalue) yield o // (2)
I do not understand exactly how (2) works. How would you de-sugar this for-statement ?
for-comprehensions of the form
for(v <- generator) yield expr
are translated into
generator.map(v => expr)
When you have a pattern match on the left, then any input values which do not match the pattern are filtered out. This means a partial function is created containing the match, and each input argument can be tested with isDefinedAt e.g.
val f: PartialFunction[JValue, JObject] = { case o#JObject(_) => o }
f.isDefinedAt(JObject(List[JField]())) //true
f.isDefinedAt(JNull) //false
This means your example will be translated into something like:
PartialFunction[JValue, List[JField]] mfun = { case JObject(o) -> o }
var jobject = jvalue.filter(mfun.isDefinedAt(_)).map(mfun)