I am trying to use fold or map operation instead of match for Option.
I have a Option val ao: Option[String] = xxxx and a function f: (String => Future[Option[T]])
If I do pattern matching is:
ao match {
case Some(t) => f(t)
case None => Future.successful(None)
}
if I do map is:
ao map f getOrElse Future.successful(None)
But when I do fold, I got some following compiler errors:
ao.fold(Future.successful(None))(t => f(t))
about complaining expression Future[Option[T]] doesn't confirm to Future[None.type]
So why map works here but fold not, did I miss something here?
The reason for this is that Scala is trying to derive the return type None.type which is kind of like Nil for lists in the sense that only one object (None) exists and is upcast in all situations because it is Option[Nothing]. To get around this, you should explicitly define the types.
Here's the Scala doc for fold :
fold[B](ifEmpty: ⇒ B)(f: (A) ⇒ B): B
Returns the result of applying f to this scala.Option's value if the scala.Option is nonempty. Otherwise, evaluates expression ifEmpty.
The compiler thinks [B] is None.type. so try calling it this way :
ao.fold[Future[Option[T]]](Future.successful(None))(t => f(t))
or calling it with a type ascription :
ao.fold(Future.successful(None: Option[T]))(t => f(t))
Related
I am trying Cats for the first time and am using Scala 3, and I am trying to implement a set of parser combinators for self-pedagogy, however; I am stuck on the definition of the tailRecM function for Monad. I have managed Functor and Applicative just fine.
I have defined my type in question as a function such that:
type Parser[A] = (input: List[Token]) => ParseResult[A]
with corresponding return types as:
type ParseResult[A] = Success[A] | Failure
case class Success[A](value: A, tokens: List[Token])
case class Failure(msg: String, tokens: List[Token])
My current definition of tailRecM is as follows:
#annotation.tailrec
def tailRecM[A, B](init: A)(fn: A => Parser[Either[A, B]]): Parser[B] =
(input: List[Token]) =>
fn(init)(input) match {
case f: Failure => f
case s: Success[Either[A, B]] => s.value match {
case Right(b) => Success(b, s.tokens)
case Left(a) => tailRecM(a)(fn) // won't compile
}
}
If I attempt to build I get "Found: Parsing.Parser[B] Required: Parsing.ParseResult[B]" for tailRecM(a)(fn)
The issue as far as I can tell stems from the fact that my type in question Parser[A] is a function type and not simply a value type? I attempted to ameliorate the issue by modifying the tailRecM recursive call to tailRecM(a)(fn)(input) but then this is obviously not stack safe, and also will not compile.
How can I resolve this issue, and more broadly, how can I implement the Monad typeclass for function types in general?
It's not possible to make tailRecM itself tail-recursive; you need to define a tail-recursive helper method
Here's how the cats library implements tailRecM for Function1
def tailRecM[A, B](a: A)(fn: A => T1 => Either[A, B]): T1 => B =
(t: T1) => {
#tailrec
def step(thisA: A): B =
fn(thisA)(t) match {
case Right(b) => b
case Left(nextA) => step(nextA)
}
step(a)
}
This is because monadic recursion is a form of mutual tail-recursion, where two methods flip back and forth calling each other. The scala compiler can't optimize that. So instead we inline the definition of the monadic part rather than calling flatMap or another method
You need to pass the input again to the tailRecM call
tailRecM(a)(fn)(input)
because tailRecM(a)(fn) returns a Parser, but you need the ParserResult from that returned Parser, as you already did in all other cases.
For the following method series of transforms the Intellij IDE seems to "understand" what is going on: it does not note any errors/warnings and correctly shows the data types:
val recsWithNames = recsWithNamesAndCnts.map(_._1)
.reduceLeft{ case (dfCum: DataFrame,dfNew: DataFrame) => dfCum.union(dfNew)}
Here is the type inference:
However it does not compile:
Error:(426, 68) missing parameter type for expanded function
The argument types of an anonymous function must be fully known. (SLS 8.5)
Expected type was: (?, org.apache.spark.sql.DataFrame) => ?
val recsWithNames = recsWithNamesAndCnts.map(_._1).reduceLeft{ case (dfCum: DataFrame,dfNew: DataFrame) => dfCum.union(dfNew)}
This has been a bit of a pattern with reduceLeft so I typically end up converting to foldLeft. In particular the following does work:
val dfs = recsWithNamesAndCnts.map(_._1)
val recsWithNames = dfs.tail.foldLeft(dfs.head){ case (dfCum: DataFrame,dfNew: DataFrame) => dfCum.union(dfNew)}
But maybe in this case someone might provide some insight on the particular nuances of reduceLeft leading to this error.
reduceLeft and foldLeft have the following signatures:
def reduceLeft[B >: A](op: (B, A) => B): B
def foldLeft[B](z: B)(op: (B, A) => B): B
Both involve an op that takes a Function2, hence both will work fine without the case match:
(dfCum: DataFrame, dfNew: DataFrame) => dfCum.union(dfNew)
// Or, shorthanded to:
_ union _
On the other hand, case (dfCum, dfNew) => dfCum.union(dfNew) is a Function1 (in particular, a partial function of Tuple2). The compiler is able to interpret it and infer type B in foldLeft, but not in reduceLeft (my guess is due to B >: A). It'll work if you help the compiler a litte:
reduceLeft[DataFrame]{ case (dfCum, dfNew) => dfCum.union(dfNew) }
I took the scala odersky course and thought that the function that Flatmap takes as arguments , takes an element of Monad and returns a monad of different type.
trait M[T] {
def flatMap[U](f: T => M[U]): M[U]
}
On Monad M[T] , the return type of function is also the same Monad , the type parameter U might be different.
However I have seen examples on internet , where the function returns a completely different Monad. I was under impression that return type of function should the same Monad. Can someone simplify the below to explain how flapmap results in the actual value instead of Option in the list.
Is the List not a Monad in Scala.
val l= List(1,2,3,4,5)
def f(x:int) = if (x>2) Some(x) else None
l.map(x=>f(x))
//Result List[Option[Int]] = List( None , None , Some(3) , Some(4) , Some(5))
l.flatMap(x=>f(x))
//Result: List(3,4,5)
Let's start from the fact that M[T]is not a monad by itself. It's a type constructor. It becomes a monad when it's associated with two operators: bind and return (or unit). There are also monad laws these operators must satisfy, but let's omit them for brevity. In Haskell the type of bind is:
class Monad m where
...
(>>=) :: m a -> (a -> m b) -> m b
where m is a type constructor. Since Scala is OO language bind will look like (first argument is self):
trait M[T] {
def bind[U](f: T => M[U]): M[U]
}
Here M === m, T === a, U === b. bind is often called flatMap. In a pure spherical world in a vacuum that would be a signature of flatMap in OO language. Scala is a very practical language, so the real signature of flatMap for List is:
final def flatMap[B, That](f: (A) ⇒ GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
It's not bind, but will work as a monadic bind if you provide f in the form of (A) => List[B] and also make sure that That is List[B]. On the other hand Scala is not going to watch your back if you provide something different, but will try to find some meaningful conversion (e.g. CanBuildFrom or something else) if it exists.
UPDATE
You can play with scalac flags (-Xlog-implicits, -Xlog-implicit-conversions) to see what's happening:
scala> List(1).flatMap { x => Some(x) }
<console>:1: inferred view from Some[Int] to scala.collection.GenTraversableOnce[?] via scala.this.Option.option2Iterable[Int]: (xo: Option[Int])Iterable[Int]
List(1).flatMap { x => Some(x) }
^
res1: List[Int] = List(1)
Hmm, perhaps confusingly, the signature you gave is not actually correct, since it's really (in simplified form):
def flatMap[B](f: (A) ⇒ GenTraversableOnce[B]): Traversable[B]
Since the compiler is open-source, you can actually see what it's doing (with its full signature):
def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit bf: CanBuildFrom[Repr, B, That]): That = {
def builder = bf(repr) // extracted to keep method size under 35 bytes, so that it can be JIT-inlined
val b = builder
for (x <- this) b ++= f(x).seq
b.result
}
So you can see that there is actually no requirement that the return type of f be the same as the return type of flatMap.
The flatmap found in the standard library is a much more general and flexible method than the monadic bind method like the flatMap from Odersky's example.
For example, the full signature of flatmap on List is
def flatMap[B, That](f: (A) ⇒ GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
Instead of requiring the function passed into flatmap to return a List, it is able to return any GenTraversableOnce object, a very generic type.
flatmap then uses the implicit CanBuildFrom mechanism to determine the appropriate type to return.
So when you use flatmap with a function that returns a List, it is a monadic bind operation, but it lets you use other types as well.
I have the following method that I would like to apply fold operation on:
def rec(id: String, elems: Seq[(String, MyCase)]) = {
elems.fold(Seq.empty[(String, Seq[String])] { elem =>
....
}
}
What I do not get is the type of the elem is Nothing and I do not understand why it should be! Any clues?
You are missing the closing parentheses before the {, that's why your IDE, probably, thinks, the type is Nothing.
Also, you are, probably, looking for foldLeft, not fold (the first parameter of the latter must match the type of elements of the sequence).
Now the (simplified) signature of .foldLeft on Seq[A] is:
foldLeft[B](b: B)(f: (B,A) => B)
As you can see, it takes a function, that transforms a Tuple2 into the type of the first parameter. The first element of the tuple has the same type as the first param, the second element is the same type as the elements of the sequence.
In your example, B is Seq[(String, Seq[String])], and the sequence elements are (String, MyCase). The type of input to the function would therefore take a horribly looking tuple like this:
(Seq[(String, Seq[String])], (String, MyCase))
This is caused by you are not only want to fold, you also you want to map the tuple, see the fold method signature:
def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1 = foldLeft(z)(op)
The input type and outtype must be same: A1,
so if want to map, you maybe want to try foldLeft:
def foldLeft[B](z: B)(op: (B, A) => B): B =
There is a generics for output type B without bounding to A.
also we can find the fold source code is calling:
def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1 = foldLeft(z)(op)
use foldLeft maybe like:
elements.fold(Seq.empty[(String, Seq[String])])((a, b) => a ++ Seq((b._1, Seq[String]())))
2nd EDIT:
I'm an idiot. A fold must return the same value as the input type.
Replace all the below folds with foldLefts for example and it works.
You can't transform the data type in a standard fold.
It was compiling for me but I didn't notice the return type was useless.
The second parameter in a fold is a function:
(ResultType, SingleElement) => Result Type
In this case
(Seq[(String, Seq[String])], Seq[(String, MyCase)]) => Seq[(String, Seq[String])]
Your code only has one input on the second parameter so the compiler doesn't know what it is. So it should look something like:
elems.foldLeft(Seq.empty[(String, Seq[MyCase])] {(zeroSeq, nextElem) =>
//zeroSeq: Seq.empty[(String, Seq[MyCase]
//nextElem: (String, MyCase)
}
EDIT:
The following for example compiles:
case class MyCase(x: String)
val elems: Seq[(String, MyCase)] = Seq(("Hi", MyCase("B")))
elems.foldLeft(Seq.empty[(String, Seq[MyCase])]){(z, s) =>
z
}
While dealing with Option in Scala what are the things I should be considering to decide whether to map or patten match? For example, if I have Option[MyClass], I can deal with it the following ways:
def getList(myOptionInstance: Option[MyClass]): List[String] =
myOptionInstance map (...) getOrElse(List.empty[String])
or
def getList(myOptionInstance: Option[MyClass]): List[String] = myOptionInstance match {
case Some(mySomeInstance) => .....
case None => List.empty[String]
}
When will I choose one over the other?
I second #rarry: fold is the preferred way to deal with this.
Some prefer pattern matching because it's "cool" (whatever it means) and sometimes easier to read.
I try to avoid using getOrElse because it does not force you to use the same type for the default value as the type wrapped in your Option:
def getOrElse[B >: A](default: ⇒ B): B
So you can write:
val v = Some(42).getOrElse("FortyTwo")
Here v has type Any. It's very easy to see the problem with such a stupid example but sometimes it's not as obvious and can lead to issues.
While fold:
def fold[B](ifEmpty: ⇒ B)(f: (A) ⇒ B): B
It forces you to return the same type for both branches.
scala> Some(42).fold("fortyTwo")(v => v)
<console>:8: error: type mismatch;
found : Int
required: String
Some(42).fold("fortyTwo")(v => v)
Pattern matching is :
slightly more efficient
not anal about subtypes (in this case #rarry had to add a type hint)
easier to read
endorsed by Martin Oderksy: https://stackoverflow.com/a/5332657/578101
I would go for this:
myOptionInstance.fold(Nil: List[String])(...)