I have the following in my Scala
import scalaz.effect.IO
val i: Iterator[IO[List[String]]] = null
val ii: Iterator[IO[List[String]]] = for{ //This does not compile
io <- i;
lst <- io
} yield lst
Why? What's wrong?
I expected the ii is completely the same as i. But it refuses to compile:
Error:(12, 11) type mismatch;
found : scalaz.effect.IO[List[String]]
required: scala.collection.GenTraversableOnce[scalaz.effect.IO[List[String]]]
lst <- io
flatMap in scala
Recall that for-comprehensions are just sugared calls to flatMap:
for {
a <- expr //expr must return M[A] such that M has a
//flatMap method: flatMap[B](f: A => N[B]) for some N
b <- f(a) //f(a) must be an N[B]
...
Your question
Here is the signature of Iterator.flatMap
def flatMap[B](f: A => GenTraversableOnce[B]): Iterator[B]
But you were attempting to supply a function returning an IO[B]:
lst <- io //You cannot just have `io` on the right here
hence the compilation error.
flatMap in scala again
Scala's flatMap <~> for-comprehension conversions as they apply to collection types (and Option) are (in my opinion) confusing as they allow you to switch between different types of Monad (e.g. List/Option/Set etc). For example, what is the type of x here?
val x =
for {
i <- List(1, 2, 3)
j <- Option(i + 1)
k <- Stream(i, j)
} yield k
Monads in scalaz
Having a closer look at scalaz.Monad's flatMap:
trait Monad[M[_]] {
def flatMap[A, B](ma: M[A])(f: A => M[B]): M[B]
}
The type of M is always fixed. That is, in this comprehension:
lazy val ma: M[A] = ???
for (a <- ma; b <- f(a)) yield b
The result type of the function f must be in M[B] for some B. Whilst sometimes this can be a little irritating, it has the advantage of being completely predictable. You never get confused about what monad your for comprehension is in.
Back to the question
It is not obvious what you want, but here are a few suggestions:
i.toStream.sequence.map(flatten) //IO[Stream[String]]
i and io must be of the same monad:
io <- i // i is an Iterator[...]
lst <- io // io is an IO[List[String]]
When using IO and for-comprehensions, one option is to use .unsafeToFuture in your for comprehension. In your example, this would be:
val ii: Iterator[IO[List[String]]] = for{
io <- i;
lst <- io.unsafeToFuture()
} yield lst
Related
Can someone please elaborate what it means when the types have to align with scala for comprehensions?
for {
..
..
}
If the calls all return Futures then it will be fine?
Just trying to understand when it works and when it doesn't work.
For-comprehension desugars to map/flatMap calls so consider their signatures. For example, consider Option#flatMap
def flatMap[B](f: (A) => Option[B]): Option[B]
We see Options continue to be involved. Now observe since
for {
a <- Some(41)
b <- Some(1)
} yield a + b
becomes
Some(41).flatMap(a => Some(1).map(b => a + b))
that means if you tried to mix monadic types inside for-comprehension such as
for {
a <- Some(41)
b <- Try(1)
} yield a + b
then it would desugar to
Some(41).flatMap(a => Try(1).map(b => a + b))
|
types do not align
but we already seen that Option#flatMap expects A => Option[B] not A => Try[B].
One place where it seems as if you can break that rule is when mixing Option with List
scala> for {
| a <- List(41)
| b <- Some(1)
| } yield (a + b)
val res0: List[Int] = List(42)
but this works because List#flatMap takes a function from A to IterableOnce and Option has been made IterableOnce in Scala 2.13
def flatMap[B](f: A => IterableOnce[B])
Note the other way around will not work though:
scala> for {
| a <- Some(41)
| b <- List(1)
| } yield a + b
b <- List(1)
^
On line 3: error: type mismatch;
found : List[Int]
required: Option[?]
In general given an effectful type F[A] then inside for-comprehension F cannot vary unless we are using a subtype, on the other hand A can indeed vary.
I've the written the following Haskell function which accepts two monadic values and combine them into a single monadic value (it's just to illustrate the degree of genericity (or generic-ness) that Haskell type-system could support).
combine x y = do
a <- x
b <- y
return (a, b)
and I tested it with three different monads:
main = do
putStrLn $ show $ combine (Just 10) (Just 20) -- Maybe a
putStrLn $ show $ combine [100] [10, 20] -- [] a
a <- combine getLine getLine -- IO a
putStrLn $ show a
And it works great as expected. Now, I want to know if Scala's type-system could allow me to write the above function without compromising the genericity. But I don't know Scala enough (though I wish to explore it). So could anyone help me convert this code into Scala?
I think this is the equivalent:
import cats._
import cats.implicits._
def combine[T, F[_]: Monad](fa: F[T], fb: F[T]) = for {
a <- fa
b <- fb
} yield (a, b)
Where Monad is from a library (cats or scalaz).
combine(Option(10), Option(20)) produces Some((10,20)) and combine(List(100), List(10, 20)) produces List((100,10), (100,20)).
EDIT: The above version is over-constrained, since it requires the two argument types to be the same. def combine[A, B, F[_]: Monad](fa: F[A], fb: F[B]) fixes that.
Your combine function is equivalent to the Scala code
for { a <- x; b <- y } yield (a,b)
So you might try defining a function:
def combine[M[_],A,B](x: M[A], y: M[B]): M[(A,B)] =
for { a <- x; b <- y } yield (a,b)
And the compiler will complain that flatMap is not a member of M[A] and map is not a member of M[B].
The thing with for is that it is a bit of compiler magic that will accept any type that implements functions called map, flatMap, and withFilter. This is in contrast to Haskell in which we can add (or let the compiler infer) a Monad constraint to let do notation work.
To expand on the answer #JoePallas gave, it is possible to make this work. In fact, the following implementation is essentially how GHC implements typeclasses. The cats and scalaz libraries provide all this stuff for you, but this is how the sausage is made:
First define the interface we need:
trait For[M[_]] {
def map[A,B](ma: M[A], f: A => B): M[B]
def flatMap[A,B](ma: M[A],f: A => M[B]): M[B]
def withFilter[A](ma: M[A],q: A => Boolean): M[A]
}
(I'm using the name For and using a slightly different interface than Monad.)
Then we provide an implicit implementation of this trait for every data type we want to support. Here's an example for Option:
implicit val optionFor = new For[Option] {
def map[A,B](ma: Option[A], f: A => B): Option[B] = ma.map(f)
def flatMap[A,B](ma: Option[A],f: A => Option[B]): Option[B] = ma.flatMap(f)
def withFilter[A](ma: Option[A],q: A => Boolean): Option[A] = ma.withFilter(q).map(a => a)
}
Then we provide an implicit conversion to a type that can apply these operations:
implicit class ForOps[M[_], A](val ma: M[A]) extends AnyVal {
def map[B](f: A => B)(implicit m: For[M]): M[B] = m.map(ma,f)
def flatMap[B](f: A => M[B])(implicit m: For[M]): M[B] = m.flatMap(ma, f)
def withFilter(q: A => Boolean)(implicit m: For[M]): M[A] = m.withFilter(ma,q)
}
And finally, we can define combine:
def combine[M[_]: For, A, B](ma: M[A], mb: M[B]): M[(A, B)] =
for { a <- ma; b <- mb } yield (a, b)
The syntax
def f[T: TC] = ???
is sugar for
def f[T](implicit unutterableName: TC[T]) = ???
An implicit argument list, if not given explicitly at the call site, will be automatically filled in by searching for values/functions with the correct types, as long as those are themselves implicit. In this case, we look for a proof that M is a monad. In the body, this value is implicit, and it has no name to access it. Implicit search can still find it. ForOps allows the 3 for operations to automagically appear on the values by using that Monad.
This is really an explicit version of how GHC implements typeclasses. In the simplest case of no optimization:
class Applicative m => Monad m where
return :: a -> m a
(>>=) :: m a -> (a -> m b) -> m b
compiles to
data Monad m = Monad {
monadSubApplicative :: Applicative m
return :: forall a. a -> m a
(>>=) :: forall a. m a -> (a -> m b) -> m b
}
and
instance Monad [] where
return = _
(>>=) = _
becomes
monadList :: Monad []
monadList = Monad {
monadSubApplicative = applicativeList
, return = _
, (>>=) = _
}
You will often hear the word "dictionary" be used to describe the underlying data type and values. And combine is
combine :: Monad m -> m a -> m b -> m (a, b)
combine (Monad _ return (>>=)) ma mb = ma >>= \a -> mb >>= \b -> return (a, b)
However, GHC applies a bunch of restrictions to the system that makes it more predictable and performs more optimization. Scala sacrifices this to allow the programmer to perform more interesting acrobatics.
For good measure, an instance like this:
newtype Compose f g a = Compose { unCompose :: f (g a) }
instance (Functor f, Functor g) => Functor (Compose f g) where
fmap f (Compose fga) = Compose $ fmap (fmap f) fga
would be done like this in Scala (using an implicit def, not a val):
trait Functor[F[_]] { def map[A, B](fa: F[A])(f: A => B): F[B] }
final case class Compose[F[_], G[_], A](val get: F[G[A]]) extends AnyVal
object Compose {
// you usually put these implicits in the associated companions
// because implicit search is picky about where it looks
implicit def functor[F[_], G[_]](implicit
functorF: Functor[F],
functorG: Functor[G]
// type lambda: use a type projection on a refinement type
// to create an anonymous type-level function
// it's universally accepted as a horrendous abuse of syntax
// you can use the kind-projector plugin to avoid writing them (directly)
) : Functor[({type L[X] = Compose[F, G, X]})#L]
= new Functor[({type L[X] = Compose[F, G, X]})#L] {
override def map[A, B](cfga: Compose[F, G, A])(f: A => B): Compose[F, G, B] =
Compose(functorF.map(cfga.get) { ga => functorG.map(ga)(f) })
}
}
Making all this stuff explicit is a bit ugly, but it works quite well.
In functional programming (and programming in general) it is good practice to use the least powerful abstraction you can find. In the example you gave, you actually don't need the power of a monad. The combine function is liftA2 from the applicative type class. Example:
import Data.Maybe
import Control.Applicative
z= Just 1
y= Just 2
liftA2 (,) z y
> Just (1,2)
In Scala you have something similar. An example from the Scalaz library which uses the same abstraction:
import scalaz._, Scalaz._
(Option(1) |#| Option(2))(Tuple2.apply)
> res3: Option[(Int, Int)] = Some((1, 2))
The reason you don't need the monad abstraction is that the values are independent of each other.
I'm reading about scalaz and noticed that we can make a list of Applicatives to be an Applicative of List.
def sequenceA[F[_]: Applicative, A](list: List[F[A]]): F[List[A]] = list match {
case Nil => (Nil: List[A]).point[F]
case x :: xs => (x |#| sequenceA(xs)) {_ :: _}
}
The question is can we do the opposite? Can we transform F[List[A]] to List[F[A]]?
This is possible if the F in question is Traversable. The code you're showing is specialized for List, but in fact it holds for all Traversable functors.
This means, that for every F that's Traversable and any G that's Applicative, we can go from F[G[A]] to G[F[A]] with sequence.
List also has an Applicative, so we can use it as our G, whereas in your example List was used as the F, the Traversable functor.
An example where what you're asking works could be Option. Here's the original direction:
val x: List[Option[Int]] = ...
val y: Option[List[Int]] = x.sequence
and the other direction:
val a: Option[List[Int]] = ...
val b: List[Option[Int]] = a.sequence
We could also write another function specializing to List:
def sequenceT[F[_]: Traverse, A](list: F[List[A]]): List[F[A]] = ...
Scalaz State monad's modify has the following signature:
def modify[S](f: S => S): State[S, Unit]
This allows the state to be replaced by state of the same type, which does not work well when the state includes a shapeless value such as a Record whose type changes as new fields are added. In that case what we need is:
def modify[S, T](f: S => T): State[T, Unit]
What is a good way to adapt Scalaz's State monad to use shapeless state so that one can use Records as opposed to, say, the dreaded Map[String, Any]?
Example:
case class S[L <: HList](total: Int, scratch: L)
def contrivedAdd[L <: HList](n: Int): State[S[L], Int] =
for {
a <- init
_ <- modify(s => S(s.total + n, ('latestAddend ->> n) :: s.scratch))
r <- get
} yield r.total
Update:
The full code for Travis's answer is here.
State is a type alias for a more generic type IndexedStateT that's specifically designed to represent functions that change the state type as state computations:
type StateT[F[_], S, A] = IndexedStateT[F, S, S, A]
type State[S, A] = StateT[Id, S, A]
While it's not possible to write your modify[S, T] using State, it is possible with IndexedState (which is another type alias for IndexedStateT that fixes the effect type to Id):
import scalaz._, Scalaz._
def transform[S, T](f: S => T): IndexedState[S, T, Unit] =
IndexedState(s => (f(s), ()))
You can even use this in for-comprehensions (which has always seemed a little odd to me, since the monadic type changes between operations, but it works):
val s = for {
a <- init[Int];
_ <- transform[Int, Double](_.toDouble)
_ <- transform[Double, String](_.toString)
r <- get
} yield r * a
And then:
scala> s(5)
res5: scalaz.Id.Id[(String, String)] = (5.0,5.05.05.05.05.0)
In your case you might write something like this:
import shapeless._, shapeless.labelled.{ FieldType, field }
case class S[L <: HList](total: Int, scratch: L)
def addField[K <: Symbol, A, L <: HList](k: Witness.Aux[K], a: A)(
f: Int => Int
): IndexedState[S[L], S[FieldType[K, A] :: L], Unit] =
IndexedState(s => (S(f(s.total), field[K](a) :: s.scratch), ()))
And then:
def contrivedAdd[L <: HList](n: Int) = for {
a <- init[S[L]]
_ <- addField('latestAdded, n)(_ + n)
r <- get
} yield r.total
(This may not be the best way of factoring out the pieces of the update operation, but it shows how the basic idea works.)
It's also worth noting that if you don't care about representing the state transformation as a state computation, you can just use imap on any old State:
init[S[HNil]].imap(s =>
S(1, field[Witness.`'latestAdded`.T](1) :: s.scratch)
)
This doesn't allow you to use these operations compositionally in the same way, but it may be all you need in some situations.
In Scala, how can I append an Option[A] to Option[A]:
Meaning:
Some("foo") ??? Some("bar") == Some("foobar")
In Haskell I'd use an applicative:
ghci>import Control.Applicative
ghci> (++) <$> (Just "foo") <*> (Just "bar")
Just "foobar"
Are there Applicatives in Scala's standard Typesafe library?
With scalaz you can do it this way:
import scalaz.syntax.applicative._
import scalaz.std.option._
val l: Option[String] = Some("foo")
val r: Option[String] = Some("bar")
val result = ^(l, r)(_ + _)
println(result) // Some("foobar")
Applicatives aren't in the standard library, but you can use for-comprehensions like monads in Haskell:
for {
l <- Some("foo")
r <- Some("bar")
} yield (l + r)
(or rewrite it with flatMap, of course). Otherwise, go with scalaz, as in Eugene's answer.
If you just want to achieve this particular syntax, you can combine an implicit class with a monad for-comprehension:
implicit class RichOption(a: Option[String]) {
def ???(b: Option[String]) = for (sa <- a; sb <- b) yield sa+sb
}
scala> Some("foo") ??? Some("bar") == Some("foobar")
res4: Boolean = true