List[OptionT[Future, Int]] to OptionT[Future, List[A]] - scala

I'm building a List of Int using an an async computation for retrieving element:
(1 to n).map(anAsyncThingy).toList
where anAsyncThingy returns OptionT[Future, Int]
The result is hence of type List[OptionT[Future, Int]]
What I'd like to get now is an OptionT[Future, List[A]]
Here's my best attempt so far (I'll add some stubs, so that it can run in a REPL)
import scalaz._; import Scalaz._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
def anAsyncThingy(x: Int): OptionT[Future, Int] = x.point[Future].liftM[OptionT]
val res = OptionT {
Future.sequence {
(1 to 3).map(anAsyncThingy(_).run).toList
}.map(_.sequence)
}
res.map(println) // List(1, 2, 3)
The above works as expected, but I feel there's a lot of room for improvement using the proper scalaz constructs, instead of jumping in and out from monad transfomers.
How can I achieve the same result a in a more straightforward way?

I found the answer myself, after some experimenting:
val res = (1 to 3).map(anAsyncThingy).toList.sequenceU
res.map(println) // List(1, 2, 3)
Yay for scalaz!
By the way, sequenceU is needed instead of sequence because scala isn't smart enough to figure out when you have
OptionT[M, A]
if you fix a type parameter (M to Future for instance)
OptionT[Future, A]
it has the shape of M[_]
The compiler keeps believing it has a M[_, _] shape, unless spoonfed (with a nasty type lambda)
Here's where scalaz's sequenceU kicks in and works around this issue using Unapply. More on the subject here.
Update
As per phadej's comment, sequence ∘ map ≡ traverse, so this can be made even more concise with traverseU:
(1 to 3).toList.traverseU(anAsyncThingy)
The same idea of sequence vs sequenceU applies to traverse vs traverseU, of course.

Related

Can cats.effect.IO be sequenced, ie is it in the Traverse typeclass?

I like to sequence collections of effects eg to turn a List[IO[Int]] into a IO[List[Int]] like so :-
scala> import cats._, cats.data._, cats.implicits._, cats.effect._
scala> val efs : List[IO[Int]] = List(IO(1),IO(2))
efs: List[cats.effect.IO[Int]] = List(IO$647684131, IO$2021068036)
scala> var efOfInts = efs.sequence
efOfInts: cats.effect.IO[List[Int]] = <function1>
scala> efOfInts.unsafeRunSync
res2: List[Int] = List(1, 2)
But is it possible to go the other way? eg to turn a IO[List[Int]] into a List[IO[Int]] ?
I cant seem to find the implicits to add sequence onto IO so now I'm wondering if it is not possible to Traverse or Sequence IO?
Has anyone done this or know why it is not allowed?
Thanks.
No, it is not allowed.
The whole idea of IO is that you cannot get values out of it without unsafeRunSync. There is no "peeking inside"
Since Traversable also means Foldable, let's look at why something like foldLeft on IO breaks the looking inside rule.
def foldLeft[A, B](fa: IO[A], b: B)(f: (B, A) => B): B
So for example if we had an x: IO[Int] (which produces 2)
foldLeft(x, 0)(_ + _), that would have to return us a 2. So we have peeked inside. foldLeft would have to unsafeRunSync, which breaks referential transparency.
So for this (and other similar) reasons, you cannot have Traverse on IO.

How to find out what happens when I use Monoid for Map in scalaz

How can I find all instances of Monoid. For example, how to know if there is a Monoid instance for Map in scalaz ?
And if yes, where it is in the source code.
I've tried the following without success
# implicitly[Monoid[Map[_, _]]]
Main.scala:1146: could not find implicit value for parameter e: scalaz.Monoid[Map[_, _]]
implicitly[Monoid[Map[_, _]]]
^
Compilation Failed
How can i see what happens (implicit conversions, …) when I execute code from the REPL, like
Map("a", 1) |+| Map("a", 1)
There is no way to find all the instances of a type class.
Specifically for Map it depends on the type of the values, because a Map[K, V] monoid instance needs a Semigroup[V] instance.
You can find the code for Map's Monoid in scalaz.std.map.
You can see the implicit conversions using reflection :
import scalaz.std.map._
import scalaz.std.anyVal._
import scalaz.syntax.semigroup._
import scala.reflect.runtime.universe._
showCode(reify { Map("a" -> 1) |+| Map("a" -> 1) }.tree)
// `package`.monoid.ToSemigroupOps(
// Predef.Map.apply(Predef.ArrowAssoc("a").->(1)))
// (map.mapMonoid(Predef.this.DummyImplicit.dummyImplicit, anyVal.intInstance))
// .|+|(Predef.Map.apply(Predef.ArrowAssoc("a").->(1)))
The Scalaz implicits at work are :
The syntax conversion ToSemigroupOps to add the |+| operation to Map.
The Monoid instance for Map[String, Int] which uses the Semigroup[Int] instance.

Better Way to Handle Nested Monad?

Given a list of names:
scala> import scala.concurrent.Future
import scala.concurrent.Future
scala> import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.ExecutionContext.Implicits.global
scala> val names: Future[List[String]] =
Future.successful( List("Joe Foo", "Jane Bar") )
names: scala.concurrent.Future[List[String]] =
scala.concurrent.impl.Promise$KeptPromise#3dddbe65
And, a method that, for a given name, returns a Future[String] for that name's hobby:
scala> def getHobby(name: String): Future[String] = Future.successful( "poker" )
getHobby: (name: String)scala.concurrent.Future[String]
With names, i.e. Future[List[String]], I can get those names' hobby via:
scala> names.map(_.map(getHobby))
res3: scala.concurrent.Future[List[scala.concurrent.Future[String]]] =
scala.concurrent.impl.Promise$DefaultPromise#42c28305
But, then I have a somewhat hard-to-read, nested monad, Future[List[Future[String]].
I can clean it up:
scala> res3.map(Future.sequence(_))
res5: scala.concurrent.Future[scala.concurrent.Future[List[String]]] =
scala.concurrent.impl.Promise$DefaultPromise#4eaa375c
scala> res5.flatMap(identity)
res6: scala.concurrent.Future[List[String]] =
scala.concurrent.impl.Promise$DefaultPromise#101bdd1c
And then gets it value.
scala> res6.value
res7: Option[scala.util.Try[List[String]]] = Some(Success(List(poker, poker)))
But, is there a cleaner, more idiomatic way to perform the above work?
One approach is to flatten the data as it's retrieved.
scala> names.flatMap(x => Future.sequence(x.map(getHobby)))
res54: scala.concurrent.Future[List[String]] = scala.concurrent.impl.Promise$DefaultPromise#45cd45aa
Just for illustration (of nested monad), you also have "Scalaz Monad Transformers", described by Rama Nallamilli.
Take Future for example, all these cases below are monads nested within a monad.
Future[Option[T]]
Future[List[T]]
Future[scalaz.\/[A, B]]
In the case of a nested monad such as Future[List[Int]], Rama suggests:
When choosing which monad transformer to use, you always choose the inner most type, in this case List[Int] is our inner most type so we will use the (Scalaz) ListT monad transformer.
The ListT apply function is as follows:
def apply[A](a: M[List[A]]) = new ListT[M, A](a)
Therefore we can use this to convert our Future[List[Int]] to the ListT monad type which in this case will be a ListT[Future, Int].
We can now write our addition in terms of the new monad type which has abstracted the mapping of the Future:
# for {
i <- ListT(x)
j <- ListT(y)
} yield i + j
res20: ListT[Future, Int] = ListT(Success(List(5, 6, 7, 6, 7, 8, 7, 8, 9)))
In summary:
monad transformers give you a powerful abstraction to work on the underlying data of a monadic type when it itself is wrapped in a monad.
It reduces code complexity and enhances readability by abstracting the wiring of drilling down into the nested datatypes.
ScalaZ provides implementations of monad transformers for many types including EitherT, ListT, OptionT and ReaderT to name a few.

generic functions that operates on collections in scala

This seems like a simple question that I was sure have been asked before, but couldn't find what I was looking for.
How can one write a function that takes a collection as an argument (or anything that can be treated as a collection), perform some operations on it, and return a collection of the same type?
e.g:
scala> def foo[Repr <% Traversable[String]](repr: Repr) = repr.map(_.size)
foo: [Repr](repr: Repr)(implicit evidence$1: Repr => Traversable[String])Traversable[Int]
this works ok on some collections:
scala> foo(Vector("Hello","World"))
res0: Traversable[Int] = Vector(5, 5)
but surprising when I tried on other collections (e.g. Option):
scala> foo(Some("HelloWorld"))
res1: Traversable[Int] = List(10)
a small problem is the return type Traversable, which ideally would be the type of whatever was given to the method. the bigger problem is the actual implementation type: an Option became a List.
it gets even worse, when tried on classes (that behaves like collections) but have no implicit in scope for them. e.g: Try:
scala> import scala.util._
import scala.util._
scala> foo(Success("HelloWorld"))
<console>:12: error: No implicit view available from scala.util.Success[String] => Traversable[String].
foo(Success("HelloWorld"))
^
So, is there a way, to write a generic function, that when given a "collection like" argument, can operate on it's elements and return the correct type?
ideally, I would like to use it on anything (even Future, and Try) , but for my specific usage, I can do with just real collections & Option.
EDIT:
to illustrate a possible solution, (which forces me to copy&paste code, and so, is not what i'm looking for) is to simply write both functions without view bounds:
scala> :paste
// Entering paste mode (ctrl-D to finish)
def foo[Repr <: Traversable[String]](repr: Repr) = repr.map(_.size)
def foo(repr: Option[String]) = repr.map(_.size)
// Exiting paste mode, now interpreting.
foo: [Repr <: Traversable[String]](repr: Repr)Traversable[Int] <and> (repr: Option[String])Option[Int]
foo: [Repr <: Traversable[String]](repr: Repr)Traversable[Int] <and> (repr: Option[String])Option[Int]
scala> foo(Vector("bar"))
res2: Traversable[Int] = Vector(3)
scala> foo(Some("bar"))
res3: Option[Int] = Some(3)
The concept of mapping is represented by functors. One way to easily provide functor implementations for common classes is to use the scalaz library:
import scala.language.higherKinds
import scalaz.Functor
import scalaz.Scalaz._
def foo[El <: String, Coll[_]](repr: Coll[El])(implicit ev: Functor[Coll]) =
repr.map(_.size)
Now, this just works for List, Vector and Future:
scala> foo(Vector("Hello","World"))
res1: scala.collection.immutable.Vector[Int] = Vector(5, 5)
scala> foo(List("Hello","World"))
res2: List[Int] = List(5, 5)
scala> import scala.concurrent.Future
scala> import scala.concurrent.ExecutionContext.Implicits.global
scala> foo(Future("HelloWorld")) andThen PartialFunction(println(_))
Success(10)
Using it with Some is a bit of a problem, because only Option has a Functor implementation, not Some:
scala> foo(Some("HelloWorld"))
<console>:12: error: could not find implicit value for parameter ev: scalaz.Functor[Some]
foo(Some("HelloWorld"))
^
So you have to provide Option instead of Some to foo:
scala> foo(Some("HelloWorld"): Option[String])
res3: Option[Int] = Some(10)
scala> foo(Option("HelloWorld"))
res4: Option[Int] = Some(10)
scala> foo("HelloWorld".some) // This is from scalaz
res5: Option[Int] = Some(10)
And scalaz doesn't have any typeclass implementations for Try, so if you want to use Functor with Try, you'll have to provide the implementation yourself:
import scala.util.Try
import scalaz.Functor
implicit object TryIsFunctor extends Functor[Try] {
def map[A, B](fa: Try[A])(f: A => B): Try[B] = fa map f
}
Then foo will work with Try, but similar to Option, the argument should have the type Try, instead of Success or Failure:
scala> foo(Try("HelloWorld"))
res9: scala.util.Try[Int] = Success(10)
Also, I believe, there are no Functor implementations in scalaz for more general collection types, like Iterable or Seq.
Out of the common higher-order functions Functor only supports map. So to use flatMap and filter you have to provide different typeclasses instead of Functor. For example, scalaz.Monad supports map and flatMap, and scalaz.MonadPlus supports map, flatMap and filter.
And if you don't want to use scalaz, you'd probably have to make something very similar yourself with typeclasses, to get a good result type instead of Traversable. For example, using CanBuildFrom from the standard library.
I do think Kolmar is right about the general problem, but Scala does support duck-typing, so you can do this:
def foo[T[V]](duck: {def map[U](value: String=>U): T[_]}) ={
duck.map(_.size)
}
foo(Vector("bar")).toVector //> res0: Vector[_$2] = List(3)
foo(Some("bar")) //> res1: Option[_$2] = Some(3)
(toVector just to force the eval of the iterator that otherwise results)

Implicit values of generic types for implicit parameters in scala

I have a method:
def pollAll[T, O](orchestrators :Seq[O], poll :(O)=>Future[T])
(implicit reduce: (T, T) => T) :Future[T] =
(Future sequence orchestrators.map(poll(_))).map(res => res.reduce(reduce))
Now, let's assume that I have a poll which returns Future[Seq[A]]. Is it at all possible to provide a generic implicit value for reduce parameter which concatenates any two sequences with common element type? Something like this:
implicit def reduceSeq[T] = (s1 :Seq[T], s2 :Seq[T]) => s1 ++ s2
The above doesn't work, because implicit methods are used for implicit convertions, and their results are not considered during lookup for implicit parameters.
I don't see anything wrong with your solution either, but scalaz defines types called Semigroup and Monoid which could help you here. It also includes Semigroup definitions for many common types like List and Int. SemiGroup defines an append operation. Monoid extends Semigroup but additionally provides a zero or an initial value. This would be useful if orchestrators could be empty, in which case you would want to foldLeft instead of reduce.
Note: I reversed your type parameters
import scalaz._
import Scalaz._
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
def pollAll[O, T : Semigroup](orchestrators :Seq[O], poll :(O)=>Future[T]) :Future[T] =
(Future sequence orchestrators.map(poll(_)))
.map(res => res.reduce(implicitly[Semigroup[T]].append(_, _)))
or sugared a bit
def pollAll2[O, T : Semigroup](orchestrators :Seq[O], poll :(O)=>Future[T]) :Future[T] =
(Future traverse orchestrators)(poll(_)).map(_.reduce(_ |+| _))
output
scala> pollAll(List(1, 2, 3, 4), (i: Int) => Future.successful(i)).foreach(println)
scala> 10
scala> pollAll2(List(1, 2, 3, 4), (i: Int) => Future.successful(i)).foreach(println)
scala> 10