I use sequenceU to turn inside type out while working with disjunctions in scalaz.
for e.g.
val res = List[\/[Errs,MyType]]
doing
res.sequenceU will give \/[Errs,List[MyType]]
Now if I have a val res2 = List[(\/[Errs,MyType], DefModel)] - List containing tuples of disjunctions; what's the right way to convert
res2 to \/[Errs,List[ (Mype,DefModel)]
As noted in the comments, the most straightforward way to write this is probably just with a traverse and map:
def sequence(xs: List[(\/[Errs, MyType], DefModel)]): \/[Errs, List[(MyType, DefModel)]] =
xs.traverseU { case (m, d) => m.map((_, d)) }
It's worth noting, though, that tuples are themselves traversable, so the following is equivalent:
def sequence(xs: List[(\/[Errs, MyType], DefModel)]): \/[Errs, List[(MyType, DefModel)]] =
xs.traverseU(_.swap.sequenceU.map(_.swap))
Note that this would be even simpler if the disjunction were on the right side of the tuple. If you're willing to make that change, you can also more conveniently take advantage of the fact that Traverse instances compose:
def sequence(xs: List[(DefModel, \/[Errs, MyType])]): \/[Errs, List[(DefModel, MyType)]] =
Traverse[List].compose[(DefModel, ?)].sequenceU(xs)
I'm using kind-projector here but you could also write out the type lambda.
Related
So I've been doing some reading about Monads in scala, and all the syntax relating to their flatMap functions and for comprehensions. Intuitively, I understand why Monads need to use the map part of the flatMap function, as usually when I use a map function its on a container of zero, one or many elements and returns the same container but with the passed in function applied to all the elements of the container. A Monad similarly is a container of zero, one or many elements.
However, what is the purpose of the flatten part of the flatMap? I cannot understand the intuition behind it. To me it seems like extra boilerplate that requires all the functions passed to flatMap to create a container / monad around their return value only to have that container instantly destroyed by the flatten part of the flatMap. I cannot think of a single example where flatMap is used which can't be simplified by being replaced simply by map. For example:
var monad = Option(5)
var monad2 = None
def flatAdder(i:Int) = Option(i + 1)
def adder(i:Int) = i + 1
// What I see in all the examples
monad.flatMap(flatAdder)
// Option[Int] = Some(6)
monad.flatMap(flatAdder).flatMap(flatAdder)
// Option[Int] = Some(7)
monad2.flatMap(flatAdder)
// Option[Int] = None
monad2.flatMap(flatAdder).flatMap(flatAdder)
// Option[Int] = None
// Isn't this a lot easier?
monad.map(adder)
// Option[Int] = Some(6)
monad.map(adder).map(adder)
// Option[Int] = Some(7)
monad2.map(adder)
// Option[Int] = None
monad2.map(adder).map(adder)
// Option[Int] = None
To me, using map by itself seems far more intuitive and simple than flatMap and the flatten part does not seem to add any sort of value. However, in Scala a large emphasis is placed on flatMap rather than map, to the point it even gets its own syntax for for comprehensions, so clearly I must be missing something. My question is: in what cases is the flatten part of flatMap actually useful? and what other advantages does flatMap have over map?
What if each element being processed could result in zero or more elements?
List(4, 0, 15).flatMap(n => primeFactors(n))
//List(2, 2, 3, 5)
What if you have a name, that might not be spelled correctly, and you want their office assignment, if they have one?
def getID(name:String): Option[EmployeeID] = ???
def getOffice(id:EmployeeID): Option[Office] = ???
val office :Option[Office] =
getID(nameAttempt).flatMap(id => getOffice(id))
You use flatMap() when you have a monad and you need to feed its contents to a monad producer. You want the result Monad[X] not Monad[Monad[X]].
The idea behind flatMap is to be able to take a value of M[A] and a function f having a type of A => M[B] and produce an M[B], if we didn't have a way to "flatten" the value then we'd always end up having an M[M[B]] which is what we don't want most of the time.
(Yes, there are some cases that you may want an M[M[_]] like when you want to create multiple instances of M[_])
I had spent weeks on trying to understand the idea behind "lifting" in scala.
Originally, it was from the example related to Chapter 4 of book "Functional Programming in Scala"
Then I found below topic "How map work on Options in Scala?"
The selected answer specify that:
def map[B](f: A => B): Option[B] = this match (Let's considered this as (*) )
So, from above code, I assume that function "map" is derived from function match. Hence, the mechanism behind "map"
is a kind of pattern matching to provide a case selection between Some, and None
Then, I created below examples by using function map for Seq, Option, and Map (Let's considered below examples as (**) )
Example 1: map for Seq
val xs = Seq(1, 2, 3)
xs.map(println)
Example 2: map for Option
val a:Option[Int] = Some(5)
a.map(println)
val b:Option[Int] = None
b.map(println)
Example 3: map for Map
val capitals = Map("France" -> "Paris", "Japan" -> "Tokyo")
capitals.map(println)
From (*) and (**), I could not know whether "map" is a pattern matching or an iteration, or both.
Thank you for helping me to understand this.
#Jwvh provided a more programming based answer but I want to dig a little bit deeper.
I certainly appreciate you trying to understand how things work in Scala, however if you really want to dig that deep, I am afraid you will need to obtain some basic knowledge of Category Theory since there is no "idea behind lifting in scala" but just the "idea behind lifting"
This is also why functions like "map" can be very confusing. Inherently, programmers are taught map etc. as operations on collections, where as they are actually operations that come with Functors and Natural Transformations (this is normally referred to as fmap in Category Theory and also Haskell).
Before I move on, the short answer is it is a pattern matching in the examples you gave and in some of them it is both. Map is defined specifically to the case, the only condition is that it maintains functoriality
Attention: I will not be defining every single term below, since I would need to write a book to build up to some of the following definitions, interested readers are welcome to research them on their own. You should be able to get some basic understanding by following the types
Let's consider these as Functors, the definition will be something along the lines of this:
In (very very) short, we consider types as objects in the category of our language. The functions between these types (type constructors) are morphisms between types in this category. The set of these transformations are called Endo-Functors (take us from the category of Scala and drop us back in the category of Scala). Functors have to have a polymorphic (which actually has a whole different (extra) definition in category theory) map function, that will take some object A, through some type constructor turn it into object B.
implicit val option: Functor[Option] = new Functor[Option] {
override def map[A,B](optA: Option[A])(f: (A) => B): Option[B] = optA match{
case Some(a) => Some(f(a))
case _ => None
}
}
implicit val seq: Functor[Seq[_]] = new Functor[Seq[_]] {
override def map[A,B](sA: Seq[A])(f: (A) => B): Seq[B] = sA match{
case a :: tail => Seq(f(a), map(tail)(f))
case Nil => Nil
}
}
As you can see in the second case, there is a little bit of both (more of a recursion than iteration but still).
Now before the internet blows up on me, I will say you cant pattern match on Seq in Scala. It works here because the default Seq is also a List. I just provided this example because it is simpler to understand. The underlying definition something along the lines of that.
Now hold on a second. If you look at these types, you see that they also have flatMap defined on them. This means they are something more special than plain Functors. They are Monads. So beyond satisfying functoriality, they obey the monadic laws.
Turns out Monad has a different kind of meaning in the core scala, more on that here: What exactly makes Option a monad in Scala?
But again very very short, this means that we are now in a category where the endofunctors from our previous category are the objects and the mappings between them are morphisms (natural transformations), this is slightly more accurate because if you think about it when you take a type and transform it, you take (carry over) all of it's internal type constructors (2-cell or internal morphisms) with it, you do not only take this sole idea of a type without it's functions.
implicit val optionMonad: Monad[Option] = new Monad[Option] {
override def flatMap[A, B](optA: Option[A])(f: (A) => Option[B]): Option[B] = optA match{
case Some(a) => f(a)
case _ => None
}
def pure[A](a: A): Option[A] = Some(a)
//You can define map using pure and flatmap
}
implicit val seqMonad: Monad[Seq[_]] = new Monad[Seq[_]] {
override def flatMap[A, B](sA: Seq[A])(f: (A) => Seq[B]): Seq[B] = sA match{
case x :: xs => f(a).append(flatMap(tail)(f))
case Nil => Nil
}
override def pure[A](a: A): Seq[A] = Seq(a)
//Same warning as above, also you can implement map with the above 2 funcs
}
One thing you can always count on is map being having pattern match (or some if statement). Why?
In order to satisfy the identity laws, we need to have some sort of a "base case", a unit object and in many cases (such as Lists) those types are gonna be what we call either a product or coproduct.
Hopefully, this did not confuse you further. I wish I could get into every detail of this but it would simply take pages, I highly recommend getting into categories to fully understand where these come from.
From the ScalaDocs page we can see that the type profile for the Standard Library map() method is a little different.
def map[B](f: (A) => B): Seq[B]
So the Standard Library map() is the means to transition from a collection of elements of type A to the same collection but the elements are type B. (A and B might be the same type. They aren't required to be different.)
So, yes, it iterates through the collection applying function f() to each element A to create each new element B. And function f() might use pattern matching in its code, but it doesn't have to.
Now consider a.map(println). Every element of a is sent to println which returns Unit. So if a is List[Int] then the result of a.map(println) is List[Unit], which isn't terribly useful.
When all we want is the side effect of sending information to StdOut then we use foreach() which doesn't create a new collection: a.foreach(println)
Function map for Option isn't about pattern matching. The match/case used in your referred link is just one of the many ways to define the function. It could've been defined using if/else. In fact, that's how it's defined in Scala 2.13 source of class Option:
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable {
self =>
...
final def map[B](f: A => B): Option[B] =
if (isEmpty) None else Some(f(this.get))
...
}
If you view Option like a "collection" of either one element (Some(x)) or no elements (None), it might be easier to see the resemblance of how map transforms an Option versus, say, a List:
val f: Int => Int = _ + 1
List(42).map(f)
// res1: List[Int] = List(43)
List.empty[Int].map(f)
// res2: List[Int] = List()
Some(42).map(f)
// res3: Option[Int] = Some(43)
None.map(f)
// res4: Option[Int] = None
I would like to create a generic function that works for both an Array as an Option:
val numbers = Array(1, 2, 3)
val numberO: Option[Int] = Some(4)
def addOnes(numbers: ???[Int]) = numbers.map(_+1)
addOnes(numbers)
addOnes(numberO)
Right now I have a separate function for each structure
def addOnesForArray(numbers: Array[Int]) = numbers.map(_+1)
def addOnesForOption(numberO: Option[Int]) = numberO.map(_+1)
addOnesForArray(numbers)
addOnesForOption(numberO)
So basically I need a superclass of Array and Option that has the functor and monad methods map, flatMap, filter, ...
You could use structural typing (aka "duck typing"), with which we could say that you need literally "something with a map", written as { def map(): T }. But that's a) ugly, b) uses reflection, and c) hard (note that map(): T is just an example; in reality you will have to match the exact signature of map, with CanBuildFrom and all that jazz).
Better way would be to reach for scalaz or cats and use category theory concepts. Just pick the least powerful abstraction that does the job. If you need just map, then it's a Functor. If you want to map with (curried) functions of more than one parameter, then it's an Applicative. If you also want flatMap, then it's a monad, etc.
Example for functor:
import scalaz._, Scalaz._
def addOne[F[Int]](f: F[Int])(implicit m: Functor[F]) = f.map(_ + 1)
val numbers = Array(1, 2, 3).toList
val numberO = Option(123)
addOne(numbers) // List(2, 3, 4)
addOne(numberO) // Some(124)
You will notice that I had to convert your array to a List because there are no typeclass instances (that I know of) for functors, applicatives, monads etc. that work on arrays. But arrays are old fashioned and invariant and really not idiomatic Scala anyway. If you get them from elsewhere, just convert them to Lists, Vectors etc. (based on your use case) and work with those from that point onwards.
In general I agree with #slouc . If you want to make existing classes kinda extend some other trait you need typeclasses.
But in your particular case it is not required since Option and Array are both Traversable:
object Example extends App {
def plus1(data: Traversable[Int]): Traversable[Int] = data.map(x => x + 1)
println(plus1(Array(1, 2, 3)))
println(plus1(Some(4)))
}
I have a model, which has some Option fields, which contain another Option fields. For example:
case class First(second: Option[Second], name: Option[String])
case class Second(third: Option[Third], title: Option[String])
case class Third(numberOfSmth: Option[Int])
I'm receiving this data from external JSON's and sometimes this data may contain null's, that was the reason of such model design.
So the question is: what is the best way to get a deepest field?
First.get.second.get.third.get.numberOfSmth.get
Above method looks really ugly and it may cause exception if one of the objects will be None. I was looking in to Scalaz lib, but didn't figure out a better way to do that.
Any ideas?
The solution is to use Option.map and Option.flatMap:
First.flatMap(_.second.flatMap(_.third.map(_.numberOfSmth)))
Or the equivalent (see the update at the end of this answer):
First flatMap(_.second) flatMap(_.third) map(_.numberOfSmth)
This returns an Option[Int] (provided that numberOfSmth returns an Int). If any of the options in the call chain is None, the result will be None, otherwise it will be Some(count) where count is the value returned by numberOfSmth.
Of course this can get ugly very fast. For this reason scala supports for comprehensions as a syntactic sugar. The above can be rewritten as:
for {
first <- First
second <- first .second
third <- second.third
} third.numberOfSmth
Which is arguably nicer (especially if you are not yet used to seeing map/flatMap everywhere, as will certainly be the case after a while using scala), and generates the exact same code under the hood.
For more background, you may check this other question: What is Scala's yield?
UPDATE:
Thanks to Ben James for pointing out that flatMap is associative. In other words x flatMap(y flatMap z))) is the same as x flatMap y flatMap z. While the latter is usually not shorter, it has the advantage of avoiding any nesting, which is easier to follow.
Here is some illustration in the REPL (the 4 styles are equivalent, with the first two using flatMap nesting, the other two using flat chains of flatMap):
scala> val l = Some(1,Some(2,Some(3,"aze")))
l: Some[(Int, Some[(Int, Some[(Int, String)])])] = Some((1,Some((2,Some((3,aze))))))
scala> l.flatMap(_._2.flatMap(_._2.map(_._2)))
res22: Option[String] = Some(aze)
scala> l flatMap(_._2 flatMap(_._2 map(_._2)))
res23: Option[String] = Some(aze)
scala> l flatMap(_._2) flatMap(_._2) map(_._2)
res24: Option[String] = Some(aze)
scala> l.flatMap(_._2).flatMap(_._2).map(_._2)
res25: Option[String] = Some(aze)
There is no need for scalaz:
for {
first <- yourFirst
second <- f.second
third <- second.third
number <- third.numberOfSmth
} yield number
Alternatively you can use nested flatMaps
This can be done by chaining calls to flatMap:
def getN(first: Option[First]): Option[Int] =
first flatMap (_.second) flatMap (_.third) flatMap (_.numberOfSmth)
You can also do this with a for-comprehension, but it's more verbose as it forces you to name each intermediate value:
def getN(first: Option[First]): Option[Int] =
for {
f <- first
s <- f.second
t <- s.third
n <- t.numberOfSmth
} yield n
I think it is an overkill for your problem but just as a general reference:
This nested access problem is addressed by a concept called Lenses. They provide a nice mechanism to access nested data types by simple composition. As introduction you might want to check for instance this SO answer or this tutorial. The question whether it makes sense to use Lenses in your case is whether you also have to perform a lot of updates in you nested option structure (note: update not in the mutable sense, but returning a new modified but immutable instance). Without Lenses this leads to lengthy nested case class copy code. If you do not have to update at all, I would stick to om-nom-nom's suggestion.
How do you use scalaz.WriterT for logging?
About monad transformers
This is a very short introduction. You may find more information on haskellwiki or this great slide by #jrwest.
Monads don't compose, meaning that if you have a monad A[_] and a monad B[_], then A[B[_]] can not be derived automatically. However in most cases this can be achieved by having a so-called monad transformer for a given monad.
If we have monad transformer BT for monad B, then we can compose a new monad A[B[_]] for any monad A. That's right, by using BT, we can put the B inside A.
Monad transformer usage in scalaz
The following assumes scalaz 7, since frankly I didn't use monad transformers with scalaz 6.
A monad transformer MT takes two type parameters, the first is the wrapper (outside) monad, the second is the actual data type at the bottom of the monad stack. Note: It may take more type parameters, but those are not related to the transformer-ness, but rather specific for that given monad (like the logged type of a Writer, or the error type of a Validation).
So if we have a List[Option[A]] which we would like to treat as a single composed monad, then we need OptionT[List, A]. If we have Option[List[A]], we need ListT[Option, A].
How to get there? If we have the non-transformer value, we can usually just wrap it with MT.apply to get the value inside the transformer. To get back from the transformed form to normal, we usually call .run on the transformed value.
So val a: OptionT[List, Int] = OptionT[List, Int](List(some(1)) and val b: List[Option[Int]] = a.run are the same data, just the representation is different.
It was suggested by Tony Morris that is best to go into the transformed version as early as possible and use that as long as possible.
Note: Composing multiple monads using transformers yields a transformer stack with types just the opposite order as the normal data type. So a normal List[Option[Validation[E, A]]] would look something like type ListOptionValidation[+E, +A] = ValidationT[({type l[+a] = OptionT[List, a]})#l, E, A]
Update: As of scalaz 7.0.0-M2, Validation is (correctly) not a Monad and so ValidationT doesn't exist. Use EitherT instead.
Using WriterT for logging
Based on your need, you can use the WriterT without any particular outer monad (in this case in the background it will use the Id monad which doesn't do anything), or can put the logging inside a monad, or put a monad inside the logging.
First case, simple logging
import scalaz.{Writer}
import scalaz.std.list.listMonoid
import scalaz._
def calc1 = Writer(List("doing calc"), 11)
def calc2 = Writer(List("doing other"), 22)
val r = for {
a <- calc1
b <- calc2
} yield {
a + b
}
r.run should be_== (List("doing calc", "doing other"), 33)
We import the listMonoid instance, since it also provides the Semigroup[List] instance. It is needed since WriterT needs the log type to be a semigroup in order to be able to combine the log values.
Second case, logging inside a monad
Here we chose the Option monad for simplicity.
import scalaz.{Writer, WriterT}
import scalaz.std.list.listMonoid
import scalaz.std.option.optionInstance
import scalaz.syntax.pointed._
def calc1 = WriterT((List("doing calc") -> 11).point[Option])
def calc2 = WriterT((List("doing other") -> 22).point[Option])
val r = for {
a <- calc1
b <- calc2
} yield {
a + b
}
r.run should be_== (Some(List("doing calc", "doing other"), 33))
With this approach, since the logging is inside the Option monad, if any of the bound options is None, we would just get a None result without any logs.
Note: x.point[Option] is the same in effect as Some(x), but may help to generalize the code better. Not lethal just did it that way for now.
Third option, logging outside of a monad
import scalaz.{Writer, OptionT}
import scalaz.std.list.listMonoid
import scalaz.std.option.optionInstance
import scalaz.syntax.pointed._
type Logger[+A] = WriterT[scalaz.Id.Id, List[String], A]
def calc1 = OptionT[Logger, Int](Writer(List("doing calc"), Some(11): Option[Int]))
def calc2 = OptionT[Logger, Int](Writer(List("doing other"), None: Option[Int]))
val r = for {
a <- calc1
b <- calc2
} yield {
a + b
}
r.run.run should be_== (List("doing calc", "doing other") -> None)
Here we use OptionT to put the Option monad inside the Writer. One of the calculations is Noneto show that even in this case logs are preserved.
Final remarks
In these examples List[String] was used as the log type. However using String is hardly ever the best way, just some convention forced on us by logging frameworks. It would be better to define a custom log ADT for example, and if needed to output, convert it to string as late as possible. This way you could serialize the log's ADT and easily analyse it later programmatically (instead of parsing strings).
WriterT has a host of useful methods to work with to ease logging, check out the source. For example given a w: WriterT[...], you may add a new log entry using w :++> List("other event"), or even log using the currently held value using w :++>> ((v) => List("the result is " + v)), etc.
There are many explicit and longish code (types, calls) in the examples. As always, these are for clarity, refactor them in your code by extracting common types and ops.
type OptionLogger[A] = WriterT[Option, NonEmptyList[String], A]
val two: OptionLogger[Int] = WriterT.put(2.some)("The number two".pure[NonEmptyList])
val hundred: OptionLogger[Int] = WriterT.put(100.some)("One hundred".pure[NonEmptyList])
val twoHundred = for {
a <- two
b <- hundred
} yield a * b
twoHundred.value must be equalTo(200.some)
val log = twoHundred.written map { _.list } getOrElse List() mkString(" ")
log must be equalTo("The number two One hundred")