Scala: Implicit evidence of Map with Traversable values? - scala

I'm trying to write a generic invert method that takes a Map from keys of type A to values that are collections of type B and converts it to a Map with keys of type B and collections of A using the same original collection type.
My goal is to make this method a member of a MyMap[A,B] class that offers extensions of the basic library methods, where Maps are implicitly converted to MyMaps. I am able to do this implicit conversion for a generic map, but I want to further specify that the invert method should only work in the case where B is a collection.
I lack the understanding of scala's collections framework to accomplish this - I've scoured the net for thorough introductory explanations of the signatures that look like a hodgepodge of Repr, CC,That, and CanBuildFrom, but I don't really understand how all these pieces fit together well enough to construct the method signature on my own. Please don't just give me the working signature for this case - I want to understand how the signatures of methods that use generic collections work in a broader sense so I'm able to do this independently going forward. Alternatively, feel free to reference an online resource that elaborates on this - I was unable to find one that was comprehensive and clear.
EDIT
I seem to have gotten it to work with the following code. If I did something wrong or you see something that can be improved, please comment & answer with a more optimal alternative.
class MyMap[A, B](val _map: Map[A, B]) {
def invert[E, CC[E]](
implicit ev1: B =:= CC[E],
ev2: CC[E] <:< TraversableOnce[E],
cbf: CanBuildFrom[CC[A], A, CC[A]]
): Map[E, CC[A]] = {
val inverted = scala.collection.mutable.Map.empty[E, Builder[A, CC[A]]]
for {
(key, values) <- _map
value <- values.asInstanceOf[CC[E]]
} {
if (!inverted.contains(value)) {
inverted += (value -> cbf())
}
inverted.get(value).foreach(_ += key)
}
return inverted.map({ case (k,v) => (k -> v.result) }).toMap
}
}

I started from your code and ended up with this:
implicit class MyMap[A, B, C[B] <: Traversable[B]](val _map: Map[A, C[B]]) {
def invert(implicit cbf: CanBuildFrom[C[A], A, C[A]]): Map[B, C[A]] = {
val inverted = scala.collection.mutable.Map.empty[B, Builder[A, C[A]]]
for ((k, vs) <- _map; v <- vs) {
inverted.getOrElseUpdate(v, cbf()) += k
}
inverted.map({ case (k, v) => (k -> v.result)}).toMap
}
}
val map = Map("a"-> List(1,2,3), "b" -> List(1,2))
println(map.invert) //Map(2 -> List(a, b), 1 -> List(a, b), 3 -> List(a))

Related

combining zip and map operations in Scala

I understand map and flatten operations can be combined into flatMap, and filter and map into collect in Scala.
Is there anyway I can combine zip/zipwithIndex with map operation?
There is no single operation in the standard library, as far as I know, but there is an extension method on various tuples, called zipped. This method returns an object which provides methods like map and flatMap, which would perform zipping in step with mapping:
(xs, ys).zipped.map((x, y) => x * y)
This object also is implicitly convertible to Traversable, so you can call more complex methods like mkString or foldLeft.
If, for some reason, you really wanted a combined version you could write one yourself.
implicit class SeqOps[A](s: Seq[A]) {
def zipWithIndex2[A1 >: A, B >: Int, That](f: (A, Int) => (A1, B))(implicit bf: CanBuildFrom[Seq[A], (A1, B), That]): That = {
val b = bf(s)
var i = 0
for (x <- s) {
b += f(x, i)
i += 1
}
b.result()
}
}
Call it like:
s.zipWithIndex2 {
case (a, b) => (a + "2", b + 2)
}
I'd really think about this twice though and most likely go with any of the other approaches that have been suggested.

Scala: different foldRight implementations in list

I've just figured out that scala (I'm on 2.12) provides completely different implementations of foldRight for immutable list and mutable list.
Immutable list (List.scala):
override def foldRight[B](z: B)(op: (A, B) => B): B =
reverse.foldLeft(z)((right, left) => op(left, right))
Mutable list (LinearSeqOptimized.scala):
def foldRight[B](z: B)(#deprecatedName('f) op: (A, B) => B): B =
if (this.isEmpty) z
else op(head, tail.foldRight(z)(op))
Now I'm just curious.
Could you please explain me why was it implemented so differently?
The override in List seems to override the foldRight in LinearSeqOptimized. The implementation in LinearSeqOptimized
def foldRight[B](z: B)(#deprecatedName('f) op: (A, B) => B): B =
if (this.isEmpty) z
else op(head, tail.foldRight(z)(op))
looks exactly like the canonical definition of foldRight as a catamorphism from your average theory book. However, as was noticed in SI-2818, this implementation is not stack-safe (throws unexpected StackOverflowError for long lists). Therefore, it was replaced by a stack-safe reverse.foldLeft in this commit. The foldLeft is stack-safe, because it has been implemented by a while loop:
def foldLeft[B](z: B)(#deprecatedName('f) op: (B, A) => B): B = {
var acc = z
var these = this
while (!these.isEmpty) {
acc = op(acc, these.head)
these = these.tail
}
acc
}
That hopefully explains why it was overridden in List. It doesn't explain why it was not overridden in other classes. I guess it's simply because the mutable data structures are used less often and quite differently anyway (often as buffers and accumulators during the construction of immutable ones).
Hint: there is a blame button in the top right corner over every file on Github, so you can always track down what was changed when, by whom, and why.

Iterable with two elements?

We have Option which is an Iterable over 0 or 1 elements.
I would like to have such a thing with two elements. The best I have is
Array(foo, bar).map{...}, while what I would like is:
(foo, bar).map{...}
(such that Scala recognized there are two elements in the Iterable).
Does such a construction exist in the standard library?
EDIT: another solution is to create a map method:
def map(a:Foo) = {...}
val (mappedFoo, mappedBar) = (map(foo), map(bar))
If all you want to do is map on tuples of the same type, a simple version is:
implicit class DupleOps[T](t: (T,T)) {
def map[B](f : T => B) = (f(t._1), f(t._2))
}
Then you can do the following:
val t = (0,1)
val (x,y) = t.map( _ +1) // x = 1, y = 2
There's no specific type in the scala standard library for mapping over exactly 2 elements.
I can suggest you the following thing (I suppose foo and bar has the same type T):
(foo, bar) // -> Tuple2[T,T]
.productIterator // -> Iterator[Any]
.map(_.asInstanceOf[T]) // -> Iterator[T]
.map(x => // some works)
No, it doesn't.
You could
Make one yourself.
Write an implicit conversion from 2-tuples to a Seq of the common supertype. But this won't yield 2-tuples from operations.
object TupleOps {
implicit def tupleToSeq[A <: C, B <: C](tuple: (A, B)): Seq[C] = Seq(tuple._1,tuple._2)
}
import TupleOps._
(0, 1).map(_ + 1)
Use HLists from shapeless. These provide operations on heterogenous lists, whereas you (probably?) have a homogeneous list, but it should work.

When using type classes, how to deal with object in different ways?

Suppose I have a type class Graph[G,V] which states that an object of type G is also a graph with vertices of type V.
Now I have an implicit that lets me treat sets of pairs of type A as a graph with vertices of type A (not being able to express unconnected vertices...). I can use the implicit by importing the following object's scope.
object TupleSetGraph{
implicit def ts2graph[A]: Graph[Set[(A,A)],A] = new Graph[Set[(A,A)],A] {
def nodes(g: Set[(A, A)]): Set[A] = g flatMap (t => Set(t._1,t._2))
def adjacent(g: Set[(A, A)], n1: A, n2: A): Boolean = g.contains((n1,n2)) || g.contains((n2,n1))
}
}
Suppose I also want to be able to map the content of the vertices, thus being able to do the following:
(_: Set[(A,A)]).map((_: A => B)): Set[(B,B)]
But there is already a map defined on Set. How to deal with the problem that the same data structure can be seen as the same thing (something having a map function) in different ways?
Sketching a possible solution :
Put the map operation in an auxiliary trait
say GraphOps (that could be Graph itself, but map signature will probably be too complex for that)
case class GraphOps[G](data: G) { def map...}
Making it easy to get the GraphOps :
object Graph {
def apply[G](data: G) = GraphOps(data)
}
With that, the call will be
Graph(set).map(f)
apply could be made implicit, but I'm not sure I want to do that (and if I did, I'm not sure it would find map properly).
Variant. Have the graph in GraphOps
we can also do
case class GraphOps[G,V](data: G, graph: Graph[G,V])
and
object Graph {
def apply[G,V](data: G)(implicit graph: Graph[G,V]) = GraphOps(data, graph)
}
The good point of that is that vertex type V is available in GraphOps
Defining the map operation
The signature you want is complex, with Set[(A,A)] returning a Set[(B,B)], but other graph implementations returning something completely different. This is similar to what is done in the collection library.
We may introduce a trait CanMapGraph[From, Elem, To], akin to CanBuildFrom
trait CanMapGrap[FromGraph, FromElem, ToGraph, ToElem] {
def map(data: FromGraph, f: FromElem => ToElem): ToGraph
}
(probably you would change this to have more elementary operations than map, so that it may be used for different operations, as done with CanBuildFrom)
Then map would be
case class GraphOps[G](data: G) {
def map[A,B](f: A, B)(implicit ev: CanMapFrom[G, A, B, G2]) : G2 =
ev.map(data, f)
}
You can define
implicit def mapPairSetToPairSet[A, B] =
new CanMapGraph[Set[(A,A)], A, Set[(B,B)], B] {
def map(set: Set[(A,A)], f: A => B) = set.map{case (x, y) => (f(x), f(y))}
}
And then you do
val theGraph = Set("A" -> "B", "BB" -> "A", "B" -> "C", "C" -> "A")
Graph(theGraph).map(s: String -> s(0).toLower)
res1: Set[(Char, Char)] = Set((a,b), (b,a), (b,c), (c,a))
A problem with that is that the type of the vertices is not known in the first argument list, the one for f, so we have to be explicit with s: String.
With the alternative GraphOps, where we get the vertex type early, A is not a parameter of Map, but of GraphOps, so it is known from the start and does not need to be explicit in f. It you do it that way, you may want to pass the graph to method map in CanMapGraph.
With the first solution, it is still easy to give the graph to the CanMapGraph.
implicit def anyGraphToSet[G,V,W](implicit graph: Graph[G,V])
= new CanMapFrom[G, V, Set[(W,W)], W] {
def map(data: G, f: V => W) =
(for {
from <- graph.nodes(data)
to <- graph.nodes(data))
if graph.adjacent(data, from, to) }
yield (from, to)).toSet
}
val x: Set[(A, A)] = ...
(x: Graph[_, _]).map(...)
seems to be the best you can do if you want the names to be the same.
As you point out, that's not what you want. This should work better:
object Graph {
def map[G, V](graph: G)(f: V => V)(implicit instance: Graph[G, V]) = ...
}
val x: Set[(A, A)] = ...
Graph.map(x)(f)
// but note that the type of argument of f will often need to be explicit, because
// type inference only goes from left to right, and implicit arguments come last
Note that you can only let f to be V => V and not V => V1. Why? Imagine that you have
implicit g1: Graph[SomeType, Int], but not implicit g2: Graph[SomeType, String]. What could Graph.map(_: SomeType)((_: Int).toString) return then? This problem can be avoided by requiring G to be a parametrized type:
trait Graph[G[_]] {
def nodes[A](g: G[A]): Set[A]
def adjacent[A](g: G[A], n1: A, n2: A): Boolean
}
object TupleSetGraph{
type SetOfPairs[A] = Set[(A,A)]
implicit def ts2graph: Graph[SetOfPairs] = new Graph[SetOfPairs] {
def nodes[A](g: Set[(A, A)]): Set[A] = g flatMap (t => Set(t._1,t._2))
def adjacent[A](g: Set[(A, A)], n1: A, n2: A): Boolean = g.contains((n1,n2)) || g.contains((n2,n1))
}
}
then you have
object Graph {
def map[G[_], V, V1](graph: G[V])(f: V => V1)(implicit instance: Graph[G]) = ...
}
If you are using type classes, then you can do something like this:
implicitly[TypeClass].map(...)
If you are using view bounds, then Alexey's answer is correct:
(...: ViewBound).map(...)

What's the deal with all the Either cruft?

The Either class seems useful and the ways of using it are pretty obvious. But then I look at the API documentation and I'm baffled:
def joinLeft [A1 >: A, B1 >: B, C] (implicit ev: <:<[A1, Either[C, B1]]):
Either[C, B1]
Joins an Either through Left.
def joinRight [A1 >: A, B1 >: B, C] (implicit ev: <:<[B1, Either[A1, C]]):
Either[A1, C]
Joins an Either through Right.
def left : LeftProjection[A, B]
Projects this Either as a Left.
def right : RightProjection[A, B]
Projects this Either as a Right.
What do I do with a projection and how do I even invoke the joins?
Google just points me to the API documentation.
This might just be a case of "paying no attention to the man behind the curtain", but I don't think so. I think this is important.
left and right are the important ones. Either is useful without projections (mostly you do pattern matching), but projections are quite worthy of attention, as they give a much richer API. You will use joins much less.
Either is often used to mean "a proper value or an error". In this respect, it is like an extended Option . When there is no data, instead of None, you have an error.
Option has a rich API. The same can be made available on Either, provided we know, in Either, which one is the result and which one is the error.
left and right projection says just that. It is the Either, plus the added knowledge that the value is respectively at left or at right, and the other one is the error.
For instance, in Option, you can map, so opt.map(f) returns an Option with f applied to the value of opt if it has a one, and still None if opt was None. On a left projection, it will apply f on the value at left if it is a Left, and leave it unchanged if it is a Right. Observe the signatures:
In LeftProjection[A,B], map[C](f: A => C): Either[C,B]
In RightProjection[A,B], map[C](f: B => C): Either[A,C].
left and right are simply the way to say which side is considered the value when you want to use one of the usual API routines.
Alternatives could have been:
set a convention, as in Haskell, where there were strong syntactical reasons to put the value at right. When you want to apply a method on the other side (you may well want to change the error with a map for instance), do a swap before and after.
postfix method names with Left or Right (maybe just L and R). That would prevent using for comprehension. With for comprehensions (flatMap in fact, but the for notation is quite convenient) Either is an alternative to (checked) exceptions.
Now the joins. Left and Right means the same thing as for the projections, and they are closely related to flatMap. Consider joinLeft. The signature may be puzzling:
joinLeft [A1 >: A, B1 >: B, C] (implicit ev: <:<[A1, Either[C, B1]]):
Either[C, B1]
A1 and B1 are technically necessary, but not critical to the understanding, let's simplify
joinLeft[C](implicit ev: <:<[A, Either[C, B])
What the implicit means is that the method can only be called if A is an Either[C,B]. The method is not available on an Either[A,B] in general, but only on an Either[Either[C,B], B]. As with left projection, we consider that the value is at left (that would be right for joinRight). What the join does is flatten this (think flatMap). When one join, one does not care whether the error (B) is inside or outside, we just want Either[C,B]. So Left(Left(c)) yields Left(c), both Left(Right(b)) and Right(b) yield Right(b). The relation with flatMap is as follows:
joinLeft(e) = e.left.flatMap(identity)
e.left.flatMap(f) = e.left.map(f).joinLeft
The Option equivalent would work on an Option[Option[A]], Some(Some(x)) would yield Some(x) both Some(None) and None would yield None. It can be written o.flatMap(identity). Note that Option[A] is isomorphic to Either[A,Unit] (if you use left projections and joins) and also to Either[Unit, A] (using right projections).
Ignoring the joins for now, projections are a mechanism allowing you to use use an Either as a monad. Think of it as extracting either the left or right side into an Option, but without losing the other side
As always, this probably makes more sense with an example. So imagine you have an Either[Exception, Int] and want to convert the Exception to a String (if present)
val result = opReturningEither
val better = result.left map {_.getMessage}
This will map over the left side of result, giving you an Either[String,Int]
joinLeft and joinRight enable you to "flatten" a nested Either:
scala> val e: Either[Either[String, Int], Int] = Left(Left("foo"))
e: Either[Either[String,Int],Int] = Left(Left(foo))
scala> e.joinLeft
res2: Either[String,Int] = Left(foo)
Edit: My answer to this question shows one example of how you can use the projections, in this case to fold together a sequence of Eithers without pattern matching or calling isLeft or isRight. If you're familiar with how to use Option without matching or calling isDefined, it's analagous.
While curiously looking at the current source of Either, I saw that joinLeft and joinRight are implemented with pattern matching. However, I stumbled across this older version of the source and saw that it used to implement the join methods using projections:
def joinLeft[A, B](es: Either[Either[A, B], B]) =
es.left.flatMap(x => x)
My suggestion is add the following to your utility package:
implicit class EitherRichClass[A, B](thisEither: Either[A, B])
{
def map[C](f: B => C): Either[A, C] = thisEither match
{
case Left(l) => Left[A, C](l)
case Right(r) => Right[A, C](f(r))
}
def flatMap[C](f: B => Either[A, C]): Either[A, C] = thisEither match
{
case Left(l) => Left[A, C](l)
case Right(r) => (f(r))
}
}
In my experience the only useful provided method is fold. You don't really use isLeft or isRight in functional code. joinLeft and joinRight might be useful as flatten functions as explained by Dider Dupont but, I haven't had occasion to use them that way. The above is using Either as right biased, which I suspect is how most people use them. Its like an Option with an error value instead of None.
Here's some of my own code. Apologies its not polished code but its an example of using Either in a for comprehension. Adding the map and flatMap methods to Either allows us to use the special syntax in for comprehensions. Its parsing HTTP headers, either returning an Http and Html error page response or a parsed custom HTTP Request object. Without the use of the for comprehension the code would be very difficult to comprehend.
object getReq
{
def LeftError[B](str: String) = Left[HResponse, B](HttpError(str))
def apply(line1: String, in: java.io.BufferedReader): Either[HResponse, HttpReq] =
{
def loop(acc: Seq[(String, String)]): Either[HResponse, Seq[(String, String)]] =
{
val ln = in.readLine
if (ln == "")
Right(acc)
else
ln.splitOut(':', s => LeftError("400 Bad Syntax in Header Field"), (a, b) => loop(acc :+ Tuple2(a.toLowerCase, b)))
}
val words: Seq[String] = line1.lowerWords
for
{
a3 <- words match
{
case Seq("get", b, c) => Right[HResponse, (ReqType.Value, String, String)]((ReqType.HGet, b, c))
case Seq("post", b, c) => Right[HResponse, (ReqType.Value, String, String)]((ReqType.HPost, b, c))
case Seq(methodName, b, c) => LeftError("405" -- methodName -- "method not Allowed")
case _ => LeftError("400 Bad Request: Bad Syntax in Status Line")
}
val (reqType, target, version) = a3
fields <- loop(Nil)
val optLen = fields.find(_._1 == "content-length")
pair <- optLen match
{
case None => Right((0, fields))
case Some(("content-length", second)) => second.filterNot(_.isWhitespace) match
{
case s if s.forall(_.isDigit) => Right((s.toInt, fields.filterNot(_._1 == "content-length")))
case s => LeftError("400 Bad Request: Bad Content-Length SyntaxLine")
}
}
val (bodyLen, otherHeaderPairs) = pair
val otherHeaderFields = otherHeaderPairs.map(pair => HeaderField(pair._1, pair._2))
val body = if (bodyLen > 0) (for (i <- 1 to bodyLen) yield in.read.toChar).mkString else ""
}
yield (HttpReq(reqType, target, version, otherHeaderFields, bodyLen, body))
}
}