Related
In the code below, version 1 gives the correct result. I make a small variation in V2. The None value had disappeared which is Ok as that is how For Expression works. But what is the reason the yield output in V2 no longer respects the data type returned by myList.lift() which is an Option (as in V1)?
val myList = List(12, 34, "ABC")
Version 1
for { i <- (0 to 3).toList } yield myList.lift(i)
// res1: List[Option[Any]] = List(Some(12), Some(34), Some(ABC), None)
Version 2
for {
i <- (0 to 3).toList
x <- myList.lift(i)
} yield x
// res2: List[Any] = List(12, 34, ABC)
Desugaring the first case:
// desugar for comprehension:
(0 to 3).toList.map(
i => myList.lift(i))
Desugaring the second case:
// desugar for comprehension:
(0 to 3).toList.flatMap(
i => myList.lift(i).map(
x => x))
// remove .map(x => x):
(0 to 3).toList.flatMap(
i => myList.lift(i))
// desugar flatMap:
(0 to 3).toList.map(
i => myList.lift(i)).flatten
The second case simplifies to the first case with a .flatten at the end, which explains the difference in the results: res2 = res1.flatten.
What's actually going on?
Scala can treat Option as a sequence:
Some(foo) --> Seq(foo)
None --> Seq()
The .flatten is just flattening the sequence of sequences.
If you're curious about the types:
scala.collection.Seq.flatten requires that the 'inner' type have an implicit conversion to GenTraversableOnce[T]
there's a global implicit conversion from Option[T] to Iterable[T]
Iterable[T] <: GenTraversableOnce[T]
What does <- mean then?
The <- in x <- myList.lift(i) doesn"t just assign a variable to a value, it "gets a value out of" myList.lift(i). When you "get a value out of" an Option[T], you get foo for Some(foo) and nothing for None. "Getting nothing" means the yield doesn"t run at all for a None, so nothing shows up in the result for the "iteration" when i = 3.
If you're curious about this "get a value out of" concept that is defined for Seq, Option, and many other types in Scala, it is defined for any Monad.
If you de-sugar the for comprehensions
Version 1
List(0, 1, 2, 3).map({ i =>
myList.lift(i)
})
Version 2
List(0, 1, 2, 3).flatMap({ i =>
myList.lift(i).map({ x => x })
})
what is the reason the yield output in V2 no longer respects the data type returned by myList.lift()
The yield does nothing to the output of List.lift:
myList.lift(i) returns Option[Any]
myList.lift(i).map({ x => x }) returns Option[Any]
It's the flatMap that flattens the Option[Any] to Any (by discarding None and un-wrapping Some(a: Any) => a)
I am trying to get the hang of the flatMap implementation in Scala. Based on the definition in Scala programming
Function returning a list of elements as its right argument. It applies the function to each list and returns the concatenation of all function results.
Now to understand this, I have following implementations
val listwords = List(List("abc"),List("def"),List("ghi"))
val res2 = listwords flatMap (_+"1")
println(res2) //output- List(L, i, s, t, (, a, b, c, ), 1, L, i, s, t, (, d, e, f, ), 1, L, i, s, t, (, g, h, i, ), 1)
val res3 = listwords flatMap (_.apply(0).toCharArray())
println(res3) //output- List(a, b, c, d, e, f, g, h, i)
Looking at first output which drives me crazy, why is List[List[String]] treated like List[String]?
After all with answer for above question, someone please help me to perform an operation which needs to pick the first character of the first string of each inner and result in a List[Char]. So given the listwords, I want the output to be List('a', 'd', 'g').
List("abc") + "1" is equivalent to List("abc").toString + "1" so it returns the string "List(a, b, c)1". The type of List.flatMap is
flatMap[B](f: (A) ⇒ GenTraversableOnce[B]): List[B]
and your function has type (List[String] => String). String extends GenTraversableOnce[Char] so your result list has type List[Char].
The code listwords flatMap (_+"1") can be rewritten as listwords flatMap (list => list.toString + "1"). So you basically transformed all lists to strings using toString method.
To obtain first characters you can use the following expression:
listwords.flatMap(_.headOption).flatMap(_.headOption)
First of all, you need to understand the difference between the map and the flatMap methods. Both of them iterate over some container and apply a function literal to every element. The difference is that the flatMap is making one more additional operation: it flattens the structure of the container. There is also a method that allows you just to do the flattening and its called flatten (so the flatMap is the equivalent of the map operation followed by the flatten operation). The second thing you have to remember is that you're modifying (mapping over) nested lists, so you need to nest your map/flatMap calls as well. Those examples should clarify all of those things to you:
scala> val wordLists = List(List("abc"),List("de"),List("f"), List())
wordLists: List[List[String]] = List(List(abc), List(de), List(f), List())
scala> val words = wordsLists.flatten
words: List[String] = List(abc, de, f)
scala> val replacedWordLists = wordsLists.map(_ => List("xyz"))
replacedWordLists: List[List[String]] = List(List(xyz), List(xyz), List(xyz), List(xyz))
scala> val replacedWords = wordsLists.map(_ => List("xyz")).flatten // Equivalent: wordsLists.flatMap(_ => List("xyz"))
replacedWords: List[String] = List(xyz, xyz, xyz, xyz)
scala> val upperCaseWordLists = wordsLists.map(_.map(_.toUpperCase))
upperCaseWordLists: List[List[String]] = List(List(ABC), List(DE), List(F), List())
scala> val upperCaseWords = wordsLists.map(_.map(_.toUpperCase)).flatten // Equivalent: wordsLists.flatMap(_.map(_.toUpperCase))
upperCaseWords: List[String] = List(ABC, DE, F)
scala> val optionalFirstLetterLists = wordLists.map(_.map(_.headOption))
optionalFirstLetterLists: List[List[Option[Char]]] = List(List(Some(a)), List(Some(d)), List(Some(f)), List())
scala> val optionalFirstLetters = wordLists.map(_.map(_.headOption)).flatten // Equivalent: wordLists.flatMap(_.map(_.headOption))
optionalFirstLetters: List[Option[Char]] = List(Some(a), Some(d), Some(f))
scala> val firstLetterLists = wordLists.map(_.map(_.headOption).flatten) // Equivalent: wordLists.map(_.flatMap(_.headOption))
firstLetterLists: List[List[Char]] = List(List(a), List(d), List(f), List())
scala> val firstLetters = wordLists.map(_.flatMap(_.headOption)).flatten // Equivalent: wordLists.flatMap(_.flatMap(_.headOption))
firstLetters: List[Char] = List(a, d, f)
_+"1" isn't doing what you think it's doing.
It's interpreted as list: List[String] => list.+("1")
Since List[String] doesn't contain such a method, the compiler looks for an implicit conversion in scope. It finds any2stringadd. (See http://docs.scala-lang.org/tutorials/tour/implicit-conversions for more on implicit conversions)
implicit final class any2stringadd[A](private val self: A) extends AnyVal {
def +(other: String): String = String.valueOf(self) + other
}
list: List[String] => list.+("1")
now turns into
list: List[String] => new any2stringadd(list).+("1")
which returns String.valueOf(list) + "1"
I need to conditionally apply a function f1 to the elements in a collection depending on the result of a function f2 that takes each element as an argument and returns a boolean. If f2(e) is true, f1(e) will be applied otherwise 'e' will be returned "as is".
My intent is to write a general-purpose function able to work on any kind of collection.
c: C[E] // My collection
f1 = ( E => E ) // transformation function
f2 = ( E => Boolean ) // conditional function
I cannot come to a solution. Here's my idea, but I'm afraid I'm in high-waters
/* Notice this code doesn't compile ~ partially pseudo-code */
conditionallyApply[E,C[_](c: C[E], f2: E => Boolean, f1: E => E): C[E] = {
#scala.annotation.tailrec
def loop(a: C[E], c: C[E]): C[E] = {
c match {
case Nil => a // Here head / tail just express the idea, but I want to use a generic collection
case head :: tail => go(a ++ (if f2(head) f1(head) else head ), tail)
}
}
loop(??, c) // how to get an empty collection of the same type as the one from the input?
}
Could any of you enlighten me?
This looks like a simple map of a Functor. Using scalaz:
def condMap[F[_],A](fa: F[A])(f: A => A, p: A => Boolean)(implicit F:Functor[F]) =
F.map(fa)(x => if (p(x)) f(x) else x)
Not sure why you would need scalaz for something so pedestrian.
// example collection and functions
val xs = 1 :: 2 :: 3 :: 4 :: Nil
def f1(v: Int) = v + 1
def f2(v: Int) = v % 2 == 0
// just conditionally transform inside a map
val transformed = xs.map(x => if (f2(x)) f1(x) else x)
Without using scalaz, you can use the CanBuildFrom pattern. This is exactly what is used in the standard collections library. Of course, in your specific case, this is probably over-engineered as a simple call to map is enough.
import scala.collection.generic._
def cmap[A, C[A] <: Traversable[A]](col: C[A])(f: A ⇒ A, p: A ⇒ Boolean)(implicit bf: CanBuildFrom[C[A], A, C[A]]): C[A] = {
val b = bf(col)
b.sizeHint(col)
for (x <- col) if(p(x)) b += f(x) else b += x
b.result
}
And now the usage:
scala> def f(i: Int) = 0
f: (i: Int)Int
scala> def p(i: Int) = i % 2 == 0
p: (i: Int)Boolean
scala> cmap(Seq(1, 2, 3, 4))(f, p)
res0: Seq[Int] = List(1, 0, 3, 0)
scala> cmap(List(1, 2, 3, 4))(f, p)
res1: List[Int] = List(1, 0, 3, 0)
scala> cmap(Set(1, 2, 3, 4))(f, p)
res2: scala.collection.immutable.Set[Int] = Set(1, 0, 3)
Observe how the return type is always the same as the one provided.
The function could be nicely encapsulated in an implicit class, using the "pimp my library" pattern.
For something like this you can use an implicit class. They were added just for this reason, to enhance libraries you can't change.
It would work like this:
object ImplicitStuff {
implicit class SeqEnhancer[A](s:Seq[A]) {
def transformIf( cond : A => Boolean)( f : A => A ):Seq[A] =
s.map{ x => if(cond(x)) f(x) else x }
}
def main(a:Array[String]) = {
val s = Seq(1,2,3,4,5,6,7)
println(s.transformIf(_ % 2 ==0){ _ * 2})
// result is (1, 4, 3, 8, 5, 12, 7)
}
}
Basically if you call a method that does not exists in the object you're calling it in (in this case, Seq), it will check if there's an implicit class that implements it, but it looks like a built in method.
I have some functions that represent choices, each choice having a distinct desirability. Consider
f1 : Seq[A] => Seq[A]
f2 : Seq[A] => Seq[A]
f3 : Seq[A] => Seq[A]
where f1 is more desirable than f2, and f3 is least desirable. I wrote this scala code to generate the results of making 2 consecutive choices, ordered from most desirable to least
def applyTwice[A](initial: Seq[A],
f1: Seq[A] => Seq[A],
f2: Seq[A] => Seq[A],
f3: Seq[A] => Seq[A]): Seq[A] = {
lazy val f1s = f1(initial).toStream
lazy val f2s = f2(initial).toStream
lazy val f3s = f3(initial).toStream
f1(f1s) ++
f2(f1s) ++ f1(f2s) ++
f2(f2s) ++
f1(f3s) ++ f3(f1s) ++
f2(f3s) ++ f3(f2s) ++
f3(f3s)
}
In general, a series of function applications is ranked by the worst function in the series. If the worst is a tie, compare the 2nd worst, and so on. For example, f4(f1(a)) would be worse than f3(f3(a)) because f4 is worse than f3. Note that it is a tie between f3(f2(a)) and f2(f3(a)).
I probably could generalize this to a variable number of functions, and (with more difficulty) a variable number of applications, but this seems like a classic problem that I just don't know the name of yet. Is this already built into some language/library? Is there a better way?
I don't think that it's a well-known thing, but it's pretty easy to generalise:
import scala.math.max
case class Fun[A](cost : Int, fun : Seq[A] => Seq[A])
def applyN[A](funs : Seq[Fun[A]], n : Int, initial : Seq[A]) =
(Seq((0, initial)) /: (1 to n)) {
case (acc, _) => for {
f <- funs
(cost, old) <- acc
} yield (max(cost, f.cost), f.fun(old))
}
scala> val funs = Seq(Fun[Int](2, _.map(_*2)), Fun[Int](3, _.map(_*3)))
funs: Seq[Fun[Int]] = List(Fun(2,), Fun(3,))
scala> applyN(funs, 2, Seq(1,2,3,4))
res0: Seq[(Int, Seq[Int])] = List((2,List(4, 8, 12, 16)), (3,List(6, 12, 18, 24)), (3,List(6, 12, 18, 24)), (3,List(9, 18, 27, 36)))
Edit: I notice I've used a simplified cost function here, which just looks at the max, but you could easily collect the list of costs and apply whatever decision you wanted to them instead.
A while back this was asked and answered on the Scala mailing list:
Kevin:
Given some nested structure: List[List[...List[T]]]
what's the best (preferably type-safe) way to flatten it to a List[T]
Jesper:
A combination of implicits and default arguments works:
case class Flat[T, U](fn : T => List[U])
implicit def recFlattenFn[T, U](implicit f : Flat[T, U] = Flat((l : T)
=> List(l))) =
Flat((l : List[T]) => l.flatMap(f.fn))
def recFlatten[T, U](l : List[T])(implicit f : Flat[List[T], U]) = f.fn(l)
Examples:
scala> recFlatten(List(1, 2, 3))
res0: List[Int] = List(1, 2, 3)
scala> recFlatten(List(List(1, 2, 3), List(4, 5)))
res1: List[Int] = List(1, 2, 3, 4, 5)
scala> recFlatten(List(List(List(1, 2, 3), List(4, 5)), List(List(6, 7))))
res2: List[Int] = List(1, 2, 3, 4, 5, 6, 7)
I have been looking at this code for a while. I cannot figure out how it works. There seems to be some recursion involved... Can anybody shed some light? Are there other examples of this pattern and does it have a name?
Oh wow, this is an old one! I'll start by cleaning up the code a bit and pulling it into line with current idiomatic conventions:
case class Flat[T, U](fn: T => List[U])
implicit def recFlattenFn[T, U](
implicit f: Flat[T, U] = Flat((xs: T) => List(xs))
) = Flat((xs: List[T]) => xs flatMap f.fn)
def recFlatten[T, U](xs: List[T3])(implicit f: Flat[List[T], U]) = f fn xs
Then, without further ado, break down the code. First, we have our Flat class:
case class Flat[T, U](fn: T => List[U])
This is nothing more than a named wrapper for the function T => List[U], a function that will build a List[U] when given an instance of type T. Note that T here could also be a List[U], or a U, or a List[List[List[U]]], etc. Normally, such a function could be directly specified as the type of a parameter. But we're going to be using this one in implicits, so the named wrapper avoids any risk of an implicit conflict.
Then, working backwards from recFlatten:
def recFlatten[T, U](xs: List[T])(implicit f: Flat[List[T], U]) = f fn xs
This method will take xs (a List[T]) and convert it to a U. To achieve this, it locates an implicit instance of Flat[T,U] and invokes the enclosed function, fn
Then, the real magic:
implicit def recFlattenFn[T, U](
implicit f: Flat[T, U] = Flat((xs: T) => List(xs))
) = Flat((xs: List[T]) => xs flatMap f.fn)
This satisfies the implicit parameter required by recFlatten, it also takes another implicit paramater. Most crucially:
recFlattenFn can act as its own implicit parameter
it returns a Flat[List[X], X], so recFlattenFn will only be implicitly resolved as a Flat[T,U] if T is a List
the implicit f can fallback to a default value if implicit resolution fails (i.e. T is NOT a List)
Perhaps this is best understood in the context of one of the examples:
recFlatten(List(List(1, 2, 3), List(4, 5)))
The type T is inferred as List[List[Int]]
implicit lookup is attempted for a `Flat[List[List[Int]], U]
this is matched by a recursively defined recFlattenFn
Broadly speaking:
recFlattenFn[List[List[Int]], U] ( f =
recFlattenFn[List[Int], U] ( f =
Flat[Int,U]((xs: T) => List(xs)) //default value
)
)
Note that recFlattenFn will only match an implicit search for a Flat[List[X], X] and the type params [Int,_] fail this match because Int is not a List. This is what triggers the fallback to the default value.
Type inference also works backwards up that structure, resolving the U param at each level of recursion:
recFlattenFn[List[List[Int]], Int] ( f =
recFlattenFn[List[Int], Int] ( f =
Flat[Int,Int]((xs: T) => List(xs)) //default value
)
)
Which is just a nesting of Flat instances, each one (except the innermost) performing a flatMap operation to unroll one level of the nested List structure. The innermost Flat simply wraps all the individual elements back up in a single List.
Q.E.D.
May be a good solution is to try to look at how the types are infered. To avoid ambiguity, let us rename the generics:
case class Flat[T, U](fn : T => List[U])
implicit def recFlattenFn[T2, U2](implicit f : Flat[T2, U2] =
Flat((l : T2) => List(l))) =
Flat((l : List[T2]) => l.flatMap(f.fn))
def recFlatten[T3, U3](l : List[T3])(implicit f : Flat[List[T3], U3]) = f.fn(l)
In the first case, res0, the type of T3 is Int you cannot infer yet the type of U3, but you know that you will need a Flat[List[Int, U3]] object that will be provided implicitly. There is only one "implicit candidate": the result of the recFlattenFn function and its type is Flat[List[T2], List[U2]]. Thus T2 = Int and U2 = U3 (that we still need to infer).
Now, if we weant to be able to use recFlatten we must provide it a parameter f. Here is the trick. You can either use an implicit of type Flat[Int, U2] or the default value of type Int => List[Int]. Let us look about the available implicits. As explained before recFlattenFn can provide a Flat[List[T2], U2] (for a new T2 and U2) object. It does not fit the expected signature of fat this point. Thus, no implicit are a good candidate here and we must use the default argument. As the type of the default argument is Int => List[Int], U2and U3 are Int and there we go.
Hope that this long prose will help. I leave you with the resolution of res1 and res2.