I'm trying to understand what kind of magic the flatten method does. Here is an example form this article
def toInt(s: String): Option[Int] = {
try {
Some(Integer.parseInt(s.trim))
} catch {
// catch Exception to catch null 's'
case e: Exception => None
}
}
scala> val strings = Seq("1", "2", "foo", "3", "bar")
strings: Seq[java.lang.String] = List(1, 2, foo, 3, bar)
scala> strings.map(toInt)
res0: Seq[Option[Int]] = List(Some(1), Some(2), None, Some(3), None)
scala> val flattenResult = mapResult.flatten
flattenResult: Seq[Int] = List(1, 2, 3)
Wow. It looks like some miracle. How does the flatten method know that we should not add Nones to the resulting collection. It's not obvious form its implementation at all:
def flatten[B](implicit asTraversable: A => /*<:<!!!*/ GenTraversableOnce[B]): CC[B] = {
val b = genericBuilder[B]
for (xs <- sequential)
b ++= asTraversable(xs).seq
b.result()
}
Couldn't you explain the sense of it... I mean what does it do in general? Maybe it does know, that None is a monadic value and use some of its "monadic" methods. But it's just a guess.
You see that flatten can be called on collections which contain values convertible to GenTraversableOnce:
def flatten[B](implicit asTraversable: A => GenTraversableOnce[B]): CC[B]
Since in Option companion object exists an implicit conversion to Iterable, it should be eligible to be flatten-ed(Iterable is subtype of GenTraversableOnce).
It converts Option to List, its implementation is obvious..
Please correct me if I'm wrong, I am too overwhelmed with Scala's collections implementation...
Flatten method receives implicitly a method which able to take an A (Option in our case) and convert it to GenTraversableOnce.
Option object has an implicit method defined in it:
implicit def option2Iterable[A](xo: Option[A]): Iterable[A] = xo.toList
And toList check if the object is empty before returning an Iterable:
def toList: List[A] =
if (isEmpty) List() else new ::(this.get, Nil)
Meaning we will get an empty list when applying option2Iterable on None.
So the line:
b ++= asTraversable(xs).seq
Is actually adding an empty seq to b when performed on None or a list with the value of the Some in case of a Some.
And this is why you can't see None / Some object in your flatten output, but only the values of the Some objects.
Related
If A has the Ordered[A] trait, I'd like to be able to have code that works like this
val collection: List[List[A]] = ... // construct a list of lists of As
val sorted = collection sort { _ < _ }
and get something where the lists have been sorted in lexicographic order. Of course, just because A has the trait Ordered[A] doesn't mean that List[A] has the trait Ordered[List[A]]. Presumably, however, the 'scala way' to do this is with an implicit def.
How do I implicitly convert a List[A] to a Ordered[List[A]], assuming A has the trait Ordered[A] (so that the code above just works)?
I have in mind using the lexicographic ordering on List[A] objects, but I'd like code that can be adapted to others orderings.
Inspired by Ben Lings' answer, I managed to work out what seems like the simplest way to sort lists lexicographically: add the line
import scala.math.Ordering.Implicits._
before doing your List[Int] comparison, to ensure that the implicit function infixOrderingOps is in scope.
(11 minutes ago I actually didn't know how to do this, I hope it's considered okay to answer my own question.)
implicit def List2OrderedList[A <% Ordered[A]](list1: List[A]): Ordered[List[A]] = {
new Ordered[List[A]] {
def compare(list2: List[A]): Int = {
for((x,y) <- list1 zip list2) {
val c = x compare y
if(c != 0) return c
}
return list1.size - list2.size
}
}
}
An important thing to note here is the 'view bound' A <% Ordered[A], which ensures that A needn't itself by an Ordered[A], just that there's a way to do this conversion. Happily, the Scala library's object Predef has an implicit conversion from Ints to RichInts, which in particular are Ordered[Int]s.
The rest of the code is just implementing lexicographic ordering.
Inspired by Ben Lings' answer, I wrote my own version of sort:
def sort[A : Ordering](coll: Seq[Iterable[A]]) = coll.sorted
which is equivalent to:
def sort[A](coll: Seq[Iterable[A]])(implicit ordering: Ordering[A]) = coll.sorted
Note that ordering is implicitly converted to Ordering[Iterable[A]].
Examples:
scala> def sort[A](coll: Seq[Iterable[A]])(implicit ordering: Ordering[A]) = coll.sorted
sort: [A](coll: Seq[Iterable[A]])(implicit ordering: Ordering[A])Seq[Iterable[A]]
scala> val coll = List(List(1, 3), List(1, 2), List(0), Nil, List(2))
coll: List[List[Int]] = List(List(1, 3), List(1, 2), List(0), List(), List(2))
scala> sort(coll)
res1: Seq[Iterable[Int]] = List(List(), List(0), List(1, 2), List(1, 3), List(2))
It was asked how to supply your own comparison function (say, _ > _ instead of _ < _). It suffices to use Ordering.fromLessThan:
scala> sort(coll)(Ordering.fromLessThan(_ > _))
res4: Seq[Iterable[Int]] = List(List(), List(2), List(1, 3), List(1, 2), List(0))
Ordering.by allows you to map your value into another type for which there is already an Ordering instance. Given that also tuples are ordered, this can be useful for lexicographical comparison of case classes.
To make an example, let's define a wrapper of an Int, apply Ordering.by(_.v), where _.v extracts the underlying value, and show that we obtain the same result:
scala> case class Wrap(v: Int)
defined class Wrap
scala> val coll2 = coll.map(_.map(Wrap(_)))
coll2: List[List[Wrap]] = List(List(Wrap(1), Wrap(3)), List(Wrap(1), Wrap(2)), List(Wrap(0)), List(), List(Wrap(2)))
scala> sort(coll2)(Ordering.by(_.v))
res6: Seq[Iterable[Wrap]] = List(List(), List(Wrap(0)), List(Wrap(1), Wrap(2)), List(Wrap(1), Wrap(3)), List(Wrap(2)))
Finally, let's do the same thing on a case class with more members, reusing the comparators for Tuples:
scala> case class MyPair(a: Int, b: Int)
defined class MyPair
scala> val coll3 = coll.map(_.map(MyPair(_, 0)))
coll3: List[List[MyPair]] = List(List(MyPair(1,0), MyPair(3,0)), List(MyPair(1,0), MyPair(2,0)), List(MyPair(0,0)), List(), List(MyPair(2,0)))
scala> sort(coll3)(Ordering.by(x => (x.a, x.b)))
res7: Seq[Iterable[MyPair]] = List(List(), List(MyPair(0,0)), List(MyPair(1,0), MyPair(2,0)), List(MyPair(1,0), MyPair(3,0)), List(MyPair(2,0)))
EDIT:
My definition of sort above is deprecated in 2.13:
warning: method Iterable in object Ordering is deprecated (since 2.13.0):
Iterables are not guaranteed to have a consistent order; if using a type
with a consistent order (e.g. Seq), use its Ordering (found in the
Ordering.Implicits object)
Use instead:
def sort[A](coll: Seq[Seq[A]])(implicit ordering: Ordering[A]) = {
import Ordering.Implicits._
coll.sorted
}
In 2.8, you should be able to just do collection.sorted. sorted takes an implicit Ordering parameter. Any type that implements Ordered has a corresponding Ordering (thanks to the implicit conversion Ordering.ordered). There is also the implicit Ordering.Iterable that makes an Iterable[T] have an Ordering if T has an Ordering.
However, if you try this it doesn't work:
scala> def sort[A <: Ordered[A]](coll: List[List[A]]) = coll.sorted
<console>:5: error: could not find implicit value for parameter ord: Ordering[List[A]]
def sort[A <: Ordered[A]](coll: List[List[A]]) = coll.sorted
^
You need to explicitly specify that you want the Ordering[Iterable[A]]:
def sort[A <: Ordered[A]](coll: List[List[A]]) = coll.sorted[Iterable[A]]
I'm not sure why the compiler can't find the Ordering[Iterable[A]] if the element type of the collection is List[A].
Inspired by Daniel's comment, here is a recursive version:
implicit def toOrdered[A <% Ordered[A]](list1: List[A]): Ordered[List[A]] = {
#scala.annotation.tailrec
def c(list1:List[A], list2:List[A]): Int = {
(list1, list2) match {
case (Nil, Nil) => 0
case (x::xs, Nil) => 1
case (Nil, y::ys) => -1
case (x::xs, y::ys) => (x compare y) match {
case 0 => c(xs, ys)
case i => i
}
}
}
new Ordered[List[A]] {
def compare(list2: List[A]): Int = c(list1, list2)
}
}
With respect to the comment:
I used to think it's more a matter of taste. Sometimes it's easier to verify correctness on a recursive function, and certainly your version is short enough that there is no compelling reason to prefer recursive.
I was intrigued by the performance implications though. So I tried to benchmark it: see http://gist.github.com/468435. I was surprised to see that the recursive version is faster (assuming I did the benchmark correctly). The results still hold true for list of about length 10.
Just because I already implemented this another way, here is a non-recursive version that does not use return:
new Ordering[Seq[String]]() {
override def compare(x: Seq[String], y: Seq[String]): Int = {
x.zip(y).foldLeft(None: Option[Int]){ case (r, (v, w)) =>
if(r.isDefined){
r
} else {
val comp = v.compareTo(w)
if(comp == 0) None
else Some(comp)
}
}.getOrElse(x.size.compareTo(y.size))
}
}
I'm new to scala and I'm trying run an operation on elements of the list and I want to add the result to the end of the same list.
right now I have
func(pos - 1 , list) :: func(pos - 1 , list).takeRight(2).sum
which does the job but is very ugly and calls "func" twice. I'd rather have something like:
func(pos - 1 , list).<somefunction>.takeright(2).sum
but I'm not sure how to go about it.
I don't see why your working code should be ugly, but you can get rid of the redundant method-call by storing the result in a local variable. It looks like you want to append the sum of the two last items to the list, so you would want to use the :+ operator instead of ::, though keep in mind that appending takes linear time whereas prepending to a list is constant, so you should prefer the latter.
So what you could do is
val intermediate: List[Int] = func(pos - 1, list)
intermediate :+ intermediate.takeRight(2).sum
However, just to show you what can be done in Scala, here an example that comes closer to your desired output:
def main(args: Array[String]) {
println(func(4, Nil).append(_.takeRight(2).sum)) // List(1, 2, 3, 5)
}
def func(i: Int, l: List[Int]): List[Int] = List(1, 2, 3)
implicit class PimpMyList(list: List[Int]) {
def append(f: List[Int] => Int): List[Int] = {
list :+ f(list)
}
}
By defining an implicit class you can simulate a richer interface and add a new method append to the interface of List. Note that now, list.append(x) is just the implicit syntax for new PimpMyList(list).append(x). append takes a function that translates a List[Int] to an Int, so we can call this with the function x => x.takeRight(2).sum as desired.
Note that we could also make our implicit class generic:
implicit class PimpMyList[T](list: List[T]) {
def append(f: List[T] => T): List[T] = {
list :+ f(list)
}
}
println(List("foo", "bar", "baz").append(_.takeRight(2).reduce(_ + _))) // List(foo, bar, baz, barbaz)
You can use match to apply some transformation to the result of some expression:
List(1,2,3) match {
case x => x :+ x.takeRight(2).sum
}
//>> List(1, 2, 3, 5)
What might be the most idiomatic way for turning any scala collection into an Option of the same collection, whereas when the collection is empty, the option would be None and otherwise just a Some of the original?
Oh, and without making scala create a memory copy of the entire collection for that humble conversion.
Option.apply is suited for translating null to Option of type None, a nice legacy-wrangling utility, but useless for idiomatic scala seeking the following semantics:
val some = Option(List(3,4,5))
val none = Option(List()) // seeking None, rather than Some(List()) here
I'd probably just go with a filter call after creating the Option:
scala> Option(List()).filter(_.isNonEmpty)
res1: Option[List[Nothing]] = None
scala> Option(List(1,2,3)).filter(_.isNonEmpty)
res2: Option[List[Int]] = Some(List(1, 2, 3))
I feel like it is not a common thing to do and can't think of any common idiomatic way, what I can suggest is adding toOption method like this:
implicit class OptList[A](val list: List[A]) extends AnyVal {
def toOption: Option[List[A]] = if (list.isEmpty) None else Some(list)
}
and use it as follows:
scala> List().toOption
res0: Option[List[Nothing]] = None
scala> List(615, 5).toOption
res1: Option[List[Int]] = Some(List(615, 5))
toOption is a method in scala that appears in some other contexts, for example Try has toOption method.
Similar to Lukasz's answer, but more complete and works for any collection (I think all collections extend from Traversable as this image suggests http://i.stack.imgur.com/Dsptl.png) as requested.
object EnrichedCollections {
implicit class CollectionOps[A <: Traversable[_]](val collection: A) extends AnyVal {
/**
* Returns None if the collection is empty, otherwise returns Some of the collection
*/
def toOption: Option[A] = if (collection.isEmpty) None else Some(collection)
}
}
import EnrichedCollections._
assert(Nil.toOption == None)
assert(Seq(1,2).toOption == Some(Seq(1,2)))
assert(Map.empty.toOption == None)
assert(Map(1 -> "hi", 2 -> "bye").toOption == Some(Map(1 -> "hi", 2 -> "bye")))
Just for fun:
val list = List(1, 2, 3, 4)
val option = list.headOption.map{_ => list}
However, I would question why you would want to do this... using isEmpty is a nice way of checking uniformly for empty lists or None values - you might find this avoids you having to convert to Option in the first place.
I'm wondering what is idiomatic way to applying some operation on the List if it is not empty, and return empty List (Nil) if list is empty.
val result= myList match {
case Nil => Nil // this one looks bad for me
case nonEmpty => myService.getByFilters(nonEmpty)
}
Just using map operation on the list will trigger loop, but I want to achieve same result as map for Option type - i.e. do something only once if List is non-empty, and do nothing if List is empty
I think your design is not quite right perhaps. You should be just able to pass any list into the getByFilters function and it should just handle lists of any length. So there should be no need for these sorts of checks.
If the design change is not possible there is nothing wrong with if:
val result = if(myList.isEmpty) Nil else myService.getByFilters(myList)
It's idiomatic because if returns values. Maybe there are other clean ways, I don't know.
If you just want to require non empty list argument you can use HList or alternatively, you can use this trick:
def takesNonEmptyList[T](head: T, tail: T *): List[T] = head :: tail.toList
You can do something fake to make it seem look idiomatic, but I would not recommend it. It's unclear and unnecessary complication:
def getByFilters(xs: List[Int]) = xs.filter(_ % 2 == 0)
val res = l.headOption.map(_ :: l.tail).map(getByFilters).getOrElse(Nil)
println(res)
prints List(2, 4)
If you really want it, you can just implement your own semantic:
implicit class MySpecialList[T](xs: List[T]) {
def mapIfNotEmpty[R](f: List[T] ⇒ List[R]): List[R] =
if (xs.isEmpty) Nil else f(xs)
}
def getStuff(xs: List[Int]) = xs.map(_ + " OK")
val x: List[Int] = List(1,2,3)
val y: List[Int] = List()
def main(args: Array[String]): Unit = {
val xx = x.mapIfNotEmpty(getStuff) // List("1 OK", "2 OK", "3 OK")
val yy = y.mapIfNotEmpty(getStuff) // List()
}
There is method headOption in List, so you could use option semantic to lift List to Option[List]:
import scala.collection.TraversableLike
implicit class TraversableOption[T <: TraversableLike[_, T]](traversable: T) {
def opt: Option[T] = traversable.headOption.map(_ => traversable)
}
you can use it as:
val result = myList.opt.fold[List[Int]](Nil)(myService.getByFilters)
By invoking each filter service separately,
myList.flatMap(filter => myService.getByFilters(List(filter)))
it gets an empty list if myList is empty. If performance may be a matter, consider also a parallel version with
myList.par
I have a requirement in which I want to implicitly convert and Iterable to Option. The requirement is if the Iterable is empty I return None otherwise I return Some(iterable).
The code below
implicit def toOption[T](iterable: Iterable[T]): Option[Iterable[T]] = if (iterable.isEmpty) None else Some(iterable)
val opt = List(6,7,8).toOption
I am getting compile error 'value toOption is not a member of
List[Int]'
What am I missing here?
Implicit conversion alone is not going to help you if you want to do this. What you need here is an implicit class :
object Rich {
implicit class RichIterable[T](it: Iterable[T]){
def toOption: Option[Iterable[T]] = if(it.isEmpty) None else Some(it)
}
}
scala> import Rich._
import Rich._
scala> List(1,2,3).toOption
res0: Option[Iterable[Int]] = Some(List(1, 2, 3))
scala> List().toOption
res1: Option[Iterable[Nothing]] = None
You have the calling syntax wrong (toOption is not a method on list this way, it's a top level method/function) and you need to be less specific in your type (so it will work with implicit conversions etc):
implicit def toOption[I <% Iterable[_]](iterable: I): Option[I] = if (iterable.isEmpty) None else Some(iterable)
val opt1: Option[List[Int]] = List(6,7,8)
val opt2: Option[List[Int]] = List()
Here I used a view (<%), meaning as long as the argument can be coerced or transformed to the Iterable type, it's ok.
Also you can also write the assignment like this:
val opt2 = List(): Option[List[Int]]
And here is some documentation on views:
http://docs.scala-lang.org/tutorials/tour/views.html