Related
USE CASE
I have a list of files that can might have a valid mime type or not.
In my code, I represent this using an Option.
I need to convert a Seq[Option[T]] to Option[Seq[T]] so that I do not process the list if some of the files are invalid.
ERROR
This is the error in the implementation below:
found : (Option[Seq[A]], Option[A]) => Option[Seq[A]]
[error] required: (Option[Any], Option[Any]) => Option[Any]
[error] s.fold(init)(liftOptionItem[A])
IMPLEMENTATION
def liftOptionItem[A](acc: Option[Seq[A]], itemOption: Option[A]): Option[Seq[A]] = {
{
acc match {
case None => None
case Some(items) =>
itemOption match {
case None => None
case Some(item) => Some(items ++ Seq(item))
}
}
}
}
def liftOption[A](s: Seq[Option[A]]): Option[Seq[A]] = {
s.fold(Some(Seq()))(liftOptionItem[A])
}
This implementation returns Option[Any] instead of Option[Seq[A] as the type of the liftOptionItem[A] does not fit in.
If you use TypeLevel Cats:
import cats.implicits._
List(Option(1), Option(2), Option(3)).traverse(identity)
Returns:
Option[List[Int]] = Some(List(1, 2, 3))
You have to use List so use a toList first:
Seq(Option(1), Option(2), Option(3)).toList.traverse(identity).map(_.toSeq)
using scalaz:
import scalaz._
import Sclaza._
val x:List[Option[Int]] = List(Option(1))
x.sequence[Option, Int] //returns Some(List(1))
val y:List[Option[Int]] = List(None, Option(1))
y.sequence[Option, Int] // returns None
If you dont want to use functional libraries like cats or Scalaz you could use a foldLeft
def seqToOpt[A](seq: Seq[Option[A]]): Option[Seq[A]] =
seq.foldLeft(Option(Seq.empty[A])){
(res, opt) =>
for {
seq <- res
v <- opt
} yield seq :+ v
}
Tail-recursive Solution: It returns None if any one of the seq element is None.
def seqToOption[T](s: Seq[Option[T]]): Option[Seq[T]] = {
#tailrec
def seqToOptionHelper(s: Seq[Option[T]], accum: Seq[T] = Seq[T]()): Option[Seq[T]] = {
s match {
case Some(head) :: Nil => Option(head +: accum)
case Some(head) :: tail => seqToOptionHelper(tail, head +: accum)
case _ => None
}
}
seqToOptionHelper(s)
}
Dealing with None in case statements is the reason for returning the Option[Seq[Any]] type in stead of Option[Seq[A]] type. We need to make the function
liftOptionItem[A] to return Option[Seq[Any]] type. And the compilation error can be fixed with the following changes in both the functions.(Because fold does not go in any particular order, there are constraints on the start value and thus return value , the foldLeft is used in stead of fold.)
def liftOptionItem[A](acc: Option[Seq[Any]], itemOption: Option[A]): Option[Seq[Any]] = {
{
acc match {
case None => Some(Nil)
case Some(items)=>
itemOption match {
case None => Some(items ++ Seq("None"))
case Some(item) => Some(items ++ Seq(item))
}
}
}
}
def liftOption[A](s: Seq[Option[A]]): Option[Seq[Any]] = {
s.foldLeft(Option(Seq[Any]()))(liftOptionItem[A])
}
Now, code compiles.
In Scala REPL:
scala> val list1 = Seq(None,Some(21),None,Some(0),Some(43),None)
list1: Seq[Option[Int]] = List(None, Some(21), None, Some(0), Some(43), None)
scala> liftOption(list1)
res2: Option[Seq[Any]] = Some(List(None, 21, None, 0, 43, None))
scala> val list2 = Seq(None,Some("String1"),None,Some("String2"),Some("String3"),None)
list2: Seq[Option[String]] = List(None, Some(String1), None, Some(String2), Some(String3), None)
scala> liftOption(list2)
res3: Option[Seq[Any]] = Some(List(None, String1, None, String2, String3, None))
There is no really "beautiful" way to make this with out scalaz or cats.
But you can try something like this.
def seqToOpt[A](seq: Seq[Option[A]]): Option[Seq[A]] = {
val flatten = seq.flatten
if (flatten.isEmpty) None
else Some(flatten)
}
How do I rewrite the following loop (pattern) into Scala, either using built-in higher order functions or tail recursion?
This the example of an iteration pattern where you do a computation (comparison, for example) of two list elements, but only if the second one comes after first one in the original input. Note that the +1 step is used here, but in general, it could be +n.
public List<U> mapNext(List<T> list) {
List<U> results = new ArrayList();
for (i = 0; i < list.size - 1; i++) {
for (j = i + 1; j < list.size; j++) {
results.add(doSomething(list[i], list[j]))
}
}
return results;
}
So far, I've come up with this in Scala:
def mapNext[T, U](list: List[T])(f: (T, T) => U): List[U] = {
#scala.annotation.tailrec
def loop(ix: List[T], jx: List[T], res: List[U]): List[U] = (ix, jx) match {
case (_ :: _ :: is, Nil) => loop(ix, ix.tail, res)
case (i :: _ :: is, j :: Nil) => loop(ix.tail, Nil, f(i, j) :: res)
case (i :: _ :: is, j :: js) => loop(ix, js, f(i, j) :: res)
case _ => res
}
loop(list, Nil, Nil).reverse
}
Edit:
To all contributors, I only wish I could accept every answer as solution :)
Here's my stab. I think it's pretty readable. The intuition is: for each head of the list, apply the function to the head and every other member of the tail. Then recurse on the tail of the list.
def mapNext[U, T](list: List[U], fun: (U, U) => T): List[T] = list match {
case Nil => Nil
case (first :: Nil) => Nil
case (first :: rest) => rest.map(fun(first, _: U)) ++ mapNext(rest, fun)
}
Here's a sample run
scala> mapNext(List(1, 2, 3, 4), (x: Int, y: Int) => x + y)
res6: List[Int] = List(3, 4, 5, 5, 6, 7)
This one isn't explicitly tail recursive but an accumulator could be easily added to make it.
Recursion is certainly an option, but the standard library offers some alternatives that will achieve the same iteration pattern.
Here's a very simple setup for demonstration purposes.
val lst = List("a","b","c","d")
def doSomething(a:String, b:String) = a+b
And here's one way to get at what we're after.
val resA = lst.tails.toList.init.flatMap(tl=>tl.tail.map(doSomething(tl.head,_)))
// resA: List[String] = List(ab, ac, ad, bc, bd, cd)
This works but the fact that there's a map() within a flatMap() suggests that a for comprehension might be used to pretty it up.
val resB = for {
tl <- lst.tails
if tl.nonEmpty
h = tl.head
x <- tl.tail
} yield doSomething(h, x) // resB: Iterator[String] = non-empty iterator
resB.toList // List(ab, ac, ad, bc, bd, cd)
In both cases the toList cast is used to get us back to the original collection type, which might not actually be necessary depending on what further processing of the collection is required.
Comeback Attempt:
After deleting my first attempt to give an answer I put some more thought into it and came up with another, at least shorter solution.
def mapNext[T, U](list: List[T])(f: (T, T) => U): List[U] = {
#tailrec
def loop(in: List[T], out: List[U]): List[U] = in match {
case Nil => out
case head :: tail => loop(tail, out ::: tail.map { f(head, _) } )
}
loop(list, Nil)
}
I would also like to recommend the enrich my library pattern for adding the mapNext function to the List api (or with some adjustments to any other collection).
object collection {
object Implicits {
implicit class RichList[A](private val underlying: List[A]) extends AnyVal {
def mapNext[U](f: (A, A) => U): List[U] = {
#tailrec
def loop(in: List[A], out: List[U]): List[U] = in match {
case Nil => out
case head :: tail => loop(tail, out ::: tail.map { f(head, _) } )
}
loop(underlying, Nil)
}
}
}
}
Then you can use the function like:
list.mapNext(doSomething)
Again, there is a downside, as concatenating lists is relatively expensive.
However, variable assignemends inside for comprehensions can be quite inefficient, too (as this improvement task for dotty Scala Wart: Convoluted de-sugaring of for-comprehensions suggests).
UPDATE
Now that I'm into this, I simply cannot let go :(
Concerning 'Note that the +1 step is used here, but in general, it could be +n.'
I extended my proposal with some parameters to cover more situations:
object collection {
object Implicits {
implicit class RichList[A](private val underlying: List[A]) extends AnyVal {
def mapNext[U](f: (A, A) => U): List[U] = {
#tailrec
def loop(in: List[A], out: List[U]): List[U] = in match {
case Nil => out
case head :: tail => loop(tail, out ::: tail.map { f(head, _) } )
}
loop(underlying, Nil)
}
def mapEvery[U](step: Int)(f: A => U) = {
#tailrec
def loop(in: List[A], out: List[U]): List[U] = {
in match {
case Nil => out.reverse
case head :: tail => loop(tail.drop(step), f(head) :: out)
}
}
loop(underlying, Nil)
}
def mapDrop[U](drop1: Int, drop2: Int, step: Int)(f: (A, A) => U): List[U] = {
#tailrec
def loop(in: List[A], out: List[U]): List[U] = in match {
case Nil => out
case head :: tail =>
loop(tail.drop(drop1), out ::: tail.drop(drop2).mapEvery(step) { f(head, _) } )
}
loop(underlying, Nil)
}
}
}
}
list // [a, b, c, d, ...]
.indices // [0, 1, 2, 3, ...]
.flatMap { i =>
elem = list(i) // Don't redo access every iteration of the below map.
list.drop(i + 1) // Take only the inputs that come after the one we're working on
.map(doSomething(elem, _))
}
// Or with a monad-comprehension
for {
index <- list.indices
thisElem = list(index)
thatElem <- list.drop(index + 1)
} yield doSomething(thisElem, thatElem)
You start, not with the list, but with its indices. Then, you use flatMap, because each index goes to a list of elements. Use drop to take only the elements after the element we're working on, and map that list to actually run the computation. Note that this has terrible time complexity, because most operations here, indices/length, flatMap, map, are O(n) in the list size, and drop and apply are O(n) in the argument.
You can get better performance if you a) stop using a linked list (List is good for LIFO, sequential access, but Vector is better in the general case), and b) make this a tiny bit uglier
val len = vector.length
(0 until len)
.flatMap { thisIdx =>
val thisElem = vector(thisIdx)
((thisIdx + 1) until len)
.map { thatIdx =>
doSomething(thisElem, vector(thatIdx))
}
}
// Or
val len = vector.length
for {
thisIdx <- 0 until len
thisElem = vector(thisIdx)
thatIdx <- (thisIdx + 1) until len
thatElem = vector(thatIdx)
} yield doSomething(thisElem, thatElem)
If you really need to, you can generalize either version of this code to all IndexedSeqs, by using some implicit CanBuildFrom parameters, but I won't cover that.
If I have a Seq, I can map over it.
val ss = Seq("1", "2", "3")
println(ss.map(s => s.toInt)) // List(1, 2, 3)
But sometimes, the function that you pass to map can fail.
val ss = Seq("1", "2", "c")
println(ss.map(s => try { Success(s.toInt) } catch { case e: Throwable => Failure(e) })) // List(Success(1), Success(2), Failure(java.lang.NumberFormatException: For input string: "c"))
This last one will return a Seq[Try[Int]]. What I really want though is a Try[Seq[Int]], where if any one of the mapping is a Failure, it stops the iteration and returns the Failure instead. If there is no error, I want it to just return all the converted elements, unpacked from the Try.
What is the idiomatic Scala way to do this?
You may be overthinking this. The anonymous function in your map is essentially the same as Try.apply. If you want to end up with Try[Seq[Int]] then you can wrap the Seq in Try.apply and map within:
scala> val ss = Try(Seq("1", "2", "c").map(_.toInt))
ss: scala.util.Try[Seq[Int]] = Failure(java.lang.NumberFormatException: For input string: "c")
If any of the toInts fails, it will throw an exception and stop executing, and become a Failure.
Not sure it's idiomatic, but I would do something like this:
import util.{Try, Success, Failure}
import collection.mutable.ListBuffer
def toInt(s: String) =
// Correct usage would be Try(s.toInt)
try {
Success(s.toInt)
}
catch {
case e: Throwable => Failure(e)
}
def convert[A](ss: Seq[String], f: String => Try[A]) = {
ss.foldLeft(Try(ListBuffer[A]())) {
case (a, s) =>
for {
xs <- a
x <- f(s)
}
yield xs :+ x
}.map(_.toSeq)
}
scala> convert(List("1", "2"), toInt)
scala.util.Try[Seq[Int]] = Success(List(1, 2))
scala> convert(List("1", "c"), toInt)
scala.util.Try[Seq[Int]] = Failure(java.lang.NumberFormatException: For input string: "c")
If you really want to exit early instead of skipping elements you can use good old recursion:
def convert[A](ss: Seq[String], f: String => Try[A]) = {
#annotation.tailrec
def loop(ss: Seq[String], acc: ListBuffer[A]): Try[Seq[A]] = {
ss match {
case h::t =>
f(h) match {
case Success(x) => loop(t, acc :+ x)
case Failure(e) => Failure(e)
}
case Nil =>
Success(acc.toSeq)
}
}
loop(ss, ListBuffer[A]())
}
Given a sequence of elements and a predicate p, I would like to produce a sequence of sequences such that, in each subsequence, either all elements satisfy p or the sequence has length 1. Additionally, calling .flatten on the result should give me back my original sequence (so no re-ordering of elements).
For instance, given:
val l = List(2, 4, -6, 3, 1, 8, 7, 10, 0)
val p = (i : Int) => i % 2 == 0
I would like magic(l,p) to produce:
List(List(2, 4, -6), List(3), List(1), List(8), List(7), List(10, 0))
I know of .span, but that method stops the first time it encounters a value that doesn't satisfy p and just returns a pair.
Below is a candidate implementation. It does what I want, but, well, makes we want to cry. I would love for someone to come up with something slightly more idiomatic.
def magic[T](elems : Seq[T], p : T=>Boolean) : Seq[Seq[T]] = {
val loop = elems.foldLeft[(Boolean,Seq[Seq[T]])]((false,Seq.empty)) { (pr,e) =>
val (lastOK,s) = pr
if(lastOK && p(e)) {
(true, s.init :+ (s.last :+ e))
} else {
(p(e), s :+ Seq(e))
}
}
loop._2
}
(Note that I do not particularly care about preserving the actual type of the Seq.)
I would not use foldLeft. It's just a simple recursion of span with a special rule if the head doesn't match the predicate:
def magic[T](elems: Seq[T], p: T => Boolean): Seq[Seq[T]] =
elems match {
case Seq() => Seq()
case Seq(head, tail # _*) if !p(head) => Seq(head) +: magic(tail, p)
case xs =>
val (prefix, rest) = xs span p
prefix +: magic(rest, p)
}
You could also do it tail-recursive, but you need to remember to reverse the output if you're prepending (as is sensible):
def magic[T](elems: Seq[T], p: T => Boolean): Seq[Seq[T]] = {
def iter(elems: Seq[T], out: Seq[Seq[T]]) : Seq[Seq[T]] =
elems match {
case Seq() => out.reverse
case Seq(head, tail # _*) if !p(head) => iter(tail, Seq(head) +: out)
case xs =>
val (prefix, rest) = xs span p
iter(rest, prefix +: out)
}
iter(elems, Seq())
}
For this task you can use takeWhile and drop combined with a little pattern matching an recursion:
def magic[T](elems : Seq[T], p : T=>Boolean) : Seq[Seq[T]] = {
def magic(elems: Seq[T], result: Seq[Seq[T]]): Seq[Seq[T]] = elems.takeWhile(p) match {
// if elems is Nil, we have a result
case Nil if elems.isEmpty => result
// if it's not, but we don't get any values from takeWhile, we take a single elem
case Nil => magic(elems.tail, result :+ Seq(elems.head))
// takeWhile gave us something, so we add it to the result
// and drop as many elements from elems, as takeWhile gave us
case xs => magic(elems.drop(xs.size), result :+ xs)
}
magic(elems, Seq())
}
Another solution using a fold:
def magicFilter[T](seq: Seq[T], p: T => Boolean): Seq[Seq[T]] = {
val (filtered, current) = (seq foldLeft (Seq[Seq[T]](), Seq[T]())) {
case ((filtered, current), element) if p(element) => (filtered, current :+ element)
case ((filtered, current), element) if !current.isEmpty => (filtered :+ current :+ Seq(element), Seq())
case ((filtered, current), element) => (filtered :+ Seq(element), Seq())
}
if (!current.isEmpty) filtered :+ current else filtered
}
I have the following code which recursively operates on each element within a List
def doMatch(list: List[Int]): Unit = list match {
case last :: Nil => println("Final element.")
case head :: tail => println("Recursing..."); doMatch(tail)
}
Now, ignoring that this functionality is available through filter() and foreach(), this works just fine. However, if I try to change it to accept any Seq[Int], I run into problems:
Seq doesn't have ::, but it does have +:, which as I understand is basically the same thing. If I try to match on head +: tail however, the compiler complains 'error: not found: value +:'
Nil is specific to List, and I'm not sure what to replace it with. I'm going to try Seq() if I ever get past the previous problem
Here is how I think the code should look, except it doesn't work:
def doMatch(seq: Seq[Int]): Unit = seq match {
case last +: Seq() => println("Final element.")
case head +: tail => println("Recursing..."); doMatch(tail)
}
Edit: So many good answers! I'm accepting agilesteel's answer as his was the first that noted that :: isn't an operator in my example, but a case class and hence the difference.
As of the ides of March 2012, this works in 2.10+:
def doMatch(seq: Seq[Int]): Unit = seq match {
case last +: Seq() => println("Final element.")
case head +: tail => println("Recursing..."); doMatch(tail)
} //> doMatch: (seq: Seq[Int])Unit
doMatch(List(1, 2)) //> Recursing...
//| Final element.
More generally, two different head/tail and init/last decomposition objects mirroring append/prepend were added for Seq in SeqExtractors:
List(1, 2) match { case init :+ last => last } //> res0: Int = 2
List(1, 2) match { case head +: tail => tail } //> res1: List[Int] = List(2)
Vector(1, 2) match { case init :+ last => last } //> res2: Int = 2
Vector(1, 2) match { case head +: tail => tail } //> res3: scala.collection.immutable.Vector[Int] = Vector(2)
Kind of cheating, but here it goes:
def doMatch(seq: Seq[Int]): Unit = seq match {
case Seq(x) => println("Final element " + x)
case Seq(x, xs#_*) => println("Recursing..." + x); doMatch(xs)
}
Don't ask me why xs* doesn't work...
There are two :: (pronounced cons) in Scala. One is an operator defined in class List and one is a class (subclass of List), which represents a non empty list characterized by a head and a tail.
head :: tail is a constructor pattern, which is syntactically modified from ::(head, tail).
:: is a case class, which means there is an extractor object defined for it.
You can actually define an object for +: to do exactly what you are looking for:
object +: {
def unapply[T](s: Seq[T]) =
if(s.nonEmpty)
Some(s.head, s.tail)
else
None
}
scala> val h +: t = Seq(1,2,3)
h: Int = 1
t: Seq[Int] = List(2, 3)
Then your code works exactly as expected.
This works because h +: t is equivalent to +:(h,t) when used for patten matching.
I don't think there is pattern matching support for arbitrary sequences in the standard library. You could do it with out pattern matching though:
def doMatch(seq: Seq[Int]) {
if (seq.size == 1) println("final element " + seq(0)) else {
println("recursing")
doMatch(seq.tail)
}
}
doMatch(1 to 10)
You can define your own extractor objects though. See http://www.scala-lang.org/node/112
object SEQ {
def unapply[A](s:Seq[A]):Option[(A, Seq[A])] = {
if (s.size == 0) None else {
Some((s.head, s.tail))
}
}
}
def doMatch(seq: Seq[Int]) {
seq match {
case SEQ(head, Seq()) => println("final")
case SEQ(head, tail) => {
println("recursing")
doMatch(tail)
}
}
}
A simple tranformation from Seq to List would do the job:
def doMatch (list: List[Int]): Unit = list match {
case last :: Nil => println ("Final element.")
case head :: tail => println ("Recursing..."); doMatch (tail)
case Nil => println ("only seen for empty lists")
}
def doMatchSeq (seq: Seq[Int]) : Unit = doMatch (seq.toList)
doMatch (List(3, 4, 5))
doMatchSeq (3 to 5)