I am writing a Scala program in which there is an operation that creates a sequence. The operation might fail, so I enclose it inside a Try. I want to do sequence creation and enumeration inside a for comprehension, so that a successfully-created sequence yields a sequence of tuples where the first element is the sequence and the second is an element of it.
To simplify the problem, make my sequence a Range of integers and define a createRange function that fails if it is asked to create a range of an odd length. Here is a simple for comprehension that does what I want.
import scala.util.Try
def createRange(n: Int): Try[Range] = {
Try {
if (n % 2 == 1) throw new Exception
else Range(0, n)
}
}
def rangeElements(n: Int) {
for {
r <- createRange(n)
x <- r
} println(s"$r\t$x")
}
def main(args: Array[String]) {
println("Range length 3")
rangeElements(3)
println("Range length 4")
rangeElements(4)
}
If you run this it correctly prints.
Range length 3
Range length 4
Range(0, 1, 2, 3) 0
Range(0, 1, 2, 3) 1
Range(0, 1, 2, 3) 2
Range(0, 1, 2, 3) 3
Now I would like to rewrite my rangeElements function so that instead of printing as a side-effect it returns a sequence of integers, where the sequence is empty if the range was not created. What I want to write is this.
def rangeElements(n: Int):Seq[(Range,Int)] = {
for {
r <- createRange(n)
x <- r
} yield (r, x)
}
// rangeElements(3) returns an empty sequence
// rangeElements(4) returns the sequence (Range(0,1,2,3), 0), (Range(0,1,2,3), 1) etc.
This gives me two type mismatch compiler errors. The r <- createRange(n) line required Seq[Int] but found scala.util.Try[Nothing]. The x <- r line required scala.util.Try[?] but found scala.collection.immutable.IndexedSeq[Int].
Presumably there is some type erasure with the Try that is messing me up, but I can't figure out what it is. I've tried various toOption and toSeq qualifiers on the lines in the for comprehension to no avail.
If I only needed to yield the range elements I could explicitly handle the Success and Failure conditions of createRange myself as suggested by the first two answers below. However, I need access to both the range and its individual elements.
I realize this is a strange-sounding example. The real problem I am trying to solve is a complicated recursive search, but I don't want to add in all its details because that would just confuse the issue here.
How do I write rangeElements to yield the desired sequences?
The problem becomes clear if you translate the for comprehension to its map/flatMap implementation (as described in the Scala Language Spec 6.19). The flatMap has the result type Try[U] but your function expects Seq[Int].
for {
r <- createRange(n)
x <- r
} yield x
createRange(n).flatMap {
case r => r.map {
case x => x
}
}
Is there any reason why you don't use the getOrElse method?
def rangeElements(n: Int):Seq[Int] =
createRange(n) getOrElse Seq.empty
The Try will be Success with a Range when n is even or a Failure with an Exception when n is odd. In rangeElements match and extract those values. Success will contain the valid Range and Failure will contain the Exception. Instead of returning the Exception return an empty Seq.
import scala.util.{Try, Success, Failure}
def createRange(n: Int): Try[Range] = {
Try {
if (n % 2 == 1) throw new Exception
else Range(0, n)
}
}
def rangeElements(n: Int):Seq[Tuple2[Range, Int]] = createRange(n) match {
case Success(s) => s.map(xs => (s, xs))
case Failure(f) => Seq()
}
scala> rangeElements(3)
res35: Seq[(Range, Int)] = List()
scala> rangeElements(4)
res36: Seq[(Range, Int)] = Vector((Range(0, 1, 2, 3),0), (Range(0, 1, 2, 3),1), (Range(0, 1, 2, 3),2), (Range(0, 1, 2,3),3))
Related
Suppose that I use a sequence of various maps and/or flatMaps to generate a sequence of collections. Is it possible to access information about the "current" collection from within any of those methods? For example, without knowing anything specific about the functions used in the previous maps or flatMaps, and without using any intermediate declarations, how can I get the maximum value (or length, or first element, etc.) of the collection upon which the last map acts?
List(1, 2, 3)
.flatMap(x => f(x) /* some unknown function */)
.map(x => x + ??? /* what is the max element of the collection? */)
Edit for clarification:
In the example, I'm not looking for the max (or whatever) of the initial List. I'm looking for the max of the collection after the flatMap has been applied.
By "without using any intermediate declarations" I mean that I do not want to use any temporary collections en route to the final result. So, the example by Steve Waldman below, while giving the desired result, is not what I am seeking. (I include this condition is mostly for aesthetic reasons.)
Edit for clarification, part 2:
The ideal solution would be some magic keyword or syntactic sugar that lets me reference the current collection:
List(1, 2, 3)
.flatMap(x => f(x))
.map(x => x + theCurrentList.max)
I'm prepared to accept the fact, however, that this simply is not possible.
Maybe just define the list as a val, so you can name it? I don't know of any facility built into map(...) or flatMap(...) that would help.
val myList = List(1, 2, 3)
myList
.flatMap(x => f(x) /* some unknown function */)
.map(x => x + myList.max /* what is the max element of the List? */)
Update: By this approach at least, if you have multiple transformations and want to see the transformed version, you'd have to name that. You could get away with
val myList = List(1, 2, 3).flatMap(x => f(x) /* some unknown function */)
myList.map(x => x + myList.max /* what is the max element of the List? */)
Or, if there will be multiple transformations, get in the habit of naming the stages.
val rawList = List(1, 2, 3)
val smordified = rawList.flatMap(x => f(x) /* some unknown function */)
val maxified = smordified.map(x => x + smordified.max /* what is the max element of the List? */)
maxified
Update 2: Watch it work in the REPL even with heterogenous types:
scala> def f( x : Int ) : Vector[Double] = Vector(x * math.random, x * math.random )
f: (x: Int)Vector[Double]
scala> val rawList = List(1, 2, 3)
rawList: List[Int] = List(1, 2, 3)
scala> val smordified = rawList.flatMap(x => f(x) /* some unknown function */)
smordified: List[Double] = List(0.40730853571901315, 0.15151641399798665, 1.5305929709857609, 0.35211231420067435, 0.644241939254793, 0.15530230501048903)
scala> val maxified = smordified.map(x => x + smordified.max /* what is the max element of the List? */)
maxified: List[Double] = List(1.937901506704774, 1.6821093849837476, 3.0611859419715217, 1.8827052851864352, 2.1748349102405538, 1.6858952759962498)
scala> maxified
res3: List[Double] = List(1.937901506704774, 1.6821093849837476, 3.0611859419715217, 1.8827052851864352, 2.1748349102405538, 1.6858952759962498)
It is possible, but not pretty, and not likely something you want if you are doing it for "aesthetic reasons."
import scala.math.max
def f(x: Int): Seq[Int] = ???
List(1, 2, 3).
flatMap(x => f(x) /* some unknown function */).
foldRight((List[Int](),List[Int]())) {
case (x, (xs, Nil)) => ((x :: xs), List.fill(xs.size + 1)(x))
case (x, (xs, xMax :: _)) => ((x :: xs), List.fill(xs.size + 1)(max(x, xMax)))
}.
zipped.
map {
case (x, xMax) => x + xMax
}
// Or alternately, a slightly more efficient version using Streams.
List(1, 2, 3).
flatMap(x => f(x) /* some unknown function */).
foldRight((List[Int](),Stream[Int]())) {
case (x, (xs, Stream())) =>
((x :: xs), Stream.continually(x))
case (x, (xs, curXMax #:: _)) =>
val newXMax = max(x, curXMax)
((x :: xs), Stream.continually(newXMax))
}.
zipped.
map {
case (x, xMax) => x + xMax
}
Seriously though, I just took this on to see if I could do it. While the code didn't turn out as bad as I expected, I still don't think it's particularly readable. I'd discourage using this over something similar to Steve Waldman's answer. Sometimes, it's simply better to just introduce a val, rather than being dogmatic about it.
You could define a mapWithSelf (resp. flatMapWithSelf) operation along these lines and add it as an implicit enrichment to the collection. For List it might look like:
// Scala 2.13 APIs
object Enrichments {
implicit class WithSelfOps[A](val lst: List[A]) extends AnyVal {
def mapWithSelf[B](f: (A, List[A]) => B): List[B] =
lst.map(f(_, lst))
def flatMapWithSelf[B](f: (A, List[A]) => IterableOnce[B]): List[B] =
lst.flatMap(f(_, lst))
}
}
The enrichment basically fixes the value of the collection before the operation and threads it through. It should be possible to generify this (at least for the strict collections), though it would look a little different in 2.12 vs. 2.13+.
Usage would look like
import Enrichments._
val someF: Int => IterableOnce[Int] = ???
List(1, 2, 3)
.flatMap(someF)
.mapWithSelf { (x, lst) =>
x + lst.max
}
So at the usage site, it's aesthetically pleasant. Note that if you're computing something which traverses the list, you'll be traversing the list every time (leading to a quadratic runtime). You can get around that with some mutability or by just saving the intermediate list after the flatMap.
One somewhat-simple way of referencing prior output within the current map/collect operation is to use a named reference outside the map, then reference it from within the map block:
var prevOutput = ... // starting value of whatever is referenced within the map
myValues.map {
prevOutput = ... // expression that references prior `prevOutput`
prevOutput // return above computed value for the map to collect
}
This draws attention to the fact that we're referencing prior elements while building the new sequence.
This would be more messy, though, if you wanted to reference arbitrarily previous values, not just the previous one.
I'm trying to print out all the factors of every number in a list.
Here is my code:
def main(args: Array[String])
{
val list_of_numbers = List(1,4,6)
def get_factors(list_of_numbers:List[Int]) : Int =
{
return list_of_numbers.foreach{(1 to _).filter {divisor => _ % divisor == 0}}
}
println(get_factors(list_of_numbers));
}
I want the end result to contain a single list that will hold all the numbers which are factors of any of the numbers in the list. So the final result should be (1,2,3,4,6). Right now, I get the following error:
error: missing parameter type for expanded function ((x$1) => 1.to(x$1))
return list_of_numbers.foreach{(1 to _).filter {divisor => _ % divisor == 0}}
How can I fix this?
You can only use _ shorthand once in a function (except for some special cases), and even then not always.
Try spelling it out instead:
list_of_numbers.foreach { n =>
(1 to n).filter { divisor => n % divisor == 0 }
}
This will compile.
There are other problems with your code though.
foreach returns a Unit, but you are requiring an Int for example.
Perhaps, you wanted a .map rather than .foreach, but that would still be a List, not an Int.
A few things are wrong here.
First, foreach takes a function A => Unit as an argument, meaning that it's really just for causing side effects.
Second your use of _, you can use _ when the function uses each argument once.
Lastly your expected output seems to be getting rid of duplicates (1 is a factor for all 3 inputs, but it only appears once).
list_of_numbers flatMap { i => (1 to i) filter {i % _ == 0 }} distinct
will do what you are looking for.
flatMap takes a function from A => List[B] and produces a simple List[B] as output, list.distinct gets rid of the duplicates.
Actually, there are several problems with your code.
First, foreach is a method which yields Unit (like void in Java). You want to yield something so you should use a for comprehension.
Second, in your divisor-test function, you've specified both the unnamed parameter ("_") and the named parameter (divisor).
The third problem is that you expect the result to be Int (in the code) but List[Int] in your description.
The following code will do what you want (although it will repeat factors, so you might want to pass it through distinct before using the result):
def main(args: Array[String]) {
val list_of_numbers = List(1, 4, 6)
def get_factors(list_of_numbers: List[Int]) = for (n <- list_of_numbers; r = 1 to n; f <- r.filter(n%_ == 0)) yield f
println(get_factors(list_of_numbers))
}
Note that you need two generators ("<-") in the for comprehension in order that you end up with simply a List. If you instead implemented the filter part in the yield expression, you would get a List[List[Int]].
I wanted to write it functionally, and the best I could do was:
list.zipWithIndex.filter((tt:Tuple2[Thing,Int])=>(tt._2%3==0)).unzip._1
to get elements 0, 3, 6,...
Is there a more readable Scala idiom for this?
If efficiency is not an issue, you could do the following:
list.grouped(3).map(_.head)
Note that this constructs intermediate lists.
Alternatively you can use a for-comprehension:
for {
(x,i) <- list zipWithIndex
if i % 3 == 0
} yield x
This is of course almost identical to your original solution, just written differently.
My last alternative for you is the use of collect on the zipped list:
list.zipWithIndex.collect {
case (x,i) if i % 3 == 0 => x
}
Not much clear, but still:
xs.indices.collect { case i if i % 3 == 0 => xs(i) }
A nice, functional solution, without creating temporary vectors, lists, and so on:
def everyNth[T](xs: List[T], n:Int): List[T] = xs match {
case hd::tl => hd::everyNth(tl.drop(n-1), n)
case Nil => Nil
}
Clojure has a take-nth function that does what you want, but I was surprised to find that there's not an equivalent method in Scala. You could code up a similar recursive solution based off the Clojure code, or you could read this blog post:
Scala collections: Filtering each n-th element
The author actually has a nice graph at the end showing the relative performance of each of his solutions.
I would do it like in Octave mathematical program.
val indices = 0 until n by 3 // Range 0,3,6,9 ...
and then I needed some way to select the indices from a collection. Obviously I had to have a collection with random-access O(1). Like Array or Vector. For example here I use Vector. To wrap the access into a nice DSL I'd add an implicit class:
implicit class VectorEnrichedWithIndices[T](v:Vector[T]) {
def apply(indices:TraversableOnce[Int]):Vector[T] = {
// some implementation
indices.toVector.map(v)
}
}
The usage would look like:
val vector = list.toVector
val every3rdElement = vector(0 until vector.size by 3)
Ah, how about this?
val l = List(10,9,8,7,6,5,4,3,2,1,0)
for (i <- (0 to l.size - 1 by 3).toList) yield l(i)
//res0: List[Int] = List(10, 7, 4, 1)
which can be made more general by
def seqByN[A](xs: Seq[A], n: Int): Seq[A] = for (i <- 0 to xs.size - 1 by n) yield xs(i)
scala> seqByN(List(10,9,8,7,6,5,4,3,2,1,0), 3)
res1: Seq[Int] = Vector(10,7,4,1)
scala> seqByN(List(10,9,8,7,6,5,4,3,2,1,0), 3).toList
res2: Seq[Int] = List(10,7,4,1)
scala> seqByN(List[Int](), 3)
res1: Seq[Int] = Vector()
But by functional do you mean only using the various List combinator functions? Otherwise, are Streams functional enough?
def fromByN[A](xs: List[A], n: Int): Stream[A] = if (xs.isEmpty) Stream.empty else
xs.head #:: fromByN(xs drop n, n)
scala> fromByN(List(10,9,8,7,6,5,4,3,2,1,0), 3).toList
res17: List[Int] = List(10, 7, 4, 1)
So this might not be the best way to tackle it but my initial thought was a for expression.
Say I have a List like
List(List('a','b','c'),List('d','e','f'),List('h','i','j'))
I would like to find the row and column for a character, say 'e'.
def findChar(letter: Char, list: List[List[Char]]): (Int, Int) =
for {
r <- (0 until list.length)
c <- (0 until list(r).length)
if list(r)(c) == letter
} yield (r, c)
If there is a more elegant way I'm all ears but I would also like to understand what's wrong with this. Specifically the error the compiler gives me here is
type mismatch; found : scala.collection.immutable.IndexedSeq[(Int, Int)] required: (Int, Int)
on the line assigning to r. It seems to be complaining that my iterator doesn't match the return type but I don't quite understand why this is or what to do about it ...
In the signature of findChar you are telling the compiler that it returns (Int, Int). However, the result of your for expression (as inferred by Scala) is IndexedSeq[(Int, Int)] as the error message indicates. The reason is that (r, c) after yield is produced for every "iteration" in the for expression (i.e., you are generating a sequence of results, not just a single result).
EDIT: As for findChar, you could do:
def findChar(letter: Char, list: List[List[Char]]) = {
val r = list.indexWhere(_ contains letter)
val c = list(r).indexOf(letter)
(r, c)
}
It is not the most efficient solution, but relatively short.
EDIT: Or reuse your original idea:
def findAll(letter: Char, list: List[List[Char]]) =
for {
r <- 0 until list.length
c <- 0 until list(r).length
if list(r)(c) == letter
} yield (r, c)
def findChar(c: Char, xs: List[List[Char]]) = findAll(c, xs).head
In both cases, be aware that an exception occurs if the searched letter is not contained in the input list.
EDIT: Or you write a recursive function yourself, like:
def findPos[A](c: A, list: List[List[A]]) = {
def aux(i: Int, xss: List[List[A]]) : Option[(Int, Int)] = xss match {
case Nil => None
case xs :: xss =>
val j = xs indexOf c
if (j < 0) aux(i + 1, xss)
else Some((i, j))
}
aux(0, list)
}
where aux is a (locally defined) auxiliary function that does the actual recursion (and remembers in which sublist we are, the index i). In this implementation a result of None indicates that the searched element was not there, whereas a successful result might return something like Some((1, 1)).
For your other ear, the question duplicates
How to capture inner matched value in indexWhere vector expression?
scala> List(List('a','b','c'),List('d','e','f'),List('h','i','j'))
res0: List[List[Char]] = List(List(a, b, c), List(d, e, f), List(h, i, j))
scala> .map(_ indexOf 'e').zipWithIndex.find(_._1 > -1)
res1: Option[(Int, Int)] = Some((1,1))
As far as I understand, the Scala for-comprehension notation relies on the first generator to define how elements are to be combined. Namely, for (i <- list) yield i returns a list and for (i <- set) yield i returns a set.
I was wondering if there was a way to specify how elements are combined independently of the properties of the first generator. For instance, I would like to get "the set of all elements from a given list", or "the sum of all elements from a given set". The only way I have found is to first build a list or a set as prescribed by the for-comprehension notation, then apply a transformation function to it - building a useless data structure in the process.
What I have in mind is a general "algebraic" comprehension notation as it exists for instance in Ateji PX:
`+ { i | int i : set } // the sum of all elements from a given set
set() { i | int i : list } // the set of all elements from a given list
concat(",") { s | String s : list } // string concatenation with a separator symbol
Here the first element (`+, set(), concat(",")) is a so-called "monoid" that defines how elements are combined, independently of the structure of the first generator (there can be multiple generators and filters, I just tried to keep the examples concise).
Any idea about how to achieve a similar result in Scala while keeping a nice and concise notation ? As far as I understand, the for-comprehension notation is hard-wired in the compiler and cannot be upgraded.
Thanks for your feedback.
About the for comprehension
The for comprehension in scala is syntactic sugar for calls to flatMap, filter, map and foreach. In exactly the same way as calls to those methods, the type of the target collection leads to the type of the returned collection. That is:
list map f //is a List
vector map f // is a Vector
This property is one of the underlying design goals of the scala collections library and would be seen as desirable in most situations.
Answering the question
You do not need to construct any intermediate collection of course:
(list.view map (_.prop)).toSet //uses list.view
(list.iterator map (_.prop)).toSet //uses iterator
(for { l <- list.view} yield l.prop).toSet //uses view
(Set.empty[Prop] /: coll) { _ + _.prop } //uses foldLeft
Will all yield Sets without generating unnecessary collections. My personal preference is for the first. In terms of idiomatic scala collection manipulation, each "collection" comes with these methods:
//Conversions
toSeq
toSet
toArray
toList
toIndexedSeq
iterator
toStream
//Strings
mkString
//accumulation
sum
The last is used where the element type of a collection has an implicit Numeric instance in scope; such as:
Set(1, 2, 3, 4).sum //10
Set('a, 'b).sum //does not compile
Note that the String concatenation example in scala looks like:
list.mkString(",")
And in the scalaz FP library might look something like (which uses Monoid to sum Strings):
list.intercalate(",").asMA.sum
Your suggestions do not look anything like Scala; I'm not sure whether they are inspired by another language.
foldLeft? That's what you're describing.
The sum of all elements from a given set:
(0 /: Set(1,2,3))(_ + _)
the set of all elements from a given list
(Set[Int]() /: List(1,2,3,2,1))((acc,x) => acc + x)
String concatenation with a separator symbol:
("" /: List("a", "b"))(_ + _) // (edit - ok concat a bit more verbose:
("" /: List("a", "b"))((acc,x) => acc + (if (acc == "") "" else ",") + x)
You can also force the result type of the for comprehension by explicitly supplying the implicit CanBuildFrom parameter as scala.collection.breakout and specifying the result type.
Consider this REPL session:
scala> val list = List(1, 1, 2, 2, 3, 3)
list: List[Int] = List(1, 1, 2, 2, 3, 3)
scala> val res = for(i <- list) yield i
res: List[Int] = List(1, 1, 2, 2, 3, 3)
scala> val res: Set[Int] = (for(i <- list) yield i)(collection.breakOut)
res: Set[Int] = Set(1, 2, 3)
It results in a type error when not specifying the CanBuildFrom explicitly:
scala> val res: Set[Int] = for(i <- list) yield i
<console>:8: error: type mismatch;
found : List[Int]
required: Set[Int]
val res: Set[Int] = for(i <- list) yield i
^
For a deeper understanding of this I suggest the following read:
http://www.scala-lang.org/docu/files/collections-api/collections-impl.html
If you want to use for comprehensions and still be able to combine your values in some result value you could do the following.
case class WithCollector[B, A](init: B)(p: (B, A) => B) {
var x: B = init
val collect = { (y: A) => { x = p(x, y) } }
def apply(pr: (A => Unit) => Unit) = {
pr(collect)
x
}
}
// Some examples
object Test {
def main(args: Array[String]): Unit = {
// It's still functional
val r1 = WithCollector[Int, Int](0)(_ + _) { collect =>
for (i <- 1 to 10; if i % 2 == 0; j <- 1 to 3) collect(i + j)
}
println(r1) // 120
import collection.mutable.Set
val r2 = WithCollector[Set[Int], Int](Set[Int]())(_ += _) { collect =>
for (i <- 1 to 10; if i % 2 == 0; j <- 1 to 3) collect(i + j)
}
println(r2) // Set(9, 10, 11, 6, 13, 4, 12, 3, 7, 8, 5)
}
}