This question already has answers here:
Iterate Over a tuple
(4 answers)
Closed 8 years ago.
In scala, we can get an iterator over a tuple as follows
val t = (1, 2)
val it = t.productIterator
and even
it.foreach( x => println(x.isInstanceOf[Int]) )
returns true, we cannot do simple operations on the iterator values without using asInstanceOf[Int], since
it.foreach( x => println(x+1) )
returns an error: type mismatch; found : Int(1) required: String
I understand the issue with Integer vs. Int, but still the validity of isInstanceOf[Int] is somewhat confusing.
What is the best way to do these operations over tuples? Notice that the tuple can have a mix of types like integers with doubles, so converting to a list might not always work.
A tuple does not have to be homogenous and the compiler didn't try to apply magic type unification across the elements1. Take (1, "hello") as an example of such a a heterogenous tuple (Tuple2[Int,String]).
This means that x is typed as Any (not Int!). Try it.foreach( (x: Int) => println(x) ), with the original tuple, to get a better error message indicating that the iterator is not unified over the types of the tuple elements (it is an Iterators[Any]). The error reported should be similar to:
error: type mismatch;
found : (Int) => Unit
required: (Any) => ?
(1, 2).productIterator.foreach( (x: Int) => println(x) )
In this particular case isInstanceOf[Int] can be used to refine the type - from the Any that the type-system gave us - because we know, from manual code inspection, that it will "be safe" with the given tuple.
Here is another look at the iterators/types involved:
(1, 2) // -> Tuple2[Int,Int]
.productIterator // -> Iterator[Any]
.map(_.asInstanceOf[Int]) // -> Iterator[Int]
.foreach(x => println(x+1))
While I would recommend treating tuples as finite sets of homogenous elements and not a sequence, the same rules can be used as when dealing with any Iterator[Any] such as using pattern matching (e.g. match) that discriminates by the actual object type. (In this case the code is using an implicit PartialFunction.)
(1, "hello").productIterator
.foreach {
case s: String => println("string: " + s)
case i: Int => println("int: " + i)
}
1 While it might be possible to make the compiler unify the types in this scenario, it sounds like a special case that requires extra work for minimal gain. Normally sequences like lists - not tuples - are used for homogenous elements and the compiler/type-system correctly gives us a good refinement for something like List(1,2) (which is typed as List[Int] as expected).
There is another type HList, that is like tuple and List all-in-one. See shapeless.
I think, you can get close to what you want:
import shapeless._
val t = 1 :: 2 :: HNil
val lst = t.toList
lst.foreach( x => println(x+1) )
Related
In the below two scenarios, I have flatMap function invoked on a list. In both the cases, the map portion of the flatMap function returns an array which has a iterator. In the first case, the code errors out where in the second case, it produces the expected result.
Scenario-1
val x = List("abc","cde")
x flatMap ( e => e.toArray)
<console>:13: error: polymorphic expression cannot be instantiated to expected type;
found : [B >: Char]Array[B]
required: scala.collection.GenTraversableOnce[?]
x flatMap ( e => e.toArray)
Scenario-2
val x = List("abc,def")
x flatMap ( e => e.split(",") )
res1: List[String] = List(abc, def) //Result
Can you please help why in the first case, it is not behaving as expected ?
I think the difference is that in scenario 1 you actually have a Array[B] where B is some yet-to-be-decided supertype of Char. Normally the compiler would look for an implicit conversion to GenTraversableOnce but because B is not yet known you run into a type inference issue / limitation.
You can help type inference by filling in B.
List("abc", "cde").flatMap(_.toArray[Char])
Or even better, you don't need flatMap in this case. Just call flatten.
List("abc", "cde").flatten
It is worth keeping in mind that Array is not a proper Scala collection so compiler has to work harder to first convert it to something that fits with the rest of Scala collections
implicitly[Array[Char] <:< GenTraversableOnce[Char]] // error (Scala 2.12)
implicitly[Array[Char] <:< IterableOnce[Char]] // error (Scala 2.13)
Hence because flatMap takes a function
String => GenTraversableOnce[Char]
but we are passing in
String => Array[Char]
the compiler first has to find an appropriate implicit conversion of Array[Char] to GenTraversableOnce[Char]. Namely this should be wrapCharArray
scala.collection.immutable.List.apply[String]("abc", "cde").flatMap[Char](((e: String) => scala.Predef.wrapCharArray(scala.Predef.augmentString(e).toArray[Char]((ClassTag.Char: scala.reflect.ClassTag[Char])))))
or shorter
List("abc", "cde").flatMap(e => wrapCharArray(augmentString(e).toArray))
however due to type inference reasons explained by Jasper-M, the compiler is unable to select wrapCharArray conversion. As Daniel puts it
Array tries to be a Java Array and a Scala Collection at the same
time. It mostly succeeds, but fail at both for some corner cases.
Perhaps this is one of those corner cases. For these reasons it is best to avoid Array unless performance or compatibility reasons dictate it. Nevertheless another approach that seems to work in Scala 2.13 is to use to(Collection) method
List("abc","cde").flatMap(_.to(Array))
scala> val x = List("abc","cde")
x: List[String] = List(abc, cde)
scala> x.flatMap[Char](_.toArray)
res0: List[Char] = List(a, b, c, c, d, e)
As the error mentions, your type is wrong.
In the first case, if you appy map not flatmap, you obtain List[Array[Char]]. If you apply flatten to that, you get a List[Char]
In the second case, if you appy map not flatmap, you obtain List[Array[String]]. If you apply flatten to that, you get a List[String]
I suppose you need to transform String to Char in your Array, in order to make it work.
I use Scala 2.13 and still have the same error.
Simple question, and sorry if this is a stupid question as I am just beginning in scala. I am getting a type mismatch error that says:
found : (AnyRef, org.apache.tinkerpop.gremlin.hadoop.structure.io.VertexWritable) => List[Object]
required: ((AnyRef, org.apache.tinkerpop.gremlin.hadoop.structure.io.VertexWritable)) => scala.collection.GenTraversableOnce[?]
But according to this post (I have a Scala List, how can I get a TraversableOnce?), a scala.collection.immutable.List is an Iterable and therefore also a GenTraversableOnce. And yet this error seems to indicate otherwise. And furthermore, when I actually look at the link in the accepted answer of that post, I don't see any reference to the word "traversable".
If the problem has to do with my inner class not being correct, then I have to say this error is extremely uninformative, since requiring that the inner class be of type "?" is obviously a vacuous statement ... Any help in understanding this would be appreciated.
Function2[X, Y, Z] is not the same thing as Function1[(X, Y), Z].
Compare these two definitions:
val f: ((Int, Int)) => Int = xy => xy._1 + xy._2
val f: (Int, Int) => Int = (x, y) => x + y
The first could also be written with a pattern-matching, that first decomposes the tuple:
val f: ((Int, Int)) => Int = { case (x, y) => x + y }
This is exactly what the error message asks you to do: provide an unary function that takes a tuple as argument, not a binary function. Note that there is the tupled-method, that does exactly that.
The return types of the functions are mostly irrelevant here, the compiler doesn't get to unify them, because it fails on the types of the inputs.
Also related:
Same story with eta-expansions: Why does my implementation of Haskell snd not compile in Scala
Why Line 1 and 2 works, but 3 fail ?¿:
val sum1= (a: Int, b: Int, c: Int) => a + b + c //OK
List(1,2,3).reduceLeft(_+_) //OK
val sum2 =(x: List) =>x.reduceLeft(_+_) //Fail
You have to add the element type to x: List, so it becomes List[Int] or List[Double]. List itself is a raw type, which is illegal in Scala. Without the type annotation, the compiler also does not know what the + operator means in the reduceLeft(_+_) part, so it has to produce an error.
The compiler is missing proof that the underlying objects inside List are a type that defines the + operator. Here's a nice way to use the underlying Scala lib to define a method that is capable of adding a List of any numerical type.
For this, you don't even need reduce, as Scala already defines sum. List is a higher kinded type construcotr, more details here.
def addList[T : Numeric](list: List[T]): T = list.sum
If I create a map:
val m = Map((4, 3))
And try to add a new key value pair:
val m_prime = m + (1, 5)
I get:
error: type mismatch;
found : Int(1)
required: (Int, ?)
val m_prime = m + (1, 5)
If I do:
val m_prime = m + ((1, 5))
Or:
val m_prime = m + (1 -> 5)
Then it works. Why doesn't the compiler accept the first example?
I am using 2.10.2
This is indeed very annoying (I run into this frequently). First of all, the + method comes from a general collection trait, taking only one argument—the collection's element type. Map's element type is the pair (A, B). However, Scala interprets the parentheses here as method call parentheses, not a tuple constructor. The explanation is shown in the next section.
To solve this, you can either avoid tuple syntax and use the arrow association key -> value instead, or use double parentheses, or use method updated which is specific to Map. updated does the same as + but takes key and value as separate arguments:
val m_prime = m updated (1, 5)
Still it is unclear why Scala fails here, as in general infix syntax should work and not expect parentheses. It appears that this particular case is broken because of a method overloading: There is a second + method that takes a variable number of tuple arguments.
Demonstration:
trait Foo {
def +(tup: (Int, Int)): Foo
}
def test1(f: Foo) = f + (1, 2) // yes, it works!
trait Baz extends Foo {
def +(tups: (Int, Int)*): Foo // overloaded
}
def test2(b: Baz) = b + (1, 2) // boom. we broke it.
My interpretation is that with the vararg version added, there is now an ambiguity: Is (a, b) a Tuple2 or a list of two arguments a and b (even if a and b are not of type Tuple2, perhaps the compiler would start looking for an implicit conversion). The only way to resolve the ambiguity is to use either of the three approaches described above.
I have learned the basic difference between foldLeft and reduceLeft
foldLeft:
initial value has to be passed
reduceLeft:
takes first element of the collection as initial value
throws exception if collection is empty
Is there any other difference ?
Any specific reason to have two methods with similar functionality?
Few things to mention here, before giving the actual answer:
Your question doesn't have anything to do with left, it's rather about the difference between reducing and folding
The difference is not the implementation at all, just look at the signatures.
The question doesn't have anything to do with Scala in particular, it's rather about the two concepts of functional programming.
Back to your question:
Here is the signature of foldLeft (could also have been foldRight for the point I'm going to make):
def foldLeft [B] (z: B)(f: (B, A) => B): B
And here is the signature of reduceLeft (again the direction doesn't matter here)
def reduceLeft [B >: A] (f: (B, A) => B): B
These two look very similar and thus caused the confusion. reduceLeft is a special case of foldLeft (which by the way means that you sometimes can express the same thing by using either of them).
When you call reduceLeft say on a List[Int] it will literally reduce the whole list of integers into a single value, which is going to be of type Int (or a supertype of Int, hence [B >: A]).
When you call foldLeft say on a List[Int] it will fold the whole list (imagine rolling a piece of paper) into a single value, but this value doesn't have to be even related to Int (hence [B]).
Here is an example:
def listWithSum(numbers: List[Int]) = numbers.foldLeft((List.empty[Int], 0)) {
(resultingTuple, currentInteger) =>
(currentInteger :: resultingTuple._1, currentInteger + resultingTuple._2)
}
This method takes a List[Int] and returns a Tuple2[List[Int], Int] or (List[Int], Int). It calculates the sum and returns a tuple with a list of integers and it's sum. By the way the list is returned backwards, because we used foldLeft instead of foldRight.
Watch One Fold to rule them all for a more in depth explanation.
reduceLeft is just a convenience method. It is equivalent to
list.tail.foldLeft(list.head)(_)
foldLeft is more generic, you can use it to produce something completely different than what you originally put in. Whereas reduceLeft can only produce an end result of the same type or super type of the collection type. For example:
List(1,3,5).foldLeft(0) { _ + _ }
List(1,3,5).foldLeft(List[String]()) { (a, b) => b.toString :: a }
The foldLeft will apply the closure with the last folded result (first time using initial value) and the next value.
reduceLeft on the other hand will first combine two values from the list and apply those to the closure. Next it will combine the rest of the values with the cumulative result. See:
List(1,3,5).reduceLeft { (a, b) => println("a " + a + ", b " + b); a + b }
If the list is empty foldLeft can present the initial value as a legal result. reduceLeft on the other hand does not have a legal value if it can't find at least one value in the list.
For reference, reduceLeft will error if applied to an empty container with the following error.
java.lang.UnsupportedOperationException: empty.reduceLeft
Reworking the code to use
myList foldLeft(List[String]()) {(a,b) => a+b}
is one potential option. Another is to use the reduceLeftOption variant which returns an Option wrapped result.
myList reduceLeftOption {(a,b) => a+b} match {
case None => // handle no result as necessary
case Some(v) => println(v)
}
The basic reason they are both in Scala standard library is probably because they are both in Haskell standard library (called foldl and foldl1). If reduceLeft wasn't, it would quite often be defined as a convenience method in different projects.
From Functional Programming Principles in Scala (Martin Odersky):
The function reduceLeft is defined in terms of a more general function, foldLeft.
foldLeft is like reduceLeft but takes an accumulator z, as an additional parameter, which is returned when foldLeft is called on an empty list:
(List (x1, ..., xn) foldLeft z)(op) = (...(z op x1) op ...) op x
[as opposed to reduceLeft, which throws an exception when called on an empty list.]
The course (see lecture 5.5) provides abstract definitions of these functions, which illustrates their differences, although they are very similar in their use of pattern matching and recursion.
abstract class List[T] { ...
def reduceLeft(op: (T,T)=>T) : T = this match{
case Nil => throw new Error("Nil.reduceLeft")
case x :: xs => (xs foldLeft x)(op)
}
def foldLeft[U](z: U)(op: (U,T)=>U): U = this match{
case Nil => z
case x :: xs => (xs foldLeft op(z, x))(op)
}
}
Note that foldLeft returns a value of type U, which is not necessarily the same type as List[T], but reduceLeft returns a value of the same type as the list).
To really understand what are you doing with fold/reduce,
check this: http://wiki.tcl.tk/17983
very good explanation. once you get the concept of fold,
reduce will come together with the answer above:
list.tail.foldLeft(list.head)(_)
Scala 2.13.3, Demo:
val names = List("Foo", "Bar")
println("ReduceLeft: "+ names.reduceLeft(_+_))
println("ReduceRight: "+ names.reduceRight(_+_))
println("Fold: "+ names.fold("Other")(_+_))
println("FoldLeft: "+ names.foldLeft("Other")(_+_))
println("FoldRight: "+ names.foldRight("Other")(_+_))
outputs:
ReduceLeft: FooBar
ReduceRight: FooBar
Fold: OtherFooBar
FoldLeft: OtherFooBar
FoldRight: FooBarOther