I was trying to reverse a List of Integers as follows:
List(1,2,3,4).foldLeft(List[Int]()){(a,b) => b::a}
My question is that is there a way to specify the seed to be some List[_] where the _ is the type automatically filled in by scala's type-inference mechanism, instead of having to specify the type as List[Int]?
Thanks
Update: After reading a bit more on Scala's type inference, I found a better answer to your question. This article which is about the limitations of the Scala type inference says:
Type information in Scala flows from function arguments to their results [...], from left to right across argument lists, and from first to last across statements. This is in contrast to a language with full type inference, where (roughly speaking) type information flows unrestricted in all directions.
So the problem is that Scala's type inference is rather limited. It first looks at the first argument list (the list in your case) and then at the second argument list (the function). But it does not go back.
This is why neither this
List(1,2,3,4).foldLeft(Nil){(a,b) => b::a}
nor this
List(1,2,3,4).foldLeft(List()){(a,b) => b::a}
will work. Why? First, the signature of foldLeft is defined as:
foldLeft[B](z: B)(f: (B, A) => B): B
So if you use Nil as the first argument z, the compiler will assign Nil.type to the type parameter B. And if you use List(), the compiler will use List[Nothing] for B.
Now, the type of the second argument f is fully defined. In your case, it's either
(Nil.type, Int) => Nil.type
or
(List[Nothing], Int) => List[Nothing]
And in both cases the lambda expression (a, b) => b :: a is not valid, since its return type is inferred to be List[Int].
Note that the bold part above says "argument lists" and not "arguments". The article later explains:
Type information does not flow from left to right within an argument list, only from left to right across argument lists.
So the situation is even worse if you have a method with a single argument list.
The only way I know how is
scala> def foldList[T](l: List[T]) = l.foldLeft(List[T]()){(a,b) => b::a}
foldList: [T](l: List[T])List[T]
scala> foldList(List(1,2,3,4))
res19: List[Int] = List(4, 3, 2, 1)
scala> foldList(List("a","b","c"))
res20: List[java.lang.String] = List(c, b, a)
Related
In the below two scenarios, I have flatMap function invoked on a list. In both the cases, the map portion of the flatMap function returns an array which has a iterator. In the first case, the code errors out where in the second case, it produces the expected result.
Scenario-1
val x = List("abc","cde")
x flatMap ( e => e.toArray)
<console>:13: error: polymorphic expression cannot be instantiated to expected type;
found : [B >: Char]Array[B]
required: scala.collection.GenTraversableOnce[?]
x flatMap ( e => e.toArray)
Scenario-2
val x = List("abc,def")
x flatMap ( e => e.split(",") )
res1: List[String] = List(abc, def) //Result
Can you please help why in the first case, it is not behaving as expected ?
I think the difference is that in scenario 1 you actually have a Array[B] where B is some yet-to-be-decided supertype of Char. Normally the compiler would look for an implicit conversion to GenTraversableOnce but because B is not yet known you run into a type inference issue / limitation.
You can help type inference by filling in B.
List("abc", "cde").flatMap(_.toArray[Char])
Or even better, you don't need flatMap in this case. Just call flatten.
List("abc", "cde").flatten
It is worth keeping in mind that Array is not a proper Scala collection so compiler has to work harder to first convert it to something that fits with the rest of Scala collections
implicitly[Array[Char] <:< GenTraversableOnce[Char]] // error (Scala 2.12)
implicitly[Array[Char] <:< IterableOnce[Char]] // error (Scala 2.13)
Hence because flatMap takes a function
String => GenTraversableOnce[Char]
but we are passing in
String => Array[Char]
the compiler first has to find an appropriate implicit conversion of Array[Char] to GenTraversableOnce[Char]. Namely this should be wrapCharArray
scala.collection.immutable.List.apply[String]("abc", "cde").flatMap[Char](((e: String) => scala.Predef.wrapCharArray(scala.Predef.augmentString(e).toArray[Char]((ClassTag.Char: scala.reflect.ClassTag[Char])))))
or shorter
List("abc", "cde").flatMap(e => wrapCharArray(augmentString(e).toArray))
however due to type inference reasons explained by Jasper-M, the compiler is unable to select wrapCharArray conversion. As Daniel puts it
Array tries to be a Java Array and a Scala Collection at the same
time. It mostly succeeds, but fail at both for some corner cases.
Perhaps this is one of those corner cases. For these reasons it is best to avoid Array unless performance or compatibility reasons dictate it. Nevertheless another approach that seems to work in Scala 2.13 is to use to(Collection) method
List("abc","cde").flatMap(_.to(Array))
scala> val x = List("abc","cde")
x: List[String] = List(abc, cde)
scala> x.flatMap[Char](_.toArray)
res0: List[Char] = List(a, b, c, c, d, e)
As the error mentions, your type is wrong.
In the first case, if you appy map not flatmap, you obtain List[Array[Char]]. If you apply flatten to that, you get a List[Char]
In the second case, if you appy map not flatmap, you obtain List[Array[String]]. If you apply flatten to that, you get a List[String]
I suppose you need to transform String to Char in your Array, in order to make it work.
I use Scala 2.13 and still have the same error.
I'm reading a bit about Scala currying here and I don't understand this example very much:
def foldLeft[B](z: B)(op: (B, A) => B): B
What is the [B] in square brackets? Why is it in brackets? The B after the colon is the return type right? What is the type?
It looks like this method has 2 parameter lists: one with a parameter named z and one with a parameter named op which is a function.
op looks like it takes a function (B, A) => B). What does the right side mean? It returns B?
And this is apparently how it is used:
val numbers = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
val res = numbers.foldLeft(0)((m, n) => m + n)
print(res) // 55
What is going on? Why wasn't the [B] needed when called?
In Scala documentation that A type (sometimes A1) is often the placeholder for a collection's element type. So if you have...
List('c','q','y').foldLeft( //....etc.
...then A becomes the reference for Char because the list is a List[Char].
The B is a placeholder for a 2nd type that foldLeft will have to deal with. Specifically it is the type of the 1st parameter as well as the type of the foldLeft result. Most of the time you actually don't need to specify it when foldLeft is invoked because the compiler will infer it. So, yeah, it could have been...
numbers.foldLeft[Int](0)((m, n) => m + n)
...but why bother? The compiler knows it's an Int and so does anyone reading it (anyone who knows Scala).
(B, A) => B is a function that takes 2 parameters, one of type B and one of type A, and produces a result of type B.
What is the [B] in square brackets?
A type parameter (that's also known as "generics" if you've seen something like Java before)
Why is it in brackets?
Because type parameters are written in brackets, that's just Scala syntax. Java, C#, Rust, C++ use angle brackets < > for similar purposes, but since arrays in Scala are accessed as arr(idx), and (unlike Haskell or Python) Scala does not use [ ... ] for list comprehensions, the square brackets could be used for type parameters, and there was no need for angular brackets (those are more difficult to parse anyway, especially in a language which allows almost arbitrary names for infix and postfix operators).
The B after the colon is the return type right?
Right.
What is the type?
Ditto. The return type is B.
It looks like this method has 2 parameter lists: one with a parameter named z and one with a parameter named op which is a function.
This method has a type parameter B and two argument lists for value arguments, correct. This is done to simplify the type inference: the type B can be inferred from the first argument z, so it does not have to be repeated when writing down the lambda-expression for op. This wouldn't work if z and op were in the same argument list.
op looks like it takes a function (B, A) => B.
The type of the argument op is (B, A) => B, that is, Function2[B, A, B], a function that takes a B and an A and returns a B.
What does the right side mean? It returns B?
Yes.
What is going on?
m acts as accumulator, n is the current element of the list. The fold starts with integer value 0, and then accumulates from left to right, adding up all numbers. Instead of (m, n) => m + n, you could have written _ + _.
Why wasn't the [B] needed when called?
It was inferred from the type of z. There are many other cases where the generic type cannot be inferred automatically, then you would have to specify the return type explicitly by passing it as an argument in the square brackets.
This is what is called polymorphism. The function can work on multiple types and you sometimes want to give a parameter of what type will be worked with. Basically the B is a type parameter and can either be given explicitly as a type, which would be Int and then it should be given in square brackets or implicitly in parentheses like you did with the 0. Read about polymorphism here Scala polymorphism
I was trying to do some stuff last night around accepting and calling a generic function (i.e. the type is known at the call site, but potentially varies across call sites, so the definition should be generic across arities).
For example, suppose I have a function f: (A, B, C, ...) => Z. (There are actually many such fs, which I do not know in advance, and so I cannot fix the types nor count of A, B, C, ..., Z.)
I'm trying to achieve the following.
How do I call f generically with an instance of (A, B, C, ...)? If the signature of f were known in advance, then I could do something involving Function.tupled f or equivalent.
How do I define another function or method (for example, some object's apply method) with the same signature as f? That is to say, how do I define a g for which g(a, b, c, ...) type checks if and only if f(a, b, c, ...) type checks? I was looking into Shapeless's HList for this. From what I can tell so far, HList at least solves the "representing an arbitrary arity args list" issue, and also, Shapeless would solve the conversion to and from tuple issue. However, I'm still not sure I understand how this would fit in with a function of generic arity, if at all.
How do I define another function or method with a related type signature to f? The biggest example that comes to mind now is some h: (A, B, C, ...) => SomeErrorThing[Z] \/ Z.
I remember watching a conference presentation on Shapeless some time ago. While the presenter did not explicitly demonstrate these things, what they did demonstrate (various techniques around abstracting/genericizing tuples vs HLists) would lead me to believe that similar things as the above are possible with the same tools.
Thanks in advance!
Yes, Shapeless can absolutely help you here. Suppose for example that we want to take a function of arbitrary arity and turn it into a function of the same arity but with the return type wrapped in Option (I think this will hit all three points of your question).
To keep things simple I'll just say the Option is always Some. This takes a pretty dense four lines:
import shapeless._, ops.function._
def wrap[F, I <: HList, O](f: F)(implicit
ftp: FnToProduct.Aux[F, I => O],
ffp: FnFromProduct[I => Option[O]]
): ffp.Out = ffp(i => Some(ftp(f)(i)))
We can show that it works:
scala> wrap((i: Int) => i + 1)
res0: Int => Option[Int] = <function1>
scala> wrap((i: Int, s: String, t: String) => (s * i) + t)
res1: (Int, String, String) => Option[String] = <function3>
scala> res1(3, "foo", "bar")
res2: Option[String] = Some(foofoofoobar)
Note the appropriate static return types. Now for how it works:
The FnToProduct type class provides evidence that some type F is a FunctionN (for some N) that can be converted into a function from some HList to the original output type. The HList function (a Function1, to be precise) is the Out type member of the instance, or the second type parameter of the FnToProduct.Aux helper.
FnFromProduct does the reverse—it's evidence that some F is a Function1 from an HList to some output type that can be converted into a function of some arity to that output type.
In our wrap method, we use FnToProduct.Aux to constrain the Out of the FnToProduct instance for F in such a way that we can refer to the HList parameter list and the O result type in the type of our FnFromProduct instance. The implementation is then pretty straightforward—we just apply the instances in the appropriate places.
This may all seem very complicated, but once you've worked with this kind of generic programming in Scala for a while it becomes more or less intuitive, and we'd of course be happy to answer more specific questions about your use case.
As per the example below, calling xs.toList.map(_.toBuffer) succeeds, but xs.toBuffer.map(_.toBuffer) fails. But when the steps in the latter are performed using an intermediate result, it succeeds. What causes this inconsistency?
scala> "ab-cd".split("-").toBuffer
res0: scala.collection.mutable.Buffer[String] = ArrayBuffer(ab, cd)
scala> res0.map(_.toBuffer)
res1: scala.collection.mutable.Buffer[scala.collection.mutable.Buffer[Char]] = ArrayBuffer(ArrayBuffer(a, b), ArrayBuffer(c, d))
scala> "ab-cd".split("-").toBuffer.map(_.toBuffer)
<console>:8: error: missing parameter type for expanded function ((x$1) => x$1.toBuffer)
"ab-cd".split("-").toBuffer.map(_.toBuffer)
^
scala> "ab-cd".split("-").toList.map(_.toBuffer)
res3: List[scala.collection.mutable.Buffer[Char]] = List(ArrayBuffer(a, b), ArrayBuffer(c, d))
Look at the definitions of toBuffer and toList:
def toBuffer[A1 >: A]: Buffer[A1]
def toList: List[A]
As you can see, toBuffer is generic, while toList is not.
The reason for this difference is - I believe - that Buffer is invariant, while List is covariant.
Let's say that we have the following classes:
class Foo
class Bar extends Foo
Because List is covariant, you can call toList on an instance of Iterable[Bar] and treat the result as a List[Foo].
If List where invariant, this would not be the case.
Buffer being invariant, if toBuffer was defined as def toBuffer: Buffer[A] you would similarly not be able to treat the result
of toBuffer (on an instance of Iterable[Bar]) as an instance of Buffer[Foo] (as Buffer[Bar] is not a sub-type of Buffer[Foo], unlike for lists).
But by declaring toBuffer as def toBuffer[A1 >: A] (notice the added type parameter A1), we get back the possibility to have toBuffer return an instance of Buffer[Foo] :
all we need is to explcitly set A1 to Foo, or let the compiler infer it (if toBuffer is called at a site where a Buffer[Foo] is expected).
I think this explains the reason why toList and toBuffer are defined differently.
Now the problem with this is that toBuffer is generic, and this can badly affect inference.
When you do this:
"ab-cd".split("-").toBuffer
You never explicitly say that A1 is String, but because "ab-cd".split("-") has unambiguously the type Array[String], the compiler knows that A is String.
It also knows that A1 >: A (in toBuffer), and without any further constraint, it will infer A1 to be exactly A, in other words String.
So in the end the whole expression returns a Buffer[String].
But here's the thing: in scala, type inference happens in an expression as a whole.
When you have something like a.b.c, you might expect that scala will infer an exact type
for a, then from that infer an exact type for a.b, and finally for a.b.c. Not so.
Type inference is deferred to the whole expression a.b.c (see scala specification "6.26.4 Local Type Inference
", "case 1: selections")
So, going back to your problem, in the expression "ab-cd".split("-").toBuffer.map(_.toBuffer), the sub-expression "ab-cd".split("-").toBuffer is not typed Buffer[String], but instead
it stays typed as something like Buffer[A1] forSome A1 >: String. In other words, A1 is not fixed, we just carry the constraint A1 >: String to the next step of inference.
This next step is map(_.toBuffer), where map is defined as map[C](f: (B) ⇒ C): Buffer[B]. Here B is actually the same as A1, but at this point A1
is still not fully known, we only know that A1 >: String.
Here lies our problem. The compiler needs to know the exact type of the anonymous function (_.toBuffer) (simply because instantiating a Function1[A,R] requires to know the exact types of A and R, just like for any generic type).
So you need to tell him explcitly somehow, as it was not able to infer it exactly.
This means you need to do either:
"ab-cd".split("-").toBuffer[String].map(_.toBuffer)
Or:
"ab-cd".split("-").toBuffer.map((_:String).toBuffer)
I am trying to understand how fold and foldLeft and the respective reduce and reduceLeft work. I used fold and foldLeft as my example
scala> val r = List((ArrayBuffer(1, 2, 3, 4),10))
scala> r.foldLeft(ArrayBuffer(1,2,4,5))((x,y) => x -- y._1)
scala> res28: scala.collection.mutable.ArrayBuffer[Int] = ArrayBuffer(5)
scala> r.fold(ArrayBuffer(1,2,4,5))((x,y) => x -- y._1)
<console>:11: error: value _1 is not a member of Serializable with Equals
r.fold(ArrayBuffer(1,2,4,5))((x,y) => x -- y._1)
Why fold didn't work as foldLeft? What is Serializable with Equals? I understand fold and foldLeft has slight different API signature in terms of parameter generic types. Please advise. Thanks.
The method fold (originally added for parallel computation) is less powerful than foldLeft in terms of types it can be applied to. Its signature is:
def fold[A1 >: A](z: A1)(op: (A1, A1) => A1): A1
This means that the type over which the folding is done has to be a supertype of the collection element type.
def foldLeft[B](z: B)(op: (B, A) => B): B
The reason is that fold can be implemented in parallel, while foldLeft cannot. This is not only because of the *Left part which implies that foldLeft goes from left to right sequentially, but also because the operator op cannot combine results computed in parallel -- it only defines how to combine the aggregation type B with the element type A, but not how to combine two aggregations of type B. The fold method, in turn, does define this, because the aggregation type A1 has to be a supertype of the element type A, that is A1 >: A. This supertype relationship allows in the same time folding over the aggregation and elements, and combining aggregations -- both with a single operator.
But, this supertype relationship between the aggregation and the element type also means that the aggregation type A1 in your example should be the supertype of (ArrayBuffer[Int], Int). Since the zero element of your aggregation is ArrayBuffer(1, 2, 4, 5) of the type ArrayBuffer[Int], the aggregation type is inferred to be the supertype of both of these -- and that's Serializable with Equals, the only least upper bound of a tuple and an array buffer.
In general, if you want to allow parallel folding for arbitrary types (which is done out of order) you have to use the method aggregate which requires defining how two aggregations are combined. In your case:
r.aggregate(ArrayBuffer(1, 2, 4, 5))({ (x, y) => x -- y._1 }, (x, y) => x intersect y)
Btw, try writing your example with reduce/reduceLeft -- because of the supertype relationship between the element type and the aggregation type that both these methods have, you will find that it leads to a similar error as the one you've described.