Note: I am using scalac. Please do not recommend to use sbt instead.
I ran into a peculiar issue that I could resolve, but I am wondering why it works that way and not the way I did it before. Here's a code snippet:
def multiply[A](r1: Vector[A], r2: Vector[A], multOp: (A,A) => A, sumOp: (A, A) => A) =
r1.zip(r2).map(multOp).reduce(sumOp)
It does not compile, resulting in an error message like:
Error:(73, 20) type mismatch;
found : (A, A) => A
required: ((A, A)) => ?
r1.zip(r2).map(multOp).reduce(sumOp)
Changing the snippet to:
def multiply[A](r1: Vector[A], r2: Vector[A], multOp: ((A,A)) => A, sumOp: (A, A) => A) =
r1.zip(r2).map(multOp).reduce(sumOp)
will resolve the issue.
Note that sumOp does work with only one pair of braces.
Why?
Method map is defined as taking a single parameter, and (A, A) => A has two. By converting two parameters of type A into one parameter which is a tuple of type (A, A), it compiles.
(A, A) => A // fails due to two params of type A
((A, A)) => A // works due to one param of type (A, A)
On the other hand, reduce is defined as taking two parameters of the same type, so it's happy to take sumOp which matches that description.
Here are the full signatures found in TraversableLike and TraversableOnce respectively:
def map[B, That](f: A => B)(implicit bf: CanBuildFrom[Repr, B, That]): That
def reduce[A1 >: A](op: (A1, A1) => A1): A1
EDIT (extra info):
Reason for this is the fact that reduce always takes a 2-arity function (that is, a function of two parameters) in order to reduce the collection to a single value by iteratively applying that function on the result of previous application and the next value. On the other hand, map always takes a 1-arity function and it maps the underlying value with that function. In case of Option, Future, etc. there's only one single underlying value, while in case of a Vector (like yours) there can be many, so it applies it to every element of the collection.
In some libraries you might come across map2 which takes a two-parameter function. For example, in order to combine two Options (actually, any applicative functors, but let's leave theory aside), you might do:
// presudocode
Option(1, 2).map2((a, b) => a + b)
which would give you an Option(3). I think this mechanism has been dropped in the favour of more easily understandable map + product
// presudocode
(Option(1) product Option(2)) map ((a, b) => a + b)
Actual scalaz syntax for the line above would be (in Scalaz 7):
(Option(1) |#| Option(2))((a, b) => a + b)
They are equally powerful principles (what one can do, exactly the same other one can do, no more, no less) so the latter one is usually preferred and sometimes it's the only one provided, but yes, you might come across map2 from time to time.
Alright, that's a bit of extra info. As far as map is concerned, just remember there's always just one single parameter coming in and one value coming out.
The first snippet will work, if you define it as:
def multiply[A](r1: Vector[A], r2: Vector[A], multOp: (A,A) => A, sumOp: (A, A) => A) =
(r1, r2).zipped.map(multOp).reduce(sumOp)
The map method of a zipped tuple takes a function with two arguments (A, A) => B, because that's the expected usage pattern.
This approach also avoids the creation of an intermediate Vector[(A, A)].
Related
Option is used for dealing with partiality in Scala, but we can also lift ordinary functions to the context of Options in order to handle errors. When implementing the function map2 I am curious on how to know when to use which functions. Consider the following implementation:
def map2[A,B,C] (ao: Option[A], bo: Option[B]) (f: (A,B) => C): Option[C] =
ao flatMap {aa =>
bo map {bb =>
f(aa, bb)
aa is of type A, and bb is of type B which is then fed to F, giving us a C. However, if we do the following:
def map2_1[A,B,C] (ao: Option[A], bo: Option[B]) (f: (A,B) => C): Option[C] =
ao flatMap {aa =>
bo flatMap {bb =>
f(aa, bb)
aa is still of type A, and bb is still of type B, yet we will have to wrap the last call in Some(f(aa, bb)) in order to get an Option[C] instead of a regular C. Why is this? What does it mean to flatten on BO here?
Last and not least, one could do the simpler:
def map2_2[A,B,C] (ao: Option[A], bo: Option[B]) (f: (A,B) => C): Option[C] = for {
as <- ao
bs <- bo
} yield(f(as,bs))
I know that for-comprehensions are syntactic sugar for ForEach'es, maps and flatmaps etc, but how do I, as a developer, know that the compiler will choose MAP with bs <- bo, and not flatMap?
I think I am on the verge of understanding the difference, yet nested flatmaps confuse me.
Taking the last question first, the developer knows what the compiler will do with for because the behaviour is defined and predictable: All <- turn into flatMap except the last one which will be either map or foreach depending on whether or not there is a yield.
The broader question seems to be about the difference between map and flatMap. The difference should be clear from the signatures e.g. for List these are the (simplified) signatures:
def map[B] (f: A => B) : List[B]
def flatMap[B](f: A => List[B]): List[B]
So map just replaces the values in a List with new values by applying f to each element of type A to generate a B.
flatMap generates a new list by concatenating the results of calling f on each element of the original List. It is equivalent to map followed by flatten (hence the name).
Intuitively, map is a one-for-one replacement whereas flatMap allows each element in the original List to generate 0 or more new elements.
I tried to do research but still not yet figure out what is the terminology of Scala, related to lower case a,b as per the code below
def curry[A, B, C](f: (A, B) => C): A => (B => C) = a => b => f(a, b)
Why is that a,b appears on the right hand side?
I know that it is a part of Algebraic Data Type but still could not find a match definition for this.
Update based on Tim's answer, "Scala knows the type of a and b from the return type A => (B => C). a is type A, b is type B."
I want to ask about how Scala knows the type of a and b, i.e: the mechanism behind? What is the terminology of this?
I guess this is a language feature. Please suggest a foundation guideline to fully understand and practice to gain intuition when reading these complex code.
Update from Mario Galic's comment: ... Scala compiler can perform type inference based on the signature of curry ... Please clarify: if the left hand side (i.e the signature) is too obvious, why we need to have the right hand side definition? I mean, there is only 1 way to infer the logic of the left hand side, then, what is the need of creating the right hand side content?
P/S: I wish that I could mark each feedback as the answer because each provides different aspect which helps me to fully grasp the meaning.
It might help to add full type annotations
def curry[A, B, C](f: (A, B) => C): A => (B => C) =
(a: A) => ((b: B) => f(a, b): C)
Note how curry is a method that takes a function as input and also returns a function as output. You might be wondering where do a and b come from in the output function
(a: A) => ((b: B) => f(a, b): C)
but note that they are just the means of declaring the parameters of the output function. You are free to give them any name, for example the following would also work
(x: A) => ((y: B) => f(x, y): C)
The key is to understand that functions are first class values in Scala, so you can pass them in as arguments to other functions and return them as return values from other functions, in the same way you would do with familiar values like say integer 42. Writing value 42 is straightforward, but writing down function value is more verbose since you have to specify the parameters like a and b but nevertheless conceptually it is still just a value. Hence we could say curry is a method that takes a value and returns a value, but these values happen to be function values.
As we all know, it's pretty easy to create a tuple: (1,'a'). And the type of said tuple is pretty simple: (Int,Char). That type designation, however, is a convenient alternative for the more verbose type designation Tuple2[Int,Char]. In fact, the tuple creation itself is a convenient alternate syntax to the more direct new Tuple2(1,'a').
It's a similar story with functions.
The type designation Char => Int is a convenient alternative to the more verbose Function1[Char,Int]. And, after studying the ScalaDocs page, we learn that a simple function like...
val ctoi = (c:Char) => c.toInt
...is the equivalent of...
val ctoi = new Function1[Char, Int] {
def apply(c: Char): Int = c.toInt
}
So, armed with this information, we can now translate...
def curry[A,B,C](f: (A, B) => C): A => (B => C) =
a => b => f(a, b)
...into its equivalent...
def curry[A,B,C](f: Function2[A,B,C]): Function1[A,Function1[B,C]] =
new Function1[A,Function1[B,C]] {
def apply(a:A) = new Function1[B,C] {def apply(b:B) = f(a,b)}
}
With this it's a little easier to see how a => b => ..., while a bit confusing at first, is actually a very convenient way to designate the names of the arguments being passed in to the hidden apply() methods.
The question doesn't really make sense, but in case this helps here is a breakdown of that line:
def curry[A, B, C](f: (A, B) => C): A => (B => C) = a => b => f(a, b)
This splits into a definition and an implementation with = inbetween. The definition is
def curry[A, B, C](f: (A, B) => C): A => (B => C)
Breaking it down further, A, B, and C are type parameters, meaning that any three types can be used when calling this function.
Next comes the single argument to the function:
f: (A, B) => C
The value of this argument is a function that takes two values (one of type A and one of type B) are returns a single value of type C.
Next comes the type of the result:
A => (B => C)
This is a function that takes a single argument of type A and returns a function that takes a single argument of type B and returns a result of type C.
So curry is a function that takes a function of type (A, B) => C) and returns a function of type A => (B => C). This implements the process known as currying (hence the name).
Now for the implementation (the other side of the =):
a => b => f(a, b)
Adding some brackets might make this clearer:
a => (b => f(a, b))
This is a function that take a and returns b => f(a, b). a is the argument for this function. So that leaves this
b => f(a, b)
This is a simple function with an argument b that returns f(a, b).
Scala knows the type of a and b from the return type A => (B => C). a is type A, b is type B.
Could start value just be a parameter in op argument list ?
Food is defined on List as
def fold[A1 >: A](z: A1)(op: (A1, A1) ⇒ A1): A1 Folds the elements
of this traversable or iterator using the specified associative binary
operator.
What would the implications of defining fold as
def fold[A1 >: A](op: (z:A1,A1, A1) ⇒ A1): A1
So in this version the initial value is passed as a value to the function instead of being curried in a separate parameter list.
If you're looking to motivate that particular signature of foldLeft, it may be worthwhile to first examine reduceLeft.
// Slightly simplified to remove the supertype constraint
def reduceLeft(f: (A, A) => A): A
reduceLeft squishes the entire collection into a single element and it takes as an argument a function that tells it how to squish each new element in the collection onto what it's got so far.
There's, however, a problem. reduceLeft is partial. In particular if the collection is empty, reduceLeft has nowhere to begin squishing things. So we can make it total, by telling reduceLeft where to begin. So we give reduceLeft an additional parameter.
def reduceLeftTotal(initial: A, f: (A, A) => A): A
Note that if we just glommed initial as another argument to f, we wouldn't fix the partiality of reduceLeft. If this is an empty collection, we still blow up.
// This doesn't get us what we want. Where does the initial `A` come from?
def reduceLeftNotWhatWeWant(f: (A, A, A) => A): A
Okay, now that we've got reduceLeftTotal, there's an immediate new avenue for generalization. Why does the thing that we're squishing all the elements of our collection onto have to have the same type as the elements? The answer is it doesn't!
def generalReduceLeftTotal[B](initial: B, f: (B, A) => A): B
Finally because type information in previous argument lists, but not previous arguments in the same list, can be used to help Scala's type inference, we can reduce the amount of explicit type annotations we need by currying.
// And we're back to foldLeft!
def foldLeft[B](initial: B)(f: (B, A) => A): B
I'm chaining transformations, and I'd like to accumulate the result of each transformation so that it can potentially be used in any subsequent step, and also so that all the results are available at the end (mostly for debugging purposes). There are several steps and from time to time I need to add a new step or change the inputs for a step.
HList seems to offer a convenient way to collect the results in a flexible but still type-safe way. But I'd rather not complicate the actual steps by making them deal with the HList and the accompanying business.
Here's a simplified version of the combinator I'd like to write, which isn't working. The idea is that given an HList containing an A, and the index of A, and a function from A -> B, mapNth will extract the A, run the function, and cons the result onto the list. The resulting extended list captures the type of the new result, so several of these mapNth-ified steps can be composed to produce a list containing the result from each step:
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At[L, index.N]):
B :: L =
f(l(index)) :: l
Incidentally, I'll also need map2Nth taking two indices and f: (A, B) => C, but I believe the issues are the same.
However, mapNth does not compile, saying l(index) has type at.Out, but f's argument should be A. That's correct, of course, so what I suppose I need is a way to provide evidence that at.Out is in fact A (or, at.Out <: A).
Is there a way to express that constraint? I believe it will have to take the form of an implicit, because of course the constraint can only be checked when mapNth is applied to a particular list and function.
You're exactly right about needing evidence that at.Out is A, and you can provide that evidence by including the value of the type member in at's type:
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At[L, index.N] { type Out = A }):
B :: L =
f(l(index)) :: l
The companion objects for type classes like At in Shapeless also define an Aux type that includes the output type as a final type parameter.
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At.Aux[L, index.N, A]):
B :: L =
f(l(index)) :: l
This is pretty much equivalent but more idiomatic (and it looks a little nicer).
Let's say I have an empty map
val map = Map[Int, Int]()
I'm confused because while the following code compiles properly:
map.foldLeft((0,0)){case((k1, v1), (k2, v2)) => (-1, -1)}
the following, seemingly exact code snippet results in compilation error:
map reduceLeft {case((k1, v1), (k2, v2)) => (-1, -1)}
The error is:
scala> map reduceLeft {case((k1, v1), (k2, v2)) => (k1, v1)}
<console>:9: error: missing parameter type for expanded function
The argument types of an anonymous function must be fully known. (SLS 8.5)
Expected type was: (?, (Int, Int)) => ?
map reduceLeft {case((k1, v1), (k2, v2)) => (k1, v1)}
It's not a huge problem, but obviously it would be nice to not have to deal with this. Do you have any ideas on what could I do differently, or do I just have to learn to accept it?
The reason this happens is that foldLeft has two parameter lists (the 1st being the initial, "priming" value and 2nd being the function) and reduceLeft has only one (the function).
Scala's type inference operates one parameter list at a time. Additionally, types inferred in one parameter list are available to guide or constrain type inference in later ones (those further right) in ways they cannot help guide or constrain type inference within a given parameter list. In this case, Scala cannot infer properly the B type in the reduceLeft signature:
def reduceLeft[B >: (A, B)](op: (B, (A, B)) ⇒ B): B
While in the fold case:
def foldLeft[B](z: B)(op: (B, (A, B)) ⇒ B): B
it binds a type to B by looking at your primer value alone ((0, 0)) which it then has available to infer the parameter types in the function (without you making them explicit).