So I have this function in Scala:
def f(a: Int)(b: Int)(c: Double)(d: Double): Double = a * c + b * d
The question is What are the three types that make the following statements compile.
def g: <Type1> = f(1)(2)(3.0)
def h: <Type2> = f(1)(2)
def k: <Type3> = f(1)
I'm still new to Scala and I am not really understanding the concept of currying. Maybe an answer to this question with some explanation will really help me. Thanks.
First, one main thing: function that takes two parameters a and b and returns a value c can be viewed as a function that takes an a and returns a function that takes b and returns c. This "change of point of view" is called currying.
Imagine a function that sums up two numbers. You give it 2 and 3, it returns 5. It can be viewed as a function that takes one number and returns a function from a number to a number. You give it a 2, it returns a function that takes some number and adds 2 to it.
Now, some types that you requested:
// pseudocode!
def g: Double => Double
= f(1)(2)(3.0) // we supply three params and are left with only one, "d"
= (d: Double) => 1 * 3.0 + 2 * d // we comply with g's type
def h: Double => Double => Double // or (Double, Double) => Double
= f(1)(2) // we supply two params and are left with two more, "c" and "d"
= (c: Double)(d: Double) => 1 * c + 2 * d // we comply with h's type
def k: Double => Double => Double => Double // or (Double, Double, Double) => Double
= f(1) // we supply one param and are left with three more, "b", "c" and "d"
= (b: Double)(c: Double)(d: Double) => 1 * c + b * d // we comply with k's type
Currying IMO is one of the most confusing concepts in Scala. The term itself comes from functional programming paradigm and, according to wikipedia, is
the technique of translating the evaluation of a function that takes
multiple arguments (or a tuple of arguments) into evaluating a
sequence of functions, each with a single argument.
which means that function call f(a, b, c) is represented by f(a)(b)(c). Looks like Scala? Not exactly. Here we have three function calls, each of them returns another function. Type of f (in Scala speak) is Int => (Int => (Double => (Double => Double))). Let's look at your f:
scala> def f(a: Int)(b: Int)(c: Double)(d: Double): Double = a * c + b * d
f: (a: Int)(b: Int)(c: Double)(d: Double)Double
As you see, there are no arrows here. What we have here is a method with multiple parameter lists. Method has no value and can't be assigned or passed anywhere, it belongs to an object. Function, on the other hand, is an object and can be assigned or passed to another method or function. In most cases omitting parameter lists is not allowed for methods:
scala> f(0)
<console>:01: error: missing argument list for method f
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing `f _` or `f(_)(_)(_)(_)` instead of `f`.
There is one exception though as error message implies: if f(0) is placed in a functional context Scala will perform automatic eta-expansion, which means it will convert your method to a function:
scala> val fl: (Int => (Double => (Double => Double))) = f(0)
fl: Int => (Double => (Double => Double)) = $$Lambda$1342/937956960#43c1614
where eta-expansion means literally this:
scala> val fl: (Int => (Double => (Double => Double))) = (b => (c => (d => f(0)(b)(c)(d))))
fl: Int => (Double => (Double => Double)) = $$Lambda$1353/799716194#52048150
Another (explicit) way to convert a method to a curried function is by using placeholder (which will give you correct types right away):
scala> f _
res11: Int => (Int => (Double => (Double => Double))) = $$Lambda$1354/1675405592#4fa649d8
scala> f(0) _
res12: Int => (Double => (Double => Double)) = $$Lambda$1355/1947050122#ba9f744
Also be aware that:
def g: Int => (Double => (Double => Double)) = f(0)
is in fact
def g: Int => (Double => (Double => Double)) = (b => (c => (d => f(0)(b)(c)(d))))
i.e. it's a method g, which creates a function on the fly and returns it. So g(0) means "call method g without parameters, get back a function and apply it to 0".
Related
Given Scala 2.12.6:
val list = List(1)
val x = 2
This works:
list.map ( y => x + y )
returning List[Int] = List(3)
and this works:
list.map ( (y: Int) => x + y )
returning the same value.
Same for this:
list.map { (y: Int) => x + y }
And same for this:
list.map { y: Int => x + y }
Yet this fails:
list.map ( y: Int => x + y )
producing the error:
error: not found: type +
list.map ( y: Int => x + y )
^
Why is Scala thinking the + is meant to indicate a type, and where is this difference between using parenthesis and curly braces documented and explained?
The Section 6.23 about anonymous functions says:
In the case of a single untyped formal parameter, (x) => e can be abbreviated to x => e. If an anonymous function (x: T) => e with a single typed parameter appears as the result expression of a block, it can be abbreviated to x: T => e.
Thus, in a block { ... }, the function literal (y: Int) => x + y can be abbreviated to just y: Int => x + y.
Without the block, the entire Int => x + y-part is treated as type ascription, so the error message actually makes sense. For example, here is a context in which the offending expression becomes valid:
type y = Unit
type x = Unit
type +[A, B] = Int
val y = (i: Int) => 42 + i
val list = List(1)
println(
list.map ( y: Int => x + y )
) // happily prints `List(43)`.
This is because there are two ys in two separate scopes (one value, one type alias), so that (y: Int => x + y) becomes (y: Int => +[x, y]), and then (y: Int => Int), which is just a type ascription enforcing that value y is indeed of function type Int => Int (which it is, so everything compiles and runs). Here is another similar example.
My suggestion: stick to the slightly more verbose (foo: Foo) => { ... } notation, it will cause fewer surprises for everyone who tries to read and to modify the code. Otherwise there is some risk that
argument types in bindings collide with type ascriptions
=> of the anonymous lambda collides with function type =>
arithmetic operation + collides with binary infix type constructor +[_,_]
values x, y collide with undefined types x, y.
The fact that same syntax can denote both types and expressions can be somewhat of a double-edged sword.
The following code works but I don't quite understand how the arguments are mapped to the parameter lists. Please be aware Im new to Scala.
import Math.abs
val tolerance = 0.0001
def isCloseEnough(x: Double, y: Double) =
abs((x - y) / x) / x < tolerance
def fixedPoint(f: Double => Double)(firstGuess: Double) = {
def iterate(guess: Double): Double = {
val next = f(guess)
if (isCloseEnough(guess, next)) next
else iterate(next)
}
iterate(firstGuess)
}
def averageDamp(f: Double => Double)(x: Double) = (x + f(x)) / 2
def sqrt(x: Double) =
fixedPoint(averageDamp(y => x / y))(1.0)
sqrt(2.0)
The body of the sqrt function above is fixedPoint(averageDamp(y => x / y))(1.0)
where
(y => x / y) maps to (f: Double => Double) of the averageDamp function and
(1.0) maps to (firstGuess: Double) of the fixedPoint function but
looks like nothing maps to (x: Double) of the averageDamp function.
Thank you in advance.
This is called currying. What's really happening is that averageDamp(y => x / y) is being interpreted as a function: Double => Double, because once you set the first parameter list to a particular set of values, you obtain a function that takes the second parameter list.
For example, consider the following code:
def multiply(x: Double)(y: Double) = x * y
val multiplyByFive: Double => Double = multiply(5)
println(multiplyByFive(2)) // 10
println(multiplyByFive(6)) // 30
As you see, the fact of applying multiply with only one parameter list out of two creates a new function of the second parameter list (here (y: Double)) that has a fixed set of values for the first parameter list (here (5)).
Another equivalent way to write this, which perhaps will be more explicit for you, is the following :
val multiplyByFive: Double => Double = multiply(5)(_)
Here we explicitly apply the function with two parameters lists, but using a wildcard for the second, which is a way to tell to the compiler to create a function that replaces the underscore by a parameter of the function.
Whilst I understand what a partially applied/curried function is, I still don't fully understand why I would use such a function vs simply overloading a function. I.e. given:
def add(a: Int, b: Int): Int = a + b
val addV = (a: Int, b: Int) => a + b
What is the practical difference between
def addOne(b: Int): Int = add(1, b)
and
def addOnePA = add(1, _:Int)
// or currying
val addOneC = addV.curried(1)
Please note I am NOT asking about currying vs partially applied functions as this has been asked before and I have read the answers. I am asking about currying/partially applied functions VS overloaded functions
The difference in your example is that overloaded function will have hardcoded value 1 for the first argument to add, i.e. set at compile time, while partially applied or curried functions are meant to capture their arguments dynamically, i.e. at run time. Otherwise, in your particular example, because you are hardcoding 1 in both cases it's pretty much the same thing.
You would use partially applied/curried function when you pass it through different contexts, and it captures/fills-in arguments dynamically until it's completely ready to be evaluated. In FP this is important because many times you don't pass values, but rather pass functions around. It allows for higher composability and code reusability.
There's a couple reasons why you might prefer partially applied functions. The most obvious and perhaps superficial one is that you don't have to write out intermediate functions such as addOnePA.
List(1, 2, 3, 4) map (_ + 3) // List(4, 5, 6, 7)
is nicer than
def add3(x: Int): Int = x + 3
List(1, 2, 3, 4) map add3
Even the anonymous function approach (that the underscore ends up expanding out to by the compiler) feels a tiny bit clunky in comparison.
List(1, 2, 3, 4) map (x => x + 3)
Less superficially, partial application comes in handy when you're truly passing around functions as first-class values.
val fs = List[(Int, Int) => Int](_ + _, _ * _, _ / _)
val on3 = fs map (f => f(_, 3)) // partial application
val allTogether = on3.foldLeft{identity[Int] _}{_ compose _}
allTogether(6) // (6 / 3) * 3 + 3 = 9
Imagine if I hadn't told you what the functions in fs were. The trick of coming up with named function equivalents instead of partial application becomes harder to use.
As for currying, currying functions often lets you naturally express transformations of functions that produce other functions (rather than a higher order function that simply produces a non-function value at the end) which might otherwise be less clear.
For example,
def integrate(f: Double => Double, delta: Double = 0.01)(x: Double): Double = {
val domain = Range.Double(0.0, x, delta)
domain.foldLeft(0.0){case (acc, a) => delta * f(a) + acc
}
can be thought of and used in the way that you actually learned integration in calculus, namely as a transformation of a function that produces another function.
def square(x: Double): Double = x * x
// Ignoring issues of numerical stability for the moment...
// The underscore is really just a wart that Scala requires to bind it to a val
val cubic = integrate(square) _
val quartic = integrate(cubic) _
val quintic = integrate(quartic) _
// Not *utterly* horrible for a two line numerical integration function
cubic(1) // 0.32835000000000014
quartic(1) // 0.0800415
quintic(1) // 0.015449626499999999
Currying also alleviates a few of the problems around fixed function arity.
implicit class LiftedApply[A, B](fOpt: Option[A => B]){
def ap(xOpt: Option[A]): Option[B] = for {
f <- fOpt
x <- xOpt
} yield f(x)
}
def not(x: Boolean): Boolean = !x
def and(x: Boolean)(y: Boolean): Boolean = x && y
def and3(x: Boolean)(y: Boolean)(z: Boolean): Boolean = x && y && z
Some(not _) ap Some(false) // true
Some(and _) ap Some(true) ap Some(true) // true
Some(and3 _) ap Some(true) ap Some(true) ap Some(true) // true
By having curried functions, we've been able to "lift" a function to work on Option for as many arguments as we need. If our logic functions had not been curried, then we would have had to have separate functions to lift A => B to Option[A] => Option[B], (A, B) => C to (Option[A], Option[B]) => Option[C], (A, B, C) => D to (Option[A], Option[B], Option[C]) => Option[D] and so on for all the arities we cared about.
Currying also has some other miscellaneous benefits when it comes to type inference and is required if you have both implicit and non-implicit arguments for a method.
Finally, the answers to this question list out some more times you might want currying.
This is a implementation of the Y-combinator in Scala:
scala> def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))(_:T)
Y: [T](func: (T => T) => (T => T))T => T
scala> def fact = Y {
| f: (Int => Int) =>
| n: Int =>
| if(n <= 0) 1
| else n * f(n - 1)}
fact: Int => Int
scala> println(fact(5))
120
Q1: How does the result 120 come out, step by step? Because the Y(func) is defined as func(Y(func)), the Y should become more and more,Where is the Y gone lost and how is the 120 come out in the peform process?
Q2: What is the difference between
def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))(_:T)
and
def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))
They are the same type in the scala REPL, but the second one can not print the result 120?
scala> def Y[T](func: (T => T) => (T => T)): (T => T) = func(Y(func))
Y: [T](func: (T => T) => (T => T))T => T
scala> def fact = Y {
| f: (Int => Int) =>
| n: Int =>
| if(n <= 0) 1
| else n * f(n - 1)}
fact: Int => Int
scala> println(fact(5))
java.lang.StackOverflowError
at .Y(<console>:11)
at .Y(<console>:11)
at .Y(<console>:11)
at .Y(<console>:11)
at .Y(<console>:11)
First of all, note that this is not a Y-combinator, since the lambda version of the function uses the free variable Y. It is the correct expression for Y though, just not a combinator.
So, let’s first put the part which computes the factorial into a separate function. We can call it comp:
def comp(f: Int => Int) =
(n: Int) => {
if (n <= 0) 1
else n * f(n - 1)
}
The factorial function can now be constructed like this:
def fact = Y(comp)
Q1:
Y is defined as func(Y(func)). We invoke fact(5) which is actually Y(comp)(5), and Y(comp) evaluates to comp(Y(comp)). This is the key point: we stop here because comp takes a function and it doesn’t evaluate it until needed. So, the runtime sees comp(Y(comp)) as comp(???) because the Y(comp) part is a function and will be evaluated only when (if) needed.
Do you know about call-by-value and call-by-name parameters in Scala? If you declare your parameter as someFunction(x: Int), it will be evaluated as soon as someFunction is invoked. But if you declare it as someFunction(x: => Int), then x will not be evaluated right away, but at the point where it is used. Second call is “call by name” and it is basically defining your x as a “function that takes nothing and returns an Int”. So if you pass in 5, you are actually passing in a function that returns 5. This way we achieve lazy evaluation of function parameters, because functions are evaluated at the point they are used.
So, parameter f in comp is a function, hence it is only evaluated when needed, which is in the else branch. That’s why the whole thing works - Y can create an infinite chain of func(func(func(func(…)))) but the chain is lazy. Each new link is computed only if needed.
So when you invoke fact(5), it will run through the body into the else branch and only at that point f will be evaluated. Not before. Since your Y passed in comp() as parameter f, we will dive into comp() again. In the recursive call of comp() we will be calculating the factorial of 4. We will then again go into the else branch of the comp function, thus effectively diving into another level of recursion (calculating factorial of 3). Note that in each function call your Y provided a comp as an argument to comp, but it is only evaluated in the else branch. Once we get to the level which calculates factorial of 0, the if branch will be triggered and we will stop diving further down.
Q2:
This
func(Y(func))(_:T)
is syntax sugar for this
x => func(Y(func))(x)
which means we wrapped the whole thing into a function. We didn’t lose anything by doing this, only gained.
What did we gain? Well, it’s the same trick as in the answer to a previous question; this way we achieve that func(Y(func)) will be evaluated only if needed since it’s wrapped in a function. This way we will avoid an infinite loop. Expanding a (single-paramter) function f into a function x => f(x) is called eta-expansion (you can read more about it here).
Here’s another simple example of eta-expansion: let’s say we have a method getSquare() which returns a simple square() function (that is, a function that calculates the square of a number). Instead of returning square(x) directly, we can return a function that takes x and returns square(x):
def square(x: Int) = x * x
val getSquare: Int => Int = square
val getSquare2: Int => Int = (x: Int) => square(x)
println(square(5)) // 25
println(getSquare(5)) // 25
println(getSquare2(5)) // 25
Hope this helps.
Complementing the accepted answer,
First of all, note that this is not a Y-combinator, since the lambda version of the function uses the free variable Y. It is the correct expression for Y though, just not a combinator.
A combinator isn't allowed to be explicitly recursive; it has to be a lambda expression with no free variables, which means that it can't refer to its own name in its definition. In the lambda calculus it is not possible to refer to the definition of a function in a function body. Recursion may only be achieved by passing in a function as a parameter.
Given this, I've copied the following implementation from rosetta code that uses some type trickery to implement Y combinator without explicit recursion. See here
def Y[A, B](f: (A => B) => (A => B)): A => B = {
case class W(wf: W => A => B) {
def get: A => B =
wf(this)
}
val g: W => A => B = w => a => f(w.get)(a)
g(W(g))
}
Hope this helps with the understanding.
I don't know the answer, but will try to guess. Since you have def Y[T](f: ...) = f(...) compiler can try to substitute Y(f) with simply f. This will create an infinite sequence of f(f(f(...))). Partially applying f you create a new object, and such substitution becomes impossible.
I am working on spark and not an expert in scala. I have got the two variants of map function. Could you please explain the difference between them.?
first variant and known format.
first variant
val.map( (x,y) => x.size())
Second variant -> This has been applied on tuple
val.map({case (x, y) => y.toString()});
The type of val is RDD[(IntWritable, Text)]. When i tried with first function, it gave error as below.
type mismatch;
found : (org.apache.hadoop.io.IntWritable, org.apache.hadoop.io.Text) ⇒ Unit
required: ((org.apache.hadoop.io.IntWritable, org.apache.hadoop.io.Text)) ⇒ Unit
When I added extra parenthesis it said,
Tuples cannot be directly destructured in method or function parameters.
Well you say:
The type of val is RDD[(IntWritable, Text)]
so it is a tuple of arity 2 with IntWritable and Text as components.
If you say
val.map( (x,y) => x.size())
what you're doing is you are essentially passing in a Function2, a function with two arguments to the map function. This will never compile because map wants a function with one argument. What you can do is the following:
val.map((xy: (IntWritable, Text)) => xy._2.toString)
using ._2 to get the second part of the tuple which is passed in as xy (the type annotation is not required but makes it more clear).
Now the second variant (you can leave out the outer parens):
val.map { case (x, y) => y.toString() }
this is special scala syntax for creating a PartialFunction that immediately matches on the tuple that is passed in to access the x and y parts. This is possible because PartialFunction extends from the regular Function1 class (Function1[A,B] can be written as A => B) with one argument.
Hope that makes it more clear :)
I try this in repl:
scala> val l = List(("firstname", "tom"), ("secondname", "kate"))
l: List[(String, String)] = List((firstname,tom), (secondname,kate))
scala> l.map((x, y) => x.size)
<console>:9: error: missing parameter type
Note: The expected type requires a one-argument function accepting a 2-Tuple.
Consider a pattern matching anonymous function, `{ case (x, y) => ... }`
l.map((x, y) => x.size)
maybe can give you some inspire.
Your first example is a function that takes two arguments and returns a String. This is similar to this example:
scala> val f = (x:Int,y:Int) => x + y
f: (Int, Int) => Int = <function2>
You can see that the type of f is (Int,Int) => Int (just slightly changed this to be returning an int instead of a string). Meaning that this is a function that takes two Int as arguments and returns an Int as a result.
Now the second example you have is a syntactic sugar (a shortcut) for writing something like this:
scala> val g = (k: (Int, Int)) => k match { case (x: Int, y: Int) => x + y }
g: ((Int, Int)) => Int = <function1>
You see that the return type of function g is now ((Int, Int)) => Int. Can you spot the difference? The input type of g has two parentheses. This shows that g takes one argument and that argument must be a Tuple[Int,Int] (or (Int,Int) for short).
Going back to your RDD, what you have is an Collection of Tuple[IntWritable, Text] so the second function will work, whereas the first one will not work.