The authors of Functional Programming in Scala give this as the definition of curry in scala:
def curry[A,B,C](f: (A, B) => C): A => (B => C) =
a => b => f(a, b)
However, if we apply it to a function taking parametric types, e.g.:
def isSorted[A](as: Array[A], ordered:(A,A)=>Boolean) =
if(as.size < 2)
true else
as.zip(as.drop(1)).map(ordered.tupled).reduce(_ && _)
Then the result wants A (in isSorted) to be nothing:
scala> curry(isSorted)
res29: Array[Nothing] => (((Nothing, Nothing) => Boolean) => Boolean) = <function1>
This is obviously not what is desired. Should curry be defined differently, or called differently, or is it not practical to implement curry in Scala?
You're running into two separate problems here. The first is that isSorted when it is passed to curry is forced to become monomorphic. The second is that Scala's type inference is failing you here.
This is one of those times where the difference between a function and a method matters in Scala. isSorted is eta-expanded into a function which in turn is a Scala value, not a method. Scala values are always monomorphic, only methods can be polymorphic. For any method of types (A, B) C (this is the syntax for a method type and is different from (A, B) => C which is a function and therefore a value), the default eta-expansion is going to result in the superclass of all functions of that arity, namely (Nothing, Nothing) => Any. This is responsible for all the Nothings you see (you don't have any Anys because isSorted is monomorphic in its return value).
You might imagine despite the monomorphic nature of Scala values though, that you could ideally do something like
def first[A, B](x: A, y: B): A = x
curry(first)(5)(6) // This doesn't compile
This is Scala's local type inference biting you. It works on separate parameter lists from left to right first is the first thing to get a type inferred and as mentioned above, it gets inferred to be (Nothing, Nothing) => Any. This clashes with Ints that follow.
As you've realized, one way of getting around this is annotating your polymorphic method that you pass to curry so that it eta-expands into the correct type. This is almost certainly the way to go.
Another thing you could possibly do (although I'm don't think it'll serve anything except pedagogical purposes) is to curry curry itself and define it as follows:
def curryTwo[A, B, C](x: A)(y: B)(f: (A, B) => C): C = f(x, y)
On the one hand, the below works now because of the left-to-right type inference.
curryTwo(5)(6)(first) // 5
On the other hand, to use curryTwo in the scenarios where you'd want to use curry, you're going to need to need to provide types to Scala's type inference engine anyway.
It turns out I can call curry like this:
curry(isSorted[Int])
Which yields:
scala> curry(isSorted[Int])
res41: Array[Int] => (((Int, Int) => Boolean) => Boolean) = <function1>
See https://stackoverflow.com/a/4593509/21640
Related
I took the scala odersky course and thought that the function that Flatmap takes as arguments , takes an element of Monad and returns a monad of different type.
trait M[T] {
def flatMap[U](f: T => M[U]): M[U]
}
On Monad M[T] , the return type of function is also the same Monad , the type parameter U might be different.
However I have seen examples on internet , where the function returns a completely different Monad. I was under impression that return type of function should the same Monad. Can someone simplify the below to explain how flapmap results in the actual value instead of Option in the list.
Is the List not a Monad in Scala.
val l= List(1,2,3,4,5)
def f(x:int) = if (x>2) Some(x) else None
l.map(x=>f(x))
//Result List[Option[Int]] = List( None , None , Some(3) , Some(4) , Some(5))
l.flatMap(x=>f(x))
//Result: List(3,4,5)
Let's start from the fact that M[T]is not a monad by itself. It's a type constructor. It becomes a monad when it's associated with two operators: bind and return (or unit). There are also monad laws these operators must satisfy, but let's omit them for brevity. In Haskell the type of bind is:
class Monad m where
...
(>>=) :: m a -> (a -> m b) -> m b
where m is a type constructor. Since Scala is OO language bind will look like (first argument is self):
trait M[T] {
def bind[U](f: T => M[U]): M[U]
}
Here M === m, T === a, U === b. bind is often called flatMap. In a pure spherical world in a vacuum that would be a signature of flatMap in OO language. Scala is a very practical language, so the real signature of flatMap for List is:
final def flatMap[B, That](f: (A) ⇒ GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
It's not bind, but will work as a monadic bind if you provide f in the form of (A) => List[B] and also make sure that That is List[B]. On the other hand Scala is not going to watch your back if you provide something different, but will try to find some meaningful conversion (e.g. CanBuildFrom or something else) if it exists.
UPDATE
You can play with scalac flags (-Xlog-implicits, -Xlog-implicit-conversions) to see what's happening:
scala> List(1).flatMap { x => Some(x) }
<console>:1: inferred view from Some[Int] to scala.collection.GenTraversableOnce[?] via scala.this.Option.option2Iterable[Int]: (xo: Option[Int])Iterable[Int]
List(1).flatMap { x => Some(x) }
^
res1: List[Int] = List(1)
Hmm, perhaps confusingly, the signature you gave is not actually correct, since it's really (in simplified form):
def flatMap[B](f: (A) ⇒ GenTraversableOnce[B]): Traversable[B]
Since the compiler is open-source, you can actually see what it's doing (with its full signature):
def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit bf: CanBuildFrom[Repr, B, That]): That = {
def builder = bf(repr) // extracted to keep method size under 35 bytes, so that it can be JIT-inlined
val b = builder
for (x <- this) b ++= f(x).seq
b.result
}
So you can see that there is actually no requirement that the return type of f be the same as the return type of flatMap.
The flatmap found in the standard library is a much more general and flexible method than the monadic bind method like the flatMap from Odersky's example.
For example, the full signature of flatmap on List is
def flatMap[B, That](f: (A) ⇒ GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
Instead of requiring the function passed into flatmap to return a List, it is able to return any GenTraversableOnce object, a very generic type.
flatmap then uses the implicit CanBuildFrom mechanism to determine the appropriate type to return.
So when you use flatmap with a function that returns a List, it is a monadic bind operation, but it lets you use other types as well.
In Scala, when using partially applied functions vs curried functions, we have to deal with a different way of handling type inference. Let me show it with an example, using a basic filtering function (examples taken from the excellent Functional Programming in Scala book):
1) Partially applied function
def dropWhile[A](l: List[A], f: A => Boolean): List[A] = l match {
case Nil => Nil
case x::xs if (f(x)) => dropWhile(xs, f)
case _ => l
}
2) Curried partially applied function
def dropWhileCurried[A](l: List[A])(f: A => Boolean): List[A] = l match {
case Nil => Nil
case x::xs if (f(x)) => dropWhileCurried(xs)(f)
case _ => l
}
Now, while the implementation is identical in both versions, the difference comes when we call these functions. While the curried version can be simply called like:
dropWhileCurried(List(1,2,3,4,5))(x => x < 3)
This same way (omitting type for x) cannot be used with the non curried one:
dropWhile(List(1,2,3,4,5), x => x < 3)
<console>:9: error: missing parameter type
dropWhile(List(1,2,3,4,5), x => x < 3)
So this form must be used instead:
dropWhile(List(1,2,3,4,5), (x: Int) => x < 3)
I understand this is the case, and I know there are other questions in SO regarding this fact, but what I am trying to understand is why this is the case. What is the reason for the Scala compiler to treat this two types of partially applied functions differently when it comes to type inference?
Firstly both of your examples are not partially applied functions. Partially applied function (do not confuse with Partial Functions) is the function of which only part of it's arguments applied, - but you have all your arguments in place.
But you can easily make the 2nd example into partially applied function (and curried): val a = dropWhileCurried(List(new B, new B))_. Now you have a which has only first argument applied, and you need to apply the 2nd to execute it: println(a(x => true)). You can do the same with 1st example: val a = dropWhile(List(new B, new B), _: B => Boolean).
Now as for the inference and why it works like that: I can only assume, but it sounds quite reasonable for me. You can think of each argument in the function as equal by its importance, but if inference would work and you wrote dropWhile(List(new B, new B), _ => true), you'd assume that _ is of type B, however this also is possible dropWhile(List(new B, new B), _: A => true if B extends A. In that case if you change the order of arguments the inference would change or it wouldn't work at all: dropWhile(_ => true, List(new B, new B)) And it would definetely make the inference quite complicated for the compiler to do as it has to scan the definition several times.
Now if you get back to the partial application and think of call dropWhileCurried(xs)(f) as always a partial application of xs to dropWhileCurried and then application of f to the result of previous operation, it starts to sounds reasonable. Compiler needs to infer type when you already wrote dropWhileCurried(xs) because this is a partial application (I'm still missing _ in the end though). So now, when the type is inferred, it can continue and apply (f) to it.
This is at least how I perceive the question. There might be more reasons to this, but this should help to understand some of the background if you won't receive any better answer.
I have a piece of code that I can't get to behave in the way I'd like. I have a class defined in the following way (stripped down for this):
class Behaviour[T](private val rule: Time => T) {
def map1[U, V](behaviour: Behaviour[U], func: (T, U) => V): Behaviour[V] = {
new Behaviour(time => func(this.at(time), behaviour.at(time)))
}
}
When playing around with this class I tried to something that I thought would be trivial:
val beh = Behaviour(time => 5)
val beh2 = Behaviour(time => 5)
beh.map1(beh2, (a, b) => a + b)
For the last line I receive the following error:
<console>:13: error: missing parameter type
beh.map1(beh2, (a, b) => a + b)
^
I can of course specify the closure parameter types and it works correctly but why doesn't type inference work here? Of course I could also specify the generic types for the function (see below for both solutions).
I thought Scala carried out a 'scan' to infer types and would see beh2 and passed into the function and assume U here to be Int. Is there some way I can fix this without specify the types of the input parameters (for the closure or the generics)?
EDIT: Examples of the two fixes I have:
beh.map1[Int, Int](beh2, (a, b) => a + b)
beh.map1(beh2, (a, b : Int) => a + b)
See this scala-debate thread for a discussion of what's going on here. The problem is that Scala's type inference happens per parameter list, not per parameter.
As Josh Suereth notes in that thread, there's a good reason for the current approach. If Scala had per-parameter type inference, the compiler couldn't infer an upper bound across types in the same parameter list. Consider the following:
trait X
class Y extends X
class Z extends X
val y = new Y
val z = new Z
def f[A](a: A, b: A): (A, A) = (a, b)
def g[A](a: A)(b: A): (A, A) = (a, b)
f(y, z) works exactly as we'd expect, but g(y)(z) gives a type mismatch, since by the time the compiler gets to the second argument list it's already chosen Y as the type for A.
One of the ways to fix this is to define multiple argument lists. So your map1 method would be defined like this:
def map1[U, V](behaviour: Behaviour[U])(func: (T, U) => V): Behaviour[V] = ...
and you can use it like this:
beh.map1(beh2)((a, b) => a + b)
beh.map1(beh2)(_ + _)
I'm not completely sure why type inference does not work in your case, but I believe that it has something to do with usage of U type parameter. You are using it twice - for the first and second argument. It's probably too complicated for compiler to figure it out. In case of 2 argument lists, U would be inferred during first argument list compilation, and the second argument list will use already inferred type.
This question already has answers here:
What's the difference between multiple parameters lists and multiple parameters per list in Scala?
(4 answers)
Closed 9 years ago.
Multiple parameters lists, e.g. def foo(a:Int)(b:Int) = {} and multiple parameters per list, e.g. def foo(a:Int, b:Int) = {} are semantically equivalent so far as I can tell, and most functional languages have only one way of declaring multiple parameters, e.g. F#.
The only reason I can figure out for supporting both these styles of function definitions is to allow syntax-like language extensions using a parameter list that has only one parameter in it.
def withBufferedWriter(file: File)(block: BufferedWriter => Unit)
can now be called with the syntax-looking
withBufferedWriter(new File("myfile.txt")) { out =>
out write "whatever"
...
}
However, there could be other ways of supporting the use of curly braces without having multiple parameter lists.
A related question: why is the use of multiple parameter lists in Scala called "currying"? Currying is usually defined as a technique for making an n-ary function unary for the sake of supporting partial application. However, in Scala one can partially apply a function without making a "curried" (multiple parameter lists with one param each) version of the function.
It makes you able to do e.g.:
scala> def foo(as: Int*)(bs: Int*)(cs: Int*) = as.sum * bs.sum * cs.sum
foo: (as: Int*)(bs: Int*)(cs: Int*)Int
scala> foo(1, 2, 3)(4, 5, 6, 7, 9)(10, 11)
res7: Int = 3906
As well as allowing you to write methods that look like part of the language (which you already spotted), it's worth noting that the type inferencer will work with one block at a time.
So in this:
def foo[T](a: T, b: T)(op: (T,T)=>T) = op(a,b)
foo(1,2){_+_}
T will first be inferred as Int, which will then be used as the type of the two underscores in the closure.
This is how the compiler then knows, with complete type safety, that the + operation is valid.
To answer your "related question," currying is simply a way of turning a function of multiple arguments, for example (A, B, C) => D, into a function which takes one argument and returns a function, e.g. A => (B => (C => D)) (parentheses shown but not necessary).
The tuple-ized form and the curried form are isomorphic, and we may translate freely between them. All of these are equivalent, but have different syntactic implications:
(A, B, C, D, E) => F
((A, B), (C, D, E)) => F
(A, B) => (C, D, E) => F
When you declare separate parameter groups, this is the kind of currying you're doing. The multi-parameter-group method is a method which returns a function... you can see this in the REPL:
scala> def foo(a:Int, b:Int)(c:Int, d:Int, e:Int):Int = 9
foo: (a: Int,b: Int)(c: Int,d: Int,e: Int)Int
scala> foo _
res4: (Int, Int) => (Int, Int, Int) => Int = <function2>
Back references in default arguments:
case class Foo(bar: Int)
def test(f: Foo)(i: Int = f.bar) = i*i
test(Foo(3))()
I know one of the motivations was implicit parameter lists. "implicit" is a property of the list, not the parameter. Another was probably case classes: only the first parameter list become case fields.
Following watching Nick Partidge's presentation on deriving scalaz, I got to looking at this example, which is just awesome:
import scalaz._
import Scalaz._
def even(x: Int) : Validation[NonEmptyList[String], Int]
= if (x % 2 ==0) x.success else "not even: %d".format(x).wrapNel.fail
println( even(3) <|*|> even(5) ) //prints: Failure(NonEmptyList(not even: 3, not even: 5))
I was trying to understand what the <|*|> method was doing, here is the source code:
def <|*|>[B](b: M[B])(implicit t: Functor[M], a: Apply[M]): M[(A, B)]
= <**>(b, (_: A, _: B))
OK, that is fairly confusing (!) - but it references the <**> method, which is declared thus:
def <**>[B, C](b: M[B], z: (A, B) => C)(implicit t: Functor[M], a: Apply[M]): M[C]
= a(t.fmap(value, z.curried), b)
So I have a few questions:
How come the method appears to take a higher-kinded type of one type parameter (M[B]) but can get passed a Validation (which has two type paremeters)?
The syntax (_: A, _: B) defines the function (A, B) => Pair[A,B] which the 2nd method expects: what is happening to the Tuple2/Pair in the failure case? There's no tuple in sight!
Type Constructors as Type Parameters
M is a type parameter to one of Scalaz's main pimps, MA, that represents the Type Constructor (aka Higher Kinded Type) of the pimped value. This type constructor is used to look up the appropriate instances of Functor and Apply, which are implicit requirements to the method <**>.
trait MA[M[_], A] {
val value: M[A]
def <**>[B, C](b: M[B], z: (A, B) => C)(implicit t: Functor[M], a: Apply[M]): M[C] = ...
}
What is a Type Constructor?
From the Scala Language Reference:
We distinguish between first-order
types and type constructors, which
take type parameters and yield types.
A subset of first-order types called
value types represents sets of
(first-class) values. Value types are
either concrete or abstract. Every
concrete value type can be represented
as a class type, i.e. a type
designator (§3.2.3) that refers to a
class1 (§5.3), or as a compound type
(§3.2.7) representing an intersection
of types, possibly with a refinement
(§3.2.7) that further constrains the
types of itsmembers. Abstract value
types are introduced by type
parameters (§4.4) and abstract type
bindings (§4.3). Parentheses in types
are used for grouping. We assume that
objects and packages also implicitly
define a class (of the same name as
the object or package, but
inaccessible to user programs).
Non-value types capture properties of
identifiers that are not values
(§3.3). For example, a type
constructor (§3.3.3) does not directly
specify the type of values. However,
when a type constructor is applied to
the correct type arguments, it yields
a first-order type, which may be a
value type. Non-value types are
expressed indirectly in Scala. E.g., a
method type is described by writing
down a method signature, which in
itself is not a real type, although it
gives rise to a corresponding function
type (§3.3.1). Type constructors are
another example, as one can write type
Swap[m[_, _], a,b] = m[b, a], but
there is no syntax to write the
corresponding anonymous type function
directly.
List is a type constructor. You can apply the type Int to get a Value Type, List[Int], which can classify a value. Other type constructors take more than one parameter.
The trait scalaz.MA requires that it's first type parameter must be a type constructor that takes a single type to return a value type, with the syntax trait MA[M[_], A] {}. The type parameter definition describes the shape of the type constructor, which is referred to as its Kind. List is said to have the kind '* -> *.
Partial Application of Types
But how can MA wrap a values of type Validation[X, Y]? The type Validation has a kind (* *) -> *, and could only be passed as a type argument to a type parameter declared like M[_, _].
This implicit conversion in object Scalaz converts a value of type Validation[X, Y] to a MA:
object Scalaz {
implicit def ValidationMA[A, E](v: Validation[E, A]): MA[PartialApply1Of2[Validation, E]#Apply, A] = ma[PartialApply1Of2[Validation, E]#Apply, A](v)
}
Which in turn uses a trick with a type alias in PartialApply1Of2 to partially apply the type constructor Validation, fixing the type of the errors, but leaving the type of the success unapplied.
PartialApply1Of2[Validation, E]#Apply would be better written as [X] => Validation[E, X]. I recently proposed to add such a syntax to Scala, it might happen in 2.9.
Think of this as a type level equivalent of this:
def validation[A, B](a: A, b: B) = ...
def partialApply1Of2[A, B C](f: (A, B) => C, a: A): (B => C) = (b: B) => f(a, b)
This lets you combine Validation[String, Int] with a Validation[String, Boolean], because the both share the type constructor [A] Validation[String, A].
Applicative Functors
<**> demands the the type constructor M must have associated instances of Apply and Functor. This constitutes an Applicative Functor, which, like a Monad, is a way to structure a computation through some effect. In this case the effect is that the sub-computations can fail (and when they do, we accumulate the failures).
The container Validation[NonEmptyList[String], A] can wrap a pure value of type A in this 'effect'. The <**> operator takes two effectful values, and a pure function, and combines them with the Applicative Functor instance for that container.
Here's how it works for the Option applicative functor. The 'effect' here is the possibility of failure.
val os: Option[String] = Some("a")
val oi: Option[Int] = Some(2)
val result1 = (os <**> oi) { (s: String, i: Int) => s * i }
assert(result1 == Some("aa"))
val result2 = (os <**> (None: Option[Int])) { (s: String, i: Int) => s * i }
assert(result2 == None)
In both cases, there is a pure function of type (String, Int) => String, being applied to effectful arguments. Notice that the result is wrapped in the same effect (or container, if you like), as the arguments.
You can use the same pattern across a multitude of containers that have an associated Applicative Functor. All Monads are automatically Applicative Functors, but there are even more, like ZipStream.
Option and [A]Validation[X, A] are both Monads, so you could also used Bind (aka flatMap):
val result3 = oi flatMap { i => os map { s => s * i } }
val result4 = for {i <- oi; s <- os} yield s * i
Tupling with `<|**|>`
<|**|> is really similar to <**>, but it provides the pure function for you to simply build a Tuple2 from the results. (_: A, _ B) is a shorthand for (a: A, b: B) => Tuple2(a, b)
And beyond
Here's our bundled examples for Applicative and Validation. I used a slightly different syntax to use the Applicative Functor, (fa ⊛ fb ⊛ fc ⊛ fd) {(a, b, c, d) => .... }
UPDATE: But what happens in the Failure Case?
what is happening to the Tuple2/Pair in the failure case?
If any of the sub-computations fails, the provided function is never run. It only is run if all sub-computations (in this case, the two arguments passed to <**>) are successful. If so, it combines these into a Success. Where is this logic? This defines the Apply instance for [A] Validation[X, A]. We require that the type X must have a Semigroup avaiable, which is the strategy for combining the individual errors, each of type X, into an aggregated error of the same type. If you choose String as your error type, the Semigroup[String] concatenates the strings; if you choose NonEmptyList[String], the error(s) from each step are concatenated into a longer NonEmptyList of errors. This concatenation happens below when two Failures are combined, using the ⊹ operator (which expands with implicits to, for example, Scalaz.IdentityTo(e1).⊹(e2)(Semigroup.NonEmptyListSemigroup(Semigroup.StringSemigroup)).
implicit def ValidationApply[X: Semigroup]: Apply[PartialApply1Of2[Validation, X]#Apply] = new Apply[PartialApply1Of2[Validation, X]#Apply] {
def apply[A, B](f: Validation[X, A => B], a: Validation[X, A]) = (f, a) match {
case (Success(f), Success(a)) => success(f(a))
case (Success(_), Failure(e)) => failure(e)
case (Failure(e), Success(_)) => failure(e)
case (Failure(e1), Failure(e2)) => failure(e1 ⊹ e2)
}
}
Monad or Applicative, how shall I choose?
Still reading? (Yes. Ed)
I've shown that sub-computations based on Option or [A] Validation[E, A] can be combined with either Apply or with Bind. When would you choose one over the other?
When you use Apply, the structure of the computation is fixed. All sub-computations will be executed; the results of one can't influence the the others. Only the 'pure' function has an overview of what happened. Monadic computations, on the other hand, allow the first sub-computation to influence the later ones.
If we used a Monadic validation structure, the first failure would short-circuit the entire validation, as there would be no Success value to feed into the subsequent validation. However, we are happy for the sub-validations to be independent, so we can combine them through the Applicative, and collect all the failures we encounter. The weakness of Applicative Functors has become a strength!