I'm reading a bit about Scala currying here and I don't understand this example very much:
def foldLeft[B](z: B)(op: (B, A) => B): B
What is the [B] in square brackets? Why is it in brackets? The B after the colon is the return type right? What is the type?
It looks like this method has 2 parameter lists: one with a parameter named z and one with a parameter named op which is a function.
op looks like it takes a function (B, A) => B). What does the right side mean? It returns B?
And this is apparently how it is used:
val numbers = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
val res = numbers.foldLeft(0)((m, n) => m + n)
print(res) // 55
What is going on? Why wasn't the [B] needed when called?
In Scala documentation that A type (sometimes A1) is often the placeholder for a collection's element type. So if you have...
List('c','q','y').foldLeft( //....etc.
...then A becomes the reference for Char because the list is a List[Char].
The B is a placeholder for a 2nd type that foldLeft will have to deal with. Specifically it is the type of the 1st parameter as well as the type of the foldLeft result. Most of the time you actually don't need to specify it when foldLeft is invoked because the compiler will infer it. So, yeah, it could have been...
numbers.foldLeft[Int](0)((m, n) => m + n)
...but why bother? The compiler knows it's an Int and so does anyone reading it (anyone who knows Scala).
(B, A) => B is a function that takes 2 parameters, one of type B and one of type A, and produces a result of type B.
What is the [B] in square brackets?
A type parameter (that's also known as "generics" if you've seen something like Java before)
Why is it in brackets?
Because type parameters are written in brackets, that's just Scala syntax. Java, C#, Rust, C++ use angle brackets < > for similar purposes, but since arrays in Scala are accessed as arr(idx), and (unlike Haskell or Python) Scala does not use [ ... ] for list comprehensions, the square brackets could be used for type parameters, and there was no need for angular brackets (those are more difficult to parse anyway, especially in a language which allows almost arbitrary names for infix and postfix operators).
The B after the colon is the return type right?
Right.
What is the type?
Ditto. The return type is B.
It looks like this method has 2 parameter lists: one with a parameter named z and one with a parameter named op which is a function.
This method has a type parameter B and two argument lists for value arguments, correct. This is done to simplify the type inference: the type B can be inferred from the first argument z, so it does not have to be repeated when writing down the lambda-expression for op. This wouldn't work if z and op were in the same argument list.
op looks like it takes a function (B, A) => B.
The type of the argument op is (B, A) => B, that is, Function2[B, A, B], a function that takes a B and an A and returns a B.
What does the right side mean? It returns B?
Yes.
What is going on?
m acts as accumulator, n is the current element of the list. The fold starts with integer value 0, and then accumulates from left to right, adding up all numbers. Instead of (m, n) => m + n, you could have written _ + _.
Why wasn't the [B] needed when called?
It was inferred from the type of z. There are many other cases where the generic type cannot be inferred automatically, then you would have to specify the return type explicitly by passing it as an argument in the square brackets.
This is what is called polymorphism. The function can work on multiple types and you sometimes want to give a parameter of what type will be worked with. Basically the B is a type parameter and can either be given explicitly as a type, which would be Int and then it should be given in square brackets or implicitly in parentheses like you did with the 0. Read about polymorphism here Scala polymorphism
As per the example below, calling xs.toList.map(_.toBuffer) succeeds, but xs.toBuffer.map(_.toBuffer) fails. But when the steps in the latter are performed using an intermediate result, it succeeds. What causes this inconsistency?
scala> "ab-cd".split("-").toBuffer
res0: scala.collection.mutable.Buffer[String] = ArrayBuffer(ab, cd)
scala> res0.map(_.toBuffer)
res1: scala.collection.mutable.Buffer[scala.collection.mutable.Buffer[Char]] = ArrayBuffer(ArrayBuffer(a, b), ArrayBuffer(c, d))
scala> "ab-cd".split("-").toBuffer.map(_.toBuffer)
<console>:8: error: missing parameter type for expanded function ((x$1) => x$1.toBuffer)
"ab-cd".split("-").toBuffer.map(_.toBuffer)
^
scala> "ab-cd".split("-").toList.map(_.toBuffer)
res3: List[scala.collection.mutable.Buffer[Char]] = List(ArrayBuffer(a, b), ArrayBuffer(c, d))
Look at the definitions of toBuffer and toList:
def toBuffer[A1 >: A]: Buffer[A1]
def toList: List[A]
As you can see, toBuffer is generic, while toList is not.
The reason for this difference is - I believe - that Buffer is invariant, while List is covariant.
Let's say that we have the following classes:
class Foo
class Bar extends Foo
Because List is covariant, you can call toList on an instance of Iterable[Bar] and treat the result as a List[Foo].
If List where invariant, this would not be the case.
Buffer being invariant, if toBuffer was defined as def toBuffer: Buffer[A] you would similarly not be able to treat the result
of toBuffer (on an instance of Iterable[Bar]) as an instance of Buffer[Foo] (as Buffer[Bar] is not a sub-type of Buffer[Foo], unlike for lists).
But by declaring toBuffer as def toBuffer[A1 >: A] (notice the added type parameter A1), we get back the possibility to have toBuffer return an instance of Buffer[Foo] :
all we need is to explcitly set A1 to Foo, or let the compiler infer it (if toBuffer is called at a site where a Buffer[Foo] is expected).
I think this explains the reason why toList and toBuffer are defined differently.
Now the problem with this is that toBuffer is generic, and this can badly affect inference.
When you do this:
"ab-cd".split("-").toBuffer
You never explicitly say that A1 is String, but because "ab-cd".split("-") has unambiguously the type Array[String], the compiler knows that A is String.
It also knows that A1 >: A (in toBuffer), and without any further constraint, it will infer A1 to be exactly A, in other words String.
So in the end the whole expression returns a Buffer[String].
But here's the thing: in scala, type inference happens in an expression as a whole.
When you have something like a.b.c, you might expect that scala will infer an exact type
for a, then from that infer an exact type for a.b, and finally for a.b.c. Not so.
Type inference is deferred to the whole expression a.b.c (see scala specification "6.26.4 Local Type Inference
", "case 1: selections")
So, going back to your problem, in the expression "ab-cd".split("-").toBuffer.map(_.toBuffer), the sub-expression "ab-cd".split("-").toBuffer is not typed Buffer[String], but instead
it stays typed as something like Buffer[A1] forSome A1 >: String. In other words, A1 is not fixed, we just carry the constraint A1 >: String to the next step of inference.
This next step is map(_.toBuffer), where map is defined as map[C](f: (B) ⇒ C): Buffer[B]. Here B is actually the same as A1, but at this point A1
is still not fully known, we only know that A1 >: String.
Here lies our problem. The compiler needs to know the exact type of the anonymous function (_.toBuffer) (simply because instantiating a Function1[A,R] requires to know the exact types of A and R, just like for any generic type).
So you need to tell him explcitly somehow, as it was not able to infer it exactly.
This means you need to do either:
"ab-cd".split("-").toBuffer[String].map(_.toBuffer)
Or:
"ab-cd".split("-").toBuffer.map((_:String).toBuffer)
I'm chaining transformations, and I'd like to accumulate the result of each transformation so that it can potentially be used in any subsequent step, and also so that all the results are available at the end (mostly for debugging purposes). There are several steps and from time to time I need to add a new step or change the inputs for a step.
HList seems to offer a convenient way to collect the results in a flexible but still type-safe way. But I'd rather not complicate the actual steps by making them deal with the HList and the accompanying business.
Here's a simplified version of the combinator I'd like to write, which isn't working. The idea is that given an HList containing an A, and the index of A, and a function from A -> B, mapNth will extract the A, run the function, and cons the result onto the list. The resulting extended list captures the type of the new result, so several of these mapNth-ified steps can be composed to produce a list containing the result from each step:
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At[L, index.N]):
B :: L =
f(l(index)) :: l
Incidentally, I'll also need map2Nth taking two indices and f: (A, B) => C, but I believe the issues are the same.
However, mapNth does not compile, saying l(index) has type at.Out, but f's argument should be A. That's correct, of course, so what I suppose I need is a way to provide evidence that at.Out is in fact A (or, at.Out <: A).
Is there a way to express that constraint? I believe it will have to take the form of an implicit, because of course the constraint can only be checked when mapNth is applied to a particular list and function.
You're exactly right about needing evidence that at.Out is A, and you can provide that evidence by including the value of the type member in at's type:
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At[L, index.N] { type Out = A }):
B :: L =
f(l(index)) :: l
The companion objects for type classes like At in Shapeless also define an Aux type that includes the output type as a final type parameter.
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At.Aux[L, index.N, A]):
B :: L =
f(l(index)) :: l
This is pretty much equivalent but more idiomatic (and it looks a little nicer).
I was trying to reverse a List of Integers as follows:
List(1,2,3,4).foldLeft(List[Int]()){(a,b) => b::a}
My question is that is there a way to specify the seed to be some List[_] where the _ is the type automatically filled in by scala's type-inference mechanism, instead of having to specify the type as List[Int]?
Thanks
Update: After reading a bit more on Scala's type inference, I found a better answer to your question. This article which is about the limitations of the Scala type inference says:
Type information in Scala flows from function arguments to their results [...], from left to right across argument lists, and from first to last across statements. This is in contrast to a language with full type inference, where (roughly speaking) type information flows unrestricted in all directions.
So the problem is that Scala's type inference is rather limited. It first looks at the first argument list (the list in your case) and then at the second argument list (the function). But it does not go back.
This is why neither this
List(1,2,3,4).foldLeft(Nil){(a,b) => b::a}
nor this
List(1,2,3,4).foldLeft(List()){(a,b) => b::a}
will work. Why? First, the signature of foldLeft is defined as:
foldLeft[B](z: B)(f: (B, A) => B): B
So if you use Nil as the first argument z, the compiler will assign Nil.type to the type parameter B. And if you use List(), the compiler will use List[Nothing] for B.
Now, the type of the second argument f is fully defined. In your case, it's either
(Nil.type, Int) => Nil.type
or
(List[Nothing], Int) => List[Nothing]
And in both cases the lambda expression (a, b) => b :: a is not valid, since its return type is inferred to be List[Int].
Note that the bold part above says "argument lists" and not "arguments". The article later explains:
Type information does not flow from left to right within an argument list, only from left to right across argument lists.
So the situation is even worse if you have a method with a single argument list.
The only way I know how is
scala> def foldList[T](l: List[T]) = l.foldLeft(List[T]()){(a,b) => b::a}
foldList: [T](l: List[T])List[T]
scala> foldList(List(1,2,3,4))
res19: List[Int] = List(4, 3, 2, 1)
scala> foldList(List("a","b","c"))
res20: List[java.lang.String] = List(c, b, a)
I have read that with a statically typed language like Scala or Haskell there is no way to create or provide a Lisp apply function:
(apply #'+ (list 1 2 3)) => 6
or maybe
(apply #'list '(list :foo 1 2 "bar")) => (:FOO 1 2 "bar")
(apply #'nth (list 1 '(1 2 3))) => 2
Is this a truth?
It is perfectly possible in a statically typed language. The whole java.lang.reflect thingy is about doing that. Of course, using reflection gives you as much type safety as you have with Lisp. On the other hand, while I do not know if there are statically typed languages supporting such feature, it seems to me it could be done.
Let me show how I figure Scala could be extended to support it. First, let's see a simpler example:
def apply[T, R](f: (T*) => R)(args: T*) = f(args: _*)
This is real Scala code, and it works, but it won't work for any function which receives arbitrary types. For one thing, the notation T* will return a Seq[T], which is a homegenously-typed sequence. However, there are heterogeneously-typed sequences, such as the HList.
So, first, let's try to use HList here:
def apply[T <: HList, R](f: (T) => R)(args: T) = f(args)
That's still working Scala, but we put a big restriction on f by saying it must receive an HList, instead of an arbitrary number of parameters. Let's say we use # to make the conversion from heterogeneous parameters to HList, the same way * converts from homogeneous parameters to Seq:
def apply[T, R](f: (T#) => R)(args: T#) = f(args: _#)
We aren't talking about real-life Scala anymore, but an hypothetical improvement to it. This looks reasonably to me, except that T is supposed to be one type by the type parameter notation. We could, perhaps, just extend it the same way:
def apply[T#, R](f: (T#) => R)(args: T#) = f(args: _#)
To me, it looks like that could work, though that may be naivety on my part.
Let's consider an alternate solution, one depending on unification of parameter lists and tuples. Let's say Scala had finally unified parameter list and tuples, and that all tuples were subclass to an abstract class Tuple. Then we could write this:
def apply[T <: Tuple, R](f: (T) => R)(args: T) = f(args)
There. Making an abstract class Tuple would be trivial, and the tuple/parameter list unification is not a far-fetched idea.
A full APPLY is difficult in a static language.
In Lisp APPLY applies a function to a list of arguments. Both the function and the list of arguments are arguments to APPLY.
APPLY can use any function. That means that this could be any result type and any argument types.
APPLY takes arbitrary arguments in arbitrary length (in Common Lisp the length is restricted by an implementation specific constant value) with arbitrary and possibly different types.
APPLY returns any type of value that is returned by the function it got as an argument.
How would one type check that without subverting a static type system?
Examples:
(apply #'+ '(1 1.4)) ; the result is a float.
(apply #'open (list "/tmp/foo" :direction :input))
; the result is an I/O stream
(apply #'open (list name :direction direction))
; the result is also an I/O stream
(apply some-function some-arguments)
; the result is whatever the function bound to some-function returns
(apply (read) (read))
; neither the actual function nor the arguments are known before runtime.
; READ can return anything
Interaction example:
CL-USER 49 > (apply (READ) (READ)) ; call APPLY
open ; enter the symbol OPEN
("/tmp/foo" :direction :input :if-does-not-exist :create) ; enter a list
#<STREAM::LATIN-1-FILE-STREAM /tmp/foo> ; the result
Now an example with the function REMOVE. We are going to remove the character a from a list of different things.
CL-USER 50 > (apply (READ) (READ))
remove
(#\a (1 "a" #\a 12.3 :foo))
(1 "a" 12.3 :FOO)
Note that you also can apply apply itself, since apply is a function.
CL-USER 56 > (apply #'apply '(+ (1 2 3)))
6
There is also a slight complication because the function APPLY takes an arbitrary number of arguments, where only the last argument needs to be a list:
CL-USER 57 > (apply #'open
"/tmp/foo1"
:direction
:input
'(:if-does-not-exist :create))
#<STREAM::LATIN-1-FILE-STREAM /tmp/foo1>
How to deal with that?
relax static type checking rules
restrict APPLY
One or both of above will have to be done in a typical statically type checked programming language. Neither will give you a fully statically checked and fully flexible APPLY.
The reason you can't do that in most statically typed languages is that they almost all choose to have a list type that is restricted to uniform lists. Typed Racket is an example for a language that can talk about lists that are not uniformly typed (eg, it has a Listof for uniform lists, and List for a list with a statically known length that can be non-uniform) -- but still it assigns a limited type (with uniform lists) for Racket's apply, since the real type is extremely difficult to encode.
It's trivial in Scala:
Welcome to Scala version 2.8.0.final ...
scala> val li1 = List(1, 2, 3)
li1: List[Int] = List(1, 2, 3)
scala> li1.reduceLeft(_ + _)
res1: Int = 6
OK, typeless:
scala> def m1(args: Any*): Any = args.length
m1: (args: Any*)Any
scala> val f1 = m1 _
f1: (Any*) => Any = <function1>
scala> def apply(f: (Any*) => Any, args: Any*) = f(args: _*)
apply: (f: (Any*) => Any,args: Any*)Any
scala> apply(f1, "we", "don't", "need", "no", "stinkin'", "types")
res0: Any = 6
Perhaps I mixed up funcall and apply, so:
scala> def funcall(f: (Any*) => Any, args: Any*) = f(args: _*)
funcall: (f: (Any*) => Any,args: Any*)Any
scala> def apply(f: (Any*) => Any, args: List[Any]) = f(args: _*)
apply: (f: (Any*) => Any,args: List[Any])Any
scala> apply(f1, List("we", "don't", "need", "no", "stinkin'", "types"))
res0: Any = 6
scala> funcall(f1, "we", "don't", "need", "no", "stinkin'", "types")
res1: Any = 6
It is possible to write apply in a statically-typed language, as long as functions are typed a particular way. In most languages, functions have individual parameters terminated either by a rejection (i.e. no variadic invocation), or a typed accept (i.e. variadic invocation possible, but only when all further parameters are of type T). Here's how you might model this in Scala:
trait TypeList[T]
case object Reject extends TypeList[Reject]
case class Accept[T](xs: List[T]) extends TypeList[Accept[T]]
case class Cons[T, U](head: T, tail: U) extends TypeList[Cons[T, U]]
Note that this doesn't enforce well-formedness (though type bounds do exist for that, I believe), but you get the idea. Then you have apply defined like this:
apply[T, U]: (TypeList[T], (T => U)) => U
Your functions, then, are defined in terms of type list things:
def f (x: Int, y: Int): Int = x + y
becomes:
def f (t: TypeList[Cons[Int, Cons[Int, Reject]]]): Int = t.head + t.tail.head
And variadic functions like this:
def sum (xs: Int*): Int = xs.foldLeft(0)(_ + _)
become this:
def sum (t: TypeList[Accept[Int]]): Int = t.xs.foldLeft(0)(_ + _)
The only problem with all of this is that in Scala (and in most other static languages), types aren't first-class enough to define the isomorphisms between any cons-style structure and a fixed-length tuple. Because most static languages don't represent functions in terms of recursive types, you don't have the flexibility to do things like this transparently. (Macros would change this, of course, as well as encouraging a reasonable representation of function types in the first place. However, using apply negatively impacts performance for obvious reasons.)
In Haskell, there is no datatype for multi-types lists, although I believe, that you can hack something like this together whith the mysterious Typeable typeclass. As I see, you're looking for a function, which takes a function, a which contains exactly the same amount of values as needed by the function and returns the result.
For me, this looks very familiar to haskells uncurryfunction, just that it takes a tuple instead of a list. The difference is, that a tuple has always the same count of elements (so (1,2) and (1,2,3) are of different types (!)) and there contents can be arbitrary typed.
The uncurry function has this definition:
uncurry :: (a -> b -> c) -> (a,b) -> c
uncurry f (a,b) = f a b
What you need is some kind of uncurry which is overloaded in a way to provide an arbitrary number of params. I think of something like this:
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE UndecidableInstances #-}
class MyApply f t r where
myApply :: f -> t -> r
instance MyApply (a -> b -> c) (a,b) c where
myApply f (a,b) = f a b
instance MyApply (a -> b -> c -> d) (a,b,c) d where
myApply f (a,b,c) = f a b c
-- and so on
But this only works, if ALL types involved are known to the compiler. Sadly, adding a fundep causes the compiler to refuse compilation. As I'm not a haskell guru, maybe domeone else knows, howto fix this. Sadly, I don't know how to archieve this easier.
Résumee: apply is not very easy in Haskell, although possible. I guess, you'll never need it.
Edit I have a better idea now, give me ten minutes and I present you something whithout these problems.
try folds. they're probably similar to what you want. just write a special case of it.
haskell: foldr1 (+) [0..3] => 6
incidentally, foldr1 is functionally equivalent to foldr with the accumulator initialized as the element of the list.
there are all sorts of folds. they all technically do the same thing, though in different ways, and might do their arguments in different orders. foldr is just one of the simpler ones.
On this page, I read that "Apply is just like funcall, except that its final argument should be a list; the elements of that list are treated as if they were additional arguments to a funcall."
In Scala, functions can have varargs (variadic arguments), like the newer versions of Java. You can convert a list (or any Iterable object) into more vararg parameters using the notation :_* Example:
//The asterisk after the type signifies variadic arguments
def someFunctionWithVarargs(varargs: Int*) = //blah blah blah...
val list = List(1, 2, 3, 4)
someFunctionWithVarargs(list:_*)
//equivalent to
someFunctionWithVarargs(1, 2, 3, 4)
In fact, even Java can do this. Java varargs can be passed either as a sequence of arguments or as an array. All you'd have to do is convert your Java List to an array to do the same thing.
The benefit of a static language is that it would prevent you to apply a function to the arguments of incorrect types, so I think it's natural that it would be harder to do.
Given a list of arguments and a function, in Scala, a tuple would best capture the data since it can store values of different types. With that in mind tupled has some resemblance to apply:
scala> val args = (1, "a")
args: (Int, java.lang.String) = (1,a)
scala> val f = (i:Int, s:String) => s + i
f: (Int, String) => java.lang.String = <function2>
scala> f.tupled(args)
res0: java.lang.String = a1
For function of one argument, there is actually apply:
scala> val g = (i:Int) => i + 1
g: (Int) => Int = <function1>
scala> g.apply(2)
res11: Int = 3
I think if you think as apply as the mechanism to apply a first class function to its arguments, then the concept is there in Scala. But I suspect that apply in lisp is more powerful.
For Haskell, to do it dynamically, see Data.Dynamic, and dynApp in particular: http://www.haskell.org/ghc/docs/6.12.1/html/libraries/base/Data-Dynamic.html
See his dynamic thing for haskell, in C, void function pointers can be casted to other types, but you'd have to specify the type to cast it to. (I think, haven't done function pointers in a while)
A list in Haskell can only store values of one type, so you couldn't do funny stuff like (apply substring ["Foo",2,3]). Neither does Haskell have variadic functions, so (+) can only ever take two arguments.
There is a $ function in Haskell:
($) :: (a -> b) -> a -> b
f $ x = f x
But that's only really useful because it has very low precedence, or as passing around HOFs.
I imagine you might be able to do something like this using tuple types and fundeps though?
class Apply f tt vt | f -> tt, f -> vt where
apply :: f -> tt -> vt
instance Apply (a -> r) a r where
apply f t = f t
instance Apply (a1 -> a2 -> r) (a1,a2) r where
apply f (t1,t2) = f t1 t2
instance Apply (a1 -> a2 -> a3 -> r) (a1,a2,a3) r where
apply f (t1,t2,t3) = f t1 t2 t3
I guess that's a sort of 'uncurryN', isn't it?
Edit: this doesn't actually compile; superseded by #FUZxxl's answer.