I want to define a function f that takes another function g. We require g to take take n Doubles (for some fixed n) and return a Double. The function call f(g) should return the specific value of n.
For example, f(Math.max) = 2 since Math.sin has type (Double, Double) => Double, and f(Math.sin) = 1 since Math.sin has type Double => Double.
How can I define f using Scala generics?
I've tried several forms without success. For example:
def f[A <: Product](g: Product => Double) = {...}
This doesn't work since we cannot extract the value of n at compile time, and cannot constrain the A to contain only Double values.
There is a pattern called Magnet Pattern, created by the Spray team. It does exectly what you want
This was a good excuse for me to look into Shapeless, something I always wanted to do at some point :)
$ git clone git#github.com:milessabin/shapeless.git
...
$ cd shapeless
(1)
Shapeless provides some abstractions over arity, and especially the representation as heterogeneous list (HList). A function of arbitrary arity can be seen as FnHList (a function that takes an HList as argument).
$ sbt shapeless-core/console
scala> import shapeless._
import shapeless._
scala> def isFunction[A](fun: A)(implicit fnh: FnHLister[A]) {}
isFunction: [A](fun: A)(implicit fnh: shapeless.FnHLister[A])Unit
scala> isFunction(math.sqrt _)
scala> isFunction(math.random _)
(2)
Now let's require that the function returns a Double:
scala> def isFunReturningDouble[A](fun: A)(implicit fnh: FnHLister[A] { type Result = Double }) {}
isFunReturningDouble: [A](fun: A)(implicit fnh: shapeless.FnHLister[A]{type Result = Double})Unit
scala> isFunReturningDouble(math.sqrt _)
scala> isFunReturningDouble(math.signum _)
<console>:12: error: could not find implicit value for parameter fnh: shapeless.FnHLister[Int => Int]{type Result = Double}
isFunReturningDouble(math.signum _)
^
(3)
The LUBConstraint type class can witness the upper bound of the argument list:
scala> def isValidFun[A, B <: HList](fun: A)(implicit fnh: FnHLister[A] { type Result = Double; type Args = B }, lub: LUBConstraint[B, Double]) {}
isValidFun: [A, B <: shapeless.HList](fun: A)(implicit fnh: shapeless.FnHLister[A]{type Result = Double; type Args = B}, implicit lub: shapeless.LUBConstraint[B,Double])Unit
scala> isValidFun(math.random _)
scala> isValidFun((i: Int) => i.toDouble)
<console>:12: error: could not find implicit value for parameter lub: shapeless.LUBConstraint[B,Double]
isValidFun((i: Int) => i.toDouble)
^
(4)
Now we still need to extract the arity somehow. On the type level this would be Length which is provided for HList. To get a runtime value, another type class ToInt is needed.
Here is the final function:
import shapeless._
def doubleFunArity[A, B <: HList, C <: Nat](fun: A)(implicit
fnh: FnHLister[A] { type Result = Double; type Args = B },
lub: LUBConstraint[B, Double],
len: Length[B] { type Out = C },
res: ToInt[C]
): Int = res()
Test:
scala> doubleFunArity(math.sqrt _)
res15: Int = 1
scala> doubleFunArity(math.random _)
res16: Int = 0
scala> val g: (Double, Double) => Double = math.max _
g: (Double, Double) => Double = <function2>
scala> doubleFunArity(g)
res17: Int = 2
Note that unfortunately many math operations are overloaded, and without strong type constraint, Scala will not give you the Double version automatically, but will use the Int version for some reason:
scala> math.max _
res18: (Int, Int) => Int = <function2>
So I need the indirection math.max _: ((Double, Double) => Double) to make this work.
Not saying that this is the best way to do it in your concrete case, but I think it was a fun exploration.
Probably the easiest solution is to use overloading as
def f(g: () => Double) = 0;
def f(g: (Double) => Double) = 1;
def f(g: (Double, Double) => Double) = 2;
def f(g: (Double, Double, Double) => Double) = 2;
// ...
println(f(Math.pow _));
println(f(Math.sin _));
(You can't check function argument/return types at run time due to type erasure, so I believe you can't create a fully generic function that would satisfy your requirements.)
Related
trouble-shooting env: sbt console (Scala 2.11.8) & spark-shell (Spark 2.3, Scala 2.11)
I have a higher-order function with a view bound type T... but that arg t: T type signature turns from T <% Double to Nothing when the function is partially applied.
Toy example to demonstrate:
// tot: T needs to work on (at least) Int, Long, Double, Float
// no common supertype -> some kind of context bound
def func[T <% Double](isValid: Boolean)(tot: T, cnt: Int): Double =
if (isValid) tot.toDouble / cnt else Double.NaN
When I try to partially apply isValid, I'd expect the result to be type (T, Int) => Double but instead the type ends up (Nothing, Int) => Double, and I cannot pass in arg tot.
val f1 = func(true)_ // f1: (Nothing, Int) => Double = <function2>
val f2 = func(false)_ // f2: (Nothing, Int) => Double = <function2>
val g1 = f1(10.0, 1)
// <console>:40: error: type mismatch;
// found : Double(10.0)
// required: Nothing
// val g1 = f1(10.0, 1)
I'm not getting any error messages when defining f1 or f2... so it's hard to interpret. It just converts arg tot: T to type Nothing.
Checking scala doc... I see scala.Nothing is a subtype of EVERY other type, so I thought maybe it was losing the view bound on T... which was maybe related to type erasure... so I tried using ClassTag...
import scala.reflect.ClassTag
def func[T <% Double](isValid: Boolean)(tot: T, cnt: Int)(implicit tag: ClassTag[T]): Double =
if (isValid) tot.toDouble / cnt else Double.NaN
That doesn't help. Same problem.
if I try using implicit num: Numeric[T] it chokes on type Nothing in a new way...
def func[T](isValid: Boolean)(tot: T, cnt: Int)( implicit num: Numeric[T] ): Double =
if (isValid) num.toDouble(tot) / cnt else Double.NaN
val f1 = func(true)_
// <console>:40: error: could not find implicit value for parameter num: Numeric[Nothing]
// val f1 = func(true)_
If I apply it all at once (using first 'func' at top), it works fine...
val g1 = func(true)(10.0, 1)
// g1: Double = 10.0
But in my real (non-toy) code, that's not an option.
What is happening here, and how can I make func work when partially applied?
EDIT [#Alexey's solution]
I can't get the preferred 'def' approach to work.
def func[T <% Double](isValid: Boolean)(tot: T, cnt: Int): Double =
if (isValid) tot.toDouble / cnt else Double.NaN
// func: [T](isValid: Boolean)(tot: T, cnt: Int)(implicit evidence$1: T => Double)Double
def f1[T <% Double]: ((T, Int) => Double) = func[T](true)_
// f1: [T](implicit evidence$1: T => Double)(T, Int) => Double
f1[Double](10.0, 1)
<console>:41: error: too many arguments for method f1: (implicit evidence$1: Double => Double)(Double, Int) => Double
f1[Double](10.0, 1)
When I try to partially apply isValid, I'd expect the result to be type (T, Int) => Double
Values can't be generic. So it can have this type for some specific T, but you don't provide exactly the parameters which would allow inferring it. You can specify e.g.
val f1 = func[TheTypeYouWant](true) _
or
val f1: (TheTypeYouWant, Int) => Double = func(true) _
If you want it to be generic, it has to be a def again:
def f1[T <% Double] = func[T](true) _
How can I make this work? My task at hand is a little bit more complicated but it boils down to this:
object Z {
class B extends Function1[Int, Int] {
def apply(i: Int): Int = i
}
def compose[T <: Function1[X, X], X](fcts: List[T]): Function1[X, X] = {
fcts.reduce(_ andThen _)
}
def test() = {
val fcts = List.empty[B]
// Unspecified type parameter X
val composed: Function1[Int, Int] = compose[B](fcts)
}
}
I don't know how to define the "compose" function to be able to receive some concrete class B and automatically infer the dependent types X
The Scala compiler does not do well when trying to infer multiple levels of type parameters like you have. Instead, it would be simpler to remove T <: Function1[X, X] and simply require a single type parameter that represents the argument and return type of the Function1.
def compose[A](fcts: List[Function1[A, A]]): Function1[A, A] = {
fcts.reduce(_ andThen _)
}
The compiler will have a much easier time simply inferring A, instead of trying to figure out what with T and X are, when X is part of type T.
val a: Int => Int = _ + 10
val b: Int => Int = _ * 2
val c: Int => Int = _ - 3
scala> val f = compose(List(a, b, c))
f: Int => Int = scala.Function1$$Lambda$1187/930987088#531ec2ca
scala> f(2)
res1: Int = 21
Note that reduce will throw an exception for an empty list of functions.
Update (2018): my prayers were answered in Dotty (Type Lambdas), so the following Q&A is more "Scala 2.x"-related
Just a simple example from Scala:
scala> def f(x: Int) = x
f: (x: Int)Int
scala> (f _)(5)
res0: Int = 5
Let's make it generic:
scala> def f[T](x: T) = x
f: [T](x: T)T
scala> (f _)(5)
<console>:9: error: type mismatch;
found : Int(5)
required: Nothing
(f _)(5)
^
Let's look at eta-expansion of polymorphic method in Scala:
scala> f _
res2: Nothing => Nothing = <function1>
Comparison with Haskell:
Prelude> let f x = x
Prelude> f 5
5
Prelude> f "a"
"a"
Prelude> :t f
f :: t -> t
Haskell did infer correct type [T] => [T] here.
More realistic example?
scala> identity _
res2: Nothing => Nothing = <function1>
Even more realistic:
scala> def f[T](l: List[T]) = l.head
f: [T](l: List[T])T
scala> f _
res3: List[Nothing] => Nothing = <function1>
You can't make alias for identity - have to write your own function. Things like [T,U](t: T, u: U) => t -> u (make tuple) are impossible to use as values. More general - if you want to pass some lambda that rely on generic type (e.g. uses generic function, for example: creates lists, tuples, modify them in some way) - you can't do that.
So, how to solve that problem? Any workaround, solution or reasoning?
P.S. I've used term polymorphic lambda (instead of function) as function is just named lambda
Only methods can be generic on the JVM/Scala, not values. You can make an anonymous instance that implements some interface (and duplicate it for every type-arity you want to work with):
trait ~>[A[_], B[_]] { //exists in scalaz
def apply[T](a: A[T]): B[T]
}
val f = new (List ~> Id) {
def apply[T](a: List[T]) = a.head
}
Or use shapeless' Poly, which supports more complicated type-cases. But yeah, it's a limitation and it requires working around.
P∀scal is a compiler plugin that provides more concise syntax for encoding polymorphic values as objects with a generic method.
The identity function, as a value, has type ∀A. A => A. To translate that into Scala, assume a trait
trait ForAll[F[_]] {
def apply[A]: F[A]
}
Then the identity function has type ForAll[λ[A => A => A]], where I use the kind-projector syntax, or, without kind-projector:
type IdFun[A] = A => A
type PolyId = ForAll[IdFun]
And now comes the P∀scal syntactic sugar:
val id = Λ[Α](a => a) : PolyId
or equivalently
val id = ν[PolyId](a => a)
("ν" is the Greek lowercase letter "Nu", read "new")
These are really just shorthands for
new PolyId {
def apply[A] = a => a
}
Multiple type parameters and parameters of arbitrary kinds are supported by P∀scal, but you need a dedicated variation on the above ForAll trait for each variant.
I really like #Travis Brown 's solution:
import shapeless._
scala> Poly(identity _)
res2: shapeless.PolyDefns.~>[shapeless.Id,shapeless.Id] = fresh$macro$1$2$#797aa352
-
scala> def f[T](x: T) = x
f: [T](x: T)T
scala> Poly(f _)
res3: shapeless.PolyDefns.~>[shapeless.Id,shapeless.Id] = fresh$macro$2$2$#664ea816
-
scala> def f[T](l: List[T]) = l.head
f: [T](l: List[T])T
scala> val ff = Poly(f _)
ff: shapeless.PolyDefns.~>[List,shapeless.Id] = fresh$macro$3$2$#51254c50
scala> ff(List(1,2,3))
res5: shapeless.Id[Int] = 1
scala> ff(List("1","2","3"))
res6: shapeless.Id[String] = 1
Poly constructor (in some cases) will give you eta-expansion into Shapeless2 Poly1 function, which is (more-less) truly generic. However it doesn't work for multi-parameters (even with multi type-parameters), so have to "implement" Poly2 with implicit + at approach (as #som-snytt suggested), something like:
object myF extends Poly2 {
implicit def caseA[T, U] = at[T, U]{ (a, b) => a -> b}
}
scala> myF(1,2)
res15: (Int, Int) = (1,2)
scala> myF("a",2)
res16: (String, Int) = (a,2)
P.S. I would really want to see it as a part of language.
It seems to do this you will need to do a bit type hinting to help the Scala type inference system.
def id[T] : T => T = identity _
So I guess if you try to pass identity as a parameter to a function call and the types of that parameter are generic then there should be no problem.
Further to my other question about reduceLeft, the signature of reduceLeft on Seq is
def reduceLeft [B >: A] (f: (B, A) ⇒ B): B
and we can call it with expressions such as
List(1,2,3,4) reduceLeft (_ + _)
In this example A is Int, so reduceLeft expects a Function2[B >: Int, Int, B]. Regardless of how reduceLeft works (which is irrelevant), how does the type inferencer know that B has a + method, when it could be of type Any?
I think the section 6.26.4 Local Type Inference of the spec sort of explains what's going on. The compiler will look for an optimal type. When the type parameter is contravariant the type chosen will be maximal (in this case Any) and otherwise (invariant or covariant) minimal (in this case Int).
There are a couple examples which I can't really relate to reduceLeft.
What I did notice is the inference seems to happen before looking at the anonymous function passed:
scala> List(1,2).reduceLeft[Any](_.toString + _)
res26: Any = 12
But If I don't help the type inferencer:
scala> List(1,2).reduceLeft(_.toString + _)
<console>:8: error: type mismatch;
found : java.lang.String
required: Int
List(1,2).reduceLeft(_.toString + _)
Edit, I'm wrong the anonymous function is taken into account, this works:
List(1,2).reduceLeft((_:Any).toString + (_:Any).toString)
There is a compiler -Ytyper-debug option that you can run on:
List(1,2).reduceLeft(_+_)
It will show you that somehow the compiler assumes the expected type of the anonymous function is (Int, Int) => Int, then it proceeds to check the _ + _ against it and succeeds and then infers B as Int. Snippet here:
typed immutable.this.List.apply[Int](1, 2).reduceLeft: [B >: Int](f: (B, Int) => B)B
adapted immutable.this.List.apply[Int](1, 2).reduceLeft: [B >: Int](f: (B, Int) => B)B to ?, undetparams=type B
typing ((x$1, x$2) => x$1.$plus(x$2)): pt = (Int, Int) => Int: undetparams=,
// some time later
typed ((x$1: Int, x$2: Int) => x$1.+(x$2)): (Int, Int) => Int
adapted ((x$1: Int, x$2: Int) => x$1.+(x$2)): (Int, Int) => Int to (Int, Int) => Int,
typed immutable.this.List.apply[Int](1, 2).reduceLeft[Int](((x$1: Int, x$2: Int) => x$1.+(x$2))): Int
I don't know why in absence of type ascription the anonymous function is assumed to be (Int, Int) => Int.
If B >: X and the compiler knows X but cannot resolve B it simply assumes B = X.
It is somewhat practical since it only has two options for B and only one is known. So absent knowing which super class it assumes that B is X. You can test the compilers decision making process with the following code.
class Y {
def bar(y:Y) = this
}
case class X( i: Int ) extends Y {
def foo(x:X)=X(i+x.i)
}
val t = new Y bar X(7)
val t2 = X(8) bar X(7)
val res = List(X(1),X(2),X(3)) reduceLeft { _ foo _ }
val res2 = List(X(1),X(2),X(3)) reduceLeft { _ bar _ } // will not compile
I would like to map the elements of a Scala tuple (or triple, ...) using a single function returning type R. The result should be a tuple (or triple, ...) with elements of type R.
OK, if the elements of the tuple are from the same type, the mapping is not a problem:
scala> implicit def t2mapper[A](t: (A,A)) = new { def map[R](f: A => R) = (f(t._1),f(t._2)) }
t2mapper: [A](t: (A, A))java.lang.Object{def map[R](f: (A) => R): (R, R)}
scala> (1,2) map (_ + 1)
res0: (Int, Int) = (2,3)
But is it also possible to make this solution generic, i.e. to map tuples that contain elements of different types in the same manner?
Example:
class Super(i: Int)
object Sub1 extends Super(1)
object Sub2 extends Super(2)
(Sub1, Sub2) map (_.i)
should return
(1,2): (Int, Int)
But I could not find a solution so that the mapping function determines the super type of Sub1 and Sub2. I tried to use type boundaries, but my idea failed:
scala> implicit def t2mapper[A,B](t: (A,B)) = new { def map[X >: A, X >: B, R](f: X => R) = (f(t._1),f(t._2)) }
<console>:8: error: X is already defined as type X
implicit def t2mapper[A,B](t: (A,B)) = new { def map[X >: A, X >: B, R](f: X => R) = (f(t._1),f(t._2)) }
^
<console>:8: error: type mismatch;
found : A
required: X
Note: implicit method t2mapper is not applicable here because it comes after the application point and it lacks an explicit result type
implicit def t2mapper[A,B](t: (A,B)) = new { def map[X >: A, X >: B, R](f: X => R) = (f(t._1),f(t._2)) }
Here X >: B seems to override X >: A. Does Scala not support type boundaries regarding multiple types? If yes, why not?
I think this is what you're looking for:
implicit def t2mapper[X, A <: X, B <: X](t: (A,B)) = new {
def map[R](f: X => R) = (f(t._1), f(t._2))
}
scala> (Sub1, Sub2) map (_.i)
res6: (Int, Int) = (1,2)
A more "functional" way to do this would be with 2 separate functions:
implicit def t2mapper[A, B](t: (A, B)) = new {
def map[R](f: A => R, g: B => R) = (f(t._1), g(t._2))
}
scala> (1, "hello") map (_ + 1, _.length)
res1: (Int, Int) = (2,5)
I’m not a scala type genius but maybe this works:
implicit def t2mapper[X, A<:X, B<:X](t: (A,B)) = new { def map[A, B, R](f: X => R) = (f(t._1),f(t._2)) }
This can easily be achieved using shapeless, although you'll have to define the mapping function first before doing the map:
object fun extends Poly1 {
implicit def value[S <: Super] = at[S](_.i)
}
(Sub1, Sub2) map fun // typed as (Int, Int), and indeed equal to (1, 2)
(I had to add a val in front of i in the definition of Super, this way: class Super(val i: Int), so that it can be accessed outside)
The deeper question here is "why are you using a Tuple for this?"
Tuples are hetrogenous by design, and can contain an assortment of very different types. If you want a collection of related things, then you should be using ...drum roll... a collection!
A Set or Sequence will have no impact on performance, and would be a much better fit for this kind of work. After all, that's what they're designed for.
For the case when the two functions to be applied are not the same
scala> Some((1, "hello")).map((((_: Int) + 1 -> (_: String).length)).tupled).get
res112: (Int, Int) = (2,5)
The main reason I have supplied this answer is it works for lists of tuples (just change Some to List and remove the get).