Let's suppose I have a def that takes multiple type parameters:
def foo[A, B, C](b: B, c: C)(implicit ev: Writer[A])
However, the intended usage is that type parameters B and C should be inferred (based on the passed-in arguments). And the caller should only need to really specify A explicitly (e.g. to have an appropriate implicit chosen by the compiler). Unfortunately, Scala only allows all or none of the type parameters to be specified by the caller. In a sense, I want the type parameters to be curried:
def foo[A][B, C]...
Is there some trick to accomplish this in Scala?
(If my specific example doesn't make complete sense I'm happy to improve it with suggestions.)
The best way I've been able to pull this off is by defining a class which holds the curried type information then uses the apply method to simulate the function call.
I've written about this here - http://caryrobbins.com/dev/scala-type-curry/
For your specific example, you'd need to put the implicit ev: Writes[A] in the signature for the apply and not in the signature for foo. This is because it causes ambiguity between explicitly passing the implicit argument or implicitly calling the apply method.
Here's an example implementation for your example -
object Example {
def foo[A]: _Foo[A] = _foo.asInstanceOf[_Foo[A]]
final class _Foo[A] private[Example] {
def apply[B, C](b: B, c: C)(implicit ev: Writes[A]): Unit = ???
}
private lazy val _foo = new _Foo[Nothing]
}
You can then supply your type parameter you wish to curry and the following arguments passed to the apply method will be inferred.
Example.foo[Int]("bar", new Object)
If you do end up needing to specify the other type parameters, you can do so by explicitly calling apply; although, I've never seen a need to do this yet.
Example.foo[Int].apply[String, Object]("bar", new Object)
If you don't wish to use the intermediate type you can also use a structural type, which I discuss in the aforementioned post; however, this requires reflectiveCalls and an inferred type signature, both of which I like to avoid.
Is there a way to retrieve the arity of a function? For instance, given the following function:
def moop[I,O](func: I => O) = {
val arity = I.???
???
}
How would I get the arity of func?
If I is a tuple, you can call the productArity method on it which is inherited from Product.
If you have an object of which you only know it is some kind of function, you will have to check against the (finite) set of FunctionN classes since there is no common supertype as for tuples. It looks like we currently have Function0 up to Function22.
I'm pretty sure I'm missing something here, since I'm pretty new to Shapeless and I'm learning, but when is the Aux technique actually required? I see that it is used to expose a type statement by raising it up into the signature of another "companion" type definition.
trait F[A] { type R; def value: R }
object F { type Aux[A,RR] = F[A] { type R = RR } }
but isn't this nearly equivalent to just putting R in the type signature of F ?
trait F[A,R] { def value: R }
implicit def fint = new F[Int,Long] { val value = 1L }
implicit def ffloat = new F[Float,Double] { val value = 2.0D }
def f[T,R](t:T)(implicit f: F[T,R]): R = f.value
f(100) // res4: Long = 1L
f(100.0f) // res5: Double = 2.0
I see that path-dependent type would bring benefits if one could use them in argument lists, but we know we can't do
def g[T](t:T)(implicit f: F[T], r: Blah[f.R]) ...
thus, we are still forced to put an additional type parameter in the signature of g. By using the Aux technique, we are also required to spend additional time writing the companion object. From a usage standpoint, it would look to a naive user like me that there is no benefit in using path-dependent types at all.
There is only one case I can think of, that is, for a given type-level computation more than one type-level result is returned, and you may want to use only one of them.
I guess it all boils down to me overlooking something in my simple example.
There are two separate questions here:
Why does Shapeless use type members instead of type parameters in some cases in some type classes?
Why does Shapeless include Aux type aliases in the companion objects of these type classes?
I'll start with the second question because the answer is more straightforward: the Aux type aliases are entirely a syntactic convenience. You don't ever have to use them. For example, suppose we want to write a method that will only compile when called with two hlists that have the same length:
import shapeless._, ops.hlist.Length
def sameLength[A <: HList, B <: HList, N <: Nat](a: A, b: B)(implicit
al: Length.Aux[A, N],
bl: Length.Aux[B, N]
) = ()
The Length type class has one type parameter (for the HList type) and one type member (for the Nat). The Length.Aux syntax makes it relatively easy to refer to the Nat type member in the implicit parameter list, but it's just a convenience—the following is exactly equivalent:
def sameLength[A <: HList, B <: HList, N <: Nat](a: A, b: B)(implicit
al: Length[A] { type Out = N },
bl: Length[B] { type Out = N }
) = ()
The Aux version has a couple of advantages over writing out the type refinements in this way: it's less noisy, and it doesn't require us to remember the name of the type member. These are purely ergonomic issues, though—the Aux aliases make our code a little easier to read and write, but they don't change what we can or can't do with the code in any meaningful way.
The answer to the first question is a little more complex. In many cases, including my sameLength, there's no advantage to Out being a type member instead of a type parameter. Because Scala doesn't allow multiple implicit parameter sections, we need N to be a type parameter for our method if we want to verify that the two Length instances have the same Out type. At that point, the Out on Length might as well be a type parameter (at least from our perspective as the authors of sameLength).
In other cases, though, we can take advantage of the fact that Shapeless sometimes (I'll talk about specifically where in a moment) uses type members instead of type parameters. For example, suppose we want to write a method that will return a function that will convert a specified case class type into an HList:
def converter[A](implicit gen: Generic[A]): A => gen.Repr = a => gen.to(a)
Now we can use it like this:
case class Foo(i: Int, s: String)
val fooToHList = converter[Foo]
And we'll get a nice Foo => Int :: String :: HNil. If Generic's Repr were a type parameter instead of a type member, we'd have to write something like this instead:
// Doesn't compile
def converter[A, R](implicit gen: Generic[A, R]): A => R = a => gen.to(a)
Scala doesn't support partial application of type parameters, so every time we call this (hypothetical) method we'd have to specify both type parameters since we want to specify A:
val fooToHList = converter[Foo, Int :: String :: HNil]
This makes it basically worthless, since the whole point was to let the generic machinery figure out the representation.
In general, whenever a type is uniquely determined by a type class's other parameters, Shapeless will make it a type member instead of a type parameter. Every case class has a single generic representation, so Generic has one type parameter (for the case class type) and one type member (for the representation type); every HList has a single length, so Length has one type parameter and one type member, etc.
Making uniquely-determined types type members instead of type parameters means that if we want to use them only as path-dependent types (as in the first converter above), we can, but if we want to use them as if they were type parameters, we can always either write out the type refinement (or the syntactically nicer Aux version). If Shapeless made these types type parameters from the beginning, it wouldn't be possible to go in the opposite direction.
As a side note, this relationship between a type class's type "parameters" (I'm using quotation marks since they may not be parameters in the literal Scala sense) is called a "functional dependency" in languages like Haskell, but you shouldn't feel like you need to understand anything about functional dependencies in Haskell to get what's going on in Shapeless.
I have next field:
var operations = Map.empty[Long, _ <: Operation]
I want to get second generic parameter upper bound extends Operation class. When I'm doing like above I have error unbound wildcard type.
How can I achieve this?
Map is defined as trait Map[A, +B], so Operation is covariant - an upper-bound type in this example.
Just say Map.empty[Long, Operation]
I'm going to address the actual error at hand. As unnecessary as it is, it will work if you define it like this:
var operations: Map[Long, _ <: Operation] = Map.empty // Or some Map that conforms
The difference is that in the above code, we're saying operations has type Map[Long, _ <: Operation] -- which is a map from Long to some type that we don't care about, so long as it is bounded above by Operation. But Map.empty is a method call which expects some actual types to be supplied as type-parameters (or will be inferred as Nothing) and not an existential.
Of course, this is all unnecessary because Map is covariant over its second type parameter. That means that if you have some Z that is a sub-type of Operation, then Map[Long, Z] is a sub-type of Map[Long, Operation].
Following watching Nick Partidge's presentation on deriving scalaz, I got to looking at this example, which is just awesome:
import scalaz._
import Scalaz._
def even(x: Int) : Validation[NonEmptyList[String], Int]
= if (x % 2 ==0) x.success else "not even: %d".format(x).wrapNel.fail
println( even(3) <|*|> even(5) ) //prints: Failure(NonEmptyList(not even: 3, not even: 5))
I was trying to understand what the <|*|> method was doing, here is the source code:
def <|*|>[B](b: M[B])(implicit t: Functor[M], a: Apply[M]): M[(A, B)]
= <**>(b, (_: A, _: B))
OK, that is fairly confusing (!) - but it references the <**> method, which is declared thus:
def <**>[B, C](b: M[B], z: (A, B) => C)(implicit t: Functor[M], a: Apply[M]): M[C]
= a(t.fmap(value, z.curried), b)
So I have a few questions:
How come the method appears to take a higher-kinded type of one type parameter (M[B]) but can get passed a Validation (which has two type paremeters)?
The syntax (_: A, _: B) defines the function (A, B) => Pair[A,B] which the 2nd method expects: what is happening to the Tuple2/Pair in the failure case? There's no tuple in sight!
Type Constructors as Type Parameters
M is a type parameter to one of Scalaz's main pimps, MA, that represents the Type Constructor (aka Higher Kinded Type) of the pimped value. This type constructor is used to look up the appropriate instances of Functor and Apply, which are implicit requirements to the method <**>.
trait MA[M[_], A] {
val value: M[A]
def <**>[B, C](b: M[B], z: (A, B) => C)(implicit t: Functor[M], a: Apply[M]): M[C] = ...
}
What is a Type Constructor?
From the Scala Language Reference:
We distinguish between first-order
types and type constructors, which
take type parameters and yield types.
A subset of first-order types called
value types represents sets of
(first-class) values. Value types are
either concrete or abstract. Every
concrete value type can be represented
as a class type, i.e. a type
designator (§3.2.3) that refers to a
class1 (§5.3), or as a compound type
(§3.2.7) representing an intersection
of types, possibly with a refinement
(§3.2.7) that further constrains the
types of itsmembers. Abstract value
types are introduced by type
parameters (§4.4) and abstract type
bindings (§4.3). Parentheses in types
are used for grouping. We assume that
objects and packages also implicitly
define a class (of the same name as
the object or package, but
inaccessible to user programs).
Non-value types capture properties of
identifiers that are not values
(§3.3). For example, a type
constructor (§3.3.3) does not directly
specify the type of values. However,
when a type constructor is applied to
the correct type arguments, it yields
a first-order type, which may be a
value type. Non-value types are
expressed indirectly in Scala. E.g., a
method type is described by writing
down a method signature, which in
itself is not a real type, although it
gives rise to a corresponding function
type (§3.3.1). Type constructors are
another example, as one can write type
Swap[m[_, _], a,b] = m[b, a], but
there is no syntax to write the
corresponding anonymous type function
directly.
List is a type constructor. You can apply the type Int to get a Value Type, List[Int], which can classify a value. Other type constructors take more than one parameter.
The trait scalaz.MA requires that it's first type parameter must be a type constructor that takes a single type to return a value type, with the syntax trait MA[M[_], A] {}. The type parameter definition describes the shape of the type constructor, which is referred to as its Kind. List is said to have the kind '* -> *.
Partial Application of Types
But how can MA wrap a values of type Validation[X, Y]? The type Validation has a kind (* *) -> *, and could only be passed as a type argument to a type parameter declared like M[_, _].
This implicit conversion in object Scalaz converts a value of type Validation[X, Y] to a MA:
object Scalaz {
implicit def ValidationMA[A, E](v: Validation[E, A]): MA[PartialApply1Of2[Validation, E]#Apply, A] = ma[PartialApply1Of2[Validation, E]#Apply, A](v)
}
Which in turn uses a trick with a type alias in PartialApply1Of2 to partially apply the type constructor Validation, fixing the type of the errors, but leaving the type of the success unapplied.
PartialApply1Of2[Validation, E]#Apply would be better written as [X] => Validation[E, X]. I recently proposed to add such a syntax to Scala, it might happen in 2.9.
Think of this as a type level equivalent of this:
def validation[A, B](a: A, b: B) = ...
def partialApply1Of2[A, B C](f: (A, B) => C, a: A): (B => C) = (b: B) => f(a, b)
This lets you combine Validation[String, Int] with a Validation[String, Boolean], because the both share the type constructor [A] Validation[String, A].
Applicative Functors
<**> demands the the type constructor M must have associated instances of Apply and Functor. This constitutes an Applicative Functor, which, like a Monad, is a way to structure a computation through some effect. In this case the effect is that the sub-computations can fail (and when they do, we accumulate the failures).
The container Validation[NonEmptyList[String], A] can wrap a pure value of type A in this 'effect'. The <**> operator takes two effectful values, and a pure function, and combines them with the Applicative Functor instance for that container.
Here's how it works for the Option applicative functor. The 'effect' here is the possibility of failure.
val os: Option[String] = Some("a")
val oi: Option[Int] = Some(2)
val result1 = (os <**> oi) { (s: String, i: Int) => s * i }
assert(result1 == Some("aa"))
val result2 = (os <**> (None: Option[Int])) { (s: String, i: Int) => s * i }
assert(result2 == None)
In both cases, there is a pure function of type (String, Int) => String, being applied to effectful arguments. Notice that the result is wrapped in the same effect (or container, if you like), as the arguments.
You can use the same pattern across a multitude of containers that have an associated Applicative Functor. All Monads are automatically Applicative Functors, but there are even more, like ZipStream.
Option and [A]Validation[X, A] are both Monads, so you could also used Bind (aka flatMap):
val result3 = oi flatMap { i => os map { s => s * i } }
val result4 = for {i <- oi; s <- os} yield s * i
Tupling with `<|**|>`
<|**|> is really similar to <**>, but it provides the pure function for you to simply build a Tuple2 from the results. (_: A, _ B) is a shorthand for (a: A, b: B) => Tuple2(a, b)
And beyond
Here's our bundled examples for Applicative and Validation. I used a slightly different syntax to use the Applicative Functor, (fa ⊛ fb ⊛ fc ⊛ fd) {(a, b, c, d) => .... }
UPDATE: But what happens in the Failure Case?
what is happening to the Tuple2/Pair in the failure case?
If any of the sub-computations fails, the provided function is never run. It only is run if all sub-computations (in this case, the two arguments passed to <**>) are successful. If so, it combines these into a Success. Where is this logic? This defines the Apply instance for [A] Validation[X, A]. We require that the type X must have a Semigroup avaiable, which is the strategy for combining the individual errors, each of type X, into an aggregated error of the same type. If you choose String as your error type, the Semigroup[String] concatenates the strings; if you choose NonEmptyList[String], the error(s) from each step are concatenated into a longer NonEmptyList of errors. This concatenation happens below when two Failures are combined, using the ⊹ operator (which expands with implicits to, for example, Scalaz.IdentityTo(e1).⊹(e2)(Semigroup.NonEmptyListSemigroup(Semigroup.StringSemigroup)).
implicit def ValidationApply[X: Semigroup]: Apply[PartialApply1Of2[Validation, X]#Apply] = new Apply[PartialApply1Of2[Validation, X]#Apply] {
def apply[A, B](f: Validation[X, A => B], a: Validation[X, A]) = (f, a) match {
case (Success(f), Success(a)) => success(f(a))
case (Success(_), Failure(e)) => failure(e)
case (Failure(e), Success(_)) => failure(e)
case (Failure(e1), Failure(e2)) => failure(e1 ⊹ e2)
}
}
Monad or Applicative, how shall I choose?
Still reading? (Yes. Ed)
I've shown that sub-computations based on Option or [A] Validation[E, A] can be combined with either Apply or with Bind. When would you choose one over the other?
When you use Apply, the structure of the computation is fixed. All sub-computations will be executed; the results of one can't influence the the others. Only the 'pure' function has an overview of what happened. Monadic computations, on the other hand, allow the first sub-computation to influence the later ones.
If we used a Monadic validation structure, the first failure would short-circuit the entire validation, as there would be no Success value to feed into the subsequent validation. However, we are happy for the sub-validations to be independent, so we can combine them through the Applicative, and collect all the failures we encounter. The weakness of Applicative Functors has become a strength!