I've tried to define a Monad (scalaz) for shapeless HList through point and bind implementation. The first problem is that HList trait is not a type constructor, but that can be solved with type lambdas, point is simple, but i couldn't find right implementation for bind, i guess i need some function of type Poly1 with some Aux/Mapper tricks, but that side of shapeless is still dark to me. HList has all functions to be a Monad, like simple List, so is it possible to implement one from Scalaz?
A monoid is a set with some operations that obey particular laws. What elements are you considering as possible HListM[A]? If you declare HListM[A] = HList, i.e. any HList, then you'll quickly find that you can't map with f: A => B, except by treating all maps as identity and you've reinvented the rather uninteresting monad Id (with a few extra but inert inhabitants).
We could make a monad with the type HListM[A] = A :: ... :: A :: HNil (though even actually expressing that type in Scala is a challenge - you'd need an auxiliary trait trait CopiesOf[N <: Nat, A] {type Out <: HList}, implicits to provide instances of this, and then an existential to actually write it (CopiesOf[N, A]#Out forSome {type N <: Nat})). Writing monad operations for this is possible, though you'd need to require shapeless auxiliary classes like Prepend at the point of operation, since there's no real way to express a "forall" type in Scala - you can declare instances of your type for _0 and Succ[N], but there's no way to prove to the compiler that there is an instance for any N <: Nat, you just have to require implicit ones whenever you need to use them.
But after a lot of work you'd end up with something isomorphic to List[A]; why not just use List[A] for that case?
Related
To me this appears as a very basic functionality, but I can't find it in current shapeless (2.3.3).
So I am looking for a type Induction[X,F[_],N <: Nat] with
Induction[X,F,Nat._0].Out =:= X
Induction[X,F,Nat._1].Out =:= F[X]
Induction[X,F,Nat._2].Out =:= F[F[X]]
...
Maybe it's also possible to chain a function along the type construction, e.g., to construct a Point instance?
No there isn't. As you observe this most likely needs a Point-like type class to be useful. I suggest adding something like this to Kittens which depends on both shapeless and Cats.
In cats there are 2 semigroup types classes: Semigroup and SemigroupK with the latter working on type constructors.
I fail to see the advantages of the latter over the former. If I look at the list instances they are providing Monoid (although there is a MonoidK), whereas NonEmptyList is providing a SemigroupK. Note that NonEmptyList is also providing a Semigroup via the following method:
implicit def catsDataSemigroupForNonEmptyList[A]: Semigroup[NonEmptyList[A]] =
SemigroupK[NonEmptyList].algebra[A]
Why the discrepancy?
Then it seems that most semigroup operations are only available on Semigroup and not SemigroupK (there's reduceK in Reducible but that's the only one I saw, and it delegates to reduce which works on Semigroup).
So, given a type T[_], what would you gain by having both a SemigroupK[T] and a Semigroup[T[A]] for some A?
Edit
There's now an issue to remove MonoidK and SemigroupK: https://github.com/typelevel/cats/issues/1932
One thing you can do with SemigroupK which you can't do with Semigroup is to compose instances for Nested:
implicit def catsDataSemigroupKForNested[F[_]: SemigroupK, G[_]]: SemigroupK[Nested[F, G, ?]]
If you try to write an equivalent for Semigroup, I think the closest you would get is
implicit def catsDataSemigroupForNested[F[_], G[_], A](implicit sg: Semigroup[F[G[A]]]): Semigroup[F[G[A]]] // or Semigroup[Nested[F, G, A]]
which is not very useful! From search, I can't see anything else which is implemented for SemigroupK and can't be done using Semigroup, but I might have missed something.
But the main point of SemigroupK is that once you have it, you can automatically get a Semigroup too, exactly like NonEmptyList does.
I want to write a function which simply binds two monads together, without fixing in advance the exact type of the monads (Lists, State monads, etc.). It seems to me that this kind of genericity is the reason why typeclasses are so powerful, and that I should be able to do it with Scalaz. Here is what I have in mind:
def f[F[_], A](m1: F[A], m2: F[A]): F[(A,A)] =
m1 >>= { a: A => m2.map{ b: A => (a,b) }}
How do specify that F[_] must implement the Monad typeclass so that I can use >>= in my function? Writing F[_] <: Monad doesn't seem to be the right approach since the types State, List, etc... which are monads don't extend the Monad trait.
It sounds like context bounds are exactly what you're looking for. f[F[_] : Monad... See What are Scala context and view bounds?
I discovered HList / KList, they are pretty cool. I have an actual use case, in which heterogenously typed and variable length containers with conserved type information would be very useful (for more info, see background below). However, I haven't understood the usage of a H/KList as method parameter, where I'm forced to fully type-annotate the parameter or loose type information.
Can H/KLists even be used as a parameter, if the full type is, of course, not known?
How to refer to a H/KList without loosing the type information?
Could "type lists" be used to refer to a tuple of heterogenous & variable length type parameters? Here it says:
... the types of the elements can be tracked separate from the actual element values. To do this we create an purely abstract type (it has no instances) which models a list of types, let's call it TList.
I played around with it, but haven't not yet understood how to use it for type annotation of an HList as parameter.
Basically, I want something like this:
implicit def hlistToTypedLink[TL](a: HList[TL]):TypedLink[TL] = new TypedLink[TL](a.map(a:X => new TypedHandle[X]))
where TL refers to the Type List and X to the type of the current element. So here a HList should be mapped to another Tuple-like container, TypedLink, parametrized by the type list TL. The elements are to be wrapped each in yet another parametrized container TypedHandle, typed with the current type X.
Is this possible?
I saw Shapeless' HList and its "unify" method but the problem remains the same: I don't know how to refer to it in the parameter list, besides variable length.
My second hope was to use KList. It applies in my case, because TypedHandle is a common container with same constructor. With KList it appears easier to type annotate, according to apocalisp:
val m = List(1, 2, 3, 4) :^: List("str1", "str2") :^: KNil
would be of type:
KCons[Int,java.lang.String :: HNil,List]
However, the problem remains the same: In the method definition, I cannot know whether it will be a
KCons[String, Int :: HNil, TH]
or a
KCons[Foo, Bar, Baz :: HNil, TH]
so I don't know how to type annotate the KList as method parameter either.
Thanks for any hints!
Background:
I'm writing scala convenience extensions for the excellent OO- & graph database hypergraphdb. Hypergraphdb's hyperedges, HGLink, are basically tuples of HGHandle's. HGHandle refer to atoms which per se are typed.
Hence HGLink per se would be heterogenously typed and of variable length. However, HGLink's implementations are till now untyped, and constructed by untyped implementations of HGHandle's. I guess java's typesystem is not expressive enough to reflect the (far superior) typesystem of hypergraphdb (which for example, also has higher kinded types).
Basically, I'm trying to bridge scala's with hypergraphdb's type systems, I'm learning a lot and till now this has been real fun. TypedHandle works great already, besides numerous other hacks.
thanks for any advice.
It's not completely clear to me what you're asking for, but your hlistToTypedLink looks like it could be handled using shapeless's HList and polymorphic function values,
scala> import shapeless._ ; import TypeOperators._
import shapeless._
import TypeOperators._
scala> class TypedHandle[T]
defined class TypedHandle
scala> class TypedLink[L <: HList](l : L)
defined class TypedLink
scala> object MkTypedHandle extends (Id ~> TypedHandle) {
| def apply[T](t : T) = new TypedHandle[T]
| }
defined module MkTypedHandle
scala> def hlistToTypedLink[L <: HList, M <: HList](a: L)
| (implicit mapper: MapperAux[MkTypedHandle.type, L, M]) =
| new TypedLink[M](a map MkTypedHandle)
hlistToTypedLink: [L <: HList, M <: HList](a: L)
(implicit mapper: MapperAux[MkTypedHandle.type,L,M])TypedLink[M]
scala> hlistToTypedLink(23 :: "foo" :: true :: HNil)
res0: TypedLink[TypedHandle[Int] :: TypedHandle[String] ::
TypedHandle[Boolean] :: HNil] = TypedLink#51fb5716
Essentially it looks like you want to map over your argument HList a, wrapping each element in a TypedHandle and then wrapping the resulting HList in a TypedLink. In all cases the wrappers are to be parametrized precisely on the types of their contents.
As seen above, this is possible using shapeless's HList map. There are two key ingredients here. First, the definition of the polymorphic function-like value MkTypedHandle which can be mapped across the HList a creating an HList of TypedLink-wrapped elements. And second, the implicit witness mapper which drives the map operation.
I better respond as you refer to my blog post. A TList is simply a HList without value storage, i.e. there is only a type representation of the list, no runtime representation. One advantage is that it facilitates different, possibly more efficient, types of value storage, for example arrays (HArray in Metascala). TLists can also be used to model union types (basically type sets), however Scala's type system is not powerful enough to do this solely on the type level (Haskell's type system is I think).
According to this question, Scala's type system is Turing complete. What resources are available that enable a newcomer to take advantage of the power of type-level programming?
Here are the resources I've found so far:
Daniel Spiewak's High Wizardry in the Land of Scala
Apocalisp's Type-Level Programming in Scala
Jesper's HList
These resources are great, but I feel like I'm missing the basics, and so do not have a solid foundation on which to build. For instance, where is there an introduction to type definitions? What operations can I perform on types?
Are there any good introductory resources?
Overview
Type-level programming has many similarities with traditional, value-level programming. However, unlike value-level programming, where the computation occurs at runtime, in type-level programming, the computation occurs at compile time. I will try to draw parallels between programming at the value-level and programming at the type-level.
Paradigms
There are two main paradigms in type-level programming: "object-oriented" and "functional". Most examples linked to from here follow the object-oriented paradigm.
A good, fairly simple example of type-level programming in the object-oriented paradigm can be found in apocalisp's implementation of the lambda calculus, replicated here:
// Abstract trait
trait Lambda {
type subst[U <: Lambda] <: Lambda
type apply[U <: Lambda] <: Lambda
type eval <: Lambda
}
// Implementations
trait App[S <: Lambda, T <: Lambda] extends Lambda {
type subst[U <: Lambda] = App[S#subst[U], T#subst[U]]
type apply[U] = Nothing
type eval = S#eval#apply[T]
}
trait Lam[T <: Lambda] extends Lambda {
type subst[U <: Lambda] = Lam[T]
type apply[U <: Lambda] = T#subst[U]#eval
type eval = Lam[T]
}
trait X extends Lambda {
type subst[U <: Lambda] = U
type apply[U] = Lambda
type eval = X
}
As can be seen in the example, the object-oriented paradigm for type-level programming proceeds as follows:
First: define an abstract trait with various abstract type fields (see below for what an abstract field is). This is a template for guaranteeing that certain types fields exist in all implementations without forcing an implementation. In the lambda calculus example, this corresponds to trait Lambda that guarantees that the following types exist: subst, apply, and eval.
Next: define subtraits that extend the abstract trait and implement the various abstract type fields
Often, these subtraits will be parameterized with arguments. In the lambda calculus example, the subtypes are trait App extends Lambda which is parameterized with two types (S and T, both must be subtypes of Lambda), trait Lam extends Lambda parameterized with one type (T), and trait X extends Lambda (which is not parameterized).
the type fields are often implemented by referring to the type parameters of the subtrait and sometimes referencing their type fields via the hash operator: # (which is very similar to the dot operator: . for values). In trait App of the lambda calculus example, the type eval is implemented as follows: type eval = S#eval#apply[T]. This is essentially calling the eval type of the trait's parameter S, and calling apply with parameter T on the result. Note, S is guaranteed to have an eval type because the parameter specifies it to be a subtype of Lambda. Similarly, the result of eval must have an apply type, since it is specified to be a subtype of Lambda, as specified in the abstract trait Lambda.
The Functional paradigm consists of defining lots of parameterized type constructors that are not grouped together in traits.
Comparison between value-level programming and type-level programming
abstract class
value-level: abstract class C { val x }
type-level: trait C { type X }
path dependent types
C.x (referencing field value/function x in object C)
C#x (referencing field type x in trait C)
function signature (no implementation)
value-level: def f(x:X) : Y
type-level: type f[x <: X] <: Y (this is called a "type constructor" and usually occurs in the abstract trait)
function implementation
value-level: def f(x:X) : Y = x
type-level: type f[x <: X] = x
conditionals
see here
checking equality
value-level: a:A == b:B
type-level: implicitly[A =:= B]
value-level: Happens in the JVM via a unit test at runtime (i.e. no runtime errors):
in essense is an assert: assert(a == b)
type-level: Happens in the compiler via a typecheck (i.e. no compiler errors):
in essence is a type comparison: e.g. implicitly[A =:= B]
A <:< B, compiles only if A is a subtype of B
A =:= B, compiles only if A is a subtype of B and B is a subtype of A
A <%< B, ("viewable as") compiles only if A is viewable as B (i.e. there is an implicit conversion from A to a subtype of B)
an example
more comparison operators
Converting between types and values
In many of the examples, types defined via traits are often both abstract and sealed, and therefore can neither be instantiated directly nor via anonymous subclass. So it is common to use null as a placeholder value when doing a value-level computation using some type of interest:
e.g. val x:A = null, where A is the type you care about
Due to type-erasure, parameterized types all look the same. Furthermore, (as mentioned above) the values you're working with tend to all be null, and so conditioning on the object type (e.g. via a match statement) is ineffective.
The trick is to use implicit functions and values. The base case is usually an implicit value and the recursive case is usually an implicit function. Indeed, type-level programming makes heavy use of implicits.
Consider this example (taken from metascala and apocalisp):
sealed trait Nat
sealed trait _0 extends Nat
sealed trait Succ[N <: Nat] extends Nat
Here you have a peano encoding of the natural numbers. That is, you have a type for each non-negative integer: a special type for 0, namely _0; and each integer greater than zero has a type of the form Succ[A], where A is the type representing a smaller integer. For instance, the type representing 2 would be: Succ[Succ[_0]] (successor applied twice to the type representing zero).
We can alias various natural numbers for more convenient reference. Example:
type _3 = Succ[Succ[Succ[_0]]]
(This is a lot like defining a val to be the result of a function.)
Now, suppose we want to define a value-level function def toInt[T <: Nat](v : T) which takes in an argument value, v, that conforms to Nat and returns an integer representing the natural number encoded in v's type. For example, if we have the value val x:_3 = null (null of type Succ[Succ[Succ[_0]]]), we would want toInt(x) to return 3.
To implement toInt, we're going to make use of the following class:
class TypeToValue[T, VT](value : VT) { def getValue() = value }
As we will see below, there will be an object constructed from class TypeToValue for each Nat from _0 up to (e.g.) _3, and each will store the value representation of the corresponding type (i.e. TypeToValue[_0, Int] will store the value 0, TypeToValue[Succ[_0], Int] will store the value 1, etc.). Note, TypeToValue is parameterized by two types: T and VT. T corresponds to the type we're trying to assign values to (in our example, Nat) and VT corresponds to the type of value we're assigning to it (in our example, Int).
Now we make the following two implicit definitions:
implicit val _0ToInt = new TypeToValue[_0, Int](0)
implicit def succToInt[P <: Nat](implicit v : TypeToValue[P, Int]) =
new TypeToValue[Succ[P], Int](1 + v.getValue())
And we implement toInt as follows:
def toInt[T <: Nat](v : T)(implicit ttv : TypeToValue[T, Int]) : Int = ttv.getValue()
To understand how toInt works, let's consider what it does on a couple of inputs:
val z:_0 = null
val y:Succ[_0] = null
When we call toInt(z), the compiler looks for an implicit argument ttv of type TypeToValue[_0, Int] (since z is of type _0). It finds the object _0ToInt, it calls the getValue method of this object and gets back 0. The important point to note is that we did not specify to the program which object to use, the compiler found it implicitly.
Now let's consider toInt(y). This time, the compiler looks for an implicit argument ttv of type TypeToValue[Succ[_0], Int] (since y is of type Succ[_0]). It finds the function succToInt, which can return an object of the appropriate type (TypeToValue[Succ[_0], Int]) and evaluates it. This function itself takes an implicit argument (v) of type TypeToValue[_0, Int] (that is, a TypeToValue where the first type parameter is has one fewer Succ[_]). The compiler supplies _0ToInt (as was done in the evaluation of toInt(z) above), and succToInt constructs a new TypeToValue object with value 1. Again, it is important to note that the compiler is providing all of these values implicitly, since we do not have access to them explicitly.
Checking your work
There are several ways to verify that your type-level computations are doing what you expect. Here are a few approaches. Make two types A and B, that you want to verify are equal. Then check that the following compile:
Equal[A, B]
with: trait Equal[T1 >: T2 <: T2, T2] (taken from apocolisp)
implicitly[A =:= B]
Alternatively, you can convert the type to a value (as shown above) and do a runtime check of the values. E.g. assert(toInt(a) == toInt(b)), where a is of type A and b is of type B.
Additional Resources
The complete set of available constructs can be found in the types section of the scala reference manual (pdf).
Adriaan Moors has several academic papers about type constructors and related topics with examples from scala:
Generics of a higher kind (pdf)
Type Constructor Polymorphism for Scala: Theory and Practice (pdf) (PhD thesis, which includes the previous paper by Moors)
Type Constructor Polymorphism Inference
Apocalisp is a blog with many examples of type-level programming in scala.
Type-Level Programming in Scala is a fantastic guided tour of some type-level programming which includes booleans, natural numbers (as above), binary numbers, heterogeneous lists, and more.
More Scala Typehackery is the lambda calculus implementation above.
ScalaZ is a very active project that is providing functionality that extends the Scala API using various type-level programming features. It is a very interesting project that has a big following.
MetaScala is a type-level library for Scala, including meta types for natural numbers, booleans, units, HList, etc. It is a project by Jesper Nordenberg (his blog).
The Michid (blog) has some awesome examples of type-level programming in Scala (from other answer):
Meta-Programming with Scala Part I: Addition
Meta-Programming with Scala Part II: Multiplication
Meta-Programming with Scala Part III: Partial function application
Meta-Programming with Scala: Conditional Compilation and Loop Unrolling
Scala type level encoding of the SKI calculus
Debasish Ghosh (blog) has some relevant posts as well:
Higher order abstractions in scala
Static typing gives you a head start
Scala implicits type classes, here I come
Refactoring into scala type-classes
Using generalized type constraints
How scalas type system words for you
Choosing between abstract type members
(I've been doing some research on this subject and here's what I've learned. I'm still new to it, so please point out any inaccuracies in this answer.)
In addition to the other links here, there are also my blog posts on type level meta programming in Scala:
Meta-Programming with Scala Part I: Addition
Meta-Programming with Scala Part II: Multiplication
Meta-Programming with Scala Part III: Partial function application
Meta-Programming with Scala: Conditional Compilation and Loop Unrolling
Scala type level encoding of the SKI calculus
As suggested on Twitter: Shapeless: An exploration of generic/polytypic programming in Scala by Miles Sabin.
Sing, a type-level metaprogramming library in Scala.
The Beginning of Type-level Programming in Scala
Scalaz has source code, a wiki and examples.