I am still trying to get my head around Shapeless (and, to a lesser extent, Scala!) and I have been writing some simple code to generate random instance data for case classes - predominantly based on the guides here: http://enear.github.io/2016/09/27/bits-of-shapeless-2/ (the example covers a JSON Writer implementation)
I have created a Generator[A] trait and created implicit implementations for simple types, and as per the example in the above link, I have also created implicit implementations to handle HList, HNil, Coproduct and CNil:
implicit def hnilGenerator = new Generator[HNil] {
override def generate(a: HNil) = HNil
}
implicit def hconsGenerator[H, T <: HList](implicit headGen: Generator[H], tailGen: Generator[T]) =
new Generator[H :: T] {
override def generate(a: H :: T) = headGen.generate(a.head) :: tailGen.generate(a.tail)
}
implicit def cnilGenerator: Generator[CNil] =
new Generator[CNil] {
override def generate(a: CNil): CNil = throw new RuntimeException("Invalid candidate configuration")
}
implicit def cconsGenerator[H, T <: Coproduct] =
new Generator[H :+: T] {
override def generate(a: H :+: T) = throw new RuntimeException("Invalid candidate configuration")
}
I can now use this code to generate a random instance based on a case class or a sealed trait:
it("should work with a case class to hlist") {
case class Test(x: IntGene, y: DoubleGene, z: BooleanGene)
val c = Generic[Test].to(Test(IntGene(), DoubleGene(), BooleanGene()))
generate(c)
}
it("what happens with sealed traits") {
sealed trait Shape
case class Square(width: Int, height: Int) extends Shape
case class Circle(radius: Int) extends Shape
val c = Generic[Shape].to(Circle(1))
generate(c)
}
Both of the above work no problem, however, if I try to make this a generic (as in parameter types) I get compilation errors not being able to find the necessary implicts:
it("should handle generics") {
case class GenericTest[A: Generic](x: A) {
def convert() = {
val c = Generic[A].to(x)
generate(c)
}
}
}
So from my understanding, because I have used the Generic context bound A, the compiler knows that is going to be available, so c must be some possible return from the call to(x) - Am I missing something in the implementation to handle that return type from the Generic shapeless call? Or have I wildly misunderstood something?
I am hoping this is possible and I have just missed something - is it that the compiler doesn't know what will be passed in (Im assuming not), or is there another possible type that needs to be handled implicitly from that to(x) call?
EDIT
Compile error added below - I'm really just trying to understand: Is it that there is some return case from the to(x) shapeless call that I have not catered for, or is it because the compiler doesn't have any idea what will be passed in and there are some types not catered for (e.g. I haven't added a Date generator implicit - and a case class could potentially have any type included? I was hoping that was not the case, and as the compiler knows nothing is actually being passed to the class/method it knows there are no issues?)
GeneratorSpec.scala:44: could not find implicit value for parameter gen: io.github.robhinds.genotype.Generator[GenericTest.this.evidence$1.Repr]
generate(c)
And my generate method is just a simple helper method that gets given the implicit Generator :
def generate[A](a: A)(implicit gen: Generator[A]) = gen.generate(a)
Your problem comes from the fact that Generic only converts a case class to a HList and a sealed trait to a Coproduct (not recursively).
So if you have a generic Generic, you have no information on what HList or Coproduct you are given, so, you cannot use your product and coproduct rules to find the wanted implicit. In some explicit case, you might have the same problem, so I will give you this as an example:
Let's say you have a case class architecture
case class Bottom(value: String)
case class Top(bot: Bottom, count: Int)
The implicit Generic[Top] will have type MyHList = Bottom :: Int :: HNil as output type, so you'll be asking for an implicit Generator[MyHList]. But since you don't have an implicit Generator[Bottom] in scope, you won't be able to use hconsgenerator.
In the generic case, it's even worse. The compiler can only infer HList for the output type of Generic[A] (and that's assuming you forget about Coproduct), so you need an implicit Generator[HList], which you cannot provide.
The solution is to give an implicit for constructs that have a generic that can itself be generated:
implicit def generic2generate[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Generator[L]): Generator[T] = new Generator[T] {
def generate(c: T) = generic.from(lGen.generate(generic.to(c)))
}
EDIT
You can now follow the implicit resolution for our Top type:
We can have a Generator[Top] using last rule, if we have a Generic.Aux[Top, L] for some L, and a Generator[L].
The only Generic.Aux[Top, _] that exists implicitly is a Generic.Aux[Top, Bottom :: Int :: HNil], so we are reduced to finding a Generator[Top, Bottom :: Int :: HNil]
using the hcons rule three times, we are reduced to finding a Generator[Bottom], a Generator[Int] and a Generator[HNil].
Generator[Int] is given (I assume) and Generator[HNil] is the first rule, so we are reduced to finding a Generator[Bottom]
the only rule that can provide one, is once again the 3rd rule, so we must find a Generator[String :: HNil], since the only Generic available is a Generic.Aux[Bottom, String :: HNil].
using the hcons rule, we are down to finding a Generator[String], which can easily be provided.
This example shows different points:
first that it may take a long time when compiling to solve all these implicits (I only gave the main points of the proof, but the compiler has to try all possible branches)
second, that this resolution can only be done for a specific Generic, it cannot be inferred generically (although this might seem counter-intuitive); even if the human mind is able to tell that it will work for every Generic, the compiler cannot process it as such.
I think you are missing the Generator type bound.
it("should handle generics") {
case class GenericTest[A: Generic : Generator](x: A) {
def convert() = {
val c = Generic[A].to(x)
generate(c)
}
}
}
Related
I'm reading through and working my way through using type classes and I came across this way of defining type classes from the Shapeless guide:
So here goes the example:
object CsvEncoder {
// "Summoner" method
def apply[A](implicit enc: CsvEncoder[A]): CsvEncoder[A] =
enc
// "Constructor" method
def instance[A](func: A => List[String]): CsvEncoder[A] =
new CsvEncoder[A] {
def encode(value: A): List[String] =
func(value)
}
// Globally visible type class instances
}
What I do not understand is the need for the apply method? What is it doing in this context above?
Later on, the guide describes how I could create a type class instance:
implicit val booleanEncoder: CsvEncoder[Boolean] =
new CsvEncoder[Boolean] {
def encode(b: Boolean): List[String] =
if(b) List("yes") else List("no")
}
is actually shortened to:
implicit val booleanEncoder: CsvEncoder[Boolean] =
instance(b => if(b) List("yes") else List("no"))
So my question now is, how does this work? What I do not get is the need for the apply method?
EDIT: I came across a blog post that describes the steps in creating type classes as below:
Define typeclass contract trait Foo.
Define a companion object Foo with a helper method apply that acts like implicitly, and a way of defining Foo instances typically from a function.
Define FooOps class that defines unary or binary operators.
Define FooSyntax trait that implicitly provides FooOps from a Foo instance.
So what is the deal with point number 2, 3 and 4?
Most of those practices came from Haskell (basically an intention to mimic Haskell's type-classes is a reason for so much boilerplate), some of it is just for convenience. So,
2) As #Alexey Romanov mentioned, companion object with apply is just for convenience, so instead of implicitly[CsvEncoder[IceCream]] you could write just CsvEncoder[IceCream] (aka CsvEncoder.apply[IceCream]()), which will return you a required type-class instance.
3) FooOps provides convenience methods for DSLs. For instance you could have something like:
trait Semigroup[A] {
...
def append(a: A, b: A)
}
import implicits._ //you should import actual instances for `Semigroup[Int]` etc.
implicitly[Semigroup[Int]].append(2,2)
But sometimes it's inconvenient to call append(2,2) method, so it's a good practice to provide a symbolic alias:
trait Ops[A] {
def typeClassInstance: Semigroup[A]
def self: A
def |+|(y: A): A = typeClassInstance.append(self, y)
}
trait ToSemigroupOps {
implicit def toSemigroupOps[A](target: A)(implicit tc: Semigroup[A]): Ops[A] = new Ops[A] {
val self = target
val typeClassInstance = tc
}
}
object SemiSyntax extends ToSemigroupOps
4) You can use it as follows:
import SemiSyntax._
import implicits._ //you should also import actual instances for `Semigroup[Int]` etc.
2 |+| 2
If you wonder why so much boilerplate, and why scala's implicit class syntax doesn't provide this functionality from scratch - the answer is that implicit class actually provides a way to create DSL's - it's just less powerful - it's (subjectively) harder to provide operation aliases, deal with more complex dispatching (when required) etc.
However, there is a macro solution that generates boilerplate automatically for you: https://github.com/mpilquist/simulacrum.
One another important point about your CsvEncoder example is that instance is convenience method for creating type-class instances, but apply is a shortcut for "summoning" (requiring) those instances. So, first one is for library extender (a way to implement interface), another one is for a user (a way to call a particular operation provided for that interface).
Another thing to note is that in shapeless the apply method is not only for cuter syntax.
Take for instance this simplified version of shapeless' Generic and some case class Foo.
trait Generic[T] {
type Repr
}
object Generic {
def apply[T](implicit gen: Generic[T]): Generic[T] { type Repr = gen.Repr } = gen
/* lots of macros to generate implicit instances omitted */
}
case class Foo(a: Int, b: String)
Now when I call Generic[Foo] I will get an instance that is typed as Generic[Foo] { type Repr = Int :: String :: HNil }. But if I call implicitly[Generic[Foo]] all the compiler knows about the result is that it's a Generic[Foo]. In other words: the concrete type of Repr is lost and I can't do anything useful with it. The reason is that implicitly is implemented as follows:
def implicitly[T](implicit e: T): T = e
That method declaration basically says: if you ask for a T I promise to give you a T, if I find one, and nothing more. So that means you'd have to ask implicitly[Generic[Foo] { type Repr = Int :: String :: HNil }] and that defeats the purpose of having automatic derivation.
Quoting the guide immediately after the object CsvEncoder definition:
The apply method ... allows us to summon a type class instance given a target type:
CsvEncoder[IceCream]
// res9: CsvEncoder[IceCream] = ...
I would like to perfom two maps over shapeless HList, I have folllowing code that works:
object doubleToInt extends Poly1 {
implicit def caseDouble = at[Double](_.toInt)
}
object intToDouble extends Poly1 {
implicit def caseInt = at[Int](_.toDouble)
}
def convert[InputType<:HList, ResultType<:HList, IntermediateType<:HList](input: InputType)(
implicit mapper1: Mapper.Aux[doubleToInt.type , InputType, IntermediateType],
mapper2: Mapper.Aux[intToDouble.type, IntermediateType, ResultType]
): ResultType = {
input.map(doubleToInt).map(intToDouble)
}
convert(2.5 :: HNil)
as we can see the type of convert contains a type parameter IntermediateType that is completely irrelevant to the caller of this function. Is it posible to hide it somehow? It's a problem because I would like to call like this:
convert[SomeType, SomeType2, _](2.5 :: HNil)
but it doesn't compile because of unbound type parameter.
The normal solution when you want to infer only some type parameters of a method call is to split it into 2: the first fixes parameters you are interested in and returns a helper with apply method which is called to infer the rest. Applying this here, you'd get
def convert[ResultType <: HList] = new ConvertTo[ResultType]
class ConvertTo[ResultType <: HList] {
def apply[InputType <: HList, IntermediateType <: HList](input: InputType)(implicit mapper1: Mapper.Aux[doubleToInt.type, InputType, IntermediateType],
mapper2: Mapper.Aux[intToDouble.type, IntermediateType, ResultType]
) = input.map(doubleToInt).map(intToDouble)
}
Usage: convertTo[SomeType].apply(2.5 :: HNil). InputType is inferred from input and then the compiler finds the implicit mapper1 and infers IntermediateType.
Or at least, ideally it would: due to the way Scala type inference works, it can't infer IntermediateType from mapper1 and then use it in mapper2, so I am not sure it'll find mapper2 correctly. In future Scala versions this should be fixed by def apply(implicit mapper1: ...)(implicit mapper2: ...), but the pull request enabling this wasn't accepted in time for 2.12.0. If this turns out to be a problem, you'll have to split ConvertTo.apply in 2 again.
Let's assume the following:
class Wrapper1 {
case class Condition(test: String)
}
object Wrapper1 extends Wrapper1
class Wrapper2 {
case class Condition[A](test: String)
}
object Wrapper2 extends Wrapper2
class Test
type T = // whatever
def test(fn: T => Wrapper1.Condition): X
def test[R](fn: T => Wrapper2.Condition[R]): X
}
The problem is that because of type erasure those methods have the exact same type after erasure. It's easy to change the signature of the second with say:
def test[R](fn: T => Wrapper2.Condition[R])(implicit ev: R =:= R): X
But that confuses the compiler and using the test method in other places is impossible. For a number of design reasons I'm trying to keep the name of this method consistent. Is there any way to successfully do this?
Seems this is NOT a duplicate of Scala double definition (2 methods have the same type erasure)
My suggestion... Just have a single method
def test[Cond: TypeTag](fn: T => Cond): X = {
if(typeOf[T] <:< typeOf[Wrapper1.Condition]) ...
else if(typeOf[T] <:< typeOf[Wrapper2.Condition[_]) ...
else ...
}
Finally able to work it out. What I was after was a way to conditionally append to an HList. Basically multiple overloads of the same method had to co-exist because they would return a different kind of HList.
To only have one method, arguments needed to dictate what the final signature of the method looked like.
So let's assume the following, the test method had to return
def test[RR](arg: Type1[RR]): Wrapper[RR :: ExistingHList]
Or respectively:
def test(arg: Type2): Wrapper[ExistingList]
The same method would have to append to an Hlist depending on whether or not the argument satisfied certain conditions. The fix was as trivial as using shapeless.ops.hlist.Prepend, which will by default returning the existing HList in the event we try to append HNil to an hlist.
Making the argument of Type2 return an HNil or implicitly convert to something that would end up being HNil was the approach that worked.
Specifically:
class Wrapper[PS <: HList] {
def test[
RR,
HL <: HList,
Out <: HList
](
condition: T => Wrapper[HL]
)(implicit prepend: Prepend.Aux[HL, PS, Out]): Wrapper[Out[
}
This approach works if the condition function will return a Wrapper[HNil] for situations where the function doesn't need to modify the end return type. Functions who do modify are simply free to build up their own HList independently.
I think I need a HList that is constrained to have all of its elements being a subtype of a certain type. LUBConstraint seems to be what I want, and indeed it does constrain the construction of such a HList - but I can't see how to get the evidence out again, so that I can map (actually, traverse, because it needs to be monadic) over the HList and call a method (that exists in the LUB type) on each of the elements.
In addition, I want the type of the HList resulting from the traverse operation to be exactly the same type as the type of the input HList.
The use case is a kind of functional "listener list" - all of the elements of the HList are "listeners" which must be notified of "events", accept or reject them, and return new versions of themselves with updated "internal state". If that was all I needed, then I could just use an ordinary immutable Scala collection. But I also want direct typed access to individual elements without using asInstanceOf - hence the motivation for trying to use HList.
In general if you have some operation that you want to perform on all of the elements in an HList, you'll want to map a polymorphic function value over the HList. For example, suppose I have the following setup:
trait Listener[L <: Listener[L]] {
def handle(s: String): Option[L]
}
class FooListener extends Listener[FooListener] {
def handle(s: String) =
if (s.size == 3) Some(this) else None
}
class BarListener extends Listener[BarListener ]{
def handle(s: String) = Some(this)
}
import shapeless._
val listeners = new FooListener :: new BarListener :: HNil
Now I want to send a String to each of these listeners and gather the results. If I just wanted to send a fixed value, this would be easy:
object event123 extends Poly1 {
implicit def listener[L <: Listener[L]] = at[L](_.handle("123"))
}
val result = listeners.map(event123)
Which will be appropriately typed as an Option[FooListener] :: Option[BarListener] :: HNil. If I'm using shapeless-contrib, I can sequence this HList:
import scalaz._, Scalaz._, shapeless.contrib.scalaz._
val sequenced: Option[FooListener :: BarListener :: HNil] = sequence(result)
Or just use traverse:
traverse(listeners)(event123)
Unfortunately there are restrictions on how polymorphic function values can be defined that mean that partial application isn't convenient, so if we don't know the String we're sending at compile time, this is a lot more complicated:
object event extends Poly1 {
implicit def listener[L <: Listener[L]] = at[(L, String)] {
case (listener, string) => listener.handle(string)
}
}
traverse(listeners.zip(listeners.mapConst("123")))(event)
Where we've zipped the elements with the string and then mapped a polymorphic function that takes tuples over the result. There are other ways you could do this using more or less the same approach, but none of them are terribly clear.
A completely different approach is just to skip the polymorphic function values and define a new type class:
trait Notifiable[L <: HList] {
def tell(s: String)(l: L): Option[L]
}
object Notifiable {
implicit val hnilNotifiable: Notifiable[HNil] = new Notifiable[HNil] {
def tell(s: String)(l: HNil) = Some(HNil)
}
implicit def hconsNotifiable[H <: Listener[H], T <: HList](implicit
tn: Notifiable[T]
): Notifiable[H :: T] = new Notifiable[H :: T] {
def tell(s: String)(l: H :: T) = for {
h <- l.head.handle(s)
t <- tn.tell(s)(l.tail)
} yield h :: t
}
}
def tell[L <: HList: Notifiable](s: String)(l: L) =
implicitly[Notifiable[L]].tell(s)(l)
And then:
val sequenced: Option[FooListener :: BarListener :: HNil] =
tell("123")(listeners)
This is less generic (it only works on Option, not arbitrary applicatives), but it doesn't require an extra dependency for sequencing, and it's arguably a little less muddled than jumping through hoops to partially apply a polymorphic function value because of weird limitations of the compiler.
Still struggling with this.types (singleton types). Assume this scenario:
trait Sys[A <: Access] {
def in[T](v: String): AccessPrepare[A]
}
trait AccessPrepare[A <: Access] {
val a: A
def apply[T](fun: a.type => T): T
}
object Ref {
def single[A <: Access, V](v: V)(implicit a: A): Ref[A, V] = ???
}
trait Ref[A, V]
trait Access {
def set(r: Ref[this.type, Int]): Unit
}
The following fails:
def test(sys: Sys[Access]): Unit =
sys.in("v1") { implicit a =>
val r = Ref.single(44)
a.set(r)
}
because apparently r is of type Ref[Access, Int] and not Ref[a.type, Int]. My guess is the problem is that I would need a line like
def single[A <: Access, V](v: V)(implicit a: A): Ref[a.type, V] = ...
which isn't compiling as due to "illegal dependent method type"...
Any ideas how I can fix this. The demand is that I do not explicitly annotate calls with types. That is, I do not want to write Ref.single[a.type, Int](44)(a) for comprehensible reasons.
EDIT
As a clarification, with reference to answer "FYI, and to close the question" in thread Constraining an operation by matching a type parameter to an argument's path-dependent type -- what I would like to have in addition is the possibility to create objects (Refs) not by using a factory method in the Access but somewhere outside (e.g. with a new statement). Because the system cannot be limited by the definition of Access, I must be able to extend it with further objects.
You have several possibilities. With Scala 2.8/2.8.1, you can use the private option -Ydependent-method-types and then your solution with
def single[ A <: Access, V ]( v: V )( implicit a: A ) : Ref[ a.type, V ] = // ...
compiles fine.
If you want to avoid dependent method types because it's a private option, you can still make your first proposal compile by explicitly typing the call to Ref.single:
val r = Ref.single[a.type, Int](44)
You need to specify the type, though, as singleton types are never inferred. You problem is not the same as, but related to, the problem that singleton types are not inferred: see How to correctly type-annotate this HList?