Using Shapeless, I tried to get a Generic[F] via:
import shapeless._
class F(x: Int)
but it failed:
scala> Generic[F]
<console>:20: error: could not find implicit value for parameter gen: shapeless.Generic[F]
Generic[F]
^
Can shapeless produce a Generic[F]? If so, how?
What representation would you want for F? You could say it should be HNil, but the x isn't be visible outside of the body of the class, so Shapeless makes the decision not to provide any instance at all. This is "just" a design decision—the right one in my view, but it's easy to imagine Shapeless providing the HNil instance for your class.
It would also be reasonably easy to define your own instance for F, which is possible because Generic is just another type class:
import shapeless._
class F(x: Int)
object F {
implicit val genericF: Generic.Aux[F, HNil] = new Generic[F] {
type Repr = HNil
def from(r: HNil): F = new F(0)
def to(t: F): HNil = HNil
}
}
As another answer notes, you can get Shapeless to provide an instance for you by changing your class to a case class. That's not the only way, though—you could also change the constructor parameter to a val or just remove it:
scala> class F(val x: Int)
defined class F
scala> shapeless.Generic[F]
res0: shapeless.Generic[F]{type Repr = shapeless.::[Int,shapeless.HNil]} = anon$macro$3$1#73e5dfa9
scala> class F()
defined class F
scala> shapeless.Generic[F]
res1: shapeless.Generic[F]{type Repr = shapeless.HNil} = anon$macro$5$1#4e0e355c
You couldn't make the parameter a var or make the class abstract, though (unless it was sealed and had case class or object implementations). Or rather you could, but then you'd have to define your own instances again, and you'd be breaking the contract in the Generic documentation, which says that the characterized type should be an "immutable data type that has a canonical way of constructing and deconstructing instances".
As far as I know the exact details of what kinds of arrangements of class definitions will get Generic instances isn't documented anywhere (and the source isn't easy reading), but it's pretty easy to test the limits you're interested in by trying out specific cases in the REPL.
Related
I am still trying to get my head around Shapeless (and, to a lesser extent, Scala!) and I have been writing some simple code to generate random instance data for case classes - predominantly based on the guides here: http://enear.github.io/2016/09/27/bits-of-shapeless-2/ (the example covers a JSON Writer implementation)
I have created a Generator[A] trait and created implicit implementations for simple types, and as per the example in the above link, I have also created implicit implementations to handle HList, HNil, Coproduct and CNil:
implicit def hnilGenerator = new Generator[HNil] {
override def generate(a: HNil) = HNil
}
implicit def hconsGenerator[H, T <: HList](implicit headGen: Generator[H], tailGen: Generator[T]) =
new Generator[H :: T] {
override def generate(a: H :: T) = headGen.generate(a.head) :: tailGen.generate(a.tail)
}
implicit def cnilGenerator: Generator[CNil] =
new Generator[CNil] {
override def generate(a: CNil): CNil = throw new RuntimeException("Invalid candidate configuration")
}
implicit def cconsGenerator[H, T <: Coproduct] =
new Generator[H :+: T] {
override def generate(a: H :+: T) = throw new RuntimeException("Invalid candidate configuration")
}
I can now use this code to generate a random instance based on a case class or a sealed trait:
it("should work with a case class to hlist") {
case class Test(x: IntGene, y: DoubleGene, z: BooleanGene)
val c = Generic[Test].to(Test(IntGene(), DoubleGene(), BooleanGene()))
generate(c)
}
it("what happens with sealed traits") {
sealed trait Shape
case class Square(width: Int, height: Int) extends Shape
case class Circle(radius: Int) extends Shape
val c = Generic[Shape].to(Circle(1))
generate(c)
}
Both of the above work no problem, however, if I try to make this a generic (as in parameter types) I get compilation errors not being able to find the necessary implicts:
it("should handle generics") {
case class GenericTest[A: Generic](x: A) {
def convert() = {
val c = Generic[A].to(x)
generate(c)
}
}
}
So from my understanding, because I have used the Generic context bound A, the compiler knows that is going to be available, so c must be some possible return from the call to(x) - Am I missing something in the implementation to handle that return type from the Generic shapeless call? Or have I wildly misunderstood something?
I am hoping this is possible and I have just missed something - is it that the compiler doesn't know what will be passed in (Im assuming not), or is there another possible type that needs to be handled implicitly from that to(x) call?
EDIT
Compile error added below - I'm really just trying to understand: Is it that there is some return case from the to(x) shapeless call that I have not catered for, or is it because the compiler doesn't have any idea what will be passed in and there are some types not catered for (e.g. I haven't added a Date generator implicit - and a case class could potentially have any type included? I was hoping that was not the case, and as the compiler knows nothing is actually being passed to the class/method it knows there are no issues?)
GeneratorSpec.scala:44: could not find implicit value for parameter gen: io.github.robhinds.genotype.Generator[GenericTest.this.evidence$1.Repr]
generate(c)
And my generate method is just a simple helper method that gets given the implicit Generator :
def generate[A](a: A)(implicit gen: Generator[A]) = gen.generate(a)
Your problem comes from the fact that Generic only converts a case class to a HList and a sealed trait to a Coproduct (not recursively).
So if you have a generic Generic, you have no information on what HList or Coproduct you are given, so, you cannot use your product and coproduct rules to find the wanted implicit. In some explicit case, you might have the same problem, so I will give you this as an example:
Let's say you have a case class architecture
case class Bottom(value: String)
case class Top(bot: Bottom, count: Int)
The implicit Generic[Top] will have type MyHList = Bottom :: Int :: HNil as output type, so you'll be asking for an implicit Generator[MyHList]. But since you don't have an implicit Generator[Bottom] in scope, you won't be able to use hconsgenerator.
In the generic case, it's even worse. The compiler can only infer HList for the output type of Generic[A] (and that's assuming you forget about Coproduct), so you need an implicit Generator[HList], which you cannot provide.
The solution is to give an implicit for constructs that have a generic that can itself be generated:
implicit def generic2generate[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Generator[L]): Generator[T] = new Generator[T] {
def generate(c: T) = generic.from(lGen.generate(generic.to(c)))
}
EDIT
You can now follow the implicit resolution for our Top type:
We can have a Generator[Top] using last rule, if we have a Generic.Aux[Top, L] for some L, and a Generator[L].
The only Generic.Aux[Top, _] that exists implicitly is a Generic.Aux[Top, Bottom :: Int :: HNil], so we are reduced to finding a Generator[Top, Bottom :: Int :: HNil]
using the hcons rule three times, we are reduced to finding a Generator[Bottom], a Generator[Int] and a Generator[HNil].
Generator[Int] is given (I assume) and Generator[HNil] is the first rule, so we are reduced to finding a Generator[Bottom]
the only rule that can provide one, is once again the 3rd rule, so we must find a Generator[String :: HNil], since the only Generic available is a Generic.Aux[Bottom, String :: HNil].
using the hcons rule, we are down to finding a Generator[String], which can easily be provided.
This example shows different points:
first that it may take a long time when compiling to solve all these implicits (I only gave the main points of the proof, but the compiler has to try all possible branches)
second, that this resolution can only be done for a specific Generic, it cannot be inferred generically (although this might seem counter-intuitive); even if the human mind is able to tell that it will work for every Generic, the compiler cannot process it as such.
I think you are missing the Generator type bound.
it("should handle generics") {
case class GenericTest[A: Generic : Generator](x: A) {
def convert() = {
val c = Generic[A].to(x)
generate(c)
}
}
}
I'm reading through and working my way through using type classes and I came across this way of defining type classes from the Shapeless guide:
So here goes the example:
object CsvEncoder {
// "Summoner" method
def apply[A](implicit enc: CsvEncoder[A]): CsvEncoder[A] =
enc
// "Constructor" method
def instance[A](func: A => List[String]): CsvEncoder[A] =
new CsvEncoder[A] {
def encode(value: A): List[String] =
func(value)
}
// Globally visible type class instances
}
What I do not understand is the need for the apply method? What is it doing in this context above?
Later on, the guide describes how I could create a type class instance:
implicit val booleanEncoder: CsvEncoder[Boolean] =
new CsvEncoder[Boolean] {
def encode(b: Boolean): List[String] =
if(b) List("yes") else List("no")
}
is actually shortened to:
implicit val booleanEncoder: CsvEncoder[Boolean] =
instance(b => if(b) List("yes") else List("no"))
So my question now is, how does this work? What I do not get is the need for the apply method?
EDIT: I came across a blog post that describes the steps in creating type classes as below:
Define typeclass contract trait Foo.
Define a companion object Foo with a helper method apply that acts like implicitly, and a way of defining Foo instances typically from a function.
Define FooOps class that defines unary or binary operators.
Define FooSyntax trait that implicitly provides FooOps from a Foo instance.
So what is the deal with point number 2, 3 and 4?
Most of those practices came from Haskell (basically an intention to mimic Haskell's type-classes is a reason for so much boilerplate), some of it is just for convenience. So,
2) As #Alexey Romanov mentioned, companion object with apply is just for convenience, so instead of implicitly[CsvEncoder[IceCream]] you could write just CsvEncoder[IceCream] (aka CsvEncoder.apply[IceCream]()), which will return you a required type-class instance.
3) FooOps provides convenience methods for DSLs. For instance you could have something like:
trait Semigroup[A] {
...
def append(a: A, b: A)
}
import implicits._ //you should import actual instances for `Semigroup[Int]` etc.
implicitly[Semigroup[Int]].append(2,2)
But sometimes it's inconvenient to call append(2,2) method, so it's a good practice to provide a symbolic alias:
trait Ops[A] {
def typeClassInstance: Semigroup[A]
def self: A
def |+|(y: A): A = typeClassInstance.append(self, y)
}
trait ToSemigroupOps {
implicit def toSemigroupOps[A](target: A)(implicit tc: Semigroup[A]): Ops[A] = new Ops[A] {
val self = target
val typeClassInstance = tc
}
}
object SemiSyntax extends ToSemigroupOps
4) You can use it as follows:
import SemiSyntax._
import implicits._ //you should also import actual instances for `Semigroup[Int]` etc.
2 |+| 2
If you wonder why so much boilerplate, and why scala's implicit class syntax doesn't provide this functionality from scratch - the answer is that implicit class actually provides a way to create DSL's - it's just less powerful - it's (subjectively) harder to provide operation aliases, deal with more complex dispatching (when required) etc.
However, there is a macro solution that generates boilerplate automatically for you: https://github.com/mpilquist/simulacrum.
One another important point about your CsvEncoder example is that instance is convenience method for creating type-class instances, but apply is a shortcut for "summoning" (requiring) those instances. So, first one is for library extender (a way to implement interface), another one is for a user (a way to call a particular operation provided for that interface).
Another thing to note is that in shapeless the apply method is not only for cuter syntax.
Take for instance this simplified version of shapeless' Generic and some case class Foo.
trait Generic[T] {
type Repr
}
object Generic {
def apply[T](implicit gen: Generic[T]): Generic[T] { type Repr = gen.Repr } = gen
/* lots of macros to generate implicit instances omitted */
}
case class Foo(a: Int, b: String)
Now when I call Generic[Foo] I will get an instance that is typed as Generic[Foo] { type Repr = Int :: String :: HNil }. But if I call implicitly[Generic[Foo]] all the compiler knows about the result is that it's a Generic[Foo]. In other words: the concrete type of Repr is lost and I can't do anything useful with it. The reason is that implicitly is implemented as follows:
def implicitly[T](implicit e: T): T = e
That method declaration basically says: if you ask for a T I promise to give you a T, if I find one, and nothing more. So that means you'd have to ask implicitly[Generic[Foo] { type Repr = Int :: String :: HNil }] and that defeats the purpose of having automatic derivation.
Quoting the guide immediately after the object CsvEncoder definition:
The apply method ... allows us to summon a type class instance given a target type:
CsvEncoder[IceCream]
// res9: CsvEncoder[IceCream] = ...
This seems to be a classic question for developers used to Scala type-level programming, but I couldn't find (or I don't know how to search for) a solution or pattern for this. Suppose I have a class like this:
abstract class TypedTest[Args <: HList](implicit val optMapper: Mapped[Args, Option]) {
type OptArgs = optMapper.Out
def options: OptArgs // to be implemented by subclasses
}
I want users of this class to instantiate it with an HList type parameter (Args) and the class provides a method to retrieve an HList instance containing an instance of each specified type inside an Option (OptArgs). I'm using shapeless Mapped type class for this. Note that I don't have an instance of Args to provide at instantiation time.
This code doesn't work, as the compiler doesn't infer the concrete type of OptArgs and even an obviously correct implementation such as def options = HNil yields a compilation error. The same code using the Aux pattern:
abstract class TypedTest[Args <: HList, OptArgs <: HList](implicit val optMapper: Mapped.Aux[Args, Option, OptArgs]) {
def options: OptArgs
}
This forces me to specify both lists at instantiation time, which makes the external API needlessly verbose. Is there an workaround for this?
This is my understanding, but I'm not 100% sure and will be happy to stand corrected.
The type member TypedTest.OptArgs is not an abstract type but a type alias. It is the same type for all subclasses of TypedTest – an alias for Mapped[Args, Option].Out, which is an abstract type and cannot be unified with any type but itself. When a subclass is created, the type member OptArgs is not overridden.
It becomes more clear when using Mapped.Aux with an existential type for Out0, which is IIUC more or less equivalent to the above:
abstract class TypedTest[Args <: HList](
implicit val optMapper: Mapped.Aux[Args, Option, T] forSome { type T }) {
type OptArgs = optMapper.Out
def options: OptArgs // to be implemented by subclasses
}
val intTest = new TypedTest[Int :: HNil] {
def options = Some(1) :: HNil
}
Error:(18, 29) type mismatch;
found : shapeless.::[Some[Int],shapeless.HNil]
required: this.OptArgs
(which expands to) T
def options = Some(1) :: HNil
Unfortunately I'm not aware of any possible solution, except adding Out as a type parameter or defining OptArgs as an abstract type and specifying it explicitly in each subclass.
I'm trying to understand how Generic works (and TypeClass too). The github wiki is very sparse on examples and documentation. Is there a canonical blog post / documentation page describing Generic and TypeClass in detail?
In concrete, what is the difference between these two methods?:
def find1[T](implicit gen: Generic[T]): Generic[T] = gen
def find2[T](implicit gen: Generic[T]): Generic[T] { type Repr = gen.Repr } = gen
given
object Generic {
type Aux[T, Repr0] = Generic[T] { type Repr = Repr0 }
def apply[T](implicit gen: Generic[T]): Aux[T, gen.Repr] = gen
implicit def materialize[T, R]: Aux[T, R] = macro GenericMacros.materialize[T, R]
}
The issues involved in how Generic and TypeClass are implemented and what they do are different enough that they probably deserve separate questions, so I'll stick to Generic here.
Generic provides a mapping from case classes (and potentially similar types) to heterogeneous lists. Any case class has a unique hlist representation, but any given hlist corresponds to a very, very large number of potential case classes. For example, if we have the following case classes:
case class Foo(i: Int, s: String)
case class Bar(x: Int, y: String)
The hlist representation provided by Generic for both Foo and Bar is Int :: String :: HNil, which is also the representation for (Int, String) and any other case classes we could define with these two types in this order.
(As a side note, LabelledGeneric allows us to distinguish between Foo and Bar, since it includes the member names in the representation as type-level strings.)
We generally want to be able to specify the case class and let Shapeless figure out the (unique) generic representation, and making Repr a type member (instead of a type parameter) allows us to do this pretty cleanly. If the hlist representation type were a type parameter, then your find methods would have to have a Repr type parameter as well, which means that you wouldn't be able to specify only the T and have the Repr inferred.
Making Repr a type member makes sense only because the Repr is uniquely determined by the first type parameter. Imagine a type class like Iso[A, B] that witnesses that A and B are isomorphic. This type class is very similar to Generic, but A doesn't uniquely dermine B—we can't just ask "what is the type that's isomorphic to A?"—so it wouldn't be useful to make B a type member (although we could if we really wanted to—Iso[A] just wouldn't really mean anything).
The problem with type members is that they're easy to forget, and once they're gone, they're gone forever. The fact that the return type of your find1 isn't refined (i.e. doesn't include the type member) means that the Generic instance it returns is pretty much useless. For example, the static type of res0 here might as well be Any:
scala> import shapeless._
import shapeless._
scala> def find1[T](implicit gen: Generic[T]): Generic[T] = gen
find1: [T](implicit gen: shapeless.Generic[T])shapeless.Generic[T]
scala> case class Foo(i: Int, s: String)
defined class Foo
scala> find1[Foo].to(Foo(1, "ABC"))
res0: shapeless.Generic[Foo]#Repr = 1 :: ABC :: HNil
scala> res0.head
<console>:15: error: value head is not a member of shapeless.Generic[Foo]#Repr
res0.head
^
When Shapeless's Generic.materialize macro creates the Generic[Foo] instance we're asking for, it's statically typed as a Generic[Foo] { type Repr = Int :: String :: HNil }, so the gen argument that the compiler hands to find1 has all the static information we need. The problem is that we then explicitly up-cast that type to a plain old unrefined Generic[Foo], and from that point on the compiler doesn't know what the Repr is for that instance.
Scala's path-dependent types give us a way not to forget the refinement without adding another type parameter to our method. In your find2, the compiler statically knows the Repr for the incoming gen, so when you say that the return type is Generic[T] { type Repr = gen.Repr }, it will be able to keep track of that information:
scala> find2[Foo].to(Foo(1, "ABC"))
res2: shapeless.::[Int,shapeless.::[String,shapeless.HNil]] = 1 :: ABC :: HNil
scala> res2.head
res3: Int = 1
To sum up: Generic has a type parameter T that uniquely determines its type member Repr, Repr is a type member instead of a type parameter so that we don't have to include it in all of our type signatures, and path-dependent types make this possible, allowing us to keep track of Repr even though it's not in our type signatures.
I have two case classes
case class StringCaseClass(argument: String)
case class IntCaseClass(argument: Int)
I want to define a structural type which will match the companion object of both of these
type HasApply1 {
def apply[A, R](argument: A): R
}
This will compile fine, but when I try to use it like this
def method(caseClass: HasApply1) {
// whatever
}
method(StringCaseClass)
I will get a compiler error
found : StringCaseClass.type
required: WithApply1
(which expands to) AnyRef{def apply[A, R](string: A): R}
Is there any way of accomplishing this? If I redefine the structural type to have concrete types for A and R it will compile correctly, but then I lose the flexiblity
#aloiscochard's comment is almost there. What he forgot to mention is that case class companion objects already implement the appropriate FunctionN trait, so you can simply do this,
scala> case class StringCaseClass(argument: String)
defined class StringCaseClass
scala> case class IntCaseClass(argument: Int)
defined class IntCaseClass
scala> def method[A, R](caseClass: A => R, a: A) = caseClass(a)
method: [A, R](caseClass: A => R, a: A)R
scala> method(StringCaseClass, "foo")
res0: StringCaseClass = StringCaseClass(foo)
scala> method(IntCaseClass, 23)
res1: IntCaseClass = IntCaseClass(23)
In general you should avoid structural typing as it's very expensive. The call will be converted into a reflection call because of limitations in the JVM. When you start using scala 2.10 structural types will result in a warning at compile time (though you could disable that using a flag).
If you're looking into a more general way to add functionality to classes that don't share an inheritance hierarchy you could use Type Classes.
Here's a quick example:
trait CanCreateRFromA[A,R]{
def createNew(a:A): R
}
implicit object CanCreateBlahFromInt extends CanCreateRFromA[Int,Blah2]{
def createNew(i:Int):Blah2 = new Blah2(i)
}
implicit object CanCreateBlah1FromString extends CanCreateRFromA[String,Blah1]{
def createNew(s:String):Blah1 = new Blah1(s)
}
case class Blah1(something:String)
case class Blah2(something:Int)
def createRFromA[A,R](a:A)(implicit tc:CanCreateRFromA[A,R])= tc.createNew(a)
Then you can call:
createRFromA(1) // This gives a Blah2
createRFromA("1") // This gives a Blah1
Again I'm not sure what you're trying to accomplish, but it probably is possible to do what you want with a type class and it will be much faster.
You didn't pass an instance of StringCaseClass to your method. What you passed there is companion object of StringCaseClass (which is automatically generated for case classes).
Try if this works: method(StringCaseClass("dummy")).