Adding a `to[Col[_]]` method for a covariant collection - scala

I am implementing a data structure. While it doesn't directly mix in any of Scala's standard collection traits, I want to include the to[Col[_]] method which, given a builder factory, can generate standard Scala collections.
Now assume this, copied from GenTraversableOnce:
trait Foo[+A] {
def to[Col[_]](implicit cbf: CanBuildFrom[Nothing, A, Col[A]]): Col[A]
}
This fails with error: covariant type A occurs in invariant position.
So how can GenTraversableOnce achieve this? I can see in the source code, that they add an annotation.unchecked.uncheckedVariance...
That looks like a dirty trick. If the typer rejects this normally, how can this be safe and switched off with uncheckedVariance?

Variance checking is a very important part of type checking and skipping it may easily cause a runtime type error. Here I can demonstrate a type populated with an invalid runtime value by printing it. I've not been able to make it crash with an type cast exception yet though.
import collection.generic.CanBuildFrom
import collection.mutable.Builder
import scala.annotation.unchecked.uncheckedVariance
trait Foo[+A] {
def toCol[Col[_]](implicit cbf: CanBuildFrom[Nothing, A, Col[A #uncheckedVariance]]): Col[A #uncheckedVariance]
}
object NoStrings extends Foo[String] {
override def toCol[Col[_]](implicit cbf: CanBuildFrom[Nothing, String, Col[String]]): Col[String] = {
val res : Col[String] = cbf().result
println("Printing a Col[String]: ")
println(res)
res
}
}
case class ExactlyOne[T](t : T)
implicit def buildExactlyOne = new CanBuildFrom[Nothing, Any, ExactlyOne[Any]] {
def apply() = new Builder[Any, ExactlyOne[Any]] {
def result = ExactlyOne({})
def clear = {}
def +=(x : Any) = this
}
def apply(n : Nothing) = n
}
val noStrings : Foo[Any] = NoStrings
noStrings.toCol[ExactlyOne]
Here println(res) with res : Col[String] prints ExactlyOne(()). However, ExactlyOne(()) does not have a type of Col[String], demonstrating a type error.
To solve the problem while respecting variance rules we can move the invariant code out of the trait and only keep covariant part, while using implicit conversion to convert from covariant trait to invariant helper class:
import collection.generic.CanBuildFrom
trait Foo[+A] {
def to[R](implicit cbf: CanBuildFrom[Nothing, A, R]): R
}
class EnrichedFoo[A](foo : Foo[A]) {
def toCol[Col[_]](implicit cbf: CanBuildFrom[Nothing, A, Col[A]]): Col[A] =
foo.to[Col[A]]
}
implicit def enrich[A](foo : Foo[A]) = new EnrichedFoo(foo)
case class Bar[A](elem: A) extends Foo[A] {
def to[R](implicit cbf: CanBuildFrom[Nothing, A, R]): R = {
val b = cbf()
b += elem
b.result()
}
}
val bar1 = Bar(3)
println(bar1.toCol[Vector])

It can because it is using the #uncheckedVariance annotation to circumpass the type system and ignore variance checking.
Simply import scala.annotation.unchecked.uncheckedVariance and annotate the type for which you want the variance checking disabled:
def to[Col[_]](implicit cbf: CanBuildFrom[Nothing, A, Col[A #uncheckedVariance]]): Col[A #uncheckedVariance]
See a more complete explanation in the related answer.

I read the link to the other question mentioned by #axel22. It still doesn't appear to be the actual reason, though (allowing GenTraversableOnce to function both for variant and invariant collections—it is covariant in A).
For example, the following works correctly without coercing the typer:
import collection.generic.CanBuildFrom
trait Foo[+A] {
def to[A1 >: A, Col[_]](implicit cbf: CanBuildFrom[Nothing, A1, Col[A1]]): Col[A1]
}
case class Bar[A](elem: A) extends Foo[A] {
def to[A1 >: A, Col[_]](implicit cbf: CanBuildFrom[Nothing, A1, Col[A1]]): Col[A1]= {
val b = cbf()
b += elem
b.result()
}
}
This would in my opinion be the correct signature. But then of course, it gets ugly:
val b = Bar(33)
b.to[Int, Vector]
So, my interpretation of the use of #uncheckedVariance is merely to avoid having to repeat the element type (as upper bound) in the to signature.
That still doesn't answer, though, if we can imagine a case which results in a runtime error from neglecting the variance?

Related

Inference of underscore types

[ I was unable to explain the problem with less verbosity. The core of the issue is that the compiler infers an underscore (_) type. In particular, [_ >: SomeType <: SomeOtherType]. I am clueless about when, how and why that is possible ]
As a scala exercise, I am trying to encode a vector of elements with a given size.
As a necessary step, I started by copying an existing encoding of natural numbers:
sealed trait Natural extends Product with Serializable {
def +(that: Natural): Natural
}
case object Zero extends Natural {
override def +(that: Natural): Natural = that
}
final case class Suc[N <: Natural](n: N) extends Natural {
override def +(that: Natural): Natural = Suc(n + that)
}
I believe the following diagram is a faithful portrayal of the type relation:
I then attempted to model a vector parameterized by a type on the elements and another type on size. To explain the problem though, I assumed a vector of ints and parameterized only the size of the vector :
import shapeless.=:!=
sealed trait Vector[+Size <: Natural] extends Product with Serializable {
def head: Int
def tail: Vector[Natural]
def plus[OtherSize >: Size <: Natural]
(that: Vector[OtherSize])
(implicit ev: OtherSize =:!= Natural): Vector[OtherSize]
}
case object VectorNil extends Vector[Zero.type] {
override def head: Nothing = throw new Exception("Boom!")
override def tail: Vector[Zero.type] = throw new Exception("Boom")
override def plus[OtherSize >: Zero.type <: Natural]
(that: Vector[OtherSize])
(implicit ev: =:!=[OtherSize, Natural]): Vector[OtherSize] = this
}
final case class VectorCons[N <: Natural](
head: Int,
tail: Vector[N]
) extends Vector[Suc[N]] {
override def plus[OtherSize >: Suc[N] <: Natural]
(that: Vector[OtherSize])
(implicit ev: =:!=[OtherSize, Natural]): Vector[OtherSize] = that
}
Notice that VectorCons[N] is actually a Vector of size Suc[N]. (extends Vector[Suc[N]]).
Method plus should add the elements of two vectors of the same size. I wanted to raise to the type level the verification that you can only sum vectors of the same size.
Notice that the conjunction of the type bounds OtherSize >: Size <: Natural with the implicit evidence should achieve that (see similar example at the bottom), but:
val foo = VectorCons(1, VectorCons(2, VectorNil))
//type -> VectorCons[Suc[Zero.type]]
// note that foo is (can be viewed) as Vector[Suc[Suc[Zero.type]], i.e
// a vector of 2 elements
val bar = VectorCons(3, VectorNil)
//type -> VectorCons[Zero.type]
val baz = foo.plus(bar)
//type -> Vector[Suc[_ >: Suc[Zero.type] with Zero.type <: Natural]] !! How is this possible ?? !!
to my frustration, baz compiles just fine!! The shapeless type constrain doesn't work; well, because OtherSize really is different from Natural; particularly, it is Suc[_ >: Suc[Zero.type] with Zero.type <: Natural].
So, I am very much confused with the type of baz! This is what allows the constraint to be bypassed.
What did the compiler infer for the type of baz/bar?
Is that an existential type ?
Is the compiler allowed to infer such things?
Is that not undesirable behavior ?
At this point, I am not concerned whether or not this is the correct encoding for a vector. Just wanted to understand how can the compiler infer that type for baz ?
p.s
1 - I am aware the return that on the implementation of method plus on VectorCons does not achieve what the plus semantics implies, but that is not important for the question.
###### Extra #######
I suspect this weird (at least for me) behavior has something to to with the natural numbers. The following code works fine!! :
[Disregard the absurd semantics of the code]
sealed trait Animal extends Product with Serializable
case class Dog() extends Animal
case class Cow() extends Animal
case object NoBox extends Animal //Not important
and,
trait Vector[+A <: Animal] {
def head: A
def plus[That >: A <: Animal]
(that: Vector[That])
(implicit ev: =:!=[That, Animal]): Vector[That]
}
case object Nil extends Vector[NoBox.type] {
def head: Nothing = throw new NoSuchElementException("Boom!")
override def plus[That >: NoBox.type <: Animal]
(that: Vector[That])(implicit ev: =:!=[That, Animal]): Vector[That] = this
}
case class Cons[A <: Animal](head: A) extends Vector[A] {
override def plus[That >: A <: Animal]
(that: Vector[That])(implicit ev: =:!=[That, Animal]): Vector[That] = that
}
whereby:
val foo = Cons(Dog())
val bar = Cons(Cow())
// Compiles
val baz = foo.plus(foo)
val baz2 = bar.plus(bar)
// Does not compile (what I would expect)
val baz3 = bar.plus(foo)
val baz4 = foo.plus(bar)
Thank you for your input,

How to construct an empty Iterable subclass, generically, and in Scala.js

Normally breakout would aid in conversion from one collection to another, but it doesn't seem to be able to infer the necessary colleciton constuctor for C:
import scala.collection.breakOut
object Utils {
implicit class IterableExtra[T, C[X] <: Iterable[X]](val list: C[T]) extends AnyVal {
def empty: C[T] = Iterable.empty[T].map(x => x)(breakOut)
}
}
Ideally this would work with minimal reflection, so that it might work in scala.js
Update I was also trying to use this in a different way, and I forgot to have the implicit at the outermost level:
def testIterableEmpty[B, I[X] <: Iterable[X]](implicit cbf: CanBuildFrom[I[B], B, I[B]]): I[B] = {
def emptyIter: I[B] = cbf().result()
emptyIter
}
scala> val x: List[Int] = testIterableEmpty[Int, List]
x: List[Int] = List()
breakOut is defined like so:
def breakOut[From, T, To](implicit b: CanBuildFrom[Nothing, T, To]): CanBuildFrom[From, T, To]
So it cannot be used to avoid passing a CanBuildFrom into your empty method - it requires one itself. Luckily, it is easy to write - you want to create a C[T] out of C[T], and the element type is T, so:
def empty(implicit cbf: CanBuildFrom[C[T], T, C[T]]): C[T] =
Iterable.empty[T].map(x => x)(breakOut)
Tho since you have a CanBuildFrom instance anyway, the implementation using it directly is straightforward too:
def empty(implicit cbf: CanBuildFrom[C[T], T, C[T]]): C[T] =
cbf().result()

Higher order type classes in scala

I am trying to understand typeclasses and so far i got to Monoids, which are simple enough:
object AbstractAlgebra {
case class Pair[A,B](a: A, b: B)
trait Monoid[T] {
def times(t1: T, t2: T): T
def unit: T
}
object Monoid {
implicit object MonoidIntPlus extends Monoid[Int] {
def times(i1: Int, i2: Int) = i1 + i2
def unit = 0
}
implicit object StringMonoid extends Monoid[String] {
def times(s1: String, s2: String) = s1 + s2
def unit = ""
}
implicit object PairOfMonoids extends Monoid[Pair[Monoid, Monoid]] = ???
}
}
I suppose my problem is the type Monoid[Pair[Monoid, Monoid]], cause I'm not really dealing with two monoid instances, only two classes that are implicit monoids, but I am not sure how to express that.
Any help or references would be appreciated
Monoid is not a type in itself. It's a type constructor, so Pair[Monoid, Monoid] makes no sense.
What you really want is actually the following: assuming that you have a Monoid type class instance for two given types A and B, then make an instance also for Pair[A, B].
This can be written as follows (the implementation is the most natural one you can derive):
implicit def monoidPair[A, B](implicit A: Monoid[A], B: Monoid[B]): Monoid[Pair[A, B]] = new Monoid[Pair[A, B]] {
def times(p1: Pair[A, B], p2: Pair[A, B]) =
Pair(A.times(p1.a, p2.a), B.times(p1.b, p2.b))
def unit = Pair(A.unit, B.unit)
}
This will do exactly what I explained before: If implicit instances for types Monoid[A] and Monoid[B] are found, then it puts a new implicit instance of type Monoid[Pair[A, B]] in scope.
Note. Your case class Pair[A, B] is already defined in Predef (although it's been deprecated since 2.11.0) as Tuple2[A, B] or (A, B).
Other note. If you don't like defining implicit instances as def or val, you can do the same with an implicit class:
implicit class MonoidPair[A, B](implicit A: Monoid[A], B: Monoid[B]) extends Monoid[Pair[A, B]] {
... //same body as before
}

Fold over HList with unknown Types

I have kind of a complex type hierarchy, but to break it down there are two base traits: Convertable and Conversion[A <: Convertable, B <: Convertable, e.g. there is a Conversion which can convert a Mealy-automaton to a Moore-automaton.
Every Conversion[A,B] has a convert(automaton: A) : B method.
Now I want to introduce the concept of smart Conversions, which are basically a List of normal Conversions, which will be performed one after another.
Therefore I have introduced an AutoConversion trait, extending a Conversion, which has a val path : HList parameter, to represent the chain of conversions, and should implement the convert method, so that AutoConversions just have to provide the list of actual Conversions to take.
I think you could implement this with a fold over the path, so here is my first try:
package de.uni_luebeck.isp.conversions
import shapeless._
import shapeless.ops.hlist.LeftFolder
trait AutoConversion[A <: Convertable, B <: Convertable] extends Conversion[A, B] {
val path: HList
object combiner extends Poly {
implicit def doSmth[C <: Convertable, D <: Convertable] =
use((conv : Conversion[C, D] , automaton : C) => conv.convert(automaton))
}
override def convert(startAutomaton: A): B = {
path.foldLeft(startAutomaton)(combiner)
}
}
This won't work, because no implicit Folder can be found, so I'm guessing I have to provide more Type information for the Compiler somewhere, but don't know where
You're right about needing more type information, and in general if you have a value with HList as a static type, it's likely you'll need to change your approach. There's essentially nothing you can do with an HList if all you know is that it's an HList (besides prepend values to it), and you'll usually only ever write HList as a type constraint.
In your case what you're describing is a kind of type-aligned sequence. Before you move forward with this approach, I'd suggest being really sure that you actually need to. One of the nice things about functions (and function-like types like your Conversion) is that they compose: you have an A => B and a B => C and you compose them into an A => C and can forget about B forever. You get a nice clean black box, which is generally exactly what you want.
In some cases, though, it can be useful to be able to compose function-like things in such a way that you can reflect on the pieces of the pipeline. I'm going to assume that this is one of those cases, but you should confirm that for yourself. If it's not, you're in luck, because what's coming is kind of messy.
I'll assume these types:
trait Convertable
trait Conversion[A <: Convertable, B <: Convertable] {
def convert(a: A): B
}
We can define a type class that witnesses that a specific HList is composed of one or more conversions whose types line up:
import shapeless._
trait TypeAligned[L <: HList] extends DepFn1[L] {
type I <: Convertable
type O <: Convertable
type Out = Conversion[I, O]
}
L contains all of the type information about the pipeline, and I and O are the types of its endpoints.
Next we need instances for this type class (note that this must be defined together with the trait above for the two to be companioned):
object TypeAligned {
type Aux[L <: HList, A <: Convertable, B <: Convertable] = TypeAligned[L] {
type I = A
type O = B
}
implicit def firstTypeAligned[
A <: Convertable,
B <: Convertable
]: TypeAligned.Aux[Conversion[A, B] :: HNil, A, B] =
new TypeAligned[Conversion[A, B] :: HNil] {
type I = A
type O = B
def apply(l: Conversion[A, B] :: HNil): Conversion[A, B] = l.head
}
implicit def composedTypeAligned[
A <: Convertable,
B <: Convertable,
C <: Convertable,
T <: HList
](implicit
tta: TypeAligned.Aux[T, B, C]
): TypeAligned.Aux[Conversion[A, B] :: T, A, C] =
new TypeAligned[Conversion[A, B] :: T] {
type I = A
type O = C
def apply(l: Conversion[A, B] :: T): Conversion[A, C] =
new Conversion[A, C] {
def convert(a: A): C = tta(l.tail).convert(l.head.convert(a))
}
}
}
And now you can write a version of your AutoConversion that keeps track of all of the type information about the pipeline:
class AutoConversion[L <: HList, A <: Convertable, B <: Convertable](
path: L
)(implicit ta: TypeAligned.Aux[L, A, B]) extends Conversion[A, B] {
def convert(a: A): B = ta(path).convert(a)
}
And you can use it like this:
case class AutoA(i: Int) extends Convertable
case class AutoB(s: String) extends Convertable
case class AutoC(c: Char) extends Convertable
val ab: Conversion[AutoA, AutoB] = new Conversion[AutoA, AutoB] {
def convert(a: AutoA): AutoB = AutoB(a.i.toString)
}
val bc: Conversion[AutoB, AutoC] = new Conversion[AutoB, AutoC] {
def convert(b: AutoB): AutoC = AutoC(b.s.lift(3).getOrElse('-'))
}
val conv = new AutoConversion(ab :: bc :: HNil)
And conv will have the expected static type (and implement Conversion[AutoA, AutoC]).

Adding a pairwise difference to generic collections - implicit resolution doesn't kick in

Ok, so I have this:
implicit final class RichIterableLike[A, Repr <: IterableLike[A, Repr]](val it: Repr)
extends AnyVal {
def pairDiff[To](implicit num: Numeric[A], cbf: CanBuildFrom[Repr, A, To]): To = {
val b = cbf(it)
val iter = it.iterator
if (iter.hasNext) {
var pred = iter.next()
while (iter.hasNext) {
import num.mkNumericOps
val succ = iter.next()
b += succ - pred
pred = succ
}
}
b.result()
}
}
This compiles, but doesn't kick in:
val stabs = IndexedSeq(1.0, 2.0, 3.0)
stabs.pairDiff
Gives: value pairDiff is not a member of IndexedSeq[Double]
Explicit conversion works:
new RichIterableLike[Double, IndexedSeq[Double]](stabs).pairDiff
... how to fix this?
EDIT
If I apply the approach of this answer, it works:
implicit final class RichIterableLike[A, CC[~] <: Iterable[~]](val it: CC[A])
extends AnyVal {
def pairDiff[To](implicit num: Numeric[A], cbf: CanBuildFrom[CC[A], A, To]): To = {
...
}
But the question remains, what is the crucial difference that makes the implicit lookup kick in in the latter case.
In order for the implicit lookup to work it needs a link between A and Repr (IterableLike demands that link). You pass it in as an argument, so that argument should be typed as Repr[A]. That means you need to modify your signature so it will look something like this:
RichIterableLike[A, Repr[X] <: IterableLike[X, Repr[X]]](val it: Repr[A])
With the above signature you say:
I have an object that accepts a type parameter, I will name that object Repr and when you pass it in I would like to capture the type parameter as well. I will name that type parameter A. As an extra condition I want the type of Repr to conform to the signature of IterableLike