Scala Generics, which of these 4 generic version is specific to which particular problem - scala

These four versions compile but i'm curious about context in which we should prefer one option rather than another.
// 1
def f[N, S <: Seq[N]](s: S)
// 2
def f[N, S[N] <: Seq[N]](s: S[N])
They are pretty similar when use 1 rather than 2
2 impose that S have N as generic parameter as 1 but then what is the difference between these two ?
Then we have more general settings.
// 3
def f[N, S[X] <: Seq[X]](s: S[N])
// 3.a
def f[N, S[X] <: Seq[X]](s: S[N]): S[Int]
From what i humbly understood 3 authorize to extract the generic container type to reuse it later and get something like 3.a.
But what is the meaning of the undeclared X generic parameter, i suppose it's a way to declare something special but i don't get it.
// 4
def f[N, S[X] <: Seq[_]](s: S[N])
I don't know what to say about 4 except than from what i know Seq[_] stands for Seq[Any]
Finally i'm simply curious to have more information about these tools and their specificity to get things done more properly.

// 2
def f[N, S[N] <: Seq[N]](s: S[N])
The idea here is that the first N parameter and N mentioned in S[N] <: Seq[N] are completely independent parameters. They are just sharing the same name.
N mentioned in S[N] is visible only in a scope of its bound <: Seq[N].
N used in parameter definition (s: S[N]) comes from first N as this is the only N parameter visible for parameter type definition. So instead of N in S[N] <: Seq[N] you can use any letter and this will not affect your parameter type at any way.
// 4
def f[N, S[X] <: Seq[_]](s: S[N])
Here you just ignored X parameter.
Edit: as #alexey-romanov mentioned in a comment. There is difference between S[X] <: Seq[X] and S[X] <: Seq[_]
Here is an example showing the difference:
def f1[N, S[X] <: Seq[X]](s: S[N]) = ""
def f2[N, S[X] <: Seq[_]](s: S[N]) = ""
type Foo[A] = Seq[Int]
val foo: Foo[String] = Seq(2,3)
//f1(foo) -- compilation error
f2(foo)
The problem here is as type constrictor is a kind of "function on types", we can define such "function" accepting one type as a parameter but returning type parametrized by another parameter not related to parameter used in type constructor. (See type Foo)
Passing foo val to f2 is fine because X is infered to String and Foo[String] is a "subtype"(actually they are equal) of Seq[Int], but when we pass foo to f1 the X is still a String but Foo[String] is not "subtype" of Seq[String] (because Foo[String]==Seq[Int] not subtype of Seq[String])
// 1
def f[N, S <: Seq[N]](s: S)
And here you said that N used in Seq[N] is the same as first parameter N. So this is the same N

Related

How to ask Scala if evidence exists for all instantiations of type parameter?

Given the following type-level addition function on Peano numbers
sealed trait Nat
class O extends Nat
class S[N <: Nat] extends Nat
type plus[a <: Nat, b <: Nat] = a match
case O => b
case S[n] => S[n plus b]
say we want to prove theorem like
for all natural numbers n, n + 0 = n
which perhaps can be specified like so
type plus_n_0 = [n <: Nat] =>> (n plus O) =:= n
then when it comes to providing evidence for theorem we can easily ask Scala compiler for evidence in particular cases
summon[plus_n_O[S[S[O]]]] // ok, 2 + 0 = 2
but how can we ask Scala if it can generate evidence for all instantiations of [n <: Nat], thus providing proof of plus_n_0?
Here is one possible approach, which is an attempt at a literal interpretation of this paragraph:
When proving a statement E:N→U about all natural numbers, it suffices to prove it for 0 and for succ(n), assuming it holds for n, i.e., we construct ez:E(0) and es:∏(n:N)E(n)→E(succ(n)).
from the HoTT book (section 5.1).
Here is the plan of what was implemented in the code below:
Formulate what it means to have a proof for a statement that "Some property P holds for all natural numbers". Below, we will use
trait Forall[N, P[n <: N]]:
inline def apply[n <: N]: P[n]
where the signature of the apply-method essentially says "for all n <: N, we can generate evidence of P[n]".
Note that the method is declared to be inline. This is one possible way to ensure that the proof of ∀n.P(n) is constructive and executable at runtime (However, see edit history for alternative proposals with manually generated witness terms).
Postulate some sort of induction principle for natural numbers. Below, we will use the following formulation:
If
P(0) holds, and
whenever P(i) holds, then also P(i + 1) holds,
then
For all `n`, P(n) holds
I believe that it should be possible to derive such induction principles using some metaprogramming facilities.
Write proofs for the base case and the induction case of the induction principle.
???
Profit
The code then looks like this:
sealed trait Nat
class O extends Nat
class S[N <: Nat] extends Nat
type plus[a <: Nat, b <: Nat] <: Nat = a match
case O => b
case S[n] => S[n plus b]
trait Forall[N, P[n <: N]]:
inline def apply[n <: N]: P[n]
trait NatInductionPrinciple[P[n <: Nat]] extends Forall[Nat, P]:
def base: P[O]
def step: [i <: Nat] => (P[i] => P[S[i]])
inline def apply[n <: Nat]: P[n] =
(inline compiletime.erasedValue[n] match
case _: O => base
case _: S[pred] => step(apply[pred])
).asInstanceOf[P[n]]
given liftCoUpperbounded[U, A <: U, B <: U, S[_ <: U]](using ev: A =:= B):
(S[A] =:= S[B]) = ev.liftCo[[X] =>> Any].asInstanceOf[S[A] =:= S[B]]
type NatPlusZeroEqualsNat[n <: Nat] = (n plus O) =:= n
def trivialLemma[i <: Nat]: ((S[i] plus O) =:= S[i plus O]) =
summon[(S[i] plus O) =:= S[i plus O]]
object Proof extends NatInductionPrinciple[NatPlusZeroEqualsNat]:
val base = summon[(O plus O) =:= O]
val step: ([i <: Nat] => NatPlusZeroEqualsNat[i] => NatPlusZeroEqualsNat[S[i]]) =
[i <: Nat] => (p: NatPlusZeroEqualsNat[i]) =>
given previousStep: ((i plus O) =:= i) = p
given liftPreviousStep: (S[i plus O] =:= S[i]) =
liftCoUpperbounded[Nat, i plus O, i, S]
given definitionalEquality: ((S[i] plus O) =:= S[i plus O]) =
trivialLemma[i]
definitionalEquality.andThen(liftPreviousStep)
def demoNat(): Unit = {
println("Running demoNat...")
type two = S[S[O]]
val ev = Proof[two]
val twoInstance: two = new S[S[O]]
println(ev(twoInstance) == twoInstance)
}
It compiles, runs, and prints:
true
meaning that we have successfully invoked the recursively defined
method on the executable evidence-term of type two plus O =:= two.
Some further comments
The trivialLemma was necessary so that summons inside of other givens don't accidentally generate recursive loops, which is a bit annoying.
The separate liftCo-method for S[_ <: U] was needed, because =:=.liftCo does not allow type constructors with upper-bounded type parameters.
compiletime.erasedValue + inline match is awesome! It automatically generates some sort of runtime-gizmos that allow us to do pattern matching on an "erased" type. Before I found this out, I was attempting to construct appropriate witness terms manually, but this does not seem necessary at all, it's provided for free (see edit history for the approach with manually constructed witness terms).

Summing "Large" Nat's

Given:
scala> import shapeless.nat.
_0 _10 _12 _14 _16 _18 _2 _21 _3 _5 _7 _9 natOps
_1 _11 _13 _15 _17 _19 _20 _22 _4 _6 _8 apply toInt
scala> import shapeless.ops.nat._
import shapeless.ops.nat._
After > 3 minutes, the following code has not compiled/run. Why's that?
scala> Sum[_22, _22]
Also, looking at the above REPL auto-completion, does _44 even exist in shapeless?
Why is it so slow?
Let's start with a smaller number. When you ask for Sum[_4, _4], the compiler is going to go looking for an instance, and it'll find these two methods:
implicit def sum1[B <: Nat]: Aux[_0, B, B] = new Sum[_0, B] { type Out = B }
implicit def sum2[A <: Nat, B <: Nat](implicit
sum: Sum[A, Succ[B]]
): Aux[Succ[A], B, sum.Out] = new Sum[Succ[A], B] { type Out = sum.Out }
The first one is clearly out since _4 is not _0. It knows that _4 is the same as Succ[_3] (more on that in a second), so it'll try sum2 with A as _3 and B as _4.
This means we need to find a Sum[_3, _5] instance. sum1 is out for similar reasons as before, so we try sum2 again, this time with A = _2 and B = _5, which means we need a Sum[_2, _6], which gets us back to sum2, with A = _1 and B = _6, which sends us looking for a Sum[_1, _7]. This is the last time we'll use sum2, with A = _0 and B = _7. This time when we go looking for a Sum[_0, _8] we'll hit sum1 and we're done.
So it's clear that for n + n we're going to have to do n + 1 implicit searches, and during each one the compiler is going to be doing type equality checks and other stuff (update: see Miles's answer for an explanation of what the biggest problem here is) that requires traversing the structure of the Nat types, so we're in exponential land. The compiler really, really isn't designed to work efficiently with types like this, which means that even for small numbers, this operation is going to take a long time.
Side note 1: implementation in Shapeless
Off the top of my head I'm not entirely sure why sum2 isn't defined like this:
implicit def sum2[A <: Nat, B <: Nat](implicit
sum: Sum[A, B]
): Aux[Succ[A], B, Succ[sum.Out]] = new Sum[Succ[A], B] { type Out = Succ[sum.Out] }
This is much faster, at least on my machine, where Sum[_18, _18] compiles in four seconds as opposed to seven minutes and counting.
Side note 2: induction heuristics
This doesn't seem to be a case where Typelevel Scala's -Yinduction-heuristics helps—I just tried compiling Shapeless with the #inductive annotation on Sum and it's still seems pretty much exactly as horribly slow as without it.
What about 44?
The _1, _2, _3 type aliases are defined in code produced by this boilerplate generator in Shapeless, which is configured only to produce values up to 22. In this case specifically, this is an entirely arbitrary limit. We can write the following, for example:
type _23 = Succ[_22]
And we've done exactly the same thing the code generator does, but going one step further.
It doesn't really matter much that Shapeless's _N aliases stop at 22, though, since they're just aliases. The important thing about a Nat is its structure, and that's independent of any nice names we might have for it. Even if Shapeless didn't provide any _N aliases at all, we could still write code like this:
import shapeless.Succ, shapeless.nat._0, shapeless.ops.nat.Sum
Sum[Succ[Succ[_0]], Succ[Succ[_0]]]
And it would be exactly the same as writing Sum[_2, _2], except that it's a lot more annoying to type.
So when you write Sum[_22, _22] the compiler isn't going to have any trouble representing the result type (i.e. 44 Succs around a _0), even though it doesn't have a _44 type alias.
Following on from Travis's excellent answer, it appears that it's the use of the member type in the definition of sum2 which is the root of the problem. With the following definition of Sum and its instances,
trait Sum[A <: Nat, B <: Nat] extends Serializable { type Out <: Nat }
object Sum {
def apply[A <: Nat, B <: Nat](implicit sum: Sum[A, B]): Aux[A, B, sum.Out] = sum
type Aux[A <: Nat, B <: Nat, C <: Nat] = Sum[A, B] { type Out = C }
implicit def sum1[B <: Nat]: Aux[_0, B, B] = new Sum[_0, B] { type Out = B }
implicit def sum2[A <: Nat, B <: Nat, C <: Nat]
(implicit sum : Sum.Aux[A, Succ[B], C]): Aux[Succ[A], B, C] =
new Sum[Succ[A], B] { type Out = C }
}
which replaces the use of the member type with an additional type variable, the compile time is 0+noise on my machine both with and without -Yinduction-heurisitics.
I think that the issue we're seeing is a pathological case for subtyping with member types.
Aside from that, the induction is so small that I wouldn't actually expect -Yinduction-heurisitics to make much of an improvement.
Update now fixed in shapeless.

List of Nat of Fixed Size and Element Bounds

Using shapeless, I'm trying to define a function:
import shapeless._
import ops.nat._
import nat._
def threeNatsLessThan3[N <: Nat](xs: Sized[List[N], N])
(implicit ev: LTEq[N, _3]) = ???
where it will only compile if the input xs is a List (of sized 3) of Nat where each element is <= 3.
But that fails to compile:
scala> threeNatsLessThan3[_3](List(_1,_2,_3))
<console>:22: error: type mismatch;
found : List[shapeless.Succ[_ >: shapeless.Succ[shapeless.Succ[shapeless._0]] with shapeless.Succ[shapeless._0] with shapeless._0 <: Serializable with shapeless.Nat]]
required: shapeless.Sized[List[shapeless.nat._3],shapeless.nat._3]
(which expands to) shapeless.Sized[List[shapeless.Succ[shapeless.Succ[shapeless.Succ[shapeless._0]]]],shapeless.Succ[shapeless.Succ[shapeless.Succ[shapeless._0]]]]a>
twoNatsFirstLtEqSecond[_3](List(_1,_2,_3))
^
How can I implement the above function correctly?
Also I would appreciate a solution using an HList too, where the HList consists only of Nat elements (if possible).
What you're typing as the signature is a List of size N which contains only elements of type N. To whit, Sized[List[N], N] denotes one of the following: List(_1), List(_2, _2), or finally List(_3, _3, _3), taking into consideration your type level constraint. That's almost what you want and explains the error the compiler is giving you:
required: shapeless.Sized[List[shapeless.nat._3],shapeless.nat._3]
To begin breaking down what you want to accomplish we need to note that you can't have a List[Nat] and also preserve the individual types. The abstractness of Nat would obscure them. So if you want to do things at compile time, you're going to have three choices: work with an HList, choose to fix the type of Nat within the list so that you have List[N] or choose to fix the size of the List[Nat] with Sized.
If you want to say that the List has size less than 3, then
def lessThanThree[N <: Nat](sz: Sized[List[Nat], N])(implicit ev: LTEq[N, _3]) = sz
If you want to say that the List has a Nat of less than three, again with a fixed N within the List:
def lessThanThree[N <: Nat, M <: Nat](sz: Sized[List[N], M])(implicit ev: LTEq[N, _3]) = sz
If you're looking to perhaps work with a Poly where you could define an at for any Nat such that it preserves the LTEq constraint, you'll need to understand that Sized does make working with map conform closer to the standard package map found on most collections, i.e. it requires a CanBuildFrom. That combined with the erasure of the individual Nat in List means that you'll have a lot of difficulty coming up with a solution which gives you the type of flexibility you're looking for.
If you were to work with an HList, you could do the following:
object LT3Identity extends Poly{
implicit def only[N <: Nat](implicit ev: LTEq[N, _3]) = at[N]{ i => i}
}
def lt3[L <: HList, M <: Nat](ls: L)(implicit lg: Length.Aux[L, M], lt: LTEq[M, _3]) = ls.map(LT3Identity)
which does both constrain your size of the Hlist to less than 3 while also only allowing HList that contain Nat of less than or equal to 3.

Why compound types with different order of components are equivalent in Scala?

According to the Scala Language Specification,
Two compound types are equivalent if the sequences of their component are pairwise equivalent, and occur in the same order, and their refinements are equivalent. Two refinements are equivalent if they bind the same names and the modifiers, types and bounds of every declared entity are equivalent in both refinements.
However, given
trait A { val a: Int }
trait B { val b: String }
I'm getting
scala> implicitly[A with B =:= B with A]
res0: =:=[A with B,B with A] = <function1>
i.e. they are considered equivalent, even though the order of components is different. Why?
I think the =:= evidence only asserts that each is the upper bound of the other.
trait A; trait B
import scala.reflect.runtime.{universe => ru}
val ab = ru.typeOf[A with B]
val ba = ru.typeOf[B with A]
ab =:= ba // false!
ab <:< ba // true!
ba <:< ab // true!
The implicit from Predef you get basically if LUB(X, Y) == X == Y, because then the implicit resolution finds =:=.tpEquals with the inferred upper bound.
This is most likely what you want, because it means that you can treat one type as the other, and that works because the members of A with B are equal to the members of B with A even if the trait linearisation in the implementations is different.

How to extract an element from an HList with a specific (parameterized) type

I'm chaining transformations, and I'd like to accumulate the result of each transformation so that it can potentially be used in any subsequent step, and also so that all the results are available at the end (mostly for debugging purposes). There are several steps and from time to time I need to add a new step or change the inputs for a step.
HList seems to offer a convenient way to collect the results in a flexible but still type-safe way. But I'd rather not complicate the actual steps by making them deal with the HList and the accompanying business.
Here's a simplified version of the combinator I'd like to write, which isn't working. The idea is that given an HList containing an A, and the index of A, and a function from A -> B, mapNth will extract the A, run the function, and cons the result onto the list. The resulting extended list captures the type of the new result, so several of these mapNth-ified steps can be composed to produce a list containing the result from each step:
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At[L, index.N]):
B :: L =
f(l(index)) :: l
Incidentally, I'll also need map2Nth taking two indices and f: (A, B) => C, but I believe the issues are the same.
However, mapNth does not compile, saying l(index) has type at.Out, but f's argument should be A. That's correct, of course, so what I suppose I need is a way to provide evidence that at.Out is in fact A (or, at.Out <: A).
Is there a way to express that constraint? I believe it will have to take the form of an implicit, because of course the constraint can only be checked when mapNth is applied to a particular list and function.
You're exactly right about needing evidence that at.Out is A, and you can provide that evidence by including the value of the type member in at's type:
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At[L, index.N] { type Out = A }):
B :: L =
f(l(index)) :: l
The companion objects for type classes like At in Shapeless also define an Aux type that includes the output type as a final type parameter.
def mapNth[L <: HList, A, B]
(l: L, index: Nat, f: A => B)
(implicit at: shapeless.ops.hlist.At.Aux[L, index.N, A]):
B :: L =
f(l(index)) :: l
This is pretty much equivalent but more idiomatic (and it looks a little nicer).