Binding Existential types in Scala - scala

This is my basic CMap which map classes (Class[T] for any T) to any type of value.
scala> type CMap = Map[Class[T] forSome{type T}, Any]
defined type alias CMap
scala> val cMap: CMap = Map(classOf[Int]->5, classOf[String]->"abc", classOf[Double]->"ddd")
cMap: CMap = Map(int -> 5, class java.lang.String -> abc, double -> ddd)
Now I want a "bound" CMap (call it CMapBind). Like CMap, it maps classes (any classes) to values (any values). But unlike the CMap, CMapBind has a type binding between the key and value, meaning I hope the following behavior:
val cMapBind: CMapBind = Map(classOf[Int]->5, classOf[String]-> "aa") // should compile
val cMapBind: CMapBind = Map(classOf[Int]->5, classOf[String]-> 0) // should fail compile
How do I implement CMapBind?
I know the following two won't work syntactically/logically.
scala> type CMapBind = Map[Class[T] forSome{type T}, T]
<console>:8: error: not found: type T
type CMapBind = Map[Class[T] forSome{type T}, T]
scala> type CMapBind = Map[Class[T], T] forSome{type T}
scala> val cMapBind: CMapBind = Map(classOf[Int]->5, classOf[String]->"str")
<console>:8: error: type mismatch;
found : scala.collection.immutable.Map[Class[_ >: String with Int],Any]
required: CMapBind
(which expands to) Map[Class[T],T] forSome { type T }
val cMapBind: CMapBind = Map(classOf[Int]->5, classOf[String]->"str")
Note that here I am using type constructor Class[T] as an example to illustrate the question. In my code I have my own types, e.g. trait Animal[S, T], class Dog extends Animal[Int, String].
Edit 1:
I should have mentioned that I am using immutable Map as an example, but what I really need is a mutable heterogeneous Map).

Let's try to implement mutable HMap. Some explanations about Scala type system is here by #MilesSabin: http://www.chuusai.com/2011/07/16/fundeps-in-scala/
The idea is to statically check the constructor (here you will see, that it's arity depends on your hands, so it is possible to generate it or smth else), and the insert method. By the way, same way immutable HMap implemented in shapeless.
import scala.collection.mutable.Map
class HMapBuilder[R[_, _]] { // constructor arity is two
def apply[K1, V1](e1: (K1, V1))(implicit ev1: R[K1, V1]) =
new HMap[R](Map(e1))
def apply[K1, V1, K2, V2](e1: (K1, V1), e2: (K2, V2))
(implicit ev1: R[K1, V1], ev2: R[K2, V2]) =
new HMap[R](Map(e1, e2))
}
So it is a constructor of our map. Evidences will statically check types of the inserting data. Next, lets wrap the default scala collection:
class HMap[R[_, _]](underlying : Map[Any, Any] = Map.empty) {
def get[K, V](k : K)(implicit ev : R[K, V]) : Option[V] =
underlying.get(k).asInstanceOf[Option[V]]
def +=[K, V](kv : (K, V))(implicit ev : R[K, V]) : HMap[R] = {
underlying += kv
this
}
def -=[K](k : K) : HMap[R] = {
underlying -= k
this
}
override def toString = underlying.toString
}
Finally wrapping HMapBuilder, to make a pleasant constructor:
object HMap {
def apply[R[_, _]] = new HMapBuilder[R]
def empty[R[_, _]] = new HMap[R]
def empty[R[_, _]](underlying : Map[Any, Any]) =
new HMap[R](underlying)
}
In result, the usage is similar to shapeless HMap:
class Mapping[K, V]
implicit def mappingFromClass[A] = new Mapping[Class[A], A]
val hm = HMap[Mapping](classOf[Int] -> 5) // ok
hm += (classOf[String] -> "string") // ok
hm += (classOf[Boolean] -> false) // ok
hm += (classOf[Double] -> "sss") // compile fail
Works as expected. I implemented only insert and remove functions, other functions is possible to define same way.

Related

scala 3: value as a class parameter and valueof

I am trying to specify a value as a class parameter:
import scala.collection.mutable
class MyMap[K,V,D <: V] {
private val impl = mutable.Map[K, V]()
def apply(k:K):V = impl.getOrElse(k, valueOf[D])
// ^^^^^^^^^^ error
def put(k:K,v:V):Unit = impl += k -> v
}
val m = MyMap[Int, Int, 0]()
m(100) // must return 0
For some reason, the above does not work. Instead, what works is the below code with this illogical unusedName:
import scala.collection.mutable
class MyMap[K,V,D <: V] {
private val impl = mutable.Map[K, V]()
def apply(k:K)(implicit unusedName:ValueOf[D]):V = impl.getOrElse(k, valueOf[D])
def put(k:K,v:V):Unit = impl += k -> v
}
val m = MyMap[Int, Int, 0]()
m(100) // must return 0
I tried to pass the implicit agrument explicitly, which in theory should be possible, but it did not work as I would expect:
scala> m(2)(0)
-- [E007] Type Mismatch Error: ----
1 |m(2)(0)
| ^
| Found: (0 : Int)
| Required: ValueOf[(0 : Int)]
scala> m(2)(ValueOf[0])
-- Error: --------------------------
1 |m(2)(ValueOf[0])
| ^^^^^^^^^^
| missing arguments for constructor ValueOf in class ValueOf
scala> m(2)(ValueOf[0](0))
val res7: Int = 0
Questions:
How does all this ValueOf/valueOf magic work?
Why didn't just valueOf[D] work? (It would be only logical)
UPDATE
the code referenced in the comments is:
import scala.collection.mutable
final class MyMap[K, V] private (default: V) {
private val impl = mutable.Map.empty[K, V]
def apply(k: K): V =
impl.getOrElse(k, default)
def put(k: K,v: V): Unit = {
impl += k -> v
}
}
object MyMap {
def empty[K, V, D <: V](using ValueOf[D]): MyMap[K, V] =
new MyMap(default = valueOf[D])
}
val m = MyMap.empty[Int, Int, 0]
m(100)
Code fragment 2:
import scala.collection.mutable
final class MyMap[K, V, D <: V](using ValueOf[D]) {
private val default = valueOf[D]
private val impl = mutable.Map.empty[K, V]
def apply(k: K): V =
impl.getOrElse(k, default)
def put(k: K,v: V): Unit = {
impl += k -> v
}
}
val m = MyMap[Int, Int, 0]()
m(100)
How does all this ValueOf/valueOf magic work?
ValueOf[T] is the proposition that there exists one distinguished value of type T.
The witness instances for all reasonable types T are generated by the compiler automatically.
For each v: ValueOf[T], the member access v.value gives the distinguished value of type T.
The method valueOf[T] is just unpacking the witnesses ValueOf[T] into the values of type T.
It could be somewhat instructive to quickly re-build this mechanism from scratch:
case class MyValueOf[T](value: T)
// For ValueOf, these proofs are generated automatically by the compiler;
// Here, we have to do it ourselves
given MyValueOf[0] = MyValueOf(0)
def myValueOf[A](using mvo: MyValueOf[A]): A = mvo.value
println(summon[MyValueOf[0]]) // MyValueOf(0), the proof
println(myValueOf[0]) // 0, the value
Why didn't just valueOf[D] work? (It would be only logical)
Because, as the signature says, the method valueOf[D] requires a ValueOf[D] as implicit argument. The instances of ValueOf[D] can be generated when the D is known. It can not be generated for arbitrary generic type D (if that were the case, then your code would work for a Map[Int, Int, Nothing], which is clearly absurd).
Regarding your REPL experiments:
m(2)(0) clearly shouldn't work, because 0 is of type Int, not ValueOf[Int].
ValueOf[0] is interpreted as an incomplete constructor invocation. If you meant the type instead of the value, you would have to use summon to materialize the value for the type, i.e. m(2)(summon[ValueOf[0]])
ValueOf[0](0) is basically the same as what the compiler would generate anyway when asked to summon[ValueOf[0]].
Regarding your Map-wrapper:
You probably want to pass the default value once, when you create the map:
import scala.collection.mutable
class MyMap[K, V, D <: V](using defaultValueOf: ValueOf[D]) {
private val impl = mutable.Map[K, V]()
def apply(k: K): V = impl.getOrElse(k, valueOf[D])
def put(k: K, v: V): Unit = impl += k -> v
}
val m = MyMap[Int, Int, 0]()
println(m(100)) // prints '0'
The wrapper itself does not seem to be very helpful, because there is already a method withDefault defined on all mutable maps.

Scalac cannot infer inductively built path-dependent type

I'm working on a port of servant-server to Scala. The idea is to use typeclass resolution to inductively build functions that can handle requests. I'm running into some weird inference issues I can't quite figure out.
object Servant {
class :>[Path, A]
trait HasServer[A] {
type ServerT
def route(a: ServerT): String
}
implicit val unitServer = new HasServer[Unit] {
type ServerT = String
def route(str: ServerT): String = str
}
implicit def subServer[A, Sub](implicit sub: HasServer[Sub]) = new HasServer[A :> Sub] {
type ServerT = A => sub.ServerT
def route(handler: ServerT): String = "handler"
}
}
With the above, the following fails to compile:
val foo = implicitly[HasServer[Int :> Unit]]
implicitly[=:=[Int => String, foo.ServerT]]
The error is:
servant.scala:33: error:
Cannot prove that Int => String =:= Main.$anon.Servant.foo.ServerT.
However, it will compile if I instantiate HasServer[Int :> Unit] directly, via:
val foo = new HasServer[Int :> Unit] {
type ServerT = Int => unitServer.ServerT
def route(handler: ServerT): String = handler(10)
}
How can I get this to compile? Thanks!
The problem is all in the definition of implicitly ...
def implicitly[T](implicit e: T) = e
implicitly[T] will only ever give you a value which is typed as T, never anything more precise. In the case above, that's HasServer[Int :> Unit] which, crucially, leaves the member type ServerT unconstrained.
This is commonly worked around by defining a per type class companion object apply method which preserves the desired refinement, eg.,
object HasServer {
def apply[T](implicit hs: HasServer[T]):
HasServer[T] { type ServerT = hs.ServerT } = hs
}
the result type here is a little unwieldy, so it's also common to combine this with the "Aux" pattern,
object HasServer {
type Aux[T, S] = HasServer[T] { type ServerT = S }
def apply[T](implicit hs: HasServer[T]): Aux[T, hs.ServerT] = hs
}
which will probably come in handy elsewhere in any case.
We can see the difference this makes to the inferred types on the REPL,
scala> implicitly[HasServer[Int :> Unit]]
res0: Servant.HasServer[Servant.:>[Int,Unit]] = ...
scala> HasServer[Int :> Unit]
res1: Servant.HasServer[Servant.:>[Int,Unit]]{type ServerT = Int => String} = ...
This refinement will be inferred as the type of a val definition, so now you'll get the desired result,
scala> val foo = HasServer[Int :> Unit]
foo: Servant.HasServer[Servant.:>[Int,Unit]]{type ServerT = Int => String} = ...
scala> implicitly[=:=[Int => String, foo.ServerT]]
res2: =:=[Int => String,foo.ServerT] = <function1>
There are a number of ways that the definition of implicitly could be improved to avoid this problem. The following is the most straightforward for reference types,
def implicitly[T <: AnyRef](implicit t: T): t.type = t
and if literal types are enabled we could removed the <: AnyRef bound and define it for all types,
def implicitly[T](implicit t: T): t.type = t
shapeless provides a the[T] operator which behaves similarly to the latter via a macro.

Type information is lost when working with high-kinded objects in Scala

I am trying to factorize some code and ended up having to work with higher-kinded types.
The following minimal example works fine:
trait Builder[M[_]] {
def build[A]: M[A]
def buildPair[A, B]: (M[A], M[B]) = (build[A], build[B])
}
class List[A]
class BuilderList extends Builder[List] {
def build[A] = new List[A]
}
val l: List[String] = (new BuilderList).build[String]
val ll: (List[String], List[Double]) = (new BuilderList).buildPair[String, Double]
defined trait Builder
defined class List
defined class BuilderList
l: List[String] = List#5c7754a7
ll: (List[String], List[Double]) = (List#62874648,List#7b0f6537)
If I now want to apply this to a type with two type arguments, say
class Map[K, V]
I would like to be able to write
trait BuilderMap[K] extends Builder[Map[K, _]] {...}
but of course this does not work because type arguments in Scala are not curried.
I found that the following trick allowed me to pass compilation:
trait PartialApplier[F[_, _], K] {
type PartiallyApplied[_] = F[K, _]
}
class BuilderMap[K] extends Builder[PartialApplier[Map, K]#PartiallyApplied] {
def build[V] = new Map[K, V]
}
But then, some strange effect happens and I can't figure out the reason why:
scala> val m: Map[Int, String] = (new BuilderMap[Int]).build[String]
m: Map[Int,String] = Map#71da0c64
scala> val mm: (Map[Int, String], Map[Int, Double]) = (new BuilderMap[Int]).buildPair[String, Double]
<console>:21: error: type mismatch;
found : (Map[Int, _], Map[Int, _])
required: (Map[Int,String], Map[Int,Double])
val mm: (Map[Int, String], Map[Int, Double]) = (new BuilderMap[Int]).buildPair[String, Double]
It seems that the functions defined in the higher-kinded trait Builder lose some type information when I use the PartialApplier trick.
Is there a way to make all this work smoothly together? Maybe the PartialApplier trick is not the right way to go!
In your example in type PartiallyApplied[_] = F[K, _] the underscores don't specify the same type.
It works, if you define PartialApplier as
trait PartialApplier[F[_, _], K] {
type PartiallyApplied[V] = F[K, V]
}
Also, you may avoid using PartialApplier at all, and define Builder type parameter like this:
class BuilderMap[K] extends Builder[({ type L[V] = Map[K, V] })#L] {
def build[V] = Map.empty[K, V]
}

Successfully implement MapLike in Scala

I'm having a dastard of a time implementing a Map in scala typed to (for our purposes) [String, Set[Foo]] that provides extra operations for the Foos in the values. The actual implementation is more complicated than presented below, but this is the gist. I need a class that implements all the Map-like functions and provides the extra-and-above aggregation on the collections that are the values of the map. The extends Map with MapLike patterns I've seen, particularly this, are not working.
What I've got so far:
import scala.collection.{immutable, Map, MapLike}
class Foo(a:Int)
class CustomMap
(val underlying:Map[String,Set[Foo]] = Map[String,Set[Foo]]())
extends Map[String, Set[Foo]] with MapLike[String, Set[Foo], CustomMap] {
override def empty = new CustomMap(underlying.empty)
def -(key: String) = new CustomMap(underlying - key)
def +[B1 >: Set[Foo]](kv: (String, B1)): scala.collection.Map[String,B1] = new CustomMap(underlying + (kv._1 -> kv._2))
def get(key: String): Option[Set[Foo]] = underlying.get(key)
def iterator: Iterator[(String, Set[Foo])] = underlying.iterator
override def size = underlying.size
def getAllFoos() = underlying.values.flatten.toSet
}
val cm1:CustomMap = new CustomMap(Map("a" -> Set(new Foo(1))))
val cm2:CustomMap = cm1 + ("a" -> Set(new Foo(2)))
println(cm2.getAllFoos)
There are issues both with the + method and accessing the extra aggregate methods.
Any pointers?
file.scala:12: error: type mismatch;
found : B1
required: Set[this.Foo]
def +[B1 >: Set[Foo]](kv: (String, B1)): scala.collection.Map[String,B1] = new CustomMap(underlying + (kv._1 -> kv._2))
^
file.scala:24: error: type mismatch;
found : scala.collection.Map[String,Set[this.Foo]]
required: this.CustomMap
val cm2:CustomMap = cm1 + ("a" -> Set(new Foo(2)))
^
two errors found
+ can't return a CustomMap because sometimes B1 won't be a Set[Foo] but some other supertype of Set[Foo]. This is the source of your errors. Map and MapLike are meant for classes that provide a map implementation that could safely have any value added to it and will return a usable Map. So a Map[String, Set[Foo]] can always have an ("", 5) put into it and become a Map[String, Any].
You can eliminate the wrapper around your underlying, as well as avoid these problems, by using the "pimp-my-library" pattern:
implicit class FooSetMap(val map: Map[String,Set[Foo]]) extends AnyVal {
def getAllFoos = map.values.flatten.toSet
}
If you are willing to use a mutable map, take a look at collection.mutable.MultiMap. It is a mixin trait which adds extra methods to subtypes of mutable.Map[A, mutable.Set[B]] for working with multimaps - you could do something similar for your needs.

What is the Scala identifier "implicitly"?

I have seen a function named implicitly used in Scala examples. What is it, and how is it used?
Example here:
scala> sealed trait Foo[T] { def apply(list : List[T]) : Unit }; object Foo {
| implicit def stringImpl = new Foo[String] {
| def apply(list : List[String]) = println("String")
| }
| implicit def intImpl = new Foo[Int] {
| def apply(list : List[Int]) = println("Int")
| }
| } ; def foo[A : Foo](x : List[A]) = implicitly[Foo[A]].apply(x)
defined trait Foo
defined module Foo
foo: [A](x: List[A])(implicit evidence$1: Foo[A])Unit
scala> foo(1)
<console>:8: error: type mismatch;
found : Int(1)
required: List[?]
foo(1)
^
scala> foo(List(1,2,3))
Int
scala> foo(List("a","b","c"))
String
scala> foo(List(1.0))
<console>:8: error: could not find implicit value for evidence parameter of type
Foo[Double]
foo(List(1.0))
^
Note that we have to write implicitly[Foo[A]].apply(x) since the compiler thinks that implicitly[Foo[A]](x) means that we call implicitly with parameters.
Also see How to investigate objects/types/etc. from Scala REPL? and Where does Scala look for implicits?
implicitly is avaliable in Scala 2.8 and is defined in Predef as:
def implicitly[T](implicit e: T): T = e
It is commonly used to check if an implicit value of type T is available and return it if such is the case.
Simple example from retronym's presentation:
scala> implicit val a = "test" // define an implicit value of type String
a: java.lang.String = test
scala> val b = implicitly[String] // search for an implicit value of type String and assign it to b
b: String = test
scala> val c = implicitly[Int] // search for an implicit value of type Int and assign it to c
<console>:6: error: could not find implicit value for parameter e: Int
val c = implicitly[Int]
^
Here are a few reasons to use the delightfully simple method implicitly.
To understand/troubleshoot Implicit Views
An Implicit View can be triggered when the prefix of a selection (consider for example, the.prefix.selection(args) does not contain a member selection that is applicable to args (even after trying to convert args with Implicit Views). In this case, the compiler looks for implicit members, locally defined in the current or enclosing scopes, inherited, or imported, that are either Functions from the type of that the.prefix to a type with selection defined, or equivalent implicit methods.
scala> 1.min(2) // Int doesn't have min defined, where did that come from?
res21: Int = 1
scala> implicitly[Int => { def min(i: Int): Any }]
res22: (Int) => AnyRef{def min(i: Int): Any} = <function1>
scala> res22(1) //
res23: AnyRef{def min(i: Int): Int} = 1
scala> .getClass
res24: java.lang.Class[_] = class scala.runtime.RichInt
Implicit Views can also be triggered when an expression does not conform to the Expected Type, as below:
scala> 1: scala.runtime.RichInt
res25: scala.runtime.RichInt = 1
Here the compiler looks for this function:
scala> implicitly[Int => scala.runtime.RichInt]
res26: (Int) => scala.runtime.RichInt = <function1>
Accessing an Implicit Parameter Introduced by a Context Bound
Implicit parameters are arguably a more important feature of Scala than Implicit Views. They support the type class pattern. The standard library uses this in a few places -- see scala.Ordering and how it is used in SeqLike#sorted. Implicit Parameters are also used to pass Array manifests, and CanBuildFrom instances.
Scala 2.8 allows a shorthand syntax for implicit parameters, called Context Bounds. Briefly, a method with a type parameter A that requires an implicit parameter of type M[A]:
def foo[A](implicit ma: M[A])
can be rewritten as:
def foo[A: M]
But what's the point of passing the implicit parameter but not naming it? How can this be useful when implementing the method foo?
Often, the implicit parameter need not be referred to directly, it will be tunneled through as an implicit argument to another method that is called. If it is needed, you can still retain the terse method signature with the Context Bound, and call implicitly to materialize the value:
def foo[A: M] = {
val ma = implicitly[M[A]]
}
Passing a subset of implicit parameters explicitly
Suppose you are calling a method that pretty prints a person, using a type class based approach:
trait Show[T] { def show(t: T): String }
object Show {
implicit def IntShow: Show[Int] = new Show[Int] { def show(i: Int) = i.toString }
implicit def StringShow: Show[String] = new Show[String] { def show(s: String) = s }
def ShoutyStringShow: Show[String] = new Show[String] { def show(s: String) = s.toUpperCase }
}
case class Person(name: String, age: Int)
object Person {
implicit def PersonShow(implicit si: Show[Int], ss: Show[String]): Show[Person] = new Show[Person] {
def show(p: Person) = "Person(name=" + ss.show(p.name) + ", age=" + si.show(p.age) + ")"
}
}
val p = Person("bob", 25)
implicitly[Show[Person]].show(p)
What if we want to change the way that the name is output? We can explicitly call PersonShow, explicitly pass an alternative Show[String], but we want the compiler to pass the Show[Int].
Person.PersonShow(si = implicitly, ss = Show.ShoutyStringShow).show(p)
Starting Scala 3 implicitly has been replaced with improved summon which has the advantage of being able to return a more precise type than asked for
The summon method corresponds to implicitly in Scala 2. It is
precisely the same as the the method in Shapeless. The difference
between summon (or the) and implicitly is that summon can return a
more precise type than the type that was asked for.
For example given the following type
trait F[In]:
type Out
def f(v: Int): Out
given F[Int] with
type Out = String
def f(v: Int): String = v.toString
implicitly method would summon a term with erased type member Out
scala> implicitly[F[Int]]
val res5: F[Int] = given_F_Int$#7d0e5fbb
scala> implicitly[res5.Out =:= String]
1 |implicitly[res5.Out =:= String]
| ^
| Cannot prove that res5.Out =:= String.
scala> val x: res5.Out = ""
1 |val x: res5.Out = ""
| ^^
| Found: ("" : String)
| Required: res5.Out
In order to recover the type member we would have to refer to it explicitly which defeats the purpose of having the type member instead of type parameter
scala> implicitly[F[Int] { type Out = String }]
val res6: F[Int]{Out = String} = given_F_Int$#7d0e5fbb
scala> implicitly[res6.Out =:= String]
val res7: res6.Out =:= String = generalized constraint
However summon defined as
def summon[T](using inline x: T): x.type = x
does not suffer from this problem
scala> summon[F[Int]]
val res8: given_F_Int.type = given_F_Int$#7d0e5fbb
scala> summon[res8.Out =:= String]
val res9: String =:= String = generalized constraint
scala> val x: res8.Out = ""
val x: res8.Out = ""
where we see type member type Out = String did not get erased even though we only asked for F[Int] and not F[Int] { type Out = String }. This can prove particularly relevant when chaining dependently typed functions:
The type summoned by implicitly has no Out type member. For this
reason, we should avoid implicitly when working with dependently typed
functions.
A "teach you to fish" answer is to use the alphabetic member index currently available in the Scaladoc nightlies. The letters (and the #, for non-alphabetic names) at the top of the package / class pane are links to the index for member names beginning with that letter (across all classes). If you choose I, e.g., you'll find the implicitly entry with one occurrence, in Predef, which you can visit from the link there.