I am trying and failing to get something like this to work in Scala 3:
type TupleK[K[*], V[*], A] = (K[A], V[A])
final class MapK[K[*], V[*]] private (val rawMap: Map[K[?], V[?]]) {
def foreach(f: TupleK[K, V, ?] => Unit): Unit = {
rawMap.foreach(f.asInstanceOf[Tuple2[K[?], V[?]] => Any])
}
}
object MapK {
def apply[K[*], V[*]](entries: TupleK[K, V, ?]*): MapK[K, V] = {
new MapK[K, V](Map(entries: _*))
}
}
With usage like this:
class Key[A]()
type Id[A] = A
val intKey = Key[Int]
val strKey = Key[String]
MapK[Key, Id](intKey -> 1, strKey -> "a")
In Scala 2 that works, just need to adjust syntax by replacing * and ? with _ (except in _* of course).
In Scala 3 however basically every line errors with "unreducible application of higher-kinded type to wildcard arguments": Scastie.
The docs say that existential types have been dropped in Scala 3, however they don't really give any non-trivial examples of how to deal with this.
The docs mention that "Existential types largely overlap with path-dependent types" – can this MapK be implemented with path-dependent types? I've read this but didn't understand how to apply that, or whether it's possible in my case.
And, if not path dependent types... then what? It seems unlikely that Scala was "simplified" to the point where it's impossible to implement this functionality anymore, so I must be missing something.
ETA: In addition to my own answers below, I made this repo and wrote this article about the various approaches to encoding MapK in Scala 3.
I was able to produce a working, albeit incredibly annoying, implementation. This pointer was especially valuable.
First, a few notes:
Type inference on this sucks on many levels. All the manual type ascribtions in the tests, and all of the implicit conversions below are needed for this to work.
Apparently Scala is not smart enough to figure out that A and type Id[A] = A are functionally the same when looking for implicits, so a combinatorial explosion of ad-hoc implicit conversions are needed. Ugly and not very scalable.
Observe the different options available in Scala 3: foreach, foreachT, and foreachK. All of them have stylistic tradeoffs.
If you can improve on any of this, please let me know. This works, but it was so much nicer in Scala 2.
MapK implementation:
class MapK[K[_], V[_]] protected(protected val rawMap: Map[Type[K], Type[V]]) {
def apply[A](key: K[A]): V[A] = {
rawMap(key).asInstanceOf[V[A]]
}
def updated[A](key: K[A], value: V[A]): MapK[K, V] = {
MapK.unsafeCoerce(rawMap.updated(key, value))
}
def updated[A](pair: (K[A], V[A])): MapK[K, V] = {
MapK.unsafeCoerce(rawMap.updated(pair._1, pair._2))
}
def foreach[A](f: ((K[A], V[A])) => Unit): Unit = {
rawMap.foreach(f.asInstanceOf[(([Type[K], Type[V]])) => Any])
}
def foreachT(f: Type.Tuple2[K, V] => Unit): Unit = {
foreach { (k, v) => f((k, v)) }
}
def foreachK(f: [A] => (K[A], V[A]) => Unit): Unit = {
foreach { (k, v) => f(k, v) }
}
}
object MapK {
def unsafeCoerce[K[_], V[_]](rawMap: Map[Type[K], Type[V]]): MapK[K, V] = {
new MapK[K, V](rawMap)
}
def apply[K[_], V[_]](entries: Type.Tuple2[K, V]*): MapK[K, V] = {
new MapK[K, V](Map(entries.asInstanceOf[Seq[(Type[K], Type[V])]]: _*))
}
}
Other methods in MapK that you might want to implement basically follow the same patterns as foreach, foreachT, or foreachK.
And now, usage:
def test(caption: String)(code: => Unit): Unit = code
def assertEquals[A](a: A, b: A): Unit = assert(a == b)
case class Key[A](label: String, default: A)
val boolKey = Key[Boolean]("bool", false)
val intKey = Key[Int]("int", 0)
val strKey = Key[String]("str", "")
val optionMap = MapK[Key, Option](boolKey -> Some(true), intKey -> Some(1), strKey -> Option("a"), strKey -> None)
val idMap = MapK[Key, Id](boolKey -> true, intKey -> 1, strKey -> "hello")
val expectedOptionValues = List[Type.Tuple3[Key, Option, Id]](
(boolKey, Some(true), false),
(intKey, Some(1), 0),
(strKey, None, "")
)
val expectedIdValues = List[Type.Tuple3[Key, Id, Id]](
(boolKey, true, false),
(intKey, 1, 0),
(strKey, "hello", "")
)
test("optionMap - apply & updated") {
assertEquals(optionMap(intKey), Some(1))
assertEquals(optionMap(strKey), None)
assertEquals(optionMap.updated(strKey, Some("yo"))(strKey), Some("yo"))
}
test("optionMap - foreach") {
var values: List[Type.Tuple3[Key, Option, Id]] = Nil
optionMap.foreach { (k, v) =>
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedOptionValues)
}
test("optionMap - foreachT") {
var values: List[Type.Tuple3[Key, Option, Id]] = Nil
optionMap.foreachT { pair => // no parameter untupling :(
values = values :+ (pair._1, pair._2, pair._1.default)
}
assertEquals(values, expectedOptionValues)
}
test("optionMap - foreachK") {
var values: List[Type.Tuple3[Key, Option, Id]] = Nil
optionMap.foreachK {
[A] => (k: Key[A], v: Option[A]) => // need explicit types :(
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedOptionValues)
}
test("idMap - apply & updated") {
assertEquals(idMap(intKey), 1)
assertEquals(idMap(strKey), "hello")
assertEquals(idMap.updated(strKey, "yo")(strKey), "yo")
}
test("idMap - foreach") {
var values: List[Type.Tuple3[Key, Id, Id]] = Nil
idMap.foreach { (k, v) =>
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedIdValues)
}
test("idMap - foreachT") {
var values: List[Type.Tuple3[Key, Id, Id]] = Nil
idMap.foreachT { pair =>
values = values :+ (pair._1, pair._2, pair._1.default)
}
assertEquals(values, expectedIdValues)
}
test("idMap - foreachK") {
var values: List[Type.Tuple3[Key, Id, Id]] = Nil
idMap.foreachK {
[A] => (k: Key[A], v: A) =>
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedIdValues)
}
And now, the support cast that makes this work:
import scala.language.implicitConversions // old style, but whatever
type Id[A] = A
type Type[F[_]] <: (Any { type T })
object Type {
type Tuple2[F[_], G[_]] <: (Any { type T })
type Tuple3[F[_], G[_], H[_]] <: (Any { type T })
}
implicit def wrap[F[_], A](value: F[A]): Type[F] =
value.asInstanceOf[Type[F]]
implicit def wrapT2[F[_], G[_], A](value: (F[A], G[A])): Type.Tuple2[F, G] =
value.asInstanceOf[Type.Tuple2[F, G]]
implicit def wrapT2_P1[F[_], A](t: (F[A], A)): Type.Tuple2[F, Id] = wrapT2[F, Id, A](t)
implicit def wrapT3[F[_], G[_], H[_], A](value: (F[A], G[A], H[A])): Type.Tuple3[F, G, H] =
value.asInstanceOf[Type.Tuple3[F, G, H]]
implicit def wrapT3_P1[F[_], G[_], A](value: (F[A], A, A)): Type.Tuple3[F, Id, Id] =
value.asInstanceOf[Type.Tuple3[F, Id, Id]]
implicit def wrapT3_P1_P2[F[_], G[_], A](value: (F[A], G[A], A)): Type.Tuple3[F, G, Id] =
value.asInstanceOf[Type.Tuple3[F, G, Id]]
implicit def unwrap[F[_]](value: Type[F]): F[value.T] =
value.asInstanceOf[F[value.T]]
implicit def unwrapT2[F[_], G[_]](value: Type.Tuple2[F, G]): (F[value.T], G[value.T]) =
value.asInstanceOf[(F[value.T], G[value.T])]
implicit def unwrapT3[F[_], G[_], H[_]](value: Type.Tuple3[F, G, H]): (F[value.T], G[value.T], H[value.T]) =
value.asInstanceOf[(F[value.T], G[value.T], H[value.T])]
Here's an alternative solution, using dependent types. In general I like it better, it's more obvious to me what's going on.
import scala.language.implicitConversions
type Id[A] = A
implicit def wrapId[A](a: A): Id[A] = a
implicit def unwrapId[A](a: Id[A]): A = a
case class Key[A](caption: String, default: A)
val boolKey = Key[Boolean]("bool", false)
val intKey = Key[Int]("int", 0)
val strKey = Key[String]("str", "")
type KTuple[K[_], V[_]] = {
type T;
type Pair = (K[T], V[T]);
}
implicit def KTuple[K[_], V[_], A](value: (K[A], V[A])): KTuple[K, V]#Pair = value.asInstanceOf[KTuple[K, V]#Pair]
implicit def KTuple_P1[K[_], A](value: (K[A], A)): KTuple[K, Id]#Pair = value.asInstanceOf[KTuple[K, Id]#Pair]
class MapK[K[_], V[_]](rawMap: Map[K[Any], V[Any]]) {
def foreachK(f: [A] => (K[A], V[A]) => Unit): Unit = {
rawMap.foreach(f.asInstanceOf[((K[Any], V[Any])) => Unit])
}
def foreach(f: KTuple[K, V]#Pair => Unit): Unit = {
rawMap.foreach { pair =>
f(pair.asInstanceOf[KTuple[K, V]#Pair])
}
}
}
object MapK {
def create[K[_], V[_]](pairs: KTuple[K, V]#Pair*): MapK[K, V] = {
val x: List[KTuple[K, V]#Pair] = pairs.toList
val y: List[(K[Any], V[Any])] = x.map(t => t.asInstanceOf[(K[Any], V[Any])])
new MapK(Map(y: _*))
}
}
val idMap = MapK.create[Key, Id](
boolKey -> false,
intKey -> 1,
strKey -> "a",
)
val optionMap = MapK.create[Key, Option](
intKey -> Some(1),
strKey -> Some("a")
)
type T3[A] = (Key[A], A, A)
var log = List[KTuple[Key, Option]#Pair]()
idMap.foreach { (k, v) =>
log = log.appended(KTuple(k, Some(v)))
}
def doSomething[A, V[_]](k: Key[A], v: V[A]): Unit = println(s"$k -> v")
optionMap.foreachK {
[A] => (k: Key[A], v: Option[A]) => {
doSomething(k, v.get)
doSomething(k, v)
log = log :+ KTuple((k, v))
}
}
I wrote up a blog post with more details, will publish soon after some editing. Still looking for better approaches and improvements though.
I'm learning about the Free monads, and I've put together a simple example in Scala where I use them to define two domain specific languages.
The first monad deals with the side effects of a repository. I have implemented an interpreter that uses the state monad to manage the state, but in a real program I'd use a database.
The second monad deals with IO.
import cats.data.State
import cats.{Id, ~>}
import cats.free.Free
import cats.free.Free.liftF
final case class Todo(title: String, body: String)
def represent(todo: Todo) = s"${todo.title}: ${todo.body}"
sealed trait CRUDActionA[T]
final case class Find(key: String) extends CRUDActionA[Option[Todo]]
final case class Add(data: Todo) extends CRUDActionA[Unit]
type CRUDAction[T] = Free[CRUDActionA, T]
def find(key: String): CRUDAction[Option[Todo]] = liftF[CRUDActionA, Option[Todo]](Find(key))
def add(data: Todo): CRUDAction[Unit] = liftF[CRUDActionA, Unit](Add(data))
type TodosState[A] = State[List[Todo], A]
val repository: CRUDActionA ~> TodosState = new (CRUDActionA ~> TodosState) {
def apply[T](fa: CRUDActionA[T]): TodosState[T] = fa match {
case Add(todo) => State.modify(todos => todos :+ todo)
case Find(title) => State.inspect(todos => todos find (_.title == title))
}
}
sealed trait IOActionA[T]
final case class Out(str: String) extends IOActionA[Unit]
type IOAction[T] = Free[IOActionA, T]
def out(str: String): IOAction[Unit] = liftF[IOActionA, Unit](Out(str))
val io: IOActionA ~> Id = new (IOActionA ~> Id) {
override def apply[A](fa: IOActionA[A]): Id[A] = fa match {
case Out(todo) => println(todo)
}
}
Then, I can put together these two "programs"
def addNewTodo: Free[CRUDActionA, Option[Todo]] = for {
_ <- add(Todo(title = "Must do", body = "Must do something"))
todo <- find("Must do")
} yield todo
def outProgram(todo: Todo): IOAction[Unit] = for {
_ <- out(represent(todo))
} yield ()
And run them doing
val (_, mayBeTodo) = (addNewTodo foldMap repository run List()).value
outProgram(mayBeTodo.get).foldMap(io)
I understand this is far from ideal, and I'd like to write a program as and an interpreter that supports:
def fullProgram = for {
_ <- add(Todo(title = "Must do", body = "Must do something"))
todo <- find("Must do") // This is an option!!!
_ <- out(represent(todo)) // But represent expects a Todo
} yield ()
So the questions are:
How can I stack the two monads together into a "fullProgram"
How can I compose the two interpreters into a new interpreter?
How do I deal with the Option[Todo] returned by find, and then passed to
represent
Answer to questions 1 & 2:
type TodoApp[A] = Coproduct[IOActionA, CRUDActionA, A]
class CRUDActions[F[_]](implicit I: Inject[CRUDActionA, F]) {
def find(key: String): Free[F, Option[Todo]] = Free.inject[CRUDActionA, F](Find(key))
def add(data: Todo): Free[F, Unit] = Free.inject[CRUDActionA, F](Add(data))
}
object CRUDActions {
implicit def crudActions[F[_]](implicit I: Inject[CRUDActionA, F]): CRUDActions[F] = new CRUDActions[F]
}
class IOActions[F[_]](implicit I: Inject[IOActionA, F]) {
def out(str: String): Free[F, Unit] = Free.inject[IOActionA, F](Out(str))
}
object IOActions {
implicit def ioActions[F[_]](implicit I: Inject[IOActionA, F]): IOActions[F] = new IOActions[F]
}
def fullProgram(implicit C : CRUDActions[TodoApp], I : IOActions[TodoApp]): Free[TodoApp, Unit] = {
for {
_ <- C.add(Todo(title = "Must do", body = "Must do something"))
todo <- C.find("Must do")
_ <- I.out(represent(todo.get))
} yield ()
}
object ConsoleCatsInterpreter extends (IOActionA ~> Id) {
def apply[A](i: IOActionA[A]) = i match {
case Out(prompt) => println(prompt).asInstanceOf[A]
}
}
object MutableListCrudInterpreter extends (CRUDActionA ~> Id) {
val data = new ListBuffer[Todo]
override def apply[A](fa: CRUDActionA[A]): Id[A] = fa match {
case Add(todo) => data.append(todo).asInstanceOf[A]
case Find(title) => data.find( _.title == title).asInstanceOf[A]
}
}
val interpreter: TodoApp ~> Id = ConsoleCatsInterpreter or MutableListCrudInterpreter
fullProgram.foldMap(interpreter)
Here is an example from the stairway book:
object Example1 {
import collection._
class PrefixMap[T]
extends mutable.Map[String, T]
with mutable.MapLike[String, T, PrefixMap[T]] {
var suffixes: immutable.Map[Char, PrefixMap[T]] = Map.empty
var value: Option[T] = None
def get(s: String): Option[T] = {
// base case, you are at the root
if (s.isEmpty) value
// recursive
else suffixes get (s(0)) flatMap (_.get(s substring 1))
}
def iterator: Iterator[(String, T)] = {
(for (v <- value.iterator) yield ("", v)) ++
(for ((chr, m) <- suffixes.iterator; (s, v) <- m.iterator) yield (chr +: s, v))
}
def +=(kv: (String, T)): this.type = {
update(kv._1, kv._2)
this
}
def -=(key: String): this.type = {
remove(key)
this
}
def withPrefix(s: String): PrefixMap[T] = {
if (s.isEmpty) this
else {
val leading = s(0)
suffixes get leading match {
case None => {
// key does not exist, create it
suffixes = suffixes + (leading -> empty)
}
case _ =>
}
// recursion
suffixes(leading) withPrefix (s substring 1)
}
}
override def update(s: String, elem: T) = {
withPrefix(s).value = Some(elem)
}
override def remove(key: String): Option[T] = {
if (key.isEmpty) {
// base case. you are at the root
val prev = value
value = None
prev
} else {
// recursive
suffixes get key(0) flatMap (_.remove(key substring 1))
}
}
override def empty = PrefixMap.empty
}
import collection.mutable.{Builder, MapBuilder}
import collection.generic.CanBuildFrom
object PrefixMap {
def empty[T] = new PrefixMap[T]
def apply[T](kvs: (String, T)*): PrefixMap[T] = {
val m: PrefixMap[T] = empty
for(kv <- kvs) m += kv
m
}
def newBuilder[T]: Builder[(String, T), PrefixMap[T]] = {
new mutable.MapBuilder[String, T, PrefixMap[T]](empty)
}
implicit def canBuildFrom[T]: CanBuildFrom[PrefixMap[_], (String, T), PrefixMap[T]] = {
new CanBuildFrom[PrefixMap[_], (String, T), PrefixMap[T]] {
def apply(from: PrefixMap[_]) = newBuilder[T]
def apply() = newBuilder[T]
}
}
}
}
I don't understand this line:
def apply(from: PrefixMap[_]) = newBuilder[T]
What's point in receiving a PrefixMap and returning a empty PrefixMap?
Read little bit more official docs
If in short: CBF can return builder with knowledge of properties of whole collection.
For example it could preinitialize some buffer of needed size to collect entries.
Or even reuse some parts of collection of known type and structure.
But by default in many case it would just collect element by element to empty collection. That's happening in your case.
Inspired by travisbrown, I'm trying to use a macro to create some "smart constructors".
Given
package mypkg
sealed trait Hello[A]
case class Ohayo[A,B](a: (A,B)) extends Hello[A]
and
val smartConstructors = FreeMacros.liftConstructors[Hello]
The macro should find all the subclasses of Hello, look at their constructors, and extract a few elements to populate this tree for the "smart constructor":
q"""
def $methodName[..$typeParams](...$paramLists): $baseType =
$companionSymbol[..$typeArgs](...$argLists)
"""
I hoped to get:
val smartConstructors = new {
def ohayo[A, B](a: (A, B)): Hello[A] = Ohayo[A, B](a)
}
but instead get:
error: type mismatch;
found : (A(in class Ohayo), B(in class Ohayo))
required: ((some other)A(in class Ohayo), (some other)B(in class Ohayo))
val liftedConstructors = FreeMacros.liftConstructors[Hello]
At a glance, the tree looks ok to me:
scala> q" new { ..$wellTyped }"
res1: u.Tree =
{
final class $anon extends scala.AnyRef {
def <init>() = {
super.<init>();
()
};
def ohayo[A, B](a: (A, B)): net.arya.constructors.Hello[A] = Ohayo[A, B](a)
};
new $anon()
}
but I guess it invisibly isn't. If I naively try to freshen up the typeParams with info.typeParams.map(p => TypeName(p.name.toString)), I get "can't splice A as type parameter" when I do the quasiquoting.
Where am I going wrong? Thanks for taking a look.
-Arya
import scala.language.experimental.macros
import scala.reflect.api.Universe
import scala.reflect.macros.whitebox
class FreeMacros(val c: whitebox.Context) {
import c.universe._
import FreeMacros._
def liftedImpl[F[_]](implicit t: c.WeakTypeTag[F[_]]): Tree = {
val atc = t.tpe
val childSymbols: Set[ClassSymbol] = subCaseClassSymbols(c.universe)(atc.typeSymbol.asClass)
val wellTyped = childSymbols.map(ctorsForSymbol(c.universe)(atc)).unzip
q"new { ..${wellTyped} }"
}
}
object FreeMacros {
def liftConstructors[F[_]]: Any = macro FreeMacros.liftedImpl[F]
def smartName(name: String): String = (
name.toList match {
case h :: t => h.toLower :: t
case Nil => Nil
}
).mkString
def subCaseClassSymbols(u: Universe)(root: u.ClassSymbol): Set[u.ClassSymbol] = {
val subclasses = root.knownDirectSubclasses
val cast = subclasses.map(_.asInstanceOf[u.ClassSymbol])
val partitioned = mapped.partition(_.isCaseClass)
partitioned match {
case (caseClasses, regularClasses) => caseClasses ++ regularClasses.flatMap(r => subCaseClassSymbols(u)(r))
}
}
def ctorsForSymbol(u: Universe)(atc: u.Type)(caseClass: u.ClassSymbol): (u.DefDef, u.DefDef) = {
import u._
import internal._
// these didn't help
// def clearTypeSymbol(s: Symbol): TypeSymbol = internal.newTypeSymbol(NoSymbol, s.name.toTypeName, s.pos, if(s.isImplicit)Flag.IMPLICIT else NoFlags)
// def clearTypeSymbol2(s: Symbol): TypeSymbol = internal.newTypeSymbol(NoSymbol, s.name.toTypeName, NoPosition, if(s.isImplicit)Flag.IMPLICIT else NoFlags)
// def clearTypeDef(d: TypeDef): TypeDef = internal.typeDef(clearTypeSymbol(d.symbol))
val companionSymbol: Symbol = caseClass.companion
val info: Type = caseClass.info
val primaryCtor: Symbol = caseClass.primaryConstructor
val method = primaryCtor.asMethod
val typeParams = info.typeParams.map(internal.typeDef(_))
// val typeParams = info.typeParams.map(s => typeDef(newTypeSymbol(NoSymbol, s.name.toTypeName, NoPosition, NoFlags)))
// val typeParams = info.typeParams.map(s => internal.typeDef(clearTypeSymbol2(s)))
val typeArgs = info.typeParams.map(_.name)
val paramLists = method.paramLists.map(_.map(internal.valDef(_)))
val argLists = method.paramLists.map(_.map(_.asTerm.name))
val baseType = info.baseType(atc.typeSymbol)
val List(returnType) = baseType.typeArgs
val methodName = TermName(smartName(caseClass.name.toString))
val wellTyped =
q"""
def $methodName[..$typeParams](...$paramLists): $baseType =
$companionSymbol[..$typeArgs](...$argLists)
"""
wellTyped
}
}
P.S. I have been experimenting with toolbox.untypecheck / typecheck per this article but haven't found a working combination.
you need using
clas.typeArgs.map(_.toString).map(name => {
TypeDef(Modifiers(Flag.PARAM),TypeName(name), List(),TypeBoundsTree(EmptyTree, EmptyTree))
}
replace
info.typeParams.map(p => TypeName(p.name.toString))
it si my code
object GetSealedSubClass {
def ol3[T]: Any = macro GetSealedSubClassImpl.ol3[T]
}
class GetSealedSubClassImpl(val c: Context) {
import c.universe._
def showInfo(s: String) =
c.info(c.enclosingPosition, s.split("\n").mkString("\n |---macro info---\n |", "\n |", ""), true)
def ol3[T: c.WeakTypeTag]: c.universe.Tree = {
//get all sub class
val subClass = c.weakTypeOf[T]
.typeSymbol.asClass.knownDirectSubclasses
.map(e => e.asClass.toType)
//check type params must ia s sealed class
if (subClass.size < 1)
c.abort(c.enclosingPosition, s"${c.weakTypeOf[T]} is not a sealed class")
// get sub class constructor params
val subConstructorParams = subClass.map { e =>
//get constructor
e.members.filter(_.isConstructor)
//if the class has many Constructor then you need filter the main Constructor
.head.map(s => s.asMethod)
//get function param list
}.map(_.asMethod.paramLists.head)
.map(_.map(e => q"""${e.name.toTermName}:${e.info} """))
val outfunc = subClass zip subConstructorParams map {
case (clas, parm) =>
q"def smartConstructors[..${
clas.typeArgs.map(_.toString).map(name => {
TypeDef(Modifiers(Flag.PARAM), TypeName(name), List(), TypeBoundsTree(EmptyTree, EmptyTree))
})
}](..${parm})=${clas.typeSymbol.name.toTermName} (..${parm})"
}
val outClass =
q"""
object Term{
..${outfunc}
}
"""
showInfo(show(outClass))
q"""{
$outClass
Term
}
"""
}
}
using like this
sealed trait Hello[A]
case class Ohayo[A, B](a: (A, B)) extends Hello[A]
object GetSealed extends App {
val a = GetSealedSubClass.ol3[Hello[_]]
val b=a.asInstanceOf[ {def smartConstructors[A, B](a: (A, B)): Ohayo[A, B]}].smartConstructors(1, 2).a
println(b)
}