Akin to this question here (Mapping over Shapeless record), I am attempting to map over a trivial shapeless record (in this case, if I encounter a value of type Int, I want to convert it to a Double).
object Main extends App {
import shapeless._ ; import syntax.singleton._ ; import record._
import ops.record._
import syntax.singleton._
case class S(int:Int,t:String)
val s = S(3,"a")
val gen = LabelledGeneric[S]
val rec = gen.to(s)
val extended = rec + ('inPrint ->> true)
val removed = rec - 'int
val keys = Keys[gen.Repr]
val options =
('awesomeString ->> "a") ::
('epicInt ->> 5:Int) ::
HNil
def intToDouble(i:Int):Double = i.toDouble
object bind extends FieldPoly {
implicit def rpb[T, K](implicit witness: Witness.Aux[K]): Case.Aux[
FieldType[K, Int],
FieldType[K, Double]
] = atField(witness)(intToDouble)
}
val z = options.map(bind)
}
When I try to compile however, I get the following error
could not find implicit value for parameter mapper: shapeless.ops.hlist.Mapper[Main.bind.type,shapeless.::[String with shapeless.record.KeyTag[Symbol with shapeless.tag.Tagged[String("awesomeString")],String],shapeless.::[Int,shapeless.HNil]]]
Is there something critical that I am missing?
There's one very minor problem with your example. You have,
val options =
('awesomeString ->> "a") ::
('epicInt ->> 5:Int) ::
HNil
If you paste this into the REPL you'll see that you've lost track of the 'epicInt key in the last element of the record. This is because the type ascription binds less tightly than the ->> operator so you have, in effect, first tagged your value with its key and then immediately thrown it away again. This leaves you with a valid HList, but unfortunately not one which is the right shape to be a record. The fix is to either drop the type ascription altogether, or use parentheses (ie. (5: Int).
The bigger problem is your use of FieldPoly here. FieldPoly is intended for situations which the key you want to operate on is statically known. That's not the case here: the type variable K is free and would have to be inferred. Unfortunately there's no way that it can be: it would have to be inferred from the first argument to atField but that, in turn, depends on K via the implicit definition of witness.
It could be that a different variant of FieldPoly that better matches your scenario would be useful. In the meantime, an ordinary Poly1 which operates on whole fields (ie. the key combined with the value) will do what you want,
trait bind0 extends Poly1 {
implicit def default[E] = at[E](identity) // default case for non-Int fields
}
object bind extends bind0 {
implicit def caseInt[K] = // case for fields with Int values
at[FieldType[K, Int]](field[K](intToDouble _))
}
Related
I’m trying to write the following:
import scala.reflect.runtime.universe._
val value: Tree = /* some AST */
val tpe = typeOf(value) // This should be the result type of the AST.
// This is pseudocode. What should
// actually go on this line?
q"""
type U = $tpe
val v: U = $value
"""
I need to capture the type of the value represented by the AST value in tpe and assign it to U. How does one do this?
Edit: Giving a type annotation for value and matching on it via quasiquotes isn't an option here. The use case is a Shapeless extensible record, which has complicated types such as String with labelled.KeyTag[1, String] :: Long with labelled.KeyTag[three, Long] :: HNil for something like val ls = (1 ->> "two") :: ("three" ->> 4L) :: HNil. Also, the value AST is programmatically generated and not a literal.
Get a ToolBox, use it to typecheck value, and ask the annotated tree for its type.
import scala.runtime.reflect.currentMirror
val toolbox = currentMirror.mkToolBox()
val tpe = TypeTree(toolbox.typecheck(value).tpe)
The code you've written states you're doing this at runtime. The use case you've stated in your comment makes it seem like you're in a compile-time macro. In that case, use the similar typecheck method in your Context. It won't typecheck otherwise; the value would be a Tree from the wrong Universe, representing the fact that the new compiler instance made by ToolBox operates in the context of the current program (which happens to be a compiler), while the reflection represented by the Context is all about the future context of the code manipulated by the containing compiler.
Try to use $tpt in q"$mods val $tname: $tpt = $expr"
val t: Tree = reify {
val value: Int = 3
}.tree
val tpe = t match {
case q"{$mods val $tname: $tpt = $expr; $_}" => tpt
} // Int
https://docs.scala-lang.org/overviews/quasiquotes/syntax-summary.html#definitions
Connected question: How does one use quasiquotes to obtain the type of a value?
I'm (in effect, this is all simplified from my working example) trying to map a tagged HList (a record) of contravariant (constant return value, polymorphic argument type) functions over a record consisting of the proper argument types of those functions.
I'm aware of ZipApply, and that seems to work for HLists, but I specifically want to do this regardless of order and specifically use the key types to select which functions to use.
Here's my code example, which isn't compiling for reasons I can't figure out:
import shapeless._
import labelled._
import ops.hlist._
import ops.record.Selector
import syntax._
import syntax.singleton._
import syntax.HListOps
object POC {
case class Foo(bar: String)
val gen = LabelledGeneric[Foo]
val funs = ('bar ->> { s: String => s.toUpperCase() }) :: HNil
val poly = new Poly1 {
implicit def apply[K, V]
(implicit selector: Selector.Aux[funs.type, K, (V) => String]) =
at[FieldType[K, V]] { v => selector(funs)(v) }
}
def run(foo: Foo)(implicit mapper: Mapper[poly.type, gen.Repr]) =
mapper(gen to foo)
println(run(Foo("bar")))
// could not find implicit value for parameter mapper: shapeless.ops.hlist.Mapper[POC.<refinement>.type,POC.gen.Repr]
}
The issue here is that poly isn't a stable identifier, or it's not stable enough, or something like that. The following will work just fine:
object poly extends Poly1 { ... }
I'm not sure I've seen a much better explanation of what's going on here than Miles's comment on this answer of mine (which is now four and a half years old).
I think I need a HList that is constrained to have all of its elements being a subtype of a certain type. LUBConstraint seems to be what I want, and indeed it does constrain the construction of such a HList - but I can't see how to get the evidence out again, so that I can map (actually, traverse, because it needs to be monadic) over the HList and call a method (that exists in the LUB type) on each of the elements.
In addition, I want the type of the HList resulting from the traverse operation to be exactly the same type as the type of the input HList.
The use case is a kind of functional "listener list" - all of the elements of the HList are "listeners" which must be notified of "events", accept or reject them, and return new versions of themselves with updated "internal state". If that was all I needed, then I could just use an ordinary immutable Scala collection. But I also want direct typed access to individual elements without using asInstanceOf - hence the motivation for trying to use HList.
In general if you have some operation that you want to perform on all of the elements in an HList, you'll want to map a polymorphic function value over the HList. For example, suppose I have the following setup:
trait Listener[L <: Listener[L]] {
def handle(s: String): Option[L]
}
class FooListener extends Listener[FooListener] {
def handle(s: String) =
if (s.size == 3) Some(this) else None
}
class BarListener extends Listener[BarListener ]{
def handle(s: String) = Some(this)
}
import shapeless._
val listeners = new FooListener :: new BarListener :: HNil
Now I want to send a String to each of these listeners and gather the results. If I just wanted to send a fixed value, this would be easy:
object event123 extends Poly1 {
implicit def listener[L <: Listener[L]] = at[L](_.handle("123"))
}
val result = listeners.map(event123)
Which will be appropriately typed as an Option[FooListener] :: Option[BarListener] :: HNil. If I'm using shapeless-contrib, I can sequence this HList:
import scalaz._, Scalaz._, shapeless.contrib.scalaz._
val sequenced: Option[FooListener :: BarListener :: HNil] = sequence(result)
Or just use traverse:
traverse(listeners)(event123)
Unfortunately there are restrictions on how polymorphic function values can be defined that mean that partial application isn't convenient, so if we don't know the String we're sending at compile time, this is a lot more complicated:
object event extends Poly1 {
implicit def listener[L <: Listener[L]] = at[(L, String)] {
case (listener, string) => listener.handle(string)
}
}
traverse(listeners.zip(listeners.mapConst("123")))(event)
Where we've zipped the elements with the string and then mapped a polymorphic function that takes tuples over the result. There are other ways you could do this using more or less the same approach, but none of them are terribly clear.
A completely different approach is just to skip the polymorphic function values and define a new type class:
trait Notifiable[L <: HList] {
def tell(s: String)(l: L): Option[L]
}
object Notifiable {
implicit val hnilNotifiable: Notifiable[HNil] = new Notifiable[HNil] {
def tell(s: String)(l: HNil) = Some(HNil)
}
implicit def hconsNotifiable[H <: Listener[H], T <: HList](implicit
tn: Notifiable[T]
): Notifiable[H :: T] = new Notifiable[H :: T] {
def tell(s: String)(l: H :: T) = for {
h <- l.head.handle(s)
t <- tn.tell(s)(l.tail)
} yield h :: t
}
}
def tell[L <: HList: Notifiable](s: String)(l: L) =
implicitly[Notifiable[L]].tell(s)(l)
And then:
val sequenced: Option[FooListener :: BarListener :: HNil] =
tell("123")(listeners)
This is less generic (it only works on Option, not arbitrary applicatives), but it doesn't require an extra dependency for sequencing, and it's arguably a little less muddled than jumping through hoops to partially apply a polymorphic function value because of weird limitations of the compiler.
I want to use a map of varying types on an unknown A:
val map: Map[Foo[A], Bar[A]] = ...
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = map(foo)
This doesn't work, because A is an unknown. I have to define it instead as:
val map: Map[Foo[_], Bar[_]] = ...
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = map(foo).asInstanceOf[Bar[Qux]]
This works, but the cast is ugly. I'd rather find a better way. I gather the answer is to use existential types with the forSome keyword, but I'm confused as to how that works. Should it be:
Map[Foo[A], Bar[A]] forSome { type A }
or:
Map[Foo[A] forSome { type A }, Bar[A]]
or:
Map[Foo[A forSome { type A }], Bar[A]]
Actually, none of these work.
Map[Foo[A], Bar[A]] forSome { type A }
is a Map where all keys are of the same type Foo[A] and values of type Bar[A] (but the type A may be different for different maps of this type); in second and third examples, A in Bar[A] is completely different from A under forSome.
This ugly workaround should work:
// need type members, so can't use tuples
case class Pair[A, B](a: A, b: B) {
type T1 = A
type T2 = B
}
type PairedMap[P <: Pair[_, _]] = Map[P#T1, P#T2]
type FooBarPair[A] = Pair[Foo[A], Bar[A]]
val map: PairedMap[FooBarPair[_]] = ...
I want to use a map of varying types on an unknown A
So, you want a variant of Map[K,V] with following interface, correct?
trait DependentMap[K[_],V[_]] {
def add[A](key: K[A], value: V[A]): DependentMap[K,V]
def get[A](key: K[A]): Option[V[A]]
}
Whether it is or not might be a bit hard to tell from the type signature, so let's create a few dummy values and see if the type checker accepts what we want it to accept and rejects what we want it to reject.
// dummy definitions just to check that the types are correct
case class Foo[+A](a: A)
case class Bar[+A](a: A)
val myMap: DependentMap[Foo,Bar] = null
myMap.add(Foo( 42), Bar( 43)) // typechecks
myMap.add(Foo("foo"), Bar("bar")) // typechecks
myMap.add(Foo( 42), Bar("bar")) // type mismatch
val r1: Option[Bar[ Int]] = myMap.get(Foo( 42)) // typechecks
val r2: Option[Bar[String]] = myMap.get(Foo("foo")) // typechecks
val r3: Option[Bar[String]] = myMap.get(Foo( 42)) // type mismatch
So far so good. But look what happens once we start to play with inheritance:
val fooInt: Foo[Int] = Foo(42)
val fooAny: Foo[Any] = fooInt
val barStr: Bar[String] = Bar("bar")
val barAny: Bar[Any] = barStr
println(fooInt == fooAny) // true
myMap.add(fooAny, barAny).get(fooInt) // Bar("bar")?
Since fooInt and fooAny are the same value, we'd naïvely expect this get to succeed. According to the type signature of get, it would have to succeed with a value of type Bar[Int], but where would this value come from? The value we put in has the type Bar[Any] and also the type Bar[String], but certainly not the type Bar[Int]!
Here's another surprising case.
val fooListInt: Foo[List[Int]] = Foo(List[Int]())
val fooListStr: Foo[List[String]] = Foo(List[String]())
println(fooListInt == fooListStr) // true!
myMap.add(fooListInt, Bar(List(42))).get(fooListStr) // Bar(List(42))?
This time fooListInt and fooListStr look like they should be distinct, but in fact they both have the value Nil, of type List[Nothing], which is a subtype of both List[Int] and List[String]. So if we want to mimic the behaviour of Map[K,V] on such a key, get would again have to succeed, but it can't since we gave it a Bar[List[Int]] and it needs to produce a Bar[List[String]].
All this to say, our DependentMap should not consider keys to be equal unless the type A given to add is also equal to the type A given to get. Here is an implementation which uses type tags to ensure that this is the case.
import scala.reflect.runtime.universe._
class DependentMap[K[_],V[_]](
inner: Map[
(TypeTag[_], Any),
Any
] = Map()
) {
def add[A](
key: K[A], value: V[A]
)(
implicit tt: TypeTag[A]
): DependentMap[K,V] = {
val realKey: (TypeTag[_], Any) = (tt, key)
new DependentMap(inner + ((realKey, value)))
}
def get[A](key: K[A])(implicit tt: TypeTag[A]): Option[V[A]] = {
val realKey: (TypeTag[_], Any) = (tt, key)
inner.get(realKey).map(_.asInstanceOf[V[A]])
}
}
And here are a few examples demonstrating that it works as expected.
scala> val myMap: DependentMap[Foo,Bar] = new DependentMap
scala> myMap.add(Foo(42), Bar(43)).get(Foo(42))
res0: Option[Bar[Int]] = Some(Bar(43))
scala> myMap.add(Foo("foo"), Bar("bar")).get(Foo("foo"))
res1: Option[Bar[String]] = Some(Bar(bar))
scala> myMap.add(Foo(42), Bar("bar")).get(Foo(42))
error: type mismatch;
scala> myMap.add[Any](Foo(42), Bar("bar")).get(Foo(42))
res2: Option[Bar[Int]] = None
scala> myMap.add[Any](Foo(42), Bar("bar")).get[Any](Foo(42))
res3: Option[Bar[Any]] = Some(Bar(bar))
scala> myMap.add(Foo(List[Int]()), Bar(List(43))).get(Foo(List[Int]()))
res4: Option[Bar[List[Int]]] = Some(Bar(List(43)))
scala> myMap.add(Foo(List[Int]()), Bar(List(43))).get(Foo(List[String]()))
res5: Option[Bar[List[String]]] = None
This works, but the cast is ugly. I'd rather find a better way.
Then you're probably disappointed that my get is implemented using a cast. Let me try to convince you that there is no other way.
Instead of a map which may or may not contain our key, let's consider a much simpler case.
def safeCast[A,B](
value: A
)(
implicit tt1: TypeTag[A], tt2: TypeTag[B]
): Option[B] = {
if (tt1 == tt2)
Some(value.asInstanceOf[B])
else
None
}
Given a type tag for A and a type tag for B, if the two type tags are equal, then we know that A and B are the same type, and therefore the typecast is safe. But how could we possibly implement this without a typecast? The equality check returns a mere boolean, not a witness of equality like in some other languages. Therefore there is nothing which statically distinguishes the two branches of the if and the compiler cannot possibly know that a cast-free conversion is legal in the "true" branch but illegal in the other.
In the more complicated case of our DependentMap, we also have to compare our type tag with the one we stored when we did an add, and so we also have to use a cast.
I gather the answer is to use existential types with the forSome keyword
I can see why you'd think so. You want a bunch of associations from key to value, where each (key, value) pair has the type (Foo[A], Bar[A]) forSome {type A}. And indeed, if you were not concerned with performance, you could store those associations in a list:
val myList: List[(Foo[A], Bar[A]) forSome {type A}]
Since the forSome is inside the brackets, this allows each entry in the list to use a different A. And since Foo[A] and Bar[A] are both to the left of the forSome, in each entry the As must match.
In the case of Map, however, there is no place to put the forSome to obtain the result you want. The problem with Map is that it has two type parameters, which prevents us from from putting them both to the left of the forSome without putting the forSome outside of the brackets. It wouldn't make sense to do so: since the two type parameters are independent, there is nothing which links one occurrence of the left type parameter to a corresponding occurrence of the right type parameter, and so there is no way to know which As should match. Consider the following counter-example:
case class NotMap[K,V](k1: K, k2: K, v1: V, v2: V, v3: V)
There isn't the same number of K values as there are V values, so in particular there is no correspondence between the K values and the V values. If there was some special syntax like Map[{Foo[A], Bar[A]} forSome {type A}] which would allow us to specify that for each entry in the Map, the As should match, then it would also be possible to use that syntax to specify the nonsense type NotMap[{Foo[A], Bar[A]} forSome {type A}]. Therefore there is no such syntax, and we need to use a type other than Map, such as DependentMap or HMap.
What about something like
def map[A]: Map[Foo[A], Bar[A]] = ...
val myMap = map[Qux]
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = myMap(foo)
Or (inspired by Alexey Romanov's answer)
type MyMap[A] = Map[Foo[A],Bar[A]]
val map:MyMap[Qux] = ...
...
val foo = new Foo[Qux]
val bar: Bar[Qux] = map(foo)
I have done a few implementations of HList now. One based on Daniel Spiewak's High Wizardry in the Land of Scala talk and another based on a post in Apocalisp blog. The goal was to have a heterogenous list of which is not heterogenous in the primary type but rather the higher kind. For example:
val requests = Request[String] :: Request[Int] :: HNil
I would be able to do a map across the list to perform the request and result in a heterogenous list of the higher kind. So:
requests.map(execute)
should equal
String :: Int :: HNil
Sadly all my attempts have resulted in an HList of Any. Here is the code from a recent attempt:
class Request[+Out](o:Out) {
type O = Out
def v:O = o
}
object HList {
trait Func[-Elem,Out] {
type Apply[E <: Elem] <: Out
def apply[N <: Elem](e:N):Apply[N]
}
sealed trait HList[Base] {
type Head <: Base
type Tail <: HList[Base]
type Map[Out,F <: Func[Base,Out]] <: HList[Out]
def head:Head
def tail:Tail
def ::[A <: Base](a:A):HList[Base]
def map[Out,F <: Func[Base,Out]](f:F):Map[Out,F]
}
case class HNil[Base]() extends HList[Base] {
type Head = Nothing
type Tail = Nothing
type Map[Out,F <: Func[Base,Out]] = HNil[Out]
def head = error("Head of an empty HList")
def tail = error("Head of an empty HList")
def ::[A <: Base](a:A) = HCons(a,this)
def map[Out,F <: Func[Base,Out]](f:F) = new HNil[Out]
}
case class HCons[Base,A <: Base,B <: HList[Base]](head: A, tail: B) extends HList[Base] {
type Head = A
type Tail = B
type Map[Out,F <: Func[Base,Out]] = HCons[Out,F#Apply[Head],Tail#Map[Out,F]]
def ::[C <: Base](c:C) = HCons(c,this)
def map[Out,F <: Func[Base,Out]](f:F) =
HCons(f(head),tail.map(f))
}
val :: = HCons
}
object Test extends Application {
import HList._
val HNil = new HNil[Request[_]]
val list = new Request[Int](1) :: new Request[String]("1") :: HNil
val (a :: b :: HNil) = list
val y:Request[String] = b
val results = list.map[Any,Unwrap.type](Unwrap)
val i:Int = results.head
}
import HList._
object Unwrap extends Func[Request[Any],Any] {
type Apply[I <: Request[Any]] = I#O
def apply[N <: Request[Any]](e:N) = null.asInstanceOf[Apply[N]]
}
The other attempt was based on the Apocalisp version which uses fold to create a new HList and again it resulted in a HList of Any types. Any tips would be appreciated.
The HList implementation in shapeless is rich enough to subsume both HList and KList functionality. It provides a map operation which applies a higher-ranked function, possibly with type-specific cases, across it's elements yielding an appropriately typed HList result,
import shapeless.Poly._
import shapeless.HList._
// Define a higher-ranked function from Sets to Options
object choose extends (Set ~> Option) {
def default[T](s : Set[T]) = s.headOption
}
// An HList of Sets
val sets = Set(1) :: Set("foo") :: HNil
// Map our choose function across it ...
val opts = sets map choose
// The resulting value
opts == Option(1) :: Option("foo") :: HNil
Note that although it's the case in the above example there's no requirement that the HList elements share a common outer type constructor, it just has to be the case that the higher-ranked function mapped with has cases for all of the types involved,
// size is a higher-ranked function from values of arbitrary type to a 'size'
// which is defined as 1 by default but which has type specific cases for
// Strings and tuples
object size extends (Id ~> Const[Int]#λ) {
def default[T](t : T) = 1
}
implicit def sizeString = size.λ[String](s => s.length)
implicit def sizeTuple[T, U](implicit st : size.λ[T], su : size.λ[U]) =
size.λ[(T, U)](t => 1+size(t._1)+size(t._2))
size(23) == 1 // Default
size("foo") == 3 // Type specific case for Strings
size((23, "foo")) == 5 // Type specific case for tuples
Now let's map this across an HList,
val l = 23 :: true :: "foo" :: ("bar", "wibble") :: HNil
val ls = l map size
ls == 1 :: 1 :: 3 :: 10 :: HNil
In this case the result type of the function being mapped is constant: it's an Int no matter what the argument type is. Consequently the resulting HList has elements all of the same type, which means that it can usefully be converted to a vanilla list,
ls.toList == List(1, 1, 3, 10)
what you need is a Klist with type constructor Request, and a natural transformation execute: Request ~> Id. All of this is detailed in the marvelous type-level programming series of posts at Apocalisp, in particular:
Natural transformation literals
Klist basics
you can checkout the code for the whole series from Mark Harrah's up repo
In your case, you'll need something like
val reqList = new Request[Int](1) :^: new Request[String]("1") :^: KNil
val exec = new (Request ~> Id) { def apply[T](reqs: Request[T]): T = reqs.execute }
val results = reqList down exec
the down method above is conceptually the same as map for a nat transf M ~> Id; you also have more general map which from a nat transf M ~> N and a Klist of kind M yields a KList of kind N.
Note that you have an example of Map with HList in the recent (October 2016, 5 years after the OP) article "Using shapeless' HLists for extra type safety (in Akka Streams)" from Mikołaj Koziarkiewicz.
//glue for the ParserStageDefs
specs.map(s => Flow[Data].map(s.parser).map(s.processor))
.foreach(broadcast ~> _ ~> merge)
The problem lies in the fact that the type information in our specs list is not preserved. Or rather, not preserved the way we want to - the type of the List elements is ParserStageDef[_ >: Int with String], so the lowest common supertype for our decorator and incrementer.
The above implies that, when mapping between the parser and processor elements, the compiler has no way to provide the actual type T that's used within the given spec.
A solution
Here's where HLists come to the rescue. Because they preserve the complete type information for each element, it's possible to define our flow very similarly to our last attempt.
First, let's replace our list with an HList:
import shapeless.ops.hlist._
import shapeless._
//...
val specs = decorator :: incrementer :: HNil
val specsSize = specs.length.toInt
Now, for the mapping from ParserStageDefs into Flows, we need to take a different approach, as the map for HList requires something called P**oly - a polymorphic function value**.
Here's how one would look like in our case:
import shapeless.PolyDefns.~>
object toFlow extends (ParserStageDef ~> ProcessingFlow) {
override def apply[T](f: ParserStageDef[T]) =
Flow[Data].map(f.parser).map(f.processor)
}
For it to work, we'll also have change ProcessingFlow to type ProcessingFlow[_] = Flow[Data, Data, _], since the polymorphic function above expects a higher-kinded type.
Now, our central statement turns out to be:
//we convert to a List[ProcessingFlow[_]] for simplicity
specs.map(toFlow).toList.foreach(broadcast ~> _ ~> merge)
and we're all set!