I'm using Scala Pickling, an automatic serialization framework for Scala.
According to the author's slides, any type T can be pickled as long as there is an implicit Pickler[T] in scope.
Here, I'm assuming she means scala.tools.nsc.io.Pickler.
However, the following does not compile:
import scala.pickling._
import scala.pickling.binary._
import scala.tools.nsc.io.Pickler
object Foo {
def bar[T: Pickler](t: T) = t.pickle
}
The error is:
[error] exception during macro expansion:
[error] scala.ScalaReflectionException: type T is not a class
[error] at scala.reflect.api.Symbols$SymbolApi$class.asClass(Symbols.scala:323)
[error] at scala.reflect.internal.Symbols$SymbolContextApiImpl.asClass(Symbols.scala:73)
[error] at scala.pickling.PickleMacros$class.pickleInto(Macros.scala:381)
[error] at scala.pickling.Compat$$anon$17.pickleInto(Compat.scala:33)
[error] at scala.pickling.Compat$.PickleMacros_pickleInto(Compat.scala:34)
I'm using Scala 2.10.2 with scala-pickling 0.8-SNAPSHOT.
Is this a bug or user error?
EDIT 1: The same error arises with both scala.pickling.SPickler and scala.pickling.DPickler.
EDIT 2: It looks like this is a bug: https://github.com/scala/pickling/issues/31
Yep, as Andy pointed out:
you need either a scala.pickling.SPickler or a scala.pickling.DPickler (static and dynamic, respectively) in order to pickle a particular type.
Those both already come in the scala.pickling package, so it's enough to just use them in your generic method signature.
You're absolutely correct that you can add an SPickler context-bound to your generic method. The only additional thing which you need (admittedly it's a bit ugly, and we're thinking about removing it) is to add a FastTypeTag context bound as well. (This is necessary for the pickling framework to know what type it's trying to pickle, as it handles primitives differently, for example.)
This is what you'd need to do to provide generic pickling/unpickling methods:
Note that for the unbar method, you need to provide an Unpickler context-bound rather than a SPickler context-bound.
import scala.pickling._
import binary._
object Foo {
def bar[T: SPickler: FastTypeTag](t: T) = t.pickle
def unbar[T: Unpickler: FastTypeTag](bytes: Array[Byte]) = bytes.unpickle[T]
}
Testing this in the REPL, you get:
scala> Foo.bar(42)
res0: scala.pickling.binary.BinaryPickle =
BinaryPickle([0,0,0,9,115,99,97,108,97,46,73,110,116,0,0,0,42])
scala> Foo.unbar[Int](res0.value)
res1: Int = 42
Looking at the project, it seems you need either an scala.pickling.SPickler or a scala.pickling.DPickler (static and dynamic, respectively) in order to pickle a particular type.
The pickle methods are macros. I suspect that if you pickle with an SPickler, the macro will require the compile time type of your class to be known.
Thus, you may need to do something similar to:
object Foo {
def bar(t: SomeClass1) = t.pickle
def bar(t: SomeClass2) = t.pickle
def bar(t: SomeClass3) = t.pickle
// etc
}
Alternatively, a DPickler may do the trick. I suspect that you'll still have to write some custom pickling logic for your specific types.
Related
I wonder if it is possible to do something like the following:
import scala.reflect.runtime.universe._
class Bar[T]
def foo[T]()(implicit ctag: reflect.ClassTag[T]) {
val clazz = classOf[Bar[ctag.runtimeClass.asInstanceOf[Class[T]]]]
}
Here the Scala compiler complains:
error: stable identifier required, but ctag.runtimeClass found.
Is there a way to get the class type with type parameters inserted from the runtime type information available in the function?
Is there a way to get the class type with type parameters inserted from the runtime type information available in the function?
classOf[Bar[T]] works for a very simple reason: it doesn't insert any runtime information! classOf[Bar[T]], classOf[Bar[String]], classOf[Bar[Int]], classOf[Bar[_]] are all the same; that's what type erasure means in JVM context (to avoid misleading, I prefer always using classOf[Bar[_]] where possible). Note that there is actually a single exception: if Bar is Array, because classOf[Array[Int]] and classOf[Array[Object]] (e.g.) are different!
classOf[T] obviously would need runtime information and so it doesn't work.
Thanks to the comment by #Mr. V I realized that its actually easier than I initially thought:
import scala.reflect.runtime.universe._
class Bar[T]
def foo[T]() {
val clazz = classOf[Bar[T]]
}
Suppose I have:
class X
{
val listPrimitive: List[Int] = null
val listX: List[X] = null
}
and I print out the return types of each method in Scala as follows:
classOf[ComplexType].getMethods().foreach { m => println(s"${m.getName}: ${m.getGenericReturnType()}") }
listPrimitive: scala.collection.immutable.List<Object>
listX: scala.collection.immutable.List<X>
So... I can determine that the listX's element type is X, but is there any way to determine via reflection that listPrimitive's element type is actually java.lang.Integer? ...
val list:List[Int] = List[Int](123);
val listErased:List[_] = list;
println(s"${listErased(0).getClass()}") // java.lang.Integer
NB. This seems not to be an issue due to JVM type erasure since I can find the types parameter of List. It looks like the scala compiler throws away this type information IFF the parameter type is java.lang.[numbers] .
UPDATE:
I suspect this type information is available, due to the following experiment. Suppose I define:
class TestX{
def f(x:X):Unit = {
val floats:List[Float] = x.listPrimitive() // type mismatch error
}
}
and X.class is imported via a jar. The full type information must be available in X.class in order that this case correctly fails to compile.
UPDATE2:
Imagine you're writing a scala extension to a Java serialization library. You need to implement a:
def getSerializer(clz:Class[_]):Serializer
function that needs to do different things depending on whether:
clz==List[Int] (or equivalently: List[java.lang.Integer])
clz==List[Float] (or equivalently: List[java.lang.Float])
clz==List[MyClass]
My problem is that I will only ever see:
clz==List[Object]
clz==List[Object]
clz==List[MyClass]
because clz is provided to this function as clz.getMethods()(i).getGenericReturnType().
Starting with clz:Class[_] how can I recover the element type information that was lost?
Its not clear to me that TypeToken will help me because its usages:
typeTag[T]
requires that I provide T (ie. at compile time).
So, one path to a solution... Given some clz:Class[_], can I determine the TypeTokens of its method's return types? Clearly this is possible as this information must be contained (somewhere) in a .class file for a scala compiler to correctly generate type mismatch errors (see above).
At the java bytecode level Ints have to be represented as something else (apparently Object) because a List can only contain objects, not primitives. So that's what java-level reflection can tell you. But the scala type information is, as you infer, present (at the bytecode level it's in an annotation, IIRC), so you should be able to inspect it with scala reflection:
import scala.reflect.runtime.universe._
val list:List[Int] = List[Int](123)
def printTypeOf[A: TypeTag](a: A) = println(typeOf[A])
printTypeOf(list)
Response to update2: you should use scala reflection to obtain a mirror, not the Class[_] object. You can go via the class name if need be:
import scala.reflect.runtime.universe._
val rm = runtimeMirror(getClass.getClassLoader)
val someClass: Class[_] = ...
val scalaMirrorOfClass = rm.staticClass(someClass.getName)
// or possibly rm.reflectClass(someClass) ?
val someObject: Any = ...
val scalaMirrorOfObject = rm.reflectClass(someObject)
I guess if you really only have the class, you could create a classloader that only loads that class? I can't imagine a use case where you wouldn't have the class, or even a value, though.
I'd like to experiment with the use of a dynamic data model with a reflective library that uses typeOf[].
I've defined a class at runtime with a Scala reflection ToolBox in 2.11:
import scala.tools.reflect.ToolBox
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{ currentMirror => cm }
def cdef() = q"case class C(v: String)"
val tb = cm.mkToolBox()
val csym = tb.define(cdef())
def newc(csym: Symbol) = q"""new ${csym}("hi")"""
val obj = tb.eval(newc(csym))
I'm able to circumvent the typeOf[] call by entering Scala reflection via the ClassSymbol instead, but that requires modifying a library over which I have no immediate control.
Is there any way that I can use it as a type parameter in a library whose entry point is typeOf[]?
I've tried:
The only way I found to go from a value to something that I could use in the type position was use Java reflection to invoke the companion class' apply method and call .type on the result:
val method_apply = obj.getClass.getMethod("apply", "".getClass)
val typeTemplate = method_apply.invoke(obj, "hello")
type MyType = typeTemplate.type
(Keeping with the naming scheme of #xeno_by and #travisbrown 's menagerie of odd types, I might call this "Frankenstein's Type", because it is made from parts, given life at the wrong time, not quite a substitute for the original, and given that this is all happening at runtime, should probably be burned with fire.)
This type alias works as a type parameter is some cases. But in the case of typeOf[MyType], the the compiler makes a TypeTag before the runtime type is defined, so typeOf[MyType] returns a type member that doesn't correspond to the runtime type/class (e.g. TypeTag[package.Example.MyType] instead of TypeTag[package.C])
Should I expect the ToolBox to have generated a TypeTag, and if so, how do I use it?
If I have to make a TypeTag at runtime, this question shows me how, but then how do I attach it to whatever I use as a type parameter?
Thanks for any ideas,
-Julian
Let's consider this code:
class A
object A{
implicit def A2Int(implicit a:A)=1
implicit def A2String(a:A)="Hello"
}
object Run extends App{
implicit val a: A =new A
import A.A2Int
// without this import this code does not compile, why ?
// why is no import needed for A2String then ?
def iWantInt(implicit i:Int)=println(i)
def iWantString(implicit s:String)=println(s)
iWantInt
iWantString(a)
}
It runs and prints:
1
Hello
Now, if we comment out the line
import A.A2Int
then we get a compilation error:
With the line commented out, why Scala cannot find A.A2String if it can find A.A2Int ?
How can this problem be fixed ?
Thanks for reading.
The difference is that when you do iWantString(a), the compiler gets some starting point to work from: you are explicitly passing a, which the compiler knows is of type A.
Given that iWantString takes a String and not an A, the compiler will search for an implicit conversion from A to String in order to insert it and make the call succeed.
Implicit lookup rules state that the compiler must look (among other places) in the companion object of class A because type A is the source type of the conversion.
This is where it finds the implicit conversion A2String.
What you must get from this is that it is only because you passed an instance of A that the compiler knew to look for implicit conversions into the companion object of A.
When you just do iWantInt, the compiler has no reason to look into A, so it won't find your method A2Int (and as no other method/value in scope provides an implicit value of type Int, compilation then fails).
For more information about implicit lookup rules, see the
See the scala specification at http://www.scala-lang.org/docu/files/ScalaReference.pdf (chapter 7.2). Here's the most relevant excerpt:
The implicit scope of a type T consists of all companion modules (§5.4) of classes
that are associated with the implicit parameter’s type.
In my app, I am keeping track of the number of credits the user has. To add some type checking, I'm using a Credits class similar to this one:
case class Credits(val numCredits: Int) extends Ordered[Credits] {
...
}
Suppose I have a function def accept(creds: Credits): Unit that I want to call. Would there be a way for me to call it with
process(Credits(100))
process(0)
but not with this?
process(10)
I.e., I'd like to provide an implicit conversion only from the literal 0 and none other. Right now, I just have val Zero = Credits(0) in the companion object and I think that's rather good practice, but I'd be interested in an answer anyway, including other comments, like:
could this be done with macro implicit conversions in 2.10?
should Credits rather extend AnyVal and not be a case class in 2.10?
This kind of compile-time checking are the good terrain to use macros, which will be available in 2.10
A very smart guy named Jason Zaugg has already implemented something similar to what you need, but it applies to regex: Regex Compile Time checking.
You might want to look to its Macrocosm to see how it is done and how you could code your own macros with the same purpose.
https://github.com/retronym/macrocosm
If you really want to know more about Macros, firstly I would say that you need to be brave because the documentation is scarce for now and the API is likely to change. Jason Zaugg works compiles fine with 2.10-M3 but I am not sure it will works with the newer version.
If you want to start with some readings:
A good entry point is the scalamacros website http://scalamacros.org/ and the SIP document https://docs.google.com/document/d/1O879Iz-567FzVb8kw6N5OBpei9dnbW0ZaT7-XNSa6Cs/edit?pli=1
If you have time, you might also wants to read Eugene Burmako presentation:
http://scalamacros.org/talks/2012-04-28-MetaprogrammingInScala210.pdf
Now, getting to the topic, Scala macros are CATs : "Compile-time AST Transformations".
The abstract syntax tree is the way the compiler represents your source code. The compiler applies consequent transformations to the AST and at the last step it actual generates the java bytecode.
Let's now look to Jason Zaugg code:
def regex(s: String): scala.util.matching.Regex = macro regexImpl
def regexImpl(c: Context)(s: c.Expr[String]): c.Expr[scala.util.matching.Regex] = {
import c.universe._
s.tree match {
case Literal(Constant(string: String)) =>
string.r // just to check
c.reify(s.splice.r)
}
}
As you seen regex is a special function which takes a String and returns a Regex, by calling macro regexImpl
A macro function receives a context in the first parameter lists, and in second argument list the parameters of the macro under the form of c.Expr[A] and returns a c.Expr[B]. Please note that c.Expr is a path dependent type, i.e. it is a class defined inside the Context, so that if you have two context the following is illegal
val c1: context1.Expr[String] = ...
val c2: context2.Expr[String] = ...
val c3: context1.Expr[String] = context2.Expr[String] // illegal , compile error
Now if you look what happens in the code:
There is a match block which matches on s.tree
If s.tree is a Literal, containing a constant String , string.r is called
What's going on here is that there is an implicit conversion from string to StringOps defined in Predef.scala, which is automatically imported in the compilation every scala source
implicit def augmentString(x: String): StringOps = new StringOps(x)
StringOps extends scala.collection.immutable.StringLike, which contains:
def r: Regex = new Regex(toString)
Since macros are executed at compile time, this will be executed at compile time, and compilation will fail if an exception will be thrown (that is the behaviour of creating a regex from an invalid regex string)
Note: unluckily the API is very unstable, if you look at http://scalamacros.org/documentation/reference.html you will see a broken link towards the Context.scala. The right link is https://github.com/scala/scala/blob/2.10.x/src/reflect/scala/reflect/makro/Context.scala
Basically, you want dependent types. Why Scala supports a limited form of dependent types in path dependent types, it can't do what you ask.
Edmondo had a great idea in suggesting macros, but it has some limitations. Since it was pretty easy, I implemented it:
case class Credits(numCredits: Int)
object Credits {
implicit def toCredits(n: Int): Credits = macro toCreditsImpl
import scala.reflect.makro.Context
def toCreditsImpl(c: Context)(n: c.Expr[Int]): c.Expr[Credits] = {
import c.universe._
n.tree match {
case arg # Literal(Constant(0)) =>
c.Expr(Apply(Select(Ident("Credits"), newTermName("apply")),
List(arg)))
case _ => c.abort(c.enclosingPosition, "Expected Credits or 0")
}
}
}
Then I started up REPL, defined accept, and went through a basic demonstration:
scala> def accept(creds: Credits) { println(creds) }
accept: (creds: Credits)Unit
scala> accept(Credits(100))
Credits(100)
scala> accept(0)
Credits(0)
scala> accept(1)
<console>:9: error: Expected Credits or 0
accept(1)
^
Now, to the problem:
scala> val x = 0
x: Int = 0
scala> accept(x)
<console>:10: error: Expected Credits or 0
accept(x)
^
In other words, I can't track properties of the value assigned to identifiers, which is what dependent types would allow me to do.
But the whole strikes me as wasteful. Why do you want just 0 to be converted? It seems you want a default value, in which case the easiest solution is to use a default value:
scala> def accept(creds: Credits = Credits(0)) { println(creds) }
accept: (creds: Credits)Unit
scala> accept(Credits(100))
Credits(100)
scala> accept()
Credits(0)
Use could use an implicit partial function:
scala> case class Credits(val numCredits: Int)
defined class Credits
scala> def process(c: Credits) = {}
process: (c: Credits)Unit
scala> implicit def i2c:PartialFunction[Int, Credits] = { case 0 => Credits(0) }
i2c: PartialFunction[Int,Credits]
Allows you
scala> process(Credits(12))
and
scala> process(0)
But:
scala> process(12)
scala.MatchError: 12 (of class java.lang.Integer)
at $anonfun$i2c$1.apply(<console>:9)
at $anonfun$i2c$1.apply(<console>:9)
at .<init>(<console>:12)
at .<clinit>(<console>)
at .<init>(<console>:11)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:704)
at scala.tools.nsc.interpreter.IMain$Request$$anonfun$14.apply(IMain.sca
la:920)
at scala.tools.nsc.interpreter.Line$$anonfun$1.apply$mcV$sp(Line.scala:4
3)
at scala.tools.nsc.io.package$$anon$2.run(package.scala:25)
at java.lang.Thread.run(Unknown Source)
Edit: But yes, the compiler will still allow process(12) resulting in a match error at runtime.