Context
I use scala 2.11.6 currently, possibly 2.11.7 in the future.
Given compiled class files in the classpath, I want to do 2 things:
Find the name of any objects that implements a certain interface:
trait Service
trait ServiceFactory {
def create(): Service
}
...
package my.package
object MyServiceFactory extends ServiceFactory {
def create(): Service = new Service()
}
Here the name would be something like my.package.MyServiceFactory as it implements the ServiceFactory trait.
Given the fully qualified name of the object I want to get the reference to the object's instance.
val factory = getInstance[ServiceFactory]("my.package.MyServiceFactory")
val service = factory.create()
Problem
The problem to both scenarios is verifying the type inheritance and making sure it is a singleton object.
Checking the class seems straight-forward but given all the documentation
I could understand, none helped me implementing something like isSingletonObject(name: String): Boolean as used in:
import scala.reflect.runtime.{universe => ru}
val rm = ru.runtimeMirror(classLoader)
def getInstance[T](name: String)(implicit tt: ru.TypeTag[T]): T = {
if (!isSingletonObject(name)) throw new RuntimeException(
s"$name does not specify a singleton object")
val moduleSym = try rm.staticModule(name).asModule
if (!(moduleSym.moduleClass.asClass.selfType <:< tt.tpe))
throw new RuntimeException("Type of loaded module " + moduleSym.fullName
+ " does not satisfy subtype relationship with "
+ tt.tpe.typeSymbol.fullName)
val mm = rm.reflectModule(moduleSym.asModule)
mm.instance.asInstanceOf[T]
}
How can one find objects and verify that a given name is really an object?
Alternative approaches to the given scenarios are also welcome.
For the first question, you could use ClassUtil library. Note that it'll find Java classes, and the ones which correspond to objects will have names ending in $class. See also Scala Reflection - Loading or finding classes based on trait.
For the second, objects are called "modules" in Scala reflection, so you don't need isSingletonObject(name); if it isn't, rm.staticModule(name).asModule in your code will fail. There doesn't seem to be a way to check if it's a compiler-generated empty companion object (isSynthetic returns false), but they will be ruled out by the subtyping check anyway (so will the static parts of Java classes, but you can also filter those out using isJava).
Related
While learning Scala, I came across interesting concept of companion object. Companion object can used to define static methods in Scala. Need few clarifications in the below Spark Scala code in regard of companion object.
class BballStatCounter extends Serializable {
val stats: StatCounter = new StatCounter()
var missing: Long = 0
def add(x: Double): BballStatCounter = {
if (x.isNaN) {
missing += 1
} else {
stats.merge(x)
}
this
}
}
object BballStatCounter extends Serializable {
def apply(x: Double) = new BballStatCounter().add(x)
}
Above code is invoked using val stat3 = stats1.map(b=>BballStatCounter(b)).
What is nature of variables stats and missing declared in the
class? Is it similar to class attributes of Python?
What is the significance of apply method in here?
Here stats and missing are class attributes and each instance of BballStatCounter will have their own copy of them just like in Python.
In Scala the method apply serves a special purpose, if any object has a method apply and if that object is used as function calling notation like Obj() then the compiler replaces that with its apply method calling, like Obj.apply() .
The apply method is generally used as a constructor in a Class Companion object.
All the collection Classes in Scala has a Companion Object with apply method, thus you are able to create a list like : List(1,2,3,4)
Thus in your above code BballStatCounter(b) will get compiled to BballStatCounter.apply(b)
stats and missing are members of the class BcStatCounter. stats is a val so it cannot be changed once it has been defined. missing is a var so it is more like a traditional variable and can be updated, as it is in the add method. Every instance of BcStatCounter will have these members. (Unlike Python, you can't add or remove members from a Scala object)
The apply method is a shortcut that makes objects look like functions. If you have an object x with an apply method, you write x(...) and the compiler will automatically convert this to x.apply(...). In this case it means that you can call BballStatCounter(1.0) and this will call the apply method on the BballStatCounter object.
Neither of these questions is really about companion objects, this is just the normal Scala class framework.
Please note the remarks in the comments about asking multiple questions.
In Java, while type arguments are erased in runtime, it is possible to find the actual type arguments passed to a superclass:
class Derived extends Base<String> {
// ...
}
ParameterizedType type = (ParameterizedType)Derived.class.getGenericSuperclass();
Type[] args = type.getActualTypeArguments(); // gives {String.class}
While I can use the same Java reflection to Scala class, It does not catch Scala's value types:
class Base[T]
class Derived extends Base[Int]
classOf[Derived]
.getGenericSuperclass
.asInstanceOf[ParameterizedType]
.getActualTypeArguments // gives {Object.class}, not {int.class}
Is it possible to determine the value type used when extending from a generic superclass? I am loading classes from a jar file so it'd be best to achieve this only using a java.lang.Class instance.
In Java reflection you won't be able to obtain Int and other AnyVal types because they are handled specially by the compiler and if they are used generically, they will be represented by Object. However, you can use Scala reflection, and it is wholly possible to go from Java reflection to Scala reflection. Here's how:
import scala.reflect.runtime.universe._
class Base[T]
class Derived extends Base[Int]
object Main extends App {
val rm = runtimeMirror(getClass.getClassLoader) // whatever class loader you're using
val derivedSym = rm.staticClass(classOf[Derived].getName)
val baseSym = rm.staticClass(classOf[Base[_]].getName)
val TypeRef(_, _, params) = derivedSym.typeSignature.baseType(baseSym)
println(s"$derivedSym extends $baseSym[${params.mkString(", ")}]")
}
Unfortunately, unless you know exactly what you are searching for, you will have hard time finding proper documentation. I have found the answer on scala-users mailing list. Scala reflection is still experimental and, AFAIK, it will probably be superseded by a better one in future Scala versions.
This is a follow up to the following question, which concerned serialization: How best to keep a cached list of member fields, one each for a family of case classes in Scala
I'm trying to generically support deserialization in the same way. One straightforward attempt is the following:
abstract class Serializer[T](implicit ctag: ClassTag[T]) {
private val fields = ctag.runtimeClass.getDeclaredFields.toList
fields foreach { _.setAccessible(true) }
implicit class AddSerializeMethod(obj: T) {
def serialize = fields.map(f => (f.getName, f.get(obj)))
}
def deserialize(data: List[(String, Any)]): T = {
val m = data toMap
val r: T = ctag.runtimeClass.newInstance // ???
fields.foreach { case f => f.set(r, m(f.getName)) }
r;
}
}
There are a couple of issues with the code:
The line with val r: T = ... has a compile error because the compiler thinks it's not guaranteed to have the right type. (I'm generally unsure of how to create a new instance of a generic class in a typesafe way -- not sure why this isn't safe since the instance of Serializer is created with a class tag whose type is checked by the compiler).
The objects I'm creating are expected to be immutable case class objects, which are guaranteed to be fully constructed if created in the usual way. However, since I'm mutating the fields of instances of these objects in the deserialize method, how can I be sure that the objects will not be seen as partially constructed (due to caching and instruction reordering) if they are published to other threads?
ClassTag's runtimeClass method returns Class[_], not Class[T], probably due to the fact generics in Scala and Java behave differently; you can try casting it forcefully: val r: T = ctag.runtimeClass.newInstance.asInstanceOf[T]
newInstance calls the default, parameterless constructor. If the class doesn't have one, newInstance will throw InstantiationException. There's no way around it, except for:
looking around for other constructors
writing custom serializers (see how Gson does that; BTW Gson can automatically serialize only classes with parameterless constructors and those classes it has predefined deserializers for)
for case classes, finding their companion object and calling its apply method
Anyhow, reflection allows for modifying final fields as well, so if you manage to create an immutable object, you'll be able to set its fields.
Scala 2.10 comes with a great reflection API. There are two entry points to it, however: runtime universe and macro context universe.
When using runtime reflection, you should import scala.reflect.runtime.universe. When using reflection inside a macro implementation, you should import universe from the context.
Is it possible to write some code that works in both environments? How should one obtain the universe?
Consider this example:
class MyReflection(val u: scala.reflect.api.Universe) {
import u._
def foo[T: TypeTag] = implicitly[TypeTag[T]].tpe.members // returns MyReflection.u.MemberScope
}
val x = new MyReflection(scala.reflect.runtime.universe)
val members: scala.reflect.runtime.universe.MemberScope = x.foo[String] // BANG! Compiler error
This won't compile because of type mismatch. Same time, it is obvious that both scala.reflect.runtime.universe.MemberScope and MyReflection.u.MemberScope in this example share the same API. Is there a way to abstract over different universes?
Or am I possibly doing something philosophically wrong with trying to export reflection artifacts (MemberScope in this example)?
You can just accept the universe as a parameter:
class MyReflection(val u: scala.reflect.api.Universe) {
import u._
def foo[T : TypeTag] = implicitly[TypeTag[T]].tpe.members
}
val x = new MyReflection(scala.reflect.runtime.universe)
Note that you'll have to refer to the universe via your instance of MyReflection to get the path-dependent types right.
val members: x.u.MemberScope = x.foo[String]
Have a look at this question for more examples and options.
There seems to be a lot of enthusiasm among Scala bloggers lately for the type classes pattern, in which a simple class has functionality added to it by an additional class conforming to some trait or pattern. As a vastly oversimplified example, the simple class:
case class Wotsit (value: Int)
can be adapted to the Foo trait:
trait Foo[T] {
def write (t: T): Unit
}
with the help of this type class:
implicit object WotsitIsFoo extends Foo[Wotsit] {
def write (wotsit: Wotsit) = println(wotsit.value)
}
The type class is typically captured at compile time with implicts, allowing both the Wotsit and its type class to be passed together into a higher order function:
def writeAll[T] (items: List[T])(implicit tc: Foo[T]) =
items.foreach(w => tc.write(w))
writeAll(wotsits)
(before you correct me, I said it was an oversimplified example)
However, the use of implicits assumes that the precise type of the items is known at compile time. I find in my code this often isn't the case: I will have a list of some type of item List[T], and need to discover the correct type class to work on them.
The suggested approach of Scala would appear to be to add the typeclass argument at all points in the call hierarchy. This can get annoying as an the code scales and these dependencies need to be passed down increasingly long chains, through methods to which they are increasingly irrelevant. This makes the code cluttered and harder to maintain, the opposite of what Scala is for.
Typically this is where dependency injection would step in, using a library to supply the desired object at the point it's needed. Details vary with the library chosen for DI - I've written my own in Java in the past - but typically the point of injection needs to define precisely the object desired.
Trouble is, in the case of a type class the precise value isn't known at compile time. It must be selected based on a polymorphic description. And crucially, the type information has been erased by the compiler. Manifests are Scala's solution to type erasure, but it's far from clear to me how to use them to address this issue.
What techniques and dependency injection libraries for Scala would people suggest as a way of tackling this? Am I missing a trick? The perfect DI library? Or is this really the sticking point it seems?
Clarification
I think there are really two aspects to this. In the first case, the point where the type class is needed is reached by direct function calls from the point where the exact type of its operand is known, and so sufficient type wrangling and syntactic sugar can allow the type class to be passed to the point it's needed.
In the second case, the two points are separated by a barrier - such as an API that can't be altered, or being stored in a database or object store, or serialised and send to another computer - that means the type class can't be passed along with its operand. In this case, given an object whose type and value are known only at runtime, the type class needs somehow to be discovered.
I think functional programmers have a habit of assuming the first case - that with a sufficiently advanced language, the type of the operand will always be knowable. David and mkniessl provided good answers for this, and I certainly don't want to criticise those. But the second case definitely does exist, and that's why I brought dependency injection into the question.
A fair amount of the tediousness of passing down those implicit dependencies can be alleviated by using the new context bound syntax. Your example becomes
def writeAll[T:Foo] (items: List[T]) =
items.foreach(w => implicitly[Foo[T]].write(w))
which compiles identically but makes for nice and clear signatures and has fewer "noise" variables floating around.
Not a great answer, but the alternatives probably involve reflection, and I don't know of any library that will just make this automatically work.
(I have substituted the names in the question, they did not help me think about the problem)
I'll attack the problem in two steps. First I show how nested scopes avoid having to declare the type class parameter all the way down its usage. Then I'll show a variant, where the type class instance is "dependency injected".
Type class instance as class parameter
To avoid having to declare the type class instance as implicit parameter in all intermediate calls, you can declare the type class instance in a class defining a scope where the specific type class instance should be available. I'm using the shortcut syntax ("context bound") for the definition of the class parameter.
object TypeClassDI1 {
// The type class
trait ATypeClass[T] {
def typeClassMethod(t: T): Unit
}
// Some data type
case class Something (value: Int)
// The type class instance as implicit
implicit object SomethingInstance extends ATypeClass[Something] {
def typeClassMethod(s: Something): Unit =
println("SomthingInstance " + s.value)
}
// A method directly using the type class
def writeAll[T:ATypeClass](items: List[T]) =
items.foreach(w => implicitly[ATypeClass[T]].typeClassMethod(w))
// A class defining a scope with a type class instance known to be available
class ATypeClassUser[T:ATypeClass] {
// bar only indirectly uses the type class via writeAll
// and does not declare an implicit parameter for it.
def bar(items: List[T]) {
// (here the evidence class parameter defined
// with the context bound is used for writeAll)
writeAll(items)
}
}
def main(args: Array[String]) {
val aTypeClassUser = new ATypeClassUser[Something]
aTypeClassUser.bar(List(Something(42), Something(4711)))
}
}
Type class instance as writable field (setter injection)
A variant of the above which would be usable using setter injection. This time the type class instance is passed via a setter call to the bean using the type class.
object TypeClassDI2 {
// The type class
trait ATypeClass[T] {
def typeClassMethod(t: T): Unit
}
// Some data type
case class Something (value: Int)
// The type class instance (not implicit here)
object SomethingInstance extends ATypeClass[Something] {
def typeClassMethod(s: Something): Unit =
println("SomthingInstance " + s.value)
}
// A method directly using the type class
def writeAll[T:ATypeClass](items: List[T]) =
items.foreach(w => implicitly[ATypeClass[T]].typeClassMethod(w))
// A "service bean" class defining a scope with a type class instance.
// Setter based injection style for simplicity.
class ATypeClassBean[T] {
implicit var aTypeClassInstance: ATypeClass[T] = _
// bar only indirectly uses the type class via writeAll
// and does not declare an implicit parameter for it.
def bar(items: List[T]) {
// (here the implicit var is used for writeAll)
writeAll(items)
}
}
def main(args: Array[String]) {
val aTypeClassBean = new ATypeClassBean[Something]()
// "inject" the type class instance
aTypeClassBean.aTypeClassInstance = SomethingInstance
aTypeClassBean.bar(List(Something(42), Something(4711)))
}
}
Note that the second solution has the common flaw of setter based injection that you can forget to set the dependency and get a nice NullPointerException upon use...
The argument against type classes as dependency injection here is that with type classes the "precise type of the items is known at compile time" whereas with dependency injection, they are not. You might be interested in this Scala project rewrite effort where I moved from the cake pattern to type classes for dependency injection. Take a look at this file where the implicit declarations are made. Notice how the use of environment variables determines the precise type? That is how you can reconcile the compile time requirements of type classes with the run time needs of dependency injection.