Can't even try to load a class in Scala - scala

This is one of those times where you go, What do you mean, it doesn't compile?
That's not a rhetorical question: What's the shortest or idiomatickest fix? For bonus points, why is it necessary?
scala> import scala.util.Try
import scala.util.Try
scala> Try { getClass.getClassLoader loadClass "scala.util.Try" }
I hope this doesn't give the game away, but here's the message:
<console>:9: error: type mismatch;
found : Class[_]
required: Class[?0(in value res0)] where type ?0(in value res0)
Note: Any >: ?0, but Java-defined class Class is invariant in type T.
You may wish to investigate a wildcard type such as `_ >: ?0`. (SLS 3.2.10)
Try { getClass.getClassLoader loadClass "scala.util.Try" }
By "investigate", do they mean like basic research, or just apply techniques already available in the literature?
I'm still waiting for that error message that concludes, "Left as an exercise for the reader."
Update:
This is an exercise for Scala 2.10.
As usual, all good things come to those that wait:
apm#mara:~/tmp$ skala
Welcome to Scala version 2.11.0-20130622-103744-990c2b024a (OpenJDK 64-Bit Server VM, Java 1.7.0_21).
Type in expressions to have them evaluated.
Type :help for more information.
scala> import scala.language.existentials
import scala.language.existentials
scala> import scala.util.Try
import scala.util.Try
scala> Try { getClass.getClassLoader loadClass "scala.util.Try" }
res0: scala.util.Try[Class[?0]] forSome { type ?0 } = Success(class scala.util.Try)

This must surely be a duplicate question. Maybe someone can point it out, or exactly how type inference is not doing what comes naturally here.
Someone left an answer (which seems to have disappeared?) with a helpful link to MacIver on existential types. Possibly, I also need help from MacGyver.
Here are some variants I tried on the way to the forum.
package classy
import scala.util._
class Foo
object Test {
/* DNC
def loadTry(n: String, loader: ClassLoader) = Try { loader loadClass n }
def loadTry(n: String, loader: ClassLoader): Try[Class[_]] = Try { loader loadClass n }
*/
def main(args: Array[String]) {
val cl = getClass.getClassLoader
println(loadTry("classy.Foo", cl))
println(loadTry("classy.Bar", cl))
println(cl loadClass "classy.Foo")
println(loadOpt("classy.Foo", cl))
println(loadTryAgain("classy.Foo", cl))
println(loadTryYetAgain("classy.Foo", cl))
}
def loadOpt(n: String, loader: ClassLoader): Option[Class[_]] =
try Some(loader loadClass n) catch {
case _: Exception => None
}
def loadTryAgain(n: String, loader: ClassLoader): Try[Class[_]] = {
val res: Option[Class[_]] = try Some(loader loadClass n) catch {
case _: Exception => None
}
res match {
case None =>
Failure(new RuntimeException(s"Warning: class not found: ${n})"))
case Some(x) =>
Success(x)
}
}
def loadTryYetAgain(n: String, loader: ClassLoader): Try[Class[_]] = {
val res = try loader loadClass n catch {
case _: Exception => null
}
res match {
case null =>
Failure(new RuntimeException(s"Warning: class not found: ${n})"))
case x =>
Success(x)
}
}
def loadTry(n: String, loader: ClassLoader) =
Try[Class[_]] {
loader loadClass n
} recoverWith {
case e: Exception =>
Failure(new RuntimeException(s"Warning: class not found: ${n} (${e.getMessage})"))
}
}

the Try is causing it. for me in scala 2.10.0:
scala> import scala.util.Try
scala> val typeName = "scala.util.Try"
error:
scala> Try(Class.forName(typeName))
<console>:10: error: type mismatch;
found : Class[_]
required: Class[?0(in value res1)] where type ?0(in value res1)
Note: Any >: ?0, but Java-defined class Class is invariant in type T.
You may wish to investigate a wildcard type such as `_ >: ?0`. (SLS 3.2.10)
Try(Class.forName(typeName))
^
no error:
scala> Try[Class[_]](Class.forName(typeName))
res2: scala.util.Try[Class[_]] = Success(class scala.util.Try)
catching also has the same problem:
scala> import scala.util.control.Exception._
scala> catching(classOf[Throwable]) opt Class.forName(typeName)
<console>:13: error: type mismatch;
found : Class[_]
required: Class[?0(in value res4)] where type ?0(in value res4)
Note: Any >: ?0, but Java-defined class Class is invariant in type T.
You may wish to investigate a wildcard type such as `_ >: ?0`. (SLS 3.2.10)
catching(classOf[Throwable]) opt Class.forName(typeName)

Related

Scala singleton objects and implicit resolution

What is the best way to make implicit resolution in scala work with singleton objects? This is especially common with Either and custom error objects.
In the code example below a method returns an application-specific error wrapped in IO. The error is represented by a singleton object extending Throwable. This code does not compile because scala is looking for an implicit for AppSpecificError.type instead of Throwable.
It is possible to put everything into variables with a specified type but it looks weird. This seems like a pretty common case, what is the best way to address it?
import cats.data.EitherT
import cats.effect.IO
import cats.implicits._
import scala.util.Random
object EitherTest {
case object AppSpecificError extends Throwable
def random: IO[Boolean] = {
IO(Random.nextBoolean())
}
def appMethod(flag: Boolean): EitherT[IO, Throwable, Int] = {
for {
flag <- EitherT.right(random)
result <- if (flag) {
EitherT.left[Int](AppSpecificError.pure[IO]) // compilation error here
} else {
EitherT.right[Throwable](10.pure[IO])
}
// can be more computations here
} yield result
}
def main(args: Array[String]): Unit = {
appMethod(true).value.unsafeRunSync() match {
case Right(s) => println("Success")
case Left(error) => println(error)
}
}
}
Error:(18, 14) type mismatch;
found : cats.data.EitherT[cats.effect.IO,_1,Int] where type _1 >: EitherTest.AppSpecificError.type <: Throwable
required: cats.data.EitherT[cats.effect.IO,Throwable,Int]
Note: _1 <: Throwable, but class EitherT is invariant in type A.
You may wish to define A as +A instead. (SLS 4.5)
result <- if (flag) {
Try to specify type parameters explicitly
EitherT.left[Int][IO, Throwable](AppSpecificError.pure[IO])
or use type ascription
EitherT.left[Int]((AppSpecificError: Throwable).pure[IO])

Prove that a runtimeClass satisfies a type Bound in Scala

I have a method that writes one of my classes Foo, which is defined as Thrift, in Parquet form.
import Foo
import org.apache.spark.rdd.RDD
import org.apache.thrift.TBase
import org.apache.hadoop.mapreduce.Job
import org.apache.parquet.hadoop.ParquetOutputFormat
import org.apache.parquet.hadoop.thrift.ParquetThriftOutputFormat
def writeThriftParquet(rdd: RDD[Foo], outputPath: String): Unit = {
val job = Job.getInstance()
ParquetThriftOutputFormat.setThriftClass(job, classOf[Foo])
ParquetOutputFormat.setWriteSupportClass(job, classOf[Foo])
rdd
.map(x => (null, x))
.saveAsNewAPIHadoopFile(
outputPath,
classOf[Void],
classOf[Foo],
classOf[ParquetThriftOutputFormat[Foo]],
job.getConfiguration)
}
This works fine, but I'd prefer to write a more generic method. I tried the (relatively) simple:
def writeThriftParquetGeneral[A <: TBase[_, _]](rdd: RDD[A], outputPath: String): Unit = {
val job = Job.getInstance()
ParquetThriftOutputFormat.setThriftClass(job, classOf[A])
ParquetOutputFormat.setWriteSupportClass(job, classOf[A])
rdd
.map(x => (null, x))
.saveAsNewAPIHadoopFile(
outputPath,
classOf[Void],
classOf[A],
classOf[ParquetThriftOutputFormat[A]],
job.getConfiguration)
}
but that fails with errors like:
class type required but A found ParquetThriftOutputFormat.setThriftClass(job, classOf[A])
class type required but A found ParquetOutputFormat.setWriteSupportClass(job, classOf[A])
To try to remedy that, I've used a ClassTag, but haven't gotten things to compile.
import scala.reflect._
implicit val ct = ClassTag[Foo](classOf[Foo])
def writeThriftParquetGeneral[A <: TBase[_, _]](rdd: RDD[A], outputPath: String)(
implicit tag: ClassTag[A]): Unit = {
val job = Job.getInstance()
// The problem line
ParquetThriftOutputFormat.setThriftClass(job, tag.runtimeClass)
// Seems OK from here
ParquetOutputFormat.setWriteSupportClass(job, tag.runtimeClass)
rdd
.map(x => (null, x))
.saveAsNewAPIHadoopFile(
outputPath,
classOf[Void],
tag.runtimeClass,
classOf[ParquetThriftOutputFormat[A]],
job.getConfiguration)
}
This fails at the line: ParquetThriftOutputFormat.setThriftClass(job, tag.runtimeClass)
[error] found : Class[_$1] where type _$1
[error] required: Class[_ <: org.apache.thrift.TBase[_, _]]
I'm surprised the compiler (Scala 2.11) isn't recognizing that tag.runtimeClass must be a classOf[A], and A satisfies the type bound by definition.
ClassTag#runtimeClass returns just a Class[_]
https://github.com/scala/scala/blob/2.13.x/src/library/scala/reflect/ClassTag.scala#L55
Class[_ <: TBase[_, _]] is an existential type different from Class[_] (actually its subtype)
implicitly[Class[_ <: TBase[_, _]] <:< Class[_]]
Try to replace the problem line with
ParquetThriftOutputFormat.setThriftClass(job, classTag.runtimeClass.asSubclass(classOf[TBase[_, _]]))

Compilation issue with type alias of higherKinds

I have a trait like
trait T{
type F[_]
def get[A](f: F[A]): A
}
But I cannot implement it
type Id[+A] = A // same as shapeless Id
object O extends T{
type F[_] = Id[_]
def get[A](f: F[A]): A = f //
}
// error: type mismatch;
// found : f.type (with underlying type O.F[A])
// required: A
// def get[A](f: F[A]): A = f
// ^
(note I think i should work if I cast f.asIntanceOf[A] but I didn't try)
I have the same problem with Future:
import scala.concurrent.Await
import scala.concurrent.Future
import scala.concurrent.duration.Duration
object O2 extends T{
type F[_] = Future[_]
def get[A](f: F[A]): A = Awaits.result(f, Duration.Inf)
}
// error: type mismatch;
// found : scala.concurrent.Future[_$1] where type _$1
// required: scala.concurrent.Awaitable[A]
// def get[A](f: F[A]): A = Await.result(f, Duration.Inf)
// ^
Can someone explain to me what's happening? Why the compiler cannot understand the actual type F[A] is using the above type alias?
You probably wanted to write
type F[X] = Id[X]
and
type F[X] = Future[X]
Example:
trait T { type F[_]; def get[A](f: F[A]): A }
object Foo extends T {
type F[X] = List[X]
def get[A](f: List[A]): A = f.head
}
I assume that you will have to wait for Dotty and full support for type-lambdas until you can drop the redundant argument on both sides.

Scala, cryptic compiler error: found: Result[_ >: Some[String] with Option[T] <: Option[Any]] required: Result[Option[T]]

This code (https://ideone.com/6y81ZT)
object Main extends App {
import scala.util.Try
case class Result[T](value :T)
def hi() = "hi"
def safer[T](f : () => T) : Try[Result[Option[T]]] = {
Try(hi)
.map((r) => Result(Some(r)))
.orElse(Try(Result(None)))
}
safer(hi)
}
Produces compilation error:
Main.scala:12: error: type mismatch; found :
scala.util.Try[Main.Result[_ >: Some[String] with Option[T] <: Option[Any]]]
required: scala.util.Try[Main.Result[Option[T]]]
How should this message be understood ?
There are a couple problems.
First, you passing hi to Try.apply, which is causing it to be nailed down to Try[String] instead of Try[T].
The second error comes from Result(Some(r)), where the compiler is expecting Result[Option[T]]. The type inside the lambda is prematurely nailed down to Result[Some[T]], which causes the orElse infer the weird compound type.
Try(f())
.map(r => Result(Option(r))) // Try[Result[Some[T]]]
.orElse(Try(Result(None))) // Try[Result[None]]
It also happens when (note the inferred type):
scala> Try(1).map(r => Result(Some(r))).orElse(Try(Result(None)))
res6: scala.util.Try[Result[_ >: Some[Int] with None.type <: Option[Int]]] = Success(Result(Some(1)))
The above wouldn't happen if Result were covariant over T (notice the desired type being inferred):
case class Result[+T](value: T)
scala> Try(1).map(r => Result(Some(r))).orElse(Try(Result(None)))
res7: scala.util.Try[Result[Option[Int]]] = Success(Result(Some(1)))
Without changing Result, this works:
def safer[T](f: () => T) : Try[Result[Option[T]]] = {
Try(f())
.map(r => Result(Option(r)))
.orElse(Try(Result(None)))
}
It's not really necessary to keep the outer Try in the return type, as you will never actually return a failure. (It is always recovered by Result(None).
You could simplify the code it you did that:
def safer[T](f: () => T) : Result[Option[T]] = Result(Try(f()).toOption)

Scala Macros: "cannot create TypeTag from a type T having unresolved type parameters"

I'm playing around with Scalas new macros and found this gist from akshaal. As it seams I did not quite get it.
Given the following trait (the fieldsMacro is more or less the same as in akshaal example)
case class Field[I <: AnyRef](name: String, get: I => Any)
type Fields[I <: AnyRef] = List[Field[I]]
trait FieldAccess {
import FieldMacors._
import Field._
import language.experimental.macros
def fields[T <: AnyRef]: Fields[T] = macro fieldsMacro[T]
def field[T <: AnyRef](name: String): Fields[T] = fields[T].headOption <-- does not work!
^
}
object FieldMacors {
import language.experimental.macros
import Field._
def fields[T <: AnyRef]: Fields[T] = macro fieldsMacro[T]
/**
* Get a list of fiels
*/
def fieldsMacro[T <: AnyRef: c.TypeTag](c: Context): c.Expr[Fields[T]] = {
import c.universe._
val instanceT = c.typeOf[T]
val fields = instanceT.members.filter(member => member.isTerm && !member.isMethod)
// transform an iterable of expr in a expr of list.
def foldIntoListExpr[T: c.TypeTag](exprs: Iterable[c.Expr[T]]): c.Expr[List[T]] =
exprs.foldLeft(reify { Nil: List[T] }) {
(accumExpr, expr) =>
reify { expr.splice :: accumExpr.splice }
}
val fieldAccessores = for (field <- fields) yield {
val name = field.name.toString.trim // Why is there a space at the end of field name?!
val nameExpr = c literal name
// Construct expression (x : $I) => x.$name
val getFunArgTree = ValDef(Modifiers(), newTermName("x"), TypeTree(instanceT), EmptyTree)
val getFunBodyTree = Select(Ident(newTermName("x")), newTermName(name))
val getFunExpr = c.Expr[T => Any](Function(List(getFunArgTree), getFunBodyTree))
reify {
Field[T](name = nameExpr.splice, get = getFunExpr.splice)
}
}
foldIntoListExpr(fieldAccessores)
}
}
the compiler complains about
'Cannot create TypeTag from a type T having unresolved type parameters'
How do I manage to get the T to the macro or must I implement another macro that uses the fieldsMacro
T: TypeTag context bound for a type parameter T means that you require type arguments provided in place of this parameter to be concrete (i.e. not contain references to untagged type parameters or abstract type members). Otherwise an error occurs.
Examples:
scala> val ru = scala.reflect.runtime.universe
ru # 6d657803: scala.reflect.api.JavaUniverse = scala.reflect.runtime.JavaUniverse#6d657803
scala> def foo[T: ru.TypeTag] = implicitly[ru.TypeTag[T]]
foo: [T](implicit evidence$1: reflect.runtime.universe.TypeTag[T])reflect.runtime.universe.TypeTag[T]
scala> foo[Int]
res0 # 7eeb8007: reflect.runtime.universe.TypeTag[Int] = TypeTag[Int]
scala> foo[List[Int]]
res1 # 7d53ccbe: reflect.runtime.universe.TypeTag[List[Int]] = TypeTag[scala.List[Int]]
scala> def bar[T] = foo[T] // T is not a concrete type here, hence the error
<console>:26: error: No TypeTag available for T
def bar[T] = foo[T]
^
scala> def bar[T] = foo[List[T]] // T being not concrete renders
// the entire compound type not concrete
<console>:26: error: No TypeTag available for List[T]
def bar[T] = foo[List[T]]
^
scala> def bar[T: TypeTag] = foo[T] // to the contrast T is concrete here
// because it's bound by a concrete tag bound
bar: [T](implicit evidence$1: reflect.runtime.universe.TypeTag[T])reflect.runtime.universe.TypeTag[T]
scala> bar[Int]
res2 # 7eeb8007: reflect.runtime.universe.TypeTag[Int] = TypeTag[Int]
scala> def bar[T: TypeTag] = foo[List[T]]
bar: [T](implicit evidence$1: reflect.runtime.universe.TypeTag[T])reflect.runtime.universe.TypeTag[List[T]]
scala> bar[Int]
res3 # 1a108c98: reflect.runtime.universe.TypeTag[List[Int]] = TypeTag[scala.List[Int]]
scala> bar[List[Int]]
res4 # 76d5989c: reflect.runtime.universe.TypeTag[List[List[Int]]] = TypeTag[scala.List[scala.List[Int]]]
Having a notion of concrete types to be enforcible at compile-time is useful. Having concrete type tags on by default is useful as well as described in https://issues.scala-lang.org/browse/SI-5884.
However as you've seen yourself, concrete type tags in macros can be a source of confusion, because typically macros should work both for concrete and non-concrete types. Therefore one should always use c.AbsTypeTag instead. Due to this reason we no longer allow c.TypeTag context bounds in 2.10.0-M7: https://github.com/scala/scala/commit/788478d3ab.
Edit. In 2.10.0-RC1 some AbsTypeTag has been renamed to WeakTypeTag. Everything else about type tags remains the same.