Can a Scala "extractor" use generics on unapply? - scala

Can't I use a generic on the unapply method of an extractor along with an implicit "converter" to support a pattern match specific to the parameterised type?
I'd like to do this (Note the use of [T] on the unapply line),
trait StringDecoder[A] {
def fromString(string: String): Option[A]
}
object ExampleExtractor {
def unapply[T](a: String)(implicit evidence: StringDecoder[T]): Option[T] = {
evidence.fromString(a)
}
}
object Example extends App {
implicit val stringDecoder = new StringDecoder[String] {
def fromString(string: String): Option[String] = Some(string)
}
implicit val intDecoder = new StringDecoder[Int] {
def fromString(string: String): Option[Int] = Some(string.charAt(0).toInt)
}
val result = "hello" match {
case ExampleExtractor[String](x) => x // <- type hint barfs
}
println(result)
}
But I get the following compilation error
Error: (25, 10) not found: type ExampleExtractor
case ExampleExtractor[String] (x) => x
^
It works fine if I have only one implicit val in scope and drop the type hint (see below), but that defeats the object.
object Example extends App {
implicit val intDecoder = new StringDecoder[Int] {
def fromString(string: String): Option[Int] = Some(string.charAt(0).toInt)
}
val result = "hello" match {
case ExampleExtractor(x) => x
}
println(result)
}

A variant of your typed string decoder looks promising:
trait StringDecoder[A] {
def fromString(s: String): Option[A]
}
class ExampleExtractor[T](ev: StringDecoder[T]) {
def unapply(s: String) = ev.fromString(s)
}
object ExampleExtractor {
def apply[A](implicit ev: StringDecoder[A]) = new ExampleExtractor(ev)
}
then
implicit val intDecoder = new StringDecoder[Int] {
def fromString(s: String) = scala.util.Try {
Integer.parseInt(s)
}.toOption
}
val asInt = ExampleExtractor[Int]
val asInt(Nb) = "1111"
seems to produce what you're asking for. One problem remains: it seems that trying to
val ExampleExtractor[Int](nB) = "1111"
results in a compiler crash (at least inside my 2.10.3 SBT Scala console).

Related

repartitionAndSortWithinPartitions is not a member of RDD[(K, V)]

I am trying to do secondary sort with Scala Spark, following this blog. And I have the following code:
case class TokenZidKey(token: String, zid: Long)
object TokenZidKey {
implicit def orderingByTokenZid[A <: TokenZidKey]: Ordering[A] = {
Ordering.by(tzk => (tzk.token, tzk.zid))
}
}
class TokenZidPartitioner(partitions: Int) extends Partitioner {
override def numPartitions: Int = partitions;
override def getPartition(key: Any): Int = key.asInstanceOf[TokenZidKey].token.hashCode() % numPartitions;
}
object SetSim {
def main(args: Array[String]) {
...
val linesWithZid = dataLines.zipWithIndex();
def mapZidByToken(r: (String, Long)): Iterable[(TokenZidKey, String)] = {
for (t <- r._1.split(" ")) yield (new TokenZidKey(t, r._2), r._1);
}
val tokensWithZid = linesWithZid.flatMap(mapZidByToken);
val tokenZidPartitioner = new TokenZidPartitioner(tokensWithZid.context.defaultParallelism);
val tokenZidsWithSecondarySort = tokensWithZid.repartitionAndSortWithinPartitions(tokenZidPartitioner).map(r => {
(r._1.token, (r._1.zid, r._2))
}).toDF("token", "zid_with_line");
}
}
where tokensWithZid is org.apache.spark.rdd.RDD[(TokenZidKey, String)], but I still got
value repartitionAndSortWithinPartitions is not a member of org.apache.spark.rdd.RDD[(TokenZidKey, String)]
Any idea why and how I fix this? Is it that my custom TokenZidKey is not recognized as a proper RDD key that is sortable?
Turns out that the code works, the issue is I tested it in spark-shell and when input on different lines, spark-shell does not register the case class and the companion object as on the same file.
So if you want to test this in spark-shell, enter the case class and companion object in a single line like the following
case class TokenZidKey(token: String, zid: Long); object TokenZidKey {
implicit def orderingByTokenZid[A <: TokenZidKey]: Ordering[A] = {
Ordering.by(tzk => (tzk.token, tzk.zid))
}
}

How to implement typesafe domain repository in scala?

I want to implement generic and typesafe domain repository. Say I have
trait Repo[Value] {
def put(value: Value): Unit
}
case class IntRepo extends Repo[Int] {
override def put(value: Int): Unit = ???
}
case class StringRepo extends Repo[String] {
override def put(value: String): Unit = ???
}
case class DomainRepo(intRepo: IntRepo, stringRepo: StringRepo) {
def putAll[?](values: ?*): Unit // what type should be here?
}
As result I want to have following api:
domainRepo.putAll(1, 2, 3, "foo", "bar") //Should work
domainRepo.putAll(1, 2, true, "foo") // should not compile because of boolean value
The question is How to achieve this?
so, I see only one way to make it typesafe. It's to do pattern matching on Any type like
def putAll(values: Seq[Any]) => Unit = values.foreach {
case str: String => stringRepo.put(str)
case int: Int => intRepo.put(int)
case _ => throw RuntimeException // Ha-Ha
}
but what if I would have 10000 of types here? it would be a mess!
another not clear for me approach for now is to use dotty type | (or) like following:
type T = Int | String | 10000 other types // wouldn't be a mess?
def putAll(t: T*)(implicit r1: Repo[Int], r2: Repo[String] ...) {
val myTargetRepo = implicitly[Repo[T]] // would not work
}
so, what do you think? is it even possible?
the easies way I've saw was
Map[Class[_], Repo[_]]
but this way allows to do a lot of errors
It seems you are looking for a type class
trait Repo[Value] {
def put(value: Value): Unit
}
implicit val intRepo: Repo[Int] = new Repo[Int] {
override def put(value: Int): Unit = ???
}
implicit val stringRepo: Repo[String] = new Repo[String] {
override def put(value: String): Unit = ???
}
case object DomainRepo {
def putAll[Value](value: Value)(implicit repo: Repo[Value]): Unit = repo.put(value)
}
If you want domainRepo.putAll(1, 2, 3, "foo", "bar") to compile and domainRepo.putAll(1, 2, true, "foo") not to compile, you can try to use heterogeneous collection (HList).
import shapeless.{HList, HNil, ::, Poly1}
import shapeless.ops.hlist.Mapper
trait Repo[Value] {
def put(value: Value): Unit
}
implicit val intRepo: Repo[Int] = new Repo[Int] {
override def put(value: Int): Unit = ???
}
implicit val stringRepo: Repo[String] = new Repo[String] {
override def put(value: String): Unit = ???
}
case object DomainRepo {
def put[Value](value: Value)(implicit repo: Repo[Value]): Unit = repo.put(value)
object putPoly extends Poly1 {
implicit def cse[Value: Repo]: Case.Aux[Value, Unit] = at(put(_))
}
def putAll[Values <: HList](values: Values)(implicit
mapper: Mapper[putPoly.type, Values]): Unit = mapper(values)
}
DomainRepo.putAll(1 :: 2 :: 3 :: "foo" :: "bar" :: HNil)
// DomainRepo.putAll(1 :: 2 :: true :: "foo" :: HNil) // doesn't compile

How to pass implicit parameter explicitly?

If we define the following function:
def methodWithImplicit(explicit: String)(implicit imp: String) = {
println(explicit + imp)
}
we can call it as follows:
methodWithImplicit("abc")("efg") //abc - explicit, efg - imp
And it works fine. Now consider the following TypeClass:
trait MyTypeClass[T] {
def accept(t: T): T
}
which is going to be used inside extractor object:
object TestExtractor {
def unapply(str: String)(implicit myTypeClass: MyTypeClass[String]): Option[String] =
if (!str.isEmpty)
Some(myTypeClass.accept(str))
else
None
}
So if we use it as follows:
implicit val myTypeClass:MyTypeClass[String] = new MyTypeClass[String] {
override def accept(t: String): Unit = t
}
"123" match {
case TestExtractor(str) => println(str)
}
It works ok. But how to pass the parameter explicitly when using with pattern matching? I tried
"123" match {
case TestExtractor(str)(myTypeClass) => println(str) //compile error
}
and
"123" match {
case TestExtractor(myTypeClass)(str) => println(str) //compile error
}
But it does not compile.
Since the left hand side seems to accept essentially nothing but trees built from stable identifiers, constant literals, and lower-case letters for variable names, I don't see any way to get closer to the desired syntax than this:
val `TestExtractor(myTypeClass)` = TestExtractor(myTypeClass)
"hello" match {
case `TestExtractor(myTypeClass)`(str) => println(str)
}
This of course requires that you define the weirdly named value TestExtractor(myTypeClass) (in backticks) right before the match-case, so you can use it as a single symbol.
Full code:
trait MyTypeClass[T] {
def accept(t: T): T
}
object TestExtractor { outer =>
def unapply(str: String)(implicit myTypeClass: MyTypeClass[String]): Option[String] =
if (!str.isEmpty)
Some(myTypeClass.accept(str))
else
None
class ExplicitTestExtractor(tc: MyTypeClass[String]) {
def unapply(t: String) = outer.unapply(t)(tc)
}
def apply(tc: MyTypeClass[String]): ExplicitTestExtractor =
new ExplicitTestExtractor(tc)
}
implicit val myTypeClass:MyTypeClass[String] = new MyTypeClass[String] {
override def accept(t: String): String = t.toUpperCase
}
val `TestExtractor(myTypeClass)` = TestExtractor(myTypeClass)
"hello" match {
case `TestExtractor(myTypeClass)`(str) => println(str)
}

Scala macro and type erasure

I'm having some problems with a macro I've written to help me log metrics represented as case class instances to to InfluxDB. I presume I'm having a type erasure problem and that the tyep parameter T is getting lost, but I'm not entirely sure what's going on. (This is also my first exposure to Scala macros.)
import scala.language.experimental.macros
import play.api.libs.json.{JsNumber, JsString, JsObject, JsArray}
abstract class Metric[T] {
def series: String
def jsFields: JsArray = macro MetricsMacros.jsFields[T]
def jsValues: JsArray = macro MetricsMacros.jsValues[T]
}
object Metrics {
case class LoggedMetric(timestamp: Long, series: String, fields: JsArray, values: JsArray)
case object Kick
def log[T](metric: Metric[T]): Unit = {
println(LoggedMetric(
System.currentTimeMillis,
metric.series,
metric.jsFields,
metric.jsValues
))
}
}
And here's an example metric case class:
case class SessionCountMetric(a: Int, b: String) extends Metric[SessionCountMetric] {
val series = "sessioncount"
}
Here's what happens when I try to log it:
scala> val m = SessionCountMetric(1, "a")
m: com.confabulous.deva.SessionCountMetric = SessionCountMetric(1,a)
scala> Metrics.log(m)
LoggedMetric(1411450638296,sessioncount,[],[])
Even though the macro itself seems to work fine:
scala> m.jsFields
res1: play.api.libs.json.JsArray = ["a","b"]
scala> m.jsValues
res2: play.api.libs.json.JsArray = [1,"a"]
Here's the actual macro itself:
import scala.language.experimental.macros
import scala.reflect.macros.blackbox.Context
object MetricsMacros {
private def fieldNames[T: c.WeakTypeTag](c: Context)= {
val tpe = c.weakTypeOf[T]
tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
}
def jsFields[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => Literal(Constant(name.toString))).toList
)
}
def jsValues[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val names = fieldNames[T](c)
Apply(
q"play.api.libs.json.Json.arr",
names.map(name => q"${c.prefix.tree}.$name").toList
)
}
}
Update
I tried Eugene's second suggestion like this:
abstract class Metric[T] {
def series: String
}
trait MetricSerializer[T] {
def fields: Seq[String]
def values(metric: T): Seq[Any]
}
object MetricSerializer {
implicit def materializeSerializer[T]: MetricSerializer[T] = macro MetricsMacros.materializeSerializer[T]
}
object Metrics {
def log[T: MetricSerializer](metric: T): Unit = {
val serializer = implicitly[MetricSerializer[T]]
println(serializer.fields)
println(serializer.values(metric))
}
}
with the macro now looking like this:
object MetricsMacros {
def materializeSerializer[T: c.WeakTypeTag](c: Context) = {
import c.universe._
val tpe = c.weakTypeOf[T]
val names = tpe.decls.collect {
case field if field.isMethod && field.asMethod.isCaseAccessor => field.asTerm.name
}
val fields = Apply(
q"Seq",
names.map(name => Literal(Constant(name.toString))).toList
)
val values = Apply(
q"Seq",
names.map(name => q"metric.$name").toList
)
q"""
new MetricSerializer[$tpe] {
def fields = $fields
def values(metric: Metric[$tpe]) = $values
}
"""
}
}
However, when I call Metrics.log -- specifically when it calls implicitly[MetricSerializer[T]] I get the following error:
error: value a is not a member of com.confabulous.deva.Metric[com.confabulous.deva.SessionCountMetric]
Why is it trying to use Metric[com.confabulous.deva.SessionCountMetric] instead of SessionCountMetric?
Conclusion
Fixed it.
def values(metric: Metric[$tpe]) = $values
should have been
def values(metric: $tpe) = $values
You're in a situation that's very close to one described in a recent question: scala macros: defer type inference.
As things stand right now, you'll have to turn log into a macro. An alternative would also to turn Metric.jsFields and Metric.jsValues into JsFieldable and JsValuable type classes materialized by implicit macros at callsites of log (http://docs.scala-lang.org/overviews/macros/implicits.html).

Casting String to Int using scala extractors

I am trying to cast a String to Int using extractors. My code looks as follows.
object Apply {
def unapply(s: String): Option[Int] = try {
Some(s.toInt)
} catch {
case _: java.lang.Exception => None
}
}
object App {
def toT[T](s: AnyRef): Option[T] = s match {
case v: T => Some(v)
case _ => None
}
def foo(param: String): Int = {
//reads a Map[String,String] m at runtime
toT[Int](m("offset")).getOrElse(0)
}
}
I get a runtime error: java.lang.String cannot be cast to java.lang.Integer. It seems the extractor is not being used at all. What should I do?
Edit: My use case is as follows. I am using play and I want to parse the query string passed in the url. I want to take the query string value (String) and use it as an Int, Double etc. For example,
val offset = getQueryStringAs[Int]("offset").getOrElse(0)
I think the biggest problem here is, that you seem to confuse casting and conversion. You have a Map[String, String] and therefore you can't cast the values to Int. You have to convert them. Luckily Scala adds the toInt method to strings through implicit conversion to StringOps.
This should work for you:
m("offset").toInt
Note that toInt will throw a java.lang.NumberFormatException if the string can not be converted to an integer.
edit:
What you want will afaik only work with typeclasses.
Here is an example:
trait StringConverter[A] {
def convert(x: String): A
}
implicit object StringToInt extends StringConverter[Int] {
def convert(x: String): Int = x.toInt
}
implicit object StringToDouble extends StringConverter[Double] {
def convert(x: String): Double = x.toDouble
}
implicit def string2StringConversion(x: String) = new {
def toT[A](implicit ev: StringConverter[A]) = ev.convert(x)
}
usage:
scala> "0.".toT[Double]
res6: Double = 0.0
There's a problem in your code, for which you should have received compiler warnings:
def toT[T](s: AnyRef): Option[T] = s match {
case v: T => Some(v) // this doesn't work, because T is erased
case _ => None
}
Now... where should Apply have been used? I see it declared, but I don't see it used anywhere.
EDIT
About the warning, look at the discussions around type erasure on Stack Overflow. For example, this answer I wrote on how to get around it -- though it's now deprecated with Scala 2.10.0.
To solve your problem I'd use type classes. For example:
abstract class Converter[T] {
def convert(s: String): T
}
object Converter {
def toConverter[T](converter: String => T): Converter[T] = new Converter[T] {
override def convert(s: String): T = converter(s)
}
implicit val intConverter = toConverter(_.toInt)
implicit val doubleConverter = toConverter(_.toDouble)
}
Then you can rewrite your method like this:
val map = Map("offset" -> "10", "price" -> "9.99")
def getQueryStringAs[T : Converter](key: String): Option[T] = {
val converter = implicitly[Converter[T]]
try {
Some(converter convert map(key))
} catch {
case ex: Exception => None
}
}
In use:
scala> getQueryStringAs[Int]("offset")
res1: Option[Int] = Some(10)
scala> getQueryStringAs[Double]("price")
res2: Option[Double] = Some(9.99)