Scala: generic parser for Enumeration values - scala

I thought it should be possible to write a generic function that works for all Enumeration values. I tried a simple parser first but I failed:
object Weekday extends Enumeration {
type Weekday = Value
val MONDAY = Value("MONDAY")
val OTHER = Value("OTHER")
implicit def valueToWeekday(v: Value): Weekday = v.asInstanceOf[Weekday]
implicit def stringToWeekday(s: String): Weekday = Weekday.withName(s)
}
object Enumerations {
import Weekday._
println("Welcome to the Scala worksheet")
def parseEnumeration[T <: Enumeration](s: String)(implicit ev: T): T#Value = {
ev.withName(s)
}
val test = parseEnumeration[Weekday]("MONDAY")
}
So how can I write a generict function taking an enumeration type as parameter and returning a Value of that type? I'm a bit confused here with the Object and the inner type with the same name.

Firstly, your implicit method valueToWeekday doesn't really do anything, as Weekday is simply an alias for Value in this context.
Secondly, your implicit method stringToWeekday is a working, albeit non-generic conversion from a string to its enumeration value.
However, it is not hard to make stringToWeekday generic. You simply need to pass the enumeration to the function, just like you do in parseEnumeration. Since you made the evidence in parseEnumeration implicit, all you need to do is put an appropriate implicit value in the context. Alternatively, you can pass the evidence explicitly.
So you can remove those implicit conversions (and the type alias, since the name-clash is slightly misleading).
object Weekday extends Enumeration {
val Monday = Value("MONDAY")
val Other = Value("OTHER")
}
The implicit way:
def parseEnumeration[T <: Enumeration](s: String)(implicit ev: T): T#Value = ev.withName(s)
implicit val evidence = Weekday
val weekday = parseEnumeration("MONDAY") // results in the value Weekday.Monday
The explicit way:
def parseEnumeration[T <: Enumeration](s: String, enumeration: T): T#Value = enumeration.withName(s)
val weekday = stringToEnumerationValue("MONDAY", Weekday) // results in the value Weekday.Monday
A third option would be to use a ClassTag as evidence, which is put in the context by the compiler through the generic parameters. However, this requires reflection to actually call the method withName and I would discourage going this way.

Related

Difference between Scala 2 implicits and Scala 3 given/using

What is the difference between the implicit keyword in Scala 2 and given+using in Scala 3? Is it just that implicit has been split up into two keywords, or are the semantics also different, and if so, how?
For the most part, they are the same. However, implicit is no longer used for multiple different concepts. The docs go into more detail, but here's a summary of them:
Using
When declaring parameters, using is just the same as implicit. However, when explicitly passing an implicit argument, you must use using:
def foo(using bar: Bar) = ???
foo(using Bar()) //Cannot use just foo(Bar()) as you would in Scala 2
You can also have implicit by-name parameters in Scala 3.
Given
Givens are also pretty similar to implicit vals/objects/methods.
One nice thing about them is that they can be anonymous, and the compiler will generate a name for them, which looks something like given_F_X_Y if the type of the given were F[X, Y]. More details here.
Another change is that the type of a given must be written explicitly - it cannot be inferred like for an implicit in Scala 2.
A given without parameters maps to an implicit object. given foo: Foo with {...} becomes just implicit object foo extends Foo {...}.
A given with parameters is akin to an implicit def that takes in only more implicit parameters.
given listOrd[T](using ord: Ord[T]): Ord[List[T]] with { ... }
//^^ this maps to this vv
class listOrd[T](implicit ord: Ord[T]) extends Ord[List[T]] { ... }
final implicit def listOrd[T](implicit ord: Ord[T]): listOrd[T] = new listOrd[T]
A given that is merely an alias becomes an implicit def if it is just a reference, or an implicit lazy val otherwise.
val foo: Foo
given Foo = foo
would become final implicit def given_Foo = foo (note the compiler-generated name), but
given foo: Foo = new Foo()
would turn into final implicit lazy val foo: Foo = new Foo() because new Foo() shouldn't be computed unnecessarily.
Instead of using an implicit def for an implicit conversion from A to B, you can now define a given Conversion[A, B] instance.
You can also still use implicit classes in Dotty, but you can directly define extension methods. While methods inside extensions cannot take their own type parameters, they are easier to use than implicit classes.
An additional change in Scala 3 - summon is a method like implicitly, but it can return a type more specific than the one being requested.
Semantics is also different. In Scala 2 Not can be defined with ambiguity trick
trait Not[A]
object Not {
implicit def default[A]: Not[A] = null
implicit def ambig[A](implicit a: A): Not[A] = null
}
implicitly[Not[Int]] // compiles
implicit val s: String = null
// implicitly[Not[String]] // doesn't compile
But in Scala 3 this doesn't work because ambiguity error is not propagated
trait Not[A]
object Not {
given [A]: Not[A] = null
given [A](using a: A): Not[A] = null
// given ambig[A](using a: A): Not[A] = null
}
summon[Not[Int]] // compiles
given String = null
summon[Not[String]] // compiles
One should use scala.util.NotGiven instead
summon[NotGiven[Int]] // compiles
given String = null
// summon[NotGiven[String]] // doesn't compile
(Tested in 3.0.0-M3-bin-20201211-dbc1186-NIGHTLY)
http://dotty.epfl.ch/docs/reference/contextual/givens.html#negated-givens
http://dotty.epfl.ch/docs/reference/changed-features/implicit-resolution.html

Scala implicit search of covariant type class replaces the type argument with Nothing. Why?

Let's use a real world example. A string parser type class whose implicit instances are created by a function that delegates the creation to a factory.
import scala.reflect.runtime.universe.TypeTag
object Test {
trait Parser[+T] { def parse(input: String): T }
implicit def summonParserOf[T](implicit factory: ParserFactory[T]): Parser[T] = factory.build
trait ParserFactory[T] { def build: Parser[T] }
implicit def summonFactoryOfParsersOf[T](implicit t: TypeTag[T]): ParserFactory[T] =
new ParserFactory[T] {
def build: Parser[T] = new Parser[T] {
def parse(input: String) = {
println("T = " + t.tpe) // this outputs "T = Int" if Parser is non variant, and "T = Nothing" if Parser is covariant on T. Why?
null.asInstanceOf[T]
}
}
}
def main(args: Array[String]): Unit = {
val parserOfInt = implicitly[Parser[Int]]
parserOfInt.parse("")
}
}
The type parameter T received by the factory is Int when Parser is non-variant, and Nothing when it is covariant. Why?
Edit 1:
The factory is not necessary. The replacement occurs before. So the test can be reduced to:
package jsfacile.test
import scala.reflect.runtime.universe.TypeTag
object Probando {
trait Parser[+T] { def parse(input: String): T }
implicit def summonParserOf[T](implicit t: TypeTag[T]): Parser[T] = new Parser[T] {
def parse(input: String): T = {
println("summon parser: T = " + t.tpe) // this outputs "T = Int" if Parser is non variant, and "T = Nothing" if Parser is covariant on T. Why?
null.asInstanceOf[T]
null.asInstanceOf[T]
}
}
def main(args: Array[String]): Unit = {
val parserOfInt = implicitly[Parser[Int]]
parserOfInt.parse("")
}
}
Parser[Nothing] is assignable to Parser[Int], but which is the purpose of choosing the lower bound instead of the upper one?
Edit 2: The answer given by #Dmytro Mitin and the useful comments below, translated to my own words and limited scope of thinking, for future reference to myself.
What stopped me to understand was the wrong idea that, when the implicit value provider is a def with parametrized result type, there is no set of living values from which the compiler has to pick one of. In that case, I thought, it just skips that step (the one that chooses the value with the most specific declared type).
And given the summoner function grants the compiler the power to build a value of any type, why not to fill the implicit parameter with a value that makes him happy. If the implicit parameter demands something assignable to a type T then give it a value of type T. Giving it Nothing, which is assignable to everything, wouldn't be nice nor useful.
The problem with that idea arises when there is more than one summoner providing values assignable to the implicit parameter type. In that case, the only consistent way to decide which summoner to chose is to deduce the set of types of the values they produce, pick a type from said set based on an established criteria (the most specific, for instance), and choose the summoner that produces it.
Scala spec says
If there are several eligible arguments which match the implicit parameter's type, a most specific one will be chosen using the rules of static overloading resolution
https://scala-lang.org/files/archive/spec/2.11/07-implicits.html#implicit-parameters
Since you defined instances like
implicit def summonParserOf[T](implicit t: TypeTag[T]): Parser[T] = ...
for covariant
trait Parser[+T] { ... }
when you look for implicitly[Parser[T]], all summonParserOf[S] (S <: T) are eligible candidates, so the compiler selects the most specific one.

Auto-Inferring a higher-kinded Aux Parameter from an Argument

On an earlier SO post I asked how to create use the Aux pattern with a higher kinded type (here, that had a great reply!). Now based on the reply, I am trying to take the abstraction a little but further and have the actual generic parameter be inferred from an argument's type. Trouble is, it seems having trouble differentiating X from X.type. Here's my code:
// The are types that I want to convert to various things
sealed trait ConversionType
trait CaseA extends ConversionType
object CaseA extends CaseA // In this case, convert to an optional
trait CaseB extends ConversionType
object CaseB extends CaseB // In this case, convert to a future etc...
trait Converter[Prefix] {
type Paramd[_]
def create[N](n:N): Paramd[N]
}
// Create the mechanism to convert from the cases, only doing case A for now...
object Converter {
type Aux[Prefix, Ret[_]] = Converter[Prefix] { type Paramd[N] = Ret[N] }
// Shouldn't `Prefix` be automatically inferred?
def apply[Prefix](prefix:Prefix)(implicit p:Converter[Prefix]): Aux[Prefix, p.Paramd] = p
implicit def makeOptionParamd: Aux[CaseA, Option] =
new Converter[CaseA] {
type Paramd[N] = Option[N]
override def create[N](n:N): Paramd[N] = Option[N](n)
}
}
// This works
val v = Converter.apply[CaseA](CaseA).create("test")
// **** This breaks! Why? ****
val vv = Converter.apply(CaseA).create("test")
The following errors occour on the broken line above:
Error:(135, 29) could not find implicit value for parameter p: Test.this.Converter[Test.this.CaseA.type]
val vv = Converter.apply(CaseA).create("test")
Error:(135, 29) not enough arguments for method apply: (implicit p: Test.this.Converter[SchemaMaker.this.CaseA.type])Test.this.Converter.Aux[SchemaMaker.this.CaseA.type,p.Paramd] in object Converter.
Unspecified value parameter p.
val vv = Converter.apply(CaseA).create("test")
So the compiler doesn't connect the dots between the object CaseA and the type CaseA? Is there any way to fix this?
Your object CaseA is at the same time
of type CaseA.type (because it's the singleton object)
of type CaseA (because it extends CaseA)
When you invoke
val vv = Converter.apply(CaseA).create("test")
the inferred type Prefix is CaseA.type (the singleton object type), and no implicit can be found for that type.
Even more explicitly:
val works = Converter.apply(CaseA: CaseA).create("test")
val fails = Converter.apply(CaseA: CaseA.type).create("test")
The absolutely shortest possible change that would force your code to compile would be to add a single - in front of Prefix to make Converter contravariant:
trait Converter[-Prefix] {
type Paramd[_]
def create[N](n:N): Paramd[N]
}
but I'm not sure whether this is what you want, because I don't know what you are attempting to achieve with the all this fancy machinery.

Spark reduceByKey with generic types (Scala)

I am trying to create some simple custom aggregate operators in Spark using Scala.
I have created a simple hierarchy of operators, with the following super-class:
sealed abstract class Aggregator(val name: String) {
type Key = Row // org.apache.spark.sql.Row
type Value
...
}
I also have a companion object, which constructs the appropriate aggregator each time. Observe that each operator is allowed to specify the Value type it wants.
Now the problem is when I try to call combineByKey:
val agg = Aggregator("SUM")
val res = rdd
.map(agg.mapper)
.reduceByKey(agg.reducer(_: agg.Value, _: agg.Value))
The error is:
value reduceByKey is not a member of org.apache.spark.rdd.RDD[(agg.Key, agg.Value)]
For my needs, Value can either be a numeric type or a tuple, hence its no bounds definition. If I replace the Value type declaration with:
type Value = Double
in Aggregator class, then everything works fine. Therefore, I suppose that the error is relevant to reduceByKey not knowing the exact Value type in compile time.
Any ideas on how to get around this?
Your RDD cannot be implicitly converted into PairRDDFunctions, because all the implicit ClassTags for keys and values are missing.
You might want to include the class tags as implicit parameters in your Aggregator:
sealed abstract class Aggregator[K: ClassTag, V: ClassTag](name: String) {
implicit val keyClassTag: ClassTag[K] = implicitly
implicit val valueClassTag: ClassTag[V] = implicitly
}
or maybe:
sealed abstract class Aggregator[K, V](name: String)(implicit kt: ClassTag[K], vt: ClassTag[V]) {
implicit val keyClassTag: ClassTag[K] = kt
implicit val valueClassTag: ClassTag[V] = vt
}
or maybe even:
sealed abstract class Aggregator(name: String) {
type K
type V
implicit def keyClassTag: ClassTag[K]
implicit def valueClassTag: ClassTag[V]
}
The last variant would shift the responsibility for providing the ClassTags to the implementor of the abstract class.
Now, when using an aggregator a of type Aggregator[K, V] in a reduceByKey, you would have to make sure that those implicitly provided class tags are in the current implicit scope:
val agg = Aggregator("SUM")
import agg._ // now the implicits should be visible
val res = rdd
.map(agg.mapper)
.reduceByKey(agg.reducer(_: agg.Value, _: agg.Value))

Scala implicit conversion problem

I am struggling with a Scala implicit conversion problem. The following code snippet illustrates my problem :
import org.junit.{ Test, Before, After };
class ImplicitsTest {
implicit def toStringWrapper(str: String) = new StringWrapper(str);
#Test
def test(){
val res1: Predicate = "str" startsWith "other";
}
}
class StringWrapper(str: String){
def startsWith(other: String): Predicate = null;
}
trait Predicate
How can I force the String literal "str" to be converted through the implicit conversion toStringWrapper to get startsWith return Predicate instead of Boolean?
The code example doesn't compile. I am aware that String has already a startsWith method, I just want to use a different one, and I thought that using implicit conversions might be a way to do it.
Scala thankfully doesn't let you sneak replacement methods in without you noticing--if you call a method on a class, and the class has that method, that's the method call you get. To do otherwise would likely cause all sorts of confusion.
That leaves you with two other options:
(1) Rename the method
(2) Add a specific method to do the conversion.
The second approach works like so: you define a class that has both the method that you want and a uniquely-named method that returns itself, and optionally an implicit conversion from that class back to string if you want to be able to use the custom item like the original string (as if it had extended String):
object ImplicitExample {
class CustomString(s: String) {
def original = s
def custom = this
def startsWith(other: String): Int = if (s.startsWith(other)) 1 else 0
}
implicit def string_to_custom(s: String) = new CustomString(s)
implicit def custom_to_string(c: CustomString) = c.original
def test = {
println("This".custom.startsWith("Thi"))
println("This".custom.length())
}
}
scala> ImplicitExample.test
1
4
scala>
An implicit conversion is triggered in Scala only if the receiver does not contain the method being invoked or if an expression has a type different than the expected type.
Since the String object above contains the method startsWith, no implicit conversion is triggered. The compiler does, however, check if the type of the right hand expression "str".startsWith("other"); (that is Boolean) can be converted to a Predicate. Since there is no such implicit conversion in scope, it reports an error.
Note, also, that implicit conversions must be unambiguous. For instance, you might try to override Scala string behaviour for some other methods using implicit conversions. Java String objects are not Scala sequences (Seq), meaning that they do not have methods such as sorted, which returns the sorted version of the sequence. If you invoke it on a string:
scala> "string" sorted
res1: String = ginrst
This works because an implicit conversion defined in the Predef object gets triggered. Note that providing your own implicit conversion results in an error:
scala> implicit def wrap(s: String) = new { def sorted = "Hi!" }
wrap: (s: String)java.lang.Object{def sorted: java.lang.String}
scala> "string" sorted
<console>:7: error: type mismatch;
found : java.lang.String
required: ?{val sorted: ?}
Note that implicit conversions are not applicable because they are ambiguous:
...