scala: Parameterize by Type Union - scala

I needed a type Union to force restriction on types, So as per answer in here, I defined my Union as:
sealed trait Value[T]
object Value{
implicit object NumberWitness extends Value[Int]
implicit object StringWitness extends Value[String]
}
Now, How do i create a list or class parameterized by this type union? Is it possible to do so?? I tried following syntax in repl, but without any luck:
scala> import Value._
import Value._
scala> def list[V: Value] = List("hello", 1)
list: [V](implicit evidence$1: Value[V])List[Any]
scala> list
<console>:18: error: ambiguous implicit values:
both object NumberWitness in object Value of type Value.NumberWitness.type
and object StringWitness in object Value of type Value.StringWitness.type
match expected type Value[V]
list
^
Or Is it possible to do so with advanced FP libraries like scalaz or cats??

This is called type class, not type union. And they are intended to allow you to write methods which work either with Int or with String, e.g.
def listOfValues[V: Value](x: V) = List(x)
listOfValues(1) // works
listOfValues("") // works
listOfValues(0.0) // doesn't work
listOfValues(1, "") // doesn't work
not to allow mixing different types.
You can do it using existential types, e.g.
case class WithValue[V: Value](x: V)
object WithValue {
implicit def withValue[V: Value](x: V) = WithValue(x)
}
def list = List[WithValue[_]]("hello", 1)
but I would not recommend actually doing that. There is quite likely a better way to solve your problem.
In particular, consider using simply
// not sealed if you need to add other types elsewhere
// can be Value[T] instead
sealed trait Value
case class IntValue(x: Int) extends Value
case class StringValue(x: Int) extends Value
// add implicit conversions to IntValue and StringValue if desired
List(StringValue("hello"), IntValue(1))

Related

Spark reduceByKey with generic types (Scala)

I am trying to create some simple custom aggregate operators in Spark using Scala.
I have created a simple hierarchy of operators, with the following super-class:
sealed abstract class Aggregator(val name: String) {
type Key = Row // org.apache.spark.sql.Row
type Value
...
}
I also have a companion object, which constructs the appropriate aggregator each time. Observe that each operator is allowed to specify the Value type it wants.
Now the problem is when I try to call combineByKey:
val agg = Aggregator("SUM")
val res = rdd
.map(agg.mapper)
.reduceByKey(agg.reducer(_: agg.Value, _: agg.Value))
The error is:
value reduceByKey is not a member of org.apache.spark.rdd.RDD[(agg.Key, agg.Value)]
For my needs, Value can either be a numeric type or a tuple, hence its no bounds definition. If I replace the Value type declaration with:
type Value = Double
in Aggregator class, then everything works fine. Therefore, I suppose that the error is relevant to reduceByKey not knowing the exact Value type in compile time.
Any ideas on how to get around this?
Your RDD cannot be implicitly converted into PairRDDFunctions, because all the implicit ClassTags for keys and values are missing.
You might want to include the class tags as implicit parameters in your Aggregator:
sealed abstract class Aggregator[K: ClassTag, V: ClassTag](name: String) {
implicit val keyClassTag: ClassTag[K] = implicitly
implicit val valueClassTag: ClassTag[V] = implicitly
}
or maybe:
sealed abstract class Aggregator[K, V](name: String)(implicit kt: ClassTag[K], vt: ClassTag[V]) {
implicit val keyClassTag: ClassTag[K] = kt
implicit val valueClassTag: ClassTag[V] = vt
}
or maybe even:
sealed abstract class Aggregator(name: String) {
type K
type V
implicit def keyClassTag: ClassTag[K]
implicit def valueClassTag: ClassTag[V]
}
The last variant would shift the responsibility for providing the ClassTags to the implementor of the abstract class.
Now, when using an aggregator a of type Aggregator[K, V] in a reduceByKey, you would have to make sure that those implicitly provided class tags are in the current implicit scope:
val agg = Aggregator("SUM")
import agg._ // now the implicits should be visible
val res = rdd
.map(agg.mapper)
.reduceByKey(agg.reducer(_: agg.Value, _: agg.Value))

Scala variable with multiple types

There is Either in scala which allow a variable to have 2 types value.
val x: Either[String, Int] = Left("apple")
However, I want to have more than 2 types for variable x e.g. {String, Int, Double, List[String] }.
e.g. val x:[type can be either String, Int, Double or List[String]]
//So that I can store either String, Int, Double, List[String] value in x.
Is there any way to achieve this?
IMO the most natural way to express this is to create an ADT (Algebraic Data Type):
sealed trait Foo
final case class Bar(s: String) extends Foo
final case class Baz(i: Int) extends Foo
final case class Fizz(d: Double) extends Foo
final case class Buzz(l: List[String]) extends Foo
And now you can pattern match on Foo:
val f: Foo = ???
f match {
case Bar(s) => // String
case Baz(i) => // Int
case Fizz(d) => // Double
case Buzz(l) => // List[String]
}
Look at shapeless coproducts
"shapeless has a Coproduct type, a generalization of Scala's Either to
an arbitrary number of choices"
Not sure what your exact use case is but check this out: scala type alias - how to have a type that represent multiple data types
Basically you create a trait represents an umbrella/parent class. Then you define multiple classes that extend the parent trait and put the parent trait as the expected parameter type in your function. Then you can pass any type that extends that parent trait.
You have to wrap the subtypes, but this allows you to have one type that represents multiple types.
Another way could be to use generic types and type bounds. These articles talk about these two subjects:
http://docs.scala-lang.org/tutorials/tour/variances.html
How do I setup multiple type bounds in Scala?
I'm still learning Scala, but hope this helps! :)

How can I write a function have a polymorphic return type based on the type argument of its type parameter?

I have some code like this:
sealed trait Foo[A] {
def value: A
}
case class StringFoo(value: String) extends Foo[String]
case class IntFoo(value: Int) extends Foo[Int]
I'd like to have a function which can use the A type given a subtype's type parameter.
// Hypothetical invocation
val i: Int = dostuff[IntFoo](param)
val s: String = dostuff[StringFoo](param)
I can't figure out how to declare dostuff in a way that works. The closest thing I can figure out is
def dostuff[B <: Foo[A]](p: Param): A
But that doesn't work because A is undefined in that position. I can do something like
def dostuff[A, B <: Foo[A]](p: Param): A
But then I have to invoke it like dostuff[String, StringFoo](param) which is pretty ugly.
It seems like the compiler should have all the information it needs to move A across to the return type, how can I make this work, either in standard scala or with a library. I'm on scala 2.10 currently if that impacts the answer. I'm open to a 2.11-only solution if it's possible there but impossible in 2.10
Another option is to use type members:
sealed trait Foo {
type Value
def value: Value
}
case class StringFoo(value: String) extends Foo { type Value = String }
case class IntFoo(value: Int) extends Foo { type Value = Int }
def dostuff[B <: Foo](p: Any): B#Value = ???
// Hypothetical invocation
val i: Int = dostuff[IntFoo](param)
val s: String = dostuff[StringFoo](param)
Note that both solutions mainly work around the syntactic restriction in Scala, that you cannot fix one type parameter of a list and have the compiler infer the other.
As you might know, if you have a parameter of type Foo[A], then you can make the method generic in just A:
def dostuff[A](p: Foo[A]): A = ???
Since that might not always be the case, we can try to use an implicit parameter that can express the relationship between A and B. Since we can't only apply some of the generic parameters to a method call (generic parameter inference is all or nothing), we have to split this into 2 calls. This is an option:
case class Stuff[B <: Foo[_]]() {
def get[A](p: Param)(implicit ev: B => Foo[A]): A = ???
}
You can check the types in the REPL:
:t Stuff[IntFoo].get(new Param) //Int
:t Stuff[StringFoo].get(new Param) //String
Another option along the same lines, but using an anonymous class, is:
def stuff[B <: Foo[_]] = new {
def apply[A](p: Param)(implicit ev: B <:< Foo[A]): A = ???
}
:t stuff[IntFoo](new Param) //Int
Here, I've used apply in stead of get, so you can apply the method more naturally. Also, as suggested in your comment, here I've used <:< in the evidence type. For those looking to learn more about this type of generalized type constraint, you can read more here.
You might also consider using abstract type members instead of generic parameters here. When struggling with generic type inference, this often provides an elegant solution. You can read more about abstract type members and their relationship to generics here.

How do I use a structural type with generic parameters?

I have two case classes
case class StringCaseClass(argument: String)
case class IntCaseClass(argument: Int)
I want to define a structural type which will match the companion object of both of these
type HasApply1 {
def apply[A, R](argument: A): R
}
This will compile fine, but when I try to use it like this
def method(caseClass: HasApply1) {
// whatever
}
method(StringCaseClass)
I will get a compiler error
found : StringCaseClass.type
required: WithApply1
(which expands to) AnyRef{def apply[A, R](string: A): R}
Is there any way of accomplishing this? If I redefine the structural type to have concrete types for A and R it will compile correctly, but then I lose the flexiblity
#aloiscochard's comment is almost there. What he forgot to mention is that case class companion objects already implement the appropriate FunctionN trait, so you can simply do this,
scala> case class StringCaseClass(argument: String)
defined class StringCaseClass
scala> case class IntCaseClass(argument: Int)
defined class IntCaseClass
scala> def method[A, R](caseClass: A => R, a: A) = caseClass(a)
method: [A, R](caseClass: A => R, a: A)R
scala> method(StringCaseClass, "foo")
res0: StringCaseClass = StringCaseClass(foo)
scala> method(IntCaseClass, 23)
res1: IntCaseClass = IntCaseClass(23)
In general you should avoid structural typing as it's very expensive. The call will be converted into a reflection call because of limitations in the JVM. When you start using scala 2.10 structural types will result in a warning at compile time (though you could disable that using a flag).
If you're looking into a more general way to add functionality to classes that don't share an inheritance hierarchy you could use Type Classes.
Here's a quick example:
trait CanCreateRFromA[A,R]{
def createNew(a:A): R
}
implicit object CanCreateBlahFromInt extends CanCreateRFromA[Int,Blah2]{
def createNew(i:Int):Blah2 = new Blah2(i)
}
implicit object CanCreateBlah1FromString extends CanCreateRFromA[String,Blah1]{
def createNew(s:String):Blah1 = new Blah1(s)
}
case class Blah1(something:String)
case class Blah2(something:Int)
def createRFromA[A,R](a:A)(implicit tc:CanCreateRFromA[A,R])= tc.createNew(a)
Then you can call:
createRFromA(1) // This gives a Blah2
createRFromA("1") // This gives a Blah1
Again I'm not sure what you're trying to accomplish, but it probably is possible to do what you want with a type class and it will be much faster.
You didn't pass an instance of StringCaseClass to your method. What you passed there is companion object of StringCaseClass (which is automatically generated for case classes).
Try if this works: method(StringCaseClass("dummy")).

Is it possible to refer to the types of Scala case class constructor arguments?

My goal is to create a trait that a case class can extend, which can process each constructor argument and then pass them as arguments to a method of the case class. All of the constructor arguments will be the same type with different type parameters, and the method will take arguments that match the type parameter of each constructor argument. You can think of it as a pre-processor of the constructor arguments. For example,
case class Foo(input1:Input[Int], input2:Input[String]) extends MagicTrait {
def run(input1:Int, input2:String) { ... }
}
Is this at all feasible? Is it feasible in a way that isn't terribly ugly (e.g. all reflection)? Is it possible to refer to the companion object of a case class in a way that is at all generic across case classes (e.g. a function that takes the output of Companion.unapply())?
Seeing as an acceptable solution allows the preprocessing functionality to be moved off the instances to an associated object the main remaining difficulty is that you want to be able to abstract over the arity and types (ie. the shape) of your case class constructors. This is possible with the HList implementation and polymorphic function values from shapeless.
First some preliminaries,
import shapeless.HList._
import shapeless.Functions._
import shapeless.Poly._
import shapeless.TypeOperators._
// Implementation of your Input wrapper
case class Input[T](value: T)
// Value extractor as a shapeless polymorphic function value
object value extends (Input ~> Id) {
def default[T](i : Input[T]) = i.value
}
We can now define a preprocessor base class which provides an apply method which takes an HList of Input types, maps the value polymorphic function across it (ie. performing the preprocessing) and then passes the resulting HList of non-Input types to the provided case class constructor (which is given in hlisted form, see below),
// Pre-processer base class
abstract class Preprocessor[In <: HList, Out <: HList, R](ctor : Out => R)
(implicit mapper : MapperAux[value.type, In, Out]) {
def apply(in : In) = ctor(in map value)
}
Now we define the case class with the post-processing component types,
case class Foo(input1 : Int, input2 : String)
and add one line of boilerplate,
object FooBuilder extends Preprocessor((Foo.apply _).hlisted)
(here the Foo companion object factory method is provided as the Preprocessor constructor argument in HListed form as required above.)
Now we can construct Foo instances using the FooBuilder.
val foo = FooBuilder(Input(23) :: Input("foo") :: HNil)
Unfortunately it isn't (currently) possible to combine the FooBuilder object with the Foo companion object: if you attempt to have the Foo companion extend Preprocessor you'll discover that the Foo factory method isn't available to be passed as the Preprocessor constructor argument.
To illustrate that this solution is really abstracting over type and arity, here's how we might add a second differently shaped case class,
case class Bar(input1 : Int, input2 : String, input3 : Boolean)
object BarBuilder extends Preprocessor((Bar.apply _).hlisted)
val bar = BarBuilder(Input(23) :: Input("foo") :: Input(true) :: HNil)
case class Input[T](value: T)
trait MagicTrait[T,U] {
val input1: Input[T]
val input2: Input[U]
def run: Unit
}
case class Foo(input1: Input[Int], input2: Input[String])
extends MagicTrait[Int, String] {
def run = println(input1.value * 2 + input2.value.toUpperCase)
}
scala> val m: MagicTrait[_,_] = Foo(Input(3), Input("hi"))
m: MagicTrait[_, _] = Foo(Input(3),Input(hi))
scala> m.run
6HI
edit:
If you want to find the types of the class parameters you can use the fact that case classes extend Product:
scala> Foo(2, "hi").productIterator.map(_.asInstanceOf[AnyRef].getClass).toList
res13: List[java.lang.Class[_]] =
List(class java.lang.Integer, class java.lang.String)
But this uses the reflection you wanted to avoid. This is why we use parameterization.
If you want to return its companion object, I'm not sure you can do this in a useful, type-safe way in the context of case classes, because companion objects don't extend an interface that specifies their extractor methods. You might be able to do something with structural types but it's probable that there's a better way to approach whatever problem it is that you're trying to solve.