Higher order operations with flattened tuples in scala - scala

I've recently come across a problem. I'm trying to flatten "tail-nested" tuples in a compiler-friendly way, and I've come up with the code below:
implicit def FS[T](x: T): List[T] = List(x)
implicit def flatten[T,V](x: (T,V))(implicit ft: T=>List[T], fv: V=>List[T]) =
ft(x._1) ++ fv(x._2)
This above code works well for flattening tuples I am calling "tail-nested" like the ones below.
flatten((1,2)) -> List(1,2)
flatten((1,(2,3))) -> List(1,2,3)
flatten((1,(2,(3,4)))) -> List(1,2,3,4)
However, I seek to make my solution more robust. Consider a case where I have a list of these higher-kinded "tail-nested" tuples.
val l = List( (1,2), (1,(2,3)), (1,(2,(3,4))) )
The inferred type signature of this would be List[(Int, Any)] and this poses a problem for an operation such as map, which would fail with:
error: No implicit view available from Any => List[Int]
This error makes sense to me because of the nature of my recursive implicit chain in the flatten function. However, I was wondering: is there any way I can make my method of flattening the tuples more robust so that higher order functions such as map mesh well with it?
EDIT:
As Bask.ws pointed out, the Product trait offers potential for a nice solution. The below code illustrates this:
def flatten(p: Product): List[_] = p.productIterator.toList.flatMap {x => x match {
case pr: Product => flatten(pr)
case _ => List(x)
}}
The result type of this new flatten call is always List[Any]. My problem would be solved if there was a way to have the compiler tighten this bound a bit. In parallel to my original question, does anyone know if it is possible to accomplish this?

UPD Compile-time fail solution added
I have one solution that may suit you. Types of your first 3 examples are resolved in compile time: Int, Tuple2[Int, Int], Tuple2[Int, Tuple2[Int, Int]]. For you example with the list you have heterogeneous list with actual type List[(Int, Any)] and you have to resolve the second type in runtime or it maybe can be done by macro. So you may want to actually write implicit def flatten[T](x: (T,Any)) as your error advises you
Here is the fast solution. It gives a couple of warnings, but it works nicely:
implicit def FS[T](x: T): List[T] = List(x)
implicit def FP[T](x: Product): List[T] = {
val res = (0 until x.productArity).map(i => x.productElement(i) match {
case p: Product => FP[T](p)
case e: T => FS(e)
case _ => sys.error("incorrect element")
})
res.toList.flatten
}
implicit def flatten[T](x: (T,Any))(implicit ft: T=>List[T], fp: Product =>List[T]) =
ft(x._1) ++ (x._2 match {
case p: Product => fp(p)
case t: T => ft(t)
})
val l = List( (1,2), (1,(2,3)), (1,(2,(3,4))) )
scala> l.map(_.flatten)
res0: List[List[Int]] = List(List(1, 2), List(1, 2, 3), List(1, 2, 3, 4))
UPD
I have researched problem a little bit more, and I have found simple solution to make homogeneus list, which can fail at compile time. It is fully typed without Any and match and looks like compiler now correctly resolves nested implicits
case class InfiniteTuple[T](head: T, tail: Option[InfiniteTuple[T]] = None) {
def flatten: List[T] = head +: tail.map(_.flatten).getOrElse(Nil)
}
implicit def toInfiniteTuple[T](x: T): InfiniteTuple[T] = InfiniteTuple(x)
implicit def toInfiniteTuple2[T, V](x: (T, V))(implicit ft: V => InfiniteTuple[T]): InfiniteTuple[T] =
InfiniteTuple(x._1, Some(ft(x._2)))
def l: List[InfiniteTuple[Int]] = List( (1,2), (1,(2,3)), (1,(2,(3,4)))) //OK
def c: List[InfiniteTuple[Int]] = List( (1,2), (1,(2,3)), (1,(2,(3,"44"))))
//Compile-time error
//<console>:11: error: No implicit view available from (Int, (Int, java.lang.String)) => InfiniteTuple[Int]
Then you can implement any flatten you want. For example, one above:
scala> l.map(_.flatten)
res0: List[List[Int]] = List(List(1, 2), List(1, 2, 3), List(1, 2, 3, 4))

Related

Is there a way in Scala to convert Seq[(String, String)] To Seq[String]?

I'm new to Scala and really coding in general so sorry if this question is a bit basic but would love some help!
I currently have a function which yields Seq[(String, String)] which I would then like to convert to Seq[String].
Depending on the iteration of the function, the dimension of the vector will be different so some outcomes are Seq[(String, String)], others are Seq[(String, String, String)] etc. So ideally I could use the same function for each output.
I have tried using .flatten, and split[","] and am not sure what other methods I could use.
Fr one sample value, output of the function looks like this:
res17: Seq[(String, String)] = Vector((u, v),(w, x))
The ideal result would be Vector(u,v,w,x)
Consider shapeless approach which provides a way of abstracting over arity in type-safe manner:
import shapeless._
import shapeless.ops.hlist
import syntax.std.tuple._
def flattenTupleN[P <: Product, L <: HList](
ps: List[P]
)(implicit gen: Generic.Aux[P, L],
toT: hlist.ToTraversable.Aux[L, List, String]
): List[String] = {
ps.flatMap(p => gen.to(p).toList)
}
val t1: List[(String, String)] = List(("u", "v"), ("w", "x"))
val t2: List[(String, String, String)] = List(("a", "b", "c"), ("d", "e", "f"))
val t3: List[(Int, Double, String)] = List((42, 3.14, "Picard"))
flattenTupleN(t1)
flattenTupleN(t2)
// flattenTupleN(t3) // compile-time error
which outputs
res0: List[String] = List(u, v, w, x)
res1: List[String] = List(a, b, c, d, e, f)
Here is an unsafe but out-of-the-box approach:
def flattenTupleNUnsafe(ps: List[Product]): List[String] =
ps.flatMap(_.productIterator.map(_.toString))
flattenTupleNUnsafe(t1)
flattenTupleNUnsafe(t2)
flattenTupleNUnsafe(t3) // compiles OK but is a bad idea!
which outputs
res2: List[String] = List(u, v, w, x)
res3: List[String] = List(a, b, c, d, e, f)
res4: List[String] = List(42, 3.14, Picard)
Note how due to loss of type-safety flattenTupleNUnsafe(t3) compiles just fine and introduces a logic error in the program.
I agree with those people who are asking for more information about your use case. Without knowing more about the data you are trying to transform, it is difficult to recommend a concise solution that addresses your needs.
Please consider what I have to say in a provisional light, as I do not know enough about what you are trying to do.
You say that the tuple you are outputting can vary. Have you considered converting everything from Tuple to Vector?
So instead of
res17: Vector[(String, String)] = Vector((u, v),(w, x))
use
res17: Vector[Vector[String]] = Vector(Vector(u, v),Vector(w, x))
You can then easily transform from the Vector of Vector to a single Vector with a call to flatMap or flatten, which based on your question, it sounds like you already know how to do.
Another impression I get from reading your question is that if you want to keep a fixed length argument list that you transform from that argument list to a Vector, you might want to look at case classes instead of Tuple.
So instead of Tuple2 or Tuple3, define an inheritance hierarchy that allows the compiler to type check your program.
Something like this:
trait MyData
case class 2MemberData(data1: String, data2: String) extends MyData
case class 3MemberData(data1: String, data2: String, data3: String) extends MyData
case class 4MemberData(data1: String, data2: String, data3: String, data4: String) extends My Data
That way your function can output a value of type Vector[MyData] that you then flatten using pattern matching. So something like
def processData: Vector[MyData]
def cleanUp(input: Vector[MyData]): Vector[String]
where cleanUp is implemented as follows:
def cleanUp(input: Vector[MyData]): Vector[String] = {
input.flatMap{ d =>
d match {
case 2MemberData(data1, data2) => Vector(data1, data2)
case 3MemberData(data1, data2, data3) => Vector(data1, data2, data3)
case 4MemberData(data1, data2, data3, data4) => Vector(data1, data2, data3, data4)
}
}
}
I'm just throwing ideas out there and don't know if what I am saying is helpful or not. It really depends on what the surrounding code looks like. If you have any questions, feel free to ask.

Extractors in for vs extractors in assignment

I have this doubt with extractors.
If I can do this:
val a :: b = List(1, 2, 3)
why I cannot do this:
val c = for ( a :: b <- List(1, 2, 3) } yield a
The translation of a for-yield expression in Scala (without using guards) is the map function. If you try and use map instead, it'll seem a little clear as to the quirk with the code:
List(1, 2, 3).map((x: Int) => ???)
When you map over a List[+A], you project each value, one at a time. You don't have the entire list at your disposable inside the higher order function.
On the contrary, when using pattern matching on the list itself, the compiler will translate your first example into (after some cleanup):
def main(args: Array[String]): Unit = {
private[this] val x$1: (Int, List[Int]) = List(1, 2, 3) match {
case (head: Int, tl: List[Int])scala.collection.immutable.::[Int]((a # _), (b # _)) => Tuple2[Int, List[Int]](a, b)
};
val a: Int = x$1._1;
val b: List[Int] = x$1._2;
()
Which is a pattern match on the List[Int], matching the case of a head and tail. It's simply syntatic sugar for regular pattern matching, same as for-yield is syntatic sugar for a map. They simply do different things.
After Yuval answer, I've understood why it doesn't work.
Equivalent syntax if interested in 1st value:
val first = for (a <- List(1, 2, 3).headOption) yield a

Why is flatMap on a Vector[Option[Int]] whose mapper function result is not a Vector[Option[Int]] valid?

For example,
Vector(Some(1), Some(2), Some(3), None).flatMap{
n => n
}
produces a Vector(1, 2, 3) instead of giving an error. As I have seen in other languages, flatMap is used when you have a mapper function that produces nesting so I would expect this to be a valid flatMap:
Vector(1, 2, 3).flatMap{
eachNum => Vector(eachNum)
}
My mapper function produces a Vector which would cause nesting (i.e. Vector(Vector(1), Vector(2), Vector(3), Vector(4))) if I used a map due to the container wrapping. However, flatMap will remove this nesting and flatten it. This makes sense when there is nesting of two identical monads.
However, I do not understand how using a flatMap with a mapper function that returns an Option makes a Vector[Option[Int]] become a Vector[Int]. Is there some sort of transformation going on (I have never seen this before), could someone explain and perhaps point me to some resources?
Thank you very much
we can use reify to see what is going on:
scala> import reflect.runtime.universe._
import reflect.runtime.universe._
scala> val v = Vector(Some(1), Some(2), Some(3), None)
v: scala.collection.immutable.Vector[Option[Int]] = Vector(Some(1), Some(2), Some(3), None)
scala> reify { v.flatMap(x => x) }
res0: reflect.runtime.universe.Expr[scala.collection.immutable.Vector[Int]] =
Expr[scala.collection.immutable.Vector[Int]]($read.v.flatMap(((x) =>
Option.option2Iterable(x)))(Vector.canBuildFrom))
This is showing us that it is using the option2Iterable conversion to convert the Option to Iterable, and Iterable is a subtype of the GenTraversableOnce type that flatMap is expecting.
The function f passed within the flatMap here:
Vector(Some(1), Some(2), Some(3), None).flatMap{
n => n
}
Is a function A => GenTraversableOnce[B] as described in the flatMap implementation:
def flatMap[B, That](f : scala.Function1[A, GenTraversableOnce[B]])
(implicit bf : CanBuildFrom[Repr, B, That])
: That = ???
The function implemented in your example n => n is:
(n: Option[Int]) => n
Where A is Option[Int] and B is Int.
Because CanBuildFrom is defined as trait CanBuildFrom[-From, -Elem, +To]:
From is Repr, and in this case Vector
Elem is B so Int
The result of flatMap is therefore Vector[Int]

Generic strongly-typed scala method to retrieve item of a particular type from a collection

Imagine I have this class:
class Example {
val list = List(new Apple(), new Orange(), Banana());
def getIfPresent[T <: Fruit] : Option[T] = list.collectFirst { case x : T => x }
}
You use it like this:
val example = new Example();
match example.getIfPresent[Apple] {
case Some(apple) => apple.someAppleSpecificMethod();
case None => println("No apple");
}
Now, of course, this doesn't work in the JVM, because of type erasure. getIfPresent just matches on the type Fruit in the collectFirst partial function, instead of the actual type specified in the call.
I have tried to get my head around type tags and class tags, and really have no idea how I would implement the above method. The examples that I see are trying to do very different things. How could I achieve a method that does what I want, either with TypeTags or some other mechanism I'm unaware of?
Edit: m-z's answer below is the full solution, but here is how it looks with my example code:
class Example {
val list = List(new Apple(), new Orange(), Banana());
def getIfPresent[T <: Fruit : ClassTag] : Option[T] = list.collectFirst { case x : T => x }
}
Just needed to add : ClassTag!
You can do this using ClassTag, to an extent.
import scala.reflect.ClassTag
// Modify to apply whatever type bounds you find necessary
// Requires Scala ~2.11.5 or greater (not sure of the exact version, but 2.11.1 does not work, and 2.11.5 does)
def findFirst[A : ClassTag](list: List[Any]): Option[A] =
list collectFirst { case a: A => a }
val l = List(1, "a", false, List(1, 2, 3), List("a", "b"))
scala> findFirst[Boolean](l)
res22: Option[Boolean] = Some(false)
scala> findFirst[Long](l)
res23: Option[Long] = None
But there are some caveats with ClassTag, in that it will only match the class, and not the type:
scala> findFirst[List[String]](l)
res24: Option[List[String]] = Some(List(1, 2, 3)) // No!
You can use a TypeTag to get around this, but it won't work with a List[Any]. Here is one possible (sort of ugly) trick:
import scala.reflect.runtime.universe.{typeOf, TypeTag}
case class Tagged[A : TypeTag](a: A) {
def tpe = typeOf[A]
}
implicit class AnyTagged[A : TypeTag](a: A) {
def tag = Tagged(a)
}
def findFirst[A : TypeTag](list: List[Tagged[_]]): Option[A] =
list collectFirst { case tag # Tagged(a) if(tag.tpe =:= typeOf[A]) => a.asInstanceOf[A] }
The only way I can think of to hold onto the TypeTag of each element is to literally hold onto it with a wrapper class. So I have to construct the list like this:
val l = List(1.tag, "a".tag, false.tag, List(1, 2, 3).tag, List("a", "b").tag)
But it works:
scala> findFirst[List[String]](l)
res26: Option[List[String]] = Some(List(a, b))
There may be a more elegant way to construct such a list with TypeTags.
For fun, you can also try to do this with shapeless using an HList and select. The difference is that instead of returning Option[A], select will return A (the type you want), but if the HList contains no A, it won't compile.
import shapeless._
val l = 1 :: "a" :: false :: List(1, 2, 3) :: List("a", "b") :: HNil
scala> l.select[Boolean]
res0: Boolean = false
scala> l.select[Boolean]
res1: Boolean = false
scala> l.select[List[String]]
res2: List[String] = List(a, b)
scala> l.select[Long]
<console>:12: error: Implicit not found: shapeless.Ops.Selector[shapeless.::[Int,shapeless.::[String,shapeless.::[Boolean,shapeless.::[List[Int],shapeless.::[List[String],shapeless.HNil]]]]], Long]. You requested an element of type Long, but there is none in the HList shapeless.::[Int,shapeless.::[String,shapeless.::[Boolean,shapeless.::[List[Int],shapeless.::[List[String],shapeless.HNil]]]]].
l.select[Long]
^

How to read the class of a Scala object extending Any but not AnyRef?

I have an heterogeneous List like the following one:
val l = List(1, "One", true)
and I need to filter its objects by extracting only the ones belonging to a given Class. For this purpose I wrote a very simple method like this:
def filterByClass[A](l: List[_], c: Class[A]) =
l filter (_.asInstanceOf[AnyRef].getClass() == c)
Note that I am obliged to add the explicit conversion to AnyRef in order to avoid this compilation problem:
error: type mismatch;
found : _$1 where type _$1
required: ?{val getClass(): ?}
Note that implicit conversions are not applicable because they are ambiguous:
both method any2stringadd in object Predef of type (x: Any)scala.runtime.StringAdd
and method any2ArrowAssoc in object Predef of type [A](x: A)ArrowAssoc[A]
are possible conversion functions from _$1 to ?{val getClass(): ?}
l filter (_.getClass() == c)
However in this way the invocation of:
filterByClass(l, classOf[String])
returns as expected:
List(One)
but of course the same doesn't work, for example, with Int since they extends Any but not AnyRef, so by invoking:
filterByClass(l, classOf[Int])
the result is just the empty List.
Is there a way to make my filterByClass method working even with Int, Boolean and all the other classes extending Any?
The collect method already does what you want. For example to collect all Ints in a collection you could write
xs collect { case x: Int => x }
This of course only works when you hardcode the type but as primitives are handled differently from reference types it is actually better to do so. You can make your life easier with some type classes:
case class Collect[A](collect: PartialFunction[Any,A])
object Collect {
implicit val collectInt: Collect[Int] = Collect[Int]({case x: Int => x})
// repeat for other primitives
// for types that extend AnyRef
implicit def collectAnyRef[A <: AnyRef](implicit mf: ClassManifest[A]) =
Collect[A]({ case x if mf.erasure.isInstance(x) => x.asInstanceOf[A] })
}
def collectInstance[A : Collect](xs: List[_ >: A]) =
xs.collect(implicitly[Collect[A]].collect)
Then you can use it without even passing a Class[A] instance:
scala> collectInstance[Int](l)
res5: List[Int] = List(1)
scala> collectInstance[String](l)
res6: List[String] = List(One)
Using isInstanceOf:
scala> val l = List(1, "One", 2)
l: List[Any] = List(1, One, 2)
scala> l . filter(_.isInstanceOf[String])
res1: List[Any] = List(One)
scala> l . filter(_.isInstanceOf[Int])
res2: List[Any] = List(1, 2)
edit:
As the OP requested, here's another version that moves the check in a method. I Couldn't find a way to use isInstanceOf and so I changed the implementation to use a ClassManifest:
def filterByClass[A](l: List[_])(implicit mf: ClassManifest[A]) =
l.filter(mf.erasure.isInstance(_))
Some usage scenarios:
scala> filterByClass[String](l)
res5: List[Any] = List(One)
scala> filterByClass[java.lang.Integer](l)
res6: List[Any] = List(1, 2)
scala> filterByClass[Int](l)
res7: List[Any] = List()
As can be seen above, this solution doesn't work with Scala's Int type.
The class of an element in a List[Any] is never classOf[Int], so this is behaving as expected. Your assumptions apparently leave this unexpected, but it's hard to give you a better way because the right way is "don't do that."
What do you think can be said about the classes of the members of a heterogenous list? Maybe this is illustrative. I'm curious how you think java does it better.
scala> def f[T: Manifest](xs: List[T]) = println(manifest[T] + ", " + manifest[T].erasure)
f: [T](xs: List[T])(implicit evidence$1: Manifest[T])Unit
scala> f(List(1))
Int, int
scala> f(List(1, true))
AnyVal, class java.lang.Object
scala> f(List(1, "One", true))
Any, class java.lang.Object
This worked for me. Is this what you want?
scala> val l = List(1, "One", true)
l: List[Any] = List(1, One, true)
scala> l filter { case x: String => true; case _ => false }
res0: List[Any] = List(One)
scala> l filter { case x: Int => true; case _ => false }
res1: List[Any] = List(1)
scala> l filter { case x: Boolean => true; case _ => false }
res2: List[Any] = List(true)
Despite my solution could be less elegant than this one I find mine quicker and easier. I just defined a method like this:
private def normalizeClass(c: Class[_]): Class[_] =
if (classOf[AnyRef].isAssignableFrom((c))) c
else if (c == classOf[Int]) classOf[java.lang.Integer]
// Add all other primitive types
else classOf[java.lang.Boolean]
So by using it in my former filterByClass method as it follows:
def filterByClass[A](l: List[_], c: Class[A]) =
l filter (normalizeClass(c).isInstance(_))
the invocation of:
filterByClass(List(1, "One", false), classOf[Int])
just returns
List(1)
as expected.
At the end, this problem reduces to find a map between a primitive and the corresponding boxed type.
Maybe a help can arrive from scala.reflect.Invocation (not included in the final version of 2.8.0), the getAnyValClass function in particular (here slightly edited)
def getAnyValClass(x: Any): java.lang.Class[_] = x match {
case _: Byte => classOf[Byte]
case _: Short => classOf[Short]
case _: Int => classOf[Int]
case _: Long => classOf[Long]
case _: Float => classOf[Float]
case _: Double => classOf[Double]
case _: Char => classOf[Char]
case _: Boolean => classOf[Boolean]
case _: Unit => classOf[Unit]
case x#_ => x.asInstanceOf[AnyRef].getClass
}
With this function the filter is as easy as
def filterByClass[T: Manifest](l:List[Any]) = {
l filter (getAnyValClass(_) == manifest[T].erasure)
}
and the invocation is:
filterByClass[Int](List(1,"one",true))