I'm looking for a way to treat an Any as an Array or Seq and iterate over it, if possible.
Currently I have some code that looks like this, taking a sequence of Any's and flattening out any Traversable or Array objects contained.
def flattenAsStrings(as: Seq[Any]): Seq[String] = {
val (travValued, other) = as.partition(a => classOf[Traversable[_]] isAssignableFrom(a.getClass))
val (arrayValued, singleValued) = other.partition(a => a.isInstanceOf[Array[_]])
val travStrings = travValued.map(_.asInstanceOf[Traversable[_]].map(_.toString)).flatMap(_.toList)
val arrayStrings = arrayValued.map(_.asInstanceOf[Array[_]].map(_.toString)).flatMap(_.toList)
singleValued.map(_.toString) ++ travStrings ++ arrayStrings
}
It feels like there mustr be a simpler way to do this in Scala, given implicit conversions and whatnot. Anyone?
Basically you want to force each element to a Seq, and then flatten them all at once. Array has an implicit conversion to Seq and both Seq and Traversable have a .toSeq method. So we can do:
val t: Traversable[Int] = List(1, 2, 3)
val a: Array[Int] = Array(4, 5, 6)
val other = "whatever"
val as: Seq[Any] = List(t, a, other)
as.flatMap{
case t: Traversable[_] => t.toSeq
case a: Array[_] => a.toSeq
case other => Seq(other)
}.map{_.toString}
//Seq[java.lang.String] = List(1, 2, 3, 4, 5, 6, whatever)
(as an aside, this is pretty ugly Scala code, you might want to consider refactoring things to get rid of using a Seq[Any] in the first place)
Related
I am a bit confused with documentation about this function/with examples of use provided. Does flatten can happen only once? like
List(List(1, 2), List(3, List(4, 5, 6))) -> List(1, 2, 3, List(4, 5, 6))
Or you somehow specify the depth of flattening, so that it could become a List(1, 2, 3, 4, 5, 6) ?
Because, for example, in JS it seems that function can flat() whatever depth you want into a 1D array. Can Scala flatten do that or it's capable of only elevating once?
I am trying to recreate that function by myself and want to mimic the required behavior and understand the reason why it may work differently.
As mentioned in the comments defining such a method in Scala 2 would not be straightforward to do in a typesafe manner. First of all, recursive types are not supported directly in Scala 2, so you'd have to operate on List[Any] and use runtime reflection to distinguish if the element is a list or an integer.
Recently released Scala 3 has many improvements in its type system, so I wondered that maybe it would be possible to implement such a method there? I tried and I think I was able to achieve usable implementation.
First of all, I wondered if it would be possible to implement recursive union type (a similar thing is possible in typescript):
type ListOr[A] = A | List[ListOr[A]]
unfortunately, such type was raising compiler error:
illegal cyclic type reference: alias ... of type ListOr refers back to the type itself
That was disappointing, but after some digging, I found that I could define such recursive type as:
type ListOr[A] = A match {
case AnyVal => AnyVal | List[ListOr[AnyVal]]
case _ => A | List[ListOr[A]]
}
and it was usable:
val ints: ListOr[Int] = List(List(1), 2, 3, List(List(4, List(5)), 6), 7, 8, List(9))
val strings: ListOr[String] = List(List("A", "B", "C"), List(List("D", List("E")), "F"), "G", "H", List("I"), "J")
so now I needed to just implement the flattening function:
//I needed class tag for A to be able to do a match
def deepFlatten[A: ClassTag](s: ListOr[A]): List[A] =
s match
case a: A => List(a)
case ls: List[_ <: ListOr[A]] => ls.flatMap(deepFlatten(_))
and it seemed to be working correctly:
#main
def main =
val i: List[Int] = deepFlatten[Int](ints) //List(1, 2, 3, 4, 5, 6, 7, 8, 9)
val j: List[String] = deepFlatten[String](strings)//List(A, B, C, D, E, F, G, H, I, J)
Obviously, such implementation could be improved (it's not tail-recursive), but it's doing its job.
Since I'm Scala 3 novice I'm not sure if that's the best implementation, but it's definitely possible to implement such arbitrarily deep flattening functions as type-safe.
Scastie with the solution.
This is an attempt of doing something similar using a simple Tree data structure.
final case class Tree[+A](value: A, children: List[Tree[A]] = Nil)
def flattenTree[A](tree: Tree[A]): List[A] = {
#annotation.tailrec
def loop(remainingNodes: List[Tree[A]], acc: List[A]): List[A] =
remainingNodes match {
case Tree(value, children) :: tail =>
loop(
remainingNodes = children reverse_::: tail,
value :: acc
)
case Nil =>
acc
}
loop(remainingNodes = tree :: Nil, acc = List.empty)
}
Which can be used like this:
val tree = Tree(
value = 1,
children = List(
Tree(value = 2),
Tree(
value = 3,
children = List(
Tree(value = 4),
Tree(value = 5),
Tree(value = 6)
)
)
)
)
val flattenedTree = flattenTree(tree)
println(flattenedTree.mkString("[", ", ", "]"))
Which will produce the following output:
[2, 4, 5, 6, 3, 1]
As you can see is a reversed DFS (whose results are also reversed). If the order doesn't matter this is a straightforward and efficient implementation, if order matters then one can play with the code.
Another approach would be to use a data structure like:
sealed trait ListOr[+A] extends Product with Serializable
final case class NestedList[+A](data: List[ListOr[A]]) extends ListOr[A]
final case class SingleValue[+A](value: A) extends ListOr[A]
You can see the code running here.
I am a newbie to Scala and I am trying to understand collectives. I have a sample Scala code in which a method is defined as follows:
override def write(records: Iterator[Product2[K, V]]): Unit = {...}
From what I understand, this function is passed an argument record which is an Iterator of type Product2[K,V]. Now what I don't understand is this Product2 a user defined class or is it a built in data structure. Moreover how do explore the key-value pair contents of Product2 and how do I iterate over them.
Chances are Product2 is a built-in class and you can easily check it if you're in modern IDE (just hover over it with ctrl pressed), or, by inspecting file header -- if there is no related imports, like some.custom.package.Product2, it's built-in.
What is Product2 and where it's defined? You can easily found out such things by utilizing Scala's ScalaDoc:
In case of build-in class you can treat it like tuple of 2 elements (in fact Tuple2 extends Product2, as you may see below), which has ._1 and ._2 accessor methods.
scala> val x: Product2[String, Int] = ("foo", 1)
// x: Product2[String,Int] = (foo,1)
scala> x._1
// res0: String = foo
scala> x._2
// res1: Int = 1
See How should I think about Scala's Product classes? for more.
Iteration is also hassle free, for example here is the map operation:
scala> val xs: Iterator[Product2[String, Int]] = List("foo" -> 1, "bar" -> 2, "baz" -> 3).iterator
xs: Iterator[Product2[String,Int]] = non-empty iterator
scala> val keys = xs.map(kv => kv._1)
keys: Iterator[String] = non-empty iterator
scala> val keys = xs.map(kv => kv._1).toList
keys: List[String] = List(foo, bar, baz)
scala> xs
res2: Iterator[Product2[String,Int]] = empty iterator
Keep in mind though, that once iterator was consumed, it transitions to empty state and can't be re-used again.
Product2 is just two values of type K and V.
use it like this:
write(List((1, "one"), (2, "two")))
the prototype can also be written like: override def write(records: Iterator[(K, V)]): Unit = {...}
To access values k of type K and v of type V.
override def write(records: Iterator[(K, V)]): Unit = {
records.map{case (k, v) => w(k, v)}
}
I have run into this scenario several times recently:
a class has an immutable (indexed?) sequence member
a factory member method creates a new instance with the sequence somewhat modified
What's an efficient way to do this?
class A( xs: IndexedSeq[Int] ) {
def another: A = {
val ys = xs.toArray.clone() // !!!
ys(7) = 13
new A(ys)
}
}
I do .toArray so that I can modify this sequence in place and .clone because I'm afraid that if the original xs was an array already, toArray will just return this and I will modify the objects (meant-to-be-immutable) values. However, this obviously makes two copies if xs was not an Array and I would really like to avoid that. Obviously, I could just check its type, but that seems very inelegant and I'm not too sure if I'd have to check against other mutable sequences which can wrap an Array. What do?
scala> val xs: Seq[Int] = Array(1, 2, 3)
ss: Seq[Int] = WrappedArray(1, 2, 3)
scala> val ys = xs.toArray
ys: Array[Int] = Array(1, 2, 3)
scala> ys(1) = 22
scala> ys
res1: Array[Int] = Array(1, 22, 3)
scala> xs
res2: Seq[Int] = WrappedArray(1, 22, 3)
If you don't really need mutability, then the best is just to ask for an immutable sequence; that way you don't need to worry about whether changing the data has side effects.
class A(xs: collection.immutable.IndexedSeq[Int]) {
def another: A = {
val ys = xs.updated(7, 13)
new A(ys)
}
}
Just using the basic JDBC interface to read some data using Scala.
In F# (using System.Data.SqlClient namespace) we could do something like this to return an immutable list from the database.
let rs = cmd.ExecuteReader()
[while rs.Read() do yield rs.GetInt32(1)]
In Scala this proves more difficult, as far as I know there is no "while" comprehension like F#. Effectively I'd like to do something close to F# in Scala without having to use mutable vars - if only because they look ugly and add to Lines of Code.
Something like this seems to be commonplace in my Scala code right now:
var result = Seq.empty[Int]
val rs = stmt.executeQuery()
while (rs.next()) {
result = result :+ rs.getInt(1) }
I would create a custom subclass of Iterator that wraps a query result. It's really easy; senia showed how.
But you could also
val rs = stmt.executeQuery
val it = Iterator.continually(if (rs.next()) Some(rs.getInt(1)) else None)
val result = it.takeWhile(_.isDefined).toList.flatten
You could use same way in scala, but I think it's ugly:
class Reader(var i: Int){
def read = { i-=1; i > 0 }
def getInt32 = i
}
val r = new Reader(10)
Stream.from(0).takeWhile{ _ => r.read}.map{ _ => r.getInt32}.toList
// List(9, 8, 7, 6, 5, 4, 3, 2, 1)
Idiomatic scala way is to convert your Reader to an Iterator:
implicit class ReaderIterator(r: Reader) extends Iterator[Int] {
def hasNext = r.read
def next = r.getInt32
}
scala> new Reader(10).toList
res0: List[Int] = List(9, 8, 7, 6, 5, 4, 3, 2, 1)
But if you are really missing this syntax you could add it:
import scala.collection.immutable.VectorBuilder
class FWhile(c: => Boolean){
def apply[T](e: => T): Seq[T] = {
val b = new VectorBuilder[T]
while (c) b += e
b.result
}
}
object FWhile{
def apply(c: => Boolean) = new FWhile(c)
}
scala> FWhile(r.read){r.getInt32}
res0: Seq[Int] = Vector(9, 8, 7, 6, 5, 4, 3, 2, 1)
You could use an implicit class together with an implicit CanBuildFrom. This does use a mutable builder, but not at the caller's side:
object MyResultSetContainer {
implicit class MyResultSet(rs: ResultSet) {
def map[T, C <: Iterable[T]](f: (ResultSet) => T)
(implicit cbf: CanBuildFrom[Nothing, T, C]): C = {
val builder = cbf()
while (rs.next()) {
builder += f(rs)
}
builder.result()
}
}
}
to be used like this:
import MyResultSetContainer._
val rs = stmnt.executeQuery("select * from pg_user")
val names = for (row <- rs) yield (row.getString(1))
println(names)
rs.close()
The for comprehension uses map under the hood, so if you prefer map directly:
val names = rs.map(row => row.getString(1))
which produces a sequence. Thanks to CanBuildFrom you can produce other collections as well by providing a type explicitly:
val names: List[String] = rs.map(row => row.getString(1))
How does CanBuildFrom work? The Scala compiler looks at the types involved in this expression: There is the resulting type, and the type returned by the function called by map. Based on this information, the Scala compiler provides a factory implicitly which can be used to create a suitable builder. So you need only one method to produce different types of collections.
If you want to return multiple values, just return a tuple:
val columns = rs.map(row => (row.getInt(2), row.getString(1)))
and the tuple can be used to create a Map directly:
val keyNamesMap: Map[Int, String] =
rs.map(row => (row.getInt(2), row.getString(1)))
This is based on the idea that a result set is a list of rows, and so the map function should be available on top of it. The implicit class is used to add the map method to the underlying result set implicitly.
I have a map using the multimap trait, like so
val multiMap = new HashMap[Foo, Set[Bar]] with MultiMap[Foo, Bar]
I would like to combine filtering this map on specific values
multiMap.values.filter(bar => barCondition)
with flattening the matching results into a list of tuples of the form
val fooBarPairs: List[(Foo, Bar)]
What would be the idiomatic way of doing this? I was hoping that Scala might provide something like an anamorphism to do this without looping, but as a complete newbie I am not sure what my options are.
Here's an example:
import collection.mutable.{HashMap, MultiMap, Set}
val mm = new HashMap[String, Set[Int]] with MultiMap[String, Int]
mm.addBinding("One", 1).addBinding("One",11).addBinding("Two",22).
addBinding("Two",222)
// mm.type = Map(Two -> Set(22, 222), One -> Set(1, 11))
I think the easiest way to get what you want is to use a for-expression:
for {
(str, xs) <- mm.toSeq
x <- xs
if x > 10
} yield (str, x) // = ArrayBuffer((Two,222), (Two,22), (One,11))
You need the .toSeq or the output type will be a Map, which would mean each mapping is overidden by subsequent elements. Use toList on this output if you need a List specifically.
Here is an example of what I think you want to do:
scala> mm
res21: scala.collection.mutable.HashMap[String,scala.collection.mutable.Set[Int]] with scala.collection.mutable.MultiMap[String,Int]
= Map(two -> Set(6, 4, 5), one -> Set(2, 1, 3))
scala> mm.toList.flatMap(pair =>
pair._2.toList.flatMap(bar =>
if (bar%2==0)
Some((pair._1, bar))
else
None))
res22: List[(String, Int)] = List((two,6), (two,4), (one,2))
Here is another, slightly more concise solution:
import collection.mutable.{HashMap, MultiMap, Set}
val mm = new HashMap[String, Set[Int]] with MultiMap[String, Int]
val f = (i: Int) => i > 10
mm.addBinding("One", 1)
.addBinding("One",11)
.addBinding("Two",22)
.addBinding("Two",222)
/* Map(Two -> Set(22, 222), One -> Set(1, 11)) */
mm.map{case (k, vs) => vs.filter(f).map((k, _))}.flatten
/* ArrayBuffer((Two,222), (Two,22), (One,11)) */