I need to implement a generic method that takes a tuple and returns a Map
Example :
val tuple=((1,2),(("A","B"),("C",3)),4)
I have been trying to break this tuple into a list :
val list=tuple.productIterator.toList
Scala>list: List[Any] = List((1,2), ((A,B),(C,3)), 4)
But this way returns List[Any] .
I am trying now to find out how to iterate over the following tuple ,for example :
((1,2),(("A","B"),("C",3)),4)
in order to loop over each element 1,2,"A",B",...etc. How could I do this kind of iteration over the tuple
What about? :
def flatProduct(t: Product): Iterator[Any] = t.productIterator.flatMap {
case p: Product => flatProduct(p)
case x => Iterator(x)
}
val tuple = ((1,2),(("A","B"),("C",3)),4)
flatProduct(tuple).mkString(",") // 1,2,A,B,C,3,4
Ok, the Any-problem remains. At least that´s due to the return type of productIterator.
Instead of tuples, use Shapeless data structures like HList. You can have generic processing, and also don't lose type information.
The only problem is that documentation isn't very comprehensive.
tuple.productIterator map {
case (a,b) => println(a,b)
case (a) => println(a)
}
This works for me. tranform is a tuple consists of dataframes
def apply_function(a: DataFrame) = a.write.format("parquet").save("..." + a + ".parquet")
transform.productIterator.map(_.asInstanceOf[DataFrame]).foreach(a => apply_function(a))
Related
I have a collection with elements that have a field field1. I want to get all field1s that are options of type MyType.
Currently this is my code.
elems.map(_.field1).map {case Some(found: MyType) => found}
I'm sure this can be done in a much nicer way.. It bugs me that I need to use map twice. Is there a way to do this with only one map/collect ?
EDIT: My code works. I'm just wondering if it can be done in a better (i.e. shorter or prettier way).
elems.flatMap(_.field1.collect { case x: MyType => x })
I believe utilising .flatMap may solve this issue for you
elems.flatMap(_.field1 match {
case myType: MyType => Some(myType)
case _ => None
}
Calling iterator before transforming the collection accumulates all the transformations into a single one so perhaps try
elems
.iterator
.flatMap(_.field1)
.collect { case v: MyType => v }
.toList
if your Seq type is case class you can use pattern matching with one collect function like so (see actual seq):
case class MyTypeWrapper(field1: Option[MyType])
case class MyType(x: String)
val elems = Seq.empty[MyTypeWrapper]
val expected: Seq[MyType] = elems.map(_.field1).map{ case Some(found: MyType) => found }
val actual: Seq[MyType] = elems.collect{ case MyTypeWrapper(Some(mt: MyType)) => mt }
// expected and actual should contains the same elements
I have list say -
List("aa","1","bb","2","cc","3","dd","4")
How to make a list of tuples with even and odd positions :
(aa,1),(bb,2),(cc,3),(dd,4)
Hope it will help.
val list = List("aa","1","bb","2","cc","3","dd","4")
val tuple =
list.grouped(2).map { e =>
(e.head,e.last)
}.toList
We should consider the case of oddly sized lists, for example, List("aa","1","bb","2","cc","3","dd"):
Should we return List((aa,1), (bb,2), (cc,3), (dd,dd))?
Should we drop the last element and return List((aa,1), (bb,2), (cc,3))?
Should we indicate error is some way, perhaps with Option?
Should we crash?
Here is an example of returning Option[List(String, String)] to indicate error case:
def maybeGrouped(list: List[String]): Option[List[(String, String)]] =
Try(
list
.sliding(2, 2)
.map { case List(a,b) => (a, b) }
.toList
).toOption
I have following list
val a = List(("name1","add1","city1",10),("name1","add1","city1",10),
("name2","add2","city2",10),("name2","add2","city2",20),("name3","add3","city3",20))
I want distinct element from above list based on first three values of tuple. Fourth value should not be consider while finding distinct elements from list.
I want following output:
val output = List(("name1","add1","city1",10),("name2","add2","city2",10),
("name3","add3","city3",20))
Is it possible to get above output?
As per my knowledge, distinct works if whole tuple/value is duplicated. I tried out with distinct like following:
val b = List(("name1","add1","city1",10),("name1","add1","city1",10),("name2","add2","city2",10),
("name2","add2","city2",20),("name3","add3","city3",20)).distinct
but it gives output as -
List(("name1","add1","city1",10),("name2","add2","city2",10),
("name2","add2","city2",20),("name3","add3","city3",20))
Any alternate approach will also appreciated.
Use groupBy like this
a.groupBy( v => (v._1,v._2,v._3)).keys.toList
This constructs a Map where each key is by definition a unique triplet as required in the lambda function above.
Should it include also the last element in the tuple, fetch the first element for each key, like this
a.groupBy( v => (v._1,v._2,v._3)).mapValues(_.head)
If the order of the output list isn't important (i.e. you are happy to get List(("name3","add3","city3",20),("name1","add1","city1",10),("name2","add2","city2",10))), the following works as specified:
a.groupBy(v => (v._1,v._2,v._3)).values.map(_.head).toList
(Due to Scala collections design, you'll see the order kept for output lists up to 4 elements, but above that size HashMap will be used.) If you do need to keep the order, you can do something like (generalizing a bit)
def distinctBy[A, B](xs: Seq[A], f: A => B) = {
val seen = LinkedHashMap.empty[B, A]
xs.foreach { x =>
val key = f(x)
if (!seen.contains(key)) { seen.update(key, x) }
}
seen.values.toList
}
distinctBy(a, v => (v._1, v._2, v._3))
You could try
a.map{case x#(name, add, city, _) => (name,add,city) -> x}.toMap.values.toList
To make sure you have the first one in list kept,
type String3 = (String, String, String)
type String3Int = (String, String, String, Int)
a.foldLeft(collection.immutable.ListMap.empty[String3, String3Int]) {
case (a, b) => if (a.contains((b._1, b._2, b._3))) {
a
} else a + ((b._1, b._2, b._3) -> b)
}.values.toList
On simple solution would be to convert the List to a Set. Sets don't contain duplicates: check the documentation.
val setOfTuples = a.toSet
println(setOfTuples)
Output: Set((1,1), (1,2), (1,3), (2,1))
If I have a following method
def getMyList :\/[Throwable,List[\/[Throwable,Int]]] ={
....
}
how to flatten type of getMyList to \/[Throwable,List[Int]]
Just flatMap and sequenceU, it's all in scalaz:
def flatten(e: \/[Throwable,List[\/[Throwable,Int]]]): \/[Throwable,List[Int]] = {
e.flatMap(a => a.sequenceU)
}
If by flatten, you mean remove the left types from List[\/[Throwable,Int]], then you can map the outer disjunction, and collect the right types:
list.map(_.collect{ case \/-(x) => x})
I don't think that some higher order "flatten" exists for /. Looks like Validateion & ValidationNEL will be better choice for this problem. However here is "dirty" solution for /, it will return first fail. If you want to accumulate failures Validation is way to go
val getMyList: \/[Throwable,List[\/[Throwable,Int]]] =
//\/-(List(-\/(new RuntimeException("test")), \/-(1)))
\/-(List(\/-(2), \/-(1)))
val flatten = getMyList.fold(\/.left, _.foldLeft(\/.right[Throwable, List[Int]](List.empty[Int])) {
case (\/-(list), \/-(i)) => \/-(list :+ i)
case (\/-(list), -\/(err)) => -\/(err)
case (-\/(err), _) => -\/(err)
})
println(flatten)
We use the following method, where .sSuccess creates a \/[_, Seq[T]] and .sFail creates a \/[Throwable, _] with all of the throwables' error messages concatenated:
implicit class CondenseEither[T](seq: Seq[\/[Throwable,T]]) = {
def condenseSeq: \/[Throwable, Seq[T]] = {
val errs = seq.filter(_.isLeft).map(_.toEither)
if(errs.isEmpty) seq.map(_.toEither).map(_.right.get).sSuccess
else errs.map(_.left.get.getMessage).mkString(", ")).sFail
}
}
There's probably a way to do this without the toEithers
I have a collection which I want to map to a new collection, however each resulting value is dependent on the value before it in some way.I could solve this with a leftFold
val result:List[B] = (myList:List[A]).foldLeft(C -> List.empty[B]){
case ((c, list), a) =>
..some function returning something like..
C -> (B :: list)
}
The problem here is I need to iterate through the entire list to retrieve the resultant list. Say I wanted a function that maps TraversableOnce[A] to TraversableOnce[B] and only evaluate members as I call them?
It seems to me to be a fairly conventional problem so Im wondering if there is a common approach to this. What I currently have is:
implicit class TraversableOnceEx[T](val self : TraversableOnce[T]) extends AnyVal {
def foldyMappyFunction[A, U](a:A)(func:(A,T) => (A,U)):TraversableOnce[U] = {
var currentA = a
self.map { t =>
val result = func(currentA, t)
currentA = result._1
result._2
}
}
}
As far as functional purity goes, you couldn't run it in parallel, but otherwise it seems sound.
An example would be;
Return me each element and if it is the first time that element has appeared before.
val elements:TraversableOnce[E]
val result = elements.mappyFoldyFunction(Set.empty[E]) {
(s, e) => (s + e) -> (e -> s.contains(e))
}
result:TraversableOnce[(E,Boolean)]
You might be able to make use of the State Monad. Here is your example re-written using scalaz:
import scalaz._, Scalaz._
def foldyMappy(i: Int) = State[Set[Int], (Int, Boolean)](s => (s + i, (i, s contains(i))))
val r = List(1, 2, 3, 3, 6).traverseS(foldyMappy)(Set.empty[Int])._2
//List((1,false), (2,false), (3,false), (3,true), (6,false))
println(r)
It is look like you need SeqView. Use view or view(from: Int, until: Int) methods for create a non-strict view of list.
I really don't understand your example as your contains check will always result to false.
foldLeft is different. It will result in a single value by aggregating all elements of the list.
You clearly need map (List => List).
Anyway, answering your question about laziness:
you should use Stream instead of List. Stream doesn't evaluate the tail before actually calling it.
Stream API