Using scala 2.12 , joining the value of map1 with key of map2 to get the value of map2
Not sure how to achieve. Please help
val t1=Map("K1"->"tempV1" , "K2"->"tempV2" , "K3"-> "tempV3","K4"->"tempV4")
val t2=Map("tempV1"->"V1" , "tempV2"->"V2" , "tempV3"-> "V3","someV4"->"V4")
//expected
Map("K1"->"V1","K2"->"V2","K3"->"V3")
t1.flatMap{case (k,v) => t2.get(v).map(k -> _)}
Or, the equivalent expression:
for {
(k,v1) <- t1
v2 <- t2.get(v1)
} yield (k,v2)
val res = t1.map(t1Tup=>(t1Tup._1,t2.getOrElse(t1Tup._2,None)))
Related
I am newbie in scala and I have a problem when merging or joining (dont know the exact term in scala) two lists by key values. In a case, key values didn't match and I took java.util.NoSuchElementException: key not found error. But I want to have just matched cases. My sample code snippet is as follows:
val users = List(
(1,1,4.0),
(1,3,4.0),
(1,6,4.0)
)
val cars = List(
(1,1,"ww"),
(1,3,"mds"),
(2,6,"pgt")
)
val usersmapped = users.map(i => ((i._1,i._2),i)).toMap
val carsmapped = cars.map(i => ((i._1,i._2),i)).toMap
val result = carsmapped.map(ar => (ar._2,usersmapped(ar._1)._3))
The error message for above code is: java.util.NoSuchElementException: key not found: (2,6)
My aim is to have something like that:
List(
(1,1,ww) -> 4.0,
(1,3,mds)-> 4.0
)
OR
List(
(1,1,ww) -> 4.0,
(1,3,mds) -> 4.0,
(1,6,null) -> 4.0 //null means no match with other list
)
Is especially the first one is possible? And how can I do that?
I tried that bu it returned none:
val result = Try(carsmapped.map(ar => (ar._2,usersmapped(ar._1)._3)))
).toOption
If you can help, I will be appreciated
Use get method instead of "()". get method returning Option:
val users = List(
(1,1,4.0),
(1,3,4.0),
(1,6,4.0)
)
val cars = List(
(1,1,"ww"),
(1,3,"mds"),
(2,6,"pgt")
)
val usersmapped = users.map(i => ((i._1,i._2),i)).toMap
val carsmapped = cars.map(i => ((i._1,i._2),i)).toMap
carsmapped
.map(ar => (ar._2, usersmapped.get(ar._1).map(_._3)))
.foreach(println)
I'm trying to do sometihng in Scala,
var tupleList = new ListBuffer[(String,List[String])]
val num1 = List("1","2")
val num2= List("3","4")
val num3= List("5","6")
tupleList+=(("Joe",num1))
tupleList+=(("Ben",num2))
tupleList+=(("Joe",num3))
**I want to union between both of the lists with name 'Joe', and create one list of numbers: 1,2,5,6 .
So I thought to use groupByKey, but how to explode the values?
mapValues?
How Can I use reduceLeft in this case?
I tried something like that:
val try3 = vre.groupMapReduce(_._1)(_._2)(_ :: _)
Thanks!
Principle problem was in using :: instead of :::.
With immutable collections your example will look like this:
val input =
("Joe" -> List("1","2")) ::
("Ben" -> List("3","4")) ::
("Joe" -> List("5","6")) ::
Nil
val result = input.groupMapReduce(_._1)(_._2)(_ ::: _)
print(result.mkString("{", ", ", "}"))
// Output:
// {Joe -> List(1, 2, 5, 6), Ben -> List(3, 4)}
How do I avoid duplicates for this code:
val lastUpdatesBuilder = List.newBuilder[(String, Int)]
val somelist = List("a","a")
for (v <- somelist) {
lastUpdatesBuilder += v -> 1
}
println(lastUpdatesBuilder.result())
Result is List((a,1), (a,1)) and I want it to be List((a,1)) only.
Here you go:
object Demo extends App {
val lastUpdatesBuilder = Set.newBuilder[(String, Int)]
val somelist = List("a","a")
for (v <- somelist) {
lastUpdatesBuilder += v -> 1
}
println(lastUpdatesBuilder.result())
}
Tho i would suggest not to use mutable set you can do something like this.
val ans = somelist.map{ key =>
key -> 1
}.toMap
println(ans)
Or you can first remove the duplicate using distinct and then create a map out of it.
val somelist = List("a","a").distinct
val ans = somelist.map{ key =>
key -> 1
}.toMap
This is what the distinct method does.
How can i traverse following RDD using Spark scala. I wants to print every value present in Seq with associated key
res1: org.apache.spark.rdd.RDD[(java.lang.String, Seq[java.lang.String])] = MapPartitionsRDD[6] at groupByKey at <console>:14
I tried following code for it.
val ss=mapfile.map(x=>{
val key=x._1
val value=x._2.sorted
var i=0
while (i < value.length) {
(key,value(i))
i += 1
}
}
)
ss.top(20).foreach(println)
I try to convert your codes as follows:
val ss = mapfile.flatMap {
case (key, value) => value.sorted.map((key, _))
}
ss.top(20).foreach(println)
Is it what you want?
I tried this and it works for the return type as mentioned.
val ss=mapfile.map(x=>{case (key, value) => value.sorted.map((key, _))}.groupByKey().map(x=>(x._1,x._2.toSeq))
ss.top(20).foreach(println)
Note: ss is of type::: org.apache.spark.rdd.RDD[(java.lang.String, Seq[java.lang.String])]
I have a two Maps and I want to concatenate them.
I tried almost all example given here Best way to merge two maps and sum the values of same key? but it ignores all values for key metrics and only stores last value.
I have downloaded scalaz-full_2.9.1-6.0.3.jar and imported it import scalaz._ but it won't works for me.
How can I concate this two maps with multiple values to same keys ?
Edit :-
Now I tried
val map = new HashMap[String, Set[String]] with MultiMap[String, String]
map.addBinding("""report_type""" , """performance""")
map.addBinding("""start_date""" ,start_date)
map.addBinding("""end_date""" , end_date)
map.addBinding("metrics" , "plays")
map.addBinding("metrics", "displays")
map.addBinding("metrics" , "video_starts")
map.addBinding("metrics" , "playthrough_25")
map.addBinding("metrics", "playthrough_50")
map.addBinding("metrics", "playthrough_75")
map.addBinding("metrics", "playthrough_100")
val map1 = new HashMap[String, Set[String]] with MultiMap[String, String]
map1.addBinding("""dimensions""" , """asset""")
map1.addBinding("""limit""" , """50""")
And tried to conver this mutable maps to immutable type using this link as
val asset_query_string = map ++ map1
val asset_query_string_map =(asset_query_string map { x=> (x._1,x._2.toSet) }).toMap[String, Set[String]]
But still I get
i_ui\config\config.scala:51: Cannot prove that (String, scala.collection.immutable.Set[String]) <:< (St
ring, scala.collection.mutable.Set[String]).
11:10:13.080 [ERROR] i.g.a.ZincCompiler$ - val asset_query_string_map =(asset_query_string map { x=> (x
._1,x._2.toSet) }).toMap[String, Set[String]]
Your problem is not related with a concatenation but with a declaration of the metrics map. It's not possible to have multiple values for a single key in a Map. Perhaps you should look at this collection:
http://www.scala-lang.org/api/2.10.3/index.html#scala.collection.mutable.MultiMap
You can't have duplicate keys in a Map.
for simple map it is impossible to have duplicates keys,if you have the duplicates keys in the map it takes the last one
but you can use MultiMap
import collection.mutable.{ HashMap, MultiMap, Set }
val mm = new HashMap[String, Set[String]] with MultiMap[String, String]
mm.addBinding("metrics","plays")
mm.addBinding("metrics","displays")
mm.addBinding("metrics","players")
println(mm,"multimap")//(Map(metrics -> Set(players, plays, displays)),multimap)
I was able to create two MultiMaps but when I tried to concatenate val final_map = map1 ++ map2
and I tried answer given here Mutable MultiMap to immutable Map
But my problem was not solved, I got
config\config.scala:51: Cannot prove that (String, scala.collection.immutable.Set[String]) <:< (St
ring, scala.collection.mutable.Set[String]).
finally it solved by
val final_map = map1 ++ map2
val asset_query_string_map = final_map.map(kv => (kv._1,kv._2.toSet)).toMap