I need to form this:
{
“item” : “hardcoded_value”,
“item2” : “hardcoded_value”,
“item3” : “hardcoded_value”,
}
In the exec block I am trying:
// list with items [“item1”, “ item2”, “ item3”]
val theList = session("itemNames").as[List[String]]
val theMap = Map.empty[String,String] // empty map
// add items from list in map
theList.foreach{ key =>
| theMap += key -> "hardcoded_value"
}
But getting error at += position.
Also tried:
theList.foreach(key => theMap += key -> "hardcoded_value" )
How to insert key and value into a map by iterating over a list? I am new to gatling and scala.
After looking at your question in more detail, I realize that you're not just asking about turning the Collection into a Map. Combining the answers from How can I use map and receive an index as well in Scala? and Scala best way of turning a Collection into a Map-by-key?, you can do the following:
List("first", "second", "third").zipWithIndex.map {
case (item, index) => ("item" + index) -> item.toString
}
>res0: List[(String, String)] = List((item0,first), (item1,second), (item2,third))
Related
Question 1: Can I use tuple as a key of a map in Scala?
Question 2: If yes , how can I create a map with a tuple as key?
Question 3: I want to convert my scala map to RDD, how would I do in the following case? I am trying to do in this way
var mapRDD = sc.parallelize(map.toList)
Is this the right way to do ?
Question 4: For this particular code snippet, when I do a println on map, it has no values.
I have not included the whole code, basically mapAgainstValue contains userId as key and list of friends as values. I want to recreate a map RDD with the following transformation in the key.
What would be the reason for empty map?
var mapAgainstValue = logData.map(x=>x.split("\t")).filter(x => x.length == 2).map(x => (x(0),x(1).split(",")))
var map:Map[String,List[String]] = Map()
var changedMap = mapAgainstValue.map{
line =>
var key ="";
for(userIds <- line._2){
if(line._1.toInt < userIds.toInt){
key =line._1.concat("-"+userIds);
}
else {
key = userIds.concat("-" + line._1);
}
map += (key -> line._2.toList)
}
}
changedMap.collect()
map.foreach(println)
Yes, you can use Tuple as a key in Map.
For example:
val userMap = Map(
(1, 25) -> "shankar",
(2, 35) -> "ramesh")
Then you can try print the output using foreach
val userMapRDD = sparkContext.parallelize(userMap.toSeq, 2)
mapRDD.foreach(element => {
println(element)
})
If you want to transform the mapRDD to something else. following code returns only age and name as tuple.
val mappedRDD = userMapRDD.map {
case ((empId: Int, age: Int), name: String) => {
(age, name)
}
}
I have some records in a List .
Now I want to create a new Map(Mutable Map) from that List with unique key for each record. I want to achieve this my reading a List and calling the higher order method called map in scala.
records.txt is my input file
100,Surender,2015-01-27
100,Surender,2015-01-30
101,Raja,2015-02-19
Expected Output :
Map(0-> 100,Surender,2015-01-27, 1 -> 100,Surender,2015-01-30,2 ->101,Raja,2015-02-19)
Scala Code :
object SampleObject{
def main(args:Array[String]) ={
val mutableMap = scala.collection.mutable.Map[Int,String]()
var i:Int =0
val myList=Source.fromFile("D:\\Scala_inputfiles\\records.txt").getLines().toList;
println(myList)
val resultList= myList.map { x =>
{
mutableMap(i) =x.toString()
i=i+1
}
}
println(mutableMap)
}
}
But I am getting output like below
Map(1 -> 101,Raja,2015-02-19)
I want to understand why it is keeping the last record alone .
Could some one help me?
val mm: Map[Int, String] = Source.fromFile(filename).getLines
.zipWithIndex
.map({ case (line, i) => i -> line })(collection.breakOut)
Here the (collection.breakOut) is to avoid the extra parse caused by toMap.
Consider
(for {
(line, i) <- Source.fromFile(filename).getLines.zipWithIndex
} yield i -> line).toMap
where we read each line, associate an index value starting from zero and create a map out of each association.
EDIT: I found this What is Scala's yield? (particularly the second, most popular, answer) to be very instructive after the accepted answer solved my problem.
==
I have a HashMap, which I want to iterate in, and for each keys, use a for loop to create new objects.
I'm trying to get a list of those new objects, but I'm always given back an empty "Unit" sequence. I'd like to understand better the behaviour of my code.
case class MyObject(one: String, two: String, three: Int)
val hm = new HashMap[String,Int]
hm += ("key" -> 3)
hm += ("key2" -> 4)
val newList = hm.map { case (key,value) =>
for (i <- 0 until value) {
new MyObject(key, "a string", i)
}}.toSeq
result:
newList:Seq[Unit] = ArrayBuffer((), ())
If I don't use any for loop inside the .map(), I have the type of structure I'm expecting:
val newList = hm.map { case (key,value) =>
new MyObject(key, "a string", value)}.toSeq
results in:
newList:Seq[MyObject] = ArrayBuffer(MyObject(key,host,3), MyObject(key2,host,4))
As I mentioned in my comment, you are missing yield on the for comprehension in your map statement. If you do not include the yield keyword then your for comprehension is purely side effecting and does not produce anything. Change it to:
for (i <- 0 until value) yield {
Now from here, you will end up with a Seq[IndexedSeq[MyObject]]. If you want to end up with just a Seq[MyObject] then you can flatten like so:
val newList = hm.map { case (key,value) =>
for (i <- 0 until value) yield {
MyObject(key, "a string", i)
}}.toSeq.flatten
}
And in fact (as pointed out by #KarolS), you can shorten this even further by replacing map with flatMap and remove the explicit flatten at the end:
val newList = hm.flatMap { case (key,value) =>
for (i <- 0 until value) yield {
MyObject(key, "a string", i)
}}.toSeq
}
I have an application which has to gather some external data first, then turn them into objects. Afterwards, it will do some analysis on the data.
I managed to gather the data and put it into a map. The map contains a unique key for each of the future objects, and a ListBuffer of the data needed to build the object.
Now I want to create a list of objects from this map, and don't know how to get my data out of the map. I haven't worked with maps before (yes, I am that new to the language), but found a question which says that, when I want to access an element of the map with head, I get a tuple of the key and the value. I hoped that I get the same when I iterate over the map with map (the method), but this doesn't appear to work. And I looked in Programming with Scala, but couldn't find a place saying what I get when I iterate over a map.
Here is an MWE for what I want to do:
//This code will gather number names from different languages and then create objects of type Number containing each name.
import scala.collection.mutable
import scala.collection.mutable.ListBuffer
class Number (val theNumber: Int, val names: List[String]) {
override def toString = theNumber + " is known as " + names.mkString(", ") + "."
}
// Construct a map holding example data
val numbersAsMap = mutable.Map.empty[Int, ListBuffer[String]]
numbersAsMap += (1 -> new ListBuffer[String])
numbersAsMap += (2 -> new ListBuffer[String])
numbersAsMap += (3 -> new ListBuffer[String])
numbersAsMap(1) += "one"
numbersAsMap(1) += "eins"
numbersAsMap(1) += "uno"
numbersAsMap(2) += "two"
numbersAsMap(2) += "zwei"
numbersAsMap(2) += "due"
numbersAsMap(3) += "three"
numbersAsMap(3) += "drei"
numbersAsMap(3) += "tre"
// Create a list of numbers
numbersAsMap map ((key, value) => new Number(key, value.toList)).toList
// error: missing parameter type
// obviously I'm not getting tuples, let's try it another way
numbersAsMap.keys map (key => new Number(key, numbersAsMap(key).toList)).toList
// it throws the same error as above :(
The map method of Map complies with the map method of other collections, so it's body only gets one parameter. In case of a Map, this is a tuple consisting of the key and the value.
So you can write:
numbersAsMap.map(kv => new Number(kv._1, kv._2.toList)).toList
If you want to name the tuple values:
numbersAsMap.map {
kv =>
val (key, value) = kv
new Number(key, value.toList)
}.toList
But there is another option to write it nicely in a single line: Use a partial function:
numbersAsMap.map { case (key, value) => new Number(key, value.toList) }.toList
A { case ... } defines a partial function; and this way you can extract the values of the tuple.
Here are two possible ways to do the map operation on your Map:
val result = numbersAsMap.map{
case (key, value) =>
new Number(key, value.toList)
}.toList
val result2 = numbersAsMap.map(kv => new Number(kv._1, kv._2.toList)).toList
I'm parallelising over a collection to count the number same item values in a List. The list in this case is uniqueSetOfLinks :
for (iListVal <- uniqueSetOfLinks.par) {
try {
val num : Int = listOfLinks.count(_.equalsIgnoreCase(iListVal))
linkTotals + iListVal -> num
}
catch {
case e : Exception => {
e.printStackTrace()
}
}
}
linkTotals is an immutable Map. To gain a reference to the total number of links do I need to update linkTotals so that it is immutable ?
I can then do something like :
linkTotals.put(iListVal, num)
You can't update immutable collection, all you can do is to combine immutable collection with addition element to get new immutable collection, like this:
val newLinkTotals = linkTotals + (iListVal -> num)
In case of collection you could create new collection of pairs and than add all pairs to the map:
val optPairs =
for (iListVal <- uniqueSetOfLinks.par)
yield
try {
val num : Int = listOfLinks.count(_.equalsIgnoreCase(iListVal))
Some(iListVal -> num)
}
catch {
case e : Exception => e.printStackTrace()
None
}
val newLinkTotals = linkTotals ++ optPairs.flatten // for non-empty initial map
val map = optPairs.flatten.toMap // in case there is no initial map
Note that you are using parallel collections (.par), so you should not use mutable state, like linkTotals += iListVal -> num.
Possible variation of #senia's answer (got rid of explicit flatten):
val optPairs =
(for {
iListVal <- uniqueSetOfLinks.par
count <- {
try
Some(listOfLinks.count(_.equalsIgnoreCase(iListVal)))
catch {
case e: Exception =>
e.printStackTrace()
None
}
}
} yield iListVal -> count) toMap
I think that you need some form of MapReduce in order to have parallel number of items estimation.
In your problem you already have all unique links. The partial intermediate result of map is simply a pair. And "reduce" is just toMap. So you can simply par-map the link to pair (link-> count) and then finally construct a map:
def count(iListVal:String) = listOfLinks.count(_.equalsIgnoreCase(iListVal))
val listOfPairs = uniqueSetOfLinks.par.map(iListVal => Try( (iListVal, count(iListVal)) ))
("map" operation is par-map)
Then remove exceptions:
val clearListOfPairs = listOfPairs.flatMap(_.toOption)
And then simply convert it to a map ("reduce"):
val linkTotals = clearListOfPairs.toMap
(if you need to check for exceptions, use Try.failure)