Hey I'm new to the language and trying to make a map that contains values used alot within the code, but currently having trouble calling the map into a parameter.
heres and example of what it looks like
val abcd = "abc"
val defg = "123"
val ghij = "456"
val map1 = Map ( "abcd" -> abcd, "defg" -> defg, "ghij" -> ghij)
that part is fine but when i want to add them all to the parameter it says there is too many arguments?
had to add
.toString
.##()
to the end of the strings and ints, for example
val settings = Array( map1("defg").##() );
Related
I have written below code, it is working for the one word but when I give the seq variable term I am not getting the output, can anyone tell me how to solve this.
val term = List("Achieving","Making")
val sc = new SparkContext("local[*]","Filter_lines")
val Lines = sc.textFile("../book.txt")
val filter_Lines = Lines.filter(l => l.contains("Making")).collect()
filter_Lines.foreach(println)
Try this -
Lines.filter(l => term.exists(t => l.contains(t))).foreach(println)
exists function on the collection accepts a function that returns true if the "l" contains any of the "t" terms.
I wish to create a custom feeder in Gatling scala, which fills the parameter element via iterating through a map.
I have the following code snippet:
val idPostFeeder = Iterator.continually(
Map("postId" -> getValues())
)
getValues is a Collection containing String elements
I tried also the following way:
val idPostFeeder = (for (i <- getFile().get(l.get(b)) yield {
Map("postId" -> s"$i")
} )
val l = getFile().keysIterator.toList;
var b = l.indexOf() until l.indexOf(mapLenght)
getFile is a Map[String, String], from which I need the values passed to the feeder.
Is there a way to fill a feeder via an iteration over a Collection or a Map?
Thank you!
for a collection of Strings, you just need to convert to a Map
getValues.map(s => Map("postId" -> s)).toIterator
you now have an iterator that maps "postId" to each value of your collection
i have the below input,
input
[level:1,firstFile:one,secondFile:secone,Flag:NA][level:1,firstFile:two,secondFile:sectwo,Flag:NA][level:2,firstFile:three,secondFile:secthree,Flag:NA]
getting below output and working fine,
List(List(one, two), List(three))
List(List(secone, sectwo), List(secthree))
However when i pass the below input i am getting the output as,
[level:1,firstFile:one,four,secondFile:secone,Flag:NA][level:1,firstFile:two,secondFile:sectwo,Flag:NA][level:2,firstFile:three,secondFile:secthree,Flag:NA]
getting output as,
List(List(), List(two), List(three))
List(List(), List(sectwo), List(secthree))
But the expected output is,
List(List(one, four, two), List(three))
List(List(secone, sectwo), List(secthree))
Code.
val validJsonRdd = sc.parallelize(Seq(input)).flatMap(x => x.replace(",", "\",\"").replace(":", "\":\"").replace("[", "{\"").replace("]", "\"}").replace("}{", "}&{").split("&"))
import org.apache.spark.sql.functions._
val df = spark.read.json(validJsonRdd).orderBy("level").groupBy("level")
.agg(collect_list("firstFile").as("firstFile"), collect_list("secondFile").as("secondFile"))
.select(collect_list("firstFile").as("firstFile"), collect_list("secondFile").as("secondFile"))
val rdd = df.collect().map(row => (row(0).asInstanceOf[Seq[Seq[String]]], row(1).asInstanceOf[Seq[Seq[String]]]))
val first = rdd(0)._1.map(x => x.toList).toList
val second = rdd(0)._2.map(x => x.toList).toList
val firstInputcolumns = first.map(_.filterNot(_ == null))
val secondInputcolumns= second.map(_.filterNot(_ == null))
println(firstInputcolumns)
println(secondInputcolumns)
Kindly help me to correct the code.
It doesn't look like your replaces are quite producing valid JSON. If you run them on the second input, for the first entry you get:
{"level":"1","firstFile":"one","four","secondFile":"secone","Flag":"NA"}
But JSON is a list of key-value pairs. You can't just have "four" sitting out on its own like that. If you want firstFile to be mapped to a list, one and four should be wrapped in square brackets, and the JSON should look like so:
{"level":"1","firstFile":["one","four"],"secondFile":"secone","Flag":"NA"}
How can I create a simple feeder in Gatling without using a csv file?
I have tried scripts from the Gatling documentation.
I have seen one example in the documentation
val random = new util.Random
val feeder = Iterator.continually(Map("email" -> random.nextString(20) + "#foo.com"))
I don't understand the above code.
I tried a script with a feeder that uses a csv file and was executed successfully. Instead of feeding data from a csv file, how do I write a feeder that can take a defined value.
As the docs state, a Feeder is just an alias for Iterator[Map[String, T]]. You just need to make sure your feeder provides an infinite stream of values as highlighted by Rüdiger Klaehn.
Since you said you already was able to run an example using the builtin csv feeder, let's convert it to our own feeder so it becomes more clear what the above custom feeder code does.
Let's look at the example that comes from the advanced tutorial:
object Search {
val feeder = csv("search.csv").random // 1, 2
val search = exec(http("Home")
.get("/"))
.pause(1)
.feed(feeder) // 3
.exec(http("Search")
.get("/computers?f=${searchCriterion}") // 4
.check(css("a:contains('${searchComputerName}')", "href").saveAs("computerURL"))) // 5
.pause(1)
.exec(http("Select")
.get("${computerURL}")) // 6
.pause(1)
}
This is the part that generates the feed:
val feeder = csv("search.csv").random // 1, 2
And this is the search.csv file:
searchCriterion,searchComputerName
Macbook,MacBook Pro
eee,ASUS Eee PC 1005PE
Let's replace that with our new custom feeder:
/* This is our list of choices, we won't ready from csv anymore */
val availableComputers = List(
Map("searchCriterion" -> "MacBook", "searchComputerName" -> "Macbook Pro"),
Map("searchCriterion" -> "eee", "searchComputerName" -> "ASUS Eee PC 1005PE")
)
/* Everytime we call this method we get a random member of availableComputers */
def pickARandomComputerInfo() = {
availableComputers(Random.nextInt(availableComputers.size))
}
/* Continually means every time you ask feeder for a new input entry,
it will call pickARandomComputerInfo to gerenate an input for you.
So iterating over feeder will never end, you will always get
something */
val feeder = Iterator.continually(pickARandomComputerInfo)
This is harder to see in your provided example, but you could split it to better understand it:
def getRandomEmailInfo() = Map("email" -> random.nextString(20) + "#foo.com")
val feeder = Iterator.continually(getRandomEmailInfo)
I have nested url parameters being passed to an endpoint, and I need these represented in as a JsValue. My initial assumption was that Play would parse them in a way similar to Rails, however parameters seem to only be split by & and =. Example:
Query params: ?test[testkey]=testvalue&test[newkey]=newvalue
Actual:
Map(
"test[testkey]" -> "testvalue" ,
"test[newkey]" -> "newvalue
)
Expected:
Map(
"test" -> Map(
"testkey" -> "testvalue" ,
"newkey" -> "newvalue"
)
)
Note that the end goal here is to be able to convert this into a JsObject.
I've started writing this myself, however simply porting the function from Rack is very un-scala-y and I feel like there has to be a quick way to get this that I am simply missing.
UPDATE
I am trying to find a generic solution which mimics the parsing that Rails uses (ie, with nested objects, lists, etc), and not just one level deep objects.
Just for fun, one option is to do something like:
import scala.util.matching.Regex
val pattern = new Regex("""(\w+)\[(\w+)\]""")
val qs : Map[String, Map[String, List[Seq[String]]]] = request.queryString.toList.map {
case (k, v) =>
pattern findFirstIn k match {
case Some(pattern(key, value)) => (key, value, v)
}
}.groupBy(_._1).mapValues(value => value.groupBy(_._2).mapValues {
value => value.map(x => x._3)
})
To convert this is to a JsValue, we can simply invoke:
import play.api.libs.json.Json
Json.toJson(qs)
This assumes that all your url params look like map[key]=value. You would have to modify the code a little to accommodate the standard key=value pattern.