I can't grasp the map method I guess..
Trying to read a file :
val messagesMap = XML.loadFile(messageXMLFile).map(parseMessageXML)
where the method parseMessageXML is defined as :
def parseMessageXML(xml : scala.xml.Node) = {
val nodes = xml \\ "add"
nodes.map({
node =>
val obj = new AdMessage(node)
println("adding an AdMessage " + obj.toString)
(obj.MessageId -> obj)
}).toMap
}
Can anybody please explain why I end up with a Seq[Map[String, AdMessage]] and not a just a Map[String, AdMessage] ?
map transforms each element of your Seq into an another element.
For instance:
scala> Seq("One", "Two", "Three").map(_.length())
res0: Seq[Int] = List(3, 3, 5)
Each String is mapped into an Int thanks to the length function. Therefore the original type is Seq[String] and the final type is Seq[Int]
In your case, parseMessageXML transforms a Node into a Map[String, AdMessage], so the original type is Seq[Node] and the final type is Seq[Map[String, AdMessage]].
In your case, assuming you just want to transform the content of the file into a Map[String, AdMessage]:
val messagesMap = parseMessageXML(XML.loadFile(messageXMLFile))
Related
I am trying to return a Map from a scala method as below.
I have two different Maps with some matching keys. I need to find the matching keys between them and pick the values out of them and put them in another Map in the way I wanted. Below is the code I wrote for the aforementioned action.
val common = rdKeys.keySet.intersect(bounds.keySet).toList
val metaColumns = getReadColumns(common, rdKeys, bounds)
def getReadColumns(common:List[String], rdKeys:scala.collection.mutable.Map[String, String], bounds:scala.collection.mutable.Map[String, String]): scala.collection.mutable.Map[String, String] = {
var metaMap = scala.collection.mutable.Map[String, String]
common.map {
c => metaMap += (c -> bounds(c) + "|" + rdKeys(c))
}
metaMap
}
But the method is giving me a compilations error:
Expression of type Seq[(String, String)] => mutable.Map[String, String] doesn't confirm to expected type mutable.Map[String, String]
All the method parameters, Maps used inside the method & the return type of the method are of mutable.Map[String, String]. I don't understand what is the mistake I did here.
Could anyone let me know what do I have to do to correct the problem ?
You got the error
Expression of type Seq[(String, String)] => mutable.Map[String, String] doesn't confirm to expected type mutable.Map[String, String]
Because of the statement scala.collection.mutable.Map[String, String] returns a function Seq[(String, String)] => mutable.Map[String, String]
You can correct it by empty method:
def getReadColumns( common:List[String],
rdKeys:scala.collection.mutable.Map[String, String],
bounds:scala.collection.mutable.Map[String, String]):scala.collection.mutable.Map[String, String] = {
val metaMap = scala.collection.mutable.Map.empty[String, String]
common.foreach {
c => metaMap.update(c, bounds(c) + "|" + rdKeys(c))
}
metaMap
}
P.S. or use c => metaMap += c -> (bounds(c) + "|" + rdKeys(c))
map preserves the collection type. You can map a List to another List, and, in the end, cast your List directly to a Map
val common = List("a", "b")
val rdKeys = Map("a" -> 1, "b" -> 1)
val bounds = Map("a" -> 10, "b" -> 10)
common // this is a list
.map(c => c -> (bounds(c) + "|" + rdKeys(c))) // this is a list
.toMap // then cast to it to a Map
This code outputs
scala.collection.immutable.Map[String,String] = Map(a -> 10|1, b -> 10|1)
In the below code, encoded is a JSON string. The JSON.parseFull() function is returning an object of the form: Some(Map(...)). I am using .get to extract the Map, but am unable to index it as the compiler sees it as type Any. Is there any to provide the compiler visibility that it is, in fact, a map?
val parsed = JSON.parseFull(encoded)
val mapped = parsed.get
You can utilize the collect with pattern matching to match on the type:
scala> val parsed: Option[Any] = Some(Map("1" -> List("1")))
parsed: Option[Any] = Some(Map(1 -> List(1)))
scala> val mapped = parsed.collect{
case map: Map[String, Any] => map
}
mapped: Option[Map[String,Any]] = Some(Map(1 -> List(1)))
You can do something like the following in the case of a List value to get values from the List:
scala> mapped.get.map{ case(k, List(item1)) => item1}
res0: scala.collection.immutable.Iterable[Any] = List(1)
I was able to use a combination of the get function and pattern matching similar to what was posted in Tanjin's response to get the desired result.
object ReadFHIR {
def fatal(msg: String) = throw new Exception(msg)
def main (args: Array[String]): Unit = {
val fc = new FhirContext()
val client = fc.newRestfulGenericClient("http://test.fhir.org/r2")
val bundle = client.search().forResource("Observation")
.prettyPrint()
.execute()
val jsonParser = fc.newJsonParser()
val encoded = jsonParser.encodeBundleToString(bundle)
val parsed = JSON.parseFull(encoded)
val mapped: Map[String, Any] = parsed.get match{
case map: Map[String, Any] => map
}
println(mapped("resourceType"))
}
}
I have a PartialFunction[String,String] and a Map[String,String].
I want to apply the partial functions on the map values and collect the entries for which it was applicaple.
i.e. given:
val m = Map( "a"->"1", "b"->"2" )
val pf : PartialFunction[String,String] = {
case "1" => "11"
}
I'd like to somehow combine _._2 with pfand be able to do this:
val composedPf : PartialFunction[(String,String),(String,String)] = /*someMagicalOperator(_._2,pf)*/
val collected : Map[String,String] = m.collect( composedPf )
// collected should be Map( "a"->"11" )
so far the best I got was this:
val composedPf = new PartialFunction[(String,String),(String,String)]{
override def isDefinedAt(x: (String, String)): Boolean = pf.isDefinedAt(x._2)
override def apply(v1: (String, String)): (String,String) = v1._1 -> pf(v1._2)
}
is there a better way?
Here is the magical operator:
val composedPf: PartialFunction[(String, String), (String, String)] =
{case (k, v) if pf.isDefinedAt(v) => (k, pf(v))}
Another option, without creating a composed function, is this:
m.filter(e => pf.isDefinedAt(e._2)).mapValues(pf)
There is a function in Scalaz, that does exactly that: second
scala> m collect pf.second
res0: scala.collection.immutable.Map[String,String] = Map(a -> 11)
This works, because PartialFunction is an instance of Arrow (a generalized function) typeclass, and second is one of the common operations defined for arrows.
I am trying to append filename to each record in the file. I thought if the RDD is Array it would have been easy for me to do it.
Some help with converting RDD type or solving this problem would be much appreciated!
In (String, String) type
scala> myRDD.first()(1)
scala><console>:24: error: (String, String) does not take parametersmyRDD.first()(1)
In Array(string)
scala> myRDD.first()(1)
scala> res1: String = abcdefgh
My function:
def appendKeyToValue(x: Array[Array[String]){
for (i<-0 to (x.length - 1)) {
var key = x(i)(0)
val pattern = new Regex("\\.")
val key2 = pattern replaceAllIn(key1,"|")
var tempvalue = x(i)(1)
val finalval = tempvalue.split("\n")
for (ab <-0 to (finalval.length -1)){
val result = (I am trying to append filename to each record in the filekey2+"|"+finalval(ab))
}
}
}
If you have a RDD[(String, String)], you can access the first tuple field of the first tuple by calling
val firstTupleField: String = myRDD.first()._1
If you want to convert a RDD[(String, String)] into a RDD[Array[String]] you can do the following
val arrayRDD: RDD[Array[String]] = myRDD.map(x => Array(x._1, x._2))
You may also employ a partial function to destructure the tuples:
val arrayRDD: RDD[Array[String]] = myRDD.map { case (a,b) => Array(a, b) }
This drives me crazy, I can't figure out why this gives me an error.
Here an example of my code:
var seqOfObjects:Seq[Map[String, String]] = Seq[Map[String, String]]()
for(item <- somelist) {
seqOfObjects += Map(
"objectid" -> item(0).toString,
"category" -> item(1),
"name" -> item(2),
"url" -> item(3),
"owneremail" -> item(4),
"number" -> item(5).toString)
}
This gives me an error saying:
Type mismatch, expected: String, actual: Map[String, String]
But a Map[String, String] is exactly what I want to append into my Seq[Map[String, String]].
Why is it saying that my variable seqOfObjects expects a String??
Anyone have a clue?
Thanks
a += b means a = a.+(b). See this answer.
There is no method + in Seq, so you can't use +=.
scala> Seq[Int]() + 1
<console>:8: error: type mismatch;
found : Int(1)
required: String
Seq[Int]() + 1
^
required: String is from string concatenation. This behavior is inherited from Java:
scala> List(1, 2, 3) + "str"
res0: String = List(1, 2, 3)str
Actually method + here is from StringAdd wrapper. See implicit method Predef.any2stringadd.
You could use :+= or +:= instead of +=.
Default implementation of Seq is List, so you should use +: and +:= instead of :+ and :+=. See Performance Characteristics of scala collections.
You could also use List instead of Seq. There is :: method in List, so you can use ::=:
var listOfInts = List[Int]()
listOfInts ::= 1
You can rewrite your code without mutable variables using map:
val seqOfObjects =
for(item <- somelist) // somelist.reverse to reverse order
yield Map(...)
To reverse elements order you could use reverse method.
Short foldLeft example:
sl.foldLeft(Seq[Map[Srting, String]]()){ (acc, item) => Map(/* map from item */) +: acc }