I am reading a file composed of lines in the format a=b.
Using Source.fromFile("file").getLines I get an Iterator[String]. I figure I need to split the strings into tuples and then form the map from the tuples - that is easy. I am not being able to go from the Iterator[String] to an Iterator[(String,String)].
How can I do this? I am a beginner to scala and not experienced with functional programming, so I am receptive to alternatives :)
You can do so by splitting the string and then creating the tuples from the first and second elements using Iterator.map:
val strings = List("a=b", "c=d", "e=f").iterator
val result: Iterator[(String, String)] = strings.map { s =>
val split = s.split("=")
(split(0), split(1))
}
If you don't mind the extra iteration and intermediate collection you can make this a little prettier:
val result: Iterator[(String, String)] =
strings
.map(_.split("="))
.map(arr => (arr(0), arr(1)))
You can transform the values returned by an Iterator using the map method:
def map[B](f: (A) ⇒ B): Iterator[B]
Maybe like this?
Source.fromFile("file").getLines.map(_.split("=").map( x => (x.head,x.tail) ) )
You might want to wrap this into Try.
This is my try:
val strings = List("a=b", "c=d", "e=f")
val map = strings.map(_.split("=")).map { case Array(f1,f2) => (f1,f2) }
Related
I have a set of Map.Entry like Set<Map.Entry<String, ConfigValue>> in scala. Now I want to get the Set either keys(String) or values(ConfigValue) in scala. Please suggest some easy solution for this problem.
Thanks
you can use .map to transform your Set[Map.Entry[String,ConfigValue]] to Set[String] and/or Set[ConfigValue]. however note that you might want to convert to List before to avoid collapsing duplicates.
So if you have
val map: Set[Map[K, V]] = ???
val keys = map.flatMap(_.keySet) will give you Set[K]
val values = map.flatMap(_.values) will give you Set[V]
In both cases duplicates will be removed.
You could create a couple of functions that describe that computation, like:
val getKeys: Set[JavaMap.Entry[String, ConfigValue]] => Set[String] = _.map(_.getKey)
val getValues: Set[JavaMap.Entry[String, ConfigValue]] => Set[ConfigValue] = _.map(_.getValue)
Then when you need to extract one or the other you can call them like so:
val setOfKeyMap: Set[Map.Entry[String, ConfigValue]] = ???
...
val setOfKeys: Set[String] = getKeys(setOfKeyMap)
val setOfValues: Set[ConfigValue] = getValues(setOfKeyMap)
in my main program i receive inputs like -
key1=value1 key2=value2
Now what I want is to create a map out of it. I know the imperative way of doing this where I would get Array[String] that can be foreach and then split by "=" and then key and value can be used to form a Map.
is there a good functional and readable way to achieve this?
Also It will be great if I can avoid mutable Map and I want to avoid initial Dummy value initialization.
def initialize(strings: Array[String]): Unit = {
val m = collection.mutable.Map("dummy" -> "dummyval")
strings.foreach(
s => {
val keyVal:Array[String] = s.split("=")
m += keyVal(0) -> keyVal(1)
})
println(m)
}
you can just use toMap().
However, converting from array to tuple is not quite trivial:
How to convert an Array to a Tuple?
scala> val ar = Array("key1=value1","key2=value2")
ar: Array[String] = Array(key1=value1, key2=value2)
scala> ar.collect(_.split("=") match { case Array(x,y) => (x,y)}).toMap
res10: scala.collection.immutable.Map[String,String] = Map(key1 -> value1, key2 -> value2)
Maybe you have to call Function.unlift for intellij
val r = ar.collect(Function.unlift(_.split("=") match { case Array(x, y) => Some(x, y)})).toMap
similar to above but using only 'map'
ar.map(_.split("=")).map(a=>(a(0), a(1))).toMap
You can use Scopt to do the command line argument parsing in a neat way.
case class TargetClass(key: Any, value: Number, lowerBound: Double, upperBound: Double)
val keys: List[Any] = List("key1", "key2", "key3")
val values: List[Number] = List(1,2,3);
val lowerBounds: List[Double] = List(0.1, 0.2, 0.3)
val upperBounds: List[Double] = List(0.5, 0.6, 0.7)
Now I want to construct a List[TargetClass] to hold the 4 lists. Does anyone know how to do it efficiently? Is using for-loop to add elements one by one very inefficient?
I tried to use zipped, but it seems that this only applies for combining up to 3 lists.
Thank you very much!
One approach:
keys.zipWithIndex.map {
case (item,i)=> TargetClass(item,values(i),lowerBounds(i),upperBounds(i))
}
You may want to consider using the lift method to deal with case of lists being of unequal lengths (and thereby provide a default if keys is longer than any of the lists?)
I realise this doesn't address your question of efficiency. You could fairly easily run some tests on different approaches.
You can apply zipped to the first two lists, to the last two lists, then to the results of the previous zips, then map to your class, like so:
val z12 = (keys, values).zipped
val z34 = (lowerBounds, upperBounds).zipped
val z1234 = (z12.toList, z34.toList).zipped
val targs = z1234.map { case ((k,v),(l,u)) => TargetClass(k,v,l,u) }
// targs = List(TargetClass(key1,1,0.1,0.5), TargetClass(key2,2,0.2,0.6), TargetClass(key3,3,0.3,0.7))
How about:
keys zip values zip lowerBounds zip upperBounds map {
case (((k, v), l), u) => TargetClass(k, v, l, u)
}
Example:
scala> val zipped = keys zip values zip lowerBounds zip upperBounds
zipped: List[(((Any, Number), Double), Double)] = List((((key1,1),0.1),0.5), (((key2,2),0.2),0.6), (((key3,3),0.3),0.7))
scala> zipped map { case (((k, v), l), u) => TargetClass(k, v, l, u) }
res6: List[TargetClass] = List(TargetClass(key1,1,0.1,0.5), TargetClass(key2,2,0.2,0.6), TargetClass(key3,3,0.3,0.7))
It would be nice if .transpose worked on a Tuple of Lists.
for (List(k, v:Number, l:Double, u:Double) <-
List(keys, values, lowerBounds, upperBounds).transpose)
yield TargetClass(k,v,l,u)
I think no matter what you use from an efficiency point of view, you will have to traverse the lists individually. The only question is, do you do it OR for the sake of readability, you use Scala idioms and let Scala do the dirty work for you :) ?
Other approaches are not necessarily more efficient. You can change the order of zipping and the order of assembling the return value of the map function as you like.
Here is a more functional way but I am not sure it will be more efficient. See comments on #wwkudu (zip with index) answer
val res1 = keys zip lowerBounds zip values zip upperBounds
res1.map {
x=> (x._1._1._1,x._1._1._2, x._1._2, x._2)
//Of course, you can return an instance of TargetClass
//here instead of the touple I am returning.
}
I am curious, why do you need a "TargetClass"? Will a touple work?
What is the best way to convert a List[String, Int] A to List[Int, String] B. I wanted to use the map function which would iterate through all items in my list A and then return a new list B however whenever I apply the map function on the list A it complains about wrong number of arguments
val listA:List[(String, Int)] = List(("graduates", 20), ("teachers", 10), ("students", 300))
val listB:List[(Int, String)] = listA.map((x:String, y:Int) => y, x)
Any suggestions? Thanks
How about this:
val listB = listA.map(_.swap)
You need to use pattern matching to get the elements of a pair. I swear a question like this was asked just a few days ago....
listA.map{case (a,b) => (b,a)}
I need to implement a generic method that takes a tuple and returns a Map
Example :
val tuple=((1,2),(("A","B"),("C",3)),4)
I have been trying to break this tuple into a list :
val list=tuple.productIterator.toList
Scala>list: List[Any] = List((1,2), ((A,B),(C,3)), 4)
But this way returns List[Any] .
I am trying now to find out how to iterate over the following tuple ,for example :
((1,2),(("A","B"),("C",3)),4)
in order to loop over each element 1,2,"A",B",...etc. How could I do this kind of iteration over the tuple
What about? :
def flatProduct(t: Product): Iterator[Any] = t.productIterator.flatMap {
case p: Product => flatProduct(p)
case x => Iterator(x)
}
val tuple = ((1,2),(("A","B"),("C",3)),4)
flatProduct(tuple).mkString(",") // 1,2,A,B,C,3,4
Ok, the Any-problem remains. At least that´s due to the return type of productIterator.
Instead of tuples, use Shapeless data structures like HList. You can have generic processing, and also don't lose type information.
The only problem is that documentation isn't very comprehensive.
tuple.productIterator map {
case (a,b) => println(a,b)
case (a) => println(a)
}
This works for me. tranform is a tuple consists of dataframes
def apply_function(a: DataFrame) = a.write.format("parquet").save("..." + a + ".parquet")
transform.productIterator.map(_.asInstanceOf[DataFrame]).foreach(a => apply_function(a))