Say i have case classes like this.
case class someClass0(content: someClass1)
case class someClass1(someContent: Option[Map[String, someClass2]])
case class someClass2(someKey: Array[Int])
I need to delete items in Map(which is immutable) by values.
This values i get through iteration.
val keys_to_remove = new ListBuffer[String]()
val keys_to_keep: List[Int] = List(100, 200)
for (x <- keys_to_keep) {
content.someContent.get.foreach {
case (key: String, value: someClass2) => {
if (!value.someKey.contains(x)) {
keys_to_remove.append(key)
}
}
}
}
So, how to keep all the structure, and delete only needed items by key?
I was trying to change type of Map like
content.someContent.map(_.to(collection.mutable.Map))
But content.someContent.get.remove(key) is not working.
What am i doing wrong?
You don't need mutability for that.
val keys_to_keep: List[String] = List("a", "b")
val res = content.someContent.map(
_.filterKeys(k => !keys_to_keep.contains(k))
)
filterKeys filters a Map by testing each entries' key against a condition.
Of course, it is important to remember that you can't test contains on a List[Int] against Strings, as the result will always be false.
Furthermore, try looking up style-guides for Scala:
Classes are usually named in upper camel case
Values and variables are usually named in lower camel case
Try it out
You can do it using - operator and foldLeft on keys to remove.
you are using get for get value, if you want do it safety, you need to use:
content.someContent.map(immutableMap =>
keys_to_remove.foldLeft(immutableMap){
(map, key) =>
map - key
}).getOrElse(Map.empty[String, SomeClass2])
this works like in this example:
import scala.collection.mutable.ListBuffer
val immutableMap = Map("a" -> 1, "b" -> 2, "c" -> 3, "d" -> 4)
val keys_to_remove: ListBuffer[String] = ListBuffer("b", "d")
println(immutableMap) // Map(a -> 1, b -> 2, c -> 3, d -> 4)
val mapWithoutKeys = keys_to_remove.foldLeft(immutableMap){
(map, key) =>
map - key
}
println(mapWithoutKeys) //Map(a -> 1, c -> 3)
Here's how you can do it:
val optionalMap = someClass0.content.someContent.map {
contentMap => contentMap - keyToBeRemoved
}
val originalStructure = someClass0.copy(content = SomeClass1(optionalMap))
Here's the Scastie
This will remove all keys and keep structure
val someClass0_copy = someClass0.copy(content = Content(someContent = someClass0.content. someContent.map(_.removedAll(keysToRemove)))
Related
I have a Scala map collection that looks something like this:
var collection = Map((A,B) -> 1)
The key is (A,B) and the value is 1.
My question: If I use collection.head._1, the result is (A,B) which is correct. But I want to extract A only, without B, as I need to compare A with some other variable. So the final result should be A stored in a different variable.
I tried to use collection.head._1(0) which results in error
Any does not take parameters
You can try:
val collection = Map(("A","B") -> 1)
collection.map{ case ((a, b),v) => a -> v}
You can use keySet to get all the keys as a Set[(String, String)] and then map it into the first element of each:
val coll: Map[(String, String), Int] =
Map(
("one", "elephant") -> 1,
("two", "elephants") -> 2,
("three", "elephants") -> 3
)
/*
val myKeys = coll.keySet.map { case (x, _) => x }
// equivalent to:
val myKeys = coll.keySet.map(tup => tup._1)
// equivalent to: */
val myKeys = coll.keySet.map(_._1) // Set(one, two, three)
I'm using Scala 2.11 and i'm trying to update the value of a key in the tree map. I tried using updated:
private val xyz = List(0, 100000, 500000, 1000000)
private val abc = List (0, 5, 25, 50)
private var a = TreeMap.empty[Int, TreeMap[Int, Int]] ++ xyz.map {
aa => aa -> (TreeMap.empty[Int, Int] ++ abc.map(bb => bb -> 0))
}
a(xyz(0)).foreach {
case (key, value) =>
if (key < 50) {
a(xyz(0)) = a(xyz(0)).updated(key, 5)
}
}
And got the error:
value update is not a member of scala.collection.immutable.TreeMap[Int,scala.collection.immutable.TreeMap[Int,Int]]
Is it possible to update it? Or could someone please help me replicate the logic using a Java Tree Map since that will also allow me to use floorEntry and ceilingEntry functions. I tried converting to java tree map and it generated a regular map, not a tree map:
private var a = TreeMap.empty[Int, TreeMap[Int, Int]] ++ xyz.map {
aa => aa -> (TreeMap.empty[Int, Int] ++ abc.map(bb => bb -> 0)).asJava
}
private var b = a.asJava
You are getting confused between var/val and mutable/immutable.
I think you correctly understood the difference between val and var, that the former is an immutable variable and later is mutable. ie, if you try to reassign the object assigned as val you will get an error.
import scala.collection.immutable.TreeMap
val tm = TreeMap(1 -> 1, 2 -> 2, 3 -> 3)
tm = TreeMap(1->2)
^
error: reassignment to val
But a var can be mutated:
import scala.collection.immutable.TreeMap
var tm = TreeMap(1 -> 1, 2 -> 2, 3 -> 3)
tm = TreeMap(1->2)
// mutated tm
Notice that in the latter case, even though we are mutating the variable, we are not mutating the collection itself, we are assigning a new TreeMap. As we were using scala.collection.immutable.TreeMap it cant be mutated.
Instead, if we had used scala.collection.mutable.TreeMap, it has an update function
import scala.collection.mutable.TreeMap
val tm = TreeMap(1 -> 1, 2 -> 2, 3 -> 3)
tm.update(1, 5)
tm //TreeMap(1 -> 5, 2 -> 2, 3 -> 3)
Once you change scala.collection.immutable.TreeMap to scala.collection.mutable.TreeMap, this will work
a(xyz(0)).foreach{ case (key, value) =>
if(key < 50){
a(xyz(0)) = a(xyz(0)).updated(key, 5) //addOne(key, 5) if 2.13+
}
}
EDIT using java.util.TreeMap
private val xyz = List(0, 100000, 500000, 1000000)
private val abc = List(0, 5, 25, 50)
import java.util.{TreeMap => JTreeMap}
val jTreeMap = xyz.foldLeft(new JTreeMap[Int, JTreeMap[Int, Int]]()) { (acc, elem) =>
acc.put(
elem,
abc.foldLeft(new JTreeMap[Int, Int]()) { (acc2, elem2) =>
acc2.put(elem2, 0)
acc2
}
)
acc
}
//Map created
jTreeMap.get(xyz.head).replaceAll{
//hack for scala 2.11.x
new java.util.function.BiFunction[Int, Int, Int]{
def apply(key: Int, value: Int) = if (value < 5) 5 else value
}
}
//value edited
It is not possible to update an immutable object, you can only create a new immutable object from the old one. So the code needs to create a new TreeMap from the original one with different values as necessary.
The code looks like this:
val newMap = a.map{
case (k, v) if k == xyz(0) =>
k -> v.map {
case (k2, v2) if k2 < 50 =>
k2 -> 5
case (k2, v2) =>
k2 -> v2
}
case (k, v) =>
k -> v
}
This breaks down to an outer map that looks for matching keys in the outer TreeMap, and an inner map that looks for matching keys in the inner TreeMap. Pattern matching (case) is used to implement the match tests, and also to extract the keys and values.
Each map has one case that selects the values to be modified, and a second case that leaves other values unchanged. The first case returns the original key with a modified value while the second case just returns the original values (k -> v).
Also note that var applies to a variable, not the contents of a variable. It indicates whether the variable can be updated to refer to a different object, but says nothing about whether the object that the variable refers to can be updated. var is rarely used in Scala because it goes against a clean functional design.
What is the best way to apply a function to each element of a Map and at the end return the same Map, unchanged, so that it can be used in further operations?
I'd like to avoid:
myMap.map(el => {
effectfullFn(el)
el
})
to achieve syntax like this:
myMap
.mapEffectOnKV(effectfullFn)
.foreach(println)
map is not what I'm looking for, because I have to specify what comes out of the map (as in the first code snippet), and I don't want to do that.
I want a special operation that knows/assumes that the map elements should be returned without change after the side-effect function has been executed.
In fact, this would be so useful to me, I'd like to have it for Map, Array, List, Seq, Iterable... The general idea is to peek at the elements to do something, then automatically return these elements.
The real case I'm working on looks like this:
calculateStatistics(trainingData, indexMapLoaders)
.superMap { (featureShardId, shardStats) =>
val outputDir = summarizationOutputDir + "/" + featureShardId
val indexMap = indexMapLoaders(featureShardId).indexMapForDriver()
IOUtils.writeBasicStatistics(sc, shardStats, outputDir, indexMap)
}
Once I have calculated the statistics for each shard, I want to append the side effect of saving them to disk, and then just return those statistics, without having to create a val and having that val's name be the last statement in the function, e.g.:
val stats = calculateStatistics(trainingData, indexMapLoaders)
stats.foreach { (featureShardId, shardStats) =>
val outputDir = summarizationOutputDir + "/" + featureShardId
val indexMap = indexMapLoaders(featureShardId).indexMapForDriver()
IOUtils.writeBasicStatistics(sc, shardStats, outputDir, indexMap)
}
stats
It's probably not very hard to implement, but I was wondering if there was something in Scala already for that.
Function cannot be effectful by definition, so I wouldn't expect anything convenient in scala-lib. However, you can write a wrapper:
def tap[T](effect: T => Unit)(x: T) = {
effect(x)
x
}
Example:
scala> Map(1 -> 1, 2 -> 2)
.map(tap(el => el._1 + 5 -> el._2))
.foreach(println)
(1,1)
(2,2)
You can also define an implicit:
implicit class TapMap[K,V](m: Map[K,V]){
def tap(effect: ((K,V)) => Unit): Map[K,V] = m.map{x =>
effect(x)
x
}
}
Examples:
scala> Map(1 -> 1, 2 -> 2).tap(el => el._1 + 5 -> el._2).foreach(println)
(1,1)
(2,2)
To abstract more, you can define this implicit on TraversableOnce, so it would be applicable to List, Set and so on if you need it:
implicit class TapTraversable[Coll[_], T](m: Coll[T])(implicit ev: Coll[T] <:< TraversableOnce[T]){
def tap(effect: T => Unit): Coll[T] = {
ev(m).foreach(effect)
m
}
}
scala> List(1,2,3).tap(println).map(_ + 1)
1
2
3
res24: List[Int] = List(2, 3, 4)
scala> Map(1 -> 1).tap(println).toMap //`toMap` is needed here for same reasons as it needed when you do `.map(f).toMap`
(1,1)
res5: scala.collection.immutable.Map[Int,Int] = Map(1 -> 1)
scala> Set(1).tap(println)
1
res6: scala.collection.immutable.Set[Int] = Set(1)
It's more useful, but requires some "mamba-jumbo" with types, as Coll[_] <: TraversableOnce[_] doesn't work (Scala 2.12.1), so I had to use an evidence for that.
You can also try CanBuildFrom approach: How to enrich a TraversableOnce with my own generic map?
Overall recommendation about dealing with passthrough side-effects on iterators is to use Streams (scalaz/fs2/monix) and Task, so they've got an observe (or some analog of it) function that does what you want in async (if needed) way.
My answer before you provided example of what you want
You can represent effectful computation without side-effects and have distinct values that represent state before and after:
scala> val withoutSideEffect = Map(1 -> 1, 2 -> 2)
withoutSideEffect: scala.collection.immutable.Map[Int,Int] = Map(1 -> 1, 2 -> 2)
scala> val withSideEffect = withoutSideEffect.map(el => el._1 + 5 -> (el._2 + 5))
withSideEffect: scala.collection.immutable.Map[Int,Int] = Map(6 -> 6, 7 -> 7)
scala> withoutSideEffect //unchanged
res0: scala.collection.immutable.Map[Int,Int] = Map(1 -> 1, 2 -> 2)
scala> withSideEffect //changed
res1: scala.collection.immutable.Map[Int,Int] = Map(6 -> 6, 7 -> 7)
Looks like the concept you're after is similar to the Unix tee
utility--take an input and direct it to two different outputs. (tee
gets its name from the shape of the letter 'T', which looks like a
pipeline from left to right with another line branching off downwards.)
Here's the Scala version:
package object mypackage {
implicit class Tee[A](a: A) extends AnyVal {
def tee(f: A => Unit): A = { f(a); a }
}
}
With that, we can do:
calculateStatistics(trainingData, indexMapLoaders) tee { stats =>
stats foreach { case (featureShardId, shardStats) =>
val outputDir = summarizationOutputDir + "/" + featureShardId
val indexMap = indexMapLoaders(featureShardId).indexMapForDriver()
IOUtils.writeBasicStatistics(sc, shardStats, outputDir, indexMap)
}
}
Note that as defined, Tee is very generic--it can do an effectful
operation on any value and then return the original passed-in value.
Call foreach on your Map with your effectfull function. You original Map will not be changed as Maps in scala are immutable.
val myMap = Map(1 -> 1)
myMap.foreach(effectfullFn)
If you are trying to chain this operation, you can use map
myMap.map(el => {
effectfullFn(el)
el
})
I have a Map instance (immutable):
val source = Map(
("foo", "spam"),
("bar", "hoge"),
("baz", "eggs"),
("qux", "corge"),
("quux", "grault")
)
and I have number of keys (Set or List) in some order which may or may not exist in source map:
baz
foo
quuuuux // does not exist in a source map
But what is the best and cleanest way to iterate over the source map with concise scala style, filter it by my keys and place filtered items into resulting map in the same order as keys are?
Map(baz -> eggs, foo -> spam)
P.S. To clarify - order of keys in resulting map must be the same as in filtration keys list
If you have:
val source = Map(
"foo" -> "spam",
"bar" -> "hoge",
"baz" -> "eggs",
"qux" -> "corge",
"quux" -> "grault"
)
and
val keys = List( "baz", "foo", "quuuux" )
Then, you can:
import scala.collection.immutable.SortedMap
SortedMap(source.toSeq:_*).filter{ case (k,v) => keys.contains(k) }
val keys = List("foo", "bar")
val map = Map("foo" -> "spam", "bar" -> "hoge", "baz" -> "eggs")
keys.foldLeft(ListMap.empty[String, String]){ (acc, k) =>
map.get(k) match {
case Some(v) => acc + (k -> v)
case None => acc
}
}
This will iterate over the keys, building a map containing only the matching keys.
Please note that you need a ListMap to preserve the ordering of keys, although the implementation of ListMap will return the elements in the opposite order they were inserted (since keys are prepended as head of the list)
LinkedHashMap would ensure exact insertion order, but it's a mutable data structure.
If you need an ordered Map, you could use something like a TreeMap with a custom key ordering. So given
import scala.collection.immutable.TreeMap
val source = Map(
("foo", "spam"),
("bar", "hoge"),
("baz", "eggs"),
("qux", "corge"),
("quux", "grault")
)
val order: IndexedSeq[String] = IndexedSeq("baz", "foo", "quuuuux")
implicit val keyOrdering: Ordering[String] = Ordering.by(order.indexOf)
You have choice, either iterate over the ordered keys:
val result1: TreeMap[String, String] = order.collect {
case key if source.contains(key) => key -> source(key)
}(collection.breakOut)
// or a bit shorter
val result2: TreeMap[String, String] = order.flatMap { key => source.get(key).map(key -> _) }(collection.breakOut)
or filter from the source map:
val result3: TreeMap[String, String] = TreeMap.empty ++ source.filterKeys(order.contains)
I am not sure which one would be the most efficient, but I suspect the flatMap one might be fastest, at least for your simple example. Though, imho, the last example is better readable than the others.
I want to sort a scala map by key length. The map looks something like:
val randomMap = Map("short" -> "randomVal1", "muchlonger" -> "randomVal2")
I want to sort randomMap such that when I iterate over it, I will start with the longest key first...so it should iterate over the "muchlonger" element first.
Convert it to a sequence of key/value pairs and apply a sorting criteria. eg.:
randomMap.toSeq.sortBy(_._1.length).reverse
(reverse because it sorts by shortest to longest by default).
One option would be to define a custom ordering for a TreeMap. TreeMap is a sorted implementation of Map
import scala.collection.immutable.TreeMap
implicit object LengthOrder extends Ordering[String] {
def compare(s1: String, s2: String) = s1.length - s2.length
}
val randomMap = TreeMap("111" -> "111", "1" -> "1", "11" -> "11")
//randomMap: TreeMap[String,String] = Map(1 -> 1, 11 -> 11, 111 -> 111)
val keys = randomMap.keys
//keys: Iterable[String] = Set(1, 11, 111)
Note that this will affect all TreeMap[String]s where LengthOrder is in scope. In your project you could nest LengthOrder in another object (or put it in its own package) and then only import it inside the specific code blocks that need it.
Edit:
#Bharadwaj made a good point about how this would destroy all but one keys that have the same length. Something like this would fix this issue:
implicit object LengthOrder extends Ordering[String] {
def compare(s1: String, s2: String) = s1.length - s2.length match {
case 0 => s1.compareTo(s2)
case x => x
}
}