I'm trying to use scala's LinkedHashMap as an LRU cache, but I'm not sure how to remove the oldest entry of such a map. I know java's LinkedHashMap has a method removeEldestEntry, but there does not seem to be a similar method for scala's implementation. I'd prefer not to convert to java's implementation just to have access to removeEldestEntry. How can I achieve this?
This will do what you want:
def removeOldestEntry[K](m: scala.collection.mutable.LinkedHashMap[K, _]): m.type =
m -= m.head._1
(Kudos to Jasper-M for pointing out that head will give the oldest entry)
You can do this in the following way:
object myApp {
def main(args: Array[String]) {
val myMap = new MyLinkedHashMap[Int,String]()
myMap.add(1, "a") // Map(1 -> a)
myMap.add(2, "b") // Map(1 -> a, 2 -> b)
myMap.add(3, "c") // Map(1 -> a, 2 -> b, 3 -> c)
myMap.add(4, "d") // Map(1 -> a, 2 -> b, 3 -> c, 4 -> d)
myMap.removeEldest // Map(2 -> b, 3 -> c, 4 -> d)
myMap.get(2) // Map(3 -> c, 4 -> d, 2 -> b)
myMap.removeEldest // Map(4 -> d, 2 -> b)
}
}
class MyLinkedHashMap[K,V] {
import scala.collection.mutable.LinkedHashMap
var map = new LinkedHashMap[K, V]()
/* adds an element to the HaskMap */
def add(key: K, value: V) {
map.put(key, value)
}
/* removes the LRU element from the HaskMap */
def removeEldest {
if (!map.isEmpty) {
map = map.drop(1)
}
}
/* gets the value for the given key and moves it to the top of the HashMap */
def get(key: K): Option[V] = {
val value = map.remove(key)
if (value != None) {
map.put(key, value.get)
}
return value
}
}
Related
I'm new in Scala, so struggling to understand Syntax. Please check below code.
def myDef(entityMap: Seq[(DataName.Value, DataFrame)]) : Seq[Map[Int,Info]] = {
val depenInfo = Seq[Map[Int,Info]]()
entityMap.foldLeft(depenInfo)((info,entity) => {
val(dataName: DataName.Value, df: DataFrame) = entity
info ++ df.createDepenInfo(dataName)
})
}
what am I getting is, Seq of tuples having to two types (DataName.Value, DataFrame) and return type of myDef is Seq of Map.
After that, create one empty Seq of Map, then feed this empty Map to entityMap.foldLeft so that it could add more values in it.
Remaining part I kind of literally stuck to understand. Can anyone please help me to understand what's happening ? If possible with any very simple example similar to above with output.
Thanks
Since there are many user defined class I don't know, I try to mock your data type as following:
import scala.collection.{immutable, Seq}
object Example {
object DataName {
type Value = Int
}
case class DataFrame(fakeData: String) {
def createDepenInfo(value: DataName.Value): Seq[Map[Int, Info]] = Seq(Map(value -> fakeData))
}
type Info = String
def myDef(entityMap: Seq[(DataName.Value, DataFrame)]): Seq[Map[Int, Info]] = {
val depenInfo = Seq[Map[Int, Info]]()
entityMap.foldLeft(depenInfo)((info: Seq[Map[Int, Info]], entity: (DataName.Value, DataFrame)) => {
// here is Pattern matching on tuples,
// here we extract (dataName: DataName.Value, df: DataFrame) from tuple entity: (DataName.Value, DataFrame)
// see: https://docs.scala-lang.org/tour/tuples.html
val (dataName: DataName.Value, df: DataFrame) = entity
// ++ is a method of Seq, it contact two Seq to one
// e.g. Seq(1,2,3) ++ Seq(4,5,6) = Seq(1,2,3,4,5,6)
info ++ df.createDepenInfo(dataName)
})
}
def main(args: Array[String]): Unit = {
val data: immutable.Seq[(DataName.Value, DataFrame)] = (1 to 5).map(i => (i, DataFrame((i + 'a').toChar.toString)))
// Vector((1,DataFrame(b)), (2,DataFrame(c)), (3,DataFrame(d)), (4,DataFrame(e)), (5,DataFrame(f)))
println(data)
val res = myDef(data)
// List(Map(1 -> b), Map(2 -> c), Map(3 -> d), Map(4 -> e), Map(5 -> f))
println(res)
}
}
raw data: Vector((1,DataFrame(b)), (2,DataFrame(c)), (3,DataFrame(d)), (4,DataFrame(e)), (5,DataFrame(f)))
let's say info ++ df.createDepenInfo(dataName) is result
info = Seq(), entity = (1,DataFrame(b)), reuslt=Seq(Map(1 -> b))
info = Seq(Map(1 -> b)), entity = (2,DataFrame(c)), result=Seq(Map(1 -> b), Map(2 -> c))
info = Seq(Map(1 -> b), Map(2 -> c)), entity = (3,DataFrame(d)), result=Seq(Map(1 -> b), Map(2 -> c), Map(3 -> d))
and so on...
You see, during each caluclation, the value info is "saved"(with a init value from deepInfo), and the entity value is "read" from entityMap.
So the final result is List(Map(1 -> b), Map(2 -> c), Map(3 -> d), Map(4 -> e), Map(5 -> f))
In your code, info is the accumulator, depenInfo is the initial value, and entity is a map item (i.e. a key-value tuple). Here's a simpler example where acc is the accumulator and kv is the key-value pair being "read" from the map.
Map(1->2).foldLeft(0)((acc, kv) => {
val (k, v) = kv;
println(s"$k, $v, $acc");
acc + k + v
})
// prints: 1, 2, 0
// result: 3
To read about the accumulator pattern: https://www.arothuis.nl/posts/accumulators-and-folds/
As for the ++, that is the operator/method in Seq (sequence) which concatenates this sequence to another sequence. Simple example with concatenating length 1 sequences together:
Seq(1) ++ Seq(2)
// Seq(1, 2)
I have defined the object below. But I don't understand why the mapValues body only executes in test1. ie. why is the output:
Calling test1
Calling test2
Mapping: One
Mapping: Two
Mapped: Map(1 -> Xx, 2 -> Xx)
I have tested it with both scala 2.10 and 2.11 with the same results.
object Test {
def test1: Option[String] = {
val map = Map(1 -> "One", 2 -> "Two")
val mapped = map.mapValues { v =>
println("Mapping: " + v)
"Xx"
}
None
}
def test2: Option[String] = {
val map = Map(1 -> "One", 2 -> "Two")
val mapped = map.mapValues { v =>
println("Mapping: " + v)
"Xx"
}
println("Mapped: " + mapped)
None
}
def main(args: Array[String]): Unit = {
println("Calling test1")
test1
println("Calling test2")
test2
}
}
mapValues actually returns a view, so the results are computed lazily. From the scaladoc for mapValues:
return a map view which maps every key of this map to f(this(key)). The resulting map wraps the original map without copying any elements.
So for example:
val mapped = Map(1 -> "One", 2 -> "Two").mapValues { v =>
println("Mapping: " + v)
"Xx"
}
On it's own this will print nothing when declared. But as soon as mapped is accessed, the values will be computed, and the statements will be printed. (In fact, the values will be re-computed every time you access mapped)
In Test.test1, there is nothing accessing mapped, so the values are never computed.
In Test.test2, you're printing out mapped, which triggers the computation of the values.
The other answer explains the problem, but as a solution, if you want a strict map, just use normal map:
val m = Map(1 -> "One", 2 -> "Two")
val mapped = m.map {
case (k,v) => k -> {
println("Mapping: " + v)
"Xx"
}
}
Alternatively, you can define your own extension method to do what you want:
import scala.collection.GenTraversableLike
import scala.collection.generic.CanBuildFrom
implicit class HasMapVals[T, U, Repr](val self: GenTraversableLike[(T, U), Repr]) extends AnyVal {
def mapVals[R, That](f: U => R)(implicit bf: CanBuildFrom[Repr, (T, R), That]) = {
self.map { case (k,v) => k -> f(v) }
}
}
val m = Map(1 -> "One", 2 -> "Two")
val mapped = m.mapVals { v =>
println("Mapping: " + v)
"Xx"
}
Why, in the below code example, does isAList's for comprehension yield a List, but the other two yield Maps? I can't think of any reason - the only difference seems to be that there is isAList's comprehension declares two variables, and the others declare one or zero.
object Weird {
def theMap: Map[Int, String] =
Map(1 -> "uno", 2 -> "dos", 3 -> "tres")
def main(args: Array[String]) {
val isAMap = for {
(key, value) <- theMap
} yield (key*2 -> value*2)
val isAlsoAMap = for {
(key, value) <- theMap
doubleKey = key*2
} yield (doubleKey -> value*2)
val isAList = for {
(key, value) <- theMap
doubleKey = key*2
doubleValue = value*2
} yield (doubleKey -> doubleValue)
println(isAMap)
println(isAlsoAMap)
println(isAList)
}
}
outputs
Map(2 -> unouno, 4 -> dosdos, 6 -> trestres)
Map(2 -> unouno, 4 -> dosdos, 6 -> trestres)
List((2,unouno), (4,dosdos), (6,trestres))
I am comparatively new to Scala, so apologies if I'm being incredibly naive about something!
Recently discussed on the ML:
https://groups.google.com/forum/#!msg/scala-internals/Cmh0Co9xcMs/D-jr9ULOUIsJ
https://issues.scala-lang.org/browse/SI-7515
Suggested workaround is to use a tuple to propagate the variables.
scala> for ((k,v) <- theMap; (dk,dv) = (k*2,v*2)) yield (dk,dv)
res8: scala.collection.immutable.Map[Int,String] = Map(2 -> unouno, 4 -> dosdos, 6 -> trestres)
More on the tupling mechanism:
What are the scoping rules for vals in Scala for-comprehensions
I create an ObservableMap, and a subscriber that just prints any events it receives (taken from here):
class MyMap extends HashMap[Int,Int] with ObservableMap[Int,Int]
class MySub extends Subscriber[Message[(Int,Int)] with Undoable, ObservableMap[Int, Int]] {
def notify(pub: ObservableMap[Int, Int], evt: Message[(Int, Int)] with Undoable) {
println(evt)
}
}
val map = new MyMap
map.subscribe(new MySub)
Using +=, ++=, and -= work as expected:
scala> map += 1 -> 1
Include(NoLo,(1,1))
res5: map.type = Map(1 -> 1)
scala> map ++= Map(2 -> 4, 3 -> 9)
Include(NoLo,(3,9))
Include(NoLo,(2,4))
res6: map.type = Map(3 -> 9, 1 -> 1, 2 -> 4)
scala> map -= 1
Remove(NoLo,(1,1))
res7: map.type = Map(3 -> 9, 2 -> 4)
But update doesn't work:
scala> map(4) = 16
scala> map
res9: MyMap = Map(3 -> 9, 4 -> 16, 2 -> 4)
Why? It looks like ObservableMap overrides +=, -=, and clear. Both ++= and update appear to be implemented in terms of += (by Growable and MapLike respectably), so why does it work on one but not the other?
The mutable version of HashMap calls update which in turn calls put, which does not call += so the observable += method doesn't get called. I'm using scala 2.9.1, but this should be the same for 2.8 on.
From HashMap:
override def put(key: A, value: B): Option[B] = {
val e = findEntry(key)
if (e == null) { addEntry(new Entry(key, value)); None }
else { val v = e.value; e.value = value; Some(v) }
}
override def update(key: A, value: B): Unit = put(key, value)
def += (kv: (A, B)): this.type = {
val e = findEntry(kv._1)
if (e == null) addEntry(new Entry(kv._1, kv._2))
else e.value = kv._2
this
}
I have a list of parent keys, each of which could possibly have zero or more associated values. I am not sure which collection to use.
I am using Map[Int,List[String]]
I am declaring the Map as
var nodes = new HashMap[Int, List[String]]
Then I have two methods to handle adding new elements. The first is to add new keys addNode and the second is to add new values addValue. Initially, the key will not have any values associated with it. Later on, during execution, new values will be associated.
def addNode(key: Int) = nodes += (key -> "")
def addValue(key: Int, value: String) = ???
I am not sure how to implement addValues
Update:
In response to #oxbow-lakes answer, This is the error I am receiving. Please note that keys need not have values associated with them.
scala> var nodes = Map.empty[Int, List[String]]
nodes: scala.collection.immutable.Map[Int,List[String]] = Map()
scala> nodes += (1->null)
scala> nodes += (1 -> ("one" :: (nodes get 1 getOrElse Nil)))
java.lang.NullPointerException
at .<init>(<console>:9)
at .<clinit>(<console>)
at .<init>(<console>:11)
at .<clinit>(<console>)
at $print(<console>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:704)
at scala.tools.nsc.interpreter.IMain$Request$$anonfun$14.apply(IMain.scala:920)
at scala.tools.nsc.interpreter.Line$$anonfun$1.apply$mcV$sp(Line.scala:43)
at scala.tools.nsc.io.package$$anon$2.run(package.scala:25)
at java.lang.Thread.run(Thread.java:680)
Update 2:
The problem with the code above is the line nodes += (1->null) the key should be associated with Nil instead. Below is the working code.
scala> var nodes = Map.empty[Int, List[String]]
nodes: scala.collection.immutable.Map[Int,List[String]] = Map()
scala> nodes += (1->Nil)
scala> nodes += (1 -> ("one" :: (nodes get 1 getOrElse Nil)))
scala> nodes
res27: scala.collection.immutable.Map[Int,List[String]] = Map(1 -> List(one))
Using MultiMap
You possibly want to use MultiMap, which is a mutable collection isomorphic to Map[K, Set[V]]. Use as follows:
import collection.mutable
val mm = new mutable.HashMap[Int, mutable.Set[String]] with mutable.MultiMap[Int, String]
Then you add your nodes:
mm addBinding (key, value)
Without MultiMap
The alternative is to stick with immutable values. Assuming you want to avoid using lenses (see scalaz), you can add nodes as follows:
nodes += (key -> (value :: (nodes get key getOrElse Nil)))
Here it is working (in response to your comment):
scala> var nodes = Map.empty[Int, List[String]]
nodes: scala.collection.immutable.Map[Int,List[String]] = Map()
scala> def addNode(key: Int, value: String) =
| nodes += (key -> (value :: (nodes get key getOrElse Nil)))
addNode: (key: Int, value: String)Unit
scala> addNode(1, "Hi")
scala> addNode(1, "Bye")
scala> nodes
res2: scala.collection.immutable.Map[Int,List[String]] = Map(1 -> List(Bye, Hi))
Using Scalaz
Using the scalaz library, you can realize that this is simply using the Empty pattern:
nodes += (key -> (value :: ~(nodes get key)))
Or you could take advantage of the fact that Map is a monoid:
nodes = nodes |+| Map(key -> List(value))
In addition to #oxbow_lakes' answer, here's a idea for how you could use an addMap method that correctly adds two maps together (ie, combining lists for matching keys, adding new lists for new keys):
class EnhancedListMap(self: Map[Int,List[String]]) {
def addMap(other: Map[Int,List[String]]) =
(this.ungroup ++ enhanceListMap(other).ungroup)
.groupBy(_._1)
.mapValues(_.map(_._2))
def ungroup() =
self.toList.flatMap{ case (k,vs) => vs.map(k -> _) }
}
implicit def enhanceListMap(self: Map[Int,List[String]]) = new EnhancedListMap(self)
And you'd use it like this:
val a = Map(1 -> List("a","b"), 2 -> List("c","d"))
val b = Map(2 -> List("e","f"), 3 -> List("g","h"))
a addMap b
//Map(3 -> List(g, h), 1 -> List(a, b), 2 -> List(c, d, e, f))
You can include addNode, addValue, and addValues the same way (to EnhancedListMap above):
def addNode(key: Int) =
if(self contains key) self else self + (key -> Nil)
def addValue(key: Int, value: String) =
self + (key -> (value :: (self get key getOrElse Nil)))
def addValues(key: Int, values: List[String]) =
self + (key -> (values ::: (self get key getOrElse Nil)))
And then use them together:
var nodes = Map.empty[Int, List[String]]
// Map()
nodes = nodes.addNode(1)
// Map(1 -> List())
nodes = nodes.addValue(1,"a")
// Map(1 -> List(a))
nodes = nodes.addValue(2,"b")
// Map(1 -> List(a), 2 -> List(b))
nodes = nodes.addValues(2,List("c","d"))
// Map(1 -> List(a), 2 -> List(c, d, b))
nodes = nodes.addValues(3,List("e","f"))
// Map(1 -> List(a), 2 -> List(c, d, b), 3 -> List(e, f))
nodes = nodes.addMap(Map(3 -> List("g","h"), 4-> List("i","j")))
// Map(1 -> List(a), 2 -> List(c, d, b), 3 -> List(e, f, g, h), 4 -> List(i, j))
I quite like the getOrElseUpdate method provided by mutable maps:
import scala.collection.mutable._
private val nodes = new HashMap[Int, Buffer[String]]
def addNode(key: Int): Unit =
nodes.getOrElseUpdate(key, new ArrayBuffer)
def addValue(key: Int, value: String): Unit =
nodes.getOrElseUpdate(key, new ArrayBuffer) += value