I'm upgrading a software scala based from scala 2.12 to scala 2.13.
Some blocks of code was broken and fixing them I found a wierd behavior in an already existing code.
#Edit
Below we have an explanation of a use case of this problem.
The problem in general is the usage of the method .map in mutable or immutable maps since Scala 2.13. At the end of the question we have an easy way to reproduce, and further we have the answer.
Obs.:
The term variables used there is a business field, its not related to
scala (or java) variables.
list is an Array[Long] containing idProject.
The return of p.getVariables is a Map[String, VariableSet].
So after the second .map (line 4), we have an Array[(Project,
Map[String, VariableSet])]
The code is:
// get the projects
list.map(projectById)
// and map their variable
.map(p => p -> p.getVariables)
// and transform them to DTOs
.map(pair => pair._2.map(set => toVariableDTO(set._2, pair._1.getId)).toArray)
// and return the union of all of them
.reduce((l, r) => l.union(r))
// and sort them by name
.sortBy(_.name.toLowerCase)
The problems comes at the 6th line, because after the upgrade it recognizes the set (pair._2.map(set => ) as type "Nothing" .
I tried line by line and it seems to work.
Like this:
val abs = list.map(projectById).map(p => p -> p.getVariables)
val ab = abs.map(pair => pair._2)
ab.map(pair => pair)
The problem here is that in the 6th line of previous example I need the reference to the project associated to that flow.
Of course, there would be room to rewrite this in another way (continuing the work on the second example), but I have many many other cases like this and would like to know if its really supposed not to work anymore of if I miss something during the upgrade.
Thanks in advance!
#Edit
Easy way to reproduce:
import scala.collection.mutable.{Map => MMap}
val mmap: MMap[String, Long] = MMap[String, Long]()
mmap.map(set => ) // Here, it recognizes 'set' as Nothing .
Looks like scala 2.13 see an element of Mutable Map as 'Nothing' ?
Well, after searching and struggling for hours, I figure out that this is one of the major changes in scala 2.13.
To use the expected behavior from the .map method, executed in map objects, we need to explicitly say that it should use the implementation from Iterable (which is the default on in scala 2.12 or lower).
We do that adding a .iterator before the .map call.
So for that, according to my "easy to reproduce" step would be like that:
import scala.collection.mutable.{Map => MMap}
val mmap: MMap[String, Long] = MMap[String, Long]()
mmap.iterator.map(set => set._2) // Now we may use the 'set' normally
I will make a few changes in my question to make it easier to find, for those who may have a similar problem.
Related
I'm new to scala and FP in general and trying to practice it on a dummy example.
val counts = ransomNote.map(e=>(e,1)).reduceByKey{case (x,y) => x+y}
The following error is raised:
Line 5: error: value reduceByKey is not a member of IndexedSeq[(Char, Int)] (in solution.scala)
The above example looks similar to staring FP primer on word count, I'll appreciate it if you point on my mistake.
It looks like you are trying to use a Spark method on a Scala collection. The two APIs have a few similarities, but reduceByKey is not part of it.
In pure Scala you can do it like this:
val counts =
ransomNote.foldLeft(Map.empty[Char, Int].withDefaultValue(0)) {
(counts, c) => counts.updated(c, counts(c) + 1)
}
foldLeft iterates over the collection from the left, using the empty map of counts as the accumulated state (which returns 0 is no value is found), which is updated in the function passed as argument by being updated with the found value, incremented.
Note that accessing a map directly (counts(c)) is likely to be unsafe in most situations (since it will throw an exception if no item is found). In this situation it's fine because in this scope I know I'm using a map with a default value. When accessing a map you will more often than not want to use get, which returns an Option. More on that on the official Scala documentation (here for version 2.13.2).
You can play around with this code here on Scastie.
On Scala 2.13 you can use the new groupMapReduce
ransomNote.groupMapReduce(identity)(_ => 1)(_ + _)
val str = "hello"
val countsMap: Map[Char, Int] = str
.groupBy(identity)
.mapValues(_.length)
println(countsMap)
Which of the following is better and why?
Case 1:
Map[String, String] result = Map.empty[String,String]
inputList.foreach(i => result += (i -> "hi") )
Case 2:
inputList.map(i => (i , "hi")).toMap
Which one is best is really a matter of use case. If this was a pipeline where I would pass/transform data and turning List[String] to Map[String, String] was just part of it, I would prefer to use .map with .toMap as it makes reading, maintenance and understanding easier
something.
.map(someMap)
.filter(condition)
.map(i => i -> "hi")
.toMap
.mapValues(something)
...
However, if this was used in some part of a framework that instead of Future or IOs used callbacks, and the only way for me to get the result would be appending it to something, .foreach would be one of options I would consider
val mutableMap = mutable.Map.empty[String,String]
someSadLibrary.onComplete { result =>
result.foreach { value =>
mutableMap += ...
}
}
If the code was on a really hot path I would probably take some immutable input, then use mutable structures to build result and finally return immutable result. That would also require .foreach and mutation (I almost never see a reason to do it in my code, more like in libraries internals).
Long story short, .map(...).toMap would be almost always better, but in some specific situations where it wouldn't be an option I would consider mutation.
Case 2
This is the functional way to implement it and is clearer about what the code is actually doing. If you are concerned about performance then using a view gives pretty much the same code as case 1 but, again, is clearer.
And to state the obvious, if case 2 is not possible then use case 1.
I wanted to create a scala compiler plugin that would prevents the call of some functions. For example System.exit.
The idea behind the scene is to let people write Scala scripts that would be interpreted on the fly. But I want to ensure that some dangerous actions are prohibited - the way it is done can definitely be discussed.
I started with the example in http://www.scala-lang.org/node/140 and started to replace the Apply section.
Doing some pattern matching I was able to extract a ClassSymbol for the right part of the compilation unit.
Then I wanted to do something like in the example that would be like:
classSymbol.tpe =:= global.typeOf[System]
unfortunately they don't match I get System.type on one side and System on the other.
Of course I could compare String values but I think that there is perhaps a nicer way to achieve this.
Has anyone any advice ?
Just in case a larger part of the code:
def apply(unit: global.CompilationUnit) {
for (global.Apply(fun, _) <- unit.body) {
fun.symbol match {
case method: global.MethodSymbol =>
val classSymbol = method.owner
println(classSymbol.fullName)
println(classSymbol.tpe =:= global.typeOf[System])
case _ => ()
}
}
Perhaps,
val jls = global.findMemberFromRoot(TermName("java.lang.System"))
Im using scala Map#get function, and for every accurate query it returns as Some[String]
IS there an easy way to remove the Some?
Example:
def searchDefs{
print("What Word would you like defined? ")
val selection = readLine
println(selection + ":\n\t" + definitionMap.get(selection))
}
When I use this method and use the following Input:
What Word would you like defined? Ontology
The returned Value is:
Ontology:
Some(A set of representational primitives with which to model a domain of knowledge or discourse.)
I would like to remove the Some() around that.
Any tips?
There are a lot of ways to deal with the Option type. First of all, however, do realize how much better it is to have this instead of a potential null reference! Don't try to get rid of it simply because you are used to how Java works.
As someone else recently stated: stick with it for a few weeks and you will moan each time you have to get back to a language which doesn't offer Option types.
Now as for your question, the simplest and riskiest way is this:
mymap.get(something).get
Calling .get on a Some object retrieves the object inside. It does, however, give you a runtime exception if you had a None instead (for example, if the key was not in your map).
A much cleaner way is to use Option.foreach or Option.map like this:
scala> val map = Map(1 -> 2)
map: scala.collection.immutable.Map[Int,Int] = Map(1 -> 2)
scala> map.get(1).foreach( i => println("Got: " + i))
Got: 2
scala> map.get(2).foreach( i => println("Got: " + i))
scala>
As you can see, this allows you to execute a statement if and only if you have an actual value. If the Option is None instead, nothing will happen.
Finally, it is also popular to use pattern matching on Option types like this:
scala> map.get(1) match {
| case Some(i) => println("Got something")
| case None => println("Got nothing")
| }
Got something
I personally like using .getOrElse(String) and use something like "None" as a default i.e. .getOrElse("None").
I faced similar issue, replaced with .Key() to resolve.
Solution:
definitionMap(selection)
In modern scala you can just map(key)
Let"s say I have a class Point with a toInt method, and I have an immutable Map[Point,V], for some type V. What is the most efficient way in Scala to convert it to an IntMap[V]? Here is my current implementation:
def pointMap2IntMap[T](points: Map[Point,T]): IntMap[T] = {
var result: IntMap[T] = IntMap.empty[T]
for(t <- points) {
result += (t._1.toInt, t._2)
}
result
}
[EDIT] I meant primarily faster, but I would also be interested in shorter versions, even if they are not obviously faster.
IntMap has a built-in factory method (apply) for this:
IntMap(points.map(p => (p._1.toInt, p._2)).toSeq: _*)
If speed is an issue, you may use:
points.foldLeft(IntMap.empty[T])((m, p) => m.updated(p._1.toInt, p._2))
A one liner that uses breakOut to obtain an IntMap. It does a map to a new collection, using a custom builder factory CanBuildFrom which the breakOut call resolves:
Map[Int, String](1 -> "").map(kv => kv)(breakOut[Map[Int, String], (Int, String), immutable.IntMap[String]])
In terms of performance, it's hard to tell, but it creates a new IntMap, goes through all the bindings and adds them to the IntMap. A handwritten iterator while loop (preceded with a pattern match to check if the source map is an IntMap) would possibly result in somewhat better performance.