The Scala Collection library has mapValues and filterKeys. The reason it doesn't have mapKeys is likely the performance aspect (with regard to HashMap implementation), as discussed here for Haskell: Why there's not mapKeys in Data.Hashmap?
However.
Performance implications aside, I find myself needing mapKeys at least as much as mapValues, simply for massaging data (i.e. I'm using a map for data abstraction, not for its fetch speed).
Am I wrong, and which data model would you use for this? Tuples?
No idea why it's not in standard library, but you can easily pimp your library with implicit class
implicit class MapFunctions[A, B](val map: Map[A, B]) extends AnyVal {
def mapKeys[A1](f: A => A1): Map[A1, B] = map.map({ case (a, b) => (f(a), b) })
}
val m = Map(1 -> "aaa", 2 -> "bbb")
println(m.mapKeys(_ + 1))
You can use scalaz:
import scalaz.Scalaz._
val m = Map(1 -> "aaa", 2 -> "bbb")
m.mapKeys(_ + 1)
In case of collisions the result may be smaller than the original Map.
I guess it might be related to collision resolution.
Different keys after map may get the same values, e.g. "Key"=>key, "KEY"=>key
It is unclear how to resolve this type of conflicts without changing the value type (Set?) or overriding values which might be undesirable.
Related
Shapeless' HListOps includes a number of useful functions for their heterogeneous HList type. I couldn't find an equivalent for HMap.
Here is my goal. I have a simple Map[String, String] which is used as an options repository in the pipeline of message processing in quite a few places of my application. I now would like to add a different (Key => Value) to this map, by transforming it to a HMap, so it could be something like:
class HOptionsMap[K, V]
implicit val intToString = new HOptionsMap[String, String]
implicit val stringToInt = new HOptionsMap[String, Instant]
So I could further use it as follows:
val hm = HMap[HOptionsMap]("placeOfIncident" -> "Toronto", "incidentDate" -> Instant.now)
Except I would like to call operations like collect, fold, filter on the above, which are not supported (unlike with HList). This is a requirement in order to not break current functionality.
Of course I could use composition here but I would be curious if this would be achievable with Shapeless library.
If HMap is an inconvenient abstraction for your use case try to use a record
import shapeless.syntax.singleton._
val hm = "placeOfIncident" ->> "Toronto" :: "incidentDate" ->> Instant.now :: HNil
You can collect, fold, filter it as any HList/record.
https://github.com/milessabin/shapeless/wiki/Feature-overview:-shapeless-2.0.0#extensible-records
I had spent weeks on trying to understand the idea behind "lifting" in scala.
Originally, it was from the example related to Chapter 4 of book "Functional Programming in Scala"
Then I found below topic "How map work on Options in Scala?"
The selected answer specify that:
def map[B](f: A => B): Option[B] = this match (Let's considered this as (*) )
So, from above code, I assume that function "map" is derived from function match. Hence, the mechanism behind "map"
is a kind of pattern matching to provide a case selection between Some, and None
Then, I created below examples by using function map for Seq, Option, and Map (Let's considered below examples as (**) )
Example 1: map for Seq
val xs = Seq(1, 2, 3)
xs.map(println)
Example 2: map for Option
val a:Option[Int] = Some(5)
a.map(println)
val b:Option[Int] = None
b.map(println)
Example 3: map for Map
val capitals = Map("France" -> "Paris", "Japan" -> "Tokyo")
capitals.map(println)
From (*) and (**), I could not know whether "map" is a pattern matching or an iteration, or both.
Thank you for helping me to understand this.
#Jwvh provided a more programming based answer but I want to dig a little bit deeper.
I certainly appreciate you trying to understand how things work in Scala, however if you really want to dig that deep, I am afraid you will need to obtain some basic knowledge of Category Theory since there is no "idea behind lifting in scala" but just the "idea behind lifting"
This is also why functions like "map" can be very confusing. Inherently, programmers are taught map etc. as operations on collections, where as they are actually operations that come with Functors and Natural Transformations (this is normally referred to as fmap in Category Theory and also Haskell).
Before I move on, the short answer is it is a pattern matching in the examples you gave and in some of them it is both. Map is defined specifically to the case, the only condition is that it maintains functoriality
Attention: I will not be defining every single term below, since I would need to write a book to build up to some of the following definitions, interested readers are welcome to research them on their own. You should be able to get some basic understanding by following the types
Let's consider these as Functors, the definition will be something along the lines of this:
In (very very) short, we consider types as objects in the category of our language. The functions between these types (type constructors) are morphisms between types in this category. The set of these transformations are called Endo-Functors (take us from the category of Scala and drop us back in the category of Scala). Functors have to have a polymorphic (which actually has a whole different (extra) definition in category theory) map function, that will take some object A, through some type constructor turn it into object B.
implicit val option: Functor[Option] = new Functor[Option] {
override def map[A,B](optA: Option[A])(f: (A) => B): Option[B] = optA match{
case Some(a) => Some(f(a))
case _ => None
}
}
implicit val seq: Functor[Seq[_]] = new Functor[Seq[_]] {
override def map[A,B](sA: Seq[A])(f: (A) => B): Seq[B] = sA match{
case a :: tail => Seq(f(a), map(tail)(f))
case Nil => Nil
}
}
As you can see in the second case, there is a little bit of both (more of a recursion than iteration but still).
Now before the internet blows up on me, I will say you cant pattern match on Seq in Scala. It works here because the default Seq is also a List. I just provided this example because it is simpler to understand. The underlying definition something along the lines of that.
Now hold on a second. If you look at these types, you see that they also have flatMap defined on them. This means they are something more special than plain Functors. They are Monads. So beyond satisfying functoriality, they obey the monadic laws.
Turns out Monad has a different kind of meaning in the core scala, more on that here: What exactly makes Option a monad in Scala?
But again very very short, this means that we are now in a category where the endofunctors from our previous category are the objects and the mappings between them are morphisms (natural transformations), this is slightly more accurate because if you think about it when you take a type and transform it, you take (carry over) all of it's internal type constructors (2-cell or internal morphisms) with it, you do not only take this sole idea of a type without it's functions.
implicit val optionMonad: Monad[Option] = new Monad[Option] {
override def flatMap[A, B](optA: Option[A])(f: (A) => Option[B]): Option[B] = optA match{
case Some(a) => f(a)
case _ => None
}
def pure[A](a: A): Option[A] = Some(a)
//You can define map using pure and flatmap
}
implicit val seqMonad: Monad[Seq[_]] = new Monad[Seq[_]] {
override def flatMap[A, B](sA: Seq[A])(f: (A) => Seq[B]): Seq[B] = sA match{
case x :: xs => f(a).append(flatMap(tail)(f))
case Nil => Nil
}
override def pure[A](a: A): Seq[A] = Seq(a)
//Same warning as above, also you can implement map with the above 2 funcs
}
One thing you can always count on is map being having pattern match (or some if statement). Why?
In order to satisfy the identity laws, we need to have some sort of a "base case", a unit object and in many cases (such as Lists) those types are gonna be what we call either a product or coproduct.
Hopefully, this did not confuse you further. I wish I could get into every detail of this but it would simply take pages, I highly recommend getting into categories to fully understand where these come from.
From the ScalaDocs page we can see that the type profile for the Standard Library map() method is a little different.
def map[B](f: (A) => B): Seq[B]
So the Standard Library map() is the means to transition from a collection of elements of type A to the same collection but the elements are type B. (A and B might be the same type. They aren't required to be different.)
So, yes, it iterates through the collection applying function f() to each element A to create each new element B. And function f() might use pattern matching in its code, but it doesn't have to.
Now consider a.map(println). Every element of a is sent to println which returns Unit. So if a is List[Int] then the result of a.map(println) is List[Unit], which isn't terribly useful.
When all we want is the side effect of sending information to StdOut then we use foreach() which doesn't create a new collection: a.foreach(println)
Function map for Option isn't about pattern matching. The match/case used in your referred link is just one of the many ways to define the function. It could've been defined using if/else. In fact, that's how it's defined in Scala 2.13 source of class Option:
sealed abstract class Option[+A] extends IterableOnce[A] with Product with Serializable {
self =>
...
final def map[B](f: A => B): Option[B] =
if (isEmpty) None else Some(f(this.get))
...
}
If you view Option like a "collection" of either one element (Some(x)) or no elements (None), it might be easier to see the resemblance of how map transforms an Option versus, say, a List:
val f: Int => Int = _ + 1
List(42).map(f)
// res1: List[Int] = List(43)
List.empty[Int].map(f)
// res2: List[Int] = List()
Some(42).map(f)
// res3: Option[Int] = Some(43)
None.map(f)
// res4: Option[Int] = None
I would like to create a generic function that works for both an Array as an Option:
val numbers = Array(1, 2, 3)
val numberO: Option[Int] = Some(4)
def addOnes(numbers: ???[Int]) = numbers.map(_+1)
addOnes(numbers)
addOnes(numberO)
Right now I have a separate function for each structure
def addOnesForArray(numbers: Array[Int]) = numbers.map(_+1)
def addOnesForOption(numberO: Option[Int]) = numberO.map(_+1)
addOnesForArray(numbers)
addOnesForOption(numberO)
So basically I need a superclass of Array and Option that has the functor and monad methods map, flatMap, filter, ...
You could use structural typing (aka "duck typing"), with which we could say that you need literally "something with a map", written as { def map(): T }. But that's a) ugly, b) uses reflection, and c) hard (note that map(): T is just an example; in reality you will have to match the exact signature of map, with CanBuildFrom and all that jazz).
Better way would be to reach for scalaz or cats and use category theory concepts. Just pick the least powerful abstraction that does the job. If you need just map, then it's a Functor. If you want to map with (curried) functions of more than one parameter, then it's an Applicative. If you also want flatMap, then it's a monad, etc.
Example for functor:
import scalaz._, Scalaz._
def addOne[F[Int]](f: F[Int])(implicit m: Functor[F]) = f.map(_ + 1)
val numbers = Array(1, 2, 3).toList
val numberO = Option(123)
addOne(numbers) // List(2, 3, 4)
addOne(numberO) // Some(124)
You will notice that I had to convert your array to a List because there are no typeclass instances (that I know of) for functors, applicatives, monads etc. that work on arrays. But arrays are old fashioned and invariant and really not idiomatic Scala anyway. If you get them from elsewhere, just convert them to Lists, Vectors etc. (based on your use case) and work with those from that point onwards.
In general I agree with #slouc . If you want to make existing classes kinda extend some other trait you need typeclasses.
But in your particular case it is not required since Option and Array are both Traversable:
object Example extends App {
def plus1(data: Traversable[Int]): Traversable[Int] = data.map(x => x + 1)
println(plus1(Array(1, 2, 3)))
println(plus1(Some(4)))
}
I encountered some unauthorized strangeness working with Scala's SortedMap[A,B]. If I declare the reference to SortedMap[A,B] "a" to be of type Map[A,B], then map operations on "a" will produce a non-sorted map implementation.
Example:
import scala.collection.immutable._
object Test extends App {
val a: Map[String, String] = SortedMap[String, String]("a" -> "s", "b" -> "t", "c" -> "u", "d" -> "v", "e" -> "w", "f" -> "x")
println(a.getClass+": "+a)
val b = a map {x => x} // identity
println(b.getClass+": "+b)
}
The output of the above is:
class scala.collection.immutable.TreeMap: Map(a -> s, b -> t, c -> u, d -> v, e -> w, f -> x)
class scala.collection.immutable.HashMap$HashTrieMap: Map(e -> w, f -> x, a -> s, b -> t, c -> u, d -> v)
The order of key/value pairs before and after the identity transformation is not the same.
The strange thing is that removing the type declaration from "a" makes this issue go away. That's fine in a toy example, but makes SortedMap[A,B] unusable for passing to methods that expect Map[A,B] parameters.
In general, I would expect higher order functions such as "map" and "filter" to not change the fundamental properties of the collections they are applied to.
Does anyone know why "map" is behaving like this?
The map method, like most of the collection methods, isn't defined specifically for SortedMap. It is defined on a higher-level class (TraversableLike) and uses a "builder" to turn the mapped result into the correct return type.
So how does it decide what the "correct" return type is? Well, it tries to give you back the return type that it started out as. When you tell Scala that you have a Map[String,String] and ask it to map, then the builder has to figure out how to "build" the type for returning. Since you told Scala that the input was a Map[String,String], the builder decides to build a Map[String,String] for you. The builder doesn't know that you wanted a SortedMap, so it doesn't give you one.
The reason it works when you leave off the the Map[String,String] type annotation is that Scala infers that the type of a is SortedMap[String,String]. Thus, when you call map, you are calling it on a SortedMap, and the builder knows to construct a SortedMap for returning.
As far as your assertion that methods shouldn't change "fundamental properties", I think you're looking at it from the wrong angle. The methods will always give you back an object that conforms to the type that you specify. It's the type that defines the behavior of the builder, not the underlying implementation. When you think about like that, it's the type that forms the contract for how methods should behave.
Why might we want this?
Why is this the preferred behavior? Let's look at a concrete example. Say we have a SortedMap[Int,String]
val sortedMap = SortedMap[Int, String](1 -> "s", 2 -> "t", 3 -> "u", 4 -> "v")
If I were to map over it with a function that modifies the keys, I run the risk of losing elements when their keys clash:
scala> sortedMap.map { case (k, v) => (k / 2, v) }
res3: SortedMap[Int,String] = Map(0 -> s, 1 -> u, 2 -> v)
But hey, that's fine. It's a Map after all, and I know it's a Map, so I should expect that behavior.
Now let's say we have a function that accepts an Iterable of pairs:
def f(iterable: Iterable[(Int, String)]) =
iterable.map { case (k, v) => (k / 2, v) }
Since this function has nothing to do with Maps, it would be very surprising if the result of this function ever had fewer elements than the input. After all, map on a Iterable should produce the mapped version of each element. But a Map is an Iterable of pairs, so we can pass it into this function. So what happens in Scala when we do?
scala> f(sortedMap)
res4: Iterable[(Int, String)] = List((0,s), (1,t), (1,u), (2,v))
Look at that! No elements lost! In other words, Scala won't surprise us by violating our expectations about how map on an Iterable should work. If the builder instead tried to produce a SortedMap based on the fact that the input was a SortedMap, then our function f would have surprising results, and this would be bad.
So the moral of the story is: Use the types to tell the collections framework how to deal with your data. If you want your code to be able to expect that a map is sorted, then you should type it as SortedMap.
The signature of map is:
def
map[B, That](f: ((A, B)) ⇒ B)(implicit bf: CanBuildFrom[Map[A, B], B, That]): That
The implicit parameter bf is used to build the resulting collection. So in your example, since the type of a is Map[String, String], the type of bf is:
val cbf = implicitly[CanBuildFrom[Map[String, String], (String, String), Map[String, String]]]
Which just builds a Map[String, String] which doesn't have any of the properties of the SortedMap. See:
cbf() ++= List("b" -> "c", "e" -> "g", "a" -> "b") result
For more information, see this excellent article: http://docs.scala-lang.org/overviews/core/architecture-of-scala-collections.html
As dyross points out, it's the Builder, which is chosen (via the CanBuildFrom) on the basis of the target type, which determines the class of the collection that you get out of a map operation. Now this might not be the behaviour that you wanted, but it does for example allow you select the target type:
val b: SortedMap[String, String] = a.map(x => x)(collection.breakOut)
(breakOut gives a generic CanBuildFrom whose type is determined by context, i.e. our type annotation.)
So you could add some type parameters that allow you accept any sort of Map or Traversable (see this question), which would allow you do do a map operation in your method while retaining the correct type information, but as you can see it's not straightforward.
I think a much simpler approach is instead to define functions that you apply to your collections using the collections' map, flatMap etc methods, rather than by sending the collection itself to a method.
i.e. instead of
def f[Complex type parameters](xs: ...)(complex implicits) = ...
val result = f(xs)
do
val f: X => Y = ...
val results = xs map f
In short: you explicitly declared a to be of type Map, and the Scala collections framework tries very hard for higher order functions such as map and filter to not change the fundamental properties of the collections they are applied to, therefore it will also return a Map since that is what you explicitly told it you wanted.
(I'm using Scala nightlies, and see the same behaviour in 2.8.0b1 RC4. I'm a Scala newcomer.)
I have two SortedMaps that I'd like to form the union of. Here's the code I'd like to use:
import scala.collection._
object ViewBoundExample {
class X
def combine[Y](a: SortedMap[X, Y], b: SortedMap[X, Y]): SortedMap[X, Y] = {
a ++ b
}
implicit def orderedX(x: X): Ordered[X] = new Ordered[X] { def compare(that: X) = 0 }
}
The idea here is the 'implicit' statement means Xs can be converted to Ordered[X]s, and then it makes sense combine SortedMaps into another SortedMap, rather than just a map.
When I compile, I get
sieversii:scala-2.8.0.Beta1-RC4 scott$ bin/scalac -versionScala compiler version
2.8.0.Beta1-RC4 -- Copyright 2002-2010, LAMP/EPFL
sieversii:scala-2.8.0.Beta1-RC4 scott$ bin/scalac ViewBoundExample.scala
ViewBoundExample.scala:8: error: type arguments [ViewBoundExample.X] do not
conform to method ordered's type parameter bounds [A <: scala.math.Ordered[A]]
a ++ b
^
one error found
It seems my problem would go away if that type parameter bound was [A <% scala.math.Ordered[A]], rather than [A <: scala.math.Ordered[A]]. Unfortunately, I can't even work out where the method 'ordered' lives! Can anyone help me track it down?
Failing that, what am I meant to do to produce the union of two SortedMaps? If I remove the return type of combine (or change it to Map) everything works fine --- but then I can't rely on the return being sorted!
Currently, what you are using is the scala.collection.SortedMap trait, whose ++ method is inherited from the MapLike trait. Therefore, you see the following behaviour:
scala> import scala.collection.SortedMap
import scala.collection.SortedMap
scala> val a = SortedMap(1->2, 3->4)
a: scala.collection.SortedMap[Int,Int] = Map(1 -> 2, 3 -> 4)
scala> val b = SortedMap(2->3, 4->5)
b: scala.collection.SortedMap[Int,Int] = Map(2 -> 3, 4 -> 5)
scala> a ++ b
res0: scala.collection.Map[Int,Int] = Map(1 -> 2, 2 -> 3, 3 -> 4, 4 -> 5)
scala> b ++ a
res1: scala.collection.Map[Int,Int] = Map(1 -> 2, 2 -> 3, 3 -> 4, 4 -> 5)
The type of the return result of ++ is a Map[Int, Int], because this would be the only type it makes sense the ++ method of a MapLike object to return. It seems that ++ keeps the sorted property of the SortedMap, which I guess it is because ++ uses abstract methods to do the concatenation, and those abstract methods are defined as to keep the order of the map.
To have the union of two sorted maps, I suggest you use scala.collection.immutable.SortedMap.
scala> import scala.collection.immutable.SortedMap
import scala.collection.immutable.SortedMap
scala> val a = SortedMap(1->2, 3->4)
a: scala.collection.immutable.SortedMap[Int,Int] = Map(1 -> 2, 3 -> 4)
scala> val b = SortedMap(2->3, 4->5)
b: scala.collection.immutable.SortedMap[Int,Int] = Map(2 -> 3, 4 -> 5)
scala> a ++ b
res2: scala.collection.immutable.SortedMap[Int,Int] = Map(1 -> 2, 2 -> 3, 3 -> 4, 4 -> 5)
scala> b ++ a
res3: scala.collection.immutable.SortedMap[Int,Int] = Map(1 -> 2, 2 -> 3, 3 -> 4, 4 -> 5)
This implementation of the SortedMap trait declares a ++ method which returns a SortedMap.
Now a couple of answers to your questions about the type bounds:
Ordered[T] is a trait which if mixed in a class it specifies that that class can be compared using <, >, =, >=, <=. You just have to define the abstract method compare(that: T) which returns -1 for this < that, 1 for this > that and 0 for this == that. Then all other methods are implemented in the trait based on the result of compare.
T <% U represents a view bound in Scala. This means that type T is either a subtype of U or it can be implicitly converted to U by an implicit conversion in scope. The code works if you put <% but not with <: as X is not a subtype of Ordered[X] but can be implicitly converted to Ordered[X] using the OrderedX implicit conversion.
Edit: Regarding your comment. If you are using the scala.collection.immutable.SortedMap, you are still programming to an interface not to an implementation, as the immutable SortedMap is defined as a trait. You can view it as a more specialised trait of scala.collection.SortedMap, which provides additional operations (like the ++ which returns a SortedMap) and the property of being immutable. This is in line with the Scala philosophy - prefer immutability - therefore I don't see any problem of using the immutable SortedMap. In this case you can guarantee the fact that the result will definitely be sorted, and this can't be changed as the collection is immutable.
Though, I still find it strange that the scala.collection.SortedMap does not provide a ++ method witch returns a SortedMap as a result. All the limited testing I have done seem to suggest that the result of a concatenation of two scala.collection.SortedMaps indeed produces a map which keeps the sorted property.
Have you picked a tough nut to crack as a beginner to Scala! :-)
Ok, brief tour, don't expect to fully understand it right now. First, note that the problem happens at the method ++. Searching for its definition, we find it at the trait MapLike, receiving either an Iterator or a Traversable. Since y is a SortedMap, then it is the Traversable version being used.
Note in its extensive type signature that there is a CanBuildFrom being passed. It is being passed implicitly, so you don't normally need to worry about it. However, to understand what is going on, this time you do.
You can locate CanBuildFrom by either clicking on it where it appears in the definition of ++, or by filtering. As mentioned by Randall on the comments, there's an unmarked blank field on the upper left of the scaladoc page. You just have to click there and type, and it will return matches for whatever it is you typed.
So, look up the trait CanBuildFrom on ScalaDoc and select it. It has a large number of subclasses, each one responsible for building a specific type of collection. Search for and click on the subclass SortedMapCanBuildFrom. This is the class of the object you need to produce a SortedMap from a Traversable. Note on the instance constructor (the constructor for the class) that it receives an implicit Ordering parameter. Now we are getting closer.
This time, use the filter filter to search for Ordering. Its companion object (click on the small "o" the name) hosts an implicit that will generate Orderings, as companion objects are examined for implicits generating instances or conversions for that class. It is defined inside the trait LowPriorityOrderingImplicits, which object Ordering extends, and looking at it you'll see the method ordered[A <: Ordered[A]], which will produce the Ordering required... or would produce it, if only there wasn't a problem.
One might assume the implicit conversion from X to Ordered[X] would be enough, just as I had before looking more carefully into this. That, however, is a conversion of objects, and ordered expects to receive a type which is a subtype of Ordered[X]. While one can convert an object of type X to an object of type Ordered[X], X, itself, is not a subtype of Ordered[X], so it can't be passed as a parameter to ordered.
On the other hand, you can create an implicit val Ordering[X], instead of the def Ordered[X], and you'll get around the problem. Specifically:
object ViewBoundExample {
class X
def combine[Y](a: SortedMap[X, Y], b: SortedMap[X, Y]): SortedMap[X, Y] = {
a ++ b
}
implicit val orderingX = new Ordering[X] { def compare(x: X, y: X) = 0 }
}
I think most people initial reaction to Ordered/Ordering must be one of perplexity: why have classes for the same thing? The former extends java.lang.Comparable, whereas the latter extends java.util.Comparator. Alas, the type signature for compare pretty much sums the main difference:
def compare(that: A): Int // Ordered
def compare(x: T, y: T): Int // Ordering
The use of an Ordered[A] requires for either A to extend Ordered[A], which would require one to be able to modify A's definition, or to pass along a method which can convert an A into an Ordered[A]. Scala is perfectly capable of doing the latter easily, but then you have to convert each instance before comparing.
On the other hand, the use of Ordering[A] requires the creation of a single object, such as demonstrated above. When you use it, you just pass two objects of type A to compare -- no objects get created in the process.
So there are some performance gains to be had, but there is a much more important reason for Scala's preference for Ordering over Ordered. Look again on the companion object to Ordering. You'll note that there are several implicits for many of Scala classes defined in there. You may recall I mentioned earlier that an implicit for class T will be searched for inside the companion object of T, and that's exactly what is going on.
This could be done for Ordered as well. However, and this is the sticking point, that means every method supporting both Ordering and Ordered would fail! That's because Scala would look for an implicit to make it work, and would find two: one for Ordering, one for Ordered. Being unable to decide which is it you wanted, Scala gives up with an error message. So, a choice had to be made, and Ordering had more going on for it.
Duh, I forgot to explain why the signature isn't defined as ordered[A <% Ordered[A]], instead of ordered[A <: Ordered[A]]. I suspect doing so would cause the double implicits failure I have mentioned before, but I'll ask the guy who actually did this stuff and had the double implicit problems whether this particular method is problematic.