I'm trying to write a small wrapper class to make the Gson library a bit more Scala friendly. Unfortunately, I'm running into a compile error when I try to get this going the way I would like.
This is the code I've got so far:
package com.test
import com.google.gson.{JsonObject, JsonElement}
import scala.collection.Iterator
import scala.collection.immutable.Map
case class GsonMap ( private val inner: JsonObject = new JsonObject )
extends Map[String, JsonElement] {
/** {#inheritDoc} */
override def iterator: Iterator[(String, JsonElement)]
= new Iterator[(String, JsonElement)] {
private val entries = inner.entrySet.iterator
override def hasNext: Boolean = entries.hasNext
override def next: (String, JsonElement) = {
val elem = entries.next
( elem.getKey, elem.getValue )
}
}
/**
* Returns a clone of the inner JsonObject
*/
private def cloneInner: JsonObject = {
val result = new JsonObject()
iterator.foreach { (item) => result.add( item._1, item._2 ) }
result
}
/** {#inheritDoc} */
override def + ( kv: (String, JsonElement) ): GsonMap = {
val cloned = cloneInner
cloned.add( kv._1, kv._2 )
GsonMap( cloned )
}
/** {#inheritDoc} */
override def get( key: String ): Option[JsonElement]
= Option( inner.get(key) )
/** {#inheritDoc} */
override def - ( key: String ): GsonMap = {
val cloned = cloneInner
cloned.remove( key )
GsonMap( cloned )
}
}
Now, I know that the + method doesn't match what is defined in the Map class. That's the problem, really. I want the + method to accept JsonElements and return a GsonMap, but I'm not really sure how to make that work. I've tried a few variations at this point, but with no luck
for reference, this is the compile error I'm receiving:
[info] Compiling 1 Scala source to target/scala-2.9.2/classes...
[error] src/main/scala/GsonMap.scala:7: class GsonMap needs to be abstract, since method + in trait Map of type [B1 >: com.google.gson.JsonElement](kv: (String, B1))scala.collection.immutable.Map[String,B1] is not defined
[error] case class GsonMap ( val inner: JsonObject = new JsonObject )
[error] ^
[error] src/main/scala/GsonMap.scala:31: method + overrides nothing
[error] override def + ( kv: (String, JsonElement) ): GsonMap = {
[error] ^
[error] two errors found
Any advice out there about this?
UPDATE:
As was also suggested below, this is one of the variations I tried:
override def +[T >: JsonElement] ( kv: (String, T) ): GsonMap = {
val cloned = cloneInner
cloned.add( kv._1, kv._2 )
GsonMap( cloned )
}
However, it fails too:
[info] Compiling 1 Scala source to target/scala-2.9.2/classes...
[error] /src/main/scala/GSON.scala:33: type mismatch;
[error] found : T
[error] required: com.google.gson.JsonElement
[error] cloned.add( kv._1, kv._2 )
[error] ^
[error] one error found
My understanding of the >: operator is that T must be a parent of JsonElement, which I don't think is what I'm looking for. In this case, this map can only ever contain instances of JsonElements, so it wouldn't be appropriate to put in parents of JsonElements.
The direct cause of your error is that your + only accepts JsonElement, while the + in the trait expects a type parameter with an upper bound of JsonElement.
override def +[T >: JsonElement] ( kv: (String, T) ): GsonMap = {
val cloned = cloneInner
cloned.add( kv._1, kv._2 )
GsonMap( cloned )
}
The reason is (as pointed out in #Frank's answer) is that Map is covariant in its value argument, i.e. if Child is a subtype of Parent, Map[String,Parent] will be a supertype of Map[String, Child], and this add definition allows you to "up-add" to a Map:
scala> class Element;
defined class Element
scala> class SubElement extends Element;
defined class SubElement
scala> val m = Map("foo"-> new SubElement)
m: scala.collection.immutable.Map[java.lang.String,SubElement] = Map(foo -> SubElement#6a63afa4)
scala> m + ("bar" -> new Element)
res0: scala.collection.immutable.Map[java.lang.String,Element] = Map(foo -> SubElement#2e7ff81e, bar -> Element#654ab15b)
scala> m + ("bar" -> new Element) + ("baz" -> "Text")
res1: scala.collection.immutable.Map[java.lang.String,java.lang.Object] = Map(foo -> SubElement#6a63afa4, bar -> Element#233d0d04, baz -> Text)
If you're trying to implement the immutable Map trait on a mutable backing object, you will have to provide this "up-casting" yourself, or you can give in to the warm embrace of the Scala standard library and instead extend mutable.Map, which already does precisely that for you. If your Java type implements the java.util.Map interface, there's even ready-made wrappers and implicit conversions in scala.collection.JavaConversions.
I don't know what you're trying to do with your custom Map, but it's fairly likely that extending Map isn't the way to go at all (the example for extending maps in the standard intro to the Scala collection library implements a new data structure) and you rather want to deal with Scala maps in most of your code and then provide an implicit to e.g. convert a Map to the GSON equivalent at the boundaries.
The error is pretty detailled and to the point: you try to overwrite something which isn't in the base class and you have not implemented a required method.
In terms of a solution, what you essentially missed is the variance annotation that Map uses. Look at the ScalaDoc for the Map class and you will see this: Map[A, +B]. This little + is causing your problems.
To understand what's going on, I'd suggest you read up on covariance, and then understand why the + method has a different type signature and does not return a Map[A, B], but instead a Map[A, B1], where B1 >: B. You should do the same, as this will also allow you to not only keep a map of invariant JsonElement objects, but profit from the covariance when you have subclasses.
The "+" method needs to have the following signature: +[B1 >: B](kv: (A, B1)): Map[A, B1]
More of an observation than an answer: your GSonMap has a constructor which receives an JsonObject and uses it internally. It also exposes the JsonObject as a public field. The problem is that JsonObject is mutable, and because of the way you expose it in GsonMap the latter also becomes mutable (that's because anyone can modify the JsonObject from the exterior).
So please consider cloning the JsonObject in the constructor and exposing inner as a method that returns a cloned copy of JsonObject instead of the internal object. In this way the immutability of the GsonMap is guaranteed.
Related
I am writing a small Scala Program which should:
Read a file (line by line) from a local FS
Parse from each line three double values
Make instances of a case class based on those three values
Pass those instances to a Binary Heap
To be able to parse Strings to both Doubles and CoordinatePoints I've came up with this trait:
trait Parseable[T] {
def parse(input: String): Either[String, T]
}
and I have a number of type object implementations for the latter:
object Parseable {
implicit val parseDouble: Parseable[Double] = new Parseable[Double] {
override def parse(input: String): Either[String, Double] = {
val simplifiedInput = input.replaceAll("[ \\n]", "").toLowerCase
try Right(simplifiedInput.toDouble) catch {
case _: NumberFormatException =>
Left(input)
}
}
}
implicit val parseInt: Parseable[Int] = new Parseable[Int] {
override def parse(input: String): Either[String, Int] = {
val simplifiedInput = input.replaceAll("[ \\n]", "").toLowerCase
try Right(simplifiedInput.toInt) catch {
case _: NumberFormatException =>
Left(input)
}
}
}
implicit val parseCoordinatePoint: Parseable[CoordinatePoint] = new Parseable[CoordinatePoint] {
override def parse(input: String): Either[String, CoordinatePoint] = {
val simplifiedInput = input.replaceAll("[ \\n]", "").toLowerCase
val unparsedPoints: List[String] = simplifiedInput.split(",").toList
val eithers: List[Either[String, Double]] = unparsedPoints.map(parseDouble.parse)
val sequence: Either[String, List[Double]] = eithers.sequence
sequence match {
case Left(value) => Left(value)
case Right(doublePoints) => Right(CoordinatePoint(doublePoints.head, doublePoints(1), doublePoints(2)))
}
}
}
}
I have a common object that delegates the call to a corresponding implicit Parseable (in the same file):
object InputParser {
def parse[T](input: String)(implicit p: Parseable[T]): Either[String, T] = p.parse(input)
}
and just for reference - this is the CoordinatePoint case class:
case class CoordinatePoint(x: Double, y: Double, z: Double)
In my main program (after having validated that the file is there, and is not empty, etc..) I want to transform each line into an instance of CoordinatePoint as follows:
import Parseable._
import CoordinatePoint._
...
private val bufferedReader = new BufferedReader(new FileReader(fileName))
private val streamOfMaybeCoordinatePoints: Stream[Either[String, CoordinatePoint]] = Stream
.continually(bufferedReader.readLine())
.takeWhile(_ != null)
.map(InputParser.parse(_))
and the error I get is this:
[error] /home/vgorcinschi/data/eclipseProjects/Algorithms/Chapter 2 Sorting/algorithms2_1/src/main/scala/ca/vgorcinschi/algorithms2_4/selectionfilter/SelectionFilter.scala:42:27: ambiguous implicit values:
[error] both value parseDouble in object Parseable of type => ca.vgorcinschi.algorithms2_4.selectionfilter.Parseable[Double]
[error] and value parseInt in object Parseable of type => ca.vgorcinschi.algorithms2_4.selectionfilter.Parseable[Int]
[error] match expected type ca.vgorcinschi.algorithms2_4.selectionfilter.Parseable[T]
[error] .map(InputParser.parse(_))
[error] ^
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed Sep 1, 2020 10:38:18 PM
I don't understand nor know where to look for why is the compiler finding Parseable[Int] and Parseable[Double] but not the only right one - Parseable[CoordinatePoint].
So I thought, ok let me give the compiler a hand by specifying the transformation function from beforehand:
private val bufferedReader = new BufferedReader(new FileReader(fileName))
val stringTransformer: String => Either[String, CoordinatePoint] = s => InputParser.parse(s)
private val streamOfMaybeCoordinatePoints: Stream[Either[String, CoordinatePoint]] = Stream
.continually(bufferedReader.readLine())
.takeWhile(_ != null)
.map(stringTransformer)
Alas this yields the same error just a bit up the code - in the function declaration.
I would love to learn what is that that causes such behavior. Both to rectify the code and for personal knowledge. At this point I am very curious.
One fix is to specify type prameter explicitly
InputParser.parse[CoordinatePoint](_)
Another is to prioritize implicits. For example
trait LowPriorityParseable1 {
implicit val parseInt: Parseable[Int] = ...
}
trait LowPriorityParseable extends LowPriorityParseable1 {
implicit val parseDouble: Parseable[Double] = ...
}
object Parseable extends LowPriorityParseable {
implicit val parseCoordinatePoint: Parseable[CoordinatePoint] = ...
}
By the way, since you put implicits into the companion object it doesn't make much sense now to import them.
In the call site of
object InputParser {
def parse[T](input: String)(implicit p: Parseable[T]): Either[String, T] = p.parse(input)
}
type parameter T is inferred (if not specified explicitly) not before the implicit is resolved (type inference and implicit resolution make impact on each other). Otherwise the following code wouldn't compile
trait TC[A]
object TC {
implicit val theOnlyImplicit: TC[Int] = null
}
def materializeTC[A]()(implicit tc: TC[A]): TC[A] = tc
materializeTC() // compiles, A is inferred as Int
So during implicit resolution compiler tries to infer types not too early (otherwise in the example with TC type A would be inferred as Nothing and implicit wouldn't be found). By the way, an exception is implicit conversions where compiler tries to infer types eagerly (sometimes this can make troubles too)
// try to infer implicit parameters immediately in order to:
// 1) guide type inference for implicit views
// 2) discard ineligible views right away instead of risking spurious ambiguous implicits
https://github.com/scala/scala/blob/2.13.x/src/compiler/scala/tools/nsc/typechecker/Implicits.scala#L842-L854
The problem that the compiler does not inference and fix type parameter T in .map(InputParser.parse(_)) before trying to find the implicit in the second parameter list.
In the compiler, there is a concrete algorithm that infers types with its own logic, constraints, and tradeoffs. In that concrete compiler version that you use it first goes to the parameter lists and infer and checks types list by list, and only at the end, it infers type parameter by returning type (I do not imply that in other versions it differs, I only point out that it is implementation behavior not a fundamental constraint).
More precisely what is going on is that type parameter T is not being inferred or specified somehow at the step of typechecking of the second parameter list. T (at that point) is existential and it can be any/every type and there is 3 different implicit object that suitable for such type.
It is just how the compiler and its type inference works for now.
There's this question for Java How to declare a map with variable generics?
I have the exact same problem.
Is there a better/Scala way of solving it?
EDIT:
I tried following the above answer, but without implementing an internal map.
private class SectionConversionMap
extends HashMap[SectionSchema[_], (Iterable[HtmlRow]) => Option[Iterable[_]]]
{
override def +[T, B1 >: ( Iterable[HtmlRow] ) =>
Option[Iterable[T]]](kv: (SectionSchema[T], B1)):
HashMap[SectionSchema[_], B1] = {
val m = super.+(kv)
m
}
}
But my IDE keeps insisting that
Expression of type HashMap[SectionSchema[_], Any] doesn't conform to expected type HashMap[SectionSchema[_], B1]
Don't extend standard collection types, it's just asking for trouble unless you know very well what you are doing. In this case the compiler will ensure all method signatures are at least as general as before, and your + isn't (note that in Binzi Cao's answer it doesn't override anything!). Just follow the source and keep the Map in a field.
Here is a simple example, maybe it can give you some hint:
import scala.collection.mutable._
class MyMap extends HashMap[Any, List[Any]] {
def put[T, B <% T](key: T, value: List[B]) = {
this += key -> value
}
}
val b = new MyMap()
b.put("q", List("q"))
b.put(12, List(12))
b.put(122, List("23")) //not compile
The last line will not compile:
No implicit view available from String => Int.
It seems you want to ovrride the standard scala lib function, however, I guess, you could not change the method return types if you want to do a override. The below is the scala Map Method
#migration("`+` creates a new map. Use `+=` to add an element to this map and return that map itself.", "2.8.0")
def + [B1 >: B] (kv: (A, B1)): Map[A, B1] = clone().asInstanceOf[Map[A, B1]] += kv
I did something similar in your code and it can compile now, hopefully it is what you needed or give you some clue:
import scala.collection.mutable._
type HtmlRow = String
type SectionSchema[T] = List[T]
private class SectionConversionMap extends HashMap[SectionSchema[_], (Iterable[_]) => Option[Iterable[_]]] {
def +[T, B1 >: ( Iterable[HtmlRow] ) => Option[Iterable[T]]](kv: (SectionSchema[T], B1)):
HashMap[SectionSchema[_], B1] = {
val m = clone().asInstanceOf[HashMap[SectionSchema[_], B1]]
m+=kv
}
}
I'm having a dastard of a time implementing a Map in scala typed to (for our purposes) [String, Set[Foo]] that provides extra operations for the Foos in the values. The actual implementation is more complicated than presented below, but this is the gist. I need a class that implements all the Map-like functions and provides the extra-and-above aggregation on the collections that are the values of the map. The extends Map with MapLike patterns I've seen, particularly this, are not working.
What I've got so far:
import scala.collection.{immutable, Map, MapLike}
class Foo(a:Int)
class CustomMap
(val underlying:Map[String,Set[Foo]] = Map[String,Set[Foo]]())
extends Map[String, Set[Foo]] with MapLike[String, Set[Foo], CustomMap] {
override def empty = new CustomMap(underlying.empty)
def -(key: String) = new CustomMap(underlying - key)
def +[B1 >: Set[Foo]](kv: (String, B1)): scala.collection.Map[String,B1] = new CustomMap(underlying + (kv._1 -> kv._2))
def get(key: String): Option[Set[Foo]] = underlying.get(key)
def iterator: Iterator[(String, Set[Foo])] = underlying.iterator
override def size = underlying.size
def getAllFoos() = underlying.values.flatten.toSet
}
val cm1:CustomMap = new CustomMap(Map("a" -> Set(new Foo(1))))
val cm2:CustomMap = cm1 + ("a" -> Set(new Foo(2)))
println(cm2.getAllFoos)
There are issues both with the + method and accessing the extra aggregate methods.
Any pointers?
file.scala:12: error: type mismatch;
found : B1
required: Set[this.Foo]
def +[B1 >: Set[Foo]](kv: (String, B1)): scala.collection.Map[String,B1] = new CustomMap(underlying + (kv._1 -> kv._2))
^
file.scala:24: error: type mismatch;
found : scala.collection.Map[String,Set[this.Foo]]
required: this.CustomMap
val cm2:CustomMap = cm1 + ("a" -> Set(new Foo(2)))
^
two errors found
+ can't return a CustomMap because sometimes B1 won't be a Set[Foo] but some other supertype of Set[Foo]. This is the source of your errors. Map and MapLike are meant for classes that provide a map implementation that could safely have any value added to it and will return a usable Map. So a Map[String, Set[Foo]] can always have an ("", 5) put into it and become a Map[String, Any].
You can eliminate the wrapper around your underlying, as well as avoid these problems, by using the "pimp-my-library" pattern:
implicit class FooSetMap(val map: Map[String,Set[Foo]]) extends AnyVal {
def getAllFoos = map.values.flatten.toSet
}
If you are willing to use a mutable map, take a look at collection.mutable.MultiMap. It is a mixin trait which adds extra methods to subtypes of mutable.Map[A, mutable.Set[B]] for working with multimaps - you could do something similar for your needs.
I've created the following class:
class Foo[T] extends collection.mutable.HashMap[T, Int] {...}
(The class has some methods, but to reproduce this problem it doesn't need any.)
When I clone an instance of this class and try to use it as another instance of Foo, I get the following type error:
scala> val f = new Foo[String]
f: Foo[String] = Map()
scala> val anotherF: Foo[String] = f.clone
<console>:9: error: type mismatch;
found : scala.collection.mutable.HashMap[String,Int]
required: Foo[String]
How do I cast the result of f.clone as an instance of Foo[String] ?
Update A commenter pointed out that I could override the clone method on HashMap. No objections to doing so, but it seems like I should be able to do so without reimplementing the actual cloning process.
The HashMap class returns HashMap objects, whether it is extended or not. If you want to change the return value, you have to extend MapLike as well.
Following the comment from Rex Kerr, the approach presented in the question is just wrong.
Instead, the scala-idiomatic approach is to use implicit definitions to add the increment method to HashMap objects:
type Counter[T] = collection.mutable.HashMap[T, Int]
class WrappedHash[T](val h: collection.mutable.HashMap[T, Int]) {
def increment(key: T) = {
val currentKeyCount: Int = h.getOrElse(key, 0)
h ++= Map(key -> (currentKeyCount + 1))
}
}
implicit def wrapHash[T](h: collection.mutable.HashMap[T, Int]) = new WrappedHash[T](h)
I'm running into some kind of quirk in the Scala type system that has me a bit stumped. I am trying to make a class that extends Map[String,String] and I can't quite figure out how to implement the + method in such a way that the compiler accepts it.
Here's the code I have now:
class ParamMap(val pairs:List[(String,String)] = Nil) extends Map[String,String] {
lazy val keyLookup = Map() ++ pairs
override def get(key: String): Option[String] = keyLookup.get(key)
override def iterator: Iterator[(String, String)] = pairs.reverseIterator
/**
* Add a key/value pair to the map
*/
override def + [B1 >: String](kv: (String, B1)) = new ParamMap(kv :: pairs)
/**
* Remove all values for the given key from the map
*/
override def -(key: String): ParamMap = new ParamMap(pairs.filterNot(_._1 == key))
/**
* Remove a specific pair from the map
*/
def -(kv: (String, String)) : ParamMap = new ParamMap(pairs - kv)
}
Scala tells me this:
type mismatch; found: (String, B1) required: (String, String)
I believe this is because B1 is allowed to be a subtype of String but my constructor expects just a String (?). My original attempt was:
override def +(kv: (String, String)) = new ParamMap(kv :: pairs)
But this complained because the type signature didn't match the trait:
class ParamMap needs to be abstract, since method + in trait Map of type [B1 >: String](kv: (String, B1))scala.collection.immutable.Map[String,B1] is not defined
method + overrides nothing
I'm new to Scala and I think I'm getting over my head here in terms of how the type system works. Perhaps I'll try messing with casting but I have a feeling there might be a "better way" that, if I know it, will save me a lot of trouble in the future.
Any ideas?
Some background about Scala's type system.
The syntax B1 >: String means that B1 is a supertype of String. So B1 is less specific, and can't be cast to a String. Conversely, B1 <: String would be a subtype relationship.
The definition of the Map trait is Map [A, +B], where A represents the type of the key and B the type of the value. The +B notation says that Map is covariant in the key type, which means that T <: S implies Map[A, T] <: Map[A, S].
The full type of the Map.+ method is + [B1 >: B] (kv: (A, B1)): Map[A, B1]. The covariance of B kind of forces the use of B1 >: B. Here's an example of how it works: given a map m: Map[String, String] adding a key-value pair with a less specific type kv : (String, Any) will result in a less specific map, (m + kv): Map[String, Any].
The last point illustrates the problem with your ParamMap definition. According to the Map interface, one should be able to add a key of type Any to a map of type ParamMap <: Map[String, String] and get back a Map[String, Any]. But you're trying to define ParamMap.+ to always return ParamMap[String, String], which is incompatible with Map.+.
One way to fix the problem is to give ParamMap an explicit type parameter, something like (warning untested),
class ParamMap[B](val pairs:List[(String,String)] = Nil) extends Map[String, B] {
...
override def + [B1 >: B](kv: (String, B1)) = new ParamMap[B1](kv :: pairs)
}
but this may not be what you want. I don't think there's a way to fix the value type as String and implement the Map[String, String] interface.
Given all the above, why does the code in your answer compile? You've actually uncovered a limitation (unsoundness) of Scala's pattern matching, and it can lead to run-time crashes. Here's a simplified example:
def foo [B1 >: String](x: B1): Int = {
val (s1: Int, s2: Int) = (x, x)
s1
}
Although this compiles, it doesn't do anything useful. In fact, it will always crash with a MatchError:
scala> foo("hello")
scala.MatchError: (hello,hello) (of class scala.Tuple2)
at .foo(<console>:9)
at .<init>(<console>:10)
at .<clinit>(<console>)
...
In your answer, you've basically told the compiler to convert a B1 instance to a String, and if the conversion doesn't work, you'll get a runtime crash. It's equivalent to an unsafe cast,
(value: B1).asInstanceOf[String]
You're correct that your constructor expects a value of type List[String, String], but the issue isn't that B1 could be a subclass of String, it's that it could be a superclass -- this is what the B1 :> String notation indicates.
At first glance, you might wonder why the parent Map class would have the method typed this way. In fact, the return type of the + method you're attempting to override there is Map[String, B1]. In the context of a general map, though, this makes sense. Suppose you had the following code:
class Parent
class Child extends Parent
val childMap = Map[String, Child]("Key" -> new Child)
val result = childMap + ("ParentKey" -> new Parent)
The type of result would then have to be Map[String, Parent]. In light of this, the type restrictions on the + method in Map makes sense, but your fixed-type map isn't capable of fulfilling what the method is designed to be able to do. Its signature allows you to pass in a value of e.g. type (String, AnyRef), but using the method definition you gave in your followup answer, you'll get a MatchError when it tries to perform the assignment to key and value.
Does that make sense?
I ran the same problem with a colleague, when trying to build a Bag[T] which is a Map[T,Int]. We found two different solutions:
Implement Traversable rather than Map with appropriate Builder and CanBuildFrom, and add the useful map methods (get,+,-). If you need to pass the collection to a function taking maps as arguments, you can use implicit conversions. Here is our full Bag implementation: https://gist.github.com/1136259
Stay simple:
object collection {
type ParamMap = Map[String,String]
object ParamMap {
def apply( pairs: List[(String,String)] = Nil ) = Map( pairs:_* )
}
}
The compiler does seem to accept this one:
override def + [B1 >: String](kv: (String, B1)) = {
val (key:String, value:String) = kv
new ParamMap((key,value) :: pairs)
}
But I don't know why that is better than the original. I suppose this is an acceptable solution if nobody has a better one.