From the following code:
type Structure = Map[String, Any]
def getStructureSourceNames(structuresDesc: Structure): Iterable[String] = {
val subColsDesc: Map[String, String] =
structuresDesc.filter(_._2.isInstanceOf[String]).asInstanceOf[Map[String, String]]
val subStructuresDesc: Map[String, Structure] = structuresDesc
.filter(_._2.isInstanceOf[Map[String, Structure]])
.asInstanceOf[Map[String, Structure]]
subColsDesc.values ++ subStructuresDesc.values.flatMap(getStructureSourceNames(_))
}
I want to pass a recursive Map of (String -> String), ia. an example of Structure is:
Map("test" -> Map(
"newid" -> "id",
"newstring" -> "string",
"toto" -> Map("newdouble" -> "double")
),
"otherid" -> "id")
The method getStructureSourceNames should return the list of "final" value, ia. browse the whole tree and for each leaft, get the String value.
When I run this code, this drives me to:
Warning:(78, 32) non-variable type argument String in type scala.collection.immutable.Map[String,Structure] (the underlying of Map[String,Structure]) is unchecked since it is eliminated by erasure
.filter(_._2.isInstanceOf[Map[String, Structure]])
Moreover, I don't like to use isInstanceOf / asInstanceOf. By googling, I found that I could use pattern matching to check for the type, and get the Map with expected typing, but I can"t find how to do it.
Would you have an example of such code?
There are 2 kinds of pattern matching:
1) pattern matching on a sealed trait (good)
2) pattern matching where patterns involve matching with arbitrary classes and equality checks (not better than instanceOf checks)
To avoid 2) you need to make the type you want to match a sealed trait:
sealed trait ConfigValue
case class StringValue(v: String) extends ConfigValue
case class MapValue(map: Map[String, ConfigValue]) extends ConfigValue
val struct: ConfigValue = MapValue(Map("key1" -> StringValue("v1"),
"key2" -> MapValue(Map("sub" -> StringValue("val")))))
def allValues(s: ConfigValue): Iterable[String] = {
s match {
case StringValue(v) => Seq(v)
case MapValue(map) => map.values.flatMap(v => allValues(v))
}
}
println(allValues(struct))
By the way your structure looks similar to json. Maybe you could reuse some json library.
Related
I have a simple method to retrieve a nested key from a hashmap. I need to pattern match on Map[String,Any] so that I can keep iterating into the nested data until I get to a flat value:
def get(map: Map[String, Any], key: String): Any = {
var fields: mutable.Seq[String] = key.split('.')
var currentKey: String = fields.head
var currentValue: Any = map
while (fields.nonEmpty && currentValue.isInstanceOf[Map[String, Any]]) {
currentKey = fields.head
fields = fields.drop(1)
currentValue match {
case m: Map[String, Any] => currentValue = m.getOrElse(currentKey, None)
case _ =>
}
}
if (fields.nonEmpty) None else currentValue
}
It works when I use it only within scala, but if it gets called from java, I get the error non-variable type argument String in type scala.collection.immutable.Map[String,Any].
I've seen some other solutions that require you to refactor code and wrap the map in a case class, but that would be very disruptive to all the code that relies on this method. Is there any simpler fix?
You cannot pattern match on Map[String,Any] because of type erasure. The compiler will have warned of this. This code is just matching on Map[_,_] so it will succeed with any key type, not just String.
So the method is inherently buggy and it appears that calling from Java is exposing bugs that did not emerge when using Scala.
Since you are not using this from Java yet, I would switch to a typesafe implementation for the Java code and then migrate the legacy code to this version as quickly as possible. While this may be disruptive, it would be fixing a design error that introduced buggy code, so it should be done sooner rather than later. (Whenever you see Any used as a value type it is likely that the design went wrong at some point)
The typesafe version is not that difficult, here is an outline implementation:
class MyMap[T] {
trait MapType
case class Value(value: T) extends MapType
case class NestedMap(map: Map[String, MapType]) extends MapType
def get(map: Map[String, MapType], key: String): Option[T] = {
def loop(fields: List[String], map: Map[String, MapType]): Option[T] =
fields match {
case Nil =>
None
case field :: rest =>
map.get(field).flatMap{
case Value(res) => Some(res)
case NestedMap(m) => loop(rest, m)
}
}
loop(key.split('.').toList, map)
}
}
In reality MyMap should actually hold the Map data rather than passing it in to get, and there would be methods for safely building nested maps.
I have in scala a
case class foo(a: Array[String], b: Map[String,Any])
I am trying to run unit tests for this but the assertEquals while storing the foo elements(actual & expected) in a Arrays.
So the last line is using assertEquals(expected.deep, actual.deep).
The map b is showing correctly, but the assertEquals is trying the match the hashcode of the Array a instead of the content. the error is get is something like this:
Array(foo([Ljava.lang.string#235543a70,Map("bar" -> "bar")))
The overall code looks like
case class Foo(a: Array[String], b: Map[String, Any])
val foo = Foo(Array("1"), Map("ss" -> 1))
val foo2 = Foo(Array("1"), Map("ss" -> 1))
org.junit.Assert.assertEquals(Array(foo).deep, Array(foo2).deep)
How do you suggest making this work?
Case classes in scala come with their own hashCode and equals methods that should not be overridden. This implementation checks equality of the content. But it relies on "content" based hashCode and equals implementations of types used within the case class.
Unfortunately, this is not the case for Arrays, which means that Arrays cannot be checked by content with default equals method.
The easiest solution is to use data collections that check equality based on the content, like Seq.
case class Foo(a: Seq[String], b: Map[String, Any])
val foo = Foo(Seq("1"), Map("ss" -> 1))
val foo2 = Foo(Seq("1"), Map("ss" -> 1))
org.junit.Assert.assertEquals(foo, foo2)
Calling deep won't help you in this case as it scans and converts nested Array only.
println(Array(foo, Array("should be converted and printed")))
println(Array(foo, Array("should be converted and printed")).deep)
produces
[Ljava.lang.Object;#188715b5
Array(Foo([Ljava.lang.String;#6eda5c9,Map(ss -> 1)), Array(should be converted and printed))
This issue occurs for any API that can return multiple classes, but in a collection of type Any.
A specific example is handling JSON using the built-in JSON parser (scala.util.parsing.json): the value returned is a Map[String,Any] because the value in each JSON key-value pair can be any JSON type.
Extracting values from these nested Maps seems to require type testing and casting, which is rather ugly. In particular, we end up with multiple functions that are identical apart from the return type (e.g. String, Double, Map...), which is used for checking and casting.
Is it possible to abstract out this type so that only one generic get[T](...): T function is required, avoiding this boilerplate?
I have been looking at TypeTag but all the examples I've found so far look at abstracting over the argument type, not the return type.
To clarify: I'm aware that there are many other JSON parsers that provide much nicer interfaces with pattern matching etc, but I'm just interested in this general problem of refactoring, for dealing with legacy interfaces that return collections of Any.
import scala.util.parsing.json._
object ParseJSON {
val text = """{"str":"A String", "num":123, "obj": { "inner":"value" }}"""
val json = JSON.parseFull(text).get.asInstanceOf[Map[String,Any]]
//> ... Map(str -> A String, num -> 123.0, obj -> Map(inner -> value))
// Three essentially identical functions:
def getString(m:Map[String,Any], k:String): Option[String] = {
m.get(k).flatMap{ v =>
if (v.isInstanceOf[String]) Some(v.asInstanceOf[String]) else None
}
}
def getDouble(m:Map[String,Any], k:String): Option[Double] = {
m.get(k).flatMap{ v =>
if (v.isInstanceOf[Double]) Some(v.asInstanceOf[Double]) else None
}
}
def getObject(m:Map[String,Any], k:String): Option[Map[String, Any]] = {
m.get(k).flatMap{ v =>
if (v.isInstanceOf[Map[_,_]]) Some(v.asInstanceOf[Map[String,Any]])
else None
}
}
getString(json, "str") //> res0: Option[String] = Some(A String)
getString(json, "num") //> res1: Option[String] = None
getObject(json, "obj")
//> res3: Option[Map[String,Any]] = Some(Map(inner -> value))
}
I initially thought this could be solved via a generic class:
class Getter[T] {
def get(m: Map[String, Any], k: String): Option[T] = {
m.get(k).flatMap { v =>
if (v.isInstanceOf[T]) Some(v.asInstanceOf[T]) else None
}
}
}
new Getter[String].get(json, "str")
but as Oleg Pyzhcov pointed out (in my now-deleted answer), type erasure prevents this from detecting whether the types are correct at runtime.
The fix to your failed attempt is quite simple:
import scala.reflect.ClassTag
class Getter[T: ClassTag] {
def get(m: Map[String, Any], k: String): Option[T] = {
m.get(k).flatMap {
case v: T => Some(v)
case _ => None
}
}
}
new Getter[String].get(json, "str")
Pattern-matching against : T is handled specially when a ClassTag[T] is available.
Unfortunately, if you want T itself to be a generic type, type erasure strikes back: Getter[List[String]] can only check if it's passed a List, not its type parameter.
You probably want to use a JSON library for the purpose, I personally suggest Circe. It handles smoothly typesafe serialization and deserialization and can be extended easily. If you are looking for polymorphic deserialization though, you will need to store the type inside the JSON and Circe typically handles these too, via support of sealed hierarchy of case classes.
I am experimenting around with Scala and Play. I want to return a Tuple, in this case a Tuple3 but could be Tuple of any size. I want to serialize the Tuple as JSON, but Play doesn't seem to know how to serialize a Tuple.
I'm just trying to do something very simple like the following
def getClient(clientId: Int) = Action {
val result = ("I", "AM", "TUPLE")
Ok(Json.toJson(result))
}
No Json serializer found for type (String, String, String). Try to implement an implicit Writes or Format for this type.
I tried something like this but it only seems to work on Tuple2.
val seq = Seq[(String,String)](("attr1"->"val1"),("attr2"->"val2"))
val s = Json.toJson(seq.map(e => Json.obj((e._1 -> e._2))))
Ok(s).as(JSON)
You can create a case class like this.
case class MyCaseClass(string1: String, string2: String, string3: String)
then you have to add an implicit writer with
implicit val myCaseClassFormat = Json.format[MyCaseClass]
then you can do Json.toJson(MyCaseClass("I", "AM", "TUPLE"))
I have 2 case-classes:
case class OutlierPortal(portal: String, timeData: Seq[OutlierPortalTimeSeriesData])
and
case class OutlierPortalTimeSeriesData(period: Timestamp, totalAmount: Double, isOutlier: Int)
or respectively a Seq[OutlierPortal]
What I want to perform is similar to Scala Macros: Making a Map out of fields of a class in Scala, but I want to map a sequence of a (nested) case-classes to Seq[Map[String, Any]].
However, new to scala I fear a bit the proposed idea of macros. Is there a "simpler" way to map this Sequence of Seq[OutlierPortal] to Seq[Map[String, Any]]
Or would you recommend to start using macros even though a beginner in scala? For me a one-way conversion (case-class -> map) is enough.
If you're looking to avoid fancy tricks, and you don't have too many total classes to write this for, you can just write the methods to create the maps yourself. I'd suggest adding methods named something like toMap to your case classes. OutlierPortalTimeSeriesData's is simple if you use the Map() constructor:
case class OutlierPortalTimeSeriesData(period: Timestamp, totalAmount: Double, isOutlier: Int) {
def toMap: Map[String, Any] = Map(
"period" -> period,
"totalAmount" -> totalAmount,
"isOutlier" -> isOutlier)
}
I suppose there's some duplication there, but at least if you ever have a reason to change the string values but not the variable names, you have the flexibility to do that.
To take a sequence of something you can call toMap on, and turn it into a Seq[Map[String, Any]], just use map:
mySeq.map { _.toMap }
We can use this both to write OutlierPortal's toMap:
case class OutlierPortal(portal: String, timeData: Seq[OutlierPortalTimeSeriesData]) {
def toMap: Map[String, Any] = Map(
"portal" -> portal,
"timeData" -> timeData.map { _.toMap })
}
and then again to convert a Seq[OutlierPortal] to a Seq[Map[String, Any]].
Depending on how you're using these objects and methods, you might want to define a trait that distinguishes classes with this method, and have your case classes extend it:
trait HasToMap { def toMap: Map[String, Any] }
case class Blah( /* ... */ ) extends HasToMap {
def toMap: /* ... */ }
}
This would let you take a value that you know you can convert to Map[String, Any] (or a sequence of them, etc.) in a method that otherwise doesn't care which particular type it is.