I'd like to convert a Map[String, Any] to a given case class, and the map can be a nested map.
The case class can have certain fields as Optional, and if the map doesn't have the field in question , it'll have a default value.
eg.
case class CC(a: Int, b: Option[Int], c: String)
Map(a -> 1, c -> "abc")
result case class : CC(1, None, "abc")
The only caveat is I need to control the typeSignatures of these fields.
ex:
Map(a ->, b -> "10", c ->"abc"). \\Note b is string here
result case class : CC(1, Some(10), "abc")
I got some code that can do this exact behavior(code below), but I cant seem to make it work for nested case class
import scala.reflect.runtime.universe._
object DynamicClassFactory {
val classLoaderMirror = runtimeMirror(getClass.getClassLoader)
}
class DynamicClassFactory[T: TypeTag] {
import DynamicClassFactory.classLoaderMirror
val tpe = typeOf[T]
val classSymbol = tpe.typeSymbol.asClass
val companion = tpe.typeSymbol.companion
val classMirror = classLoaderMirror reflectClass classSymbol
val instanceMirror = classLoaderMirror.reflectModule(companion.asModule)
val objMirror = classLoaderMirror.reflect(instanceMirror.instance)
val constructorSymbol = tpe.decl(termNames.CONSTRUCTOR)
val constructorArgs = constructorSymbol.asMethod.paramLists.flatten.map{
p => (p.name.decodedName.toString, p.typeSignature)
}
val defaultConstructor =
if (constructorSymbol.isMethod) constructorSymbol.asMethod
else {
val ctors = constructorSymbol.asTerm.alternatives
ctors.map { _.asMethod }.find { _.isPrimaryConstructor }.get
}
val constructorMethod = classMirror reflectConstructor defaultConstructor
def buildParameters: List[String] = constructorArgs.map ( x => x._1 )
def getCompanionParam(name: String): Any =
if(classSymbol.isCaseClass) {
objMirror.reflectMethod(instanceMirror.symbol.info.decl(TermName(name)).asMethod)("test")
}
def getDefaults = {
val defaults = companion.typeSignature.members.filter { m => m.isMethod && m.name.toString.contains("apply$default")}.toList.map { x => (x.asMethod.name.toTermName.toString.replaceAll("[^0-9]", "").toInt-1, getCompanionParam(x.asMethod.name.toTermName.toString)) }.toMap
val defaultValues = scala.collection.mutable.Map[String, Any]()
for( (x, i) <- buildParameters.view.zipWithIndex ) {
if(defaults.contains(i)) {
defaultValues(x) = defaults(i)
}
}
defaultValues.toMap
}
def buildWith(args: Seq[_]): T = {
constructorMethod(args: _*).asInstanceOf[T]
}
def buildSafe(configuration: Map[String, Any]): T = {
val config = getDefaults ++ configuration
val safeConfig = safeTypeConvert(config, constructorArgs.toMap)
buildWith(constructorArgs.map { x => safeConfig(x._1) })
}
def safeTypeConvert(v: Map[String, Any], t: Map[String, Type]): Map[String, Any] = {
val o = scala.collection.mutable.Map[String, Any]()
val z = v.filter{
case (k, v) =>
t.contains(k)
}
z.foreach {
case (k, v) =>
val typeS: Type = t(k)
try {
if (typeS <:< typeOf[Option[String]]) {
v match {
case x: Int => o(k) = Some(x.toString)
case y: Option[Nothing] => o(k) = None
case _ => o(k) = Some(v.asInstanceOf[String])
}
}
else if (typeS <:< typeOf[Option[Int]]) {
v match {
case x: String => o(k) = Some(x.toFloat.toInt)
case y: Option[Nothing] => o(k) = None
case _ => o(k) = Some(v.asInstanceOf[Int])
}
}
else if (typeS <:< typeOf[Option[Boolean]]) {
v match {
case x: String => o(k) = Some(x.toLowerCase.toBoolean)
case y: Option[Nothing] => o(k) = None
case _ => o(k) = v
}
}
else if (typeS <:< typeOf[Int]) {
v match {
case x: String => o(k) = x.toFloat.toInt // this is to handle decimals returned from json
case _ => o(k) = v.asInstanceOf[Int]
}
} else if (typeS =:= typeOf[String]) {
v match {
case x: Int => o(k) = x.toString
case _ => o(k) = v.asInstanceOf[String]
}
} else if (typeS =:= typeOf[Boolean]) {
v match {
case x: String => o(k) = x.toLowerCase.toBoolean
case _ => o(k) = v
}
}
else if (typeS <:< typeOf[List[_]]) {
if (v == Nil) {
o(k) = List[String]()
} else {
o(k) = v.asInstanceOf[List[_]]
}
} else if (typeS <:< typeOf[Map[_, _]]) {
if (v == None) {
o(k) = Map()
} else o(k) = v
}
else {
throw new Exception(s"sdf $k must be of type ${typeS.typeSymbol.name.decoded}, got ${v.getClass.getName}: ${v.toString}")
}
} catch {
case e#(_: ClassCastException | _: NumberFormatException | _: IllegalArgumentException) =>
throw new Exception(s"$k must be of type ${typeS.typeSymbol.name.decoded}, got ${v.getClass.getName}: ${v.toString}")
}
}
return o.toMap
}
}
The above works great for a map which is not a nested map, but fails for a nested map
case class Address(addLine1: String, addLine2: Optional[String], zip: Long)
case class Inner(id: Option[String], addr: Option[Address], isPath: Option[Boolean])
case class Outer(name: String, addtlnInfo: Inner)
val map: Map[String, Any] = Map("name" -> "john", "addtlnInfo" -> Map("id" -> 123, "addr" -> Map("addLine1" -> "901 Lincoln St", "zip" -> "80101")))
val ins = new DynamicClassFactory[Outer]
ins.buildSafe(map)
I am trying and failing to get something like this to work in Scala 3:
type TupleK[K[*], V[*], A] = (K[A], V[A])
final class MapK[K[*], V[*]] private (val rawMap: Map[K[?], V[?]]) {
def foreach(f: TupleK[K, V, ?] => Unit): Unit = {
rawMap.foreach(f.asInstanceOf[Tuple2[K[?], V[?]] => Any])
}
}
object MapK {
def apply[K[*], V[*]](entries: TupleK[K, V, ?]*): MapK[K, V] = {
new MapK[K, V](Map(entries: _*))
}
}
With usage like this:
class Key[A]()
type Id[A] = A
val intKey = Key[Int]
val strKey = Key[String]
MapK[Key, Id](intKey -> 1, strKey -> "a")
In Scala 2 that works, just need to adjust syntax by replacing * and ? with _ (except in _* of course).
In Scala 3 however basically every line errors with "unreducible application of higher-kinded type to wildcard arguments": Scastie.
The docs say that existential types have been dropped in Scala 3, however they don't really give any non-trivial examples of how to deal with this.
The docs mention that "Existential types largely overlap with path-dependent types" – can this MapK be implemented with path-dependent types? I've read this but didn't understand how to apply that, or whether it's possible in my case.
And, if not path dependent types... then what? It seems unlikely that Scala was "simplified" to the point where it's impossible to implement this functionality anymore, so I must be missing something.
ETA: In addition to my own answers below, I made this repo and wrote this article about the various approaches to encoding MapK in Scala 3.
I was able to produce a working, albeit incredibly annoying, implementation. This pointer was especially valuable.
First, a few notes:
Type inference on this sucks on many levels. All the manual type ascribtions in the tests, and all of the implicit conversions below are needed for this to work.
Apparently Scala is not smart enough to figure out that A and type Id[A] = A are functionally the same when looking for implicits, so a combinatorial explosion of ad-hoc implicit conversions are needed. Ugly and not very scalable.
Observe the different options available in Scala 3: foreach, foreachT, and foreachK. All of them have stylistic tradeoffs.
If you can improve on any of this, please let me know. This works, but it was so much nicer in Scala 2.
MapK implementation:
class MapK[K[_], V[_]] protected(protected val rawMap: Map[Type[K], Type[V]]) {
def apply[A](key: K[A]): V[A] = {
rawMap(key).asInstanceOf[V[A]]
}
def updated[A](key: K[A], value: V[A]): MapK[K, V] = {
MapK.unsafeCoerce(rawMap.updated(key, value))
}
def updated[A](pair: (K[A], V[A])): MapK[K, V] = {
MapK.unsafeCoerce(rawMap.updated(pair._1, pair._2))
}
def foreach[A](f: ((K[A], V[A])) => Unit): Unit = {
rawMap.foreach(f.asInstanceOf[(([Type[K], Type[V]])) => Any])
}
def foreachT(f: Type.Tuple2[K, V] => Unit): Unit = {
foreach { (k, v) => f((k, v)) }
}
def foreachK(f: [A] => (K[A], V[A]) => Unit): Unit = {
foreach { (k, v) => f(k, v) }
}
}
object MapK {
def unsafeCoerce[K[_], V[_]](rawMap: Map[Type[K], Type[V]]): MapK[K, V] = {
new MapK[K, V](rawMap)
}
def apply[K[_], V[_]](entries: Type.Tuple2[K, V]*): MapK[K, V] = {
new MapK[K, V](Map(entries.asInstanceOf[Seq[(Type[K], Type[V])]]: _*))
}
}
Other methods in MapK that you might want to implement basically follow the same patterns as foreach, foreachT, or foreachK.
And now, usage:
def test(caption: String)(code: => Unit): Unit = code
def assertEquals[A](a: A, b: A): Unit = assert(a == b)
case class Key[A](label: String, default: A)
val boolKey = Key[Boolean]("bool", false)
val intKey = Key[Int]("int", 0)
val strKey = Key[String]("str", "")
val optionMap = MapK[Key, Option](boolKey -> Some(true), intKey -> Some(1), strKey -> Option("a"), strKey -> None)
val idMap = MapK[Key, Id](boolKey -> true, intKey -> 1, strKey -> "hello")
val expectedOptionValues = List[Type.Tuple3[Key, Option, Id]](
(boolKey, Some(true), false),
(intKey, Some(1), 0),
(strKey, None, "")
)
val expectedIdValues = List[Type.Tuple3[Key, Id, Id]](
(boolKey, true, false),
(intKey, 1, 0),
(strKey, "hello", "")
)
test("optionMap - apply & updated") {
assertEquals(optionMap(intKey), Some(1))
assertEquals(optionMap(strKey), None)
assertEquals(optionMap.updated(strKey, Some("yo"))(strKey), Some("yo"))
}
test("optionMap - foreach") {
var values: List[Type.Tuple3[Key, Option, Id]] = Nil
optionMap.foreach { (k, v) =>
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedOptionValues)
}
test("optionMap - foreachT") {
var values: List[Type.Tuple3[Key, Option, Id]] = Nil
optionMap.foreachT { pair => // no parameter untupling :(
values = values :+ (pair._1, pair._2, pair._1.default)
}
assertEquals(values, expectedOptionValues)
}
test("optionMap - foreachK") {
var values: List[Type.Tuple3[Key, Option, Id]] = Nil
optionMap.foreachK {
[A] => (k: Key[A], v: Option[A]) => // need explicit types :(
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedOptionValues)
}
test("idMap - apply & updated") {
assertEquals(idMap(intKey), 1)
assertEquals(idMap(strKey), "hello")
assertEquals(idMap.updated(strKey, "yo")(strKey), "yo")
}
test("idMap - foreach") {
var values: List[Type.Tuple3[Key, Id, Id]] = Nil
idMap.foreach { (k, v) =>
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedIdValues)
}
test("idMap - foreachT") {
var values: List[Type.Tuple3[Key, Id, Id]] = Nil
idMap.foreachT { pair =>
values = values :+ (pair._1, pair._2, pair._1.default)
}
assertEquals(values, expectedIdValues)
}
test("idMap - foreachK") {
var values: List[Type.Tuple3[Key, Id, Id]] = Nil
idMap.foreachK {
[A] => (k: Key[A], v: A) =>
values = values :+ (k, v, k.default)
}
assertEquals(values, expectedIdValues)
}
And now, the support cast that makes this work:
import scala.language.implicitConversions // old style, but whatever
type Id[A] = A
type Type[F[_]] <: (Any { type T })
object Type {
type Tuple2[F[_], G[_]] <: (Any { type T })
type Tuple3[F[_], G[_], H[_]] <: (Any { type T })
}
implicit def wrap[F[_], A](value: F[A]): Type[F] =
value.asInstanceOf[Type[F]]
implicit def wrapT2[F[_], G[_], A](value: (F[A], G[A])): Type.Tuple2[F, G] =
value.asInstanceOf[Type.Tuple2[F, G]]
implicit def wrapT2_P1[F[_], A](t: (F[A], A)): Type.Tuple2[F, Id] = wrapT2[F, Id, A](t)
implicit def wrapT3[F[_], G[_], H[_], A](value: (F[A], G[A], H[A])): Type.Tuple3[F, G, H] =
value.asInstanceOf[Type.Tuple3[F, G, H]]
implicit def wrapT3_P1[F[_], G[_], A](value: (F[A], A, A)): Type.Tuple3[F, Id, Id] =
value.asInstanceOf[Type.Tuple3[F, Id, Id]]
implicit def wrapT3_P1_P2[F[_], G[_], A](value: (F[A], G[A], A)): Type.Tuple3[F, G, Id] =
value.asInstanceOf[Type.Tuple3[F, G, Id]]
implicit def unwrap[F[_]](value: Type[F]): F[value.T] =
value.asInstanceOf[F[value.T]]
implicit def unwrapT2[F[_], G[_]](value: Type.Tuple2[F, G]): (F[value.T], G[value.T]) =
value.asInstanceOf[(F[value.T], G[value.T])]
implicit def unwrapT3[F[_], G[_], H[_]](value: Type.Tuple3[F, G, H]): (F[value.T], G[value.T], H[value.T]) =
value.asInstanceOf[(F[value.T], G[value.T], H[value.T])]
Here's an alternative solution, using dependent types. In general I like it better, it's more obvious to me what's going on.
import scala.language.implicitConversions
type Id[A] = A
implicit def wrapId[A](a: A): Id[A] = a
implicit def unwrapId[A](a: Id[A]): A = a
case class Key[A](caption: String, default: A)
val boolKey = Key[Boolean]("bool", false)
val intKey = Key[Int]("int", 0)
val strKey = Key[String]("str", "")
type KTuple[K[_], V[_]] = {
type T;
type Pair = (K[T], V[T]);
}
implicit def KTuple[K[_], V[_], A](value: (K[A], V[A])): KTuple[K, V]#Pair = value.asInstanceOf[KTuple[K, V]#Pair]
implicit def KTuple_P1[K[_], A](value: (K[A], A)): KTuple[K, Id]#Pair = value.asInstanceOf[KTuple[K, Id]#Pair]
class MapK[K[_], V[_]](rawMap: Map[K[Any], V[Any]]) {
def foreachK(f: [A] => (K[A], V[A]) => Unit): Unit = {
rawMap.foreach(f.asInstanceOf[((K[Any], V[Any])) => Unit])
}
def foreach(f: KTuple[K, V]#Pair => Unit): Unit = {
rawMap.foreach { pair =>
f(pair.asInstanceOf[KTuple[K, V]#Pair])
}
}
}
object MapK {
def create[K[_], V[_]](pairs: KTuple[K, V]#Pair*): MapK[K, V] = {
val x: List[KTuple[K, V]#Pair] = pairs.toList
val y: List[(K[Any], V[Any])] = x.map(t => t.asInstanceOf[(K[Any], V[Any])])
new MapK(Map(y: _*))
}
}
val idMap = MapK.create[Key, Id](
boolKey -> false,
intKey -> 1,
strKey -> "a",
)
val optionMap = MapK.create[Key, Option](
intKey -> Some(1),
strKey -> Some("a")
)
type T3[A] = (Key[A], A, A)
var log = List[KTuple[Key, Option]#Pair]()
idMap.foreach { (k, v) =>
log = log.appended(KTuple(k, Some(v)))
}
def doSomething[A, V[_]](k: Key[A], v: V[A]): Unit = println(s"$k -> v")
optionMap.foreachK {
[A] => (k: Key[A], v: Option[A]) => {
doSomething(k, v.get)
doSomething(k, v)
log = log :+ KTuple((k, v))
}
}
I wrote up a blog post with more details, will publish soon after some editing. Still looking for better approaches and improvements though.
The following code is producing run time error as below. Could reason why the following error. Please explain.
Exception in thread "main" scala.MatchError: Some(Some(List(17))) (of class scala.Some)
at com.discrete.CountingSupp$.$anonfun$tuplesWithRestrictions1$1(CountingSupp.scala:43)
def tuplesWithRestrictions1(): (Int, Map[Int, Option[List[Int]]]) = {
val df = new DecimalFormat("#")
df.setMaximumFractionDigits(0)
val result = ((0 until 1000) foldLeft[(Int, Map[Int, Option[List[Int]]])] ((0, Map.empty[Int, Option[List[Int]]]))) {
(r: (Int, Map[Int, Option[List[Int]]]), x: Int) => {
val str = df.format(x).toCharArray
if (str.contains('7')) {
import scala.math._
val v = floor(log10(x)) - 1
val v1 = (pow(10, v)).toInt
val m: Map[Int, Option[List[Int]]] = (r._2).get(v1) match {
case None => r._2 + (v1 -> Some(List(x)))
case Some(xs: List[Int]) => r._2 updated(x, Some(x::xs))
}
val f = (r._1 + 1, m)
f
} else r
}
}
result
}
Return type of .get on map is
get(k: K): Option[V]
Scala doc
/** Optionally returns the value associated with a key.
*
* #param key the key value
* #return an option value containing the value associated with `key` in this map,
* or `None` if none exists.
*/
def get(key: K): Option[V]
Now,
r._2.get(v1) returns an option of Value. So the final return type would be Option[Option[List[Int]]].
You are trying to pattern match for Option[T] but the real value type is Option[Option[Int]] which is not captured in the match.
Use r._2(v1) to extract the value and match. Throws exception when v1 is not found in map.
Match inside map providing default value.
r._2.get(k1).map {
case None => r._2 + (v1 -> Some(List(x)))
case Some(value) => r._2 updated(x, Some(x::xs))
}.getOrElse(defaultValue)
def tuplesWithRestrictions1(): (Int, Map[Int, List[Int]]) = {
val df = new DecimalFormat("#")
df.setMaximumFractionDigits(0)
val result = ((0 until 1000) foldLeft[(Int, Map[Int, List[Int]])] ((0, Map.empty[Int, List[Int]]))) {
(r: (Int, Map[Int, List[Int]]), x: Int) => {
val str = df.format(x).toCharArray
if (str.contains('7')) {
import scala.math._
val v = floor(log10(x))
val v1 = (pow(10, v)).toInt
val m: Map[Int, List[Int]] = (r._2).get(v1) match {
case Some(xs: List[Int]) => r._2 updated(v1, x :: xs)
case None => r._2 + (v1 -> List(x))
}
val f = (r._1 + 1, m)
f
} else r
}
}
result
}
I have the following issue
Given this list in input , I want to concateneate integers for each line having the same title,
val listIn= List("TitleB,Int,11,0",
"TitleB,Int,1,0",
"TitleB,Int,1,0",
"TitleB,Int,3,0",
"TitleA,STR,3,0",
"TitleC,STR,4,5")
I wrote the following function
def sumB(list: List[String]): List[String] = {
val itemPattern = raw"(.*)(\d+),(\d+)\s*".r
list.foldLeft(ListMap.empty[String, (Int,Int)].withDefaultValue((0,0))) {
case (line, stri) =>
val itemPattern(k,i,j) = stri
val (a, b) = line(k)
line.updated(k, (i.toInt + a, j.toInt + b))
}.toList.map { case (k, (i, j)) => s"$k$i,$j" }
}
Expected output would be:
List("TitleB,Int,16,0",
"TitleA,STR,3,0",
"TitleC,STR,4,5")
Since you are looking to preserve the order of the titles as they appear in the input data, I would suggest you to use LinkedHashMap with foldLeft as below
val finalResult = listIn.foldLeft(new mutable.LinkedHashMap[String, (String, String, Int, Int)]){ (x, y) => {
val splitted = y.split(",")
if(x.keySet.contains(Try(splitted(0)).getOrElse(""))){
val oldTuple = x(Try(splitted(0)).getOrElse(""))
x.update(Try(splitted(0)).getOrElse(""), (Try(splitted(0)).getOrElse(""), Try(splitted(1)).getOrElse(""), oldTuple._3+Try(splitted(2).toInt).getOrElse(0), oldTuple._4+Try(splitted(3).toInt).getOrElse(0)))
x
}
else {
x.put(Try(splitted(0)).getOrElse(""), (Try(splitted(0)).getOrElse(""), Try(splitted(1)).getOrElse(""), Try(splitted(2).toInt).getOrElse(0), Try(splitted(3).toInt).getOrElse(0)))
x
}
}}.mapValues(iter => iter._1+","+iter._2+","+iter._3+","+iter._4).values.toList
finalResult should be your desired output
List("TitleB,Int,16,0", "TitleA,STR,3,0", "TitleC,STR,4,5")
I have this code where I'm trying to call a partial function. When I build my project I get an error stating missing parameter type ++ headerExtractor.applyOrElse(event, _ => Map.empty).
I've looked at other posts, but I feel like this should be working. What am I doing wrong?
I'm calling headerExtractor here
private def publishToProfileExchange(customer: Schema.Customer, event: Schema.CustomerEvent): Future[Done] = {
val messageType: String = "ProfileEvent"
val headers: Map[String, AnyRef] = Map(
"profileId" -> customer.id.toString,
"eventType" -> event.eventType,
"type" -> messageType
) ++ headerExtractor.applyOrElse(event, _ => Map.empty)
...
//valid return values would be here
}
private val headerExtractor: PartialFunction[Schema.CustomerEvent, Map[String, AnyRef]] = {
case x if x.eventType.equalsIgnoreCase("message") && (x.eventData \ "incoming").asOpt[Boolean].getOrElse(false) =>
Map("incomingMessage" -> "true")
}
you have to provide the type of paramater on your .applyOrElse(). See example below
case class CustomerEvent(eventType: String, eventData: String)
val headerExtractor = new PartialFunction[CustomerEvent, Map[String, AnyRef]] {
def apply(x: CustomerEvent): Map[String, AnyRef] = Map("incomingMessage" -> "true")
def isDefinedAt(x: CustomerEvent): Boolean = x.eventType.equalsIgnoreCase("message")
}
assert(headerExtractor.applyOrElse(CustomerEvent("message", "{}"), (customerEvent: CustomerEvent) => Map.empty)
== Map("incomingMessage" -> "true"))
// or simply do (_: CustomerEvent)
assert(headerExtractor.applyOrElse(CustomerEvent("message", "{}"), (_: CustomerEvent) => Map.empty)
== Map("incomingMessage" -> "true"))
negative case
assert(headerExtractor.applyOrElse(CustomerEvent("something-else", "{}"), (_: CustomerEvent) => Map.empty)
== Map())