I'm creating shapeless HMap using the code like below from scala-exercises.
import shapeless.HMap
class BiMapIS[K, V]
implicit val intToString = new BiMapIS[Int, String]
implicit val stringToInt = new BiMapIS[String, Int]
val hm = HMap[BiMapIS](23 -> "foo", "bar" -> 13)
I would like to create HMap from variable arguments as below (I'm having long list of arguments so just checking whether I can simplify the code littlebit) -
import shapeless.{HMap, HNil}
import java.util.{List => JList}
val entities: JList[(_, _)] = ???
class BiMapIS[K, V]
implicit val intToString = new BiMapIS[Int, String]
implicit val stringToInt = new BiMapIS[String, Int]
import collection.JavaConverters._
val entitiesSeq = entities.asScala.toList
val hm = HMap[BiMapIS](entitiesSeq:_*)
Is there any way I can create HMap from variable args?
I'm using shapless 2.33 with scala 2.12 https://mvnrepository.com/artifact/com.chuusai/shapeless_2.12/2.3.3
Try
val entitiesSeq = entities.asScala.toMap[Any, Any]
val hm = new HMap[BiMapIS](entitiesSeq)
Related
I thought we can rely on implicit conversion which converts scala.Double to java.lang.Double. So I tried the following:
import scala.collection.JavaConverters._
object Main extends App {
def main(args: Array[String]) = {
val m = Map("10" -> 20.0)
doSome(m.asJava) //error. Type mismatch found: java.util.Map[String,scala.Double]
// required: java.util.Map[String,java.lang.Double]
doSome2(m.asJava)
}
def doSome(m: java.util.Map[java.lang.String, java.lang.Double]) = println(m)
def doSome2(m: java.util.Map[java.lang.String, Double]) = println(m)
}
Why doesn't it work? What would be the idiomatic way to perform such a conversion?
You need the boxed version of double:
import scala.collection.JavaConverters._
m.mapValues(Double.box).asJava
The implicits are able to convert a value of Double to java.lang.Double, but not a Map[String,Double] to java.util.Map[String,java.lang.Double].
String requires no conversion because String is a java.lang.String while Double is a double (primitive).
It seems that for String, you don't need to do any conversion, but is not the case for Double.
You can use the method double2Double which is defined in Predef to convert to java.double.
import scala.collection.JavaConverters._
m.map { case (k, v) => k -> double2Double(v) }.asJava
or another way is to do asInstanceOf to convert it to Java map directly.
The issue here is that scala.Double is a primitive (equivalent to Java double) while java.lang.Double is a class. They are not at all the same thing. The JavaConverters converts between Java Map and Scala Map, not their contents. This works:
import scala.collection.JavaConverters._
object Main {
def main(args: Array[String]): Unit = {
val m = Map("10" -> 20.0)
doSome(mapConv(m))
doSome2(m.asJava)
}
def doSome(m: java.util.Map[java.lang.String, java.lang.Double]) = println(m)
def doSome2(m: java.util.Map[java.lang.String, Double]) = println(m)
def mapConv(msd: Map[String, Double]): java.util.Map[java.lang.String, java.lang.Double] = {
msd.map { case (k, v) => (k -> new java.lang.Double(v)) }.asJava
}
}
I am new to scala and just understanding how can I transform via Map call. The foo function is not getting called. What is that I am missing ?
import org.apache.spark.rdd.RDD
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
object Expt {
def main(args: Array[String]): Unit = {
var conf = new SparkConf().setAppName("Test").setMaster("local[*]")
val sc = new SparkContext(conf)
val a1 = sc.parallelize(List((1,"one"),(2,"two"),(3,"three"))
val a3 = a1.map(x => Expt.foo(x))
}
def foo(x: (Int,String)) : (Int,String) = {
println(x)
x
}
}
You don't execute any action so map is never evaluated. Also println in map usually won't have visible effect.
trigger computation using collect or foreach (do some action)
scala> val a1 = sc.parallelize(List((1,"one"),(2,"two"),(3,"three")))
a1: org.apache.spark.rdd.RDD[(Int, String)] = ParallelCollectionRDD[6] at parallelize at <console>:24
scala> val a3 = a1.map(x => foo(x))
a3: org.apache.spark.rdd.RDD[(Int, String)] = MapPartitionsRDD[7] at map at <console>:28
scala> a3.collect
(1,one)
(2,two)
(3,three)
res3: Array[(Int, String)] = Array((1,one), (2,two), (3,three))
scala> a3.foreach(_ => ())
(1,one)
(3,three)
(2,two)
Given a (data structure that operates like a) Map. Is there a type level way of inserting to it? That is:
val myMap: Map[Int, String] = Map(1 -> "a", 2 -> "b")
val x: Int = 5
val y: Int = 99
val myMap2 = myMap + (x -> "e")
What I'm hoping for is that myMap2 will have some type wherein I can safely do something like myMap2.retrieve(x) and have it compile and return "e". But myMap2.retrieve(y) should not even compile.
It's possible with Shapeless if your keys have different types:
import shapeless._
object _1
object _2
object _3
object _4
//-------------Map definition starts here----------------
class StaticMap1[K, V]
trait StaticMap1Like {
type M[T] <: StaticMap1[T, String]
private def add[T]: M[T] = (new StaticMap1[T, String] {}).asInstanceOf[M[T]]
implicit val __1 = add[_1.type] //add key to StaticMap1
implicit val __2 = add[_2.type] //add key to StaticMap1
}
object StaticMap1 extends StaticMap1Like
val hm = HMap[StaticMap1](_1 -> "a", _2 -> "b") //add values
//-------------Map definition ends here-----------------
scala> hm.get(_1)
res0: Option[String] = Some(a)
scala> hm.get(_2)
res1: Option[String] = Some(b)
scala> hm.get(_3) //compile-time error
<console>:18: error: could not find implicit value for parameter ev: BiMapIS[shapeless.nat._3,V]
hm.get(_3)
^
And key insertion:
//----Adding element _3 -> "c" starts here--------------
class StaticMap2[K, V] extends StaticMap1[K, V]
trait StaticMap2Like extends StaticMap1Like {
type M[T] <: StaticMap2[T, String]
private def add[T] = new StaticMap2[T, String] {}.asInstanceOf[M[T]]
implicit val __3 = add[_3.type] //add key to StaticMap2
}
object StaticMap2 extends StaticMap2Like
val hm2 = hm.asInstanceOf[HMap[StaticMap2]] + (_3 -> "c")
//----Adding element ends here---------------------------
scala> hm2.get(_3)
res6: Option[String] = Some(c)
scala> hm2.get(_2)
res7: Option[String] = Some(b)
scala> hm2.get(_1)
res8: Option[String] = Some(a)
scala> hm2.get(_4)
<console>:21: error: could not find implicit value for parameter ev: StaticMap2[_4.type,V]
hm2.get(_4)
^
scala> hm.get(_3) //old `hm` still working
<console>:17: error: could not find implicit value for parameter ev: StaticMap1[_3.type,V]
hm.get(_3)
^
But:
don't use Shapeless native nat here - it won't work as it can't distinguish nat._1 from nat._2 (at least for my version of Shapeless)
adding element to а Map also wouldn't be so convinient, as user is gonna have to add an implicit for every new key
Each time you use myMap2.get(y) you will get an Option of String for any y if it is an Int.
In case you try to use any other type, you will have a compilation exception.
e.g.: myMap2.get("y") will throw a type mismatch.
So you can't compile if the key type is not the right one.
If I have a Shapeless HMap[MappingA] (with implicits properly defined for the type MappingA[K, V]), can I type-safely transform/map it to a HMap[MappingB].
class MappingA[K, V]
implicit val intToString = new MappingA[Int, String]
implicit val stringToInt = new MappingA[String, Int]
class MappingB[K, V]
implicit val longToString = new MappingA[Long, String]
implicit val stringToLong = new MappingA[String, Long]
val hm1 = HMap[MappingA](1 -> "one", "two" -> 2)
// How to...
val hm2: HMap[MappingB] = ??? // transform/map hm1
// expected for hm2 in this basic example
// HMap[MappingB](1L -> "one", "two" -> 2L)
I would like to be able to perform the following, but it fails in the call to useMap. How can I perform this conversion?
scala> import scala.collection.JavaConversions._
import scala.collection.JavaConversions._
scala> import scala.collection.JavaConverters._
import scala.collection.JavaConverters._
scala> def useMap(m: java.util.Map[java.lang.Integer, java.lang.Float]) = m
useMap: (m: java.util.Map[Integer,Float])java.util.Map[Integer,Float]
scala> val v: Map[Int, Float] = Map()
v: Map[Int,Float] = Map()
scala> useMap(v)
<console>:10: error: type mismatch;
found : scala.collection.immutable.Map[Int,scala.Float]
required: java.util.Map[Integer,java.lang.Float]
useMap(v)
^
It seems to work with Map[String, String], but not Map[Int, Float].
The solution linked to by #pagoda_5b works: Scala convert List[Int] to a java.util.List[java.lang.Integer]
scala> import collection.JavaConversions._
import collection.JavaConversions._
scala> val m = mapAsJavaMap(Map(1 -> 2.1f)).asInstanceOf[java.util.Map[java.lang.Integer, java.lang.Float]]
m: java.util.Map[Integer,Float] = {1=2.1}
scala> m.get(1).getClass
res2: Class[_ <: Float] = class java.lang.Float
Use scala predefined float2Float and use CollectionConverters to perform conversion explicitly.
import scala.jdk.CollectionConverters._
// Prior to Scala 2.13: import scala.collection.JavaConverters._
val scalaMap = Map(1 -> 1.0F)
val javaMap = scalaMap.map{ case (k, v) => k -> float2Float(v) }.asJava
Implicit conversions are sometimes hard to debug/understand and therefore I prefer explicit conversions as follows:
import scala.collection.JavaConversions.mapAsJavaMap
val scalaMap = Map(1 -> 1.0F)
val javaMap = mapAsJavaMap(scalaMap)
scala> v.asJava
<console>:16: error: type mismatch;
found : java.util.Map[Int,scala.Float]
required: java.util.Map[Integer,java.lang.Float]
The type of v is Map[scala.Int, scala.Float], not Map[java.lang.Integer, java.lang.Float].
You could try this:
import java.lang.{Float => JFloat}
useMap(v.map{ case (k, v) => (k: Integer) -> (v: JFloat)})