I have a HList of Symbols. I want to convert this into HMap with the HList values as the keys and the value mapping set to a generated object based on the key. The generated object holds state hence though default mapping can be known subsequent state is not.
Well... the thing is that Shapeless HMap is not as straight forward as HList, but the following can get you started
import shapeless._
import poly._
val hList = 'omg1 :: 'omg2 :: 'omg3 :: HNil
// lets assume that you want to map Symbols to kv-pairs String -> Int
// hence your HMap will have kv-pairs "omg1" -> 1, "omg2" -> 2...
class BiMapIS[K, V]
implicit val stringToInt = new BiMapIS[String, Int]
object myFoldPolyFunc extends Poly {
implicit def caseSymbol = use(
(hmap: HMap[BiMapIS], elem: Symbol) =>
hmap + (elem.name -> elem.name.last.toString.toInt)
)
}
val hMap = hList.foldLeft(HMap.empty[BiMapIS])(myFoldPolyFunc)
val v1 = hMap.get("omg1")
// Some(1)
val v2 = hMap.get("omg2")
// Some(2)
Without further information and based on a very superficial knowledge about HList (my only perception is a List of Tuple2), I guess you could do that via iteration. A possible pseudo code would be:
// Presume tuple._1 is the key
val hmap = hlist.map{ tuple => (tuple._1, doSomething(tuple._1))}.toMap
Related
I am a newbie to Scala and I am trying to understand collectives. I have a sample Scala code in which a method is defined as follows:
override def write(records: Iterator[Product2[K, V]]): Unit = {...}
From what I understand, this function is passed an argument record which is an Iterator of type Product2[K,V]. Now what I don't understand is this Product2 a user defined class or is it a built in data structure. Moreover how do explore the key-value pair contents of Product2 and how do I iterate over them.
Chances are Product2 is a built-in class and you can easily check it if you're in modern IDE (just hover over it with ctrl pressed), or, by inspecting file header -- if there is no related imports, like some.custom.package.Product2, it's built-in.
What is Product2 and where it's defined? You can easily found out such things by utilizing Scala's ScalaDoc:
In case of build-in class you can treat it like tuple of 2 elements (in fact Tuple2 extends Product2, as you may see below), which has ._1 and ._2 accessor methods.
scala> val x: Product2[String, Int] = ("foo", 1)
// x: Product2[String,Int] = (foo,1)
scala> x._1
// res0: String = foo
scala> x._2
// res1: Int = 1
See How should I think about Scala's Product classes? for more.
Iteration is also hassle free, for example here is the map operation:
scala> val xs: Iterator[Product2[String, Int]] = List("foo" -> 1, "bar" -> 2, "baz" -> 3).iterator
xs: Iterator[Product2[String,Int]] = non-empty iterator
scala> val keys = xs.map(kv => kv._1)
keys: Iterator[String] = non-empty iterator
scala> val keys = xs.map(kv => kv._1).toList
keys: List[String] = List(foo, bar, baz)
scala> xs
res2: Iterator[Product2[String,Int]] = empty iterator
Keep in mind though, that once iterator was consumed, it transitions to empty state and can't be re-used again.
Product2 is just two values of type K and V.
use it like this:
write(List((1, "one"), (2, "two")))
the prototype can also be written like: override def write(records: Iterator[(K, V)]): Unit = {...}
To access values k of type K and v of type V.
override def write(records: Iterator[(K, V)]): Unit = {
records.map{case (k, v) => w(k, v)}
}
I need to produce an extensible record given an HList of keys and a map of values, here's a MWE of what I'm trying to achieve (you can copy/paste this in any REPL with shapeless 2.0 available, in order to reproduce the issue)
import shapeless._; import syntax.singleton._; import record._
case class Foo[T](column: Symbol)
val cols = Foo[String]('column1) :: HNil
val values = Map("column1" -> "value1")
object toRecord extends Poly1 {
implicit def Foo[T] = at[Foo[T]] { foo =>
val k = foo.column.name
val v = values.get(k)
(k ->> v)
}
}
val r = cols.map(toRecord)
// r: shapeless.::[Option[String] with shapeless.record.KeyTag[k.type,Option[String]] forSome { val k: String },shapeless.HNil] = Some(value1) :: HNil
val value = r("column1")
// error: No field String("column1") in record shapeless.::[Option[String] with shapeless.record.KeyTag[k.type,Option[String]] forSome { val k: String },shapeless.HNil]
val value = r("column1")
If I try defining the record manually everything works as expected
val q = ("column1" ->> Some("value1")) :: HNil
// q: shapeless.::[Some[String] with shapeless.record.KeyTag[String("column1"),Some[String]],shapeless.HNil] = Some(value1) :: HNil
q("column1")
// Some[String] = Some(value1)
Clearly the difference is that in one case the KeyTag has type
KeyTag[String("column1"), Some[String]]
and in the (non-working) other
KeyTag[k.type,Option[String]] forSome { val k: String }
I sense the issue is with the string k not being statically known, but I have no clue on how to fix this.
Generally speaking, is there a way of dynamically generating an extensible record from a list of keys?
I fear the answer is to use a macro, but I'd be glad if another solution existed.
This isn't too bad if you can change your Foo definition a bit to allow it to keep track of the singleton type of the column key (note that I've removed the unused T type parameter):
import shapeless._; import syntax.singleton._; import record._
case class Foo[K <: Symbol](column: Witness.Aux[K])
val cols = Foo('column1) :: HNil
val values = Map("column1" -> "value1")
object toRecord extends Poly1 {
implicit def atFoo[K <: Symbol] = at[Foo[K]] { foo =>
field[K](values.get(foo.column.value.name))
}
}
val r = cols.map(toRecord)
And then:
scala> val value = r('column1)
value: Option[String] = Some(value1)
Note that I've changed your string key ("column1") to a symbol, since that's what we've put into the record.
I need a Map where I put different types of values (Double, String, Int,...) in it, key can be String.
Is there a way to do this, so that I get the correct type with map.apply(k) like
val map: Map[String, SomeType] = Map()
val d: Double = map.apply("double")
val str: String = map.apply("string")
I already tried it with a generic type
class Container[T](element: T) {
def get: T = element
}
val d: Container[Double] = new Container(4.0)
val str: Container[String] = new Container("string")
val m: Map[String, Container] = Map("double" -> d, "string" -> str)
but it's not possible since Container takes an parameter. Is there any solution to this?
This is not straightforward.
The type of the value depends on the key. So the key has to carry the information about what type its value is. This is a common pattern. It is used for example in SBT (see for example SettingsKey[T]) and Shapeless Records (Example). However, in SBT the keys are a huge, complex class hierarchy of its own, and the HList in shapeless is pretty complex and also does more than you want.
So here is a small example of how you could implement this. The key knows the type, and the only way to create a Record or to get a value out of a Record is the key. We use a Map[Key, Any] internally as storage, but the casts are hidden and guaranteed to succeed. There is an operator to create records from keys, and an operator to merge records. I chose the operators so you can concatenate Records without having to use brackets.
sealed trait Record {
def apply[T](key:Key[T]) : T
def get[T](key:Key[T]) : Option[T]
def ++ (that:Record) : Record
}
private class RecordImpl(private val inner:Map[Key[_], Any]) extends Record {
def apply[T](key:Key[T]) : T = inner.apply(key).asInstanceOf[T]
def get[T](key:Key[T]) : Option[T] = inner.get(key).asInstanceOf[Option[T]]
def ++ (that:Record) = that match {
case that:RecordImpl => new RecordImpl(this.inner ++ that.inner)
}
}
final class Key[T] {
def ~>(value:T) : Record = new RecordImpl(Map(this -> value))
}
object Key {
def apply[T] = new Key[T]
}
Here is how you would use this. First define some keys:
val a = Key[Int]
val b = Key[String]
val c = Key[Float]
Then use them to create a record
val record = a ~> 1 ++ b ~> "abc" ++ c ~> 1.0f
When accessing the record using the keys, you will get a value of the right type back
scala> record(a)
res0: Int = 1
scala> record(b)
res1: String = abc
scala> record(c)
res2: Float = 1.0
I find this sort of data structure very useful. Sometimes you need more flexibility than a case class provides, but you don't want to resort to something completely type-unsafe like a Map[String,Any]. This is a good middle ground.
Edit: another option would be to have a map that uses a (name, type) pair as the real key internally. You have to provide both the name and the type when getting a value. If you choose the wrong type there is no entry. However this has a big potential for errors, like when you put in a byte and try to get out an int. So I think this is not a good idea.
import reflect.runtime.universe.TypeTag
class TypedMap[K](val inner:Map[(K, TypeTag[_]), Any]) extends AnyVal {
def updated[V](key:K, value:V)(implicit tag:TypeTag[V]) = new TypedMap[K](inner + ((key, tag) -> value))
def apply[V](key:K)(implicit tag:TypeTag[V]) = inner.apply((key, tag)).asInstanceOf[V]
def get[V](key:K)(implicit tag:TypeTag[V]) = inner.get((key, tag)).asInstanceOf[Option[V]]
}
object TypedMap {
def empty[K] = new TypedMap[K](Map.empty)
}
Usage:
scala> val x = TypedMap.empty[String].updated("a", 1).updated("b", "a string")
x: TypedMap[String] = TypedMap#30e1a76d
scala> x.apply[Int]("a")
res0: Int = 1
scala> x.apply[String]("b")
res1: String = a string
// this is what happens when you try to get something out with the wrong type.
scala> x.apply[Int]("b")
java.util.NoSuchElementException: key not found: (b,Int)
This is now very straightforward in shapeless,
scala> import shapeless._ ; import syntax.singleton._ ; import record._
import shapeless._
import syntax.singleton._
import record._
scala> val map = ("double" ->> 4.0) :: ("string" ->> "foo") :: HNil
map: ... <complex type elided> ... = 4.0 :: foo :: HNil
scala> map("double")
res0: Double with shapeless.record.KeyTag[String("double")] = 4.0
scala> map("string")
res1: String with shapeless.record.KeyTag[String("string")] = foo
scala> map("double")+1.0
res2: Double = 5.0
scala> val map2 = map.updateWith("double")(_+1.0)
map2: ... <complex type elided> ... = 5.0 :: foo :: HNil
scala> map2("double")
res3: Double = 5.0
This is with shapeless 2.0.0-SNAPSHOT as of the date of this answer.
I finally found my own solution, which worked best in my case:
case class Container[+T](element: T) {
def get[T]: T = {
element.asInstanceOf[T]
}
}
val map: Map[String, Container[Any]] = Map("a" -> Container[Double](4.0), "b" -> Container[String]("test"))
val double: Double = map.apply("a").get[Double]
val string: String = map.apply("b").get[String]
(a) Scala containers don't track type information for what's placed inside them, and
(b) the return "type" for an apply/get method with a simple String parameter/key is going to be static for a given instance of the object the method is to be applied to.
This feels very much like a design decision that needs to be rethought.
I don't think there's a way to get bare map.apply() to do what you'd want. As the other answers suggest, some sort of container class will be necessary. Here's an example that restricts the values to be only certain types (String, Double, Int, in this case):
sealed trait MapVal
case class StringMapVal(value: String) extends MapVal
case class DoubleMapVal(value: Double) extends MapVal
case class IntMapVal(value: Int) extends MapVal
val myMap: Map[String, MapVal] =
Map("key1" -> StringMapVal("value1"),
"key2" -> DoubleMapVal(3.14),
"key3" -> IntMapVal(42))
myMap.keys.foreach { k =>
val message =
myMap(k) match { // map.apply() in your example code
case StringMapVal(x) => "string: %s".format(x)
case DoubleMapVal(x) => "double: %.2f".format(x)
case IntMapVal(x) => "int: %d".format(x)
}
println(message)
}
The main benefit of the sealted trait is compile-time checking for non-exhaustive matches in pattern matching.
I also like this approach because it's relatively simple by Scala standards. You can go off into the weeds for something more robust, but in my opinion you're into diminishing returns pretty quickly.
If you want to do this you'd have to specify the type of Container to be Any, because Any is a supertype of both Double and String.
val d: Container[Any] = new Container(4.0)
val str: Container[Any] = new Container("string")
val m: Map[String, Container[Any]] = Map("double" -> d, "string" -> str)
Or to make things easier, you can change the definition of Container so that it's no longer type invariant:
class Container[+T](element: T) {
def get: T = element
override def toString = s"Container($element)"
}
val d: Container[Double] = new Container(4.0)
val str: Container[String] = new Container("string")
val m: Map[String, Container[Any]] = Map("double" -> d, "string" -> str)
There is a way but it's complicated. See Unboxed union types in Scala. Essentially you'll have to type the Map to some type Int |v| Double to be able to hold both Int and Double. You'll also pay a high price in compile times.
I'd like to link 2 columns of unique identifiers and be able to get a first column value by a second column value as well as a second column value by a first column value. Something like
Map(1 <-> "one", 2 <-> "two", 3 <-> "three")
Is there such a facility in Scala?
Actually I need even more: 3 columns to select any in a triplet by another in a triplet (individual values will never be met more than once in the entire map). But a 2-column bidirectional map can help too.
Guava has a bimap that you can use along with
import scala.collection.JavaConversions._
My BiMap approach:
object BiMap {
private[BiMap] trait MethodDistinctor
implicit object MethodDistinctor extends MethodDistinctor
}
case class BiMap[X, Y](map: Map[X, Y]) {
def this(tuples: (X,Y)*) = this(tuples.toMap)
private val reverseMap = map map (_.swap)
require(map.size == reverseMap.size, "no 1 to 1 relation")
def apply(x: X): Y = map(x)
def apply(y: Y)(implicit d: BiMap.MethodDistinctor): X = reverseMap(y)
val domain = map.keys
val codomain = reverseMap.keys
}
val biMap = new BiMap(1 -> "A", 2 -> "B")
println(biMap(1)) // A
println(biMap("B")) // 2
Of course one can add syntax for <-> instead of ->.
Here's a quick Scala wrapper for Guava's BiMap.
import com.google.common.{collect => guava}
import scala.collection.JavaConversions._
import scala.collection.mutable
import scala.languageFeature.implicitConversions
class MutableBiMap[A, B] private (
private val g: guava.BiMap[A, B] = new guava.HashBiMap[A, B]()) {
def inverse: MutableBiMap[B, A] = new MutableBiMap[B, A](g.inverse)
}
object MutableBiMap {
def empty[A, B]: MutableBiMap[A, B] = new MutableBiMap()
implicit def toMap[A, B] (x: MutableBiMap[A, B]): mutable.Map[A,B] = x.g
}
I have a really simple BiMap in Scala:
case class BiMap[A, B](elems: (A, B)*) {
def groupBy[X, Y](pairs: Seq[(X, Y)]) = pairs groupBy {_._1} mapValues {_ map {_._2} toSet}
val (left, right) = (groupBy(elems), groupBy(elems map {_.swap}))
def apply(key: A) = left(key)
def apply[C: ClassTag](key: B) = right(key)
}
Usage:
val biMap = BiMap(1 -> "x", 2 -> "y", 3 -> "x", 1 -> "y")
assert(biMap(1) == Set("x", "y"))
assert(biMap("x") == Set(1, 3))
I don't think it exists out of the box, because the generic behavior is not easy to extract
How to handle values matching several keys in a clean api?
However for specific cases here is a good exercise that might help. It must be updated because no hash is used and getting a key or value is O(n).
But the idea is to let you write something similar to what you propose, but using Seq instead of Map...
With the help of implicit and trait, plus find, you could emulate what you need with a kind of clean api (fromKey, fromValue).
The specificities is that a value is not supposed to appear in several places... In this implementation at least.
trait BiMapEntry[K, V] {
def key:K
def value:V
}
trait Sem[K] {
def k:K
def <->[V](v:V):BiMapEntry[K, V] = new BiMapEntry[K, V]() { val key = k; val value = v}
}
trait BiMap[K, V] {
def fromKey(k:K):Option[V]
def fromValue(v:V):Option[K]
}
object BiMap {
implicit def fromInt(i:Int):Sem[Int] = new Sem[Int] {
def k = i
}
implicit def fromSeq[K, V](s:Seq[BiMapEntry[K, V]]) = new BiMap[K, V] {
def fromKey(k:K):Option[V] = s.find(_.key == k).map(_.value)
def fromValue(v:V):Option[K] = s.find(_.value == v).map(_.key)
}
}
object test extends App {
import BiMap._
val a = 1 <-> "a"
val s = Seq(1 <-> "a", 2 <-> "b")
println(s.fromKey(2))
println(s.fromValue("a"))
}
Scala is immutable and values are assigned as reference not copy, so memory footprint will for reference/pointer storage only, which it's better to use to two maps, with type A being key for first and type being B being key for second mapped to B and A respectively, than tun time swapping of maps. And the swapping implementation also has it's own memory footprint and the newly swapped hash-map will also be there in memory till the execution of parent call back and the garbage collector call. And if the the swapping of map is required frequently than virtually your are using equally or more memory than the naive two maps implementation at starting.
One more approach you can try with single map is this(will work only for getting key using mapped value):
def getKeyByValue[A,B](map: Map[A,B], value: B):Option[A] = hashMap.find((a:A,b:B) => b == value)
Code for Scala implementation of find by key:
/** Find entry with given key in table, null if not found.
*/
#deprecatedOverriding("No sensible way to override findEntry as private findEntry0 is used in multiple places internally.", "2.11.0")
protected def findEntry(key: A): Entry =
findEntry0(key, index(elemHashCode(key)))
private[this] def findEntry0(key: A, h: Int): Entry = {
var e = table(h).asInstanceOf[Entry]
while (e != null && !elemEquals(e.key, key)) e = e.next
e
}
Is there a way in Scala to convert a List[Int] to java.util.List[java.lang.Integer]?
I'm interfacing with Java (Thrift).
JavaConversions supports List --> java.util.List, and implicits exist between Int --> java.lang.Integer, but from what I can tell I would still need an extra pass to manually do the conversion:
val y = List(1)
val z: java.util.List[Integer] = asList(y) map { (x: Int) => x : java.lang.Integer }
Apparently you need both conversions. However, you can group them in a single implicit conversion:
implicit def toIntegerList( lst: List[Int] ) =
seqAsJavaList( lst.map( i => i:java.lang.Integer ) )
Example:
scala> def sizeOf( lst: java.util.List[java.lang.Integer] ) = lst.size
scala> sizeOf( List(1,2,3) )
res5: Int = 3
Because the underlying representation of Int is Integer you can cast directly to java.util.List[java.lang.Integer]. It will save you an O(n) operation and some implicit stuff.
import collection.JavaConversions._
class A {
def l() = asList(List(1,2)).asInstanceOf[java.util.List[java.lang.Integer]]
}
Then you can use from Java like this:
A a = new A();
java.util.List<Integer> l = a.l();
Note that on 2.9.0 ,I get a deprecation warning on asList (use seqAsJavaList instead)
Did you try:
val javalist = collection.JavaConversions.asJavaList (y)
I'm not sure, whether you need a conversion Int=>Integer or Int=>int here. Can you try it out?
Update:
The times, they are a changing. Today you'll get a deprecated warning for that code. Use instead:
import scala.collection.JavaConverters._
val y = List (1)
> y: List[Int] = List(1)
val javalist = (y).asJava
> javalist: java.util.List[Int] = [1]
This doesn't have an implicit at the outmost layer, but I like this generic approach and have implemented it for a couple of types of collections (List, Map).
import java.util.{List => JList}
import scala.collection.JavaConverters._
def scalaList2JavaList[A, B](scalaList: List[A])
(implicit a2bConversion: A => B): JList[B] =
(scalaList map a2bConversion).asJava
Since an implicit conversion from Int to Integer is part of standard lib, usage in this case would just look like this:
scalaList2JavaList[Int, Integer](someScalaList)
In the other direction!
(since I have these available anyway as they were my original implementations...)
import java.util.{List => JList}
import scala.collection.JavaConversions._
def javaList2ScalaList[A, B](javaList: JList[A])
(implicit a2bConversion: A => B): List[B] =
javaList.toList map a2bConversion
Usage:
javaList2ScalaList[Integer, Int](someJavaList)
This can then be re-used for all lists so long as an implicit conversion of the contained type is in scope.
(And in case you're curious, here is my implementation for map...)
def javaMap2ScalaMap[A, B, C, D](javaMap: util.Map[A, B])(implicit a2cConversion: A => C, b2dConversion: B => D): Map[C, D] =
javaMap.toMap map { case (a, b) => (a2cConversion(a), b2dConversion(b)) }
Starting Scala 2.13, the standard library includes scala.jdk.CollectionConverters which provides Scala to Java implicit collection conversions.
Which we can combine with java.lang.Integer::valueOf to convert Scala's Int to Java's Integer:
import scala.jdk.CollectionConverters._
List(1, 2, 3).map(Integer.valueOf).asJava
// java.util.List[Integer] = [1, 2, 3]
I was trying to pass a Map[String, Double] to a Java method. But the problem was JavaConversions converts the Map to a java Map, but leaves the scala Double as is, instead of converting it to java.lang.Double. After a few hours of seaching I found [Alvaro Carrasco's answer])https://stackoverflow.com/a/40683561/1612432), it is as simple as doing:
val scalaMap = // Some Map[String, Double]
val javaMap = scalaMap.mapValues(Double.box)
After this, javaMap is a Map[String, java.lang.Double]. Then you can pass this to a java function that expects a Map<String, Double> and thanks to implicit conversions the Scala Map will be converted to java.util.Map
In your case would be the same, but with Int.box:
val y = List(1)
val javay = y.map(Int.box)