I'm new to scala and I'm trying to do something like this in a clean way.
I have a method that takes in several optional parameters. I want to create a map and only add items to the map if the optional parameter has a value. Here's a dummy example:
def makeSomething(value1: Option[String], value2: Option[String], value3: Option[String]): SomeClass = {
val someMap: Map[String, String] =
value1.map(i => Map("KEY_1_NAME" -> i.toString)).getOrElse(Map())
}
In this case above, we're kind of doing what I want but only if we only care about value1 - I would want this done for all of the optional values and have them put into the map. I know I can do something brute-force:
def makeSomething(value1: Option[String], value2: Option[String], value3: Option[String]): SomeClass = {
// create Map
// if value1 has a value, add to map
// if value2 has a value, add to map
// ... etc
}
but I was wondering if scala any features that would help me able to clean this up.
Thanks in advance!
You can create a Map[String, Option[String]] and then use collect to remove empty values and "extract" the present ones from their wrapping Option:
def makeSomething(value1: Option[String], value2: Option[String], value3: Option[String]): SomeClass = {
val someMap: Map[String, String] =
Map("KEY1" -> value1, "KEY2" -> value2, "KEY3" -> value3)
.collect { case (key, Some(value)) => key -> value }
// ...
}
.collect is one possibility. Alternatively, use the fact that Option is easily convertible to a Seq:
value1.map("KEY1" -> _) ++
value2.map("KEY2" -> _) ++
value3.map("KEY3" -> _) toMap
Related
Few days ago I started to learn Cats and I want to implement method appendOptional for Map[String, _: Show].
I started with the following idea:
def appendOptional[T: Show](to: Map[String, String], values: (String, Option[T])*): Map[String, String] =
values.foldLeft(values) {
case (collector, (key, Some(value))) =>
collector + (key -> implicitly[Show[T]].show(value))
case (collector, _) => collector
}
And to use it like:
def createProps(initial: Map[String, String], name: Option[String], age: Option[Int])
val initial = Map("one" -> "one", "two" -> "two")
val props = appendOptional(initial, "name" -> name, "age" -> age)
I understand that this approach is quite naive and straightforward because implicitly[Show[T]].show(value) will actually lookup for Show[Any].
Also, I had an idea to accept HList with context bound, but I have not found any example of that.
One more variant is to create a lot of overloaded methods (as it is done in a lot of libraris):
def appendOptional[T1: Show, T2: Show](to: Map[String, String], v1: (String, Option[T1], v2: (String, Option[T2])))
Question: Is there a way to define context bound for varargs functions?
Strictly speaking, the first is the correct way to define the bound; varargs don't mean the type of arguments varies, only their number.
The way to achieve what you want with varying types is more involved and requires packing Show instances together with values. E.g.
case class HasShow[A](x: A)(implicit val ev: Show[A])
def appendOptional(to: Map[String, String], values: (String, Option[HasShow[_]])*): Map[String, String] =
values.foldLeft(values) {
// value.ev.show(value.x)) can be extracted into a method on HasShow as well
case (collector, (key, Some(value: HasShow[a]))) =>
collector + (key -> value.ev.show(value.x))
case (collector, _) => collector
}
val props = appendOptional(initial, "name" -> name.map(HasShow(_)), "age" -> age.map(HasShow(_)))
You can sprinkle some more implicit conversions for HasShow to simplify the call-site, but this way you can see what's going on better.
For this specific case I think a better and simpler solution would be
implicit class MapOp(self: Map[String, String]) extends AnyVal {
def appendOptional[A: Show](key: String, value: Option[A]) =
value.fold(self)(x => self + (key -> Show.show(x)))
}
val props = initial.appendOptional("name", name).appendOptional("age", age)
I'm having an error
error: Cannot prove that (Int, String, String, String, String, Double, String) <:< (T, U).
}.collect.toMap
when executing my application having the following code snippet.
val trains = sparkEnvironment.sc.textFile(dataDirectoryPath + "/trains.csv").map { line =>
val fields = line.split(",")
// format: (trainID,trainName,departure,arrival,cost,trainClass)
(fields(0).toInt, fields(1),fields(2),fields(3),fields(4).toDouble,fields(5))
}.collect.toMap
What could be the cause and can anyone please suggest a solution ?
if you want to do toMap on Seq, you show have a Seq of Tuple2. ScalaDoc of toMap states :
This method is unavailable unless the elements are members of Tuple2,
each ((T, U)) becoming a key-value pair in the map
So you should do:
val trains = sparkEnvironment.sc.textFile(dataDirectoryPath + "/trains.csv").map { line =>
val fields = line.split(",")
// format: (trainID,trainName,departure,arrival,cost,trainClass)
(fields(0).toInt, // first element of Tuple2 -> "key"
(fields(1),fields(2),fields(3),fields(4).toDouble,fields(5)) // 2nd element of Tuple2 -> "value"
)
}.collect.toMap
such that your map-statwment returns RDD[(Int, (String, String, String, String, Double, String))]
I expect to return a map containing value of different datatypes such as
(key -> String) and (key -> Int), but i can have Map either of
Map[String,String] or Map[String,Int].
I can't use class because number and order of keys are not fixed.
Is there any way to wrap String and Int to a generic class so that i can return map as Map[String,Any]
You can use HMap as #Harnish suggested, but there is an alternative in the scala library: Map[String, Either[Int, String]]. It applies only if you know that the types either one or another and nothing more.
The type Either[Int, String] can be created either by Left(5) or Right("Hello"). Then you can use match to test the value:
x match {
case Left(n) => println(s"$n is a number")
case Right(s) => println(s"$s is a string")
}
Updated
Example:
val dict = scala.collection.mutable.Map[String, Either[String, Int]]()
dict += ("a" -> Right(5))
dict += ("b" -> Left("Hello"))
dict map {
case (key, Right(n)) => println(s"For $key: $n is integer")
case (key, Left(s)) => println(s"For $key: $s is string")
}
I'm not sure if you can do this with the standard collections library, however it is possible using shapeless HMap (Heterogenous map). This is the example given in the docs, which closely matches what you have described:
// Key/value relation to be enforced: Strings map to Ints and vice versa
class BiMapIS[K, V]
implicit val intToString = new BiMapIS[Int, String]
implicit val stringToInt = new BiMapIS[String, Int]
val hm = HMap[BiMapIS](23 -> "foo", "bar" -> 13)
//val hm2 = HMap[BiMapIS](23 -> "foo", 23 -> 13) // Does not compile
scala> hm.get(23)
res0: Option[String] = Some(foo)
scala> hm.get("bar")
res1: Option[Int] = Some(13)
Note, it doesn't give you an Any, instead you have to specify what is valid in your key/value pairs. I'm not sure if that's helpful to you or not...
I have this scala method that build a Map from some parameters:
def foo(name: Option[String], age: Option[Int], hasChilds: Option[Boolean],
childs: Option[List[Map[String, Any]]]): Map[String,Any] = {
var m = Map[String, Any]()
if (!name.isEmpty) m += ("name" -> name.get)
if (!age.isEmpty) m += ("age" -> age.get)
if (!hasChilds.isEmpty) m += ("hasChilds" -> hasChilds.get)
if (!childs.isEmpty) m += ("childs" -> childs.get)
m
}
I wonder if there is a way to refactor the code in more functional style?
Is it possible to eleminate the using of var in this case?
One approach includes the flattening of an immutable Map, like this,
def foo(name: Option[String],
age: Option[Int],
hasChilds: Option[Boolean],
childs: Option[List[Map[String, Any]]]): Map[String,Any] = {
Map( ("name" -> name),
("age" -> age),
("hasChilds" -> hasChilds),
("childs" -> childs)).collect { case(a,Some(b)) => (a,b) }
}
Another approach can be
def foo(....) =
Map("name" -> name, "age" -> age, "hasChilds" -> hasChilds, "childs" -> childs)
.filter(_._2 != None).mapValues(_.get)
As pointed out by #Dider, this can also be done, similar to #enzyme solution
Map("name" -> name, "age" -> age, "hasChilds" -> hasChilds, "childs" -> childs)
.collect {case (k, Some(v)) => (k,v) }
Scala generally favors typed data and immutability, and you're fighting against both of those here. I don't know what the context is for this map, but I think it would be more idiomatic to use a case clase with optional parameters. For example:
case class Person(name: String, age: Option[Int], children: Seq[Person]) {
def hasChildren: Boolean = !children.isEmpty
}
Now you might call this as follows with an optional name variable.
val name: Option[String] = Option("suud")
val age: Option[Int] = Option(25)
val children: Seq[Person] = Seq.empty
val suud: Option[Person] = name map {Person(_, age, children)}
The way that foo is written, it's possible to pass in an empty list of children with a boolean parameter which says the map has children. Writing hasChildren as a method of a case class guards against this, because the boolean method depends on the collection it's meant to provide information about.
If you really insist on using a map here, you can either use a MapBuilder to get an immutable map, or just import and use the mutable map.
import scala.collection.mutable.MapBuilder
val builder: MapBuilder[String, Any, Map[String, Any]] = new MapBuilder(Map.empty)
if (!name.isEmpty) builder += ("name" -> name.get)
if (!age.isEmpty) builder += ("age" -> age.get)
if (!hasChilds.isEmpty) builder += ("hasChilds" -> hasChilds.get)
if (!childs.isEmpty) builder += ("childs" -> childs.get)
builder.result
Now the result of the builder is an immutable map. If you really need a mutable map, you can just:
import scala.collection.mutable
val m = mutable.Map[String, Any]()
Now you've got a mutable map which can be converted to an immutable map with its toMap method.
Is it possible to map the key value pairs of a Map to a Scala constructor with named parameters?
That is, given
class Person(val firstname: String, val lastname: String) {
...
}
... how can I create an instance of Person using a map like
val args = Map("firstname" -> "John", "lastname" -> "Doe", "ignored" -> "value")
What I am trying to achieve in the end is a nice way of mapping Node4J Node objects to Scala value objects.
The key insight here is that the constructor arguments names are available, as they are the names of the fields created by the constructor. So provided that the constructor does nothing with its arguments but assign them to fields, then we can ignore it and work with the fields directly.
We can use:
def setFields[A](o : A, values: Map[String, Any]): A = {
for ((name, value) <- values) setField(o, name, value)
o
}
def setField(o: Any, fieldName: String, fieldValue: Any) {
// TODO - look up the class hierarchy for superclass fields
o.getClass.getDeclaredFields.find( _.getName == fieldName) match {
case Some(field) => {
field.setAccessible(true)
field.set(o, fieldValue)
}
case None =>
throw new IllegalArgumentException("No field named " + fieldName)
}
Which we can call on a blank person:
test("test setFields") {
val p = setFields(new Person(null, null, -1), Map("firstname" -> "Duncan", "lastname" -> "McGregor", "age" -> 44))
p.firstname should be ("Duncan")
p.lastname should be ("McGregor")
p.age should be (44)
}
Of course we can do better with a little pimping:
implicit def any2WithFields[A](o: A) = new AnyRef {
def withFields(values: Map[String, Any]): A = setFields(o, values)
def withFields(values: Pair[String, Any]*): A = withFields(Map(values :_*))
}
so that you can call:
new Person(null, null, -1).withFields("firstname" -> "Duncan", "lastname" -> "McGregor", "age" -> 44)
If having to call the constructor is annoying, Objenesis lets you ignore the lack of a no-arg constructor:
val objensis = new ObjenesisStd
def create[A](implicit m: scala.reflect.Manifest[A]): A =
objensis.newInstance(m.erasure).asInstanceOf[A]
Now we can combine the two to write
create[Person].withFields("firstname" -> "Duncan", "lastname" -> "McGregor", "age" -> 44)
You mentioned in the comments that you're looking for a reflection based solution. Have a look at JSON libraries with extractors, which do something similar. For example, lift-json has some examples,
case class Child(name: String, age: Int, birthdate: Option[java.util.Date])
val json = parse("""{ "name": null, "age": 5, "birthdate": null }""")
json.extract[Child] == Child(null, 5, None)
To get what you want, you could convert your Map[String, String] into JSON format and then run the case class extractor. Or you could look into how the JSON libraries are implemented using reflection.
I guess you have domain classes of different arity, so here it is my advice. (all the following is ready for REPL)
Define an extractor class per TupleN, e.g. for Tuple2 (your example):
class E2(val t: Tuple2[String, String]) {
def unapply(m: Map[String,String]): Option[Tuple2[String, String]] =
for {v1 <- m.get(t._1)
v2 <- m.get(t._2)}
yield (v1, v2)
}
// class E3(val t: Tuple2[String,String,String]) ...
You may define a helper function to make building extractors easier:
def mkMapExtractor(k1: String, k2: String) = new E2( (k1, k2) )
// def mkMapExtractor(k1: String, k2: String, k3: String) = new E3( (k1, k2, k3) )
Let's make an extractor object
val PersonExt = mkMapExtractor("firstname", "lastname")
and build Person:
val testMap = Map("lastname" -> "L", "firstname" -> "F")
PersonExt.unapply(testMap) map {Person.tupled}
or
testMap match {
case PersonExt(f,l) => println(Person(f,l))
case _ => println("err")
}
Adapt to your taste.
P.S. Oops, I didn't realize you asked about named arguments specifically. While my answer is about positional arguments, I shall still leave it here just in case it could be of some help.
Since Map is essentially just a List of tuples you can treat it as such.
scala> val person = args.toList match {
case List(("firstname", firstname), ("lastname", lastname), _) => new Person(firstname, lastname)
case _ => throw new Exception
}
person: Person = Person(John,Doe)
I made Person a case class to have the toString method generated for me.