I am new to Scala and am trying to get to grips with Option. I am trying to
sum the double values members of the following list according to their string keys:
val chmembers = List(("Db",0.1574), ("C",1.003), ("Db",15.4756), ("D",0.003), ("Bb",1.4278), ("D",13.0001))
Summing as:
List((D,13.0031), (Db,15.633000000000001), (C,1.003), (Bb,1.4278))
My current code is
def getClassWeights: List[(Option[String], Option[Double])] = {
chMembers.flatMap(p => Map(p.name -> p.weight))
.groupBy(_._1).mapValues(_.map(_._2)sum).toList
}
However this will not compile and returns:
'Could not find implicit value for parameter num:
Numeric[Option[Double]]'.
I don't understand where 'Numeric' comes from or how handle it.
I would be grateful for any suggestions. Thanks in advance.
Numeric is used to perform the sum. It is a type class which implements some common numeric operations for Int, Float, Double etc.
I am not sure why you want to use Option, I do not think it will help you solving your problem. Your code can be simplified to:
val chmembers = List(("Db",0.1574), ("C",1.003), ("Db",15.4756), ("D",0.003), ("Bb",1.4278), ("D",13.0001))
def getClassWeights: List[(String, Double)] = {
chmembers.groupBy(_._1).mapValues(_.map(_._2).sum).toList
}
you can do it like this:
val chmembers = List(("Db",0.1574), ("C",1.003), ("Db",15.4756), ("D",0.003), ("Bb",1.4278), ("D",13.0001))
chmembers.groupBy(_._1).mapValues(_.map(_._2).sum)
output
//Map(D -> 13.0031, Db -> 15.633000000000001, C -> 1.003, Bb -> 1.4278)
Related
I am trying to get the sum of arguments using currying in Scala. This is a very basic question so please don't go hard on it.
If you can answer the question then please otherwise let it be:
object MultiSum extends App {
var res=0
def sum(f:Int=>Int)(x:Int*)={f(x)}
val a=List(1,2,3,4,5,6,7,8,9)
val b=sum(x=>x+x)(a:_*)
}
So val b will be storing the result of all the numbers passed in the List in var a
Thanks.
Well... first of all... I think you should call your function collect and not sum. Now, lets help you write this collect.
So... your collect function takes two arguments,
1 - A collector function.
2 - Things to accumulate.
So... your collector function needs to know two things... first is already collected and second is next item to collect... and it will produce the next collected.
Which means that for collection of Int, your collector function will have a signature (Int, Int) => Int.
So, your collect function should have following signature,
def collect(collector: (Int, Int) => Int)(items: Int*): Int = ???
Now... lets come to implement it.
1 - Using reduce
def collect(collector: (Int, Int) => Int)(items: Int*): Int =
items.reduce(collector)
2 - Using fold
def collect(collector: (Int, Int) => Int)(items: Int*): Int =
items.fold(0)(collector)
Now, you can use it like this,
val l = List(1,2,3,4,5,6,7,8,9)
val b = collect((i1, i2) => i1 + i2)(l: _*)
I have a use-case where I need to define a new enum type LongShort but I need it in a way to also carry the sign so it can be directly used in mathematical expressions e.g.
object LongShortType extends Enumeration {
type Type = Value
val Long = Value(+1)
val Short = Value(-1)
}
I'd then like to use it like this:
val longShort = LongShortType.Short
val numberOfContracts: Int = 10
val vanillaOptionNotional: Double = longShort*numberOfContracts
but this leads to compiler error cannot resolve symbol * ... is there a way to extract the value of the enum? Or am I not understanding how enum types work?
The type of LongShortType.Short isn't Int, it's Value. You can either extract the underlying id of the value:
val longShort = LongShortType.Short.id
Which is a little ugly. Or you could not use an enum type at all:
object LongShortType {
val Long = 1
val Short = -1
}
And then your equation would work as is.
OK I worked out a solution to accomplish what I wanted without any compromisse and by that I mean that this solution has all the advantages of using Scala enum e.g. the withName and still allows me to define extra features on it:
object LongShortType extends Enumeration {
type Type = LongShortVal
val Long = Value("Long", +1)
val Short = Value("Short", -1)
case class LongShortVal(name: String, sign: Int) extends Val(nextId, name)
protected final def Value(name: String, sign: Int) = new LongShortVal(name, sign)
}
and now can do:
val longShort = LongShortType.Short
val numberOfContracts: Int = 10
val vanillaOptionNotional: Double = longShort.sign*numberOfContracts
and can also do:
val longShort = LongShort.withName("Long") // returns LongShort.Long
I am trying to define an accumulator variable of type String in Scala shell (driver) but I keep getting the following error:-
scala> val myacc = sc.accumulator("Test")
<console>:21: error: could not find implicit value for parameter param: org.apache.spark.AccumulatorParam[String]
val myacc = sc.accumulator("Test")
^
This seems to be no issue for Int or Double type of accumulator.
Thanks
That's because Spark by default provides only accumulators of type Long, Double and Float. If you need something else you have to extend AccumulatorParam.
import org.apache.spark.AccumulatorParam
object StringAccumulatorParam extends AccumulatorParam[String] {
def zero(initialValue: String): String = {
""
}
def addInPlace(s1: String, s2: String): String = {
s"$s1 $s2"
}
}
val stringAccum = sc.accumulator("")(StringAccumulatorParam)
val rdd = sc.parallelize("foo" :: "bar" :: Nil, 2)
rdd.foreach(s => stringAccum += s)
stringAccum.value
Note:
In general you should avoid using accumulators for tasks where data may grow significantly over time. Its behavior will similar to group an collect and in the worst case scenario can fail due to lack of resources. Accumulators are useful mostly for simple diagnostics tasks like keeping track of basic statistics.
Is there a way to get the Type of a field with scala reflection?
Let's see the standard reflection example:
scala> class C { val x = 2; var y = 3 }
defined class C
scala> val m = ru.runtimeMirror(getClass.getClassLoader)
m: scala.reflect.runtime.universe.Mirror = JavaMirror ...
scala> val im = m.reflect(new C)
im: scala.reflect.runtime.universe.InstanceMirror = instance mirror for C#5f0c8ac1
scala> val fieldX = ru.typeOf[C].declaration(ru.newTermName("x")).asTerm.accessed.asTerm
fieldX: scala.reflect.runtime.universe.TermSymbol = value x
scala> val fmX = im.reflectField(fieldX)
fmX: scala.reflect.runtime.universe.FieldMirror = field mirror for C.x (bound to C#5f0c8ac1)
scala> fmX.get
res0: Any = 2
Is there a way to do something like
val test: Int = fmX.get
That means can I "cast" the result of a reflection get to the actual type of the field? And otherwise: is it possible to do a reflection set from a string? In the example something like
fmx.set("10")
Thanks for hints!
Here's the deal... the type is not known at compile time, so, basically, you have to tell the compiler what the type it's supposed to be. You can do it safely or not, like this:
val test: Int = fmX.get.asInstanceOf[Int]
val test: Int = fmX.get match {
case n: Int => n
case _ => 0 // or however you want to handle the exception
}
Note that, since you declared test to be Int, you have to assign an Int to it. And even if you kept test as Any, at some point you have to pick a type for it, and it is always going to be something static -- as in, in the source code.
The second case just uses pattern matching to ensure you have the right type.
I'm not sure I understand what you mean by the second case.
How can I sort my Tag elements by id ?
I don't get how to define the orderById function...
case class Tag(id: Int, name: String, ttype: String)
val orderByID = Ordering[??].on[Tag](? => ? -> ?)
val mySet: SortedSet[Tag] = SortedSet()(orderByID) ;
I think you need:
val orderByID = Ordering.by[Tag, Int](_.id)
val mySet: SortedSet[Tag] = SortedSet()(orderByID)
Ordering.on turns an Ordering[U] into an Ordering[T] given a function to produce a U from a T. In this case, we need to start with an Ordering[Int], then provide a Tag => Int function:
val orderByID = Ordering[Int].on[Tag](_.id)
But as #Lee has said, Ordering.by is a more straightforward approach here.