Best practices for mixing in Scala concurrent.Map - scala

The ScalaDoc says this about concurrentMap: "Deprecated (Since version 2.10.0) Use scala.collection.concurrent.Map instead." Unfortunately, the rest of the Scala docs has not been updated and still references concurrentMap.
I tried to mix in concurrent.Map into a HashMap, with the following results:
scala> val mmap = new mutable.HashMap[String, String] with collection.concurrent.Map[String, String]
<console>:16: error: object creation impossible, since:
it has 4 unimplemented members.
/** As seen from anonymous class $anon, the missing signatures are as follows.
* For convenience, these are usable as stub implementations.
*/
def putIfAbsent(k: String,v: String): Option[String] = ???
def remove(k: String,v: String): Boolean = ???
def replace(k: String,v: String): Option[String] = ???
def replace(k: String,oldvalue: String,newvalue: String): Boolean = ???
val mmap = new mutable.HashMap[String, String] with collection.concurrent.Map[String, String]
So we see that instead of a simple mixin, some methods must also be implemented. Is this the best way to use concurrent.Map, or is there a better way?

The scala.collection.concurrent.Map trait is not meant to be mixed-in with an existing mutable Scala Map to obtain a thread-safe version of the map instance. The SynchronizedMap mixin existed for this purpose before 2.11, but is now deprecated.
Currently, Scala has the scala.collection.concurrent.TrieMap implementation for the scala.collection.concurrent.Map interface, but can wrap Java classes as well.
The scala.collection.concurrent.Map, in versions prior to 2.10 known as scala.collection.mutable.ConcurrentMap, interface is used when you:
want to implement your own concurrent, thread-safe Map from scratch
want to wrap an existing Java concurrent map implementation:
E.g:
import scala.collection._
import scala.collection.convert.decorateAsScala._
import java.util.concurrent.ConcurrentHashMap
val map: concurrent.Map[String, String] = new ConcurrentHashMap().asScala
want to write generic code that works concurrent maps, and don't want to commit to a specific implementation:
E.g.:
import scala.collection._
def foo(map: concurrent.Map[String, String]) = map.putIfAbsent("", "")
foo(new concurrent.TrieMap)
foo(new java.util.concurrent.ConcurrentSkipListMap().asScala)
you could implement your own wrapper around a single-threaded mutable map implementation by using synchronized (but you would need to ensure that your program is accessing the mutable map only through this wrapper and never directly).
E.g.:
class MySynchronizedMap[K, V](private val underlying: mutable.Map[K, V])
extends concurrent.Map[K, V] {
private val monitor = new AnyRef
def putIfAbsent(k: K,v: V): Option[String] = monitor.synchronized {
underlying.get(k) match {
case s: Some[V] => s
case None =>
underlying(k) = v
None
}
}
def remove(k: K, v: V): Boolean = monitor.synchronized {
underlying.get(k) match {
case Some(v0) if v == v0 => underlying.remove(k); true
case None => false
}
}
// etc.
}

Unless you want to implement a concurrent mutable hash map yourself, you have to use scala.collection.concurrent.TrieMap.

Taken from this comment directly:
https://stackoverflow.com/a/49689669/7082628
In 2018, apparently you can just do this:
import java.util.concurrent.ConcurrentHashMap
val m: ConcurrentHashMap[String,MyClass] = new ConcurrentHashMap

By "simple mixin", perhaps you're asking if the trait can be used as a decorator as shown here for SynchronizedMap, and the answer is apparently not.
Implementations include TrieMap and the wrapper for Java's ConcurrentMap (of which there are two implementations). (Java also offers ConcurrentSkipListSet as Set.)
Also see this roll-your-own question.
They have you covered on the conversion side of things, if that's what you were used to:
scala> import java.util.concurrent._
import java.util.concurrent._
scala> import collection.JavaConverters._
import collection.JavaConverters._
scala> val m = new ConcurrentHashMap[String, Int]
m: java.util.concurrent.ConcurrentHashMap[String,Int] = {}
scala> val mm = m.asScala
mm: scala.collection.concurrent.Map[String,Int] = Map()
scala> mm.replace("five",5)
res0: Option[Int] = None
scala> mm.getClass
res1: Class[_ <: scala.collection.concurrent.Map[String,Int]] = class scala.collection.convert.Wrappers$JConcurrentMapWrapper

Update for 2021 and Scala 2.13:
You need to use different implicit conversion when wrapping the Java concurrent map implementation:
import java.util.concurrent.ConcurrentHashMap
import scala.collection.concurrent
import scala.jdk.CollectionConverters._
val map: concurrent.Map[String, String] = new ConcurrentHashMap().asScala

Related

How to make method return the same generic as the input?

I want to split a string delimited by commas and use the result as either a Seq or a Set:
def splitByComma(commaDelimited: String): Array[String]
= commaDelimited.trim.split(',')
def splitByCommaAsSet(commaDelimited: String): Set[String]
= splitByComma(commaDelimited).toSet
def splitByCommaAsSeq(commaDelimited: String): Seq[String]
= splitByComma(commaDelimited).toSeq
val foods = "sushi,taco,burrito"
val foodSet = splitByCommaAsSet(foods)
// foodSet: scala.collection.immutable.Set[String] = Set(sushi, taco, burrito)
val foodSeq = splitByCommaAsSeq(foods)
// foodSeq: Seq[String] = List(sushi, taco, burrito)
However, this is quite repetitive. Ideally, I would want to have something like genericSplitByComma[Set](foods) that just returns a Set when I pass Set in, and returns a Seq when I pass Seq.
#KrzysztofAtłasik's answer works great for Scala 2.12.
This is a solution for 2.13. (Not completely sure if this is the best way).
import scala.collection.Factory
import scala.language.higherKinds
def splitByComma[C[_]](commaDelimited: String)(implicit f: Factory[String, C[String]]): C[String] =
f.fromSpecific(commaDelimited.split(","))
// Or, as Dmytro stated, which I have to agree looks better.
commaDelimited.split(",").to(f)
Which you can use like this:
splitByComma[Array]("hello,world!")
// res: Array[String] = Array(hello, world!)
splitByComma[Set]("hello,world!")
// res: Set[String] = Set(hello, world!)
splitByComma[List]("hello,world!")
// res: List[String] = List(hello, world!)
There's a method in Scala called to which can transform arbitrary collection to another as long as there is typeclass called CanBuildFrom in scope.
import scala.collection.generic.CanBuildFrom
import scala.languageFeature.higherKinds
def genericSplitByComma[S[_]](s: String)(implicit cbf: CanBuildFrom[Nothing, String, S[String]]): S[String] = {
s.split(",").to[S]
}
genericSplitByComma[Set]("Hello, hello") //Set(Hello, hello)
genericSplitByComma[List]("Hello, hello") //List(Hello, hello)
genericSplitByComma[Array]("Hello, hello") //Array(hello, world!)
We don't need to constrain S[_] because this function won't compile if there is no suitable CanBuildFrom in scope. For example, this will fail:
genericSplitByComma[Option]("Hello, hello")
Below will also fail because our type constructor S[_] accepts only one type argument and the map expects two:
genericSplitByComma[Map]("Hello, hello")
As Luis Miguel Mejía Suárez and Dmytro Mitin noticed, there was major refactor in collections in just-released Scala 2.13, so it will work up to Scala 2.12.
There's a simple workaround for this. Not exactly the requested syntax but just as concise and it should be 2.13 compatible.
def simpleSplitByComma(coll :Iterable[String]) =
coll.flatMap(_.trim.split(","))
simpleSplitByComma(Set("hello,world")) //res0: Set(hello, world)
simpleSplitByComma(Seq("bellow,world")) //res1: List(bellow, world)
simpleSplitByComma(Array("fellow,old")) //res2: ArrayBuffer(fellow, old)
simpleSplitByComma(Stream("end,of,the,world")) //res3: Stream(end, ?)

How make implicit Ordered on java.time.LocalDate

I want to use java.time.LocalDate and java.time.LocalDateTime with an implicit Ordered like:
val date1 = java.time.LocalDate.of(2000, 1, 1)
val date2 = java.time.LocalDate.of(2010, 10, 10)
if (date1 < date2) ...
import scala.math.Ordering.Implicits._ doesn't work, because LocalDate inherits from Comparable<ChronoLocalDate> instead of Comparable<LocalDate>.
How can I write my own imlicit Orderd to use <, <=, >, >= operators/methods to compare LocalDate's?
Edit:
I found a way with use of an implicit class:
import java.time.{LocalDate}
object MyDateTimeUtils {
implicit class MyLocalDateImprovements(val ld: LocalDate)
extends Ordered[LocalDate] {
def compare(that: LocalDate): Int = ld.compareTo(that)
}
}
// Test
import MyDateTimeUtils._
val d1 = LocalDate.of(2016, 1, 1)
val d2 = LocalDate.of(2017, 2, 3)
if (d1 < d2) println("d1 is less than d2")
But I would prefer a way like Scala is doing for all Java classes which implements Comparable<T>. You just have to
import scala.math.Ordering.Implicits._ in your code. Scala implements it like:
implicit def ordered[A <% Comparable[A]]: Ordering[A] = new Ordering[A] {
def compare(x: A, y: A): Int = x compareTo y
}
But unfortunately LocalDate implements Comparable<ChronoLocalDate> instead of Comparable<LocalDate>. I couldn't find a way to modify the above implicit ordered method to fit with LocalDate/Comparable<ChronoLocalDate>. Any idea?
You can use Ordering.by to create ordering for any type, given a function from that type to something that already has an Ordering - in this case, to Long:
implicit val localDateOrdering: Ordering[LocalDate] = Ordering.by(_.toEpochDay)
Comparing by LocalDate.toEpochDay is clear, though maybe relatively slow...
The answer by #tzach-zohar is great, in that it's most obvious what is going on; you're ordering by Epoch Day:
implicit val localDateOrdering: Ordering[LocalDate] = Ordering.by(_.toEpochDay)
However, if you look at the implementation of toEpochDay you'll see that it's relatively involved - 18 lines of code, with 4 divisions, 3 conditionals and a call to isLeapYear() - and the resulting value isn't cached, so it gets recalculated with each comparison, which might be expensive if there were a large number of LocalDates to be sorted.
...making use of LocalDate.compareTo is probably more performant...
The implementation of LocalDate.compareTo is simpler - just 2 conditionals, no division - and it's what you'd be getting with that implicit conversion of java.lang.Comparable to scala.math.Ordering that scala.math.Ordering.Implicits._ offers, if only it worked! But as you said, it doesn't, because LocalDate inherits from Comparable<ChronoLocalDate> instead of Comparable<LocalDate>. One way to take advantage of it might be...
import scala.math.Ordering.Implicits._
implicit val localDateOrdering: Ordering[LocalDate] =
Ordering.by(identity[ChronoLocalDate])
...which lets you order LocalDates by casting them to ChronoLocalDates, and using the Ordering[ChronoLocalDate] that scala.math.Ordering.Implicits._ gives you!
...and in the end it looks best with the lambda syntax for SAM types
The lambda syntax for SAM types, introduced with Scala 2.12, can make really short work of constructing a new Ordering :
implicit val localDateOrdering: Ordering[LocalDate] = _ compareTo _
...and I think this ends up being my personal favourite! Concise, still fairly clear, and using (I think) the best-performing comparison method.
A slight modification to the implicit ordered should do the trick.
type AsComparable[A] = A => Comparable[_ >: A]
implicit def ordered[A: AsComparable]: Ordering[A] = new Ordering[A] {
def compare(x: A, y: A): Int = x compareTo y
}
Now every type which is comparable to itself or to a supertype of itself should have an Ordering.
Here is the solution that I use:
define two implicits. The first one for making an Ordering[LocalDate] available. And a second one for giving LocalDate a compare method which comes in very handy. I typically put these in package objects in a library I can just include where I need them.
package object net.fosdal.oslo.odatetime {
implicit val orderingLocalDate: Ordering[LocalDate] = Ordering.by(d => (d.getYear, d.getDayOfYear))
implicit class LocalDateOps(private val localDate: LocalDate) extends AnyVal with Ordered[LocalDate] {
override def compare(that: LocalDate): Int = Ordering[LocalDate].compare(localDate, that)
}
}
with both of these defined you can now do things like:
import net.fosdal.oslo.odatetime._
val bool: Boolean = localDate1 < localDate1
val localDates: Seq[LocalDate] = ...
val sortedSeq = localDates.sorted
Alternatively... you could just use my library (v0.4.3) directly. see: https://github.com/sfosdal/oslo
Here's my solution for java.time.LocalDateTime
implicit val localDateTimeOrdering: Ordering[LocalDateTime] =
Ordering.by(x => x.atZone(ZoneId.of("UTC")).toEpochSecond)
one possible implementation in Scala 3 is to use a Conversion:
given toOrderedLocalDate: Conversion[LocalDate, Ordered[LocalDate]] with
def apply(x: LocalDate): Ordered[LocalDate] = (y: LocalDate) => x.compareTo(y)

How do you implicitly convert to Java autoboxed types in Scala?

I've seen how we can go from Scala to Java, e.g. from Scala can't multiply java Doubles?:
implicit def javaToScalaDouble(d: java.lang.Double): Double = d.doubleValue
I tried something naive to go in the other direction:
implicit def toJavaDouble(d: Double): java.lang.Double = new java.lang.Double(d)
However, I still get a compilation error when a complex type:
Error:(123, 99) type mismatch;
found : java.util.Map[String,java.util.function.Function[edu.xxx.SimulationTimestep,scala.Double]]
required: java.util.Map[String,java.util.function.Function[edu.xxx.SimulationTimestep,java.lang.Double]]
EconomicTimeSeriesCSVWriter.write(timesteps(simulationId, cowId), new File(selected), datasetFunctions, isParallelizable)
The problem is with the type of datasetFunctions; it is defined as:
val datasetFunctions: util.Map[String, Function[SimulationTimestep, Double]] = ...
I've also imported JavaConversions._, though I don't think that seems to help with autoboxed types from what I can see so far.
Edited to add a simplified example:
import java.util
import scala.collection.JavaConversions._
type O = java.lang.Object
type JD = java.lang.Double
implicit def javaToScalaDouble(d: JD): Double = d.doubleValue
implicit def scalaToJavaDouble(d: Double): JD = new JD(d)
def myJavaFun(in: util.Map[O, JD]): Unit = {}
val myMap: util.Map[O, Double] = new util.HashMap[O, Double]()
//Doesn't work
// myJavaFun(myMap)
//Doesn't seem to work either
myJavaFun(mapAsJavaMap[O, JD](myMap.map(x => x).toSeq.toMap))
// A more real example (if it should make any difference)
//def myJavaFun2(in: util.Map[String, Function[O, Double]]): Unit = {}
Two problems:
Having an implicit B => C won't automatically let you use a Map[A, B] as a Map[A, C]. You need to do something like .mapValues(x => x) to provide some place for the implicit to apply.
The implicits between scala.Double and java.lang.Double are already defined in Predef. Your redefinitions of them create an ambiguity which prevents them from applying.
Your example, fixed and trimmed down:
import scala.collection.JavaConversions._
def myJavaFun(in: java.util.Map[Object, java.lang.Double]): Unit = {}
val myMap = new java.util.HashMap[Object, Double]()
myJavaFun(mapAsJavaMap(myMap.mapValues(x => x)))

Polymorphic instantiation in Scala using TypeTag and ClassTag

In Scala 2.9 one could implement polymorphic instantiation as
def newInstance[T](implicit m: Manifest[T]) =
m.erasure.newInstance.asInstanceOf[T]
but as of 2.10 Manifest is being replaced with TypeTag,
and it is not clear to me how to achieve something similar with TypeTag.
I would prefer if the TypeTag version preserved all available type information.
I know that the above only works for traits/classes that do not require constructor args,
and ven then it does not always work, but it works well enough for what I need.
If I can do better the new reflection APIs that would be great.
TypeTag is not yet a replacement for Manifest because it's a part of experimental and unstable Scala reflection. You definitely shouldn't use it for production as of now.
For the use case you showed, where only runtime class is needed (not full type information with generics etc.), Scala 2.10 introduced ClassTag, which you can use like this:
def newInstance[T: ClassTag] =
implicitly[ClassTag[T]].runtimeClass.newInstance.asInstanceOf[T]
or:
def newInstance[T](implicit ct: ClassTag[T]) =
ct.runtimeClass.newInstance.asInstanceOf[T]
Anyway, Manifest isn't deprecated yet, so I guess you can still use it.
EDIT:
Using TypeTag to achieve the same:
import scala.reflect.runtime.universe._
def newInstance[T: TypeTag] = {
val clazz = typeTag[T].mirror.runtimeClass(typeOf[T])
clazz.newInstance.asInstanceOf[T]
}
The above solution still uses some Java reflection. If we want to be puristic and use only Scala reflection, this is the solution:
def newInstance[T: TypeTag]: T = {
val tpe = typeOf[T]
def fail = throw new IllegalArgumentException(s"Cannot instantiate $tpe")
val noArgConstructor = tpe.member(nme.CONSTRUCTOR) match {
case symbol: TermSymbol =>
symbol.alternatives.collectFirst {
case constr: MethodSymbol if constr.paramss == Nil || constr.paramss == List(Nil) => constr
} getOrElse fail
case NoSymbol => fail
}
val classMirror = typeTag[T].mirror.reflectClass(tpe.typeSymbol.asClass)
classMirror.reflectConstructor(noArgConstructor).apply().asInstanceOf[T]
}
If you want to support passing args, here's a trick I do with 2.11:
def newInstance[T : ClassTag](init_args: AnyRef*): T = {
classTag[T].runtimeClass.getConstructors.head.newInstance(init_args: _*).asInstanceOf[T]
}
Example usage:
scala> case class A(x:Double, y:Int)
defined class A
scala> newInstance[A](4.5.asInstanceOf[Object],3.asInstanceOf[Object])
res1: A = A(4.5,3)

Implicit conversion of a generic container for an implicit parameter in Scala

Is there a way to make this work? (Scala 2.8.1)
class A
def f(implicit a: A) = 0
class Vendor[T](val v: T)
implicit val vendor = new Vendor(new A)
implicit def vendorToVal[T](implicit v: Vendor[T]) = v.v
f
The error is: 'diverging implicit expansion for type A starting with method vendorToVal'
This is related to Lift 2.2 dependency injection, the real code looks like this:
class UserStore(implicit db: DbAccess)
object DependencyFactory extends Factory {
implicit val db = new FactoryMaker[DbAccess](Model) {}
import db._ // implicit conversion would allow to remove this import
implicit val userStore = new FactoryMaker[UserStore](new UserStore) {}
}
This question is related to: Is there a way to implicitly convert an implicit parameter in Scala?
The problem is caused with vendorToVal method - I observed the same behavior many times, when I've been using implicit parameters in implicit type-parametrized methods. Unfortunately, I've found no simple and elegant glue in 2.8._.
Some interesting threads, related to the topic:
http://scala-programming-language.1934581.n4.nabble.com/scala-Why-is-this-a-diverging-implicit-td1998156.html
http://www.scala-lang.org/node/6847
In Scala 2.9 trunk, you can do this:
scala> class A
defined class A
scala> def f(implicit a: A) = 0
f: (implicit a: A)Int
scala>
scala> class Vendor[T](val v: T)
defined class Vendor
scala> implicit def value[T: Vendor] = implicitly[Vendor[T]].v
value: [T](implicit evidence$1: Vendor[T])T
scala> implicit val vendor = new Vendor(new A)
vendor: Vendor[A] = Vendor#bbb2d0
scala> f
res0: Int = 0
Calling f will search for a value of type A, and find the implicit value[A], which requires an evidence parameter of type Vendor[A]. It resolves this evidence parameter to vendor.
I don't think implicits were that powerful in 2.8.1.