I'm using a ConcurrentHashMap in Scala and I would like to use the computeIfAbsent() method but can't figure out the syntax for the second argument. Can someone show me what would be the proper syntax?
When running the following code
val data = new ConcurrentHashMap[String, LongAdder]
data.computeIfAbsent("bob", k: String => new LongAdder()).increment()
I'm getting the following error
Type mismatch, expected: Function[_ >: String, _ <: LongAdder], actual: (String) => Any
Thanking you in advance
Francis
The problem is that you're using java.util.concurrent.ConcurrentHashMap, which accepts java.util.function.Function as a parameter for computeIfAbsent() instead of scala.Function1 which you pass to it.
Since scala doesn't support lambda conversion for functional interfaces as Java does (at least not without the -Xexperimental flag), you can solve this by implementing a java.util.function.Function explicitly:
val data = new ConcurrentHashMap[String, LongAdder]
val adderSupplier = new java.util.function.Function[String, LongAdder]() {
override def apply(t: String): LongAdder = new LongAdder()
}
data.computeIfAbsent("bob", adderSupplier).increment()
Alternatively, if you need this more often, you may write a utility conversion function or even an implicit conversion:
object FunctionConverter {
implicit def scalaFunctionToJava[From, To](function: (From) => To): java.util.function.Function[From, To] = {
new java.util.function.Function[From, To] {
override def apply(input: From): To = function(input)
}
}
}
import FunctionConverter._
val data = new ConcurrentHashMap[String, LongAdder]()
data.computeIfAbsent("bob", (k: String) => new LongAdder()) // <- implicit conversion applied here
If you enable -Xexperimental flag you can use scala anonymous function notation for this:
scala> val data = new java.util.concurrent.ConcurrentHashMap[String, Int]
data: java.util.concurrent.ConcurrentHashMap[String,Int] = {}
scala> data.computeIfAbsent("bob", _.size)
res0: Int = 3
Note that you still can't pass regular scala Function
scala> val f: String => Int = _.size
f: String => Int = <function1>
scala> data.computeIfAbsent("bob", f)
<console>:13: error: type mismatch;
found : String => Int
required: java.util.function.Function[_ >: String, _ <: Int]
data.computeIfAbsent("bob", f)
^
But eta-expansion will work
scala> def a(s: String): Int = s.size
a: (s: String)Int
scala> data.computeIfAbsent("bob", a)
res3: Int = 3
Related
Scala compiler detects the following two map functions as duplicates conflicting with each other:
class ADT {
def map[Output <: AnyVal](f: Int => Output): List[Output] = ???
def map[Output >: Null <: AnyRef](f: Int => Output): List[Output] = ???
}
The class type of Output parameter is different. First one limits to AnyVal and second one limits to AnyRef. How can I differentiate them?
The problem is not differentiating AnyVal from AnyRef so much as getting around the fact that both method signatures become the same after erasure.
Here is a neat trick to get around this kind of problem. It is similar to what #som-snytt did, but a bit more generic, as it works for other similar situations as well (e.g. def foo(f: Int => String): String = ??? ; def foo(f: String => Int): Int = ??? etc.):
class ADT {
def map[Output <: AnyVal](f: Int => Output): List[Output] = ???
def map[Output >: Null <: AnyRef](f: Int => Output)(implicit dummy: DummyImplicit): List[Output] = ???
}
The cutest thing is that this works "out of the box". Apparently, a DummyImplicit is a part of standard library, and you always have the thing in scope.
You can have more than two overloads this way too by just adding more dummies to the list.
scala 2.13.0-M5> :pa
// Entering paste mode (ctrl-D to finish)
object X {
def map[Output <: AnyVal](f: Int => Output) = 1
def map[O](f: Int => O)(implicit ev: O <:< AnyRef) = 2
}
// Exiting paste mode, now interpreting.
defined object X
scala 2.13.0-M5> X.map((x: Int) => x*2)
res0: Int = 1
scala 2.13.0-M5> X.map((x: Int) => "")
res1: Int = 2
You could use a typeclass for that map method.
Using your exact example:
trait MyTC[Output]{
def map(f: Int => Output): List[Output]
}
object MyTC{
def apply[A](a : A)(implicit ev : MyTC[A]) : MyTC[A] = ev
implicit def anyRefMyTc[A <: AnyRef] : MyTC[A] = new MyTC[A]{
def map(f: Int => A): List[A] = { println("inside sub-AnyRef"); List.empty }
}
implicit def anyValMyTc[A <: AnyVal] : MyTC[A] = new MyTC[A]{
def map(f: Int => A): List[A] = { println("inside sub-AnyVal"); List.empty }
}
}
import MyTC._
val r1 = Option("Test1")
val r2 = List(5)
val v1 = true
val v2 = 6L
// The functions here are just to prove the point, and don't do anything.
MyTC(r1).map(_ => None)
MyTC(r2).map(_ => List.empty)
MyTC(v1).map(_ => false)
MyTC(v2).map(_ => 10L)
That would print:
inside sub-AnyRef
inside sub-AnyRef
inside sub-AnyVal
inside sub-AnyVal
The advantage of this approach is that, should you then choose to specialise the behaviour further for just some specific type (e.g. say you want to do something specific for Option[String]), you can do that easily:
// This is added to MyTC object
implicit val optMyTc : MyTC[Option[String]] = new MyTC[Option[String]]{
def map(f: Int => Option[String]): List[Option[String]] = { println("inside Option[String]"); List.empty }
}
Then, re-running the code will print:
inside Option[String]
inside sub-AnyRef
inside sub-AnyVal
inside sub-AnyVal
I don't know why the following code can not compile, this is the error message:
Error:(29, 7) no type parameters for method flatMap: (f: String => Option[B])Option[B] exist so that it can be applied to arguments (String => Some[Class[?0]] forSome { type ?0 <: org.apache.hadoop.io.compress.CompressionCodec })
--- because ---
argument expression's type is not compatible with formal parameter type;
found : String => Some[Class[?0]] forSome { type ?0 <: org.apache.hadoop.io.compress.CompressionCodec }
required: String => Option[?B]
a.flatMap(codecClassName => {
^
and code
def f(a: Option[String]): Unit = {
a.flatMap(codecClassName => {
val codecFactory = new CompressionCodecFactory(new Configuration())
val codecClass = codecFactory.getCodecClassByName(codecClassName)
if (codecClass == null) {
throw new RuntimeException("Unknown or not supported codec:" + codecClassName)
}
Some(codecClass)
})
}
This seems to be related to the fact that getClass and classOf are not returning the exact same thing. See Scala equivalent of Java java.lang.Class<T> Object for more details.
Looking around for a workaround I came across Scala Getting class type from string representation.
So how about:
val codecClass = Manifest.classType(codecFactory.getCodecClassByName(codecClassName))
This should work. flatMap involves both map and flatten, so it may need more type annotations in some cases. The overall code works after annotation of the function parameter, i.e. (codecClassName: String).
Note the other change -- that flatMap with an inner function returning an Option type is the same as a map if that function returns what's inside the Option (i.e. flattens the option) (see below).
import org.apache.hadoop.conf.Configuration
import org.apache.hadoop.io.compress.{CompressionCodec, CompressionCodecFactory}
...
def f(a: Option[String]): Option[Class[_ <: CompressionCodec]] = {
a.map((codecClassName: String) => {
val codecFactory = new CompressionCodecFactory(new Configuration())
val codecClass = codecFactory.getCodecClassByName(codecClassName)
if (codecClass == null) {
throw new RuntimeException("Unknown or not supported codec:" + codecClassName)
}
codecClass
})
}
To show the relationship between flatMap and map as described above:
scala> val opt: Option[Int] = Some(1)
opt: Option[Int] = Some(1)
scala> opt.map((i: Int) => i + 1)
res0: Option[Int] = Some(2)
scala> val opt2: Option[Int] = None
opt2: Option[Int] = None
scala> opt.flatMap((i: Int) => Some(i + 1))
res1: Option[Int] = Some(2)
scala> opt2.map((i: Int) => i + 1)
res3: Option[Int] = None
scala> opt2.flatMap((i: Int) => Some(i + 1))
res2: Option[Int] = None
Essentially, what I would like to do is write overloaded versions of "map" for a custom class such that each version of map differs only by the type of function passed to it.
This is what I would like to do:
object Test {
case class Foo(name: String, value: Int)
implicit class FooUtils(f: Foo) {
def string() = s"${f.name}: ${f.value}"
def map(func: Int => Int) = Foo(f.name, func(f.value))
def map(func: String => String) = Foo(func(f.name), f.value)
}
def main(args: Array[String])
{
def square(n: Int): Int = n * n
def rev(s: String): String = s.reverse
val f = Foo("Test", 3)
println(f.string)
val g = f.map(rev)
val h = g.map(square)
println(h.string)
}
}
Of course, because of type erasure, this won't work. Either version of map will work alone, and they can be named differently and everything works fine. However, it is very important that a user can call the correct map function simply based on the type of the function passed to it.
In my search for how to solve this problem, I cam across TypeTags. Here is the code I came up with that I believe is close to correct, but of course doesn't quite work:
import scala.reflect.runtime.universe._
object Test {
case class Foo(name: String, value: Int)
implicit class FooUtils(f: Foo) {
def string() = s"${f.name}: ${f.value}"
def map[A: TypeTag](func: A => A) =
typeOf[A] match {
case i if i =:= typeOf[Int => Int] => f.mapI(func)
case s if s =:= typeOf[String => String] => f.mapS(func)
}
def mapI(func: Int => Int) = Foo(f.name, func(f.value))
def mapS(func: String => String) = Foo(func(f.name), f.value)
}
def main(args: Array[String])
{
def square(n: Int): Int = n * n
def rev(s: String): String = s.reverse
val f = Foo("Test", 3)
println(f.string)
val g = f.map(rev)
val h = g.map(square)
println(h.string)
}
}
When I attempt to run this code I get the following errors:
[error] /src/main/scala/Test.scala:10: type mismatch;
[error] found : A => A
[error] required: Int => Int
[error] case i if i =:= typeOf[Int => Int] => f.mapI(func)
[error] ^
[error] /src/main/scala/Test.scala:11: type mismatch;
[error] found : A => A
[error] required: String => String
[error] case s if s =:= typeOf[String => String] => f.mapS(func)
It is true that func is of type A => A, so how can I tell the compiler that I'm matching on the correct type at runtime?
Thank you very much.
In your definition of map, type A means the argument and result of the function. The type of func is then A => A. Then you basically check that, for example typeOf[A] =:= typeOf[Int => Int]. That means func would be (Int => Int) => (Int => Int), which is wrong.
One of ways of fixing this using TypeTags looks like this:
def map[T, F : TypeTag](func: F)(implicit ev: F <:< (T => T)) = {
func match {
case func0: (Int => Int) #unchecked if typeOf[F] <:< typeOf[Int => Int] => f.mapI(func0)
case func0: (String => String) #unchecked if typeOf[F] <:< typeOf[String => String] => f.mapS(func0)
}
}
You'd have to call it with an underscore though: f.map(rev _). And it may throw match errors.
It may be possible to improve this code, but I'd advise to do something better. The simplest way to overcome type erasure on overloaded method arguments is to use DummyImplicit. Just add one or several implicit DummyImplicit arguments to some of the methods:
implicit class FooUtils(f: Foo) {
def string() = s"${f.name}: ${f.value}"
def map(func: Int => Int)(implicit dummy: DummyImplicit) = Foo(f.name, func(f.value))
def map(func: String => String) = Foo(func(f.name), f.value)
}
A more general way to overcome type erasure on method arguments is to use the magnet pattern. Here is a working example of it:
sealed trait MapperMagnet {
def map(foo: Foo): Foo
}
object MapperMagnet {
implicit def forValue(func: Int => Int): MapperMagnet = new MapperMagnet {
override def map(foo: Foo): Foo = Foo(foo.name, func(foo.value))
}
implicit def forName(func: String => String): MapperMagnet = new MapperMagnet {
override def map(foo: Foo): Foo = Foo(func(foo.name), foo.value)
}
}
implicit class FooUtils(f: Foo) {
def string = s"${f.name}: ${f.value}"
// Might be simply `def map(func: MapperMagnet) = func.map(f)`
// but then it would require those pesky underscores `f.map(rev _)`
def map[T](func: T => T)(implicit magnet: (T => T) => MapperMagnet): Foo =
magnet(func).map(f)
}
This works because when you call map, the implicit magnet is resolved at compile time using full type information, so no erasure happens and no runtime type checks are needed.
I think the magnet version is cleaner, and as a bonus it doesn't use any runtime reflective calls, you can call map without underscore in the argument: f.map(rev), and also it can't throw runtime match errors.
Update:
Now that I think of it, here magnet isn't really simpler than a full typeclass, but it may show the intention a bit better. It's a less known pattern than typeclass though. Anyway, here is the same example using the typeclass pattern for completeness:
sealed trait FooMapper[F] {
def map(foo: Foo, func: F): Foo
}
object FooMapper {
implicit object ValueMapper extends FooMapper[Int => Int] {
def map(foo: Foo, func: Int => Int) = Foo(foo.name, func(foo.value))
}
implicit object NameMapper extends FooMapper[String => String] {
def map(foo: Foo, func: String => String) = Foo(func(foo.name), foo.value)
}
}
implicit class FooUtils(f: Foo) {
def string = s"${f.name}: ${f.value}"
def map[T](func: T => T)(implicit mapper: FooMapper[T => T]): Foo =
mapper.map(f, func)
}
I'd like to dynamically insert a type parameter, i.e. List[T], where T can only be found at run-time.
Normally you can create bindings like this if you already have an existing type parameter T (not the case), or during run-time using TypeTags.
The latter sounds like the way to go, but this shifts the question to "how do I get my TypeTag here".
I know how to create a TypeTag using if you have your type T (typeTag[T]), or if you have an instance of your type T (see getTypeTag below).
However, I have neither; what I do have of my type in question is a reflect.runtime.universe.Type. Might it be possible to convert this into a TypeTag so I can dynamically insert my type parameter?
Bonus points if the List can be made dynamic as well.
scala> import scala.reflect.runtime.{universe => ru}
scala> def getTypeTag[T: ru.TypeTag](obj: T) = ru.typeTag[T]
scala> def insertTag[T](tt: ru.TypeTag[T]) = List[T]_
scala> val fn = (a: String) => "foo"
fn: String => String = <function1>
scala> val pars = getTypeTag(fn).tpe.typeArgs.init(0)
pars: reflect.runtime.universe.Type = String
scala> insertTag(pars)
<console>:21: error: type mismatch;
found : reflect.runtime.universe.Type
required: reflect.runtime.universe.TypeTag[?]
insertTag(pars)
^
I haven't figured out how to convert a Type to TypeTag. If your goal is to get TypeTag just for function parameters, you can modify your function getTypeTag to accept functions:
import scala.reflect.runtime.{universe => ru}
def getTypeTag[T: ru.TypeTag](obj: (T) => Any) = ru.typeTag[T]
def insertTag[T](tt: ru.TypeTag[T]) = List[T] _
val fn = (a: String, b: Int) => "foo"
var parTypeTag = getTypeTag(fn.tupled)
insertTag(parTypeTag) // Seq[(String, Int)] => List[(String, Int)] = <function1>
I am trying to define a Map literal with key: String, value: (Any)=>String. I tried the following, but get a syntax error:
def foo(x: Int): String = /...
def bar(x: Boolean): String = /...
val m = Map[String, (Any) => String]("hello" -> foo, "goodbye" -> bar)
Funny that no one actually gave a type that would work. Here's one such:
def foo(x: Int): String = x.toString
def bar(x: Boolean): String = x.toString
val m = Map[String, (Nothing) => String]("hello" -> foo, "goodbye" -> bar)
The reason why it works this way is because Function1 is contra-variant on the input, so (Nothing) => String is a superclass of (Int) => String. It is also co-variant on the output, so (Nothing) => Any would be a superclass to any other Function1.
Of course, you can't use it like that. Without manifests, you can't even uncover what the original type of Function1 is. You could try something like this, though:
def f[T : Manifest](v: T) = v -> manifest[T]
val m = Map[String, ((Nothing) => String, Manifest[_])]("hello" -> f(foo), "goodbye" -> f(bar))
val IntManifest = manifest[Int]
val BooleanManifest = manifest[Boolean]
val StringManifest = manifest[String]
m("hello")._2.typeArguments match {
case List(IntManifest, StringManifest) =>
m("hello")._1.asInstanceOf[(Int) => String](5)
case List(BooleanManifest, StringManifest) =>
m("hello")._1.asInstanceOf[(Boolean) => String](true)
case _ => "Unknown function type"
}
Int => String is not a subclass of Any => String, rather, the contrary. You can't put (replace) an Int => String function when a code expects Any => String, since that code can apply the function with, say, "hi".
#Ben suggestion works, but how is it useful? you can't invoke the function once you get it from the Map.
If you really want to do this, maybe define foo as a partial function:
val foo: PartialFunction[Any, String] = {case i: Int => ....}
Obviously, this will fail at runtime if you pass it a string, but you can always test if the function is ok for use with your parameter by using isDefinedAt. (another alternative may be manifests, but I don't see the value here)
If I let the compiler infer it I seem to get an illegal type:
scala> val m = Map("hello" -> foo _, "goodbye" -> bar _)
m: scala.collection.immutable.Map[java.lang.String,(Boolean with Int) => String] =
Map((hello,<function1>), (goodbye,<function1>))
scala> m("hello")(8)
<console>:9: error: type mismatch;
found : Int(8)
required: Boolean with Int
m("hello")(8)
scala> var q = new Boolean with Int
<console>:5: error: illegal inheritance from final class Boolean
var q = new Boolean with Int
Anyway, what you want is not the type Any but a generic of "any type" which is _:
scala> val mm = Map[String, (_) => String]("hello" -> foo _, "goodbye" -> bar _)
mm: scala.collection.immutable.Map[String,Function1[_, String]] =
Map((hello,<function1>), (goodbye,<function1>))
I just posted a question about how to invoke such functions because I don't actually know.
Trait Function1 is contravariant for parameter, so def foo(x: Int): String is not a (Any) => String. So the following would work:
scala> def baz(x: Any): String = "baz"
baz: (x: Any)String
scala> val m2 = Map[String, (String) => String]("hello" -> baz)
m2: scala.collection.immutable.Map[String,(String) => String] = Map((hello,<function1>))
This is how I did it to fulfill a similar requirement.
object MapToMethods {
private def increment(x: Int): Int = x+1
private def decrement(x: Int): Int = x-1
val m: Map[String, Int => Int] =Map("increment" -> increment, "decrement" ->decrement)
println(m("increment")(2)) //prints 3
println(m("decrement")(3)) //prints 2
}