I would like to use functions in for-comprehension with input/return values of type JValue, that is defined within json4s. The following snippet:
def func1(...):org.json4s.JValue
def func2(val1:org.json4s.JValue):...
for {
aJValue <- func1(...)
whatever <- func2(aJValue)
} yield (wathever)
yields the compiler error
Cannot resolve symbol flatMap
at the first "<-" symbol. How to fix this?
Many Thanks!
Related
SBT is throwing the following error:
value split is not a member of (String, String)
[error] .filter(arg => arg.split(delimiter).length >= 2)
For the following code block:
implicit def argsToMap(args: Array[String]): Map[String, String] = {
val delimiter = "="
args
.filter(arg => arg.split(delimiter).length >= 2)
.map(arg => arg.split(delimiter)(0) -> arg.split(delimiter)(1))
.toMap
}
Can anyone explain what might be going on here?
Some details:
java version "1.8.0_191"
sbt version 1.2.7
scala version 2.11.8
I've tried both on the command line and also with intellij. I've also tried Java 11 and Scala 2.11.12 to no avail.
I'm not able to replicate this on another machine (different OS, SBT, IntelliJ, etc. though) and I can also write a minimal failing case:
value split is not a member of (String, String)
[error] Array("a", "b").map(x => x.split("y"))
The issue is that the filter method is added to arrays via an implicit.
When you call args.filter(...), args is converted to ArrayOps via the Predef.refArrayOps implicit method.
You are defining a implicit conversion from Array[String] to Map[(String, String)].
This implicit has higher priority than Predef.refArrayOps and is therefore used instead.
So args is converted into a Map[(String, String)]. The filter method of that Map would expect a function of type (String, String) => Boolean as parameter.
I believe what happened is that the implicit method is getting invoked a bit too eagerly. That is, the Tuple2 that's seemingly coming out of nowhere is the result of the implicit function converting each String into a key/value pair. The implicit function was recursively calling itself. I found this out after eventually getting a stack overflow with some other code that was manipulating a collection of Strings.
The following code snippet
import util.control.TailCalls._
for {(num, ch) <- done((3, '3'))
} yield num
fails to compile with error message:
value withFilter is not a member of util.control.TailCalls.TailRec[(Int, Char)]
I am using Scala 2.12.7. How to avoid this error? (IntelliJ Idea 18.3.1 with Scala plugin v2018.3.4 does not show error.)
To avoid the call to withFilter and keep the current syntax, it helps if a compiler plugin is used to treat for comprehensions differently. An option is using better-monadic-for.
Adding this to the build.sbt file should make the code in the question compile:
addCompilerPlugin("com.olegpy" %% "better-monadic-for" % "0.3.0-M4")
(Though it has other -usually positive- effects too, please check its documentation.)
Another option is implementing withFilter with an extension method, for example like this (and having it in scope at usage site):
implicit class TailCallsExtension[A](t: TailRec[A]) {
def withFilter(pred: A => Boolean): TailRec[A] = t.flatMap(a => if (pred(a)) t else done(a))
}
Seemingly there is no filtering in the code, but actually the pattern match in Scala for comprehensions (before <-) are translated as a call to withFilter. TailCalls does not support withFilter, so this will not compile. The following rewrite compiles though:
import util.control.TailCalls._
done((3, '3')).map{ case (num, ch) => num}
json4 has the following types:
sealed abstract class JValue
case class JString(s: String) extends JValue
//etc
I have the following json value:
val json: JValue = JString("hi")
and I use it in a for-comprehension as such:
val token = for {
JString(s) <- json
} yield s
Here is the question:
As it is, the token will be evaluated as List("hi") namely an instance of the type List[String]. My understanding was that it should instead be Option[String]. why Option -> List?
IntelliJ's type-"inference" helper, suggests setting the type JValue for the result of the for-comprehension. When you that however, you get a compile error. What's exactly at fault here? and why is the confusion happening?
Why List[String]?
Well, as you can see here, that's what the json4s authors chose. They could have gone with Vector[String], or Array[String], but decided upon List[String]. I assume they had good reasons.
What's withFilter() got to do with it?
As we all learned on the first day of Scala class, for is not a control-flow language construct, but is actually syntactic sugar used to prettify nested map/flatMap constructs.
for {
b <- a // flatMap()
if b.isSomeCondition // withFilter()
c <- b // map()
} yield c
It turns out that withFilter() is also employed when pattern matching in a for construct. So this...
for {
JString(s) <- json
} yield s
...gets translated into this (roughly).
json.withFilter{
case JString((s # _)) => true
case _ => false
}.map{
case JString(s # _) => s
}
The authors of json4s decided that withFilter() should return a List and thus that's the result of your for comprehension.
IntelliJ's confusion.
IntelliJ's syntax checker and suggestion engine is pretty good, but it's not perfect. Scala code can get rather complicated and it's not uncommon for IntelliJ to get confused. Trust the compiler. If it compiles without warning then just ignore the faulty type suggestions.
I am scratching my head on an example I've seen in breeze's documentation about distributions.
After creating a Rand instance, they show that you can do the following:
import breeze.stats.distributions._
val pois = new Poisson(3.0);
val doublePoi: Rand[Double] = for(x <- pois) yield x.toDouble
Now, this is very cool, I can get a Rand object that I can get Double instead of Int when I call the samples method. Another example might be:
val abc = ('a' to 'z').map(_.toString).toArray
val letterDist: Rand[String] = for(x <- pois) yield {
val i = if (x > 26) x % 26 else x
abc(i)
}
val lettersSamp = letterDist.samples.take(20)
println(letterSamp)
The question is, what is going on here? Rand[T] is not a collection, and all the for/yield examples I've seen so far work on collections. The scala docs don't mention much, the only thing I found is translating for-comprehensions in here.
What is the underlying rule here? How else can this be used (doesn't have to be a breeze related answer)
Scala has rules for translating for and for-yield expressions to the equivalent flatMap and map calls, optionally also applying filters using withFilter and such. The actual specification for how you translate each for comprehension expression into the equivalent method calls can be found in this section of the Scala Specification.
If we take your example and compile it we'll see the underlying transformation happen to the for-yield expression. This is done using scalac -Xprint:typer command to print out the type trees:
val letterDist: breeze.stats.distributions.Rand[String] =
pois.map[String](((x: Int) => {
val i: Int = if (x.>(26))
x.%(26)
else
x;
abc.apply(i)
}));
Here you can see that for-yield turns into a single map passing in an Int and applying the if-else inside the expression. This works because Rand[T] has a map method defined:
def map[E](f: T => E): Rand[E] = MappedRand(outer, f)
For comprehensions are just syntactic sugar for flatMap, map and withFilter. The main requirement for use in a for comprehension is that those methods are implemented. They are therefore not limited to collections E.g. some common non-collections used in for comprehensions are Option, Try and Future.
In your case, Poisson seems to inherit from a trait called Rand
https://github.com/scalanlp/breeze/blob/master/math/src/main/scala/breeze/stats/distributions/Rand.scala
This trait has map, flatmap, and withFilter defined.
Tip: If you use an IDE like IntelliJ - you can press alt+enter on your for comprehension, and chose convert to desugared expression and you will see how it expands.
In the following code :
val expression : String = "1+1";
for (expressionChar <- expression) {
println(expressionChar)
}
the output is "1+1"
How is each character being accessed here ? What is going on behind the scenes in Scala. Using Java I would need to use the .charAt method , but not in Scala, why ?
In scala, a for comprehension:
for (p <- e) dosomething
would be translated to
e.foreach { x => dosomething }
You can look into Scala Reference 6.19 for more details.
However, String in scala is just a Java String, which doesnot have a method foreach.
But there is an implicit conversion defined in Predef which convert String to StringOps which indeed have a foreach method.
So, finally the code for(x <- "abcdef") println(x) be translated to:
Predef.augmentString("abcdef").foreach(x => println(x))
You can look int the scala reference or Scala Views for more information.
scala> for (c <- "1+1") println(c)
1
+
1
Is the same as
scala> "1+1" foreach (c => println(c))
1
+
1
Scalas for-comprehension is not a loop but a composition of higher order functions that operate on the input. The rules on how the for-comprehension is translated by the compiler can be read here.
Furthermore, in Scala Strings are collections. Internally they are just plain old Java Strings for which there exists some implicit conversions to give them the full power of the Scala collection library. But for users they can be seen as collections.
In your example foreach iterates over each element of the string and executes a closure on them (which is the body of the for-comprehension or the argument in the foreach call).
Even more information about for-comprehensions can be found in section 7 of the StackOverflow Scala Tutorial.