I have:
val words = List("all", "the", "words", "all", "the", "counts", "all", "day")
val map = Exercise02.count(words.iterator)
val expected = Map("all" -> 3, "the" -> 2, "words" -> 1, "counts" -> 1, "day" -> 1)
where Exercise02.count is java.util.Iterator[String] => Map[String, Int] and merely produces a count of each word in the input java.util.Iterator[String].
I wrote a test:
object Exercise02Spec extends FlatSpec with Inspectors {
val words = List("all", "the", "words", "all", "the", "counts", "all", "day")
val map = Exercise02.count(words.iterator)
val expected = Map("all" -> 3, "the" -> 2, "words" -> 1, "counts" -> 1, "day" -> 1)
"count" should "count the occurrence of each word" in {
forAll (map) { kv => assert(kv._2 === expected(kv._1)) }
// forAll (map) { (k: String, v: Int) => assert(v === expected(k)) }
}
}
The first line compiles just fine, and the test passes. If I replace the first line with the commented out second line, I get a compilation error.
sbt reports: found : (String, Int) => Unit, required: ((String, Int)) => Unit
IntelliJ IDEA reports: Type mismatch, expected: ((String, Int)) => Unit, actual: (String, Int) => Unit
Why is this? And how do I fix it?
The method you're using accepts a function that transforms a single argument to a single output. What you're telling Scala in your second statement is that map should accept a function with two arguments! There's an important difference between a single argument that just happens to be a tuple of size two, and two distinct arguments.
So you have single argument, but you (and Scala) know it's a tuple. To access those two elements, you have to destructure or pattern match your argument against the tuple you want. You just do that by writing forAll (map) { case (k: String, v: Int) => assert(v === expected(k)) }. You're saying the argument you receive should match the pattern of a tuple (String, Int) and you want the first element bound to k and the second to v. You could probably eliminate mentioning the types explicitly here. Note that this is similar to the syntax used in pattern matching, which is essentially what you're doing here.
In the 2nd example you're defining a two argument function, not a single tuple argument function. Hence why the error is saying it is expecting the double parenthesis around the argument.
Related
I'm having an error
error: Cannot prove that (Int, String, String, String, String, Double, String) <:< (T, U).
}.collect.toMap
when executing my application having the following code snippet.
val trains = sparkEnvironment.sc.textFile(dataDirectoryPath + "/trains.csv").map { line =>
val fields = line.split(",")
// format: (trainID,trainName,departure,arrival,cost,trainClass)
(fields(0).toInt, fields(1),fields(2),fields(3),fields(4).toDouble,fields(5))
}.collect.toMap
What could be the cause and can anyone please suggest a solution ?
if you want to do toMap on Seq, you show have a Seq of Tuple2. ScalaDoc of toMap states :
This method is unavailable unless the elements are members of Tuple2,
each ((T, U)) becoming a key-value pair in the map
So you should do:
val trains = sparkEnvironment.sc.textFile(dataDirectoryPath + "/trains.csv").map { line =>
val fields = line.split(",")
// format: (trainID,trainName,departure,arrival,cost,trainClass)
(fields(0).toInt, // first element of Tuple2 -> "key"
(fields(1),fields(2),fields(3),fields(4).toDouble,fields(5)) // 2nd element of Tuple2 -> "value"
)
}.collect.toMap
such that your map-statwment returns RDD[(Int, (String, String, String, String, Double, String))]
I am trying to create a map of lambda functions in Scala
val identity = ((x:Any) => x)
val propmap = Map("references-count" -> identity,
"title" -> ((x:List[String]) => x(0)),
"score" -> identity,
"issued" -> ((x:List[Any]) => x(0)))
when I type propmap("score") or propmap("title") the output I get is the same: <function1>.
Running identity(10.1) returns the expected result. However
val f1 = propmap("score")
f1(10.9)
results in:
Name: Unknown Error
Message: <console>:29: error: type mismatch;
found : Double(10.9)
required: List[String]
f1(10.9)
^
StackTrace:
Seemingly the function is overwritten. Where to go for the besung immutabilty?
The problem is the type of propmap.
Scala infers the type as String, (List[String] => Any)
The reason for that is that scala needs to infer a type which matches ALL values. The String as key is obvious but for the function, it needs to find a function type which matches all functions. Since all functions have 1 parameter this would be function1.
If you look at the definition of function1 you would see it is:
trait Function1[-T1, +R] extends AnyRef
This means that it needs to find the first type to be the most constrictive. In this case this is List[String] and therefore it expects a function List[String]=>Any (this actually makes sense because you want a type you can use on ALL functions).
So scala automatically converts your identity function to be (x: List[String] => x) in practice and therefore when you try to pass it a number it fails.
Possible solutions:
The first solution as mentioned by #KotWarm would be to use asInstanceOf:
val f1 = propmap("score").asInstanceOf[Any ⇒ Any]
println(f1(10.9))
The second solution would be to rewrite the functions to use Any, for example:
val propmap = Map("references-count" -> identity,
"title" -> ((x: Any) => x.asInstanceOf[List[String]](0)),
"score" -> identity,
"issued" -> ((x: Any) => x.asInstanceOf[List[Any]](0)))
Because scalac determined the type of collection as
propmap: scala.collection.immutable.Map [String, List [String] => Any]
Determine the type of your collection explicitly so that the compiler knows what you want to get
Here is an example code
val identity = ((x:Any) => x)
val propmap = Map[String,_ => _]("references-count" -> identity,
"title" -> ((x:List[String]) => x(0)),
"score" -> identity,
"issued" -> ((x:List[Any]) => x(0)))
But to execute the methods you must cast the type
val f1 = propmap("score").asInstanceOf[Any ⇒ Any]
println(f1(10.9))
This drives me crazy, I can't figure out why this gives me an error.
Here an example of my code:
var seqOfObjects:Seq[Map[String, String]] = Seq[Map[String, String]]()
for(item <- somelist) {
seqOfObjects += Map(
"objectid" -> item(0).toString,
"category" -> item(1),
"name" -> item(2),
"url" -> item(3),
"owneremail" -> item(4),
"number" -> item(5).toString)
}
This gives me an error saying:
Type mismatch, expected: String, actual: Map[String, String]
But a Map[String, String] is exactly what I want to append into my Seq[Map[String, String]].
Why is it saying that my variable seqOfObjects expects a String??
Anyone have a clue?
Thanks
a += b means a = a.+(b). See this answer.
There is no method + in Seq, so you can't use +=.
scala> Seq[Int]() + 1
<console>:8: error: type mismatch;
found : Int(1)
required: String
Seq[Int]() + 1
^
required: String is from string concatenation. This behavior is inherited from Java:
scala> List(1, 2, 3) + "str"
res0: String = List(1, 2, 3)str
Actually method + here is from StringAdd wrapper. See implicit method Predef.any2stringadd.
You could use :+= or +:= instead of +=.
Default implementation of Seq is List, so you should use +: and +:= instead of :+ and :+=. See Performance Characteristics of scala collections.
You could also use List instead of Seq. There is :: method in List, so you can use ::=:
var listOfInts = List[Int]()
listOfInts ::= 1
You can rewrite your code without mutable variables using map:
val seqOfObjects =
for(item <- somelist) // somelist.reverse to reverse order
yield Map(...)
To reverse elements order you could use reverse method.
Short foldLeft example:
sl.foldLeft(Seq[Map[Srting, String]]()){ (acc, item) => Map(/* map from item */) +: acc }
This is cross-posted from the coursera functional programming course because there's a lot less activity on that forum.
I wrote the following code (parts are redacted because it's homework):
type Occurrences = List[(Char, Int)]
def subtract(x: Occurrences, y: Occurrences): Occurrences = {
val mx: Map[Char, Int] = x toMap
y.foldLeft (redacted) (redacted => simple expression using updated and -)) toList
}
This produces the following compile error:
type mismatch; found : Map[Char,Int] required: <:<[(Char, Int), (?, ?)]
However if I add a copy of the third line, without the toList, in between via a val statement, the error goes away:
type Occurrences = List[(Char, Int)]
def subtract(x: Occurrences, y: Occurrences): Occurrences = {
val mx: Map[Char, Int] = x toMap
val foo: Map[Char, Int] = y.foldLeft (redacted) (redacted => simple expression using updated and -))
y.foldLeft (redacted) (redacted => simple expression using updated and -)) toList
}
I'm guessing this has something to do with giving some kind of extra hint to the type checker, but does anyone know specifically why this happens?
Below follows a few examples and some explanations on why it happens.
First, a working and a non-working cases:
scala> { List('a -> 1, 'b -> 2).toMap
| println("aaa") }
aaa
scala> { List('a -> 1, 'b -> 2) toMap
| println("aaa") }
<console>:9: error: type mismatch;
found : Unit
required: <:<[(Symbol, Int),(?, ?)]
println("aaa") }
^
This happens because the syntax "obj method arg" is considered to be "obj.method(arg)" and so is "obj method \n arg", this way the argument can be written in the next line. Notice below:
scala> { val x = List('a -> 1, 'b -> 2) map
| identity
|
| println(x) }
List(('a,1), ('b,2))
It's the same as List('a -> 1, 'b -> 2).map(identity).
Now for the weird error message found : Unit, required: <:<[(Symbol, Int),(?, ?)]. It happens that toMap actually takes one argument, here is it's signature:
def toMap[T, U](implicit ev: <:<[A,(T, U)]): Map[T,U],
but it's an implicit argument, so doesn't need to be provided explicitly in this case. But when you use the obj method \n arg syntax it fills the method argument. In the above non-working example the argument is println which has type Unit, hence it is not accepted by the compiler.
One workaround is to have two \n to separate the lines:
scala> { List('a -> 1, 'b -> 2) toMap
|
| println("aaa") }
aaa
You can also use a ; to separate the lines.
#RexKerr & #DidierDupont are right, you're having issues because you called toMap like a binary operator, so the compiler freaked out.
My two cents: you should probably read the Suffix Notation section of the Scala Style Guide.
I'm not sure why this doesn't work:
scala> case class Loader(n: String, x: String, l: List[String])
scala> val m: Map[String, (List[String])=>Loader] =
| Map("x" -> Loader("x", "x1", _:List[String]))
<console>:8: error: type mismatch;
found : (List[String]) => (java.lang.String, Loader)
required: (String, (List[String]) => Loader)
Map("x" -> Loader("x", "x1", _:List[String]))
but this does?
scala> Loader("t", "x", _:List[String])
res7: (List[String]) => Loader = function1>
scala> val m = Map("x" -> res7)
m: scala.collection.immutable.Map[java.lang.String,(List[String]) => Loader] =
Map((String,function1>))
One more victim of the overload of _ in Scala. Consider this:
f(_, 5) + 1 // Partial function application
f(_ + 1, 5) // Closure
In the first case, _ is replacing the entire parameter. In this case, it stands for a partial application of f. In practice, it's equivalent to x => f(x, 5) + 1, as the whole expression that contains f is turned into a closure.
In the second case, _ is part of an expression. In this case, the whole expression is turned into a closure, up to any expression delimiter -- by which I mean that if the expression is nested inside another, only the inner expression is turned into a closure. In practice, it is equivalent to f(x => x + 1, 5).
The parser was not sure where to put the beginning of the anonymous function. Sometimes you can solve this by adding another pair of parentheses (though not always):
val m: Map[String, (List[String])=>Loader] =
Map("x" -> (Loader("x", "x1", _:List[String])))
I don’t see any ambiguities here, so it might just not have been smart enough to figure it out. I think, the parser overlooked the possibility to have an anonymous function just after the -> (which also is a library construct and uses implicit magic and all the wicket stuff which makes the little parser’s mind loop).
When you write it as an explicit tuple, it’ll work fine.
val m: Map[String, (List[String])=>Loader] =
Map(("x", Loader("x", "x1", _:List[String])))