I have a partition method that creates tuple of two sets of string.
def partition(i:Int) = {
dictionary.keySet.partition(dictionary(_)(i) == true)
}
I also have a map that maps integer to the return value from the partition method.
val m = Map[Int, (Set[String], Set[String])]()
for (i <- Range(0, getMaxIndex())) {
m(i) = partition(i)
}
The issue is that I have type mismatch error, but the error message does not make sense to me.
What might be wrong?
This is the code:
import scala.collection.mutable.Map
import scala.collection.{BitSet}
case class Partition(dictionary:Map[String, BitSet]) {
def max(x:Int, y:Int) = if (x > y) x else y
def partition(i:Int) = {
dictionary.keySet.partition(dictionary(_)(i) == true)
}
def getMaxIndex() = {
val values = dictionary.values
(0 /: values) ((m, bs) => max(m, bs.last))
}
def get() = {
val m = Map[Int, (Set[String], Set[String])]()
for (i <- Range(0, getMaxIndex())) {
m(i) = partition(i)
}
m
}
}
When I compile your example, the error is clear:
<console>:64: error: type mismatch;
found : (scala.collection.Set[String], scala.collection.Set[String])
required: (scala.collection.immutable.Set[String], scala.collection.immutable.Set[String])
m(i) = partition(i)
^
Looking into the API, the keySet method of a mutable map does not guarantee that the returned set is immutable. Compare this with keySet on an immutable Map—it does indeed return an immutable set.
Therefore, you could either
use an immutable Map and a var
force the result of your partition method to return an immutable set (e.g. toSet)
define the value type of your map to be collection.Set instead of Predef.Set which is an alias for collection.immtuable.Set.
To clarify these types, it helps to specify an explicit return type for your public methods (get and partition)
Related
I'm fresh with scala and udf, now I would like to write a udf which accept 3 parameters from a dataframe columns(one of them is array), for..loop current array, parse and return a case class which will be used afterwards. here's a my code roughly:
case class NewFeatures(dd: Boolean, zz: String)
val resultUdf = udf((arrays: Option[Row], jsonData: String, placement: Int) => {
for (item <- arrays) {
val aa = item.getAs[Long]("aa")
val bb = item.getAs[Long]("bb")
breakable {
if (aa <= 0 || bb <= 0) break
}
val cc = item.getAs[Long]("cc")
val dd = cc > 0
val jsonData = item.getAs[String]("json_data")
val jsonDataObject = JSON.parseFull(jsonData).asInstanceOf[Map[String, Any]]
var zz = jsonDataObject.getOrElse("zz", "").toString
NewFeatures(dd, zz)
}
})
when I run it, it will get exception:
java.lang.UnsupportedOperationException: Schema for type Unit is not supported
how should I modify above udf
First of all, try better naming for your variables, for instance in your case, "arrays" is of type Option[Row]. Here, for (item <- arrays) {...} is basically a .map method, using map on Options, you should provide a function, that uses Row and returns a value of some type (~= signature: def map[V](f: Row => V): Option[V], what you want in your case: def map(f: Row => NewFeatures): Option[NewFeature]). While you're breaking out of this map in some circumstances, so there's no assurance for the compiler that the function inside map method would always return an instance of NewFeatures. So it is Unit (it only returns on some cases, and not all).
What you want to do could be enhanced in something similar to this:
val funcName: (Option[Row], String, Int) => Option[NewFeatures] =
(rowOpt, jsonData, placement) => rowOpt.filter(
/* your break condition */
).map { row => // if passes the filter predicate =>
// fetch data from row, create new instance
}
I am trying to use a Scala HashMap like below, and when I try to operate on the HashMap, I get type mismatches. Besides using Java HashMap, how can I write this?
import scala.collection.mutable._
object Solution {
def twoSum(nums: Array[Int], target: Int): Array[Int] = {
var lookupTable = new HashMap()
for(i <- nums.indices) {
if (lookupTable.contains(target - nums(i)))
return Array(lookupTable(i), i)
lookupTable.put(nums(i), i)
}
throw new Exception
}
}
In the absence of a type ascription, an empty HashMap is a HashMap[Nothing, Nothing], which, because mutable HashMaps are invariant means you have a collection which you can't put anything into or get anything out of.
There are thus two ways to fix this:
You can use a mutable HashMap[Int, Int]:
val lookupTable = HashMap[Int, Int]()
Alternatively, you can use an immutable HashMap, which is covariant in the value type and technically invariant in the key type, but there's a way around that:
var lookupTable = scala.collection.immutable.HashMap()
for (i <- nums.indices) {
if (lookupTable.contains(target - nums(i))) return Array(lookupTable(i), i)
else lookupTable = lookupTable ++ Seq((nums(i), i))
}
def longestPalindrome(s: String): String = {
val f = Array.ofDim[Boolean](1001,1001);
var len = s.length();
for (i<-0 to len-1; j<-i to len-1) {
f(i,j) = i match {
case j => true ;
case x if x==j-1 => if (s(i) == s(j)) true else false;
case y if y<j-1 => if (s(i) == s(j)) f(i+1,j-1) else false;
}
}
}
It complains about this line f(i,j) = i match { :
Line 5: error: too many arguments for method update: (i: Int, x: Array[Boolean])Unit
What does it means? I just assign the value of match expression to a 2-dim array, what error happens?
Your problem is that you are trying to access the elements of the array with f(i,j).
You should use f(i)(j)
scala> val f = Array.ofDim[Boolean](1001,1001);
f: Array[Array[Boolean]] = Array(Array(false, false, false, false....
scala> f(1,1)
<console>:13: error: too many arguments for method apply: (i: Int)Array[Boolean] in class Array
f(1,1)
^
scala> f(1)(1)
res1: Boolean = false
The error you are experiencing is due to the fact that you are not using the right syntax to access a 2D array.
To put it into context, the syntax to access an arbitrary nth element in the array is
array(n)
and if you think about it, a 2D array is just an array of arrays, meaning that to access the inner array you could do something like
val inner = nested(n)
inner(m)
Or in short
nested(n)(m)
Can't we use scala flatMap method on List of integers (i.e) List[Int]?
I am getting compile time error for the below code
object FlatMapExample {
def main(args:Array[String])
{
val numberList = List(1,2,3)
val mappedList = numberList.map { elem => elem*2 }
println(mappedList)
val flatMappedList = numberList.flatMap { elem => elem*2 }//compile time error
println(flatMappedList)
}
}
Compile time error:
type mismatch ; found: Int required :scala.collection.GenTraversableOnce[?]
flatMap() assumes you are returning a collection of values rather than a single element. Thus these would work:
val list = List(1,2,3)
list.flatMap(elem => List(elem * 2)) // List (2,4,6)
If you just want to multiply by two, use map.
Does Scala provide a means of accessing primitives by reference (e.g., on the heap) out of the box? E.g., is there an idiomatic way of making the following code return 1?:
import scala.collection.mutable
val m = new mutable.HashMap[String, Int]
var x = m.getOrElseUpdate("foo", 0)
x += 1
m.get("foo") // The map value should be 1 after the preceding update.
I expect I should be able to use a wrapper class like the following as the map's value type (thus storing pointers to the WrappedInts):
class WrappedInt(var theInt:Int)
...but I'm wondering if I'm missing a language or standard library feature.
You can't do that with primitives or their non-primitives counter parts in Java nor Scala. Don't see any other way but use the WrappedInt.
If your goal is to increment map values by key, than you can use some nicer solutions instead of wrapper.
val key = "foo"
val v = m.put(key, m.getOrElse(key, 0) + 1)
or another approach would be to set a default value 0 for the map:
val m2 = m.withDefault(_ => 0)
val v = m2.put(key, m2(key) + 1)
or add extension method updatedWith
implicit class MapExtensions[K, V](val map: Map[K, V]) extends AnyVal {
def updatedWith(key: K, default: V)(f: V => V) = {
map.put(key, f(map.getOrElse(key, default)))
}
}
val m3 = m.updatedWith("foo", 0) { _ + 1 }