This question already has answers here:
How to pass Scala array into Scala vararg method?
(3 answers)
What does `:_*` (colon underscore star) do in Scala?
(4 answers)
Closed 5 years ago.
I am encountering this : _* notation in many spark-scala answers, but couldn't find any documentation. What does it mean actually? An example of such usage is in the answer to this question
How to use DataFrame filter with isin in Spark Java?
line:
df.filter(col("something").isin(list: _*)
To understand it, lets take an example
scala> def echo(args: String*) =
for (arg <- args) println(arg)
echo: (args: String*)Unit
scala> val arr = Array("What's", "up", "doc?")
arr: Array[String] = Array(What's, up, doc?)
scala> echo(arr)
<console>:14: error: type mismatch;
found : Array[String]
required: String
echo(arr)
scala> echo(arr: _ *)
What's
up
doc?
This notation,arr:_* tells the compiler to pass each element of arr as its own argument to echo , rather than all of it as a single argument.
Related
This question already has answers here:
How do I get around type erasure on Scala? Or, why can't I get the type parameter of my collections?
(11 answers)
Closed 2 years ago.
Why does the following piece of code not work as expected? By looking at the code, I was thinking that it might return "a list of numbers" as numbers is a list with numbers. but I ran the code and got the output as "list of strings". Guess I cleared most of the confusion around here.
val numbers: List[Int] = List(1,2,3)
val numbersMatch: String = numbers match {
case listOfStrings: List[String] => "a list of strings"
case listOfNumbers: List[Int] => "a list of numbers"
case _ => ""
}
println(numbersMatch)
This code will print that it is a list of string which is a result of the way generics are compiled on most JVM languages.
They use so called "Type erasure" which drops generics. Since the generic type is not available at runtime, Scala cannot differentiate between List[String] and List[Int] and therefore the first case matches. If you compile your code, Scala will actually warn you about this:
warning: fruitless type test: a value of type List[Int] cannot also be a List[String] (the underlying of List[String]) (but still might match its erasure)
case listOfStrings: List[String] => "a list of strings"
This question already has answers here:
What does => and () => mean in Scala [duplicate]
(5 answers)
Closed 6 years ago.
While looking at an open source code I found out that sometimes some people use syntax like this:
Seq[Date => String]
Can you please explain what does this mean and how it is used? I am new to Scala.
Seq[Date => String]
Is a sequence of functions from Date (taking in a parameter of type Date) to String (returning a String). It is syntactic sugar for Function1[Date, String]:
Seq[Function1[Date, String]]
For example, one could use:
val x = List[Date => String](date => date.toString)
Which, when invoked, would print the toString method of the Date class.
It means this is a sequence of Functions from Date to String. In Scala, functions are first-class citizens, which means (among other things) that functions have types. A => B is the notation describing the type of a function that takes an argument of type A and returns a value of type B.
For example, you can write:
val f1: Date => String = d => d.toString
def f2(d: Date): String = d.toString
val s: Seq[Date => String] = Seq(f1, f2)
This question already has answers here:
Error with varargs for function-objects in Scala?
(2 answers)
Closed 7 years ago.
I am quite new to Scala (and Spark, if this is somehow Spark-specific), so please forgive the super simple question.
To me, it seems like this code should compile just fine:
sqlContext.udf.register("json_extract_string", (rawJson: String, keyPath: String*) => {
[String]UDFs.jsonExtract(rawJson, keyPath:_*)
})
Yet compiling gives the error:
Error:(31, 89) ')' expected but identifier found.
sqlContext.udf.register("json_extract_string", (rawJson: String, keyPath: String*) => {
^
Why is this?
The function being called looks like this:
object UDFs {
def jsonExtract[T: Manifest](rawJson: String, keyPath: String*): Option[T] = {
implicit val formats = DefaultFormats
val json = parse(rawJson)
keyPath.foldLeft(json)(_ \ _).extractOpt[T]
}
}
In scala it is not permitted for anonymous functions to have variable length arguments, see this answer Scala: How do I define an anonymous function with a variable argument list?
There is a shorter form of what you're trying to express which should work:
sqlContext.udf.register("json_extract_string", UDFs.jsonExtract[String]_)
This:
[String]UDFs.jsonExtract(rawJson, keyPath:_*)
is not valid Scala.
If you need to cast, you have to explicitly call asInstanceOf:
UDFs.jsonExtract(rawJson, keyPath:_*).asInstanceOf[String]
But typically such casting is a code smell and a sign that you've gone down the wrong path.
This question already has answers here:
What does param: _* mean in Scala?
(3 answers)
Closed 8 years ago.
Hi I have following code
CookieMock(response, email).cookies: _*
.cookies is type def cookies: scala.Seq[Cookie].
What does :_* mean in Scala?
Thanks
If you are familiar with Java
here is the same explanation in Java:
varargs
Because * is not a type, you add the underscore.
def printInts(ints: Int*) = ints.mkString(",")
printInts(1,2,3)
//printInts(List(1,2,3)) //type mismatch; found : List[Int] required: Int
printInts(List(1,2,3): _*)
paste this to codebrew.io this will clarify.
The : is type ascription. _* is the type you ascribe when you need a Seq[A] to be treated as A*.
http://docs.scala-lang.org/style/types.html
The following are examples of ascription:
Nil: List[String]
Set(values: _*)
"Daniel": AnyRef
Ascription is basically just an up-cast performed at compile-time for the sake of the type checker. Its use is not common, but it does happen on occasion. The most often seen case of ascription is invoking a varargs method with a single Seq parameter. This is done by ascribing the _* type (as in the second example above).
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Syntax sugar: _*
I wrote a function that gets passed a format string (for String.format(...)) and a varargs array of parameters (among other things). The method looks like this:
def myMethod(foo: Number, formatStr: String, params: Any*): Unit = {
// .. some stuff with foo
println(formatStr, params.map(_.asInstanceOf[AnyRef]) : _*)
}
I got the syntax for the params argument here. It works! But how? I do not understand the syntax of the second argument to println, particularly the ending part (: _*). It is obviously calling map and expanding the array to a sequence of AnyRefs.
Generally, the : notation is used for type ascription, forcing the compiler to see a value as some particular type. This is not quite the same as casting.
val b = 1 : Byte
val f = 1 : Float
val d = 1 : Double
In this case, you're ascribing the special varargs type _*. This mirrors the asterisk notation used for declaring a varargs parameter and can be used on a variable of any type that subclasses Seq[T]:
def myMethod(params: Any*) = ... //varargs parameter, use as an Array[Any]
val list = Seq("a", 42, 3.14) //a Seq[Any]
myMethod(list : _*)
The ending part : _* converts a collection into vararg parameters.
It looks weird, I know.