This question already has answers here:
Error with varargs for function-objects in Scala?
(2 answers)
Closed 7 years ago.
I am quite new to Scala (and Spark, if this is somehow Spark-specific), so please forgive the super simple question.
To me, it seems like this code should compile just fine:
sqlContext.udf.register("json_extract_string", (rawJson: String, keyPath: String*) => {
[String]UDFs.jsonExtract(rawJson, keyPath:_*)
})
Yet compiling gives the error:
Error:(31, 89) ')' expected but identifier found.
sqlContext.udf.register("json_extract_string", (rawJson: String, keyPath: String*) => {
^
Why is this?
The function being called looks like this:
object UDFs {
def jsonExtract[T: Manifest](rawJson: String, keyPath: String*): Option[T] = {
implicit val formats = DefaultFormats
val json = parse(rawJson)
keyPath.foldLeft(json)(_ \ _).extractOpt[T]
}
}
In scala it is not permitted for anonymous functions to have variable length arguments, see this answer Scala: How do I define an anonymous function with a variable argument list?
There is a shorter form of what you're trying to express which should work:
sqlContext.udf.register("json_extract_string", UDFs.jsonExtract[String]_)
This:
[String]UDFs.jsonExtract(rawJson, keyPath:_*)
is not valid Scala.
If you need to cast, you have to explicitly call asInstanceOf:
UDFs.jsonExtract(rawJson, keyPath:_*).asInstanceOf[String]
But typically such casting is a code smell and a sign that you've gone down the wrong path.
Related
This question already has answers here:
What is the eta expansion in Scala?
(2 answers)
Closed 4 years ago.
New to Scala, have searched far and wide for clarification on some ScalaMock syntax. As per this guide, I keep seeing the following general testing pattern:
(myClass.myMethod _).expects()
What exactly is happening here? What function does the class/method/space/underscore serve? How does the compiler treat this?
The appended _ forces the conversion of a method into a function.
To understand why this is necessary, let's try to re-build a tiny piece of Scalamock, namely the expects method. The expects method seems to be invoked on methods of mocked objects. But methods / functions do not have an expects method to begin with. Therefore, we have to use the "pimp my library"-pattern to attach the method expects to functions. We could do something like this:
implicit class ExpectsOp[A, B](f: A => B) {
def expects(a: A): Unit = println("it compiles, ship it...")
}
Now let's define a class Bar with method baz:
class Bar {
def baz(i: Int): Int = i * i
}
and also an instance of Bar:
val bar = new Bar
Let's see what happens if you try to invoke expects on bar.baz:
(bar.baz).expects(42)
error: missing argument list for method baz in class Bar
Unapplied methods are only converted to functions when a function type is expected. You can make this conversion explicit by writing baz _ or baz(_) instead of baz.
So, it doesn't work without explicit conversion into a function, and we have to enforce this conversion by appending an _:
(bar.baz _).expects(42) // prints: "it compiles, ship it..."
This question already has answers here:
In Scala, is there an easy way to convert a case class into a tuple?
(5 answers)
Closed 5 years ago.
I have a case class with two fields and a function which accepts the same arguments. Is where a way to pass contents of case class to this function without explicitly specifying all fields or ugly unapplying?
case class User(name: String, age: Int)
val user = User("Peter", 23)
def func(name: String, age: Int) = {...}
func(user.name, user.age) //works
Function.tupled(func(_,_))(User.unapply(user).get) //works, but ugly
// func(_,_) is explicit because it can be a method of a class
The simplest vanilla Scala solution is the one Tom suggested in the comments:
(func _).tupled(User.unapply(user).get)
If happen to have Shapeless in scope (maybe it's already a dependency), you could also use it to convert user to a tuple:
import shapeless._
val gen = Generic[User]
(func _).tupled(gen.to(user).tupled)
This question already has answers here:
Scala foreach strange behaviour
(5 answers)
Closed 7 years ago.
Given these case classes:
case class FeatureDistance(id: Long, distance: Double)
case class SearchResult(score: Float, id: Long)
Why does this not compile?
val distances = List[FeatureDistance](FeatureDistance(1L, 10f))
val results = distances.map(SearchResult(0f, _.id))
But this does:
val results = distances.map(fd => SearchResult(0f, fd.id))
The compilation error says: missing parameter type for expanded function ((x$3) => x$3.id)
Is it because _ is only scoped to the map function so it's not visible in the SearchResult.apply call?
After doing a bit of research, I found a post on the old scala forums that contains this quote:
When you use "_" as a place holder for an anonymous parameter of a function, the scope of that function is the innermost parenthesis containing it.
So, it's just a question of scope. I suspect this has to do with problems that could otherwise result from having nested function calls that use more than one underscore. For instance:
//suppose we have some x:List[List[Int]]
x.map(_.map(_ + 1))
This question already has answers here:
How can I chain implicits in Scala?
(3 answers)
Closed 8 years ago.
A simple example:
class A
class B
class C
object testobject {
val a = new A
implicit def b(a:A):B = new B
implicit def c(b:B) = new C
val b:B = a
val c:C = a
}
The last line doesn't compile. We have A=>B and B=>C implicit conversions defined but that doesn't infer A=>C.
It would be really nice to be able to have layers of implicit conversions work.
My particular problem. too long to post fully is actually from a web framework. I want to do something like:
A => Secure[A] => Format[A]
with the following
implicit def secure[A](a:A):Secure[A] = ???
implicit def format[A](sec:Secure[A]):Format[A] = ???
So I want to handle security and formatting through implicit magic, and only secured outputs can be formatted.
Has anybody found any tricks to make this, or something like this work?
This question is answered quite well here:
How can I chain implicits in Scala?
In short Scala does not chain implicit conversions, it only looks for direct conversion A => B. However, using additional implicit parameters the desired effect can be achieved.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Syntax sugar: _*
I wrote a function that gets passed a format string (for String.format(...)) and a varargs array of parameters (among other things). The method looks like this:
def myMethod(foo: Number, formatStr: String, params: Any*): Unit = {
// .. some stuff with foo
println(formatStr, params.map(_.asInstanceOf[AnyRef]) : _*)
}
I got the syntax for the params argument here. It works! But how? I do not understand the syntax of the second argument to println, particularly the ending part (: _*). It is obviously calling map and expanding the array to a sequence of AnyRefs.
Generally, the : notation is used for type ascription, forcing the compiler to see a value as some particular type. This is not quite the same as casting.
val b = 1 : Byte
val f = 1 : Float
val d = 1 : Double
In this case, you're ascribing the special varargs type _*. This mirrors the asterisk notation used for declaring a varargs parameter and can be used on a variable of any type that subclasses Seq[T]:
def myMethod(params: Any*) = ... //varargs parameter, use as an Array[Any]
val list = Seq("a", 42, 3.14) //a Seq[Any]
myMethod(list : _*)
The ending part : _* converts a collection into vararg parameters.
It looks weird, I know.