While creating a map of String to partial functions I ran into unexpected behavior. When I create a partial function as a map element it works fine. When I allocate to a val it invokes instead. Trying to invoke the check generates an error. Is this expected? Am I doing something dumb? Comment out the check() to see the invocation. I am using scala 2.7.7
def PartialFunctionProblem() = {
def dream()() = {
println("~Dream~");
new Exception().printStackTrace()
}
val map = scala.collection.mutable.HashMap[String,()=>Unit]()
map("dream") = dream() // partial function
map("dream")() // invokes as expected
val check = dream() // unexpected invocation
check() // error: check of type Unit does not take parameters
}
For convenience, Scala lets you omit empty parens when calling a method, but it's clever enough to see that the expected type in the first case is ()=>Unit, so it doesn't remove all the parens for you; instead, it converts the method into a function for you.
In the val check case, however, it looks just like a function call result getting assigned to a variable. In fact, all three of these do the exact same thing:
val check = dream
val check = dream()
val check = dream()()
If you want to turn the method into a function, you place _ after the method in place of the argument list(s). Thus,
val check = dream() _
will do what you want.
Well, the problem is that you got it all wrong. :-)
Here are some conceptual mistakes:
def dream()() = {
println("~Dream~");
new Exception().printStackTrace()
}
This is not a partial function. This is a curried method with two empty parameter lists which returns Unit.
val map = scala.collection.mutable.HashMap[String,()=>Unit]()
The type of the values in this map is not partial function, but function. Specifically, Function0[Unit]. A partial function would have type PartialFunction[T, R].
map("dream") = dream() // partial function
What happens here is that Scala converts the partially applied method into a function. This is not a simple assignment. Scala does the conversion because the type inferencer can guess the correct type.
val check = dream() // unexpected invocation
Here there's no expected type to help the type inferencer. However, empty parameter lists can be ommitted, so this is just a method call.
Related
I have created an UDF in Scala and when I was trying to register this UDF with just function name it was showing me error.
Not Working
def IPConvertUDF = spark.udf.register("IPConvertUDF", IPConvert)
Error
error: missing argument list for method IPConvert
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing `IPConvert _` or `IPConvert(_)` instead of `IPConvert`.
def IPConvertUDF = spark.udf.register("IPConvertUDF", IPConvert)
And so I added extra _ after method name and it worked.
Works perfectly
def IPConvertUDF = spark.udf.register("IPConvertUDF", IPConvert _)
Would someone be able to explain be what is the meaning of extra _ after the method name?
Well the short answer is, you are trying to pass a method where a function is expected as an argument. Methods are not functions.
Let's dig a bit deeper.
Lets try with simple add function first
val add:(Int,Int) => Int = (val1,val2) => val1+val2
spark.udf.register("add",add)
The above code compiled without any error. The reason is add is a function.
Now lets try the same add as a method
def add(val1:Int,val2:Int): Int ={
val1+val2
}
spark.udf.register("add",add)
Now you get an error saying error: missing argument list for method add.
As I mentioned, register(..) is expecting function and methods cannot be passed as arguments.
_ is a shorthand for partially applied function.In other words , add method is converted into partially applied function and that's the reason the error has disappeared.
spark.udf.register("add",add _)
I'd like to call a function returned by a function with an implicit parameter, simply and elegantly. This doesn't work:
def resolveA(implicit a: A): String => String = { prefix =>
s"$prefix a=$a"
}
case class A(n: Int)
implicit val a = A(1)
println(resolveA("-->")) // won't compile
I've figured out what's going on: Scala sees the ("-->") and thinks it's an attempt to explicitly fill in the implicit parameter list. I want to pass that as the prefix argument, but Scala sees it as the a argument.
I've tried some alternatives, like putting an empty parameter list () before the implicit one, but so far I've always been stopped by the fact that Scala thinks the argument to the returned function is an attempt to fill in the implicit parameter list of resolveA.
What's a nice way to do what I'm trying to do here, even if it's not as nice as the syntax I tried above?
Another option would be to use the apply method of the String => String function returned by resolveA. This way the compiler won't confuse the parameter lists, and is a little shorter than writing implicltly[A].
scala> resolveA[A].apply("-->")
res3: String = --> a=A(1)
I was looking at the FoldLeft and FoldRight methods and the operator version of the method was extremely peculiar which was something like this (0 /: List.range(1,10))(+).
For right associative functions with two parameter lists one would expect the syntax to be something like this((param1)(param2) op HostClass).
But here in this case it is of the syntax (param1 op HostClass)(param2). This causes ambiguity with another case where a right associative function returns another function that takes a single parameter.
Because of this ambiguity the class compiles but fails when the function call is made as shown below.
class Test() {
val func1:(String => String) = { (in) => in * 2 }
def `test:`(x:String) = { println(x); func1 }
def `test:`(x:String)(y:String) = { x+" "+y }
}
val test = new Test
(("Foo") `test:` test)("hello")
<console>:10: error: ambiguous reference to overloaded definition,
both method test: in class Test of type (x: String)(y: String)String
and method test: in class Test of type (x: String)String => String
match argument types (String)
(("Foo") `test:` test)("hello")
so my questions are
Is this an expected behaviour or is it a bug?
Why the two parameter list right associative function call has been designed the way it is, instead of what I think to be more intuitive syntax of ((param1)(param2) op HostClass)?
Is there a workaround to call either of the overloaded test: function without ambiguity.
The Scala's Type System considers only the first parameter list of the function for type inference. Hence to uniquely identify one of the overloaded method in a class or object the first parameter list of the method has to be distinct for each of the overloaded definition. This can be demonstrated by the following example.
object Test {
def test(x:String)(y:Int) = { x+" "+y.toString() }
def test(x:String)(y:String) = { x+" "+y }
}
Test.test("Hello")(1)
<console>:9: error: ambiguous reference to overloaded definition,
both method test in object Test of type (x: String)(y: String)String
and method test in object Test of type (x: String)(y: Int)String
match argument types (String)
Test.test("Hello")(1)
Does it really fail at runtime? When I tested it, the class compiles, but the call of the method test: does not.
I think that the problem is not with the operator syntax, but with the fact that you have two overloaded functions, one with just one and the other with two parameter lists.
You will get the same error with the dot-notation:
test.`test:`("Foo")("hello")
If you rename the one-param list function, the ambiguity will be gone and
(("Foo") `test:` test)("hello")
will compile.
I have defined two partial functions (hashes), which I expect to take an optional second Boolean parameter:
def SHA1 = hash(MessageDigest.getInstance("SHA-1"))_
def MD5 = hash(MessageDigest.getInstance("MD5"))_
private def hash(algorithm:HashAlgorithm)(s:String, urlencode:Boolean = false) = {
val form = if (urlencode) "%%%02X" else "%02X"
(algorithm.digest(s.getBytes) map(form format _)).mkString
}
When I call the function with both parameters, it compiles, but with just one parameter I get a compilation error:
// First 3 tests are fine
val test1 = hash(MessageDigest.getInstance("SHA-1"))("foo", true)
val test2 = hash(MessageDigest.getInstance("SHA-1"))("foo")
val test3 = SHA1("foo", true)
// not enough arguments for method apply: (v1: String, v2: Boolean)String in trait Function2. Unspecified value parameter v2.
val test4 = SHA1("foo")
I just refactored this to use partial functions, and before I refactored I could force the hash function to use the default without any problem.
Any ideas why the partial function implementation fails to permit default arguments? Am I doing something wrong using both partial functions and currying together?
When you use partial application to generate a function, you lose the ability to call the default. A method is a static thing, so the compiler knows where to look up the default value; a function can be passed around into different contexts, so the compiler will not in general have the information it needs to be able to apply the default parameter.
To think about it another way, functions only know how many arguments they have. There's just one method, apply, that you pass parameters into; otherwise you'd need some way (different types, presumably) to distinguish, for example, Function2-that-must-take-two-parameters and Function2-that-can-be-called-with-one-parameter-also-because-there-is-a-stored-default.
I have a class which takes a function
case class FunctionParser1Arg[T, U](func:(T => U))
def testFunc(name1:String):String = name1
val res = FunctionParser1Arg(testFunc)
I would like to know the type signature information on the function from inside the case class. I want to know both the parameter name and type. I have had success in finding the type using the runtime mirror objects, but not the name. Any suggestions?
Ok, let's say you got the symbol for the instance func points to:
import scala.reflect.runtime.universe._
import scala.reflect.runtime.{currentMirror => m}
val im = m reflect res.func // Instance Mirror
You can get the apply method from its type members:
val apply = newTermName("apply")
val applySymbol = im.symbol.typeSignature member apply
And since we know it's a method, make it a method symbol:
val applyMethod = applySymbol.asMethod
It's parameters can be found through paramss, and we know there's only one parameter on one parameter list, so we can get the first parameter of the first parameter list:
val param = applyMethod.paramss(0)(0)
Then what you are asking for is:
val name = param.name.decoded // if you want "+" instead of "$plus", for example
val type = param.typeSignature
It's possible that you think that's the wrong answer because you got x$1 instead of name1, but what is passed to the constructor is not the named function testFunc, but, instead, an anonymous function representing that method created through a process called eta expansion. You can't find out the parameter name of the method because you can't pass the method.
If that's what you need, I suggest you use a macro instead. With a macro, you'll be able to see exactly what is being passed at compile time and get the name from it.