Consider the following function:
import java.util.concurrent.Callable;
def callable[T]( operation: =>T) : Callable[T] = {
new Callable[T] {
def call : T = operation
}
}
In the REPL, this code does what I want:
scala> val myCallable = callable {
| println("Side effect!");
| "Hi!"
| }
myCallable: java.util.concurrent.Callable[String] = $anon$1#11ba4552
scala> myCallable.call
Side effect!
res3: String = Hi!
scala> myCallable.call
Side effect!
res4: String = Hi!
The by-name parameter is not evaluated until the function 'call' is called, and is re-evaluated every time that function is called. That is the behavior I want.
But in the spec, it says the following about by-name parameters:
"the corresponding argument is not evaluated at the point of function application, but instead is evaluated at each use within the function."
From this description, it is unclear that I can rely on the behavior I want. What does a "use within the function mean"? How do I know that this refers to the point at which my Callable is called (sometime in the indefinite future), rather than the point at which it's defined (very much "within the function")?
The code is doing what I want. But I'd rest easier if I was sure this behavior is reliable, not a bug that might be "fixed" in some future version of scala.
This is not a bug - that behaviour is as intended. You can think of "evaluated at each use within the function" recursively as "evaluated at each use within an expression in the function when that expression is evaluated".
"The function" is the function you're passing your parameter to. What this passage tries to warn you about is this:
scala> def byName(arg: => String) = arg + arg
byName: (arg: => String)java.lang.String
scala> byName({println("hi") ; "foo" })
hi
hi
res0: java.lang.String = foofoo
i.e. your side effect will happen every time you reference the argument. Since you're only doing it once, that's not that relevant to your case (except for the point of evaluation, which is inside the function, not at the call site).
To expand on the previous answers and clarify the way to avoid this, you can capture the value in a val inside the function if you only want it to be evaluated once. By doing this you are causing evaluation of the "by name" parameter and are using the computed value more than once rather than causing 2 evaluations of the same expression.
scala> def byName(arg: => String) = {val computedArg = arg; computedArg + computedArg}
byName: (arg: => String)java.lang.String
scala> byName({"println("hi") ; "foo" })
hi
res0: java.lang.String = foofoo
In case you need to do that in the future...
Lastly, to get truly lazy evaluation of a method parameter (zero or one evaluations only), you can do this:
def m1(i: => Int) = {
lazy val li = i
...
// any number of static or dynamic references to li
...
}
Related
In the Scala Odersky book, he has an example explaining partial functions of page 295. It starts with this function:
val second: List[Int] => Int = {
case x :: y :: _ => y
}
So the above function will succeed if you pass it a three element list but not an empty list.
second(List(5,6,7))
works but not
second(List())
The above will throw a MatchError: List
Here is the part that is confusing to me. Odersky writes:
If you want to check whether a partial function is defined, you must first tell the compiler that you know you are working with partial functions.
Why would I want to check whether a partial function is defined. What is a partial function? Is it a function that only applies to some values?
The type List[Int] => Int includes all functions from lists of integers to integers, whether or not the functions are partial. The type that only includes partial functions from lists of integers to integers is written PartialFunction[List[Int], Int].
So the above function returns a function of type List[Int] => Int, I see that, but why do we need to change this function to type PartialFunction[List[Int], Int]?
Here is the function redefined:
val second: PartialFunction[List [Int], Int] = {
case x :: y :: _ => y
}
I don't really get it. What is the benefit? Why do we want to check whether a partial function is defined? What does that even mean?
A partial function is any function, which takes only a single argument, that is defined (i.e. valid) only for a certain range of its argument's values. For example, Math.asin is defined only for argument values in the range [-1.0, 1.0] and is undefined for values outside of that range - so it is a partial function. For example, if we call Math.asin(5.0), we get NaN returned, meaning that the function is not defined for that argument.
Note that a partial function doesn't necessarily have to throw an exception; it just needs to do something other than return a valid value.
A key principle of functional programming is referential transparency (RT), meaning that we should be able to replace an expression (such as a function call) with the value of that expression, without changing the meaning of the program. (For more on this topic, I highly recommend that you read Functional Programming in Scala by Chiusano and Bjarnason.) Clearly, that breaks down if an exception is thrown or if an invalid value is returned. For calls to partial functions to be referentially transparent, we can only call them with argument values for which they are defined, or we need to elegantly handle the undefined values. So how can we tell if a partial function is defined for some arbitrary argument value?
In Scala we can express partial functions as a subclass of scala.PartialFunction that allows us to answer this question.
Let's look at your example in a Scala REPL session...
$ scala
Welcome to Scala 2.12.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_171).
Type in expressions for evaluation. Or try :help.
scala> val second: List[Int] => Int = {
| case x :: y :: _ => y
| }
<console>:11: warning: match may not be exhaustive.
It would fail on the following inputs: List(_), Nil
val second: List[Int] => Int = {
^
second: List[Int] => Int = $$Lambda$3181/1894473818#27492c62
So what did we just do? We defined second as a reference to a function that takes a List[Int] argument and returns an Int (the second value in the list).
You'll notice that the Scala compiler recognizes that this is not going to match all cases and warns you of the fact. This is a partial function, in the sense that it will fail for some arguments, but it's not an instance of scala.PartialFunction, as we can verify as follows:
scala> second.isInstanceOf[PartialFunction[List[Int], Int]]
res0: Boolean = false
Incidentally, the type List[Int] => Int is a shorthand for scala.Function1[List[Int], Int] and so seconds type is an instance of that type:
scala> second.isInstanceOf[Function1[List[Int], Int]]
res1: Boolean = true
Calling this version of the function produces the results you indicate:
scala> second(List(1, 2, 3))
res1: Int = 2
scala> second(Nil)
scala.MatchError: List() (of class scala.collection.immutable.Nil$)
at .$anonfun$second$1(<console>:11)
at .$anonfun$second$1$adapted(<console>:11)
... 36 elided
The problem is that if we just have some list value, l, and don't know what is in that list, we don't know whether we'll get an exception if we pass it to the function referenced by second. Now, we could put the call in a try block and catch any exception, but that's verbose and not good functional programming style. Ideally, we'd like to know whether we can call the function first to avoid an exception. Unfortunately, there's no way to tell from a Function1 instance:
scala> second.isDefinedAt(Nil)
<console>:13: error: value isDefinedAt is not a member of List[Int] => Int
second.isDefinedAt(Nil)
^
What we need is to declare second to have the type PartialFunction[List[Int], Int] as follows:
scala> val second: PartialFunction[List[Int], Int] = {
| case x :: y :: _ => y
| }
second: PartialFunction[List[Int],Int] = <function1>
(BTW, note that you have a typo in your question for this code - the above is how this should be defined.)
Now we do not have any warnings! We've told the compiler that this is a PartialFunction instance, so the compiler knows that its undefined for some arguments, so warnings are superfluous. We can now verify that fact:
scala> second.isInstanceOf[PartialFunction[List[Int], Int]]
res6: Boolean = true
We can now also verify whether it's defined for particular values:
scala> second.isDefinedAt(Nil)
res7: Boolean = false
scala> second.isDefinedAt(List(1, 2))
res9: Boolean = true
and so on. (The Scala compiler, as described in the book, is able to implement this magical isDefinedAt function for us.)
So, does that mean we should now write code like this:
def getSecondValue(l: List[Int]): Option[Int] = {
// Check if second is defined for this argument. If so, call it and wrap in Some.
if(second.isDefinedAt(l)) Some(second(l))
// Otherwise, we do not have a second value.
else None
}
Well, that's a little verbose too. Fortunately, once second is a PartialFunction instance, we can rewrite the above as:
def getSecondValue(l: List[Int]): Option[Int] = second.lift(l)
The lift method turns a partial function into a complete function that returns a defined value for every argument: if the argument to second is defined, then we get a Some(value); otherwise, we get None.
You'll find the concept of partial functions, and PartialFunction, more useful as you become more familiar with functional programming. If you don't get it right now, don't worry; all will become clear.
A partial function is a function that does not provide an answer for every possible input value it can be given. It provides an answer only for a subset of possible data, and defines the data it can handle. In Scala, a partial function can also be queried to determine if it can handle a particular value.
As a simple example, imagine a normal function that divides one number by another:
val divide = (x: Int) => 42 / x
As defined, this function blows up when the input parameter is zero:
scala> divide(0)
java.lang.ArithmeticException: / by zero
Although you can handle this particular situation by catching and throwing an exception, Scala lets you define the divide function as a PartialFunction. When doing so, you also explicitly state that the function is defined when the input parameter is not zero:
val divide = new PartialFunction[Int, Int] {
def apply(x: Int) = 42 / x
def isDefinedAt(x: Int) = x != 0
}
https://alvinalexander.com/scala/how-to-define-use-partial-functions-in-scala-syntax-examples
You can refer to the above link.
I've written a simple code in Scala with implicit conversion of Function1 to some case class.
object MyApp extends App{
case class FunctionContainer(val function:AnyRef)
implicit def cast(function1: Int => String):FunctionContainer = new FunctionContainer(function1)
def someFunction(i:Int):String = "someString"
def abc(f : FunctionContainer):String = "abc"
println(abc(someFunction))
}
But it doesn't work. Compiler doesn't want to pass someFunction as an argument to abc. I can guess its reasons but don't know exactly why it doesn't work.
When you use a method name as you have, the compiler has to pick how to convert the method type to a value. If the expected type is a function, then it eta-expands; otherwise it supplies empty parens to invoke the method. That is described here in the spec.
But it wasn't always that way. Ten years ago, you would have got your function value just by using the method name.
The new online spec omits the "Change Log" appendix, so for the record, here is the moment when someone got frustrated with parens and introduced the current rules. (See Scala Reference 2.9, page 181.)
This has not eliminated all irksome anomalies.
Conversions
The rules for implicit conversions of methods to functions (§6.26) have been tightened. Previously, a parameterized method used as a value was always implicitly converted to a function. This could lead to unexpected results when method arguments were forgotten. Consider for instance the statement below:
show(x.toString)
where show is defined as follows:
def show(x: String) = Console.println(x)
Most likely, the programmer forgot to supply an empty argument list () to toString. The previous Scala version would treat this code as a partially applied method, and expand it to:
show(() => x.toString())
As a result, the address of a closure would be printed instead of the value of s. Scala version 2.0 will apply a conversion from partially applied method to function value only if the expected type of the expression is indeed a function type. For instance, the conversion would not be applied in the code above because the expected type of show’s parameter is String, not a function type. The new convention disallows some previously legal code. Example:
def sum(f: int => double)(a: int, b: int): double =
if (a > b) 0 else f(a) + sum(f)(a + 1, b)
val sumInts = sum(x => x) // error: missing arguments
The partial application of sum in the last line of the code above will not be converted to a function type. Instead, the compiler will produce an error message which states that arguments for method sum are missing. The problem can be fixed by providing an expected type for the partial application, for instance by annotating the definition of sumInts with its type:
val sumInts: (int, int) => double = sum(x => x) // OK
On the other hand, Scala version 2.0 now automatically applies methods with empty parameter lists to () argument lists when necessary. For instance, the show expression above will now be expanded to
show(x.toString())
Your someFunction appears as a method here.
You could try either
object MyApp extends App{
case class FunctionContainer(val function:AnyRef)
implicit def cast(function1: Int => String):FunctionContainer = new FunctionContainer(function1)
val someFunction = (i:Int) => "someString"
def abc(f : FunctionContainer):String = "abc"
println(abc(someFunction))
}
or
object MyApp extends App{
case class FunctionContainer(val function:AnyRef)
implicit def cast(function1: Int => String):FunctionContainer = new FunctionContainer(function1)
def someFunction(i:Int): String = "someString"
def abc(f : FunctionContainer):String = "abc"
println(abc(someFunction(_: Int)))
}
By the way: implicitly casting such common functions to something else can quickly lead to problems. Are you absolutely sure that you need this? Wouldn't it be easier to overload abc?
You should use eta-expansion
println(abc(someFunction _))
def first[A] :Tuple2[A,_] => A = ( pair :Tuple2[A,_] ) => pair._1
val name = first( ("Anna", 23) )
"If you take a closer look at line 2, what you see here is a method call which returns a newly created function of type Tuple2[String,Any] => String (since the compiler kicks in and infers the needed type for applying to person). Although the whole expression looks like an ordinary method call, it’s in fact a method call (to a factory method without any parameter) and a function call which follows afterwards. " -- this is the explanation of the above code.
I am not able to reason about the first step of the above process (the process creating a function object). Can someone write out a "human compiler" procedure explicitly?
EDIT: I think the fully expanded logic for line 2 should be the following two lines
val firstAsFunc= first[String];
val name = firstAsFunc(("Anna", 23))
I'm not sure to break it down further. Here's what I can think of -- I hope you get it, or that someone else is feeling more clever than I.
scala> val func = first[String] // method call
func: Tuple2[String, _] => String = <function1>
scala> val name = func( ("Anna", 23) )
name: String = Anna
The problem with the above is that func is really a getter -- a method call itself -- so I'm hardly changing anything.
EDIT
I'm not sure what you mean by formal parameter. The method first doesn't have value parameters, just type parameters. Trying to pass a value parameter to it would be a syntactical error.
When you say
(pair: Tuple2[A,_]) => pair._1
the compiler decides that you are actually saying
new Function1[Tuple2[A,_], A] {
def apply(pair: Tuple2[A,_]) = pair._1
}
That is, the first method creates a new object (of type Function1) with a method called apply which then is transparently called when you say first(...). (You would get the same thing if you wrote first.apply(...).)
(Note: Tuple2[A,_] can itself be abbreviated (A,_).)
I'm not 100% sure that I understand which bit of the process you're asking about - are you asking about what a function object is? I'll answer that question on the assumption that it is :-)
A function object is an object that derives from one of the FunctionN (Function0, Function1 etc.) traits and implements an apply method. So your example could be rewritten:
scala> def first[A]: Tuple2[A, _] => A = new Function1[Tuple2[A, _], A] { def apply(pair: Tuple2[A, _]) = pair._1 }
first: [A]=> Tuple2[A, _] => A
scala> val name = first( ("Anna", 23) )
name: java.lang.String = Anna
You can see that a function is actually an instance of FunctionN like so:
scala> def foo(x: Int, y: Double): String = "x = "+ x.toString +", "+ y.toString
foo: (x: Int, y: Double)String
scala> (foo _).isInstanceOf[Function2[_, _, _]]
res1: Boolean = true
If you take a closer look at line 2, what you see here is a method call which returns a newly created function of type Tuple2[String,Any] => String
This explanation is wrong. Line 2 does not "return a newly created function". The function is created on line 1, as explained by Rex Kerr.
Although the whole expression [on line 2] looks like an ordinary method call, it’s in fact a method call (to a factory method without any parameter) and a function call which follows afterwards.
I don't believe this is true; there is no hidden factory method going on, because the Function1 object has already been created on line 1.
One of the questions I was asking what is the formal parameter for method first.
See Wikipedia > Parameter # Computer Science
In the file Parsers.scala (Scala 2.9.1) from the parser combinators library I seem to have come across a lesser known Scala feature called "lazy arguments". Here's an example:
def ~ [U](q: => Parser[U]): Parser[~[T, U]] = { lazy val p = q // lazy argument
(for(a <- this; b <- p) yield new ~(a,b)).named("~")
}
Apparently, there's something going on here with the assignment of the call-by-name argument q to the lazy val p.
So far I have not been able to work out what this does and why it's useful. Can anyone help?
Call-by-name arguments are called every time you ask for them. Lazy vals are called the first time and then the value is stored. If you ask for it again, you'll get the stored value.
Thus, a pattern like
def foo(x: => Expensive) = {
lazy val cache = x
/* do lots of stuff with cache */
}
is the ultimate put-off-work-as-long-as-possible-and-only-do-it-once pattern. If your code path never takes you to need x at all, then it will never get evaluated. If you need it multiple times, it'll only be evaluated once and stored for future use. So you do the expensive call either zero (if possible) or one (if not) times, guaranteed.
The wikipedia article for Scala even answers what the lazy keyword does:
Using the keyword lazy defers the initialization of a value until this value is used.
Additionally, what you have in this code sample with q : => Parser[U] is a call-by-name parameter. A parameter declared this way remains unevaluated, until you explicitly evaluate it somewhere in your method.
Here is an example from the scala REPL on how the call-by-name parameters work:
scala> def f(p: => Int, eval : Boolean) = if (eval) println(p)
f: (p: => Int, eval: Boolean)Unit
scala> f(3, true)
3
scala> f(3/0, false)
scala> f(3/0, true)
java.lang.ArithmeticException: / by zero
at $anonfun$1.apply$mcI$sp(<console>:9)
...
As you can see, the 3/0 does not get evaluated at all in the second call. Combining the lazy value with a call-by-name parameter like above results in the following meaning: the parameter q is not evaluated immediately when calling the method. Instead it is assigned to the lazy value p, which is also not evaluated immediately. Only lateron, when p is used this leads to the evaluation of q. But, as p is a val the parameter q will only be evaluated once and the result is stored in p for later reuse in the loop.
You can easily see in the repl, that the multiple evaluation can happen otherwise:
scala> def g(p: => Int) = println(p + p)
g: (p: => Int)Unit
scala> def calc = { println("evaluating") ; 10 }
calc: Int
scala> g(calc)
evaluating
evaluating
20
What exactly is the difference between:
scala> def foo = 5
foo: Int
and
scala> def foo() = 5
foo: ()Int
Seems that in both cases, I end up with a variable foo which I can refer to without parenthesis, always evaluating to 5.
You're not defining a variable in either case. You're defining a method. The first method has no parameter lists, the second has one parameter list, which is empty. The first of these should be
called like this
val x = foo
while the second should be called like this
val x = foo()
However, the Scala compiler will let you call methods with one empty parameter list without the parentheses, so either form of call will work for the second method. Methods without parameter lists cannot be called with the parentheses
The preferred Scala style is to define and call no-argument methods which have side-effects with the parentheses. No-argument methods without side-effects should be defined and called without the parentheseses.
If you actually which to define a variable, the syntax is
val foo = 5
Before anything else is said, def does not define a field, it defines a method.
In the second case, you can omit parenthesis because of a specific feature of Scala. There are two differences of interest here: one mechanical, and one of recommended usage.
Beginning with the latter, it is recommended usage to use empty parameter list when there are side effects. One classic example is close(). You'd omit parenthesis if there are no side effects to calling the element.
Now, as a practical difference -- beyond possible weird syntactic mix-ups in corner cases (I'm not saying there are, just conjecturing) -- structural types must follow the correct convention.
For example, Source had a close method without parenthesis, meaning a structural type of def close(): Unit would not accept Source. Likewise, if I define a structural method as def close: Unit, then Java closeable objects will not be accepted.
What does it mean when I use def to define a field in Scala
You can't define a field using def.
Seems that in both cases, I end up with a variable foo which I can refer to without parenthesis, always evaluating to 5.
No, in both cases you end up with a method foo, which you can call without parentheses.
To see that, you can use javap:
// Main.scala
object Main {
def foo1 = 5
def foo2() = 5
}
F:\MyProgramming\raw>scalac main.scala
F:\MyProgramming\raw>javap Main
Compiled from "main.scala"
public final class Main extends java.lang.Object{
public static final int foo2();
public static final int foo1();
}
However, see http://tommy.chheng.com/index.php/2010/03/when-to-call-methods-with-or-without-parentheses-in-scala/
Additionally to the answers already given I'd like to stress two points:
The possibility to define methods without a parameter list is a way to realize the Uniform Access Principle. This allows to hide the difference between fields and methods, which makes later implementation changes easier.
You can call a method defined as def foo() = 5 using foo, but you can't call a method defined as def foo = 5 using foo()
I'm surprised that nobody mentioned anything about the laziness difference.
While val is evaluated only once at the time of definition, def is evaluated only when we access it and evaluated every-time we access it. See example below:
scala> def foo = {
| println("hi")
| 5
| }
foo: Int
scala> val onlyOnce = foo
scala> def everyTime = foo
scala> onlyOnce
res0: Int = 5
scala> onlyOnce
res1: Int = 5
scala> everyTime
hi
res2: Int = 5
scala> everyTime
hi
res3: Int = 5