I used to think that private val and private final val are same, until I saw section 4.1 in Scala Reference:
A constant value definition is of the form
final val x = e
where e is a constant expression (§6.24). The final modifier must be present and no type annotation may be given. References to the constant value x are themselves treated as constant expressions; in the generated code they are replaced by the definition’s right-hand side e.
And I have written a test:
class PrivateVal {
private val privateVal = 0
def testPrivateVal = privateVal
private final val privateFinalVal = 1
def testPrivateFinalVal = privateFinalVal
}
javap -c output:
Compiled from "PrivateVal.scala"
public class PrivateVal {
public int testPrivateVal();
Code:
0: aload_0
1: invokespecial #19 // Method privateVal:()I
4: ireturn
public int testPrivateFinalVal();
Code:
0: iconst_1
1: ireturn
public PrivateVal();
Code:
0: aload_0
1: invokespecial #24 // Method java/lang/Object."<init>":()V
4: aload_0
5: iconst_0
6: putfield #14 // Field privateVal:I
9: return
}
The byte code is just as Scala Reference said: private val is not private final val.
Why doesn't scalac just treat private val as private final val? Is there any underlying reason?
So, this is just a guess, but it was a perennial annoyance in Java that final static variables with a literal on the right-hand side get inlined into bytecode as constants. That engenders a performance benefit sure, but it causes binary compatibility of the definition to break if the "constant" ever changed. When defining a final static variable whose value might need to change, Java programmers have to resort to hacks like initializing the value with a method or constructor.
A val in Scala is already final in the Java sense. It looks like Scala's designers are using the redundant modifier final to mean "permission to inline the constant value". So Scala programmers have complete control over this behavior without resorting to hacks: if they want an inlined constant, a value that should never change but is fast, they write "final val". if they want flexibility to change the value without breaking binary compatibility, just "val".
I think the confusion here arises from conflating immutability with the semantics of final. vals can be overridden in child classes and therefore can't be treated as final unless marked as such explicitly.
#Brian The REPL provides class scope at the line level. See:
scala> $iw.getClass.getPackage
res0: Package = package $line3
scala> private val x = 5
<console>:5: error: value x cannot be accessed in object $iw
lazy val $result = `x`
scala> private val x = 5; println(x);
5
Related
My understanding was that all non-capturing lambdas shouldn't require object creation at use site, because one can be created as a static field and reused. In principle, the same could be true for lambdas constituting of a class method call - only the field would be non static. I never actually tried to dig any deeper into it; now I am looking at the bytecode, don't see one in the enclosing class and don't have a good idea where to look? I see though that the lambda factory is different than in Java, so this should have a clear answer - at least for a given Scala version.
My motivation is simple: profiling is very time consuming. Introducing method values (or in general, lambdas capturing only the state of the enclosing object) as private class fields is less clean and more work than writing them inline and, in general, not good code. But when writing areas known (with high likelihood) to be a hot spot, it's a very simple optimisation that can be performed straight away without any real impact on the programmer's time. It doesn't make sense though if no new object is created anyway.
Take for example:
def alias(x :X) = aliases.getOrElse(x, x)
def alias2(x :X) = aliases.getOrElse(x, null) match {
case null => x
case a => a
}
The first lambda (a Function0) must be a new object because it captures method parameter x, while the second one returns a constant (null) and thus doesn't really have to. It is also less messy (IMO) than a private class field, which pollutes the namespace, but I would like to be able to know for sure - or have a way of easily confirming my expectations.
The following proves that at least some of the time, the answer is "no":
scala 2.13.4> def foo = () => 1
def foo: () => Int
scala 2.13.4> foo eq foo
val res5: Boolean = true
Looking at the bytecode produced by this code:
import scala.collection.immutable.ListMap
object ByName {
def aliases = ListMap("Ein" -> "One", "Zwei" -> "Two", "Drei" -> "Three")
val default = "NaN"
def alias(x: String) = aliases.getOrElse(x, x)
def alias2(x: String) = aliases.getOrElse(x, null) match {
case null => x
case a => a
}
def alias3(x: String) = aliases.getOrElse(x, default)
}
The compiler generates static methods for the by-name parameters. They look like this:
public static final java.lang.String $anonfun$alias$1(java.lang.String);
Code:
0: aload_0
1: areturn
public static final scala.runtime.Null$ $anonfun$alias2$1();
Code:
0: aconst_null
1: areturn
public static final java.lang.String $anonfun$alias3$1();
Code:
0: getstatic #26 // Field MODULE$:LByName$;
3: invokevirtual #138 // Method default:()Ljava/lang/String;
6: areturn
The naive approach would have been for the compiler to generate anonymous classes that implement the Function0 interface. However, this would cause bytecode-bloat. Instead the compiler defers creating these anonymous inner classes until runtime via invokedynamic instructions.
Exactly how Scala uses these invokedynamic instructions is beyond my knowledge. It's possible that they cache the generated Function0 object somehow, but my guess is that the invokedynamic call is sufficiently optimized that it's faster to just generate a new one every time. Allocating short lived objects is cheap, and the cost is most often overestimated. Reusing an existing object might even be slower than creating a new one if it means cache misses.
I also want to point out that this is a implementation detail, and likely to change at any time. The Scala compiler devs and JVM devs know what they are doing, so you are probably better off trusting that their implementation balances performance well.
What is the reason for vals not (?) being automatically final in singleton objects? E.g.
object NonFinal {
val a = 0
val b = 1
def test(i: Int) = (i: #annotation.switch) match {
case `a` => true
case `b` => false
}
}
results in:
<console>:12: error: could not emit switch for #switch annotated match
def test(i: Int) = (i: #annotation.switch) match {
^
Whereas
object Final {
final val a = 0
final val b = 1
def test(i: Int) = (i: #annotation.switch) match {
case `a` => true
case `b` => false
}
}
Compiles without warnings, so presumably generates the faster pattern matching table.
Having to add final seems pure annoying noise to me. Isn't an object final per se, and thus also its members?
This is addressed explicitly in the specification, and they are automatically final:
Members of final classes or objects are implicitly also final, so
the final modifier is generally redundant for them, too. Note, however, that
constant value definitions (§4.1) do require an explicit final modifier, even if
they are defined in a final class or object.
Your final-less example compiles without errors (or warnings) with 2.10-M7, so I'd assume that there's a problem with the #switch checking in earlier versions, and that the members are in fact final.
Update: Actually this is more curious than I expected—if we compile the following with either 2.9.2 or 2.10-M7:
object NonFinal {
val a = 0
}
object Final {
final val a = 0
}
javap does show a difference:
public final class NonFinal$ implements scala.ScalaObject {
public static final NonFinal$ MODULE$;
public static {};
public int a();
}
public final class Final$ implements scala.ScalaObject {
public static final Final$ MODULE$;
public static {};
public final int a();
}
You see the same thing even if the right-hand side of the value definitions isn't a constant expression.
So I'll leave my answer, but it's not conclusive.
You're not asking "why aren't they final", you're asking "why aren't they inlined." It just happens that final is how you cue the compiler that you want them inlined.
The reason they are not automatically inlined is separate compilation.
object A { final val x = 55 }
object B { def f = A.x }
When you compile this, B.f returns 55, literally:
public int f();
0: bipush 55
2: ireturn
That means if you recompile A, B will be oblivious to the change. If x is not marked final in A, B.f looks like this instead:
0: getstatic #19 // Field A$.MODULE$:LA$;
3: invokevirtual #22 // Method A$.x:()I
6: ireturn
Also, to correct one of the other answers, final does not mean immutable in scala.
To address the central question about final on an object, I think this clause from the spec is more relevant:
A constant value definition is of the form final val x = e
where e is a constant expression (§6.24). The final modifier must be present and no type annotation may be given. References to the constant value x are themselves treated as constant expressions; in the generated code they are replaced by the definition’s right-hand side e.
Of significance:
No type annotation may be given
The expression e is used in the generated code (by my reading, as the original unevaluated constant expression)
It sounds to me like the compiler is required by the spec to use these more like macro replacements rather than values that are evaluated in place at compile time, which could have impacts on how the resulting code runs.
I think it is particularly interesting that no type annotation may be given.
This, I think points to our ultimate answer, though I cannot come up with an example that shows the runtime difference for these requirements. In fact, in my 2.9.2 interpreter, I don't even get the enforcement of the first rule.
For performance and safety I would like to implement a fixed-size vector which is both immutable and specialized (I need fast arithmetics). My first idea was to use the #specialized annotation (because I need both integers and reals).
Here is a first try:
package so
class Vec[#specialized A] private[so] ( ary: Array[A] ) {
def apply( i: Int ) = ary(i)
}
However, when I analyze the resulting bytecode with javap, I can see that the elements are still boxed. For instance:
public double apply$mcD$sp(int);
Code:
0: aload_0
1: iload_1
2: invokevirtual #33; //Method apply:(I)Ljava/lang/Object;
5: invokestatic #83; //Method scala/runtime/BoxesRunTime.unboxToDouble:(Ljava/lang/Object;)D
8: dreturn
It looks like arrays are not specialized which seems silly, because arrays are specialized on the JVM.
Is there something I can still do to reach my goal ?
You are likely looking at the code compiled to Vec.class. According to this thread the specialization occurs in subclasses. This can be verified in the REPL:
scala> class Vec[#specialized A] ( ary: Array[A] ) {
| def apply( i: Int ) = ary(i)
| }
defined class Vec
scala> new Vec( Array[Int](1) ).getClass
res0: java.lang.Class[_ <: Vec[Int]] = class Vec$mcI$sp
As you can see for Int it is using the subclass Vec$mcI$sp. And if you run javap on that class you will see that it is infact specializing the code properly. This is what the apply method looks like in Vec$mcI$sp.class using javap:
public int apply(int);
flags: ACC_PUBLIC
Code:
stack=2, locals=2, args_size=2
0: aload_0
1: iload_1
2: invokevirtual #13 // Method apply$mcI$sp:(I)I
5: ireturn
Which I suppose is what you want when using Int.
Is there a way in scala to get the arguments back from a already partially applied function?
Does this even make sense, should be done, or fits into any use case?
example:
def doStuff(lower:Int,upper:Int,b:String)=
for(turn <- lower to upper) println(turn +": "+b)
Imagine that at one point I know the 'lower' argument and I get a function of applying it to 'doStuff'
val lowerDoStuff = doStuff(3,_:Int,_:String)
Is there a way for me to get that 3 back ?
(for the sake of example, imagine that I am inside a function which only received 'lowerDoStuff' and now needs to know the first argument)
Idiomatic scala is prefered to introspection/reflection (if possible).
Idiomatic Scala: no, you can't. You have specifically said that the first argument is no longer relevant. If the compiler can make it disappear entirely, that's best: you say you have a function that depends on an int and a string, and you haven't made any promises about what generated it. If you really need that value, but you also really need to pass a 2-argument function, you can do it by hand:
class Function2From3[A,B,C,Z](f: (A,B,C) => Z, val _1: A) extends Function2[B,C,Z] {
def apply(b: B, c: C) = f(_1, b, c)
}
val lowerDoStuff = new Function2From3(doStuff _, 3)
Now when you get the function later on, you can pattern match to see if it's a Function2From3, and then read the value:
val f: Function2[Int,String,Unit] = lowerDoStuff
f match {
case g: Function2From3[_,_,_,_] => println("I know there's a "+g._1+" in there!")
case _ => println("It's all Greek to me.")
}
(if it's important to you that it be an integer, you can remove A as a generic parameter and make _1 be an integer--and maybe just call it lower while you're at it).
Reflection: no, you can't (not in general). The compiler's smarter than that. The generated bytecode (if we wrap your code in class FuncApp) is:
public final void apply(int, java.lang.String);
Signature: (ILjava/lang/String;)V
Code:
0: aload_0
1: getfield #18; //Field $outer:LFuncApp;
4: iconst_3
5: iload_1
6: aload_2
7: invokevirtual #24; //Method FuncApp.doStuff:(IILjava/lang/String;)V
10: return
Notice the iconst_3? That's where your 3 went--it disappeared into the bytecode. There's not even a hidden private field containing the value any more.
As I read, Scala immutable val doesn't get translated to Java final for various reasons. Does this mean that accessing a val from an other Thread must be guarded with synchronization in order to guarantee visibility?
the assignment to val itself is fine from a multi-threading point of view, because you have to assign val a value when you declare it and that value can't be changed in the future (so if you do a val s="hello", s is "hello" from its birth on: no thread will ever read another value).
There are a couple of caveats, however:
1 - if you assign an instance of a mutable class to val, val by itself will not "protect" the internal state of the class from changing.
class Foo(s:String) { var thisIsMutable=s }
// you can then do this
val x = new Foo("hello")
x.thisIsMutable="goodbye"
// note that val guarantees that x is still the same instance of Foo
// reassigning x = new Foo("goodbye") would be illegal
2 - you (or one of your libraries...) can change a val via reflection. If this happens two threads could indeed read a different value for your val
import java.lang.reflect.Field
class Foo { val foo=true } // foo is immutable
object test {
def main(args: Array[String]) {
val f = new Foo
println("foo is " + f.foo) // "foo is true"
val fld = f.getClass.getDeclaredField("foo")
fld.setAccessible(true)
fld.setBoolean(f, false)
println("foo is " + f.foo) // "foo is false"
}
}
As object members, once initialized, vals never change their values during the lifetime of the object. As such, their values are guaranteed to be visible to all threads provided that the reference to the object didn't escape in the constructor. And, in fact, they get Java final modifiers as illustrated below:
object Obj {
val r = 1
def foo {
val a = 1
def bar = a
bar
}
}
Using javap:
...
private final int r;
...
public void foo();
...
0: iconst_1
1: istore_1
2: aload_0
3: iload_1
4: invokespecial #31; //Method bar$1:(I)I
7: pop
...
private final int bar$1(int);
...
0: iload_1
1: ireturn
...
As method locals, they are used only within the method, or they're being passed to a nested method or a closure as arguments (see lifted bar$1 above). A closure might be passed on to another thread, but it will only have a final field with the value of the local val. Therefore, they are visible from the point where they are created to all other threads and synchronization is not necessary.
Note that this says nothing about the object the val points to - it itself may be mutable and warrant synchronization.
In most cases the above cannot be violated via reflection - the Scala val member declaration actually generates a getter with the same name and a private field which the getter accesses. Trying to use reflection to modify the field will result in the NoSuchFieldException. The only way you could modify it is to add a specialized annotation to your class which will make the specialized fields protected, hence accessible to reflection. I cannot currently think of any other situation that could change something declared as val...