The fine Twitter util library has the following Java class, which is extended by a Scala class that reads the volatile field and updates it using AtomicReferenceFieldUpdater. Access must be at least private, i.e., must allow other.state.
Is the motivational claim in the comment true? How true? ("How" means in what way and to what degree.)
public class IVarField<A> {
/** This is needed because we cannot create a field like this in Scala. */
volatile State<A> state;
}
Something like the following works.
Making the field "private [this]" turns it into the desired field reference ("getfield").
Access from a different instance is through an accessor "state()" which, if made #inline, will happily loosen the access restriction to the field.
That also means that the updater (which happens to reside in the companion module) can also access it.
(Because the normal accessor for the var is not emitted for an object-private member, you can define your own, using parens. But you don't need to use parens at the call site, other.state. Your uniform access principle dollars at work. Or Swiss francs.)
Also notice the totally hip usage of `new` in backticks for a param name. I don't even know how to make the ticks show up in this markup. Because both params are the same type, one is likely to want to write cas(expect=prev, `new`=changed), so I might have used next instead, but if this markup supported a hipster thumbs-up, I'd give it right now. :+1: Did it work? Can anyone see my thumb? [I guess I saw that on github. Hubster, not hipster.]
object IVar {
// TODO: retrieve mangled name by reflection
private final val stateField = "com$twitter$concurrent$IVar$$state"
private val stateUpd = AtomicReferenceFieldUpdater.newUpdater(
classOf[IVar[_]], classOf[State[_]], stateField)
}
final class IVar[A] { //}extends IVarField[A] {
import IVar._
#volatile private[this] var state: State[A] = initState: State[A]
#inline private final def state(): State[A] = this.state
override def toString = "Ivar#%s(state=%s)".format(hashCode, state)
#inline private[this]
def cas(expect: State[A], `new`: State[A]) = stateUpd.compareAndSet(this, expect, `new`)
def depth: Int = {
#tailrec
def loop(iv: IVar[_], d: Int): Int = iv.state match {
case Linked(iv) => loop(iv, d + 1)
case _ => d
}
loop(this, 0)
}
// etc
}
Showing that state() is really inlined:
private final int loop$1(com.twitter.concurrent.IVar, int);
flags: ACC_PRIVATE, ACC_FINAL
Code:
stack=3, locals=5, args_size=3
0: aload_1
// Field com$twitter$concurrent$IVar$$state:Lcom/twitter/concurrent/ivar/State;
1: getfield #16
4: astore_3
Not only is it "OK to ask and answer your own questions",
https://blog.stackoverflow.com/2011/07/its-ok-to-ask-and-answer-your-own-questions/
it's infinitely more satisfying.
(More power to Daniel Sobral, Rex Kerr, Retronym and the rest of the Justice League.)
Related
A use case needs to compare the actual type of two type parameters at runtime time. That might be easily done with TypeTag, but it is deprecated in scala-3.
Therefore I tried with TypeTest but it requires the existence of an instance of the type, which I don't have.
So, I implemented TypeId, my own version of TypeTag. It is way way more limited and simple. It only supports type comparison and works only if a single instance of TypeId exists for each involved type.
import scala.collection.{mutable, immutable}
object TypeId {
private var idSequencer: Int = 0
private val typeIds: mutable.HashMap[Int, TypeId[?]] = mutable.HashMap.empty
private def registerNew[A](tm: TypeId[A]): Int = this.synchronized {
idSequencer += 1
typeIds.put(idSequencer, tm)
idSequencer
}
def knownTypeIds: immutable.HashMap[Int, TypeId[?]] = this.synchronized(immutable.HashMap.from(typeIds))
}
trait TypeId[A] {
val id: Int = registerNew(this)
def equalsTo(that: TypeId[?]): Boolean = this.id == that.id
}
As you can see, it requires synchronization to be thread-safe, which is not convenient; and the value of knownTypeIds is vulnerable to race conditions.
I wonder if it is possible to create the unique ids at compile using a macro. I suppose that to do that I need some kind of compile-time global variable to implement the sequencer. Does scala-3 macros support compile time global variables or an alternative to achieve my goal?
EDIT:
A couple of minutes after asking my question I came up with a very simple way to achieve my goal by making the identifier a string with the extended name of the type. Such a name can easily be generated in a macro.
However, I leave the question to know if scala-3 macros support compile-time global variables.
Here is the code that achieves my goal using the name of the type instead of sequential integer.
import scala.collection.{immutable}
object TypeId {
#volatile
private var typeIds: immutable.Set[String] = immutable.HashSet.empty
def knownTypeIds: immutable.Set[String] = typeIds
private inline def registerNew[A]: String = {
val id = TypeIdMacros.createId[A]
this.synchronized(typeIds += id)
id
}
}
trait TypeId[A] {
val id: String = registerNew[A]
def equalsTo(that: TypeId[?]): Boolean = this.id == that.id
}
import scala.quoted.{Quotes, Type, Expr}
object TypeIdMacros {
inline def createId[A]: String = ${createIdImpl[A]}
def createIdImpl[A](using typeA: Type[A], quotes: Quotes): Expr[String] =
Expr(Type.show[A])
}
It would be a very bad idea, because of incremental compilation - it's quite hard to tell exactly what will be compiled or when. You could easily get something that works on a clean compile, but fails on partial re-compiles during development as not every bit of code that should be triggering the sequencer gets fired.
(I'm not sure if it's doable at all, and there might be other reasons to avoid it, but this is a general reason to avoid any sort of imperative code in macros.)
My understanding was that all non-capturing lambdas shouldn't require object creation at use site, because one can be created as a static field and reused. In principle, the same could be true for lambdas constituting of a class method call - only the field would be non static. I never actually tried to dig any deeper into it; now I am looking at the bytecode, don't see one in the enclosing class and don't have a good idea where to look? I see though that the lambda factory is different than in Java, so this should have a clear answer - at least for a given Scala version.
My motivation is simple: profiling is very time consuming. Introducing method values (or in general, lambdas capturing only the state of the enclosing object) as private class fields is less clean and more work than writing them inline and, in general, not good code. But when writing areas known (with high likelihood) to be a hot spot, it's a very simple optimisation that can be performed straight away without any real impact on the programmer's time. It doesn't make sense though if no new object is created anyway.
Take for example:
def alias(x :X) = aliases.getOrElse(x, x)
def alias2(x :X) = aliases.getOrElse(x, null) match {
case null => x
case a => a
}
The first lambda (a Function0) must be a new object because it captures method parameter x, while the second one returns a constant (null) and thus doesn't really have to. It is also less messy (IMO) than a private class field, which pollutes the namespace, but I would like to be able to know for sure - or have a way of easily confirming my expectations.
The following proves that at least some of the time, the answer is "no":
scala 2.13.4> def foo = () => 1
def foo: () => Int
scala 2.13.4> foo eq foo
val res5: Boolean = true
Looking at the bytecode produced by this code:
import scala.collection.immutable.ListMap
object ByName {
def aliases = ListMap("Ein" -> "One", "Zwei" -> "Two", "Drei" -> "Three")
val default = "NaN"
def alias(x: String) = aliases.getOrElse(x, x)
def alias2(x: String) = aliases.getOrElse(x, null) match {
case null => x
case a => a
}
def alias3(x: String) = aliases.getOrElse(x, default)
}
The compiler generates static methods for the by-name parameters. They look like this:
public static final java.lang.String $anonfun$alias$1(java.lang.String);
Code:
0: aload_0
1: areturn
public static final scala.runtime.Null$ $anonfun$alias2$1();
Code:
0: aconst_null
1: areturn
public static final java.lang.String $anonfun$alias3$1();
Code:
0: getstatic #26 // Field MODULE$:LByName$;
3: invokevirtual #138 // Method default:()Ljava/lang/String;
6: areturn
The naive approach would have been for the compiler to generate anonymous classes that implement the Function0 interface. However, this would cause bytecode-bloat. Instead the compiler defers creating these anonymous inner classes until runtime via invokedynamic instructions.
Exactly how Scala uses these invokedynamic instructions is beyond my knowledge. It's possible that they cache the generated Function0 object somehow, but my guess is that the invokedynamic call is sufficiently optimized that it's faster to just generate a new one every time. Allocating short lived objects is cheap, and the cost is most often overestimated. Reusing an existing object might even be slower than creating a new one if it means cache misses.
I also want to point out that this is a implementation detail, and likely to change at any time. The Scala compiler devs and JVM devs know what they are doing, so you are probably better off trusting that their implementation balances performance well.
I'm trying to access variables of an object by using strings whose values are determined at run time.
Basically the object is used as a dictionary, and now I want to dynamically poll values from that dictionary
A simplified example:
object Something extends scala.AnyRef {
final val zero = 0
final val one = 1
final val two = 2
}
I would then principally like to do the following
Somthing."zero"
and get 0 in return.
Ideally I would like to use scala reflection for this
WARNING: Please heed Jörg W Mittag's advice before following my answer:
Scala heavily prefers type-safe compile-time reflection. So, using
reflection in this way is not the Scala way. The Scala way would
probably be to use compile-time reflection to generate all those
pattern match cases
Using reflection Something.getClass.getDeclaredMethods.foreach(println) gives:
public final int example.Something$.zero()
public final int example.Something$.one()
public final int example.Something$.two()
so to get the names we can do
Something.getClass.getDeclaredMethods.map(_.getName) // Array(zero,one,two)
and to get the value
Something.getClass.getMethod("zero").invoke(Something) // 0
Defining the following implicit class
implicit class AccessMembersByReflectionName(something: Something.type) {
def get(name: String): AnyRef = {
Something.getClass.getMethod(name).invoke(Something)
}
}
cleans up the call site
Something.get("zero") // 0
As #n.m. already pointed, if you only want to retrieve values of the same type given an string key, you should use a Dictionary.
However, if for some reason you still need an object, you could accomplish something similar of what you want with this method.
Object Something {
val Zero: Int = 0
val One: Int = 1
val Three: Int = 3
def getValueByString(valName: String): Int = valName.toLowerCase match {
case "zero" => Zero
case "one" => One
case "three" => Three
}
}
EDIT
I just read that you can modify the object, you still can make a function that will receive the object and the valName as a string.
What is the reason for vals not (?) being automatically final in singleton objects? E.g.
object NonFinal {
val a = 0
val b = 1
def test(i: Int) = (i: #annotation.switch) match {
case `a` => true
case `b` => false
}
}
results in:
<console>:12: error: could not emit switch for #switch annotated match
def test(i: Int) = (i: #annotation.switch) match {
^
Whereas
object Final {
final val a = 0
final val b = 1
def test(i: Int) = (i: #annotation.switch) match {
case `a` => true
case `b` => false
}
}
Compiles without warnings, so presumably generates the faster pattern matching table.
Having to add final seems pure annoying noise to me. Isn't an object final per se, and thus also its members?
This is addressed explicitly in the specification, and they are automatically final:
Members of final classes or objects are implicitly also final, so
the final modifier is generally redundant for them, too. Note, however, that
constant value definitions (§4.1) do require an explicit final modifier, even if
they are defined in a final class or object.
Your final-less example compiles without errors (or warnings) with 2.10-M7, so I'd assume that there's a problem with the #switch checking in earlier versions, and that the members are in fact final.
Update: Actually this is more curious than I expected—if we compile the following with either 2.9.2 or 2.10-M7:
object NonFinal {
val a = 0
}
object Final {
final val a = 0
}
javap does show a difference:
public final class NonFinal$ implements scala.ScalaObject {
public static final NonFinal$ MODULE$;
public static {};
public int a();
}
public final class Final$ implements scala.ScalaObject {
public static final Final$ MODULE$;
public static {};
public final int a();
}
You see the same thing even if the right-hand side of the value definitions isn't a constant expression.
So I'll leave my answer, but it's not conclusive.
You're not asking "why aren't they final", you're asking "why aren't they inlined." It just happens that final is how you cue the compiler that you want them inlined.
The reason they are not automatically inlined is separate compilation.
object A { final val x = 55 }
object B { def f = A.x }
When you compile this, B.f returns 55, literally:
public int f();
0: bipush 55
2: ireturn
That means if you recompile A, B will be oblivious to the change. If x is not marked final in A, B.f looks like this instead:
0: getstatic #19 // Field A$.MODULE$:LA$;
3: invokevirtual #22 // Method A$.x:()I
6: ireturn
Also, to correct one of the other answers, final does not mean immutable in scala.
To address the central question about final on an object, I think this clause from the spec is more relevant:
A constant value definition is of the form final val x = e
where e is a constant expression (§6.24). The final modifier must be present and no type annotation may be given. References to the constant value x are themselves treated as constant expressions; in the generated code they are replaced by the definition’s right-hand side e.
Of significance:
No type annotation may be given
The expression e is used in the generated code (by my reading, as the original unevaluated constant expression)
It sounds to me like the compiler is required by the spec to use these more like macro replacements rather than values that are evaluated in place at compile time, which could have impacts on how the resulting code runs.
I think it is particularly interesting that no type annotation may be given.
This, I think points to our ultimate answer, though I cannot come up with an example that shows the runtime difference for these requirements. In fact, in my 2.9.2 interpreter, I don't even get the enforcement of the first rule.
As I read, Scala immutable val doesn't get translated to Java final for various reasons. Does this mean that accessing a val from an other Thread must be guarded with synchronization in order to guarantee visibility?
the assignment to val itself is fine from a multi-threading point of view, because you have to assign val a value when you declare it and that value can't be changed in the future (so if you do a val s="hello", s is "hello" from its birth on: no thread will ever read another value).
There are a couple of caveats, however:
1 - if you assign an instance of a mutable class to val, val by itself will not "protect" the internal state of the class from changing.
class Foo(s:String) { var thisIsMutable=s }
// you can then do this
val x = new Foo("hello")
x.thisIsMutable="goodbye"
// note that val guarantees that x is still the same instance of Foo
// reassigning x = new Foo("goodbye") would be illegal
2 - you (or one of your libraries...) can change a val via reflection. If this happens two threads could indeed read a different value for your val
import java.lang.reflect.Field
class Foo { val foo=true } // foo is immutable
object test {
def main(args: Array[String]) {
val f = new Foo
println("foo is " + f.foo) // "foo is true"
val fld = f.getClass.getDeclaredField("foo")
fld.setAccessible(true)
fld.setBoolean(f, false)
println("foo is " + f.foo) // "foo is false"
}
}
As object members, once initialized, vals never change their values during the lifetime of the object. As such, their values are guaranteed to be visible to all threads provided that the reference to the object didn't escape in the constructor. And, in fact, they get Java final modifiers as illustrated below:
object Obj {
val r = 1
def foo {
val a = 1
def bar = a
bar
}
}
Using javap:
...
private final int r;
...
public void foo();
...
0: iconst_1
1: istore_1
2: aload_0
3: iload_1
4: invokespecial #31; //Method bar$1:(I)I
7: pop
...
private final int bar$1(int);
...
0: iload_1
1: ireturn
...
As method locals, they are used only within the method, or they're being passed to a nested method or a closure as arguments (see lifted bar$1 above). A closure might be passed on to another thread, but it will only have a final field with the value of the local val. Therefore, they are visible from the point where they are created to all other threads and synchronization is not necessary.
Note that this says nothing about the object the val points to - it itself may be mutable and warrant synchronization.
In most cases the above cannot be violated via reflection - the Scala val member declaration actually generates a getter with the same name and a private field which the getter accesses. Trying to use reflection to modify the field will result in the NoSuchFieldException. The only way you could modify it is to add a specialized annotation to your class which will make the specialized fields protected, hence accessible to reflection. I cannot currently think of any other situation that could change something declared as val...