In D, can I initialize directly on declaration and expect the initializer expressions are part of the constructor?
I came from C# and there this is the case. But with DMD 2.071.0 Im getting other behavior.
class Other { }
class Test { Other nonStaticMember = new Other; }
void test()
{
auto t1 = new Test;
auto t2 = new Test;
// Assert is failing, the two Test instances are
// being initialized to the same reference
// instead of execute the Other constructor twice.
assert(t1.nonStaticMember !is t2.nonStaticMember);
}
If this is the intented behavior it should be documented here: https://dlang.org/spec/class.html right?
This code doesn't do in D what it would do in C#.
In your example, Other is instantiated during compilation.
I.e. Other is instantiated once, during compilation, and is placed in the program's data segment. Then, nonStaticMember's initial value will, by default, point to that instance of Other, for all Test instances.
So, everything is working exactly as designed, even if it may be surprising when coming from other languages.
If this is the intented behavior it should be documented here: https://dlang.org/spec/class.html right?
Perhaps, but note that this behavior is not at all specific to classes. A pointer to any value allocated on the heap, as the initial value of a global or local static variable, will behave the same. Whenever the value of an expression is demanded during compilation (and that includes initializers for global/static variables), D attempts to evaluate it at compile-time. A few years ago, this has been extended to allocating values on the "heap" too, which then end up in the program's initial data segment.
Related
The code below can be run in a Swift Playground:
import UIKit
func aaa(_ key: UnsafeRawPointer!, _ value: Any! = nil) {
print(key)
}
func bbb(_ key: UnsafeRawPointer!) {
print(key)
}
class A {
var key = "aaa"
}
let a = A()
aaa(&a.key)
bbb(&a.key)
Here's the result printed on my mac:
0x00007fff5dce9248
0x00007fff5dce9220
Why the results of two prints differs? What's more interesting, when I change the function signature of bbb to make it the same with aaa, the result of two prints are the same. And if I use a global var instead of a.key in these two function calls, the result of two prints are the same. Does anyone knows why this strange behavior happens?
Why the results of two prints differs?
Because for each function call, Swift is creating a temporary variable initialised to the value returned by a.key's getter. Each function is called with a pointer to their given temporary variable. Therefore the pointer values will likely not be the same – as they refer to different variables.
The reason why temporary variables are used here is because A is a non-final class, and can therefore have its getters and setters of key overridden by subclasses (which could well re-implement it as a computed property).
Therefore in an un-optimised build, the compiler cannot just pass the address of key directly to the function, but instead has to rely on calling the getter (although in an optimised build, this behaviour can change completely).
You'll note that if you mark key as final, you should now get consistent pointer values in both functions:
class A {
final var key = "aaa"
}
var a = A()
aaa(&a.key) // 0x0000000100a0abe0
bbb(&a.key) // 0x0000000100a0abe0
Because now the address of key can just be directly passed to the functions, bypassing its getter entirely.
It's worth noting however that, in general, you should not rely on this behaviour. The values of the pointers you get within the functions are a pure implementation detail and are not guaranteed to be stable. The compiler is free to call the functions however it wishes, only promising you that the pointers you get will be valid for the duration of the call, and will have pointees initialised to the expected values (and if mutable, any changes you make to the pointees will be seen by the caller).
The only exception to this rule is the passing of pointers to global and static stored variables. Swift does guarantee that the pointer values you get will be stable and unique for that particular variable. From the Swift team's blog post on Interacting with C Pointers (emphasis mine):
However, interaction with C pointers is inherently
unsafe compared to your other Swift code, so care must be taken. In
particular:
These conversions cannot safely be used if the callee
saves the pointer value for use after it returns. The pointer that
results from these conversions is only guaranteed to be valid for the
duration of a call. Even if you pass the same variable, array, or
string as multiple pointer arguments, you could receive a different
pointer each time. An exception to this is global or static stored
variables. You can safely use the address of a global variable as a
persistent unique pointer value, e.g.: as a KVO context parameter.
Therefore if you made key a static stored property of A or just a global stored variable, you are guaranteed to the get same pointer value in both function calls.
Changing the function signature
When I change the function signature of bbb to make it the same with aaa, the result of two prints are the same
This appears to be an optimisation thing, as I can only reproduce it in -O builds and playgrounds. In an un-optimised build, the addition or removal of an extra parameter has no effect.
(Although it's worth noting that you should not test Swift behaviour in playgrounds as they are not real Swift environments, and can exhibit different runtime behaviour to code compiled with swiftc)
The cause of this behaviour is merely a coincidence – the second temporary variable is able to reside at the same address as the first (after the first is deallocated). When you add an extra parameter to aaa, a new variable will be allocated 'between' them to hold the value of the parameter to pass, preventing them from sharing the same address.
The same address isn't observable in un-optimised builds due to the intermediate load of a in order to call the getter for the value of a.key. As an optimisation, the compiler is able to inline the value of a.key to the call-site if it has a property initialiser with a constant expression, removing the need for this intermediate load.
Therefore if you give a.key a non-determininstic value, e.g var key = arc4random(), then you should once again observe different pointer values, as the value of a.key can no longer be inlined.
But regardless of the cause, this is a perfect example of how the pointer values for variables (which are not global or static stored variables) are not to be relied on – as the value you get can completely change depending on factors such as optimisation level and parameter count.
inout & UnsafeMutable(Raw)Pointer
Regarding your comment:
But since withUnsafePointer(to:_:) always has the correct behavior I want (in fact it should, otherwise this function is of no use), and it also has an inout parameter. So I assume there are implementation difference between these functions with inout parameters.
The compiler treats an inout parameter in a slightly different way to an UnsafeRawPointer parameter. This is because you can mutate the value of an inout argument in the function call, but you cannot mutate the pointee of an UnsafeRawPointer.
In order to make any mutations to the value of the inout argument visible to the caller, the compiler generally has two options:
Make a temporary variable initialised to the value returned by the variable's getter. Call the function with a pointer to this variable, and once the function has returned, call the variable's setter with the (possibly mutated) value of the temporary variable.
If it's addressable, simply call the function with a direct pointer to the variable.
As said above, the compiler cannot use the second option for stored properties that aren't known to be final (but this can change with optimisation). However, always relying on the first option can be potentially expensive for large values, as they'll have to be copied. This is especially detrimental for value types with copy-on-write behaviour, as they depend on being unique in order to perform direct mutations to their underlying buffer – a temporary copy violates this.
To solve this problem, Swift implements a special accessor – called materializeForSet. This accessor allows the callee to either provide the caller with a direct pointer to the given variable if it's addressable, or otherwise will return a pointer to a temporary buffer containing a copy of the variable, which will need to be written back to the setter after it has been used.
The former is the behaviour you're seeing with inout – you're getting a direct pointer to a.key back from materializeForSet, therefore the pointer values you get in both function calls are the same.
However, materializeForSet is only used for function parameters that require write-back, which explains why it's not used for UnsafeRawPointer. If you make the function parameters of aaa and bbb take UnsafeMutable(Raw)Pointers (which do require write-back), you should observe the same pointer values again.
func aaa(_ key: UnsafeMutableRawPointer) {
print(key)
}
func bbb(_ key: UnsafeMutableRawPointer) {
print(key)
}
class A {
var key = "aaa"
}
var a = A()
// will use materializeForSet to get a direct pointer to a.key
aaa(&a.key) // 0x0000000100b00580
bbb(&a.key) // 0x0000000100b00580
But again, as said above, this behaviour is not to be relied upon for variables that are not global or static.
The code below can be run in a Swift Playground:
import UIKit
func aaa(_ key: UnsafeRawPointer!, _ value: Any! = nil) {
print(key)
}
func bbb(_ key: UnsafeRawPointer!) {
print(key)
}
class A {
var key = "aaa"
}
let a = A()
aaa(&a.key)
bbb(&a.key)
Here's the result printed on my mac:
0x00007fff5dce9248
0x00007fff5dce9220
Why the results of two prints differs? What's more interesting, when I change the function signature of bbb to make it the same with aaa, the result of two prints are the same. And if I use a global var instead of a.key in these two function calls, the result of two prints are the same. Does anyone knows why this strange behavior happens?
Why the results of two prints differs?
Because for each function call, Swift is creating a temporary variable initialised to the value returned by a.key's getter. Each function is called with a pointer to their given temporary variable. Therefore the pointer values will likely not be the same – as they refer to different variables.
The reason why temporary variables are used here is because A is a non-final class, and can therefore have its getters and setters of key overridden by subclasses (which could well re-implement it as a computed property).
Therefore in an un-optimised build, the compiler cannot just pass the address of key directly to the function, but instead has to rely on calling the getter (although in an optimised build, this behaviour can change completely).
You'll note that if you mark key as final, you should now get consistent pointer values in both functions:
class A {
final var key = "aaa"
}
var a = A()
aaa(&a.key) // 0x0000000100a0abe0
bbb(&a.key) // 0x0000000100a0abe0
Because now the address of key can just be directly passed to the functions, bypassing its getter entirely.
It's worth noting however that, in general, you should not rely on this behaviour. The values of the pointers you get within the functions are a pure implementation detail and are not guaranteed to be stable. The compiler is free to call the functions however it wishes, only promising you that the pointers you get will be valid for the duration of the call, and will have pointees initialised to the expected values (and if mutable, any changes you make to the pointees will be seen by the caller).
The only exception to this rule is the passing of pointers to global and static stored variables. Swift does guarantee that the pointer values you get will be stable and unique for that particular variable. From the Swift team's blog post on Interacting with C Pointers (emphasis mine):
However, interaction with C pointers is inherently
unsafe compared to your other Swift code, so care must be taken. In
particular:
These conversions cannot safely be used if the callee
saves the pointer value for use after it returns. The pointer that
results from these conversions is only guaranteed to be valid for the
duration of a call. Even if you pass the same variable, array, or
string as multiple pointer arguments, you could receive a different
pointer each time. An exception to this is global or static stored
variables. You can safely use the address of a global variable as a
persistent unique pointer value, e.g.: as a KVO context parameter.
Therefore if you made key a static stored property of A or just a global stored variable, you are guaranteed to the get same pointer value in both function calls.
Changing the function signature
When I change the function signature of bbb to make it the same with aaa, the result of two prints are the same
This appears to be an optimisation thing, as I can only reproduce it in -O builds and playgrounds. In an un-optimised build, the addition or removal of an extra parameter has no effect.
(Although it's worth noting that you should not test Swift behaviour in playgrounds as they are not real Swift environments, and can exhibit different runtime behaviour to code compiled with swiftc)
The cause of this behaviour is merely a coincidence – the second temporary variable is able to reside at the same address as the first (after the first is deallocated). When you add an extra parameter to aaa, a new variable will be allocated 'between' them to hold the value of the parameter to pass, preventing them from sharing the same address.
The same address isn't observable in un-optimised builds due to the intermediate load of a in order to call the getter for the value of a.key. As an optimisation, the compiler is able to inline the value of a.key to the call-site if it has a property initialiser with a constant expression, removing the need for this intermediate load.
Therefore if you give a.key a non-determininstic value, e.g var key = arc4random(), then you should once again observe different pointer values, as the value of a.key can no longer be inlined.
But regardless of the cause, this is a perfect example of how the pointer values for variables (which are not global or static stored variables) are not to be relied on – as the value you get can completely change depending on factors such as optimisation level and parameter count.
inout & UnsafeMutable(Raw)Pointer
Regarding your comment:
But since withUnsafePointer(to:_:) always has the correct behavior I want (in fact it should, otherwise this function is of no use), and it also has an inout parameter. So I assume there are implementation difference between these functions with inout parameters.
The compiler treats an inout parameter in a slightly different way to an UnsafeRawPointer parameter. This is because you can mutate the value of an inout argument in the function call, but you cannot mutate the pointee of an UnsafeRawPointer.
In order to make any mutations to the value of the inout argument visible to the caller, the compiler generally has two options:
Make a temporary variable initialised to the value returned by the variable's getter. Call the function with a pointer to this variable, and once the function has returned, call the variable's setter with the (possibly mutated) value of the temporary variable.
If it's addressable, simply call the function with a direct pointer to the variable.
As said above, the compiler cannot use the second option for stored properties that aren't known to be final (but this can change with optimisation). However, always relying on the first option can be potentially expensive for large values, as they'll have to be copied. This is especially detrimental for value types with copy-on-write behaviour, as they depend on being unique in order to perform direct mutations to their underlying buffer – a temporary copy violates this.
To solve this problem, Swift implements a special accessor – called materializeForSet. This accessor allows the callee to either provide the caller with a direct pointer to the given variable if it's addressable, or otherwise will return a pointer to a temporary buffer containing a copy of the variable, which will need to be written back to the setter after it has been used.
The former is the behaviour you're seeing with inout – you're getting a direct pointer to a.key back from materializeForSet, therefore the pointer values you get in both function calls are the same.
However, materializeForSet is only used for function parameters that require write-back, which explains why it's not used for UnsafeRawPointer. If you make the function parameters of aaa and bbb take UnsafeMutable(Raw)Pointers (which do require write-back), you should observe the same pointer values again.
func aaa(_ key: UnsafeMutableRawPointer) {
print(key)
}
func bbb(_ key: UnsafeMutableRawPointer) {
print(key)
}
class A {
var key = "aaa"
}
var a = A()
// will use materializeForSet to get a direct pointer to a.key
aaa(&a.key) // 0x0000000100b00580
bbb(&a.key) // 0x0000000100b00580
But again, as said above, this behaviour is not to be relied upon for variables that are not global or static.
I just read that the init method can't be used as a value. Meaning:
var x = SomeClass.someClassFunction // ok
var y = SomeClass.init // error
Example found on Language reference
Why should it be like that? Is it a way to enforce language level that too dirty tricks come into place, because of some cohertion or maybe because it interferes with another feature?
Unlike Obj-C, where the init function can be called multiple times without problems, in Swift there actually is no method called init.
init is just a keyword meaning "the following is a constructor". The constructor is called always via MyClass() during the creation of a new instance. It's never called separately as a method myInstance.init(). You can't get a reference to the underlying function because it would be impossible to call it.
This is also connected with the fact that constructors cannot be inherited. Code
var y = SomeClass.init
would also break subtyping because the subtypes are not required to have the same initializers.
Why should it be like that?
init is a special member, not a regular method.
Beyond that, there's no reason that you'd ever need to store init in a variable. The only objects that could use that function in a valid way are instances of the class where that particular init is defined, and any such object will have already been initialized before you could possibly make the assignment.
Initializers don't have a return value. In order to assign it to something, it should be able to return something - and it doesn't.
Note that this question and similar ones have been asked before, such as in Forward References - why does this code compile?, but I found the answers to still leave some questions open, so I'm having another go at this issue.
Within methods and functions, the effect of the val keyword appears to be lexical, i.e.
def foo {
println(bar)
val bar = 42
}
yielding
error: forward reference extends over definition of value bar
However, within classes, the scoping rules of val seem to change:
object Foo {
def foo = bar
println(bar)
val bar = 42
}
Not only does this compile, but also the println in the constructor will yield 0 as its output, while calling foo after the instance is fully constructed will result in the expected value 42.
So it appears to be possible for methods to forward-reference instance values, which will, eventually, be initialised before the method can be called (unless, of course, you're calling it from the constructor), and for statements within the constructor to forward-reference values in the same way, accessing them before they've been initialised, resulting in a silly arbitrary value.
From this, a couple of questions arise:
Why does val use its lexical compile-time effect within constructors?
Given that a constructor is really just a method, this seems rather inconsistent to entirely drop val's compile-time effect, giving it its usual run-time effect only.
Why does val, effectively, lose its effect of declaring an immutable value?
Accessing the value at different times may result in different results. To me, it very much seems like a compiler implementation detail leaking out.
What might legitimate usecases for this look like?
I'm having a hard time coming up with an example that absolutely requires the current semantics of val within constructors and wouldn't easily be implementable with a proper, lexical val, possibly in combination with lazy.
How would one work around this behaviour of val, getting back all the guarantees one is used to from using it within other methods?
One could, presumably, declare all instance vals to be lazy in order to get back to a val being immutable and yielding the same result no matter how they are accessed and to make the compile-time effect as observed within regular methods less relevant, but that seems like quite an awful hack to me for this sort of thing.
Given that this behaviour unlikely to ever change within the actual language, would a compiler plugin be the right place to fix this issue, or is it possible to implement a val-alike keyword with, for someone who just spent an hour debugging an issue caused by this oddity, more sensible semantics within the language?
Only a partial answer:
Given that a constructor is really just a method ...
It isn't.
It doesn't return a result and doesn't declare a return type (or doesn't have a name)
It can't be called again for an object of said class like "foo".new ("bar")
You can't hide it from an derived class
You have to call them with 'new'
Their name is fixed by the name of the class
Ctors look a little like methods from the syntax, they take parameters and have a body, but that's about all.
Why does val, effectively, lose its effect of declaring an immutable value?
It doesn't. You have to take an elementary type which can't be null to get this illusion - with Objects, it looks different:
object Foo {
def foo = bar
println (bar.mkString)
val bar = List(42)
}
// Exiting paste mode, now interpreting.
defined module Foo
scala> val foo=Foo
java.lang.NullPointerException
You can't change a val 2 times, you can't give it a different value than null or 0, you can't change it back, and a different value is only possible for the elementary types. So that's far away from being a variable - it's a - maybe uninitialized - final value.
What might legitimate usecases for this look like?
I guess working in the REPL with interactive feedback. You execute code without an explicit wrapping object or class. To get this instant feedback, it can't be waited until the (implicit) object gets its closing }. Therefore the class/object isn't read in a two-pass fashion where firstly all declarations and initialisations are performed.
How would one work around this behaviour of val, getting back all the guarantees one is used to from using it within other methods?
Don't read attributes in the Ctor, like you don't read attributes in Java, which might get overwritten in subclasses.
update
Similar problems can occur in Java. A direct access to an uninitialized, final attribute is prevented by the compiler, but if you call it via another method:
public class FinalCheck
{
final int foo;
public FinalCheck ()
{
// does not compile:
// variable foo might not have been initialized
// System.out.println (foo);
// Does compile -
bar ();
foo = 42;
System.out.println (foo);
}
public void bar () {
System.out.println (foo);
}
public static void main (String args[])
{
new FinalCheck ();
}
}
... you see two values for foo.
0
42
I don't want to excuse this behaviour, and I agree, that it would be nice, if the compiler could warn consequently - in Java and Scala.
So it appears to be possible for methods to forward-reference instance
values, which will, eventually, be initialised before the method can
be called (unless, of course, you're calling it from the constructor),
and for statements within the constructor to forward-reference values
in the same way, accessing them before they've been initialised,
resulting in a silly arbitrary value.
A constructor is a constructor. You are constructing the object. All of its fields are initialized by JVM (basically, zeroed), and then the constructor fills in whatever fields needs filling in.
Why does val use its lexical compile-time effect within constructors?
Given that a constructor is really just a method, this seems rather
inconsistent to entirely drop val's compile-time effect, giving it its
usual run-time effect only.
I have no idea what you are saying or asking here, but a constructor is not a method.
Why does val, effectively, lose its effect of declaring an immutable value?
Accessing the value at different times may result in different
results. To me, it very much seems like a compiler implementation
detail leaking out.
It doesn't. If you try to modify bar from the constructor, you'll see it is not possible. Accessing the value at different times in the constructor may result in different results, of course.
You are constructing the object: it starts not constructed, and ends constructed. For it not to change it would have to start out with its final value, but how can it do that without someone assigning that value?
Guess who does that? The constructor.
What might legitimate usecases for this look like?
I'm having a hard time coming up with an example that absolutely
requires the current semantics of val within constructors and wouldn't
easily be implementable with a proper, lexical val, possibly in
combination with lazy.
There's no use case for accessing the val before its value has been filled in. It's just impossible to find out whether it has been initialized or not. For example:
class Foo {
println(bar)
val bar = 10
}
Do you think the compiler can guarantee it has not been initialized? Well, then open the REPL, put in the above class, and then this:
class Bar extends { override val bar = 42 } with Foo
new Bar
And see that bar was initialized when printed.
How would one work around this behaviour of val, getting back all the
guarantees one is used to from using it within other methods?
Declare your vals before using them. But note that constuctor is not a method. When you do:
println(bar)
inside a constructor, you are writing:
println(this.bar)
And this, the object of the class you are writing a constructor for, has a bar getter, so it is called.
When you do the same thing on a method where bar is a definition, there's no this with a bar getter.
How does an assembler handle classes and objects when a programme is compiled? And how is this stored in RAM and the executable file?
At first, memory is allocated according to the class' size, for example 20 bytes. In these 20 bytes, all variables of the class are stored. But there are two problems:
How do pointers to classes work?
How does a called method know which object it belongs to?
Could you explain this to me? (If using example code, I prefer Objective-C, C++ and x86 Assembly)
The assembler has no clue what a class is, it only assembles machine code, which the occasional macro tossed in. For all intents and purposes a class is merly a struct with an optional vftable, with all the handling and class 'special features' (virtualism, polymorphism, inheiritanc etc) being done in the intermediate stage, when IR code is created. Memory would be allocated the same as a struct, variable, array or any other data 'blob' (statically or dynamically, taking alignment, const'ness and packing into account), except for the support code to handle stack & static based dtor unwinding(done again at the IR level), ctors, and static initialization(though static initialization can happen for more than class objects). I suggest you give the dragon book a read through (the first eight chapters would cover it), to get a clearer picture of how a compiler and assembler work, seeing as these things are not handled by the assembler, but by the compiler front and/or back ends, depending on how the compiler an its IL are structured.
(2) Member functions are rewritten by the compiler. Consider class A as follows.
class A {
int i;
public:
A () : i (0) { }
void f (int a, char *b) { i = a; }
}
Then what the compiler makes of A::f looks something like this (pseudocode):
void A::f (A * const this, int a, char *b) { this->i = a; }
Consider now a call to A::f.
A a;
a.f (25, "");
The compiler generates code similar to this (pseudocode):
A a;
A::f (&a, 25, "");
In other words, the compiler works the hidden this pointer into every non-static member function and each call receives a reference to the instance that it was called upon. This, in particular, means that you can invoke non-static member functions on NULL pointers.
A *a = NULL;
a->f (25, "");
The crash only occurs when you actually reference non-static class member variables.
The resulting crash report also illustrates the answer to question (1). In many cases, you'll not crash on address 0, but an offset of that. That offset corresponds to the offset that the accessed variable has in the memory layout the compiler chose for class A (in this case, many compilers will actually offset it with 0x0 and class A will in memory not be distinguishable from struct A { int i; };). Basically, pointers to classes are pointers to the equivalent C struct. Member functions are implemented as free functions taking an instance reference as first argument. All and any access checks with regard to public, protected and private members is done upfront by the compiler, the generated assembly has no clue about any of those concepts. In fact, early versions of C++ are said to have been sets of clever C macros.
The memory layout (typically) changes a bit when you have virtual functions.