import Foundation
func address(o:UnsafeRawPointer) -> Int {
return Int(bitPattern: o)
}
var originArray = [1,2,3]
var firstArray = originArray
//q.append(4)
print(NSString.init(format: "originArray:%p", address(o: &originArray)))
print(NSString.init(format: "firstArray:%p", address(o: &firstArray)))
Debug Log:
originArray:0x100b087b0
firstArray:0x100b088c0
Above, it's my test code.I think that I don't modify originArray which like appending or reduceing element. they should point same address.but why deference
Your code is printing the addresses of the array buffers (Array is a special case when passing a value to a pointer parameter). However, in Swift 3, the compiler assumed that the presence of the & operator meant that the buffer was being passed as mutable memory, so (unnecessarily) made it unique (by copying) before passing its pointer value, despite that pointer value being passed as an UnsafeRawPointer. That's why you see different addresses.
If you remove the & operator and pass the arrays directly:
func address(_ p: UnsafeRawPointer) {
print(p)
}
var originArray = [1, 2, 3]
var firstArray = originArray
address(originArray) // 0x00000000016e71c0
address(firstArray) // 0x00000000016e71c0
You'll now get the same addresses, as the compiler now assumes that address(_:) will not modify the memory of the buffers passed, as they're being passed to an UnsafeRawPointer parameter.
In Swift 4, this inconsistency is fixed, and the compiler no longer makes the buffer unique before passing its pointer values to an UnsafeRawPointer parameter, even when using the & operator, so your code exhibits expected behaviour.
Although, it's worth noting that the above method isn't guaranteed to produce stable pointer values when passing the same array to multiple pointer parameters.
From the Swift blog post "Interacting with C Pointers":
Even if you pass the same variable, array, or string as multiple pointer arguments, you could receive a different pointer each time.
I believe this guarantee cannot be met for arrays in two cases (there may be more):
If the array is viewing elements in non-contiguous storage
Swift's Array can view elements in non-contiguous storage, for example when it is wrapping an NSArray. In such a case, when passing it to a pointer parameter, a new contiguous buffer will have to be created, therefore giving you a different pointer value.
If the buffer is non-uniquely referenced when passed as mutable memory
As mentioned earlier, when passing an array to a mutable pointer parameter, its buffer will first be made unique in order to preserve value semantics, as it's assumed the function will perform a mutation of the buffer.
Therefore, if the buffer was copied, you'll get a different pointer value to if you had passed the array to an immutable pointer parameter.
Although neither of these two points are applicable in the example you give, it's worth bearing in mind that the compiler still doesn't guarantee you stable pointer values to the array's buffer when passing to pointer parameters.
For results that are guaranteed to be reliable, you should use the withUnsafeBytes(_:) method on a ContiguousArray:
var originArray: ContiguousArray = [1, 2, 3]
var firstArray = originArray
originArray.withUnsafeBytes { print($0.baseAddress!) } // 0x0000000102829550
firstArray.withUnsafeBytes { print($0.baseAddress!) } // 0x0000000102829550
This is because withUnsafeBytes(_:) is documented as accepting:
A closure with an UnsafeRawBufferPointer parameter that points to the contiguous storage for the array. If no such storage exists, it is created.
And ContiguousArray guarantees that:
[it] always stores its elements in a contiguous region of memory
And just like Array, ContiguousArray uses copy-on-write in order to have value semantics, so you can still use it to check when the array's buffer is copied upon a mutation taking place:
var originArray: ContiguousArray = [1, 2, 3]
var firstArray = originArray
originArray.withUnsafeBytes { print($0.baseAddress!) } // 0x0000000103103eb0
firstArray.withUnsafeBytes { print($0.baseAddress!) } // 0x0000000103103eb0
firstArray[0] = 4
originArray.withUnsafeBytes { print($0.baseAddress!) } // 0x0000000103103eb0
firstArray.withUnsafeBytes { print($0.baseAddress!) } // 0x0000000100e764d0
You're printing the address of the variable itself, not the address of the array buffer it's pointing to.
You can get the address of the arrays' buffers like so:
var originArray = [1, 2, 3]
var firstArray = originArray
print("originArray: \(originArray.withUnsafeBytes { $0.baseAddress! })")
print("firstArray: \(firstArray.withUnsafeBytes { $0.baseAddress! })")
Now the same value is printed, unless you modify one of the arrays.
Related
I'm experimenting with iteration on an array using a for .. in .. loop. My question is related to the case where the collection is changed within the loop body.
It seems that the iteration is safe, even if the list shrinks in the meantime. The for iteration variables successively take the values of the (indexes and) elements that were in the array at the start of the loop, despite the changes made on the flow. Example:
var slist = [ "AA", "BC", "DE", "FG" ]
for (i, st) in slist.enumerated() { // for st in slist gives a similar result
print ("Index \(i): \(st)")
if st == "AA" { // at one iteration change completely the list
print (" --> check 0: \(slist[0]), and 2: \(slist[2])")
slist.append ("KLM")
slist.insert(st+"XX", at:0) // shift the elements in the array
slist[2]="bc" // replace some elements to come
print (" --> check again 0: \(slist[0]), and 2: \(slist[2])")
slist.remove(at:3)
slist.remove(at:3)
slist.remove(at:1) // makes list shorter
}
}
print (slist)
This works very well, the iteration being made on the values [ "AA", "BC", "DE", "FG" ] even if after the first iteration the array is completely changed to ["AAXX", "bc", "KLM"]
I wanted to know if I can safely rely on this behavior. Unfortunately, the language guide does not tell anything about iterating on a collection when the collection is modified. And the for .. in section doesn't address this question either. So:
Can I safely rely on a guarantee about this iteration behavior provided in the language specifications ?
Or am I simply lucky with the current version of Swift 5.4? In this case, is there any clue in the language specification that one cannot take it for granted? And is there a performance overhead for this iteration behavior (e.g. some copy) compared to indexed iteration?
The documentation for IteratorProtocol says "whenever you use a for-in loop with an array, set, or any other collection or sequence, you’re using that type’s iterator." So, we are guaranteed that a for in loop is going to be using .makeIterator() and .next() which is defined most generally on Sequence and IteratorProtocol respectively.
The documentation for Sequence says that "the Sequence protocol makes no requirement on conforming types regarding whether they will be destructively consumed by iteration." As a consequence, this means that an iterator for a Sequence is not required to make a copy, and so I do not think that modifying a sequence while iterating over it is, in general, safe.
This same caveat does not occur in the documentation for Collection, but I also don't think there is any guarantee that the iterator makes a copy, and so I do not think that modifying a collection while iterating over it is, in general, safe.
But, most collection types in Swift are structs with value semantics or copy-on-write semantics. I'm not really sure where the documentation for this is, but this link does say that "in Swift, Array, String, and Dictionary are all value types... You don’t need to do anything special — such as making an explicit copy — to prevent other code from modifying that data behind your back." In particular, this means that for Array, .makeIterator() cannot hold a reference to your array because the iterator for Array does not have to "do anything special" to prevent other code (i.e. your code) from modifying the data it holds.
We can explore this in more detail. The Iterator type of Array is defined as type IndexingIterator<Array<Element>>. The documentation IndexingIterator says that it is the default implementation of the iterator for collections, so we can assume that most collections will use this. We can see in the source code for IndexingIterator that it holds a copy of its collection
#frozen
public struct IndexingIterator<Elements: Collection> {
#usableFromInline
internal let _elements: Elements
#usableFromInline
internal var _position: Elements.Index
#inlinable
#inline(__always)
/// Creates an iterator over the given collection.
public /// #testable
init(_elements: Elements) {
self._elements = _elements
self._position = _elements.startIndex
}
...
}
and that the default .makeIterator() simply creates this copy.
extension Collection where Iterator == IndexingIterator<Self> {
/// Returns an iterator over the elements of the collection.
#inlinable // trivial-implementation
#inline(__always)
public __consuming func makeIterator() -> IndexingIterator<Self> {
return IndexingIterator(_elements: self)
}
}
Although you might not want to trust this source code, the documentation for library evolution claims that "the #inlinable attribute is a promise from the library developer that the current definition of a function will remain correct when used with future versions of the library" and the #frozen also means that the members of IndexingIterator cannot change.
Altogether, this means that any collection type with value semantics and an IndexingIterator as its Iterator must make a copy when using using for in loops (at least until the next ABI break, which should be a long-way off). Even then, I don't think Apple is likely to change this behavior.
In Conclusion
I don't know of any place that it is explicitly spelled out in the docs "you can modify an array while you iterate over it, and the iteration will proceed as if you made a copy" but that's also the kind of language that probably shouldn't be written down as writing such code could definitely confuse a beginner.
However, there is enough documentation lying around which says that a for in loop just calls .makeIterator() and that for any collection with value semantics and the default iterator type (for example, Array), .makeIterator() makes a copy and so cannot be influenced by code inside the loop. Further, because Array and some other types like Set and Dictionary are copy-on-write, modifying these collections inside a loop will have a one-time copy penalty as the body of the loop will not have a unique reference to its storage (because the iterator will). This is the exact same penalty that modifying the collection outside the loop with have if you don’t have a unique reference to the storage.
Without these assumptions, you aren't guaranteed safety, but you might have it anyway in some circumstances.
Edit:
I just realized we can create some cases where this is unsafe for sequences.
import Foundation
/// This is clearly fine and works as expected.
print("Test normal")
for _ in 0...10 {
let x: NSMutableArray = [0,1,2,3]
for i in x {
print(i)
}
}
/// This is also okay. Reassigning `x` does not mutate the reference that the iterator holds.
print("Test reassignment")
for _ in 0...10 {
var x: NSMutableArray = [0,1,2,3]
for i in x {
x = []
print(i)
}
}
/// This crashes. The iterator assumes that the last index it used is still valid, but after removing the objects, there are no valid indices.
print("Test removal")
for _ in 0...10 {
let x: NSMutableArray = [0,1,2,3]
for i in x {
x.removeAllObjects()
print(i)
}
}
/// This also crashes. `.enumerated()` gets a reference to `x` which it expects will not be modified behind its back.
print("Test removal enumerated")
for _ in 0...10 {
let x: NSMutableArray = [0,1,2,3]
for i in x.enumerated() {
x.removeAllObjects()
print(i)
}
}
The fact that this is an NSMutableArray is important because this type has reference semantics. Since NSMutableArray conforms to Sequence, we know that mutating a sequence while iterating over it is not safe, even when using .enumerated().
The slist.enumerate() create a new instance of EnumeratedSequence<[String]>
To create an instance of EnumeratedSequence, call enumerated() on a sequence or collection. The following example enumerates the elements of an array. reference
If you remove the .enumerate() produce the same result, any st has the old value. This occurs because the for-in loop generates a new instance of IndexingIterator<[String]>.
Whenever you use a for-in loop with an array, set, or any other collection or sequence, you’re using that type’s iterator. Swift uses a sequence’s or collection’s iterator internally to enable the for-in loop language construct. reference
About the questions:
You would be able to remove all the elements and still perform the loop safe because a new instance is generated to perform the interactions.
Swift uses the iterator internally to enable for-in then there's no overhead to compare. Logically that the larger the array the performance will be affected.
For example, without this second overload, loading an Array will yield "UnsafeRawBufferPointer.load out of bounds". Is there a way to handle both cases without overloads?
let bytes: [UInt8] = [1, 0, 1, 0]
bytes.load() as Int32 // 0x1_00_01
bytes.load() as [Int16] // [1, 1]
public extension ContiguousBytes {
func load<T>(_: T.Type = T.self) -> T {
withUnsafeBytes { $0.load(as: T.self) }
}
func load<Element>(_: [Element].Type = [Element].self) -> [Element] {
withUnsafeBytes { .init($0.bindMemory(to: Element.self)) }
}
}
In terms of having two overloads: you unfortunately can't avoid them. ContiguousBytes.withUnsafeBytes gives you access to the underlying storage of a type which implements ContiguousBytes, so the buffer given by, say, [UInt8].withUnsafeBytes will be the actual buffer the array instance is using to store data, not a pointer to itself in memory (e.g. its length, capacity, storage pointer, etc.).
The reason your first overload can't work is that you'd be calling the equivalent of:
withUnsafeBytes { rawBuffer in
rawBuffer.load(as: [Int16].self)
}
where load(as:) is attempting to read [1, 0, 1, 0] as if those contents were the length, capacity, and buffer pointer of a single [Int16] instance, but that's not the case — it's only the contents of bytes.
You need the second overload in order to create a new array with the contents of that UnsafeRawBufferPointer.
As for the actual implementation of your array variant, as of SE-0333 Expand usability of withMemoryRebound coming in Swift 5.7, the correct way to do this would be to use withMemoryRebound(to:_:), as the unconditional bindMemory(to:) is unsafe if the buffer was already bound to a different type. withMemoryRebound(to:_:) is now always safe, even if the buffer is bound, so long as you meet alignment requirements:
/// A raw buffer may represent memory that has been bound to a type.
/// If that is the case, then T must be layout compatible with the
/// type to which the memory has been bound. This requirement does not
/// apply if the raw buffer represents memory that has not been bound
/// to any type.
This would look like
func load<Element>(_: [Element].Type = [Element].self) -> [Element] {
withUnsafeBytes { rawBuffer in
rawBuffer.withMemoryRebound(to: Element.self) { typedBuffer in
Array(typedBuffer)
}
}
}
UnsafeRawBufferPointer.withMemoryRebound(to:_:) will do the math on binding the raw bytes to the right stride of Element, though keep in mind that this is only valid to do if the alignment of the raw buffer matches the alignment requirements of Element — otherwise you'll either crash or trigger undefined behavior.
Since Xcode 11.4 I get the warning message "Initialization of 'UnsafeMutableRawPointer' results in a dangling pointer"
for the following code where I read an SIMD4 from an MTLTexture into an array:
let texArray = Array<SIMD4<Float>>(repeating: SIMD4<Float>(repeating: 0), count: 1)
texture.getBytes(UnsafeMutableRawPointer(mutating: texArray), bytesPerRow: (MemoryLayout<SIMD4<Float>>.size * texture.width), from: region, mipmapLevel: 0)
Can somebody figure out how to create the array pointer to get rid of the warning ?
Thanks
TLDR
make text array mutable (use var instead of let) and use withUnsafeMutableBytes
var texArray = Array<SIMD4<Float>>(repeating: SIMD4<Float>(repeating: 0), count: 1)
texArray.withUnsafeMutableBytes { texArrayPtr in
texture.getBytes(texArrayPtr.baseAddress!, bytesPerRow: (MemoryLayout<SIMD4<Float>>.size * texture.width), from: region, mipmapLevel: 0)
}
Explanation
The warning was introduced because the compiler can't make sure that data backing the pointer will not be deallocated. Consider you have a function (e.g. implemented in C) manipulating some data pointed to.
func f(_ a: UnsafeMutablePointer<Int>){
a[0] = 42
}
Then it must be made sure that memory was not deallocated until the end of the call. So when calling this function in the following way it is not safe
var a: = [1]
p: UnsafeMutablePointer<Int>(&a)
// at this point the compiler may optimise and deallocate 'a' since it is not used anymore
f(p)
Currently this won't be an issue as far as I know since local variables will not be deallocated before the end of the scope.
One can illustrate the possible issue by introducing a nested scope in the following way
var p: UnsafeMutablePointer<Int>?
do {
var a = [1]
p = UnsafeMutablePointer<Int>(&a)
} // "a" will be deallocated here
// now "p" is a dangling pointer the compiler warned you of
var b = [0] // compiler will use same memory as for "a", so manipulating memory of "p" won't segfault
f(p!) // manipulate memory
print(b[0]) // prints 42 although "b" was initialised to 0
Due to the fact that b allocates the same memory that a was using before, the memory of b is modified by the call to f(p!). So b[0] is 42 although it was initialised to 0 and not explicitly modified.
With this illustration it should become clear why there are methods withUnsafeMutableBytes and withUnsafeMutableBufferPointer on Swift arrays and global functions withUnsafeMutablePointer plus immutable variants. (I personally find it confusing that methods must be used on arrays and and global functions on structs.)
These functions make sure that memory is not deallocated (or reused) for the scope of the closure (I also created gist with some examples).
I'm currently reading the excellent Advanced Swift book from objc.io, and I'm running into something that I don't understand.
If you run the following code in a playground, you will notice that when modifying a struct contained in a dictionary a copy is made by the subscript access, but then it appears that the original value in the dictionary is replaced by the copy. I don't understand why. What exactly is happening ?
Also, is there a way to avoid the copy ? According to the author of the book, there isn't, but I just want to be sure.
import Foundation
class Buffer {
let id = UUID()
var value = 0
func copy() -> Buffer {
let new = Buffer()
new.value = self.value
return new
}
}
struct COWStruct {
var buffer = Buffer()
init() { print("Creating \(buffer.id)") }
mutating func change() -> String {
if isKnownUniquelyReferenced(&buffer) {
buffer.value += 1
return "No copy \(buffer.id)"
} else {
let newBuffer = buffer.copy()
newBuffer.value += 1
buffer = newBuffer
return "Copy \(buffer.id)"
}
}
}
var array = [COWStruct()]
array[0].buffer.value
array[0].buffer.id
array[0].change()
array[0].buffer.value
array[0].buffer.id
var dict = ["key": COWStruct()]
dict["key"]?.buffer.value
dict["key"]?.buffer.id
dict["key"]?.change()
dict["key"]?.buffer.value
dict["key"]?.buffer.id
// If the above `change()` was made on a copy, why has the original value changed ?
// Did the copied & modified struct replace the original struct in the dictionary ?
dict["key"]?.change() // Copy
is semantically equivalent to:
if var value = dict["key"] {
value.change() // Copy
dict["key"] = value
}
The value is pulled out of the dictionary, unwrapped into a temporary, mutated, and then placed back into the dictionary.
Because there's now two references to the underlying buffer (one from our local temporary value, and one from the COWStruct instance in the dictionary itself) – we're forcing a copy of the underlying Buffer instance, as it's no longer uniquely referenced.
So, why doesn't
array[0].change() // No Copy
do the same thing? Surely the element should be pulled out of the array, mutated and then stuck back in, replacing the previous value?
The difference is that unlike Dictionary's subscript which comprises of a getter and setter, Array's subscript comprises of a getter and a special accessor called mutableAddressWithPinnedNativeOwner.
What this special accessor does is return a pointer to the element in the array's underlying buffer, along with an owner object to ensure that the buffer isn't deallocated from under the caller. Such an accessor is called an addressor, as it deals with addresses.
Therefore when you say:
array[0].change()
you're actually mutating the actual element in the array directly, rather than a temporary.
Such an addressor cannot be directly applied to Dictionary's subscript because it returns an Optional, and the underlying value isn't stored as an optional. So it currently has to be unwrapped with a temporary, as we cannot return a pointer to the value in storage.
In Swift 3, you can avoid copying your COWStruct's underlying Buffer by removing the value from the dictionary before mutating the temporary:
if var value = dict["key"] {
dict["key"] = nil
value.change() // No Copy
dict["key"] = value
}
As now only the temporary has a view onto the underlying Buffer instance.
And, as #dfri points out in the comments, this can be reduced down to:
if var value = dict.removeValue(forKey: "key") {
value.change() // No Copy
dict["key"] = value
}
saving on a hashing operation.
Additionally, for convenience, you may want to consider making this into an extension method:
extension Dictionary {
mutating func withValue<R>(
forKey key: Key, mutations: (inout Value) throws -> R
) rethrows -> R? {
guard var value = removeValue(forKey: key) else { return nil }
defer {
updateValue(value, forKey: key)
}
return try mutations(&value)
}
}
// ...
dict.withValue(forKey: "key") {
$0.change() // No copy
}
In Swift 4, you should be able to use the values property of Dictionary in order to perform a direct mutation of the value:
if let index = dict.index(forKey: "key") {
dict.values[index].change()
}
As the values property now returns a special Dictionary.Values mutable collection that has a subscript with an addressor (see SE-0154 for more info on this change).
However, currently (with the version of Swift 4 that ships with Xcode 9 beta 5), this still makes a copy. This is due to the fact that both the Dictionary and Dictionary.Values instances have a view onto the underlying buffer – as the values computed property is just implemented with a getter and setter that passes around a reference to the dictionary's buffer.
So when calling the addressor, a copy of the dictionary's buffer is triggered, therefore leading to two views onto COWStruct's Buffer instance, therefore triggering a copy of it upon change() being called.
I have filed a bug over this here. (Edit: This has now been fixed on master with the unofficial introduction of generalised accessors using coroutines, so will be fixed in Swift 5 – see below for more info).
In Swift 4.1, Dictionary's subscript(_:default:) now uses an addressor, so we can efficiently mutate values so long as we supply a default value to use in the mutation.
For example:
dict["key", default: COWStruct()].change() // No copy
The default: parameter uses #autoclosure such that the default value isn't evaluated if it isn't needed (such as in this case where we know there's a value for the key).
Swift 5 and beyond
With the unofficial introduction of generalised accessors in Swift 5, two new underscored accessors have been introduced, _read and _modify which use coroutines in order to yield a value back to the caller. For _modify, this can be an arbitrary mutable expression.
The use of coroutines is exciting because it means that a _modify accessor can now perform logic both before and after the mutation. This allows them to be much more efficient when it comes to copy-on-write types, as they can for example deinitialise the value in storage while yielding a temporary mutable copy of the value that's uniquely referenced to the caller (and then reinitialising the value in storage upon control returning to the callee).
The standard library has already updated many previously inefficient APIs to make use of the new _modify accessor – this includes Dictionary's subscript(_:) which can now yield a uniquely referenced value to the caller (using the deinitialisation trick I mentioned above).
The upshot of these changes means that:
dict["key"]?.change() // No copy
will be able to perform an mutation of the value without having to make a copy in Swift 5 (you can even try this out for yourself with a master snapshot).
After reading:
https://github.com/rodionovd/SWRoute/wiki/Function-hooking-in-Swift
https://github.com/rodionovd/SWRoute/blob/master/SWRoute/rd_get_func_impl.c
I understood that Swift function pointer is wrapped by swift_func_wrapper and swift_func_object (according to the article in 2014).
I guess this still works in Swift 3, but I couldn't find which file in https://github.com/apple/swift best describes these structs.
Can anyone help me?
I believe these details are mainly part of the implementation of Swift's IRGen – I don't think you'll find any friendly structs in the source showing you the full structure of various Swift function values. Therefore if you want to do some digging into this, I would recommend examining the IR emitted by the compiler.
You can do this by running the command:
xcrun swiftc -emit-ir main.swift | xcrun swift-demangle > main.irgen
which will emit the IR (with demangled symbols) for a -Onone build. You can find the documentation for LLVM IR here.
The following is some interesting stuff that I've been able to learn from going through the IR myself in a Swift 3.1 build. Note that this is all subject to change in future Swift versions (at least until Swift is ABI stable). It goes without saying that the code examples given below are only for demonstration purposes; and shouldn't ever be used in actual production code.
Thick function values
At a very basic level, function values in Swift are simple things – they're defined in the IR as:
%swift.function = type { i8*, %swift.refcounted* }
which is the raw function pointer i8*, along with a pointer to its context %swift.refcounted*, where %swift.refcounted is defined as:
%swift.refcounted = type { %swift.type*, i32, i32 }
which is the structure of a simple reference-counted object, containing a pointer to the object's metadata, along with two 32 bit values.
These two 32 bit values are used for the reference count of the object. Together , they can either represent (as of Swift 4):
The strong and unowned reference count of the object + some flags, including whether the object uses native Swift reference counting (as opposed to Obj-C reference counting), and whether the object has a side table.
or
A pointer to a side table, which contains the above, plus the weak reference count of the object (on forming a weak reference to an object, if it doesn't already have a side table, one will be created).
For further reading on the internals of Swift reference counting, Mike Ash has a great blog post on the subject.
The context of a function usually adds extra values onto the end of this %swift.refcounted structure. These values are dynamic things that the function needs upon being called (such as any values that it has captured, or any parameters that it has been partially applied with). In quite a few cases, function values won't need a context, so the pointer to the context will simply be nil.
When the function comes to be called, Swift will simply pass in the context as the last parameter. If the function doesn't have a context parameter, the calling convention appears to allow it to be safely passed anyway.
The storing of the function pointer along with the context pointer is called a thick function value, and is how Swift usually stores function values of known type (as opposed to a thin function value which is just the function pointer).
So, this explains why MemoryLayout<(Int) -> Int>.size returns 16 bytes – because it's made up of two pointers (each being a word in length, i.e 8 bytes on a 64 bit platform).
When thick function values are passed into function parameters (where those parameters are of non-generic type), Swift appears to pass the raw function pointer and context as separate parameters.
Capturing values
When a closure captures a value, this value will be put into a heap-allocated box (although the value itself can get stack-promoted in the case of a non-escaping closure – see later section). This box will be available to the function through the context object (the relevant IR).
For a closure that just captures a single value, Swift just makes the box itself the context of the function (no need for extra indirection). So you'll have a function value which looks like a ThickFunction<Box<T>> from the following structures:
// The structure of a %swift.function.
struct ThickFunction<Context> {
// the raw function pointer
var ptr: UnsafeRawPointer
// the context of the function value – can be nil to indicate
// that the function has no context.
var context: UnsafePointer<Context>?
}
// The structure of a %swift.refcounted.
struct RefCounted {
// pointer to the metadata of the object
var type: UnsafeRawPointer
// the reference counting bits.
var refCountingA: UInt32
var refCountingB: UInt32
}
// The structure of a %swift.refcounted, with a value tacked onto the end.
// This is what captured values get wrapped in (on the heap).
struct Box<T> {
var ref: RefCounted
var value: T
}
In fact, we can actually verify this for ourselves by running the following:
// this wrapper is necessary so that the function doesn't get put through a reabstraction
// thunk when getting typed as a generic type T (such as with .initialize(to:))
struct VoidVoidFunction {
var f: () -> Void
}
func makeClosure() -> () -> Void {
var i = 5
return { i += 2 }
}
let f = VoidVoidFunction(f: makeClosure())
let ptr = UnsafeMutablePointer<VoidVoidFunction>.allocate(capacity: 1)
ptr.initialize(to: f)
let ctx = ptr.withMemoryRebound(to: ThickFunction<Box<Int>>.self, capacity: 1) {
$0.pointee.context! // force unwrap as we know the function has a context object.
}
print(ctx.pointee)
// Box<Int>(ref:
// RefCounted(type: 0x00000001002b86d0, refCountingA: 2, refCountingB: 2),
// value: 5
// )
f.f() // call the closure – increment the captured value.
print(ctx.pointee)
// Box<Int>(ref:
// RefCounted(type: 0x00000001002b86d0, refCountingA: 2, refCountingB: 2),
// value: 7
// )
ptr.deinitialize()
ptr.deallocate(capacity: 1)
We can see that by calling the function between printing out the value of the context object, we can observe the changing in value of the captured variable i.
For multiple captured values, we need extra indirection, as the boxes cannot be stored directly as the given function's context, and may be captured by other closures. This is done by adding pointers to the boxes to the end of a %swift.refcounted.
For example:
struct TwoCaptureContext<T, U> {
// reference counting header
var ref: RefCounted
// pointers to boxes with captured values...
var first: UnsafePointer<Box<T>>
var second: UnsafePointer<Box<U>>
}
func makeClosure() -> () -> Void {
var i = 5
var j = "foo"
return { i += 2; j += "b" }
}
let f = VoidVoidFunction(f: makeClosure())
let ptr = UnsafeMutablePointer<VoidVoidFunction>.allocate(capacity: 1)
ptr.initialize(to: f)
let ctx = ptr.withMemoryRebound(to:
ThickFunction<TwoCaptureContext<Int, String>>.self, capacity: 1) {
$0.pointee.context!.pointee
}
print(ctx.first.pointee.value, ctx.second.pointee.value) // 5 foo
f.f() // call the closure – mutate the captured values.
print(ctx.first.pointee.value, ctx.second.pointee.value) // 7 foob
ptr.deinitialize()
ptr.deallocate(capacity: 1)
Passing functions into parameters of generic type
You'll note that in the previous examples, we used a VoidVoidFunction wrapper for our function values. This is because otherwise, when being passed into a parameter of generic type (such as UnsafeMutablePointer's initialize(to:) method), Swift will put a function value through some reabstraction thunks in order to unify its calling convention to one where the arguments and return are passed by reference, rather than value (the relevant IR).
But now our function value has a pointer to the thunk, rather than the actual function we want to call. So how does the thunk know which function to call? The answer is simple – Swift puts the function that we want to the thunk to call in the context itself, which will therefore look like this:
// the context object for a reabstraction thunk – contains an actual function to call.
struct ReabstractionThunkContext<Context> {
// the standard reference counting header
var ref: RefCounted
// the thick function value for the thunk to call
var function: ThickFunction<Context>
}
The first thunk that we go through has 3 parameters:
A pointer to where the return value should be stored
A pointer to where the arguments for the function are located
The context object which contains the actual thick function value to call (such as shown above)
This first thunk just extracts the function value from the context, and then calls a second thunk, with 4 parameters:
A pointer to where the return value should be stored
A pointer to where the arguments for the function are located
The raw function pointer to call
The pointer to the context of the function to call
This thunk now retrieves the arguments (if any) from the argument pointer, then calls the given function pointer with these arguments, along with its context. It then stores the return value (if any) at the address of the return pointer.
Like in the previous examples, we can test this like so:
func makeClosure() -> () -> Void {
var i = 5
return { i += 2 }
}
func printSingleCapturedValue<T>(t: T) {
let ptr = UnsafeMutablePointer<T>.allocate(capacity: 1)
ptr.initialize(to: t)
let ctx = ptr.withMemoryRebound(to:
ThickFunction<ReabstractionThunkContext<Box<Int>>>.self, capacity: 1) {
// get the context from the thunk function value, which we can
// then get the actual function value from, and therefore the actual
// context object.
$0.pointee.context!.pointee.function.context!
}
// print out captured value in the context object
print(ctx.pointee.value)
ptr.deinitialize()
ptr.deallocate(capacity: 1)
}
let closure = makeClosure()
printSingleCapturedValue(t: closure) // 5
closure()
printSingleCapturedValue(t: closure) // 7
Escaping vs. non-escaping capture
When the compiler can determine that the capture of a given local variable doesn't escape the lifetime of the function it's declared in, it can optimise by promoting the value of that variable from the heap-allocated box to the stack (this is a guaranteed optimisation, and occurs in even -Onone). Then, the function's context object need only store a pointer to the given captured value on the stack, as it is guaranteed not to be needed after the function exits.
This can therefore be done when the closure(s) capturing the variable are known not to escape the lifetime of the function.
Generally, an escaping closure is one that either:
Is stored in a non-local variable (including being returned from the function).
Is captured by another escaping closure.
Is passed as an argument to a function where that parameter is either marked as #escaping, or is not of function type (note this includes composite types, such as optional function types).
So, the following are examples where the capture of a given variable can be considered not to escape the lifetime of the function:
// the parameter is non-escaping, as is of function type and is not marked #escaping.
func nonEscaping(_ f: () -> Void) {
f()
}
func bar() -> String {
var str = ""
// c doesn't escape the lifetime of bar().
let c = {
str += "c called; "
}
c();
// immediately-evaluated closure obviously doesn't escape.
{ str += "immediately-evaluated closure called; " }()
// closure passed to non-escaping function parameter, so doesn't escape.
nonEscaping {
str += "closure passed to non-escaping parameter called."
}
return str
}
In this example, because str is only ever captured by closures that are known not to escape the lifetime of the function bar(), the compiler can optimise by storing the value of str on the stack, with the context objects storing only a pointer to it (the relevant IR).
So, the context objects for each of the closures1 will look like Box<UnsafePointer<String>>, with pointers to the string value on the stack. Although unfortunately, in a Schrödinger-like manner, attempting to observe this by allocating and re-binding a pointer (like before) triggers the compiler to treat the given closure as escaping – so we're once again looking at a Box<String> for the context.
In order to deal with the disparity between context objects that hold pointer(s) to the captured values rather than holding the values in their own heap-allocated boxes – Swift creates specialised implementations of the closures that take pointers to the captured values as arguments.
Then, a thunk is created for each closure that simply takes in a given context object, extracts the pointer(s) to the captured values from it, and passes this onto the specialised implementation of the closure. Now, we can just have a pointer to this thunk along with our context object as the thick function value.
For multiple captured values that don't escape, the additional pointers are simply added onto the end of the box, i.e
struct TwoNonEscapingCaptureContext<T, U> {
// reference counting header
var ref: RefCounted
// pointers to captured values (on the stack)...
var first: UnsafePointer<T>
var second: UnsafePointer<U>
}
This optimisation of promoting the captured values from the heap to the stack can be especially beneficial in this case, as we're no longer having to allocate separate boxes for each value – such as was the case previously.
Furthermore it's worth noting that lots of cases with non-escaping closure capture can be optimised much more aggressively in -O builds with inlining, which can result in context objects being optimised away entirely.
1. Immediately-evaluated closures actually don't use a context object, the pointer(s) to the captured values are just passed directly to it upon calling.