It seems logical to me that escaping closures would capture structs by copying. But if that was the case, the following code makes no sense and should not compile:
struct Wtf {
var x = 1
}
func foo(){
var wtf = Wtf()
DispatchQueue.global().async {
wtf.x = 5
}
Thread.sleep(forTimeInterval: 2)
print("x = \(wtf.x)")
}
Yet it compiles successfully and even prints 5 when foo is called. How is this possible?
While it might make sense for a struct to be copied, as your code demonstrates, it is not. That's a powerful tool. For example:
func makeCounter() -> () -> Int {
var n = 0
return {
n += 1 // This `n` is the same `n` from the outer scope
return n
}
// At this point, the scope is gone, but the `n` lives on in the closure.
}
let counter1 = makeCounter()
let counter2 = makeCounter()
print("Counter1: ", counter1(), counter1()) // Counter1: 1 2
print("Counter2: ", counter2(), counter2()) // Counter2: 1 2
print("Counter1: ", counter1(), counter1()) // Counter1: 3 4
If n were copied into the closure, this couldn't work. The whole point is the closure captures and can modify state outside itself. This is what separates a closure (which "closes over" the scope where it was created) and an anonymous function (which does not).
(The history of the term "close over" is kind of obscure. It refers to the idea that the lambda expression's free variables have been "closed," but IMO "bound" would be a much more obvious term, and is how we describe this everywhere else. But the term "closure" has been used for decades, so here we are.)
Note that it is possible to get copy semantics. You just have to ask for it:
func foo(){
var wtf = Wtf()
DispatchQueue.global().async { [wtf] in // Make a local `let` copy
var wtf = wtf // To modify it, we need to make a `var` copy
wtf.x = 5
}
Thread.sleep(forTimeInterval: 2)
// Prints 1 as you expected
print("x = \(wtf.x)")
}
In C++, lambdas have to be explicit about how to capture values, by binding or by copying. But in Swift, they chose to make binding the default.
As to why you're allowed to access wtf after it's been captured by the closure, that's just a lack of move semantics in Swift. There's no way in Swift today to express "this variable has been passed to something else and may no longer be accessed in this scope." That's a known limitation of the language, and a lot of work is going into fix it. See The Ownership Manifesto for more.
Related
Suppose I have a Swift class like this:
#objc final MyClass : NSObject
{
let classPropertyString = "A class property"
func doStuff()
{
let localString = "An object local to this function"
DispatchQueue.global(qos: .userInitiated).async { [classPropertyString] in
// Do things with 'classPropertyString' and 'localString'
}
}
}
My question is: when I write a capture list, am I responsible for EXHAUSTIVELY listing all the things to which I want the closure to hold a strong reference?
In other words, if I omit localString from my capture list (as I've done here), will the closure still automatically capture a strong reference to it or am I in for a bad time?
There are several minor quirks with your question that make it tricky to answer clearly, but I think I understand the underlying concern, and the short answer is "no." But your example is impossible, so the answer is "it's impossible." And if it were possible, there'd be no strong reference (nor would there be a need for one), so the question still would be kind of "it's impossible." Even so, let's walk through what's going on here.
First, closure can't reference localString unless it's reassigned somehow in the comment inside doStuff(). closure is assigned at a level where localString is not in scope. Closures can only capture variables that are in scope when they are assigned, not when they're called. But let's go back to the original version of this question, before it was edited. That version did have the case you're describing:
#objc final myClass : NSObject
{
let classPropertyString = "A class property"
func doStuff()
{
let localString = "An object local to this function"
DispatchQueue.global(qos: .userInitiated).async { [classPropertyString] in // (1)
// Do things with 'classPropertyString' and 'localString'
}
// (2)
}
}
There's no problems here. classPropertyString is copied into the closure, avoiding any retain loops. localString is referenced by the closure, and so it's preserved as long as the closure exists.
Because you listed classPropertyString in the capture list, it is evaluated at point (1) and copied into the closure. Because you implicitly captured localString, it is treated as a reference. See Capture Lists in the Swift Programming Language Reference for some excellent examples of exactly how this works in different cases.
In no case (*) will Swift allow the underlying storage for something you're using in a closure to disappear behind your back. That's why the typical concern is excessive retains (memory leaks) rather than dangling references (crashes).
(*) "In no case" here is a lie. There are several ways that Swift will allow it, but almost all of them involve "Unsafe" which is your warning about that. The major exception is unowned, and of course anything involving ! types. And Swift is not typically thread-safe, so you need to be careful about that...
The last comment about thread-safety is a place where the subtle distinctions between implicit and explicit captures can really matter. Consider this case where you modify an implicitly captured value on two queues:
func doStuff() -> String
{
var localString = "An object local to this function"
DispatchQueue.global(qos: .userInitiated).async {
localString = "something else"
callFunction(localString)
}
localString = "even more changes"
return localString
}
What happens in that case? Good grief, never do that. I believe it's undefined behavior and that localString could be anything including corrupted memory, at least in the most general case (it might be defined behavior for calling .async; I'm not sure). But don't do it.
But for your normal cases, there is no reason to explicitly capture local variables. (I sometimes wish Swift had gone the C++ way and said it was required, but it isn't.)
Ok, one more way implicit and explicit are different that might drive home how they work. Consider a stateful closure like this (I build these pretty often):
func incrementor() -> () -> Int {
var n = 0
return {
n += 1
return n
}
}
let inc = incrementor()
inc() // 1
inc() // 2
inc() // 3
let inc2 = incrementor()
inc2() // 1
See how the local variable n is captured by the closure, and can be modified after it goes out of scope. And see how inc2 has its own version of that local variable. Now try that with explicit capture.
func incrementor() -> () -> Int {
var n = 0
return { [n] in // <---- add [n]
n += 1 // Left side of mutating operator isn't mutable: 'n' is an immutable capture
return n
}
}
Explicit captures are copies and they're immutable. Implicit captures are references, and so have the same mutability as the thing they reference.
I got some unexpected behavior using UnsafeMutablePointer on an observed property in a struct I created (on Xcode 10.1, Swift 4.2). See the following playground code:
struct NormalThing {
var anInt = 0
}
struct IntObservingThing {
var anInt: Int = 0 {
didSet {
print("I was just set to \(anInt)")
}
}
}
var normalThing = NormalThing(anInt: 0)
var ptr = UnsafeMutablePointer(&normalThing.anInt)
ptr.pointee = 20
print(normalThing.anInt) // "20\n"
var intObservingThing = IntObservingThing(anInt: 0)
var otherPtr = UnsafeMutablePointer(&intObservingThing.anInt)
// "I was just set to 0."
otherPtr.pointee = 20
print(intObservingThing.anInt) // "0\n"
Seemingly, modifying the pointee on an UnsafeMutablePointer to an observed property doesn't actually modify the value of the property. Also, the act of assigning the pointer to the property fires the didSet action. What am I missing here?
Any time you see a construct like UnsafeMutablePointer(&intObservingThing.anInt), you should be extremely wary about whether it'll exhibit undefined behaviour. In the vast majority of cases, it will.
First, let's break down exactly what's happening here. UnsafeMutablePointer doesn't have any initialisers that take inout parameters, so what initialiser is this calling? Well, the compiler has a special conversion that allows a & prefixed argument to be converted to a mutable pointer to the 'storage' referred to by the expression. This is called an inout-to-pointer conversion.
For example:
func foo(_ ptr: UnsafeMutablePointer<Int>) {
ptr.pointee += 1
}
var i = 0
foo(&i)
print(i) // 1
The compiler inserts a conversion that turns &i into a mutable pointer to i's storage. Okay, but what happens when i doesn't have any storage? For example, what if it's computed?
func foo(_ ptr: UnsafeMutablePointer<Int>) {
ptr.pointee += 1
}
var i: Int {
get { return 0 }
set { print("newValue = \(newValue)") }
}
foo(&i)
// prints: newValue = 1
This still works, so what storage is being pointed to by the pointer? To solve this problem, the compiler:
Calls i's getter, and places the resultant value into a temporary variable.
Gets a pointer to that temporary variable, and passes that to the call to foo.
Calls i's setter with the new value from the temporary.
Effectively doing the following:
var j = i // calling `i`'s getter
foo(&j)
i = j // calling `i`'s setter
It should hopefully be clear from this example that this imposes an important constraint on the lifetime of the pointer passed to foo – it can only be used to mutate the value of i during the call to foo. Attempting to escape the pointer and using it after the call to foo will result in a modification of only the temporary variable's value, and not i.
For example:
func foo(_ ptr: UnsafeMutablePointer<Int>) -> UnsafeMutablePointer<Int> {
return ptr
}
var i: Int {
get { return 0 }
set { print("newValue = \(newValue)") }
}
let ptr = foo(&i)
// prints: newValue = 0
ptr.pointee += 1
ptr.pointee += 1 takes place after i's setter has been called with the temporary variable's new value, therefore it has no effect.
Worse than that, it exhibits undefined behaviour, as the compiler doesn't guarantee that the temporary variable will remain valid after the call to foo has ended. For example, the optimiser could de-initialise it immediately after the call.
Okay, but as long as we only get pointers to variables that aren't computed, we should be able to use the pointer outside of the call it was passed to, right? Unfortunately not, turns out there's lots of other ways to shoot yourself in the foot when escaping inout-to-pointer conversions!
To name just a few (there are many more!):
A local variable is problematic for a similar reason to our temporary variable from earlier – the compiler doesn't guarantee that it will remain initialised until the end of the scope it's declared in. The optimiser is free to de-initialise it earlier.
For example:
func bar() {
var i = 0
let ptr = foo(&i)
// Optimiser could de-initialise `i` here.
// ... making this undefined behaviour!
ptr.pointee += 1
}
A stored variable with observers is problematic because under the hood it's actually implemented as a computed variable that calls its observers in its setter.
For example:
var i: Int = 0 {
willSet(newValue) {
print("willSet to \(newValue), oldValue was \(i)")
}
didSet(oldValue) {
print("didSet to \(i), oldValue was \(oldValue)")
}
}
is essentially syntactic sugar for:
var _i: Int = 0
func willSetI(newValue: Int) {
print("willSet to \(newValue), oldValue was \(i)")
}
func didSetI(oldValue: Int) {
print("didSet to \(i), oldValue was \(oldValue)")
}
var i: Int {
get {
return _i
}
set {
willSetI(newValue: newValue)
let oldValue = _i
_i = newValue
didSetI(oldValue: oldValue)
}
}
A non-final stored property on classes is problematic as it can be overridden by a computed property.
And this isn't even considering cases that rely on implementation details within the compiler.
For this reason, the compiler only guarantees stable and unique pointer values from inout-to-pointer conversions on stored global and static stored variables without observers. In any other case, attempting to escape and use a pointer from an inout-to-pointer conversion after the call it was passed to will lead to undefined behaviour.
Okay, but how does my example with the function foo relate to your example of calling an UnsafeMutablePointer initialiser? Well, UnsafeMutablePointer has an initialiser that takes an UnsafeMutablePointer argument (as a result of conforming to the underscored _Pointer protocol which most standard library pointer types conform to).
This initialiser is effectively same as the foo function – it takes an UnsafeMutablePointer argument and returns it. Therefore when you do UnsafeMutablePointer(&intObservingThing.anInt), you're escaping the pointer produced from the inout-to-pointer conversion – which, as we've discussed, is only valid if it's used on a stored global or static variable without observers.
So, to wrap things up:
var intObservingThing = IntObservingThing(anInt: 0)
var otherPtr = UnsafeMutablePointer(&intObservingThing.anInt)
// "I was just set to 0."
otherPtr.pointee = 20
is undefined behaviour. The pointer produced from the inout-to-pointer conversion is only valid for the duration of the call to UnsafeMutablePointer's initialiser. Attempting to use it afterwards results in undefined behaviour. As matt demonstrates, if you want scoped pointer access to intObservingThing.anInt, you want to use withUnsafeMutablePointer(to:).
I'm actually currently working on implementing a warning (which will hopefully transition to an error) that will be emitted on such unsound inout-to-pointer conversions. Unfortunately I haven't had much time lately to work on it, but all things going well, I'm aiming to start pushing it forwards in the new year, and hopefully get it into a Swift 5.x release.
In addition, it's worth noting that while the compiler doesn't currently guarantee well-defined behaviour for:
var normalThing = NormalThing(anInt: 0)
var ptr = UnsafeMutablePointer(&normalThing.anInt)
ptr.pointee = 20
From the discussion on #20467, it looks like this will likely be something that the compiler does guarantee well-defined behaviour for in a future release, due to the fact that the base (normalThing) is a fragile stored global variable of a struct without observers, and anInt is a fragile stored property without observers.
I'm pretty sure the problem is that what you're doing is illegal. You can't just declare an unsafe pointer and claim that it points at the address of a struct property. (In fact, I don't even understand why your code compiles in the first place; what initializer does the compiler think this is?) The correct way, which gives the expected results, is to ask for a pointer that does point at that address, like this:
struct IntObservingThing {
var anInt: Int = 0 {
didSet {
print("I was just set to \(anInt)")
}
}
}
withUnsafeMutablePointer(to: &intObservingThing.anInt) { ptr -> Void in
ptr.pointee = 20 // I was just set to 20
}
print(intObservingThing.anInt) // 20
I am going through the following tutorial on RxSwift:
http://adamborek.com/thinking-rxswift/
and having trouble understanding the following pattern:
searchBar.rx.text.orEmpty
------------> .flatMap { [spotifyClient] query in
return spotifyClient.rx.search(query: query)
}.map { tracks in
return tracks.map(TrackRenderable.init)
}
This square brackets input parameter: [spotifyClient] query seems very weird for me. I looked over official Apple documentation for closures and functions and I can not see any examples of such input parameters. In Objective C this would not bother me much, but it is Swift. Could anyone explain, what this parameter means here?
You will need to understand the variable capturing of closure idea.
Consider this example:
struct Calculator {
var a: Int
var b: Int
var sum: Int {
return a + b
}
}
Then you use this as:
let calculator = Calculator(a: 3, b: 5)
// You define a closure where you will use this calculator instance
let closure = {
// closure captures the variables that are declared prior to the declaration of the closure.
// your calculator instance is being captured here
// it's default variable capture
print("The result is \(calculator.sum)")
}
closure() // Prints "The result is 8"
Till now, everything is okay. You get what's expected.
Now consider you declare the calculator instance as var because in some point you need to mutate it's state. This is the case where complexity arises. Look:
var calculator = Calculator(a: 3, b: 5)
let closure = {
print("The result is \(calculator.sum)")
}
// You change the state of your calculator instance anytime before the closure gets executed
calculator.b = 20
// When the closure actually executes, you will be affected by any changes outside the closure
closure() // Prints "The result is 23"
So, the default variable capture isn't really helping you, instead it's creating problem in your case.
If you want to prevent this behaviour and print 8 even if the properties change after their capturing inside the closure, we can explicitly capture the variable with a capture list like this:
// [calculator] is your capture list
let closure = { [calculator] in
print("The result is \(calculator.sum)")
}
// change anything with calculator instance
calculator.b = 20
// execute the closure
closure() // Prints "The result is 8"
Capture List keeps immutable copy of the variable(s). Thanks to this copy, further changes to calculator, outside the closure, will not affect the closure.
You can capture multiple variables at once, hence it's called Capture List. Example:
let closure = { [variable1, variable2, variable3] in
print(variable1)
print(variable2)
print(variable3)
}
I recommend you read this article Capturing Values In Swift Closures.
Now, in your case spotifyClient is an instance of a class that may be responsible to make API calls. This instance may need some changes for calling different APIs. So, to prevent the affect of any changes to spotifyClient outside this closure you capture this instance in a Capture List.
Capture List vs. Parameter List:
You are confusing the parameter list with the capture list. The generic syntax is:
{ [capture list] (parameter list) in
...
...
}
Now take a look at the modified version of the above example:
let closure: (String)-> Void = { [calculator] stringParameter in // When using single parameter, you can always omit the () parentheses
print("\(stringParameter). The result is \(calculator.sum)")
}
// change anything with calculator instance
calculator.b = 20
// execute the closure
closure("Hey") // Prints "Hey. The result is 8"
Global functions are closures that have name and do not capture any values.
In swift a function is a spatial form of closure. The different is the function has a name and if it is an global function it cannot capture constants and variables from the surrounding context.
However, I found that the global function can also capture the constants and variables from the surrounding context also (please see the sample code below)
let referenceInt = 10
func addOne () -> Int {
return referenceInt + 1 //captured the constant referenceInt
}
let fooA = addOne
let fooB = addOne
let fooC = addOne
print(fooA()) //prints 11
print(fooB()) //prints 11
print(fooC()) //prints 11, (func addOne captured referenceInt ?)
print(referenceInt) //prints 10
Problems:
I believe I didn't fully understand the following concepts:
Simply define a function in playground (like addOne() -> Int here) may not means it is a global function
Having the wrong understanding of the "Capture" for this cases, this is not a capture at all, (but why?)
Helps I'm looking for:
I would be very appreciate you could point out which part I understand wrongly and would be even great that you can give me some explanation.
Thanks
PS:
This question might be a duplication of this one, however, I still post it since there is no clean answer on it yet and my question pushed the question a bit further. However, if you still want to close it, I respect that and I willing to learn from you.
First, I would question your premise. A closure is a closure. All functions in Swift are closures, and they all capture in just the same way.
Second, I don't see what your code has to do with capturing or closures. You are not doing anything in your code that tests whether anything is being captured. The assignment of the type let fooA = addOne doesn't do anything interesting, and the code inside addOne doesn't do anything interesting either. You are merely adding two values at the time the code runs. Certainly the code inside addOne is permitted to refer to the global variable referenceInt, but that is merely because it is in scope. You aren't doing anything here that elicits the special powers of a closure.
Here's a modification of your code that does show capturing in action:
struct Test {
var referenceInt = 10
func addOne () -> Int {
return referenceInt + 1 // capture
}
mutating func test() {
let fooA = self.addOne
let fooB = self.addOne
let fooC = self.addOne
referenceInt = 100 // :)
print(fooA()) //prints 11
print(fooB()) //prints 11
print(fooC()) //prints 11
print(referenceInt) //prints 100
}
}
var t = Test()
t.test()
We change referenceInt before calling fooA and so on. Calling fooA still gives 11, because the value of self.referenceInt was captured before we changed referenceInt.
I know about the ampersand as a bit operation but sometimes I see it in front of variable names. What does putting an & in front of variables do?
It works as an inout to make the variable an in-out parameter. In-out means in fact passing value by reference, not by value. And it requires not only to accept value by reference, by also to pass it by reference, so pass it with & - foo(&myVar) instead of just foo(myVar)
As you see you can use that in error handing in Swift where you have to create an error reference and pass it to the function using & the function will populate the error value if an error occur or pass the variable back as it was before
Why do we use it? Sometimes a function already returns other values and just returning another one (like an error) would be confusing, so we pass it as an inout. Other times we want the values to be populated by the function so we don't have to iterate over lots of return values, since the function already did it for us - among other possible uses.
It means that it is an in-out variable. You can do something directly with that variable. It is passed by address, not as a copy.
For example:
var temp = 10
func add(inout a: Int){
a++
}
add(inout:&temp)
temp // 11
There's another function of the ampersand in the Swift language that hasn't been mentioned yet. Take the following example:
protocol Foo {}
protocol Bar {}
func myMethod(myVar: Foo & Bar) {
// Do something
}
Here the ampersand syntax is stating that myVar conforms to both the Foo and Bar protocol.
As another use case, consider the following:
func myMethod() -> UIViewController & UITableViewDataSource {
// Do something
}
Here we're saying that the method returns a class instance (of UIViewController) that conforms to a certain protocol (UITableViewDataSource). This is rendered somewhat obsolete with Swift 5.1's Opaque Types but you may see this syntax in pre-Swift 5.1 code from time to time.
If you put & before a variable in a function, that means this variable is inout variable.
#Icaro already described what it means, I will just give an example to illustrate the difference between inout variables and in variables:
func majec(inout xValue:Int, var yValue:Int) {
xValue = 100
yValue = 200
}
var xValue = 33
var yValue = 33
majec(&xValue, yValue: yValue)
xValue //100
yValue //33
As noted in other answers, you use prefix & to pass a value to an inout parameter of a method or function call, as documented under Functions > Function Argument Labels and Parameter Names > In-Out Parameters in The Swift Programming Language. But there's more to it than that.
You can, in practice, think about Swift inout parameters and passing values to them as being similar to C or C++ pass-by-address or pass-by-reference. In fact, the compiler will optimize many uses of inout parameters down to roughly the same mechanics (especially when you're calling imported C or ObjC APIs that deal in pointers). However, those are just optimizations — at a semantic level, inout really doesn't pass addresses around, which frees the compiler to make this language construct more flexible and powerful.
For example, here's a struct that uses a common strategy for validating access to one of its properties:
struct Point {
private var _x: Int
var x: Int {
get {
print("get x: \(_x)")
return _x
}
set {
print("set x: \(newValue)")
_x = newValue
}
}
// ... same for y ...
init(x: Int, y: Int) { self._x = x; self._y = y }
}
(In "real" code, the getter and setter for x could do things like enforcing minimum/maximum values. Or x could do other computed-property tricks, like talking to a SQL database under the hood. Here we just instrument the call and get/set the underlying private property.)
Now, what happens when we pass x to an inout parameter?
func plusOne(num: inout Int) {
num += 1
}
var pt = Point(x: 0, y: 1)
plusOne(num: &pt.x)
// prints:
// get x: 0
// set x: 1
So, even though x is a computed property, passing it "by reference" using an inout parameter works the same as you'd expect it to if x were a stored property or a local variable.
This means that you can pass all sorts of things "by reference" that you couldn't even consider in C/C++/ObjC. For example, consider the standard library swap function, that takes any two... "things" and switches their values:
var a = 1, b = 2
swap(&a, &b)
print(a, b) // -> 2 1
var dict = [ "Malcolm": "Captain", "Kaylee": "Mechanic" ]
swap(&dict["Malcolm"], &dict["Kaylee"])
print(dict) // -> ["Kaylee": "Captain", "Malcolm": "Mechanic"], fanfic ahoy
let window1 = NSWindow()
let window2 = NSWindow()
window1.title = "window 1"
window2.title = "window 2"
var windows = [window1, window2]
swap(&windows[0], &windows[1])
print(windows.map { $0.title }) // -> ["window 2", "window 1"]
The the way inout works also lets you do fun stuff like using the += operator on nested call chains:
window.frame.origin.x += 10
... which is a whole lot simpler than decomposing a CGRect just to construct a new one with a different x coordinate.
This more nuanced version of the inout behavior, called "call by value result", and the ways it can optimize down to C-style "pass by address" behavior, is covered under Declarations > Functions > In-Out Parameters in The Swift Programming Language.