I am going through the following tutorial on RxSwift:
http://adamborek.com/thinking-rxswift/
and having trouble understanding the following pattern:
searchBar.rx.text.orEmpty
------------> .flatMap { [spotifyClient] query in
return spotifyClient.rx.search(query: query)
}.map { tracks in
return tracks.map(TrackRenderable.init)
}
This square brackets input parameter: [spotifyClient] query seems very weird for me. I looked over official Apple documentation for closures and functions and I can not see any examples of such input parameters. In Objective C this would not bother me much, but it is Swift. Could anyone explain, what this parameter means here?
You will need to understand the variable capturing of closure idea.
Consider this example:
struct Calculator {
var a: Int
var b: Int
var sum: Int {
return a + b
}
}
Then you use this as:
let calculator = Calculator(a: 3, b: 5)
// You define a closure where you will use this calculator instance
let closure = {
// closure captures the variables that are declared prior to the declaration of the closure.
// your calculator instance is being captured here
// it's default variable capture
print("The result is \(calculator.sum)")
}
closure() // Prints "The result is 8"
Till now, everything is okay. You get what's expected.
Now consider you declare the calculator instance as var because in some point you need to mutate it's state. This is the case where complexity arises. Look:
var calculator = Calculator(a: 3, b: 5)
let closure = {
print("The result is \(calculator.sum)")
}
// You change the state of your calculator instance anytime before the closure gets executed
calculator.b = 20
// When the closure actually executes, you will be affected by any changes outside the closure
closure() // Prints "The result is 23"
So, the default variable capture isn't really helping you, instead it's creating problem in your case.
If you want to prevent this behaviour and print 8 even if the properties change after their capturing inside the closure, we can explicitly capture the variable with a capture list like this:
// [calculator] is your capture list
let closure = { [calculator] in
print("The result is \(calculator.sum)")
}
// change anything with calculator instance
calculator.b = 20
// execute the closure
closure() // Prints "The result is 8"
Capture List keeps immutable copy of the variable(s). Thanks to this copy, further changes to calculator, outside the closure, will not affect the closure.
You can capture multiple variables at once, hence it's called Capture List. Example:
let closure = { [variable1, variable2, variable3] in
print(variable1)
print(variable2)
print(variable3)
}
I recommend you read this article Capturing Values In Swift Closures.
Now, in your case spotifyClient is an instance of a class that may be responsible to make API calls. This instance may need some changes for calling different APIs. So, to prevent the affect of any changes to spotifyClient outside this closure you capture this instance in a Capture List.
Capture List vs. Parameter List:
You are confusing the parameter list with the capture list. The generic syntax is:
{ [capture list] (parameter list) in
...
...
}
Now take a look at the modified version of the above example:
let closure: (String)-> Void = { [calculator] stringParameter in // When using single parameter, you can always omit the () parentheses
print("\(stringParameter). The result is \(calculator.sum)")
}
// change anything with calculator instance
calculator.b = 20
// execute the closure
closure("Hey") // Prints "Hey. The result is 8"
Related
It seems logical to me that escaping closures would capture structs by copying. But if that was the case, the following code makes no sense and should not compile:
struct Wtf {
var x = 1
}
func foo(){
var wtf = Wtf()
DispatchQueue.global().async {
wtf.x = 5
}
Thread.sleep(forTimeInterval: 2)
print("x = \(wtf.x)")
}
Yet it compiles successfully and even prints 5 when foo is called. How is this possible?
While it might make sense for a struct to be copied, as your code demonstrates, it is not. That's a powerful tool. For example:
func makeCounter() -> () -> Int {
var n = 0
return {
n += 1 // This `n` is the same `n` from the outer scope
return n
}
// At this point, the scope is gone, but the `n` lives on in the closure.
}
let counter1 = makeCounter()
let counter2 = makeCounter()
print("Counter1: ", counter1(), counter1()) // Counter1: 1 2
print("Counter2: ", counter2(), counter2()) // Counter2: 1 2
print("Counter1: ", counter1(), counter1()) // Counter1: 3 4
If n were copied into the closure, this couldn't work. The whole point is the closure captures and can modify state outside itself. This is what separates a closure (which "closes over" the scope where it was created) and an anonymous function (which does not).
(The history of the term "close over" is kind of obscure. It refers to the idea that the lambda expression's free variables have been "closed," but IMO "bound" would be a much more obvious term, and is how we describe this everywhere else. But the term "closure" has been used for decades, so here we are.)
Note that it is possible to get copy semantics. You just have to ask for it:
func foo(){
var wtf = Wtf()
DispatchQueue.global().async { [wtf] in // Make a local `let` copy
var wtf = wtf // To modify it, we need to make a `var` copy
wtf.x = 5
}
Thread.sleep(forTimeInterval: 2)
// Prints 1 as you expected
print("x = \(wtf.x)")
}
In C++, lambdas have to be explicit about how to capture values, by binding or by copying. But in Swift, they chose to make binding the default.
As to why you're allowed to access wtf after it's been captured by the closure, that's just a lack of move semantics in Swift. There's no way in Swift today to express "this variable has been passed to something else and may no longer be accessed in this scope." That's a known limitation of the language, and a lot of work is going into fix it. See The Ownership Manifesto for more.
I'm rewriting some code to append jobs to an array of closures rather than execute them directly:
var someObject: SomeType?
var jobsArray: [() -> ()] = []
// before rewriting
doExpensiveOperation(someObject!.property)
// 1st attempt at rewriting
jobsArray.append {
doExpensiveOperation(someObject!.property)
}
However, because the value of someObject might change before the closure is executed, I'm now adding a closure list as follows:
// 2nd attempt at rewriting
jobsArray.append { [someObject] in
doExpensiveOperation(someObject!.property)
}
Hopefully then if, say, someObject is subsequently set to nil before the closure executes, the closure will still access the intended instance.
But what's the neatest way to deal with the possibility that the value of .property might change before the closure is executed? Is there a better way than this?
// 3rd attempt at rewriting
let p = someObject!.property
jobsArray.append {
doExpensiveOperation(p)
}
I'm not very keen on this solution because it means changing the original line of code. I'd prefer this but it doesn't work:
// 4th attempt at rewriting
jobsArray.append { [someObject!.property] in
doExpensiveOperation(someObject!.property)
}
Pretty new to Swift, so all guidance gratefully received. Thanks!
A capture list such as [someObject] is actually syntactic sugar for [someObject = someObject], where the right hand side can be an arbitrary expression that gets bound to a new constant upon the closure being formed.
Therefore one option is to write your example as:
jobsArray.append { [property = someObject!.property] in
doExpensiveOperation(property)
}
After reading:
https://github.com/rodionovd/SWRoute/wiki/Function-hooking-in-Swift
https://github.com/rodionovd/SWRoute/blob/master/SWRoute/rd_get_func_impl.c
I understood that Swift function pointer is wrapped by swift_func_wrapper and swift_func_object (according to the article in 2014).
I guess this still works in Swift 3, but I couldn't find which file in https://github.com/apple/swift best describes these structs.
Can anyone help me?
I believe these details are mainly part of the implementation of Swift's IRGen – I don't think you'll find any friendly structs in the source showing you the full structure of various Swift function values. Therefore if you want to do some digging into this, I would recommend examining the IR emitted by the compiler.
You can do this by running the command:
xcrun swiftc -emit-ir main.swift | xcrun swift-demangle > main.irgen
which will emit the IR (with demangled symbols) for a -Onone build. You can find the documentation for LLVM IR here.
The following is some interesting stuff that I've been able to learn from going through the IR myself in a Swift 3.1 build. Note that this is all subject to change in future Swift versions (at least until Swift is ABI stable). It goes without saying that the code examples given below are only for demonstration purposes; and shouldn't ever be used in actual production code.
Thick function values
At a very basic level, function values in Swift are simple things – they're defined in the IR as:
%swift.function = type { i8*, %swift.refcounted* }
which is the raw function pointer i8*, along with a pointer to its context %swift.refcounted*, where %swift.refcounted is defined as:
%swift.refcounted = type { %swift.type*, i32, i32 }
which is the structure of a simple reference-counted object, containing a pointer to the object's metadata, along with two 32 bit values.
These two 32 bit values are used for the reference count of the object. Together , they can either represent (as of Swift 4):
The strong and unowned reference count of the object + some flags, including whether the object uses native Swift reference counting (as opposed to Obj-C reference counting), and whether the object has a side table.
or
A pointer to a side table, which contains the above, plus the weak reference count of the object (on forming a weak reference to an object, if it doesn't already have a side table, one will be created).
For further reading on the internals of Swift reference counting, Mike Ash has a great blog post on the subject.
The context of a function usually adds extra values onto the end of this %swift.refcounted structure. These values are dynamic things that the function needs upon being called (such as any values that it has captured, or any parameters that it has been partially applied with). In quite a few cases, function values won't need a context, so the pointer to the context will simply be nil.
When the function comes to be called, Swift will simply pass in the context as the last parameter. If the function doesn't have a context parameter, the calling convention appears to allow it to be safely passed anyway.
The storing of the function pointer along with the context pointer is called a thick function value, and is how Swift usually stores function values of known type (as opposed to a thin function value which is just the function pointer).
So, this explains why MemoryLayout<(Int) -> Int>.size returns 16 bytes – because it's made up of two pointers (each being a word in length, i.e 8 bytes on a 64 bit platform).
When thick function values are passed into function parameters (where those parameters are of non-generic type), Swift appears to pass the raw function pointer and context as separate parameters.
Capturing values
When a closure captures a value, this value will be put into a heap-allocated box (although the value itself can get stack-promoted in the case of a non-escaping closure – see later section). This box will be available to the function through the context object (the relevant IR).
For a closure that just captures a single value, Swift just makes the box itself the context of the function (no need for extra indirection). So you'll have a function value which looks like a ThickFunction<Box<T>> from the following structures:
// The structure of a %swift.function.
struct ThickFunction<Context> {
// the raw function pointer
var ptr: UnsafeRawPointer
// the context of the function value – can be nil to indicate
// that the function has no context.
var context: UnsafePointer<Context>?
}
// The structure of a %swift.refcounted.
struct RefCounted {
// pointer to the metadata of the object
var type: UnsafeRawPointer
// the reference counting bits.
var refCountingA: UInt32
var refCountingB: UInt32
}
// The structure of a %swift.refcounted, with a value tacked onto the end.
// This is what captured values get wrapped in (on the heap).
struct Box<T> {
var ref: RefCounted
var value: T
}
In fact, we can actually verify this for ourselves by running the following:
// this wrapper is necessary so that the function doesn't get put through a reabstraction
// thunk when getting typed as a generic type T (such as with .initialize(to:))
struct VoidVoidFunction {
var f: () -> Void
}
func makeClosure() -> () -> Void {
var i = 5
return { i += 2 }
}
let f = VoidVoidFunction(f: makeClosure())
let ptr = UnsafeMutablePointer<VoidVoidFunction>.allocate(capacity: 1)
ptr.initialize(to: f)
let ctx = ptr.withMemoryRebound(to: ThickFunction<Box<Int>>.self, capacity: 1) {
$0.pointee.context! // force unwrap as we know the function has a context object.
}
print(ctx.pointee)
// Box<Int>(ref:
// RefCounted(type: 0x00000001002b86d0, refCountingA: 2, refCountingB: 2),
// value: 5
// )
f.f() // call the closure – increment the captured value.
print(ctx.pointee)
// Box<Int>(ref:
// RefCounted(type: 0x00000001002b86d0, refCountingA: 2, refCountingB: 2),
// value: 7
// )
ptr.deinitialize()
ptr.deallocate(capacity: 1)
We can see that by calling the function between printing out the value of the context object, we can observe the changing in value of the captured variable i.
For multiple captured values, we need extra indirection, as the boxes cannot be stored directly as the given function's context, and may be captured by other closures. This is done by adding pointers to the boxes to the end of a %swift.refcounted.
For example:
struct TwoCaptureContext<T, U> {
// reference counting header
var ref: RefCounted
// pointers to boxes with captured values...
var first: UnsafePointer<Box<T>>
var second: UnsafePointer<Box<U>>
}
func makeClosure() -> () -> Void {
var i = 5
var j = "foo"
return { i += 2; j += "b" }
}
let f = VoidVoidFunction(f: makeClosure())
let ptr = UnsafeMutablePointer<VoidVoidFunction>.allocate(capacity: 1)
ptr.initialize(to: f)
let ctx = ptr.withMemoryRebound(to:
ThickFunction<TwoCaptureContext<Int, String>>.self, capacity: 1) {
$0.pointee.context!.pointee
}
print(ctx.first.pointee.value, ctx.second.pointee.value) // 5 foo
f.f() // call the closure – mutate the captured values.
print(ctx.first.pointee.value, ctx.second.pointee.value) // 7 foob
ptr.deinitialize()
ptr.deallocate(capacity: 1)
Passing functions into parameters of generic type
You'll note that in the previous examples, we used a VoidVoidFunction wrapper for our function values. This is because otherwise, when being passed into a parameter of generic type (such as UnsafeMutablePointer's initialize(to:) method), Swift will put a function value through some reabstraction thunks in order to unify its calling convention to one where the arguments and return are passed by reference, rather than value (the relevant IR).
But now our function value has a pointer to the thunk, rather than the actual function we want to call. So how does the thunk know which function to call? The answer is simple – Swift puts the function that we want to the thunk to call in the context itself, which will therefore look like this:
// the context object for a reabstraction thunk – contains an actual function to call.
struct ReabstractionThunkContext<Context> {
// the standard reference counting header
var ref: RefCounted
// the thick function value for the thunk to call
var function: ThickFunction<Context>
}
The first thunk that we go through has 3 parameters:
A pointer to where the return value should be stored
A pointer to where the arguments for the function are located
The context object which contains the actual thick function value to call (such as shown above)
This first thunk just extracts the function value from the context, and then calls a second thunk, with 4 parameters:
A pointer to where the return value should be stored
A pointer to where the arguments for the function are located
The raw function pointer to call
The pointer to the context of the function to call
This thunk now retrieves the arguments (if any) from the argument pointer, then calls the given function pointer with these arguments, along with its context. It then stores the return value (if any) at the address of the return pointer.
Like in the previous examples, we can test this like so:
func makeClosure() -> () -> Void {
var i = 5
return { i += 2 }
}
func printSingleCapturedValue<T>(t: T) {
let ptr = UnsafeMutablePointer<T>.allocate(capacity: 1)
ptr.initialize(to: t)
let ctx = ptr.withMemoryRebound(to:
ThickFunction<ReabstractionThunkContext<Box<Int>>>.self, capacity: 1) {
// get the context from the thunk function value, which we can
// then get the actual function value from, and therefore the actual
// context object.
$0.pointee.context!.pointee.function.context!
}
// print out captured value in the context object
print(ctx.pointee.value)
ptr.deinitialize()
ptr.deallocate(capacity: 1)
}
let closure = makeClosure()
printSingleCapturedValue(t: closure) // 5
closure()
printSingleCapturedValue(t: closure) // 7
Escaping vs. non-escaping capture
When the compiler can determine that the capture of a given local variable doesn't escape the lifetime of the function it's declared in, it can optimise by promoting the value of that variable from the heap-allocated box to the stack (this is a guaranteed optimisation, and occurs in even -Onone). Then, the function's context object need only store a pointer to the given captured value on the stack, as it is guaranteed not to be needed after the function exits.
This can therefore be done when the closure(s) capturing the variable are known not to escape the lifetime of the function.
Generally, an escaping closure is one that either:
Is stored in a non-local variable (including being returned from the function).
Is captured by another escaping closure.
Is passed as an argument to a function where that parameter is either marked as #escaping, or is not of function type (note this includes composite types, such as optional function types).
So, the following are examples where the capture of a given variable can be considered not to escape the lifetime of the function:
// the parameter is non-escaping, as is of function type and is not marked #escaping.
func nonEscaping(_ f: () -> Void) {
f()
}
func bar() -> String {
var str = ""
// c doesn't escape the lifetime of bar().
let c = {
str += "c called; "
}
c();
// immediately-evaluated closure obviously doesn't escape.
{ str += "immediately-evaluated closure called; " }()
// closure passed to non-escaping function parameter, so doesn't escape.
nonEscaping {
str += "closure passed to non-escaping parameter called."
}
return str
}
In this example, because str is only ever captured by closures that are known not to escape the lifetime of the function bar(), the compiler can optimise by storing the value of str on the stack, with the context objects storing only a pointer to it (the relevant IR).
So, the context objects for each of the closures1 will look like Box<UnsafePointer<String>>, with pointers to the string value on the stack. Although unfortunately, in a Schrödinger-like manner, attempting to observe this by allocating and re-binding a pointer (like before) triggers the compiler to treat the given closure as escaping – so we're once again looking at a Box<String> for the context.
In order to deal with the disparity between context objects that hold pointer(s) to the captured values rather than holding the values in their own heap-allocated boxes – Swift creates specialised implementations of the closures that take pointers to the captured values as arguments.
Then, a thunk is created for each closure that simply takes in a given context object, extracts the pointer(s) to the captured values from it, and passes this onto the specialised implementation of the closure. Now, we can just have a pointer to this thunk along with our context object as the thick function value.
For multiple captured values that don't escape, the additional pointers are simply added onto the end of the box, i.e
struct TwoNonEscapingCaptureContext<T, U> {
// reference counting header
var ref: RefCounted
// pointers to captured values (on the stack)...
var first: UnsafePointer<T>
var second: UnsafePointer<U>
}
This optimisation of promoting the captured values from the heap to the stack can be especially beneficial in this case, as we're no longer having to allocate separate boxes for each value – such as was the case previously.
Furthermore it's worth noting that lots of cases with non-escaping closure capture can be optimised much more aggressively in -O builds with inlining, which can result in context objects being optimised away entirely.
1. Immediately-evaluated closures actually don't use a context object, the pointer(s) to the captured values are just passed directly to it upon calling.
I am trying to see if I can use structs for my model and was trying this. When I call vm.testClosure(), it does not change the value of x and I am not sure why.
struct Model
{
var x = 10.0
}
var m = Model()
class ViewModel
{
let testClosure:() -> ()
init(inout model: Model)
{
testClosure =
{
() -> () in
model.x = 30.5
}
}
}
var vm = ViewModel(model:&m)
m.x
vm.testClosure()
m.x
An inout argument isn't a reference to a value type – it's simply a shadow copy of that value type, that is written back to the caller's value when the function returns.
What's happening in your code is that your inout variable is escaping the lifetime of the function (by being captured in a closure that is then stored) – meaning that any changes to the inout variable after the function has returned will never be reflected outside that closure.
Due to this common misconception about inout arguments, there has been a Swift Evolution proposal for only allowing inout arguments to be captured by #noescape closures. As of Swift 3, your current code will no longer compile.
If you really need to be passing around references in your code – then you should be using reference types (make your Model a class). Although I suspect that you'll probably be able to refactor your logic to avoid passing around references in the first place (however without seeing your actual code, it's impossible to advise).
(Edit: Since posting this answer, inout parameters can now be compiled as a pass-by-reference, which can be seen by looking at the SIL or IR emitted. However you are unable to treat them as such due to the fact that there's no guarantee whatsoever that the caller's value will remain valid after the function call.)
Instances of the closure will get their own, independent copy of the captured value that it, and only it, can alter. The value is captured in the time of executing the closure. Let see your slightly modified code
struct Model
{
var x = 10.0
mutating func modifyX(newValue: Double) {
let this = self
let model = m
x = newValue
// put breakpoint here
//(lldb) po model
//▿ Model
// - x : 30.0
//
//(lldb) po self
//▿ Model
// - x : 301.0
//
//(lldb) po this
//▿ Model
// - x : 30.0
}
}
var m = Model()
class ViewModel
{
let testClosure:() -> ()
init(inout model: Model)
{
model.x = 50
testClosure =
{ () -> () in
model.modifyX(301)
}
model.x = 30
}
}
let mx = m.x
vm.testClosure()
let mx2 = m.x
Here is what Apple says about that.
Classes and Structures
A value type is a type that is copied when it is assigned to a
variable or constant, or when it is passed to a function. [...] All
structures and enumerations are value types in Swift
Methods
Structures and enumerations are value types. By default, the properties of a value type cannot be modified from within its instance methods.
However, if you need to modify the properties of your structure or
enumeration within a particular method, you can opt in to mutating
behaviour for that method. The method can then mutate (that is,
change) its properties from within the method, and any changes that it
makes are written back to the original structure when the method ends.
The method can also assign a completely new instance to its implicit
self property, and this new instance will replace the existing one
when the method ends.
Taken from here
I was surprised by the following playground I created after seeing some unexpected behavior in my code:
import Foundation
let bytes:[UInt8] = [20, 30, 40, 50, 60, 70]
var stream = bytes.generate()
func consumeTwo(var stream:IndexingGenerator<[UInt8]>) {
print(stream.next())
print(stream.next())
}
consumeTwo(stream) // This prints 20 and 30
print(stream.next()) // This prints 20!? I expected 40!
I had thought that marking the stream argument as var to the consumeTwo() function, the stream state/position would be shared/updated as it moved from function to function. But that appears to not be the case.
Does this mean I need to make that an inout? And pass with the ampersand? When does one use the var if that's the case?
More generally... what is the right/idiomatic way to create a stream over a byte array which can be passed from function to function (e.g. a decoder) and preserve the position of the stream as it is passed around?
+1 for archaic English in the question title. :)
When you use var in a function signature, you create a local copy of that value. It's the same as if you did this:
func consumeTwo(stream: IndexingGenerator<[UInt8]>) {
var localStream = stream
print(localStream.next())
print(localStream.next())
}
When the parameter is a reference type (i.e. a class), the duplicate "value" is a duplicate reference to the same object. But the thing you get from Array.generate() is a value type, so your local copy is a separate iterator with separate state.
Does this mean I need to make that an inout? And pass with the ampersand?
Yes — for your simple example, inout (and pass with &) is a simple and idiomatic way to do this:
func consumeTwo(inout stream:IndexingGenerator<[UInt8]>) {
print(stream.next())
print(stream.next())
}
consumeTwo(&stream) // This prints 20 and 30
print(stream.next()) // This prints 40
Remember: when you want to modify a value type inside a function and later see those modifications outside the function, use inout. And the & goes with it, so that it's clear both inside the function and at the call site that this behavior is happening.
When does one use the var if that's the case?
Use var for parameters only when you want to make a copy that's local to the function invocation. Admittedly, the use cases for this are few. Here's a contrived (and completely unnecessary) one:
func bananify(var strings: [String]) {
for i in 1.stride(to: strings.count, by: 2) {
strings[i] = "banana"
}
print(strings.joinWithSeparator(" "))
}
let words = ["foo", "bar", "bas", "zap", "asdf"]
bananify(words) // "foo banana bas banana asdf\n"
If you find this confusing, you're not the only one. For this reason, removing the ability to use var for parameters is a planned change for Swift 3.
More generally... what is the right/idiomatic way to create a stream over a byte array which can be passed from function to function (e.g. a decoder) and preserve the position of the stream as it is passed around?
As user3441734 notes, you can indeed create and use a reference-type iterator instead. Or you can write a reference type that holds and manages an iterator. For your hypothetical case of sharing a stream among several subsystems of a program, this is probably a good approach — representing a shared resource is one of the canonical cases for using reference types.
you wrote "It makes me wish that Generators were objects instead of structs."
there is no trouble define some generator as reference type ...
class G: AnyGenerator<Int> {
var i = 0
override func next() -> Int? {
return i++
}
}
let g = G()
func foo(gen: G)->Void {
print(gen.next())
print(gen.next())
}
foo(g)
print(g.next())
/*
Optional(0)
Optional(1)
Optional(2)
*/