Was trying to fix a 300MB memory-leak, and after finding leak-reason;
(Which was calls to NSString's stringFromUTF8String:, from C++ thread (without #autoreleasepool-block wrapper))
I edited the code, to enforce reference-counting (instead of auto-release), something like below:
public func withNSString(
_ chars: UnsafePointer<Int8>,
_ callback: (NSString) -> Void
) {
let result: NSString = NSString(utf8String: chars)!;
callback(result);
}
As personal policy, with a Unit-Test, like:
import Foundation
import XCTest
#testable import MyApp
class AppTest: XCTestCase {
func testWithNSString_hasNoMemoryLeak() {
weak var weakRef: NSString? = nil
autoreleasepool {
let chars = ("some data" as NSString).utf8String!;
withNSString(chars, { strongRef in
weakRef = strongRef;
XCTAssertNotNil(weakRef);
})
// Checks if reference-counting is used.
XCTAssertNil(weakRef); // Fails, so no reference-counting.
}
// Checks if autoreleased.
XCTAssertNil(weakRef); // Fails, OMG! what is this?
}
}
But now, not even auto-release seems to work anymore (-_- )
Why does last XCTAssertNil call fail?
(In other words, how can I fix memory-leaks?)
The problem is that you're using a very short string. It's getting inlined onto the stack, so it's not released until the entire stack frame goes out of scope. If you made the string a little bit longer (2 characters longer), this would behave the way you expect. This is an implementation detail, of course, and could change due to different versions of the compiler, different versions of the OS, different optimization settings, or different architectures.
Keep in mind that testing this kind of thing with static strings of any kind can be tricky, since static strings are placed into the binary. So if the compiler notices that you've indirectly made a pointer to a static string, then it might optimize out the indirection and not release it.
In none of these cases is there a memory leak, though. Your memory leak is more likely in the calling code of withNSString. I would mostly suspect that you're not properly dealing with the bytes passed as chars. We would need to see more about why you think there's a leak to evaluate that. (Foundation also has some small leaks, and Instruments has false positives on leaks, so if you're chasing an allocation that is smaller than 50 bytes and doesn't recur on every operation, you probably are chasing ghosts.)
Note that this is a bit dangerous:
let chars = ("some data" as NSString).utf8String!
withNSString(chars, { strongRef in
The utf8String inner pointer is not promised to live longer than the NSString, and Swift is free to destroy objects after their last reference (which may be before they go out of scope). As the docs note:
This C string is a pointer to a structure inside the string object, which may have a lifetime shorter than the string object and will certainly not have a longer lifetime. Therefore, you should copy the C string if it needs to be stored outside of the memory context in which you use this property.
In this case the object is a constant string, which is in the binary and cannot be destroyed. But in more general cases this is is a classic cause of crashes. I would highly recommend moving away from the NSString interfaces and using String. It offers utf8CString, which returns a proper ContinguousArray, which is much safer.
let chars = "some data".utf8CString
chars.withUnsafeBufferPointer { buffer in
withNSString(buffer.baseAddress!, { strongRef in
weakRef = strongRef;
XCTAssertNotNil(weakRef);
})
}
withUnsafeBufferPointer ensures that chars cannot be destroyed before the block completes.
You can also ensure the lifetime of the string if needed (this is mostly useful for fixing older code you don't want to rewrite in safer ways):
let string = "some data"
withExtendedLifetime(string) {
let chars = string.utf8CString
chars.withUnsafeBufferPointer { buffer in
withNSString(buffer.baseAddress!, { strongRef in
weakRef = strongRef;
XCTAssertNotNil(weakRef);
})
}
}
Related
I was doing some research on mutexes and came across the following Swift code:
class Lock {
private var mutex: pthread_mutex_t = {
var mutex = pthread_mutex_t()
pthread_mutex_init(&mutex, nil)
return mutex
}()
func someFunc() {
pthread_mutex_lock(&mutex)
defer { pthread_mutex_unlock(&mutex) }
...
}
}
The code defines and initializes a pthread_mutex_t within the closure, then assigns the returned value to a class property. It then lock and unlocks within several of the functions as shown.
Since one should also call pthread_mutex_destroy, it implies that some sort of allocation is occurring within the mutex which may or may not reference the address of the original value.
In effect, the mutex is initialized in one place and stored in another.
The question is whether or not it's safe or correct to do this?
What if the mutex initializer needed arguments?
private var mutex: pthread_mutex_t = {
var recursiveMutex = pthread_mutex_t()
var recursiveMutexAttr = pthread_mutexattr_t()
pthread_mutexattr_init(&recursiveMutexAttr)
pthread_mutexattr_settype(&recursiveMutexAttr, PTHREAD_MUTEX_RECURSIVE)
pthread_mutex_init(&recursiveMutex, &recursiveMutexAttr)
return recursiveMutex
}()
The later strikes me as definitely being incorrect, as the attribute storage whose address is passed into the mutex will disappear when the closure collapses.
It's not, this code is broken.
To work, the pthread_mutex_t would need to be initialized in-place within a class instance, and never copied out. The class would need to expose lock/unlock methods which operator on the instance-variable in-place.
Value Types
Note that pthread_mutex_t, pthread_rwlock_t, and os_unfair_lock are value types, not reference types. That means that if you use = on them, you make a copy. This is important, because these types can't be copied! If you copy one of the pthread types, the copy will be unusable and may crash when you try to use it.
– By Mike Ash, Friday Q&A 2017-10-27: Locks, Thread Safety, and Swift: 2017 Edition
Check out https://cocoawithlove.com/blog/2016/06/02/threads-and-mutexes.html
And this example usage of pthread_mutex_t in Swift: https://github.com/mattgallagher/CwlUtils/blob/0bfc4587d01cfc796b6c7e118fc631333dd8ab33/Sources/CwlUtils/CwlMutex.swift#L60-L105
I have the following class, grossly simplified
class MyClass {
var largeArray: [Int] = []
init() {
largeArray.reserveCapacity(10000000)
... lots of code to add 10000000 various elements to largeArray
}
func mutateArray(idx: Int) {
largeArray[idx] = someVal
}
}
Surprisingly, when profiling this code, calls to mutateArray turned out to be very expensive, with most of the time spent
in _ArrayBufferProtocol.init(copying:), and some in _swift_release_dealloc. The time spent is proportional to the number of calls to mutateArray, indicating
that this happens every time the method is called.
Why is this happening? Is there any way to avoid it?
Your array's buffer leaks out of its MyClass encapsulation somewhere.
If largeArray is initialized within a MyClass object, it has sufficient capacity reserved up front, and you've never let anyone else have access to your class, or alias it yourself, then you couldn't possibly cause a CoW copy.
You should set var largeArray to private. Not only will that enforce the capsulation you're in need of, but it'll also show you what else is accessing this.
Suppose I have a Swift class like this:
#objc final MyClass : NSObject
{
let classPropertyString = "A class property"
func doStuff()
{
let localString = "An object local to this function"
DispatchQueue.global(qos: .userInitiated).async { [classPropertyString] in
// Do things with 'classPropertyString' and 'localString'
}
}
}
My question is: when I write a capture list, am I responsible for EXHAUSTIVELY listing all the things to which I want the closure to hold a strong reference?
In other words, if I omit localString from my capture list (as I've done here), will the closure still automatically capture a strong reference to it or am I in for a bad time?
There are several minor quirks with your question that make it tricky to answer clearly, but I think I understand the underlying concern, and the short answer is "no." But your example is impossible, so the answer is "it's impossible." And if it were possible, there'd be no strong reference (nor would there be a need for one), so the question still would be kind of "it's impossible." Even so, let's walk through what's going on here.
First, closure can't reference localString unless it's reassigned somehow in the comment inside doStuff(). closure is assigned at a level where localString is not in scope. Closures can only capture variables that are in scope when they are assigned, not when they're called. But let's go back to the original version of this question, before it was edited. That version did have the case you're describing:
#objc final myClass : NSObject
{
let classPropertyString = "A class property"
func doStuff()
{
let localString = "An object local to this function"
DispatchQueue.global(qos: .userInitiated).async { [classPropertyString] in // (1)
// Do things with 'classPropertyString' and 'localString'
}
// (2)
}
}
There's no problems here. classPropertyString is copied into the closure, avoiding any retain loops. localString is referenced by the closure, and so it's preserved as long as the closure exists.
Because you listed classPropertyString in the capture list, it is evaluated at point (1) and copied into the closure. Because you implicitly captured localString, it is treated as a reference. See Capture Lists in the Swift Programming Language Reference for some excellent examples of exactly how this works in different cases.
In no case (*) will Swift allow the underlying storage for something you're using in a closure to disappear behind your back. That's why the typical concern is excessive retains (memory leaks) rather than dangling references (crashes).
(*) "In no case" here is a lie. There are several ways that Swift will allow it, but almost all of them involve "Unsafe" which is your warning about that. The major exception is unowned, and of course anything involving ! types. And Swift is not typically thread-safe, so you need to be careful about that...
The last comment about thread-safety is a place where the subtle distinctions between implicit and explicit captures can really matter. Consider this case where you modify an implicitly captured value on two queues:
func doStuff() -> String
{
var localString = "An object local to this function"
DispatchQueue.global(qos: .userInitiated).async {
localString = "something else"
callFunction(localString)
}
localString = "even more changes"
return localString
}
What happens in that case? Good grief, never do that. I believe it's undefined behavior and that localString could be anything including corrupted memory, at least in the most general case (it might be defined behavior for calling .async; I'm not sure). But don't do it.
But for your normal cases, there is no reason to explicitly capture local variables. (I sometimes wish Swift had gone the C++ way and said it was required, but it isn't.)
Ok, one more way implicit and explicit are different that might drive home how they work. Consider a stateful closure like this (I build these pretty often):
func incrementor() -> () -> Int {
var n = 0
return {
n += 1
return n
}
}
let inc = incrementor()
inc() // 1
inc() // 2
inc() // 3
let inc2 = incrementor()
inc2() // 1
See how the local variable n is captured by the closure, and can be modified after it goes out of scope. And see how inc2 has its own version of that local variable. Now try that with explicit capture.
func incrementor() -> () -> Int {
var n = 0
return { [n] in // <---- add [n]
n += 1 // Left side of mutating operator isn't mutable: 'n' is an immutable capture
return n
}
}
Explicit captures are copies and they're immutable. Implicit captures are references, and so have the same mutability as the thing they reference.
Below is my source code, every time I execute the function, the memory usage increases dramatically. Please help to point out what is the problem.
func loadfontsFromDatabase(code:String)->[String] {
let documentsPath : AnyObject = NSSearchPathForDirectoriesInDomains(.documentDirectory,.userDomainMask,true)[0] as AnyObject
let databasePath = documentsPath.appending("/bsmcoding.sqlite")
let contactDB = FMDatabase(path: databasePath as String)
var c:[String]=[]
let querySQL = "SELECT FONT FROM BSMCODE WHERE BSMCODE.CODE = '\(code)' ORDER BY NO DESC"
NSLog("query:\(querySQL)")
let results:FMResultSet? = Constants.contactDB?.executeQuery(querySQL, withArgumentsIn: nil)
while (results?.next())! {
c.append((results?.string(forColumn: "FONT"))!)
}
results?.close()
return c
}
There's nothing here that would account for any substantial memory loss. I would suggest using the "Debug Memory Graph" feature in Xcode 8 to identify what objects are being created and not being released, but I suspect the problem rests elsewhere in your code. Or use Instruments to track it down what's leaking and debug from there. See https://stackoverflow.com/a/30993476/1271826.
There are unrelated issues here, though:
You are creating local contactDB, but you never open it and you never use it. It will be released when the routine exits, but it's completely unnecessary if you're going to use Constants.contactDB, anyway.
I'd advise against using string interpolation when building your SQL. Use ? placeholder and pass the code in as a parameter. This is much safer, in case the code ever contained something that couldn't be represented in SQL statement. (This is especially true if the code was supplied by the user, in which case you'd be susceptible to SQL injection attacks or innocent input errors that could lead to crashes.)
For example, you could do something like:
func loadfontsFromDatabase(code: String) -> [String] {
var c = [String]()
let querySQL = "SELECT FONT FROM BSMCODE WHERE BSMCODE.CODE = ? ORDER BY NO DESC"
let results = try! Constants.contactDB!.executeQuery(querySQL, values: [code])
while results.next() {
c.append((results.string(forColumn: "FONT"))!)
}
return c
}
If you don't like the forced unwrapping, you can do optional unwrapping if you want, but personally I'd rather know immediately when debugging during the development phase if there's some logic mistake (e.g. the contactDB wasn't open, the SQL is incorrect, etc.). But you can do optional binding and add the necessary guard statements if you want. But don't just do optional binding and silently return a value suggesting that everything is copacetic, leaving you with a debugging challenge of tracking down the problem if you don't get what you expected.
But the key point is to avoid inserting values into your SQL directly. Use ? placeholders.
OK... so I have no idea why this happens but:
Compare the following two lines:
let pointCurve: [AnyObject] = self.curve.map{NSValue(point:$0)}
and
let pointCurve: [NSPoint] = self.curve.map{$0}
In either case, the variable is local and not used at all after assignment. The line resides in a method that is called repeatedly and very quickly. The first case results in terrible and ever faster growing of memory usage. But when I change it to the second, the memory stats are flat as a disc.
You may say, "oh, you're not doing anything in the second line". So I tried the following:
var pointCurve: [AnyObject] = []
for c in self.curve {
pointCurve.append(NSValue(point:NSPoint(x:1, y:1))
}
vs
var pointCurve: [NSPoint] = []
for c in self.curve {
pointCurve.append(NSPoint(x: 1, y: 1))
}
Now I see the exact same results. The culprit seems to be NSValue. I checked with Instruments that a whole bunch of NSConcreteValues are allocated, and I read online these are related to NSValue. But I didn't find anything about them causing memory leaks.
The question is what can I do about this. I'm supposed to send an array of points to some ObjC code, and until I figure out how to fix this, I can't do it without huge performance issues.
Try:
func pointCurvy() {
autoreleasepool {
let pointCurve: [AnyObject] = self.curve.map{NSValue(point:$0)}
// Do something with pointCurve.
}
}