High Memory Allocation debugging with Apple Instruments - swift

I have an app written in swift which works fine initially, but throughout time the app gets sluggish. I have opened an instruments profiling session using the Allocation and Leaks profile.
What I have found is that the allocation increases dramatically, doing something that should only overwrite the current data.
The memory in question is in the group < non-object >
Opening this group gives hundreds of different allocations, with the responsible library all being libvDSP. So with this I can conclude it is a vDSP call that is not releasing the memory properly. However, double clicking on any of these does not present me with any code, but the raw language I do not understand.
The function that callas vDSP is wrapped like this:
func outOfPlaceComplexFourierTransform(
setup: FFTSetup,
resultSize:Int,
logSize: UInt,
direction: FourierTransformDirection) -> ComplexFloatArray {
let result = ComplexFloatArray.zeros(count:resultSize)
self.useAsDSPSplitComplex { selfPointer in
result.useAsDSPSplitComplex { resultPointer in
vDSP_fft_zop(
setup,
&selfPointer,
ComplexFloatArray.strideSize,
&resultPointer,
ComplexFloatArray.strideSize,
logSize,
direction.rawValue)
}
}
return result
}
This is called from another function:
var mags1 = ComplexFloatArray.zeros(count: measurement.windowedImpulse!.count)
mags1 = (measurement.windowedImpulse?.outOfPlaceComplexFourierTransform(setup: fftSetup, resultSize: mags1.count, logSize: UInt(logSize), direction: ComplexFloatArray.FourierTransformDirection(rawValue: 1)!))!
Within this function, mags1 is manipulated and overwrites an existing array. It was my understanding that mags1 would be deallocated once this function has finished, as it is only available inside this function.
This is the function that is called, many times per second at times. Any help would be appreciated, as what should only take 5mb, very quickly grows by two hundred megabytes in a couple of seconds.
Any pointers to either further investigate the source of the leak, or to properly deallocate this memory once finished would be appreciated.

I cannot believe I solved this so quickly after posting this. (I genuinely had several hours of pulling my hair out).
Not included in my code here, I was creating a new FFTSetup every time this was called. Obviously this is memory intensive, and it was not reusing this memory.
In instruments looking at the call tree I was able to see the function utilising this memory.

Related

Memory usage growing seems to be linked to Metal objects

I am currently building an app that uses Metal to efficiently render triangles and evaluate a fitness function on textures, I noticed that the memory usage of my Metal app is growing and I can't really understand why.
First of all I am surprised to see that in debug mode, according to Xcode debug panel, memory usage grows really slowly (about 20 MB after 200 generated images), whereas it grows way faster in release (about 100 MB after 200 generated images).
I don't store the generated images (at least not intentionally... but maybe there is some leak I am unaware of).
I am trying to understand where the leak (if it one) comes from but I don't really know where to start, I took a a GPU Frame capture to see the objects used by Metal and it seems suspicious to me:
Looks like there are thousands of objects (the list is way longer than what you can see on the left panel).
Each time I draw an image, there is a moment when I call this code:
trianglesVerticesCoordiantes = device.makeBuffer(bytes: &positions, length: bufferSize , options: MTLResourceOptions.storageModeManaged)
triangleVerticiesColors = device.makeBuffer(bytes: &colors, length: bufferSize, options: MTLResourceOptions.storageModeManaged)
I will definitely make it a one time allocation and then simply copy data into this buffer when needed, but could it cause the memory leak or not at all ?
EDIT with screenshot of instruments :
EDIT #2 : Tons of command encoder objects present when using Inspector:
EDIT #3 : Here is what seems to be the most suspect memory graph when analysed which Xcode visual debugger:
And some detail :
I don't really know how to interpret this...
Thank you.

How to clean the memory properly (conceptual, SWIFT)

I have a loop where I add objects (.append) to an array and remove them after a while in a Background-Task (dispatch_async(backgroundQueue, {)
During this loop my array has theoretically a max amount of objects in it, because I am removing old objects.
What I observe is, that the memory cunsumption goes high (over 400MB) till I get a didReceiveMemoryWarning Warning and my didReceiveMemoryWarning is called and sometimes the app just crashes. So it looks like my cleaning doesn't work, BUT
When stopping in the middle of the loop and waiting for some seconds (say 10 seconds) I see the memory slowly going down to my expected value (60MB).
So I guess I conceptually doing this wrong.
How to do this properly ? Is there a way to force cleaning memory like in Java forcegc ?
When cleaning the objects in the main task (dispatch_async(dispatch_get_main_queue(),{) the memory doesn't reach the limit, but the GUI is stocking.

“EXC_BAD_ACCESS: Unable to restore previously selected frame” - stack variables not getting cleaned up

With reference to the question here.
The solutions provided in that thread, include:
Being smart about where and how many variables you declare, that would go on the stack.
Disable Guard Malloc if needed.
Also, in general:
3. Make sure you are releasing a variable you have actually allocated memory for!!
I have none of the above issues. There are very few stack-allocated variables that I use within functions, say 2-3 within each. But because the functions are called in a loop several times, it seems to trigger an exception.
And here's the kicker, this is not just happening for statically allocated variables, but stuff on the heap too! I'm getting the page errors on Auto-released declarations of NSNumber variables that I use a couple of inside the loop, as well as a dictionary that I juDIciously alloc and release within the scope of the function, every single time it's called.
So why is this happening, and why the heck are heap variables getting affected? I don't get it at all, please throw some light on this. :)
I'm on IOS5 with XCode 4.2, iPhone/iPad simulator.
Thanks!
Regards,
Dev
Edit: Sample code
- (void)doSomething {
NSInteger fun = 3;
NSInteger time = 4;
NSInteger overload = fun*time;
NSString *string = [NSString stringWithFormat:#"%d",overload];
NSObject *myCustomObject = [[NSObject alloc] init];
[myCustomDictionary setObject:myCustomObject forKey:string];
[myCustomObject release];
//myCustomDictionary is an iVar, alloced in the class's init method, and released in dealloc and not touched anywhere in between
}
//doSomething gets called several times through the course of execution as the state of the view changes, the user interacts with it etc, often 2-3 times during one state change.
The code is purposely vague, but at the same time, it is EXACTLY as simple as in the sample. As is the rest of the code in the entire project. Several functions, each doing a small amount of work, as nicely self contained memory-wise as this one.
I faced EXC_BAD_ACCESS issues before, and at that point had referred to this question. However in my case, I was not creating multiple variables on the stack within a loop, they were getting created repeatedly by a function which gets called several times through the course of the execution. Ideally, the variables should have just got destroyed at the end of function-scope. Not sure why that didn't happen.
Anyway, to resolve that and to prevent multiple allocations from happening, I ended up declaring my stack-allocated variables as all static. That is bad practice, but that's exactly what I had to do to get it working. And it was working until I ended up facing the issue AGAIN with the "doSomething" function.
So the difficulty in "doSomething", was that I did not have only stack-allocated variables getting created, but heap stuff too. So I first started getting EXC_BAD_ACCESS on the NSInteger variables, at which point I tried fixing it again, by declaring them as static. It worked, but now EXC_BAD_ACCESS started occurring on the auto-released variable and finally the custom-allocated variable - which is when I got stumped. I have been following all the rules of memory management, and I'm having stack AND heap variables fubaring all over me. If it was only heap stuff, or stack stuff inside a loop, I could understand there's a mistake SOMEwhere. But here, it's neither, these are perfectly innocent variables getting allocated on the stack inside ONE function that is NOT called in a loop, and regular auto-released variables that never ever get retains or releases thrown at them from another place in the code. What makes it all even worse, is that the failure points are random - not just in this function but practically every one that gets called several times through the course of execution of the project.
Edit2: Turns out, in this case, it's my fault. See my answer for details. Sorry for wasting people's time. :\
We can't really help you without seeing the whole stack trace.
EXC_BAD_ACCESS doesn't mean anything in order to troubleshoot we need to know what the exception is.
Based on my experience when you don't get a stack tarce it means you are double releasing. Zombies is the way to find your double release.
Go to: Product -> Profile and then select "Zombies" from the list.
Run the app and perform any task that causes the crash, if the problem is a double release a pop-up would appear. Select the arrow in the popup and it tells you exactly what object is being double released, and it shows you the retain cycle.
Without any code posted in your question, and seeing that you are using iOS 5 and Xcode 4.2 my best advice for you is, in Xcode, go to Edit>Refactor>Convert to Objective-C ARC and wave all your memory management headaches goodbye.
ARC does all the memory management for you. You do not need to retain, release or write dealloc methods. You don't have to worry about memory management in most cases. You will leave mysterious EXC_BAD_ACCESS crashes behind. The way it works is super efficient. The compiler puts in the retains and releases for you and then optimises. You never even have to see the code.
Ah geez I don't believe this.
The issues faced earlier with the stack-allocated NSInteger variables holds, but in this case, it was entirely my fault.
"doSomething" was part of a long chain of events, that, due to some silly negligence on my part, ends up looping a bunch of times, which is what, rightfully and as it should be, leads to the application running out of memory. Whether it's with the stack variables or with the malloced ones sitting on the heap, with an infinite loop, it's gonna crash one way or another. :)
So the crash is perfectly fair, just got confused in this case with a previous issue that turned out to be unrelated. :(
Sorry for wasting everybody's time.

What can I do to find out what's causing my program to consume lots of memory over time?

I have an application using POE which has about 10 sessions doing various tasks. Over time, the app starts consuming more and more RAM and this usage doesn't go down even though the app is idle 80% of the time. My only solution at present is to restart the process often.
I'm not allowed to post my code here so I realize it is difficult to get help but maybe someone can tell me what I can do find out myself?
Don't expect the process size to decrease. Memory isn't released back to the OS until the process terminates.
That said, might you have reference loops in data structures somewhere? AFAIK, the perl garbage collector can't sort out reference loops.
Are you using any XS modules anywhere? There could be leaks hidden inside those.
A guess: your program executes a loop for as long as it is running; in this loop it may be that you allocate memory for a buffer (or more) each time some condition occurs; since the scope is never exited, the memory remains and will never be cleaned up. I suggest you check for something like this. If it is the case, place the allocating code in a sub that you call from the loop and where it will go out of scope, and get cleaned up, on return to the loop.
Looks like Test::Valgrind is a tool for searching for memory leaks. I've never used it myself though (but I used plain valgrind with C source).
One technique is to periodically dump the contents of $POE::Kernel::poe_kernel to a time- or sequence-named file. $poe_kernel is the root of a tree spanning all known sessions and the contents of their heaps. The snapshots should monotonically grow if the leaked memory is referenced. You'll be able to find out what's leaking by diff'ing an early snapshot with a later one.
You can export POE_ASSERT_DATA=1 to enable POE's internal data consistency checks. I don't expect it to surface problems, but if it does I'd be very happy to receive a bug report.
Perl can not resolve reference rings. Either you have zombies (which you can detect via ps axl) or you have a memory leak (reference rings/circle)
There are a ton of programs to detect memory leaks.
strace, mtrace, Devel::LeakTrace::Fast, Devel::Cycle

iPhone OS memory problem - how to debug?

I have a pretty weird problem in my iPhone app which is, I think, related to memory getting corrupted:
At one point, I need to sort an array, which I do with -[sortArrayUsingFunction].
The result is not correct unless I either allocate some memory with something like void *test = malloc(2 * sizeof( int )) before the method call or have, e.g., a call to NSLog() (which is never invoked) in the sorting function.
In other words: the sorting only works if I slightly increase the memory that was used before calling the sorting function. I think this is because at some point, memory gets corrupted.
How do you debug something like this?
It sounds like some of your code is using already released objects. A lot of help with debugging this kind of errors is provided in Apple’s great Mac OS X Debugging Magic tech note, especially the foundation part.
For your case I'd disable autorelease pools (setting the environment variable NSEnableAutoreleasePool=NO) or use the zombie feature (NSZombieEnabled=YES) to find places where you send messages to released objects.
Try running your program in the simulator under Valgrind:
http://valgrind.org/
And how to use it under the simulator:
http://landonf.bikemonkey.org/code/iphone/iPhone_Simulator_Valgrind.20081224.html
You may have to change the VALGRIND path in the code example depending on where it gets installed.
Such things can be a challenge to debug. There are some tools for detecting out-of-bounds accesses and such on other platforms, so I presume there would be something for the iPhone, however I don't know of any.
Perhaps you should store two copies of the array, and compare them for differences. Print out the differences. The nature of the "junk" that was introduced to one of the arrays might give a hint as to where it came from.
Also just go through the code that runs before this point, and re-read it (or better yet, get someone else to read it). You might spot a bug.