I have a loop where I add objects (.append) to an array and remove them after a while in a Background-Task (dispatch_async(backgroundQueue, {)
During this loop my array has theoretically a max amount of objects in it, because I am removing old objects.
What I observe is, that the memory cunsumption goes high (over 400MB) till I get a didReceiveMemoryWarning Warning and my didReceiveMemoryWarning is called and sometimes the app just crashes. So it looks like my cleaning doesn't work, BUT
When stopping in the middle of the loop and waiting for some seconds (say 10 seconds) I see the memory slowly going down to my expected value (60MB).
So I guess I conceptually doing this wrong.
How to do this properly ? Is there a way to force cleaning memory like in Java forcegc ?
When cleaning the objects in the main task (dispatch_async(dispatch_get_main_queue(),{) the memory doesn't reach the limit, but the GUI is stocking.
Related
I have an app written in swift which works fine initially, but throughout time the app gets sluggish. I have opened an instruments profiling session using the Allocation and Leaks profile.
What I have found is that the allocation increases dramatically, doing something that should only overwrite the current data.
The memory in question is in the group < non-object >
Opening this group gives hundreds of different allocations, with the responsible library all being libvDSP. So with this I can conclude it is a vDSP call that is not releasing the memory properly. However, double clicking on any of these does not present me with any code, but the raw language I do not understand.
The function that callas vDSP is wrapped like this:
func outOfPlaceComplexFourierTransform(
setup: FFTSetup,
resultSize:Int,
logSize: UInt,
direction: FourierTransformDirection) -> ComplexFloatArray {
let result = ComplexFloatArray.zeros(count:resultSize)
self.useAsDSPSplitComplex { selfPointer in
result.useAsDSPSplitComplex { resultPointer in
vDSP_fft_zop(
setup,
&selfPointer,
ComplexFloatArray.strideSize,
&resultPointer,
ComplexFloatArray.strideSize,
logSize,
direction.rawValue)
}
}
return result
}
This is called from another function:
var mags1 = ComplexFloatArray.zeros(count: measurement.windowedImpulse!.count)
mags1 = (measurement.windowedImpulse?.outOfPlaceComplexFourierTransform(setup: fftSetup, resultSize: mags1.count, logSize: UInt(logSize), direction: ComplexFloatArray.FourierTransformDirection(rawValue: 1)!))!
Within this function, mags1 is manipulated and overwrites an existing array. It was my understanding that mags1 would be deallocated once this function has finished, as it is only available inside this function.
This is the function that is called, many times per second at times. Any help would be appreciated, as what should only take 5mb, very quickly grows by two hundred megabytes in a couple of seconds.
Any pointers to either further investigate the source of the leak, or to properly deallocate this memory once finished would be appreciated.
I cannot believe I solved this so quickly after posting this. (I genuinely had several hours of pulling my hair out).
Not included in my code here, I was creating a new FFTSetup every time this was called. Obviously this is memory intensive, and it was not reusing this memory.
In instruments looking at the call tree I was able to see the function utilising this memory.
I have deleted some variables in jupiter notebook using del list_of_df. But we realize the contents still occupies memory. so we tried %reset list_of_df , but the previous variable names are already not there... Is there nothing we could do but to restart the kernel to recollect the memory? Thanks
Further:
In a more general, I might have lost track of what I have deleted from a huge jupiter notebook codes. Is it possible to check what have been deleted but still occupying the memory and delete it?
Python isn't like C (for example) in which you have to manually free memory. Instead, all memory allocation and deallocation tasks are handled automatically in the background by a garbage collection (GC) routine. GC uses lazy evaluation, which means that the memory probably won't be freed right away, but will instead be freed only when it "needs" to be (in the ideal case, anyway).
You shouldn't use this in production code, but if you really want to, after you del your list you can force GC to run using the gc module:
import gc
gc.collect()
It might not actually work/deallocate the memory, though, for many different reasons. In general, it's better to just let Python manage memory automatically and not interfere.
Is it possible to manually clear out the contents of an object from memory?
In particular, I'm dealing with NSData. I've tried using data.length = 0 and data.setData(NSData).
I know ARC will come in and clean up after it is out of scope to whom it belongs, but is it possible to manually force this process when I want?
I think you have some misconceptions about ARC I'd like to clear up. The goal of ARC is is to ensure memory leaks don't occur. It's responsible for tracking the object over its life cycle, and ensuring it's "freed" when no references remain to it.
It's important to note that the memory being "freed" does not imply "writing over it all with 0s".
It simply means that memory will be designated as unused. The freed memory becomes a candidate for allocation when the system needs to allocate memory to new objects.
There's no guarentee, however, that this reallocation will happen, thus it's very possible for your freed memory to contain your original data, and never be overwritten.
I'm trying to loop through an array and copy across data, but after 1023 loops, I get an exc bad access message thrown and I have a feeling it might be to do with my memory.
In my loop, I need to append data to my totalValues array, so I did this:
memcpy(totalValues + totalCopied, tempBuffer, 600 * sizeof(float));
This is done inside a loop and totalCopied keeps track of how much data has been appended to totalValues so that I know where to write from when the loop hits memcpy again. I'm not sure why I get the "exc bad access" error, but my theory is that the memory is not contiguous and, therefore, the totalValues + totalCopied line might be causing trouble. I'm not sure if an error would be thrown in this case, or if the memory would just be overwritten anyway. The interesting thing is, it always occurs after 1023 loops. If I remove the 'memcpy' line, the program loops through without any problems. Any ideas what could be causing this?
EDIT - The cause was that the memory allocation was hard coded for another file. Normally, I won't know the length of the file before the memory allocation, so how can I ensure that enough memory is allocated at runtime?
Sounds like you're writing more bytes than totalValues can contain. Show us how you're allocating it.
Incidentally, we usually do this kind of thing with NSData objects on iOS.
I have an application using POE which has about 10 sessions doing various tasks. Over time, the app starts consuming more and more RAM and this usage doesn't go down even though the app is idle 80% of the time. My only solution at present is to restart the process often.
I'm not allowed to post my code here so I realize it is difficult to get help but maybe someone can tell me what I can do find out myself?
Don't expect the process size to decrease. Memory isn't released back to the OS until the process terminates.
That said, might you have reference loops in data structures somewhere? AFAIK, the perl garbage collector can't sort out reference loops.
Are you using any XS modules anywhere? There could be leaks hidden inside those.
A guess: your program executes a loop for as long as it is running; in this loop it may be that you allocate memory for a buffer (or more) each time some condition occurs; since the scope is never exited, the memory remains and will never be cleaned up. I suggest you check for something like this. If it is the case, place the allocating code in a sub that you call from the loop and where it will go out of scope, and get cleaned up, on return to the loop.
Looks like Test::Valgrind is a tool for searching for memory leaks. I've never used it myself though (but I used plain valgrind with C source).
One technique is to periodically dump the contents of $POE::Kernel::poe_kernel to a time- or sequence-named file. $poe_kernel is the root of a tree spanning all known sessions and the contents of their heaps. The snapshots should monotonically grow if the leaked memory is referenced. You'll be able to find out what's leaking by diff'ing an early snapshot with a later one.
You can export POE_ASSERT_DATA=1 to enable POE's internal data consistency checks. I don't expect it to surface problems, but if it does I'd be very happy to receive a bug report.
Perl can not resolve reference rings. Either you have zombies (which you can detect via ps axl) or you have a memory leak (reference rings/circle)
There are a ton of programs to detect memory leaks.
strace, mtrace, Devel::LeakTrace::Fast, Devel::Cycle