I've heard rumors that calling malloc leads to so called "dirty memory", which you can see in the VM Tracker instrument.
Now, rumors also say one must try to keep the amount of dirty memory as low as possible. But what they didn't talk much about was how to undirty it again.
Sometimes there's no other option than using malloc(). Heck, I love malloc(). For example when creating audio sources for OpenAL, one must malloc() a lot of data.
So: When my app calls malloc() and free() all over the place, I always believed that's fine. Am I having a huge problem when doing that? Or will free() always "clean it up"? I'm a bit confused because some very big guys at a very big company warned that malloc() must be avoided as much as possible, because of this dirty memory problem.
Maybe someone can un-confuse me about that.
I seriously doubt this is true. All memory allocation in Cocoa is eventually done via malloc. so sayeth Apple's Memory Usage Performance Guidelines. Quoting from that document:
Because memory is such a fundamental resource, Mac OS X and iOS both
provide several ways to allocate it. Which allocation techniques you
use will depend mostly on your needs, but in the end all memory
allocations eventually use the malloc library to create the memory.
Even Cocoa objects are allocated using the malloc library eventually.
I don't know about your big guys at your big company, but I've known big guys at big companies that didn't know squat. Just sayin'. Documentation trumps rumors every time. :)
I don't know what they mean by "dirty" and "clean". Possibly they are referring to the problem of fragmentation. Doing lots of allocs and frees can cause fragmentation problems, but it really depends on the usage patterns and block sizes you are allocating. In general, don't worry about using malloc and free. If you a real reason to avoid the standard allocator, you can use your own allocator. Then you just call malloc once for a huge block that you can use as the basis of your custom allocator.
If you malloc and free same size memory blocks multiple times, the memory will be reused instead of accumulating dirty VM pages. So it's perfectly safe as long as you know the max of all possible fragment sizes ever allocated by your app at any one time, and that keeps your app under the OS kill limit.
You are correct - free() will simply clean it up again.
Related
I have been having some memory problems with an app. Im getting to a stage now where nothing really gives me a solid answer in terms of memory.
At first I used the Allocations profiler which I dont think seems to work that well at all, I think this is due to the fact most of my code is in Obj-C++ meaning it cant track the memory correctly.
With the Allocations profiler it tells me the app uses 32mb of memory and around this point it says it has low memory and sometimes crashes out. However, within other parts of the applications its used up to 40mb and never crashed out.
I found this code chunk:
http://landonf.bikemonkey.org/code/iphone/Determining_Available_Memory.20081203.html
Which appears to tell me im using 70mb of memory, when I get the low memory warning it says I have 2mb - 4mb left of unused memory. Which seems more reasonable, but its almost double what the profiler says!
The only thing I can think of is just ignoring it all and reducing the amount of memory used by my app as much as possible.
Ignore it all and reduce the size of your app is actually a good way to proceed. Make sure that you're responding to memory warnings by purging anything in memory that you don't need. Remember that different devices have different amounts of memory, and you may need to use even less than you think, at least if you want to support those older devices.
My app starts with about 3 MB Live Bytes, and after 10 minutes of use it reaches 6 MB and also gets some Low Memory Warnings. But there are no Memory Leaks discovered by the Leaks instrument.
What can be the cause for this?
As #viggio24 says, there may well be an "abandoned memory" problem such as cashes that are not released retain cycles, etc.
Try Heapshot Analysis, bbum has a great tutorial here.
Basically you take a Heapshot, run some procedure, take another Heapshot for several iterations. This will help find memory that lost but not a leak. I use this method often,
I have used Heapshot many times to great advantage, many thanks to bum.
You are just using a lot of memory. A leak occurs when you load something into memory and then stop referencing it before you have released it. In your case it sounds like you are loading items into memory that you still have active references to so they don't show up as leaks.
This is a typical case of "abandoned memory". Not a leak in the strict sense of the definition, but almost. This means that you have retained some objects, you have not lost the references to them (so they are not leaked and analyzer and instruments don't trigger them) but you simply forgot to release them.
This is typical when you do some sort of internal caching (e.g.: images) and then you don't clean this cache of data which is no more needed.
I have a navigation based template, when I run my application using Instruments, the very first memory leak comes under:
Leak Object: Malloc 128 bytes
Responsible Library: CoreGraphics
Responsible Frame: open_handle_to_dylib_path
I don't know where this leak is coming from and how remove this.
If it's a default leak, then I think I don't need to worry about it. But if it's not then I have to find a way to remove the leak.
Expanding on deanWombourne perfectly correct answer a bit..
The Leaks Instrument shows you memory that isn't going to be free'd in the normal course of the app (because there aren't any references to it). This in itself is no biggie, it will be free'd when the app exits. A lot of the framework code will allocate and leave these very small chunks of memory allocated. I have no idea if they are mistakes or essential to how the app runs. Whatever, we must accept that they are completely normal.
Leaks will identify these chunks of memory as 'Leaks' and that sounds bad, but this is not really the 'Leaks' that the instrument is there to help you identify.
The 'real' leaks are in the code that can be run many times and which allocate some memory that is never freed, so over time will consume more and more memory until all memory is used and your app will crash.
So if you have an app that no matter how long you use it for or no matter how you use it, it 'leaks' 128 bytes in an apple framework you generally don't have to worry.
However, if you have an app that say, every time you click a button it allocates a new string which is never released - no matter how many bytes the string is - if the user pressed the button enough times this would consume all the memory available to app and eventually crash it. This is the kind of leak you need to watch out for.
The leaks instrument realistically can't tell the difference between the two kinds, but you need to be able to. You might want a kind of singleton object, for example, that there is only ever one instance of, and that needs to exist for the entire lifetime of your app. You create the object on app startup and realistically you never need to free this object, it can be killed when the app exits. Leaks will flag it as a leak, and some other developers that you work with who assume that this means you don't know what you are doing will run to your boss like a little child and say "He's writing really leaky code, and that's reeeeally bad". And your boss, who isn't a programmer will take him seriously because it does sound bad and anyway he scraped a 2.2 in CS from a reputable University so he must know what he's talking about. When really it is completely reasonable and exactly what you meant to do.
So, use the Leaks instrument to find bugs in your code that will ruin your app. Don't worry about every byte found 'Leaking' in an Apple framework.
If it's a one off 128 byte leak then don't worry about it at all, there are better ways to spend your time than thinking about this :)
I was curious, when one goes ahead and executes their code for leaks checking with Instruments -- is it prudent to hit all of the parts of the app manually to ensure memory leaks are occurring in that respective area? For instance, I believe I have some memory leaks in my application, deep within a UINavigationController tree. Do I go ahead and run the app checking for leaks, while manually drilling down on the iPhone to get to that part of the app? Is Instruments smart enough to find it on its own? What's the proper way of going about it?
Thanks for any insight!
No, Instruments just monitors the memory allocations of your code, it does not "go" anywhere, unless your app goes there. Actually a leak is nothing more than a piece of memory to that no reference exists anymore; thus it cannot be freed anymore, since how are you going to free it in the future if you cannot even reference to it any longer?
Instruments won't find all memory leaks that way, though. If you keep references to the memory, just never use them to free the memory, Instruments won't see this as a leak, because it cannot foresee if you are going to ever free it up in the future or not. As you still could free it up, it's not considered a leak. So if you have a memory issue, it might be beneficial to not just look for leaks, but also to monitor how much memory your application is "collecting" over the time. If this permanently rises even though it shouldn't, you might still have a leak, just none where you lose the references to the memory.
Typically, I would focus on the module suspected of causing the leaks and then widen the scope afterward. While I have not used Instruments on a Mac, I have used Purify and the native Windows heap tools to do memory leak tracing in Windows programs.
After you have identified the source of your major leak it's never a bad idea to run the program with varying test input and generally check the program for other leaks. Even small leaks with a particular set of data can lead to much larger leaks down the road.
I am working on a 2D iPhone game using OpenGL ES and I keep hitting the 24 MB memory limit – my application keeps crashing with the error code 101. I tried real hard to find where the memory goes, but the numbers in Instruments are still much bigger than what I would expect.
I ran the application with the Memory Monitor, Object Alloc, Leaks and OpenGL ES instruments. When the application gets loaded, free physical memory drops from 37 MB to 23 MB, the Object Alloc settles around 7 MB, Leaks show two or three leaks a few bytes in size, the Gart Object Size is about 5 MB and Memory Monitor says the application takes up about 14 MB of real memory. I am perplexed as where did the memory go – when I dig into the Object Allocations, most of the memory is in the textures, exactly as I would expect. But both my own texture allocation counter and the Gart Object Size agree that the textures should take up somewhere around 5 MB.
I am not aware of allocating anything else that would be worth mentioning, and the Object Alloc agrees. Where does the memory go? (I would be glad to supply more details if this is not enough.)
Update: I really tried to find where I could allocate so much memory, but with no results. What drives me wild is the difference between the Object Allocations (~7 MB) and real memory usage as shown by Memory Monitor (~14 MB). Even if there were huge leaks or huge chunks of memory I forget about, the should still show up in the Object Allocations, shouldn’t they?
I’ve already tried the usual suspects, ie. the UIImage with its caching, but that did not help. Is there a way to track memory usage “debugger-style”, line by line, watching each statement’s impact on memory usage?
What I have found so far:
I really am using that much memory. It is not easy to measure the real memory consumption, but after a lot of counting I think the memory consumption is really that high. My fault.
I found no easy way to measure the memory used. The Memory Monitor numbers are accurate (these are the numbers that really matter), but the Memory Monitor can’t tell you where exactly the memory goes. The Object Alloc tool is almost useless for tracking the real memory usage. When I create a texture, the allocated memory counter goes up for a while (reading the texture into the memory), then drops (passing the texture data to OpenGL, freeing). This is OK, but does not always happen – sometimes the memory usage stays high even after the texture has been passed on to OpenGL and freed from “my” memory. This means that the total amount of memory allocated as shown by the Object Alloc tool is smaller than the real total memory consumption, but bigger than the real consumption minus textures (real – textures < object alloc < real). Go figure.
I misread the Programming Guide. The memory limit of 24 MB applies to textures and surfaces, not the whole application. The actual red line lies a bit further, but I could not find any hard numbers. The consensus is that 25–30 MB is the ceiling.
When the system gets short on memory, it starts sending the memory warning. I have almost nothing to free, but other applications do release some memory back to the system, especially Safari (which seems to be caching the websites). When the free memory as shown in the Memory Monitor goes zero, the system starts killing.
I had to bite the bullet and rewrite some parts of the code to be more efficient on memory, but I am probably still pushing it. If I were to design another game, I would certainly think of some resource paging. With the current game it’s quite hard, because the thing is in motion all the time and loading the textures gets in the way, even if done in another thread. I would be much interested in how other people solve this issue.
Please note that these are just my views that do not have to be much accurate. If I find out something more to say on this topic, I will update the question. I’ll keep the question open in case somebody who understands the issue would care to answer, since these all are more workarounds and guesses than anything else.
I highly doubt this is a bug in Instruments.
First, read this blog post by Jeff Lamarche about openGL textures:
has a simple example of how to load
textures without causing leaks
gives understanding of how "small"
images, get once they are loaded
into openGL, actually use "a lot" of memory
Excerpt:
Textures, even if they're made from
compressed images, use a lot of your
application's memory heap because they
have to be expanded in memory to be
used. Every pixel takes up four bytes,
so forgetting to release your texture
image data can really eat up your
memory quickly.
Second, it is possible to debug texture memory with Instruments. There are two profiling configurations: OpenGL ES Analyzer and OpenGL ES Driver. You will need to run these on the device, as the simulator doesn't use OpenGL. Simply choose Product->Profile from XCode and look for these profiles once Instruments launches.
Armed with that knowledge, here is what I would do:
Check that you're not leaking memory -- this will obviously cause this problem.
Ensure your'e not accessing autoreleased memory -- common cause of crashes.
Create a separate test app and play with loading textures individually (and in combination) to find out what texture (or combination thereof) is causing the problem.
UPDATE: After thinking about your question, I've been reading Apple's OpenGL ES Programming Guide and it has very good information. Highly recommended!
One way is to start commenting out code and checking to see if the bug still happens. Yes it is tedious and elementary, but it might help if you knew where the bug was.
Where it is crashing is why it is crashing, etc.
Hrmm, that's not many details, but if leaks doesn't show you where the leaks are, there are two important options:
[i] Leaks missed a leak
[ii] The memory isn't actually being leaked
fixing [i] is quite hard, but as Eric Albert said filing a bug report with Apple will help. [ii] means that the memory you're using is still accessible somewhere, but perhaps you've forgotten about it. Are any lists growing, without throwing out old entries? Are any buffers being realloc()ed a lot?
For those seeing this after the year 2012:
The memory really loaded into device's physical memory is the Resident Memory in VM Tracker Instrument.
Allocation Instrument only marks the memory created by malloc/[NSObject alloc] and some framework buffer, for example, decompressed image bitmap is not included in Allocation Instrument but it always takes most of your memory.
Please Watch WWDC 2012 Session 242 iOS App Performance: Memory to get the information from Apple.
This doesn't specifically help you, but if you find that the memory tools don't provide all the data you need, please file a bug at bugreport.apple.com. Attach a copy of your app and a description of how the tools are falling short of your analysis and Apple will see if they can improve the tools. Thanks!