I'm looking at a crash dump. Some variables seem perfectly viewable in windbg, while others just say "memory access error". What causes this? Why do some variables have sensical values while others simply list ?
It appears that all the problems are associated with following pointers. I'm certain that while many of these pointers are uninitialized the vast majority of them should be pointing somewhere valid. Based on the nature of this crash (a simple null ptr dereference) I'm fairly certain the whole process hasn't gone out to lunch.
Mini-dumps are fairly useless, they don't contain a snapshot of all in use memory. Instead, all they contain are some critical structures/lists (e.g. the loaded module list) and the contents of the crashing stack.
So, any pointer that you try to follow in the dump will just give you question marks. Grab a full memory dump instead and you'll be able to see what these buffers point to.
-scott
If they are local pointer variables, what is most likely happening is that the pointers are not initialized, or that stack location has been reused to contain another variable, that may not be a pointer. In both cases, the pointer value may point to a random, unreadable portion of memory.
Related
I have 2 pieces of code :
It works normal
with ADOTemp do
begin
SQL.Clear;
SQL.Add('INSERT INTO documents');
SQL.Add('(document_date,fk_id_status,money_direction,');
SQL.Add('paid,addition,saving,fk_id_base,fk_id_user)');
SQL.Add('VALUES ');
SQL.Add('(CONVERT(DATE,GETDATE(),103),:pfk_id_status,:pmoney_direction,');
SQL.Add('0,0,0,'+IntToStr(p_id_base)+',:pfk_id_user)');
Parameters.ParamByName('pfk_id_status').Value := p_id_status;
Parameters.ParamByName('pmoney_direction').Value := p_money_direction;
// Parameters.ParamByName('p').Value := p_id_base;
Parameters.ParamByName('pfk_id_user').Value := fMain.ApplicationVariablers.user_id;
ExecSQL;
end;
It does not work and returns a terrible error
with ADOTemp do
begin
SQL.Clear;
SQL.Add('INSERT INTO documents');
SQL.Add('(document_date,fk_id_status,money_direction,');
SQL.Add('paid,addition,saving,fk_id_base,fk_id_user)');
SQL.Add('VALUES ');
SQL.Add('(CONVERT(DATE,GETDATE(),103),:pfk_id_status,:pmoney_direction,');
SQL.Add('0,0,0,:p,:pfk_id_user)');
Parameters.ParamByName('pfk_id_status').Value := p_id_status;
Parameters.ParamByName('pmoney_direction').Value := p_money_direction;
Parameters.ParamByName('p').Value := p_id_base;
Parameters.ParamByName('pfk_id_user').Value := fMain.ApplicationVariablers.user_id;
ExecSQL;
end;
Error on image, line 1917
It seems that this question is in reference to your earlier question: Delphi, error :"Access violation at address xxxxxxxx. Read of address yyyyyyyy", at AdoQuery.SQL.Text:='''
You had added and accepted an answer there along the lines seen in this question. The error message, an AV in msvcrt.dll is indicative of a serious programming error. Perhaps a heap corruption. Perhaps something else. The change you made that stops the error occurring does not really fix the problem. The problem will still be there, lying dormant. You just got (un)lucky that the change you made appeared to fix the problem.
Access violations are not always reproducible. That's just their nature. When you encounter one you need to understand why it happens. Simply using trial and error to re-organise code from a completely different location will never lead to the real solution.
What you should be doing is tracking down the real cause of the problem. Do that, fix it, and either version of the code above will work. This is probably not the answer you are wanting to get. You may very well not want to accept what I say. But, speaking from experience, until you recognise that you have a more serious problem, you will make no headway here.
Looking at your code in the bitmap...I'm pretty sure you access violation is because of what David just said...you have serious Memory problems in your code...Your not freeing your objects...that you are creating...Anytime you pass a nil reference to Owner in a TComponent Constructor...your telling the compiler...that you know what your doing and will free it when your finished with it...Matter of fact good practice is to always free objects that you declare and use exclusively in the scope of your method.
procedure TForm1.MyMethod;
var
a_MyComp: TMyComp;
begin
a_MyComp := TMyComp.Create(nil);
Try
//use my a_MyComp...
Finally
a_MyComp.Free;
End;
end;
Check out your code in CreateNewDocument...you'll notice that your not freeing your AdoTemp.
-Rick
Access violation exceptions (AVs) tell you that you have made a mistake in the memory access of your program. However, the way access violations work: they are unable to guarantee always detecting the error every time you make a mistake. (Sometimes you get un-lucky and no access violations are raised, but the mistake is still there causing other things to quietly go wrong inside your application.)
You can think of the memory available to your program being represented as below (where "." means the memory is not allocated to anything, and "A" means the memory is allocated to something within your program: e.g. object, local variable, parameter, machine code).
[.......AAA..AAAA.....AA......A...A...........AAA....A.......AA.........AAAAA]
Suppose you create some object; this will require memory to be allocated for the object itself. If the object in turn creates child objects, this will also be allocated in memory. (I'll use "O" and "C")
/-ref--\
[.......AAA..AAAAO....AAC.....A...A.....C.....AAA....A.......AA.........AAAAA]
\-ref------------------/
Note that within the memory allocated to O, it might hold references to its child objects.
Conversely to the above, whenever an object is destroyed, its memory is deallocated. Let us suppose you have made a mistake in your memory access, and something has destroyed one of O's child objects before O has finished using it.
/-ref--\
[.......AAA..AAAAO....AAC.....A...A...........AAA....A.......AA.........AAAAA]
\-ref------------------/
If O now tries to use its second child object, you will get an access violation. However, you might be un-lucky and not get an access violation showing your earlier mistake IF:
You destroy O without it trying to do anything to the second child object.
Or you first create a new object that happens to be allocated in the exact same place the child object was.
The second situation tends to be worse than the first, because every time C2 is used on the assumption that it is the correct child of O: unexpected results are produced and an incorrect values are written in memory. These incorrect values may be important data, or references to other objects (making the problem get worse over time).
So: Whenever yo do get an access violation, thank your lucky stars and hunt down the root cause of the problem.
NB! NB! I cannot stress the importance of the above enough.
Investigating your particular problem
First note, when an exception pauses in the debugger, the code usually points to the next line that would have been executed if not for the exception. (This doesn't affect anything here, because the previous line is also ADOTemp.SQL.Add; - the point is be prepared to consider the previous line as the cause of an exception - and test to confirm it!)
Useful tip: When an exception is thrown within Delphi/Third Party code, it can be useful to build with Debug DCU's or recompile Third Party source with debug information to get closer to the actual line raising the exception. (This can even be useful in situations like this where the error appears to be inside a Microsoft DLL.)
Looking at your screen-shot, the exception is thrown from a brand new instance of TADOQuery. Now there really isn't any sensible explanation for a brand new query to be throwing access violations when simply adding text to its SQL query. This strongly implies a corruption problem as described earlier.
One possibility would be another thread interfering with this one. (NOTE: even if your application isn't multi-threaded, the ADO objects do have built-in support for asynchronous operations.) However, I'm going to ignore that possibility for now because threading issues tend to be less consistent than you've implied this one is.
So, assuming you have a memory access problem somewhere else that is only manifesting here with a brand new TADOQuery (used correctly at least up to the point of the AV) - what other objects are interacting with this query to possibly cause corruption?
ADOTemp.Connection := fMain.ADOConnection;
There is a very strong possibility that commenting out the above line would also eliminate your access violation. So what are the possible problems:
fMain might have been destroyed prematurely and is now a dangling pointer meaning the code to return ADOConnection could do any of a number of unexpected things.
Any of the code backing fMain.ADOConnection may be referencing corrupted memory or dangling pointers; this includes the returned connection itself.
NOTE: One very common cause of invalid memory access is to have a function such as GetADOConnection that does not correctly initialise its Result. So in some cases it returns a 'random' address in memory, resulting in all sorts of unexpected behaviour when something tries to use the connection.
Is your ADOConnection created on a different thread, and being used by multiple threads?
PS: Don't forget to follow Rick's advice, and make sure you destroy the query when you've finished using it. Who knows, there might be an internal bug in ADO that when it runs out of a particular internal resource (due to queries not being destroyed), that causes ADO to start throwing AVs.
NOTE: You should be able to test my theory without altering functionality by simply changing the ADOTemp.Connection := ... line. Simply assign a connection string to the query instead of a connection object. However, if that solves (or more correctly stated: hides) the problem, please follow my advice, and hunt down the root cause of the AV.
SO community,
I have been scratching my head lately around two memory issues I am running into with some of my perl scripts and I am hoping I am finding some help/pointers here to better understand what is going on.
Questionable observation #1:
I am running the same perl script on different server instances (local laptop macosx, dedicated server hardware, virtual server hardware) and am getting significantly varying results in the traced memory consumption. Just after script initialization one instance would report be a memory consumption of the script of 210 MB compared to 330 MB on another box which is a fluctuation of over 60%. I understand that the malloc() function in charge of "garbage collection" for Perl is OS specific but are there deviations normal or should I be looking more closely at what is going on?
Questionable observation #2:
One script that is having memory leaks is relatively trivial:
foreach(#dataSamples) {
#memorycheck_1
my $string = subRoutine($_);
print FILE $string;
#memorycheck_2
}
All variables in the subRoutine are kept local and should be out of scope once the subroutine finishes. Yet when checking memory usage at #memorycheck_1 and #memorycheck_1 there is a significant memory leak.
Is there any explanation for that? Using Devel::Leak it seems there are leaked pointers which I have a hard time understanding where they would be coming from. Is there an easy way to translate the response of Devel::Leak into something that can actually give me pointers from where those leaked references origin?
Thanks
You have two different questions:
1) Why is the memory footprint not the same across various environments?
Well, are all the OS involved 64 bit? Or is there a mix? If one OS is 32 bit and the other 64 bit, the variation is to be expected. Or, as #hobbs notes in the comments, is one of the perls compiled with threads support whereas another is not?
2) Why does the memory footprint change between check #1 and check #2?
That does not necessarily mean there is a memory leak. Perl won't give back memory to the OS. The memory footprint of your program will be the largest footprint it reaches and will not go down.
Neither of these points is Perl specific. For more detail, you'll need to show more detail.
See also Question 7.25 in the C FAQ and further reading mentioned in that FAQ entry.
The most common reason for a memory leak in Perl is circular references. The simplest form would be something along the lines of:
sub subRoutine {
my( $this, $that );
$this = \$that;
$that = \$this;
return $_[0];
}
Now of course people reading that are probably saying, "Why would anyone do that?" And one generally wouldn't. But more complex data structures can contain circular references pretty easily, and we don't even blink an eye at them. Consider double-linked lists where each node refers to the node to its left and its right. It's important to not let the last explicit reference to such a list pass out of scope without first breaking the circular references contained in each of its nodes, or you'll get a structure that is inaccessible but can't be garbage collected because the reference count to each node never falls to zero.
Per Eric Strom's excellent suggestion, the core module Scalar::Util has a function called weaken. A reference that has been weakened won't hold a reference count to the entity it refers to. This can be helpful for preventing circular references. Another strategy is to implement your circular-reference-wielding datastructure within a class where an object method explicitly breaks the circular reference. Either way, such data structures do require careful handling.
Another source of trouble is poorly written XS modules (nothing against XS authors; it's just really tricky to write XS modules well). What goes on behind the closed doors of an XS module may be a memory leak.
Until we see what's happening inside of subRoutine we can only guess whether or not there's actually an issue, and what the source of the issue may be.
I have an application using POE which has about 10 sessions doing various tasks. Over time, the app starts consuming more and more RAM and this usage doesn't go down even though the app is idle 80% of the time. My only solution at present is to restart the process often.
I'm not allowed to post my code here so I realize it is difficult to get help but maybe someone can tell me what I can do find out myself?
Don't expect the process size to decrease. Memory isn't released back to the OS until the process terminates.
That said, might you have reference loops in data structures somewhere? AFAIK, the perl garbage collector can't sort out reference loops.
Are you using any XS modules anywhere? There could be leaks hidden inside those.
A guess: your program executes a loop for as long as it is running; in this loop it may be that you allocate memory for a buffer (or more) each time some condition occurs; since the scope is never exited, the memory remains and will never be cleaned up. I suggest you check for something like this. If it is the case, place the allocating code in a sub that you call from the loop and where it will go out of scope, and get cleaned up, on return to the loop.
Looks like Test::Valgrind is a tool for searching for memory leaks. I've never used it myself though (but I used plain valgrind with C source).
One technique is to periodically dump the contents of $POE::Kernel::poe_kernel to a time- or sequence-named file. $poe_kernel is the root of a tree spanning all known sessions and the contents of their heaps. The snapshots should monotonically grow if the leaked memory is referenced. You'll be able to find out what's leaking by diff'ing an early snapshot with a later one.
You can export POE_ASSERT_DATA=1 to enable POE's internal data consistency checks. I don't expect it to surface problems, but if it does I'd be very happy to receive a bug report.
Perl can not resolve reference rings. Either you have zombies (which you can detect via ps axl) or you have a memory leak (reference rings/circle)
There are a ton of programs to detect memory leaks.
strace, mtrace, Devel::LeakTrace::Fast, Devel::Cycle
I have a pretty weird problem in my iPhone app which is, I think, related to memory getting corrupted:
At one point, I need to sort an array, which I do with -[sortArrayUsingFunction].
The result is not correct unless I either allocate some memory with something like void *test = malloc(2 * sizeof( int )) before the method call or have, e.g., a call to NSLog() (which is never invoked) in the sorting function.
In other words: the sorting only works if I slightly increase the memory that was used before calling the sorting function. I think this is because at some point, memory gets corrupted.
How do you debug something like this?
It sounds like some of your code is using already released objects. A lot of help with debugging this kind of errors is provided in Apple’s great Mac OS X Debugging Magic tech note, especially the foundation part.
For your case I'd disable autorelease pools (setting the environment variable NSEnableAutoreleasePool=NO) or use the zombie feature (NSZombieEnabled=YES) to find places where you send messages to released objects.
Try running your program in the simulator under Valgrind:
http://valgrind.org/
And how to use it under the simulator:
http://landonf.bikemonkey.org/code/iphone/iPhone_Simulator_Valgrind.20081224.html
You may have to change the VALGRIND path in the code example depending on where it gets installed.
Such things can be a challenge to debug. There are some tools for detecting out-of-bounds accesses and such on other platforms, so I presume there would be something for the iPhone, however I don't know of any.
Perhaps you should store two copies of the array, and compare them for differences. Print out the differences. The nature of the "junk" that was introduced to one of the arrays might give a hint as to where it came from.
Also just go through the code that runs before this point, and re-read it (or better yet, get someone else to read it). You might spot a bug.
Perl uses reference counting for GC, and it's quite easy to make a circular reference by accident. I see that my program seems to be using more and more memory, and it will probably overflow after a few days.
Is there any way to debug memory leaks in Perl? Attaching to a program and getting numbers of objects of various types would be a good start. If I knew which objects are much more numerous than expected I could check all references to them and hopefully fix the leak.
It may be relevant that Perl never gives memory back to the system by itself: It's all up to malloc() and all the rules associated with that.
Knowing how malloc() allocates memory is important to answering the greater question, and it varies from system to system, but in general most malloc() implementations are optimized for programs allocating and deallocating in stack-like orders. Perl uses reference-counting for tracking memory which means that deallocations which means (unlike a GC-based language which uses malloc() underneath) it is actually not all that difficult to tell where deallocation is going to occur, and in what order.
It may be that you can reorganize your program to take advantage of this fact- by calling undef($old_object) explicitly - and in the right order, in a manner similar to the way C-programmers say free(old_object);
For long-running programs (days, months, etc), where I have loads of load/copy/dump cycles, I garbage-collect using exit() and exec(), and where it's otherwide unfeasible, I simply pack up my data structures (using Storable) and file descriptors (using $^F) and exec($0) - usually with an environment variable set like $ENV{EXEC_GC_MODE}, and you may need something similar even if you don't have any leaks of your own simply because Perl is leaking small chunks that your system's malloc() can't figure out how to give back.
Of course, if you do have leaks in your code, then the rest of my advice is somewhat more relevant. It was originally posted to another question on this subject, but it didn't explicitly cover long-running programs.
All perl program memory leaks will either be an XS holding onto a reference, or a circular data structure. Devel::Cycle is a great tool for finding circular references, if you know what structures are likely to contain the loops. Devel::Peek can be used to find objects with a higher-than-expected reference count.
If you don't know where else to look, Devel::LeakTrace::Fast could be a good first place, but you'll need a perl built for debugging.
If you suspect the leak is inside XS-space, it's much harder, and Valgrind will probably be your best bet. Test::Valgrind may help you lower the amount of code you need to search, but this won't work on Windows, so you'd have to port (at least the leaky portion) to Linux in order to do this.
Devel::Gladiator is another useful tool in this space.
Seems like the cpan module Devel::Cycle is what you are looking for. It requires making some changes to your code, but it should help you find your references without too many problems.
valgrind is a great linux application, which locates memory leaks in running code. If your Perl code runs on linux, you should check it out.
In addition to the other comments, you may find my Perl Memory Use talk at LPW2013 useful. I'd recommend watching the screencast as it explains the slides and has some cute visuals and some Q&A at the end.
I'd also suggest looking at Paul Evans Devel::MAT module which I mention in the talk.