What i understand:
Setting the NX bit high for a page will mean that whenever some program tries to execute the code in that page, it will cause a segmentation fault.
There is also a Valid-Invalid bit, and accessing a page marked Invalid will also cause a segmentation fault.
So the question is:
Why is the NX bit not redundant when you already have a valid-invalid bit ? What does it mean to mark a page both Valid and NX ?
First, I am going to change the terminology slightly. Instead of using terms like valid/invalid, I am going to replace them with present/not present.
A page that is marked as present can be accessed in some fashion (read, write or execute). A page that is marked as not present is not directly accessible (if at all). If it can be accessed, it must first be loaded from some type of backing store (disk/flash/...) into memory.
The NX bit only controls whether the page of memory has execute permissions. Why is controlling the execute permissions important? It helps in locking down the system to help prevent the execution of arbitrary code.
So, a page that is marked as both NX and not present is one that does not have execute permissions, but may need to be loaded from backing store.
If you do not have an NX bit (especially on your data pages), then if some clever hacker figures out how to jump to code in the data section while in supervisor/kernel mode, it makes their job much easier to execute arbitrary code on your system.
Hope this helps.
Related
In an Operating Systems course, the instructor introduced PSW and PC when he talked about Interrupt Handling.
His explanation was
PC holds the address of the next instruction to be fetched
PSW contains execution status information
But later I searched online and found that PSW = PC + status register. This makes me quite confused.
On the one hand, I am not sure what "execution status information" refers to. On the other hand, if PSW has the functions of a PC, why do we still need it?
Appreciate any explanation.
This isn't really standardized terminology. Most architectures have some register that plays the role of a status word, containing bits to indicate things like whether an add instruction caused a carry. But different architectures give it different names, and what exactly is included can vary widely. I'm not aware of any architecture that includes the program counter as part of their status word, but if they want to do that, well, who's going to stop them?
This is the kind of thing where you just have to look at the definition given by whatever book or article you are reading (or infer it from context), and realize that a different author may use the word differently.
In general, interrupts are hardware level subroutine calls. They do the same thing as a subroutine call (change the algorithm that the processor is executing) however they do it without warning the "executing code" that they are now operating.
In order to not damage the "executing code" all information that it was using must be stored. This includes the Program Counter (usually saved to the stack by the interrupt hardware in the same way that a subroutine call does) and all of the registers that the interrupt function will alter- these must be saved by pushing them onto the stack. The registers etc must be restored before the return from interrupt (RETI) instruction - the PC is restored by the RETI itself.
The PSW (often called the flag register) is a very important register and must generally be saved first. It contains bits like Zero (the last calculation resulted in a zero result) Carry (the last calculation resulted in a carry ie the result number is bigger than the register can hold) and several other flags. I suggest that you read the data sheet of an 8 bit microcontroller for an idea of what these flags might be. suffice it to say that these flags are needed in order to perform conditional jumps. And whilst they will often be ignored you can't take that chance.
You are probably correct in Your instructor using the term PSW to mean all all of the registers.
The subject of interrupts contains concepts that are common to subroutine calls in general (e.g. don't leave data that you don't want overwritten in a register before entering a subroutine). And later on in operating systems, the concept of context switches that occur during multi-tasking.
Peter
Is there a way to limit the amount of time an evaluation is allowed to run? Or limit the amount of memory that MatLab is allowed to take up so it doesn't freeze my laptop?
Let's answer your questions one at a time:
Question #1 - Can I limit the amount of time MATLAB takes to execute a script?
As far as I know, this is not possible. If you want to do this, you would need a multi-threaded environment where one thread does the actual job while another thread keeps an eye on the timer... but even with that functionality, AFAIK, MATLAB does not have this supported. The only way to stop your script from running is if you punch Ctrl + C / Cmd + C. Depending on what is actually being executed... for example a MEX script or a LAPACK routine, or just a simple MATLAB script, it may work by just pushing it once... or you may have to mash the sequence like a maniac.
(Note: The above image was introduced to try and be funny. If you don't know where that image is from, it's from the movie Flashdance and one of the songs from the soundtrack is She's a maniac, where I've also provided a YouTube link to the song above.)
See this post for more details: How can I interrupt MATLAB when it gets really really busy?
Question #2 - Can we limit the amount of memory that MATLAB uses?
Yes you can. From what I have seen in your posts, you're using Windows. You can change this by changing the page size of the virtual memory that is used for your computer. Specifically, instead of allowing it to grow dynamically, you could set it to be a certain size and once MATLAB exhausts that, it'll give you an out-of-memory error rather than freezing your computer.
See this post from MathWorks forums for more insight:
http://www.mathworks.com/matlabcentral/answers/12695-put-a-limit-on-memory-matlab-uses
Also see this guide from MathWorks on how to handle out-of-memory errors:
http://www.mathworks.com/help/matlab/matlab_prog/resolving-out-of-memory-errors.html
Finally, take a look at this link on how to change / modify the page size of your computer via Windows:
http://windows.microsoft.com/en-ca/windows/change-virtual-memory-size#1TC=windows-7
I have 2 pieces of code :
It works normal
with ADOTemp do
begin
SQL.Clear;
SQL.Add('INSERT INTO documents');
SQL.Add('(document_date,fk_id_status,money_direction,');
SQL.Add('paid,addition,saving,fk_id_base,fk_id_user)');
SQL.Add('VALUES ');
SQL.Add('(CONVERT(DATE,GETDATE(),103),:pfk_id_status,:pmoney_direction,');
SQL.Add('0,0,0,'+IntToStr(p_id_base)+',:pfk_id_user)');
Parameters.ParamByName('pfk_id_status').Value := p_id_status;
Parameters.ParamByName('pmoney_direction').Value := p_money_direction;
// Parameters.ParamByName('p').Value := p_id_base;
Parameters.ParamByName('pfk_id_user').Value := fMain.ApplicationVariablers.user_id;
ExecSQL;
end;
It does not work and returns a terrible error
with ADOTemp do
begin
SQL.Clear;
SQL.Add('INSERT INTO documents');
SQL.Add('(document_date,fk_id_status,money_direction,');
SQL.Add('paid,addition,saving,fk_id_base,fk_id_user)');
SQL.Add('VALUES ');
SQL.Add('(CONVERT(DATE,GETDATE(),103),:pfk_id_status,:pmoney_direction,');
SQL.Add('0,0,0,:p,:pfk_id_user)');
Parameters.ParamByName('pfk_id_status').Value := p_id_status;
Parameters.ParamByName('pmoney_direction').Value := p_money_direction;
Parameters.ParamByName('p').Value := p_id_base;
Parameters.ParamByName('pfk_id_user').Value := fMain.ApplicationVariablers.user_id;
ExecSQL;
end;
Error on image, line 1917
It seems that this question is in reference to your earlier question: Delphi, error :"Access violation at address xxxxxxxx. Read of address yyyyyyyy", at AdoQuery.SQL.Text:='''
You had added and accepted an answer there along the lines seen in this question. The error message, an AV in msvcrt.dll is indicative of a serious programming error. Perhaps a heap corruption. Perhaps something else. The change you made that stops the error occurring does not really fix the problem. The problem will still be there, lying dormant. You just got (un)lucky that the change you made appeared to fix the problem.
Access violations are not always reproducible. That's just their nature. When you encounter one you need to understand why it happens. Simply using trial and error to re-organise code from a completely different location will never lead to the real solution.
What you should be doing is tracking down the real cause of the problem. Do that, fix it, and either version of the code above will work. This is probably not the answer you are wanting to get. You may very well not want to accept what I say. But, speaking from experience, until you recognise that you have a more serious problem, you will make no headway here.
Looking at your code in the bitmap...I'm pretty sure you access violation is because of what David just said...you have serious Memory problems in your code...Your not freeing your objects...that you are creating...Anytime you pass a nil reference to Owner in a TComponent Constructor...your telling the compiler...that you know what your doing and will free it when your finished with it...Matter of fact good practice is to always free objects that you declare and use exclusively in the scope of your method.
procedure TForm1.MyMethod;
var
a_MyComp: TMyComp;
begin
a_MyComp := TMyComp.Create(nil);
Try
//use my a_MyComp...
Finally
a_MyComp.Free;
End;
end;
Check out your code in CreateNewDocument...you'll notice that your not freeing your AdoTemp.
-Rick
Access violation exceptions (AVs) tell you that you have made a mistake in the memory access of your program. However, the way access violations work: they are unable to guarantee always detecting the error every time you make a mistake. (Sometimes you get un-lucky and no access violations are raised, but the mistake is still there causing other things to quietly go wrong inside your application.)
You can think of the memory available to your program being represented as below (where "." means the memory is not allocated to anything, and "A" means the memory is allocated to something within your program: e.g. object, local variable, parameter, machine code).
[.......AAA..AAAA.....AA......A...A...........AAA....A.......AA.........AAAAA]
Suppose you create some object; this will require memory to be allocated for the object itself. If the object in turn creates child objects, this will also be allocated in memory. (I'll use "O" and "C")
/-ref--\
[.......AAA..AAAAO....AAC.....A...A.....C.....AAA....A.......AA.........AAAAA]
\-ref------------------/
Note that within the memory allocated to O, it might hold references to its child objects.
Conversely to the above, whenever an object is destroyed, its memory is deallocated. Let us suppose you have made a mistake in your memory access, and something has destroyed one of O's child objects before O has finished using it.
/-ref--\
[.......AAA..AAAAO....AAC.....A...A...........AAA....A.......AA.........AAAAA]
\-ref------------------/
If O now tries to use its second child object, you will get an access violation. However, you might be un-lucky and not get an access violation showing your earlier mistake IF:
You destroy O without it trying to do anything to the second child object.
Or you first create a new object that happens to be allocated in the exact same place the child object was.
The second situation tends to be worse than the first, because every time C2 is used on the assumption that it is the correct child of O: unexpected results are produced and an incorrect values are written in memory. These incorrect values may be important data, or references to other objects (making the problem get worse over time).
So: Whenever yo do get an access violation, thank your lucky stars and hunt down the root cause of the problem.
NB! NB! I cannot stress the importance of the above enough.
Investigating your particular problem
First note, when an exception pauses in the debugger, the code usually points to the next line that would have been executed if not for the exception. (This doesn't affect anything here, because the previous line is also ADOTemp.SQL.Add; - the point is be prepared to consider the previous line as the cause of an exception - and test to confirm it!)
Useful tip: When an exception is thrown within Delphi/Third Party code, it can be useful to build with Debug DCU's or recompile Third Party source with debug information to get closer to the actual line raising the exception. (This can even be useful in situations like this where the error appears to be inside a Microsoft DLL.)
Looking at your screen-shot, the exception is thrown from a brand new instance of TADOQuery. Now there really isn't any sensible explanation for a brand new query to be throwing access violations when simply adding text to its SQL query. This strongly implies a corruption problem as described earlier.
One possibility would be another thread interfering with this one. (NOTE: even if your application isn't multi-threaded, the ADO objects do have built-in support for asynchronous operations.) However, I'm going to ignore that possibility for now because threading issues tend to be less consistent than you've implied this one is.
So, assuming you have a memory access problem somewhere else that is only manifesting here with a brand new TADOQuery (used correctly at least up to the point of the AV) - what other objects are interacting with this query to possibly cause corruption?
ADOTemp.Connection := fMain.ADOConnection;
There is a very strong possibility that commenting out the above line would also eliminate your access violation. So what are the possible problems:
fMain might have been destroyed prematurely and is now a dangling pointer meaning the code to return ADOConnection could do any of a number of unexpected things.
Any of the code backing fMain.ADOConnection may be referencing corrupted memory or dangling pointers; this includes the returned connection itself.
NOTE: One very common cause of invalid memory access is to have a function such as GetADOConnection that does not correctly initialise its Result. So in some cases it returns a 'random' address in memory, resulting in all sorts of unexpected behaviour when something tries to use the connection.
Is your ADOConnection created on a different thread, and being used by multiple threads?
PS: Don't forget to follow Rick's advice, and make sure you destroy the query when you've finished using it. Who knows, there might be an internal bug in ADO that when it runs out of a particular internal resource (due to queries not being destroyed), that causes ADO to start throwing AVs.
NOTE: You should be able to test my theory without altering functionality by simply changing the ADOTemp.Connection := ... line. Simply assign a connection string to the query instead of a connection object. However, if that solves (or more correctly stated: hides) the problem, please follow my advice, and hunt down the root cause of the AV.
I'm looking at some slightly confused code that's attempted a platform abstraction of prefetch instructions, using various compiler builtins. It appears to be based on powerpc semantics initially, with Read and Write prefetch variations using dcbt and dcbtst respectively (both of these passing TH=0 in the new optional stream opcode).
On ia64 platforms we've got for read:
__lfetch(__lfhint_nt1, pTouch)
wherease for write:
__lfetch_excl(__lfhint_nt1, pTouch)
This (read vs. write prefetching) appears to match the powerpc semantics fairly well (with the exception that ia64 allows for a temporal hint).
Somewhat curiously the ia32/amd64 code in question is using
prefetchnta
Not
prefetchnt1
as it would if that code were to be consistent with the ia64 implementations (#ifdef variations of that in our code for our (still live) hpipf port and our now dead windows and linux ia64 ports).
Since we are building with the intel compiler I should be able to many of our ia32/amd64 platforms consistent by switching to the xmmintrin.h builtins:
_mm_prefetch( (char *)pTouch, _MM_HINT_NTA )
_mm_prefetch( (char *)pTouch, _MM_HINT_T1 )
... provided I can figure out what temporal hint should be used.
Questions:
Are there read vs. write ia32/amd64 prefetch instructions? I don't see any in the instruction set reference.
Would one of the nt1, nt2, nta temporal variations be preferred for read vs. write prefetching?
Any idea if there would have been a good reason to use the NTA temporal hint on ia32/amd64, yet T1 on ia64?
Are there read vs. write ia32/amd64 prefetch instructions? I don't see any in the instruction set reference.
Some systems support the prefetchw instructions for writes
Would one of the nt1, nt2, nta temporal variations be preferred for read vs. write prefetching?
If the line is exclusively used by the calling thread, it shouldn't matter how you bring the line, both reads and writes would be able to use it. The benefit for prefetchw mentioned above is that it will bring the line and give you ownership on it, which may take a while if the line was also used by another core. The hint level on the other hand is orthogonal with the MESI states, and only affects how long would the prefetched line survive. This matters if you prefetch long ahead of the actual access and don't want to prefetch to get lost in that duration, or alternatively - prefetch right before the access, and don't want the prefetches to thrash your cache too much.
Any idea if there would have been a good reason to use the NTA temporal hint on ia32/amd64, yet T1 on ia64?
Just speculating - perhaps the larger caches and aggressive memory BW are more vulnerable to bad prefetching and you'd want to reduce the impact through the non-temporal hint. Consider that your prefetcher is suddenly set loose to fetch anything it can, you'd end up swamped in junk prefetches that would through away lots of useful cachelines. The NTA hint makes them overrun each other, leaving the rest undamaged.
Of course this may also be just a bug, I can't tell for sure, only whoever developed the compiler, but it might make sense for the reason above.
The best resource I could find on x86 prefetching hint types was the good ol' article What Every Programmer Should Know About Memory.
For the most part on x86 there aren't different instructions for read and write prefetches. The exceptions seem to be those that are non-temporal aligned, where a write can bypass the cache but as far as I can tell, a read will always get cached.
It's going to be hard to backtrack through why the earlier code owners used one hint and not the other on a certain architecture. They could be making assumptions about how much cache is available on processors in that family, typical working set sizes for binaries there, long term control flow patterns, etc... and there's no telling how much any of those assumptions were backed up with good reasoning or data. From the limited background here I think you'd be justified in taking the approach that makes the most sense for the platform you're developing on now, regardless what was done on other platforms. This is especially true when you consider articles like this one, which is not the only context where I've heard that it's really, really hard to get any performance gain at all with software prefetches.
Are there any more details known up front, like typical cache miss ratios when using this code, or how much prefetches are expected to help?
I've been working on a bit of JavaScript code that, under certain conditions, lazy-loads a couple of different libraries (Clicky Web Analytics and the Sizzle selector engine).
This script is downloaded millions of times per day, so performance optimization is a major concern. To date, I've employed a couple of flags like script_loading and script_loaded to try to ensure that I don't load either library more than once (by "load," I mean requesting the scripts after page load by inserting a <script> element into the DOM).
My question is: Rather than rely on these flags, which have gotten a little unwieldy and hard to follow in my code (think callbacks and all of the pitfalls of asynchronous code), is it cross-browser safe (i.e., back to IE 6) and not detrimental to performance to just call a simple function to insert a <script> element whenever I reach a code branch that needs one of these libraries?
The latter would still ensure that I only load either library when I need it, and would also simplify and reduce the weight of my code base, but I need to be absolutely sure that this won't result in additional, unnecessary browser requests.
My hunch is that appending a <script> element multiple times won't be harmful, as I assume browsers should recognize a duplicate src URL and rely on a local cached copy. But, you know what happens when we assume...
I'm hoping that someone is familiar enough with the behavior of various modern (and not-so-modern, such as IE 6) browsers to be able to speak to what will happen in this case.
In the meantime, I'll write a test to try to answer this first-hand. My hesitation is just that this may be difficult and cumbersome to verify with certainty in every browser that my script is expected to support.
Thanks in advance for any help and/or input!
Got an alternative solution.
At the point where you insert the new script element in the DOM, could you not do a quick scan of existing script elements to see if there is another one with the same src? If there is, don't insert another?
Javascript code on the same page can't run multithreaded, so you won't get any race conditions in the middle of this or anything.
Otherwise you are just relying on the caching behaviour of current browsers (and HTTP proxies).
The page is processed as a stream. If you load the same script multiple times, it will be run every time it is included. Obviously, due to the browser cache, it will be requested from the server only once.
I would stay away from this approach of inserting script tags for the same script multiple times.
The way I solve this problem is to have a "test" function for every script to see if it is loaded. E.g. for sizzle this would be "function() { return !!window['Sizzle']; }". The script tag is only inserted if the test function returns false.
Each time you add a script to your page,even if it has the same src the browser may found it on the local cache or ask the server if the content is changed.
Using a variable to check if the script is included is a good way to reduce loading and it's very simple:
for example this may works for you:
var LOADED_JS=Object();
function js_isIncluded(name){//returns true if the js is already loaded
return LOADED_JS[name]!==undefined;
}
function include_js(name){
if(!js_isIncluded(name)){
YOUR_LAZY_LOADING_FUNCTION(name);
LOADED_JS[name]=true;
}
}
you can also get all script elements and check the src,my solution is better because it hase the speed and simplicity of an hash array and the script src has an absolute path even if you set it with a relative path.
you may also want to init the array with the scripts normally loaded(without lazy loading)on the page init to avoid double request.
For what it's worth, if you define the scripts as type="module", they will only be loaded and executed once.