I was trying to understand the concept of Page Size Extension, used in x86 processor but was not able to relate it with the page fault mechanism. From my understanding, when a page fault occurs, the virtual address is written in a register and an error code is pushed onto the stack. But if we are using page size extension, then how does the page fault handler comes to know what page size needs to be allocated.Can anyone help me with this?
There is a bit in the page directory. Intel calls this the PS bit. (Page size?) If the bit is set, it is a large page. If clear, a small page.
While Intel allows both page sizes to be in use simultaneously, I would wager that few OS implementations would support mixed page sizes.
Related
I find that during TLB missing process, some architecture use hardware to handle it while some use the OS. But when it comes to page fault, most of them use the OS instead of hardware.
I tried to find the answer but didn't find any article explains why.
Could anyone help with this?
Thanks.
If the hardware could handle it on its own, it wouldn't need to fault.
The whole point is that the OS hasn't wired the page into the hardware page tables, e.g. because it's not actually in memory at all, or because the OS needs to catch an attempt to write so the OS can implement copy-on-write.
Page faults come in three categories:
valid (the process logically has the memory mapped, but the OS was lazy or playing tricks):
hard: the page needs to be paged in from disk, either from swap space or from a disk file (e.g. a memory mapped file, like a page of an executable or shared library). Usually the OS will schedule another task while waiting for I/O.
soft: no disk access required, just for example allocating + zeroing a new physical page to back a virtual page that user-space just tried to write. Or copy-on-write of a writeable page that multiple processes had mapped, but where changes by one shouldn't be visible to the other (like mmap(MAP_PRIVATE)). This turns a shared page into a private dirty page.
invalid: There wasn't even a logical mapping for that page. A POSIX OS like Linux will deliver SIGSEGV signal to the offending process/thread.
The hardware doesn't know which is which, all it knows was that a page walk didn't find a valid page-table entry for that virtual address, so it's time to let the OS decide what to do next. (i.e. raise a page-fault exception which runs the OS's page-fault handler.) valid/invalid are purely software/OS concepts.
These example reasons are not an exhaustive list. e.g. an OS might remove the hardware mapping for a page without actually paging it out, just to see if the process touches it again soon. (In which case it's just a cheap soft page fault. But if not, then it might actually page it out to disk. Or drop it if it's clean.)
For HW to be able to fully handle a page fault, we'd need data structures with a hardware-specified layout that somehow lets hardware know what to do in some possible situations. Unless you build a whole kernel into the CPU microcode, it's not possible to have it handle every page fault, especially not invalid ones which require reading the OS's process / task-management data structures and delivering a signal to user-space. Either to a signal handler if there is one, or killing the process.
And especially not hard page faults, where a multi-tasking OS will let some other process run while waiting for the disk to DMA the page(s) into memory, before wiring up the page tables for this process and letting it retry the faulting load or store instruction.
I am reading Chap9 of Operating System Concepts and the concept of pure demand paging is described as follows:
In the extreme case, we can start executing a process with no pages in
memory. When the operating system sets the instruction pointer to the first instruction of the process, which is on a non-memory-resident page, the process
immediately faults for the page....
But if NONE of the pages, particularly the pages containing code, are in memory, how does the OS know where the program counter is in the first place? Is program counter set as part of process creation by inspecting the program image on disk? If so, I would assume the OS knows the format of the binary image and can directly access that info on disk. And it will only make sense if somehow this info is stored in the part of the program image not needed during program execution, if OS decides not to bring the page containing this info into memory.
To summarize, I would like to know:
How is program counter set for a new process if using pure demand paging?
Is any real OS using pure demand paging and what benefit does it have?
How does an executable's binary format (e.g. ELF, PE formats) help the OS do demand paging (OS needs to know where the first page is at least?)
What exactly happens when I open an application or a program which is not cached in the main memory.
a) How does the OS know where to look for the program?
b) If suppose all the pages cannot be loaded then does the address of the rest of the pages or at least starting address of the rest of the pages is maintained in the PCB?
c) Also is any information regarding the application is present in main memory, assuming it is never accessed before and it is not a critical component that has to be present in memory.
Any answers, follow-up questions, clarifications are welcome.
Edit: I have went through many links online but none states exactly what happens or who maintains the information. Most of the places it is stated that the program would be brought in the main memory by the page fault handler, I am looking for something more specific.
Read about page tables and page faults. That's the mechanism behind it. If you want something very specific, download x86 CPU manuals from intel or AMD and read the chapters about the same.
On WindowsCE 6.0 the default OS paging settings are a bit small as discussed here
http://blogs.msdn.com/b/ce_base/archive/2008/01/19/paging-and-the-windows-ce-paging-pool.aspx
I cannot convince OS providers to fix such issues so I'm interested in whether there is a suggested workaround from the application side where we are impacted by excessive paging/thrashing.
Theoretically, a smaller exe would help but I'm not convinced of that. I'm also experimenting with avoiding all memory mapping of files.
Any other suggestions?
You can set the paging pool size when creating the OS image, see here.
If your OS image is third party (or you don't control it), then you can try to set the paging parameters at runtime using IOCTL_HAL_GET_POOL_PARAMETERS IOCTL call.
Also check this link for more information about WinCE paging pools.
I have two applications (processes) running under Windows XP that share data via a memory mapped file. Despite all my efforts to eliminate per iteration memory allocations, I still get about 10 soft page faults per data transfer. I've tried every flag there is in CreateFileMapping() and CreateFileView() and it still happens. I'm beginning to wonder if it's just the way memory mapped files work.
If anyone there knows the O/S implementation details behind memory mapped files I would appreciate comments on the following theory: If two processes share a memory mapped file and one process writes to it while another reads it, then the O/S marks the pages written to as invalid. When the other process goes to read the memory areas that now belong to invalidated pages, this causes a soft page fault (by design) and the O/S knows to reload the invalidated page. Also, the number of soft page faults is therefore directly proportional to the size of the data write.
My experiments seem to bear out the above theory. When I share data I write one contiguous block of data. In other words, the entire shared memory area is overwritten each time. If I make the block bigger the number of soft page faults goes up correspondingly. So, if my theory is true, there is nothing I can do to eliminate the soft page faults short of not using memory mapped files because that is how they work (using soft page faults to maintain page consistency). What is ironic is that I chose to use a memory mapped file instead of a TCP socket connection because I thought it would be more efficient.
Note, if the soft page faults are harmless please note that. I've heard that at some point if the number is excessive, the system's performance can be marred. If soft page faults intrinsically are not significantly harmful then if anyone has any guidelines as to what number per second is "excessive" I'd like to hear that.
Thanks.