What is the relationship between address space and page table? I know that each process should have a page table which maps between virtual address to physical address. But what does an address space do? in os161, address space looks like:
struct addrespace {
vaddr_t as_vbase1;
paddr_t as_pbase1;
size_t as_npages1;
vaddr_t as_vbase2;
paddr_t as_pbase2;
size_t as_npages2;
paddr_t as_stackpbase;
}
we translate the virtual address (vaddr) to physical address using: (assume vaddr in segment 1)
paddr = vaddr - as_vbase1 + as_pbase1
it seems that we can get the physical address from the virtual address using the addrespace. If we can use addrespace to do the virtual to physical memory mapping, why do we need the page table?
Looking forward to your help! Thanks!
Firstly, thanks a lot for this question. Even though I am still a newbie to OS161 and struggling to understand the code, I will tell you what I have understood till now. Please feel free to correct me.
We need a page table to keep track of all the pages assigned to our process, not just because we need a translation from virtual to physical address translation.
The page table also keeps track of the pages if they are in memory and if the required page is on the disk, which would trigger a page fault. In that case we should allocate a new page, load a page from the disk, update page table, and update TLB entries.
Any page undergoes different states like free, dirty (should be written to the disk), etc. There are certain pages which are never supposed to be swapped out and they always remain in memory. Page tables also keep track of these states too.
Even this article might help you..
Why one page table per process
Related
The top-level page table will not occupy all its table items. What happens when I access a table item with "in / out of place" of 0?
I see that the processing method given in the book is to force a page missing interrupt, but this page missing interrupt is different from what I think. It may directly end the process. Why not remap it?
In addition, if the access to the top-level page table is random (I think the virtual address can be large), For example, in this case in the figure, it is obvious that there are many empty table items in the top-level page table.Isn't it easy to miss?
I think I can extract the key point of the problem, that is, is there a certain virtual address after the program is compiled? Can the operating system try to ensure that it does not cross the border to access unmapped addresses?
I think I have found the answer. After compiling, the program will set logical addresses for them, and the page table items of the top-level page table also correspond to the logical memory distribution of the program. For example, the program body is at the bottom, the corresponding page table item is 0, and the last item of the page table corresponds to the stack area of the program. Once the program accesses the page table items without mapping, it means that it accesses an address area that should not be accessed, I wonder if my idea is correct?
I'm sorry that my English level may make it difficult for you to understand. Thank you for your answer!
In an OS that uses virtual memory, each process has a page table. Each page table maps a process's virtual memory pages to the system's physical memory pages and indicates whether a given page is currently valid (loaded in memory) or not.
Suppose that memory is running low and the OS needs to choose a page to evict from physical memory. There are different algorithms for this. For example, FIFO, LRU. Once the OS chooses a page to evict, how does it invalidate any existing references to the page?
If the victim page is currently used by the active process, then the OS must invalidate the mapping in the current process's page table. If the victim page is currently used by another process, the OS must invalidate the mapping in the other process's page table. In any case, how does the OS determine which page table to update (if any), and how does it know where the mapping lives in that page table without doing a linear search?
There is a detailed description of x86 page table structure beginning at slide 22 of this presentation:
http://www.scs.stanford.edu/12au-cs140/notes/l8.pdf
I also found some helpful overviews of virtual memory:
http://www.cs.uic.edu/~jbell/CourseNotes/OperatingSystems/9_VirtualMemory.html
http://www.cs.umd.edu/class/sum2003/cmsc311/Notes/Memory/virtual.html
Similar Stack Overflow question without an answer:
how does linux update page table after context switch
Actually, the thing you are asking about called reverse mapping. For example, in Linux, you can find usefull function try_to_unmap_anon Inside page descriptor there is a field called mapping. This field is anon_vma for annonymous pages. As you can see this is not just ordinary struct, but also a list entry. There might be several anon_vmas for one page (see try_to_unmap_anon):
list_for_each_entry(vma, &anon_vma->head, anon_vma_node)
exactly one per page mapping. All this vmas linked into list. That is how kernel knows which processes (and their page tables) are in play.
Now about how kernel determines the virtual address ... again the answer could be found here: vma_address
233 pgoff_t pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT);
234 unsigned long address;
235
236 address = vma->vm_start + ((pgoff - vma->vm_pgoff) << PAGE_SHIFT);
So we can answer your question shortly now: in order not to do the page tables scan, kernel stores everything it needs (for quick fetch) in page descriptor (struct page).
Usually an OS will pick a process for whom to reduce memory first. Then it will pick pages from within that process. Then it knows the page table to use.
You suggest picking a page then finding which process. That normally occurs in systems that have pageable kernels. In that case, the system page table is shared by all processes.
When a page table entry(PTE) is not marked as valid, it means the data needed is not in memory, but on the disk. So now page fault happens and the OS is responsible to load this page of data from the disk to memory.
My question is, how does the OS know the exact disk address?
You are asking in a system dependent manner. A PTE not marked as valid may mean the address does not exist at all in the process address. A system may have another bit to indicate that the address is valid but logical to physical mapping does not exist.
The operating system needs to maintain a table of where it put the data.
The data can exist in a number of places.
1. It might be uninitialized data that has no mapping anywhere. Respond to the page fault by clearing a physical page and mapping it to the process address space.
It might be in the page file.
Some systems have a separate swap file.
It might be in the executable or shared library file.
The answer given in 2014 is correct. All the processor knows is that the page is missing - or sometimes that it had incorrect permission (e.g., write to a read-only page). At that point the processor generates a "page fault" exception which the kernel gets and now has to handle.
In some cases, this page fault will need to be passed all the way to the application, in Linux as a SIGSEGV ("segmentation violation") signal, e.g., when the user uses a null pointer. But, as you said, more usually, the kernel should and can handle the page fault. The kernel keeps, in its own tables (not inside the page table which is a structure with a specific format dictated by the processor) information about what each virtual-memory page is supposed contain. The following are some of the things the kernel may realize about the faulting page by consulting its own tables. This is not an exhaustive list.
This might be a page mmap()ed from disk. This case includes an application's explicit use of mmap(), but also happens when you run an executable, or use shared libraries - those are also mapped from disk - so the page fault can also happen when the processor executes instructions, not just when reading and writing. The kernel keeps a list of these mappings, so when it gets the page fault it can figure out where on disk it needs to read to get the missing page. So it reads from disk, and when getting the data it puts it in a new page in memory, and sets the page table entry (PTE) to point to this new page with the data, and resumes the application thread - where the faulting instruction is retried and now succeeds.
This may have been a page swapped out to disk. Again, the kernel keeps a table of which pages were swapped out, and where in the swap partition (or swap file, or whatever) this page now lives.
This might have been a write attempt to a "copy on write" page. The kernel needs to make a copy of the source page, and change the address in the page table to point to the new copy, and then allow the write. For example when you allocate a large area of memory, it can point to an existing "zero-filled" page, and only ever allocated when you first write to pages. Another example after fork() the new process's pages are all copy-on-write pages pointing to the original process's pages, and will only be actually copied when first written (by either process).
However, since you are looking for credible sources, maybe you want to read an explanation how the Linux kernel, specifically, does this, for example in:
https://vistech.net/~champ/online-docs/books/linuxkernel2/060.htm.
It is the same as virtual memory addressing.
The addresses that appear in programs are the virtual addresses or program addresses. For every memory access, either to fetch an instruction or data, the CPU must translate the virtual address to a real physical address. A virtual memory address can be considered to be composed of two parts: a page number and an offset into the page. The page number determines which page contains the information and the offset specifies which byte within the page. The size of the offset field is the log base 2 of the size of a page.
If the virtual address is valid, the system checks to see if a page frame is free. If no frames are free, the page replacement algorithm is run to remove a page.
At first I thought there is only one page table for the whole system. But there are actually one page table per process? What is the point of having multiple page table instead of one page table.
I am implementing part of os161
A page table usually has a fixed number of entries and therefore describes only a portion of the entire virtual address space. This is why you need multiple of them to cover the entire address space. Now, in many OSes processes have individual (in other words, not shared with others) virtual address spaces, which helps to protect processes from one another. This is another reason for having multiple page tables.
Page table translates from virtual to physical page addresses. Since each process has its own virtual address space and usually maps the same virtual address to a different physical address it needs a separate page table. Curiously, multiple processes may map different virtual addresses to the same physical memory. This can be used to implement shared libraries or inter-process communication.
Is it possible to provide physical address for a given virtual address in a direct way to the TLB on x86-64 architectures in long mode?
For example, lets say, I put zeros in PML4E, so a page fault exception will be triggered because an invalid address will be found, during the exception can the CPU tell the TLB by using some instruction that this virtual address is located at X physical page frame?
I want to do this because by code I can easily tell where the physical address would be, and this way avoid expensive page walk.
No, you need to put a page to the TLB. To be precise, you need to create/update appropriate PTE (with PDE and PDPE if needed). Everything around MMU management is somehow based on page tables and TLB. Even user/supervisor protection mode is done as a special flag of mapped page.
Why do you think that "page walk" is expensive operation? It is not expensive at all. To determine the PTE that must be updated you need to dereference only 4 pointers: PML4E -> PDPE -> PDE -> PTE. These entries are just indices in related tables. To get PML4E you need to use 39-47 bits of address taken during page fault handling and use the value as an index in PML4 table. To get PDPE you need 30-39 bits of an address as an index in PDE table and so on. It's not the thing that can slow down your system. I think allocation of a physical page takes more time than that.