At first I thought there is only one page table for the whole system. But there are actually one page table per process? What is the point of having multiple page table instead of one page table.
I am implementing part of os161
A page table usually has a fixed number of entries and therefore describes only a portion of the entire virtual address space. This is why you need multiple of them to cover the entire address space. Now, in many OSes processes have individual (in other words, not shared with others) virtual address spaces, which helps to protect processes from one another. This is another reason for having multiple page tables.
Page table translates from virtual to physical page addresses. Since each process has its own virtual address space and usually maps the same virtual address to a different physical address it needs a separate page table. Curiously, multiple processes may map different virtual addresses to the same physical memory. This can be used to implement shared libraries or inter-process communication.
Related
I've been researching (on SO and elsewhere) the relationship between virtual addresses and physical addresses. I would appreciate it if someone could confirm if my understanding of this concept is correct.
The page table is classified as 'virtual space' and contains the virtual addresses of each page. It then maps to the 'physical space', which contains the physical addresses of each page.
A wikipedia diagram to make my explanation clearer:
https://upload.wikimedia.org/wikipedia/commons/3/32/Virtual_address_space_and_physical_address_space_relationship.svg
Is my understanding of this concept correct?
Thank you.
Not entirely correct.
Each program has its own virtual address space. Technically, there is only one address space, the physical random-access memory. Therefore it's called "virtual" because to the user program it seems as if it has its own address space.
Now, take the instruction mov 0x1234, %eax (AT&T) or MOV EAX, [0x1234] (Intel) as an example:
The CPU sends the virtual address 0x1234 to one of its parts, the MMU.
The MMU obtains the corresponding physical address from the page table. This process of adjusting the address is also lovingly called "massaging."
The CPU retrieves the data from the RAM location the physical address refers to.
The concrete translation process depends heavily on the actual architecture and CPU.
The page table is classified as 'virtual space' and contains the virtual addresses of each page. It then maps to the 'physical space', which contains the physical addresses of each page.
This is not really correct. The page table defines a logical address space consisting of pages. The page table maps the logical pages to physical page frames they indicate that the page frame does not [yet] exist in memory in which case you have a virtual mapping. A page is VIRTUAL when memory is simulated using disk space.
In the olde days, page tables always established a virtual address space. Now it is becoming increasingly common (e.g. embedded system) to use logical address translation without virtual memory (paging). Thus, the terms "virtual memory" and "logical memory" are frequently conflated.
The physical address space exists only to the operating system. The process sees only a logical address space.
That is a bit of an oversimplification because the process becomes the operating system after an exception or interrupt and the kernel operates within a common logical address range. However, the operating system kernel does have to manage physical memory to some degree.
For example, some aspect of the page tables must use physical addresses. If the page tables used all logical addresses, then you'd have a chicken and egg problem for address translation. Various hardware systems address this problem in different ways.
Finally, the diagram you link to is a very poor illustration.
When a page table entry(PTE) is not marked as valid, it means the data needed is not in memory, but on the disk. So now page fault happens and the OS is responsible to load this page of data from the disk to memory.
My question is, how does the OS know the exact disk address?
You are asking in a system dependent manner. A PTE not marked as valid may mean the address does not exist at all in the process address. A system may have another bit to indicate that the address is valid but logical to physical mapping does not exist.
The operating system needs to maintain a table of where it put the data.
The data can exist in a number of places.
1. It might be uninitialized data that has no mapping anywhere. Respond to the page fault by clearing a physical page and mapping it to the process address space.
It might be in the page file.
Some systems have a separate swap file.
It might be in the executable or shared library file.
The answer given in 2014 is correct. All the processor knows is that the page is missing - or sometimes that it had incorrect permission (e.g., write to a read-only page). At that point the processor generates a "page fault" exception which the kernel gets and now has to handle.
In some cases, this page fault will need to be passed all the way to the application, in Linux as a SIGSEGV ("segmentation violation") signal, e.g., when the user uses a null pointer. But, as you said, more usually, the kernel should and can handle the page fault. The kernel keeps, in its own tables (not inside the page table which is a structure with a specific format dictated by the processor) information about what each virtual-memory page is supposed contain. The following are some of the things the kernel may realize about the faulting page by consulting its own tables. This is not an exhaustive list.
This might be a page mmap()ed from disk. This case includes an application's explicit use of mmap(), but also happens when you run an executable, or use shared libraries - those are also mapped from disk - so the page fault can also happen when the processor executes instructions, not just when reading and writing. The kernel keeps a list of these mappings, so when it gets the page fault it can figure out where on disk it needs to read to get the missing page. So it reads from disk, and when getting the data it puts it in a new page in memory, and sets the page table entry (PTE) to point to this new page with the data, and resumes the application thread - where the faulting instruction is retried and now succeeds.
This may have been a page swapped out to disk. Again, the kernel keeps a table of which pages were swapped out, and where in the swap partition (or swap file, or whatever) this page now lives.
This might have been a write attempt to a "copy on write" page. The kernel needs to make a copy of the source page, and change the address in the page table to point to the new copy, and then allow the write. For example when you allocate a large area of memory, it can point to an existing "zero-filled" page, and only ever allocated when you first write to pages. Another example after fork() the new process's pages are all copy-on-write pages pointing to the original process's pages, and will only be actually copied when first written (by either process).
However, since you are looking for credible sources, maybe you want to read an explanation how the Linux kernel, specifically, does this, for example in:
https://vistech.net/~champ/online-docs/books/linuxkernel2/060.htm.
It is the same as virtual memory addressing.
The addresses that appear in programs are the virtual addresses or program addresses. For every memory access, either to fetch an instruction or data, the CPU must translate the virtual address to a real physical address. A virtual memory address can be considered to be composed of two parts: a page number and an offset into the page. The page number determines which page contains the information and the offset specifies which byte within the page. The size of the offset field is the log base 2 of the size of a page.
If the virtual address is valid, the system checks to see if a page frame is free. If no frames are free, the page replacement algorithm is run to remove a page.
I was going through few of the lectures conducted at UC Berkeley on Virtual Memory
# https://www.youtube.com/results?search_query=computer+science+194+-+lecture+14
"Computer Science 194 - Lecture 13_ Memory Management_ Mechanisms on various architectures, NUMA" and "Computer Science 194 - Lecture 14_ Virtual Memory Management, Swapping, Page Cache"
Its an excellent lecture. However , I am slightly confused on one point.
The lecture explains how segmentation and paging have been combined for addressing VM.
It goes on to explain that the current systems use as the structure of VM address.
It also mentioned that the virtual space visible to a process is private and the address ranges for each process remains same. Each process views its address space to start at 0 and extend till 4G. with different segment areas within this 4G space.
Questions :
Now if the address ranges for each VM space is same , How is it that two processes referring to the highest level lookup table - the PageTblPtr using "segment Number" as the index , are uniquely able to identify a row in this table ... as the segment address/number for each of the process may be same... lets say Process A's and B's have both data segment starting at address 'x' within there VM space .
Does that also mean that there could be as many as 6 entries in the PagetblPtr for a process for each - 6 possible segments - CS, DS, ...etc?
Where is the PageTblPtr maintained ?
Best Regards,
Varun
I'm not going to watch an 83-minute video lecture just to get the exact definitions of terms like PageTblPtr, but I'll try to provide some general answers:
Modern operating systems don't use segmentation. It's just a big flat address space, and all the segment selectors are set to encompass the whole thing. I don't think x86 processors even support segmentation when running in 64-bit mode.
Each process has its own page table, which defines the mappings for that process's virtual address space. Two processes can have different data at the same virtual address because their respective page tables point to different physical pages for the same virtual page.
Page tables are owned and managed by the OS kernel. When the kernel performs a task switch (changing which process it's running), one of the things it must do is activate the new process's page table instead of the old.
What is the relationship between address space and page table? I know that each process should have a page table which maps between virtual address to physical address. But what does an address space do? in os161, address space looks like:
struct addrespace {
vaddr_t as_vbase1;
paddr_t as_pbase1;
size_t as_npages1;
vaddr_t as_vbase2;
paddr_t as_pbase2;
size_t as_npages2;
paddr_t as_stackpbase;
}
we translate the virtual address (vaddr) to physical address using: (assume vaddr in segment 1)
paddr = vaddr - as_vbase1 + as_pbase1
it seems that we can get the physical address from the virtual address using the addrespace. If we can use addrespace to do the virtual to physical memory mapping, why do we need the page table?
Looking forward to your help! Thanks!
Firstly, thanks a lot for this question. Even though I am still a newbie to OS161 and struggling to understand the code, I will tell you what I have understood till now. Please feel free to correct me.
We need a page table to keep track of all the pages assigned to our process, not just because we need a translation from virtual to physical address translation.
The page table also keeps track of the pages if they are in memory and if the required page is on the disk, which would trigger a page fault. In that case we should allocate a new page, load a page from the disk, update page table, and update TLB entries.
Any page undergoes different states like free, dirty (should be written to the disk), etc. There are certain pages which are never supposed to be swapped out and they always remain in memory. Page tables also keep track of these states too.
Even this article might help you..
Why one page table per process
Is it possible to provide physical address for a given virtual address in a direct way to the TLB on x86-64 architectures in long mode?
For example, lets say, I put zeros in PML4E, so a page fault exception will be triggered because an invalid address will be found, during the exception can the CPU tell the TLB by using some instruction that this virtual address is located at X physical page frame?
I want to do this because by code I can easily tell where the physical address would be, and this way avoid expensive page walk.
No, you need to put a page to the TLB. To be precise, you need to create/update appropriate PTE (with PDE and PDPE if needed). Everything around MMU management is somehow based on page tables and TLB. Even user/supervisor protection mode is done as a special flag of mapped page.
Why do you think that "page walk" is expensive operation? It is not expensive at all. To determine the PTE that must be updated you need to dereference only 4 pointers: PML4E -> PDPE -> PDE -> PTE. These entries are just indices in related tables. To get PML4E you need to use 39-47 bits of address taken during page fault handling and use the value as an index in PML4 table. To get PDPE you need 30-39 bits of an address as an index in PDE table and so on. It's not the thing that can slow down your system. I think allocation of a physical page takes more time than that.