I am taking an operating systems class, and they introduced the concept of address translation. While a program is running, all memory accesses will be translated from virtual to physical. My question is: a lot of memory addresses are given to the program by the OS, and thus can be the physical addresses themselves. What types of memory requests does the program initiate all by itself (without an address given by the OS), and thus would be virtual addresses?
The address of the stack is pre-set by the OS in the stack pointer before the program starts running, and so the stack pointer can hold the physical address. Heap addresses returned by malloc are returned by the OS -- thus, the OS can return the physical address, since that address is stored in some variable and is transparent to the program. So what addresses does the program itself access that need to be translated to physical addresses? So far, the only examples I can think of are: 1) instruction addresses (jump commands have the instruction address hardcoded in the program code) and 2) maybe static variable addresses (if it's not stored in a register by the OS). Are there any more examples/am I missing something?
Thanks!
Maybe the simplest example why address translation is extremely useful:
The physical address space is usually partitioned into pages, e.g. 4k large.
Processes have their own virtual address space, that is also partitioned into pages of the same size.
Address translation, which is done by a memory management unit under control of the operating system, maps any virtual page to any physical page, for every page independently.
It is thus possible to combine arbitrary pages of a fragmented physical memory to continuous virtual pages.
This allows to use the physical memory much more effectively than without address translation.
Related
Upon reading this chapter from "Operating Systems: Three Easy Pieces" book, I'm confused of this excerpt:
If in contrast the kernel were located entirely in physical memory, it would be quite hard to do things like swap pages of the page table to disk;
I've been trying to make sense of it for days but still can't get as to how kernel virtual memory helps in making swap pages of the page table easier. Wouldn't it be the same if the kernel would live completely in physical memory as the pages of different page tables' processes would end up in physical memory in the end anyway (and thus be swapped to disk if needed)? How is it different when page tables reside in kernel virtual memory vs. in kernel-owned physical memory?
Let's assume the kernel needs to access a string of characters from user-space (e.g. maybe a file name passed to an open() system call).
With the kernel in virtual memory; the kernel would have to check that the virtual address of the string is sane (e.g. not an address for kernel's own data) and also guard against a different thread in user-space modifying the data the pointer points to (so that the string can't change while the kernel is using it, possibly after the kernel checked that the string itself is valid but before the kernel uses the string). In theory (and likely in practice) this could all be hidden by a "check virtual address range" function. Apart from those things; kernel can just use the string like normal. If the data is in swap space, then kernel can just let its own page fault handler fetch the data when kernel attempts to access it the data and not worry about it.
With kernel in physical memory; the kernel would still have to do those things (check the virtual address is sane, guard against another thread modifying the data the pointer points to). In addition; it would have to convert the virtual address to a physical addresses itself, and ensure the data is actually in RAM itself. In theory this could still be hidden by a "check virtual address range" function that also converts the virtual address into physical address(es).
However, the "contiguous in virtual memory" data (the string) may not be contiguous in physical memory (e.g. first half of the string in one page with the second half of the string in a different page with a completely unrelated physical address). This means that kernel would have to deal with this problem too (e.g. for a string, even things like strlen() can't work), and it can't be hidden in a "check (and convert) virtual address range" function to make it easier for the rest of the kernel.
To deal with the "not contiguous in physical memory" problem there's mostly only 2 possibilities:
a) A set of "get char/short/int.. at offset N using this list of physical addresses" functions; or
b) Refuse to support it in the kernel; which mostly just shifts unwanted burden to user-space (e.g. the open() function in a C library copying the file name string to a new page if the original string crossed a page boundary before calling the kernel's open() system call).
In an ideal world (which is what we live in now with kernels that are hundreds of megabytes in size running on machines with gigabytes of physical RAM) the kernel would never swap even parts of itself. But in the old days, when physical memory was a constraint, the less of the kernel in physical memory, the more the application could be in physical memory. The more the application is in physical memory, the fewer page faults in user space.
THe linux kernel has been worked over fairly extensively to keep it compact. Case in point: kernel modules. You can load a module using insmod or modprobe, and that module will become resident, but if nothing uses is, after a while it will get swapped out, and that's no big deal because nothing is using it.
As far as I read from the book(correct me I am wrong) after a compiler puts the compiled code in the storage, the CPU creates logical addresses and those logical addresses are mapped to physical memory through MMU(Memory Management Unit).Also I know that CPU directly cannot access anything other than the physical memory.
Then how does CPU produce logical addresses for the process in the first place?
It sounds like you have a bit of confusion about what things do.
The operating system defines the logical address space by setting up page tables that logical pages to physical page frames. The operating system loads hardware registers of the CPU so that it knows about the page tables it has defined.
This use of page tables to define logical address spaces is an integral part of a modern CPU. In some systems, the only use of physical addresses is within page tables.
The compiler generates an object code file that describes the instructions and data used and created.
The linker combines object code into an executable file that defines how the program will be loaded into memory.
The loader reads the instructions in an executable file and sets up the logical addresses space to run the program. The loader calls system routines that set up the page tables the define the logical addresses space.
For example, where the executable has read-only data, the loader will call OS routines to creates read-only pages in the logical address space and map them to the data in the executable file.
Consider a hypothetical situation in which we come up with an Operating System having the same sized physical and virtual address space.Then what are the advantages or disadvantages of putting such a scheme.For starters special hardware support for MMU(Memory Management Unit) is not required.What shall be other implications.
The need for virtual memory doesn't really lie in sizes, but in the fact that it allows a process to not care about other processes.
Each can use their virtual memory freely and the OS makes sure to store all the data in different parts of the physical memory. There are other advantages too, such as swapping certain pages to external memory without the process knowing it.
Suppose the following. I have a system with virtual memory with one lever paging, I have a MMU and the TLB thing is controled by software.
Ok.. so imagine I'm a process, and I want to read a word in RAM of virtual address vaddr.
So, the CPU gives the MMU vaddr, the MMU checks in the TLB if there's an entry with the (suppose) 5 most significant bits of vaddr. If it's there... all's ok, it calculates the physical address and all goes fine.
Now.. suppose it wasn't in the TLB. In this case, the MMU makes an interrupt (a page fault).
Ok.. now I'm in the handler of the page fault.
In the PBR (page base register), I have the address of the start of the page table.
My question here. Is this address a physical one?. I guess yes, because if it were virtual, would means two things:
1) Must be reserved somehow in the virtual adress space of the process (never heard of something like that)
2) If this address is not in the TLB, would couse again a pagefault and I'll have an infinite loop.
Same question about addresses in tables. If I had a two level paging. The address in an entry in the first level table (that points to the second level table), is somehow virtual or physical?
Thanks.
Homework?
In any case, such things are in detail described in the CPU's architecture manual (and you haven't even written which CPU you're talking about - x86 does not generate page faults on TLB misses).
as #zvrba pointed out that behaviour is defined by CPU implementation, so this question is not answerable per se.
General things to consider though:
In the PBR (page base register), I have the address of the start of the page table. My question here. Is this address a physical one?
yes, that has to be physical address, otherwise MMU would need to resolve virtual-to-physical address for a translation table first.
1) Must be reserved somehow in the virtual adress space of the process
wrong, virtual should be 'translated'. So reservation doesn't make sense.
2) If this address is not in the TLB, would couse again a pagefault and I'll have an infinite loop
good try but still wrong thinking, even if addresses holding translation tables are mapped, you still need to get physical address first to read them (TLB tables, I mean) out of memory. Otherwise you are stuck on getting virtual-to-physical for first level TLB, not ever reaching point to get your general RAM access.By the way, TLB tables are usually mapped into virtual address space, that's how OS mapping/unmapping memory pages for applications.
If I had a two level paging. The address in an entry in the first level table (that points to the second level table), is somehow virtual or physical?
page walk is by physical addresses, no matter how many levels are there. See above.
I have a system with virtual memory with one lever paging... So, the CPU gives the MMU vaddr, the MMU checks in the TLB if there's an entry with the (suppose) 5 most significant bits of vaddr
5 most significant bits... hmm... Let's say that's 32b system so 5 most significant mean only bits [31..27] has effect. That concludes you have a page size 2^27 = 128MB. Why to bother with MMU if you could have only 32 pages mapped??? Keep MMU off!!!
It's widely known that the most significant mmap() feature is that file mapping is shared between many processes. But it's not less widely known that every process has its own address space.
The question is where are memmapped files (more specifically, its data) truly kept, and how processes can get access to this memory?
I mean not *(pa+i) and other high-level stuff, but I mean the internals of the process.
This happens at the virtual memory management layer in the operating system. When you memory map a file, the memory manager basically treats the file as if it were swap space for the process. As you access pages in your virtual memory address space, the memory mapper has to interpret them and map them to physical memory. When you cross a page boundary, this may cause a page fault, at which time the OS must map a chunk of disk space to a chunk of physical memory and resolve the memory mapping. With mmap, it simply does so from your file instead of its own swap space.
If you want lots of details of how this happens, you'll have to tell us which operating system you're using, as implementation details vary.
This is very implementation-dependent, but the following is one possible implementation:
When a file is a first memory-mapped, the data isn't stored anywhere at first, it's still on disk. The virtual memory manager (VMM) allocates a range of virtual memory addresses to the process for the file, but those addresses aren't immediately added to the page table.
When the program first tries to read or write to one of those addresses, a page fault occurs. The OS catches the page fault, figures out that that address corresponds to a memory-mapped file, and reads the appropriate disk sector into an internal kernel buffer. Then, it maps the kernel buffer into the process's address space, and restarts the user instruction that caused the page fault. If the faulting instruction was a read, we're all done for now. If it was a write, the data is written to memory, and the page is marked as dirty. Subsequent reads or writes to data within the same page do not require reading/writing to/from disk, since the data is in memory.
When the file is flushed or closed, any pages which have been marked dirty are written back to disk.
Using memory-mapped files is advantageous for programs which read or write disk sectors in a very haphazard manner. You only read disk sectors which are actually used, instead of reading the entire file.
I'm not really sure what you are asking, but mmap() sets aside a chunk of virtual memory to hold the given amount of data (usually. It can be file-backed sometimes).
A process is an OS entity, and it gains access to memory mapped areas through the OS-proscribed method: calling mmap().
The kernel has internal buffers representing chunks of memory. Any given process is assigned a memory mapping in its own address space which refers to that buffer. A number of proccesses may have their own mappings, but they all end up resolving to the same chunk (via the kernel buffer).
This is a simple enough concept, but it can get a little tricky when processes write. To keep things simple in the read-only case there's usually a copy-on-write functionality that's only used as needed.
Any data will be in some form of memory or others, some cases in HDD, in embedded systems may be some flash memory or even the ram (initramfs), barring the last one, the data in the memory are frequently cached in the RAM, RAM is logical divided into pages and the kernel maintains a list of descriptors which uniquely identify an page.
So at best accessing data would be accessing the physical pages. Process gets there own process address space which consists of many vm_are_struct which identifies a mapped section in the address space. In a call to mmap, new vm_area_struct may be created or may be merged with an existing one if the addresses are adjacent.
A new virtual address is returned to the call to mmap. Also new page tables are created which consists the mapping of the newly created virtual addresses to the physical address where the real data resides. mapping can be done on a file, or anonymously like malloc. The process address space structure mm_struct uses the pointer of pgd_t (Page global directory) to reach the physical page and access the data.