First of all, is virtual memory a hardware feature of the system, or is it implemented solely by OS?
During link-time relocation, the linker assigns run-time addresses to each section and each symbol, in the generated executable
Do those run-time addresses correspond to virtual addresses?
What if the system for which the executable is generated, does not use virtual memory?
Next, if virtual memory is not used, then the application's address space is limited to the physical address space allocated for it by OS after load-time relocation
Does page fault occur if no virtual memory is used?
I think, it does: in case if the physical page containing the requested physical address has not yet been stored in RAM, then page fault should occur, which is serviced by OS page fault handler
Finally, is paging possible without virtual memory?
I'm asking because paging is always mentioned together with virtual memory, but it seems that the presence of virtual memory is not required to do paging
Thanks
Wow, a lot of questions.
Where is virtual memory implemented? The underlying hardware needs to support virtual memory. Remember, when you access a memory address in your program, the CPU needs some way to obtain the data belonging to this address. If you only have physical access, then the operation is directly sent to the memory controller. In systems with virtual memory you have an MMU (memory management unit), which translates a virtual address into a physical one. (Note, that some microcontrollers provide a stripped-down version, called a Memory-Protection Unit (MPU), which does not provide this translation step, but at least allows access rights checking.)
Do link-time addresses correspond to virtual addresses at runtime? In general, link-time addresses correspond to runtime virtual addresses. However, there is a mode where this is not the case: position-independent code. Here, the virtual addresses are determined at load time by a dynamic linker. This approach is usually used to load dynamically linked libraries (DLL / .so) to an application. For more details on that topic, you migth want to check out "Linkers and Loaders".
What if my target system does not have virtual memory? If your system does not support virtual memory, from the compiler's/loader's perspective nothing really changes: you still need to generate code to access memory. The only difference is that your CPU does no additional translation from a virtual to a physical address anymore.
Are there page faults if there is no virtual memory? There are no page faults if you don't have virtual memory. However, in case of an MPU you might still see access violations reported by the hardware, if your application tries to access an address it is not supposed to read/write. Note, that physical addresses (better: data pointed to by physical addresses) don't need to be loaded into RAM. They are just pointers into the RAM which is already there.
Is paging possible without virtual memory? 'Paging' and 'Virtual Memory' are often used to denote the same thing. However, paging may also refer to the concept of splitting memory into chunks of the same size - pages. The answer to your question depends on what you mean by paging. ;)
Related
What is the need of virtual memory on 64 bit microprocessor? As i know it can address around 16exabyte memory. So why do we still need paging.
Thanks in advance
In addition to providing virtual memory, paging is used to control memory protection, to provide separation between different applications and between applications and the operating system. Paging also allow different applications to use the same linear address to access different memory locations.
The memory pager is also capable of doing other very useful things, such as mapping a file to memory and paging only the blocks that are actually used from disk, mapping the same data into multiple processes with copy-on-write, giving each program only as much physical RAM as it actually uses, implementing shared memory, memory-mapped I/O and virtualization.
The main reason to have virtual memory is to be able to work with more data than the system has physical memory, but most of the underlying infrastructure (with the significant exception of the paging algorithm) would be needed anyway, and has hardware support.
In the future we may see paging go away. One other problem is that we have systems with 8GB of physical memory with 64 bit processors. As soon as you need more than 8GB of memory, you have to resort to paging. It should not be that long until we have computer systems that have terrabytes of memory and paging will not be necessary.
In that case we will need new operating systems and even new computer systems to take advantage of such large memory.
The question is conceptual/theoretical, nothing about anything I am working practically on.
I understand in the virtual memory layout you have the heap, stack, data, memory mapped, etc sections
I was wondering how would it work if for example on macosx the driver wants to access one of the registers in the memory mapped IO(MMIO) region ?
I would assume that it would need to some how know the physical page number to where the MMIO region is present, but how would it get this? Any function call ?
Also, what if after a while the page was swapped out and in to a different page number ? Would it need to check/fetch the page number to the MMIO registers every time ?
How MMIO pages are discovered depends on the kind of device and platform we're talking about. On something as sophisticated and complex as a Mac, schemes like ACPI are used for assigning and enumerating MMIO addresses. Some addresses are most likely also hardcoded in Firmware and accessible to the OS via the Firmware's API. On simple embedded platforms, you'll often just find MMIO ranges hard-coded to some specific physical address.
PCI devices advertise their MMIO ranges in the configuration space, and system software (Firmware and/or OS) can decide where in physical address space the device's MMIO ranges should be located.
If the (x86/-64) CPU is running in paged/long mode, which for a Mac is true even in (EFI) firmware, the MMIO pages need to be mapped into virtual memory address space using the page table, in order to be software-accessible.
Also, what if after a while the page was swapped out and in to a
different page number ?
A MMIO-backed virtual page will not be swapped out - the only reason a virtual page would be swapped out would be because the system is low on unused physical system memory, and the OS needs to reclaim some. Removing an MMIO-backed page from the page table doesn't free any system memory.
I basically understand how virtual memory works but I'm wondering whether there are some situations where virtual memory is not used, especially for kernel address space.
Thanks!
Only on older systems, I don't think that most current systems don't use it (unless it's a very specific device where all the functionality is in kernel context).
In Windows, with reference:
In modern operating systems such as Windows, applications and many
system processes always reference memory by using virtual memory
addresses. Virtual memory addresses are automatically translated to
real (RAM) addresses by the hardware. Only core parts of the operating
system kernel bypass this address translation and use real memory
addresses directly.
Virtual memory is always being used, even when the memory that is
required by all running processes does not exceed the volume of RAM
that is installed on the system.
I have a slight doubt regarding virtual memory.
Normally, it is up to the OS to provide virtual memory to using disk space to expand the amount of memory which appears to be available for applications.
The OS will clear physical memory by copying the data to disk and restoring when needed.
However, it is possible for an application to manage its own “virtual memory” rather than the OS, for example by writing objects to a file then destroying them?
If so, is allowing application to manage its own virtual memory for advantageous or allowing the OS to provide?
Most applications would not be able to even know that they are being managed using virtual memory because the operating system would perform address translation on every memory request made by your application.
This is a task definitely best left to the operating system unless you are working in a very low-level environment (in which case you are probably writing your own operating system anyway).
Aside from the fact that this requires kernel privileges to accomplish, you would need to take care not to corrupt other process' memory.
The operating system is the best place for this kind of logic.
It is not just not advantageous for the application to manage its own virtual memory, it is not possible with standard operating systems (Windows, Unix, Linux, Mac OS X, etc.).
Translation from virtual address to physical address is done by the Memory Management Unit of the system, which is typically firmware, not strictly part of the operating system software.
The only part of the process done by the operating system software is handling of page faults (swapping units of virtual memory to and from backing store), when the address translation finds a reference to a virtual address that is not currently mapped in physical memory.
What could be advantageous is for an application to minimize its use of virtual memory by writing out its own data to disk rather than allocating larger amounts of virtual memory. However, this will only yield a benefit if the application's disk i/o is more efficient than the operating system page handler's disk i/o - an unlikely scenario these days.
Virtual memory is a good concept currently used by modern operating systems. But I was stuck answering a question and was not sure enough about it. Here is the question:
Suppose there are only a few applications running on a machine, such that the
physical memory of system is more than the memory required by all the
applications. To support virtual memory, the OS needs to do a lot work. So if
the running applications all fit in the physical memory, is virtual memory
really needed?
(Furthermore, the applications running together will always fit in RAM.)
Even when the memory usage of all applications fits in physical memory, virtual memory is still useful. VM can provide these features:
Privileged memory isolation (every app can't touch the kernel or memory-mapped hardware devices)
Interprocess memory isolation (one app can't see another app's memory)
Static memory addresses (e.g. every app has main() at address 0x0800 0000)
Lazy memory (e.g. pages in the stack are allocated and set to zero when first accessed)
Redirected memory (e.g. memory-mapped files)
Shared program code (if more than one instance of a program or library is running, its code only needs to be stored in memory once)
While not strictly needed in this scenario, virtual memory is about more than just providing "more" memory than is physically available (swapping). For example, it helps avoiding memory fragmentation (from an application point of view) and depending on how dynamic/shared libraries are implemented, it can help to avoid relocation (relocation is when the dynamic linker needs to adapt pointers in a library or executable that was just loaded).
A few more points to consider:
Buggy apps that don't handle failures in the memory allocation code
Buggy apps that leak allocated memory
Virtual memory reduces severity of these bugs.
The other replies list valid reasons why virtual memory is useful but
I would like to answer the question more directly : No, virtual memory
is not needed in the situation you describe and not using virtual
memory can be the right trade-off in such situations.
Seymour Cray took the position that "Virtual memory leads to virtual
performance." and most (all?) Cray vector machines lacked virtual
memory. This usually leads to higher performance on the process level
(no translations needed, processes are contiguous in RAM) but can lead
to poorer resource usage on the system level (the OS cannot utilize
RAM fully since it gets fragmented on the process level).
So if a system is targeting maximum performance (as opposed to maximum
resource utilization) skipping virtual memory can make sense.
When you experience the severe performance (and stability) problems
often seen on modern Unix-based HPC cluster nodes when users
oversubscribe RAM and the system starts to page to disk, there is a
certain sympathy with the Cray model where the process either starts
and runs at max performance, or it doesn't start at all.