The question is conceptual/theoretical, nothing about anything I am working practically on.
I understand in the virtual memory layout you have the heap, stack, data, memory mapped, etc sections
I was wondering how would it work if for example on macosx the driver wants to access one of the registers in the memory mapped IO(MMIO) region ?
I would assume that it would need to some how know the physical page number to where the MMIO region is present, but how would it get this? Any function call ?
Also, what if after a while the page was swapped out and in to a different page number ? Would it need to check/fetch the page number to the MMIO registers every time ?
How MMIO pages are discovered depends on the kind of device and platform we're talking about. On something as sophisticated and complex as a Mac, schemes like ACPI are used for assigning and enumerating MMIO addresses. Some addresses are most likely also hardcoded in Firmware and accessible to the OS via the Firmware's API. On simple embedded platforms, you'll often just find MMIO ranges hard-coded to some specific physical address.
PCI devices advertise their MMIO ranges in the configuration space, and system software (Firmware and/or OS) can decide where in physical address space the device's MMIO ranges should be located.
If the (x86/-64) CPU is running in paged/long mode, which for a Mac is true even in (EFI) firmware, the MMIO pages need to be mapped into virtual memory address space using the page table, in order to be software-accessible.
Also, what if after a while the page was swapped out and in to a
different page number ?
A MMIO-backed virtual page will not be swapped out - the only reason a virtual page would be swapped out would be because the system is low on unused physical system memory, and the OS needs to reclaim some. Removing an MMIO-backed page from the page table doesn't free any system memory.
Related
In order to be executed by the cpu, a program must be loaded into RAM. A program is just a sequence of machine instructions (like the x86 instruction set) that a processor can understand (because it physically implements their semantic through logic gates).
I can more or less understand how a local instruction (an instruction executed inside the cpu chipset) such as 'ADD R1, R2, R3' works. Even how the cpu interfaces with the ram through the northbridge chipset using the data bus and the address bus is clear enough to me.
What I am struggling with is the big picture.
For example how can a file be saved into an hard disk?
Let's say that the motherboard uses a SATA interface to communicate with the HDD.
Does this mean that this SATA interface has an instruction set which can be used by the cpu by preparing SATA instructions written in the correct format?
Does the same apply with the PCI interface, the AGP interface and so on?
Is all the hardware communication basically accomplished through determining a stardard interface for some task and implementing it (by the companies that create hardware chipsets) with an instruction set that any other hardware components can query?
Is my high level understanding of hardware and software interaction correct?
Nearly. It's actually more general than an instruction.
Alot of these details are architecture specific, so I will stick to a high level general overview of how this can be done.
The CPU can read and write to RAM without a problem correct? You can issue instructions that read and write to any memory address. So rather than try to extend the CPU to understand every possible hardware interface out there, hardware manufacturers simply map sections of the address space (where RAM normally would be) to hardware.
Say for example you want to save a file to a hard drive. This is a possible sequence of command that would occur:
The command register of the hard drive controller is address 0xF00, an address that is outside of RAM but accessible to the CPU
Write the instruction to the command register that indicates we want to write to the hard drive.
There could be conceivably an address register at 0xF01 that tells the hard drive controller where to save the data
Tell the hard drive controller that the data I want to write is at some address in RAM, and initiate the write sequence.
There are a myriad of other ways this can be conceivably be done, but the important thing to note is that it is simply using the instructions that the CPU already has for using RAM.
All of this can be done by the CPU without any special instructions on the CPU side, just read and write to an address. You can imagine this being extended, there is a special place in the address space for the USB controller that contains a list of USB devices, there is a special place for the PCI device list, each PCI devices has several registers that can be read and written to instruct them to do things.
Essentially the role of a device driver is to know how these special registers are to be read and written, what kind of commands devices can accept, etc. Often, as is the case with many graphics cards, what these registers do is not documented to the public and so we rely on their drivers to run the cards correctly.
I basically understand how virtual memory works but I'm wondering whether there are some situations where virtual memory is not used, especially for kernel address space.
Thanks!
Only on older systems, I don't think that most current systems don't use it (unless it's a very specific device where all the functionality is in kernel context).
In Windows, with reference:
In modern operating systems such as Windows, applications and many
system processes always reference memory by using virtual memory
addresses. Virtual memory addresses are automatically translated to
real (RAM) addresses by the hardware. Only core parts of the operating
system kernel bypass this address translation and use real memory
addresses directly.
Virtual memory is always being used, even when the memory that is
required by all running processes does not exceed the volume of RAM
that is installed on the system.
Address of video memory (0xB8000), who is mapping video memory to this address?
The routine which is copying the data from the address and putting to the screen, Is it a processor inbuilt function (Is this driver comes with the processor)?
What happens when you write to the address:
That area of the address space is not mapped to RAM, instead it gets sent across the system bus to your VGA card. The BIOS set this up with your VGA card at boot time (Lots of address ranges are memory mapped to various devices). No code is being executed on the CPU to plot the pixels when you write to this area of the address space. The VGA card receives this information instead of your RAM and does this itself.
If you wanted you could look BIOS functions calls and have it reconfigure the hardware so you could plot pixels instead of place characters at the video address instead. You could even probe it to see if it supports VESA and switch to a nice 1280*768 32bpp resolution. The BIOS would then map an area of the address space of your choosing to the VGA card for you.
More about the BIOS:
The BIOS is a program that comes with your motherboard that your CPU executes when it first powers up. It sets up all the hardware, maps all the memory mapped devices, creates various useful tables, assigns IO ports, hooks interrupts up to a bunch of routines it leaves in memory. It then loads your bootsector from a device and jumps to your OS code.
The left behind routines and data structures enable you to get your OS off the ground. You can load sectors off a disk, write text to the screen, get information about the system (Memory maps, ACPI tables, MP Tables .etc). Without these routines and data structures, it would be a lot harder if not impossible to make an acceptable bootsector and have all the information about the system to build a functional kernel.
However the routines are dated, slow and have very restrictive limitations. For one the routines left in memory are 16bit real mode code, so as soon as you switch to 32bit protected mode you have to switch back constantly or use VM86 mode to access them (Completely unaccessible in 64bit mode, Apparently emulating the instructions with a modified linux x86emu library is an option though). So the routines are generally very slow. So you will need to write your own drivers from scratch if you move away from real mode programming.
In most cases, the PC monitor is a VGA-compatible device, which by standard includes a mode for setting text buffer (32 KB sized) through MMIO starting from the address of 0xB8000.
The picture above summarizes how MMIO works.
First of all, is virtual memory a hardware feature of the system, or is it implemented solely by OS?
During link-time relocation, the linker assigns run-time addresses to each section and each symbol, in the generated executable
Do those run-time addresses correspond to virtual addresses?
What if the system for which the executable is generated, does not use virtual memory?
Next, if virtual memory is not used, then the application's address space is limited to the physical address space allocated for it by OS after load-time relocation
Does page fault occur if no virtual memory is used?
I think, it does: in case if the physical page containing the requested physical address has not yet been stored in RAM, then page fault should occur, which is serviced by OS page fault handler
Finally, is paging possible without virtual memory?
I'm asking because paging is always mentioned together with virtual memory, but it seems that the presence of virtual memory is not required to do paging
Thanks
Wow, a lot of questions.
Where is virtual memory implemented? The underlying hardware needs to support virtual memory. Remember, when you access a memory address in your program, the CPU needs some way to obtain the data belonging to this address. If you only have physical access, then the operation is directly sent to the memory controller. In systems with virtual memory you have an MMU (memory management unit), which translates a virtual address into a physical one. (Note, that some microcontrollers provide a stripped-down version, called a Memory-Protection Unit (MPU), which does not provide this translation step, but at least allows access rights checking.)
Do link-time addresses correspond to virtual addresses at runtime? In general, link-time addresses correspond to runtime virtual addresses. However, there is a mode where this is not the case: position-independent code. Here, the virtual addresses are determined at load time by a dynamic linker. This approach is usually used to load dynamically linked libraries (DLL / .so) to an application. For more details on that topic, you migth want to check out "Linkers and Loaders".
What if my target system does not have virtual memory? If your system does not support virtual memory, from the compiler's/loader's perspective nothing really changes: you still need to generate code to access memory. The only difference is that your CPU does no additional translation from a virtual to a physical address anymore.
Are there page faults if there is no virtual memory? There are no page faults if you don't have virtual memory. However, in case of an MPU you might still see access violations reported by the hardware, if your application tries to access an address it is not supposed to read/write. Note, that physical addresses (better: data pointed to by physical addresses) don't need to be loaded into RAM. They are just pointers into the RAM which is already there.
Is paging possible without virtual memory? 'Paging' and 'Virtual Memory' are often used to denote the same thing. However, paging may also refer to the concept of splitting memory into chunks of the same size - pages. The answer to your question depends on what you mean by paging. ;)
I am trying to write a very thin hypervisor that would have the following restrictions:
runs only one operating system at a time (ie. no OS concurrency, no hardware sharing, no way to switch to another OS)
it should be able only to isolate some portions of RAM (do some memory translations behind the OS - let's say I have 6GB of RAM, I want Linux / Win not to use the first 100MB, see just 5.9MB and use them without knowing what's behind)
I searched the Internet, but found close to nothing on this specific matter, as I want to keep as little overhead as possible (the current hypervisor implementations don't fit my needs).
What you are looking for already exists, in hardware!
It's called IOMMU[1]. Basically, like page tables, adding a translation layer between the executed instructions and the actual physical hardware.
AMD calls it IOMMU[2], Intel calls it VT-d (please google:"intel vt-d" I cannot post more than two links yet).
[1] http://en.wikipedia.org/wiki/IOMMU
[2] http://developer.amd.com/documentation/articles/pages/892006101.aspx
Here are a few suggestions / hints, which are necessarily somewhat incomplete, as developing a from-scratch hypervisor is an involved task.
Make your hypervisor "multiboot-compliant" at first. This will enable it to reside as a typical entry in a bootloader configuration file, e.g., /boot/grub/menu.lst or /boot/grub/grub.cfg.
You want to set aside your 100MB at the top of memory, e.g., from 5.9GB up to 6GB. Since you mentioned Windows I'm assuming you're interested in the x86 architecture. The long history of x86 means that the first few megabytes are filled with all kinds of legacy device complexities. There is plenty of material on the web about the "hole" between 640K and 1MB (plenty of information on the web detailing this). Older ISA devices (many of which still survive in modern systems in "Super I/O chips") are restricted to performing DMA to the first 16 MB of physical memory. If you try to get in between Windows or Linux and its relationship with these first few MB of RAM, you will have a lot more complexity to wrestle with. Save that for later, once you've got something that boots.
As physical addresses approach 4GB (2^32, hence the physical memory limit on a basic 32-bit architecture), things get complex again, as many devices are memory-mapped into this region. For example (referencing the other answer), the IOMMU that Intel provides with its VT-d technology tends to have its configuration registers mapped to physical addresses beginning with 0xfedNNNNN.
This is doubly true for a system with multiple processors. I would suggest you start on a uniprocessor system, disable other processors from within BIOS, or at least manually configure your guest OS not to enable the other processors (e.g., for Linux, include 'nosmp'
on the kernel command line -- e.g., in your /boot/grub/menu.lst).
Next, learn about the "e820" map. Again there is plenty of material on the web, but perhaps the best place to start is to boot up a Linux system and look near the top of the output 'dmesg'. This is how the BIOS communicates to the OS which portions of physical memory space are "reserved" for devices or other platform-specific BIOS/firmware uses (e.g., to emulate a PS/2 keyboard on a system with only USB I/O ports).
One way for your hypervisor to "hide" its 100MB from the guest OS is to add an entry to the system's e820 map. A quick and dirty way to get things started is to use the Linux kernel command line option "mem=" or the Windows boot.ini / bcdedit flag "/maxmem".
There are a lot more details and things you are likely to encounter (e.g., x86 processors begin in 16-bit mode when first powered-up), but if you do a little homework on the ones listed here, then hopefully you will be in a better position to ask follow-up questions.