Address of video memory (0xB8000), who is mapping video memory to this address?
The routine which is copying the data from the address and putting to the screen, Is it a processor inbuilt function (Is this driver comes with the processor)?
What happens when you write to the address:
That area of the address space is not mapped to RAM, instead it gets sent across the system bus to your VGA card. The BIOS set this up with your VGA card at boot time (Lots of address ranges are memory mapped to various devices). No code is being executed on the CPU to plot the pixels when you write to this area of the address space. The VGA card receives this information instead of your RAM and does this itself.
If you wanted you could look BIOS functions calls and have it reconfigure the hardware so you could plot pixels instead of place characters at the video address instead. You could even probe it to see if it supports VESA and switch to a nice 1280*768 32bpp resolution. The BIOS would then map an area of the address space of your choosing to the VGA card for you.
More about the BIOS:
The BIOS is a program that comes with your motherboard that your CPU executes when it first powers up. It sets up all the hardware, maps all the memory mapped devices, creates various useful tables, assigns IO ports, hooks interrupts up to a bunch of routines it leaves in memory. It then loads your bootsector from a device and jumps to your OS code.
The left behind routines and data structures enable you to get your OS off the ground. You can load sectors off a disk, write text to the screen, get information about the system (Memory maps, ACPI tables, MP Tables .etc). Without these routines and data structures, it would be a lot harder if not impossible to make an acceptable bootsector and have all the information about the system to build a functional kernel.
However the routines are dated, slow and have very restrictive limitations. For one the routines left in memory are 16bit real mode code, so as soon as you switch to 32bit protected mode you have to switch back constantly or use VM86 mode to access them (Completely unaccessible in 64bit mode, Apparently emulating the instructions with a modified linux x86emu library is an option though). So the routines are generally very slow. So you will need to write your own drivers from scratch if you move away from real mode programming.
In most cases, the PC monitor is a VGA-compatible device, which by standard includes a mode for setting text buffer (32 KB sized) through MMIO starting from the address of 0xB8000.
The picture above summarizes how MMIO works.
Related
I'm interested in the details of how operating systems work and perhaps writing my own.
From what I've gathered, the BIOS/UEFI is supposed to handle setting up the hardware, and do things like memory-mapping (or IO ports for) the graphics card and other IO devices like audio and ethernet.
My question is, how does the kernel know how to access and (re)configure these devices when it's passed control from the bootloader? Are there just conventions like 'the graphics card is always memory mapped from X to Y address space'? Are you at the mercy of a hardware manufacturer writing a driver for an operating system which knows how the hardware will be initialized?
That seems wrong, so maybe the kernel code includes instructions which somehow iterate through all the bus-connected devices. But what instructions can accomplish that? Is the PCI(e) controller also a memory-mapped device? How do you begin querying and setting up the system?
My primary reference has been the Intel 64 Architectures Software Developer's Manual, which has excellent documentation on how the CPU works, but doesn't describe how the system is setup.
I never wrote a firmware so I don't really know how that works in general. You probably have some memory detection done like an actual memory iteration that is done and some interrogation of PCI devices that are memory mapped in RAM. You also probably have some information in the Developer's manuals as to how you should get some information about memory and stuff like that.
In the end, the kernel doesn't need to bother about that because the firmware takes care to do all that and to provide temporary drivers before the kernel is set up completely.
The firmware passes information to the kernel using the ACPI tables so that is the convention you are looking for. The UEFI firmware launches the /efi/boot/bootx64.efi EFI app from the hard-disk automatically. It calls the main function of that app often called the bootloader. When you write that application, often with frameworks such as EDK2 or GNU-EFI, you can thus use the temporary drivers to get some information like the location of the RSDP which points to all other ACPI tables.
The ACPI convention specifies a language that is AML which, when your kernel interprets, tells it all about hardware. You thus have all the required information there to load drivers and such.
PCI (which is everything nowadays) is another thing. It works with memory mapped IO but the ACPI tables (the MCFG) is helpful to find the beginning of the configuration space for PCI devices that take the form of memory mapped registers.
As to graphics cards, you probably don't want to start with those. They are complex and, at first, you should probably stick to the framebuffer returned by UEFI and at least write a driver for xHCI which is the PCI host controller responsible to interact with USB including keyboards and mouses.
since we know that operating system works in protected mode
and BIOS works in a Real mode ( 16 bit). so when an interrupt is called from
either Operating System or application program, does CPU switch mode back and forth everytime?
In general; hardware is capable of doing lots of things at once (playing sounds while generating 3D graphics while sending data to network while transferring data to multiple disks while waiting for user input while all CPUs are busy doing actual processing); and BIOS functions are not capable of allowing more than one thing to happen at time (e.g. will waste 100% of CPU time waiting for a hard disk controller to transfer data while the CPU does nothing and while nothing else can use any other BIOS service for anything).
For this reason alone, BIOS services are not usable and not used by any modern OS (except briefly during boot).
Of course it's not the only reason - no IO prioritisation, no support for hot-plug of any kind, no support for power management, no support for system management (e.g. https://en.wikipedia.org/wiki/S.M.A.R.T. ), no support for GPU, no support for any sound card (other than "PC speaker beep"), no support for networking (excluding PXE), no support for IO APICs, etc. Ironically, out of all of the problems, "BIOS services are designed for real mode" is the least important problem because it's the only problem that an OS can work around.
Instead, each OS has native drivers that don't have any of these limitations.
Note: this is partly why it's relatively easy for modern operating systems to support UEFI (where none of the BIOS services exist at all) - for most, they replace the boot loader/boot code and don't need to change much else.
i knew that kernel space is dedicated to run the kernels ,while user space for user applications.
but after reading the following paragraph from one of the papers related to OS I am confused,
suppose that we have gesture recognition system and that The design decomposes the system into four components, implemented as separate programs:
catusb: Captures image data from cameras connected on a USB bus.
xform: Uses image-processing algorithms such as noise filtering and geometric transformations to transform images in the camera
perspective to a point cloud in the coordinate system of the screen
or user. Inherently data-parallel.
detect: Detects gestures in a point cloud and features significant data parallelism.
hidinput: Accepts gestures found by the detect program and sends them to the OS as human interface
device (HID) input.
next they mentioned the following flow of execution :
"reading data from a camera requires copying image buffers out of kernel space to user space.
Writing to the pipe connecting catusb to xform causes the same buffer to be written back into kernel space. To run xform on the GPU, the system must read buffers out of kernel space into user space, where a user-mode runtime such as CUDA must subsequently write the buffer back into kernel space and transfer it to the GPU and back."
so what i want to know is:
if the camera buffer is a memory space resides in RAM so when reading
data from camera why to read it from kernel space i don't understand ,is it because that buffer itself resides within the kernel space(in other word ,is the hardware device's memory mapped to RAM resides within the kernel space)?,
and even if the camera buffer resides on the camera's chip so
reading is done by kernel copying that buffer from camera chip to
user space directly, am i right?if so why its mentioned in the
paragraph that copying buffer out of kernel to user space.
The operating system's job is allow a lot of components, both hardware and software, to play nice with each other. In general, userland programs can't directly manipulate peripherals nor interfere with each other. I'm not familiar with the specific setup that you're citing, but it doesn't sound unusual.
The USB camera notifies the operating system that it has a new frame. When the kernel (driver) notices this it, will copy the frame with I/O commands into RAM. Since this RAM was allocated by the driver, the userland programs won't be able to see or read it due to virtual memory. To summarise it quickly, the address &0x1000 in the kernel and the address &0x1000 in a program are actually physically distinct locations in RAM. The kernel will then copy the frame into the memory of any process that is expecting input from the camera and then notify it (in this case catusb).
Likewise, since xform, detect and hdinput exist as separate processes, they must use inter-process communication. Since the operating system must ensure the isolation of the programs, each process will leverage the kernel to achieve this.
There's nothing unusual here. I imagine they are just spelling it out because gesture recognition is time-critical and doing it this way has some overhead.
In order to be executed by the cpu, a program must be loaded into RAM. A program is just a sequence of machine instructions (like the x86 instruction set) that a processor can understand (because it physically implements their semantic through logic gates).
I can more or less understand how a local instruction (an instruction executed inside the cpu chipset) such as 'ADD R1, R2, R3' works. Even how the cpu interfaces with the ram through the northbridge chipset using the data bus and the address bus is clear enough to me.
What I am struggling with is the big picture.
For example how can a file be saved into an hard disk?
Let's say that the motherboard uses a SATA interface to communicate with the HDD.
Does this mean that this SATA interface has an instruction set which can be used by the cpu by preparing SATA instructions written in the correct format?
Does the same apply with the PCI interface, the AGP interface and so on?
Is all the hardware communication basically accomplished through determining a stardard interface for some task and implementing it (by the companies that create hardware chipsets) with an instruction set that any other hardware components can query?
Is my high level understanding of hardware and software interaction correct?
Nearly. It's actually more general than an instruction.
Alot of these details are architecture specific, so I will stick to a high level general overview of how this can be done.
The CPU can read and write to RAM without a problem correct? You can issue instructions that read and write to any memory address. So rather than try to extend the CPU to understand every possible hardware interface out there, hardware manufacturers simply map sections of the address space (where RAM normally would be) to hardware.
Say for example you want to save a file to a hard drive. This is a possible sequence of command that would occur:
The command register of the hard drive controller is address 0xF00, an address that is outside of RAM but accessible to the CPU
Write the instruction to the command register that indicates we want to write to the hard drive.
There could be conceivably an address register at 0xF01 that tells the hard drive controller where to save the data
Tell the hard drive controller that the data I want to write is at some address in RAM, and initiate the write sequence.
There are a myriad of other ways this can be conceivably be done, but the important thing to note is that it is simply using the instructions that the CPU already has for using RAM.
All of this can be done by the CPU without any special instructions on the CPU side, just read and write to an address. You can imagine this being extended, there is a special place in the address space for the USB controller that contains a list of USB devices, there is a special place for the PCI device list, each PCI devices has several registers that can be read and written to instruct them to do things.
Essentially the role of a device driver is to know how these special registers are to be read and written, what kind of commands devices can accept, etc. Often, as is the case with many graphics cards, what these registers do is not documented to the public and so we rely on their drivers to run the cards correctly.
I am trying to write a very thin hypervisor that would have the following restrictions:
runs only one operating system at a time (ie. no OS concurrency, no hardware sharing, no way to switch to another OS)
it should be able only to isolate some portions of RAM (do some memory translations behind the OS - let's say I have 6GB of RAM, I want Linux / Win not to use the first 100MB, see just 5.9MB and use them without knowing what's behind)
I searched the Internet, but found close to nothing on this specific matter, as I want to keep as little overhead as possible (the current hypervisor implementations don't fit my needs).
What you are looking for already exists, in hardware!
It's called IOMMU[1]. Basically, like page tables, adding a translation layer between the executed instructions and the actual physical hardware.
AMD calls it IOMMU[2], Intel calls it VT-d (please google:"intel vt-d" I cannot post more than two links yet).
[1] http://en.wikipedia.org/wiki/IOMMU
[2] http://developer.amd.com/documentation/articles/pages/892006101.aspx
Here are a few suggestions / hints, which are necessarily somewhat incomplete, as developing a from-scratch hypervisor is an involved task.
Make your hypervisor "multiboot-compliant" at first. This will enable it to reside as a typical entry in a bootloader configuration file, e.g., /boot/grub/menu.lst or /boot/grub/grub.cfg.
You want to set aside your 100MB at the top of memory, e.g., from 5.9GB up to 6GB. Since you mentioned Windows I'm assuming you're interested in the x86 architecture. The long history of x86 means that the first few megabytes are filled with all kinds of legacy device complexities. There is plenty of material on the web about the "hole" between 640K and 1MB (plenty of information on the web detailing this). Older ISA devices (many of which still survive in modern systems in "Super I/O chips") are restricted to performing DMA to the first 16 MB of physical memory. If you try to get in between Windows or Linux and its relationship with these first few MB of RAM, you will have a lot more complexity to wrestle with. Save that for later, once you've got something that boots.
As physical addresses approach 4GB (2^32, hence the physical memory limit on a basic 32-bit architecture), things get complex again, as many devices are memory-mapped into this region. For example (referencing the other answer), the IOMMU that Intel provides with its VT-d technology tends to have its configuration registers mapped to physical addresses beginning with 0xfedNNNNN.
This is doubly true for a system with multiple processors. I would suggest you start on a uniprocessor system, disable other processors from within BIOS, or at least manually configure your guest OS not to enable the other processors (e.g., for Linux, include 'nosmp'
on the kernel command line -- e.g., in your /boot/grub/menu.lst).
Next, learn about the "e820" map. Again there is plenty of material on the web, but perhaps the best place to start is to boot up a Linux system and look near the top of the output 'dmesg'. This is how the BIOS communicates to the OS which portions of physical memory space are "reserved" for devices or other platform-specific BIOS/firmware uses (e.g., to emulate a PS/2 keyboard on a system with only USB I/O ports).
One way for your hypervisor to "hide" its 100MB from the guest OS is to add an entry to the system's e820 map. A quick and dirty way to get things started is to use the Linux kernel command line option "mem=" or the Windows boot.ini / bcdedit flag "/maxmem".
There are a lot more details and things you are likely to encounter (e.g., x86 processors begin in 16-bit mode when first powered-up), but if you do a little homework on the ones listed here, then hopefully you will be in a better position to ask follow-up questions.