since we know that operating system works in protected mode
and BIOS works in a Real mode ( 16 bit). so when an interrupt is called from
either Operating System or application program, does CPU switch mode back and forth everytime?
In general; hardware is capable of doing lots of things at once (playing sounds while generating 3D graphics while sending data to network while transferring data to multiple disks while waiting for user input while all CPUs are busy doing actual processing); and BIOS functions are not capable of allowing more than one thing to happen at time (e.g. will waste 100% of CPU time waiting for a hard disk controller to transfer data while the CPU does nothing and while nothing else can use any other BIOS service for anything).
For this reason alone, BIOS services are not usable and not used by any modern OS (except briefly during boot).
Of course it's not the only reason - no IO prioritisation, no support for hot-plug of any kind, no support for power management, no support for system management (e.g. https://en.wikipedia.org/wiki/S.M.A.R.T. ), no support for GPU, no support for any sound card (other than "PC speaker beep"), no support for networking (excluding PXE), no support for IO APICs, etc. Ironically, out of all of the problems, "BIOS services are designed for real mode" is the least important problem because it's the only problem that an OS can work around.
Instead, each OS has native drivers that don't have any of these limitations.
Note: this is partly why it's relatively easy for modern operating systems to support UEFI (where none of the BIOS services exist at all) - for most, they replace the boot loader/boot code and don't need to change much else.
Peripheral devices require drivers to work in a computer system (operating system).
Does a CPU need a driver to work?
Same question for a main memory?
The answer is no.
The reason is that the motherboard comes with an (upgradable) BIOS, which takes care of making sure the CPU features function correctly (obviously, an AMD processor won't work on an Intel motherboard). You can upgrade the BIOS, but that should be avoided until, ... reasons of course.
Same goes for memory, it does not require a driver either.
Just so that you know, if you ever tried overclocking you can notice that you can alter the way the RAM functions, ganged/unganged mods and so on. My point is that there is already an interface established using code allowing you to make changes in real time, isn't that the very purpose we even have drivers, to be able to use a peripheral with expected outcome.
On the other hand, peripheral devices are just extensions, which the motherboard does not know how to handle, hence needing a set of instructions i.e. drivers.
In a modern system both memory and the CPU require kernel mode code — as do devices — to function.
Memory requires management of virtual memory tables. The CPU requires maintenance of process control structures.
In the business, such code is not called a "driver".
Generally, one thinks of a device driver as being kernel mode code that responds to devices through the interrupt vector.
That said, on some systems there are "printer drivers" that do not fit that definition of driver.
In short, do memory and CPU have something called a "driver"? No.
Do they have something analogous to a driver? Yes.
In order to be executed by the cpu, a program must be loaded into RAM. A program is just a sequence of machine instructions (like the x86 instruction set) that a processor can understand (because it physically implements their semantic through logic gates).
I can more or less understand how a local instruction (an instruction executed inside the cpu chipset) such as 'ADD R1, R2, R3' works. Even how the cpu interfaces with the ram through the northbridge chipset using the data bus and the address bus is clear enough to me.
What I am struggling with is the big picture.
For example how can a file be saved into an hard disk?
Let's say that the motherboard uses a SATA interface to communicate with the HDD.
Does this mean that this SATA interface has an instruction set which can be used by the cpu by preparing SATA instructions written in the correct format?
Does the same apply with the PCI interface, the AGP interface and so on?
Is all the hardware communication basically accomplished through determining a stardard interface for some task and implementing it (by the companies that create hardware chipsets) with an instruction set that any other hardware components can query?
Is my high level understanding of hardware and software interaction correct?
Nearly. It's actually more general than an instruction.
Alot of these details are architecture specific, so I will stick to a high level general overview of how this can be done.
The CPU can read and write to RAM without a problem correct? You can issue instructions that read and write to any memory address. So rather than try to extend the CPU to understand every possible hardware interface out there, hardware manufacturers simply map sections of the address space (where RAM normally would be) to hardware.
Say for example you want to save a file to a hard drive. This is a possible sequence of command that would occur:
The command register of the hard drive controller is address 0xF00, an address that is outside of RAM but accessible to the CPU
Write the instruction to the command register that indicates we want to write to the hard drive.
There could be conceivably an address register at 0xF01 that tells the hard drive controller where to save the data
Tell the hard drive controller that the data I want to write is at some address in RAM, and initiate the write sequence.
There are a myriad of other ways this can be conceivably be done, but the important thing to note is that it is simply using the instructions that the CPU already has for using RAM.
All of this can be done by the CPU without any special instructions on the CPU side, just read and write to an address. You can imagine this being extended, there is a special place in the address space for the USB controller that contains a list of USB devices, there is a special place for the PCI device list, each PCI devices has several registers that can be read and written to instruct them to do things.
Essentially the role of a device driver is to know how these special registers are to be read and written, what kind of commands devices can accept, etc. Often, as is the case with many graphics cards, what these registers do is not documented to the public and so we rely on their drivers to run the cards correctly.
While understanding each by itself (or maybe not), it looks like I'm far from understanding the practical differences between the two.
Per my understanding, a BSP is a package of drivers and configuration settings that allows a kernel image to boot up a board (and is part of it).
The individual device driver, operates on a specific component (hardware), interfacing on one side with the core kernel and on the other side with the device itself.
Looking at the Linux kernel, it is unclear to me where the BSP role starts and the device driver role ends. Specifically, I am used to see one BSP per board per image, however, the generic Linux kernel may be loaded on any architecture family with the same image (it is clear that for different families there are different images: x86, amd64, arm, etc.), where the specific board and peripherals drivers are loaded per need from the initrd.
Is there a BSP for the common Linux kernel distributions? Or is BSP relevant just for special cases boards?
Is this behavior similar on other kernels? VxWorks?
And the last one, is it common to merge different BSP/s in order to generate a single image that will fit different boards?
I see the relationship between BSPs and devices drivers as "has-a". Board support packages include device drivers.
The differences between BSPs & kernels isn't easy to distinguish. A kernel translates instructions to the hardware. Kernels are often written to particular families of hardware, so they're not as portable or generic as they seem. It amounts to different permutations of the code for each architecture family.
The BSP acts as sort of the inverse: it provides the tools & instructions to work with that board's specific set of hardware. In specific, controlled situations, the kernel could do this work. But the BSP enables any compatible kernel/OS/application stack to use that board, by following its configuration instructions.
If you just need to access CPU cycles & memory, maybe a few protocols (USB, Ethernet, and a couple of video types), a kernel with wide architecture support is fantastic, and there was a time when the breadth of that hardware abstraction was penultimately valued. But now, consider that the board may have a suite of sensors (accelerometer, magnetometer, gyroscope, light, proximity, atmospheric pressure, etc), telephony, there may be multiple CPUs, multiple GPUs, and so on. A kernel can be written to provide VGA, DVI, HDMI, DisplayPort, and several permutations of CPU/GPU combinations, if/when someone uses those particular hardware packages, but it's not practical to write support for all theoretical contexts, compared to utilizing a BSP that's built for a specific board. And even then, that would be for one kernel; the board is capable of supporting Linux, Windows, Android, Symbian, whatever.
That's why efforts like Yocto exist, to further decouple kernel and hardware. BSPs make hardware sets extensible beyond a kernel, OS, and application stack or two, while kernels make a particular OS or application stack portable over multiple hardware architectures.
Based on my experience, BSP is a much larger scope. It includes bootloader, rootfs, kernel, drivers, etc., which means having a BSP makes your board capable of booting itself up. Drivers make devices working and are just a part of BSP.
Drivers is not equal to BSP.
Today things are modular to increase reusability, and software development for embedded systems normally breaks down into three layers.
Kernel (which contain task handling, scheduling, and memory management)
Stack (which is the upper layer on the device drivers and provides protocol implementations for I²C, SPI, Ethernet, SDIO, serial, file system, networking, etc.)
BSP = Device drivers (which provide access to any controller's registers on hardware like registers of I²C, SDIO, SPI, Ethernet MAC address, UART (serial) and interrupt handling (ISR).
Board support package (device driver) is a software layer which changes with every board, keeping the other two software layers unchanged.
There is a conceptual link between board support packages and a HAL (Hardware Abstraction Layer) in the sense that the device drivers / kernel modules perform the hardware abstraction and the board support package provides an interface to the device drivers or is the hardware abstraction layer itself.
So basically a BSP has a functionality similar to the BIOS in the DOS era:
Additionally the BSP is supposed to perform the following operations
Initialize the processor
Initialize the bus
Initialize the interrupt controller
Initialize the clock
Initialize the RAM settings
Configure the segments
Load and run bootloader from flash
From: Board support package (Wikipedia)
The BIOS in modern PCs initializes and tests the system hardware
components, and loads a boot loader from a mass memory device which
then initializes an operating system. In the era of DOS, the BIOS
provided a hardware abstraction layer for the keyboard, display, and
other input/output (I/O) devices [device drivers] that standardized an interface to
application programs and the operating system. More recent operating
systems do not use the BIOS after loading, instead accessing the
hardware components directly.
Source: BIOS (Wikipedia)
Another aspect is the usage of device trees in BSPs, the device tree is a unifying or standardizing concept to describe the hardware of a machine:
U-boot boot loader and getting ready to ship
Doug Abbott, in Linux for Embedded and Real-Time Applications (Fourth
Edition), 2018
Device Trees
One of the biggest problems with porting an operating system such as
Linux to a new platform is describing the hardware. That is because
the hardware description is scattered over perhaps several dozen or so
device drivers, the kernel, and the boot loader, just to name a few.
The ultimate result is that these various pieces of software become
unique to each platform, the number of configuration options grows,
and every board requires a unique kernel image.
There have been a number of approaches to addressing this problem. The
notion of a “board support package” or BSP attempts to gather all of
the hardware-dependent code in a few files in one place. It could be
argued that the entire arch/ subtree of the Linux kernel source tree
is a gigantic board support package.
Take a look at the arch/arm/ subtree of the kernel. In there you will
find a large number of directories of the form mach-* and plat-*,
presumably short for “machine” and “platform,” respectively. Most of
the files in these directories provide configuration information for a
specific implementation of the ARM architecture. And of course, each
implementation describes its configuration differently.
Would not it be nice to have a single language that could be used to
unambiguously describe the hardware of a computer system? That is the
premise, and promise, of device trees.
The peripheral devices in a system can be characterized along a number
of dimensions. There are, for example, character vs block devices.
There are memory mapped devices, and those that connect through an
external bus such as I2C or USB. Then there are platform devices and
discoverable devices.
Discoverable devices are those that live on external busses, such as
PCI and USB, that can tell the system what they are and how they are
configured. That is, they can be “discovered” by the kernel. Having
identified a device, it is a fairly simple matter to load the
corresponding driver, which then interrogates the device to determine
its precise configuration.
Platform devices, on the other hand, lack any mechanism to identify
themselves. System on Chip (SoC) implementations, such as the Sitara,
are rife with these platform devices—system clocks, interrupt
controllers, GPIO, serial ports, to name a few. The device tree
mechanism is particularly useful for managing platform devices.
The device tree concept evolved in the PowerPC branch of the kernel,
and that is where it seems to be used the most. In fact, it is now a
requirement that all PowerPC platforms pass a device tree to the
kernel at boot time. The text representation of a device tree is a
file with the extension .dts. These .dts files are typically found in
the kernel source tree at arch/$ARCH/boot/dts.
A device tree is a hierarchical data structure that describes the
collection of devices and interconnecting busses of a computer system.
It is organized as nodes that begin at a root represented by “/,” just
like the root file system. Every node has a name and consists of
“properties” that are name-value pairs. It may also contain “child”
nodes.
Listing 15.1 is a sample device tree taken from the devicetree.org
website. It does nothing beyond illustrating the structure. Here we
have two nodes named node1 and node2. node1 has two child nodes, and
node2 has one child. Properties are represented by name=value. Values
can be strings, lists of strings, one or more numeric values enclosed
by square brackets, or one or more “cells” enclosed in angle brackets.
The value can also be empty if the property conveys a Boolean value by
its presence or absence.
Source: Board Support Package (ScienceDirect)
Via device tree overlays kernel modules can be loaded at boot time, i.e., on Raspberry Pi adding dtoverlay=lirc-rpi to /boot/config.txt loads the lirc-pi kernel module/device driver:
A future default config.txt may contain a section like this:
# Uncomment some or all of these to enable the optional hardware interfaces
#dtparam=i2c_arm=on
#dtparam=i2s=on
#dtparam=spi=on
If you have an overlay that defines some parameters, they can be
specified either on subsequent lines like this:
dtoverlay=lirc-rpi
dtparam=gpio_out_pin=16
dtparam=gpio_in_pin=17
dtparam=gpio_in_pull=down
Source: Configuration (Raspberry Pi Documentation)
When building BSPs with Yocto, all the hardware information that is scattered over device drivers, the kernel, the boot loader, etc. is gathered. Here is the developer's guide how this can be done in Yocto: Yocto Project Board Support Package Developer's Guide
[Yocto documentation]... states that the BSP “…is concerned with the
hardware-specific components only. At the end-distribution point, you
can ship the BSP combined with a build system and other tools.
However, it is important to maintain the distinction that these are
separate components that happen to be combined in certain end
products.”
Source: Board Support Package: what is it?
A board support package includes everything that is needed to use the board by an application. These include device drivers for the devices on the board and utility software for application programmers. A windowing environment is also available on multi-media boards. System engineers can further add extensions to the board. Some applications require reimplementing some part of the BSP for enhancements. Here the BSP plays a role of a reference implementation or a starting point for such requirements.
The confusion lies in the business model. The reference or development board is not an end/consumer product like a mobile device. It plays an important role to design and develop a product like iPhone or Samsung Galaxy.
A generic BSP will lack optimization in most cases therefore you can only expect a generic BSP for the newbie model or where optimization is left for you to be done. In case of cheap boards the BSP is quite generic because the producer will put less investment into it.
Don't stress much on the terms of kernel and user-space as there are also microkernels available. Here the drivers are part of user-space! Again think of a low-power board which only has one piece of code without any kernel. So it boils down to software that supports the board to do its job.
The driver is a program which says to the kernel like the behavior of the device... The device may be USB devices or camera or Bluetooth or it can be anything.
Based on the size of operation we classify into three char, block, network. But it only gives access to each device...It configures only the device alone not configure the memory, CPU speed. It does not give the instruction for that processor or controller. It is work on that processor or controller. Who enables that microcontroller who define the functionalities, who gives the starting point of the microcontroller. Who gives instructions. Now comes the answer like BSP.
BSP is a board support package which enables the bootloader. It gives the behavior of the system.
Consider the two scenarios,
One is having pc in pc having a USB connector option. All are okay. This is the first scenario
Second is I am having a pc, board alone in the board having USB. The board should talk to USB. What can I do?
In this case, I have a pc with an OS, so I do not need to think about the behaviour of the system. So I just enable the behavior of the device with system OS
In this, the board means that processor with all peripherals. In this case, we do not have an OS, so we need to make or we need to enable the behavior of that device.
I am trying to write a very thin hypervisor that would have the following restrictions:
runs only one operating system at a time (ie. no OS concurrency, no hardware sharing, no way to switch to another OS)
it should be able only to isolate some portions of RAM (do some memory translations behind the OS - let's say I have 6GB of RAM, I want Linux / Win not to use the first 100MB, see just 5.9MB and use them without knowing what's behind)
I searched the Internet, but found close to nothing on this specific matter, as I want to keep as little overhead as possible (the current hypervisor implementations don't fit my needs).
What you are looking for already exists, in hardware!
It's called IOMMU[1]. Basically, like page tables, adding a translation layer between the executed instructions and the actual physical hardware.
AMD calls it IOMMU[2], Intel calls it VT-d (please google:"intel vt-d" I cannot post more than two links yet).
[1] http://en.wikipedia.org/wiki/IOMMU
[2] http://developer.amd.com/documentation/articles/pages/892006101.aspx
Here are a few suggestions / hints, which are necessarily somewhat incomplete, as developing a from-scratch hypervisor is an involved task.
Make your hypervisor "multiboot-compliant" at first. This will enable it to reside as a typical entry in a bootloader configuration file, e.g., /boot/grub/menu.lst or /boot/grub/grub.cfg.
You want to set aside your 100MB at the top of memory, e.g., from 5.9GB up to 6GB. Since you mentioned Windows I'm assuming you're interested in the x86 architecture. The long history of x86 means that the first few megabytes are filled with all kinds of legacy device complexities. There is plenty of material on the web about the "hole" between 640K and 1MB (plenty of information on the web detailing this). Older ISA devices (many of which still survive in modern systems in "Super I/O chips") are restricted to performing DMA to the first 16 MB of physical memory. If you try to get in between Windows or Linux and its relationship with these first few MB of RAM, you will have a lot more complexity to wrestle with. Save that for later, once you've got something that boots.
As physical addresses approach 4GB (2^32, hence the physical memory limit on a basic 32-bit architecture), things get complex again, as many devices are memory-mapped into this region. For example (referencing the other answer), the IOMMU that Intel provides with its VT-d technology tends to have its configuration registers mapped to physical addresses beginning with 0xfedNNNNN.
This is doubly true for a system with multiple processors. I would suggest you start on a uniprocessor system, disable other processors from within BIOS, or at least manually configure your guest OS not to enable the other processors (e.g., for Linux, include 'nosmp'
on the kernel command line -- e.g., in your /boot/grub/menu.lst).
Next, learn about the "e820" map. Again there is plenty of material on the web, but perhaps the best place to start is to boot up a Linux system and look near the top of the output 'dmesg'. This is how the BIOS communicates to the OS which portions of physical memory space are "reserved" for devices or other platform-specific BIOS/firmware uses (e.g., to emulate a PS/2 keyboard on a system with only USB I/O ports).
One way for your hypervisor to "hide" its 100MB from the guest OS is to add an entry to the system's e820 map. A quick and dirty way to get things started is to use the Linux kernel command line option "mem=" or the Windows boot.ini / bcdedit flag "/maxmem".
There are a lot more details and things you are likely to encounter (e.g., x86 processors begin in 16-bit mode when first powered-up), but if you do a little homework on the ones listed here, then hopefully you will be in a better position to ask follow-up questions.