In Operating System Design the kernel is most always mapped to a high virtual memory address, thus gaining control of the upper memory part. The space left below is for applications running in user space, as described in an excellent way in "Linux 3/1 virtual address split".
What I'd like to know is, why is this design decision made or why the kernel does not use the lower part of memory? This is not really clear to me, or maybe I've overseen something.
Edit: This question regards virtual addresses and not physical.
Some advantages of/reasons for such a design:
Applications don't need to care about the size and location of the kernel and may pretend they're the only ones in the memory, starting at around 0 and spanning upwards, with minimal or no code and data relocations. Applications are hence easier to design and implement and they may be less likely to have bugs related to mememory management.
Applications may use smaller/shorter addresses/pointers and hence save some memory.
In the x86 CPU, 16-bit and 32-bit address spaces start at virtual address 0 and end at around 1MB (for real and virtual 8086 modes), 16 MB (16-bit protected mode on i80286+) and 4 GB (32-bit mode, unreal mode). Placing the kernel at lower addresses will reduce the range of addresses available to applications (ex: 16-bit app in 32-bit mode or 32-bit app in 64-bit mode) and/or complicate their memory management. Moving the kernel to the top of the virtual address space generally makes sense on the x86.
There may be other reasons, usually platform-specific. On some platforms there may be little to no difference between the two options. Yet on others the preferred kernel location may be at the lower virtual addresses. Details matter.
Related
Virtual address is described as linear address in some places, and logical address in others.
I'd like to know which one is right with the clear concept of virtual address.
The concept of virtual addresses is that you have a fake/pretend address space and convert/map that somehow to the real/physical address space for one or more reasons (to improve flexibility, to improve portability, to improve security, etc). How this is implemented in practice doesn't really effect the theoretical concept.
For the implementation of the concept on 80x86; virtual addresses are converted into linear addresses using segmentation, then linear addresses are converted into physical addresses using paging. However; segmentation can be configured so that "virtual = linear" (by setting segment bases to zero and segment limits to max., including in 64-bit code if FS and GS are configured so that they do nothing); and paging can be disabled resulting in "linear = physical"; and if neither segmentation nor paging are used you end up with "virtual = linear = physical".
Most operating systems for 80x86 don't use segmentation but do use paging; so virtual addresses can be described as linear addresses for most operating systems (and most applications) on 80x86; but "technically can" isn't a good reason for increasing confusion and almost nobody would call them linear addresses (instead of virtual addresses) without a reason - normally you'd only see the word "linear" used if the difference might matter.
For logical addresses, I have no idea where you saw that, and without context I'd (correctly or incorrectly) assume it's related to storage space and has nothing to do with memory (e.g. "logical block address" as an alternative to "cylinder, head, sector addressing" for old hard disks).
The three basic concept you need to know:
Physical - An actual, specific device
Logical - A redirection to a device
Virtual - A simulated device
In ye olde days before large memory system, virtual and logical were often conflated in regard to addresses. In reality, there is no such thing as a virtual address. A logical address can map to a nothing at all, a physical address, or memory that is simulated virtually.
You can have virtual memory that is accessed by logical addresses.
Why is there a portion of virtual memory reserved for OS? Why is it limited to a certain size? This seems to be a universally known fact because when I googled I didn't find anyone asking similar questions.
If the OS segment (the part in VM reserved for OS) is accessed, what happens?
How does the OS segment affect the translation between virtual and physical memory?
For example if your virtual memory is 128KB, the first 32KB is allocated for seg 0 and the last 32KB for seg 1. Then you reserve the first 16KB for the OS seg. What happens to seg 0? Does its size shrink to 16KB because 16KB has been changed to OS seg? Or does it stays the same?
Why is there a portion of virtual memory reserved for OS? Why is it limited to a certain size? This seems to be a universally known fact because when I googled I didn't find anyone asking similar questions.
The reason some area of the logical address space is reserved for the OS is because the same physical memory is shared by all processes and it needs to be at the same location.
When an interrupt occurs, any process can be running. So the kernel mode handler needs to be in the same location.
Usually the reserved OS area is so large that the actual OS will never come close to using it all. So it is not really limited in size.
If the OS segment is accessed, what happens?
That depends upon how it is accessed. If a process accesses it in kernel mode (system call, interrupt, exception), that is normal. If it accesses the reserved area in user mode, it usually triggers an access violation of some kind. Some systems may make some areas of system memory readable from user mode but usually is all write protected.
How does the OS segment affect the translation between virtual and physical memory?
This is system dependent. Some systems make the user page tables pageable. The user page tables can then be in pageable areas in the system address space. In other words, the page tables are in virtual/logical memory, giving an additional translation for user addresses that does not occur for system addresses
Doing the same for the system address space would cause a chicken and egg problem. In such a system, the system page tables would be in physical locations (another reason everyone uses the same address range for system space).
Other systems use physical addresses for all page tables. In case, they translation is the same.
For example if your virtual memory is 128KB, the first 32KB is allocated for seg 0 and the last 32KB for seg 1. Then you reserve the first 16KB for the OS seg. What happens to seg 0? Does its size shrink to 16KB because 16KB has been changed to OS seg? Or does it stays the same?
This is not a good example. Virtual memory is never this small. Imagine a 32-bit system. The virtual address space is 4GB. The system assigns the first 3 GB to user the user space and the last 1 GB to the system space.
All processes share the same 1GB system space. They have there own, unique 3 GB user space.
I'm learning that segmentation in operating systems is based on dividing different segments (for a program, these could represent a symbol table, the source text, the stack...) into units that start at logical memory address 0. This is the virtual address that the MMU (?) uses to get the real in addition to the offset.
An apparent benefit of segmentation is that, since each segment starts at address 0, multiple processes can take advantage of a single segment simultaneously (an example is the shared library).
However, I don't see how else segmentation can benefit programmers. What would be some examples?
Thanks!
Segmentation provides NO benefit to programmers. Segmentation is a kludge that developed to overcome architectural limits. The 16-bit PDP-11 computers could only address 64K of memory. The use of a segmentation allowed the programmer to map memory in and out of the address space to access more memory.
The 8086 chip was retrograde. IBM set the computer industry back by years using it for the PC rather than 68000. The 8086 used segments to reduce the size of instructions. Rather than using 32-bits for an address, instructions could use am offset from a segment register.
In 64-bit mode, the abomination of segments in the Intel processors finally goes away.
Does 32 bit mean ram size should be 4GB ? or can a computer with say 32GB ram also have 32 bit provided adress space does not exceed 32 bit ?
When we say 32-bit windows or 64-bit OS, which part of OS exactly differs between the two ? I mean does some part of kernel differ ? if yes then which part ?
NOTE: this question is not a duplicate. please dont vote to close
No 32-bit does not necessarily refer to the size of the address bus. If the address bus is 32-bit then certainly the maximum RAM in the system is 4 gb, or 2^32. There have been several examples of 32-bit machines that could exceed 4gb of RAM, however, by using a concept of Page-Extended Addressing (PAE) That was introduces in the mid 1990s.
Another examples where this comes into play is the first IBM PC. It used a 16-bit microprocessor known as the 8088. The 8088 had a 20-bit address line and as such had the capacity of 2^20 (1MB) of RAM.
When we speak of a microprocessor having a certain number of 'bits', such as a 16-bit microprocessor or a 32-bit microprocessor, we are primarily referring to the basic data unit that the processor can handle at a time. This is determined by the size of the processor registers, which are the areas of the processor used for holding data for calculations and decisions.
Because there is a fundamental difference in how machine code is used to grab and process data in a 32-bit vs a 64-bit system, All code must be compiled specifically for the machine you want it to run on. This is why there are two version of many x86 operating systems. There is often one for 32-bit and one for 64-bit x86. x86 microprocessors have a legacy of backwards compatibility and are therefore able to run in 16, 32, or 64-bit modes. This means that you can run 32-bit windows on a 64-bit processor. If this backwards compatibility wasn't build in, however, this would not be possible.
So, as far as which part of the kernel differs, the answer is all of it. The same is true for desktop applications that are coded for 64-bit machines. If they have two versions, the entire code is different as the compiler optimizes for one or the other.
I'm reading "Understanding Linux Kernel". This is the snippet that explains how Linux uses Segmentation which I didn't understand.
Segmentation has been included in 80 x
86 microprocessors to encourage
programmers to split their
applications into logically related
entities, such as subroutines or
global and local data areas. However,
Linux uses segmentation in a very
limited way. In fact, segmentation
and paging are somewhat redundant,
because both can be used to separate
the physical address spaces of
processes: segmentation can assign a
different linear address space to each
process, while paging can map the same
linear address space into different
physical address spaces. Linux prefers
paging to segmentation for the
following reasons:
Memory management is simpler when all
processes use the same segment
register values that is, when they
share the same set of linear
addresses.
One of the design objectives of Linux
is portability to a wide range of
architectures; RISC architectures in
particular have limited support for
segmentation.
All Linux processes running in User
Mode use the same pair of segments to
address instructions and data. These
segments are called user code segment
and user data segment , respectively.
Similarly, all Linux processes running
in Kernel Mode use the same pair of
segments to address instructions and
data: they are called kernel code
segment and kernel data segment ,
respectively. Table 2-3 shows the
values of the Segment Descriptor
fields for these four crucial
segments.
I'm unable to understand 1st and last paragraph.
The 80x86 family of CPUs generate a real address by adding the contents of a CPU register called a segment register to that of the program counter. Thus by changing the segment register contents you can change the physical addresses that the program accesses. Paging does something similar by mapping the same virtual address to different real addresses. Linux using uses the latter - the segment registers for Linux processes will always have the same unchanging contents.
Segmentation and Paging are not at all redundant. The Linux OS fully incorporates demand paging, but it does not use memory segmentation. This gives all tasks a flat, linear, virtual address space of 32/64 bits.
Paging adds on another layer of abstraction to the memory address translation. With paging, linear memory addresses are mapped to pages of memory, instead of being translated directly to physical memory. Since pages can be swapped in and out of physical RAM, paging allows more memory to be allocated than what is physically available. Only pages that are being actively used need to be mapped into physical memory.
An alternative to page swapping is segment swapping, but it is generally much less efficient given that segments are usually larger than pages.
Segmentation of memory is a method of allocating multiple chunks of memory (per task) for different purposes and allowing those chunks to be protected from each other. In Linux a task's code, data, and stack sections are all mapped to a single segment of memory.
The 32-bit processors do not have a mode bit for disabling
segmentation, but the same effect can be achieved by mapping the
stack, code, and data spaces to the same range of linear addresses.
The 32-bit offsets used by 32-bit processor instructions can cover a
four-gigabyte linear address space.
Aditionally, the Intel documentation states:
A flat model without paging minimally requires a GDT with one code and
one data segment descriptor. A null descriptor in the first GDT entry
is also required. A flat model with paging may provide code and data
descriptors for supervisor mode and another set of code and data
descriptors for user mode
This is the reason for having a one pair of CS/DS for kernel privilege execution (ring 0), and one pair of CS/DS for user privilege execution (ring 3).
Summary: Segmentation provides a means to isolate and protect sections of memory. Paging provides a means to allocate more memory that what is physically available.
Windows uses the fs segment for local thread storage.
Therefore, wine has to use it, and the linux kernel needs to support it.
Modern operating systems (i.e. Linux, other Unixen, Windows NT, etc.) do not use the segmentation facility provided by the x86 processor. Instead, they use a flat 32 bit memory model. Each user mode process has it's own 32 bit virtual address space.
(Naturally the widths are expanded to 64 bits on x86_64 systems)
Intel first added segmentation on the 80286, and then paging on the 80386. Unix-like OSes typically use paging for virtual memory.
Anyway, since paging on x86 didn't support execute permissions until recently, OpenWall Linux used segmentation to provide non-executable stack regions, i.e. it set the code segment limit to a lower value than the other segment's limits, and did some emulation to support trampolines on the stack.