I am creating a kernel for x64 which is booting with UEFI. While the kernel has to be loaded at a low-ish address (I believe, because UEFI requires identity mapped pages so it cannot be mapped higher than the highest physical address), I want to relocate up to the end of memory. During this process I intend on creating new paging structures and in order to reduce memory consumption, I wanted to reuse the page tables used to map the image in the lower half. However, these page tables will only exist if 4k paging is used by UEFI, so my question is whether or not UEFI is required to use 4k paging on x64. I believe the answer is no, but I hope otherwise and wanted to see if this is true.
Now I understand UEFI allocates memory via BootServices->AllocatePage in 4k chunks it refers to as pages, but is this required to translate to the actual mapping structure used? I noticed that in section 2.3.6 of the UEFI 2.8 specification, the section referring to AArch64 calling conventions, it states
MMU configuration: Implementations must use only 4k pages [...]
There is no similar denotation in section 2.3.4, on the x64 calling conventions, which is why I believe the answer is no.
EDIT:
Based upon what I've already seen and the comment by Peter Cordes, I believe the standard does not specify exactly what it should be. Thus a revised version of the question is: Does the standard specify 4k translation granularity? If not, do most UEFI vendors on x64 use 4k pages?
Related
I am trying to write a program where I scan a processes memory and can also write to these addresses(just like cheat engine). However I did some research and found out that the memory I was reading is virtual memory I can read this memory but I can't write to it and to translate it I need page tables. So my question is where can I find these page tables and is there any other way to write using the virtual address I get?
Virtual memory is an elaborate illusion. What you think is read/write RAM may actually be data in swap space, or "ready only, copy on write", or something else.
To maintain the illusion, and for security, and for compatibility (e.g. 32-bit program running on a 64-bit CPU with a 64-bit kernel); user-space is not given access to page tables.
An OS or kernel might provide an abstract interface to some of the information (with suitable restrictions and limitations for security). One example of this would be the VirtualQuery() and VirtualQueryEx() functions in Windows (see https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualqueryex ).
In a similar way, an OS or kernel might provide an abstract interface to alter a page's permissions (with suitable restrictions and limitations for security). One example of this would be the VirtualProtect() function in Windows (see https://learn.microsoft.com/en-us/windows/win32/api/memoryapi/nf-memoryapi-virtualprotect ).
... and is there any other way to write using the virtual address I get?
If your CPU is an 80x86 CPU that supports Intel's transactional extensions; you can misuse "transactions" to suppress page faults (make them cause a "transaction abort" instead of triggering a page fault).
This won't allow you to write to a read-only or "not present" page; but will allow you to attempt to write without being detected by the OS.
The user's program in main memory consists of machine instructions and
data. In contrast, the control memory holds a fixed microprogram that
cannot be altered by the occasional user. The microprogram consists of
microinstructions that specify various internal control signals for
execution of register microoperations. Each machine instruction
initiates a series of micro instructions in control memory. These
microsinstructions generates microoperations to fetch the instruction
for main memory; to evaluate the effective address, to execute the
operation specified by the instruction, and to return control the
fetch phase in order to repeat the cycle for the next instruction
I don't exactly understand here the difference between machine instruction, microinstruction and micropeerations. i certainly do understand that microinstructions according to the paragraph given are the intermediate level of instructions but which of the other 2 is the one that is more close to the machine language. Are CLA, ADD, STA, BUN, BSA, AND etc machine instructions or microoperations?
A CPU presents itself to the outside as a device capable of executing machine instructions. For example,
mov (%esi,%ebx,4), %edx
is a machine instruction that moves 4 bytes of data at address ESI+4*EBX into register EDX. Machine instructions are public - they are published by CPU manufacturer in a user manual. Compilers such as gcc will output files that contain machine instructions, and these will typically end up in EXE/DLL files.
If you look closely at the above instruction, you will see that it is a fairly complex operation. It involves some arithmetic (multiplying and addition) to get the memory address, then moving data from that address into a register. From CPU's perspective, it would also make sense to use the arithmetical unit that is already there. So it makes natural sense to break down this instruction into microinstructions. In essence, mov instruction is implemented internally by CPU as a microprogram written in microinstructions. This is, however, an implementation detail of a CPU. Microinstructions are internal to CPU and they are invisible to anybody except to CPU manufacturer.
Microinstructions have several benefits:
they simplify internal CPU architecture, design and testing, thus lowering cost per unit
they make it easy to create rich and powerful sets of machine instructions (you just have to combine microinstrcutions in different ways)
they provide a consistent machine language across different CPUs (e.g. Xeon and Pentium both implement basic x86_64 instruction set even though they are very different in hardware)
create optimizations (i.e. the same instruction on one CPU can be implemented by a hardware, the other can be emulated in microinstructions)
fix bugs (e.g. you can fix Spectre vulnerability while the machine is running and without buying a new CPU and opening your server)
For more information, see https://en.wikipedia.org/wiki/Micro-operation
I think the answer to your question is in these three sentences:
The user's program in main memory consists of machine instructions and data
Each machine instruction initiates a series of micro-instructions in control memory.
These micro-instructions generate micro-operations.
So:
The user supplies machine instructions
Those get translated into micro-instructions
Those get translated into micro-operations
The mnemonics you mentioned are what the user might use to write or read a list of machine instructions (the actual instructions just being patterns of bits understood by the processor). The "occasional user" (i.e. everyone other than the chip's designer) never needs to deal directly in micro-instructions or micro-operations, so would never know individual names for them.
I have located RSDP by searching the valid string.
I know the table contains a pointer direct me to XSDT.
However, I compared the "Length" and the entries in XSDT, only FACP, APIC, SSDT tables are located.
Does this mean that the system does not contain MCFG table?
Therefore I can not using memory map to access PCIe configuration space?
Thanks a lot!
MCFG is not a part of ACPI specification. The table is described in the PCI Firmware specification document.
Section 2.7.2 states
The ACPI MCFG table describes the location of the PCI Express configuration space, and this table will be present in a firmware implementation compliant to this specification version 3.0 (or later).
So, it means your firmware is not compliant to the PCI firmware specification v3.0 or later.
Reading further the section 4.1 about ECAM
On PC-compatible systems, the enhanced configuration access mechanism allows PCI configuration space to be accessed using memory primitives rather than I/O-based primitives (CF8/CFC mechanism).
So, it means in your case, if you are talking about PC-compatible systems, that only type 1 (CF8/CFC) access to the PCI configuration space is available (you may not reach space beyond 256 bytes).
Of course it may be just a bug in firmware that for some reason forgot to describe it. On x86 you may try to access traditional ECAM window (starting from 0xE0000000) and check if it works (be sure the memory region is marked as reserved in the OS).
In this article we can see that 16 Bits systems have different memory models.
Through that answer we know that COM application always uses the Tiny Model (all segments are in the same one) but for the other executables what make the operating system uses one model or another?
I did not see in the MS-DOS Header any flag that would help for a choice so how does MS-DOS determines what memory model to use?
The selection of memory model is necessary as compiler option, not OS related. You can assume that DOS always works with Large memory model (far pointers for CS and DS).
I have learned that in an operating system (Linux), the memory management unit (MMU) can translate a virtual address (VA) to a physical address (PA) via the page table data structure. It seems that page is the smallest data unit that is managed by the VM. But how about the block? Is it also the smallest data unit transfered between the disk and the system memory?
What is the difference between pages and blocks?
A block is the smallest unit of data that an operating system can either write to a file or read from a file.
What exactly is a page?
Pages are used by some operating systems instead of blocks. A page is basically a virtual block. And, pages have a fixed size – 4K and 2K are the most commonly used sizes. So, the two key points to remember about pages is that they are virtual blocks and they have fixed sizes.
Why pages may be used instead of blocks
Pages are used because they make processing easier when there are many storage devices, because each device may support a different block size. With pages the operating system can deal with just a fixed size page, rather than try to figure out how to deal with blocks that are all different sizes. So, pages act as sort of a middleman between operating systems and hardware drivers, which translate the pages to the appropriate blocks. But, both pages and blocks are used as a unit of data storage.
http://www.programmerinterview.com/index.php/database-sql/page-versus-block/
Generally speaking, the hard-disk is one of those devices called "block-devices" as opposed to "character-devices" because the unit of transferring data is in the block.
Even if you want only a single character from a file, the OS and the drive will get you a block and then give you access only to what you asked for while the rest remains in a specific cache/buffer.
Note: The block size, however, can differ from one system to another.
To clear a point:
Yes, any data transferred between the hard disk and the RAM is usually sent in blocks rather than actual bytes.
Data which is stored in RAM is managed, typically, by pages yes; of course the assembly instructions only know byte addresses.