ARM's aarch64 has an AT (Address Translate) instruction that runs a virtual address through a stage of address translation returning a physical address in PAR_EL1, along with status to indicate whether the translation exists. See ARMv8 ARM, Section C5.5.
The question is: does x86_64 have the equivalent? Intel's System Programming Guide (Volume 3, Chapter 5) talks about pointer validation, but these methods seem to apply to segment-level protection, and there do not appear to be any page-level protection pointer validation instructions.
Is anybody aware of an ARMv8-AT-like instruction for x86_64?
No, the x86-64 instruction set does not have an instruction to perform physical-to-virtual address translation. It only has basic instructions like setting the page directory register, invalidating addresses, and enabling paging.
If you want this functionality on x86-64, I'm afraid you need to be in supervisor mode to do so. You would read the CR3 register, possibly change a few page table mappings to access the physical addresses you need, and perform the address translation by manually walking the page directory and tables.
Your question raises a question in response: For what purpose do you need to know about virtual-to-physical address translations? Paging is supposed to be transparent to application programs, and it is rare to have a good reason to know the physical memory address corresponding to a particular virtual memory address.
Related
I've been experimenting with some UEFI/Kernel code and am working on the various PCI-Express elements. I have obtained the MCFG ACPI table, Enumerated all PCI devices into my own structures and have access to all the devices MMIO regions and the full 4kb configuration space.
For this specific PCIe device which I have identified I have followed the configuration space as follows:
Test capability list bit, assuming it is set,
use offset 0x34, follow the pointers until I find a PCI Express configuration capability (ID = 0x10).
From here register 0x0c (Link capabilities) specifies the max link width as x16 and the max link speed as 3 (which is an index into the supported link speed vector and equates to 8.0 GT/s or Gen3 speed which the device is capable of).
The link status register is showing that the negotiated link width is x16, however the link speed is 1 (2.5 GT/s).
What I've tried is using the Link Control 2 Register to set the Target Speed to 3 then set Bit 5 on the Link Control Register to trigger a re-training. I also disable the autonomous link speed to ensure the device remains at the selected speed.
I then wait a small duration (1 second for testing) then poll the Link Status register checking for the Link Training bit to clear. This seems to clear immediately regardless of the above delay and when checking the Link Status register again the link speed is still 1.
I have checked several of the registers for error notifications and haven't spotted anything yet.
Clearly I need to find the correct process to establish a new link speed on the device, possibly configure de-emphasis values or apply the same link speed settings to upstream devices/bridges.
Any advice would be hugely appreciated.
I am reading about logical and physical addressing. I am confused that when a binary file is run, does it pass first through the CPU where the logical address are generated or is it directly copied to the physical memory ?
when a binary file is run, does it pass first through the CPU where the logical address are generated or is it directly copied to the physical memory ?
Typically some code somewhere loads the executable file's headers into memory, and then uses information from the headers to figure out where various pieces of the file (sections - e.g. .text, .data, etc) should be in virtual memory and what each virtual page's virtual permissions should be (if writes are allowed, if execution is allowed).
After this, areas of the virtual address space are set up. Often this is done by memory mapping the relevant part of the file into the virtual address space, without actually loading them into physical memory. In this case each page's actual permissions don't reflect the page's virtual permissions (e.g. a "read/write" page might be "not present" initially, and when software tries to read from the page you'll get a page fault and the page fault handler might fetch the page from disk and change the page to "present, read only"; and then later when software tries to write to the page you might get a second page fault and the page fault handler might do a "copy on write" so that anything else using the same physical page isn't effected and then make the new copy "read/write" so that it matches the original virtual permissions).
While this is happening; the OS could (depending on amount of free physical RAM and whether storage devices have more important data to transfer) be prefetching the file data from disk (e.g. into VFS cache), and could be "opportunistically" updating the process' page tables to avoid the overhead of page faults for pages that have been prefetched.
However; if the OS knows that the file was on unreliable and/or removable media it may decide that using memory mapped files is a bad idea and may actually load the needed executable's sections into memory before executing it; and an OS could have other features that cause the file to be loaded into RAM before it's executed (e.g. if the OS checks that an executable file's digital signature is correct before allowing the file to be executed, then the entire file probably needs to be loaded into memory to allow the digital signature can be checked, and in that case the entire file is likely to still be in memory when virtual address space is set up).
You need to read an entire book on these topics and spend several weeks on that
But Operating Systems: Three Easy Pieces is a good book, and it is freely downloadable.
Once you have read it, look perhaps also into osdev.org for practical things. And don't forget free software OSes such as Linux, e.g.
https://kernelnewbies.org/
Be aware of copy-on-write and virtual address space...
An executable files is generally an interpreted program itself that is executed by the loader. The executable contains instructions that tell the loader how the program should exist in VIRTUAL memory. By that, I mean the instructions in the executable define how the initial VIRTUAL representation of process address space.
So when the executable starts, there is only a virtual representation of the address space in secondary storage. As the program executes, it starts page faulting repeatedly to load pages into memory. After the initial load, the page fault rate dies down.
The executable NORMALLY only contains logical addresses.
I can see from my OS the informations about my hard disk, RAM and CPU. But I've never told my OS these info.
How does my OS know it?
Is there some place in the hard disk or CPU or RAM that stores this kind of information?
Is there some standard about the format of this kind of information?
SMBIOS (formerly known as DMI) contains much of this information. SMBIOS is a a data structure/API that is part of the BIOS/UEFI firmware and contains info like brand and model of the computer, etc.
The rest is gathered by the OS querying hardware directly.
Answer grabbed from superuser by Mokubai.
You don't need to tell it because each device already knows (or has a way) to identify itself.
If you get the idea that every device is accessed via address and data lines, and in some cases only data lines then you come to the relaisation that in those data lines you need some kind of "protocol" that determines just how you talk to those devices.
In amongst that protocol you have commands that say "read this" and "send that" or "put this over there". It is also relatively easy to have a command that says "identify yourself" which, rather than reading a block of disk or memory or painting a pixel a particular colour, will return a premade string or set of strings that tell the driver or operating system what that device is. Using a series of identity commands you could discover a device type, it's capabilities and what driver might be able to work with it.
You don't need to tell a device what it is, because it already knows. And you don't need to tell the operating system what it is because it can ask the device itself.
You don't tell people what they're called and how they talk, you ask them.
Each device has it's own protocol for these messages, and they don't store the details of other devices because to do so would be insane and near useless given that you can remove any device at any time. Your hard drive doesn't need to store information about your memory or graphics card except for the driver that the operating system uses to talk to it with.
The PC UEFI specification would define a core set of system specifications that every computer has, allowing the processor to be powered up and for a program stored in an EEPROM to begin the asbolute basic system probing necessary to determine the processor, set up the RAM, find a disk and display and thus continue to boot the computer.
From there the UEFI system would hand over to the operating system which would have more detailed probing and identification procedures, but it all starts at the most basic "I have a processor, what is around me?" situation.
I am currently learning about Virtual Memory within the OS. I recently learned that access rights are stored in the page tables and so I am wondering if you can modify your own page tables? Does the hardware enforce protection from this?
Yes, you can modify your page tables—to some degree. Most operating system have system services to allow you to map and unmap pages to your address space (thus modifying your page tables).
Because page tables are stored in the system address space invariably with access limited to kernel mode, you have to modify the page tables in kernel mode. That means doing it through a system service that executes in kernel mode.
Of course you are limited to the types of modification you can make by the system services.
No, you (as user code) cannot directly modify the page tables for your processs, or any other process.
The page tables are manages exclusively by the kernel. They are stored in physical memory which is not mapped into userspace.
The hardware (specifically the MMU) enforces this protection just as it protects all of the kernel data and code.
There are two case when working on an embedded system.
Embedded system have limited resources like as ARM Cortex M0 Microcontroller with 12 K Flash.
Case 1 :
Common function/module usage for Bootloader and Firmware :
Bootloader and Firmware may need to use same module and function to prevent code duplication. Otherwise, same code will be included both Firmware and Bootloader twice.
We can prevent this by specify the function address and call this function by calling functions by addresses. This is one of the solutions.
Is there any smart method to provide common function usage?
Case 2 :
Sometimes, we need to upgrade firmware. One of the duties of bootloader is firmware upgrades. We can easily upgrade the firmware by overwrite the old one.
As we saw, two case can be implemented separately. But when we merge they, some problems are appeared.
Question :
Bootloader's are generally static objects but firmware's are can be modified. Therefore, common functions are generally located at Bootloaders. But when we need update a common module/function, how can we do?
What are the general or smart approaches which bootloader, firmware structured embedded systems? In Addition, for limited resources.
To discrete common modules/functions, Can one or more additional areas solve this problems.
Firmware, Bootloader and Library(New Area)?
I want to learn general approaches. Is there any paper, book and source about advanced Firmware management?
Thanks
If you share code between your bootloader and your mainline firmware application, then your bootloader will be using this code when it flashes the application space. In order to prevent this condition you must sacrifice the ability to update the common code, otherwise your bootloader will crash.
With only 12k of flash, it's pretty ambitious to expect a bootloader and mainline application to fit. You might consider writing the bootloader in assembly (gasp!). Some Cortex M0 parts (such as the NXP LPC11xx family) have an additional boot ROM which stores some useful functions and help alleviate some of the memory constraints.
Your question states the problem correctly - you cannot have your cake and eat it. Either:
1. You go for a small memory footprint and do not include firmware upgrade logic in the bootloader (i.e bootloader might just validate application image CRC etc but nothing more complicated). Here you could share functions to save space. OR
2. The bootloader has firmware upgrade functionality. Here you have to have shared functions compiled both into app and bootloader. The shared functions should be small - probably not a huge overhead but you need the space that this would take - if you dont have it then you need to go for more memory.
There is no way to share functionality and do firmware upgrade from bootloader reliably.
In lights of the current discussion about security in the firmware update process I would like to add the following for clarification:
Sharing code between the bootloader and the app will open yet another door for the potential attack, so you really want to avoid that.
The bootloader part is the part you actually do not want to change ever, this should be as static as possible. If the bootloader is broken, in-the-field-updates become nearly impossible or at least insecure.
Having said that, you might want to use a different approach.
You could create a maintenance mode for your device.
This mode opens the JTAG interface and allows direct access to memory. Than the service technician could apply the update.
Now you "only" need to secure the activation of the maintenance mode.
The following could work:
Use a UART interface to communicate the activation.
Maintenance system sends its own id and requests maintenance mode via UART
The id of the maintenance system, a random number and a unique system id are sent back to the maintenance system.
The maintenance system sends this id-sequence to your certification server.
If the unique system id and the maintenance systems id is correct, the server will create a signature of the information received and send it back to the maintenance system.
Your system now will receive the signature via UART
Your system verifies the signature against the previously send id-string with a public key stored during production
On a successfull verification maintenance mode is entered
To add security, you definitely want to put some effort into the maintenance systems id following a similar scheme. The ID should basically depend on MAC-address or another unique hardware id and a signature of the same. The ID should be created in a secure environment during production process of the maintenance system. The unique hardware id should be something visible to the outside world, so the server could actually verify, whether the ID received matches with the maintenance system communicating with the server.
This whole setup would give you a secure firmware update without a bootloader.
To have secure firmware updates, common understanding is, that you need a authentication system based on asymmetric encryption like RSA. If you need the verification code anyway, the above will exchange a bootloader capable of accepting updates with a simple UART interface, saving some resources in the process.
Is this something you were looking for?
A commercial bootloader in my experience uses between 4 and 8k of flash memory depending on flash algorithm and a couple of other things. I have been sticking with the same vendor throughout my carreer, so this might vary from your experience.
A digital signature system optimized for embedded systems uses approximately 4.5kByte in flash memory (for an example, see here: https://www.segger.com/emlib-emsecure.html ) and no more RAM than the stack.
You see, that 12k is really really low in terms of having a system which can be updated securely in the field. And even more so, if you want the system to be updated using a bootloader.