I'm working on a minimal hobby operating-system. I want to write a driver for xHCI in order to support USB keyboards and mouses. I'm using QEMU for virtualization. I'm passing the -device qemu-xhci parameter to qemu so that it emulates the xHCI. I was wondering if the modern operating-systems use the MCFG ACPI table to find the configuration space of PCI (and the registers of the xHCI)? I was also wondering how to enable PCI-Express in QEMU so that I can find a MCFG table in RAM so that I can support the latest technology in my hobby OS.
To find all xHCI controllers you search PCI configuration space for devices ("functions") with matching "class/subclass/progID" values (see note 2); which means that you have to find a way to access PCI configuration space first.
On 80x86; there are 3 possible ways to access PCI configuration space - 2 that use IO ports ("mechanism #1" and the deprecated "mechanism #2"), and one that maps PCI configuration space into the physical address space (called "Enhanced Configuration Access Mechanism").
If the Enhanced Configuration Access Mechanism is supported; the MCFG ACPI table describes how PCI configuration space is mapped into the physical address space. Primarily; PCI buses are described as "groups of buses", where each group (defined by a "starting bus number" and "total buses in this group" pair) has a base physical address, and the correct physical address for a PCI function is determined by finding information for the relevant group of buses for the requested bus number, then doing a calculation like:
physical_address = base_physical_address_for_group +
(bus_number - starting_bus_number_for_group) << 20 +
device_number << 15 +
function_number << 12 +
offset;
Note 1: because most operating systems use virtual memory it's possible for an OS to create a nice "virtually linear" mapping of the ("possibly physically disjoint") physical memory areas described by MCFG ACPI table (while using the same page full of zeros mapped as read-only to fill any gaps in the "virtually linear mapping"); so that the OS can use a simplified approach (with no need to find information for the relevant group of buses) like:
virtual_address = PCI_config_space_base_virtual_address +
bus_number << 20 +
device_number << 15 +
function_number << 12 +
offset;
Note 2: An OS doesn't/shouldn't literally search PCI configuration space each time it wants to start a device driver for one specific type of device. Instead an OS typically enumerates PCI buses once during boot (and possibly after boot in response to a notification if "hot-plug PCI" is supported) and starts device drivers based on the results of that enumeration. In other words, it's more like "I found an xHCI controller and need to start the appropriate driver" and not like "I want to start an xHCI driver and need to find the appropriate device/s".
Related
I am writing an operating system for the raspberry pi.
I have a problem with the sdcard driver for the custom sdhost controller (emmc2)
of the raspberry-pi 4 (Cortex-A72, ARMv8-A, bcm2711 chipset).
Without using sdma everything works. With sdma, read works, but after writing sectors, the data on the sdcard sometimes contains invalid data.
For the sdma data transfer, I use a transfer buffer with a device type memory attribute (nGnRnE). When I use a fresh data buffer for the dma write transfer, the data on the sdcard is correct. But when I reuse the same buffer for the next write, then the sector on the sdcard partially contains data from the previous buffer content.
Maybe this is a cache coherency problem. I have enabled all caches (I and D). In the ARM manuals there is a talk of the SCU (snoop control unit) and I don't know whether I have do bother about the SCU.
My questions are:
Is it necessary to enable the SCU on a Cortex-A72 and how can this be done ?
What other things have to be kept in mind when using dma for device access ?
I found the solution for my problem:
On the raspberry pi 4 (bcm2711 chip), the physical addresses that are written into the registers of a dma engine must be legacy master addresses. The legacy master addresses are in the range 0xC0000000-0xFFFFFFFF. So I have to add 0xC0000000 to the values of the physical addresses that are written into the registers of the sdhci controller.
The documentation can be found here:
https://datasheets.raspberrypi.org/bcm2711/bcm2711-peripherals.pdf
1.2. Address map
1.2.4. Legacy master addresses
The answer to the other SCU question is: it is not necessary to enable the SCU on the Raspberry Pi 4 when the caches are enabled.
Hi I'm trying to access the PCIe configuration space with MMIO method on a kernel base.
Before I drop my question, my platform is Windows 10, VirtualBox 6.0.10.
My virtual machine set as default except the following:
chipset choosed ICH9
Core number set to 4
Memory set to 1GB
Added IDE controller(no HD connected)
After boot, the printing shows that valid memory address are:0x0~0x9FC00 and 0x100000~0x3FEF0000 as displayed in following screen shot.
While type 1 is RAM, 2 is ROM or Reserved, 3 is ACPI Reclaim Memory and 4 is ACPI NVS Memory.
Furthermore I retrieved base address of the PCIe configuration memory map base address from MCFG as showed in the following screen shot.
It can be seen that:
The configuration space base is 0x3F000000 which overlap with the valid memory space.
The first 8 byte of 0x3F000000~0x3F000008 is all 0, which should be first 8 byte of bus:0, device:0, function:0.
So whether I should not use VirtualBox, or I should do some other operations to enable PCIe MMIO accessibility of configuration space?
Thanks so much!!
You've probably parsed the "MFCG ACPI table" incorrectly, or used the wrong (virtual?) address for the "MFCG ACPI table" and forgot to check signature and checksum.
The "Base_addr:" doesn't make sense, and the "Start_PCI_bus: 0, End_PCI_bus: 0" doesn't make sense either.
Hi I'm writing a kernel and plan to use MSI interrupt for PCI devices.
However, I'm also quite confused by the documentations.
My understanding about MSI are as follow:
From PCI device point of view:
Documentations indicate that I
need to find Capabillty ID = 0x05 to locate 3 registers: Message control (MCR), Message Address (MAR) and Message Data (MDR) registers
MCR provide control functionality for MSI interrupt,
MAR provide the physical address the PCI device
will write once interrupt occurs
MDR forms out the actual data it will write into the physical address
From CPU point of view:
Documentation shows that Message Address register contains fixed top of 0xFEE, and following by destination ID (LAPIC ID) and other controlling bits as follow:
The Message Data register will contain the following information, including the interrupt vector:
After reading all of these, I am thinking if the APIC_ID is 0x0h would the Message Address conflict with the Local APIC memory mapping? Although the address of FEE00000~FEE00010 are reserved.
In addition, is it true that the vector number in MDR is corresponding to the IDT vector number. In other words, if I put MAR = 0xFEE0000C (Destination ID = 0, Using logical APIC ID) and MDR = 0x0032 (edge trigger, Vector = 50) and enable the MSI interrupt, then once the device issues an interrupt CPU would correspondingly run the function pointed by IDT[50]? After that I write 0h to EOI register to end it?
Finally, according to the documentation, the upper 32 bit of MAR is not used? Can anyone help on this?
Thanks a lot!
Your understanding of how to detect and program MSI in a PCI (or PCIe) device is correct.*
The message address controls the destination (which CPU the interrupt is sent to), while the message data contains the vector number. For normal interrupts, all bits of the message data should be 0 except for the low 8 bits, which contain the vector.
The vector is an index into the IDT, so if the message data is 0x0032, the interrupt is delivered through entry 50 of the IDT.**
If the Destination ID in an interrupt message is 0, the Message Address of the MSI does match the default address of the local APIC, but they do not conflict, because the APIC can only be written by the CPU and MSIs can only be written by devices.
On x86 platforms, the upper 32 bits of the message address must be 0. This can be done by setting the upper part of the message address to 0 or by programming the device to use a 32-bit message address (in which case the upper message address register is not used). The PCI spec was designed to work with systems where 64-bit MSI addresses are used, but x86 systems never use the upper 32 bits of the message address.
Reprogramming the APIC base address by writing to the APIC_BASE MSR does not affect the address range used for MSI; it is always 0xFEExxxxx.
* You should also look at the MSI-X capability, because some devices support MSI-X but not MSI. MSI-X is a bit more flexible, which inevitably makes it a bit more complicated.
** When using the MSI capability, the message data isn't exactly the value in the Message Data Register (MDR). The MSI capability allows the device to use several contiguous vectors. When the device sends an interrupt message, it replaces the low bits of the MDR with a different value depending on the interrupt cause within the device.
I'm struggling to understand where to get the address of a device on a device tree? As an example how do I know that I should set <0x00900000 0x20000> in here.
Is memory mapped IO done in the hardware (the processor itself) or in software and do I just have to pass the right address in the device tree?
Is the address hardcoded on the processor or can I just set an arbitrary address? I cannot find anything in my reference manual about setting a certain address in the device tree
These kind of addresses can be found in the Reference Manual of the processor.
You can find the link here.
Take a look at the chapter 48 (OCRAM On-chip RAM Memory Controller) and more specifically at the section 48.2.1 (page 4118):
The total on-chip RAM size for the chip is 128 Kbytes, organized as 16K x 64 bits,mapped from 0x00900000 to 0x0091FFFF
This is where come from the values <0x00900000 0x20000> from the dtsi file, corresponding to the base address and the offset.
These values are in dts/dtsi file provided by the chip maker.
Can you access PCI cards(32 bits) in "real mode" ? Isn't "real mode" 16 bits? I have a developer claiming he can only access hardware in Real mode. But PCI is 32bits...
Yes, you can.
IO ports 0xCF8 and 0xCF9 act as index and data registers for accessing PCI Config space. The address to be written to index register (i.e. 0xCF8) has a fixed predefined format (refer PCI spec). To access pci config data, one writes to index register and then reads from data register.
The Index register is a DWORD (32-bit) register and the format is:
Byte-3 = 0x80
Byte-2 = Bus No
Byte-1 = Upper 5 bits as DEVICE no, and lower 3 bits as FUNCTION no.
Byte-0 = Register no. to read from config space
So to read from Bus:0 Device:0 Func:0 register:0 in real mode, you would say:
IoPortWrite32(0xCF8, 0x80000000);
ValueRead = IoPortRead32(0xCFC);
Hope this helps!
Thanks,
Rohit