This is from Documentation/devicetree/bindings/pci/pci-iommu.txt (linux-5.10.0)
PCI root complex
================
Optional properties
-------------------
- iommu-map: Maps a Requester ID to an IOMMU and associated IOMMU specifier
data.
The property is an arbitrary number of tuples of
(rid-base,iommu,iommu-base,length).
Any RID r in the interval [rid-base, rid-base + length) is associated with
the listed IOMMU, with the IOMMU specifier (r - rid-base + iommu-base).
- iommu-map-mask: A mask to be applied to each Requester ID prior to being
mapped to an IOMMU specifier per the iommu-map property.
Example (1)
===========
/ {
#address-cells = <1>;
#size-cells = <1>;
iommu: iommu#a {
reg = <0xa 0x1>;
compatible = "vendor,some-iommu";
#iommu-cells = <1>;
};
pci: pci#f {
reg = <0xf 0x1>;
compatible = "vendor,pcie-root-complex";
device_type = "pci";
/*
* The sideband data provided to the IOMMU is the RID,
* identity-mapped.
*/
iommu-map = <0x0 &iommu 0x0 0x10000>;
};
};
I can't understand the meaning of Any RID r in the interval [rid-base, rid-base + length) is associated with the listed IOMMU, with the IOMMU specifier (r - rid-base + iommu-base). I understand this iommu-map specifies the relationship between the master device (requesting transaction from the PCIe bus to this root complex) and the assigned iommu for the device. This example says, rid 0 ~ 0x10000 is assigned to iommu specifier 0 ~ 0x10000. The document says rid is formed by {bus,device,function} (16 bits).
How is this single number iommu specifier determined by the PCIe core device? Is it device dependent(with the driver's settig)? In arm64 case, the smmu-v3(arm's type of iommu) needs stream ID (and substream ID which is mapped from PASID in PCIe TLP) and when this #iommu-cells is 1, I don't know how this stream ID is passed from the PCIe core device to smmu-v3. If anyone could clarify it, it would be much grateful.
Related
I ask it here to check if my guess is correct.
This page says
Currently we have devm_clk_get() which gives a managed-resource clk
(by name), or of_clk_get() which gives an unmanaged resource clk (by
id).
So I guess clk_get is selecting the clock using 'clock-names' (which is input to a clock consumer device) in the device tree, and of_clk_get is for selecting the clock using 'clock id' (which is the first number in the index-name pair of the clock input specifier.
And I see these lines in a driver drivers/tty/serial/bcm63xx_uart.c (linux 5.15.68).
clk = clk_get(&pdev->dev, "refclk"); // first try
if (IS_ERR(clk) && pdev->dev.of_node)
clk = of_clk_get(pdev->dev.of_node, 0); // second try
It looks like it first try to get the input clock from the name "refclk" and if it fails, it trys to get the clock from the clock speicifer, 0 meaning the first input clock. Is my understanding correct? Are both tries using device tree information?
I see in arch/arm64/boot/dts/broadcom/bcm4908/bcm4908.dtsi,
clocks {
periph_clk: periph_clk {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <50000000>;
clock-output-names = "periph";
};
};
uart0: serial#640 {
compatible = "brcm,bcm6345-uart";
reg = <0x640 0x18>;
interrupts = <GIC_SPI 32 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&periph_clk>;
clock-names = "refclk";
status = "okay";
};
I have set an 8-bit bus on (PD0:PD7) on the stm32 MCU to send addresses to another chip (0:255). I am interested if a function like below would work for fast change in addresses. I could not find an example directly showing register to be equal to integer so I want to confirm it would work. I need a function to which I will give an integer value for the address (0:255) and it will set the 8 pins of the bus with this value:
void chipbus(uint16_t bus8){
GPIOD->regs->BSRR = bus8; // set all the '1' in bus8 to high
GPIOD->regs->BRR = 255-bus8; // 255-bus8 inverts the 8 bits
// BRR to set the new '1' to low
}
If this solution works, I am curious also if I change the bus to ports PD5:PD12 would my function work as:
void chipbus(uint16_t bus8){
GPIOD->regs->BSRR = bus8*32; // set all '1' in bus8 to high
// multiply by 32 to shift 5 bits/pins
GPIOD->regs->BRR = (255-bus8)*32; // 255-bus8 inverts the 8 bits
// BRR to set the new '1' to low
}
Thank you!
Yes, both should work. However, a more recognisable but equivalent formulation would be:
void chipbus(uint16_t bus8) {
GPIOD->regs->BSRR = bus8; // set all the '1' in bus8 to high
GPIOD->regs->BRR = (~bus8) & 0xFF; // inverts the 8 bits // BRR to set the new '1' to low
}
void chipbus(uint16_t bus8) {
GPIOD->regs->BSRR = bus8<<5; // set all '1' in bus8 to high, shift 5 bits
GPIOD->regs->BRR = ((~bus8)&0xFF)<<5; // inverts the 8 bits
}
But, there is an even faster way. BSRR is a 32bit register than can both set and reset. You can combine the two write accesses into one:
void chipbus(uint16_t bus8) {
GPIOD->regs->BSRR =
(bus8<<5) | (((~bus8) & 0xFF) << (16+5));
}
Happy bit-fiddling!
Yes, it'd definitely work. However, as others have noted, it's advisable to set the outputs in a single operation.
Taking advantage of the full 32 bit BSRR register, it can be done without inverting the data bits:
GPIOD->regs->BSRR = bus8 | (0xFF << 16);
or
GPIOD->regs->BSRR = (bus8 << 5) | (0xFF << (16 + 5));
because BSRR has the functionality to set some bits and reset some others in a single write operation, and when both the set and reset bits for a particular pin is set, the output becomes 1.
I wouldn't recommend using this approach. Writing to BSRR and BRR as two separate steps means that the bus will transition through an unintended state where some bits are still set from the previous value.
Instead, consider writing directly to the GPIO output data register (ODR). If you need to preserve the original value of the upper bits in the port, you can do that on the CPU side:
GPIOD->regs->ODR = (GPIOD->regs->ODR & 0xff00) | (bus8 & 0x00ff);
I was wondering how some specific details in the device relate to the hardware and where to find this information(like schematic, datasheet etc).
An example of a usb node is given below:
In the picture above I was wondering how do you find CLK_BUS_OHCI2 or RST_BUS_EHCI2 on the hardware. If you go to the include files you get a value (CLK_BUS_OHCI2 = 39), but I am not sure how that relates to actual hardware. Like which register or which pin etc.
Nowadays I'm working with a similar platform, Allwinner A64 and I'm trying to understand how DTS, clocks, resets work. Fortunately your question came up in the search engine results. Gaurav Pathak's answer clarified most of the things for me and I want to extend a little bit from his point to help those who are trying to fill in the gap about bridging DTS and hardware.
I will refer to Linux kernel 5.7 and Allwinner A64 User Manual (v1.1).
There's a resemblance between uart4 and ehci2 in the means of node definition; for instance, we have property of clocks and resets in both of them. But ccu clock output usage were implemented different.
Let's take a closer look at uart4 node.
uart4: serial#1c29000 {
compatible = "snps,dw-apb-uart";
reg = <0x01c29000 0x400>;
interrupts = <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>;
reg-shift = <2>;
reg-io-width = <4>;
clocks = <&ccu CLK_BUS_UART4>;
resets = <&ccu RST_BUS_UART4>;
status = "disabled";
};
Now we already know that CLK_BUS_UART4 is included from include/dt-bindings/clock/sun50i-a64-ccu.h.
<&ccu CLK_BUS_UART4>
A further search for CLK_BUS_UART4 reveals that include/dt-bindings/clock/sun50i-a64-ccu.h is also indirectly (through drivers/clk/sunxi-ng/ccu-sun50i-a64.h) included from drivers/clk/sunxi-ng/ccu-sun50i-a64.c.
Then it was possible to refer to CLK_BUS_UART4 like below (L807 # ccu-sun50i-a64.c)
[CLK_BUS_UART4] = &bus_uart4_clk.common.hw
But what is bus_uart4_clk? Let's see.
It is defined at L381 # ccu-sun50i-a64.c. This is how it looks:
static SUNXI_CCU_GATE(bus_uart4_clk, "bus-uart4", "apb2",
0x06c, BIT(20), 0);
SUNXI_CCU_GATE is a macro to manage gating register of Clock Control Unit. 4th argument, 0x06c refers to the register offset and 5th argument BIT(20) implies the Nth bit (20, in this case) at the offset of 0x06c is adjusted to drive bus_uart4_clk.
Same thing applies for the resets property. To give a specific example; just search for RST_BUS_UART4 and 0x2d8 at Linux kernel source then look at the bottom of Page 142 of the A64 User Manual.
I hope there were not too much nonsense... Please correct me if I'm wrong.
Well, as far as I know in the below structure
ehci2: usb#01c1c000 {
compatible = "allwinner,sun8i-h3-ehci", "generic-ehci";
reg = <0x01c1c000 0x100>;
interrupts = <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&ccu CLK_BUS_EHCI2>, <&ccu CLK_BUS_OHCI2>;
resets = <&ccu RST_BUS_EHCI2>, <&ccu RST_BUS_OHCI2>;
phys = <&usbphy 2>;
phy-names = "usb";
status = "disabled";
};
clocks = <&ccu CLK_BUS_EHCI2>, <&ccu CLK_BUS_OHCI2>; represents consumer clocks i.e. input clocks and is called "phandle + clock specifier pairs". As you mentioned CLK_BUS_OHCI2 has value 39, that means USB controller will take input clock from the output 39 of ccu clock source.
In the dtsi file from where you posted the above screenshot, there should be a structure that defines the ccu for example like this below:
ccu: clk#01c20060 {
#clock-cells = <1>;
compatible = "allwinner,sun7i-a20-ahb-gates-clk";
reg = <0x01c20060 0x8>;
clocks = <&ahb>;
clock-output-names = "ahb_usb0", "ahb_ehci0",
"ahb_ohci0", "ahb_ehci1", "ahb_ohci1",
"ahb_ss", "ahb_dma", "ahb_bist", "ahb_mmc0",
"ahb_mmc1", "ahb_mmc2", "ahb_mmc3", "ahb_ms",
"ahb_nand", "ahb_sdram", "ahb_ace",
"ahb_emac", "ahb_ts", "ahb_spi0", "ahb_spi1",
"ahb_spi2", "ahb_spi3", "ahb_sata",
"ahb_hstimer", "ahb_ve", "ahb_tvd", "ahb_tve0",
"ahb_tve1", "ahb_lcd0", "ahb_lcd1", "ahb_csi0",
"ahb_csi1", "ahb_hdmi1", "ahb_hdmi0",
"ahb_de_be0", "ahb_de_be1", "ahb_de_fe0",
"ahb_de_fe1", "ahb_gmac", "ahb_ehci2",
"ahb_mali";
};
In the above structure there are multiple clock source outputs, so clock source 39 should be utilised by the above USB controller for taking clock input, note that #clock-cells = <1>; represents multiple clock output and #clock-cells = <0>; is for single clock output.
The ccu structure is just an example.
I am developing a network driver (RTL8139) for a selfmade operating system and have problems in writing values to the PCI configuration space registers.
I want to change the value of the interrupt line (offset 0x3c) to get another IRQ number and enable bus master (set bit 2) of the command register (offset 0x04).
When I read back the values I see that my values are correctly written. But instead of using the new IRQ number (in my case 6) it uses the old value (11).
Also bus master for DMA is not working (my packets I would like to send have the correct size (set by IO-Port) but they have no content (all values just 0, transceive buffer is on physical memory and has non zero values). This always worked with my code as expected after I double checked the physical address. I need this to let the network controller access to my physical memory where my buffers for receive/transceive are. (Bus mastering needed for RTL8139)
Do I have to do something else to confirm my changes to the PCI-device?
As emulator I use qemu.
For reading/writing I wrote the following functions:
uint16_t pci_read_word(uint16_t bus, uint16_t slot, uint16_t func, uint16_t offset)
{
uint64_t address;
uint64_t lbus = (uint64_t)bus;
uint64_t lslot = (uint64_t)slot;
uint64_t lfunc = (uint64_t)func;
uint16_t tmp = 0;
address = (uint64_t)((lbus << 16) | (lslot << 11) |
(lfunc << 8) | (offset & 0xfc) | ((uint32_t)0x80000000));
outportl (0xCF8, address);
tmp = (uint16_t)((inportl (0xCFC) >> ((offset & 2) * 8)) & 0xffff);
return (tmp);
}
uint16_t pci_write_word(uint16_t bus, uint16_t slot, uint16_t func, uint16_t offset, uint16_t data)
{
uint64_t address;
uint64_t lbus = (uint64_t)bus;
uint64_t lslot = (uint64_t)slot;
uint64_t lfunc = (uint64_t)func;
uint32_t tmp = 0;
address = (uint64_t)((lbus << 16) | (lslot << 11) |
(lfunc << 8) | (offset & 0xfc) | ((uint32_t)0x80000000));
outportl (0xCF8, address);
tmp = (inportl (0xCFC));
tmp &= ~(0xFFFF << ((offset & 0x2)*8)); // reset the word at the offset
tmp |= data << ((offset & 0x2)*8); // write the data at the offset
outportl (0xCF8, address); // set address again just to be sure
outportl(0xCFC,tmp); // write data
return pci_read_word(bus,slot,func,offset); // read back data;
}
I hope somebody can help me.
The "interrupt line" field (at offset 0x03C in PCI configuration space) literally does nothing.
The Full Story
PCI cards could use up to 4 "PCI IRQs" at the PCI slot; and use them in order (so if you have ten PCI cards that all have one IRQ, then they'll all use the first PCI IRQ at the slot).
There's a tricky "barber pole" arrangement to connect "PCI IRQ at the slot" to "PCI IRQ at the host controller" that is designed to reduce IRQ sharing. If you have ten PCI cards that all have one IRQ, then they'll all use the first PCI IRQ at the slot, but the first IRQ at each PCI slot will be connected to a different PCI IRQ at the host controller. All of this is hard-wired and can not be changed by software.
To complicate things more; to connect PCI IRQs (from the PCI host controller) to the legacy PIC chips, a special "PCI IRQ router" was added. In theory the configuration of the "PCI IRQ router" can be changed by software (if you can find the documentation that describes a table that describes the location, capabilities and restrictions of the "PCI IRQ router").
Without the firmware's help it'd be impossible for an OS to figure out which PIC chip input a PCI device actually uses. For that reason, the firmware figures it out during boot and then stores "PIC chip input number" somewhere for the OS to find. That is what the "interrupt line" register is - it's just an 8-bit register that can store anything you like (that the BIOS/firmware uses to store "PIC chip input number").
I am trying to understand the following from a DTS file. Am very new to OS/Kernel.
cpus {
#address-cells = <1>;
#size-cells = <0>;
PowerPC,8313#0 {
device_type = "cpu";
reg = <0x0>;
d-cache-line-size = <32>;
i-cache-line-size = <32>;
d-cache-size = <16384>;
i-cache-size = <16384>;
timebase-frequency = <0>;
bus-frequency = <0>;
clock-frequency = <0>;
};
};
Can anyone provide brief explanation of the above?
I understand the following.
cache block size or cache line size: the amount of data that gets transferred on a cache miss.
instruction cache (I-cache): cache that can only hold instructions.
data cache (D-cache): cache that can only hold data.
Also what does i-cache-line-size mean?
d-cache-line-size = <32>;
i-cache-line-size = <32>;
d-cache-size = <16384>;
i-cache-size = <16384>;
In certain dts files there are comments like from boot loader as follows.
cpus {
#address-cells = <1>;
#size-cells = <0>;
PowerPC,8313#0 {
device_type = "cpu";
reg = <0x0>;
d-cache-line-size = <32>;
i-cache-line-size = <32>;
d-cache-size = <16384>;
i-cache-size = <16384>;
timebase-frequency = <0>; // from bootloader
bus-frequency = <0>; // from bootloader
clock-frequency = <0>; // from bootloader
};
};
How do i find out from which file in bootloader?
Bootloader used is U-boot.
Thanks.
The DTS snippet describes the PowerPC 8313 CPU.
From the Datasheet of PowerPC 8313,
7.1.5.2 Cache Units The e300c3 provides 16-Kbyte, four-way set-associative instruction and data caches. The cache block is 32
bytes long ...
Further more,
7.1.6 Bus Interface Unit (BIU) Because the caches are on-chip, write-back caches, the most common transactions are burst-read memory
operations, burst-write memory operations, ...
... Memory accesses can occur in single-beat (1–8 bytes) and four-beat burst (32 bytes) data transfers on the 64-bit data bus.
Essentially the DTS snippet configures :
the cache-size (to 16KB)
to make complete use of the on-board cache
the line-size (to 32 bytes)
to effectively use the BIU for fastest possible transfers between the CPU and the on-chip cache.
Update: Regarding your query about where to start?...
The BEST book to start with is always the Technical Reference Manual of the processor in question. It will be filled with lot of jargon specific to your hardware that you will need to patiently go through and understand.
In parallel start brushing-up your understanding of the Linux Kernel with books like "Linux Device Drivers 3e", "Understanding the Linux Kernel" and "Professional Linux Kernel Architecture".
The boot-loaders are usually written with the hardware in mind and implemented using similar semantics as the Linux Kernel i.e. borrow heavily from it. Tons of stuff is available in random blogs all over the internet. Being active on Stackoverflow and mailing-lists like kernelnewbies is a good way to find them on regular basis.