I am using the latest Openwrt trunk firmware (kernel 4.3) and have successfully compiled the driver for its CryptoEngine, an internal ipsec accelerator of MT7621 Soc (which as far as I understood is on an internal bus called AMBA / APB).
The driver seems to work (CryptoEngine gets configured successfully by sending and receiving packets, so it does "react" on memory mapped registers).
However the driver is unable to hook up the irq (so it works only in polling mode), because request_irq always fails with error code 89 (EDESTADDRREQ Destination address required) on IRQ 19 (driver log does not give any error before) and I can't understand why (I am not a linux device driver expert, just a basic understanding, let's say that I managed to recompile the driver).
Any idea why this happens ? Could it be a problem with the board dts file ? Here follows the dmesg log and a link to the dts files used (Dmesg Log too long to include, Soc dtsi (dev.openwrt.org/browser/trunk/target/linux/ramips/dts/mt7621.dtsi) and Board dts))
CryptoEngine memory map 0x1E004000-0x1E004FFF
root#OpenWrt:/# lspci
00:00.0 PCI bridge: Device 0e8d:0801 (rev 01)
00:01.0 PCI bridge: Device 0e8d:0801 (rev 01)
00:02.0 PCI bridge: Device 0e8d:0801 (rev 01)
01:00.0 Network controller: MEDIATEK Corp. MT7662E 802.11ac PCI Express Wireless Network Adapter
02:00.0 Network controller: MEDIATEK Corp. MT7662E 802.11ac PCI Express Wireless Network Adapter
03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01)
root#OpenWrt:/# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
8: 63126 63253 63143 63987 MIPS GIC Local 1 timer
10: 1513 0 0 0 MIPS GIC 10 1e100000.ethernet
11: 2 0 0 0 MIPS GIC 11 mt76pci
29: 0 0 0 0 MIPS GIC 29 xhci-hcd:usb1
31: 2 0 0 0 MIPS GIC 31 mt76pci
33: 7976 0 0 0 MIPS GIC 33 serial
63: 1178 0 0 0 MIPS GIC 63 IPI call
64: 0 1305 0 0 MIPS GIC 64 IPI call
65: 0 0 1257 0 MIPS GIC 65 IPI call
66: 0 0 0 1087 MIPS GIC 66 IPI call
67: 144709 0 0 0 MIPS GIC 67 IPI resched
68: 0 25822 0 0 MIPS GIC 68 IPI resched
69: 0 0 97134 0 MIPS GIC 69 IPI resched
70: 0 0 0 43037 MIPS GIC 70 IPI resched
root#OpenWrt:/# cat /proc/ioports
ffffffff-ffffffff : /pcie#1e140000
Related
My test setup looks as following:
Ubuntu 22.4
Kernel 5.15.1025 Realtime
I210 enp1s0 (10.1.180.98)
I225 enp2s0 (10.1.180.97)
Netgear GS108 Switch
enp1s0 and enp2s0 are connected to the switch
sending UDP Packets over enp1s0 to multicast address 224.0.0.22
listening on enp2s0 (-> external loop back)
open62541 UDP pubsub
General:
ifconig
enp1s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.1.180.98 netmask 255.255.255.0 broadcast 10.1.180.255
inet6 fe80::36fc:cf83:b6f7:e7eb prefixlen 64 scopeid 0x20<link>
ether 00:07:32:a5:c3:88 txqueuelen 1000 (Ethernet)
RX packets 10823 bytes 3936173 (3.9 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 287226 bytes 29921782 (29.9 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x7fe00000-7fe1ffff
enp2s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.1.180.97 netmask 255.255.255.0 broadcast 10.1.180.255
inet6 fe80::a22:bab1:5e74:d3ad prefixlen 64 scopeid 0x20<link>
ether 00:07:32:a5:c3:89 txqueuelen 1000 (Ethernet)
RX packets 287442 bytes 29411683 (29.4 MB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3506 bytes 174754 (174.7 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device memory 0x7fc00000-7fcfffff
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 10698 bytes 924534 (924.5 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 10698 bytes 924534 (924.5 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
# netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 10.1.180.10 0.0.0.0 UG 0 0 0 enp1s0
0.0.0.0 10.1.180.10 0.0.0.0 UG 0 0 0 enp2s0
10.1.180.0 0.0.0.0 255.255.255.0 U 0 0 0 enp2s0
10.1.180.0 0.0.0.0 255.255.255.0 U 0 0 0 enp1s0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 enp2s0
# netstat -g
IPv6/IPv4 Group Memberships
Interface RefCnt Group
--------------- ------ ---------------------
lo 1 224.0.0.251
lo 1 224.0.0.1
enp1s0 1 224.0.0.251
enp1s0 1 224.0.0.1
enp2s0 1 224.0.0.22
enp2s0 1 224.0.0.251
enp2s0 1 224.0.0.1
lo 1 ff02::fb
lo 1 ip6-allnodes
lo 1 ff01::1
enp1s0 1 ff02::fb
enp1s0 1 ff02::1:fff7:e7eb
enp1s0 1 ip6-allnodes
enp1s0 1 ff01::1
enp2s0 1 ff02::fb
enp2s0 1 ff02::1:ff74:d3ad
enp2s0 1 ip6-allnodes
enp2s0 1 ff01::1
# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
befor sending:
# netstat -i
Kernel Interface table
Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
enp1s0 1500 12397 0 0 0 572786 0 0 0 BMRU
enp2s0 1500 562000 0 0 0 4015 0 0 0 BMRU
lo 65536 12782 0 0 0 12782 0 0 0 LRU
# netstat -s -u
IcmpMsg:
InType3: 6576
OutType3: 6576
Udp:
5710 packets received
902 packets to unknown port received
0 packet receive errors
576693 packets sent
0 receive buffer errors
0 send buffer errors
IgnoredMulti: 259
UdpLite:
IpExt:
InMcastPkts: 110
OutMcastPkts: 567399
InBcastPkts: 259
InOctets: 54256683
OutOctets: 52916072
InMcastOctets: 10142
OutMcastOctets: 50498445
InBcastOctets: 19383
InNoECTPkts: 574627
MPTcpExt:
# ethtool -S enp2s0 | grep rx
rx_packets: 561920
rx_bytes: 59893407
rx_broadcast: 5508
rx_multicast: 556412
rx_crc_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
rx_long_byte_count: 59893407
rx_smbus: 0
os2bmc_rx_by_bmc: 0
os2bmc_rx_by_host: 0
rx_hwtstamp_cleared: 0
rx_lpi_counter: 0
rx_errors: 0
rx_length_errors: 0
rx_over_errors: 0
rx_frame_errors: 0
rx_fifo_errors: 0
rx_queue_0_packets: 561750
rx_queue_0_bytes: 57629925
rx_queue_0_drops: 0
rx_queue_0_csum_err: 0
rx_queue_0_alloc_failed: 0
rx_queue_1_packets: 0
rx_queue_1_bytes: 0
rx_queue_1_drops: 0
rx_queue_1_csum_err: 0
rx_queue_1_alloc_failed: 0
rx_queue_2_packets: 148
rx_queue_2_bytes: 13290
rx_queue_2_drops: 0
rx_queue_2_csum_err: 0
rx_queue_2_alloc_failed: 0
rx_queue_3_packets: 22
rx_queue_3_bytes: 2512
rx_queue_3_drops: 0
rx_queue_3_csum_err: 0
rx_queue_3_alloc_failed: 0
after sending:
# netstat -i
Kernel Interface table
Iface MTU RX-OK RX-ERR RX-DRP RX-OVR TX-OK TX-ERR TX-DRP TX-OVR Flg
enp1s0 1500 12465 0 0 0 618087 0 0 0 BMRU
enp2s0 1500 607349 0 0 0 4031 0 0 0 BMRU
lo 65536 12800 0 0 0 12800 0 0 0 LRU
# netstat -s -u
IcmpMsg:
InType3: 6588
OutType3: 6588
Udp:
5715 packets received
902 packets to unknown port received
0 packet receive errors
621972 packets sent
0 receive buffer errors
0 send buffer errors
IgnoredMulti: 263
UdpLite:
IpExt:
InMcastPkts: 112
OutMcastPkts: 612677
InBcastPkts: 263
InOctets: 58289081
OutOctets: 56953872
InMcastOctets: 10222
OutMcastOctets: 54527991
InBcastOctets: 19816
InNoECTPkts: 619936
MPTcpExt:
# ethtool -S enp2s0 | grep rx
rx_packets: 607351
rx_bytes: 64748001
rx_broadcast: 5666
rx_multicast: 601685
rx_crc_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
rx_long_byte_count: 64748001
rx_smbus: 0
os2bmc_rx_by_bmc: 0
os2bmc_rx_by_host: 0
rx_hwtstamp_cleared: 0
rx_lpi_counter: 0
rx_errors: 0
rx_length_errors: 0
rx_over_errors: 0
rx_frame_errors: 0
rx_fifo_errors: 0
rx_queue_0_packets: 607176
rx_queue_0_bytes: 62302224
rx_queue_0_drops: 0
rx_queue_0_csum_err: 0
rx_queue_0_alloc_failed: 0
rx_queue_1_packets: 0
rx_queue_1_bytes: 0
rx_queue_1_drops: 0
rx_queue_1_csum_err: 0
rx_queue_1_alloc_failed: 0
rx_queue_2_packets: 153
rx_queue_2_bytes: 13861
rx_queue_2_drops: 0
rx_queue_2_csum_err: 0
rx_queue_2_alloc_failed: 0
rx_queue_3_packets: 22
rx_queue_3_bytes: 2512
rx_queue_3_drops: 0
rx_queue_3_csum_err: 0
rx_queue_3_alloc_failed: 0
Dropwatch output is as followed:
# sudo dropwatch -l ksa
2 drops at igmp_rcv+10c (0xffffffff9dd7202c) [software]
1 drops at unix_stream_connect+36a (0xffffffff9ddbb10a) [software]
2 drops at ip_rcv_finish_core.constprop.0+19c (0xffffffff9dd1930c) [software]
2048 drops at ip_rcv_finish_core.constprop.0+19c (0xffffffff9dd1930c) [software]
2036 drops at ip_rcv_finish_core.constprop.0+19c (0xffffffff9dd1930c) [software]
1 drops at __udp4lib_lib_mcast_deliver+31f (0xffffffff9dd5d67f) [software]
1 drops at __udp4lib_lib_mcast_deliver+31f (0xffffffff9dd5d67f) [software]
If I run this setup (exatly same UDP packets with tcpdump) with a real second windows device, receiving works. But this "external loopback" dosn't receive anything (I want to create so a TSN setup, so the windows machine is no option).
If I don't specify the interface for receiving, I get the packets (but don't know if they come from the loopback)
Following steps I tried without success:
Disabling RP_FILTER (in any combination for all available interfaces)
promisc mode on (but the ethtool output says that there is no problem on the NIC side)
What did I missed?
Best regards,
Patrick
My goal is to send UDP multicast packets on the first interface and receive them on the second interface (for performance analysis and for simulating a current missing Master hardware).
Why I am not seeing any SWAP usage in below scenario, even though the virtual memory is showing 20gig ?
Tasks: 186 total, 1 running, 185 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32880880k total, 17555744k used, 15325136k free, 300268k buffers
Swap: 16383996k total, 0k used, 16383996k free, 2287700k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10314 abcuser 20 0 20.3g 6.8g 25m S 0.7 21.8 23:47.38 java
25119 abcuser 20 0 20.3g 6.7g 25m S 0.3 21.4 23:38.25 java
total used free shared buffers cached
Mem: 31 16 14 0 0 2
-/+ buffers/cache: 14 17
Swap: 15 0 15
Total: 46 16 30
Regards
Sanjeev
My WinDBG version is 10.0.10240.9 AMD64 and while casually debugging some native memory dump I realized that my !heap command behaves different than described and I am unable to figure out why.
There are plenty of resources mentioning !heap -s:
https://msdn.microsoft.com/en-us/library/windows/hardware/ff563189%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
http://windbg.info/doc/1-common-cmds.html
When I execute !heap -s
I get this truncated list:
0:000> !heap -s
************************************************************************************************************************
NT HEAP STATS BELOW
************************************************************************************************************************
LFH Key : 0x000000c42ceaf6ca
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-------------------------------------------------------------------------------------
Virtual block: 0000000003d40000 - 0000000003d40000 (size 0000000000000000)
... many more virtual blocks
0000000000b90000 00000002 3237576 3220948 3237576 20007 1749 204 359 0 LFH
0000000000010000 00008000 64 8 64 5 1 1 0 0
... more heaps
-------------------------------------------------------------------------------------
Ok fine, b90000 looks big but contrary to those docs above and !heap -s -? I cannot get information for this heap, each of those commands produce the exact same output as seen above (as if I would not specify anything after -s):
!heap -s b90000
!heap -s -h b90000
!heap -s 1
I get a load of virtual blocks and a dump of all heaps instead of the single specified one.
Anyone having the same issue?
My "Windows Debugger Version 10.0.10586.567 AMD64" behaved like yours, but
“Microsoft (R) Windows Debugger Version 6.3.9600.16384 AMD64” I have in in:
C:\Program Files\Windows Kits\8.1\Debuggers\x64
0:000> !heap -s -h 0000000000220000
Walking the heap 0000000000220000 ..................Virtual block: 0000000015f20000 - 0000000015f20000 (size 0000000000000000)
Virtual block: 000000001b2e0000 - 000000001b2e0000 (size 0000000000000000)
Virtual block: 000000001f1e0000 - 000000001f1e0000 (size 0000000000000000)
Virtual block: 0000000023c10000 - 0000000023c10000 (size 0000000000000000)
Virtual block: 000000001c060000 - 000000001c060000 (size 0000000000000000)
Virtual block: 000000001ddc0000 - 000000001ddc0000 (size 0000000000000000)
0: Heap 0000000000220000
Flags 00000002 - HEAP_GROWABLE
Reserved memory in segments 226880 (k)
Commited memory in segments 218204 (k)
Virtual bytes (correction for large UCR) 218740 (k)
Free space 12633 (k) (268 blocks)
External fragmentation 5% (268 free blocks)
Virtual address fragmentation 0% (30 uncommited ranges)
Virtual blocks 6 - total 0 KBytes
Lock contention 0
Segments 1
Low fragmentation heap 00000000002291e0
Lock contention 0
Metadata usage 90112 bytes
Statistics:
Segments created 993977
Segments deleted 992639
Segments reused 0
Block cache:
3: 1024 bytes ( 17, 0)
4: 2048 bytes ( 42, 0)
5: 4096 bytes ( 114, 0)
6: 8192 bytes ( 231, 2)
7: 16384 bytes ( 129, 9)
8: 32768 bytes ( 128, 11)
9: 65536 bytes ( 265, 58)
10: 131072 bytes ( 357, 8)
11: 262144 bytes ( 192, 49)
Buckets info:
Size Blocks Seg Empty Aff Distribution
------------------------------------------------
------------------------------------------------
Default heap Front heap Unused bytes
Range (bytes) Busy Free Busy Free Total Average
------------------------------------------------------------------
0 - 1024 577 140 1035286 11608 10563036 10
1024 - 2048 173 3 586 374 27779 36
2048 - 3072 17 19 47 224 1605 25
3072 - 4096 20 12 1 126 348 16
4096 - 5120 35 3 1 30 677 18
5120 - 6144 2 8 0 0 33 16
6144 - 7168 5 9 0 0 56 11
7168 - 8192 0 11 0 0 0 0
8192 - 9216 14 0 0 15 236 16
9216 - 10240 1 0 0 0 8 8
12288 - 13312 1 0 0 0 17 17
14336 - 15360 1 0 0 18 1 1
15360 - 16384 1 0 0 0 32 32
16384 - 17408 10 0 0 0 160 16
22528 - 23552 1 0 0 0 9 9
23552 - 24576 2 0 0 0 32 16
27648 - 28672 1 0 0 0 8 8
30720 - 31744 0 1 0 0 0 0
32768 - 33792 18 0 0 0 250 13
33792 - 34816 0 1 0 0 0 0
39936 - 40960 0 2 0 0 0 0
40960 - 41984 0 1 0 0 0 0
43008 - 44032 0 2 0 0 0 0
44032 - 45056 0 5 0 0 0 0
45056 - 46080 0 1 0 0 0 0
46080 - 47104 0 2 0 0 0 0
47104 - 48128 0 1 0 0 0 0
49152 - 50176 0 3 0 0 0 0
50176 - 51200 1 0 0 0 16 16
51200 - 52224 0 4 0 0 0 0
57344 - 58368 0 1 0 0 0 0
58368 - 59392 0 1 0 0 0 0
62464 - 63488 0 1 0 0 0 0
63488 - 64512 200 1 0 0 3200 16
64512 - 65536 0 1 0 0 0 0
65536 - 66560 1029 2 0 0 10624 10
79872 - 80896 100 0 0 0 900 9
131072 - 132096 9 0 0 0 144 16
193536 - 194560 1 0 0 0 9 9
224256 - 225280 1 0 0 0 16 16
262144 - 263168 49 27 0 0 784 16
327680 - 328704 1 0 0 0 17 17
384000 - 385024 0 1 0 0 0 0
523264 - 524288 1 5 0 0 23 23
------------------------------------------------------------------
Total 2271 268 1035921 12395 10610020 10
This might be a walkaround,
can’t answer why the win 10 version don’t work :-(
I need to know what tm->when means, but proc(5) doesn't mention anything helpful,
So, does it store the creation time of the socket? The number seems to be decreasing each time I view the file.
root#ubuntu-vm:~# cat /proc/net/tcp
sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode
0: 00000000:0CEA 00000000:0000 0A 00000000:00000000 00:00000000 00000000 104 0 17410 1 dddb6d00 100 0 0 10 -1
1: 00000000:0016 00000000:0000 0A 00000000:00000000 00:00000000 00000000 0 0 7959 1 dddb4500 100 0 0 10 -1
2: B238A8C0:0016 0138A8C0:9C96 01 00000000:00000000 02:00061444 00000000 0 0 8243 4 daa3c000 20 4 27 10 16
3: B238A8C0:0CEA 0138A8C0:8753 01 00000000:00000000 02:0009C787 00000000 104 0 19467 2 daa3e300 20 4 18 10 -1
From Exploring the /proc/net/ Directory
The tr field indicates whether a timer is active for this socket. A value of zero indicates the timer is not active. The tm->when field indicates the time remaining (in jiffies) before timeout occurs.
i have an installation on memcache which i want to use in my production environment but when i have ran a couple of tests it seems that memcache doesn't free up memory even after it has used up all of it allocated memory, Also i logged in and ran a flush_all command but the objects are still in the cache.
Here are outputs from some tests
memcached-tool
memcache-top v0.6 (default port: 11211, color: on, refresh: 3 seconds)
INSTANCE USAGE HIT % CONN TIME EVICT/s READ/s WRITE/s
127.0.0.1:11211 427.1% 0.0% 18 1.4ms 0.0 244 261.0K
AVERAGE: 427.1% 0.0% 18 1.4ms 0.0 244 261.0K
TOTAL: 4.3MB/ 1.0MB 18 1.4ms 0.0 244 261.0K
memcached-tool 127.0.0.1:11211 display
No Item_Size Max_age Pages Count Full? Evicted Evict_Time OOM
1 560B 4s 1 1872 yes 0 0 15488
2 704B 32s 1 559 no 0 0 0
3 880B 4s 1 1191 yes 0 0 1335
4 1.1K 9s 1 116 no 0 0 0
5 1.4K 21s 1 14 no 0 0 0
6 1.7K 4s 1 17 no 0 0 0
7 2.1K 84s 1 24 no 0 0 0
8 2.7K 130s 1 60 no 0 0 0
9 3.3K 25s 1 290 no 0 0 0
10 4.2K 9s 1 194 no 0 0 0
11 5.2K 9s 1 116 no 0 0 0
15 12.7K 816s 1 1 no 0 0 0
16 15.9K 769s 1 5 no 0 0 0
18 24.8K 786s 1 1 no 0 0 0
21 48.5K 816s 1 1 no 0 0 0
memcached-tool 127.0.0.1:11211 stats
127.0.0.1:11211 Field Value
accepting_conns 1
auth_cmds 0
auth_errors 0
bytes 4478060
bytes_read 23964596
bytes_written 546642860
cas_badval 0
cas_hits 0
cas_misses 0
cmd_flush 0
cmd_get 240894
cmd_set 4504
conn_yields 0
connection_structures 21
curr_connections 18
curr_items 4461
decr_hits 0
decr_misses 0
delete_hits 0
delete_misses 0
evictions 0
get_hits 43756
get_misses 197138
incr_hits 0
incr_misses 0
limit_maxbytes 1048576
listen_disabled_num 0
pid 8731
pointer_size 64
reclaimed 0
rusage_system 5.047232
rusage_user 4.311344
threads 4
time 1306247929
total_connections 3092
total_items 4504
uptime 1240
version 1.4.5
-m tells memcached how much RAM to use for item storage (in megabytes). Note
carefully that this isn't a global
memory limit, so memcached will use a
few % more memory than you tell it to.
Set this to safe values. Setting it to
less than 48 megabytes does not work
properly in 1.4.x and earlier. It will
still use the memory.
Source: https://github.com/memcached/memcached/wiki/ConfiguringServer#commandline-arguments