Is there a way to get the IBM XL C/C++ Compiler, Metal Option to make an __asm DS declaration at the global scope _NOT_ be an EXTERNAL? - zos

I am creating several similar (but not the same) invocations of the list form of the WTO macro in the global scope of my program:
__asm(
"xxxUtilsStaticWtp WTO TEXT=*,ROUTCDE=(11),LINKAGE=,MF=L \n" // Static list form WTO
: "DS"(xxxUtilsStaticWtp) // DS name to locate list form of WTO
); ...
__asm(
"xxxUtilsStaticSWto WTO TEXT=*,ROUTCDE=(2,10),DESC=(3),LINKAGE=,MF=L \n" // Static list form WTO
: "DS"(xxxUtilsStaticSWto) // DS name to locate list form of WTO
);
(Yes, I know that one has the label inside the assembler statement declaration and one does not, it does not appear to matter either way).
When this gets compiled, the compiler emits this information about these variables:
xxxUtilsStaticSWto Class = static, Length = 256
Type = struct __ASM_DS_256
3180-10:741, 3211-10:770
xxxUtilsStaticWtp Class = static, Length = 256
Type = struct __ASM_DS_256
3089-10:650, 3136-10:695
Plus this error message:
ERROR CCN3244 XXXXX.XXXXX.H(XXXUTILS):741 External variable XXXUTILS cannot be redefined.
(Note that it works fine if one of the declarations is removed).
If I understand this correctly, the compiler is shortening the name down to eight characters (I am not using GOFF) so both of these are colliding with name XXXUTILS.
I know that I can keep other kinds of declarations at the global scope from being EXTERNAL by adding "static" to the declaration, e.g.
static int MyRoutine(int Input) { <code> }
or
static int AnInt = 5;
I spent some quality time with Inline assembly statements (IBM extension), but could not divine how to somehow squeeze the word "static" into the __asm statement or if there was another way to do it.
Anybody have any ideas on how to accomplish this?
Thank you,
Scott Fagen

Ok, further investigation/playing around has lead me to believe that this might be a compiler bug.  If I rename one of the DS variables from xxxUtils... to xxyUtils... everything compiles correctly.  Removing the "first eight character collision" eliminates the problem (which is how I will solve it for now).  
There is nothing in the assembler listing that even references xxxUtils or xxyUtils, which leads me to believe that the symbols are being somehow incorrectly tagged while the compiler is building up the information to create the $STATIC area of the module:
001CA8 2713 $STATIC DS 0D 000000
.....
01D70 2751 ##LAB#5 EQU * 000000
2752 WTO TEXT=*,ROUTCDE=(11),LINKAGE=,MF=L 000000
001D70 2754+ DS 0F 01-WTO
001D70 0008 2755+ DC AL2(8) TEXT LENGTH #YA17152 01-WTO
001D72 8010 2756+ DC B'1000000000010000' MCSFLAGS 01-WTO
001D74 00000000 2757+ DC AL4(0) MESSAGE TEXT ADDRESS #L5A 01-WTO
001D78 02 2758+ DC AL1(2) VERSION LEVEL #PJC 01-WTO
001D79 00 2759+ DC B'00000000' MISCELLANEOUS FLAGS #L2A 01-WTO
001D7A 00 2760+ DC AL1(0) REPLY LENGTH #L2A 01-WTO
001D7B 68 2761+ DC AL1(104) LENGTH OF WPX #L5C 01-WTO
001D7C 0080 2762+ DC B'0000000010000000' EXTENDED MCS FLAGS #L2A 01-WTO
001D7E 0000 2763+ DC AL2(0) RESERVED #L2A 01-WTO
001D80 00000000 2764+ DC AL4(0) REPLY BUFFER ADDRESS #P7C 01-WTO
001D84 00000000 2765+ DC AL4(0) REPLY ECB ADDRESS #P7C 01-WTO
001D88 00000000 2766+ DC AL4(0) CONNECT ID #01C 01-WTO
001D8C 0000 2767+ DC B'0000000000000000' DESCRIPTOR CODES #L2A 01-WTO
001D8E 0000 2768+ DC AL2(0) RESERVED #L2A 01-WTO
001D90 0020000000000000 2769+ DC XL16'00200000000000000000000000000000' X01-WTO
001D98 0000000000000000 + EXTENDED ROUTING CODES #L2A
001DA0 0000 2770+ DC B'0000000000000000' MESSAGE TYPE #L2A 01-WTO
001DA2 0000 2771+ DC AL2(0) MESSAGE'S PRIORITY #L2A 01-WTO
001DA4 4040404040404040 2772+ DC CL8' ' JOB ID #L2A 01-WTO
001DAC 4040404040404040 2773+ DC CL8' ' JOB NAME #L2A 01-WTO
001DB4 4040404040404040 2774+ DC CL8' ' RETRIEVAL KEY #L2A 01-WTO
001DBC 00000000 2775+ DC AL4(0) TOKEN FOR DOM #P1C 01-WTO
001DC0 00000000 2776+ DC AL4(0) CONSOLE ID #P1C 01-WTO
001DC4 4040404040404040 2777+ DC CL8' ' SYSTEM NAME #L2A 01-WTO
001DCC 4040404040404040 2778+ DC CL8' ' CONSOLE NAME #L3A 01-WTO
001DD4 00000000 2779+ DC AL4(0) REPLY CONSOLE NAME/ID ADDR #L3A 01-WTO
001DD8 00000000 2780+ DC AL4(0) CART ADDRESS #L4C 01-WTO
001DDC 00000000 2781+ DC AL4(0) WSPARM ADDRESS #L6C 01-WTO
00070 2782 ##LAB#5L EQU *-##LAB#5 000000
2783 &DSMAC SETC '##LAB#5' 000000
2784 &DSSIZE SETA 256 000000
2785 &MSIZE SETA ##LAB#5L 000000
2786 AIF (&DSSIZE GE &MSIZE).##OK#5 000000
2787 .##OK#5 ANOP 000000
001DE0 01DE0 01E70 2788 ORG $STATIC+456 000000
01E70 2789 ##LAB#6 EQU * 000000
2790 WTO TEXT=*,ROUTCDE=(2,10),DESC=(3),LINKAGE=,MF=L 000000
001E70 2792+ DS 0F 01-WTO
001E70 0008 2793+ DC AL2(8) TEXT LENGTH #YA17152 01-WTO
001E72 8010 2794+ DC B'1000000000010000' MCSFLAGS 01-WTO
001E74 00000000 2795+ DC AL4(0) MESSAGE TEXT ADDRESS #L5A 01-WTO
001E78 02 2796+ DC AL1(2) VERSION LEVEL #PJC 01-WTO
001E79 00 2797+ DC B'00000000' MISCELLANEOUS FLAGS #L2A 01-WTO
001E7A 00 2798+ DC AL1(0) REPLY LENGTH #L2A 01-WTO
001E7B 68 2799+ DC AL1(104) LENGTH OF WPX #L5C 01-WTO
001E7C 0080 2800+ DC B'0000000010000000' EXTENDED MCS FLAGS #L2A 01-WTO
001E7E 0000 2801+ DC AL2(0) RESERVED #L2A 01-WTO
001E80 00000000 2802+ DC AL4(0) REPLY BUFFER ADDRESS #P7C 01-WTO
001E84 00000000 2803+ DC AL4(0) REPLY ECB ADDRESS #P7C 01-WTO
001E88 00000000 2804+ DC AL4(0) CONNECT ID #01C 01-WTO
001E8C 2000 2805+ DC B'0010000000000000' DESCRIPTOR CODES #L2A 01-WTO
001E8E 0000 2806+ DC AL2(0) RESERVED #L2A 01-WTO
001E90 4040000000000000 2807+ DC XL16'40400000000000000000000000000000' X01-WTO
001E98 0000000000000000 + EXTENDED ROUTING CODES #L2A
001EA0 0000 2808+ DC B'0000000000000000' MESSAGE TYPE #L2A 01-WTO
001EA2 0000 2809+ DC AL2(0) MESSAGE'S PRIORITY #L2A 01-WTO
001EA4 4040404040404040 2810+ DC CL8' ' JOB ID #L2A 01-WTO
001EAC 4040404040404040 2811+ DC CL8' ' JOB NAME #L2A 01-WTO
001EB4 4040404040404040 2812+ DC CL8' ' RETRIEVAL KEY #L2A 01-WTO
001EBC 00000000 2813+ DC AL4(0) TOKEN FOR DOM #P1C 01-WTO
001EC0 00000000 2814+ DC AL4(0) CONSOLE ID #P1C 01-WTO
001EC4 4040404040404040 2815+ DC CL8' ' SYSTEM NAME #L2A 01-WTO
001ECC 4040404040404040 2816+ DC CL8' ' CONSOLE NAME #L3A 01-WTO
001ED4 00000000 2817+ DC AL4(0) REPLY CONSOLE NAME/ID ADDR #L3A 01-WTO
001ED8 00000000 2818+ DC AL4(0) CART ADDRESS #L4C 01-WTO
001EDC 00000000 2819+ DC AL4(0) WSPARM ADDRESS #L6C 01-WTO
00070 2820 ##LAB#6L EQU *-##LAB#6 000000
----- rest of $STATIC -----
The xxxUtils list form WTO is at x'1D70' and the xxyUtils list form is at x'1E70'.
A search of the assembler listing for xxxUtils or xxyUtils yields nothing.
I'd expect that this is a bug of some sort...
Thanks,
Scott

Related

Perl: Reproducing a UDP packet

I'm trying to emulate the UDP output of a particular piece of software (a very simple remote control for an audio player). Capturing its output with tcpdump -X was no problem, and resulted in packets like the ones below:
Turn the device off...
18:30:03.623499 IP x.x.x.1.27490 > x.x.x.2.23500: UDP, length 32
0x0000: 4500 003c 42e6 0000 8011 92ee 0a02 2810 E..<B.........(.
0x0010: 0a02 28c9 6b62 5bcc 0028 b213 0500 007f ..(.kb[..(......
0x0020: 0100 0000 1800 0000 0000 0000 0000 0000 ................
0x0030: 0000 0000 0100 0000 0200 0000 ............
Turn the device on...
18:30:06.222808 IP x.x.x.1.27490 > x.x.x.2.23500: UDP, length 32
0x0000: 4500 003c 42e7 0000 8011 92ed 0a02 2810 E..<B.........(.
0x0010: 0a02 28c9 6b62 5bcc 0028 b213 0500 007f ..(.kb[..(......
0x0020: 0100 0000 1800 0000 0000 0000 0000 0000 ................
0x0030: 0000 0000 0200 0000 0100 0000 ............
Now I'm trying to reproduce these packets in Perl, so I can send them via IO::Socket::INET and have them arrive looking the same as the above.
I've been fooling around with the Perl pack function, but I'm afraid my understanding of what I'm looking at is insufficient to craft a working template.
Am I even on the right track here?
To turn the device off, send the following string to 10.2.40.201:23500 using UDP:
"\x05\x00\x00\x7f"."\x01\x00\x00\x00"."\x18\x00\x00\x00"."\x00\x00\x00\x00".
"\x00\x00\x00\x00"."\x00\x00\x00\x00"."\x01\x00\x00\x00"."\x02\x00\x00\x00"
To turn it on,
"\x05\x00\x00\x7f"."\x01\x00\x00\x00"."\x18\x00\x00\x00"."\x00\x00\x00\x00".
"\x00\x00\x00\x00"."\x00\x00\x00\x00"."\x02\x00\x00\x00"."\x01\x00\x00\x00"
IP packet structure
UDP packet structure
The first byte tells gives us important information.
45
Version: 4 (IPv4)
IHL: 5 (IP header is 20 bytes long, including this byte.)
Now we know how long the header is.
00 00 3c 42 e6 00 00 80 11 92 ee 0a 02 28 10
0a 02 28 c9
DSCP: 0
ECN: 0
Total length: 0x003c (60)
Identification: 0x42e6
Flags: 0
Fragment offset: 0
TTL: 0x80 (128)
Protocol: 0x11 (UDP)
Header checksum: 0x92ee
Source IP: 0x0a022810 (10.2.40.16)
Destination IP: 0x0a0228c9 (10.2.40.201)
It's an unfragmented packet, so it's the full packet.
The next 60 - 20 = 40 bytes is the IP packet payload.
It's a UDP packet, so the next 8 bytes form the UDP packet header.
6b 62 5b cc 00 28 b2 13
Source port: 0x6b62 (27490)
Destination port: 0x5bcc (23500)
Length (IP header excluded): 0x0028 (40)
Checksum: 0xb213
The next 40 - 8 = 32 bytes is the UDP packet payload.
05 00 00 7f 01 00 00 00 18 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00
To send this message, you'd use UDP send the following string to 10.2.40.201:23500:
"\x05\x00\x00\x7f"."\x01\x00\x00\x00"."\x18\x00\x00\x00"."\x00\x00\x00\x00".
"\x00\x00\x00\x00"."\x00\x00\x00\x00"."\x01\x00\x00\x00"."\x02\x00\x00\x00"
Here, I divided into four-byte strings for readability. How you build the string is irrelevant. You could use pack in a million different ways to generate this string, but there's no way to know which one is significant without knowing the protocol.
Now, you asked to replicate the packets, but that's surely unnecessary. You only need to send the same message. There are going to be differences in the headers. You might be able to control some of them (e.g. by binding the socket to 10.2.40.16:27490). But others might be more difficult. The TTL field decreases as the the packet moves through the network. The packet can become fragmented (divided into smaller packets called fragments). etc. But none of that should be relevant.

Enabling 802.11w mode with hostapd

I'm trying to setup a WiFi Access Point with a Raspberry Pi 3B+ having 802.11w enabled.
Kernel version: Linux efb-ap-0 4.19.66-Re4son-v7+ #1 SMP Sun Aug 18 22:25:39 AEST 2019 armv7l GNU/Linux
Driver: brcmfmac
hostapd (Deb package): 2:2.9-1 armel
During the 4-Way Handshake, wpa_supplicant immediatly disconnects at the 3/4 msg, with following logs:
wlan0: WPA: IE in 3/4 msg does not match with IE in Beacon/ProbeResp (src=b8:27:eb:3b:3f:0e)
WPA: RSN IE in Beacon/ProbeResp - hexdump(len=28): 30 1a 01 00 00 0f ac 04 01 00 00 0f ac 04 01 00 00 0f ac 06 c0 00 00 00 00 0f ac 06
WPA: RSN IE in 3/4 msg - hexdump(len=26): 30 18 01 00 00 0f ac 04 01 00 00 0f ac 04 02 00 00 0f ac 02 00 0f ac 06 c0 00
Comparing 3/4 msg hexdump and Beacon hexdump via Wireshark shows that the Beacon contains the following additional fields that are not in the 3/4 msg: PMKID Count (0x00 00)+ PMKID List + Group Management Cipher Suite
(0x00 0f ac 06).
Why is the 3/4 msg not matching the Beacon ? Is this an issue in hostapd ? in driver ? in hostapd<->driver communication ?
Thanks for any information about that.
You can find below the hostapd.conf content:
interface=wlan0
driver=nl80211
logger_syslog=-1
logger_syslog_level=2
auth_algs=1
wpa_pairwise=CCMP
rsn_pairwise=CCMP
wpa=2
hw_mode=g
ieee80211w=2
ssid=XXXXXXXXXX
channel=1
wpa_key_mgmt=WPA-PSK-SHA256
wpa_passphrase=XXXXXXXXXX
And the wpa_supplicant.conf used to connect:
ctrl_interface=DIR=/var/run/
network={
ssid="XXXXXXXX"
proto=RSN
scan_ssid=1
key_mgmt=WPA-PSK-SHA256
pairwise=CCMP
psk="XXXXXXXX"
ieee80211w=2
}
Note: this thread is a duplicate from a message I had posted on hostap mailing list for which I didn't have answer: http://lists.infradead.org/pipermail/hostap/2019-November/040764.html

Wifi connection gets rejected with CTRL-EVENT-ASSOC-REJECT

This problem is with Ubuntu 16.04 running on a mobile device BQ E4.5.
I have at home a Wifi AP (AVM FRITZ!Box WLAN 3170) and a bunch of Linux and FreeBSD laptops and 4 phones BQ E4.5 connected to it. All do fine, but since some month one of the E4.5 running the latest developer channel can not connect anymore to my AP. It started (IIRC) somewhere in December, first being unstable on connect and after some days it could not connect at all anymore. The connection is always rejected from the AP with a package CTRL-EVENT-ASSOC-REJECT (see debug log below). As this E4.5 is only my test device and as this can connect without any problem to all other AP, I mostly don't care. Really, I have not found any other AP, not in public, not in my office, to which the device can't connect. That's why I think the fault must be in the AP or in the radio environment in my house (I tested all channels), but as this AP does not offer any log for debugging, I can't debug this from this side.
I managed to monitor and raised the debug level with wpa-cli in the E4.5
via "adb shell" and have here the log of one connection attempt.
Is there any Wifi expert to see from the log why it gets rejected?
Is this problem somewhere known in UBports or UBphone (Canonical) land?
PS: Please don't think in AP's blacklist (as this AP hasn't one) or in
invalid psk. I run Wifi for many years and know what I do :-)
1548368340.906062: wlan0: Starting AP scan for wildcard SSID
1548368340.906260: wlan0: Determining shared radio frequencies (max len 1)
1548368340.906338: wlan0: Shared frequencies (len=0): completed iteration
1548368340.906410: wlan0: Add radio work 'scan'#0xb8a29310
1548368340.906475: wlan0: First radio work item in the queue - schedule start immediately
1548368340.906584: wlan0: Starting radio work 'scan'#0xb8a29310 after 0.000099 second wait
1548368340.906834: wlan0: nl80211: scan request
1548368340.906930: nl80211: Scan SSID - hexdump_ascii(len=0): [NULL]
1548368340.908515: Scan requested (ret=0) - scan timeout 30 seconds
1548368340.908740: nl80211: Event message available
1548368340.908840: nl80211: Drv Event 33 (NL80211_CMD_TRIGGER_SCAN) received for wlan0
1548368340.908912: wlan0: nl80211: Scan trigger
1548368340.908981: wlan0: Event SCAN_STARTED (47) received
1548368340.909053: wlan0: Own scan request started a scan in 0.000342 seconds
1548368340.911945: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1
1548368341.325323: RTM_NEWLINK: ifi_index=10 ifname=wlan0 wext ifi_family=0 ifi_flags=0x1003 ([UP])
1548368341.325791: RTM_NEWLINK: ifi_index=10 ifname=wlan0 wext ifi_family=0 ifi_flags=0x1003 ([UP])
1548368341.325972: nl80211: Event message available
1548368341.326090: nl80211: Drv Event 34 (NL80211_CMD_NEW_SCAN_RESULTS) received for wlan0
1548368341.326164: wlan0: nl80211: New scan results available
1548368341.326226: nl80211: Scan probed for SSID ''
1548368341.326302: nl80211: Scan included frequencies: 2412 2417 2422 2427 2432 2437 2442 2447 2452 2457 2462 2467 2472
1548368341.326366: wlan0: Event SCAN_RESULTS (3) received
1548368341.326436: wlan0: Scan completed in 0.417385 seconds
1548368341.327931: nl80211: Received scan results (6 BSSes)
1548368341.328181: wlan0: BSS: Start scan result update 17
1548368341.328675: BSS: last_scan_res_used=6/32
1548368341.328775: wlan0: New scan results available (own=1 ext=0)
1548368341.338042: WPS: AP[0] d4:21:22:e0:24:8b type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338203: WPS: AP[1] e0:28:6d:26:32:9d type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338272: WPS: AP[2] cc:ce:1e:b1:69:38 type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338334: WPS: AP[3] 10:b1:f8:fb:fb:38 type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338396: WPS: AP[4] 7c:ff:4d:7c:5c:29 type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338529: WPS: AP[5] 24:69:a5:5d:cb:5e type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338762: WPS: AP[6] 30:d3:2d:74:86:41 type=0 tries=0 last_attempt=-1 sec ago blacklist=0
1548368341.338873: wlan0: Radio work 'scan'#0xb8a29310 done in 0.432282 seconds
1548368341.338945: wlan0: Selecting BSS from priority group 0
1548368341.339039: wlan0: 0: 00:1c:4a:06:17:f5 ssid='tarara' wpa_ie_len=24 rsn_ie_len=20 caps=0x411 level=-42
1548368341.339112: wlan0: selected based on RSN IE
1548368341.339194: wlan0: selected BSS 00:1c:4a:06:17:f5 ssid='tarara'
1548368341.339303: wlan0: Considering connect request: reassociate: 0 selected: 00:1c:4a:06:17:f5 bssid: 00:00:00:00:00:00 pending: 00:00:00:00:00:00 wpa_state: SCANNING ssid=0xb8a28738 current_ssid=(nil)
1548368341.339377: wlan0: Request association with 00:1c:4a:06:17:f5
1548368341.339437: wlan0: Re-association to the same ESS
1548368341.339495: WPA: Unrecognized EAPOL-Key Key Data IE - hexdump(len=8): 00 06 74 61 72 61 72 61
1548368341.339565: WPA: Unrecognized EAPOL-Key Key Data IE - hexdump(len=3): 03 01 03
1548368341.339625: WPA: Unrecognized EAPOL-Key Key Data IE - hexdump(len=14): 05 0c 00 03 00 00 00 00 00 00 00 00 00 00
1548368341.339707: WPA: Unrecognized EAPOL-Key Key Data IE - hexdump(len=3): 2a 01 04
1548368341.339766: WPA: RSN IE in EAPOL-Key - hexdump(len=22): 30 14 01 00 00 0f ac 02 01 00 00 0f ac 04 01 00 00 0f ac 02 00 00
1548368341.339863: WPA: WPA IE in EAPOL-Key - hexdump(len=26): dd 18 00 50 f2 01 01 00 00 50 f2 02 01 00 00 50 f2 02 01 00 00 50 f2 02 00 00
1548368341.339981: wlan0: Add radio work 'connect'#0xb8a29310
1548368341.340044: wlan0: First radio work item in the queue - schedule start immediately
1548368341.340112: RSN: Ignored PMKID candidate without preauth flag
1548368341.340224: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1
1548368341.358253: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1/BSSs/7
1548368341.364150: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1/BSSs/19
1548368341.368290: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1/BSSs/20
1548368341.368562: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1/BSSs/1
1548368341.371943: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1/BSSs/2
1548368341.372207: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1/BSSs/6
1548368341.372566: wlan0: Starting radio work 'connect'#0xb8a29310 after 0.032518 second wait
1548368341.372608: wlan0: Trying to associate with SSID 'tarara'
1548368341.372638: wlan0: Cancelling scan request
1548368341.372667: wlan0: WPA: clearing own WPA/RSN IE
1548368341.372697: wlan0: Automatic auth_alg selection: 0x1
1548368341.372725: wlan0: Overriding auth_alg selection: 0x1
1548368341.372752: RSN: PMKSA cache search - network_ctx=0xb8a28738 try_opportunistic=0
1548368341.372778: RSN: Search for BSSID 00:1c:4a:06:17:f5
1548368341.372805: RSN: No PMKSA cache entry found
1548368341.372835: wlan0: RSN: using IEEE 802.11i/D9.0
1548368341.372868: wlan0: WPA: Selected cipher suites: group 8 pairwise 16 key_mgmt 2 proto 2
1548368341.372897: wlan0: WPA: Selected mgmt group cipher 32
1548368341.372923: WPA: set AP WPA IE - hexdump(len=26): dd 18 00 50 f2 01 01 00 00 50 f2 02 01 00 00 50 f2 02 01 00 00 50 f2 02 00 00
1548368341.372973: WPA: set AP RSN IE - hexdump(len=22): 30 14 01 00 00 0f ac 02 01 00 00 0f ac 04 01 00 00 0f ac 02 00 00
1548368341.373022: wlan0: WPA: using GTK TKIP
1548368341.373050: wlan0: WPA: using PTK CCMP
1548368341.373077: wlan0: WPA: using KEY_MGMT WPA-PSK
1548368341.373105: wlan0: WPA: not using MGMT group cipher
1548368341.373131: WPA: Set own WPA IE default - hexdump(len=22): 30 14 01 00 00 0f ac 02 01 00 00 0f ac 04 01 00 00 0f ac 02 00 00
1548368341.373288: wlan0: State: SCANNING -> ASSOCIATING
1548368341.373319: nl80211: Set wlan0 operstate 0->0 (DORMANT)
1548368341.373347: netlink: Operstate: ifindex=10 linkmode=-1 (no change), operstate=5 (IF_OPER_DORMANT)
1548368341.373507: wlan0: set_disable_max_amsdu: -1
1548368341.373546: wlan0: set_ampdu_factor: -1
1548368341.373575: wlan0: set_ampdu_density: -1
1548368341.373603: wlan0: set_disable_ht40: 0
1548368341.373630: wlan0: set_disable_sgi: 0
1548368341.373658: wlan0: set_disable_ldpc: 0
1548368341.373688: wlan0: Determining shared radio frequencies (max len 1)
1548368341.373718: wlan0: Shared frequencies (len=0): completed iteration
1548368341.373744: nl80211: Set mode ifindex 10 iftype 2 (STATION)
1548368341.373848: nl80211: Unsubscribe mgmt frames handle 0x3029e831 (mode change)
1548368341.388203: nl80211: Subscribe to mgmt frames with non-AP handle 0xb8a160b8
1548368341.388312: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=040a
1548368341.388510: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=040b
1548368341.388617: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=040c
1548368341.388704: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=040d
1548368341.388789: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=090a
1548368341.388872: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=090b
1548368341.388955: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=090c
1548368341.389037: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=090d
1548368341.389120: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=0409506f9a09
1548368341.389204: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=7f506f9a09
1548368341.389285: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=0801
1548368341.389367: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=06
1548368341.389450: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=0a07
1548368341.389532: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=0a11
1548368341.389614: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=1101
1548368341.389695: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=1102
1548368341.389776: nl80211: Register frame type=0xd0 (WLAN_FC_STYPE_ACTION) nl_handle=0xb8a160b8 match=0505
1548368341.389868: nl80211: Key management set PSK
1548368341.389897: nl80211: Connect (ifindex=10)
1548368341.389929: * bssid_hint=00:1c:4a:06:17:f5
1548368341.389959: * freq_hint=2422
1548368341.389985: * SSID - hexdump_ascii(len=6):
74 61 72 61 72 61 tarara
1548368341.390044: * IEs - hexdump(len=22): 30 14 01 00 00 0f ac 02 01 00 00 0f ac 04 01 00 00 0f ac 02 00 00
1548368341.390090: * WPA Versions 0x2
1548368341.390115: * pairwise=0xfac04
1548368341.390139: * group=0xfac02
1548368341.390163: * akm=0xfac02
1548368341.390188: * htcaps - hexdump(len=26): 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1548368341.390236: * htcaps_mask - hexdump(len=26): 63 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
1548368341.390286: * vhtcaps - hexdump(len=12): 00 00 00 00 00 00 00 00 00 00 00 00
1548368341.390321: * vhtcaps_mask - hexdump(len=12): 00 00 00 00 00 00 00 00 00 00 00 00
1548368341.390357: * Auth Type 0
1548368341.410056: nl80211: Connect request send successfully
1548368341.410178: wlan0: Setting authentication timeout: 10 sec 0 usec
1548368341.410219: EAPOL: External notification - EAP success=0
1548368341.410251: EAPOL: External notification - EAP fail=0
1548368341.410278: EAPOL: External notification - portControl=Auto
1548368341.410450: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1
1548368341.541040: nl80211: Event message available
1548368341.541434: nl80211: Drv Event 46 (NL80211_CMD_CONNECT) received for wlan0
1548368341.541556: nl80211: Connect event (status=1 ignore_next_local_disconnect=0)
1548368341.541637: wlan0: Event ASSOC_REJECT (13) received
1548368341.541719: wlan0: CTRL-EVENT-ASSOC-REJECT bssid=00:1c:4a:06:17:f5 status_code=1
1548368341.541799: wlan0: Radio work 'connect'#0xb8a29310 done in 0.169226 seconds
1548368341.541860: Added BSSID 00:1c:4a:06:17:f5 into blacklist
1548368341.541925: Continuous association failures - consider temporary network disabling
1548368341.542001: wlan0: CTRL-EVENT-SSID-TEMP-DISABLED id=0 ssid="tarara" auth_failures=2 duration=20 reason=CONN_FAILED
1548368341.542068: wlan0: Blacklist count 11 --> request scan in 10000 ms
1548368341.542134: wlan0: Setting scan request: 10.000000 sec
1548368341.542205: wlan0: State: ASSOCIATING -> DISCONNECTED
1548368341.542260: nl80211: Set wlan0 operstate 0->0 (DORMANT)
1548368341.542318: netlink: Operstate: ifindex=10 linkmode=-1 (no change), operstate=5 (IF_OPER_DORMANT)
1548368341.543474: EAPOL: External notification - portEnabled=0
1548368341.543583: EAPOL: External notification - portValid=0
1548368341.543641: EAPOL: External notification - EAP success=0
1548368341.548586: dbus: flush_object_timeout_handler: Timeout - sending changed properties of object /fi/w1/wpa_supplicant1/Interfaces/1

Create PCAP file from values in a database

I have a database filled with a lot of logged IPV4 messages. It is used to get queries like: "give me all messages from MacAddress ... that were logged in the period ... to ... that have ..."
Some queries will result in a huge amount of logged messages. Therefore we decided to make a PCAP file if such a request was made.
"Please create a PCAP file containing all logged messages from your
database that ..."
So upon request, my service should fetch the requested data from the database (in pages) and create a PCAP file filled with the data fetched from the database. Later callers can ask for a read-only OWIN stream to this file
The service can create such a file. The problem is that it is not recognized as a proper WireShark file.
I've read Libcap File Format. Whenever I have to create a file filled with LoggedMessages I fill a binary file as follows.
Global Header
Per logged message:
A packet header
Packet data with:
Ethernet Frame: Destination Mac, Source Mac, EtherType (0x800)
IPV4 header
Logged Data
Wireshark starts complaining about the file when it attempts to read the Ethertype. It says this is a Length. Definition of Ethernet Frame with EtherType
So below I show the start of my file. Hexadecimal format per byte + my interpretation of it. After that the comments from wireshark
The created stream starts with the Global Header: a 32 bytes structure. First the hexadecimal values then the interpretation:
=== Global Header ====
D4 C3 B2 A1 02 00 04 00
00 00 00 00 00 00 00 00
FF FF 00 00 01 00 00 00
Magic number A1B2C3D4 (Original Time Precision)
Version: 2 - 4
ThisZone 0
sigFigs 0
snapLen 0000FFFF
datalinkType 1
Note that the magic number has the LSB first, indicating that every multi-byte number will have the least significant byte first. So a 2 byte value of 0x1234 will have in memory first 34 then 12.
After that the Packets should come. Every time one Packet Header, followed by one Packet Data
=== Packet header ===
09 89 58 5A C8 85 0B 00
6B 00 00 00 6B 00 00 00
Timestamp: 1515751689.7551446 (usec precision)
Number of saved bytes (incl_len) 107 bytes (0x006b)
Actual packet length (orig_len) 107 bytes (0x006b)
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 00 08 45 00
5D 00 00 00 00 00 FF 00 - E0 0D 8A 84 77 44 E0 2B
9C FB 4D 43 D5 8A 00 00 - 00 00 41 41 41 41 41 41
41 41 41 41 41 41 41 41 - 41 41 41 41 41 41 41 41
// etc, until total 107 bytes
The packet data consists of a Mac Header, IPV4 header and a couple of 0x41 as data
=== Mac Header ===
Destination Mac: CF:31:59:D3:E7:98
Source Mac: 53:39:17:F0:A9:9C
Ether type: 0800
Note that the magic number showed that every multi-byte number has the LSB first, so the two bytes 00 08 will have a 16-bit meaning of 0x0800
If you look at the PCAP file interpretation I show below, then the problem starts here: the Ether Type is not interpreted as Ether Type, but as length.
After remark in one of the answers, I tried to reverse the two byte ether type from 00 08 into 08 00 (MSB first), but that made the problems worse.
=== IPV4 header ===
- 45 00 5D 00
- 00 00 00 00
- FF 00 E0 0D
- 8A 84 77 44
- E0 2B 9C FB
Specification of the IPV4 header structure
DWORD 0
- bits 00..04: version; bits 04..07 IP Header Length: 04 05
- bits 08..13 DSCP; bits 14..15 ECN: 00
- bits 16..31 Total Length (header + Payload): 93 (005D)
DWORD 1
- bits 00..15 Identification: 0000
- bits 16..18 Flags; bits 19..31 offset: 0000
DWORD 2
- bits 00..07 Time to Live FF
- bits 08..15 Protocol; used protocol 00
- bits 16..31 Header Checksum 3552 (0DE0)
DWORD 3 and 4
Source IP: 138.132.119.68
Destination IP: 224.43.156.251
Bacause wireshark complains about checksum, I verify as follows:
Verify checksum:
Header: 0045 005D 0000 0000 00FF 0DE0 848A 4477 2BE0 FB9C
69 + 93 + 0 + 0 + 255 + 3552 + 33930 + 17527 + 11232 + 64412 = 131070 (01FFFE)
0001 + FFFE = FFFF
1's complement: 0000 (checksum ok)
This is what WireShark (version 2.4.4) makes of it:
The following seems normal:
Frame 1: 107 bytes on wire (856 bits), 107 bytes captured (856 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Jan 12, 2018 11:08:09.755144000 W. Europe Standard Time
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1515751689.755144000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 107 bytes (856 bits)
Capture Length: 107 bytes (856 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:llc:data]
[Coloring Rule Name: Checksum Errors]
[Coloring Rule String [truncated]: eth.fcs.status=="Bad" ||
ip.checksum.status=="Bad" || tcp.checksum.status=="Bad" ||
udp.checksum.status=="Bad" || sctp.checksum.status=="Bad" ||
mstp.checksum.status=="Bad" || cdp.checksum.status=="Bad" ||]
Here comes the first problem: EtherType is interpreted as Length
IEEE 802.3 Ethernet
Destination: cf:31:59:d3:e7:98 (cf:31:59:d3:e7:98)
Source: 53:39:17:f0:a9:9c (53:39:17:f0:a9:9c)
Length: 8
Padding: ff00e00d8a847744e02b9cfb4d43d58a0000000041414141...
Trailer: 414141414141414141414141414141414141414141414141...
Frame check sequence: 0x41414141 incorrect, should be 0xe19cae36
[FCS Status: Bad]
After the length, which I meant as an EtherType, comes a lot of padding, instead of interpretation of my 5 DWORDs.
The link to the Ethernet Frame in wikipedia I showed says:
The EtherType field is two octets long and it can be used for two
different purposes. Values of 1500 and below mean that it is used to
indicate the size of the payload in octets, while values of 1536 and
above indicate that it is used as an EtherType, to indicate which
protocol is encapsulated in the payload of the frame.
My value if 0x0800 = 2048. This certainly is above 1536
For example, an EtherType value of 0x0800 signals that the frame
contains an IPv4 datagram.
If value 0x0800 the incorrect value? Or is my error somewhere else?
Looks like your ethertype has the wrong byte order. It should be:
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 08 00 XX XX

Initial Image bytes

If I debug an asm file with Windbg the image is loaded at a location in memory, in my case its 00400000. If I dump the data from 00400000 I can see my assembly code is loaded in the image file at and offset of 10000 from this address and not at 00400000.
Here's the dump for the start of the image.
00400000 -> 00905a4d 00000003 00000004 0000ffff
00400010 -> 000000b8 00000000 00000040 00000000
00400020 -> 00000000 00000000 00000000 00000000
00400030 -> 00000000 00000000 00000000 000000c0
So, my question is what is this header at the start of the image and what is it used for? It looks like all images have this header.
Thanks in advance.
What you have posted looks like a PE header in 32 bit format:
0:000> dd 00400000
00400000 00905a4d 00000003 00000004 0000ffff
00400010 000000b8 00000000 00000040 00000000
00400020 00000000 00000000 00000000 00000000
00400030 00000000 00000000 00000000 00000080
If you dump it as bytes, it will show the ASCII characters along with it and you can easily recognize the "MZ" header.
0:000> db 00400000
00400000 4d 5a 90 00 03 00 00 00-04 00 00 00 ff ff 00 00 MZ..............
00400010 b8 00 00 00 00 00 00 00-40 00 00 00 00 00 00 00 ........#.......
00400020 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
00400030 00 00 00 00 00 00 00 00-00 00 00 00 80 00 00 00 ................
You can use the !dh command to decode it:
0:000> !dh 00400000
File Type: EXECUTABLE IMAGE
FILE HEADER VALUES
14C machine (i386)
3 number of sections
5911398E time date stamp Mon May 8 20:37:50 2017
0 file pointer to symbol table
0 number of symbols
E0 size of optional header
22 characteristics
Executable
App can handle >2gb addresses
[...]
If you don't want to use WinDbg, there's Stud_PE where you can load an executable and it will analyze it for you.