WM-Bus extended layer decoding - aes

I am trying to decrypt wm-bus telegram from Kamstrup Multical21 in C1 mode with Extended Link Layer.
The payload together with ELL info is following:
23 44 2D 2C 45 45 71 63 1B 16 8D 20 6A 31 FB 7C 20 39 A3 79 60 4B 90 BD FC BE 8D D8 CB 18 CE 77 DC 41 CE 8C
Analysing CI = 8D I found that there is a ELL with following data:
CI (1 byte) CC(1 byte) ACC(1 byte) SN(4 bytes) CRC(2 bytes)
8D 20 6A 31 FB 7C 20 39 A3
The documentation says that the buffer which should be decrypted shall contain CRC from ELL, i.e:
39 A3 79 60 4B 90 BD FC BE 8D D8 CB 18 CE 77 DC 41 CE 8C
I have got the AES key from the Manufacturer:
B9 7A 6D 4E C2 74 A4 6D 87 0E 31 27 D9 A0 AF 63
Initialization vector for ELL shall be:
M-field A-field CC-field SN-field FN BC
2D 2C 45 45 71 63 1B 16 20 31 FB 7C 20 00 00 00
After decrypting, I get the following result:
08 3a 5f ce b2 8d 51 97 94 a2 5b fb 61 ab 2e c0
e4 20 c8 2a 43 ff 3a 75 6f 93 d0 ac 8c 79 b7 a1
Since there is no 2F 2F in the beginning, something is wrong!
Can somebody help me and tell what I have done wrong?
Thanks in advance.

I had a look in the latest Kamstrup docs ("Wireless M-Bus Communication Kamstrup Water Meters - MULTICAL® 21 and flowIQ® water meters Mode C1 according to EN 13757-4:2013")
When I decrypt your packet I find:
25877968217E8E01000000000000000000
Firstly, it seems the Kamstrup decrypted packets does not start with 2F 2F.
The first 2 bytes of the decrypted packet is supposedly the PLCRC (I can't confirm that right now - don't have immediate access to the standard that defines the crc polynomial algorithm), and then the next byte is 79, which means it is a Compact Frame, then the next 4 bytes are 2 more CRCs, and then the next 2 bytes 0100 is probably the Info, which is manufacturer specific and I don't know how to interpret that yet.
This meter is probably R type 1, right? (on the face place, the "Con.:" parameter's 3rd last digit should be a 1) So its format would be [Info][Volume][Target Volume] - 2 bytes, 4 bytes, 4 bytes - I kind of assume that, since this packet is a compact packet, so I don't get the actual format the long packet would have, e.g. number of decimals - which normally you'd need - but your values are zeroes? so decimals doesn't matter. (the 'long' packet of course is every 6th packet or so?)
The IV I get is:
2D2C454571631B162031FB7C20000000
which is exactly the same as yours.
The encrypted packet I use is:
39A379604B90BDFCBE8DD8CB18CE77DC41
so I exclude the CE and 8C you had on yours?
When I put them in, the decrypted packet becomes:
25877968217E8E01000000000000000000BB49
which is pretty much the same packet with some more crc stuff at the back, I suspect, so I really do not get what you do to decrypt, since your result is completely different?
Ok, maybe you use AES/CBC/NoPadding, as in OpenMUC.
Kamstrup uses AES/CTR/NoPadding. That is how they don't have to decrypt multiples of 16 byte blocks? The way that looks in my Java code is as follows:
Cipher cipher = Cipher.getInstance("AES/CTR/NoPadding");

the hints here are very helpfull. There's one obstacle I stumbled across with the given message. The Length-Field is wrong and there are 2 bytes of garbage at the end.
I guess the original message was encoded in frame format B. That means the length field includes the frame CRCs and should be corrected after the CRCs are removed. After correcting the length to 0x21 (33 bytes + L-Field), I get the correct message and also can verify that the first 2 bytes of the decoded message contain the CRC16 of the remaining message.

Related

Random symbols in Source window instead of Russian characters in RStudio

I have been googling and stackoverflowing (yes, that is the word now) on how to fix the problem with wrong encoding. However, I could not find the solution.
I am trying to load .Rmd file with UTF-8 encoding which basically has Russian characters in it. They do not show properly. Instead, the code lines in the Source window look like so:
Initially, I created this .Rmd file long ago on my previous laptop. Now, I am using another one and I cannot spot the issue here.
I have already tried to use some Sys.setlocale() commands with no success whatsoever.
I run RStudio on Windows 10.
Edited
This is the output of readBin('raw[1].Rmd', raw(), 10000). Slice from 2075 to 2211:
[2075] 64 31 32 2c 20 71 68 35 20 3d 3d 20 22 d0 a0 d1 9a d0 a0 d0 88 d0 a0
e2 80 93 d0 a0 d0 8e d0 a0 d1 99
[2109] d0 a0 d1 9b d0 a0 e2 84 a2 22 29 3b 20 64 31 32 6d 24 71 68 35 20 3d
20 4e 55 4c 4c 0d 0a 64 31 35 6d
[2143] 20 3d 20 66 69 6c 74 65 72 28 64 31 35 2c 20 74 68 35 20 3d 3d 20 22
d0 a0 d1 9a d0 a0 d0 88 d0 a0 e2
[2177] 80 93 d0 a0 d0 8e d0 a0 d1 99 d0 a0 d1 9b d0 a0 e2 84 a2 22 29 3b 20
64 31 35 6d 24 74 68 35 20 3d 20
Thank you.
Windows doesn't have very good support for UTF-8. Likely your local encoding is something else.
RStudio normally reads files using the system encoding. If that is wrong, you can use "File | Reopen with encoding..." to re-open the file using a different encoding.
Edited to add:
The first line of the sample output looks like UTF-8 encoding with some Cyrillic letters, but not Russian-language text. I decode it as "d12, qh5 == \"РњРЈР–РЎРљ". Is that what RStudio gave you when you re-opened the file, declaring it as UTF-8?

Create PCAP file from values in a database

I have a database filled with a lot of logged IPV4 messages. It is used to get queries like: "give me all messages from MacAddress ... that were logged in the period ... to ... that have ..."
Some queries will result in a huge amount of logged messages. Therefore we decided to make a PCAP file if such a request was made.
"Please create a PCAP file containing all logged messages from your
database that ..."
So upon request, my service should fetch the requested data from the database (in pages) and create a PCAP file filled with the data fetched from the database. Later callers can ask for a read-only OWIN stream to this file
The service can create such a file. The problem is that it is not recognized as a proper WireShark file.
I've read Libcap File Format. Whenever I have to create a file filled with LoggedMessages I fill a binary file as follows.
Global Header
Per logged message:
A packet header
Packet data with:
Ethernet Frame: Destination Mac, Source Mac, EtherType (0x800)
IPV4 header
Logged Data
Wireshark starts complaining about the file when it attempts to read the Ethertype. It says this is a Length. Definition of Ethernet Frame with EtherType
So below I show the start of my file. Hexadecimal format per byte + my interpretation of it. After that the comments from wireshark
The created stream starts with the Global Header: a 32 bytes structure. First the hexadecimal values then the interpretation:
=== Global Header ====
D4 C3 B2 A1 02 00 04 00
00 00 00 00 00 00 00 00
FF FF 00 00 01 00 00 00
Magic number A1B2C3D4 (Original Time Precision)
Version: 2 - 4
ThisZone 0
sigFigs 0
snapLen 0000FFFF
datalinkType 1
Note that the magic number has the LSB first, indicating that every multi-byte number will have the least significant byte first. So a 2 byte value of 0x1234 will have in memory first 34 then 12.
After that the Packets should come. Every time one Packet Header, followed by one Packet Data
=== Packet header ===
09 89 58 5A C8 85 0B 00
6B 00 00 00 6B 00 00 00
Timestamp: 1515751689.7551446 (usec precision)
Number of saved bytes (incl_len) 107 bytes (0x006b)
Actual packet length (orig_len) 107 bytes (0x006b)
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 00 08 45 00
5D 00 00 00 00 00 FF 00 - E0 0D 8A 84 77 44 E0 2B
9C FB 4D 43 D5 8A 00 00 - 00 00 41 41 41 41 41 41
41 41 41 41 41 41 41 41 - 41 41 41 41 41 41 41 41
// etc, until total 107 bytes
The packet data consists of a Mac Header, IPV4 header and a couple of 0x41 as data
=== Mac Header ===
Destination Mac: CF:31:59:D3:E7:98
Source Mac: 53:39:17:F0:A9:9C
Ether type: 0800
Note that the magic number showed that every multi-byte number has the LSB first, so the two bytes 00 08 will have a 16-bit meaning of 0x0800
If you look at the PCAP file interpretation I show below, then the problem starts here: the Ether Type is not interpreted as Ether Type, but as length.
After remark in one of the answers, I tried to reverse the two byte ether type from 00 08 into 08 00 (MSB first), but that made the problems worse.
=== IPV4 header ===
- 45 00 5D 00
- 00 00 00 00
- FF 00 E0 0D
- 8A 84 77 44
- E0 2B 9C FB
Specification of the IPV4 header structure
DWORD 0
- bits 00..04: version; bits 04..07 IP Header Length: 04 05
- bits 08..13 DSCP; bits 14..15 ECN: 00
- bits 16..31 Total Length (header + Payload): 93 (005D)
DWORD 1
- bits 00..15 Identification: 0000
- bits 16..18 Flags; bits 19..31 offset: 0000
DWORD 2
- bits 00..07 Time to Live FF
- bits 08..15 Protocol; used protocol 00
- bits 16..31 Header Checksum 3552 (0DE0)
DWORD 3 and 4
Source IP: 138.132.119.68
Destination IP: 224.43.156.251
Bacause wireshark complains about checksum, I verify as follows:
Verify checksum:
Header: 0045 005D 0000 0000 00FF 0DE0 848A 4477 2BE0 FB9C
69 + 93 + 0 + 0 + 255 + 3552 + 33930 + 17527 + 11232 + 64412 = 131070 (01FFFE)
0001 + FFFE = FFFF
1's complement: 0000 (checksum ok)
This is what WireShark (version 2.4.4) makes of it:
The following seems normal:
Frame 1: 107 bytes on wire (856 bits), 107 bytes captured (856 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Jan 12, 2018 11:08:09.755144000 W. Europe Standard Time
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1515751689.755144000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 107 bytes (856 bits)
Capture Length: 107 bytes (856 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:llc:data]
[Coloring Rule Name: Checksum Errors]
[Coloring Rule String [truncated]: eth.fcs.status=="Bad" ||
ip.checksum.status=="Bad" || tcp.checksum.status=="Bad" ||
udp.checksum.status=="Bad" || sctp.checksum.status=="Bad" ||
mstp.checksum.status=="Bad" || cdp.checksum.status=="Bad" ||]
Here comes the first problem: EtherType is interpreted as Length
IEEE 802.3 Ethernet
Destination: cf:31:59:d3:e7:98 (cf:31:59:d3:e7:98)
Source: 53:39:17:f0:a9:9c (53:39:17:f0:a9:9c)
Length: 8
Padding: ff00e00d8a847744e02b9cfb4d43d58a0000000041414141...
Trailer: 414141414141414141414141414141414141414141414141...
Frame check sequence: 0x41414141 incorrect, should be 0xe19cae36
[FCS Status: Bad]
After the length, which I meant as an EtherType, comes a lot of padding, instead of interpretation of my 5 DWORDs.
The link to the Ethernet Frame in wikipedia I showed says:
The EtherType field is two octets long and it can be used for two
different purposes. Values of 1500 and below mean that it is used to
indicate the size of the payload in octets, while values of 1536 and
above indicate that it is used as an EtherType, to indicate which
protocol is encapsulated in the payload of the frame.
My value if 0x0800 = 2048. This certainly is above 1536
For example, an EtherType value of 0x0800 signals that the frame
contains an IPv4 datagram.
If value 0x0800 the incorrect value? Or is my error somewhere else?
Looks like your ethertype has the wrong byte order. It should be:
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 08 00 XX XX

How to unpack NTP UDP packet using pcap

I can read UDP packet using
void my_callback(u_char *useless, const struct pcap_pkthdr* pkthdr, const u_char* packet)
I have hexa output of my packet:
08 00 27 E5 B5 3B 52 54 00 12 35 02 08 00 45 00 00 4C 7C E7 00 00 40 11 3C 28 5B BD 59 C6 0A 00 02 0F 00 7B 00 7B 00 38 B7 9D 24 02 03 E8 00 00 04 A8 00 00 07 51 83 BC 03 DC DC C5 CC 47 F1 F1 69 C3 DC C5 CF 37 D2 5F A7 F5 DC C5 CF 38 3C 2D C2 CF DC C5 CF 38 3C 32 0B 9A
I know, that it is NTP packet.
How can I extrath data? Cut ethernet frames, etc..
Thank you for your help.
I am using pcap c++.
If you read pcap you get raw packet from the network device. Several options may be there:
Packet is read from ethernet device
Packet is read from vlan device
Packet is read from some other device
What kind of device is used during pcap defines what protocol header is first in your packet. To know it you can look at link layer type field of global pcap header.
Once you defined first protocol header you need to open protocol specification and find:
Size of header (in your case it looks like regular ethenet header - 14 bytes 08 00 27 E5 B5 3B 52 54 00 12 35 02 08 00)
How to find encapsulated packet type (in your case last 08 00 means IP)
Once you found IP header (45 00 00 4C 7C E7 00 00 40 11 ...) you can determine IP header length:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL |Type of Service| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time to Live | Protocol | Header Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Address |
Here you need:
IHL defines sizeof IP header. This is lower 4 bits of first byte of IP header. In your case it is 0x5. This means 5 words or 20 bytes.
Protocol defines what data is encapsulated in IP header. In your case 0x11 (IPPROTO_UDP)
After that you can get UDP header (8 bytes) check ports if you need it and parse NTP header in according to NTP specification.
In your example total shift of the NTP header will be 14+20+8 bytes.

Inspecting binary over sockets

I'm using WireShark to inspect data sent/received over a web-socket, however, all I see is nonsense.
0000 1c 74 0d 7d 42 24 d8 5d e2 26 c1 7d 08 00 45 00 .t.}B$.].&.}..E.
0010 00 3c 75 4e 40 00 80 06 22 eb c0 a8 01 c0 4f 89 .<uN#...".....O.
0020 50 91 c4 f1 0f 78 72 e5 d0 f4 ea 5e 6e e2 50 18 P....xr....^n.P.
0030 00 40 91 b3 00 00 c2 8e 6d 06 87 95 7f 76 78 62 .#......m....vxb
0040 92 f9 54 2a 92 f9 b4 95 6c 06 ..T*....l.
I've seen this type of output before. The left is a line of binary, and the right is the decoded string (ASCII), right?
Is this data obfuscated/encrypted?
Is it possible to get cogent information from my socket?
Also, what do the [FIN] and [MASKED] flags mean?
If you copy and paste the data you supplied into a text file and append a line beginning with 0050 with nothing following it, you can then run text2pcap -a infile.txt outfile.pcap to convert the data to a pcap file that Wireshark can read and decode for you.
See the text2pcap man page for more information about this tool.
I have done this and the packet appears to just be a simple TCP segment. There is no [FIN] or [MASKED] flag, only PSH and ACK. For information about these TCP flags, refer to RFC 793, section 3.1, as well as the other RFC's mentioned at the top, which update this one.

Decoding an arbitrary block of NSData?

If I have an arbitrary block of NSData as a hex value, is there a way to determine what the object might have been before it was archived or serialized? I don't mind a few guess and check methods, but I need some pointers in the right direction.
I have an NSData object with some hex in it. What methods of NSData should I look at? Are there other classes to try as well
Don't want to scare people away from answering, but I have a file of game data which was likely encoded using a Cocoa Touch class. The data, when viewed in a hex editor, shows gibberish and a username, which leads me to suspect that it's an archived or encoded object of sorts. I have copied the hex from the hex editor into a sample project which I am using to try and unarchive the data.
I don't believe this is related to the 3d format, the file extension is arbitrary.
Here's the data. I'm hoping it doesn't get lost in translation:
'µköXN[ÎÀü÷h/F9ó9Vìñ°ceE¸z¶=Hmoshbermú«ó¼Ppù#ÝVÔ=4â®L,K;Êç;ASÀ&Ë÷ëÓ%È;Úf¬G}tmQ;µéüø_87´y©ã©!߶óQòAçÛl©âSG4S½3ýJת9äô¡wxiD²M¼ÏB]39øþ:óñ7ª¾÷躣È3Ï¢ÍEFÍ¢ª»r]BmÁ'Ò+åygÞÅQ?luó>÷ú¼è6¸|}[¼[¶Ñ¦g!\OÎÒJSE..pSß&_ÈEäø)6òëó¨¼2¶ð°æà`ï7Ë=Ã¥:cƧ=L4qG-"µ(ÐÝïß ÓãXkÀ4fzæ·p\ññT<tu¥Æ©;Ìn4£³Ï¢ÌFåG´
And the corresponding hex:
27 B5 6B F6 01 00 00 00 58 4E 5B CE C0 FC F7 68 2F 46 86 87 83 39 F3 39 9E 56 EC F1 B0 63 9E 65 45 B8 7A B6 3D 07 99 48 6D 6F 73 68 62 65 72 6D 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90 86 FA 03 0E AB F3 BC 0B 50 70 F9 23 DD 87 56 03 D4 3D 34 90 E2 AE 4C 2C 94 9E 8E 15 4B 0C 83 8C 3B 03 CA E7 3B 1B 41 53 C0 26 04 CB F7 EB D3 25 C8 3B DA 66 8A AC 47 7D 8A 7F 74 6D 51 3B B5 19 E9 FC F8 5F 38 37 B4 11 0C 79 A9 12 E3 A9 21 DF B6 F3 51 F2 41 E7 DB 85 02 9F 6C A9 E2 53 47 1F 34 86 53 BD 33 FD 4A D7 AA 39 C3 A4 F4 A1 77 78 69 44 B2 4D BC CF 42 5D 33 39 F8 FE 97 3A 81 F3 F1 10 37 AA BE 86 91 F7 1F E8 83 BA A3 C8 33 CF 1D A2 CD 45 7F 46 1F CD A2 AA BB 1A 72 5D 42 02 6D C1 0F 27 D2 2B E5 0B 79 67 DE C5 1A 51 3F 14 6C 75 F3 3E F7 FA BC E8 36 8E B8 7C 02 1C 7D 01 00 92 8C 19 5B BC 5B B6 D1 A6 67 7F 21 5C 84 13 4F CE 0C D2 4A 53 19 82 45 1B 2E 2E 96 70 53 DF 26 5F C8 1C 45 8F E4 F8 29 36 F2 EB 9D 95 F3 A8 BC 32 B6 F0 B0 E6 91 98 1A E0 99 60 EF 37 CB 3D C3 A5 3A 63 0C C6 A7 3D 4C 34 71 47 2D 22 B5 28 D0 DD EF DF 09 D3 E3 58 6B C0 17 34 66 7A E6 B7 70 5C F1 F1 54 3C 74 94 75 A5 C6 15 A9 9E 14 3B CC 15 10 83 6E 34 A3 B3 CF 0F A2 9C CC 8E 46 8C E5 00 00 47 B4 17 05 00 00 00 00
If anyone cares to help figure this out it would be much appreciated.
If I have an arbitrary block of NSData as a hex value, is there a way to determine what the object might have been before it was archived or serialized?
Not really. That's about as 'trivial' as reading arbitrary files correctly without the use of a UTI, extension, MIME type. Of course, your program would also need to support reading of all those files/formats.
I don't mind a few guess and check methods, but I need some pointers in the right direction.
You need to narrow your problem/inputs down, if you don't want an impossibly difficult task.
I have an NSData object with some hex in it. What methods of NSData should I look at?
It's just a data blob of length bytes. It could represent anything -- if you don't know where it came from.
Are there other classes to try as well?
Perhaps you would start by saving all your data via NSCoder or another serializer/archiver which offers some introspection and support for you to enter your own information (which would be comparable to a UTI or MIME type).
Edit:
Don't want to scare people away from answering, but I have a file of game data which was likely encoded using a Cocoa Touch class. The data, when viewed in a hex editor, shows gibberish and a username, which leads me to suspect that it's an archived or encoded object of sorts. I have copied the hex from the hex editor into a sample project which I am using to try and unarchive the data.
Using these APIs, the data may be represented multiple ways. You're probably facing something within the domain of 1) a proprietary file format through 2) a keyed archive.
The latter is easier for nontrivial data representations. You would need to define any objc classes you do not have available when unarchived. In that case, a few sample representations would offer a rough outline of the data structures you will need (under conventional implementations). It could also be an archive similar to an NSDictionary, if the unarchiver is capable of opening it. This is a problem which is easier than with other langs, since archiving often falls back on keys and values mapped to members in Cocoa.
Edit2:
The file came from the Draw Something directory. It's called gamedata.i3d
(shrug)
Try using NSKeyedUnarchiver to read it. It's not uncommon to use just the standard Foundation containers like NSArray, NSDictionary, and NSString to store data, so you might get lucky. That obviously won't work if custom classes were involved, but it might be worth 15 minutes of your time to try it.