I have a VBscript code that processes utf-8 files.
It works perfectly. Except there is a problem with the source files in that sometimes the input (despite the script is clearly labeled to be used for utf-8 files) is unicode-LE.
This then creates a corrupt output of course. I am putting in a check for the BOM to ensure no Unicode-LE files are opened incorrectly. But I already have files that got corrupted this way.
Is there a way to seamlessly revert the damage? Meaning to read it back "incorrectly" and saving it correctly?
Here is the code:
Private Sub UnicodeToUTF8(ByVal InFName, ByVal OutFName)
Dim strText
With CreateObject("ADODB.Stream")
.Open
.Type = adTypeBinary
.LoadFromFile InFName
.Type = adTypeText
.Charset = "utf-8"
'Read Unicode source file
strText = .ReadText(adReadAll)
'Process file
strText = OffsetTCs(strText)
'Output UTF-8 file
.Position = 0
.SetEOS
.Charset = "utf-8"
.WriteText strText, adWriteChar
.SaveToFile OutFName, adSaveCreateOverWrite
.Close
End With
End Sub
Edit:
I tried this script to save the day, but it reports an error on the file.Write data line. It does show the ASCII content properly in the message box, but not the Chinese characters:
Dim fso, file, data
Set fso = CreateObject("Scripting.FileSystemObject")
Set file = fso.OpenTextFile("damaged_Test.sub", 1, False, -1)
data = ""
data = file.ReadAll
MsgBox(data)
Set file = fso.OpenTextFile("output.txt", 2, True)
file.Write data
Here is the hex dump of the damaged file:
EF BB BF EF BF BD EF BF BD 5B 00 53 00 63 00 72 00 69 00 70 00 74 00 20 00 49 00 6E 00 66 00 6F 00 5D 00 0D 00 0A 00 3B 00 0D 00 0A 00 54 00 69 00 74 00 6C 00 65 00 3A 00 20 00 20 00 28 00 29 00 0D 00 0A 00 4F 00 72 00 69 00 67 00 69 00 6E 00 61 00 6C 00 20 00 53 00 63 00 72 00 69 00 70 00 74 00 3A 00 20 00 0D 00 0A 00 4F 00 72 00 69 00 67 00 69 00 6E 00 61 00 6C 00 20 00 54 00 69 00 6D 00 69 00 6E 00 67 00 3A 00 20 00 0D 00 0A 00 53 00 63 00 72 00 69 00 70 00 74 00 54 00 79 00 70 00 65 00 3A 00 20 00 76 00 34 00 2E 00 30 00 38 00 0D 00 0A 00 43 00 6F 00 6C 00 6C 00 69 00 73 00 69 00 6F 00 6E 00 73 00 3A 00 20 00 4E 00 6F 00 72 00 6D 00 61 00 6C 00 0D 00 0A 00 50 00 6C 00 61 00 79 00 52 00 65 00 73 00 58 00 3A 00 20 00 31 00 32 00 38 00 30 00 0D 00 0A 00 50 00 6C 00 61 00 79 00 52 00 65 00 73 00 59 00 3A 00 20 00 37 00 32 00 30 00 0D 00 0A 00 50 00 6C 00 61 00 79 00 44 00 65 00 70 00 74 00 68 00 3A 00 20 00 30 00 0D 00 0A 00 54 00 69 00 6D 00 65 00 72 00 3A 00 20 00 31 00 30 00 30 00 2E 00 30 00 30 00 30 00 30 00 0D 00 0A 00 0D 00 0A 00 5B 00 56 00 34 00 20 00 53 00 74 00 79 00 6C 00 65 00 73 00 5D 00 0D 00 0A 00 46 00 6F 00 72 00 6D 00 61 00 74 00 3A 00 20 00 4E 00 61 00 6D 00 65 00 2C 00 20 00 46 00 6F 00 6E 00 74 00 6E 00 61 00 6D 00 65 00 2C 00 20 00 46 00 6F 00 6E 00 74 00 73 00 69 00 7A 00 65 00 2C 00 20 00 50 00 72 00 69 00 6D 00 61 00 72 00 79 00 43 00 6F 00 6C 00 6F 00 75 00 72 00 2C 00 20 00 53 00 65 00 63 00 6F 00 6E 00 64 00 61 00 72 00 79 00 43 00 6F 00 6C 00 6F 00 75 00 72 00 2C 00 20 00 54 00 65 00 72 00 74 00 69 00 61 00 72 00 79 00 43 00 6F 00 6C 00 6F 00 75 00 72 00 2C 00 20 00 42 00 61 00 63 00 6B 00 43 00 6F 00 6C 00 6F 00 75 00 72 00 2C 00 20 00 42 00 6F 00 6C 00 64 00 2C 00 20 00 49 00 74 00 61 00 6C 00 69 00 63 00 2C 00 20 00 42 00 6F 00 72 00 64 00 65 00 72 00 53 00 74 00 79 00 6C 00 65 00 2C 00 20 00 4F 00 75 00 74 00 6C 00 69 00 6E 00 65 00 2C 00 20 00 53 00 68 00 61 00 64 00 6F 00 77 00 2C 00 20 00 41 00 6C 00 69 00 67 00 6E 00 6D 00 65 00 6E 00 74 00 2C 00 20 00 4D 00 61 00 72 00 67 00 69 00 6E 00 4C 00 2C 00 20 00 4D 00 61 00 72 00 67 00 69 00 6E 00 52 00 2C 00 20 00 4D 00 61 00 72 00 67 00 69 00 6E 00 56 00 2C 00 20 00 41 00 6C 00 70 00 68 00 61 00 4C 00 65 00 76 00 65 00 6C 00 2C 00 20 00 45 00 6E 00 63 00 6F 00 64 00 69 00 6E 00 67 00 0D 00 0A 00 53 00 74 00 79 00 6C 00 65 00 3A 00 20 00 50 00 75 00 62 00 6C 00 69 00 63 00 2C 00 44 00 46 00 4B 00 61 00 69 00 53 00 68 00 75 00 20 00 53 00 74 00 64 00 20 00 57 00 35 00 2C 00 35 00 32 00 2C 00 31 00 32 00 35 00 37 00 31 00 38 00 37 00 32 00 2C 00 31 00 32 00 35 00 37 00 31 00 38 00 37 00 32 00 2C 00 31 00 32 00 35 00 37 00 31 00 38 00 37 00 32 00 2C 00 2D 00 32 00 31 00 34 00 37 00 34 00 38 00 33 00 36 00 34 00 30 00 2C 00 2D 00 31 00 2C 00 30 00 2C 00 31 00 2C 00 33 00 2C 00 33 00 2C 00 32 00 2C 00 31 00 32 00 38 00 2C 00 31 00 32 00 38 00 2C 00 37 00 32 00 2C 00 30 00 2C 00 31 00 33 00 36 00 0D 00 0A 00 0D 00 0A 00 5B 00 45 00 76 00 65 00 6E 00 74 00 73 00 5D 00 0D 00 0A 00 46 00 6F 00 72 00 6D 00 61 00 74 00 3A 00 20 00 4D 00 61 00 72 00 6B 00 65 00 64 00 2C 00 20 00 53 00 74 00 61 00 72 00 74 00 2C 00 20 00 45 00 6E 00 64 00 2C 00 20 00 53 00 74 00 79 00 6C 00 65 00 2C 00 20 00 4E 00 61 00 6D 00 65 00 2C 00 20 00 4D 00 61 00 72 00 67 00 69 00 6E 00 4C 00 2C 00 20 00 4D 00 61 00 72 00 67 00 69 00 6E 00 52 00 2C 00 20 00 4D 00 61 00 72 00 67 00 69 00 6E 00 56 00 2C 00 20 00 45 00 66 00 66 00 65 00 63 00 74 00 2C 00 20 00 54 00 65 00 78 00 74 00 0D 00 0A 00 44 00 69 00 61 00 6C 00 6F 00 67 00 75 00 65 00 3A 00 20 00 4D 00 61 00 72 00 6B 00 65 00 64 00 3D 00 30 00 2C 00 30 00 3A 00 30 00 30 00 3A 00 30 00 38 00 2E 00 36 00 30 00 2C 00 30 00 3A 00 30 00 30 00 3A 00 31 00 31 00 2E 00 31 00 30 00 2C 00 50 00 75 00 62 00 6C 00 69 00 63 00 2C 00 30 00 31 00 2C 00 30 00 30 00 30 00 30 00 2C 00 30 00 30 00 30 00 30 00 2C 00 30 00 30 00 30 00 30 00 2C 00 2C 00 EF BF BD 65 74 5E EF BF BD 5F 02 6A 0D 00 0A 00 44 00 69 00 61 00 6C 00 6F 00 67 00 75 00 65 00 3A 00 20 00 4D 00 61 00 72 00 6B 00 65 00 64 00 3D 00 30 00 2C 00 30 00 3A 00 30 00 30 00 3A 00 31 00 31 00 2E 00 32 00 30 00 2C 00 30 00 3A 00 30 00 30 00 3A 00 31 00 35 00 2E 00 31 00 33 00 2C 00 50 00 75 00 62 00 6C 00 69 00 63 00 2C 00 30 00 31 00 2C 00 30 00 30 00 30 00 30 00 2C 00 30 00 30 00 30 00 30 00 2C 00 30 00 30 00 30 00 30 00 2C 00 2C 00 EF BF BD 65 74 5E EF BF BD 5F 02 6A 0D 00 0A
Related
Sample of my df is:
+------------------------------------------------------------------------------------------------------------------+
|id | binary_col |
+------------------------------------------------------------------------------------------------------------------+
| 1 | [08 01 10 0D 00 0E CC 93 01 00 00 00 01 00 00 00 00 00 00 00 80 FF BF 40 00 00 00 00 00 00 F0 3F BE 2B 00 00]|
| 2 | [08 01 10 0D 00 0E CC 93 01 00 00 00 01 00 00 00 00 00 00 00 F0 FF BF 40 00 00 00 00 00 00 F0 3F 57 66 00 00]|
| 3 | [08 01 10 0D 00 0E CC 93 01 00 00 00 01 00 00 00 00 00 00 00 C0 FF BF 40 00 00 00 00 00 00 F0 3F D5 69 00 00]|
| 4 | [08 01 10 0D 00 0E CC 93 01 00 00 00 01 00 00 00 00 00 00 00 80 FF BF 40 00 00 00 00 00 00 F0 3F 5A 60 00 00]|
+------------------------------------------------------------------------------------------------------------------+
with these schema (df.printSchema())
|-- id: int (nullable = true)
|-- binary_col: binary (nullable = true)
And I want to filter only the values with [08 01 10 0D 00 0E CC 93 01 00 00 00 01 00 00 00 00 00 00 00 80 FF BF 40 00 00 00 00 00 00 F0 3F BE 2B 00 00] (It doesn't work filtering id=1 because there are other ids in the df)
I've tried to cast binary to bigint to filter later like here: Spark: cast bytearray to bigint
by doing df.withColumn('casted_bin', F.conv(F.hex(F.col("binary_col")), 16, 10).cast("bigint")).show(truncate=False) but it didn't work.
How can I filter any kind of binary data type?
Note: I had asked previously here (How to filter Pyspark column with binary data type?) but it was a very simple binary data and the answer generated the binary from a numeric value while now I don't know how to generate the numeric value.
I'm trying to reverse engineer the CAN bus codes generated by the steering wheel of a car. I'm interested in a single message, the one that holds the information about what button of the steering wheel has been pressed. I have done that for many different cars but this one is special (?)
I found the message that holds the data has for ID 0x3D4. So I logged this specific message many times with/without a different combination of the steering wheel buttons.
The way car works (most of the time) is that a single bit will toggle from 0 to 1 when a button is held down. And go back to 0 when the button is released. This message has a DLC of 8 so there would be potentially enough room for 64 buttons.
Now for this car, they have used 4 bits of the second byte for a counter counting from 0 to F. The first byte is the one that is causing a problem. I don't understand how it is generated. It seems to be dependant on the value of the other bytes but does not follow a standard CRC algo... It seems that it also depends on the position of the bytes... Pressing on buttons in the car will change a bit in the data which then changes the value of the first byte. If I generate a fake command to the car without changing the first byte, the car will not react to it.
I would like to find how this first byte is generated so I can generate it myself.
The DLC: 8
CAN bus high speed 500kb/s
CA 0E 80 00 00 04 00 00
1F 0F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
4A 01 81 00 00 04 00 00
01 02 81 00 00 04 00 00
4C 03 81 00 00 04 00 00
DC 04 81 00 00 04 00 00
37 05 81 00 00 04 00 00
1E 06 81 00 00 04 00 00
8B 07 80 00 00 04 00 00
25 08 80 00 00 04 00 00
70 09 80 00 00 04 00 00
6B 0A 80 00 00 04 00 00
9B 0B 81 00 00 04 00 00
BE 0C 81 00 00 04 00 00
41 0D 81 00 00 04 00 00
3E 0E 81 00 00 04 00 00
1F 0F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
F5 02 80 00 00 04 00 00
B8 03 80 00 00 04 00 00
I also found these following messages that share the same first byte:
7F 2D 84 00 00 04 00 00
7F 07 81 00 00 04 00 00
7F 03 82 00 00 04 00 00
4C 03 81 00 00 04 00 00
4C 00 84 00 00 04 00 00
4C 07 82 00 00 04 00 00
BE 0C 81 00 00 04 00 00
BE 01 80 00 00 04 00 00
BE 0F 84 00 00 04 00 00
BE 23 88 00 00 04 00 00
More data:
70 09 80 00 00 04 00 00
6B 0A 80 00 00 04 00 00
9B 0B 81 00 00 04 00 00
BE 0C 81 00 00 04 00 00
41 0D 81 00 00 04 00 00
3E 0E 81 00 00 04 00 00
EB 0F 81 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
F5 02 80 00 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
04 05 82 00 00 04 00 00
4B 06 84 00 00 04 00 00
2A 07 84 00 00 04 00 00
26 18 84 00 00 04 00 00
73 19 84 00 00 04 00 00
68 1A 84 00 00 04 00 00
6C 1B 84 00 00 04 00 00
49 1C 84 00 00 04 00 00
B6 1D 84 00 00 04 00 00
C9 1E 84 00 00 04 00 00
BE 0F 84 00 00 04 00 00
4C 00 84 00 00 04 00 00
1F 01 84 00 00 04 00 00
54 02 84 00 00 04 00 00
19 03 84 00 00 04 00 00
89 04 84 00 00 04 00 00
09 25 84 00 00 04 00 00
20 26 84 00 00 04 00 00
41 27 84 00 00 04 00 00
EF 28 84 00 00 04 00 00
BA 29 84 00 00 04 00 00
A1 2A 84 00 00 04 00 00
A5 2B 84 00 00 04 00 00
80 2C 84 00 00 04 00 00
7F 2D 84 00 00 04 00 00
00 2E 84 00 00 04 00 00
D5 2F 84 00 00 04 00 00
27 20 84 00 00 04 00 00
74 21 84 00 00 04 00 00
3F 22 84 00 00 04 00 00
19 03 84 00 00 04 00 00
2B 14 84 00 00 04 00 00
C0 15 84 00 00 04 00 00
E9 16 84 00 00 04 00 00
88 17 84 00 00 04 00 00
26 18 84 00 00 04 00 00
73 19 84 00 00 04 00 00
68 1A 84 00 00 04 00 00
6C 1B 84 00 00 04 00 00
49 1C 84 00 00 04 00 00
B6 1D 84 00 00 04 00 00
C9 1E 84 00 00 04 00 00
1C 1F 84 00 00 04 00 00
EE 10 84 00 00 04 00 00
BD 11 84 00 00 04 00 00
F6 12 84 00 00 04 00 00
BB 13 84 00 00 04 00 00
2B 14 84 00 00 04 00 00
0C 15 88 00 00 04 00 00
25 16 88 00 00 04 00 00
44 17 88 00 00 04 00 00
EA 18 88 00 00 04 00 00
BF 19 88 00 00 04 00 00
A4 1A 88 00 00 04 00 00
02 0B 88 00 00 04 00 00
27 0C 88 00 00 04 00 00
D8 0D 88 00 00 04 00 00
CC 2E 88 00 00 04 00 00
19 2F 88 00 00 04 00 00
EB 20 88 00 00 04 00 00
B8 21 88 00 00 04 00 00
F3 22 88 00 00 04 00 00
BE 23 88 00 00 04 00 00
2E 24 88 00 00 04 00 00
C5 25 88 00 00 04 00 00
EC 26 88 00 00 04 00 00
8D 27 88 00 00 04 00 00
23 28 88 00 00 04 00 00
76 29 88 00 00 04 00 00
6D 2A 88 00 00 04 00 00
69 2B 88 00 00 04 00 00
4C 2C 88 00 00 04 00 00
B3 2D 88 00 00 04 00 00
CC 2E 88 00 00 04 00 00
72 0F 88 00 00 04 00 00
80 00 88 00 00 04 00 00
D3 01 88 00 00 04 00 00
98 02 88 00 00 04 00 00
D5 03 88 00 00 04 00 00
45 04 88 00 00 04 00 00
AE 05 88 00 00 04 00 00
4B 06 84 00 00 04 00 00
2A 07 84 00 00 04 00 00
E2 08 82 00 00 04 00 00
B7 09 82 00 00 04 00 00
AC 0A 82 00 00 04 00 00
A8 0B 82 00 00 04 00 00
8D 0C 82 00 00 04 00 00
72 0D 82 00 00 04 00 00
0D 0E 82 00 00 04 00 00
D8 0F 82 00 00 04 00 00
2A 00 82 00 00 04 00 00
79 01 82 00 00 04 00 00
32 02 82 00 00 04 00 00
7F 03 82 00 00 04 00 00
EF 04 82 00 00 04 00 00
04 05 82 00 00 04 00 00
2D 06 82 00 00 04 00 00
4C 07 82 00 00 04 00 00
E2 08 82 00 00 04 00 00
B7 09 82 00 00 04 00 00
AC 0A 82 00 00 04 00 00
A8 0B 82 00 00 04 00 00
8D 0C 82 00 00 04 00 00
72 0D 82 00 00 04 00 00
0D 0E 82 00 00 04 00 00
D8 0F 82 00 00 04 00 00
2A 00 82 00 00 04 00 00
79 01 82 00 00 04 00 00
32 02 82 00 00 04 00 00
7F 03 82 00 00 04 00 00
EF 04 82 00 00 04 00 00
04 05 82 00 00 04 00 00
2D 06 82 00 00 04 00 00
Following data have been logged while I tried all the buttons available. + a few combinations of the buttons.
76 00 A0 00 00 04 00 00
25 01 A0 00 00 04 00 00
6E 02 A0 00 00 04 00 00
23 03 A0 00 00 04 00 00
B3 04 A0 00 00 04 00 00
58 05 A0 00 00 04 00 00
71 06 A0 00 00 04 00 00
10 07 A0 00 00 04 00 00
BE 08 A0 00 00 04 00 00
EB 09 A0 00 00 04 00 00
04 0A A1 00 00 04 00 00
00 0B A1 00 00 04 00 00
25 0C A1 00 00 04 00 00
DA 0D A1 00 00 04 00 00
A5 0E A1 00 00 04 00 00
84 0F A0 00 00 04 00 00
76 00 A0 00 00 04 00 00
25 01 A0 00 00 04 00 00
6E 02 A0 00 00 04 00 00
23 03 A0 00 00 04 00 00
B3 04 A0 00 00 04 00 00
9F 05 A2 00 00 04 00 00
D0 06 A4 00 00 04 00 00
B1 07 A4 00 00 04 00 00
D3 08 A8 00 00 04 00 00
86 09 A8 00 00 04 00 00
9D 0A A8 00 00 04 00 00
99 0B A8 00 00 04 00 00
70 0C A4 00 00 04 00 00
8F 0D A4 00 00 04 00 00
F0 0E A4 00 00 04 00 00
25 0F A4 00 00 04 00 00
B1 00 A2 00 00 04 00 00
E2 01 A2 00 00 04 00 00
A9 02 A2 00 00 04 00 00
23 03 A0 00 00 04 00 00
B3 04 A0 00 00 04 00 00
C3 05 80 00 00 04 00 00
30 06 90 00 00 04 00 00
51 07 90 00 00 04 00 00
25 08 80 00 00 04 00 00
70 09 80 00 00 04 00 00
6B 0A 80 00 00 04 00 00
6F 0B 80 00 00 04 00 00
4A 0C 80 00 00 04 00 00
B5 0D 80 00 00 04 00 00
CA 0E 80 00 00 04 00 00
1F 0F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
01 02 81 00 00 04 00 00
4C 03 81 00 00 04 00 00
DC 04 81 00 00 04 00 00
37 05 81 00 00 04 00 00
1E 06 81 00 00 04 00 00
7F 07 81 00 00 04 00 00
D1 08 81 00 00 04 00 00
70 09 80 00 00 04 00 00
6B 0A 80 00 00 04 00 00
6F 0B 80 00 00 04 00 00
4A 0C 80 00 00 04 00 00
B5 0D 80 00 00 04 00 00
CA 0E 80 00 00 04 00 00
1F 0F 80 00 00 04 00 00
2A 00 82 00 00 04 00 00
79 01 82 00 00 04 00 00
32 02 82 00 00 04 00 00
7F 03 82 00 00 04 00 00
EF 04 82 00 00 04 00 00
04 05 82 00 00 04 00 00
2D 06 82 00 00 04 00 00
2A 07 84 00 00 04 00 00
84 08 84 00 00 04 00 00
D1 09 84 00 00 04 00 00
CA 0A 84 00 00 04 00 00
CE 0B 84 00 00 04 00 00
27 0C 88 00 00 04 00 00
D8 0D 88 00 00 04 00 00
A7 0E 88 00 00 04 00 00
72 0F 88 00 00 04 00 00
4C 00 84 00 00 04 00 00
1F 01 84 00 00 04 00 00
32 02 82 00 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
C3 05 80 00 00 04 00 00
71 06 A0 00 00 04 00 00
10 07 A0 00 00 04 00 00
BE 08 A0 00 00 04 00 00
69 09 C0 00 00 04 00 00
72 0A C0 00 00 04 00 00
F4 0B A0 00 00 04 00 00
D1 0C A0 00 00 04 00 00
2E 0D A0 00 00 04 00 00
51 0E A0 00 00 04 00 00
84 0F A0 00 00 04 00 00
76 00 A0 00 00 04 00 00
25 01 A0 00 00 04 00 00
6E 02 A0 00 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
C3 05 80 00 00 04 00 00
EA 06 80 00 00 04 00 00
8B 07 80 00 00 04 00 00
25 08 80 00 00 04 00 00
0F 09 80 04 00 04 00 00
14 0A 80 04 00 04 00 00
B8 0B 80 06 00 04 00 00
9D 0C 80 06 00 04 00 00
62 0D 80 06 00 04 00 00
1D 0E 80 06 00 04 00 00
C8 0F 80 06 00 04 00 00
3A 00 80 06 00 04 00 00
C1 01 80 04 00 04 00 00
8A 02 80 04 00 04 00 00
C7 03 80 04 00 04 00 00
57 04 80 04 00 04 00 00
BC 05 80 04 00 04 00 00
95 06 80 04 00 04 00 00
F4 07 80 04 00 04 00 00
5A 08 80 04 00 04 00 00
0F 09 80 04 00 04 00 00
14 0A 80 04 00 04 00 00
10 0B 80 04 00 04 00 00
35 0C 80 04 00 04 00 00
CA 0D 80 04 00 04 00 00
B5 0E 80 04 00 04 00 00
60 0F 80 04 00 04 00 00
92 00 80 04 00 04 00 00
C1 01 80 04 00 04 00 00
8A 02 80 04 00 04 00 00
C7 03 80 04 00 04 00 00
57 04 80 04 00 04 00 00
BC 05 80 04 00 04 00 00
95 06 80 04 00 04 00 00
F4 07 80 04 00 04 00 00
5A 08 80 04 00 04 00 00
0F 09 80 04 00 04 00 00
14 0A 80 04 00 04 00 00
10 0B 80 04 00 04 00 00
35 0C 80 04 00 04 00 00
CA 0D 80 04 00 04 00 00
B5 0E 80 04 00 04 00 00
60 0F 80 04 00 04 00 00
92 00 80 04 00 04 00 00
C1 01 80 04 00 04 00 00
8A 02 80 04 00 04 00 00
C7 03 80 04 00 04 00 00
57 04 80 04 00 04 00 00
BC 05 80 04 00 04 00 00
95 06 80 04 00 04 00 00
23 07 80 02 00 04 00 00
8D 08 80 02 00 04 00 00
D8 09 80 02 00 04 00 00
C3 0A 80 02 00 04 00 00
C7 0B 80 02 00 04 00 00
E2 0C 80 02 00 04 00 00
1D 0D 80 02 00 04 00 00
62 0E 80 02 00 04 00 00
B7 0F 80 02 00 04 00 00
45 00 80 02 00 04 00 00
16 01 80 02 00 04 00 00
5D 02 80 02 00 04 00 00
10 03 80 02 00 04 00 00
80 04 80 02 00 04 00 00
6B 05 80 02 00 04 00 00
42 06 80 02 00 04 00 00
23 07 80 02 00 04 00 00
25 08 80 00 00 04 00 00
70 09 80 00 00 04 00 00
6B 0A 80 00 00 04 00 00
6F 0B 80 00 00 04 00 00
4A 0C 80 00 00 04 00 00
1D 0D 80 02 00 04 00 00
B5 0E 80 04 00 04 00 00
60 0F 80 04 00 04 00 00
3A 00 80 06 00 04 00 00
69 01 80 06 00 04 00 00
22 02 80 06 00 04 00 00
6F 03 80 06 00 04 00 00
FF 04 80 06 00 04 00 00
14 05 80 06 00 04 00 00
3D 06 80 06 00 04 00 00
5C 07 80 06 00 04 00 00
F2 08 80 06 00 04 00 00
A7 09 80 06 00 04 00 00
BC 0A 80 06 00 04 00 00
B8 0B 80 06 00 04 00 00
9D 0C 80 06 00 04 00 00
62 0D 80 06 00 04 00 00
1D 0E 80 06 00 04 00 00
C8 0F 80 06 00 04 00 00
3A 00 80 06 00 04 00 00
69 01 80 06 00 04 00 00
22 02 80 06 00 04 00 00
6F 03 80 06 00 04 00 00
FF 04 80 06 00 04 00 00
14 05 80 06 00 04 00 00
3D 06 80 06 00 04 00 00
5C 07 80 06 00 04 00 00
F2 08 80 06 00 04 00 00
A7 09 80 06 00 04 00 00
BC 0A 80 06 00 04 00 00
10 0B 80 04 00 04 00 00
35 0C 80 04 00 04 00 00
1D 0D 80 02 00 04 00 00
62 0E 80 02 00 04 00 00
B7 0F 80 02 00 04 00 00
45 00 80 02 00 04 00 00
16 01 80 02 00 04 00 00
5D 02 80 02 00 04 00 00
10 03 80 02 00 04 00 00
80 04 80 02 00 04 00 00
6B 05 80 02 00 04 00 00
42 06 80 02 00 04 00 00
23 07 80 02 00 04 00 00
8D 08 80 02 00 04 00 00
D8 09 80 02 00 04 00 00
C3 0A 80 02 00 04 00 00
C7 0B 80 02 00 04 00 00
E2 0C 80 02 00 04 00 00
1D 0D 80 02 00 04 00 00
62 0E 80 02 00 04 00 00
B7 0F 80 02 00 04 00 00
45 00 80 02 00 04 00 00
16 01 80 02 00 04 00 00
5D 02 80 02 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
C3 05 80 00 00 04 00 00
EA 06 80 00 00 04 00 00
4C 07 82 00 00 04 00 00
E2 08 82 00 00 04 00 00
B7 09 82 00 00 04 00 00
AC 0A 82 00 00 04 00 00
A8 0B 82 00 00 04 00 00
8D 0C 82 00 00 04 00 00
72 0D 82 00 00 04 00 00
0D 0E 82 00 00 04 00 00
D8 0F 82 00 00 04 00 00
2A 00 82 00 00 04 00 00
79 01 82 00 00 04 00 00
32 02 82 00 00 04 00 00
7F 03 82 00 00 04 00 00
EF 04 82 00 00 04 00 00
04 05 82 00 00 04 00 00
2D 06 82 00 00 04 00 00
4C 07 82 00 00 04 00 00
E2 08 82 00 00 04 00 00
B7 09 82 00 00 04 00 00
AC 0A 82 00 00 04 00 00
A8 0B 82 00 00 04 00 00
8D 0C 82 00 00 04 00 00
72 0D 82 00 00 04 00 00
0D 0E 82 00 00 04 00 00
D8 0F 82 00 00 04 00 00
2A 00 82 00 00 04 00 00
79 01 82 00 00 04 00 00
32 02 82 00 00 04 00 00
7F 03 82 00 00 04 00 00
EF 04 82 00 00 04 00 00
04 05 82 00 00 04 00 00
2D 06 82 00 00 04 00 00
4C 07 82 00 00 04 00 00
E2 08 82 00 00 04 00 00
B7 09 82 00 00 04 00 00
AC 0A 82 00 00 04 00 00
A8 0B 82 00 00 04 00 00
8D 0C 82 00 00 04 00 00
72 0D 82 00 00 04 00 00
0D 0E 82 00 00 04 00 00
1F 0F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
57 12 80 00 00 04 00 00
1A 13 80 00 00 04 00 00
8A 14 80 00 00 04 00 00
61 15 80 00 00 04 00 00
48 16 80 00 00 04 00 00
29 17 80 00 00 04 00 00
87 18 80 00 00 04 00 00
D2 19 80 00 00 04 00 00
C9 1A 80 00 00 04 00 00
CD 1B 80 00 00 04 00 00
E8 1C 80 00 00 04 00 00
17 1D 80 00 00 04 00 00
68 1E 80 00 00 04 00 00
BD 1F 80 00 00 04 00 00
4F 10 80 00 00 04 00 00
1C 11 80 00 00 04 00 00
57 12 80 00 00 04 00 00
1A 13 80 00 00 04 00 00
8A 14 80 00 00 04 00 00
61 15 80 00 00 04 00 00
48 16 80 00 00 04 00 00
29 17 80 00 00 04 00 00
87 18 80 00 00 04 00 00
D2 19 80 00 00 04 00 00
C9 1A 80 00 00 04 00 00
CD 1B 80 00 00 04 00 00
E8 1C 80 00 00 04 00 00
17 1D 80 00 00 04 00 00
CA 0E 80 00 00 04 00 00
1F 0F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
F5 02 80 00 00 04 00 00
B8 03 80 00 00 04 00 00
43 24 80 00 00 04 00 00
A8 25 80 00 00 04 00 00
81 26 80 00 00 04 00 00
E0 27 80 00 00 04 00 00
4E 28 80 00 00 04 00 00
1B 29 80 00 00 04 00 00
00 2A 80 00 00 04 00 00
04 2B 80 00 00 04 00 00
21 2C 80 00 00 04 00 00
DE 2D 80 00 00 04 00 00
A1 2E 80 00 00 04 00 00
74 2F 80 00 00 04 00 00
86 20 80 00 00 04 00 00
D5 21 80 00 00 04 00 00
9E 22 80 00 00 04 00 00
D3 23 80 00 00 04 00 00
43 24 80 00 00 04 00 00
A8 25 80 00 00 04 00 00
81 26 80 00 00 04 00 00
E0 27 80 00 00 04 00 00
4E 28 80 00 00 04 00 00
1B 29 80 00 00 04 00 00
00 2A 80 00 00 04 00 00
04 2B 80 00 00 04 00 00
21 2C 80 00 00 04 00 00
DE 2D 80 00 00 04 00 00
A1 2E 80 00 00 04 00 00
74 2F 80 00 00 04 00 00
86 20 80 00 00 04 00 00
D5 21 80 00 00 04 00 00
F5 02 80 00 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
C3 05 80 00 00 04 00 00
EA 06 80 00 00 04 00 00
8B 07 80 00 00 04 00 00
F3 48 80 00 00 04 00 00
A6 49 80 00 00 04 00 00
BD 4A 80 00 00 04 00 00
B9 4B 80 00 00 04 00 00
9C 4C 80 00 00 04 00 00
63 4D 80 00 00 04 00 00
1C 4E 80 00 00 04 00 00
C9 4F 80 00 00 04 00 00
3B 40 80 00 00 04 00 00
68 41 80 00 00 04 00 00
23 42 80 00 00 04 00 00
6E 43 80 00 00 04 00 00
FE 44 80 00 00 04 00 00
15 45 80 00 00 04 00 00
3C 46 80 00 00 04 00 00
5D 47 80 00 00 04 00 00
F3 48 80 00 00 04 00 00
A6 49 80 00 00 04 00 00
BD 4A 80 00 00 04 00 00
B9 4B 80 00 00 04 00 00
9C 4C 80 00 00 04 00 00
63 4D 80 00 00 04 00 00
1C 4E 80 00 00 04 00 00
C9 4F 80 00 00 04 00 00
3B 40 80 00 00 04 00 00
68 41 80 00 00 04 00 00
23 42 80 00 00 04 00 00
6E 43 80 00 00 04 00 00
FE 44 80 00 00 04 00 00
15 45 80 00 00 04 00 00
3C 46 80 00 00 04 00 00
5D 47 80 00 00 04 00 00
F3 48 80 00 00 04 00 00
A6 49 80 00 00 04 00 00
BD 4A 80 00 00 04 00 00
6F 0B 80 00 00 04 00 00
4A 0C 80 00 00 04 00 00
B5 0D 80 00 00 04 00 00
CA 0E 80 00 00 04 00 00
1F 0F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
3D 81 80 00 00 04 00 00
76 82 80 00 00 04 00 00
3B 83 80 00 00 04 00 00
AB 84 80 00 00 04 00 00
40 85 80 00 00 04 00 00
69 86 80 00 00 04 00 00
08 87 80 00 00 04 00 00
A6 88 80 00 00 04 00 00
F3 89 80 00 00 04 00 00
E8 8A 80 00 00 04 00 00
EC 8B 80 00 00 04 00 00
C9 8C 80 00 00 04 00 00
36 8D 80 00 00 04 00 00
49 8E 80 00 00 04 00 00
9C 8F 80 00 00 04 00 00
6E 80 80 00 00 04 00 00
3D 81 80 00 00 04 00 00
76 82 80 00 00 04 00 00
3B 83 80 00 00 04 00 00
AB 84 80 00 00 04 00 00
40 85 80 00 00 04 00 00
69 86 80 00 00 04 00 00
08 87 80 00 00 04 00 00
A6 88 80 00 00 04 00 00
F3 89 80 00 00 04 00 00
E8 8A 80 00 00 04 00 00
EC 8B 80 00 00 04 00 00
C9 8C 80 00 00 04 00 00
36 8D 80 00 00 04 00 00
49 8E 80 00 00 04 00 00
9C 8F 80 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
F5 02 80 00 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
04 05 82 00 00 04 00 00
4B 06 84 00 00 04 00 00
2A 07 84 00 00 04 00 00
84 08 84 00 00 04 00 00
D1 09 84 00 00 04 00 00
CA 0A 84 00 00 04 00 00
CE 0B 84 00 00 04 00 00
EB 0C 84 00 00 04 00 00
14 0D 84 00 00 04 00 00
6B 0E 84 00 00 04 00 00
BE 0F 84 00 00 04 00 00
4C 00 84 00 00 04 00 00
1F 01 84 00 00 04 00 00
54 02 84 00 00 04 00 00
19 03 84 00 00 04 00 00
89 04 84 00 00 04 00 00
62 05 84 00 00 04 00 00
4B 06 84 00 00 04 00 00
2A 07 84 00 00 04 00 00
84 08 84 00 00 04 00 00
D1 09 84 00 00 04 00 00
CA 0A 84 00 00 04 00 00
CE 0B 84 00 00 04 00 00
EB 0C 84 00 00 04 00 00
14 0D 84 00 00 04 00 00
6B 0E 84 00 00 04 00 00
BE 0F 84 00 00 04 00 00
4C 00 84 00 00 04 00 00
1F 01 84 00 00 04 00 00
54 02 84 00 00 04 00 00
19 03 84 00 00 04 00 00
89 04 84 00 00 04 00 00
62 05 84 00 00 04 00 00
4B 06 84 00 00 04 00 00
2A 07 84 00 00 04 00 00
48 08 88 00 00 04 00 00
1D 09 88 00 00 04 00 00
06 0A 88 00 00 04 00 00
02 0B 88 00 00 04 00 00
27 0C 88 00 00 04 00 00
D8 0D 88 00 00 04 00 00
A7 0E 88 00 00 04 00 00
72 0F 88 00 00 04 00 00
80 00 88 00 00 04 00 00
D3 01 88 00 00 04 00 00
98 02 88 00 00 04 00 00
D5 03 88 00 00 04 00 00
45 04 88 00 00 04 00 00
AE 05 88 00 00 04 00 00
87 06 88 00 00 04 00 00
E6 07 88 00 00 04 00 00
48 08 88 00 00 04 00 00
1D 09 88 00 00 04 00 00
06 0A 88 00 00 04 00 00
CE 0B 84 00 00 04 00 00
EB 0C 84 00 00 04 00 00
72 0D 82 00 00 04 00 00
0D 0E 82 00 00 04 00 00
D8 0F 82 00 00 04 00 00
ED 00 80 00 00 04 00 00
BE 01 80 00 00 04 00 00
F5 02 80 00 00 04 00 00
B8 03 80 00 00 04 00 00
28 04 80 00 00 04 00 00
C3 05 80 00 00 04 00 00
EA 06 80 00 00 04 00 00
8B 07 80 00 00 04 00 00
Thanks!
If your goal is to send an arbitrary known code to the car, a simple way is to use a map to store all the valid 1st bytes, and send it accordingly.
If your goal is to utilize those other bits previously not used (~5 bytes of them), then unfortunately it is not possible from the data you provided. One reason is that there is no way for us to know the ordering of the ~40 currently unused bits even if we knew the algorithm used to generate the 1st byte.
The space of 1 byte values is small, only 256. For any code that you want to send, you could just try 256 times each with a different 1st byte. If any one does anything, then it's probably it. If none does, then maybe that code is invalid altogether?
I can say what it is not. The first byte is not a CRC, nor is it any linear combination of the provided remaining bits over GF(2). It may be some other function of the following bits/bytes.
I am trying to import a key into the card, but it is giving response as 6F00 (UNKNOWN ERROR).The procedure i followed to import a key is
Load the (MUSCLE) applet
Initialize the applet
Verify the pin
create the object with id (FF FF FF FE):
-> B0 5A 00 00 0E FF FF FF FE 00 00 00 44 00 00 00 00 00 00 00
<- 90 00
write into the object
-> B0 54 00 00 8D FF FF FF FE 00 00 00 00 84 00 01 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
<- 90 00
Import key
-> B0 32 04 00 07 00 00 FF FF 00 00 00 00
<- 6F 00
Please provide a solution for the above problem.
If you are still looking for the solution: 7 bytes seems to be a bit high for importing a key... ;)
The ACL in the data block is only six bytes, so this might cause your error. The following "optional parameters" are AFAIK completely unused.
Have a file that appears to have plaintext headers in them that I would like to extract and convert to plaintext.
Using HEXedit, this is what I'm seeing, which is in a file:
3a40 - 31 65 33 38 00 00 00 00 00 00 00 00 00 00 00 00 - 1e38............
3a50 - 00 00 00 00 00 00 00 00 00 00 0a 00 74 00 65 00 - ............t.e.
3a60 - 78 00 74 00 2f 00 61 00 73 00 63 00 69 00 69 00 - x.t./.a.s.c.i.i.
3a70 - 00 00 18 00 61 00 66 00 66 00 79 00 6d 00 65 00 - ....a.f.f.y.m.e
3a80 - 74 00 72 00 69 00 78 00 2d 00 61 00 72 00 72 00 - t.r.i.x.-.a.r.r
3a90 - 61 00 79 00 2d 00 62 00 61 00 72 00 63 00 6f 00 - a.y.-.b.a.r.c.o.
3aa0 - 64 00 65 00 00 00 64 00 40 00 35 00 32 00 30 00 - d.e...d.#.5.2.0.
3ab0 - 38 00 32 00 36 00 30 00 30 00 39 00 31 00 30 00 - 8.2.6.0.0.9.1.0.
3ac0 - 37 00 30 00 36 00 31 00 31 00 31 00 38 00 31 00 - 7.0.6.1.1.1.8.1.
3ad0 - 31 00 34 00 31 00 32 00 31 00 33 00 34 00 35 00 - 1.4.1.2.1.3.4.5.
3ae0 - 35 00 30 00 39 00 38 00 39 00 00 00 00 00 00 00 - 5.0.9.8.9.......
3af0 - 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 - ................
3b00 - 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0a 00 - ................
and this is the output I'd like to get:
text/ascii affymetrix-array-barcode d#52082600910706111811412134550989
Try with the iconv command. Something like this should work:
tail -c +6 input.txt | iconv -f UTF16 -t ASCII >output.txt
Then split on the null bytes.
Granted, I'm no wiz, but this does the job if all your files look very similar to the one you just posted:
use strict;
open FILE, 'file.dat';
binmode FILE;
my ($chunk, $buf, $n);
seek FILE, 28, 0;
while (($n=read FILE, $chunk, 16)) { $buf .= $chunk; }
my #s=split(/\0\0/, $buf, 4);
print "$s[0] $s[1] $s[2]\n";
close (FILE);
A perl solution might be interesting, but wouldn't the unix strings command give you the plaintext portion of the file?
I have a file like this:
01 00 01 14 c0 00 01 10 01 00 00 16 00 00 00 64
00 00 00 65 00 00 01 07 40 00 00 22 68 61 6c 2e
6f 70 65 6e 65 74 2e 63 6f 6d 3b 30 30 30 30 30
30 30 30 32 3b 30 00 00 00 00 01 08 40 00 00 1e
68 61 6c 2e 6f 70 65 6e 65 74 2d 74 65 6c 65 63
6f 6d 2e 6c 61 6e 00 00 00 00 01 28 40 00 00 21
72 65 61 6c 6d 31 2e 6f 70 65 6e 65 74 2d 74 65
6c 65 63 6f 6d 2e 6c 61 6e 00 00 00 00 00 01 25
40 00 00 1e 68 61 6c 2e 6f 70 65 6e 65 74 2d 74
65 6c 65 63 6f 6d 2e 6c 61 6e 00 00 00 00 01 1b
40 00 00 20 72 65 61 6c 6d 2e 6f 70 65 6e 65 74
2d 74 65 6c 65 63 6f 6d 2e 6c 61 6e 00 00 01 02
40 00 00 0c 01 00 00 16 00 00 01 a0 40 00 00 0c
00 00 00 01 00 00 01 9f 40 00 00 0c 00 00 00 00
00 00 01 16 40 00 00 0c 00 00 00 00 00 00 01 bb
40 00 00 28 00 00 01 c2 40 00 00 0c 00 00 00 00
00 00 01 bc 40 00 00 13 31 39 37 37 31 31 31 32
32 33 31 00
I am reading the file and then finding certain octets and replacing them with tags:
while(<FH>){
$line =~ s/(00 00 00 64)/<incr4> /g;
$line =~ s/(00 00 00 65)/<incr4> /g;
$line =~ s/(30 30 30 30 30 32)/<incr6ascii:999999:0>/g;
$line =~ s/(31 31 32 32 33 31)/<incr6ascii:999999:0>/g;
print OUTPUT $line;
}
So for example, 00 00 00 64 would be replaced by the <incr4> tag. This was working fine, but it doesn't seem to able to match over multiple lines any more. For example the pattern 31 31 32 32 33 31 runs over multiple lines, and the regular expression doesn't seem to catch it. I tried using /m /s pattern modifiers to ignore new lines but they didn't match it either. The only way around it I can come up with, is to read the whole file into a string using:
undef $/;
my $whole_file = <FH>;
my $line = $whole_file;
$line =~ s/(00 00 00 64)/<incr4> /g;
$line =~ s/(00 00 00 65)/<incr4> /g;
$line =~ s/(30 30 30 30 30 32)/<incr6ascii:999999:0>/g;
$line =~ s/(31 31 32 32 33 31)/<incr6ascii:999999:0>/g;
print OUTPUT $line;
This works, the tags get inserted correctly, but the structure of the file is radically altered. It is all dumped out on a single line. I would like to retain the structure of the file as it appears here. Any ideas as to how I might do this?
/john
The trick here is to match the class of all space like characters \s:
my $file = do {local (#ARGV, $/) = 'filename.txt'; <>}; # slurp file
my %tr = ( # setup a translation table
'00 00 00 64' => '<incr4>',
'00 00 00 65' => '<incr4>',
'00 30 30 30 30 32' => '<incr6ascii:999999:0>',
'31 31 32 32 33 31' => '<incr6ascii:999999:0>',
);
for (keys %tr) {
my $re = join '\s+' => split; # construct new regex
$file =~ s{($re)}{
$1 =~ /\n/ ? "\n$tr{$_}" : $tr{$_} # if octets contained \n, add \n
}ge # match multiple times, execute the replacement block as perl code
}
print $file;