Some Hexademical numbers get modified when writing to a file - encoding

Question

I'm writing a program which packs hexadecimal strings into bytes and writes them on disk. I expect the hexdump of the file to be same as the hexadecimal things. I'm doing this in Clojure:
(defn- hex-char->int
[hex-char]
(-> hex-char
str
(Integer/parseInt 16)))
(defn- pack
[hex-1 hex-2]
(-> hex-1
(bit-shift-left 4)
(bit-or hex-2)
unchecked-char))
(defn- hex-str->packed-bytes
[hex-str]
(->> hex-str
(map hex-char->int)
(partition 2)
(mapv (partial apply pack))))
(defn write-bytes
[bs]
(with-open [f (io/output-stream "test.txt")]
(.write f (.getBytes bs))))
(defn test-write
[hex-str]
(->> hex-str
hex-str->packed-bytes
(apply str)
write-bytes))
This program works well for hex couples from "00" to "7f". I can see the same hex numbers when I hexdump the output file.
But for characters from "80" to "ff", this doesn't work. The hexdump for "80" is "c280" and for "ff" it is "c3bf".
This gets solved if I don't convert to characters and directly write with bytes, so I assume that this is related to encoding.
I even found this: https://superuser.com/questions/1349494/filling-file-with-0xff-gives-c3bf-in-osx
But I want to understand how to solve this in Clojure's context.
Pasting the hexdump of `000f101f202f303f404f505f606f707f808f909fa0afb0bfc0cfd0dfe0eff0ff" for reference:
00000000 00 0f 10 1f 20 2f 30 3f 40 4f 50 5f 60 6f 70 7f |.... /0?#OP_`op.|
00000010 c2 80 c2 8f c2 90 c2 9f c2 a0 c2 af c2 b0 c2 bf |................|
00000020 c3 80 c3 8f c3 90 c3 9f c3 a0 c3 af c3 b0 c3 bf |................|
00000030
Please help me solve this.
Thanks! :)

Answer 1

As you suspected, the problem is in encoding. I'm guessing that the problem is happening when you (apply str) in test-write. So, I slightly re-wrote your code as follows:
user> (defn- hex-char->int
[hex-char]
(-> hex-char
str
(Integer/parseInt 16)))
#'user/hex-char->int
user> (defn- pack
[hex-1 hex-2]
(-> hex-1
(bit-shift-left 4)
(bit-or hex-2)))
#'user/pack
user> (defn- hex-str->packed-bytes
[hex-str]
(->> hex-str
(map hex-char->int)
(partition 2)
(mapv (partial apply pack))))
#'user/hex-str->packed-bytes
user> (defn write-bytes
[bs]
(with-open [f (io/output-stream "test.txt")]
(.write f bs)))
#'user/write-bytes
user> (defn test-write
[hex-str]
(->> hex-str
hex-str->packed-bytes
(mapv unchecked-byte)
(byte-array)
write-bytes))
#'user/test-write
user> (test-write "000f101f202f303f404f505f606f707f808f909fa0afb0bfc0cfd0dfe0eff0ff")
nil
user>
And showing the contents of the resultant file in hex:
dorabs-imac:example dorab$ od -h test.txt
0000000 0f00 1f10 2f20 3f30 4f40 5f50 6f60 7f70
0000020 8f80 9f90 afa0 bfb0 cfc0 dfd0 efe0 fff0
0000040