struct ValueType {
var member: Int
}
class ReferenceType {
var member: Int
init(member: Int) {
self.member = member
}
}
var valueTypeObject = ValueType(member: 3)
var referenceTypeObject = ReferenceType(member: 4)
withUnsafePointer(to: &referenceTypeObject) {
print("referenceTypeObject address: \($0)")
}
withUnsafePointer(to: &valueTypeObject) {
print("valueTypeObject address: \($0)")
}
When executing the above code, the address of each object appears like this.
valueTypeObject address: 0x0000000100008218
referenceTypeObject address: 0x0000000100008220
First, if I view memory of valueTypeObject address (0x0000000100008218), I can check the 03 value within 64 bits that I actually allocated (03 00 00 00 00 00 00 00 00 00 00 00 00 00. Maybe data is stored as little endian.)
Next, if I view memory of referenceTypeObject address (0x0000000100008220), I can check 0x000000010172f8b0 is stored in 64bit. (I don't know why right side of ..r..... is also highlighted, and what it is 🤔)
I know that the referenceTypeObject is reference type, so the actual value is in the heap area. So I can guess 0x000000010172f8b0 is an address that stores the actual value that I assigned (in this case, 4.)
But how does Swift know that this is the address that points to heap area instead of 0x000000010172f8b0 value that can be assied by me?
In addition, if I view memory of address 0x000000010172f8b0 where the actual value is stored, there are some 32 bytes values in front of the value that I allocated (in this case, 4). What are those?
I want to disassemble a list of executables with an output format for the instructions that I can customize. As a prior web developer, iced-x86 with Node.js seems like an easy way to start. I want to use this.
The code I'm using is exactly what's listed on the documentation for iced-x86:
Disassemble (decode and format instructions), although I'm loading the Uint8Array from my own exe and set the RIP to 0x0, I'm not sure how this value should be decided - it seems like an arbitrary equivalent to uint64_t exampleRip = 0x00007FFAC46ACDA4:
// iced-x86 features needed: --features "decoder nasm"
const { Decoder, DecoderOptions, Formatter, FormatterSyntax } = require("iced-x86");
/*
This code produces the following output:
00007FFAC46ACDA4 48895C2410 mov [rsp+10h],rbx
00007FFAC46ACDA9 4889742418 mov [rsp+18h],rsi
00007FFAC46ACDAE 55 push rbp
00007FFAC46ACDAF 57 push rdi
00007FFAC46ACDB0 4156 push r14
00007FFAC46ACDB2 488DAC2400FFFFFF lea rbp,[rsp-100h]
00007FFAC46ACDBA 4881EC00020000 sub rsp,200h
00007FFAC46ACDC1 488B0518570A00 mov rax,[rel 7FFA`C475`24E0h]
00007FFAC46ACDC8 4833C4 xor rax,rsp
00007FFAC46ACDCB 488985F0000000 mov [rbp+0F0h],rax
00007FFAC46ACDD2 4C8B052F240A00 mov r8,[rel 7FFA`C474`F208h]
00007FFAC46ACDD9 488D05787C0400 lea rax,[rel 7FFA`C46F`4A58h]
00007FFAC46ACDE0 33FF xor edi,edi
*/
const exampleBitness = 64;
const exampleRipLo = 0xC46ACDA4;
const exampleRipHi = 0x00007FFA;
const exampleCode = new Uint8Array([
0x48, 0x89, 0x5C, 0x24, 0x10, 0x48, 0x89, 0x74, 0x24, 0x18, 0x55, 0x57, 0x41, 0x56, 0x48, 0x8D,
0xAC, 0x24, 0x00, 0xFF, 0xFF, 0xFF, 0x48, 0x81, 0xEC, 0x00, 0x02, 0x00, 0x00, 0x48, 0x8B, 0x05,
0x18, 0x57, 0x0A, 0x00, 0x48, 0x33, 0xC4, 0x48, 0x89, 0x85, 0xF0, 0x00, 0x00, 0x00, 0x4C, 0x8B,
0x05, 0x2F, 0x24, 0x0A, 0x00, 0x48, 0x8D, 0x05, 0x78, 0x7C, 0x04, 0x00, 0x33, 0xFF
]);
const hexBytesColumnByteLength = 10;
const decoder = new Decoder(exampleBitness, exampleCode, DecoderOptions.None);
// You have to enable the bigint feature to get i64/u64 APIs, not all browsers support BigInt
decoder.ipLo = exampleRipLo;
decoder.ipHi = exampleRipHi;
// This decodes all bytes. There's also `decode()` which decodes the next instruction,
// `decodeInstructions(count)` which decodes `count` instructions and `decodeOut(instruction)`
// which overwrites an existing instruction.
const instructions = decoder.decodeAll();
// Create a nasm formatter. It supports: Masm, Nasm, Gas (AT&T) and Intel (XED).
// There's also `FastFormatter` which uses less code (smaller wasm files).
// const formatter = new FastFormatter();
const formatter = new Formatter(FormatterSyntax.Nasm);
// Change some options, there are many more
formatter.digitSeparator = "`";
formatter.firstOperandCharIndex = 10;
// Format the instructions
instructions.forEach(instruction => {
const disasm = formatter.format(instruction);
// Eg. "00007FFAC46ACDB2 488DAC2400FFFFFF lea rbp,[rsp-100h]"
let line = ("0000000" + instruction.ipHi.toString(16)).substr(-8).toUpperCase() +
("0000000" + instruction.ipLo.toString(16)).substr(-8).toUpperCase();
line += " ";
const startIndex = instruction.ipLo - exampleRipLo;
exampleCode.slice(startIndex, startIndex + instruction.length).forEach(b => {
line += ("0" + b.toString(16)).substr(-2).toUpperCase();
});
for (let i = instruction.length; i < hexBytesColumnByteLength; i++)
line += " ";
line += " ";
line += disasm;
console.log(line);
});
// Free wasm memory
instructions.forEach(instruction => instruction.free());
formatter.free();
decoder.free();
Meanwhile, I'm also disassembling the same file with Ghidra to check my work.
Ghidra outputs the correct disassembly of the header:
//
// Headers
// ram:00400000-ram:004001ff
//
assume DF = 0x0 (Default)
IMAGE_DOS_HEADER_00400000 XREF[1]: 004000b4(*)
00400000 4d 5a 90 IMAGE_DO
00 03 00
00 00 04
00400000 4d 5a char[2] "MZ" e_magic
00400002 90 00 dw 90h e_cblp Bytes of last page
00400004 03 00 dw 3h e_cp Pages in file
00400006 00 00 dw 0h e_crlc Relocations
00400008 04 00 dw 4h e_cparhdr Size of header in
0040000a 00 00 dw 0h e_minalloc Minimum extra para
0040000c ff ff dw FFFFh e_maxalloc Maximum extra para
0040000e 00 00 dw 0h e_ss Initial (relative)
00400010 b8 00 dw B8h e_sp Initial SP value
00400012 00 00 dw 0h e_csum Checksum
00400014 00 00 dw 0h e_ip Initial IP value
00400016 00 00 dw 0h e_cs Initial (relative)
00400018 40 00 dw 40h e_lfarlc File address of re
0040001a 00 00 dw 0h e_ovno Overlay number
While iced_x86 outputs the following, clearing not correctly handling the header:
0000000000000000 4D5A pop r10
0000000000000002 90 nop
0000000000000003 0003 add [rbx],al
0000000000000005 0000 add [rax],al
0000000000000007 000400 add [rax+rax],al
000000000000000A 0000 add [rax],al
000000000000000C FFFF (bad)
000000000000000E 0000 add [rax],al
0000000000000010 B800000000 mov eax,0
0000000000000015 0000 add [rax],al
0000000000000017 004000 add [rax],al
000000000000001A 0000 add [rax],al
I'm newly learning Assembly; I don't know how the header data needs to be disassembled differently from the the rest of the instructions. How can I handle this? Is there a feature of iced_x86 I should be using?
What I've tried:
I'm looking through documentation on the topic, and found this representation in C:
struct DOS_Header
{
// short is 2 bytes, long is 4 bytes
char signature[2] = { 'M', 'Z' };
short lastsize;
short nblocks;
short nreloc;
short hdrsize;
short minalloc;
short maxalloc;
void *ss; // 2 byte value
void *sp; // 2 byte value
short checksum;
void *ip; // 2 byte value
void *cs; // 2 byte value
short relocpos;
short noverlay;
short reserved1[4];
short oem_id;
short oem_info;
short reserved2[10];
long e_lfanew; // Offset to the 'PE\0\0' signature relative to the beginning of the file
}
Which seems straightforward enough. But I'm still unsure whether I need to write code for this in the Node.js script, or if the iced_x86 library supports it already.
I'm using the RootCA's SHA1 thumbprint which is still valid from api.github.com
full original source code.
I do not understand why the connection has failed, i tried changing the thumbprint to capital instead still doesn't work, i've also added client.setFingerprint();
const char* host = "api.github.com";
const int httpsPort = 443;
// Use web browser to view and copy
// SHA1 fingerprint of the certificate
const char* fingerprint = "5f b7 ee 06 33 e2 59 db ad 0c 4c 9a e6 d3 8f 1a 61 c7 dc 25";
// Use WiFiClientSecure class to create TLS connection
WiFiClientSecure client;
client.setFingerprint(fingerprint);
Do you have idea why first and secod lines below do not produce BOM to the file and third line does? I thought UTF-16LE is correct encoding name and that encoding does no create BOM automatically to beginning of the file.
new File("foo-wo-bom.txt").withPrintWriter("utf-16le") {it << "test"}
new File("foo-bom1.txt").withPrintWriter("UnicodeLittleUnmarked") {it << "test"}
new File("foo-bom.txt").withPrintWriter("UTF-16LE") {it << "test"}
Another samples
new File("foo-bom.txt").withPrintWriter("UTF-16LE") {it << "test"}
new File("foo-bom.txt").getBytes().each {System.out.format("%02x ", it)}
prints
ff fe 74 00 65 00 73 00 74 00
and with java
PrintWriter w = new PrintWriter("foo.txt","UTF-16LE");
w.print("test");
w.close();
FileInputStream r = new FileInputStream("foo.txt");
int c;
while ((c = r.read()) != -1) {
System.out.format("%02x ",c);
}
r.close();
prints
74 00 65 00 73 00 74 00
With Java is does not produce BOM and with Groovy there is BOM.
There appears to be a difference in behavior with withPrintWriter. Try this out in your GroovyConsole
File file = new File("tmp.txt")
try {
String text = " "
String charset = "UTF-16LE"
file.withPrintWriter(charset) { it << text }
println "withPrintWriter"
file.getBytes().each { System.out.format("%02x ", it) }
PrintWriter w = new PrintWriter(file, charset)
w.print(text)
w.close()
println "\n\nnew PrintWriter"
file.getBytes().each { System.out.format("%02x ", it) }
} finally {
file.delete()
}
It outputs
withPrintWriter
ff fe 20 00
new PrintWriter
20 00
This is because calling new PrintWriter calls the Java constructor, but calling withPrintWriter eventually calls org.codehaus.groovy.runtime.ResourceGroovyMethods.writeUTF16BomIfRequired(), which writes the BOM.
I'm uncertain whether this difference in behavior is intentional. I was curious about this, so I asked on the mailing list. Someone there should know the history behind the design.
Edit: GROOVY-7465 was created out of the aforementioned discussion.
I've been reading about Unicode and UTF-8 in the last couple of days and I often come across a bitwise comparison similar to this :
int strlen_utf8(char *s)
{
int i = 0, j = 0;
while (s[i])
{
if ((s[i] & 0xc0) != 0x80) j++;
i++;
}
return j;
}
Can someone clarify the comparison with 0xc0 and checking if it's the most significant bit ?
Thank you!
EDIT: ANDed, not comparison, used the wrong word ;)
It's not a comparison with 0xc0, it's a logical AND operation with 0xc0.
The bit mask 0xc0 is 11 00 00 00 so what the AND is doing is extracting only the top two bits:
ab cd ef gh
AND 11 00 00 00
-- -- -- --
= ab 00 00 00
This is then compared to 0x80 (binary 10 00 00 00). In other words, the if statement is checking to see if the top two bits of the value are not equal to 10.
"Why?", I hear you ask. Well, that's a good question. The answer is that, in UTF-8, all bytes that begin with the bit pattern 10 are subsequent bytes of a multi-byte sequence:
UTF-8
Range Encoding Binary value
----------------- -------- --------------------------
U+000000-U+00007f 0xxxxxxx 0xxxxxxx
U+000080-U+0007ff 110yyyxx 00000yyy xxxxxxxx
10xxxxxx
U+000800-U+00ffff 1110yyyy yyyyyyyy xxxxxxxx
10yyyyxx
10xxxxxx
U+010000-U+10ffff 11110zzz 000zzzzz yyyyyyyy xxxxxxxx
10zzyyyy
10yyyyxx
10xxxxxx
So, what this little snippet is doing is going through every byte of your UTF-8 string and counting up all the bytes that aren't continuation bytes (i.e., it's getting the length of the string, as advertised). See this wikipedia link for more detail and Joel Spolsky's excellent article for a primer.
An interesting aside by the way. You can classify bytes in a UTF-8 stream as follows:
With the high bit set to 0, it's a single byte value.
With the two high bits set to 10, it's a continuation byte.
Otherwise, it's the first byte of a multi-byte sequence and the number of leading 1 bits indicates how many bytes there are in total for this sequence (110... means two bytes, 1110... means three bytes, etc).