Following links explain x86-32 system call conventions for both UNIX (BSD flavor) & Linux:
http://www.int80h.org/bsdasm/#system-calls
http://www.freebsd.org/doc/en/books/developers-handbook/x86-system-calls.html
But what are the x86-64 system call conventions on both UNIX & Linux?
Further reading for any of the topics here: The Definitive Guide to Linux System Calls
I verified these using GNU Assembler (gas) on Linux.
Kernel Interface
x86-32 aka i386 Linux System Call convention:
In x86-32 parameters for Linux system call are passed using registers. %eax for syscall_number. %ebx, %ecx, %edx, %esi, %edi, %ebp are used for passing 6 parameters to system calls.
The return value is in %eax. All other registers (including EFLAGS) are preserved across the int $0x80.
I took following snippet from the Linux Assembly Tutorial but I'm doubtful about this. If any one can show an example, it would be great.
If there are more than six arguments,
%ebx must contain the memory
location where the list of arguments
is stored - but don't worry about this
because it's unlikely that you'll use
a syscall with more than six
arguments.
For an example and a little more reading, refer to http://www.int80h.org/bsdasm/#alternate-calling-convention. Another example of a Hello World for i386 Linux using int 0x80: Hello, world in assembly language with Linux system calls?
There is a faster way to make 32-bit system calls: using sysenter. The kernel maps a page of memory into every process (the vDSO), with the user-space side of the sysenter dance, which has to cooperate with the kernel for it to be able to find the return address. Arg to register mapping is the same as for int $0x80. You should normally call into the vDSO instead of using sysenter directly. (See The Definitive Guide to Linux System Calls for info on linking and calling into the vDSO, and for more info on sysenter, and everything else to do with system calls.)
x86-32 [Free|Open|Net|DragonFly]BSD UNIX System Call convention:
Parameters are passed on the stack. Push the parameters (last parameter pushed first) on to the stack. Then push an additional 32-bit of dummy data (Its not actually dummy data. refer to following link for more info) and then give a system call instruction int $0x80
http://www.int80h.org/bsdasm/#default-calling-convention
x86-64 Linux System Call convention:
(Note: x86-64 Mac OS X is similar but different from Linux. TODO: check what *BSD does)
Refer to section: "A.2 AMD64 Linux Kernel Conventions" of System V Application Binary Interface AMD64 Architecture Processor Supplement. The latest versions of the i386 and x86-64 System V psABIs can be found linked from this page in the ABI maintainer's repo. (See also the x86 tag wiki for up-to-date ABI links and lots of other good stuff about x86 asm.)
Here is the snippet from this section:
User-level applications use as integer registers for passing the
sequence %rdi, %rsi, %rdx, %rcx,
%r8 and %r9. The kernel interface uses %rdi, %rsi, %rdx, %r10, %r8 and %r9.
A system-call is done via the syscall instruction. This clobbers %rcx and %r11 as well as the %rax return value, but other registers are preserved.
The number of the syscall has to be passed in register %rax.
System-calls are limited to six arguments, no argument is passed
directly on the stack.
Returning from the syscall, register %rax contains the result of
the system-call. A value in the range between -4095 and -1 indicates
an error, it is -errno.
Only values of class INTEGER or class MEMORY are passed to the kernel.
Remember this is from the Linux-specific appendix to the ABI, and even for Linux it's informative not normative. (But it is in fact accurate.)
This 32-bit int $0x80 ABI is usable in 64-bit code (but highly not recommended). What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code? It still truncates its inputs to 32-bit, so it's unsuitable for pointers, and it zeros r8-r11.
User Interface: function calling
x86-32 Function Calling convention:
In x86-32 parameters were passed on stack. Last parameter was pushed first on to the stack until all parameters are done and then call instruction was executed. This is used for calling C library (libc) functions on Linux from assembly.
Modern versions of the i386 System V ABI (used on Linux) require 16-byte alignment of %esp before a call, like the x86-64 System V ABI has always required. Callees are allowed to assume that and use SSE 16-byte loads/stores that fault on unaligned. But historically, Linux only required 4-byte stack alignment, so it took extra work to reserve naturally-aligned space even for an 8-byte double or something.
Some other modern 32-bit systems still don't require more than 4 byte stack alignment.
x86-64 System V user-space Function Calling convention:
x86-64 System V passes args in registers, which is more efficient than i386 System V's stack args convention. It avoids the latency and extra instructions of storing args to memory (cache) and then loading them back again in the callee. This works well because there are more registers available, and is better for modern high-performance CPUs where latency and out-of-order execution matter. (The i386 ABI is very old).
In this new mechanism: First the parameters are divided into classes. The class of each parameter determines the manner in which it is passed to the called function.
For complete information refer to : "3.2 Function Calling Sequence" of System V Application Binary Interface AMD64 Architecture Processor Supplement which reads, in part:
Once arguments are classified, the registers get assigned (in
left-to-right order) for passing as follows:
If the class is MEMORY, pass the argument on the stack.
If the class is INTEGER, the next available register of the
sequence %rdi, %rsi, %rdx, %rcx, %r8 and %r9 is used
So %rdi, %rsi, %rdx, %rcx, %r8 and %r9 are the registers in order used to pass integer/pointer (i.e. INTEGER class) parameters to any libc function from assembly. %rdi is used for the first INTEGER parameter. %rsi for 2nd, %rdx for 3rd and so on. Then call instruction should be given. The stack (%rsp) must be 16B-aligned when call executes.
If there are more than 6 INTEGER parameters, the 7th INTEGER parameter and later are passed on the stack. (Caller pops, same as x86-32.)
The first 8 floating point args are passed in %xmm0-7, later on the stack. There are no call-preserved vector registers. (A function with a mix of FP and integer arguments can have more than 8 total register arguments.)
Variadic functions (like printf) always need %al = the number of FP register args.
There are rules for when to pack structs into registers (rdx:rax on return) vs. in memory. See the ABI for details, and check compiler output to make sure your code agrees with compilers about how something should be passed/returned.
Note that the Windows x64 function calling convention has multiple significant differences from x86-64 System V, like shadow space that must be reserved by the caller (instead of a red-zone), and call-preserved xmm6-xmm15. And very different rules for which arg goes in which register.
Perhaps you're looking for the x86_64 ABI?
www.x86-64.org/documentation/abi.pdf (404 at 2018-11-24)
www.x86-64.org/documentation/abi.pdf (via Wayback Machine at 2018-11-24)
Where is the x86-64 System V ABI documented? - https://github.com/hjl-tools/x86-psABI/wiki/X86-psABI is kept up to date (by HJ Lu, one of the ABI maintainers) with links to PDFs of the official current version.
If that's not precisely what you're after, use 'x86_64 abi' in your preferred search engine to find alternative references.
Linux kernel 5.0 source comments
I knew that x86 specifics are under arch/x86, and that syscall stuff goes under arch/x86/entry. So a quick git grep rdi in that directory leads me to arch/x86/entry/entry_64.S:
/*
* 64-bit SYSCALL instruction entry. Up to 6 arguments in registers.
*
* This is the only entry point used for 64-bit system calls. The
* hardware interface is reasonably well designed and the register to
* argument mapping Linux uses fits well with the registers that are
* available when SYSCALL is used.
*
* SYSCALL instructions can be found inlined in libc implementations as
* well as some other programs and libraries. There are also a handful
* of SYSCALL instructions in the vDSO used, for example, as a
* clock_gettimeofday fallback.
*
* 64-bit SYSCALL saves rip to rcx, clears rflags.RF, then saves rflags to r11,
* then loads new ss, cs, and rip from previously programmed MSRs.
* rflags gets masked by a value from another MSR (so CLD and CLAC
* are not needed). SYSCALL does not save anything on the stack
* and does not change rsp.
*
* Registers on entry:
* rax system call number
* rcx return address
* r11 saved rflags (note: r11 is callee-clobbered register in C ABI)
* rdi arg0
* rsi arg1
* rdx arg2
* r10 arg3 (needs to be moved to rcx to conform to C ABI)
* r8 arg4
* r9 arg5
* (note: r12-r15, rbp, rbx are callee-preserved in C ABI)
*
* Only called from user space.
*
* When user can change pt_regs->foo always force IRET. That is because
* it deals with uncanonical addresses better. SYSRET has trouble
* with them due to bugs in both AMD and Intel CPUs.
*/
and for 32-bit at arch/x86/entry/entry_32.S:
/*
* 32-bit SYSENTER entry.
*
* 32-bit system calls through the vDSO's __kernel_vsyscall enter here
* if X86_FEATURE_SEP is available. This is the preferred system call
* entry on 32-bit systems.
*
* The SYSENTER instruction, in principle, should *only* occur in the
* vDSO. In practice, a small number of Android devices were shipped
* with a copy of Bionic that inlined a SYSENTER instruction. This
* never happened in any of Google's Bionic versions -- it only happened
* in a narrow range of Intel-provided versions.
*
* SYSENTER loads SS, ESP, CS, and EIP from previously programmed MSRs.
* IF and VM in RFLAGS are cleared (IOW: interrupts are off).
* SYSENTER does not save anything on the stack,
* and does not save old EIP (!!!), ESP, or EFLAGS.
*
* To avoid losing track of EFLAGS.VM (and thus potentially corrupting
* user and/or vm86 state), we explicitly disable the SYSENTER
* instruction in vm86 mode by reprogramming the MSRs.
*
* Arguments:
* eax system call number
* ebx arg1
* ecx arg2
* edx arg3
* esi arg4
* edi arg5
* ebp user stack
* 0(%ebp) arg6
*/
glibc 2.29 Linux x86_64 system call implementation
Now let's cheat by looking at a major libc implementations and see what they are doing.
What could be better than looking into glibc that I'm using right now as I write this answer? :-)
glibc 2.29 defines x86_64 syscalls at sysdeps/unix/sysv/linux/x86_64/sysdep.h and that contains some interesting code, e.g.:
/* The Linux/x86-64 kernel expects the system call parameters in
registers according to the following table:
syscall number rax
arg 1 rdi
arg 2 rsi
arg 3 rdx
arg 4 r10
arg 5 r8
arg 6 r9
The Linux kernel uses and destroys internally these registers:
return address from
syscall rcx
eflags from syscall r11
Normal function call, including calls to the system call stub
functions in the libc, get the first six parameters passed in
registers and the seventh parameter and later on the stack. The
register use is as follows:
system call number in the DO_CALL macro
arg 1 rdi
arg 2 rsi
arg 3 rdx
arg 4 rcx
arg 5 r8
arg 6 r9
We have to take care that the stack is aligned to 16 bytes. When
called the stack is not aligned since the return address has just
been pushed.
Syscalls of more than 6 arguments are not supported. */
and:
/* Registers clobbered by syscall. */
# define REGISTERS_CLOBBERED_BY_SYSCALL "cc", "r11", "cx"
#undef internal_syscall6
#define internal_syscall6(number, err, arg1, arg2, arg3, arg4, arg5, arg6) \
({ \
unsigned long int resultvar; \
TYPEFY (arg6, __arg6) = ARGIFY (arg6); \
TYPEFY (arg5, __arg5) = ARGIFY (arg5); \
TYPEFY (arg4, __arg4) = ARGIFY (arg4); \
TYPEFY (arg3, __arg3) = ARGIFY (arg3); \
TYPEFY (arg2, __arg2) = ARGIFY (arg2); \
TYPEFY (arg1, __arg1) = ARGIFY (arg1); \
register TYPEFY (arg6, _a6) asm ("r9") = __arg6; \
register TYPEFY (arg5, _a5) asm ("r8") = __arg5; \
register TYPEFY (arg4, _a4) asm ("r10") = __arg4; \
register TYPEFY (arg3, _a3) asm ("rdx") = __arg3; \
register TYPEFY (arg2, _a2) asm ("rsi") = __arg2; \
register TYPEFY (arg1, _a1) asm ("rdi") = __arg1; \
asm volatile ( \
"syscall\n\t" \
: "=a" (resultvar) \
: "0" (number), "r" (_a1), "r" (_a2), "r" (_a3), "r" (_a4), \
"r" (_a5), "r" (_a6) \
: "memory", REGISTERS_CLOBBERED_BY_SYSCALL); \
(long int) resultvar; \
})
which I feel are pretty self explanatory. Note how this seems to have been designed to exactly match the calling convention of regular System V AMD64 ABI functions: https://en.wikipedia.org/wiki/X86_calling_conventions#List_of_x86_calling_conventions
Quick reminder of the clobbers:
cc means flag registers. But Peter Cordes comments that this is unnecessary here.
memory means that a pointer may be passed in assembly and used to access memory
For an explicit minimal runnable example from scratch see this answer: How to invoke a system call via syscall or sysenter in inline assembly?
Make some syscalls in assembly manually
Not very scientific, but fun:
x86_64.S
.text
.global _start
_start:
asm_main_after_prologue:
/* write */
mov $1, %rax /* syscall number */
mov $1, %rdi /* stdout */
mov $msg, %rsi /* buffer */
mov $len, %rdx /* len */
syscall
/* exit */
mov $60, %rax /* syscall number */
mov $0, %rdi /* exit status */
syscall
msg:
.ascii "hello\n"
len = . - msg
GitHub upstream.
Make system calls from C
Here's an example with register constraints: How to invoke a system call via syscall or sysenter in inline assembly?
aarch64
I've shown a minimal runnable userland example at: https://reverseengineering.stackexchange.com/questions/16917/arm64-syscalls-table/18834#18834 TODO grep kernel code here, should be easy.
Calling conventions defines how parameters are passed in the registers when calling or being called by other program. And the best source of these convention is in the form of ABI standards defined for each these hardware. For ease of compilation, the same ABI is also used by userspace and kernel program. Linux/Freebsd follow the same ABI for x86-64 and another set for 32-bit. But x86-64 ABI for Windows is different from Linux/FreeBSD. And generally ABI does not differentiate system call vs normal "functions calls".
Ie, here is a particular example of x86_64 calling conventions and it is the same for both Linux userspace and kernel: http://eli.thegreenplace.net/2011/09/06/stack-frame-layout-on-x86-64/ (note the sequence a,b,c,d,e,f of parameters):
Performance is one of the reasons for these ABI (eg, passing parameters via registers instead of saving into memory stacks)
For ARM there is various ABI:
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.subset.swdev.abi/index.html
https://developer.apple.com/library/ios/documentation/Xcode/Conceptual/iPhoneOSABIReference/iPhoneOSABIReference.pdf
ARM64 convention:
http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055b/IHI0055B_aapcs64.pdf
For Linux on PowerPC:
http://refspecs.freestandards.org/elf/elfspec_ppc.pdf
http://www.0x04.net/doc/elf/psABI-ppc64.pdf
And for embedded there is the PPC EABI:
http://www.freescale.com/files/32bit/doc/app_note/PPCEABI.pdf
This document is good overview of all the different conventions:
http://www.agner.org/optimize/calling_conventions.pdf
Warning - it is rather long question.
Background
I'm working on my small project for instrumenting pe files. At the moment my main focus is to extend .text section of an executable. Not add a new segment, not modify the entry point, but really extend existing .text section.
My approach is very naive. I fully rely on https://learn.microsoft.com/en-us/windows/win32/debug/pe-format and so I'm happy to hear suggestions on how I can improve. Moreover maybe you will spot something I've missed. For now I'm:
fixing all raw/virtual addresses of sections that are after .text
section making sure they are aligned correctly,
fixing some
fields in headers such as size of data, size of code, checksum,
iterating over every data dictionary and fixing all fields which
have RVA.
When comparing files in pe-bear everything seems to be correctly updated.
Current state
I am able to extend small project written by myself which use TLS, exceptions, imports, delay imports, sefe seh and resources. But I fail to extend real, big applications avaible on internet.
Notepad++
One of the applications which I'm unable to extend is Notepad. By this I mean that modified Notepad++ binary crashes. I've traced down a place which seems to be the reason.
push ebp
mov ebp, esp
mov ebx, dword ptr ss:[ebp+8] ; <-- 1) this argument in orginal is equal to 2. In permutated executable it is equal to some huge number
xor ecx, ecx
push edi
xor eax, eax
lea edi, dword ptr ds:[ebx*4 + 0x641fdc] ; <-- and so this causes ACCESS VOLATATION
Some screenshots:
x64dbg - orginal Notepad++ before 1). Base: 0x41000. The ebx will become: 0x2.
x64dbg - extended Notepad++ before 1). Base: 0x42000. The ebx will become: 0x00177FF4
0x146dde rva address is a relocation. Here is a screenshot of this relocation in orginal Notepad++
And in the modified one. Note that only the relocation value has changed from: 0x611fdc to 0x621fdc (which is expected behaviour):
And finaly screenshot of sections:
orginal:
modified:
Link to orginal Notepad++ binary: https://notepad-plus-plus.org/downloads/v7.8.6/
Link to modified Notepad++ binary: https://www.dropbox.com/s/hokh4ulmtgn7om1/notepad%2B%2B.permutated.exe?dl=0
I am creating an 8-bit CPU. I have basic instructions like mov, ld, st, add, sub, mult, jmp. I am trying to put my instructions together. First I move the base address of a value into register 1 (R1). I then want to load register 2 (R2) with the value. So my instructions look:
1 mov R1, 0xFFFF
2 ld R2, [R1+0]
My opcode definitions are:
ld: 0001
mov: 1111
Register codes are:
R1: 0001
R2: 0010
So my instructions in binary look like:
1 mov R1, 0xFFFF = 1111 0001 0xFFFF
2 ld R2, [R1+0] = 0001 00010
But on my second direction for load, how can I ensure the value stored at the memory location I moved to R1 is going to be used. This is my first time doing anything with computer architecture, so I am a little lost.
how can I ensure the value stored at the memory location I moved to R1 is going to be used.
By building your hardware to correctly handle the read-after-write hazard (https://en.wikipedia.org/wiki/Hazard_(computer_architecture)#Data_hazards).
Either
make it a simple non-pipelined CPU where one instruction writes back to registers before the next instruction reads any registers
detect the dependency and stall the pipeline
bypass forwarding. (https://en.wikipedia.org/wiki/Hazard_(computer_architecture)#Eliminating_hazards)
Is it possible to alter the way that an existing x86-64 binary references and/or calls one particular function. Specifically, is it possible to alter the binary such nothing happens (similar to a nop) at the times when that function would normally have executed?
I realize that there are powerful speciality tools out there (ie decompilers/disassemblers) for just this sort of task, but what I'm really wondering is if the executable formats are human-readable "enough" to be able to do this sort of thing (on small programs, at least) with just vim and a hex editor.
Are certain executable file formats (eg mach-o, elf, whatever the heck windows uses, etc.) more readable than others? Are they all just completely incomprehensible gibberish? Any expert views and/or good jumping off points/references would be greatly appreciated.
Disclaimer
Someone came by and quickly downvoted the initial version of this question, so I want to make this perfectly clear: I am not interested in disabling any serial or security checks or anything of the sort. Originally I had wanted a program to stop making a really irritating noise, but now I'm just curious about how compilers and executables work.
I'm in this for the educational value, and I think that other people on SE will be interested in the answer. However, I appreciate that others might not be as comfortable with this topic. If you have a concern about something I've said, please leave a comment and I promise I'll change my post.
This is trivial to do when the function in question is in the binary itself and uses standard calling conventions. Example:
void make_noise() { printf("Quack!\n"); }
int fn1() { puts("fn1"); make_noise(); return 1; }
int fn2() { puts("fn2"); make_noise(); return 2; }
int main() { puts("main"); return fn1() + fn2() - 3; }
gcc -w t.c -o a.out && ./a.out
This outputs (expected):
main
fn1
Quack!
fn2
Quack!
Now let's get rid of the noise:
gdb -q --write ./a.out
(gdb) disas/r make_noise
Dump of assembler code for function make_noise:
0x000000000040052d <+0>: 55 push %rbp
0x000000000040052e <+1>: 48 89 e5 mov %rsp,%rbp
0x0000000000400531 <+4>: bf 34 06 40 00 mov $0x400634,%edi
0x0000000000400536 <+9>: e8 d5 fe ff ff callq 0x400410 <puts#plt>
0x000000000040053b <+14>: 5d pop %rbp
0x000000000040053c <+15>: c3 retq
End of assembler dump.
This tells us a few things:
The function that we want to get rid of starts at address 0x40052d
The op-code of retq instruction is 0xC3.
Let's patch retq as the first instruction of make_noise, and see what happens:
(gdb) set *(char*)0x40052d = 0xc3
(gdb) disas make_noise
Dump of assembler code for function make_noise:
0x000000000040052d <+0>: retq
0x000000000040052e <+1>: mov %rsp,%rbp
0x0000000000400531 <+4>: mov $0x400634,%edi
0x0000000000400536 <+9>: callq 0x400410 <puts#plt>
0x000000000040053b <+14>: pop %rbp
0x000000000040053c <+15>: retq
End of assembler dump.
It worked!
(gdb) q
Segmentation fault (core dumped) ## This is a long-standing GDB bug
And now let's run patched binary:
$ ./a.out
main
fn1
fn2
Voila! No noise.
If the function is in a different binary, LD_PRELOAD techniques mentioned by Florian Weimer is usually easier than binary patching.
ELF dynamic linking implementations often support LD_PRELOAD and LD_AUDIT modules, which can both intercept calls into another shared object. LD_AUDIT offers more control, and exists on GNU/Linux (but the Solaris documentation is the canonical reference).
For calls within the same shared object, this may not be possible if the target function is not exported (or the call is executed via a hidden alias; glibc does this a lot). If you have debugging information, you can use systemtap to intercept the call. If the function is inlined, intercepting the call might not be possible even with systemtap because there is no exact place in the instruction stream where the call takes place.
I am making my first steps in lc3 assembly programming and I noticed that every time I try to store a negative value in memory, for example using "ST" instruction, there is some kind of error. In this memory location is stored "TRAP xFF" instead...
Anybody know how can I get over it??
You're getting that error because your variables are apart of the run-time code. It's usually best practice to put your variables at the end of your code after the HALT command.
.ORIG x3000
MAIN
LD R0, VAR1
NOT R0, R0
ADD R0, R0, #1
ST R0, VAR2
HALT
VAR1 .FILL #5
VAR2 .FILL #0
.END
The reason you were getting those errors is because when you were storing numbers into your variables the simulator thought they were commands. The trap command has an opcode of 1111 which also a negative number. When the simulator ran into your variable it couldn't figure out what type of TRAP command it was, thus the error. By preventing the simulator from running your variables you won't get that error.