In particular the print command typically (80-90% failure rate) does not work
I've verified already:
https://developer.apple.com/library/content/qa/qa1947/_index.html
Example 1
(lldb) p prevMsg
error: Couldn't materialize: couldn't get the value of runOnce: extracting data from value failed error: errored out in DoExecute, couldn't PrepareToExecuteJITExpression
Example 2 A more typical example that puts you in a stone age of computing:
(lldb) p activeNetworkRequests
error: Execution was interrupted, reason: EXC_BAD_ACCESS (code=1, address=0x1700530). The process has been returned to the state before expression evaluation.
This seems to have gotten progressively worse since Xcode 7.
Printing variables scoped from the enclosing function of a closure are particularly hopeless.
The code base is not small, about 15K lines. It would not be practical to isolate and reproduce all the code here.
Surely others are experiencing this?
UPDATE: I'm told about the merits of expression --unwind-on-error=0 -- variable-in-question, presumably for example2
UPDATE 2:
Code:
Util.log("Returning \(key) from file cache", [.Caches])
Output:
08:03:11.201 v2.0.64d other TwoStageCache.swift objectForKey(_:completion:)[95]: Returning https://example.server.com/Storage/Retrieve?FileName=accounts/person#domain.com/resource/47a58660-26d1-11e7-8e7f-c9f4cd679b03.html from file cache
(So the value of key is fine)
(lldb) fr var key
(URL) key = unable to read data
(lldb) print key
error: Execution was interrupted, reason: EXC_BAD_ACCESS (code=1, address=0x1d787583).
The process has been returned to the state before expression evaluation.
If we look at the crash:
(lldb) expression --unwind-on-error=0 -- key
libobjc.A.dylib`objc_retain:
0x22562b0 <+0>: pushl %ebp
0x22562b1 <+1>: movl %esp, %ebp
0x22562b3 <+3>: subl $0x8, %esp
0x22562b6 <+6>: calll 0x22562bb ; <+11>
0x22562bb <+11>: popl %ecx
0x22562bc <+12>: movl 0x8(%ebp), %eax
0x22562bf <+15>: testl %eax, %eax
0x22562c1 <+17>: je 0x22562e1 ; <+49>
0x22562c3 <+19>: movl (%eax), %edx
-> 0x22562c5 <+21>: testb $0x2, 0x10(%edx)
From:
1 $__lldb_expr(UnsafeMutablePointer<Any>) -> ()
2 Beta Viewer`#objc AppDelegate.init() -> AppDelegate:
3 sharedEnchantment`partial apply for TwoStageCache.(objectForKey(URL, completion : (imgData : Data?, err : BBError?) -> ()) -> ()).(closure #1)
4 sharedEnchantment`thunk:
Sorry in advance for the essay, but hopefully the info will be worth the read...
lldb has two ways of looking at variables(*): print and frame variable.
print isn't really meant primarily for printing variables - that's just a side effect of what it really does. print is an alias for expression which gives you a little more sense of what it is: a full expression evaluator which runs the expression you pass at the point where you are stopped in your code.
It builds a context that emulates the code at the current pc (including the Class/Protocol context) and then takes the snippet you pass it, compiles it in that context, JIT's the result, inserts the JIT'ed code into the process you are debugging and runs it. That's quite powerful - you can change values, call functions in your program, introduce new functions, new types, etc. But there is also a lot of machinery just to get it going, and with swift some of this machinery is tricky to get right.
frame variable can only print locals and arguments in the current frame (with the -g flag it can also print globals & statics). It can't call functions, or any of the other fancy things print can do. It does understand a limited subset of the variable access syntax, so:
(lldb) frame variable foo.bar.baz
will work. But under the covers, all it needs to do is read the debug information to find the variable, its type, and where it is in memory, and then it can extract the value from that information. So it is faster and more robust for what it does do - which is a large percentage of the work people generally ask print to do.
Note, you can get "object printing" for variables you access with frame variable by using the -O flag, and it supports the same formatting options for results as print. For context, the Xcode "Locals" view is roughly equivalent to calling frame variable.
I tend to use frame variable for simple locals printing, but even if you like to use one command for all your needs - which will be print - it's good to know that there's a fallback if print is failing for some reason.
Back to your examples...
Example 1: one of the things print does in Swift is introduce all the visible local variables into the context of the expression, so they are available to your code. The error in Example 1 is because one of the local variables couldn't be realized - maybe it was a only specified by a protocol conformance and we couldn't figure out what it really was - so we failed building the context, which means the parse or JIT steps failed. The print code does a pre-scan for this sort of failure and omits failing locals, but you've found a case this scan misses.
frame variable would have probably also failed to print runOnce but since it doesn't depend on the current context, the inability to do that wouldn't have affected your ability to print other variables.
If you can reproduce this issue, even if you can't make the project available to us we can often figure out what's going on from lldb's debugging log. So drive the debug session to the point where the print is going to fail, and do:
(lldb) log enable -f /tmp/lldb-log.txt lldb expr types
then run the failing expression. Then grab that log, and file a bug as described here:
https://swift.org/contributing/#reporting-bugs
Example 2: Was activeNetworkRequests a property? Those require us to call the "get" method to access them, and I have seen a few cases where lldb doesn't emit the code to call the property getters correctly. The log above will show us the code that was emitted, and we might be able to tell from there what went wrong. Of course, if you can make a test case you can send with the bug that is always best, but that's often not possible...
(*)For gdb users this is pretty close to the info locals vrs. print...
Related
Is it possible to alter the way that an existing x86-64 binary references and/or calls one particular function. Specifically, is it possible to alter the binary such nothing happens (similar to a nop) at the times when that function would normally have executed?
I realize that there are powerful speciality tools out there (ie decompilers/disassemblers) for just this sort of task, but what I'm really wondering is if the executable formats are human-readable "enough" to be able to do this sort of thing (on small programs, at least) with just vim and a hex editor.
Are certain executable file formats (eg mach-o, elf, whatever the heck windows uses, etc.) more readable than others? Are they all just completely incomprehensible gibberish? Any expert views and/or good jumping off points/references would be greatly appreciated.
Disclaimer
Someone came by and quickly downvoted the initial version of this question, so I want to make this perfectly clear: I am not interested in disabling any serial or security checks or anything of the sort. Originally I had wanted a program to stop making a really irritating noise, but now I'm just curious about how compilers and executables work.
I'm in this for the educational value, and I think that other people on SE will be interested in the answer. However, I appreciate that others might not be as comfortable with this topic. If you have a concern about something I've said, please leave a comment and I promise I'll change my post.
This is trivial to do when the function in question is in the binary itself and uses standard calling conventions. Example:
void make_noise() { printf("Quack!\n"); }
int fn1() { puts("fn1"); make_noise(); return 1; }
int fn2() { puts("fn2"); make_noise(); return 2; }
int main() { puts("main"); return fn1() + fn2() - 3; }
gcc -w t.c -o a.out && ./a.out
This outputs (expected):
main
fn1
Quack!
fn2
Quack!
Now let's get rid of the noise:
gdb -q --write ./a.out
(gdb) disas/r make_noise
Dump of assembler code for function make_noise:
0x000000000040052d <+0>: 55 push %rbp
0x000000000040052e <+1>: 48 89 e5 mov %rsp,%rbp
0x0000000000400531 <+4>: bf 34 06 40 00 mov $0x400634,%edi
0x0000000000400536 <+9>: e8 d5 fe ff ff callq 0x400410 <puts#plt>
0x000000000040053b <+14>: 5d pop %rbp
0x000000000040053c <+15>: c3 retq
End of assembler dump.
This tells us a few things:
The function that we want to get rid of starts at address 0x40052d
The op-code of retq instruction is 0xC3.
Let's patch retq as the first instruction of make_noise, and see what happens:
(gdb) set *(char*)0x40052d = 0xc3
(gdb) disas make_noise
Dump of assembler code for function make_noise:
0x000000000040052d <+0>: retq
0x000000000040052e <+1>: mov %rsp,%rbp
0x0000000000400531 <+4>: mov $0x400634,%edi
0x0000000000400536 <+9>: callq 0x400410 <puts#plt>
0x000000000040053b <+14>: pop %rbp
0x000000000040053c <+15>: retq
End of assembler dump.
It worked!
(gdb) q
Segmentation fault (core dumped) ## This is a long-standing GDB bug
And now let's run patched binary:
$ ./a.out
main
fn1
fn2
Voila! No noise.
If the function is in a different binary, LD_PRELOAD techniques mentioned by Florian Weimer is usually easier than binary patching.
ELF dynamic linking implementations often support LD_PRELOAD and LD_AUDIT modules, which can both intercept calls into another shared object. LD_AUDIT offers more control, and exists on GNU/Linux (but the Solaris documentation is the canonical reference).
For calls within the same shared object, this may not be possible if the target function is not exported (or the call is executed via a hidden alias; glibc does this a lot). If you have debugging information, you can use systemtap to intercept the call. If the function is inlined, intercepting the call might not be possible even with systemtap because there is no exact place in the instruction stream where the call takes place.
I'm writing some shellcode, and I would like to know how to properly call a function from assembly that resides in libc.Note that the address below is the address of the system function. I'm certain I've got this address correct, because if I overwrite the return address with this address, system is called successfully, but it seems to segfault within the assembly (which is contained in a buffer). If anything is unclear, let me know. I'm trying to get a libc function call working on a executable stack, andI'm pulling my hair out here. Note that the code reaches the buffer okay, and starts going through the appropriate nop sled, but segfaults on the call instruction (code below).
mov 0xf7ff7fa5b640, %rax
mov (hex representation of /bin/sh), %rdi
call *%rax
You can set a breakpoint on the closing } of a Scala method, but it's pointless to do so because it won't be hit, apparently.
I would still like to set it there. So I thought, "how about I put in a no-op before that line, and set a breakpoint on that?"
But since evidently Eclipse is not warning me when I try to set a breakpoint that will never be hit (because there is no code there), I therefore can't rely on Eclipse telling me if a no-op has been optimised out (particularly as I'm not even using the same version of Scala to run the code as the Eclipse Scala plugin is using).
So is there a short no-operation statement or expression that I can use here which is guaranteed not to be optimised away by the Scala compiler, in all circumstances - and guaranteed not to be optimised away by a JIT in a way that prevents a breakpoint on it being hit? I guess it has to be an expression rather than a statement in my case, because this method returns a useful value, not Unit.
It's not exactly a no-op, but a logging statement (or trace expression - i.e. a logging statement which additionally returns the value it has just logged) is guaranteed not to be optimised away!
The only trouble with that is, you have to manually step over the debug statement or trace expression each time you hit the breakpoint, if you want to see what it actually outputs. Unless it's just outputting the value of a variable, in which case you can find that variable in the Variables tab in Eclipse.
And stepping over a trace expression, at least in my code, with the Eclipse Scala plugin 2.1 milestone 2, in "old-style debugging mode", is no small matter - because it's completely broken, due to this issue!
Before I realised what was going on, it took me about 20 tries of pounding the F6 and F7 keys before I finally managed to step over it! And that was just for stepping over it once!
And remember, the next line is the } - so the obvious workaround, setting another breakpoint on the next line - won't work!
Theoretically you should just be able to Inspect the value being traced, rather than trying to step over the trace. Unfortunately, that doesn't work for me either - it says class not found.
After swaping compiler to LLVM 3.0 and libc++ to have C++11 suport (unique_ptr is a little jewel :)) I have noticed gdb is not working properly when I try to print an object information. What I have tested is the following:
1) I have a std::vector < std::unique_ptr > member variable. If I set a breakpoint inside the class containing this member var and do a "print _reusableEntities.size()" I get this:
Breakpoint 2, GameObjectMgr::createGameObject (this=0x0) at /Users/hexdump/Dropbox/Games prototypes/Re-World/src/ReWorld/Engine/EntityManagement/GameObjectMgr.mm:92
92 go=std::move(_reusableEntities.back());
The program being debugged stopped while in a function called from GDB.
When the function (GameObjectMgr::createGameObject()) is done executing, GDB will silently
stop (instead of continuing to evaluate the expression containing
the function call).
But if I have a line of code like:
int size=_reusableEntities.size()
it reads the size ok. It really seems a gdb problem when inspecting the vector size. In the other hand I can see how the vector contains correct values in the Local vars windows.
Another thing that is really weird is why the error shows information related to createGameObject function because It has nothing to do with the gdb command call and I wasn't inside it when trying to print the values.
It is really weird and a pain in the ass when debugging.
2) I have F7 configured to step over functions. This worked until I made this changes, now, it doesn't matter if I push F6 (step into) or F7 (step over) I'm always getting inside unique_ptr code when copying, or getting raw pointer from it :/.
If anyone could guess what is happening I would be really grateful, anyway, I know it is really difficult to guess from just my explanation. If anyone needs more info, please, ask for it.
I've only recently began using the debugger extensively, so I'm not sure if this is a limitation.
When I debug on the iPhone, the variables aren't up to date unless I explicitly view it (ctrl+click -> view variable as expression). Is there a way to view actual variables without viewing explicitly?
Can you clarify your question? You should only be viewing data while the program is stopped -- examining data while it's running, if it works at all, is much less useful. Make sure to set a breakpoint, and then examine data once you've hit the breakpoint.
An alternative to using Xcode's built-in debugging features is to use the gdb console. Type ⌘-Shift-R, or select "Debugging Console" from the menu to open the console. Then, you can type commands like:
# View a variable
print var
# View this object's member variable
print self->memberVar
# Ask an Objective-C object to print itself:
print-object self
You should use print with primitive types (int, char*, etc.) and POD types (structs); you should use print-object with Objective-C objects (NSString, etc.). For more information about print and print-object, type
help print
help print-object
You can also use the abbreviations p and po for print and print-object respectively.