I'm working with an Objective-C wrapper around SQLite that I didn't write, and documentation is sparse...
It's not FMDB. The people writing this wrapper weren't aware of FMDB when writing this code.
It seems that the code is suffering from a bug where database connections are being accessed from multiple threads -- which according to the SQLite documentation won't work if the if SQLite is compiled with SQLITE_THREADSAFE 2.
I have tested the libsqlite3.dylib provided as part of the iPhone SDK and seen that it is compiled in this manner, using the sqlite_threadsafe() routine.
Using the provided sqlite library, the code regularly hits SQLITE_BUSY and SQLITE_LOCKED return codes when performing routines.
To combat this, I added some code to wait a couple of milliseconds and try again, with a maximum retry count of 50. The code didn't contain any retry logic prior to this.
Now when a sqlite call returns SQLITE_BUSY or SQLITE_LOCKED, the retry loop is invoked and the retry returns SQLITE_MISUSE. Not good.
Grasping at straws, I replaced the provided sqlite library with a version compiled by myself setting SQLITE_THREADSAFE to 1 - which according to the documentation means sqlite is safe to be used in a multithreaded environment, effectively serialising all of the operations. It incurs a performance hit, that which I haven't measured, but it ridded the app of the SQLITE_MISUSE happening and seemed to not need the retry logic as it never hit a busy or locked state.
What I would rather do is fix the problem of accessing a single db connection from multiple threads, but I can't for the life of me find where it's occurring.
So if anyone has any tips on locating multithreaded bugs I would be extremely appreciative.
Thanks in advance.
I haven't tested this to see if it works, but you could try wrapping access to the db connection in a function and then using Instruments to log accesses to that function which, IIRC, should let you get a thread ID and a stack trace. That should give you a bit of a handle on where it's coming from. Alternatively you could just set a breakpoint on it but that might take a bit longer.
Related
I work on an average (~ 20k lines of code, Objective-C mixed with C++), and I am figthing to hunt down an EXC_BAD_ACCESS error.
I have tried all the common techniques (like enabling NSZombie, guard edges,etc.) So far, I have ruled out the possibility to access a released object, and the double-free error.
It seems that something writes on a memory space where it shouldn't. But, as many memory errors, it's not happening all the time, and it's not crashing always in the same place.
(Sometimes I receive the "object was modified after being freed" message).
Sometimes, the overwritten memory belongs to the allocator, and it crashes on malloc, or on free().
And, of course, some changes in the app may influence the bug's behaviour - if I try to comment out parts of the code, the error appears less often, so it's more difficult to find it.
Finally, I have been looking into using valgrind, but it seems that all those who used it worked on the simulator. but my code must run on the actual device (some code is ARM-specific)
Are there any general tips on how to debug such errors?
Note: The app involves video processing, so the amount of memory used is fairly large.
There are some special tools available on the XCode. You could try to use them in order to analyse your code.
http://developer.apple.com/library/mac/#featuredarticles/StaticAnalysis/index.html
It will produce some warning in case of invalid objects usage so it could help you to find a problem.
If you feel that the C++ code is causing the issue you could copy the C++ out of your iPhone project and create a Mac project. With this you could set up various stress tests. And, you should be able to use valgrind as well.
I am currently programming a iPhone-app for my maturity research. But there is an behavior I don't understand: Sometimes when i compile my project there is:
Thread 1: Program received signal : "EXC_BAD_ACCESS".
But when I compile the same code a second, or a third time the code just runs fine and i can't get why. I use some MonteCarloSimulation but when it fails it fails executing one of the first 100 simulations. But when every thing runs fine it executes 1000000 simulations without an error.. Really strange isn't it?
Do you have any idea? Can this be an issue of Xcode or arc?
All other things just work perfect.
Do you have to get any further information? I can also send you my code as an email.
This usually means you're trying to access an object that has already been deallocated.
In order to debug these things, Objective C uses something called "NSZombie" that will keep those objects around so you can at least see what it is that's trying to be called. See this question for some details on how to use it.
This is typically caused by accessing some memory that's been corrupted, chances are you have a reference to an object which has been deleted. A lot of the time you may find that the memory where the object was located has not yet been overwritten, so when you attempt to access that memory your data is still intact and there is no problem, hence it working some of the time.
Another scenario would be that you've got some code writing into memory using a bad reference, so you're writing into an area you shouldn't be. Depending on the memory layout when the program starts, this could have no effect some of the time but cause something catastrophic at other times.
I have read a few tutorials and did some search but I must be missing something.
I've had a bunch of different problems with sqlite on my iphone app the last few days.
The most common problem was crashing when I was doing inserts at the same time on different tables, not on the same table.
I can achieve several inserts using threads by the way, so I am sure there is concurrency.
So does the db lock the whole DB or just the table I am inserting information into?
I'm also looking into another possible problems who might be causing the crashes but this little piece of information would help a lot.
After further tests I can say with certainty that you can perform multiple inserts at the same time in the database as long as it is on different tables. Multiple inserts at the same time on the same table will throw an error.
The crashes I was having earlier seem to be related to the fact that I was init + alloc the db on each view I wanted to use it and then releasing it on each view.
After init+alloc only on the app delegate it solved the crashes.
Although you can have multiple instances of the same DB throughout your App it seems to be preferable to just having one in the App Delegate and accessing it as you need it.
This might not be entirely correct but was my conclusion after several hours searching and testing.
I hope this helps other people.
There are 5 types of locks, but all are related to the whole database file and not a table-lock, or not to mention row-lock.
Are you calling sqlite3_finalize at the proper times when you're done? I've noticed that failure to call sqlite3_finalize in sqlite will pretty much bring things to a halt.
I trying to write a a crash report feature that when you launch the app after a crash, it will offer to send the crash report to the server. I can't find how to get the crash log within the app. I saw there is a framework that doing so (PLCrashReporter), however this framework is large and I don't need most of it's features.
Does anyone knows how to simply access the log?
Thanks,
Guy.
I guess I don't have the karma to add a comment to Nimrod Gat's answer, so I have to provide my follow-up here. I'll try to make it worthy of a standalone answer.
It's very, very difficult to write a safe, correct, and reliable crash reporter, especially one that runs directly in-process. The code referenced in Nimrod Gat's answer is not correct and honestly, that blog post should be retracted. Signal handlers must only run async-safe code, and that code isn't async-safe:
http://www.cocoadev.com/index.pl?SignalSafety
Crash handling is even more complicated than normal signal handling, because that you can't expect the process to continue to run successfully after your signal handler returns.
It's tempting to think you can just hack together a simpler solution, and it will work some of the time, but there's a good reason people like Google's engineers have thousands of LoC dedicated to reliable crash reporting:
http://code.google.com/p/google-breakpad/
On iOS, you should just use PLCrashReporter. On other platforms (such as Mac OS X) you should use Google Breakpad. There's no point in re-inventing this wheel unless you're going to do it not only correctly, but better than what already exists.
I had this similar issue and the PLCrashReported seemed too complicated for what I wanted to do.
Note that you can't access the crash report generated by Apple, the PLCrashReport generates it's own reports and store them in the user's cache folder.
Eventually, I used the following example:
http://cocoawithlove.com/2010/05/handling-unhandled-exceptions-and.html
it's very simple and easy to use, just register exception and signal handlers using:
NSSetUncaughtExceptionHandler(&HandleException);
signal(SIGABRT, SignalHandler);
signal(SIGILL, SignalHandler);
signal(SIGSEGV, SignalHandler);
signal(SIGFPE, SignalHandler);
signal(SIGBUS, SignalHandler);
signal(SIGPIPE, SignalHandler);
and get the stack trace using the backtrace method in UncaughtExceptionHandler class.
Maybe a better solution will be to use a fully specialized end-2-end solution/service? Such as http://apphance.com. It is currently in closed beta testing phase, but you can ask for participation and we'll get back to you pretty quickly. The only thing you need to do is to register for an API key and embed a small framework library into your app (or .jar file in android). Then you have remote access not only to crashlogs but also to a debug logs generated by the application - which makes it much more useful. It is for now targeted to be used during testing, but there will soon be a lite version that you will be able to embed into app-store-released application.
Inside the framework we are doing all the magic of plugging-into the apple's framework and getting the crashlog information, decoding stack traces, even handling out-of-memory cases. All the comments by #nupark hold very much true: We spend countless hours on making it works seamlessly - thread-safeness, making sure that we can do the job of saving everything within the time required by Apple's framework before your app get finally killed, getting stack trace from out-of-memory case (that one was really difficult). The same for android - we've done some clever tricks there to make sure it's really working fine.
Disclaimer: I am CTO of Polidea, company which is behind apphance and co-creator of the solution.
There are a bunch of (SAAS) E2E solutions that you may be very happy to know.
Very very simple to integrate into your application
Have Fun...
crashlytics (Free and my preferred)
hockeyapp
bugSense
Crittercism
In our days you may use the built-in crash reports (iOS & Android)
iOS (Itunes connect) - Viewing Crash Reports
Understanding Crash Reports
on iPhone OS
Reading Android Market Crash Reports
Is it possible to read from a SQLite db while it's being written to?
I'm aware that access is blocked for writes when it's being written to, but is that the same for reads?
It's a little convoluted, but check out the File Locking and Concurrency documentation for SQLite. It sounds like if the db is in exclusive locked mode, that is the only time reads are not allowed. Besides unlocked, when the db is not even open.
I'm not 100% sure on it, but that's what I think it means.
After some reading around, and looking through the FMDB code, I discovered that I wasn't using the SQLITE_BUSY and SQLITE_LOCKED return values correctly.
According to the FMDB code, one should loop for a limited number of retries, waiting for a short time bewteen.
So I implemented that in my code, and everything seems to work fine.
Yes, although all the relevent multithreading issues need to be account for. I generally open multiple handles to the same DB file via sqlite3_open_v2().