I have a very strange memory leak problem, it seems that sqlite3_step is doing some nasty stuff :|
I spent almost 4 hours trying to fix this but no luck till now :(
Here it is the code:
[dbList removeAllObjects];
sqlite3_stmt *statement = nil;
const char *sql = "SELECT * FROM dbs ORDER by rowOrder;";
if (sqlite3_prepare_v2(dbHandler, sql, -1, &statement, NULL) == SQLITE_OK)
{
while (sqlite3_step(statement) == SQLITE_ROW)
{
DatabaseEntry *entry = [[DatabaseEntry alloc] init];
entry.databaseID = sqlite3_column_int(statement, 0);
entry.databaseTitle = [NSString stringWithFormat:#"%s", (char *)sqlite3_column_text(statement, 1)];
entry.databaseProtected = sqlite3_column_int(statement, 3);
entry.databaseFileName = [NSString stringWithFormat:#"%s", (char *)sqlite3_column_text(statement, 2)];
entry.databaseOrder = sqlite3_column_double(statement, 4);
[dbList addObject:entry];
[entry release];
}
}
sqlite3_finalize(statement);
The problem seems to be with my query, if I remove the "ORDER by rowOrder" part, everything seems to be just fine, also I'm using sqlcipher, and I'm wondering if that might cause this leak ?!
Thanks a lot for your attention !!!
Update: Hey Andy, I was wrong. I started looking into this more closely after running through some scenarios in Leaks. It looks like a bad merge from the upstream SQLite source missed two pager cleanup calls. The issue caused page cache to remain allocated after the pager was closed. It probably wouldn't affect most programs, but I would still definitely recommend pulling down the latest source code that fixes the issue from GitHub at http://github.com/sjlombardo/sqlcipher
Related
I'm writing an app using a SQLite db to store information. My db table includes 5 fields: gameID (the primary key, int), gameName (Varchar), isLocked, isHidden, and isFavorite (all bools). I've set up the db using SQLite Manager in FireFox.
+ (void) getInitialDataToDisplay:(NSString *)dbPath {
FightingGamesGuideAppDelegate *appDelegate = (FightingGamesGuideAppDelegate *)[[UIApplication sharedApplication] delegate];
if (sqlite3_open([dbPath UTF8String], &database) == SQLITE_OK) {
//My second thought is that this line is incorrect. (see below)
const char *sql = "select gameID, gameName, isLocked, isHidden, isFavorite from GameTable";
sqlite3_stmt *selectstmt;
if(sqlite3_prepare_v2(database, sql, -1, &selectstmt, NULL) == SQLITE_OK) {
while(sqlite3_step(selectstmt) == SQLITE_ROW) {
NSInteger primaryKey = sqlite3_column_int(selectstmt, 0);
Games *coffeeObj = [[Games alloc] initWithPrimaryKey:primaryKey];
//getGameName
coffeeObj.gameName = [NSString stringWithUTF8String:(char *)sqlite3_column_text(selectstmt, 1)];
//I think these three lines are where the problem is
coffeeObj.isLocked=(BOOL)sqlite3_column_int(selectstmt, 2);
coffeeObj.isHidden=(BOOL)sqlite3_column_int(selectstmt, 3);
coffeeObj.isFavorite=(BOOL)sqlite3_column_int(selectstmt, 4);
//This line is only to check if the data loaded properly
NSLog(#"%i, %i, %i",isLocked, isHidden, isFavorite);
[appDelegate.coffeeArray addObject:coffeeObj];
[coffeeObj release];
}
}
}
else{
sqlite3_close(database); //Even though the open call failed, close the database connection to release all the memory.
}
}
My question: Is this how to get a bool from a SQLite db? Is there a better way, such as using an int as a bool instead? If my code is all correct, do you have any other suggestion for how I can do this?
I know that the code aside from those three lines (and the following line) work properly. In the db, I've randomly assigned the three bools to YES or NO. I've also tried 1 and 0, to no effect.
If the lines are correct, my second thought is that line 9 is wrong. I modified it from "const ... ="select gameID, gameName from GameTable", and gameID and gameName still work properly.
My third thought is that I'm not updating the db correctly. I won't go into that, besides to ask if there's a specific "save" button/list item. If I update a gameName, that shows in my app, suggesting the db saves automatically.
In looking through the other questions, I've only found suggestions that CoreData is a better idea. However, if I'm on the right track, I only need these three lines to work, and I'm done (with this part).
Aside-I know that nothing is done with the bool variables besides printing them to the console. That's my next challenge.
Thank you for your time!
In addition to p.campbell's answer (+1), you could recall your BOOL value like this if you are storing YES/NO in your DB:
coffeeObj.isLocked=[[NSString stringWithUTF8String:(char *)sqlite3_column_text(selectstmt, 2)] isEqualToString:#"YES"];
coffeeObj.isHidden=[[NSString stringWithUTF8String:(char *)sqlite3_column_text(selectstmt, 3)] isEqualToString:#"YES"];
coffeeObj.isFavorite=[[NSString stringWithUTF8String:(char *)sqlite3_column_text(selectstmt, 4)] isEqualToString:#"YES"];
The SQLite datatypes page has the answer you're looking for:
SQLite does not have a separate Boolean storage class. Instead, Boolean values are stored as integers 0 (false) and 1 (true).
I want to completely clean all the contents in a SQLite3 table in my iPhone app. My MySQL experience told me that I should use TRUNCATE. But it seems that SQLite3 doesn't support TRUNCATE (at least I got an error when preparing statement when I use the sentence TRUNCATE my_table_Name).
So I turn to DELETE FROM. Two questions: 1) Will DELETE FROM clean my table in a thorough way? 2) I tried the following code and it worked. But I highly doubted that there are unnecessary or even wrong codes in it. Can anyone help me take a look at it?
Thanks in advance.
Di
-(void)deleteAllUser {
NSString *ns_sql = [NSString stringWithFormat:#"DELETE FROM %#", [Config USER_TABLE_NAME]];
const char *sql = [ns_sql cStringUsingEncoding:NSUTF8StringEncoding];
sqlite3_stmt *statement = nil;
sqlite3* db = ...; //get the database instance
if(sqlite3_prepare_v2(db, sql, -1, &statement, nil) != SQLITE_OK) {
return;
}
//Without this line, table is not modified
int code = sqlite3_step(statement);
if (code == SQLITE_ROW) {
//Do nothing here...
}
sqlite3_finalize(statement);
}
I think
if (code == SQLITE_ROW) {
//Do nothing here...
}
this code should be like
if (code == SQLITE_DONE) {
//Do nothing here...
}
If you want to get any message then you can return YES from here and NO from else(if you change return type to BOOL from void).This can help you in showing messaeges.
I'm inserting a batch of 100 records, each containing a dictonary containing arbitrarily long HTML strings, and by god, it's slow. On the iphone, the runloop is blocking for several seconds during this transaction. Is my only recourse to use another thread? I'm already using several for acquiring data from HTTP servers, and the sqlite documentation explicitly discourages threading with the database, even though it's supposed to be thread-safe... Is there something I'm doing extremely wrong that if fixed, would drastically reduce the time it takes to complete the whole operation?
NSString* statement;
statement = #"BEGIN EXCLUSIVE TRANSACTION";
sqlite3_stmt *beginStatement;
if (sqlite3_prepare_v2(database, [statement UTF8String], -1, &beginStatement, NULL) != SQLITE_OK) {
printf("db error: %s\n", sqlite3_errmsg(database));
return;
}
if (sqlite3_step(beginStatement) != SQLITE_DONE) {
sqlite3_finalize(beginStatement);
printf("db error: %s\n", sqlite3_errmsg(database));
return;
}
NSTimeInterval timestampB = [[NSDate date] timeIntervalSince1970];
statement = #"INSERT OR REPLACE INTO item (hash, tag, owner, timestamp, dictionary) VALUES (?, ?, ?, ?, ?)";
sqlite3_stmt *compiledStatement;
if(sqlite3_prepare_v2(database, [statement UTF8String], -1, &compiledStatement, NULL) == SQLITE_OK)
{
for(int i = 0; i < [items count]; i++){
NSMutableDictionary* item = [items objectAtIndex:i];
NSString* tag = [item objectForKey:#"id"];
NSInteger hash = [[NSString stringWithFormat:#"%#%#", tag, ownerID] hash];
NSInteger timestamp = [[item objectForKey:#"updated"] intValue];
NSData *dictionary = [NSKeyedArchiver archivedDataWithRootObject:item];
sqlite3_bind_int( compiledStatement, 1, hash);
sqlite3_bind_text( compiledStatement, 2, [tag UTF8String], -1, SQLITE_TRANSIENT);
sqlite3_bind_text( compiledStatement, 3, [ownerID UTF8String], -1, SQLITE_TRANSIENT);
sqlite3_bind_int( compiledStatement, 4, timestamp);
sqlite3_bind_blob( compiledStatement, 5, [dictionary bytes], [dictionary length], SQLITE_TRANSIENT);
while(YES){
NSInteger result = sqlite3_step(compiledStatement);
if(result == SQLITE_DONE){
break;
}
else if(result != SQLITE_BUSY){
printf("db error: %s\n", sqlite3_errmsg(database));
break;
}
}
sqlite3_reset(compiledStatement);
}
timestampB = [[NSDate date] timeIntervalSince1970] - timestampB;
NSLog(#"Insert Time Taken: %f",timestampB);
// COMMIT
statement = #"COMMIT TRANSACTION";
sqlite3_stmt *commitStatement;
if (sqlite3_prepare_v2(database, [statement UTF8String], -1, &commitStatement, NULL) != SQLITE_OK) {
printf("db error: %s\n", sqlite3_errmsg(database));
}
if (sqlite3_step(commitStatement) != SQLITE_DONE) {
printf("db error: %s\n", sqlite3_errmsg(database));
}
sqlite3_finalize(beginStatement);
sqlite3_finalize(compiledStatement);
sqlite3_finalize(commitStatement);
The thing that you need to be aware of is that the SQLite documentation warns you away from accessing/writing to the database from multiple threads. As long as you access the database from a single thread, you'll be fine. It doesn't matter if that thread is your program's main thread or some other thread.
Keep in mind that compiled version of SQLite on the iPhone has its threading mode set to "multi-thread" which, according to the documentation, "disables mutexing on database connection and prepared statement objects. The application is responsible for serializing access to database connections and prepared statements but other mutexes are enabled so that SQLite will be safe to use in a multi-threaded environment as long as no two threads attempt to use the same database connection at the same time." So, if you do decide to put this transaction on another thread, be careful of what else you try to do with the database.
That being said, I'd first follow Yonel's advice and switch to "BEGIN" AND "COMMIT". If that doesn't help, move the transaction to another thread. Working with "blobs" can be pretty slow, from what I've heard.
Did you try the same as your code but with "BEGIN" and "COMMIT" instead of "BEGIN EXCLUSIVE TRANSACTION" and "COMMIT TRANSACTION" ?
I'm simply using BEGIN and COMMIT and it's pretty much faster than committing for each transaction so I guess it's working with those keywords.
http://www.sqlite.org/lang_transaction.html
I see a lot of cases where developers new to the iPhone believe the code is slow when it is simply a case of asking lightweight hardware to do to much processing. Processing several hundred (thousands?) of "arbitrarily long HTML strings" might to heavy a task for the iPhone to carry out in a timely fashion.
Remember that the iPhone isn't a very powerful piece of hardware. It pulls off all the nifty graphics with dedicated hardware whose computational power you can't access for other task. Even if you optimize the code it maybe way slower than you would intuitively expect based on your experience with full fledged laptops and desktops.
Instead of guessing where the bottleneck is, I suggest that you profile the code with Instruments (or even just using NSLog with timestamps) to see exactly where the code is spending most of its time.
A better approach to avoid blocking problems is to use an asynchronous callbacks.
Try using Enorm EGO sqlite wrapper https://github.com/jdp-global/egodatabase
Have a look at my Readme section for EGODatabaseRequest - asynchronous requests /inserts to db.
2) Add the requestDidSucceed /requestDidFail callback methods.
-(void)requestDidSucceed:(EGODatabaseRequest*)request withResult:(EGODatabaseResult*)result
idx++
if ([items count]<idx) [self insertRow];
}
-(void)requestDidFail:(EGODatabaseRequest*)request withError:(NSError*)error{
NSLog(#"WARNING requestDidFail");
}
-(void)insertRow{
NSMutableDictionary* item = [items objectAtIndex:idx];
NSInteger hash = [[NSString stringWithFormat:#"%#%#", tag, ownerID] hash];
NSInteger timestamp = [[item objectForKey:#"updated"] intValue];
NSData *dictionary = [NSKeyedArchiver archivedDataWithRootObject:item];
NSString *qry = [NSString stringWithFormat:#"INSERT OR REPLACE INTO item (hash, tag, owner, timestamp, dictionary) VALUES (%#, %#, %#, %#, %#);",NUMBER(hash),[tag UTF8String],[ownerID UTF8String],NUMBER(timestamp),dictionary];
// be sure to use NSNumbers not NSIntegers
EGODatabaseRequest* request = [[EGODatabaseRequest alloc] initWithQuery:qry parameters:nil];
request.delegate = self;
request.database = appDelegate.database;
request.requestKind = EGODatabaseUpdateRequest; // use update not select
[request fire];
[request release];
}
I am developing a app for a movie in which I need to fetch data from the database considering some constraints. It works perfectly on the first occasion, but when I try to fetch data 2nd time it throws a runtime exception( the app crashes).I have to bind 3 placeholders. 2 are text and 1 is integer type. Here's the code which I am using to fetch data from the database.
-(void) Data2
{
databaseName = #"Cinema1.sqlite";
NSArray *documentPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory,NSUserDomainMask, YES);
NSString *documentsDir = [documentPaths objectAtIndex:0];
databasePath =[documentsDir stringByAppendingPathComponent:databaseName];
[self checkAndCreateDatabase];
sqlite3 *database;
if (sqlite3_open([databasePath UTF8String], &database) == SQLITE_OK)
{
if(detailStmt == nil)
{
const char *sql = "Select PVR,Fame,Cinemax,Big from listing where UPPER(State) = UPPER(?) and UPPER(City) = UPPER(?) and ZIP = ?";
if(sqlite3_prepare_v2(database, sql, -1, &detailStmt, NULL) == SQLITE_OK)
{
NSLog(#"Hiiiiiii");
sqlite3_bind_text(detailStmt, 1, [t1 UTF8String], -1, SQLITE_TRANSIENT);
sqlite3_bind_text(detailStmt, 2, [t2 UTF8String], -2, SQLITE_TRANSIENT);
sqlite3_bind_int(detailStmt, 3, t3);
if(SQLITE_DONE != sqlite3_step(detailStmt))
{
NSLog(#"Helllloooooo");
NSString *pvr= [NSString stringWithUTF8String:(char *)sqlite3_column_text(detailStmt, 0)];
NSString *fame= [NSString stringWithUTF8String:(char *)sqlite3_column_text(detailStmt, 1)];;
NSString *cinemax = [NSString stringWithUTF8String:(char *)sqlite3_column_text(detailStmt, 2)];
NSString *big= [NSString stringWithUTF8String:(char *)sqlite3_column_text(detailStmt, 3)];
pvr1 = pvr;
fame1 = fame;
cinemax1 = cinemax;
big1 = big;
NSLog(#"PVR %# Fame %# Cinemax %# Big %#",pvr1,fame1,cinemax1,big1);
}
}
sqlite3_finalize(detailStmt);
}
}
sqlite3_close(database);}
Can anyone help me with this.
You are using the sqlite3_prepare_v2() and sqlite3_finalize() wrong. You only prepare a statement once and then you can bind values to it as much as you'd like. When you're done you call sqlite3_reset(). When you're completely done with this statement and won't ever use it again (i.e. you quit the app), then you use finalize.
Your app crashes because you finalize the statement, but the pointer detailStmt still points to the position where your statement once was, and tries to access that region of the memory.
See also here: http://www.sqlite.org/c3ref/prepare.html
As Toby points out, the variables pvr, fame, cinemax, and big (and the reassigned pvr1, fame1, cinemax1, and big1) are autoreleased when returned from -stringWithUTF8String:, but you never retain them. Any access to these variables after this point will cause a crash.
You say that you are seeing a thrown exception. It might be helpful to know what that exception is, by examining the console output. Also, you can enable a breakpoint on objc_exception_throw in libobjc.A.dylib, then debug with breakpoints on, to find the exact line at which this exception occurs.
In my limited Iphone experience when something runs once and then crashes on the next iteration, generally you are releasing memory you shouldnt be or not releasing memory you should be. Try looking at your memory allocations for problems.
im having problems with leak memory. The instruments show line "pagamento_" is a malloc. i dont have idea how resolve this.
i have following code: * dados is NSMutableArray, and im alloc in other place... and do release in dealloc.
NSString *path = [self caminho];
if (sqlite3_open([path UTF8String], &Banco) == SQLITE_OK){
if (sqlite3_prepare_v2(Banco, [sql UTF8String], -1, &stmt, NULL) == SQLITE_OK) {
int row = sqlite3_step(stmt);
while(row == SQLITE_ROW) {
...
if([tabela isEqual:#"Pagamento"]){
pagamento_ = [[Pagamento alloc]init];
pagamento_.codigo = sqlite3_column_int(stmt, 0);
pagamento_.codNomePgto = sqlite3_column_int(stmt, 1);
pagamento_.codCategoria = sqlite3_column_int(stmt, 2);
pagamento_.vencimento = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 3)];
pagamento_.repeticaoPagamento = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 4)];
pagamento_.dataTermino = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 5)];
pagamento_.vctoFDS = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 6)];
pagamento_.valorPrevisto = [NSString stringWithFormat:#"%4.2f",sqlite3_column_double(stmt, 7)];
pagamento_.valorPago = [NSString stringWithFormat:#"%4.2f",sqlite3_column_double(stmt, 8)];
pagamento_.dataPgto = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 9)];
pagamento_.anotacoes =[NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 10)];
pagamento_.debitoAutomatico = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 11)];
pagamento_.nome = [NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 12)];
pagamento_.numSerie = sqlite3_column_int(stmt, 13);
pagamento_.codFavorecido = sqlite3_column_int(stmt, 14);
pagamento_.favorecido =[NSString stringWithUTF8String:(char *)sqlite3_column_text(stmt, 15)];
pagamento_.valor = [NSString stringWithFormat:#"%4.2f",sqlite3_column_double(stmt, 16)];
[dados addObject:pagamento_];
[pagamento_ release];
}
row = sqlite3_step(stmt);
}
sqlite3_finalize(stmt);
sqlite3_close(Banco);
return [dados copy];
anybody know how i resolve this??
thanks
return [dados copy];
This is causing a leak since you aren't releasing the original array. Since dados is an NSMutbaleArray that is, in theory, allocated in your -init and released in your -dealloc, copying dados on return from the code above is a good idea, but you should use:
return [[dados copy] autorelease];
If your application is crashing when you just return dados (or when you do the above), it is because you aren't managing memory correctly. As suggested by Ram, use the static analyzer and fix any problems it identifies first (build and analyze in Xcode on Snow Leopard).
If your app still crashes, then turn on NSZombies (google can show you how) and see if that catches it.
Unless you need to target iPhone OS 2.x or have a really really esoteric need to do so, you should use Core Data instead of SQLite. It'll undoubtedly be faster and save you significant development time.
You're properly releasing pagamento_ after adding it to the dados array, but you're returning a copy of dados from this method. That is most likely a mistake. Unless the caller of this method knows to release that array, it will be leaked, causing the pagamento_ object to be leaked as well. You probably should do this:
return [[dados copy] autorelease];
You almost certainly want to return [[dados copy] autorelease]. Just returning the result of copy will normally be a memory leak under the Cocoa memory management rules.
Try using the LLVM Clang Static analyser, for details see this link