Recently I implemented a cache component in my iPhone app, that uses SQLite.
I'm writing and reading the data natively w/o any wrapper framework.
The problem is that after using the app for a while it's beginning to get SQLITE_CORRUPT status code in response to a any SQL statement I perform in the DB.
I'm using SQLiteManager as my DB mgmt tool. When the DB becomes corrupted I'm trying to analyze and inspect via the SQLiteManager but it doesn't even allow me to browse through the data in the table and just outputs a laconic message that says data is corrupted.
Anyone can assist with this one ? First of all - How the hell the DB gets to that situation and second - how or where can I see the data anyway ?
Thanks,
Guy.
You can refer this official documentation. Here they have mentioned there are very few chances for db to be corrupted.
They have also list out the chances how DB can be corrupted. And from there refer following example which can cause the problem to the db.
CREATE TABLE ex1(a INTEGER PRIMARY KEY, b);
INSERT INTO ex1 VALUES(1,2);
INSERT INTO ex1 VALUES(2,3);
CREATE TRIGGER ex1_tr1 AFTER UPDATE ON ex1 BEGIN
DELETE FROM ex1 WHERE a=old.b;
END;
UPDATE ex1 SET b=b+1;
In the example above, the first cycle of the UPDATE causes the trigger to fire and delete the second row of the ex1 table. When the second cycle of the UPDATE loop runs, it attempts to process the second row of the ex1 table. SQLite recognized that the second row had been deleted so it aborts the second cycle, but it was failing to clean up after itself properly which could lead to database corruption on subsequent cycles of the loop.
So, Basically you need to find out the situation where this has happened in your code, By debugging or by following steps through your application or any other mechanism.
Related
I've had a strange bug pop up -- when I write to a partitioned table, then immediately after do a select on that same table, I get an error like:
./2018.04.23/ngbarx/cunadj. OS reports: Operation not permitted.
This error does not appear if, after writing the table, I wait a few seconds. This to me seems like pointing towards there being a caching situation, where q responds before an operation is complete, but afaik everything I am doing should be synchronous.
I would love to understand what I am doing wrong / what is causing this error exactly / which commands are executing asynchronously.
The horrible details:
Am writing from Python connected to q synchronously using the qpython3 package
The q session is launched with slaves i.e. -s 4
To write the partitioned table, I am using the unofficial function .Q.dcfgnt which can be found here
I write to a q session that was initialized with a database directory as is usual when dealing with partitioned tables
After writing the table with .Q.dcfgnt, but before doing the select, I also do .Q.chk`:/db/; system"l /db/"; .Q.cn table in that order, just to be sure the table is up and ready to use in the q session. These might be both overkill and in the wrong order, but I believe they are all synchronous calls afaik, please correct me if I am wrong.
The trigger to the error is a 10#select from table; I understand
why this is a bad idea to do in general on a partitioned table, but
also from my understanding it shouldn't be triggering the particular error that I am getting.
I have a .csv file, comma-delimited (located at C:/). I am using the DB2 LOAD utility to load data present in the CSV file in a DB2 table.
LOAD CLIENT FROM C:\Users\somepath\FileName.csv of del
MODIFIED BY NOCHARDEL COLDEL, insert into SchemaName.TABLE_NAME;
CSV file has 25 rows. After the utility completed I got an error message for NOCHARDEL. My table has all 25 rows properly loaded. Now when I try to execute an insert/update/delete statement on any of the tables present in that schema I am getting following error.
Lookup Error - DB2 Database Error: ERROR [55039] [IBM][DB2/AIX64] SQL0290N Table space access is not allowed.
Could you please help me whether I am making any mistake or missing a parameter that is causing lock on the table.
Earlier while loading the file similar situation occurred, where DBA confirmed that Table space in question is in “load in progress” state
Changes generated by the DB2 LOAD utility are not logged (one of the side-effects of its high performance). If the database crashes immediately after the load it will be impossible to recover the table that was loaded by replaying log records, because there are no such records. For this reason the tablespace containing the loaded table is automatically placed in the BACKUP PENDING mode, forcing you to take a backup of that tablespace or the entire database to ensure it is fully recoverable.
There are options that you can specify for the LOAD command that can help you avoid this situation in the future:
NONRECOVERABLE -- this option does not place the tablespace into the BACKUP PENDING mode, but, as its name implies, the table you're loading to becomes non-recoverable in case of a crash, and your only option in that situation will be to drop and re-create the table.
COPY YES -- this option creates a copy of the table prior to loading, which can be used to recover the table to its pre-LOAD state in case of a crash.
If you are only loading 25 records, I suggest you use the IMPORT utility instead -- it does not have these restrictions because it is fully logged (at the price of lower performance, which for 25 records won't matter).
Thanks #mustaccio. I had 60 Million rows to insert. I was using 25 as sample to check the outcome.
To add another point, we later came to know that this is a known DB2 bug that keeps the load in progress state (DB2 is unable to acknowledge that the load has completed and the session remains open indefinitely) and place the table space in backup pending state.
Recovery is the only option to release the table space once it is in pending state.
This issue is fixed in fix pack 10 as per the DB2 team (we are yet to deploy and test). Mean while NONRECOVERABLE key word is working fine for us
The reason why your table is stuck in the LOAD IN PROGRESS state is the NOCHARDEL error happening at the end of the LOAD.
Have you tried restarting the database? This should reinitialize all table spaces and remove any rogue states.
http://www-01.ibm.com/support/docview.wss?uid=swg1IC65395
http://www-01.ibm.com/support/docview.wss?uid=swg21427102
I'm investigating the scalibility of Sequalize in a production app, specifically the increment function, to see how well it can handle when a row could theortically be updated several times simultaneously (say, the totals row of something). My question is can the sequalize increment operator be trusted for these little addition operations that could be concurrent?
We're using Postgres on the backend, but I'm not familar with the internals of Postgres and how it would handle this type of scenerio (heroku postgres will be the production host, if it matters).
The Docs / The Code
The sql ran by Sequealize according to the code comments
SET column = column + X
It's hard to say without complete SQL examples, but I'd say this will likely serialize all transactions that call it on the same object.
If you update an object, the db takes a row update lock that's only released at commit/rollback time. Other updates/deletes block on this lock until the first tx commits or rolls back.
I need to load large amount of data on a table in DB2 database. I am using CLI load mode on a table written in C using SQLSetStmtAttr function. Select statements does not work (the table gets locked) when it is set.
When the loading of the data completes I am doing load mode off. After that the table becomes accessible so that i can perform select from db2 command line tools (or control center).
But the problem is when my C program crashes or fails before doing load mode off. The table is always locked. I have to drop the table and all previous data is lost.
My question is whether there is a way to recover the table?
DBMS Documentation is your friend. You can read the description of SQL0668N (or any other error!) to find out what reason code 3 means, as well as how to fix it.
Basically, when a LOAD operation fails, you need to perform some cleanup on the table – either restart or terminate it. This can be done using the LOAD utility from outside of your program (e.g., LOAD from /dev/null of del TERMINATE into yourtable nonrecoverable) but you can also do it programatically.
Typically you would do this using the db2Load() API, and setting the piLongActionString member of the db2LoadStruct parameter you pass to db2Load(), with the same RESTART or TERMINATE operation.
It looks like you can set the SQL_ATTR_LOAD_INFO statement to the same db2LoadStruct when using a CLI Load, too, but I am not sure if this would actually work to complete a load restart / terminate.
I created a set of partitioned tables in Postgres, and started inserting a lot of rows via the master table. When the load process blew up on me, I realized I should have declared the id row BIGSERIAL (BIGINT with a sequence, behind the scenes), but inadvertently set it as SERIAL (INTEGER). Now that I have a couple of billion rows loaded, I am trying to ALTER the column to BIGINT. The process seems to be working, but is taking a long time. So, in reality, I don't really know if it is working or it is hung. I'd rather not restart the entire load process again.
Any suggestions?
When you update a row to alter it in PostgreSQL, that writes out a new copy of the row and then does some cleanup later to remove the original. This means that trying to fix the problem by doing updates can take longer than just loading all the data in from scratch again--it's more disk I/O than loading a new copy, and some extra processing time too. The only situation where you'd want to do an update instead of a reload is when the original load was very inefficient, for example if a slow client programs is inserting the data and it's the bottleneck on the process.
To figure out if the process is still working, see if it's using CPU when you run top (UNIX-ish systems) or the Task Manager (Windows). On Linux, "top -c" will even show you what the PostgreSQL client processes are doing. You probably just expected it to take less time than the original load, which it won't, and it's still running rather than hung up.
Restart it (clarifying edit: restart the entire load process again).
Altering a column value requires a new row version, and all indexes pointing to the old version to be updated to point to the new version.
Additionally, see how much of the advise on populating databases you can follow.
Correction from #archnid:
altering the type of the column will trigger a table rewrite, so the row versioning isn't a big problem, but it will still take lots of disk space temporarily. you can usually monitor progress by looking at which files in the database directory are being appended to...