When I ran an infinite loop accidentally, restart and run all cells command did not execute all cells. Some cells did not show the output and also the cell numbering was altered. What should I do.
I tried running all cells again but some cells did not show any output.
Cells from in between the notebook were running and altered the numbering.
Related
Is there a way to check why my database is increasing its size so fast? I had a growth of 30GB in less than a week and all my process and procedures are so slow now, with the same amount of transactions as usual.
Or is there a way to check the size of the table?
If you have a table that you suspect might be growing you can run the following command from the command line.
gstat <DATABASE NAME|ALIAS> -t <TABLE_NAME>
Additional information on gstat can be found here. https://firebirdsql.org/file/documentation/html/en/firebirddocs/gstat/firebird-gstat.html
You can have this command run on a schedule and save the results so you can see what is happening over time. Although, this command might take a while to run based on the size of the table.
When creating an index concurrently in Postgres, how do you make the statement run in the background? Once I run the query in psql, the statement does not return and I'm not able to quit the process and disconnect SSH to server.
Edit: I understand we could use something like tmux to keep the shell alive in the background. But I'm trying to understand if Postgres' CONCURRENT index operation does not return immediately.
Yes, although the index is created concurrently, the DDL itself will not return immediately.
Ref: https://gist.github.com/bryanrite/36714b13e0aece2f6c43#safe
Add an index concurrently (Example), Note: it will still take a long time to run the migration, but it won't write-lock the table.
Use screen in linux,
screen -S session_name
then execute the command in psql.
You can detach from the screen session at any time by typing:
Ctrl+a d
To resume your session use
screen -r
Read this for more info
This way you can let your process run in the background even though you disconnect your session.
I have a cronjob that is kicked off every night that involves building up a temporary table, dropping the current table on Redshift, and swapping in the temporary table for the old one. More than half of the time, this specific job gets stuck when dropping the existing table and behaving as if there is some pending transaction that is stopping the drop from going through.
This is just one of dozens of jobs that uses the exact same script to run overnight, none of which have ever had this issue; however, there are a few minor differences:
The box that this particular job is running on is a different box from all of the other production jobs, as this one is currently in a testing state.
The S3 key used on this box is different from the other box.
In addition to the fact that I have never seen this on any other job, this issue has been extremely difficult to troubleshoot for the following reasons:
I have not been able to replicate this issue by running the script manually on the same box it is currently being run on; the script executes as expected, with the table drop occurring in mere seconds. The only difference I can think of here is that I'm executing the script as ubuntu whereas the cronjob is executed from root.
I have not had any success identifying or terminating the sessions that are causing the drop to stall; I've looked high and low on Stack Overflow (this is the most applicable question with answers - redshift drop or truncate table very very slow), the Redshift docs, and otherwise, but nothing I've found has been the answer. When I see that the job is stalled, I've checked the following tables on Redshift and usually find that things are in the following state:
The temporary table has been created, but the old version of the destination table is still there.
The stv_locks table shows that that there are three processes running, with the lock_status of "Holding write lock," "Holding delete lock," and "Holding insert lock" respectively. The process ID associated with these is NOT the ID related to the current job.
The stv_tr_conflict table shows nothing.
The stv_recents table shows the drop with a status of Running.
The query that's supposedly creating the lock described above shows up in the svl_qlog as finished, so that seems to contradict the stv_locks table.
Using pg_terminate_backend to stop the associated process does not actually remove the session when querying stv_sessions, but DOES free up something that allows the job to finish.
Any help in figuring out what exactly is going on here would be greatly appreciated!
I faced the same problem, I just rebot RS then it works again normally.
I need to load large amount of data on a table in DB2 database. I am using CLI load mode on a table written in C using SQLSetStmtAttr function. Select statements does not work (the table gets locked) when it is set.
When the loading of the data completes I am doing load mode off. After that the table becomes accessible so that i can perform select from db2 command line tools (or control center).
But the problem is when my C program crashes or fails before doing load mode off. The table is always locked. I have to drop the table and all previous data is lost.
My question is whether there is a way to recover the table?
DBMS Documentation is your friend. You can read the description of SQL0668N (or any other error!) to find out what reason code 3 means, as well as how to fix it.
Basically, when a LOAD operation fails, you need to perform some cleanup on the table – either restart or terminate it. This can be done using the LOAD utility from outside of your program (e.g., LOAD from /dev/null of del TERMINATE into yourtable nonrecoverable) but you can also do it programatically.
Typically you would do this using the db2Load() API, and setting the piLongActionString member of the db2LoadStruct parameter you pass to db2Load(), with the same RESTART or TERMINATE operation.
It looks like you can set the SQL_ATTR_LOAD_INFO statement to the same db2LoadStruct when using a CLI Load, too, but I am not sure if this would actually work to complete a load restart / terminate.
I created a set of partitioned tables in Postgres, and started inserting a lot of rows via the master table. When the load process blew up on me, I realized I should have declared the id row BIGSERIAL (BIGINT with a sequence, behind the scenes), but inadvertently set it as SERIAL (INTEGER). Now that I have a couple of billion rows loaded, I am trying to ALTER the column to BIGINT. The process seems to be working, but is taking a long time. So, in reality, I don't really know if it is working or it is hung. I'd rather not restart the entire load process again.
Any suggestions?
When you update a row to alter it in PostgreSQL, that writes out a new copy of the row and then does some cleanup later to remove the original. This means that trying to fix the problem by doing updates can take longer than just loading all the data in from scratch again--it's more disk I/O than loading a new copy, and some extra processing time too. The only situation where you'd want to do an update instead of a reload is when the original load was very inefficient, for example if a slow client programs is inserting the data and it's the bottleneck on the process.
To figure out if the process is still working, see if it's using CPU when you run top (UNIX-ish systems) or the Task Manager (Windows). On Linux, "top -c" will even show you what the PostgreSQL client processes are doing. You probably just expected it to take less time than the original load, which it won't, and it's still running rather than hung up.
Restart it (clarifying edit: restart the entire load process again).
Altering a column value requires a new row version, and all indexes pointing to the old version to be updated to point to the new version.
Additionally, see how much of the advise on populating databases you can follow.
Correction from #archnid:
altering the type of the column will trigger a table rewrite, so the row versioning isn't a big problem, but it will still take lots of disk space temporarily. you can usually monitor progress by looking at which files in the database directory are being appended to...