What's the difference between functions: pg_cancel_backend(pid int) and pg_terminate_backend(pid int)? For me they work pretty the same.
pg_cancel_backend() cancels the running query while pg_terminate_backend() terminates the entire process and thus the database connection.
When a program creates a database connection and sends queries, you can cancel one query without destroying the connection and stopping the other queries. If you destroy the entire connection, everything will be stopped.
Related
I am new to reactive world and trying to understand how the db-connections works under the hood with vertx-sql-clients.
I am using io.vertx:vertx-mysql-client, io.vertx:vertx-oracle-client and io.vertx:vertx-pg-client in my project.
As this is reactive database client, I can understand that one thread can handle multiple db-connections objects. I have set pool.maxSize to 1 and executed concurrently 5000 times a function, which gets the connection from the pool, executes a select query in the database and fetches some rows from the DB.
Now my question is, even though I have configured a single db-connection in the pool, still it can handle my 5000 requests concurrently, so how this works under the hood? is all the selects run over single database connection? if so, how it can handle a transaction management with single db-connection?
I don't understand some of Postgres mechanism and it makes me quite upset.
I usually use DBeaver as SQL client to query external pg base. If run create.. or insert.. queries and then connection for some reason is broken or invalidated, the pid is still running and finishes transaction.
But for some more complicated PL/pgSQL functions (with temp tables, loops, inserts, etc.) we wrote, breaking connection always causes process termination (it disappears from session list just before making next sql operation, eg. inserting a row in logtable). No matter if it's DBeaver editor or psql command.
I know that maybe disconnecting is critical problem, which should be eliminated and maybe I shouldn't expect process to successfully continue, but I do:) Or just to know why it happened and is it possible to prevent it?
If the network connection fails, the database server can detect that in two ways:
if it tries to send data to the client, it will figure out pretty quickly that the connection is down
if it tries to receive data from the client, it will only notice when the kernel's TCP keepalive mechanism has determined that the connection is down
When you say that sometimes execution of a function is terminated right away, I would say that is because the function returned data to the client.
In the case where a query keeps running, it is not attempting to return any data yet.
There is no cure for the former, but in PostgreSQL v14 you can prevent the latter by setting client_connection_check_interval. In addition, you have to set the PostgreSQL keepalive parameters so that the dead connection becomes known quickly.
See my article for more.
When the Go driver detects that a context is cancelled, I think it will idle the connection and return it to the pool. Does that imply the running DB operation that it started will also be killed on the server? Or does that continue on?
You can read the official documentation about how the driver uses the context here.
Relevant section: Socket Read and Write:
When the driver retrieves a connection for an operation, it sets the socket’s read or write deadline to either the Context deadline or socket timeout, whichever is shorter.
If you cancel the Context after the execution of the Read() or Write() function but before its deadline, the behavior of the driver differs based on version.
The driver generates a separate goroutine to listen for Context cancellation when the Read() or Write() function is in progress. If the goroutine detects a cancellation, it closes the connection. The pending Read() or Write() function returns an error which the driver overwrites with the context.Canceled error.
Note that closing the connection just means putting it back to the pool as idle. There is no explict statement about cancelling the operation on the server, but obviously a signal is sent that the client abandons the operation and no longer monitors / cares for the result. The MongoDB server would be dumb not to abort the initiated operation if it can be. For example if the operation was a query, that can be aborted. If it was a write operation (such as insert or update), that may not be aborted.
Our PostgreSQL 10.1 server ran out of connections today because a monitor process that was calling
select pg_database_size('databasename');
was getting stuck. It was NOT getting an obvious Lock. It just never returned. The monitor dutifully logged in every few minutes, over and over until we ran out of connections. When I run the query for other databases it works, but not for our main database.
Killing the calling process did not clear the query.
select pg_cancel_backend(1234)
doesn't kill the query. Nor does
select pg_terminate_backend(1234)
Ditto if I run the query by hand, nothing kills it in the database.
I will probably have to restart the database server to recover from this. However I'd like to prevent it from happening again.
What is this function doing that would resist signals and never return (like 8 hours after being invoked)? Is there any way to clear them from the process table without restarting the database and breaking the users who still have the few remaining connections still active in the system?
A postgres client is running a possibly long-running query. The client receives a SIGTERM. Ideally, I'd like the client to trap the SIGTERM, clean up its query by sending a pg_cancel_backend(pid) and then quit.
This would seem to be a common concern with a client-server database architecture, but I can't find any solutions. Instead, the query will continue until completion, consuming unnecessary resources.
How can the client determine with certainty the PID of its backend query? (I don't mean interactively view the backend queries and choosing the one that appears to match the query that was issued.) For example, are PIDs associated with persistent database connections?
How does this work when using psql when ctrl-C is pressed?
You can use pg_backend_pid() to get PID for current connection.
Details here.