can someone ELI5 what CONN_MAX_AGE does? I thought it worked like this:
1) Request #1 comes in, opens connection 1 to database
2) Request #1 uses connection 1 to do some work
3) Request #1 completes. Because CONN_MAX_AGE is non-zero (and the age has not been reached), the connection is left open.
4) Request #2 comes in, and Django re-uses connection #1 to the database.
But that doesn't seem to be happening. I have a page on my site that does an AJAX poll every 15 seconds. In my development environment, I see the number of open connections (select count(*) from pg_stat_activity), slowly grow, until eventually I get
OperationalError: FATAL: sorry, too many clients already
So I'm wondering where I've gone awry. Is CONN_MAX_AGE only used to keep connections open within a single HTTP request?
UPDATE:
Looking more carefully at the docs, I see this:
The development server creates a new thread for each request it
handles, negating the effect of persistent connections. Don’t enable
them during development.
Ah, so that seems to imply that a connection "belong to" a thread. (And the thread may open/close the connection, based on the value of CONN_MAX_AGE).
Related
I have a huge database where, in some places, we use Postgres notifications. We noticed that the queue size is increasing. The way we check is executing this simple command: select pg_notification_queue_usage();.
When it reaches 100% then all messages are gone. The problem I have is that I don't know who listens to notifications and what channels we have there. I identified only two services that listen for those notifications but it seems that's not the case.
My task is to find other places where we use notifications (consume or produce) to find the root cause. How can I do it?
The only thing I found about it is the query select pg_notification_queue_usage(); but it seems that Postgres doesn't provide other useful functions related to this feature.
I did some experiments regarding it. I launched a local Postgres instance and started publishing notifications there. Everything worked as expected. When I did it once again but without actual consuming notifications, the queue size started to grow. That's what I expected, tho.
Then, I restarted the process and the queue size dropped to 0. That's exactly what the docs say about it.
A session's listen registrations are automatically cleared when the session ends.
On the production, we did exactly the same - we restarted known services but the notification queue didn't drop to 0 as we expected.
It means, there's something else listening to one of the channels but it doesn't consume it or does it too slow.
Is there any way of identifying such listeners?
For some client side procedures, I implement remote logging to log the calling of the procedure. The log is printed several times with different thread id, even though the procedure is only called once. Some rpc requests are sent to the sever a few times which causes some database session problem. Is it normal? Is there anyway to avoid it?
Thanks
This is not normal, and suggests there is a bug on your client causing it to send the same call more than once. Try adding logging on the client where you invoke the RPC call, and possibly add breakpoints to confirm why it is being called twice.
My best guess with no other information would be that you have more than one event handler wired up to the same button, or something like that.
--
More specifically, your servlet container starts multiple threads to handle incoming requests - if two requests come in close succession, they might be handled by different threads.
As you noted, this can cause problems with a database, where two simultaneous calls could be made to change the same data, especially if you have some checks to ensure that a servlet call cannot accidentally overwrite some newer data. This is almost certainly a bug in your client code, and debugging it should start there.
I implemented an UWP Server Socket following the sample here and it correctly works.
Now I want to make the app able to continuously accept requests, but I expect that when the app is suspendeded and a client sends a request, the server is not able to respond. If I am correct, what is the best way to avoid this status change? If possible, I would prefer a solution with Extended Execution instead of implementing a Background Task, but I don't know if the following code in the OnSuspending method is enough to keep the app in the Running status:
var newSession = new ExtendedExecutionSession();
newSession.Reason = ExtendedExecutionReason.Unspecified;
newSession.Revoked += SessionRevoked;
I saw people calling a "LongRunningWork()" function in other samples, but in my case the code to execute is already defined in the code-behind of the view as shown in the link above, so I would like simply keeping the app always running. Keep in mind that it is a LOB application, so I don't have Store limits.
We run a postgres server v9.2.8, and use epgsql (erlang) as a client library. And in some cases, which we had on production but weren't able to reproduce in dev environment, we're loosing data.
A function in our application (it should be killed) allows an operator to change session parameters on a running connection. Since connection is usually always busy on production, a "SET SESSION bla-bla" query always crashes pgsql_connection process.
Before crashing, pgsql_connection sends a "Terminate" ('X') signal via pgsql_sock (a wrapper around tcp socket) to a backend. At the same time another erlang process (let's call it "worker") is waiting for a response from postgres backend using the same socket.
Now the question: is it possible that upon receiving a "Terminate" signal from a client, backend can cancel last transaction even if it has sent an "OK" on "COMMIT" statement already?
Because if it is possible, a worker will have a chance to report to the main application process about successfully written transaction while indeed the transaction has been cancelled.
Or, where can I read more details about this? Documentation says (http://www.postgresql.org/docs/9.2/static/protocol-flow.html):
For either normal or abnormal termination, any open transaction is
rolled back, not committed. One should note however that if a frontend
disconnects while a non-SELECT query is being processed, the backend
will probably finish the query before noticing the disconnection. If
the query is outside any transaction block (BEGIN ... COMMIT sequence)
then its results might be committed before the disconnection is
recognized.
– not a crystal clear statement.
Now the question: is it possible that upon receiving a "Terminate" signal from a client, backend can cancel last transaction even if it has sent an "OK" on "COMMIT" statement already?
No. that is fundamentally impossible. If it's committed, it's committed, and there's no going back. That's what "commit" means.
The only time Pg might return success before the commit hits disk and is persistent is if you told it to by setting synchronous_commit = off.
If you're seeing anything different happening then most likely it's a result of attempting to share a single connection between multiple processes (as you establish the connection before fork()) without proper locking or other mutual exclusion to ensure that the connection is locked while a command is in-flight.
Note that the reverse isn't true, which might be what you're thinking of with the quoted documentation passage. A transaction can get committed without returning a successful OK to the client if the client goes away (crashes, loses connection, etc) after issuing the commit command.
What the application is doing, where it sends out-of-sync messages on the wire protocol, is totally broken. It's guaranteed to cause unpredictable problems. The protocol is somewhat robust, so you're not likely to get things like an unintended commit, but you're very likely to get transactions aborted or whole sessions disconnected suddenly.
If you need to be able to roll back/abort committed transactions, then your application design has problems. You're not really ready to commit when you say COMMIT. You would have the same problem if the app process crashed or the whole server crashed between Pg committing the transaction and you doing whatever you need to do.
If you cannot fix the app design to avoid this then you will have to use two-phase transactions, either directly using PREPARE TRANSACTION then COMMIT PREPARED, or indirectly via the XA API. This has significant costs in performance and management overhead, but it's the only option if you need to do special work after database commit but before you're really "done".
The docs you quote are talking about the case where the app has sent a COMMIT but then disconnects before receiving the backend's acknowledgement of the commit. Because TCP/IP is buffered there's no guarantee the COMMIT got flushed to Pg, and if it did there's no guarantee it doesn't accompany the RST that terminates the connection. So in this specific case it's somewhat uncertain whether the transaction will commit or not. An application for which this is a problem would need to have a way of checking whether the last unit of work committed or not when it resumes work, or if it can't do that use two-phase transactions. The docs you quote say nothing about being able to cancel a commit after it's completed, because you can't. Ever.
Assuming that the app has to do some kind of extra work after commit, like moving a file or sending an email or doing work on another data store, then you're probably going to need two-phase transactions. Even then you're vulnerable to issues unless all parties in the distributed transaction support two phase commit, because your "other bit" could get done then your worker or server could crash before the confirmation of its completion is sent to the database to finish phase II of the commit.
You can keep your own two phase commit log of sorts in the DB instead of using true 2PC:
Do the main database work and write a record to the work log table that says "I've done the work in the database and I'm about to do the next part".
Do the next part; and
Update the work log to say the next part is done.
... but this has the same problem, where a crash between parts 2 and 3 causes the app to forget that it did part 2 and repeat it on startup. If you can't live with that, you need to find a way to make part 2 commit completion verifiable, so you can tell if it's done or not, or find a way to make it capable of doing 2-phase commit.
To learn more about this topic, read about XA, distributed transactions, two-phase commit, etc.
Say I have an application that wants a persistent connection to a server. How do I implement connection/re-connection logic so that I'm not wasting resources (power/bandwidth) and I have fast reconnect time when connectivity appears/improves? If I only use connectivity notifications, I can get stuck on problems not related to the local network.
Bonus if you could show me the C# version.
This is a very "huge" question. I can say that we use an O/R Mapper and each "query" to the database needs an object called PersistenceBroker. This class is in charge of all the DB Stuff related to connecting, authenticating etc.
We've written a PersistenceBrokerFactory.GetCurrentBroker() which returns the "working" broker. If the DB suddenly fails (for whatever reason), the CONN object will "timeout()" after 30secs (or whatever you define). If that happens, we show the user that he/she is offline and display a reconnect button.
On the other hand, to provide a visual indication that the user has connectivity, we have a thread running in the background, that checks for Internet connectivity every 15 seconds. We do 1 ping to google.com. ;) If that fails, we assume Internet is somehow broken, and we update a status bar.
I could show you all that code for the network health monitor if you wanted. I took some bits from google and other I made myself :)