I have Postgres 9.2 on my Openshift Online cartridge. Using Pgadmin3, I have enabled (by ticking the box) the autovuum setting for postgresql.conf. However, the autovacuum does not seem to be running.
Here is what I have:
ps -ef | grep -i vacuum
No autovacuum process is shown.
Using psql console, show autovacuum, says that its value is ON
Using psql console, SELECT schemaname, relname, last_vacuum, last_autovacuum from FROM pg_stat_user_tables; gives no value in last_vacuum and last_autovacuum column even though I did a manual Vacuum via Maintenance function using pgadmin3.
The properties tab on the db in pgAdminIII says AUTOVACUUM value of 'not running'
What do I miss?
EDIT
I also cannot access the postgresql.conf on Openshift Online when trying to find the file on the server - hoping to manually edit the file instead of using pgAdminIII.
-- Found this https://www.openshift.com/forums/openshift/how-do-i-set-maxpreparedtransactions-on-my-postgresql-cartridge I am now able to view/edit my postgresql.conf. Apparently the autovacuum is on already so the conf has the right setting.
When issue pg_ctl restart -m fast I got
LOG: could not bind socket for statistics collector: Permission denied
LOG: trying another address for the statistics collector
LOG: could not bind socket for statistics collector: Permission denied
LOG: trying another address for the statistics collector
LOG: could not bind socket for statistics collector: Cannot assign requested address LOG: trying another address for the statistics collector
LOG: could not bind socket for statistics collector: Cannot assign requested address LOG: disabling statistics collector for lack of working socket
WARNING: autovacuum not started because of misconfiguration
HINT: Enable the "track_counts" option.
LOG: database system was shut down at 2014-04-22 09:58:19 GMT
LOG: database system is ready to accept connections
Though track_counts is already set to on in postgresql.conf
Sorry for being so stupid but any help or pointers are much appreciated.
Thank you in advance.
i ran into a similar issue and found a helpful hint in this discussion:
... for some insane reason, openshit disabled localhost, and autovacuum only connects to localhost, I suppose it makes sense that they wouldn't want to be trying to vacuum a remote db... but openshit breaks autovacuum.
one solution i've found (and that i'll probably use) is to manually add a cronjob that does a forced vacuum. here is a batch-script that looks promising but be careful with the side-effects that a forced vacuum might involve (depending on you app of course).
Patching postgres to use the OPENSHIFT_PG_HOST environment variable instead of localhost seems to solve the problem: pgstat.patch.
Related
In postgresql version 9.2.1, the database didn't accept any commands to avoid wraparouond dataloss.The following error occured in the pg_log,
ERROR: database is not accepting commands to avoid wraparound data loss in database "XXX"
HINT: Stop the postmaster and use a standalone backend to vacuum that database.
You might also need to commit or roll back old prepared transactions.
I executed vacuum full for the database XXX in standalone backend mode.After that i tried to restart the pgsql, now pgsql server is rejecting connnections. while executing the pg_isready command, the host is rejecting connections.
Is there anything i have to do after completing the vacuum process? what are the possible reasons for the postgres server is failed to start ? Thanks in advance.
In single user mode, run
SELECT datname, datfrozenxid FROM pg_database;
to see which databases need to be vacuumed (those with the smallest values).
Run VACUUM (FREEZE) in these databases, not VACUUM (FULL).
We have postgresql replication on different server. So today I was doing some optimization on replication cluster postgresql.conf
After doing replication, I restarted postgresql with this command:
pg_ctlcluster 9.2 main2 restart
But instead of restarting, it gave this error:
The PostgreSQL server failed to start. Please check the log output.
And checking the log, I see this:
2015-06-16 12:18:16 EEST [10655]: [2-1] LOG: received smart shutdown request
2015-06-16 12:18:16 EEST [10661]: [2-1] FATAL: terminating walreceiver process due to administrator command
2015-06-16 12:18:16 EEST [10658]: [1-1] LOG: shutting down
2015-06-16 12:18:16 EEST [10658]: [2-1] LOG: database system is shut down
Checking log now it shows the last restart log, but log does not update, when I try to start server. It says check the log, but there is no new information, even if I try to start server again.
P.S. Do I need to do anything on master?
Update
Changing postgresql.conf settings back, started replication. But from error it is hard to tell what was wrong.
here are settings I changed (after they changing, they were the same as on master. When I commented it, only then I could start replication):
shared_buffers = 1536MB
effective_cache_size = 3072MB
checkpoint_segments = 15
checkpoint_completion_target = 0.9
autovacuum = on
track_counts = on
work_mem = 25MB
So as I said, after commenting these, I could start it. But don't get it why it won't let start with these settings.
If I were you, and if upgrade is an option, the first thing I would do is to upgrade to PostgreSQL 9.4 (or newer). There's a good reason for do this when it comes to replication - a new feature called "replication slots" (see the announcement).
In short: replication slots are more robust and easier to implement than WAL archiving (you obviously use according to your logs).
In this post you'll find a comprehensive guide on implementing the feature.
I have an app running on Heroku. This app has an Postgres 9.2.4 (Dev) addon installed. To access my online database I use Navicat Postgres. Sometimes Navicat doesn't cleanly close connections it sets up with the Postgres database. The result is that after a while there are 20+ open connections to the Postgres database. My Postgres installs only allows 20 simultanious connections. So with the 20+ open connections my Postgress database is now unreachable (too many connections).
I know this is a problem of Navicat and I'm trying to solve this on that end. But if it happens (that there are too many connections), how can I solve this (e.g. close all connections).
I've tried all of the following things, without result.
Closed Navicat & restarted my computer (OS X 10.9)
Restarted my Heroku application (heroku restart)
Tried to restart the online database, but I found out there is no option to do this
Manually closed all connections from OS X to the IP of the Postgres server
Restarted our router
I think it's obvious there are some 'dead' connections at the Postgres side. But how do I close them?
Maybe have a look at what heroku pg:kill can do for you? https://devcenter.heroku.com/articles/heroku-postgresql#pg-ps-pg-kill-pg-killall
heroku pg:killall will kill all open connections, but that may be a blunt instrument for your needs.
Interestingly, you can actually kill specific connections using heroku's dataclips.
To get a detailed list of connections, you can query via dataclips:
SELECT * FROM pg_stat_activity;
In some cases, you may want to kill all connections associated with an IP address (your laptop or in my case, a server that was now destroyed).
You can see how many connections belong to each client IP using:
SELECT client_addr, count(*)
FROM pg_stat_activity
WHERE client_addr is not null
AND client_addr <> (select client_addr from pg_stat_activity where pid=pg_backend_Tid())
GROUP BY client_addr;
which will list the number of connections per IP excluding the IP that dataclips itself uses.
To actually kill the connections, you pass their "pid" to pg_terminate_backend(). In the simple case:
SELECT pg_terminate_backend(1234)
where 1234 is the offending PID you found in pg_stat_activity.
In my case, I wanted to kill all connections associated with a (now dead) server, so I used:
SELECT pg_terminate_backend(pid), host(client_addr)
FROM pg_stat_activity
WHERE host(client_addr) = 'IP HERE'
1). First login into Heroku with your correct id (in case you have multiple accounts) using heroku login.
2). Then, run heroku apps to get a list of your apps and copy the name of the one which is having the PostgreSQL db installed.
3). Finally, run heroku pg:killall --app appname to get all the connections terminated.
From the Heroku documentation (emphasis is mine):
FATAL: too many connections for role
FATAL: too many connections for role "[role name]"
This occurs on Starter Tier (dev and basic) plans, which have a max connection limit of 20 per user. To resolve this error, close some connections to your database by stopping background workers, reducing the number of dynos, or restarting your application in case it has created connection leaks over time. A discussion on handling connections in a Rails application can be found here.
Because Heroku does not provide superuser access your options are rather limited to the above.
Restart server
heroku restart --app <app_name>
It will close all connection and restart.
As the superuser (eg. "postgres"), you can kill every session but your current one with a query like this:
select pg_cancel_backend(pid)
from pg_stat_activity
where pid <> pg_backend_pid();
If they do not go away, you might have to use a stronger "kill", but certainly test with pg_cancel_backend() first.
select pg_terminate_backend(pid)
from pg_stat_activity
where pid <> pg_backend_pid();
I am working on this Heroku tutorial and I have a question about autovacuum process. There is still no tables or data and autovacuum launcher is hung for over half an hour now. Below is the cmd window.
I found a similar question here but could not find the answer. Does anyone know what is going on?
As a summary: After the last line LOG: autovacuum launcher started the cursor is blinking under the last line but nothing is happenning for over half an hour.
C:\Users\a>initdb pg
The files belonging to this database system will be owned by user "a".
This user must also own the server process.
The database cluster will be initialized with locale "English_United States.1252
".
The default database encoding has accordingly been set to "WIN1252".
The default text search configuration will be set to "english".
Data page checksums are disabled.
initdb: directory "pg" exists but is not empty
If you want to create a new database system, either remove or empty
the directory "pg" or run initdb
with an argument other than "pg".
C:\Users\a>postgres -D pg &
LOG: database system was interrupted; last known up at 2013-10-05 13:46:39 EDT
LOG: database system was not properly shut down; automatic recovery in progress
LOG: record with zero length at 0/17704F8
LOG: redo is not required
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
EDIT
As suggested in bma's comments, I removed the pg directory and the project directory and I did the same steps again and I got the same result: LOG: autovacuum launcher started and it hangs. What am I going wrong?
EDIT
I opened a new command window and started the repl there and I was able to execute
C:\Users\a\CLOJURE\shouter>lein repl
user=> (require '[clojure.java.jdbc :as sql])
nil
But the next command gave the following error:
user=> (sql/with-connection (System/getenv "DATABASE_URL")
#_=> (sql/create-table :testing [:data :text]))
user=> IllegalArgumentException db-spec null is missing a required parameter cl
ojure.java.jdbc/get-connection (jdbc.clj:192)
How can I fix this?
EDIT
It turned out that 'export' is for unix and in Windows I needed to use 'set'. See this related question.
What is happening is that you are starting postgres directly, with the pg directory, and seeing log output. It would be better to install it as a service on Windows, and use the default data directory. This is what the normal installer does.
However your current approach is going to just run the db until you close the terminal window at which point it will die (hence the recovery at the beginning). You don't have hung processes. That's just log output.
I got this error from an Play 2.0.3 java application. How could I restart Heroku Postgres Dev DB? I could not find any instructions to restart the DB on Heroku help center.
app[web.1]: Caused by: org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
The error mesage you have there isn't a reason to restart the database; it isn't a database problem. Your application is holding too many connections, probably because you forgot to set up its connection pool. That isn't a DB server problem and you can fix it without restarting the DB server.
If you stop your Play application or reconfigure its connection pool the problem will go away.
Another option is to put your Heroku instance in maintenance mode then take it out again.
Since heroku doesn't allow you to connect as a superuser (for good reasons) you can't use that reserved superuser slot to connect and manage connections like you would with normal PostgreSQL.
See also:
Heroku "psql: FATAL: remaining connection slots are reserved for non-replication superuser connections"
http://wiki.postgresql.org/wiki/Number_Of_Database_Connections
If you're a non-heroku user who found this:
With normal PostgreSQL you can disconnect your client from the server end end using a PostgreSQL connection to your server. See how it says there's a slot reserved for "superuser connections" ? Connect to Pg as a superuser (postgres user by default) using PgAdmin-III or psql.
Once you're connected you can see other clients with:
SELECT * FROM pg_stat_activity;
If you want to terminate every connection except your own you can run:
SELECT procpid, pg_terminate_backend(procpid)
FROM pg_stat_activity WHERE procpid <> pg_backend_pid();
Add AND datname = current_database and/or AND usename = <target-user-name> as appropriate.
I think I should have just added this in reply to the previous answer, but I couldn't figure out how to do that, so...
As an update to Liron Yahdav's comment in the accepted answer's thread: the "non-heroku users who found this" solution worked for me on a Heroku PostgreSQL dev database, but with a slight modification to the query Liron provided. Here is my modified query: SELECT pid, pg_terminate_backend(pid) FROM pg_stat_activity WHERE pid <> pg_backend_pid() AND usename='<your_username>';
It seems that procpid has changed to pid.
There is no way to restart the whole database. Though, heroku offers a simple way to stop all connections which solves the problem in the majority of cases:
heroku pg:killall