PG::ConnectionBad Postgres Cluster down - postgresql

Digitalocean disabled my droplet's internet access. After fixing the error (rollback to older backup) they restored the internet access. But afterwards I constantly get an error when deploying, I can't seem to get my Postgres database up and running.
I'm getting an error each time I try to deploy my application.
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
So I used SSH to login to my server and check if my Postgres was actually running with:
pg_lsclusters
Results into:
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 down postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
Postgres server status
So my Postgres server seems to be down. I tried putting it 'up' again with:
pg_ctlcluster 9.5 main start After doing so I got the error: Insecure directory in $ENV{PATH} while running with -T switch at /usr/bin/pg_ctlcluster line 403.
And /usr/bin/pg_ctlcluster on line 403 says:
system 'systemctl', 'is-active', '-q', "postgresql\#$version-$cluster";
But I'm not to sure what the problem could be here and how I could fix this.
Update
I also tried updating the permissions on /bin to 755 as mentioned here. Sadly that did not fix my problem.
Update 2
I changed the /usr/bin to 755. Now when I try pg_ctlcluster 9.5 main start, I get this:
Job for postgresql#9.5-main.service failed because the control process exited with error code. See "systemctl status postgresql#9.5-main.service" and "journalctl -xe" for details.
And inside the systemctl status postgresql#9.5-main.service:
postgresql#9.5-main.service - PostgreSQL Cluster 9.5-main
Loaded: loaded (/lib/systemd/system/postgresql#.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2018-01-28 17:32:38 EST; 45s ago
Process: 22473 ExecStart=postgresql#%i --skip-systemctl-redirect %i start (code=exited, status=1/FAILURE)
Jan 28 17:32:08 *url* systemd[1]: Starting PostgreSQL Cluster 9.5-main...
Jan 28 17:32:38 *url* postgresql#9.5-main[22473]: The PostgreSQL server failed to start.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Control process exited, code=exited status=1
Jan 28 17:32:38 *url* systemd[1]: Failed to start PostgreSQL Cluster 9.5-main.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Unit entered failed state.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Failed with result 'exit-code'.
Thanks!

You better not mix systemctl and pg_ctlcluster. Let systemctl makes the calls to pg_ctlcluster with the right user and permissions. You should start your postgresql instance with
sudo systemctl start postgresql#9.5-main.service
Also, check the errors in the startup log. You can post them too, to help you figure out what's going on.
Your systemctl status also outputs that the service is disable, so, when the server reboots, you will have to start the service manually. To enable it run:
sudo systemctl enable postgresql#9.5-main.service
I hope it helps

It is mainly because /etc/hosts file is somehow changed.I have removed extra space inside /etc/hosts file.Use cat /etc/hosts
Add these lines into the file
127.0.0.1 localhost
127.0.1.1 your-host-name
::1 ip6-localhost ip6-loopback
And I have given permission 644 to /etc/hosts file.It is working for me even after the reboot of the system.

Related

How can I upgrade PostgreSQL from version 11 to version 13?

I'm trying to upgrade PostgreSQL from 11 to 13 on a Debian system, but it fails. I have a single cluster that needs to be upgraded:
$ sudo -u postgres pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
11 main 5432 online postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
Here's what I've tried to upgrade it:
$ sudo -u postgres pg_upgradecluster 11 main
Stopping old cluster...
Warning: stopping the cluster using pg_ctlcluster will mark the systemd unit as failed. Consider using systemctl:
sudo systemctl stop postgresql#11-main
Restarting old cluster with restricted connections...
Notice: extra pg_ctl/postgres options given, bypassing systemctl for start operation
Error: cluster configuration already exists
Error: Could not create target cluster
After this, the system is left in an unusable state:
$ sudo systemctl status postgresql#11-main.service
● postgresql#11-main.service - PostgreSQL Cluster 11-main
Loaded: loaded (/lib/systemd/system/postgresql#.service; enabled-runtime; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2022-06-14 06:48:20 CEST; 19s ago
Process: 597 ExecStart=/usr/bin/pg_ctlcluster --skip-systemctl-redirect 11-main start (code=exited, status=0/SUCCE>
Process: 4508 ExecStop=/usr/bin/pg_ctlcluster --skip-systemctl-redirect -m fast 11-main stop (code=exited, status=>
Main PID: 684 (code=exited, status=0/SUCCESS)
CPU: 1.862s
Jun 14 06:47:23 argos systemd[1]: Starting PostgreSQL Cluster 11-main...
Jun 14 06:47:27 argos systemd[1]: Started PostgreSQL Cluster 11-main.
Jun 14 06:48:20 argos postgresql#11-main[4508]: Cluster is not running.
Jun 14 06:48:20 argos systemd[1]: postgresql#11-main.service: Control process exited, code=exited, status=2/INVALIDARG>
Jun 14 06:48:20 argos systemd[1]: postgresql#11-main.service: Failed with result 'exit-code'.
Jun 14 06:48:20 argos systemd[1]: postgresql#11-main.service: Consumed 1.862s CPU time.
$ sudo systemctl start postgresql#11-main.service
Job for postgresql#11-main.service failed because the service did not take the steps required by its unit configuration.
See "systemctl status postgresql#11-main.service" and "journalctl -xe" for details.
Luckily, rebooting the system brought the old cluster back online, but nothing has been upgraded. Why does the upgrade fail? What are "the steps required by its unit configuration"? How can I upgrade PostgreSQL with minimal downtime?
I found the source of my problem: a configuration file owned by the wrong user (root instead of postgres) that could not be removed by the pg_dropcluster command because I ran it as the user postgres.
For future reference, here are the correct steps to upgrade a PostgreSQL cluster from 11 to 13:
Verify the current cluster is the still the old version:
$ pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
11 main 5432 online postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
13 main 5434 down postgres /var/lib/postgresql/13/main /var/log/postgresql/postgresql-13-main.log
Run pg_dropcluster 13 main as user postgres:
$ sudo -u postgres pg_dropcluster 13 main
Warning: systemd was not informed about the removed cluster yet.
Operations like "service postgresql start" might fail. To fix, run:
sudo systemctl daemon-reload
Run the pg_upgradecluster command as user postgres:
$ sudo -u postgres pg_upgradecluster 11 main
Verify that everything works, and that the only online cluster is now 13:
$ pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
11 main 5434 down postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
13 main 5432 online postgres /var/lib/postgresql/13/main /var/log/postgresql/postgresql-13-main.log
Drop the old cluster:
$ sudo -u postgres pg_dropcluster 11 main
Uninstall the previous version of PostgreSQL:
$ sudo apt remove 'postgresql*11'
The Debian packages create a cluster automatically when you install the server package, so get rid of that:
pg_dropcluster 13 main
Then stop the v11 server and try again.

Unable to start PostgreSQL 12 Server

I want to setup PostgreSQL 12 with PostGIS 3 on Ubuntu 20.04 for the purpose of creating an OSM Tile Server. I want to have 2 different clusters, one for a regular PSQL database and another for OSM data. I can't seem to get the one for the OSM data up and running:
When I run pg_lsclusters, I get the following:
Ver Cluster Port Status Owner Data directory Log file
12 main 5433 online postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log
12 osm_psql_db 5432 down postgres /var/lib/postgresql/12/2TB1/osm_psql_db /var/log/postgresql/postgresql-12-osm_psql_db.log
When I run journalctl -xe, I get the following:
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: Dependency failed for PostgreSQL Cluster 12-osm_psql_db.
-- Subject: A start job for unit postgresql#12-osm_psql_db.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit postgresql#12-osm_psql_db.service has finished with a failure.
--
-- The job identifier is 9566 and the job result is dependency.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: postgresql#12-osm_psql_db.service: Job postgresql#12-osm_psql_db.service/start failed with result 'dependency'.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: var-lib-postgresql-12-osm_psql_db.mount: Job var-lib-postgresql-12-osm_psql_db.mount/start failed with result 'dependency'.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: dev-disk-by\x2dlabel-osm_psql_db.device: Job dev-disk-by\x2dlabel-osm_psql_db.device/start failed with result 'timeout'.
Mar 13 11:47:43 cdil-MS-7B92 PackageKit[27900]: daemon quit
Mar 13 11:47:43 cdil-MS-7B92 systemd[1]: packagekit.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit packagekit.service has successfully entered the 'dead' state.
Any idea what could be holding me up?
*** EXTRA INFO JUST IN CASE ***
In terms of how I set up everything, I installed the following packages:
sudo apt install postgresql-12 postgresql-contrib postgis postgresql-12-postgis-3
Because the OSM data is quite large, I want to store that particular cluster on another hard disk. It's called "2TB1" and it's been mounted to /var/lib/postgresql/12/2TB1 because I realized that the postgres user needed access to the data_directory folder and all parent folders leading up to it.
To do so I modified the permissions of the new hard drive:
sudo chown -R postgres:postgres /var/lib/postgresql/12/2TB1
Next, I created the new db cluster instance:
sudo pg_createcluster 12 osm_psql_db -d /var/lib/postgresql/12/2TB1/osm_psql_db -p 5432
I start the new instance:
sudo pg_ctlcluster 12 osm_psql_db start
I get the following error:
A dependency job for postgresql#12-osm_psql_db.service failed. See 'journalctl -xe' for details.
For anyone that stumbles upon the same issue... I tracked the problem down to the *.service file referencing the wrong mount point for the database cluster location. Here's what I did:
Enable the new service (not sure if this is needed, but what the heck...)
sudo systemctl enable postgresql#12-osm_psql_db
Edit the postgresql#12-osm_psql_db.service
sudo systemctl edit --full postgresql#12-osm_psql_db.service
Change
RequiresMountsFor=/etc/postgresql/%I /var/lib/postgresql/%I
To
RequiresMountsFor=/etc/postgresql/%I /var/lib/postgresql/12/2TB1/osm_psql_db
As part of the service script, %I expands to VERSION/CLUSTER which in my case would have been 12/osm_psql_db. Since I was choosing to place the DB on another SSD and the database can't reside in the root directory of a disk, the mount location in the *.service file needed to be updated to 12/2TB1/osm_psql_db. This would not be necessary if you were storing all your databases on a single hard disk.

Failed to start PostgreSQL Cluster 10-main when booting

when I try to boot Ubuntu, it never finishes the boot process because it appears the message "Failed to start PostgreSQL Cluster 10-main." I also get the same message with 9.5-main. But lets focus on 10.
When I execute:
systemctl status postgresql#10-main.service
I get the following message:
postgresql#10-main.service - PostgreSQL Cluster 10-main
Loaded: loaded (/lib/systemd/system/postgresql#.service; indirect; vendor preset: enabled)
Active: failed (Result: protocol) since Wed 2020-02-19 17:57:22 CET; 30 min ago
Process: 1602 ExecStart=/usr/bin/pg_ctlcluster --skip-systemctl-redirect 10-main start (code_exited, status=1/FAILURE)
PC_info systemd[1]: Starting PostgreSQL Cluster 10-main...
PC_info postgresql#10-main[1602]: Error: /usr/lib/postgresql/10/bin/pg_ctl /usr/lib/postgresql/10/bin/pg_ctl start -D /var/lib/postgresql/10/main -l /var/log/postgresql/postgresql-19-main.log -s -o -c config_file="/etc/postgresql/10/main/postgresql.conf" exit with status 1:
PC_info systemd[1]: postgresql#10-main.service: Can't open PID file /var/run/postgresql/10-main.pid (yet?) after start: No such file or directory
PC_info systemd[1]: postgresql#10-main.service: Failed with result 'protocol'.
PC_info systemd[1]: Failed to start PostgreSQL Cluster 10-main.
PC_info is information about my computer (user, date..) not relevant
I got this error from one day to an other without touching anything related to Database Servers.
I tried to fix it by my self but nothing worked
Writing the command
service postgresql#10-main start
I get
Job for postgresql#10-main.service failed because the service did not take the steps required by its unit configuration
See "systemctl status postgresql#10-main.service" and "journalctl -xe" for details.
Running this two command I get the message from the beginning.
Anyone has an idea of what is happening? How I can fix it?
I had same issue, I followed below steps,
Error status :
pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
10 main 5432 down postgres /var/lib/postgresql/10/main
/var/log/postgresql/postgresql-10-main.log
Applied Solution :
sudo chmod 700 -R /var/lib/postgresql/10/main
sudo -i -u postgres
postgres#abc:~$ /usr/lib/postgresql/10/bin/pg_ctl restart -D /var/lib/postgresql/10/main
After Solution status :
pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
10 main 5432 online postgres /var/lib/postgresql/10/main
/var/log/postgresql/postgresql-10-main.log
as mentioned by #gruentee in comment above,
/usr/lib/postgresql/10/bin/pg_ctl restart -D /var/lib/postgresql/10/main -l /var/log/postgresql/postgresql-10-main.log -s -o '-c config_file="/etc/postgresql/10/main/postgresql.conf"'
started the postgresSql DB. Dont forget to
sudo -i -u postgres
before issuing above command
I was facing the same challenge, I try a lot of methods but none work for me till I try these commands below
sudo apt-get -y install postgresql
sudo systemctl start postgresql#15-main.service
pg_lsclusters
then everything started working fine, but please the version of postgresql i'm using is 15 that's you are seeing 15 in the second command so in your case you substitute it with the version you are using

postgresql could not properly start exited with active (exited)

I installed posgresql from digitalocean and in the end of installation prints the below command in terminal
/usr/lib/postgresql/10/bin/pg_ctl -D /var/lib/postgresql/10/main -l logfile start
I tried to run it with sudo root user and also with switching to postgres user but gives me below error
waiting for server to start..../bin/sh: 1: cannot create logfile:
Permission denied stopped waiting pg_ctl: could not start server
but when i check the status it says
● postgresql.service - PostgreSQL RDBMS Loaded: loaded
(/lib/systemd/system/postgresql.service; enabled; vendor preset:
enabled) Active: active (exited) since Thu 2018-05-31 13:11:18 UTC;
56s ago Main PID: 3698 (code=exited, status=0/SUCCESS)
Tasks: 0 (limit: 2362) CGroup: /system.slice/postgresql.service
May 31 13:11:18 staging systemd1: Starting PostgreSQL RDBMS... May
31 13:11:18 staging systemd1: Started PostgreSQL RDBMS.
Status is not running except is exited.what the above command do and how can i run it ? I haven't faced it in previous versions
The idea is that you supply your actual log file instead of logfile, but I recommend that you configure logging properly in postgresql.conf and use pg_ctl without the -l option.
Set logging_collector to on.
Set log_filename to postgresql-%a.log.
Set log_rotation_size to 0.
Set log_truncate_on_rotation to on.
Then you'll get the log files in the log subdirectory of your PostgreSQL data directory, and they will be rotated on a weekly basis.

'service postgresql start' fails to start postgres service on Fedora

Newcomer to postgres here!
I edited pg_hba.conf as mentioned here , but when I try to restart postgresql service, the attempt fails. Below is the command line output with all the information I could gather.
[root#arunpc modules]# service postgresql restart
Redirecting to /bin/systemctl restart postgresql.service
Job failed. See system logs and 'systemctl status' for details.
[root#arunpc modules]# systemctl status postgresql.service
postgresql.service - PostgreSQL database server
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled)
Active: failed since Sun, 08 Apr 2012 21:29:06 +0530; 14s ago
Process: 12228 ExecStop=/usr/bin/pg_ctl stop -D ${PGDATA} -s -m fast (code=exited, status=0/SUCCESS)
Process: 12677 ExecStart=/usr/bin/pg_ctl start -D ${PGDATA} -s -o -p ${PGPORT} -w -t 300 (code=exited, status=1/FAILURE)
Process: 12672 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
Main PID: 12184 (code=exited, status=0/SUCCESS)
CGroup: name=systemd:/system/postgresql.service
[root#arunpc modules]# tail /var/log/messages
....
Apr 8 21:29:06 arunpc systemd[1]: postgresql.service: control process exited, code=exited status=1
Apr 8 21:29:06 arunpc systemd[1]: Unit postgresql.service entered failed state.
Apr 8 21:29:06 arunpc pg_ctl[12677]: pg_ctl: could not start server
Apr 8 21:29:06 arunpc pg_ctl[12677]: Examine the log output.
FWIW, here is the configuration file (pg_hba.conf) used:
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all postgres ident sameuser
local all all ident sameuser
# IPv4 local connections:
host all all 127.0.0.1 password
# IPv6 local connections:
host all all ::1 password
What could be the error here? It used to work fine before I made the edit (and since this was a development machine, I brilliantly didn't make any backup).
I would also like to get a more detailed log output. The log message in /var/log/messages file does ask me to "Examine the log output" - which log output would this be? What other troubleshooting steps can I take?
Many thanks in advance!
Depending on your startup script, it might redirect the postmaster's output to a file. This is usually server.log in the PGDATA directory. Things I'd try:
Comment out everything in pg_hba.conf and retry. If the problem is a syntax error in that file, then commenting out the offending line will allow the server to start and then you'll be able to uncomment one at a time until you find the error.
Start postmaster directly from the shell without sending it to the background. Just run postmaster -D <pgdata dir> and it should spew some more helpful logs.