Could not access file "pglogical" while trying to install pglogical - postgresql

I'm following instructions from https://github.com/2ndQuadrant/pglogical to install pglogical on postgres 12 on Centos 8. The install seems be successful:
yum -y install postgresql12-pglogical
Last metadata expiration check: 0:21:30 ago on Wed 30 Sep 2020 09:32:13 PM CDT.
Dependencies resolved.
=====================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
=====================================================================================================================================================================================================================================================
Installing:
postgresql12-pglogical x86_64 2.3.2-1.el8 2ndquadrant-dl-default-release-pg12 145 k
Installing dependencies:
postgresql12 x86_64 12.4-1PGDG.rhel8 pgdg12 1.6 M
postgresql12-server x86_64 12.4-1PGDG.rhel8 pgdg12 5.2 M
Transaction Summary
=====================================================================================================================================================================================================================================================
Install 3 Packages
Total download size: 7.0 M
Installed size: 29 M
Downloading Packages:
(1/3): postgresql12-12.4-1PGDG.rhel8.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01
(2/3): postgresql12-pglogical-2.3.2-1.el8.x86_64.rpm 117 kB/s | 145 kB 00:01
(3/3): postgresql12-server-12.4-1PGDG.rhel8.x86_64.rpm 4.0 MB/s | 5.2 MB 00:01
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 5.3 MB/s | 7.0 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : postgresql12-12.4-1PGDG.rhel8.x86_64 1/3
Running scriptlet: postgresql12-12.4-1PGDG.rhel8.x86_64 1/3
failed to link /usr/bin/psql -> /etc/alternatives/pgsql-psql: /usr/bin/psql exists and it is not a symlink
failed to link /usr/bin/clusterdb -> /etc/alternatives/pgsql-clusterdb: /usr/bin/clusterdb exists and it is not a symlink
failed to link /usr/bin/createdb -> /etc/alternatives/pgsql-createdb: /usr/bin/createdb exists and it is not a symlink
failed to link /usr/bin/createuser -> /etc/alternatives/pgsql-createuser: /usr/bin/createuser exists and it is not a symlink
failed to link /usr/bin/dropdb -> /etc/alternatives/pgsql-dropdb: /usr/bin/dropdb exists and it is not a symlink
failed to link /usr/bin/dropuser -> /etc/alternatives/pgsql-dropuser: /usr/bin/dropuser exists and it is not a symlink
failed to link /usr/bin/pg_basebackup -> /etc/alternatives/pgsql-pg_basebackup: /usr/bin/pg_basebackup exists and it is not a symlink
failed to link /usr/bin/pg_dump -> /etc/alternatives/pgsql-pg_dump: /usr/bin/pg_dump exists and it is not a symlink
failed to link /usr/bin/pg_dumpall -> /etc/alternatives/pgsql-pg_dumpall: /usr/bin/pg_dumpall exists and it is not a symlink
failed to link /usr/bin/pg_restore -> /etc/alternatives/pgsql-pg_restore: /usr/bin/pg_restore exists and it is not a symlink
failed to link /usr/bin/reindexdb -> /etc/alternatives/pgsql-reindexdb: /usr/bin/reindexdb exists and it is not a symlink
failed to link /usr/bin/vacuumdb -> /etc/alternatives/pgsql-vacuumdb: /usr/bin/vacuumdb exists and it is not a symlink
Running scriptlet: postgresql12-server-12.4-1PGDG.rhel8.x86_64 2/3
Installing : postgresql12-server-12.4-1PGDG.rhel8.x86_64 2/3
Running scriptlet: postgresql12-server-12.4-1PGDG.rhel8.x86_64 2/3
Installing : postgresql12-pglogical-2.3.2-1.el8.x86_64 3/3
Running scriptlet: postgresql12-pglogical-2.3.2-1.el8.x86_64 3/3
Verifying : postgresql12-pglogical-2.3.2-1.el8.x86_64 1/3
Verifying : postgresql12-12.4-1PGDG.rhel8.x86_64 2/3
Verifying : postgresql12-server-12.4-1PGDG.rhel8.x86_64 3/3
Installed:
postgresql12-12.4-1PGDG.rhel8.x86_64 postgresql12-pglogical-2.3.2-1.el8.x86_64 postgresql12-server-12.4-1PGDG.rhel8.x86_64
Complete!
But when I try to restart postgres, I get this error
systemctl restart postgresql
Job for postgresql.service failed because the control process exited with error code.
See "systemctl status postgresql.service" and "journalctl -xe" for details.
Relevant portions of the journalctl -xe
-- Unit postgresql.service has begun starting up.
Sep 30 21:54:59 aba postmaster[305963]: 2020-10-01 02:54:59.825 UTC [305963] FATAL: could not access file "pglogical": No such file or directory
Sep 30 21:54:59 aba postmaster[305963]: 2020-10-01 02:54:59.825 UTC [305963] LOG: database system is shut down
Sep 30 21:54:59 aba systemd[1]: postgresql.service: Main process exited, code=exited, status=1/FAILURE
Sep 30 21:54:59 aba systemd[1]: postgresql.service: Failed with result 'exit-code'.
Sep 30 21:54:59 aba systemd[1]: Failed to start PostgreSQL database server.
-- Subject: Unit postgresql.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit postgresql.service has failed.
--
-- The result is failed.
I am lost!

Your session log tells the the server was installed as a prerequisite, but the "link" messages insinuate that there was already an incompatible client version in place. Probably you had installed PostgreSQL from the CentOS packages, but the pglogical RPMs pulled in the PGDG packages.
The error message probably means that shared_preload_libraries contains pglogical, but pglogical.so could not be found in the lib directory.
Presumably the installation process edited the configuration in your old server installation, but installed the shared object in the new one.
Upshot: you cannot use those pglogical binaries with your installation. Either switch to the PGDG RPMs or build pglogical from source.
You see that there is a certain amount of conjecture in my deductions, but that should help you solve the problem.

Related

How can I upgrade PostgreSQL from version 11 to version 13?

I'm trying to upgrade PostgreSQL from 11 to 13 on a Debian system, but it fails. I have a single cluster that needs to be upgraded:
$ sudo -u postgres pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
11 main 5432 online postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
Here's what I've tried to upgrade it:
$ sudo -u postgres pg_upgradecluster 11 main
Stopping old cluster...
Warning: stopping the cluster using pg_ctlcluster will mark the systemd unit as failed. Consider using systemctl:
sudo systemctl stop postgresql#11-main
Restarting old cluster with restricted connections...
Notice: extra pg_ctl/postgres options given, bypassing systemctl for start operation
Error: cluster configuration already exists
Error: Could not create target cluster
After this, the system is left in an unusable state:
$ sudo systemctl status postgresql#11-main.service
● postgresql#11-main.service - PostgreSQL Cluster 11-main
Loaded: loaded (/lib/systemd/system/postgresql#.service; enabled-runtime; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2022-06-14 06:48:20 CEST; 19s ago
Process: 597 ExecStart=/usr/bin/pg_ctlcluster --skip-systemctl-redirect 11-main start (code=exited, status=0/SUCCE>
Process: 4508 ExecStop=/usr/bin/pg_ctlcluster --skip-systemctl-redirect -m fast 11-main stop (code=exited, status=>
Main PID: 684 (code=exited, status=0/SUCCESS)
CPU: 1.862s
Jun 14 06:47:23 argos systemd[1]: Starting PostgreSQL Cluster 11-main...
Jun 14 06:47:27 argos systemd[1]: Started PostgreSQL Cluster 11-main.
Jun 14 06:48:20 argos postgresql#11-main[4508]: Cluster is not running.
Jun 14 06:48:20 argos systemd[1]: postgresql#11-main.service: Control process exited, code=exited, status=2/INVALIDARG>
Jun 14 06:48:20 argos systemd[1]: postgresql#11-main.service: Failed with result 'exit-code'.
Jun 14 06:48:20 argos systemd[1]: postgresql#11-main.service: Consumed 1.862s CPU time.
$ sudo systemctl start postgresql#11-main.service
Job for postgresql#11-main.service failed because the service did not take the steps required by its unit configuration.
See "systemctl status postgresql#11-main.service" and "journalctl -xe" for details.
Luckily, rebooting the system brought the old cluster back online, but nothing has been upgraded. Why does the upgrade fail? What are "the steps required by its unit configuration"? How can I upgrade PostgreSQL with minimal downtime?
I found the source of my problem: a configuration file owned by the wrong user (root instead of postgres) that could not be removed by the pg_dropcluster command because I ran it as the user postgres.
For future reference, here are the correct steps to upgrade a PostgreSQL cluster from 11 to 13:
Verify the current cluster is the still the old version:
$ pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
11 main 5432 online postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
13 main 5434 down postgres /var/lib/postgresql/13/main /var/log/postgresql/postgresql-13-main.log
Run pg_dropcluster 13 main as user postgres:
$ sudo -u postgres pg_dropcluster 13 main
Warning: systemd was not informed about the removed cluster yet.
Operations like "service postgresql start" might fail. To fix, run:
sudo systemctl daemon-reload
Run the pg_upgradecluster command as user postgres:
$ sudo -u postgres pg_upgradecluster 11 main
Verify that everything works, and that the only online cluster is now 13:
$ pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
11 main 5434 down postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
13 main 5432 online postgres /var/lib/postgresql/13/main /var/log/postgresql/postgresql-13-main.log
Drop the old cluster:
$ sudo -u postgres pg_dropcluster 11 main
Uninstall the previous version of PostgreSQL:
$ sudo apt remove 'postgresql*11'
The Debian packages create a cluster automatically when you install the server package, so get rid of that:
pg_dropcluster 13 main
Then stop the v11 server and try again.

Unable to start PostgreSQL 12 Server

I want to setup PostgreSQL 12 with PostGIS 3 on Ubuntu 20.04 for the purpose of creating an OSM Tile Server. I want to have 2 different clusters, one for a regular PSQL database and another for OSM data. I can't seem to get the one for the OSM data up and running:
When I run pg_lsclusters, I get the following:
Ver Cluster Port Status Owner Data directory Log file
12 main 5433 online postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log
12 osm_psql_db 5432 down postgres /var/lib/postgresql/12/2TB1/osm_psql_db /var/log/postgresql/postgresql-12-osm_psql_db.log
When I run journalctl -xe, I get the following:
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: Dependency failed for PostgreSQL Cluster 12-osm_psql_db.
-- Subject: A start job for unit postgresql#12-osm_psql_db.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- A start job for unit postgresql#12-osm_psql_db.service has finished with a failure.
--
-- The job identifier is 9566 and the job result is dependency.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: postgresql#12-osm_psql_db.service: Job postgresql#12-osm_psql_db.service/start failed with result 'dependency'.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: var-lib-postgresql-12-osm_psql_db.mount: Job var-lib-postgresql-12-osm_psql_db.mount/start failed with result 'dependency'.
Mar 13 11:47:37 cdil-MS-7B92 systemd[1]: dev-disk-by\x2dlabel-osm_psql_db.device: Job dev-disk-by\x2dlabel-osm_psql_db.device/start failed with result 'timeout'.
Mar 13 11:47:43 cdil-MS-7B92 PackageKit[27900]: daemon quit
Mar 13 11:47:43 cdil-MS-7B92 systemd[1]: packagekit.service: Succeeded.
-- Subject: Unit succeeded
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- The unit packagekit.service has successfully entered the 'dead' state.
Any idea what could be holding me up?
*** EXTRA INFO JUST IN CASE ***
In terms of how I set up everything, I installed the following packages:
sudo apt install postgresql-12 postgresql-contrib postgis postgresql-12-postgis-3
Because the OSM data is quite large, I want to store that particular cluster on another hard disk. It's called "2TB1" and it's been mounted to /var/lib/postgresql/12/2TB1 because I realized that the postgres user needed access to the data_directory folder and all parent folders leading up to it.
To do so I modified the permissions of the new hard drive:
sudo chown -R postgres:postgres /var/lib/postgresql/12/2TB1
Next, I created the new db cluster instance:
sudo pg_createcluster 12 osm_psql_db -d /var/lib/postgresql/12/2TB1/osm_psql_db -p 5432
I start the new instance:
sudo pg_ctlcluster 12 osm_psql_db start
I get the following error:
A dependency job for postgresql#12-osm_psql_db.service failed. See 'journalctl -xe' for details.
For anyone that stumbles upon the same issue... I tracked the problem down to the *.service file referencing the wrong mount point for the database cluster location. Here's what I did:
Enable the new service (not sure if this is needed, but what the heck...)
sudo systemctl enable postgresql#12-osm_psql_db
Edit the postgresql#12-osm_psql_db.service
sudo systemctl edit --full postgresql#12-osm_psql_db.service
Change
RequiresMountsFor=/etc/postgresql/%I /var/lib/postgresql/%I
To
RequiresMountsFor=/etc/postgresql/%I /var/lib/postgresql/12/2TB1/osm_psql_db
As part of the service script, %I expands to VERSION/CLUSTER which in my case would have been 12/osm_psql_db. Since I was choosing to place the DB on another SSD and the database can't reside in the root directory of a disk, the mount location in the *.service file needed to be updated to 12/2TB1/osm_psql_db. This would not be necessary if you were storing all your databases on a single hard disk.

Failed to load nf_conntrack

[root#name ~]# systemctl status firewalld -l
* firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
Nov 17 18:47:24 strike325 systemd[1]: Starting firewalld - dynamic firewall daemon...
Nov 17 18:47:25 strike325 systemd[1]: Started firewalld - dynamic firewall daemon.
Nov 17 18:47:25 strike325 firewalld[1176]: WARNING: ipset not usable, disabling ipset usage in firewall.
Nov 17 18:47:26 strike325 firewalld[1176]: ERROR: Failed to load nf_conntrack module: modprobe: ERROR: could not find module by name='nf_conntrack'
modprobe: ERROR: could not insert 'nf_conntrack': Function not implemented
modprobe: ERROR: Error running install command for nf_conntrack
modprobe: ERROR: could not insert 'nf_conntrack': Operation not permitted
Nov 17 18:47:26 strike325 firewalld[1176]: E
RROR: Raising SystemExit in run_server
Nov 17 19:47:16 strike325 systemd[1]: Starting firewalld - dynamic firewall daemon...
Nov 17 19:47:17 strike325 systemd[1]: Started firewalld - dynamic firewall daemon.
Nov 17 19:47:17 strike325 firewalld[2689]: WARNING: ipset not usable, disabling ipset usage in firewall.
Nov 17 19:47:18 strike325 firewalld[2689]: ERROR: Failed to load nf_conntrack module: modprobe: ERROR: could not find module by name='nf_conntrack'
modprobe: ERROR: could not insert 'nf_conntrack': Function not implemented
modprobe: ERROR: Error running install command for nf_conntrack
modprobe: ERROR: could not insert 'nf_conntrack': Operation not permitted
Nov 17 19:47:18 strike325 firewalld[2689]: ERROR: Raising SystemExit in run_server
I've recently purchased a VPS using Centos 7 (x64) and I'm having some trouble with my firewalld. I found the fix here but unfortunately it's no longer working for me. Any help would be appreciated as I haven't been able to find any permanent fixes.
Other fix attempts so far:
restart dbus
restart firewalld
Reverting and locking the version of firewalld (temporary fix)
I was going to post my fix in a while now (I'm supporting a number of servers, all affected with the same issue), but haven't had the time.
But your question made me motivated to finally write it down.
The solution from this post is:
create a script which generates modules.builtin file specific to your current kernel
create a SystemD unit which automatically recreates the same before FirewallD during startup (useful in case the kernel is upgraded).
In this way, you permanently solve the issue without hacking a bit of FirewallD code.
Workaround is to downgrade firewalld to 7.6:
wget http://vault.centos.org/7.6.1810/os/x86_64/Packages/firewalld-0.5.3-5.el7.noarch.rpm http://vault.centos.org/7.6.1810/os/x86_64/Packages/firewalld-filesystem-0.5.3-5.el7.noarch.rpm http://vault.centos.org/7.6.1810/os/x86_64/Packages/python-firewall-0.5.3-5.el7.noarch.rpm
yum downgrade firewalld-0.5.3-5.el7.noarch.rpm firewalld-filesystem-0.5.3-5.el7.noarch.rpm python-firewall-0.5.3-5.el7.noarch.rpm
and then lock firewalld from yum feature updates:
yum -y install yum-versionlock
yum versionlock firewalld firewalld-filesystem python-firewall
you can find more info in this thread.

CATALINA_PID was set but the specified file does not exist

I was following this guide to setup tomcat on Ubuntu 16.04.2 LTS Xenial (https://www.digitalocean.com/community/tutorials/how-to-install-apache-tomcat-8-on-ubuntu-16-04).
Got to the point of the first launch:
sudo systemctl start tomcat
and received an error:
~$ sudo systemctl start tomcat
Job for tomcat.service failed because the control process exited with error code. See "systemctl status tomcat.service" and "journalctl -xe" for details.
journalctl -xe
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit tomcat.service has begun starting up.
Feb 27 15:23:42 76672.local startup.sh[20551]: Tomcat started.
Feb 27 15:23:42 76672.local shutdown.sh[20563]: $CATALINA_PID was set but the specified file does not exist. Is Tomcat running? Stop aborted.
Feb 27 15:23:42 76672.local systemd[1]: tomcat.service: Control process exited, code=exited status=1
Feb 27 15:23:42 76672.local systemd[1]: Failed to start Tomcat 9 servlet container.
-- Subject: Unit tomcat.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit tomcat.service has failed.
--
-- The result is failed.
CATALINA_PID expected to be in a directory /opt/tomcat/latest/temp/tomcat.pid
I gave rights to do everyone for everything for this dir /opt/tomcat and all subfolders/files:
~$ namei -l /opt/tomcat
f: /opt/tomcat
drwxr-xr-x root root /
drwxr-xr-x root root opt
drwxrwxrwx tomcat tomcat tomcat
Does anyone can point me to the possible solutions?
UPD:
Using bin/startup.sh Tomcat successfully starts.
When you use CATALINA_PID in tomcat.service file please make sure to use the path inside of the double-quotes.
Environment="CATALINA_PID=/opt/tomcat/latest/temp/tomcat.pid"
Like below
[Unit]
Description=Tomcat 9 servlet container
After=network.target
[Service]
Type=forking
User=tomcat
Group=tomcat
Environment="/usr/lib/jvm/java-1.8.0-openjdk-amd64"
Environment="JAVA_OPTS=-Djava.security.egd=file:///dev/urandom -Djava.awt.headless=true"
Environment="CATALINA_BASE=/opt/tomcat/"
Environment="CATALINA_HOME=/opt/tomcat/"
Environment="CATALINA_PID=/opt/tomcat/temp/tomcat.pid"
Environment="CATALINA_OPTS=-Xms512M -Xmx1024M -server -XX:+UseParallelGC"
ExecStart=/opt/tomcat/bin/startup.sh
ExecStop=/opt/tomcat/bin/shutdown.sh
[Install]
WantedBy=multi-user.target
congratulations!

PG::ConnectionBad Postgres Cluster down

Digitalocean disabled my droplet's internet access. After fixing the error (rollback to older backup) they restored the internet access. But afterwards I constantly get an error when deploying, I can't seem to get my Postgres database up and running.
I'm getting an error each time I try to deploy my application.
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
So I used SSH to login to my server and check if my Postgres was actually running with:
pg_lsclusters
Results into:
Ver Cluster Port Status Owner Data directory Log file
9.5 main 5432 down postgres /var/lib/postgresql/9.5/main /var/log/postgresql/postgresql-9.5-main.log
Postgres server status
So my Postgres server seems to be down. I tried putting it 'up' again with:
pg_ctlcluster 9.5 main start After doing so I got the error: Insecure directory in $ENV{PATH} while running with -T switch at /usr/bin/pg_ctlcluster line 403.
And /usr/bin/pg_ctlcluster on line 403 says:
system 'systemctl', 'is-active', '-q', "postgresql\#$version-$cluster";
But I'm not to sure what the problem could be here and how I could fix this.
Update
I also tried updating the permissions on /bin to 755 as mentioned here. Sadly that did not fix my problem.
Update 2
I changed the /usr/bin to 755. Now when I try pg_ctlcluster 9.5 main start, I get this:
Job for postgresql#9.5-main.service failed because the control process exited with error code. See "systemctl status postgresql#9.5-main.service" and "journalctl -xe" for details.
And inside the systemctl status postgresql#9.5-main.service:
postgresql#9.5-main.service - PostgreSQL Cluster 9.5-main
Loaded: loaded (/lib/systemd/system/postgresql#.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Sun 2018-01-28 17:32:38 EST; 45s ago
Process: 22473 ExecStart=postgresql#%i --skip-systemctl-redirect %i start (code=exited, status=1/FAILURE)
Jan 28 17:32:08 *url* systemd[1]: Starting PostgreSQL Cluster 9.5-main...
Jan 28 17:32:38 *url* postgresql#9.5-main[22473]: The PostgreSQL server failed to start.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Control process exited, code=exited status=1
Jan 28 17:32:38 *url* systemd[1]: Failed to start PostgreSQL Cluster 9.5-main.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Unit entered failed state.
Jan 28 17:32:38 *url* systemd[1]: postgresql#9.5-main.service: Failed with result 'exit-code'.
Thanks!
You better not mix systemctl and pg_ctlcluster. Let systemctl makes the calls to pg_ctlcluster with the right user and permissions. You should start your postgresql instance with
sudo systemctl start postgresql#9.5-main.service
Also, check the errors in the startup log. You can post them too, to help you figure out what's going on.
Your systemctl status also outputs that the service is disable, so, when the server reboots, you will have to start the service manually. To enable it run:
sudo systemctl enable postgresql#9.5-main.service
I hope it helps
It is mainly because /etc/hosts file is somehow changed.I have removed extra space inside /etc/hosts file.Use cat /etc/hosts
Add these lines into the file
127.0.0.1 localhost
127.0.1.1 your-host-name
::1 ip6-localhost ip6-loopback
And I have given permission 644 to /etc/hosts file.It is working for me even after the reboot of the system.