postgresql archive permission denied - postgresql

We have installed postgres v12 on ubuntu 20.04 (with apt install -y postgresql postgresql-contrib) and wish to enable archiving to /data/db/postgres/archive by setting the following in postgresql.conf:
max_wal_senders=2
wal_keep_segments=256
wal_sender_timeout=60s
archive_mode=on
archive_command=cp %p /data/db/postgres/archive/%f
However the postgres service fails to write there:
2022-11-15 15:02:26.212 CET [392860] FATAL: archive command failed with exit code 126
2022-11-15 15:02:26.212 CET [392860] DETAIL: The failed archive command was: archive_command=cp pg_wal/000000010000000000000002 /data/db/postgres/archive/000000010000000000000002
2022-11-15 15:02:26.213 CET [392605] LOG: archiver process (PID 392860) exited with exit code 1
sh: 1: pg_wal/000000010000000000000002: Permission denied
This directory /data/db/postgres/archive/ is owned by the postgres user and when we su postgres we are able to create and delete files without a problem.
Why can the postgresql service (running as postgres) not write to a directory it owns?
Here are the permissions on all the parents of the archive directory:
drwxr-xr-x 2 postgres root 6 Nov 15 14:59 /data/db/postgres/archive
drwxr-xr-x 3 root root 21 Nov 15 14:29 /data/db/postgres
drwxr-xr-x 3 root root 22 Nov 15 14:29 /data/db
drwxr-xr-x 5 root root 44 Nov 15 14:29 /data

2022-11-15 15:02:26.212 CET [392860] DETAIL: The failed archive command was: archive_command=cp pg_wal/000000010000000000000002 /data/db/postgres/archive/000000010000000000000002
So, your archive_command is apparently set to the peculiar string archive_command=cp %p /data/db/postgres/archive/%f.
After the %variables are substituted, the result is passed to the shell. The shell does what it was told, which is to set the (unused) environment variable 'archive_command' to be 'cp', and then tries to execute the file pg_wal/000000010000000000000002, which is not allowed to because it doesn't have the execute bit set.
I don't know how you managed to get such a deformed archive_command, but it didn't come from anything you showed us.

Related

How to fix "psql: error: xxx and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?"

How to fix this error?
psql: error: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
We use Docker, Postgres12, Wercker.
When we run this command in Wercker, we can't create a database.
But we can create it in my local Ubuntu environment.
sudo docker exec docker_postgres_1 psql -U postgres -c 'create database postgres_dev'
First, you need to make sure the socket file is located in /var/run/postgresql/.s.PGSQL.5432. To check that
$ cat /var/run/postgresql/.s.PGSQL.5432
if result shows something, then the problem is anything else. But, if file is not there you need to check /tmp dir (specially for OSX Homebrew users)
$ cd /tmp
$ l
total 16
drwxrwxrwt 7 root wheel 224B Mar 11 08:03 .
drwxr-xr-x 6 root wheel 192B Jan 23 18:35 ..
-rw-r--r-- 1 root wheel 65B Nov 7 22:59 .BBE72B41371180178E084EEAF106AED4F350939DB95D3516864A1CC62E7AE82F
srwxrwxrwx 1 shiva wheel 0B Mar 11 08:03 .s.PGSQL.5432
-rw------- 1 shiva wheel 57B Mar 11 08:03 .s.PGSQL.5432.lock
drwx------ 3 shiva wheel 96B Mar 10 17:11 com.apple.launchd.C1tUB2MvF8
drwxr-xr-x 2 root wheel 64B Mar 10 17:10 powerlog
Now, there are two ways you can solve the error
Solution One
You can change the application configuration to see for sockets at /tmp/.s.PGSQL.5432
For Rails Users
# config/database.yml
default: &default
adapter: postgresql
pool: 5
# port:
timeout: 5000
encoding: utf8
# min_messages: warning
socket: /tmp/.s.PGSQL.5432
Solution Two
You can create symlinks to the expected location
$ sudo mkdir /var/pgsql_socket
$ sudo ln /tmp/.s.PGSQL.5432 /var/pgsql_socket/
```
Then the error should go.
Hope this helps.

Cannot run psql in PostgreSQL 9.5

I am using PostgreSQL 9.5 on Ubuntu 16.04 LTS.
I receive the below error when I type psql:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
On checking the logs in /var/log/postgresql/postgresql-9.5-main.log, I see the error as:
2018-11-26 13:17:41 IST [3508-1] FATAL: could not access private key file "/etc/ssl/private/ssl-cert-snakeoil.key": Permission denied
Below are the permissions of the /etc/ssl/private and ssl-cert-snakeoil.key files:
vivek#vivek-ThinkPad-E480:~$ ls -l /etc/ssl
total 36
drwxr-xr-x 2 root root 20480 Nov 22 13:06 certs
-rwxr-xr-x 1 root root 10835 Dec 8 2017 openssl.cnf
drwxr--r-- 2 root ssl-cert 4096 Nov 22 13:06 private
vivek#vivek-ThinkPad-E480:~$ sudo ls -l /etc/ssl/private
total 4
-rw-r----- 1 root ssl-cert 1704 Nov 22 13:06 ssl-cert-snakeoil.key
The postgres user is also added to the group ssl-cert.
vivek#vivek-ThinkPad-E480:~$ getent group ssl-cert
ssl-cert:x:112:postgres
NOTE: I found that there is no server.key present in /var/lib/postgresql/9.5/main.
I also posted this on DBA Stackexchange, but no response as yet.
Can anyone guide me in the right direction in setting permissions?
That can never work, and your server will not be able to start, because the OS user postgres has no permissions to access files in etc/ssl/private.
To allow users in the group ssl-cert to access files in the directory, run
chmod g+x /etc/ssl/private
While you're at it, make sure that /etc/ssl has the required permissions.
To test if everything works, become user postgres and try to read the file.

openshift postgres persistent volume permissions

The postgres image I am currently deploying with openshift is generally working great. However I need to persistently store the database data (of course) and to do so i created a persistent volume claim and mounted it to the postgres data directory like so:
- mountPath: /var/lib/pgsql/data/userdata
name: db-storage-volume
and
- name: db-storage-volume
persistentVolumeClaim:
claimName: db-storage
The problem I am facing now is that the initdb script wants to change the permission of that data folder, but it cant and the directory is assigned to a very weird user/group, as the output of ls -la /var/lib/pgsql/data indicates (including the failing command output):
total 12
drwxrwxr-x. 3 postgres root 21 Aug 30 13:06 .
drwxrwx---. 3 postgres root 17 Apr 5 09:55 ..
drwxrwxrwx. 2 nobody nobody 12288 Jun 26 11:11 userdata
chmod: changing permissions of '/var/lib/pgsql/data/userdata': Permission denied
How can I handle this? I mean the permissions are enough to read/write but initdb (and the base images initialization functions) really want to change the permission of that folder.
Just as I had sent my question I had an idea and it turns out it worked:
Change the mount to the parent folder /var/lib/pgsql/data/
Modify my entry script to include a mkdir /var/lib/pgsql/data/userdata when it runs first (aka the folder does not exist yet)
Now it is:
total 16
drwxrwxrwx. 3 nobody nobody 12288 Aug 30 13:19 .
drwxrwx---. 3 postgres root 17 Apr 5 09:55 ..
drwxr-xr-x. 2 1001320000 nobody 4096 Aug 30 13:19 userdata
Which works. Notice that the folder itself is still owned by nobody:nobody and is 777, but the created userdata folder is owned by the correct user.

Is the server running locally and accepting connections on Unix domain socket “/var/run/postgresql/.s.PGSQL.5432”? in Odoo

After some time the Postgres database stopped working of my live server. I'm working on this server from last 8 months. Now suddenly it's stopped working.
when I try to enter the command, psql produces an error
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
enter image description here
I'm using odoo8.
First, you need to make sure the socket file is located in /var/run/postgresql/.s.PGSQL.5432. To check that
$ cat /var/run/postgresql/.s.PGSQL.5432
if result shows something, then the problem is anything else. But, if file is not there you need to check /tmp dir (specially for OSX Homebrew users)
$ cd /tmp
$ l
total 16
drwxrwxrwt 7 root wheel 224B Mar 11 08:03 .
drwxr-xr-x 6 root wheel 192B Jan 23 18:35 ..
-rw-r--r-- 1 root wheel 65B Nov 7 22:59 .BBE72B41371180178E084EEAF106AED4F350939DB95D3516864A1CC62E7AE82F
srwxrwxrwx 1 shiva wheel 0B Mar 11 08:03 .s.PGSQL.5432
-rw------- 1 shiva wheel 57B Mar 11 08:03 .s.PGSQL.5432.lock
drwx------ 3 shiva wheel 96B Mar 10 17:11 com.apple.launchd.C1tUB2MvF8
drwxr-xr-x 2 root wheel 64B Mar 10 17:10 powerlog
Now, there are two ways you can solve the error
Solution One
You can change the application configuration to see for sockets at /tmp/.s.PGSQL.5432
For Rails Users
# config/database.yml
default: &default
adapter: postgresql
pool: 5
# port:
timeout: 5000
encoding: utf8
# min_messages: warning
socket: /tmp/.s.PGSQL.5432
Solution Two
You can create symlinks to the expected location
$ sudo mkdir /var/pgsql_socket
$ sudo ln /tmp/.s.PGSQL.5432 /var/pgsql_socket/
Then the error should go.
Hope this helps.
Note: Your default socket directory may not be /tmp
Did you update/upgrade your database?
Did you start a docker container that interfered with any of your data-store/socket file locations?
This probably doesn't fit your situation exactly, but maybe it will provide some insight:
Sometimes when you try
sudo systemctl start postgresql.service
and the systemd status says it is started but you still get that error message when trying to connect, try this instead:
sudo pg_ctlcluster <version> <cluster> <action>
which in my case had been
sudo pg_ctlcluster 13 main start

Where does PostgreSQL store the database?

Where are the files for a PostgreSQL database stored?
To see where the data directory is, use this query.
show data_directory;
To see all the run-time parameters, use
show all;
You can create tablespaces to store database objects in other parts of the filesystem. To see tablespaces, which might not be in that data directory, use this query.
SELECT *, pg_tablespace_location(oid) FROM pg_tablespace;
On Windows7 all the databases are referred by a number in the file named pg_database under C:\Program Files (x86)\PostgreSQL\8.2\data\global. Then you should search for the folder name by that number under C:\Program Files (x86)\PostgreSQL\8.2\data\base. That is the content of the database.
As suggested in "PostgreSQL database default location on Linux", under Linux you can find out using the following command:
ps aux | grep postgres | grep -- -D
Open pgAdmin and go to Properties for specific database. Find OID and then open directory
<POSTGRESQL_DIRECTORY>/data/base/<OID>
There should be your DB files.
Under my Linux installation, it's here: /var/lib/postgresql/8.x/
You can change it with initdb -D "c:/mydb/"
Everyone already answered but just for the latest updates. If you want to know where all the configuration files reside then run this command in the shell
SELECT name, setting FROM pg_settings WHERE category = 'File Locations';
I'd bet you're asking this question because you've tried pg_ctl start and received the following error:
pg_ctl: no database directory specified and environment variable PGDATA unset
In other words, you're looking for the directory to put after -D in your pg_ctl start command.
In this case, the directory you're looking for contains these files.
PG_VERSION pg_dynshmem pg_multixact
pg_snapshots pg_tblspc postgresql.conf
base pg_hba.conf pg_notify
pg_stat pg_twophase postmaster.opts
global pg_ident.conf pg_replslot
pg_stat_tmp pg_xlog postmaster.pid
pg_clog pg_logical pg_serial
pg_subtrans postgresql.auto.conf server.log
You can locate it by locating any of the files and directories above using the search provided with your OS.
For example in my case (a HomeBrew install on Mac OS X), these files are located in /usr/local/var/postgres. To start the server I type:
pg_ctl -D /usr/local/var/postgres -w start
... and it works.
Postgres stores data in files in its data directory. Follow the steps below to go to a database and its files:
The database corresponding to a postgresql table file is a directory. The location of the entire data directory can be obtained by running SHOW data_directory.
in a UNIX like OS (eg: Mac) /Library/PostgreSQL/9.4/data
Go inside the base folder in the data directory which has all the database folders: /Library/PostgreSQL/9.4/data/base
Find the database folder name by running (Gives an integer. This is the database folder name):
SELECT oid from pg_database WHERE datname = <database_name>;
Find the table file name by running (Gives an integer. This is the file name):
SELECT relname, relfilenode FROM pg_class WHERE relname = <table_name>;
This is a binary file. File details such as size and creation date time can be obtained as usual. For more info read this SO thread
The location of specific tables/indexes can be adjusted by TABLESPACEs:
CREATE TABLESPACE dbspace LOCATION '/data/dbs';
CREATE TABLE something (......) TABLESPACE dbspace;
CREATE TABLE otherthing (......) TABLESPACE dbspace;
On Mac: /Library/PostgreSQL/9.0/data/base
The directory can't be entered, but you can look at the content via: sudo du -hc data
picmate's answer is right. on windows the main DB folder location is (at least on my installation)
C:\PostgreSQL\9.2\data\base\
and not in program files.
his 2 scripts, will give you the exact directory/file(s) you need:
SELECT oid from pg_database WHERE datname = <database_name>;
SELECT relname, relfilenode FROM pg_class WHERE relname = <table_name>;
mine is in datname 16393 and relfilenode 41603
On Windows, the PGDATA directory that the PostgresSQL docs describe is at somewhere like C:\Program Files\PostgreSQL\8.1\data. The data for a particular database is under (for example) C:\Program Files\PostgreSQL\8.1\data\base\100929, where I guess 100929 is the database number.
I'm running postgres (9.5) in a docker container (on CentOS, as it happens), and as Skippy le Grand Gourou mentions in a comment above, the files are located in /var/lib/postgresql/data/.
$ docker exec -it my-postgres-db-container bash
root#c7d61efe2a5d:/# cd /var/lib/postgresql/data/
root#c7d61efe2a5d:/var/lib/postgresql/data# ls -lh
total 56K
drwx------. 7 postgres postgres 71 Apr 5 2018 base
drwx------. 2 postgres postgres 4.0K Nov 2 02:42 global
drwx------. 2 postgres postgres 18 Dec 27 2017 pg_clog
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_commit_ts
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_dynshmem
-rw-------. 1 postgres postgres 4.4K Dec 27 2017 pg_hba.conf
-rw-------. 1 postgres postgres 1.6K Dec 27 2017 pg_ident.conf
drwx------. 4 postgres postgres 39 Dec 27 2017 pg_logical
drwx------. 4 postgres postgres 36 Dec 27 2017 pg_multixact
drwx------. 2 postgres postgres 18 Nov 2 02:42 pg_notify
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_replslot
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_serial
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_snapshots
drwx------. 2 postgres postgres 6 Sep 16 21:15 pg_stat
drwx------. 2 postgres postgres 63 Nov 8 02:41 pg_stat_tmp
drwx------. 2 postgres postgres 18 Oct 24 2018 pg_subtrans
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_tblspc
drwx------. 2 postgres postgres 6 Dec 27 2017 pg_twophase
-rw-------. 1 postgres postgres 4 Dec 27 2017 PG_VERSION
drwx------. 3 postgres postgres 92 Dec 20 2018 pg_xlog
-rw-------. 1 postgres postgres 88 Dec 27 2017 postgresql.auto.conf
-rw-------. 1 postgres postgres 21K Dec 27 2017 postgresql.conf
-rw-------. 1 postgres postgres 37 Nov 2 02:42 postmaster.opts
-rw-------. 1 postgres postgres 85 Nov 2 02:42 postmaster.pid
A single Terminal command: pg_lsclusters, (using Ubuntu)
What you need is under Data directory:
Ver Cluster Port Status Owner Data directory Log file
10 main 5432 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log
11 main 5433 online postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log