I am newbie to docker. Today when I restarted my laptop, I found that all my docker images got deleted. Now i am installing the images again but when i executing the command for the postgres. I am getting this error while executing
mkdir -p $HOME/docker/volumes/postgres
docker run --name mydb -e POSTGRES_PASSWORD=abcd -d -p 5433:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres:12.7
PostgreSQL Database directory appears to contain a database; Skipping initialization
FATAL: could not create lock file "postmaster.pid": No such file or directory
Also if I can get my existing database that would be wonderful.
I have enabled WAL archiving in postgres configuration file. When I restarted Postgres service WAL recovery is not working. There was no wal recovery entries in logs.
Steps I followed:
Created directory for wal:
mkdir -p /var/lib/pgsql/wals/
mkdir -p /var/lib/pgsql/backups/
chown postgres:postgres -R /var/lib/pgsql/backups/
chown postgres:postgres -R /var/lib/pgsql/wals/
Edited the postgresql.conf with the below changes:
wal_level=archive
archive_mode=on
archive_command = 'test ! -f /var/lib/pgsql/wals/%f && cp %p /var/lib/pgsql/wals/%f'
sudo service postgresql restart 10
sudo su - postgres
pg_basebackup -D /var/lib/pgsql/data #created base backup
tar -C /var/lib/pgsql/data/ -czvf /var/lib/pgsql/backups/pg_basebackup_backup.tar.gz .
Deleted two rows of data in my database and stopped the postgres service
sudo service postgresql stop 10
Extracted the Basebackup
tar xvf /var/lib/pgsql/backups/pg_basebackup_backup.tar.gz -C /var/lib/pgsql/data
Created recovery.conf with the below content and restarted postgres service
echo "restore_command = 'cp /var/lib/pgsql/wals/%f %p'">/var/lib/pgsql/recovery.conf
cp /var/lib/pgsql/recovery.conf /var/lib/pgsql/data/
sudo service postgresql stop 10
sudo service postgresql start 10
There was no wal recovery entries in logs and the two rows which I deleted didn't get restored.
pg_basebackup can catch all WAL segments during backup. I use basebackup into tar format with "-X stream" option and all works well. See here - pg_basebackup – bash script for backup and archiving on Google storage
It works excellent - I backup database 4.5+ TB big which takes almost 2 days.
Restoration is described here - pg_basebackup / pg-barman – restore tar backup
All works - we already had incidents when we had to restore from these backups.
The WAL containing the two deleted rows probably hasn't been archived yet.
archive_command is run whenever a 16 MB WAL segment is full or something forces a log switch.
To be able to recover the latest changes that have not been archived yet, you have to copy the contents of pg_wal to the pg_wal directory in the restored base backup.
You can also consider streaming replication or pg_receivewal if you cannot afford to lose transactions.
When trying to perform pg_basebackup on a replica, I always get the following message:
postgres#db1:~/10$ pg_basebackup -h foo.bar.com -U repluser -D /var/lib/postgresql/10/main -v -P
pg_basebackup: initiating base backup, waiting for checkpoint to complete
I've tried waiting, but nothing happens. Is it possible to speed up the process?
Call pg_basebackup with the option --checkpoint=fast to force a fast checkpoint rather than waiting for a spread one to complete.
It's possible to force a checkpoint to complete. To do so, run CHECKPOINT; on the master server:
$ sudo su - postgres
$ psql
postgres=# CHECKPOINT;
Wanted to point out that while the accepted answer of running 'CHECKPOINT' command on the primary server is correct, it is not meant to be run during normal operation, this is according to postgres documentation which you can see here:
https://www.postgresql.org/docs/current/sql-checkpoint.html
So be sure and not do this while the server is processing normal operations from your apps etc.
I restarted my Postgres server but now.
I checked my "pgstartup.log" log file. This says:
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
Success. You can now start the database server using:
/usr/bin/postgres -D /var/lib/pgsql/data
/usr/bin/pg_ctl -D /var/lib/pgsql/data -l logfile start
FATAL: could not open lock file "/tmp/.s.PGSQL.5432.lock": Permission denied
FATAL: could not open lock file "/tmp/.s.PGSQL.5432.lock": Permission denied
Do you think deleting /tmp/.s.PGSQL.5432.lock would work ?
Postgres cannot write a file to /tmp because of permissions set on /tmp directory.
As root user execute in terminal:
chmod 1777 /tmp
PostgreSQL normally deletes the lock file when terminates correctly.
It is probably due to another PostgreSQL instance running with a different user that has been terminated abnormally (a kill -9 to postmaster).
So, if you are sure no Postgres processes are running, you can probably delete that file without any issue. You should also check with the ipcs command if there is any stale shared memory segment, and in that case delete it with ipcrm.
Probably the be best way to address all these things at once is rebooting the server.
P.S.: never kill -9 any PostgreSQL process.
It looks like you probably have another PostgreSQL instance running on the same port as a different user, or you previously started this PostgreSQL instance as a different user then stopped it uncleanly.
Check the ownership of /tmp/.s.PGSQL.5432.lock:
ls -l /tmp/.s.PGSQL.5432.lock
Does it match the user you're running PostgreSQL as?
It's relatively harmless to delete the lock files in /tmp/. (Never, ever delete the postmaster.pid lock file though). If the other PostgreSQL instance is still running you'll lose the ability to connect to it over a unix socket, or you might get an error about being unable to bind to port 5432 on tcp.
I agree with #mnencia that a server reboot is the best option if it's easy and practical.
If you know that no other Postgres processes are running, please delete these 2 files and try again:
$ sudo rm /tmp/.s.PGSQL.5432.lock
$ sudo rm /tmp/.s.PGSQL.5432
Then, you would be able to run the server as a background process with the command:
$ pg_ctl -D /usr/local/var/postgres start
If you are in OS X, put the alias in the.bash_profile as below:
alias pgb='pg_ctl -D /usr/local/var/postgres start'
Now, source it with the command:
$ source ~/.bash_profile
The Postgres server will run with the command:
$ pgb
To me it was permission problem for database file, It was group/world readable. And that was wrong! Database file should be 0700. Aft
Thanks for the suggestions.
First I tried changing the permission of the lock file but it didn't work
Later I deleted the lock file which resolved my issue.
Thanks
Thanks.
I was trying to install postgres on my mac.I was receiving
FATAL: could not open lock file "/tmp/.s.PGSQL.5432.lock": Permission denied
After deleting the /tmp/.s.PGSQL.5432.lock file, the server started working.
Yes, I had the same issue, and I fixed it by running this command
$ sudo rm /tmp/.s.PGSQL.5432.lock
$ sudo rm /tmp/.s.PGSQL.5432
$ pg_ctl -D /usr/local/var/postgres start
I killed the Postgress service running on port 5432 which is something you should never do, this can be done unknowingly. So running the above commands would remove the lock file, and now when you start a new Postgres server, it would create a new process for you
Not sure, but I think it's the lock file that another Postgres instance is working on.
Restart the PC, and then that instance will leave access to that lock file. And the Postgres server will start. It works for me.
I want to create a database with the data files and the wal on different filesystems. I want the wal on a separate server over NFS, to avoid a loss of data in case of a fs/disk crash.
Where is the wal written?
Can I force it to a different location than the default via the configuration?
I'm on 9.1 if that matters.
Thanks.
The WAL files are written to the directory pg_xlog inside of the data directory. Starting with Postgres 10, this directory was renamed to pg_wal
E.g. /var/lib/postgresql/10/main/pg_wal
See the manual for details:
http://www.postgresql.org/docs/9.1/static/wal-configuration.html
http://www.postgresql.org/docs/current/static/wal-configuration.html
If I'm not mistaken, this directory name can not be changed. But it can be a symbolic link that points to a different disk.
As a matter of fact this is actually recommended to tune WAL performance (See here: http://wiki.postgresql.org/wiki/Installation_and_Administration_Best_practices#WAL_Directory)
To Copy the WAL Directory to another file path/disk drive, follow these steps below:
Descriptive Steps
Turn off Postgres to protect against corruption
Copy WAL directory (by default on Ubuntu - /var/lib/postgresql/<version>/main/pg_wal) to new file path using rsync. It will preserve file/folder permissions and folder structure with the -a flag. You should leave off the training slash.
Verify the contents copied correctly
Rename pg_wal to pg_wal-backup in the Postgres data directory ($PG_DATA)
Create a symbolic link to the new path to pg_wal in the Postgres data directory ($PG_DATA) and update the permissions of the symbolic link to be the postgres user
Start Postgres and verify that you can connect to the database
Optionally, delete the pg_wal-backup directory in the Postgres data directory ($PG_DATA)
Matching Commands
sudo service postgresql stop
sudo rsync -av /var/lib/postgresql/12/main/pg_wal /<new_path>
ls -la /<new_path>
sudo mv /var/lib/postgresql/12/main/pg_wal /var/lib/postgresql/12/main/pg_wal-backup
sudo ln -s /<new_path> /var/lib/postgresql/12/main/pg_wal
sudo chown -h postgres:postgres /var/lib/postgresql/12/main/pg_wal
sudo service postgresql start && sudo service postgresql status
# Verify DB connection using your db credentials/information
psql -h localhost -U postgres -p 5432
rm -rf /var/lib/postgresql/12/main/pg_wal-backup