I do a backup of postgresql 9.4 database as per the postgres doc of WAL archiving.
After backup I create 2 records in DB.
Now when I try to restore the DB, the last 2 records which I created above is not coming up.
WAL archive steps:
cd /etc/postgresql/9.4/
mkdir archives
mkdir backups
chown postgres:postgres archives
chown postgres:postgres backups
cd /etc/postgresql/9.4/main/
echo 'max_wal_senders=1' >> postgresql.conf
echo 'wal_level=hot_standby' >> postgresql.conf
echo 'archive_mode=on' >> postgresql.conf
echo "archive_command='test ! -f /etc/postgresql/9.4/archives/%f && cp %p /etc/postgresql/9.4/archives/%f'" >> postgresql.conf
echo 'local replication postgres trust' >> pg_hba.conf
service postgresql restart
Backup steps:
cd /etc/postgresql/9.4/backups
rm -rf *
pg_basebackup --xlog -U postgres --format=t -D /etc/postgresql/9.4/backups/
Restore steps:
service postgresql stop
cd /var/lib/postgresql/9.4/
if [ ! -d "/var/lib/postgresql/9.4/tmp/" ]
then
mkdir tmp
else
rm -rf tmp
fi
mkdir tmp
mv /var/lib/postgresql/9.4/main/* /var/lib/postgresql/9.4/tmp/
cd /var/lib/postgresql/9.4/main/
rm -rf *
cd /etc/postgresql/9.4/backups
tar -xf base.tar -C /var/lib/postgresql/9.4/main/
cd /var/lib/postgresql/9.4/main/
FROMDIR="/etc/postgresql/9.4/archives/"
TODIR="/var/lib/postgresql/9.4/tmp/pg_xlog/"
if [ ! -d "$FROMDIR" ]
then
echo "Directory $FROMDIR does not exist!!"
exit
fi
if [ ! -d "$TODIR" ]
then
echo "Directory $TODIR does not exist!!"
exit
fi
cd $FROMDIR
for i in `find . -type f`
do
if [ ! -f $TODIR/$i ]
then
echo "copying file $i"
cp $i /var/lib/postgresql/9.4/main/pg_xlog/$i
fi
done
cd /var/lib/postgresql/9.4/main/pg_xlog/
chown -R postgres:postgres *
cd /var/lib/postgresql/9.4/main/
FILE="recovery.done"
if [ -f $FILE ]
then
mv $FILE recovery.conf
else
echo "restore_command = 'cp /etc/postgresql/9.4/archives/%f %p'" >> recovery.conf
fi
su postgres service postgresql start
exit
Changes appear in archive (/etc/postgresql/9.4/archives/ in your case) when the current WAL segment (usually 16 Mb) is filled up. Let me quote the documentation:
The archive_command is only invoked for completed WAL segments. Hence,
if your server generates little WAL traffic (or has slack periods
where it does so), there could be a long delay between the completion
of a transaction and its safe recording in archive storage. To limit
how old unarchived data can be, you can set archive_timeout to force
the server to switch to a new WAL segment file periodically. When this
parameter is greater than zero, the server will switch to a new
segment file whenever this many seconds have elapsed since the last
segment file switch, and there has been any database activity,
including a single checkpoint. (Increasing checkpoint_timeout will
reduce unnecessary checkpoints on an idle system.) Note that archived
files that are closed early due to a forced switch are still the same
length as completely full files. Therefore, it is unwise to use a very
short archive_timeout — it will bloat your archive storage.
If you just want to test the restore process, you can simply do select pg_switch_xlog(); after creating some records to force switch to a new WAL segment. Then verify that a new file appeared in the archive directory.
Also, you need not copy files from the archive directory to pg_xlog/. Restore_command will do it for you.
Related
I have a situation where I want to speed up deployment time by caching the git resources into a shared PVC.
This is the bash script I use for checkout the resource and save into a share PVC folder
#!/bin/bash
src="$1"
dir="$2"
echo "Check for existence of directory: $dir"
if [ -d "$dir" ]
then
echo "$dir found, no need to clone the git"
else
echo "$dir not found, clone $src into $dir"
mkdir -p $dir
chmod -R 777 $dir
git clone $src $dir
echo "cloned $dir"
fi
Given I have a Deployment with more than 1 pods and each of them have an initContainer. The problem with this approach is all initContainers will start almost at the same time.
They all check for the existence of the git resource directory. Let's say first deployment we dont have the git directory yet. Then it will create the directory, then clone the resource. Now, the second and third initContainers see that the directory is already there so they finish immediately.
Is there a way to make other initContainers wait for the first one to finish?
After reading the kubernetes documentation, I don't think it's supported by default
Edit 1:
The second solution I can think of is to deploy with 1 pod only, after a successful deployment, we will scale it out automatically. However I still don't know how to do this
I have found a workaround. The idea is to create a lock file, and write a script to wait until the lock file exists. In the initContainer, I prepare the script like this
#!/bin/bash
src="$1"
dir="$2"
echo "Check for existence of directory: $dir/src"
if [ -d "$dir/src" ]
then
echo "$dir/src found, check if .lock is exist"
until [ ! -f $dir/.lock ]
do
sleep 5
echo 'After 5 second, .lock is still there, I will check again'
done
echo "Finish clone in other init container, I can die now"
exit
else
echo "$dir not found, clone $src into $dir"
mkdir -p $dir/src
echo "create .lock, make my friends wait for me"
touch $dir/.lock
ls -la $dir
chmod -R 777 $dir
git clone $src $dir/src
echo "cloned $dir"
echo "remove .lock now"
rm $dir/.lock
fi
This kind of like a cheat, but it works. The script will make other initContainers wait until the .lock is remove. By then, the project is cloned already.
New to a certain Postgres implementation done by someone else and need help figuring out an issue.
We have the following archive command configured, If I understand correctly then the archive command copies WAL files to a mounted storage /mnt/database:
archive_command = 'if { egrep -q " /mnt/database .* rw," /proc/mounts ;} && { ! pgrep test -u postgres ;} ; then test ! -f /mnt/database/%f && cp %p /mnt/database/%f ; else exit 1; fi'
We then have a cron job to move corrupted WALs out of the way:
find /mnt/database -type f -regextype posix-extended -regex ".*[A-Z0-9]{24}$" -mmin +60 -size -16777216c -exec logger "Trimming Postgres WAL Logs" \; -exec find /var/lib/pgsql/9.6/data/pg_xlog/{} -type f \; -exec mv {} {}.incomplete \;
The issue we are having is the /mnt/database keeps filling up and we need to extend the disk every few days. Is that because we have excessive WAL writing or too many corrupted WAL files ?
The live WAL in 'pg_wal' cleans itself up automatically. Your WAL archive, '/mnt/database/' here, does not. It is up to you to arrange for that to get cleaned up based on your organization's retention policy.
If your policy is to keep WAL forever, then you need to get enough storage space to do that. If you have some other policy, you would need to understand what it is (and describe it to us, if you want our help implementing it)
Neither of the commands you show seem to be related to retention.
i am in no way a db admin, so please don't shoot me if i'm doing it completly wrong ...
I have to add some archiving to a productive postgres database (newest version in docker container) and trying to build some scripts to use with WAL.
The idea is to have a weekly script, that does a full backup to a new directory and then creates a symlink to this new directory that is used by the WAL script to write it's logs. Also the weekly script will delete old backups older than 30 days.
I would be very happy for any comments on this...
db settings
wal_level = replica
archive_mode = on
archive_command = '/archive/archive_wal.sh "%p" "%f"'
archive_timeout = 300
weekly script:
#!/bin/bash
#create base archive dir
#base_arch_dir=/tmp/archive/
base_arch_dir=/archive/
if [ ! -d "$base_arch_dir" ]; then
mkdir "$base_arch_dir"
chown -R postgres:postgres "$base_arch_dir"
fi
#create dir for week
dir="$base_arch_dir"$(date '+%Y_%m_%d__%H_%M_%S')
if [ ! -d "$dir" ]; then
mkdir "$dir"
chown -R postgres:postgres "$dir"
fi
#change/create the symlink
newdir="$base_arch_dir"wals
ln -fsn "$dir" "$newdir"
chown -R postgres:postgres "$newdir"
#do the base backup to the wals dir
if pg_basebackup -D "$newdir" -F tar -R -X fetch -z -Z 9 -U postgres; then
find "$base_arch_dir"* -type d -mtime +31|xargs rm -rf
fi
crchive script:
#!/bin/bash
set -e
arch_dir=/archive/wals
arch_log="$arch_dir/arch.log"
if [ ! -d "$arch_dir" ]; then
echo arch_dir '"$arch_dir"' does not exist >> "$arch_log"
exit -1
fi
#get the variables from postgres
p=$1
f=$2
if [ -f "$arch_dir"/$f.xz ]; then
echo wal file '"$arch_dir"/$f.xz' already exists
exit -1
fi
pxz -2 -z --keep -c $p > "$arch_dir"/$f.xz
Thank you in advance
It's not terribly difficult to put together your own archiving scripts, but there are a few things you need to keep track of, because when you need your backups you really need them. There are some packaged backup systems for PostgreSQL. You may find these two a good place to start, but others are available.
https://www.pgbarman.org/
https://pgbackrest.org/
I'm trying out this tutorial which is made for linux postgresql server. I followed the configuration set ups but I dont know how to translate this command in windows. Also, whats the equivalent of /etc folder and /var folder in windows installation of postgres?
echo Stopping PostgreSQL
sudo service postgresql stop
echo Cleaning up old cluster directory
sudo -u postgres rm -rf /var/lib/postgresql/9.2/main
echo Starting base backup as replicator
sudo -u postgres pg_basebackup -h 1.2.3.4 -D /var/lib/postgresql/9.2/main -U
replicator -v -P
echo Writing recovery.conf file
sudo -u postgres bash -c "cat > /var/lib/postgresql/9.2/main/recovery.conf
<<- _EOF1_
standby_mode = 'on'
primary_conninfo = 'host=1.2.3.4 port=5432 user=replicator
password=thepassword sslmode=require'
trigger_file = '/tmp/postgresql.trigger'
_EOF1_"
echo Startging PostgreSQL
sudo service postgresql start
The full tutorial is here
My answer assumes several things that you did not tell us:
the Postgres version is 9.6 (9.2 is pretty outdated and will be end-of-live in 6 months)
Postgres (64bit) was installed to: c:\Program Files\PostgreSQL\9.6
you are using the default service name postgresql-x64-9.6
the data directory is c:\ProgramData\Postgres\9.6
you already have a Postgres 9.6 installation on a different (64bit) Windows system that should be the master for this replicated instance
your current Windows user that you use to run this has the necessary privileges to stop and start the service.
So the script in your question would be roughly like the following:
rem add the Postgres bin directory to the path
rem so that references to the utilities do not
rem need to be full qualified
PATH=%PATH%;c:\Program Files\PostgreSQL\9.6\bin
rem Adjust to the real service name here
echo Stopping PostgreSQL
net stop postgresql-x64-9.6
echo Cleaning up old cluster directory
rem Adjust the data directory to your environment
rem You can check the current service configuration
rem to see where the data directory is
set DATADIR=c:\ProgramData\Postgres\9.6
rmdir /s /q "%DATADIR%"
echo Creating base backup
rem adjust IP address and username according to your master server
rem with 9.6 pg_basebackup can create the recovery.conf automatically
pg_basebackup -h 1.2.3.4 -D "%DATADIR%" -U replicator -v -P --write-recovery-conf -X stream
echo Starting PostgreSQL
net start postgresql-x64-9.6
Again: you have to adjust paths and names to your environment!
I am trying to backup postgres databases. I am running a cron job to do so. Issue is that postgres runs under user postgres and I dont think I can run a cron job under ubuntu user. I tried to create a cron job under postgres user and that also did not work. My script, if login as postgres user works just fine.
Here is my script
#!/bin/bash
# Location to place backups.
backup_dir="/home/postgres-backup/"
#String to append to the name of the backup files
backup_date=`date +%d-%m-%Y`
#Numbers of days you want to keep copie of your databases
number_of_days=30
databases=`psql -l -t | cut -d'|' -f1 | sed -e 's/ //g' -e '/^$/d'`
for i in $databases; do
if [ "$i" != "template0" ] && [ "$i" != "template1" ]; then
echo Dumping $i to $backup_dir$i\_$backup_date
pg_dump -Fc $i > $backup_dir$i\_$backup_date
fi
done
find $backup_dir -type f -prune -mtime +$number_of_days -exec rm -f {} \;
if I do
sudo su - postgres
I see
-rwx--x--x 1 postgres postgres 570 Jan 12 20:48 backup_all_db.sh
and when I do
./backup_all_db.sh
it gets backed up in /home/postgres-backup/
however with cronjob its not working, regardless if I add the cron job under postgres or under ubuntu.
here is my cronjob
0,30 * * * * /var/lib/pgsql/backup_all_db.sh 1> /dev/null 2> /home/cron.err
Will appreciate any help
Enable user to run cron jobs
If the /etc/cron.allow file exists, then users must be listed in it in order to be allowed to run the crontab command. If the /etc/cron.allow file does not exist but the /etc/cron.deny file does, then users must not be listed in the /etc/cron.deny file in order to run crontab.
In the case where neither file exists, the default on current Ubuntu (and Debian, but not some other Linux and UNIX systems) is to allow all users to run jobs with crontab.
Add cron jobs
Use this command to add a cron job for the current user:
crontab -e
Use this command to add a cron job for a specified user (permissions are required):
crontab -u <user> -e
Additional reading
man 5 crontab
Crontab in Ubuntu: https://help.ubuntu.com/community/CronHowto