I have to do a point in time recovery in postgres should i have all the wal logs that are generated or is it fine to have few recent wal log files ? I tried recovery but I'm not sure if all the files are recovered
Related
I am trying to perform a Point-In-Time Recovery using the WAL_ARCHIVE process. The archive command is added to the postgresql.conf file and I can see the WAL being archived in the backup-archive directory. When I try to start the service I get PANIC: could not locate a valid checkpoint record
I am using the below step-by-step process.
low level api basebackup
SELECT pg_start_backup('label', true, false);
copying the data directory of my cluster
tar -zcvpf basebkPostgres20230110New.tgz /PostgreSQL/13/data
closing my basebackup
SELECT * FROM pg_stop_backup(false, true);
Stopping the postgres service
Removing the current's cluster data directory
Restoring the backed up data directory
Removing the contents of the pg_wal directory
Setting the restore_command in the postgresql.conf file
Starting the postgres service
You forgot the backup_label file and recovery.signal. You have to capture the result of pg_stop_backup (or pg_backup_stop from v15 on) and create backup_label from the contents. That file has to be in the restored data directory. Also, you have to create recovery.signal in the data directory, so that PostgreSQL starts in archive recovery mode and reads your restore_command.
Without restore_command, PostgreSQL uses the WAL in pg_wal, which is empty. Without backup_label, PostgreSQL thinks that it can recover from the checkpoint indicated by the control file pg_control. Even if that worked, the result would be a corrupted database, since you have to recover from the start of the backup.
recovery.signal is documented here (step 7), and backup_label is documented here (step 4).
As per replication slot definition, it is a feature in PostgreSQL that ensure that the master server will retain the WAL logs that are needed by the replicas even when they are disconnected from server.
Is there any effect if I run pg_archivecleanup command every 15th day of month to free my storage. Does it has any effect on replication slot, since it is tracing WAL file which is required by standby server?
Because I run pg_archivecleanup removing WAL file from last checkpoint but I am not sure whether it is removing WAl file that is required for other replica.
If not removing then how it is actually tracing it?
I am looking for explanation from experts.
When you run pg_archivecleanup, PostgreSQL will delete all WAL segments that are older than the WAL segment you specify as argument. This will ignore replication slots, you you may end up removing WAL segments that may still be needed by standby servers to catch up (if they do that using restore_command).
Note that this normally not a problem, because pg_archivecleanup deletes WAL segments from the archive, while replication slots deal with WAL segments on the primary server (in the pg_wal directory), which are not affected by pg_archivecleanup. Now since the standby consumes WAL directly from the primary (as specified in primary_conninfo), it does not have to rely on the WAL archives.
Can we perform point in time recovery more than one time using same recovery.conf file, because the recovery.conf file changes to recovery.done after a one restoration of wal file.
What if I want to do another wal file restoration at a different time using same recovery.conf file.I can't do that? Or do I have to again do a pg_basebackup and then create a new recovery file each time in my data directory of Postgres to retore next wal file
Once recovery is done, you cannot go back.
You have to restore the backup again and start from scratch.
The only alternative is using pg_rewind, but that can only reset a cluster to the state of another cluster (and you probably don't have that other cluster).
I try to configure backuping database in postgresql with pg_basebackup and WAL logs.
For now I created full backup once a week and want to backup wal logs too. But, as I understand, posgresql writes them all the time. So, how can I copy them and be shure that they are not corrupted?
Thanks
You set archive_command to a shell command that copies the WAL file to a safe archive location, so that burden is mostly on you.
When PostgreSQL runs archive_command, it assumes that the WAL file is not corrupted. Only a PostgreSQL bug or a bug in the storage system could cause a corrupted WAL segment.
There is no better protection against PostgreSQL bugs than always running the latest bugfix release, and you can invest in storage hardware that will at least detect failure.
You can also write your archive_command with a certain amount of paranoia, e.g. by comparing the md5sum of the WAL segment and its archive copy.
Another idea is to write two copies of the WAL file to different storage systems.
I have set up a Postgres Hot Standby server by Streaming Replication. But My Standby server is asking for an old wal archive log which is currently not in Master's pg_xlog directory. But the file exists in the wal archive backup directory.
How can I configure Standby to read this file from backup directory? Or any way to manually copy this file to Standby Server ?
Any help will be appreciated.
You would have to add a restore_command to recovery.conf that can restore files from the WAL archive.
Then restart the standby, and it should be able to recover.
When the standby cannot get the required WAL via streaming replication, it tries restore_command. When that fails, it tries streaming replication again, and so on in an endless loop.