When I take a backup of a database (SQL Server), is there any way that I can include a scheduled job in the backup?
I have a database with stored procedures and a maintenance job that runs some of the stored procedures nightly.
I would like to achieve a minimal effort to schedule the job, when the .bak file is restored into a server, back as a database.
I don't have the quick button to click for your problem, but I think (not sitting in front of it right now) You can right click a job and get a script for its creation including scheduling specifics. I don't know how to include a job in a backup and how to restore it, though. I think restoring a job would require a script with the CREATE for the job to be run.
You can also back up the msdb database. The msdb database is where all the jobs live, it is one of the system databases, and then restore your database plus msdb.
Related
On rundeck backup guide, noted that is mandatory to stop rundeck to take full backup when using data file. Now, that guide don't show any secure method to backup full rundeck instance (rundeck server + database) when using MariaDB, PostgreSQL, or any supported database as a backend.
In a real production scenario, not seems to be possible to stop rdeck on a daily basis.
Can anybody share best pratices to take a hot full backup on rdeck installation without stop rdeck?
Is there any secure and supported way to achive a full consistent rdeck projects and jobs definitions and database on a daily basis ?
In this post, answer is not clear, because question don't describe what kinbd of backend are used.
The documentation suggests the instance shutdown because some execution could be active, and that means a potentially active transaction in the middle of the "hot backup process" which means a potential data corruption in your backup. Is the safest way to backup your database.
If you want to do a "hot" backup you can export your projects (with all content, including jobs) and keys.
I've got a dev and test database for a project, i.e. databases that I use to either run my project or run tests, locally. They're both in the same cluster ('instance' – I come from Redmond).
Note that my local cluster is different than the cluster that hosts the production database.
How should I configure those databases with respect to archiving the WAL files?
I'd like to be able to 'build' or 'rebuild' either of those databases by restoring from a base backup and running seed data scripts.
But how should I configure the databases or the cluster for archiving WAL files? I understand that I need them if I want to recover the database. I think that's unlikely (as I didn't even know about 'WAL' or their files, or that, presumably they're shared by all of the databases in the same cluster, which seems weird and scary coming from Microsoft SQL Server.)
In the event that I rebuild one of the databases, I should delete the WAL files since the base backup – how can I do that?
But I also don't want to have to worry about the size of the WAL files growing indefinitely. I don't want to be forced to rebuild just to save space. What can I do to prevent this?
My local cluster only contains a single dev and test database for my project, i.e. losing data from one of these databases is (or should be) no big deal. Even having to recreate the cluster itself, and the two databases, is fine and not an issue if it's even just easier than otherwise to restore the two databases to a 'working' condition for local development and testing.
In other words, I don't care about the data in either database. I will ensure – separate from WAL archiving – that I can restore either database to a state sufficient for my needs.
Also, I'd like to document (e.g. in code) how to configure my local cluster and the two databases so that other developers for the same project can use the same setup for their local clusters. These clusters are all distinct from the cluster that hosts the production database.
Rather than trying to manage your WAL files manually, it's generally recommended that you let a third-party app take care of that for you. There are several options, but pg_backrest is the most popular of the open-source offerings out there.
Each database instance writes its WAL stream, chopped in segments of 16MB.
Every other relational database does the same thing, even Microsoft SQL Server (the differences are in the name and organization of these files).
The WAL contains the physical information required to replay transactions. Imagine it as information like: "in file x, block 2734, change 24 bytes at offset 543 as follows: ..."
With a base backup and this information you can restore any given point in time in the life of the database since the end of the base backup.
Each PostgreSQL cluster writes its own "WAL stream". The files are named with long weird hexadecimal numbers that never repeat, so there is no danger that a later WAL segment of a cluster can conflict with an earlier WAL segment of the same cluster.
You have to make sure that WAL is archived to a different machine, otherwise the exercise is pretty useless. If you have several clusters on the same machine, make sure that you archive them to different directories (or locations in general), because the names of the WAL segments of different clusters will collide.
About retention: You want to keep around your backups for some time. Once you get rid of a base backup, you can also get rid of all WAL segments from before that base backup. There is the pg_archivecleanup executable that can help you get rid of all archived WAL segments older than a given base backup.
I'd like to be able to 'build' or 'rebuild' either of those databases by restoring from a base backup and running seed data scripts.
Where is the basebackup coming from? If you are restoring the PROD base backup and running the seed scripts over it, then you don't need WAL archiving at all on test/dev. But then what you get will be a clone of PROD, which means it will not have different databases for test and for dev in the same instance, since (presumably) PROD doesn't have that.
If the base backup is coming from someplace else, you will have to describe what it is. That will dictate your WAL needs.
Trying to run one instance with both test and dev on it seems like a false economy to me. Just run two instances.
Setting archive_mode=off will entirely disable a wal archive. There will still be "live" WAL files in the pg_wal or pg_xlog directory, but these get removed/recycled automatically after each checkpoint--you should not need to manage these, other than by controlling how often checkpoints take place (and making sure you don't have any replication slots hanging around). The WAL archive and the live WAL files are different things. The live WAL files are mandatory and are needed to automatically recover from something like a power failure. The WAL archive may be needed to manually recover from a hard-drive crash or the total destruction of your server, and probably isn't needed at all on dev/test.
I would like to recover (restore) a GCP Cloud Postgres SQL up to the last 15 minutes for DR purposed (RPO is 15 minutes). That means the database must be backed up (typically the transaction log) every 15 minutes. Is this possible in Cloud Postgres and if so, what's the process?
In addition I am concerned about someone or an application bug deleting data. This just happened to us on another system. Ideally, it would be very beneficial to restore the DB backup and the 15 minute incremental transaction log backups to another DB and pick and choose the data that needs to be recovered. Is this possible?
the database must be backed up (typically the transaction log) every
15 minutes.
You can not back up the transaction log, you only back up instance Data. To back up your instance every 15 minutes, the automatic backup would not be enough. Thus, you must use on-demand backup and trigger it every 15 minutes.
restore the DB backup and the 15 minute incremental transaction log
backups to another DB
Yes, you can restore an instance backup to another instance
pick and choose the data that needs to be recovered.
Not in an easy way. You could implement this in some way by not creating an on-demand back up but instead export a SQL dump file or a CSV file and then running your own process to obtain the data you need.
Having said that, and your reference to DR (Disaster Recovery), I would like to point out that Cloud SQL has Automatic Fail-Over Replicas called High Availability. Also, for replication you could use Read-Replicas
I have been using PG Backups add-on recently and everything has worked fine, however this morning the backup process triggered at 10:00 A.M. in the morning generating some blocks and timeouts in my application.
Is there a way to specify the schedule of the backups made with this add-on? I've been searching and haven't found anything specific.
Use Cron for Manual Backup Scheduling
Heroku gives you two types of backups: automated and user-initiated. Each plan has a different number of daily, weekly, and manual backups that are retained. You can't control when the automated backups occur with PG Backups Auto, but you can use cron to trigger a "manual" backup at any time.
For example:
# Trigger a "manual" backup every four hours.
0 */4 * * * source $HOME/database_credentials; heroku pgbackups:capture
See Creating a Backup for more information about using the pgbackups command.
No, there is no way to do it currently, aside from using an external process to fire the calls.
An email to support might reveal more.
While the original question is old, Heroku does have a schedule option for PGBackups now:
https://devcenter.heroku.com/articles/heroku-postgres-backups#scheduling-backups
We have a 5 node replication set up on our development server. We are looking for a way to allow developers to back up a subset of data in a mongo db and restore this to their local development enviroments.
We have looked into the clonedb and the mongodump utils, but both only allow for a backup/dump of the complete database. Due to the possible size of the database, we need an option that allows us to limit the data being backed up or restored.
Do any know of a util or way to achieve this?
I just stumbled upon this question again and decided to add a description of our backup strategy we opt in for:
Current back up strategy for our mongo db this server consist of 2 setups; backup via delayed passive secondarynode and daily backup using mongodump (takes journalling and oplog into play).
Besides our normal production nodes, we have setup another secondary node with a priority of 0 (this can either be on its own server or piggy backing off another mongo server but using a seperate port), hidden as true and a delay of 7200 seconds (2hours). This slave is there for "butter fingers", when some one accidentally drops a database or clears a collection, we have 2 hours before these changes replicate to this passive secondary. The passive secondary can NOT be used for READING or WRITING. It's role is simply a back up node. We also use this node for nightly backup to prevent unnecessary overhead on any of the other nodes.
The nightly backup is set to run every night at 23:00 via a cron tab. The command simply executes a script setup in /opt/auto-mongo-backup. This script can be found at https://github.com/jaconel/automongobackup (originally found it at https://github.com/micahwedemeyer/automongobackup). This script allows for a single nightly cron to cover weekly backups and monthly backups. Back ups are saved at /var/backups/mongodb.
Hope this helps some one out.