I have a small app, and I run backups manually using heroku pgbackups:capture on my dev machine.
I'd like to use the Heroku Scheduler send these backups to my own S3 bucket.
The thing is: pg_dump is not an available on Heroku boxes, and heroku pgbackups:capture is a local CLI command, also not available.
Is there another way to achieve this using Scheduler?
Related
I have recently started working on an existing Heroku environment.
How can I tell if there are database backups scheduled?
Assuming you are using Heroku Postgres, you can view backup schedules with the following command:
heroku pg:backups:schedules
You might have to provide the --app argument so Heroku knows which app you're interested in.
I'm using the Heroku CLI pg:pull command to migrate a Heroku Postgres connected database from one Heroku app (my-source-app) to another (my-target-app) - both of which are in my control.
First, I clear the database on the target application;
heroku pg:reset -a my-target-app
Then initiate the pg:pull
heroku pg:pull DATABASE $(heroku config:get DATABASE_URL -a my-target-app) --exclude-table-data='table5;table9' -a my-source-app
It seems to start working (transferring schema then data table-by-table), but is very slow. The original db is ~20GB; large, but not unreasonable. If I monitor the size of the target database (via the Heroku dashboard) it seems to fill at only about 35MB/minute.
My questions;
Is this command routing the data through my local machine or is it direct machine-to-machine?
Is there a way to "detach" from the process, and later monitor it (as I can with Heroku's run:detached command) so I don't need to remain online for the duration?
Is there a better approach for migrating the data here (such as creating a follower and switching it over to the new app somehow; I've tried this without success)
Answering the specific questions;
The data was not copied via my local machine while running the command.
In the end, I remained connected while the pg:pull operation completed; there doesn't seem to be a way to detach.
A similar feature (which copies everything across) is pg:copy - see docs - which was a viable alternative here.
We can create create backups from the Developer Portal for MongoDB, but I'm wondering if this is exposed in any other way through the CLI?
Also, how can I access the backup to inspect it on my machine for example?
There is a CLI tool for making backups: mongodump (in the mongodb-org-tools package)
See also https://docs.mongodb.com/manual/core/backups/
Yes there are two CLI keystrokes to backup and restore
Backup
Mongodump backupPath
Restore
Mongorestore -d databasename backupPath
Ex- mongodump C:/user/desktop/backup
mongorestore -d DB1 C:/user/desktop/backup/DB1
Turns out you can't do anything outside of the UI. As per their docs: https://docs.developer.swisscom.com/devguide-sc/services/backups.html
There is an API for Service Instance backups and restores (the interface you see in developer portal). See other question Are mongodb backups made automatically?. There is even a CLI plugin to automate Developer Portal backups. Try it with cf install-plugin -r CF-Community "Swisscom Application Cloud".
You can't download the backups to your local computer. For that you you need to use cf ssh tunnel and mongodump/mongorestore. There is a guide to migrate Swisscom MongoDB to other DB provider (like your own computer), see here for the exact commands.
hi I take backup of my heroku database using PGBackups but its provide me backup on there pre define places .now I wont to take that backup on my remote location folder on S3 or some other remote storage.
how can we do that periodically like every week or month it's own/automatically.
my app on Ruby on Rails help me to achieve this.
The latest backup is always available to you by entering heroku pgbackups:url. So, set up a cron job or the equivalent that fetches that URL once a week or once a day.
You could write a rake or Ruby script and call it with Heroku Scheduler (a free addon), or use a different remote machine to pull the backup, or do a shell script and:
curl -O `heroku pgbackups:url`
Here is a Gem that appears to do what you want: https://coderwall.com/p/w4wpvw
I'd like to have my master Postgres DB, which is is hosted on Heroku, replicate down to a slave DB on my laptop. Is this possible?
Heroku's documentation talks about both master and slave hosted within Heroku:
https://devcenter.heroku.com/articles/heroku-postgres-follower-databases
Someone else asked whether it's possible to have the master outside Heroku and a slave inside Heroku (it's not):
Follow external database from Heroku
I haven't seen an answer for the reverse -- having the master in Heroku and the slave outside.
Why do I want this? To speed up development. With my app running locally and the DB in the cloud, the round-trip is long so data access is slow. Most data access is read-only. If I could have a local slave, it would speed things up significantly.
Related: what if my laptop is disconnected for a while? Would that cause problems for the master?
You cannot make a follower (slave) outside of the Heroku network – followers need superuser access to create, which Heroku Postgres doesn't provide you, so you are limited to running a follower on Heroku.
If you want to pull down a copy locally for use/inspection, you can do so with pgbackups: https://devcenter.heroku.com/articles/heroku-postgres-import-export
I'd highly recommend the program Parity for this.
It copies down the last Heroku backup to your local machine with a nice command line interface:
development restore production
I'd rather just pull the production database's contents from Heroku every now and then.
$ heroku db:pull
You can speed that up with a rake task.
# lib/tasks/deployment.rake
namespace :production do
desc 'Pull the production DB to the local env'
task :pull_db do
puts 'Pulling PRODUCTION db to local...'
system 'heroku db:pull --remote MY_REMOTE_NAME --confirm MY_APP_NAME'
puts 'Pulled production db to local'
end
end
You can call rake production:pull_db to overwrite your local development database.