Our database on Heroku has grown to size which requies three pgbackups URLs to download. I know how to download this database backup to local computer, merge the parts together and do restore, but what to do if I want to copy database from one Heroku app to other?
If you want to copy from one application from another you should be able to fork your database across applications. You can do this with:
heroku addons:add heroku-postgresql:crane --fork HEROKU_POSTGRESQL_SOURCE_OF_OLD_DB_AS_URL --app newapphere
Related
I'm using the Heroku CLI pg:pull command to migrate a Heroku Postgres connected database from one Heroku app (my-source-app) to another (my-target-app) - both of which are in my control.
First, I clear the database on the target application;
heroku pg:reset -a my-target-app
Then initiate the pg:pull
heroku pg:pull DATABASE $(heroku config:get DATABASE_URL -a my-target-app) --exclude-table-data='table5;table9' -a my-source-app
It seems to start working (transferring schema then data table-by-table), but is very slow. The original db is ~20GB; large, but not unreasonable. If I monitor the size of the target database (via the Heroku dashboard) it seems to fill at only about 35MB/minute.
My questions;
Is this command routing the data through my local machine or is it direct machine-to-machine?
Is there a way to "detach" from the process, and later monitor it (as I can with Heroku's run:detached command) so I don't need to remain online for the duration?
Is there a better approach for migrating the data here (such as creating a follower and switching it over to the new app somehow; I've tried this without success)
Answering the specific questions;
The data was not copied via my local machine while running the command.
In the end, I remained connected while the pg:pull operation completed; there doesn't seem to be a way to detach.
A similar feature (which copies everything across) is pg:copy - see docs - which was a viable alternative here.
I'm currently in the process of switching my cloud server from Heroku to Digital Ocean. However is there a way to migrate the database from the heroku server to the digital ocean one? I use postgresql for my database
I hope you already got a solution, but in case you didn’t, I’ll provide a simple guide on how I did it. I am going to assume that you have already created a postgres database on digitalocean. Also you need navigate to your project directory and log in to heroku using the heroku cli. And, you need to have postgresql installed or a psql client. Installing postgresql would do it as it comes with psql.
Step 1: Create a backup and download the backup from heroku postgres
heroku pg:backups:capture --app <app_name>
heroku pg:backups:download --app <app_name>
The first command will create a backup of your database and the second command will download it to your current directory, its a .dump file. If you would like to read more, here is an article.
Step 2: Connect to your remote (digital ocean’s) database using psql
Before you can do this, you need to go and add your machine you are connecting from to the list of database’s list of trusted sources, If you don’t, you’ll get a Connection Timed Out error. That’s because the database’s firewall doesn’t allow you to connect to the database from your local machine or resource (for security reasons).
Step 3: Import the Database
pg_restore -d "postgresql://<database_username>:<database_password>#<host>:<port>/<database>?sslmode=require" --jobs 4 -c "/path/to/dump_file.dump"
This will import your database from your dump file. Just substitute the variables will your connection parameters that you get from your dashboard. If you would like to read more, here is another article for this step.
Another thing to make clear is, sometimes, you will see some harmless error messages when running this command, but it will push through anyway. To learn more about pg_restore read this article.
And that’s it, your database has been migrated. Now, can you confirm it worked?, well, as for me, I used pgAdmin to connect to the remote database and I saw the tables and data as expected.
Hope this helps anyone with the same problem :)
we are using Heroku as our Cloud platform service
Now I have 3 apps 1 as production and the other two as staging, development.
I made some changed in staging app and I need to copy the database (Postgres) from production to staging.
I found some guidelines in Heroku documentation but it's not clear.
can any one share his experience with me?
the database reach 25 GB Now
As always, it depends. I see two ways:
heroku backups / restore: just use the last backup you have, or create a new one, then restore it to the staging server.
forking the database: on the staging app, delete the postgres addon, and recreate it as a fork (aka copy) of the production server. The --fast option makes it even faster when the data doesn't have to be 100% up-to-date.
Since 25 GB is already over the recommended size for normal Heroku backups (20 GB), I would recommend going for option 2.
(though we are using Heroku backups for a 200GB database without any issues).
I am running scheduled backups on my Heroku application (rails 4). I downloaded the backup that ran last night and I'd like to view the contents of the file. When I open it in textedit, I can see the tables but then it's just a mix of characters and letters.
I have scheduled backups running every evening on my Heroku PG db so I just had to download the backup from the previous day. To download the backup I ran:
heroku pg:backups public-url --app foobar
I then visited the AWS S3 link provided by Heroku and downloaded the file onto my computer.
How would I go about either viewing the contents of the backup in terminal or any other GUI without restoring the backup into my Heroku application?
We're having trouble with the heroku fork command so are manually trying to create a staging environment. I tried creating a new database off of a backup from our prod db but the created db has no rows and is only 6.4MB. The actual backup is 15.7 GB.
I did this via the web console clicking "restore".
Whats the right way to do this?
From the command line, you want to do:
heroku pgbackups:restore DATABASE -a example-staging `heroku pgbackups:url -a example`
We use this command every few days, whenever we want the staging database to be replaced with the production database. This comes from: https://devcenter.heroku.com/articles/migrate-heroku-postgres-with-pgbackups#transfer-to-target-database