I'd like to have my master Postgres DB, which is is hosted on Heroku, replicate down to a slave DB on my laptop. Is this possible?
Heroku's documentation talks about both master and slave hosted within Heroku:
https://devcenter.heroku.com/articles/heroku-postgres-follower-databases
Someone else asked whether it's possible to have the master outside Heroku and a slave inside Heroku (it's not):
Follow external database from Heroku
I haven't seen an answer for the reverse -- having the master in Heroku and the slave outside.
Why do I want this? To speed up development. With my app running locally and the DB in the cloud, the round-trip is long so data access is slow. Most data access is read-only. If I could have a local slave, it would speed things up significantly.
Related: what if my laptop is disconnected for a while? Would that cause problems for the master?
You cannot make a follower (slave) outside of the Heroku network – followers need superuser access to create, which Heroku Postgres doesn't provide you, so you are limited to running a follower on Heroku.
If you want to pull down a copy locally for use/inspection, you can do so with pgbackups: https://devcenter.heroku.com/articles/heroku-postgres-import-export
I'd highly recommend the program Parity for this.
It copies down the last Heroku backup to your local machine with a nice command line interface:
development restore production
I'd rather just pull the production database's contents from Heroku every now and then.
$ heroku db:pull
You can speed that up with a rake task.
# lib/tasks/deployment.rake
namespace :production do
desc 'Pull the production DB to the local env'
task :pull_db do
puts 'Pulling PRODUCTION db to local...'
system 'heroku db:pull --remote MY_REMOTE_NAME --confirm MY_APP_NAME'
puts 'Pulled production db to local'
end
end
You can call rake production:pull_db to overwrite your local development database.
Related
I have recently started working on an existing Heroku environment.
How can I tell if there are database backups scheduled?
Assuming you are using Heroku Postgres, you can view backup schedules with the following command:
heroku pg:backups:schedules
You might have to provide the --app argument so Heroku knows which app you're interested in.
I'm using the Heroku CLI pg:pull command to migrate a Heroku Postgres connected database from one Heroku app (my-source-app) to another (my-target-app) - both of which are in my control.
First, I clear the database on the target application;
heroku pg:reset -a my-target-app
Then initiate the pg:pull
heroku pg:pull DATABASE $(heroku config:get DATABASE_URL -a my-target-app) --exclude-table-data='table5;table9' -a my-source-app
It seems to start working (transferring schema then data table-by-table), but is very slow. The original db is ~20GB; large, but not unreasonable. If I monitor the size of the target database (via the Heroku dashboard) it seems to fill at only about 35MB/minute.
My questions;
Is this command routing the data through my local machine or is it direct machine-to-machine?
Is there a way to "detach" from the process, and later monitor it (as I can with Heroku's run:detached command) so I don't need to remain online for the duration?
Is there a better approach for migrating the data here (such as creating a follower and switching it over to the new app somehow; I've tried this without success)
Answering the specific questions;
The data was not copied via my local machine while running the command.
In the end, I remained connected while the pg:pull operation completed; there doesn't seem to be a way to detach.
A similar feature (which copies everything across) is pg:copy - see docs - which was a viable alternative here.
we are using Heroku as our Cloud platform service
Now I have 3 apps 1 as production and the other two as staging, development.
I made some changed in staging app and I need to copy the database (Postgres) from production to staging.
I found some guidelines in Heroku documentation but it's not clear.
can any one share his experience with me?
the database reach 25 GB Now
As always, it depends. I see two ways:
heroku backups / restore: just use the last backup you have, or create a new one, then restore it to the staging server.
forking the database: on the staging app, delete the postgres addon, and recreate it as a fork (aka copy) of the production server. The --fast option makes it even faster when the data doesn't have to be 100% up-to-date.
Since 25 GB is already over the recommended size for normal Heroku backups (20 GB), I would recommend going for option 2.
(though we are using Heroku backups for a 200GB database without any issues).
My understanding is that Heroku Postgres runs on top of AWS. Is it possible to configure which datacenter your database is running in? I'm also wondering if the database files are stored on an encrypted filesystem.
Yes, Heroku runs on AWS. But you are not able to specify which datacenter to run your database. For encryption look at http://www.postgresql.org/docs/current/static/pgcrypto.html.
Heroku runs out of Amazon US-East - once you've add a postgres db to your app heroku config will give you the database connection URL which you would be able to tracert on to see where it is
Using this article: Heroku dev environments
I successfully made two separate Heroku apps, one test, and one prod. This each have their own remotes on my local development box. But now I don't know how to separate the postgres tables of my now distinct development and prod applications.
This is the command I use to create a postgres table with my Heroku app:
heroku addons:add heroku-postgresql
But then I lost access to my original postgres DB. I would like this setup so I don't inadvertently hose my prod DB from my development branch.
TL;DR: How do i keep distinct postgres databases with a single Heroku application that has multiple environments?
EDIT #1: I found that I can call:
heroku addons:add heroku-postgresql:dev
multiple times to create several dev databases. What is the best practice for pointing my dev vs. prod apps to each database without having to hard code my database links like this:
//this function connects to the Heroku postgres db
function pg_connection_string(){
return "dbname=dcs1k5588jbfad host=ec2-54-243-224-162.compute-1.amazonaws.com port=5432 user=###user_name### password=######### sslmode=require";
}
Is there a Heroku-fung-shui of swapping database pointers?
You can set the environment variable or heroku config variable DATABASE_URL to anything you wish. This is what Heroku applications expect to be the default database for that application.
If you're using a production Database you can Fork from your production app to a test app to get a copy of the data as well: https://devcenter.heroku.com/articles/heroku-postgres-fork