I've decided to save time on the ops side of things and move to Heroku. I'm planning to have a production dyno on Heroku with a postgres database AND another dyno that reads from the same database.
However when I opened the settings of postgres, it said:
Database Credentials
Get credentials for manual connections to this database.
Please note that these credentials are not permanent.
Heroku rotates credentials periodically and updates applications where this database is attached.
What's a good way to go about this?
From Heroku Documentation,
Credentials
Do not copy and paste database credentials to a separate environment or into your application’s code. The database URL is managed by Heroku and will change under some circumstances such as:
User initiated database credential rotations using heroku pg:credentials:rotate.
Catastrophic hardware failure leading to Heroku Postgres staff recovering your database on new hardware.
Automated failover events on HA enabled plans.
It is best practice to always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app-name) your_process
This way, you ensure your process or application always has correct database credentials.
May be attaching the same database to two heroku-apps will better suit you. In this way, pg creds will be auto-managed by heroku.
I am also using this technique. I have one client-facing app and another operation-app sharing the same database instance.
You can either do this using UI or via CLI
see Share database between 2 apps in Heroku
Related
I have a PostgreSQL DB hosted in heroku and I want to get access from other applications which aren't hosted in heroku, but I saw in the DB settings that the credentials are not permanent.
How can I get access from other application always having updated credentials?
Heroku recommends using the Heroku CLI to fetch fresh credentials every time you run your external application:
Always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app) your_process
This way, you ensure your process or application always has correct database credentials.
In this example, your_process will see an environment variable called DATABASE_URL that is set to the same value as the DATABASE_URL config far on the Heroku app called your-app.
Since you are using Python, here is one way to access that value:
import os
database_url = os.getenv("DATABASE_URL", default="some_default_for_local_development")
I'm using the Heroku CLI pg:pull command to migrate a Heroku Postgres connected database from one Heroku app (my-source-app) to another (my-target-app) - both of which are in my control.
First, I clear the database on the target application;
heroku pg:reset -a my-target-app
Then initiate the pg:pull
heroku pg:pull DATABASE $(heroku config:get DATABASE_URL -a my-target-app) --exclude-table-data='table5;table9' -a my-source-app
It seems to start working (transferring schema then data table-by-table), but is very slow. The original db is ~20GB; large, but not unreasonable. If I monitor the size of the target database (via the Heroku dashboard) it seems to fill at only about 35MB/minute.
My questions;
Is this command routing the data through my local machine or is it direct machine-to-machine?
Is there a way to "detach" from the process, and later monitor it (as I can with Heroku's run:detached command) so I don't need to remain online for the duration?
Is there a better approach for migrating the data here (such as creating a follower and switching it over to the new app somehow; I've tried this without success)
Answering the specific questions;
The data was not copied via my local machine while running the command.
In the end, I remained connected while the pg:pull operation completed; there doesn't seem to be a way to detach.
A similar feature (which copies everything across) is pg:copy - see docs - which was a viable alternative here.
I have a rails app on heroku that is using a Postgre database. My database has > 40 tables and > 10,000 rows. I would like to delete a lot of data, but it would be much easier if I was able to view and interact with it in a GUI table. I can access my data in rails console, but it's taking too long.
pgweb is a great cross-platform GUI, and it's easy to connect to your Heroku Postgres when launching from the command line.
I installed via Homebrew on a Mac (brew install pgweb), but instructions for other platforms are listed on the site. Here's how I launch pgweb connected to a Heroku Postgres DB:
heroku config:get DATABASE_URL | xargs pgweb --url
And if you want to connect to your localhost:
pgweb --host localhost
I'm a little late here, but this may help someone else who stumbles across this thread...
If you go to your Heroku app's dashboard (through the website) > settings > "Reveal Config Vars" > DATABASE_URL, and paste that URL into the browser.
I use TablePlus for database management, when I paste the link into the browser it asks if it can open TablePlus and then I can edit my production database in real time just like I would in development.
I'm not sure what pasting the URL into the browser will do if you don't have TablePlus. I assume it will request to open any other SQL management app you might have.
As slumdog wrote in the comment to your question, you can use pgAdmin, which comes with your local Postgres installation.
This article explains how to connect your remote heroku db with pgAdmin, using heroku credentials: https://medium.com/#vapurrmaid/getting-started-with-heroku-postgres-and-pgadmin-run-on-part-2-90d9499ed8fb
From the article:
"pgAdmin is a GUI for postgresql databases that can be used to access and modify databases that not only exist locally, but also remotely. For a fresh install of pgAdmin, the dashboard likely contains only one server. This is your local server...
We have to configure a new remote server with its credentials.
right click server(s) > create > server …
Fill out the following:
Name: This is solely for you. Name it whatever you want, I chose ‘Heroku-Run — On’
Under the connection tab: hostname/address. If you go back to your datastores ‘reveal credentials’, this is the host credential. It should look like --**...amazonaws.com
Keep the port at 5432, unless your credentials list otherwise
Maintenance database — this is the database field in the credentials
Username — this is the user field in the credentials
Password — the password field in the credentials. I highly advise checking save password so that you don’t have to copypasta this every time you want to connect.
In the SSL tab, mark SSL mode as require
At this point, if we were to hit ‘save’ (please don’t), something very strange would happen. You’d see hundreds if not thousands of databases appear in pgAdmin. This has to do with how Heroku configures their servers. You’ll still only have access to your specific database, not those of others. In order to avoid parsing so many databases, we have to white list only those databases we care about.
go to the Advanced tab and under db restriction copy the database name (it’s the same value as the Maintenance database field filled earlier)."
Article contains other usefull guidelines and screenshots.
Try GUI of DBWeaver.
https://dbeaver.io/
Download it, after that you can connect your heroku postgres using Database Credentials data.
You can use Heroku's hosted DB viewer on the Overview pane of your dashboard:
Create and click the Dataclip:
Dataclip GUI is fairly easy to use, we can type and customize SQL queries at the top etc.
My understanding is that Heroku Postgres runs on top of AWS. Is it possible to configure which datacenter your database is running in? I'm also wondering if the database files are stored on an encrypted filesystem.
Yes, Heroku runs on AWS. But you are not able to specify which datacenter to run your database. For encryption look at http://www.postgresql.org/docs/current/static/pgcrypto.html.
Heroku runs out of Amazon US-East - once you've add a postgres db to your app heroku config will give you the database connection URL which you would be able to tracert on to see where it is
Can I connect to a Heroku Postgres database via an web/application without the risk of dropping a table?
I'm building a Heroku application for a third party which uses Heroku Postgres for the backend. The third party are very security sensitive so I'm looking at applying "Layered security" throughout the application. So for example checking for SQL injection attacks at the web/application layer. Applying a "Layered security" approach I should also secure the database in case a potential SQL injection attack is missed, which might drop a database table.
In other systems I have built there would be a minimum of two users in the database. Firstly the database administrator who creates/drops tables, index, triggers, etc and the application user who would run with less privileges than the database administrator who could only insert and update records for example.
Within the Heroku Postgres setup there doesn't appear to be a way to create another user with less privileges (without the “drop table” option). So the application must connect with the default Heroku Postgres user and therefore the risk of a “drop table” might exist.
I'm running the Heroku Postgres Crane add-on.
Has anyone come up against this or got any creative work arounds for this scenario?
With Heroku Postgres you do only have a single account to connect with. One option that does exist for this type of functionality is to create a follower on Heroku Postgres. A follower is asynchronously kept up to date (usually only a second or so behind) and is read only. This would allow you to grant access to the follower to those that need it while not providing them with the details for the leader db.