I have a PostgreSQL DB hosted in heroku and I want to get access from other applications which aren't hosted in heroku, but I saw in the DB settings that the credentials are not permanent.
How can I get access from other application always having updated credentials?
Heroku recommends using the Heroku CLI to fetch fresh credentials every time you run your external application:
Always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app) your_process
This way, you ensure your process or application always has correct database credentials.
In this example, your_process will see an environment variable called DATABASE_URL that is set to the same value as the DATABASE_URL config far on the Heroku app called your-app.
Since you are using Python, here is one way to access that value:
import os
database_url = os.getenv("DATABASE_URL", default="some_default_for_local_development")
Related
Originally when I provisioned a database on Heroku (free one), then I my app got a new environment variable called DATABASE_URL that points to a newly created database and my app uses this one.
Today when I provisioned a new database on Heroku (paid) via the UI, I got a new environment variable called HEROKU_POSTGRESQL_MAUVE_URL that points to new database.
How do I point my Heroku app to this newly provisioned database HEROKU_POSTGRESQL_MAUVE_URL?
I expected my database URL to be swapped with HEROKU_POSTGRESQL_MAUVE_URL but that didn't happen.
This happens when your app has multiple databases:
As part of the provisioning process, a DATABASE_URL config var is added to your app’s configuration. DATABASE_URL contains the URL your app uses to access the database. If your app already has a Heroku Postgres database and you’ve provisioned another one, this config var’s name instead has the format HEROKU_POSTGRESQL_<COLOR>_URL (for example, HEROKU_POSTGRESQL_YELLOW_URL
You can promote your new database to make it the primary database:
pg:promote updates the value of the DATABASE_URL config var with the newly promoted database’s connection string. It also creates an alternate attachment for the old primary database, assigned with a new HEROKU_POSTGRESQL_<color>_URL config var. The promotion process triggers a release and restarts the app.
For example:
heroku pg:promote HEROKU_POSTGRESQL_MAUVE_URL -a your-app
You should continue to use the DATABASE_URL environment variable to connect to your database.
I developed a FastAPI app in a Virtual Environment using an SQLite database but deployed it on Heroku with a PostgresSQL database on Heroku as suggested in the tutorial. Although it worked on my PC, adding PostegresSQL as an addon & replacing the value of SQLALCHEMY_DATABASE_URL in the database.py broke everything. Note that I've properly frozen the dependencies on the requirements.txt file. Yet I can't figure out what went wrong.
For further clarification, I've pushed my code to GitHub & it can be accessed at this repository - Self_calculation.
If you are using Postgres Addon on Heroku, probably your solution is simple.
Use os.environ to get the connection parameters , don't try to connect directly, it's Heroku's recommended solution from Heroku Postgres
import os
DATABASE_URL = os.environ.get('DATABASE_URL')
I just need to run this:
1)heroku git:remote -a my_heroku_app_name
2)heroku logs --tail
after that I could see my problem.
In my case I forgot to change postgres url in alembic.ini file
I'm trying to import a local Postgresql database to Heroku and I'm following these steps https://devcenter.heroku.com/articles/heroku-postgres-import-export#import-to-heroku-postgres.
I have successfully:
created a dump
uploaded it to an S3 Bucket
created from AWS CLI a signed link
ran the command heroku pg:backups:restore '<SIGNED URL>' DATABASE_URL (adding -a with my app name).
The process to restore a backup starts correctly but then exits with this code:
! An error occurred and the backup did not finish.
!
! Could not initialize transfer
!
! Run heroku pg:backups:info r011 for more details.
Opening the log shows:
Database: BACKUP
Finished at: 2020-01-09 18:49:30 +0000
Status: Failed
Type: Manual
Backup Size: 0.00B (0% compression)
=== Backup Logs
2020-01-09 18:49:30 +0000 Could not initialize transfer
I've tried:
re-uploading the file to the bucket,
generating a new signed link,
putting the app in maintenance mode,
I've created a user in my IAM management service with full S3 access and saved the credentials in the app environment as from https://devcenter.heroku.com/articles/s3
Not sure where to go from here but would appreciate any help. (I'm on the hobby plan therefore I can't ask Heroku's support for help)
Edit: I also tried:
deleting and recreating the S3 Bucket
installing version 1 of the AWS CLI to see if by chance the structure of a presigned link had changed
Edit 2: Since I could not find a solution I've opted to migrate the hosting entirely on AWS for the moment
Make sure that your credentials on your machine that are stored in ~/.aws/ the default value is set to the credentials you created for your heroku configs. Then also make sure the signed url is created with those credentials and configs. I had to set my default credentials to the credentials I put in my heroku configs. Then I also had to set my default region in ~/.aws/config to match the bucket location. Should work after that.
Here are some instructions if you are on mac or linux.
Sorry Windows people. I would assume it is something similar.
Create new access id and key in IAM on AWS
Set heroku configs to use those credentials heroku config:set AWS_ACCESS_KEY_ID=xxx AWS_SECRET_ACCESS_KEY=yyy
Optional (You may have to set the bucket name in heroku config too)
On your machine set your credentials you just created to the default in ~/.aws/credentials
On your machine set your default region that corresponds to your bucket in ~/.aws/config
Create signed URL aws s3 presign s3://your-bucket-address/your-object
Run restore heroku pg:backups:restore '<SIGNED URL>' DATABASE_URL
Had the exact same error and made these 2 adjustments. In the S3 console click on the file you want to use for the backup. You should see the name fo your file followed by 4 tabs. In the General information tab, do the following:
Click on Make public to make the file available for download.
Get the URL for that object where it says URL of object
(should be something like https://mybucket.s3.amazonaws.com/my.file, you can test if it works by pasting that url in a new Chrome tab and hitting that url. That should trigger the download of your file)
Once the previous check is working you can proceed to
heroku pg:backups restore 'https://mybucket.s3.amazonaws.com/my.file' DATABASE_URL
I ran into the same issue and discovered the issue was that I had my bucket's region set as us-east rather than us-east-1.
I built and deployed a Node.js Postgres app to Heroku and can not get to any of my endpoints via the Heroku site except the root GET route. Curiously, when I run Heroku local web ALL my endpoints behave exactly as they should. I can successfully perform CRUD on the app running via Heroku local web. However, when I try, for instance, to create a user using the Heroku URL, it returns an empty error message. Yet, when I check the associated database I find that the user was indeed created. Other than returning an empty error message when I try to either create a user or sign it, the app correctly responds with the different errors I programmed. For example, when I tweak my login details or try to register the same user I earlier tried to register it correctly says the user already exists!. Still, when I try to log in that same existing user I get a blank error message. Note that I created both the Heroku PostgreSQL database and my local PostgreSQL database from exactly the same queries. Please, can you help me through this bottleneck? I am using Postman to test my APIs.
Test to sign in user on Heroku app running on the local machine: success!
Same exact test with Heroku URL: cryptic error.
Ok, so after a lot of researching and fiddling around I discovered the solution. I did not add keys from my .env file to Heroku as config vars found under the settings tab of the Heroku User dashboard. Manually adding my environment variables resolved the matter. Now my app is working both on my local machine and via the Heroku URL.
I've decided to save time on the ops side of things and move to Heroku. I'm planning to have a production dyno on Heroku with a postgres database AND another dyno that reads from the same database.
However when I opened the settings of postgres, it said:
Database Credentials
Get credentials for manual connections to this database.
Please note that these credentials are not permanent.
Heroku rotates credentials periodically and updates applications where this database is attached.
What's a good way to go about this?
From Heroku Documentation,
Credentials
Do not copy and paste database credentials to a separate environment or into your application’s code. The database URL is managed by Heroku and will change under some circumstances such as:
User initiated database credential rotations using heroku pg:credentials:rotate.
Catastrophic hardware failure leading to Heroku Postgres staff recovering your database on new hardware.
Automated failover events on HA enabled plans.
It is best practice to always fetch the database URL config var from the corresponding Heroku app when your application starts. For example, you may follow 12Factor application configuration principles by using the Heroku CLI and invoke your process like so:
DATABASE_URL=$(heroku config:get DATABASE_URL -a your-app-name) your_process
This way, you ensure your process or application always has correct database credentials.
May be attaching the same database to two heroku-apps will better suit you. In this way, pg creds will be auto-managed by heroku.
I am also using this technique. I have one client-facing app and another operation-app sharing the same database instance.
You can either do this using UI or via CLI
see Share database between 2 apps in Heroku