I updated my DB plan on heroku quite some time ago, following this clear tutorial: https://devcenter.heroku.com/articles/upgrade-heroku-postgres-with-pgbackups
So now I have 2 DB running:
$ heroku pg:info
=== HEROKU_POSTGRESQL_NAVY_URL (DATABASE_URL)
Plan: Crane
Status: Available
Data Size: 26.1 MB
Tables: 52
PG Version: 9.2.6
Connections: 8
Fork/Follow: Available
Rollback: Unsupported
Created: 2013-11-04 09:42 UTC
Region: eu-west-1
Maintenance: not required
=== HEROKU_POSTGRESQL_ORANGE_URL
Plan: Dev
Status: available
Connections: 0
PG Version: 9.2.7
Created: 2013-08-13 20:05 UTC
Data Size: 11.8 MB
Tables: 49
Rows: 7725/10000 (In compliance, close to row limit) - refreshing
Fork/Follow: Unsupported
Rollback: Unsupported
Region: Europe
I keep receiving mails saying that I'm close to rate limit on HEROKU_POSTGRESQL_ORANGE_URL. I'd rather delete it, but I'd like to make sure I'm not going to loose any data. Heroku is not clear about it:
The original database will continue to run (and incur charges) even after the upgrade. If desired, remove it after the upgrade is successful.
But can I be 100% sure that all the data in the HEROKU_POSTGRESQL_ORANGE_URL is duplicated in the HEROKU_POSTGRESQL_NAVY_URL? Because if HEROKU_POSTGRESQL_ORANGE_URL were a follower of HEROKU_POSTGRESQL_NAVY_URL, its data should be as big as the first one.
So I just need a confirmation.
Thanks
It sounds to me like the upgrade dumped and reloaded the DB. So the new DB is a copy of the old one. If that's the case, it will contain all data from the old one at the time it was copied - but if you kept on adding new data to the old database, that data wouldn't appear in the new one.
I strongly recommend that before dropping the DB you:
Disable access to it except for pg_dump
Dump it with pg_dump (or use Heroku's tools to do that)
... and only then delete it.
That way, if you discover you've made a mistake, you have a dump to restore.
Related
Can we take base backup from postgresql old version db to new version db.
For Example I have two DB servers. One server is having old version Postgres 9.x and another one is running Postgres 13.
If I want to take base backup from postgres 9.x for Postgres 13, can we do that directly using the pg_basebackup command in Postgres 13 DB server.
Is there any other way to complete the above task in Postgres.
I have a plan below to complete the above task if the base backup does not work for lower to higher versions. But it is a slow process.
DB A has a lower version and DB B has a higher version.
So to take base backup, updating the A DB version same as B DB is the only option.
Once the DB A updated successfully, Then start the base backup from A DB with updated Postgres version to the B DB.
We can refer to the below site for the DB update process.
https://www.migops.com/blog/upgrading-postgresql-9-6-to-postgresql-13/
The DB update process will take time to complete. In that time the write process will not work on the primary DB. All the write process will be affected because of this. If any issue happens in the update process, almost everything will be a mess.
I will suggest the old version DB to the new version db base backup process to complete this activity.
Do you guys have any suggestions?
You are mixing up physical file-system level backup (pg_basebackup) and logical backup, also known as dump or export (pg_dump). You can only use the latter for upgrading. A pg_basebackup can only be restored to the same major PostgreSQL version.
I'm on the Heroku Hobby Tier Postgres. After a redeploy I got
psql: FATAL: out of shared memory
HINT: You might need to increase max_locks_per_transaction.
when trying to access my psql database.
pg:info shows
Plan: Hobby-basic
Status: Unavailable, operator notified
Connections: 2/20
PG Version: 10.6
Created: 2018-07-02 18:38 UTC
Data Size: 1.38 GB
Tables: 78
Rows: 4643980/10000000 (In compliance)
Fork/Follow: Unsupported
Rollback: Unsupported
Continuous Protection: Off
is there something I can do to resolve this myself from Heroku?
Heroku routinely performs maintenance on the Heroku Postgres resource if you are on on the hobby tier. That is what happened in this situation. They are not obligated to notify you.
It's also important to note that this should only cause a minimal amount of downtime:
Maintenance windows are 4 hours long. Heroku attempts to begin the maintenance as close to the beginning of the specified window as possible. The duration of the maintenance can vary, but usually, your database is only offline for 10 – 60 seconds. Source
If it is longer, this is likely not your problem.
I have been logging bitcoin order books and trades for the past 20 days in mongodb. 1 database [bitmicro], 1 collection for trades, 1 collection for books.
now suddently all the data is gone and the logger has started from scratch since yesterday.
The mongo log doesnt show any entries since October. And a new database appeared called Warning exactly when the data loss happened
> show dbs
Warning 0.078GB
bitmicro 0.453GB
local 0.078GB
After some reading it might be the case, that the filesize of the collection became too big and ubuntu deleted it?
Since the log /var/log/mongodb/mongod.log doesnt show any entries since starting the server, how can I find out what happened?
the existence of the Warning database implies that your database was accessed by a malicious 3rd party, and your database is exposed to the internet without any authentication.
https://docs.mongodb.com/manual/administration/security-checklist/
I have three different databases for my different environments (hsprd, hstst,hstrn). hsprd is my production environment with live data.
Every so often, a request comes through to restore production data to hstrn or hstst. I typically run this command (after stopping, then dropping the db):
db2 restore db hsprd taken at 20140331180002 to /dbs into hstrn newlogpath /dbs/log/hstrn without rolling forward;
When running this, I receive this message:
SQL2537N Roll-forward is required following the Restore.
Could someone advise how to fix this?
Thanks.
edit: My backups are here:
(/home/dbtmp/backups)> ll
total 22791416
-rwxrwxr-x 1 hsprd cics 11669123072 Mar 31 18:03 HSPRD.0.hsprd.NODE0000.CATN0000.20140331180002.001
After restoring my database and omitting without rolling forward, I receive this message when trying to query the database:
SQL1117N A connection to or activation of database "HSTRN" cannot be made
because of ROLL-FORWARD PENDING. SQLSTATE=57019
When I try to rollforward, with this command, I receive this response:
(/home/dbtmp/backups)> db2 rollforward db hstrn to end of backup and complete;
SQL4970N Roll-forward recovery on database "HSTRN" cannot reach the specified
stop point (end-of-log or point-in-time) on database partition(s) "0".
Roll-forward recovery processing has halted on log file "S0006353.LOG".
The first error suggests that you are restoring an online backup, which must be rolled forward. Alternatively, use an offline backup image, then you can include the without rolling forward option.
The second error means that you need to issue the ROLLFORWARD command before you can use the database restored from an online backup.
Finally the third error means that the ROLLFORWARD command is unable to find the logs required for it to succeed. Assuming the logs are included in the backup image, you'll need to specify the LOGTARGET option on the RESTORE command to extract them, presumably to the NEWLOGPATH location.
we are using Heroku Postgres with Ruby on Rails 3.2.
A few days before, we deleted important data by mistake using 'heroku run db:load' with misconfigured data.yml, that is, drop tables and the recreate tables with almost no data.
Backup data is only available 2 weeeks before, so we lost data of 2 weeks.
So We need to recover not by PG Backup/pg_dump but by postgresql's system data files.
I think, the only way to recover data is to restore data from xlog or archive file, but of course we don't have permission to be Super User/Replication Role to copy postgres database on heroku (or Amazon EC2) to local server.
Is there anyone who confronted such a case and resolved the problem?
Your only option is the backups provided by the PgBackups service (if you had that running). If not, Heroku support might have more options available.
At a minimum, you will have some data loss, but you can guarantee you won't do it again ;)