C:\Users\krishnava\Downloads\git>heroku pg:backups:restore "https://s3.amazonaws.com/backup_xxx"
DATABASE_URL
! WARNING: Destructive Action
! This command will affect the app gcesalem
! To proceed, type gcesalem or re-run this command with --confirm appname
> appname
Starting restore of https://s3.amazonaws.com/backup_xxx
to postgresql-round-xxx... done
Use Ctrl-C at any time to stop monitoring progress; the backup will continue restoring.
Use heroku pg:backups to check progress.
Stop a running restore with heroku pg:backups:cancel.
Restoring... !
! An error occurred and the backup did not finish.
!
! waiting for restore to complete
! pg_restore finished with errors
! waiting for download to complete
! download finished with errors
! please check the source URL and ensure it is publicly accessible
!
! Run heroku pg:backups:info r006 for more details.
Info
C:\Users\krishnava\Downloads\git>heroku pg:backups:info r006
=== Backup r006
Database: BACKUP
Started at: 2019-07-16 15:34:40 +0000
Finished at: 2019-07-16 15:34:40 +0000
Status: Failed
Type: Manual
Backup Size: 0.00B (0% compression)
=== Backup Logs
2019-07-16 15:34:40 +0000 pg_restore: [archiver] did not find magic string in file header
2019-07-16 15:34:40 +0000 waiting for restore to complete
2019-07-16 15:34:40 +0000 pg_restore finished with errors
2019-07-16 15:34:40 +0000 waiting for download to complete
2019-07-16 15:34:40 +0000 download finished with errors
2019-07-16 15:34:40 +0000 please check the source URL and ensure it is publicly accessible- -
Instead of doing like this, You can export a copy of the local database and import it to Heroku.
For export from local database,
pg_dump <DATABASE_NAME> > <FILENAME>.sql
This will ask you to enter your database password. But in Windows, this will ask the User password, because the default user name is the system user name. For this you have to specify your username
pg_dump -U <USER_NAME> <DATABASE_NAME> > <FILENAME>.sql
For your case the command will be like this:
pg_dump -U postgres gce > gce.sql
After exporting the local database, you can upload this directly to heruko.
heroku pg:psql --app <APP_NAME> < gce.sql
Related
If I run heroku pg:backups --app MY_APP, I see:
=== Backups
No backups. Capture one with heroku pg:backups:capture
=== Restores
No restores found. Use heroku pg:backups:restore to restore a backup
=== Copies
ID Started at Status Size From To
──── ───────────────────────── ─────────────────────────────────── ──────── ───────── ───────────
c001 2021-02-23 15:47:54 +0000 Completed 2021-02-23 15:48:27 +0000 509.16KB From Name Target Name
If I run heroku pg:backups:info c001 --app MY_APP, I see:
=== Backup c001
Database: From Name
Started at: 2021-02-23 15:47:54 +0000
Finished at: 2021-02-23 15:48:27 +0000
Status: Completed
Type: Manual
Original DB Size: 11.53MB
Backup Size: 509.16KB (96% compression)
=== Backup Logs
However, when I try to restore this backup using heroku pg:backups:restore c001 --app MY_APP, it tells me that Backup c001 not found for MY_APP.
Is there any way I can restore this copy? For context, my database was in the free plan and was deleted by Heroku, so as far as I know this is the last backup I have.
Thank you!
Backups start with b00* and are listed in the top section.
I am not sure what "Copies" are, did not find anything in their docs.
❯ heroku pg:backups -r prod
=== Backups
ID Created at Status Size Database
──── ───────────────────────── ─────────────────────────────────── ──────── ────────
b008 2021-05-04 06:57:18 +0000 Completed 2021-05-04 06:57:20 +0000 664.15KB DATABASE
b007 2021-05-04 05:33:50 +0000 Completed 2021-05-04 05:33:52 +0000 663.53KB DATABASE
=== Restores
No restores found. Use heroku pg:backups:restore to restore a backup
=== Copies
ID Started at Status Size From To
──── ───────────────────────── ─────────────────────────────────── ──────── ───────── ───────────
c009 2022-06-23 03:28:14 +0000 Completed 2022-06-23 03:28:46 +0000 232.80KB From Name Target Name
c002 2020-05-27 15:29:37 +0000 Completed 2020-05-27 15:29:40 +0000 112.59KB From Name Target Name
c001 2020-05-26 19:43:26 +0000 Completed 2020-05-26 19:43:37 +0000 112.97KB From Name Target Name
❯ heroku pg:backups:restore c009 --app <app>
▸ Backup c009 not found for ⬢ <app>
❯ heroku pg:backups:restore b008 --app <app>
▸ WARNING: Destructive Action
▸ This command will affect the app <app>
▸ To proceed, type <app> or re-run this command with --confirm
▸ <app>
...
I've just sit down to my work, forgetting that I've upgraded brew in the 2020 – so of course I have a postgresql problem.
The last version I had was 12.x. Now I have 13.1 installed:
$ brew info postgres
Warning: Treating postgres as a formula. For the cask, use homebrew/cask/postgres
postgresql: stable 13.1 (bottled), HEAD
Object-relational database system
https://www.postgresql.org/
/usr/local/Cellar/postgresql/13.1 (3,217 files, 42.6MB) *
Poured from bottle on 2020-12-26 at 14:36:54
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/postgresql.rb
License: PostgreSQL
==> Dependencies
Build: pkg-config ✔
Required: icu4c ✔, krb5 ✔, openssl#1.1 ✔, readline ✔
==> Options
--HEAD
Install HEAD version
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
This formula has created a default database cluster with:
initdb --locale=C -E UTF-8 /usr/local/var/postgres
For more details, read:
https://www.postgresql.org/docs/13/app-initdb.html
To have launchd start postgresql now and restart at login:
brew services start postgresql
Or, if you don't want/need a background service you can just run:
pg_ctl -D /usr/local/var/postgres start
==> Analytics
install: 181,394 (30 days), 574,628 (90 days), 1,984,700 (365 days)
install-on-request: 176,623 (30 days), 554,629 (90 days), 1,910,250 (365 days)
build-error: 0 (30 days)
Following the output, I'm trying to run brew postgresql-upgrade-database, but it does not work – here is the output:
$ brew postgresql-upgrade-database --debug --verbose
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/postgresql.rb
==> Upgrading postgresql data from 12 to 13...
/usr/local/Cellar/postgresql#12/12.5/bin/pg_ctl -w -D /usr/local/var/postgres start
waiting for server to start....2021-01-04 17:37:33.282 CET [8190] FATAL: database files are incompatible with server
2021-01-04 17:37:33.282 CET [8190] DETAIL: The database cluster was initialized with PG_CONTROL_VERSION 1300, but the server was compiled with PG_CONTROL_VERSION 1201.
2021-01-04 17:37:33.282 CET [8190] HINT: It looks like you need to initdb.
2021-01-04 17:37:33.282 CET [8190] LOG: database system is shut down
stopped waiting
pg_ctl: could not start server
Examine the log output.
==> Moving postgresql data from /usr/local/var/postgres to /usr/local/var/postgres.old...
==> Creating database...
/usr/local/opt/postgresql/bin/initdb /usr/local/var/postgres
The files belonging to this database system will be owned by user "nyc".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Europe/Warsaw
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/local/opt/postgresql/bin/pg_ctl -D /usr/local/var/postgres -l logfile start
==> Migrating and upgrading data...
/usr/local/opt/postgresql/bin/pg_upgrade -r -b /usr/local/Cellar/postgresql#12/12.5/bin -B /usr/local/opt/postgresql/bin -d /usr/local/var/postgres.old -D /usr/local/var/postgres -j 4
Performing Consistency Checks
-----------------------------
Checking cluster versions ok
The source cluster was not shut down cleanly.
Failure, exiting
Error: Upgrading postgresql data from 12 to 13 failed!
==> Removing empty postgresql initdb database...
==> Moving postgresql data back from /usr/local/var/postgres.old to /usr/local/var/postgres...
Error: Failure while executing; `/usr/local/opt/postgresql/bin/pg_upgrade -r -b /usr/local/Cellar/postgresql#12/12.5/bin -B /usr/local/opt/postgresql/bin -d /usr/local/var/postgres.old -D /usr/local/var/postgres -j 4` exited with 1.
/usr/local/Homebrew/Library/Homebrew/utils.rb:303:in `safe_system'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/postgresql-upgrade-database.rb:126:in `block in postgresql_upgrade_database'
/usr/local/Homebrew/Library/Homebrew/extend/pathname.rb:318:in `block in cd'
/usr/local/Homebrew/Library/Homebrew/extend/pathname.rb:318:in `chdir'
/usr/local/Homebrew/Library/Homebrew/extend/pathname.rb:318:in `cd'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/postgresql-upgrade-database.rb:125:in `postgresql_upgrade_database'
/usr/local/Homebrew/Library/Homebrew/brew.rb:124:in `<main>'
How can I fix this?
I am trying to configure Barman to backup. When I do a barman check replica I keep getting:
Server replica:
WAL archive: FAILED (please make sure WAL shipping is setup)
PostgreSQL: OK
superuser: OK
wal_level: OK
directories: OK
retention policy settings: OK
backup maximum age: FAILED (interval provided: 1 day, latest backup age: No available backups)
compression settings: OK
failed backups: OK (there are 0 failed backups)
minimum redundancy requirements: FAILED (have 0 backups, expected at least 2)
ssh: OK (PostgreSQL server)
not in recovery: FAILED (cannot perform exclusive backup on a standby)
archive_mode: OK
archive_command: OK
continuous archiving: OK
archiver errors: OK
I am using Postgresql 9.6 and barman 2.1; I am not sure as to what the issue is could someone help?
Here is my Barman server configuration:
description = "Database backup"
conninfo = host=<db-ip> user=postgres dbname=db
backup_method = rsync
ssh_command = ssh postgres#<db-ip>
archiver = on
barman check tries to confirm that archiving is set up correctly by asserting that there's actually something in the archive. However, WAL segments are generally only archived once they're filled up, and if your server is idle, this is never going to happen.
To work around this, Barman provides a command to force a segment switch, wait for the completed WAL to show up, and then archive it immediately:
barman switch-xlog --force --archive replica
in brief
Barman's incoming_wals_directory and Postgresql.conf's archive_command not matched as described in details here
details
Another cause is that the not matched between
Barman's incoming_wals_directory
Postgresql.conf's archive_command
Bash util to check
barman#backup $ barman show-server pg | grep incoming_wals_directory
# output1
# > incoming_wals_directory: /var/lib/barman/pg/incoming
postgres#pg $ cat /etc/postgresql/10/main/postgresql.conf | grep archive_command
# output2
# > archive_command = 'rsync -a %p barman#staging:/var/lib/barman/pg/incoming/%f'
We must have same path in :output1 and :output2
Make them matched if they don't and don't forget to restart postgres afterward.
I installed two postgresql(s), 9.5 should use port 5433, 9.3.4 should use port 5432:
me#camb:/usr/lib/postgresql/9.5/bin$ ls -al psql
-rwxr-xr-x 1 root root 564464 Jan 7 14:54 psql
me#camb:/opt/pkgs/postgresql-9.3.4/bin$ ls -al psql
-rwxr-xr-x 1 root root 405960 Aug 5 18:52 psql
me#camb:/opt/pkgs/postgresql-9.3.4/bin$ cat /etc/group
postgres:x:200:
me#camb:/opt/pkgs/postgresql-9.3.4/bin$ groups postgres
postgres : postgres ssl-cert
I wrote a script to install postgresql-9.5 and migrate from 9.3.4 to 9.5
Problem 0: postgresql-9.5 can not start sucessfully and why status is online
Output during install:
* Starting PostgreSQL 9.5 database server
* The PostgreSQL server failed to start. Please check the log output:
2016-01-19 19:41:49 UTC [16523-1] LOG: database system was shut down at 2016-01-19 19:41:48 UTC
2016-01-19 19:41:49 UTC [16523-2] LOG: MultiXact member wraparound protections are now enabled
2016-01-19 19:41:49 UTC [16522-1] LOG: database system is ready to accept connections
2016-01-19 19:41:49 UTC [16527-1] LOG: autovacuum launcher started
2016-01-19 19:41:49 UTC [16529-1] [unknown]#[unknown] LOG: incomplete startup packet
...fail!
invoke-rc.d: initscript postgresql, action "start" failed.
dpkg: error processing postgresql-9.5 (--configure):
subprocess installed post-installation script returned error exit status 1
dpkg: dependency problems prevent configuration of postgresql:
postgresql depends on postgresql-9.5; however:
Package postgresql-9.5 is not configured yet.
dpkg: error processing postgresql (--configure):
dependency problems - leaving unconfigured
Processing triggers for libc-bin ...
No apport report written because the error message indicates its a followup error from a previous failure.
ldconfig deferred processing now taking place
Errors were encountered while processing:
postgresql-9.5
postgresql
E: Sub-process /usr/bin/dpkg returned an error code (1)
Error while installing Postgresql-9.5:1
Then I did some tests:
me#camb:/usr/lib/postgresql/9.5/bin$ sudo service postgresql start
* Starting PostgreSQL 9.5 database server [ OK ]
me#camb:/usr/lib/postgresql/9.5/bin$ sudo service postgresql status
9.5/main (port 5433): online
What do the above errors?
By the way, another error is:
ERROR: role "postgres" already exists
Problem 1: Why I cannot use /usr/lib/postgresql/9.5/bin/psql to loginto the database, but I can do this by /opt/pkgs/postgresql-9.3.4/bin/psql
me#camb:/usr/lib/postgresql/9.5/bin$ sudo -u postgres /opt/pkgs/postgresql-9.3.4/bin/psql -p 5433 reporting_central postgres
psql (9.3.4, server 9.5.0)
WARNING: psql major version 9.3, server major version 9.5.
Some psql features might not work.
Type "help" for help.
reporting_central=# \q
me#camb:/usr/lib/postgresql/9.5/bin$ sudo -u postgres /opt/pkgs/postgresql-9.3.4/bin/psql -p 5432 reporting_central postgres
psql (9.3.4)
Type "help" for help.
reporting_central=# \q
me#camb:/usr/lib/postgresql/9.5/bin$ sudo -u postgres /usr/lib/postgresql/9.5/bin/psql -p 5433 reporting_central postgres
/usr/lib/postgresql/9.5/bin/psql: symbol lookup error: /usr/lib/postgresql/9.5/bin/psql: undefined symbol: PQsslInUse
me#camb:/usr/lib/postgresql/9.5/bin$ sudo -u postgres /usr/lib/postgresql/9.5/bin/psql -p 5432 reporting_central postgres
/usr/lib/postgresql/9.5/bin/psql: symbol lookup error: /usr/lib/postgresql/9.5/bin/psql: undefined symbol: PQsslInUse
The error is "undefined symbol: PQsslInUse". what does this mean? Why I cannot use /usr/lib/postgresql/9.5/bin/psql to loginto the database, but I can do this by /opt/pkgs/postgresql-9.3.4/bin/psql?
Thanks
By the way, I run the following to do migration:
sudo -u postgres /opt/pkgs/postgresql-9.3.4/bin/pg_dumpall -p 5432 | sudo -u postgres /opt/pkgs/postgresql-9.3.4/bin/psql -p 5433
Thanks
You have to use the same libpq library as the one used in psql 9.5.
You should do the following:
export LD_LIBRARY_PATH=/usr/lib/postgresql/9.5/lib:$LD_LIBRARY_PATH
sudo /sbin/ldconfig /usr/lib/postgresql/9.5/lib
You mess up with different versions of postgresql.
To start/use 9.5, you must use the appropriate binaries: initdb, pg_ctl and psql (for example):
mkdir /usr/lib/postgresql/9.5/data
/usr/lib/postgresql/9.5/bin/initdb -D ../data/
/usr/lib/postgresql/9.5/bin/pg_ctl -D ../data/ -l logfile start
/usr/lib/postgresql/9.5/bin/psql -p 5432 -d template1
More info: https://www.postgresql.org/docs/devel/static/install-post.html
and PQsslInUse issue: http://postgresql.nabble.com/psql-error-on-postgresql-9-0-psql-symbol-lookup-error-psql-undefined-symbol-PQconnectdbParams-td5104930.html
The error is "undefined symbol: PQsslInUse"
I just ran into this problem myself. I discovered that the reason was because I compiled Postgres 9.5 from scratch into a particular directory and then decided that I wanted it elsewhere and moved the install to a new directory. Down the line when I went to restore my database with psql, I got the above error. I corrected the problem by deleting my installation directory, changing the PREFIX to the correct location, recompiling and reinstalling Postgres. psql worked fine after this.
I had the same problem
psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility
And the solution is :
export LD_LIBRARY_PATH=$PGXZ_HOME/lib:${LD_LIBRARY_PATH}
I had a similar issue and finally found a solution:
My user did not have permission for the postgres installation.
What worked for me was to switch to the postgres user:
sudo su - postgres
psql
and then it worked no issues.
I am trying to create a hot_standby server, and I receive the following error after pg_basebackup completes. Notice I use a shell script, replicator.sh, to start the replication. Can anyone give me some insight?
My specs:
Debian Wheezy 7.6
Postgresql 9.3
Database size: ~115GB
Error:
postgres#database-master:/etc/postgresql/9.3/main$ sh replicator.sh
Stopping PostgreSQL
[ ok ] Stopping PostgreSQL 9.3 database server: main.
Cleaning up old cluster directory
Starting base backup as replicator
Password:
113720266/113720266 kB (100%), 1/1 tablespace
NOTICE: WAL archiving is not enabled; you must ensure that all required WAL segments are copied through other means to complete the backup
pg_basebackup: base backup completed
Starting Postgresql
[....] Starting PostgreSQL 9.3 database server: main[....] The PostgreSQL server failed to start.
Please check the log output: 2014-09-11 17:56:33 UTC LOG: database system was interrupted; last
known up at 2014-09-11 16:54:29 UTC 2014-09-11 17:56:33 UTC LOG: creating missing WAL directory
"pg_xlog/archive_status" 2014-09-11 17:56:33 UTC LOG: incomplete startup packet 2014-09-11 17:56:33
UTC LOG: invalid checkpoint record 2014-09-11 17:56:33 UTC FATAL: could not locate required
checkpoint record 2014-09-11 17:56:33 UTC HINT: If you are not restoring from a backup, try
removing the file "/var/lib/p[FAILesql/9.3/main/backup_label". 2014-09-11 17:56:33 UTC LOG: startup
process (PID 21972) exited with exit code 1 2014-09-11 17:56:33 UTC LOG: aborting startup due to
startup process failure ... failed! failed!
Contents of replicator.sh:
#!/bin/bash
echo Stopping PostgreSQL
/etc/init.d/postgresql stop
echo Cleaning up old cluster directory
rm -rf /var/lib/postgresql/9.3/main
echo Starting base backup as replicator
pg_basebackup -h 123.456.789.123 -D /var/lib/postgresql/9.3/main -U replicator -v -P
echo Writing recovery.conf file
sudo -u postgres bash -c "cat > /var/lib/postgresql/9.3/main/recovery.conf <<- _EOF1_
standby_mode = 'on'
primary_conninfo = 'host=123.456.789.123 port=5432 user=replicator password=XXXXX sslmode=require'
trigger_file = '/tmp/postgresql.trigger'
_EOF1_
"
echo Starting Postgresql
/etc/init.d/postgresql start
Thank you,
Jake
My best guess from the above is that the pg_basebackup failed and your shell script doesn't check for error return codes or use set -e to automatically abort after errors, so it just carried on regardless.
It's also possible that you don't have WAL archiving configured, or don't have a restore_command set in the replica. In that case, the transaction logs required to start the base backup will not be available and startup will fail.
I strongly recommend that you:
Use pg_basebackup -X stream so that the required transaction logs get copied along with the backup; and
Use set -e in your shell script, or test for errors with a suitable if ! pg_basebackup .... ; then block.