we are facing error during postgres upgrade version from 11 to 12 .
Please suggest what I do to resolve this issue.
Error:
Performing Consistency Checks on Old Live Server
------------------------------------------------
Checking cluster versions ok
old and new pg_controldata WAL segment sizes are invalid or do not match
Failure, exiting
bash-4.1$
.....................
command:
/usr/pgsql-12/bin/pg_upgrade --old-datadir /var/lib/pgsql/11/data/ --new-datadir /var/lib/pgsql/12/data/ --old-bindir /usr/pgsql-11/bin/ --new-bindir /usr/pgsql-12/bin/ --check
Related
I installed postgres 14 beta and i want to upgrade to 14 new release. I have just installed the new release pg 14 and when i try to start the pg 14 beta i get the error below. May i know the correct procedure of upgrading from beta to new release
-bash-4.2$ /usr/pgsql-14/bin/pg_ctl -D /var/lib/pgsql/14/data -l logfile start
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
-bash-4.2$ cat logfile
2021-10-27 13:19:29.507 UTC [5112] FATAL: database files are incompatible with server
2021-10-27 13:19:29.507 UTC [5112] DETAIL: The database cluster was initialized with CATALOG_VERSION_NO 202106151, but the server was compiled with CATALOG_VERSION_NO 202107181.
2021-10-27 13:19:29.507 UTC [5112] HINT: It looks like you need to initdb.
2021-10-27 13:19:29.507 UTC [5112] LOG: database system is shut down
There has been a change in the catalog version since v14 beta 1, so you have to use fump/restore of pg_upgrade to upgrade.
Install 14.0, create a new cluster and use the method of your choice to upgrade.
I've just sit down to my work, forgetting that I've upgraded brew in the 2020 – so of course I have a postgresql problem.
The last version I had was 12.x. Now I have 13.1 installed:
$ brew info postgres
Warning: Treating postgres as a formula. For the cask, use homebrew/cask/postgres
postgresql: stable 13.1 (bottled), HEAD
Object-relational database system
https://www.postgresql.org/
/usr/local/Cellar/postgresql/13.1 (3,217 files, 42.6MB) *
Poured from bottle on 2020-12-26 at 14:36:54
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/postgresql.rb
License: PostgreSQL
==> Dependencies
Build: pkg-config ✔
Required: icu4c ✔, krb5 ✔, openssl#1.1 ✔, readline ✔
==> Options
--HEAD
Install HEAD version
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
This formula has created a default database cluster with:
initdb --locale=C -E UTF-8 /usr/local/var/postgres
For more details, read:
https://www.postgresql.org/docs/13/app-initdb.html
To have launchd start postgresql now and restart at login:
brew services start postgresql
Or, if you don't want/need a background service you can just run:
pg_ctl -D /usr/local/var/postgres start
==> Analytics
install: 181,394 (30 days), 574,628 (90 days), 1,984,700 (365 days)
install-on-request: 176,623 (30 days), 554,629 (90 days), 1,910,250 (365 days)
build-error: 0 (30 days)
Following the output, I'm trying to run brew postgresql-upgrade-database, but it does not work – here is the output:
$ brew postgresql-upgrade-database --debug --verbose
/usr/local/Homebrew/Library/Homebrew/brew.rb (Formulary::FormulaLoader): loading /usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/Formula/postgresql.rb
==> Upgrading postgresql data from 12 to 13...
/usr/local/Cellar/postgresql#12/12.5/bin/pg_ctl -w -D /usr/local/var/postgres start
waiting for server to start....2021-01-04 17:37:33.282 CET [8190] FATAL: database files are incompatible with server
2021-01-04 17:37:33.282 CET [8190] DETAIL: The database cluster was initialized with PG_CONTROL_VERSION 1300, but the server was compiled with PG_CONTROL_VERSION 1201.
2021-01-04 17:37:33.282 CET [8190] HINT: It looks like you need to initdb.
2021-01-04 17:37:33.282 CET [8190] LOG: database system is shut down
stopped waiting
pg_ctl: could not start server
Examine the log output.
==> Moving postgresql data from /usr/local/var/postgres to /usr/local/var/postgres.old...
==> Creating database...
/usr/local/opt/postgresql/bin/initdb /usr/local/var/postgres
The files belonging to this database system will be owned by user "nyc".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Europe/Warsaw
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/local/opt/postgresql/bin/pg_ctl -D /usr/local/var/postgres -l logfile start
==> Migrating and upgrading data...
/usr/local/opt/postgresql/bin/pg_upgrade -r -b /usr/local/Cellar/postgresql#12/12.5/bin -B /usr/local/opt/postgresql/bin -d /usr/local/var/postgres.old -D /usr/local/var/postgres -j 4
Performing Consistency Checks
-----------------------------
Checking cluster versions ok
The source cluster was not shut down cleanly.
Failure, exiting
Error: Upgrading postgresql data from 12 to 13 failed!
==> Removing empty postgresql initdb database...
==> Moving postgresql data back from /usr/local/var/postgres.old to /usr/local/var/postgres...
Error: Failure while executing; `/usr/local/opt/postgresql/bin/pg_upgrade -r -b /usr/local/Cellar/postgresql#12/12.5/bin -B /usr/local/opt/postgresql/bin -d /usr/local/var/postgres.old -D /usr/local/var/postgres -j 4` exited with 1.
/usr/local/Homebrew/Library/Homebrew/utils.rb:303:in `safe_system'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/postgresql-upgrade-database.rb:126:in `block in postgresql_upgrade_database'
/usr/local/Homebrew/Library/Homebrew/extend/pathname.rb:318:in `block in cd'
/usr/local/Homebrew/Library/Homebrew/extend/pathname.rb:318:in `chdir'
/usr/local/Homebrew/Library/Homebrew/extend/pathname.rb:318:in `cd'
/usr/local/Homebrew/Library/Taps/homebrew/homebrew-core/cmd/postgresql-upgrade-database.rb:125:in `postgresql_upgrade_database'
/usr/local/Homebrew/Library/Homebrew/brew.rb:124:in `<main>'
How can I fix this?
I ran
brew postgresql-upgrade-database
and after a long series of updates the final results are:
==> Migrating and upgrading data...
Performing Consistency Checks
-----------------------------
Checking cluster versions ok
Checking database user is the install user ok
Checking database connection settings ok
Checking for prepared transactions ok
Checking for reg* data types in user tables ok
Checking for contrib/isn with bigint-passing mismatch ok
Checking for tables WITH OIDS ok
Checking for invalid "sql_identifier" user columns ok
Creating dump of global objects ok
Creating dump of database schemas
ok
Checking for presence of required libraries fatal
Your installation references loadable libraries that are missing from the
new installation. You can add these libraries to the new installation,
or remove the functions using them from the old installation. A list of
problem libraries is in the file:
loadable_libraries.txt
Failure, exiting
Error: Upgrading postgresql data from 11 to 12 failed!
==> Removing empty postgresql initdb database...
==> Moving postgresql data back from /usr/local/var/postgres.old to /usr/local/var/postgres...
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
Error: Failure while executing; `/usr/local/opt/postgresql/bin/pg_upgrade -r -b /usr/local/Cellar/postgresql#11/11.9/bin -B /usr/local/opt/postgresql/bin -d /usr/local/var/postgres.old -D /usr/local/var/postgres -j 16` exited with 1.
The only notes about this I could find were about pg developers mulling whether to print out the offending databases involved: it was unclear what the fix would be. Any hints ?
Update There is no loadable_libraries.txt in the directory that this was run.
I am creating an answer based on the comment from #LaurenzAlbe. The loadable_libraries.txt was not placed in the current directory from which the command
was run . Instead let's go find where it went:
$find / -name loadable_libraries.txt
/usr/local/var/log/loadable_libraries.txt
So what's in that file?
06:22:24/cidervuong2 $cat /usr/local/var/log/loadable_libraries.txt
could not load library "$libdir/pg_background": ERROR: could not access file "$libdir/pg_background": No such file or directory
Database: bluej
I looked into that error:
ERROR: could not access file "$libdir/pg_background"
and there is no useful information. I went ahead and nuked postgresql again and this time it installed cleanly.
brew uninstall --ignore-dependencies postgres
brew install postgres
brew services start postgresql # actually already started
and pg came up cleanly this time
502 84118 1 0 6:33AM ?? 0:00.03 /usr/local/opt/postgresql/bin/postgres -D /usr/local/var/postgres
502 84120 84118 0 6:33AM ?? 0:00.00 postgres: checkpointer
502 84121 84118 0 6:33AM ?? 0:00.03 postgres: background writer
502 84122 84118 0 6:33AM ?? 0:00.01 postgres: walwriter
502 84123 84118 0 6:33AM ?? 0:00.01 postgres: autovacuum launcher
502 84124 84118 0 6:33AM ?? 0:00.01 postgres: stats collector
502 84125 84118 0 6:33AM ?? 0:00.00 postgres: logical replication launcher
I have installed postgresql and done all the setup and created db and tables yesterday but today morning I got sudden this error
I have also uninstalled and installed postgresql but still same error. And also I have observed that when installed freshly it is working fine but once I shutdown or restart my system I'm getting the below error.
psql:
could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I have tried all the solutions given in all the mediums but I'm not able to resolve the issue so I'm posting this question again with my outputs.
The status of postgresql
**sudo /etc/init.d/postgresql status**
postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
Active: active (exited) since Wed 2019-10-30 11:40:39 IST; 5min ago
Process: 7666 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 7666 (code=exited, status=0/SUCCESS)
Tasks: 0
Memory: 0B
CPU: 0
CGroup: /system.slice/postgresql.service
When I run the pg_lsclusters
**pg_lsclusters**
Ver Cluster Port Status Owner Data directory Log file
11 main 5432 down postgres /var/lib/postgresql/11/main /var/log/postgresql/postgresql-11-main.log
**dpkg -l | grep postgres**
ii pgdg-keyring 2018.2 all keyring for apt.postgresql.org
ii postgresql 11+207.pgdg16.04+1 all object-relational SQL database (supported version)
ii postgresql-11 11.5-3.pgdg16.04+1 amd64 object-relational SQL database, version 11 server
ii postgresql-client-11 11.5-3.pgdg16.04+1 amd64 front-end programs for PostgreSQL 11
ii postgresql-client-common 207.pgdg16.04+1 all manager for multiple PostgreSQL client versions
ii postgresql-common 207.pgdg16.04+1 all PostgreSQL database-cluster manager
ii postgresql-contrib 11+207.pgdg16.04+1 all additional facilities for PostgreSQL (supported version)
I even tried to restart the postgres still same issue.
**sudo /etc/init.d/postgresql status**
I had exactly this error message, and this is how I resolved it.
In the log file at /var/log/postgresql/postgresql-13-main.log, these lines were present:
2021-09-15 08:26:04.936 UTC [46258] LOG: invalid value for parameter "lc_messages": "en_US.UTF-8"
2021-09-15 08:26:04.936 UTC [46258] LOG: invalid value for parameter "lc_monetary": "en_US.UTF-8"
2021-09-15 08:26:04.937 UTC [46258] LOG: invalid value for parameter "lc_numeric": "en_US.UTF-8"
2021-09-15 08:26:04.937 UTC [46258] LOG: invalid value for parameter "lc_time": "en_US.UTF-8"
2021-09-15 08:26:04.937 UTC [46258] FATAL: configuration file "/etc/postgresql/13/main/postgresql.conf" contains errors
pg_ctl: could not start server
Examine the log output.
This is caused by missing locales, with these possible fixes:
localedef allegedly fixes this according but I had no luck with this.
localedef -i en_US -f UTF-8 en_US.UTF-8
change postgresql.conf and use a locale that does exist (en_GB in my case)
lc_messages = 'en_GB.UTF-8' # locale for system error message strings
lc_monetary = 'en_GB.UTF-8' # locale for monetary formatting
lc_numeric = 'en_GB.UTF-8' # locale for number formatting
lc_time = 'en_GB.UTF-8' # locale for time formatting
post settings
port = 5432 # (change requires restart)
Port was wrong on my PostgreSQL 13 default config (was 5433) and required a change back to the default 5432. I presume this was a result of upgrading from v11 so it gave it a new port number.
After these changes, PostgreSQL needs to restart and it should now function correctly without errors in the log, and the psql command working as expected.
Try this:
sudo -u postgres -i
And then:
initdb --locale $LANG -E UTF8 -D '/var/lib/postgres/data/'
exit
This has worked for me so far
I'm attempting a whole-planet OSM data import on an AWS EC2. During or possibly after the "Ways" processing i receive the following message:
"Failed to read from node cache: Input/output error"
The EC2 has the following specs:
type: i3.xlarge
memory: 30.5 Gb
vCPUs: 4
Postgresql: v9.5.6
PostGIS: 2.2
In addition to the root volume, I have mounted 900GB SSD and a 2TB HHD (high throughput). The Postgresql data directory is on the HHD. I have commanded the osm2pgsql to write the flat-nodes file the SSD.
Here is my osm2pgsql command:
osm2pgsql -c -d gis --number-processes 4 --slim -C 20000 --flat-nodes /data-cache/flat-node-cache/flat.nodes /data-postgres/planet-latest.osm.pbf
I run the above command as user renderaccount that is a member of the following groups renderaccount ubuntu postgres. The flat-nodes file appears to be successfully created at /data-cache/flat-node-cache/flat.nodes and has this profile:
ubuntu#ip-172-31-25-230:/data-cache/flat-node-cache$ ls -l
total 37281800
-rw------- 1 renderaccount renderaccount 38176555024 Apr 13 05:45 flat.nodes
Has anyone run into and or resolved this? I suspect maybe a permissions issue? I notice now that since this last osm2pgsql failure, the mounted SSD that is the destination of the flat-nodes file has been converted to a "read-only" file system - which sounds like may happen when there are i/o errors on the mounted volume(?).
Also, does osm2pgsql write to a log that I could acquire additional info?
UPDATE: dmesg output:
[ 6206.884412] blk_update_request: I/O error, dev nvme0n1, sector 66250752
[ 6206.890813] EXT4-fs warning (device nvme0n1): ext4_end_bio:329: I/O error -5 writing to inode 14024706 (offset 10871640064 size 8388608 starting block 8281600)
[ 6206.890817] Buffer I/O error on device nvme0n1, logical block 8281344
After researching the above output, it appears it might be a bug in Ubuntu 16.04. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668129?comments=all
This was an error with Ubuntu 16.04 writing to the volume nvme0n1. Solved by this https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1668129/comments/29