I followed these instructions:
Upgrade PostgreSQL from 9.6 to 10.0 on Ubuntu 16.10
And got the upgrade done without issue. I kept the old cluster - however with identical queries on the new cluster im getting:
wait_event_type: IO
wait_event: DataFileRead
These queries that hang are largish in that they join 10's millions of rows. I have double checked postgresql.conf and parameters where present in both the 9.6 version and 10 version are identical except:
"bytea_output"
"client_encoding"
"hot_standby"
"max_connections"
"max_replication_slots"
"password_encryption"
"port"
"server_version"
"server_version_num"
"superuser_reserved_connections"
"wal_level"
I have both clusters up on the same machine - tables I'm querying are sitting on the same tablespaces, queries are identical, config is more or less identical - is there something I'm missing with postgres 10?
Many Thanks
Related
I have a Postgres database instance in GCP running on version 9.6.
I want to upgrade the Postgres version to a newer version, and I use GCPs "Database migration" for that purpose.
I have 2 databases in the instance and they fill around 800 GB in total.
The problem is that the migration gets in stuck. There are no errors in the migration log.
Copy of monitoring
In short, my question is:
How can I check in what phase is the migration and what is the issue?
Thanks.
I worked on pgadmin 4 and QGIS in localhost port 5432 with version 13 of PostgreSQL creating and updating table in a spatial database. I installed PostgreSQL 14 a few days ago but finally I was not using it, I worked only on PostgreSQL 13 database. So, today, I decided to uninstall PostgreSQL 14. However, after that, I lose all tables and data I've done until then on the database in PostgreSQL 13 and I don't know why because I let PostgreSQL 13 and uninstall only version 14. I cannot access to my work in QGIS as well. Can anybody help me to understand what's happened and is it possible to get back my work and how?
I still have the folder called data from PostgreSQL 13 and 14 in OS. I would like to know if it's possible to extract tables I have done before uninstallation from the data folders of PostgreSQL 14, with files in base.
Postgres data folders
base folder
Thank you if anyone can tell me and how if it's possible.
I'm trying to insert a few thousand records in a simple table structure into a postgres table.
In my c# unit test I create a fresh database from the latest docker alpine image, and then using NPGSQL create a table try and insert a few thousand record.
After about 11k records the container seems to hang, and eventually the test times out. The container logs show nothing after it stops inserting records.
At first I thought it was NPGSQL, however the same test on a normal (non docker) database works fine, so it must be related to the docker image?
I've tried several different settings for docker memory/cpu's and also shared_buffers and max_connections for example for the postgres db. The test hangs on the same number of inserts regardless.
I realise bulk inserts are better done with the COPY command, but for this test multiple inserts are relevant.
NpgSql 5.0.7
PostgreSQL 13.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20210424) 10.3.1 20210424, 64-bit
After noticing that more operations were timing out.
I uninstalled docker, rebooted my machine and reinstalled docker. This seems to have solved the issue for now.
I'm trying to upgrade my postgres database from 9.6 to 10 without any downtime. Can this be done?
As "a_horse_with_no_name" mentioned logical replication is a very good choice in your situation.
Only problem is 9.6 does not have internal implementation yet so you would have to use extension "pglogical" on both DB - here I found some description - https://rosenfeld.herokuapp.com/en/articles/infrastructure/2017-11-10-upgrading-postgresql-from-9-6-to-10-with-minimal-downtime-using-pglogical - skip parts about Docker and see how pglogical works.
The only slight problem is pglogical must be added into "shared_preload_libraries" parameter and postgresql service must be restarted which can be sometimes difficult on production...
I did quite a lot of tests with pglogical (here are some notes - http://postgresql.freeideas.cz/pglogical-postgresql-9-6-small-hints-debian/) although at the end I never used pglogical on production. So I do not have experiences from long usage.
But I presume some problems can be similar to the internal implementation of logical replication in PG 10 and 11. So here are my notes from my current usage of internal logical replication on PG 11 - http://postgresql.freeideas.cz/setting-logical-replication-is-not-entirely-straight-forward/ - maybe something from it would help you.
My recommendation for you would be:
make a hot backup copy of your PG 9.6 database on some other machine on cloud VM with exactly the same OS and if possible disks types and configuration using pg_basebackup - you can find some inspiration here:
http://postgresql.freeideas.cz/pg_basebackup-bash-script-backup-archiving-google-storage/
http://postgresql.freeideas.cz/pg_basebackup-experiences/
or if you already use pg_basebackup for tar backups of your db restore latest backup on other machine or VM (http://postgresql.freeideas.cz/pg_basebackup-pgbarman-restore-tar-backup/)
start is as normal server (not as hot standby) and test pglogical on this copy of your DB against some testing installation of PG 10 - test it as close to the production environment as possible including at least simulated DML operations of similar intensity - this will show you differences in load on machine / VM.
I highly recommend to set monitoring for example using telegraf + influxdb + Grafana (easiest implementation by my opinion) to be able to analyze CPU and memory usage later - this can be very crucial part for usage on production !
after hopefully short and successful tests implement it and celebrate your success :-) and please write about you experiences. Because I believe a lot of people would welcome it.
I need to use the new jsonb field in postgres so I decided to upgrade to 9.5 in my ununtu 16.04 LTS. I followed this tutorial , but when it gets to the command pg_upgradecluster 9.3 main, I get the error that cluster configuration already exists. I checked other tutorials and found that I could get the list of clusters with pg_lsclusters. But the command always shows only the 9.3 cluster. Any attempt to create the 9.5 cluster always results to cluster configuration already exists. I purged the 9.5 and installed again multiple times but the same result. At last I decided to purge all the postgres installations I have after backing up my 9.3 cluster with pg_dumpall.
Now I have decided to install the 9.5 freshly but I still get the same thing. When I install pg_lsclusters shows nothing and after starting postgres psql always returns psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Right now I'm so tired and frustrated. Please if anyone has any possible solution or suggestion, help
The problem seemed to be from the version 9.5 in the postgres repositories. I installed 9.6 and everything worked like magic. Had I known there was a stable 9.6 earlier, I wouldn't have wasted 6 hours of my life in something I could have done for 10 minutes. Anyway, I'm happy I have have a working postgres with jsonb field