My OpenProject management software is installed with default postgresql 10.
Currently the postgresql DB is 12, It is having lot of new features.
I want to upgrade my Postgres DB without losing the data in the DB.
My system is ubuntu 18.04 and hosted openproject.
I searched the internet and could not find a step by step to upgrade postgresql.
Can you please guide me to install new DB and all data should be in the new DB.
thanks for your help.
A) First create a backup of all the databases for that (You can continue from B if you dont need a backup)
Log in as postgres user
sudo su postgres
Create a backup .sql file for all the data you have in all the databases
pg_dumpall > backup.sql
B) Upgrade to PostgreSQL12
update packages and install postgres 12
sudo apt-get update
sudo apt-get install postgresql-12 postgresql-server-dev-12
Stop the postgresql service
sudo systemctl stop postgresql.service
migrate the data
/usr/lib/postgresql/12/bin/pg_upgrade \
--old-datadir=/var/lib/postgresql/10/main \
--new-datadir=/var/lib/postgresql/12/main \
--old-bindir=/usr/lib/postgresql/10/bin \
--new-bindir=/usr/lib/postgresql/12/bin \
--old-options '-c config_file=/etc/postgresql/10/main/postgresql.conf' \
--new-options '-c config_file=/etc/postgresql/12/main/postgresql.conf'
Switch to regular user
exit
Swap the ports the old and new postgres versions.
#change port to 5432
sudo vim /etc/postgresql/12/main/postgresql.conf
#change port to 5433
sudo vim /etc/postgresql/10/main/postgresql.conf
Start the postgresql service
sudo systemctl start postgresql.service
Log in as postgres user
sudo su postgres
Check your new postgres version
psql -c "SELECT version();"
Run the generated new cluster script
./analyze_new_cluster.sh
Return as a normal(default user) user and cleanup up the old version's mess
#uninstalls postgres packages
sudo apt-get remove postgresql-10 postgresql-server-dev-10
#removes the old postgresql directory
sudo rm -rf /etc/postgresql/10/
#login as postgres user
sudo su postgres
#delete the old cluster data
./delete_old_cluster.sh
Congrads! Your postgresql version is now upgraded, If everything works well in B, we dont have to apply the backup as we have already migrated the data from the older version to the newer version, the backup is just in case if anything goes wrong.
NOTE: Change the postgresql.conf and pg_hba.conf as per your requirement
PS: Feel free to comment your issues, suggestions or anyother modifications you would like to suggest
Backup the database
psql --version sudo -u postgres psql pg_dumpall > alldbs.sql
(this command will backup all databases from the postgresql db)
Then exit from postgres user, and:
Inside a terminal run these commands:
sudo systemctl stop postgres
sudo apt-get install -y postgresql-12 postgresql-server-dev-12 postgresql-contrib-12 libpq-dev postgresql-12-hypopg
sudo pg_dropcluster 12 main --stop
sudo pg_upgradecluster 10 main
sudo pg_dropcluster 10 main --stop
restart the postgresql service:
sudo systemctl restart postgresql
login to the postgres:
su - postgres
to check the version:
psql --version
I have done using the above steps and I could update the DBand restore all data.
Postgresql Upgrade using pg_upgrade
Step - 1 (Find the locale language and encoding of source Postgresql server)
Log in to Postgresql database as postgres user
Execute the query SHOW LC_COLLATE;
Step - 2 ( Stop the existing/source Postgresql server)
Find the running Postgresql service using $ sudo systemctl list-units | grep postgres
Stop the service $ sudo service postgresql-<source-version>.service stop
Step - 3 (Install target Postgresql server)
Configure repos and install (lot of tutorials available in google)
Step - 4 (Update the locale language and encoding)
$ /usr/pgsql-<target-version>/bin/initdb -D /var/lib/pgsql/<target-version>/data --locale=<add-your-encoding>
Step - 5 (Check the source to target upgrade has any potential issues)
Run the command as postgres user, $sudo su postgres
$ /usr/pgsql-<target-version>/bin/pg_upgrade --old-bindir /usr/pgsql-<source-version>/bin --new-bindir /usr/pgsql-<target-version>/bin --old-datadir /var/lib/pgsql/<source-version>/data --new-datadir /var/lib/pgsql/12/data --link --jobs=2 --check
If it's ok we can move to the next step, If not fix the issues before proceeding to the next step
Expected output: Clusters are compatible
Step - 6 (Upgrade the source to target Postgresql version using the link option)
link option is pretty faster than the migration
Run the command as postgres user, $sudo su postgres
$ /usr/pgsql-<target-version>/bin/pg_upgrade --old-bindir /usr/pgsql-<source-version>/bin --new-bindir /usr/pgsql-<target-version>/bin --old-datadir /var/lib/pgsql/<source-version>/data --new-datadir /var/lib/pgsql/<target-version>/data --link
Expected output: Upgrade Complete
Thanks.
ENJOY !!!
I am using Ubuntu server. I am trying to install postgresql on it. when I try to start postgres using
sudo service postgresql start
it result me into
*No PostgreSQL clusters exist; see "man pg_createcluster"
and when I try to create cluster using command pg_createcluster --start 9.6 main (*9.6 is my postgres version)
The result is as follows
Creating new PostgreSQL cluster 9.6/main ...
/usr/lib/postgresql/9.6/bin/initdb -D /var/lib/postgresql/9.6/main --
auth-local peer --auth-host md5
initdb: could not look up effective user ID 108: Permission denied
Error: initdb failed
I am not able to figure out what is the issue please help.
It looks like your /etc/passwd is not readable. I'm not sure which distro would require it, but it is a problem for initdb.
I believe fast solution would be running as root chmod 644 /etc/passwd
And follow with
/usr/lib/postgresql/9.6/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start
command
My database is over 600 GB and my current volume is only 1 TB, so that probably limits my options.
My config files are here:
/etc/postgresql/9.6/main
My database is here:
/mnt/1TB/postgresql/9.6/main
Edit -
This guide worked for me. The only addition I needed to make was to download libicu55 manually and install it, and I had to grant postgres 1777 permission for my /tmp/ folder. I was also saving the data folder to a different drive, so I had to use the command:
pg_upgradecluster -m upgrade 10 main /mnt/1TB/postgresql/10
https://gist.github.com/delameko/bd3aa2a54a15c50c723f0eef8f583a44
A Step-by-Step Guide
Make a backup. Make sure that your database is not being updated.
pg_dumpall > outputfile
Install Postgres 10. Follow instructions on this page: https://www.postgresql.org/download/linux/ubuntu/
Then run sudo apt-get install postgresql-10. A newer version will be installed side-by-side with the earlier version.
Run pg_lsclusters:
Ver Cluster Port Status Owner Data directory Log file
9.6 main 5432 online postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log
10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log
There already is a cluster main for 10 (since this is created by default on package installation). This is done so that a fresh installation works out of the box without the need to create a cluster first, but of course it clashes when you try to upgrade 9.6/main when 10/main also exists. The recommended procedure is to remove the 10 cluster with pg_dropcluster and then upgrade with pg_upgradecluster.
Stop the 10 cluster and drop it:
sudo pg_dropcluster 10 main --stop
Stop all processes and services writing to the database. Stop the database:
sudo systemctl stop postgresql
Upgrade the 9.6 cluster:
sudo pg_upgradecluster -m upgrade 9.6 main
Start PostgreSQL again
sudo systemctl start postgresql
Run pg_lsclusters . Your 9.6 cluster should now be "down", and the 10 cluster should be online at 5432:
Ver Cluster Port Status Owner Data directory Log file
9.6 main 5433 down postgres /var/lib/postgresql/9.6/main /var/log/postgresql/postgresql-9.6-main.log
10 main 5432 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log
First, check that everything works fine. After that, remove the 9.6 cluster:
sudo pg_dropcluster 9.6 main --stop
Some notes on pg_upgradecluster
This guide works fine for upgrading from 9.5 to 10.1. When upgrading from an older version, consider omitting -m upgrade on the step #6:
sudo pg_upgradecluster 9.6 main
If you have a really big cluster, you may use pg_upgradecluster with a --link option, so that the upgrade will be in-place. However, this is dangerous — you can lose the cluster in an event of failure. Just don't use this option if not necessary, as -m upgrade is already fast enough.
Based on:
Docs: Upgrading a PostgreSQL Cluster
Gist #1: delameko/upgrade-postgres-9.5-to-9.6.md
Gist #2: johanndt/upgrade-postgres-9.3-to-9.5.md
What happens if I interrupt or cancel pg_upgradecluster?
Ubuntu manpage for pg_upgradecluster
Update
This guide works fine for upgrading from 9.6 to 11 and from 10 to 11, as well as from 10 to 13.
Almost worked out of the box. I'm on 17.10 artful. While you can put that in the .deb file, it wont work - Postgres only update the non LTS releases if they break the LTS release. So put Zesty into that deb and you are good to go.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I'm using the PostgreSQL database for my Ruby on Rails application (on Mac OS X 10.9).
Are there any detailed instructions on how to upgrade PostgreSQL database?
I'm afraid I will destroy the data in the database or mess it up.
Assuming you've used home-brew to install and upgrade Postgres, you can perform the following steps.
Stop current Postgres server:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Initialize a new 10.1 database:
initdb /usr/local/var/postgres10.1 -E utf8
run pg_upgrade (note: change bin version if you're upgrading from something other than below):
pg_upgrade -v \
-d /usr/local/var/postgres \
-D /usr/local/var/postgres10.1 \
-b /usr/local/Cellar/postgresql/9.6.5/bin/ \
-B /usr/local/Cellar/postgresql/10.1/bin/
-v to enable verbose internal logging
-d the old database cluster configuration directory
-D the new database cluster configuration directory
-b the old PostgreSQL executable directory
-B the new PostgreSQL executable directory
Move new data into place:
cd /usr/local/var
mv postgres postgres9.6
mv postgres10.1 postgres
Restart Postgres:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Check /usr/local/var/postgres/server.log for details and to make sure the new server started properly.
Finally, re-install the rails pg gem
gem uninstall pg
gem install pg
I suggest you take some time to read the PostgreSQL documentation to understand exactly what you're doing in the above steps to minimize frustrations.
Despite all answers above, here goes my 5 cents.
It works on any OS and from any-to-any postgres version.
Stop any running postgres instance;
Install the new version and start it; Check if you can connect to the new version as well;
Change old version's postgresql.conf -> port from 5432 to 5433;
Start the old version postgres instance;
Open a terminal and cd to the new version bin folder;
Run pg_dumpall -p 5433 -U <username> | psql -p 5432 -U <username>
Stop old postgres running instance;
Here is the solution for Ubuntu users
First we have to stop postgresql
sudo /etc/init.d/postgresql stop
Create a new file called /etc/apt/sources.list.d/pgdg.list and add below line
deb http://apt.postgresql.org/pub/repos/apt/ utopic-pgdg main
Follow below commands
wget -q -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install postgresql-9.4
sudo pg_dropcluster --stop 9.4 main
sudo /etc/init.d/postgresql start
Now we have everything, just need to upgrade it as below
sudo pg_upgradecluster 9.3 main
sudo pg_dropcluster 9.3 main
That's it. Mostly upgraded cluster will run on port number 5433. Check it with below command
sudo pg_lsclusters
If you are using homebrew and homebrew services, you can probably just do:
brew services stop postgresql
brew upgrade postgresql
brew postgresql-upgrade-database
brew services start postgresql
I think this might not work completely if you are using advanced postgres features, but it worked perfectly for me.
Update: This process is the same for upgrading 9.5 through at least 11.5; simply modify the commands to reflect versions 9.6 and 10, where 9.6 is the old version and 10 is the new version. Be sure to adjust the "old" and "new" directories accordingly, too.
I just upgraded PostgreSQL 9.5 to 9.6 on Ubuntu and thought I'd share my findings, as there are a couple of OS/package-specific nuances of which to be aware.
(I didn't want to have to dump and restore data manually, so several of the other answers here were not viable.)
In short, the process consists of installing the new version of PostgreSQL alongside the old version (e.g., 9.5 and 9.6), and then running the pg_upgrade binary, which is explained in (some) detail at https://www.postgresql.org/docs/9.6/static/pgupgrade.html .
The only "tricky" aspect of pg_upgrade is that failure to pass the correct value for an argument, or failure to be logged-in as the correct user or cd to the correct location before executing a command, may lead to cryptic error messages.
On Ubuntu (and probably Debian), provided you are using the "official" repo, deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main, and provided you haven't changed the default filesystem paths or runtime options, the following procedure should do the job.
Install the new version (note that we specify the 9.6, explicitly):
sudo apt install postgresql-9.6
Once installation succeeds, both versions will be running side-by-side, but on different ports. The installation output mentions this, at the bottom, but it's easy to overlook:
Creating new cluster 9.6/main ...
config /etc/postgresql/9.6/main
data /var/lib/postgresql/9.6/main
locale en_US.UTF-8
socket /var/run/postgresql
port 5433
Stop both server instances (this will stop both at the same time):
sudo systemctl stop postgresql
Switch to the dedicated PostgreSQL system user:
su postgres
Move into his home directory (failure to do this will cause errors):
cd ~
pg_upgrade requires the following inputs (pg_upgrade --help tells us this):
When you run pg_upgrade, you must provide the following information:
the data directory for the old cluster (-d DATADIR)
the data directory for the new cluster (-D DATADIR)
the "bin" directory for the old version (-b BINDIR)
the "bin" directory for the new version (-B BINDIR)
These inputs may be specified with "long names", to make them easier to visualize:
-b, --old-bindir=BINDIR old cluster executable directory
-B, --new-bindir=BINDIR new cluster executable directory
-d, --old-datadir=DATADIR old cluster data directory
-D, --new-datadir=DATADIR new cluster data directory
We must also pass the --new-options switch, because failure to do so results in the following:
connection to database failed: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/lib/postgresql/.s.PGSQL.50432"?
This occurs because the default configuration options are applied in the absence of this switch, which results in incorrect connection options being used, hence the socket error.
Execute the pg_upgrade command from the new PostgreSQL version:
/usr/lib/postgresql/9.6/bin/pg_upgrade --old-bindir=/usr/lib/postgresql/9.5/bin --new-bindir=/usr/lib/postgresql/9.6/bin --old-datadir=/var/lib/postgresql/9.5/main --new-datadir=/var/lib/postgresql/9.6/main --old-options=-cconfig_file=/etc/postgresql/9.5/main/postgresql.conf --new-options=-cconfig_file=/etc/postgresql/9.6/main/postgresql.conf
Logout of the dedicated system user account:
exit
The upgrade is now complete, but, the new instance will bind to port 5433 (the standard default is 5432), so keep this in mind if attempting to test the new instance before "cutting-over" to it.
Start the server as normal (again, this will start both the old and new instances):
systemctl start postgresql
If you want to make the new version the default, you will need to edit the effective configuration file, e.g., /etc/postgresql/9.6/main/postgresql.conf, and ensure that the port is defined as such:
port = 5432
If you do this, either change the old version's port number to 5433 at the same time (before starting the services), or, simply remove the old version (this will not remove your actual database content; you would need to use apt --purge remove postgresql-9.5 for that to happen):
apt remove postgresql-9.5
The above command will stop all instances, so you'll need to start the new instance one last time with:
systemctl start postgresql
As a final point of note, don't forget to consider pg_upgrade's good advice:
Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade so,
once you start the new server, consider running:
./analyze_new_cluster.sh
Running this script will delete the old cluster's data files:
./delete_old_cluster.sh
The user manual covers this topic in depth. You can:
pg_upgrade in-place; or
pg_dump and pg_restore.
If in doubt, do it with dumps. Don't delete the old data directory, just keep it in case something goes wrong / you make a mistake; that way you can just go back to your unchanged 9.3 install.
For details, see the manual.
If you're stuck, post a detailed question explaining how you're stuck, where, and what you tried first. It depends a bit on how you installed PostgreSQL too, as there are several different "distributions" of PostgreSQL for OS X (unfortunately). So you'd need to provide that info.
Standing on the shoulders of the other poor creatures trodding through this muck, I was able to follow these steps to get back up and running after an upgrade to Yosemite:
Assuming you've used home-brew to install and upgrade Postgres, you can perform the following steps.
Stop current Postgres server:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Initialize a new 9.4 database:
initdb /usr/local/var/postgres9.4 -E utf8
Install postgres 9.3 (as it was no longer present on my machine):
brew install homebrew/versions/postgresql93
Add directories removed during Yosemite upgrade:
mkdir -p /usr/local/var/postgres/{pg_tblspc,pg_twophase,pg_stat_tmp}/touch /usr/local/var/postgres/{pg_tblspc,pg_twophase,pg_stat_tmp}/.keep
run pg_upgrade:
pg_upgrade -v -d /usr/local/var/postgres -D /usr/local/var/postgres9.4 -b /usr/local/Cellar/postgresql93/9.3.5/bin/ -B /usr/local/Cellar/postgresql/9.4.0/bin/
Move new data into place:
cd /usr/local/var
mv postgres postgres9.3
mv postgres9.4 postgres
Restart Postgres:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Check /usr/local/var/postgres/server.log for details and to make sure the new server started properly.
Finally, re-install related libraries?
pip install --upgrade psycopg2
gem uninstall pg
gem install pg
Looks like the solution has been baked into Homebrew now:
$ brew info postgresql
...
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
....
On Windows I kept facing different errors messages when trying to use pg_upgrade.
Saved a lot of time for me to just:
Backup DB
Uninstall all copies of PostgreSQL
Install 9.5
Restore DB
This did it for me.
https://gist.github.com/dideler/60c9ce184198666e5ab4
Short and to the point. I honestly don't aim to understand the guts of PostgreSQL, I want to get stuff done.
My solution for upgrading from Postgresql 11 to Postgresql 12 on Windows 10 is the following.
As a first remark you will need to be able stop and start the Postgresql service. You can do this by the following commands in Powershell.
Start:
pg_ctl start -D “d:\postgresql\11\data”
Stop:
pg_ctl stop -D “d:\postgresql\11\data”
Status:
pg_ctl status -D “d:\postgresql\11\data”
It would be wise to make a backup before doing the upgrade. The Postgresql 11 instance must be running. Then to copy the globals do
pg_dumpall -U postgres -g -f d:\bakup\postgresql\11\globals.sql
and then for each database
pg_dump -U postgres -Fc <database> > d:\backup\postgresql\11\<database>.fc
or
pg_dump -U postgres -Fc -d <database> -f d:\backup\postgresql\11\<database>.fc
If not already done install Postgresql 12 (as Postgresql 11 is also installed this will be on port 5433)
Then to do the upgrade as follows:
1) Stop Postgresql 11 service (see above)
2) Edit the postgresql.conf file in d:\postgresql\12\data and change port = 5433 to port = 5432
3) Edit the windows user environment path (windows start then type env) to point to Postgresql 12 instead of Postresql 11
4) Run upgrade by entering the following command.
pg_upgrade `
-b “c:\program files\postgresql\11\bin” `
-B “c:\program files\postgresql\12\bin” `
-d “d:\postgresql\11\data” `
-D “d:\postgresql\12\data” --username=postgres
(In powershell use backtick (or backquote) ` to continue the command on the next line)
5) and finally start the new Postgresql 12 service
pg_ctl start -D “d:\postgresql\12\data”
My solution was to do a combination of these two resources:
https://gist.github.com/tamoyal/2ea1fcdf99c819b4e07d
and
http://www.gab.lc/articles/migration_postgresql_9-3_to_9-4
The second one helped more then the first one. Also to not, don't follow the steps as is as some are not necessary.
Also, if you are not being able to backup the data via postgres console, you can use alternative approach, and backup it with pgAdmin 3 or some other program, like I did in my case.
Also, the link: https://help.ubuntu.com/stable/serverguide/postgresql.html
Helped to set the encrypted password and set md5 for authenticating the postgres user.
After all is done, to check the postgres server version run in terminal:
sudo -u postgres psql postgres
After entering the password run in postgres terminal:
SHOW SERVER_VERSION;
It will output something like:
server_version
----------------
9.4.5
For setting and starting postgres I have used command:
> sudo bash # root
> su postgres # postgres
> /etc/init.d/postgresql start
> /etc/init.d/postgresql stop
And then for restoring database from a file:
> psql -f /home/ubuntu_username/Backup_93.sql postgres
Or if doesn't work try with this one:
> pg_restore --verbose --clean --no-acl --no-owner -h localhost -U postgres -d name_of_database ~/your_file.dump
And if you are using Rails do a bundle exec rake db:migrate after pulling the code :)
For Mac via homebrew:
brew tap petere/postgresql,
brew install <formula> (eg: brew install petere/postgresql/postgresql-9.6)
Remove old Postgres:
brew unlink postgresql
brew link -f postgresql-9.6
If any error happen, don't forget to read and follow brew instruction in each step.
Check this out for more: https://github.com/petere/homebrew-postgresql
On Windows 10 since I had npm, I installed rimraf package. npm install rimraf -g
Backup all your databases one by one using command pg_dump -U $username --format=c --file=$mydatabase.sqlc $dbname
Then Installed Latest PostgreSQL Version i.e. 11.2 which prompted me to use port 5433 this time.
Followed by Uninstall of older versions of PostgreSQL mine was 10. Note the uninstaller may give a warning of not deleting folder C:\PostgreSQL\10\data. That's why we have the next step using rimraf to permanently delete the folder and it's sub-folders.
change into PostgreSQL install directory and ran the command rimraf 10. 10 is a directory name. Note use your older version of PostgreSQL i.e. 9.5 or something.
Now add C:\PostgreSQL\pg11\bin, C:\PostgreSQL\pg11\lib into the Windows environmental variables. Note my new installed version is 11 thus why I am using pg11.
Navigate to C:\PostgreSQL\data\pg11 then open postgresql.conf edit port = 5433 to port = 5432
That's it. Open cmd and type psql -U postgres
You can now restore all your backed databases one by one using the command pg_restore -U $username --dbname=$databasename $filename
I think this is best link for your solution to update postgres to 9.6
https://sandymadaan.wordpress.com/2017/02/21/upgrade-postgresql9-3-9-6-in-ubuntu-retaining-the-databases/