Related
I'm new to Postgres.
I updated the Dockerfile I use and successfully installed Postgresql on it. (My image runs Ubuntu 16.04 and I'm using Postgres 9.6.)
Everything worked fine until I tried to move the database to a Volume with docker-compose (that was after making a copy of the container's folder with cp -R /var/lib/postgresql /somevolume/.)
The issue is that Postgres just keeps crashing, as witnessed by supervisord:
2017-07-26 18:55:38,346 INFO exited: postgresql (exit status 1; not expected)
2017-07-26 18:55:39,355 INFO spawned: 'postgresql' with pid 195
2017-07-26 18:55:40,430 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-07-26 18:55:40,763 INFO exited: postgresql (exit status 1; not expected)
2017-07-26 18:55:41,767 INFO spawned: 'postgresql' with pid 197
2017-07-26 18:55:42,841 INFO success: postgresql entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-07-26 18:55:43,179 INFO exited: postgresql (exit status 1; not expected)
(and so on…)
Logs
It's not clear to me what's happening as /var/log/postgresql remains empty.
chown?
I suspect it has to do with the user. If I compare the data folder inside the container and the copy I made of it to the volume, the only difference is that the original is owned by postgres while the copy is owned by root.
I tried running chown -R postgres:postgres on the copy. The operation was performed successfully, however postmaster.pid remains owned by root and I think that would be the issue.
Questions
How can I get more information about the cause of the crash?
How can I make it so that postmaster.id be owned by postgres ?
Should I consider running postgres with root instead?
Any hint welcome.
EDIT: links to the Dockerfile and the docker-compose.xml.
I'll answer my own question:
Logs & errors
What made matters more complicated was that I was not getting any specific error message.
To change that, I disabled the [program:postgresql] section in supervisord and, instead, started postgres manually from the command-line (thanks to Miguel Marques for setting me on the right track with his comment.)
Then I finally got some useful error messages:
2017-08-02 08:27:09.134 UTC [37] LOG: could not open temporary statistics file "/var/run/postgresql/9.6-main.pg_stat_tmp/global.tmp": No such file or directory
Fixing the configuration
I fixed the error above with this, eventually adding them to my Dockerfile:
mkdir -p /var/run/postgresql/9.6-main.pg_stat_tmp
chown postgres.postgres /var/run/postgresql/9.6-main.pg_stat_tmp -R
(Kudos to this guy for the fix.)
To make the data permanent, I also had to do this, for the volume to be accessible by postgres:
mkdir -p /var/lib/postgresql/9.6/main
chmod 700 /var/lib/postgresql/9.6/main
I also used initdb to initialize the data directory. BEWARE! This will erase any data found in that folder. Like so:
rm -R /var/lib/postgresql/9.6/main/*
ls /var/lib/postgresql/9.6/main/
/usr/lib/postgresql/9.6/bin/initdb -D /var/lib/postgresql/9.6/main
Testing
After the above, I could finally run postgres properly. I used this command to run it and test from the command-line:
su postgres
/usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/main -c config_file=/etc/postgresql/9.6/main/postgresql.conf # as per the Docker docs
To test, I kept it running and then, from another prompt, checked everything ran fine with this:
su postgres
psql
CREATE TABLE cities ( name varchar(80), location point );
INSERT INTO cities VALUES ('San Francisco', '(-194.0, 53.0)');
select * from cities; # repeat this command after restarting the container to check that the data does persist
…making sure to restart the container and test again to check the data did persist.
And then finally restored the [program:postgresql] section in supervisord, rebuilt the image and restarted the container, making sure everything ran fine (in particular supervisord: tail /var/log/supervisor/supervisord.log), which it did.
(The command I used inside of supervisord.conf is also /usr/lib/postgresql/9.6/bin/postgres -D /var/lib/postgresql/9.6/main -c config_file=/etc/postgresql/9.6/main/postgresql.conf, as per this Docker article and other postgres+supervisord examples. Other options would have been using pg_ctl or an init.d script, but it's not clear to me why/when one would use those.)
I spent a lot of time on this. Hopefully the detailed answer will help someone down the line.
P.S.: I did end up producing a minimal example of my issue. If that can help anyone, here they are: Dockerfile, supervisord.conf and docker-compose.yml.
I do not know if this would be another way to achieve the same result (I'm new on Docker and Postgres too), but have you try the oficial repository image for Postgres (https://hub.docker.com/_/postgres/)?
I'm getting the data out of the container setting the environment variable PGDATA to '/var/lib/postgresql/data/pgdata' and binding this to an external volume on the run command:
docker run --name bd_TEST --network=my_network --restart=always -e POSTGRES_USER="superuser" -e POSTGRES_PASSWORD="myawesomepass" -e PGDATA="/var/lib/postgresql/data/pgdata" -v /var/local/db_data:/var/lib/postgresql/data/pgdata -itd -p 5432:5432 postgres:9.6
When the volume is empty, all the files are created by the image startup script, and if they already exist, the database start to used it.
From past experience I can see what may be a problem. I can't say if this will help but it is worth a try.
I would have added this as a comment, but I can't because my rep isn't hight enough.
I've spied a couple problems with how you have structured your statements in your Dockerfile. You have installed various things multiple times and also updated sporadically through the code. In my own files i've noticed that this can lead to somewhat random behaviour of my services and installation because of the different layers.
This may not seem to solve your problem directly, but cleaning up your file as is outlined in the best practices has solved many Dockerfile problems for me in the past.
One of the first places upon finding such problems is to start here at the best practices for RUN. This has helped me solve tricky problems in the past and I hope it'll solve or at least make it easier.
Pay special attention to this part:
After building the image, all layers are in the Docker cache. Suppose you later modify apt-get install by adding extra package:
FROM ubuntu:14.04
RUN apt-get update
RUN apt-get install -y curl nginx
Docker sees the initial and modified instructions as identical and reuses the cache from previous
steps. As a result the apt-get update is NOT executed because the
build uses the cached version. Because the apt-get update is not run,
your build can potentially get an outdated version of the curl and
nginx packages.
After reading this I would start by consolidating all your dependencies.
In my case, having the same error, I debugged it until I found out:
the disk was full and I increased the diskspace to solve this.
(stupid error, easy fix - maybe reading this here helps someone not wasting time)
also linking this questiong for other options:
Supervisord "exit status 1 not expected" running php script
https://serverfault.com/questions/537773/supervisor-process-exits-with-exit-status-1-not-expected/1076115#1076115
I'm trying to run psql on my Vagrant machine, but I get this error:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting connections on
Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Note: Vagrant 1.9.2
Box: ubuntu/trusty64, https://atlas.hashicorp.com/ubuntu/boxes/trusty64
EDIT
Commands I've used in order to install and run postgres:
sudo apt-get update
sudo apt-get install postgresql
sudo su postgres
psql -d postgres -U postgres
I've had this same issue, related to the configuration of my pg_hba.conf file (located in /etc/postgresql/9.6/main). Please note that 9.6 is the postgresql version I am using.
The error itself is related to a misconfiguration of postgresql, which causes the server to crash before it starts.
I would suggest following these instructions:
Certify that postgresql service is running, using sudo service postgresql start
Run pg_lsclusters from your terminal
Check what is the cluster you are running, the output should be something like:
Version - Cluster Port Status Owner Data directory
9.6 ------- main -- 5432 online postgres /var/lib/postgresql/9.6/main
Disregard the '---' signs, as they are being used there only for alignment.
The important information are the version and the cluster. You can also check whether the server is running or not in the status column.
Copy the info from the version and the cluster, and use like so:
pg_ctlcluster <version> <cluster> start, so in my case, using version 9.6 and cluster 'main', it would be pg_ctlcluster 9.6 main start
If something is wrong, then postgresql will generate a log, that can be accessed on /var/log/postgresql/postgresql-<version>-main.log, so in my case, the full command would be sudo nano /var/log/postgresql/postgresql-9.6-main.log.
The output should show what is the error.
2017-07-13 16:53:04 BRT [32176-1] LOG: invalid authentication method "all"
2017-07-13 16:53:04 BRT [32176-2] CONTEXT: line 90 of configuration file "/etc/postgresql/9.5/main/pg_hba.conf"
2017-07-13 16:53:04 BRT [32176-3] FATAL: could not load pg_hba.conf
Fix the errors and restart postgresql service through sudo service postgresql restart and it should be fine.
I have searched a lot to find this, credit goes to this post.
Best of luck!
I had the same issue but non of the answers here helped.
How I fixed it (mac)
Try to start postgresql with pg_ctl -D /usr/local/var/postgres start
Look for the Error Message that says something like FATAL: could not open directory "pg_tblspc": No such file or directory.
Create that missing directory mkdir /usr/local/var/postgres/pg_tblspc
Repeat from step one until you created all missing directories
When done and then trying to start postgresql again it might say FATAL: lock file "postmaster.pid" already exists
Delete postmaster.pid: rm /usr/local/var/postgres/postmaster.pid
Start postgres with: pg_ctl -D /usr/local/var/postgres start
Done ✨
These two steps solved it for me on Mac:
rm /usr/local/var/postgres/postmaster.pid
brew services restart postgresql
For M1 Macs:
rm /opt/homebrew/var/postgres/postmaster.pid
brew services restart postgresql
In case you face this issue (reported by #luckyguy73): psql: FATAL: database "postgresql" does not exist
You can run
brew postgresql-upgrade-database
to fix it.
I am just posting this for anyone who is feeling lost and hopeless as I did when I found this question. It seems that sometimes by editing some psotgresql-related config files, one can accidentally change the permissions of the file:
Note how pg_hba.conf belongs to root, and users cannot even read it. This causes postgres to not be able to open this file and therefore not be able to start the server, throwing the error seen in the original question.
By running
sudo chmod +r pg_hba.conf
I was able to make this file once again accessible to the postgres user and then after running
sudo service postgresql start
Was able to get the server running again.
WARNING: This will remove the database
Use command:
rm -rf /usr/local/var/postgres && initdb /usr/local/var/postgres -E utf8
WARNING: This will remove the database
Within zsh:
rm -rf /usr/local/var/postgres && initdb /usr/local/var/postgres -E utf8
This is the only thing that worked for me after countless hours trouble shooting.
Does the /etc/postgresql/9.6/main/postgresql.conf show that port being assigned? On my default Xubuntu Linux install, mine showed port = 5433 for some reason as best as I can remember, but I did comment out the line in that same file that said listen_addresses = 'localhost' and uncommented the line listen_addresses = '*'. So maybe start and check there. Hope that helps.
This works for me:
pg_ctl -D /usr/local/var/postgresql#9.6 stop;
brew services stop postgresql#9.6;
brew services start postgresql#9.6;
I was able to solve the issue by running:
sudo systemctl start postgresql#9.5-main
In my case Postgres was managed through Homebrew Services (i.e. started via brew services start postgresql#10 Terminal command for the Postgres 10 that I use), and for that setup I had to discover a couple of essential steps to do before I could apply any advice in this thread. So I want to share just that piece as it may help someone who has the same setup.
NOTE: all the commands below are to be run in Terminal.
To give a quick background: After upgrading to macOS Big Sur I discovered that Postgres wasn't working and running psql results in the error mentioned in the original question above. I tried to start Postgres (via the brew services start postgresql#10 command), this resulted in a message Service postgresql#10 already started. If I tried to restart it (via the brew services restart postgresql#10) I got a message that it was stopped and then started successfully. But! This was a misleading message, and I spent quite some time searching for config issues etc. before discovering that the service was not started successfully in reality.
So, the way to investigate this is:
Make sure the service is started by running the brew services start postgresql#10 (the latter argument may be different depending on what your Homebrew package name is e.g. postgresql#12 or plain postgresql).
Run brew services list. This is the command that gives you the true state of the service. In my case it said that Postgres' status is error:
Name Status User Plist
postgresql#10 error Denis /Users/Denis/Library/LaunchAgents/homebrew.mxcl.postgresql#10.plist
redis started Denis /Users/Denis/Library/LaunchAgents/homebrew.mxcl.redis.plist
To investigate further open the config shown in the same command output in Plist column (I used nano /Users/Denis/Library/LaunchAgents/homebrew.mxcl.postgresql#10.plist to check it).
In the config look for the StandardErrorPath key, and open the file located in the value of that key, i.e. in the <string> tag following the key. In my case it was /usr/local/var/log/postgresql#10.log.
Open that log and check the latest error (I used nano /usr/local/var/log/postgresql#10.log and then Alt+/ to go to the end of the file).
Voila. That is the real error to investigate, which you can then look for in the previous answers or google for. I'm not covering the rest here, as the goal of this answer is to show how to find the real error if you use Homebrew Services to run Postgres. (In my case it was the lock file "postmaster.pid" already exists already covered in the previous answers, plus the path to check right in the error message, in my case /usr/local/var/postgresql#10).
In my case it was the lockfile postmaster.id that was not deleted properly during the last system crash that caused the issue. Deleting it with sudo rm /usr/local/var/postgres/postmaster.pid and restarting Postgres solved the problem.
I recommend you should clarify port that postgres.
In my case I didn't know which port postgres was running on.
lsof -i | grep 'post'
then you can know which port is listening.
psql -U postgres -p "port_in_use"
with port option, might be answer. you can use psql.
If non of the above answers are not working for you, then please try this one,
Many people have mentioned many solutions to this problem! But all of them forgot that, the same problem will arise when your disk don't have enough space or the space you are assigned for postgres is full
Check your system storage, if its full free up some space! then restart your postgres by sudo service postgresql restart or do a stop and start sudo service posgresql stop then sudo service postgresql start
This will solve the issue, it solved for me
I occasionally have the same issue but mostly after macOS upgrades. Shutting down and migrating to the new version usually fixes it for me(make changes according to your version). So first upgrade your postgresql
brew services stop postgresql#12
brew services start postgresql#12
brew postgresql-upgrade-database
This is mostly a temporary fix but since I couldn't find a better solution this works for me.
Update: If the issue says that another postmaster is running then try removing it from that location(your postmaster.pid location will be displayed to you)
rm /usr/local/var/postgres/postmaster.pid
Open your database manager and execute this script
update pg_database set datallowconn = 'true' where datname = 'your_database_name';
I had the same error when I create the SQL db in a VM. I had changed the default value of /etc/postgresql/9.3/main/postgresql.conf shared_buffers = 200MB to 75% of my total RAM. Well, I forgot to actually allocate that RAM in the VM. When I gave the command to make a new database, I received the same error.
Powered off, gave the baby its bottle (RAM) and presto, it worked.
The same thing happened to me as I had changed something in the /etc/hosts file. After changing it back to 127.0.0.1 localhost it worked for me.
just reinstall your pgsql with direct version sudo apt-get install postgresql-9.5 (u must remove the package before install new one)
I had similar problems just a while ago. After trying more than 5 suggestions I decided to go back to the basics and start from the beginning. Which meant removing my postgresql installation and following this guide upon re-installing postgresql. https://help.ubuntu.com/lts/serverguide/postgresql.html
Ubuntu 20
This Problem happened to me, as ubuntu pre-installed version of Postgresql-9.6 server was always down and after trying all the above answers it didn't start.
Solution:
I installed another version of Postgresql which is postgresql-13, using this command: sudo apt install postgresql it will install the latest version of postgresql.
I see if the server is online or down using this command: pg_lsclusters if the new version of postgresql is online, we will proceed to remove the old version of postgresql.
we will see all packages that are installed related to postgresql, using this command: dpkg -l | grep postgresql
Remove the old version, which is here postgresql-9.6. Using this command:
sudo apt-get --purge remove postgresql-9.6 postgresql-client-9.6 replace 9.6 with your old version number. Final remaining packages related to the latest Version 13:
Restart your postgresql latest version server, which is here postgresql-13. Using this command: sudo systemctl restart postgresql#13-main replace 13 in the command with your latest version number.
Now, if you try psql command you will get an error related to your user, as in the image:
To Remove the above error, The installation procedure created a user account called postgres that is associated with the default Postgres role, to switch over to the postgres account use this command: sudo -u postgres psql this command will log you into the interactive Postgres session. You can also set your password for this user using this command \password postgres.
Then change the Port to the deafult port of postgresql, which is 5432 as all application will try to connect to postgresql using this port by default, using this command: sudo nano /etc/postgresql/13/main/postgresql.conf, it will open postgresql configuration file, then search for port and change it to 5432. After that you need to restart the server using this command sudo systemctl restart postgresql#13-main. Note, Replace 13 in the command with your latest version.
If you want to create your own User/Role, use this command: sudo -u postgres createuser --interactive. The script will prompt you with some choices, as in the image and based on your responses, it will execute the correct Postgres commands to create a user to your specifications.
Tutorial: For more information on postgresql related commands
I couldn't connect using the psql command and kept getting the error Cannot connect to Server: No such file or directory.
Step 1: Check the status of the Postgres cluster
$ pg_lsclusters
Step 2: Restart the Postgres cluster
$ sudo pg_ctlcluster 12 main start
Make sure to replace 12 with your version of Postgres
Step 3: Check again and connect
$ pg_lsclusters
$ sudo -i -u postgres
$ psql
I got this error when I restored my database from last pg_basebackup backup file. After that when I tried to connect database(psql), I was getting the same error. The error was resolved, when I updated pg_hba.conf file and wherever "peer" authentication was there I replaced that with "md5" and then restarted postgres services. After that, the problem was resolved.
This error happened to me after my mac mini got un-plugged (so forced shutdown), and all I had to do to fix it was restart
I have the same issue with postgres 11 on my mac. I get this error every time after restart
psql: could not connect to server: No such file or directory
Is the server running locally and accepting connections on
Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
As a temporary fix I do
brew services stop postgresql#11
brew services start postgresql#11
My problem happened after a brew update so I've ran
pg_ctl -D /usr/local/var/postgres start
and I've got this result:
FATAL: database files are incompatible with server 2021-07-07 13:27:21.692 CEST [70896] DETAIL: The data directory was initialized by PostgreSQL version 12, which is not compatible with this version 13.2. stopped waiting
I've ran
brew postgresql-upgrade-database
FATAL: could not load server certificate file "/etc/ssl/certs/ssl-cert-snakeoil.pem": No such file or directory
LOG: database system is shut down
pg_ctl: could not start server
I have a missing ssl-cert-snakeoil.pem file so i created it using make-ssl-cert generate-default-snakeoil --force-overwrite And it worked fine.
In my case, I had to run journalctl -xe, and it showed that my disk was full. I then deleted some .gz items from /var/log and I could again restart the postgresql.
I'm on Kali Linux. I had to remove the brew version of postgresql with
brew uninstall postgresql
sudo -u postgres psql got me into root postgres
Simply running these commands from the installation steps in the official PostgreSQL docs worked for me (I'm on Fedora 33):
# Optionally initialize the database and enable automatic start:
sudo /usr/pgsql-13/bin/postgresql-13-setup initdb
sudo systemctl enable postgresql-13
sudo systemctl start postgresql-13
RHEL Installation link
kali users pls do this
sudo service postgresql restart
This is really frustrating me. I have a DO VPS with ubuntu 14.04 (64) installed.
I installed VestaCP as control panel on that and have hosted some PHP based personal project.
I also installed meteor on it but never used, now when I am trying to create a project and run it ('meteor create rt' then 'cd rt' then 'meteor')
It is giving the following error :
[[[[[ /home/admin/code/rt ]]]]]
=> Started proxy.
Unexpected mongo exit code 1. Restarting.
Unexpected mongo exit code 1. Restarting.
Unexpected mongo exit code 1. Restarting.
Can't start Mongo server.
root#RD:/home/admin/code/rt#
Could anyone please help? Please ask me for more informations if required.
**** EDIT ****
I created a fresh DigitalOcean server and it is giving the same error on that. Some issue with Digital Ocean? File System of Digital Ocean? I am confused. I am trying it on different flavours of Linux and same result. All are fresh linux installations.
I finally got the solution. Posting it here for others.
This was the problem as a few environment variables which mongodb looks for while starting was not set
Set the variables LC_ALL and LANG and it works fine (mostly setting LC_ALL will do)
first, type locale command and see the output, you will see that it will say something about LC_ALL not set.
Now, add these two lines in /etc/environment and it worked.
LC_ALL=en_US.UTF-8
LANG=en_US.UTF-8
This solution is for Ubuntu 12.04 +
Other variants may require similar work.
Unexpected mongo exit code 1 is still an uncaught exception as far as i think.
You can try by updating your c/c++ compilers uptodate. Have a look here.
It says :
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get install gcc-4.6
sudo apt-get install g++-4.6
All the best!
So we have narrowed the issue down to meteor's mongo installation on your box (though I think we were pretty sure of this all along). Let's attempt to debug that a bit. The way I have done this in the past is to try to open meteor's mongo with the mongod provided by meteor. You will perform these procedures without running the meteor server. This should give you the warning that is causing Mongo to exit. First you need to find this. In my instance installed on Mint (which should be similar to Ubuntu) it is at:
~/.meteor/packages/meteor-tool/.1.1.3.4sddkj++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/mongodb/bin/mongod
You can look at that location on your Ubuntu box or you can run something like this to get the location:
find ~/.meteor/ -name mongod
Once you find the location then go to the directory of your meteor project you are attempting to run and in that directory you should find this location:
<your meteor project>/.meteor/local
cd into that directory and run the following command:
~/.meteor/packages/meteor-tool/.1.1.3.4sddkj++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/mongodb/bin/mongod --dbpath ./
From there you can analyze the output (or update the question so we can see the output) and this should show you the mongo error you are receiving on startup and allow you to fix it.
I've got the same issues trying to start a meteor app and exactly the mongodb server is being terminated in an unexpectly manner. Generally the virtual linux server from some dealers like the one you mentioned are coming without a swap partition (check in /etc/fstab file) so if you have not enough memory to allocate MongoDB server then meteor app can't be started. You can create a swap partition or instal swapspace
sudo apt-get install swapspace
After that I was able to start the meteor app... Just be patient as swap memory is not as faster as RAM.
Since due some "smart" StackExchange policy I cannot up-vote or comment to working solution...)
Quoted answer works also on Digital Ocean on CentOS 7 x64 vmlinuz-3.10.0-123.8.1.el7.x86_64
first, type locale command and see the output, you will see that it will say something about LC_ALL not set.
Now, add these two lines in /etc/environment and it worked.
I changed the locale setting to match my needs.
Fixed on my Debian 8 with the following bash command, (use sudo if needed)
localedef -i en_US -f UTF-8 en_US.UTF-8
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I'm using the PostgreSQL database for my Ruby on Rails application (on Mac OS X 10.9).
Are there any detailed instructions on how to upgrade PostgreSQL database?
I'm afraid I will destroy the data in the database or mess it up.
Assuming you've used home-brew to install and upgrade Postgres, you can perform the following steps.
Stop current Postgres server:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Initialize a new 10.1 database:
initdb /usr/local/var/postgres10.1 -E utf8
run pg_upgrade (note: change bin version if you're upgrading from something other than below):
pg_upgrade -v \
-d /usr/local/var/postgres \
-D /usr/local/var/postgres10.1 \
-b /usr/local/Cellar/postgresql/9.6.5/bin/ \
-B /usr/local/Cellar/postgresql/10.1/bin/
-v to enable verbose internal logging
-d the old database cluster configuration directory
-D the new database cluster configuration directory
-b the old PostgreSQL executable directory
-B the new PostgreSQL executable directory
Move new data into place:
cd /usr/local/var
mv postgres postgres9.6
mv postgres10.1 postgres
Restart Postgres:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Check /usr/local/var/postgres/server.log for details and to make sure the new server started properly.
Finally, re-install the rails pg gem
gem uninstall pg
gem install pg
I suggest you take some time to read the PostgreSQL documentation to understand exactly what you're doing in the above steps to minimize frustrations.
Despite all answers above, here goes my 5 cents.
It works on any OS and from any-to-any postgres version.
Stop any running postgres instance;
Install the new version and start it; Check if you can connect to the new version as well;
Change old version's postgresql.conf -> port from 5432 to 5433;
Start the old version postgres instance;
Open a terminal and cd to the new version bin folder;
Run pg_dumpall -p 5433 -U <username> | psql -p 5432 -U <username>
Stop old postgres running instance;
Here is the solution for Ubuntu users
First we have to stop postgresql
sudo /etc/init.d/postgresql stop
Create a new file called /etc/apt/sources.list.d/pgdg.list and add below line
deb http://apt.postgresql.org/pub/repos/apt/ utopic-pgdg main
Follow below commands
wget -q -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install postgresql-9.4
sudo pg_dropcluster --stop 9.4 main
sudo /etc/init.d/postgresql start
Now we have everything, just need to upgrade it as below
sudo pg_upgradecluster 9.3 main
sudo pg_dropcluster 9.3 main
That's it. Mostly upgraded cluster will run on port number 5433. Check it with below command
sudo pg_lsclusters
If you are using homebrew and homebrew services, you can probably just do:
brew services stop postgresql
brew upgrade postgresql
brew postgresql-upgrade-database
brew services start postgresql
I think this might not work completely if you are using advanced postgres features, but it worked perfectly for me.
Update: This process is the same for upgrading 9.5 through at least 11.5; simply modify the commands to reflect versions 9.6 and 10, where 9.6 is the old version and 10 is the new version. Be sure to adjust the "old" and "new" directories accordingly, too.
I just upgraded PostgreSQL 9.5 to 9.6 on Ubuntu and thought I'd share my findings, as there are a couple of OS/package-specific nuances of which to be aware.
(I didn't want to have to dump and restore data manually, so several of the other answers here were not viable.)
In short, the process consists of installing the new version of PostgreSQL alongside the old version (e.g., 9.5 and 9.6), and then running the pg_upgrade binary, which is explained in (some) detail at https://www.postgresql.org/docs/9.6/static/pgupgrade.html .
The only "tricky" aspect of pg_upgrade is that failure to pass the correct value for an argument, or failure to be logged-in as the correct user or cd to the correct location before executing a command, may lead to cryptic error messages.
On Ubuntu (and probably Debian), provided you are using the "official" repo, deb http://apt.postgresql.org/pub/repos/apt/ xenial-pgdg main, and provided you haven't changed the default filesystem paths or runtime options, the following procedure should do the job.
Install the new version (note that we specify the 9.6, explicitly):
sudo apt install postgresql-9.6
Once installation succeeds, both versions will be running side-by-side, but on different ports. The installation output mentions this, at the bottom, but it's easy to overlook:
Creating new cluster 9.6/main ...
config /etc/postgresql/9.6/main
data /var/lib/postgresql/9.6/main
locale en_US.UTF-8
socket /var/run/postgresql
port 5433
Stop both server instances (this will stop both at the same time):
sudo systemctl stop postgresql
Switch to the dedicated PostgreSQL system user:
su postgres
Move into his home directory (failure to do this will cause errors):
cd ~
pg_upgrade requires the following inputs (pg_upgrade --help tells us this):
When you run pg_upgrade, you must provide the following information:
the data directory for the old cluster (-d DATADIR)
the data directory for the new cluster (-D DATADIR)
the "bin" directory for the old version (-b BINDIR)
the "bin" directory for the new version (-B BINDIR)
These inputs may be specified with "long names", to make them easier to visualize:
-b, --old-bindir=BINDIR old cluster executable directory
-B, --new-bindir=BINDIR new cluster executable directory
-d, --old-datadir=DATADIR old cluster data directory
-D, --new-datadir=DATADIR new cluster data directory
We must also pass the --new-options switch, because failure to do so results in the following:
connection to database failed: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/lib/postgresql/.s.PGSQL.50432"?
This occurs because the default configuration options are applied in the absence of this switch, which results in incorrect connection options being used, hence the socket error.
Execute the pg_upgrade command from the new PostgreSQL version:
/usr/lib/postgresql/9.6/bin/pg_upgrade --old-bindir=/usr/lib/postgresql/9.5/bin --new-bindir=/usr/lib/postgresql/9.6/bin --old-datadir=/var/lib/postgresql/9.5/main --new-datadir=/var/lib/postgresql/9.6/main --old-options=-cconfig_file=/etc/postgresql/9.5/main/postgresql.conf --new-options=-cconfig_file=/etc/postgresql/9.6/main/postgresql.conf
Logout of the dedicated system user account:
exit
The upgrade is now complete, but, the new instance will bind to port 5433 (the standard default is 5432), so keep this in mind if attempting to test the new instance before "cutting-over" to it.
Start the server as normal (again, this will start both the old and new instances):
systemctl start postgresql
If you want to make the new version the default, you will need to edit the effective configuration file, e.g., /etc/postgresql/9.6/main/postgresql.conf, and ensure that the port is defined as such:
port = 5432
If you do this, either change the old version's port number to 5433 at the same time (before starting the services), or, simply remove the old version (this will not remove your actual database content; you would need to use apt --purge remove postgresql-9.5 for that to happen):
apt remove postgresql-9.5
The above command will stop all instances, so you'll need to start the new instance one last time with:
systemctl start postgresql
As a final point of note, don't forget to consider pg_upgrade's good advice:
Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade so,
once you start the new server, consider running:
./analyze_new_cluster.sh
Running this script will delete the old cluster's data files:
./delete_old_cluster.sh
The user manual covers this topic in depth. You can:
pg_upgrade in-place; or
pg_dump and pg_restore.
If in doubt, do it with dumps. Don't delete the old data directory, just keep it in case something goes wrong / you make a mistake; that way you can just go back to your unchanged 9.3 install.
For details, see the manual.
If you're stuck, post a detailed question explaining how you're stuck, where, and what you tried first. It depends a bit on how you installed PostgreSQL too, as there are several different "distributions" of PostgreSQL for OS X (unfortunately). So you'd need to provide that info.
Standing on the shoulders of the other poor creatures trodding through this muck, I was able to follow these steps to get back up and running after an upgrade to Yosemite:
Assuming you've used home-brew to install and upgrade Postgres, you can perform the following steps.
Stop current Postgres server:
launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Initialize a new 9.4 database:
initdb /usr/local/var/postgres9.4 -E utf8
Install postgres 9.3 (as it was no longer present on my machine):
brew install homebrew/versions/postgresql93
Add directories removed during Yosemite upgrade:
mkdir -p /usr/local/var/postgres/{pg_tblspc,pg_twophase,pg_stat_tmp}/touch /usr/local/var/postgres/{pg_tblspc,pg_twophase,pg_stat_tmp}/.keep
run pg_upgrade:
pg_upgrade -v -d /usr/local/var/postgres -D /usr/local/var/postgres9.4 -b /usr/local/Cellar/postgresql93/9.3.5/bin/ -B /usr/local/Cellar/postgresql/9.4.0/bin/
Move new data into place:
cd /usr/local/var
mv postgres postgres9.3
mv postgres9.4 postgres
Restart Postgres:
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist
Check /usr/local/var/postgres/server.log for details and to make sure the new server started properly.
Finally, re-install related libraries?
pip install --upgrade psycopg2
gem uninstall pg
gem install pg
Looks like the solution has been baked into Homebrew now:
$ brew info postgresql
...
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
....
On Windows I kept facing different errors messages when trying to use pg_upgrade.
Saved a lot of time for me to just:
Backup DB
Uninstall all copies of PostgreSQL
Install 9.5
Restore DB
This did it for me.
https://gist.github.com/dideler/60c9ce184198666e5ab4
Short and to the point. I honestly don't aim to understand the guts of PostgreSQL, I want to get stuff done.
My solution for upgrading from Postgresql 11 to Postgresql 12 on Windows 10 is the following.
As a first remark you will need to be able stop and start the Postgresql service. You can do this by the following commands in Powershell.
Start:
pg_ctl start -D “d:\postgresql\11\data”
Stop:
pg_ctl stop -D “d:\postgresql\11\data”
Status:
pg_ctl status -D “d:\postgresql\11\data”
It would be wise to make a backup before doing the upgrade. The Postgresql 11 instance must be running. Then to copy the globals do
pg_dumpall -U postgres -g -f d:\bakup\postgresql\11\globals.sql
and then for each database
pg_dump -U postgres -Fc <database> > d:\backup\postgresql\11\<database>.fc
or
pg_dump -U postgres -Fc -d <database> -f d:\backup\postgresql\11\<database>.fc
If not already done install Postgresql 12 (as Postgresql 11 is also installed this will be on port 5433)
Then to do the upgrade as follows:
1) Stop Postgresql 11 service (see above)
2) Edit the postgresql.conf file in d:\postgresql\12\data and change port = 5433 to port = 5432
3) Edit the windows user environment path (windows start then type env) to point to Postgresql 12 instead of Postresql 11
4) Run upgrade by entering the following command.
pg_upgrade `
-b “c:\program files\postgresql\11\bin” `
-B “c:\program files\postgresql\12\bin” `
-d “d:\postgresql\11\data” `
-D “d:\postgresql\12\data” --username=postgres
(In powershell use backtick (or backquote) ` to continue the command on the next line)
5) and finally start the new Postgresql 12 service
pg_ctl start -D “d:\postgresql\12\data”
My solution was to do a combination of these two resources:
https://gist.github.com/tamoyal/2ea1fcdf99c819b4e07d
and
http://www.gab.lc/articles/migration_postgresql_9-3_to_9-4
The second one helped more then the first one. Also to not, don't follow the steps as is as some are not necessary.
Also, if you are not being able to backup the data via postgres console, you can use alternative approach, and backup it with pgAdmin 3 or some other program, like I did in my case.
Also, the link: https://help.ubuntu.com/stable/serverguide/postgresql.html
Helped to set the encrypted password and set md5 for authenticating the postgres user.
After all is done, to check the postgres server version run in terminal:
sudo -u postgres psql postgres
After entering the password run in postgres terminal:
SHOW SERVER_VERSION;
It will output something like:
server_version
----------------
9.4.5
For setting and starting postgres I have used command:
> sudo bash # root
> su postgres # postgres
> /etc/init.d/postgresql start
> /etc/init.d/postgresql stop
And then for restoring database from a file:
> psql -f /home/ubuntu_username/Backup_93.sql postgres
Or if doesn't work try with this one:
> pg_restore --verbose --clean --no-acl --no-owner -h localhost -U postgres -d name_of_database ~/your_file.dump
And if you are using Rails do a bundle exec rake db:migrate after pulling the code :)
For Mac via homebrew:
brew tap petere/postgresql,
brew install <formula> (eg: brew install petere/postgresql/postgresql-9.6)
Remove old Postgres:
brew unlink postgresql
brew link -f postgresql-9.6
If any error happen, don't forget to read and follow brew instruction in each step.
Check this out for more: https://github.com/petere/homebrew-postgresql
On Windows 10 since I had npm, I installed rimraf package. npm install rimraf -g
Backup all your databases one by one using command pg_dump -U $username --format=c --file=$mydatabase.sqlc $dbname
Then Installed Latest PostgreSQL Version i.e. 11.2 which prompted me to use port 5433 this time.
Followed by Uninstall of older versions of PostgreSQL mine was 10. Note the uninstaller may give a warning of not deleting folder C:\PostgreSQL\10\data. That's why we have the next step using rimraf to permanently delete the folder and it's sub-folders.
change into PostgreSQL install directory and ran the command rimraf 10. 10 is a directory name. Note use your older version of PostgreSQL i.e. 9.5 or something.
Now add C:\PostgreSQL\pg11\bin, C:\PostgreSQL\pg11\lib into the Windows environmental variables. Note my new installed version is 11 thus why I am using pg11.
Navigate to C:\PostgreSQL\data\pg11 then open postgresql.conf edit port = 5433 to port = 5432
That's it. Open cmd and type psql -U postgres
You can now restore all your backed databases one by one using the command pg_restore -U $username --dbname=$databasename $filename
I think this is best link for your solution to update postgres to 9.6
https://sandymadaan.wordpress.com/2017/02/21/upgrade-postgresql9-3-9-6-in-ubuntu-retaining-the-databases/
Ever since I changed the dbpath in /etc/mongodb.conf, MongoDB has not been starting automatically, nor using the new dbpath. Prior to the change, MongoDB would be running when the computer started and I was able to simply run the command mongo to get into the console or start my Ruby on Rails server with no issues.
After I made the modification (in order to switch to a new drive with more space), the only way I can get everything to work is by manually running the command mongod --config /etc/mongodb.conf. If I don't run that, it doesn't seem like the service is running and running without the --config option give me the following error: ERROR: dbpath (/data/db/) does not exist. even though the config file says nothing about data/db.
Some other notes:
In addition to changing /etc/mongodb.conf, I moved all files out of /var/lib/mongodb and into /home/nick/appdev/mongodb.
I changed the owner and group from root to nick. Tried changing it back, but it didn't seem to fix anything.
I'm running Ubuntu 12.10 Beta 1 and Mongo 2.2.0 with Ruby on Rails 3.2.8
A late follow up on the above question...
I had a similar issue after moving the db to an ebs on ec2.
It turns out that just running mongod still directs the dbpath to /data/db/ (which exists).
The /etc/mongodb.conf is completely ignored unless specifically directed to.
I manage to work around this by using the directive --config or just the --dbpath(both work)
But was left wondering where does mongod takes it defaults from...?!
I was unable to locate and override these defaults anywhere.
Anyone ?
Note:
I am really annoyed by this behaviour of mongod...This is just bad design,and bad documentation.
It turns out that I needed to set the owner and group to mongodb. When I transferred the files to the new directory, I had set the owner and group to my user account nick and also tried root, neither of which worked.
To do so, here are the following commands:
sudo chown mongodb /home/nick/appdev/mongodb -R
sudo chgrp mongodb /home/nick/appdev/mongodb -R
To confirm that it worked, you can check the file permissions with:
ls -l /home/nick/appdev/mongodb
After checking all permission in the data, journal and log folders as suggested, my problem was solved by giving permission to a lock file in the /tmp folder
sudo chown mongod:mongod mongodb-27017.sock
I was running it as a AWS Amazon Linux instance. I figured that out by executing as the mongod user as below, and then, researching the error code. It might be useful for other troubleshooting.
sudo -S -u mongod mongod -f /etc/mongod.conf
MongoDB 1.6 is very old and the latest production version is 2.2, which contains a large amount of bug fixes and enhancements since 1.6.
Am I correct that you haven't installed 1.6 via a package manager such as yum or aptitude? I don't believe there are packages for 1.6 at present afaik. Therefore, mongod is behaving correctly as you have not started MongoDB with a control script.
Please see this link on configuration file options.