Mongodb in Docker container, bad shutdown now won't start: exit 100 - mongodb

One of the annoying things about running Mongodb with docker-compose is if you stop your docker-machine without first stopping the compose containers (eg if you're running a local dev copy and reboot your laptop...) then Mongo gets in a bad state.
When you try and start mongodb again it fails:
$ docker-compose logs mongodb
Attaching to dockerenvironment_mongodb_1
mongodb_1 | about to fork child process, waiting until server is ready for connections.
mongodb_1 | forked process: 12
mongodb_1 | ERROR: child process failed, exited with error number 100
The general advice for this seems to be 'delete the lock file'
However I can't do that because the container has stopped already, I can't exec into it.
If I do (i.e. start a fresh container)
$ docker-compose run mongodb
root#a65901f9fc3d:/# ls /data/db
root#a65901f9fc3d:/#
...there's no lock file to delete.
I have tried also
$ docker rm -v dockerenvironment_mongodb_1
but when I start it fails again with Exit 100
I don't know what else to try, can anyone help?

Related

Error while running MongoDB (i.e. the mongod process) manually as a background process

After installing mongodb using Homebrew I started it by running this in the terminal
brew services start mongodb-community#6.0
which worked just fine after stopping it and I further
tried to run MongoDB (i.e. the mongod process) manually as a background process, by running this
mongod --config /opt/homebrew/etc/mongod.conf --fork
and I got this error
about to fork child process, waiting until server is ready for connections.
forked process: 43989
ERROR: child process failed, exited with 14
To see additional information in this output, start without the "--fork" option.```

how to remove non existent docker image

I am trying to run repair command for mongod, but daemon gave conflict error, so i removed the container and again run the repair command and daemon gave again same conflict error, this time i removed container with the container id displayed in the error and daemon says "no such id".
So, can anyone let me know that how can i remove this container so that i can successfully run the repair command.
I am displaying my docker commands as below for reference
Below is my docker ps result
root#ip-172-31-6-252:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
NAMES
1f7bdd83dac0 mongo:latest "/entrypoint.sh mong 23 hours ago Up 23 hours 27017/tcp
cpx.db
11e2123f7e2a centralpx/cpx.server:latest "/run.sh" 2 weeks ago Up 2 weeks 0.0.0.0:80->80/tcp
cpx.server.live
4008c7772f63 centralpx/cpx-ftp "/bin/sh -c '/usr/sb 7 months ago Up 4 months 0.0.0.0:21->21/tcp, 0.0.0.0:30000-30009->30000-30009/tcp cpx.ftp
Below is my docker images result
root#ip-172-31-6-252:~# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
mongo latest 21e69f355287 8 days ago 366.4 MB
centralpx/cpx.server latest 894a3c5fce73 2 weeks ago 429 MB
centralpx/cpx-ftp latest e35ba5efa239 9 months ago 425.5 MB
Now i get attached to cpx.db container and run shutdown command (i need to shutdown before running repair command)
root#ip-172-31-6-252:~# docker exec -it cpx.db /bin/bash
root#1f7bdd83dac0:/# mongod --shutdown
killing process with pid: 1
FATA[0026] Error response from daemon: Container 1f7bdd83dac037293d5086e86a3df7117b4b6eb2a3478d65848643eff9c4d568 is not running: Exited (0) Less than a second ago
root#ip-172-31-6-252:~#
Now below is my repair command
root#ip-172-31-6-252:~# sudo docker run -it -p 28001:27017 --name cpx.db mongo:latest mongod --dbpath /data/db --repair
FATA[0000] Error response from daemon: Conflict. The name "cpx.db" is already in use by container 8b2a8c98971c. You have to delete (or rename) that container to be able to reuse that name.
The above command gives conflict error, so we removed the container "cpx.db", below is the docker commands
root#ip-172-31-6-252:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1f7bdd83dac0 mongo:latest "/entrypoint.sh mong 23 hours ago Exited (0) 3 minutes ago cpx.db
11e2123f7e2a centralpx/cpx.server:latest "/run.sh" 2 weeks ago Up 2 weeks 0.0.0.0:80->80/tcp cpx.server.live
4008c7772f63 centralpx/cpx-ftp "/bin/sh -c '/usr/sb 7 months ago Up 4 months 0.0.0.0:21->21/tcp, 0.0.0.0:30000-30009->30000-30009/tcp cpx.ftp
root#ip-172-31-6-252:~# docker rm cpx.db
cpx.db
root#ip-172-31-6-252:~# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
11e2123f7e2a centralpx/cpx.server:latest "/run.sh" 2 weeks ago Up 2 weeks 0.0.0.0:80->80/tcp cpx.server.live
4008c7772f63 centralpx/cpx-ftp "/bin/sh -c '/usr/sb 7 months ago Up 4 months 0.0.0.0:21->21/tcp, 0.0.0.0:30000-30009->30000-30009/tcp cpx.ftp
Then we again run repair command as the conflicting container is removed and below is our command and output
root#ip-172-31-6-252:~# sudo docker run -it -p 28001:27017 --name cpx.db mongo:latest mongod --dbpath /data/db --repair
FATA[0000] Error response from daemon: Conflict. The name "cpx.db" is already in use by container 8b2a8c98971c. You have to delete (or rename) that container to be able to reuse that name.
Daemon again gave conflict error
We again removed the container with id shown in conflict message as below, and it shows no such id. Please refer below
root#ip-172-31-6-252:~# docker rm 8b2a8c98971c
Error response from daemon: no such id: 8b2a8c98971c
FATA[0000] Error: failed to remove one or more containers
So, can any one help us to remove this container which doesn't exists or can anyone help us to get rid of this issue.
As you pointed the problem in command of instantiating the stopped container so i tried to correct my mistake in the command and run this command
sudo docker mongod --dbpath /data/db --repair
this command gives error
docker: 'mongod' is not a docker command. See 'docker --help'
and if i remove docker from command and run this command
sudo mongod --dbpath /data/db --repair
then it gives error
sudo: mongod: command not found
Can i ask that if my below command is wrong as i am using the stopped container
root#ip-172-31-6-252:~# sudo docker run -it -p 28001:27017 --name cpx.db mongo:latest mongod --dbpath /data/db --repair
Then if i removed the container and running this command then why this command is not executing and giving conflict error?
EDIT:
Based on recent reply and explanation i had updated my command as below just check it and let me know if there is any correction from your side
sudo docker run -it -p 28000:27017 --name cpx.db1 -v /home/ubuntu/data/cpx.db:/data/db -d mongo:latest mongod --dbpath /data/db --repair
My new process will be as below:
1) go to the running container and stop the mongodb and this will automatically stop the running container
2) run the updated repair command as below
sudo docker run -it -p 28000:27017 --name cpx.db1 -v /home/ubuntu/data/cpx.db:/data/db -d mongo:latest mongod --dbpath /data/db --repair
This command will create a new container named as "cpx.db1" and it will use volume to mount the db and run repair command
3) I'll remove this new container "cpx.db1" as i want to use the old one.
Let me know if i am wrong
Thanks a lot in advance.
Thanks
EDIT:
I run the command and i think it worked as it doesn't give any error but it executed very fast so i am confused, i am hereby stating my commands and output for your reference
I entered in db container
docker exec -it cpx.db /bin/bash
I run shutdown command for mongodb
mongod --shutdown
This was the output (as only process was running in container so after killing this one process i was out of the container)
killing process with pid: 1
FATA[0015] Error response from daemon: Container bd910137a3957c79b304dbbbd221317c909e6779de01ed6f780857e3914c577c is not running: Exited (0) Less than a second ago
Then i run the repair command as below
sudo docker run -it -p 28000:27017 --name cpx.db1 -v /home/ubuntu/data/cpx.db:/data/db -d mongo:latest mongod --dbpath /data/db --repair
This was the output
d6b61222c7145f178e95974c87f95cb06fc8aa5c0c1adc929050ca172ab5f73f
Then i start the old container
docker start cpx.db
And db get started
There was no error but i am confused that whether repair command run successfully or not? Can you check my edited post and let me know your views .
Your current problem :
sudo docker run -it -p 28001:27017 --name cpx.db mongo:latest mongod --dbpath /data/db --repair
Here is the problem.
You're not actually reusing the same container but instanciating a new one. Hence the conflict since you're trying to give to a new container an already taken name.
The container you previously stopped is still stopped, and you don't use it with this command line.
This command line also sums up your problem : you don't understand the difference between images and containers. You're trying to manipulate an image where you should actually manipulate a container (the one you stopped).
What you could do :
As far as I know, it is not possible to restart a container with a different process that the one initially used as entrypoint (If I'm wrong I hope someone else will write an answer to explain how to do so).
But first be aware that storing a whole database in a container is a bad design since you can't access it easily, and you lose it at the container removing.
How you started your container isn't clear to me, but if you didn't do so you should store your database in a mounted volume.
This way your database is persistent, you can remove or stop your container (or even not) and still have access to your database (even from the host for instance).
If you use volumes you can stop your container, execute your repair operation from your host -or from another container if you need the same environment- and just then restarting your first container should be enough.
To use volumes :
here is a short example of the volume usage :
docker run -ti -P -v /host/path:/container/path image sh
This way you run a shell and the /host/path directory from your host will be mounted on the /container/path location inside your container : A change in the one (from host or container) appears in the other.
There are more informations in the link I gave.
Containers vs. images, a nuance important to understand :
You seem to be mistaking images and containers (containers are used to run processes), and you really need to understand the difference.
With docker, an image is what you instanciate your containers from.
You can have several containers instanciated from the same image. If an image was a baking tin, then a container would be a cake.
You probably don't have to remove an image but a running container (assuming you have a conflict because you try to run a container using a port already used by an other running container).
How to remove a container :
To list the containers :
docker ps -a (-a permits to list stopped containers as well).
Once you get the container's id, you can pass it to the docker stop command and then remove the stopped container with the docker rm command (You can use the container's name as well).
Removing containers (docker rm) and removing images (docker rmi) are two whole different things.
EDIT :
1.
sudo docker mongod --dbpath /data/db --repair
mongod is not a docker command. (run, build, ps, etc. are on the other hand).
2.
sudo mongod --dbpath /data/db --repair
Here, you don't even use docker. You simply try to run mongod directly on your host.
3.
sudo docker run -it -p 28001:27017 --name cpx.db mongo:latest mongod --dbpath /data/db --repair
From the manual :
docker run :
Run a command in a new container
So you're not reusing the stopped container, just instantiating a new one.
Keep in mind that REMOVING and STOPPING containers aren't the same.
Assuming you already used volumes to store your database files in first place your host, just :
Stop the container.
Launch a new container with a different name to proceed to your repair operation (still use volumes).
This container will exit at the end of the process, you may remove it.
Restart the container from (1), or even launch a new one (you'll need to delete the old one if you want to reuse the same name in such case, it's probably better to do so if you want to use a fresh one).
Also keep in mind that if you didn't use volumes in first place, the database content is probably not directly accessible from your host filesystem (it's contained in your container, so it's not persistent).
If so, the simplest way is probably to create a new database that would be stored on your host so you can have persistent and easily accessible data.
If you do so, use volumes to access this database (the one on your host) from within your container.

Docker container mongod error when starting via ssh

I have installed mongodb on a docker container together with openssh on ubuntu 14.04. The container is running with ssh but when I ssh into the container I get the following error when trying to start mongod.
root#430f9502ba2d:~# service mongod start
Rather than invoking init scripts through /etc/init.d, use the service(8)
utility, e.g. service mongod start
Since the script you are attempting to invoke has been converted to an
Upstart job, you may also use the start(8) utility, e.g. start mongod
Also start mongod does not affect anything.
Tried looking at this also Mongo daemon doesn't run by service mongod start without it helping.
mongod --config /your/path/to/mongod.conf doesn't seem to work also, just locks up.
The error below is standard as of course there is no mongod server running.
root#430f9502ba2d:/# mongo
MongoDB shell version: 2.6.9
connecting to: test
2015-05-07T20:49:56.213+0000 warning: Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused
2015-05-07T20:49:56.214+0000 Error: couldn't connect to server 127.0.0.1:27017 (127.0.0.1), connection attempt failed at src/mongo/shell/mongo.js:146
exception: connect failed
The problem here is your approach. Docker does not have an init system like you are used to on traditional systems. What docker does is replace PID 1 with the process you specify in the CMD or ENTRYPOINT Dockerfile commands. For now, ignore ENTRYPOINT, because it replaces what your CMD is run with (normally, it's /bin/sh -c). You need to instruct docker to start your mongod service in your Dockerfile with the CMD command, like:
CMD usr/bin/mongod
And when you run your container, mongod will be your PID 1. Now, you're probably wondering at this point "But what about my SSH server?" and the answer is: Don't run an SSH server on your docker containers. There are some use cases where running an SSH server is okay, but almost all of the "normal" reasons (debug, C&C, etc) are nullified with the "best practice" for getting a shell on your container:
docker exec -it myContainer /bin/bash
This will drop you into a shell on your running container. The recommendation here for managing configuration and changes in your docker container is to use something like Ansible. However, remember that docker containers are ephemeral, and you shouldn't be restarting services and changing configuration state on them. If you need a config change, change the Dockerfile or config data, and then start a new container. Good luck! Here is a little more information on Dockerizing MongoDB, but keep in mind that the method described there alters the ENTRYPOINT in the Dockerfile, which is a little more involved and requires a better understanding of what's going on in Dockerfiles.
This is really helpful. I was trying to make old Ansible playbooks work with Docker by creating several blank containers and let Ansible do the rest.
It works through command
mongod --dbpath /var/lib/mongodb --smallfiles

How to stop mongo DB in one command

I need to be able to start/stop MongoDB on the cli. It is quite simple to start:
./mongod
But to stop mongo DB, I need to run open mongo shell first and then type two commands:
$ ./mongo
use admin
db.shutdownServer()
So I don't know how to stop mongo DB in one line. Any help?
Starting and Stopping MongoDB is covered in the MongoDB manual. It explains the various options of stopping MongoDB through the shell, cli, drivers etc. It also details the risks of incorrectly stopping MongoDB (such as data corruption) and talks about the different kill signals.
Additionally, if you have installed MongoDB using a package manager for Ubuntu or Debian then you can stop mongodb (currently mongod in ubuntu) as follows:
Upstart: sudo service mongod stop
Sysvinit: sudo /etc/init.d/mongod stop
Or on Mac OS X
Find PID of mongod process using $ top
Kill the process by $ kill <PID> (the Mongo docs have more info on this)
Or on Red Hat based systems:
service mongod stop
Or on Windows if you have installed as a service named MongoDB:
net stop MongoDB
And if not installed as a service (as of Windows 7+) you can run:
taskkill /f /im mongod.exe
To learn more about the problems of an unclean shutdown, how to best avoid such a scenario and what to do in the event of an unclean shutdown, please see: Recover Data after an Unexpected Shutdown.
If you literally want a one line equivalent to the commands in your original question, you could alias:
mongo --eval "db.getSiblingDB('admin').shutdownServer()"
Mark's answer on starting and stopping MongoDB via services is the more typical (and recommended) administrative approach.
mongod --dbpath /path/to/your/db --shutdown
More info at official: http://docs.mongodb.org/manual/tutorial/manage-mongodb-processes/
If the server is running as the foreground process in a terminal, this can be done by pressing
Ctrl-C
Another way to cleanly shut down a running server is to use the shutdown command,
> use admin
> db.shutdownServer();
Otherwise, a command like kill can be used to send
the signal. If mongod has 10014 as its PID, the command would be
kill -2 10014
I followed the official MongoDB documentation for stopping with signals. One of the following commands can be used (PID represents the Process ID of the mongod process):
kill PID
which sends signal 15 (SIGTERM), or
kill -2 PID
which sends signal 2 (SIGINT).
Warning from MongoDB documentation:
Never use kill -9 (i.e. SIGKILL) to terminate a mongod instance.
If you have more than one instance running or you don't care about the PID, you could use pkill to send the signal to all running mongod processes:
pkill mongod
or
pkill -2 mongod
or, much more safer, only to the processes belonging to you:
pkill -U $USER mongod
or
pkill -2 -U $USER mongod
NOTE:
If the DB is running as another user, but you have administrative rights, you have invoke the above commands with sudo, in order to run them. E.g.:
sudo pkill mongod
sudo pkill -2 mongod
PS
Note: I resorted to using signals because mongod --shutdown, although mentioned in the current MongoDB documentation, did not work on my machine (macOS, mongodb v3.4.10, installed with homebrew):
Error parsing command line: unrecognised option '--shutdown'
Update 2022-05-10
meanwhile option --shutdown is marked in the documentation as "Supported on Linux only".
PPS
(macOS specific) Before anyone wonders: no, I could not stop it with command
brew services stop mongodb
because I did not start it with
brew services start mongodb
I had started mongod with a custom command line :-)
Use mongod --shutdown
According to the official doc : manage-mongodb-processes/
:D
create a file called mongostop.bat
save the following code in it
mongo admin --eval "db.shutdownServer()"
run the file mongostop.bat and you successfully have mongo stopped
My special case is:
previously start mongod by:
sudo -u mongod mongod -f /etc/mongod.conf
now, want to stop mongod.
and refer official doc Stop mongod Processes, has tried:
(1) shutdownServer but failed:
> use admin
switched to db admin
> db.shutdownServer()
2019-03-06T14:13:15.334+0800 E QUERY [thread1] Error: shutdownServer failed: {
"ok" : 0,
"errmsg" : "shutdown must run from localhost when running db without auth",
"code" : 13
} :
_getErrorWithCode#src/mongo/shell/utils.js:25:13
DB.prototype.shutdownServer#src/mongo/shell/db.js:302:1
#(shell):1:1
(2) --shutdown still failed:
# mongod --shutdown
There doesn't seem to be a server running with dbpath: /data/db
(3) previous start command adding --shutdown:
sudo -u mongod mongod -f /etc/mongod.conf --shutdown
killing process with pid: 30213
failed to kill process: errno:1 Operation not permitted
(4) use service to stop:
service mongod stop
and
service mongod status
show expected Active: inactive (dead) but mongod actually still running, for can see process from ps:
# ps -edaf | grep mongo | grep -v grep
root 30213 1 0 Feb04 ? 03:33:22 mongod --port PORT --dbpath=/var/lib/mongo
and finally, really stop mongod by:
# sudo mongod -f /etc/mongod.conf --shutdown
killing process with pid: 30213
until now, root cause: still unknown ...
hope above solution is useful for your.
Building on the answer from stennie:
mongo --eval "db.getSiblingDB('admin').shutdownServer();quit()"
I found that mongo was trying to reconnect to the db after the server shut down, which would cause a delay and error messages. Adding quit() after shutdown speeds it up and reduces the messages, but there is still one.
I also want to add context - I'm starting and stopping mongod as part of test cases for other code, so registering as a service does not fit the use case. I am hoping to end up with something that runs on all platforms (this tested in windows right now). I'm using mongod 3.6.9
One liners to start or stop mongodb service using command line;
To start the service use: NET START MONGODB
To stop the service use: NET STOP MONGODB
I use this myself, it does work.
From the given commands I think you're on Linux.
Start MongoDB:
$ sudo service mongod start
mongod start/running, process XXXXX
Check the Status:
$ sudo service mongod status
mongod start/running, process XXXXX
Stop MongoDB:
$ sudo service mongod stop
mongod stop/waiting
Using homebrew (recommended way):
To start:
brew services start mongodb-community
To stop:
brew services stop mongodb-community
I simply did:
quit();
Please note I'm using mongo 3.0.
Mongo
in the terminal window on your mac, press control+c
I use this startup script on Ubuntu.
#!/bin/sh
### BEGIN INIT INFO
# Provides: mongodb
# Required-Sart:
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: mongodb
# Description: mongo db server
### END INIT INFO
. /lib/lsb/init-functions
PROGRAM=/opt/mongo/bin/mongod
MONGOPID=`ps -ef | grep 'mongod' | grep -v grep | awk '{print $2}'`
test -x $PROGRAM || exit 0
case "$1" in
start)
log_begin_msg "Starting MongoDB server"
ulimit -v unlimited.
ulimit -n 100000
/opt/mongo/bin/mongod --fork --quiet --dbpath /data/db --bind_ip 127.0.0.1 --rest --config /etc/mongod.conf.
log_end_msg 0
;;
stop)
log_begin_msg "Stopping MongoDB server"
if [ ! -z "$MONGOPID" ]; then
kill -15 $MONGOPID
fi
log_end_msg 0
;;
status)
;;
*)
log_success_msg "Usage: /etc/init.d/mongodb {start|stop|status}"
exit 1
esac
exit 0
Windows
In PowerShell, it's: Stop-Service MongoDB
Then to start it again: Start-Service MongoDB
To verify whether it's started, run: net start | findstr MongoDB.
Note: Above assumes MongoDB is registered as a service.
Kindly take advantage of the Task Manager provided by your OS for a quick and easy solution. Below is the screengrab from/for Windows 10. Right-click on the highlighted process and select stop. Select start, if already stopped.
Please Note: Internally the commands are doing the same thing which you have to do manually using a GUI (Task Manager), provided by Windows/your OS. Though, this approach to be used for study/practice purpose to get started and you won't be blocked due to this.
To start
sudo /etc/init.d/mongodb start
To stop
sudo /etc/init.d/mongodb stop
CTRL + C
on the windows command line

How do I fix Postgres so it will start after an abrupt shutdown?

Due to a sudden power outage, the Postgres server running on my local machine shut down abruptly. After rebooting, I tried to restart Postgres and I get this error:
$ pg_ctl -D /usr/local/pgsql/data restart
pg_ctl: PID file "/usr/local/pgsql/data/postmaster.pid" does not exist
Is server running?
starting server anyway
server starting
$:/usr/local/pgsql/data$ LOG: database system shutdown was interrupted at 2009-02-28 21:06:16
LOG: checkpoint record is at 2/8FD6F8D0
LOG: redo record is at 2/8FD6F8D0; undo record is at 0/0; shutdown FALSE
LOG: next transaction ID: 0/1888104; next OID: 1711752
LOG: next MultiXactId: 2; next MultiXactOffset: 3
LOG: database system was not properly shut down; automatic recovery in progress
LOG: redo starts at 2/8FD6F918
LOG: record with zero length at 2/8FFD94A8
LOG: redo done at 2/8FFD9480
LOG: could not fsync segment 0 of relation 1663/1707047/1707304: No such file or directory
FATAL: storage sync failed on magnetic disk: No such file or directory
LOG: startup process (PID 5465) exited with exit code 1
LOG: aborting startup due to startup process failure
There is no postmaster.pid file in the data directory. What possibly could be the reason for this sort of behavior and of course what is the way out?
You'd need to pg_resetxlog. Your database can be in an inconsistent state after this though, so dump it with pg_dumpall, recreate and import back.
A cause for this could be:
You have not turned off hardware
write cache on disk, which often
prevents the OS from making sure data is written before it reports successful write to application. Check with
hdparm -I /dev/sda
If it shows "*" before "Write cache" then this could be the case. Source of PostgreSQL has a program src/tools/fsync/test_fsync.c, which tests speed of syncing data with disk. Run it - if it reports all times shorter than, say, 3 seconds than your disk is lying to OS - on a 7500rpm disks a test of 1000 writes to the same place would need at least 8 seconds to complete (1000/(7500rpm/60s)) as it can only write once per route. You'd need to edit this test_fsync.c if your database is on another disk than /var/tmp partition - change
#define FSYNC_FILENAME "/var/tmp/test_fsync.out"
to
#define FSYNC_FILENAME "/usr/local/pgsql/data/test_fsync.out"
Your disk is failing and has a bad block, check with badblocks.
You have a bad RAM, check with memtest86+ for at least 8 hours.
Reading a few similar messages in the archives of the PostgreSQL
mailing list ("storage sync failed on magnetic disk: No such file or
directory") seems to indicate that there is a very serious hardware
trouble, much worse than a simple power failure. You may have to prepare yourself to restore from backups.
Had db corruption too, my actions
docker run -it --rm -v /path/to/db:/var/lib/postgresql/data postgres:10.3 bash
su - postgres
/usr/lib/postgresql/10/bin/pg_resetwal -D /var/lib/postgresql/data -f
I had this same problem and I was about to dump, reinstall and import from db dump (a really painfull process), however I just tried this as the last resource and it worked!
brew services start postgresql
Then I restarted and that was it.
Run start instead of restart.
Execute the below command:
$pg_ctl -D /usr/local/pgsql/data start
Had this problem a couple of times, when my laptop turned off unexpectedly, when on very low battery while running PSQL in the background.
My solution after searching all over was, Hard delete and Reinstall, then import data from db dump.
Steps for Mac with brew to uninstall and reinstall psql 9.6
brew uninstall postgresql#9.6
rm -rf rm -rf /usr/local/var/postgresql#9.6
rm -rf .psql.local .psql_history .psqlrc.local l.psqlrc .pgpass
brew install postgresql#9.6
echo 'export PATH="/usr/local/opt/postgresql#9.6/bin:$PATH"' >> ~/.bash_profile
source ~/.bash_profile
brew services start postgresql#9.6
createuser -s postgres
createuser {ENTER_YOUR_USER_HERE} --interactive
As others stated, a stop + start instead of a restart worked for me. In a Docker environment this would be:
docker stop <container_name>
docker start <container_name>
or when using Docker Compose:
docker-compose stop
docker-compose start