MacOS restart problem with postgresql due to postmaster.pid file sticking around - postgresql

When hydro dies and my MacOS machine reboots, PG never restarts nice.
brew services restart postgresql
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
At this point PG remains dead.
in /usr/local/var/postgres the postmaster.pid file is still around, presumably from the time the machine died. I was thinking the brew services restart command would scan for the existance of that and remove it, but no. So I have to manually remove that and then run the command again, at which point PG is once again responsive and working.
Is there not an easier way with MacOS to ensure a restart is a restart without manually having to delete the old postmaster.pid file?

Related

brew services. Cant start service. get "Bootstrap failed: 5: Input/output error"

running brew services start mongodb-community which yields:
Bootstrap failed: 5: Input/output error
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/<myUserName>/Library/LaunchAgents/homebrew.mxcl.mongodb-community.plist` exited with 5.
when I run launchctl load -w /Users/<myUserName>/Library/LaunchAgents/homebrew.mxcl.mongodb-community.plist
yields
Load failed: 5: Input/output error
I recently upgraded macOs to BigSur 11.5.2.
Ive uninstall and reinstalled homebrew and xcode.
try this
brew services restart <mongodb-community#4.4>
I am able to run mongodb through homebrew. I believe this solved my issue:
launchctl unload -w ~/Library/LaunchAgents/homebrew.mxcl.mongodb-community.plist
followed by
launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mongodb-community.plist
I saw this issue with MySQL and the solution was my current profile was not the owner of my homebrew folder.
I had to change it because I have a second profile set up for another work space, but they share the same homebrew path.
I would suggest anyone who sees this error try running
sudo chown -R $(whoami) $(brew --prefix)/*
And then seeing if that will resolve it. Again I didn't have this issue with MongoDB, but I saw this exact error with MySQL when my profile didn't own the brew folder.
Try this if you tried to restart your service, but it still doesn't work
stop your service
brew services stop mongodb-community
restart your macOS
start your service again
brew services start mongodb-community
To see the real errors, you need to run the service without brew
I had a similar problem, but instead of mongodb, I could not start apache(httpd).
$ brew services start httpd
Bootstrap failed: 5: Input/output error
Try re-running the command as root for richer errors.
Error: Failure while executing; `/bin/launchctl bootstrap gui/501
/Users/john/Library/LaunchAgents/homebrew.mxcl.httpd.plist` exited with
5.
FAIL: 1
Unfortunately brew doesn't show the user what causes this error. At first I tried to use the tips from the answers in this thread, but neither reinstalling apache nor unload -w/load -w helped.
I also found nothing in the brew logs. I decided to investigate apache itself. To do this I ran apache without brew:
$ /opt/homebrew/Cellar/httpd/2.4.51/bin/apachectl start
httpd: Syntax error on line 66 of /opt/homebrew/etc/httpd/httpd.conf: Cannot load /opt/homebrew/Cellar/php#5.6/5.6.40_3/lib/httpd/modules/libphp5.so into server: dlopen(/opt/homebrew/Cellar/php#5.6/5.6.40_3/lib/httpd/modules/libphp5.so, 0x000A): tried: '/opt/homebrew/Cellar/php#5.6/5.6.40_3/lib/httpd/modules/libphp5.so' (no such file), '/usr/local/lib/libphp5.so' (no such file), '/usr/lib/libphp5.so' (no such file)
FAIL: 1
Then it was finally clear what was the reason why httpd service was not starting. It turned out that after the php upgrade, the php version had changed and I forgot to write the new php version in the apache configuration file.
I quickly fixed this and apache started up.
It seems that the mongodb-community is already started. So do restart it instead of starting, by the following command:
brew services restart mongodb-community
I faced a similar problem with postgresql 11 and restarting the service did not help. The issue arose when my MacBook restarted abruptly.
After checking the ports and trying out all the possible solutions, I decided to look into the logs.
tail -n 100 /usr/local/var/log/postgresql#11.log
This gave me the last 100 lines of the log where I found this:
FATAL: lock file "postmaster.pid" already exists
HINT: Is another postmaster (PID 479) running in data directory "/usr/local/var/postgresql#11"?
Clearly, the issue was with postmaster.pid so I deleted it:
rm /usr/local/var/postgresql#11/postmaster.pid
Started the postgres service again:
brew services start postgresql#11
and voila! It worked.
I had similar error in regard to postgres.
$ brew services start postgresql
Bootstrap failed: 5: Input/output error
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/yrk/Library/LaunchAgents/homebrew.mxcl.postgresql.plist` exited with 5.
I've tried restart and it looked promising:
$ brew services restart postgres
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
But there still was a problem when trying to connect:
$ psql -d postgres -U apiuser
psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
Is the server running locally and accepting connections on that socket?
Checking postgres log as mentioned by #mansoor.khan helped, in my case it was:
tail /usr/local/var/log/postgres.log.
And the issue was clear:
$ tail /usr/local/var/log/postgres.log
2022-05-19 14:18:46.029 CEST [32374] FATAL: database files are incompatible with server
2022-05-19 14:18:46.029 CEST [32374] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.3.
2022-05-19 14:18:56.071 CEST [32676] FATAL: database files are incompatible with server
2022-05-19 14:18:56.071 CEST [32676] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.3.
2022-05-19 14:19:06.111 CEST [32885] FATAL: database files are incompatible with server
2022-05-19 14:19:06.111 CEST [32885] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.3.
2022-05-19 14:19:16.128 CEST [33102] FATAL: database files are incompatible with server
2022-05-19 14:19:16.128 CEST [33102] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.3.
2022-05-19 14:19:26.166 CEST [33434] FATAL: database files are incompatible with server
2022-05-19 14:19:26.166 CEST [33434] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.3.
Now the problem is: how to revert to previous version or upgrade old data to new version of postgres DB, but this is for another topic.
run something like launchctl remove homebrew.mxcl.mongodb-community before the load command solved my problem
I had this when upgrading from 4.4 to 5.0 and I suspect it’s similar for you. Lets assume you were upgrading 5.x to 6.x then the resolution would be ...
check your mongo log, e.g.
vi /usr/local/var/log/mongodb/mongo.log
Look for any errors relating to file/directory permissions and fix those, e.g. though I’m not quite sure whether ‘wheel’ or ‘admin’ is right.
sudo chown yourloginname:wheel thedirectory/*
Roll back to your previous working version of mongo (e.g. 5.x)
Start Mongo 5
Start a Mongo Shell and type
db.adminCommand( { setFeatureCompatibilityVersion: "5.0" } )
Check the logs to verify the database has detected 5.0
Uninstall 5.x
Re-install 6.x
Restart / start 6.x
It should now work?
In my case, another httpd process was running on the Mac. Killed with monitor tool allowed me to start httpd.
I had a similar problem on mysql.
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/gerd/Library/LaunchAgents/homebrew.mxcl.mysql.plist` exited with 5.
In my case the mysqld miss the osx network connection permission. After restart OSX it appear the network permissions dialog. I granted the permission to msqld and the problem was fixed!
Happened with redis. Realized there was a bind setting in the config, binding the server to an incorrect ip address.
I faced the same problem. This commands helped me
brew uninstall rabbitmq
brew uninstall erlang
brew install rabbitmq
brew services restart rabbitmq
You may run into this if the service is already running, but as another user.
For example, you may have run sudo brew services start example and are now running brew services info example. It tells you it's not running and you think "hm, it should be running" and enter brew services start example.
Potentially example is already bound to, say, port 7777 and now you ask for another program to be bound to the same port, leading to a failing service start.
In short: If you are binding to a port, check if the same service is already running as another user and bound to the same port.

Postgres data still "in use" after server stop

I am running postgresql in a docker container. Now I wanted to add checksums to the database cluster. So I stopped the docker container and waited some time. But the pg_checksums tool is still complaining:
pg_checksums: error: cluster must be shut down
There is no postgres or similar running any longer, with docker or not.
Renaming the file postmaster.pid did not change anything.
What du I need to do to convince pg_checksums that it can savely work on the cluster data?
I'm using postgresql 12 and Docker version 19.03.13, build 4484c46d9d on a CentOS 8 machine.
You need to shutdown the database cleanly. Shutting down the container itself apparently did not do that. A tool which does not attach to PostgreSQL's shared memory has no way to know whether a database has crashed, or is still running. So you need a clean shutdown.

postgresql#10-main.service failed to load

I have been racking my brain on this issue, and probably have tried every possible solution ( fix, purge, reinstall), but postgresql doesn't start
rupin#linuxbox:~$ sudo /etc/init.d/postgresql restart
[sudo] password for rupin:
Restarting PostgreSQL 10 database server
Failed to issue method call: Unit postgresql#10-main.service failed to load: No such file or directory. See system logs and 'systemctl status postgresql#10-main.service' for details.
The Log file is empty. There is no postmaster.pid file that could be to blame.
My Ubuntu Laptop crashed and the DB was active when the system crashed.
Can someone advise what I can do to fix this issue and start the server again?
**Update **
rupin#linuxbox:~$ systemctl status postgresql#10-main.service
Failed to issue method call: No such interface 'org.freedesktop.DBus.Properties' on object at path /org/freedesktop/systemd1/unit/postgresql_4010_2dmain_2eservice
Become OS user postgres.
Try to start PostgreSQL with
pg_ctl start -D /etc/postgres/10/main
(Use the path where postgresql.conf resides.)
Then you will see the error and you will know what to do about it.
I followed the instructions of the answer from https://askubuntu.com/questions/873091/postgresql-fails-to-reinstall-after-upgrading-ubuntu-12-04-to-14-04
I ended up removing systemd
sudo apt-get purge systemd && sudo apt-get autoremove
And I was able to reinstall postgres. Unfortunately, I lost my data/cant trace it. It is okay for me, because it is my local DB

FATAL ERROR lock file "postmaster.pid" already exists

I have recently installed PostGIS on my Mac (El Capitan 10.11.4, Postgres is version 9.5.1) using Homebrew, and I am following these instructions - http://morphocode.com/how-to-install-postgis-on-mac-os-x/
When I try to start Postgres using
pg_ctl -D /usr/local/var/postgres start
I get the following error:
$ FATAL: lock file "postmaster.pid" already exists
HINT: Is another postmaster (PID 280) running in data directory "/usr/local/var/postgres"?
So I spent a few hours researching how to address this, but to no avail.
Notably, I tried to kill the PID as recommended in an answer on Superuser - https://superuser.com/questions/553045/fatal-lock-file-postmaster-pid-already-exists- (in the case above, I ran kill 208), but as soon as I tried to start Postgres again, I got the same error, albeit with a different PID number. I saw a few people recommended deleting the postmaster.pid file, but I feel like maybe I should save that as a last resort...
Admittedly part of the reason I'm not sure how to fix this is that I'm not really clear on what the postmaster even is - I'm just starting to learn about all of this.
Hopping into a Postgres database via the psql db_name command works just fine, for what it's worth.
Posting this in case it helps someone else:
I was having this same problem as the OP after a hard reboot when my laptop crashed. What helped me was running the following command to see what PID was associated with postmaster.pid:
cat /usr/local/var/postgres/postmaster.pid
The first number that appears will be the PID. Looking in Activity Monitor, I was able to see that Postgres was running, but without a PID number that matched the one shown.
Instead of the steps outlined in the answer referenced on Superuser, I restarted my laptop properly and then opened up Terminal and ran
brew services restart postgresql
This worked without having to remove postmaster.pid, which I saw a few other posts recommend. Sometimes it's the simple solutions that work.
I add here what worked for me, after a long time of searching:
Delete the postmaster.pid file:
rm /usr/local/var/postgres/postmaster.pid
Restart your postgres:
brew services restart postgresql
Hope this helps someone ...
Update 8/2022:
As Mike commented, for M1 Mac you would replace stage 1 with:
rm /opt/homebrew/var/postgresql/postmaster.pid
With M1 and specify Postgres Version # 14
rm -rf /opt/homebrew/var/postgresql#14/postmaster.pid
It often happens to me in OSx, when my system shutdown unexpectedly.
You can just remove the file postmaster.pid.
cd Library/Application Support/Postgres/var-{postgres-version}
and remove the postmaster.pid file
in case you use brew then your path should be something like:
/usr/local/var/postgres/postmaster.pid
restart the Postgres by using this command
pg_ctl -D /usr/local/var/postgres restart
Since you can connect to the database, you don't need to start the server again - it's already running.
pg_ctl is used to control the PostgreSQL server. Since your server is already started, your command:
pg_ctl -D /usr/local/var/postgres start
Returns an error, saying that there is a lock on postmaster.pid - which is true since there is already a server running under that PID.
There are two ways:
The most basic way - skip that step, your server is already running!
Executing a needless operation - stopping the server, and then starting it again.
You could stop your server doing :
pg_ctl -D /usr/local/var/postgres stop
So that you won't have the lock on postmaster anymore and you could use your command to start it again.
Postmaster is the main PostgreSQL process. You're trying to start PostgreSQL that's already running (and you're saying yourself you can connect to it). Just skip that step of your process.
When the system shutdown unexpectedly, my postgres crashs and i'm unable to connect to it.
What worked for me was:
1˚ Check postgres log:
tail -n 10000 /usr/local/var/log/postgres.log
2˚ Find the PID of postgress, should look like this:
FATAL: lock file "postmaster.pid" already exists
HINT: Is another postmaster (PID 707) running in data directory "/usr/local/var/postgres"?
3˚ Kill that process:
kill 707
4˚ Restart your postgres
brew services restart postgresql
After those steps i was able to connect to the database within my rails application.
If you got no important data to lose :
sudo killAll postgres
brew services restart postgresql
AGAIN : You could get data corrupted by doing this !
do it at your own risk !
I am using mac and these step work for me:-
step1: cd Library/Application\ Support/Postgres
(most commonly your Postgres installation will be located here)
step2: cd var-13
(if you are using version 12 then use cd var-12. Hope got the point)
step3: ls
(As you can see among the files you find the postmaster.pid, perfect.)
step4: rm postmaster.pid
When you have removed the stale postmaster.pid file you can restart PostgreSQL and everything should work as normal.
My OSX laptop had shutdown unexpectedly, and I was getting a stale postmaster.pid error in the PostgresApp. Shutting down my laptop and turning it back on again solved the problem.
After running the following commands
rm /usr/local/var/postgres/postmaster.pid
brew services restart postgresql
The error lock file "postmaster.pid" already exists comes up again.
When we run launchctl list | grep postgres
28618 0 homebrew.mxcl.postgresql
The existing file "postmaster.pid" was created by this daemon process hosted by launchctl.
We try to stop the homebrew.mxcl.postgresql through
sudo launchctl stop homebrew.mxcl.postgresql
launchctl disable homebrew.mxcl.postgresql
Unfortunately, none of them could stop the homebrew.mxcl.postgresql.
The reason is Disable and enable an agent using (persists between boots)
https://apple.stackexchange.com/questions/105892/disable-services-in-osx-services-msc
launchctl enable <name> or launchctl disable <name>
Two ways to solve it when the error lock file "postmaster.pid" already exists comes up again
In order to stop an agent immediately through
launchctl kill homebrew.mxcl.postgresql
Restart your desktop and run brew services start postgresql#14. Now, PostgreSQL could start successfully.
Hope it could help someone who met the same issue again.
This worked for me. First locate postmaster.pid (for me it was in the var directory as seen below, although it will be different on depending on your operating system). Then get rid of postmaster.pid, then kill the postgres process, then start/restart postgres service.
cd /var/lib/pgsql/data/
rm postmaster.pid
sudo pkill -u postgres
sudo systemctl start postgresql.service
If you have installed postgres with brew then simply run the following command and it will manage everything
brew services restart postgresql

Mongo 2.6.3 does not start on init

On fresh install of Ubuntu 14.04, I have installed mongodb 2.6.3 from mongo package,
problem is, that it does not start on reboot. It starts normally when I do
sudo service mongod start
and there are no errors in log file.
but when I reboot, log file has no new entries and
service mongod status
produces
mongod stop/waiting
I have tried mongod --repair, with no effect. I have tried fresh install of Ubuntu 14.04, as I thought that my older many times upgraded OS might have something wrong with it, but nothing changed.
With older version mongo that comes from ubuntu package manager, everything was starting up nicely.
Can you please give some advice on where to look to fix this issue and make mongod start up when system is booted up. Thanks!
Had the same problem. In the log file there was an:
************
Unclean shutdown detected
Please visit http://dochub.mongodb.org/core/repair for recovery instructions.
************
To solve this error, I just removed the lock file and it worked again.
sudo rm /var/lib/mongodb/mongod.lock
I believe the issue is that mongo upstart job checks if mongo is installed by the package manager. You need to edit the upstart job to use the new from-source binary location (which I think is /opt/mongodb/bin/mongod).