postgresql not installed but homebrew reports it is - postgresql

I've never had an issue like this before - normally brew install postgres works and everything is fine.
I've had instances with this machine where I see "Mac OSX restarted due to a problem" and usually there is a stale postmaster.pid file that I can delete and restart postgresql and everything is fine.
Not any longer - there is something out of whack and I don't know how homebrew works well enough to troubleshoot this.
➜ ~ pg_ctl
zsh: command not found: pg_ctl
➜ ~ brew info postgresql
postgresql: stable 13.1 (bottled), HEAD
Object-relational database system
https://www.postgresql.org/
Not installed
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/postgresql.rb
License: PostgreSQL
==> Dependencies
Build: pkg-config ✔
Required: icu4c ✔, krb5 ✔, openssl#1.1 ✔, readline ✔
==> Options
--HEAD
Install HEAD version
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
This formula has created a default database cluster with:
initdb --locale=C -E UTF-8 /usr/local/var/postgres
For more details, read:
https://www.postgresql.org/docs/13/app-initdb.html
To have launchd start postgresql now and restart at login:
brew services start postgresql
Or, if you don't want/need a background service you can just run:
pg_ctl -D /usr/local/var/postgres start
==> Analytics
install: 188,386 (30 days), 549,953 (90 days), 1,863,554 (365 days)
install-on-request: 180,714 (30 days), 528,971 (90 days), 1,789,016 (365 days)
build-error: 0 (30 days)
➜ ~ ls -ltra /usr/local/var
total 0
drwxrwxr-x 4 myhandle admin 128 Aug 21 2019 homebrew
drwxr-xr-x 4 myhandle admin 128 Apr 18 2020 log
drwxr-xr-x 15 root wheel 480 Dec 1 17:32 ..
drwxrwxr-x 4 root wheel 128 Dec 2 08:56 .
I have exactly the issue reported here - https://discourse.brew.sh/t/postgress-install-fails-in-postinstall-step/6209
The only answer says "I ran the install script on the old installation" which means nothing to me - I tried brew postinstall postgres#12 imagining this to be what is meant by the "old installation" but I get the exact same result:
➜ ~ brew postinstall postgresql
==> Postinstalling postgresql
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall postgresql`
➜ ~ brew postinstall postgresql#12
==> Postinstalling postgresql#12
Warning: The post-install step did not complete successfully
You can try again using `brew postinstall postgresql#12`
Any ideas?
UPDATE: it's starting to feel like /usr/local/var and /usr/local/bin have gotten confused by either me, homebrew, or the postgres cask because the error messages don't match the reality:
➜ ~ pg_ctl -D /usr/local/var/postgres start
pg_ctl: directory "/usr/local/var/postgres" does not exist
➜ ~ pg_ctl -D /usr/local/bin/postgres start
pg_ctl: could not open PID file "/usr/local/bin/postgres/postmaster.pid": Not a directory
➜ ~ mkdir /usr/local/bin/postgres
mkdir: /usr/local/bin/postgres: File exists
Why does homebrew suggest running pg_ctl from /usr/local/var but postgres process does not exist here after fresh install? Why do I find it in /usr/local/bin ? Then when I try to launch from bin/ it's asking for the postmaster.pid file in bin/ but of course there is no postgres directory in bin/ as these are the processes themselves...

I've run these steps and postgresql has started working - I'll admit that this is a case of "better lucky than good" because I was just stabbing in the dark here -
➜ ~ sudo mkdir /usr/local/var/postgres
➜ ~ sudo chown -R `whoami` /usr/local/var/postgres
➜ ~ initdb -D /usr/local/var/postgres
The files belonging to this database system will be owned by user "tyler".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/Chicago
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /usr/local/var/postgres -l logfile start
➜ ~ brew services restart postgresql
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
➜ ~ brew postinstall postgresql
==> Postinstalling postgresql
Success
Then I could get back into Rails-world and migrate the DB properly - I surmise that this means the homebrew installation step (perhaps the postinstall step that kept failing) wasn't creating the /usr/local/var/postgres directory as it should have, or it was trying with insufficient permissions to do so and failing silently.
Once I created the directory manually and chown'd it, things started to work as they should.

For MacOS: monterey I followed these steps:
Check if postgres is installed or not:
brew list | grep postgres
Initialize DB:
initdb /opt/homebrew/var/postgresql
Start DB:
pg_ctl -D /opt/homebrew/var/postgresql -l logfile start
checking if the db started enter below command:
psql postgres

Related

Macbook M1 - Postgres Installation - "/tmp/.s.PGSQL.5432" - Failure while executing; `/bin/launchctl bootstrap

I just got a new Macbook Pro M1 and have been trying to install Postgres with homebrew. After trying a bunch of different approaches from installing different versions of Postgres to trying rm /usr/local/var/postgres/postmaster.pid I've had no luck. I am at the point now where I can't even successfully install Postgres without an error message. When I uninstall and then reinstall use brew install postgres I get:
➜ ~ brew install postgres
==> Downloading https://ghcr.io/v2/homebrew/core/postgresql/manifests/14.1_1
Already downloaded: /Users/landonlapensee/Library/Caches/Homebrew/downloads/34e850b9c47867f84334b48fe844f7a171369d777b5c118a4119297f7c215147--postgresql-14.1_1.bottle_manifest.json
==> Downloading https://ghcr.io/v2/homebrew/core/postgresql/blobs/sha256:027c8b48406c3d732241426e0f5d2caf9f48cb4d2d38610b8f5d46f0adf7a89f
Already downloaded: /Users/landonlapensee/Library/Caches/Homebrew/downloads/b74d7a88df70e29d3cd5714896c2dd5b16a1e8c01188164026a4b5762f18234b--postgresql--14.1_1.monterey.bottle.tar.gz
==> Pouring postgresql--14.1_1.monterey.bottle.tar.gz
==> /usr/local/Cellar/postgresql/14.1_1/bin/initdb --locale=C -E UTF-8 /usr/local/var/postgres
Last 15 lines from /Users/landonlapensee/Library/Logs/Homebrew/postgresql/post_install.01.initdb:
Data page checksums are disabled.
fixing permissions on existing directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 20
selecting default shared_buffers ... 400kB
selecting default time zone ... America/Vancouver
creating configuration files ... ok
running bootstrap script ... 2021-12-07 10:07:39.032 PST [52558] FATAL: could not create shared memory segment: Cannot allocate memory
2021-12-07 10:07:39.032 PST [52558] DETAIL: Failed system call was shmget(key=1895806, size=56, 03600).
2021-12-07 10:07:39.032 PST [52558] HINT: This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMALL parameter. You might need to reconfigure the kernel with larger SHMALL.
The PostgreSQL documentation contains more information about shared memory configuration.
child process exited with exit code 1
initdb: removing contents of data directory "/usr/local/var/postgres"
Warning: The post-install step did not complete successfully
You can try again using:
brew postinstall postgresql
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
This formula has created a default database cluster with:
initdb --locale=C -E UTF-8 /usr/local/var/postgres
For more details, read:
https://www.postgresql.org/docs/14/app-initdb.html
To restart postgresql after an upgrade:
brew services restart postgresql
Or, if you don't want/need a background service you can just run:
/usr/local/opt/postgresql/bin/postgres -D /usr/local/var/postgres
==> Summary
🍺 /usr/local/Cellar/postgresql/14.1_1: 3,304 files, 43.9MB
==> Running `brew cleanup postgresql`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).
Then if I run brew services start postgres I get:
➜ ~ brew services start postgres
Bootstrap failed: 5: Input/output error
Try re-running the command as root for richer errors.
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/landonlapensee/Library/LaunchAgents/homebrew.mxcl.postgresql.plist` exited with 5
If I run psql I get:
Error: Failure while executing; `/bin/launchctl bootstrap gui/501 /Users/landonlapensee/Library/LaunchAgents/homebrew.mxcl.postgresql.plist` exited with 5.
➜ ~ psql
psql: error: connection to server on socket "/tmp/.s.PGSQL.5432" failed: No such file or directory
Is the server running locally and accepting connections on that socket?
And if I run ➜ ~ rm /usr/local/var/postgres/postmaster.pid I get:
rm: /usr/local/var/postgres/postmaster.pid: No such file or directory
I am so lost on this issue. I am trying to set this up to use for a ruby on rails environment. Does anybody know what could be causing this issue? (I tried so many different threads/commands that I think I have created root files that are causing conflicts)
Or does anybody of any suggestions of any paid services that could walk me through fixing this?
BIG THANKS!
I had the same problem. Apparently it did not start the server after reinstalling because there was another server, postgresql#13 (the previous version before updating) still running. I killed the process and then it started the updated server.

Brew Postgresql Starts But Process Is Not Running

I have installed Postgres via Brew on a Mac. Then, I have tried to start it
| => brew services restart postgres
Stopping `postgresql`... (might take a while)
==> Successfully stopped `postgresql` (label: homebrew.mxcl.postgresql)
==> Successfully started `postgresql` (label: homebrew.mxcl.postgresql)
After that, I inspect to see if the process is running
(base) ________________________________________________________________________________
| ~/ # MacBook-Pro (user)
| => ps -ef | grep postgres
501 61568 561 0 5:00PM ttys000 0:00.00 grep postgres
In addition, brew services produces the following
| => brew services
Name Status User Plist
dnsmasq unknown root /Library/LaunchDaemons/homebrew.mxcl.dnsmasq.plist
emacs stopped
postgresql error root /Library/LaunchDaemons/homebrew.mxcl.postgresql.plist
postgresql#12 stopped
rabbitmq stopped
unbound stopped
Any help on how to troubleshoot this is appreciated.
Unlike a builtin pkg manger in Linux, Homebrew on macOS doesn't require root privilege during most brew command. The only exception is sudo brew services when you're starting a services listen on port < 1024.
Answer to your question: you messed up file permission with sudo brew services.
The solution works for x86 Mac. I've not tested it on M1 mac. (Homebrew stores related files under /usr/local on x86 Mac, under /opt/homebrew under M1 Mac.)
### Run in Bash or Zsh
# Stop postgresql
sudo brew services stop postgresql
# In case service file is copied to ~/Library/LaunchAgents as well
brew services stop postgresql
# Fix permission of homebrew files
sudo chown -R $USER:admin $(brew --prefix)/*
# Remove socket like `/tmp/.s.PGSQL.5432`
# Restart postgresql without sudo
brew services start postgresql
Besides above fix, you may also need to
Do a db migration from psql 12 -> psql 13, check brew info postgresl for detail
Start psql manually and check the start log
# Change /usr/local to /opt/homebrew on M1 Mac
# Check the start log
pg_ctl -D /usr/local/var/postgres start
# to stop it
pg_ctl -D /usr/local/var/postgres stop
sudo and brew
I'm not saying postgresql can't be started as root, but the Homebrew and brew services are not designed to work with sudo.
And sudo brew services changes some of the files' owner to root. You'd better change them back being owned by yourself.
Here's an example (check the Warning part)
❯ sudo brew services start adguardhome
Warning: Taking root:admin ownership of some adguardhome paths:
/usr/local/Cellar/adguardhome/0.106.3/bin
/usr/local/Cellar/adguardhome/0.106.3/bin/AdGuardHome
/usr/local/opt/adguardhome
/usr/local/opt/adguardhome/bin
/usr/local/var/homebrew/linked/adguardhome
This will require manual removal of these paths using `sudo rm` on
brew upgrade/reinstall/uninstall.
Warning: adguardhome must be run as non-root to start at user login!
==> Successfully started `adguardhome` (label: homebrew.mxcl.adguardhome)
sudo brew services is not that intelligent enough to change the files' onwer bach to $USER automatically.
I've written a script to do this: brew-fix-perm. But this is not enough in your case. You have to change back the ownership of postgres configuration files under $(brew --prefix)/var/postgres. That's why I put sudo chown -R $USER:admin $(brew --prefix)/*.
Related Answer
I've known a stubborn guy stick with sudo brew services to start mysql. And a more detailed answer about brew services is there. You may wanna have a check.
Short answer:
brew services stop postgresql
sudo chown -R $(whoami) $(brew --prefix)/*
brew services start postgresql

Postgres does not know where to find the server configuration file. After "brew install postgresql"

I have installed PostgreSQL on MacOS Big Sur version 11.3 with brew install postgresql.
When I run the postgres command I get this error
postgres does not know where to find the server configuration file.`
You must specify the --config-file or -D invocation option or set the PGDATA environment variable.
Where do I find the config file?
Checking from the package formula postgresql.rb by brew cat postgresql, the datadir is ${HOMEBREW_PREFIX}/var/postgres. (HOMEBREW_PREFIX is not an exported env var, it's used by homebrew internally)
/usr/local/var/postgres on x86 Mac
/opt/homebrew/var/postgres on M1 Mac
This dir should be created automatically during brew install postgresql.
BTW, the canonical way to start psql with Homebrew should be brew services start postgresql, which uses launchd on macOS to manage services. (For newbie on macOS, launchd could be seen as a systemd alternative on macOS.)
Run brew info postgresql and some additional tips is displayed in the Caveats part of the output.
❯ brew info postgresql
postgresql: stable 13.3 (bottled), HEAD
Object-relational database system
https://www.postgresql.org/
/usr/local/Cellar/postgresql/13.3 (3,225 files, 38.8MB) *
Poured from bottle on 2021-05-14 at 09:37:24
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/postgresql.rb
License: PostgreSQL
==> Dependencies
Build: pkg-config ✔
Required: icu4c ✔, krb5 ✔, openssl#1.1 ✔, readline ✔
==> Options
--HEAD
Install HEAD version
==> Caveats
To migrate existing data from a previous major version of PostgreSQL run:
brew postgresql-upgrade-database
This formula has created a default database cluster with:
initdb --locale=C -E UTF-8 /usr/local/var/postgres
For more details, read:
https://www.postgresql.org/docs/13/app-initdb.html
To have launchd start postgresql now and restart at login:
brew services start postgresql
Or, if you don't want/need a background service you can just run:
pg_ctl -D /usr/local/var/postgres start
On my Mac (Big Sur), it's located at /usr/local/var/postgres/postgresql.conf. The command I use to start the server is:
pg_ctl -D /usr/local/var/postgres start

initb fails for brew postgresql-upgrade-database

I am using brew postgresql-upgrade-database to upgrade postgresql from 11.8 to 12 and migrate the data. The process failed with the following output
Upgrading postgresql data from 11 to 12...
Stopping postgresql#11... (might take a while)
==> Successfully stopped postgresql#11 (label: homebrew.mxcl.postgresql#11)
waiting for server to shut down.... done
server stopped
==> Moving postgresql data from /usr/local/var/postgres to /usr/local/var/postgres.old...
==> Creating database...
Error: Upgrading postgresql data from 11 to 12 failed!
==> Moving postgresql data back from /usr/local/var/postgres.old to /usr/local/var/postgres...
Error: Failure while executing; /usr/local/Cellar/postgresql/12.4/bin/initdb --lc-collate=en_US.UTF-8 --lc-ctype=en_US.UTF-8 --lc-messages=en_US.UTF-8 --lc-monetary=en_US.UTF-8 --lc-numeric=en_US.UTF-8 --lc-time=en_US.UTF-8 -E\ UTF8 /usr/local/var/postgres exited with 1.
How do I fix this?
There is an [issue][1] using brew postgresql-upgrade-database. I will update this answer when the issue is resolved.
As at July 21, 2021 this issue is still open. My workaround is below:
WARNING this will wipe out your databases so they have to be reloaded or re-initialized.
brew uninstall --force postgresql
brew install postgresql#12
cd /usr/local/var/
rm -rf postgres
initdb /usr/local/var/postgres
When doing the brew install read the post install message and make sure that you have the correct paths etc in your .zshrc (or equivalent file e.g. .bashrc). If you do not have this set your path correctly, the initdb command may work.
[1]: http://github.com/Homebrew/homebrew-core/issues/60686

I can't start server PostgreSQL 11: "pg_ctl: could not start server"

I am in CentOS Linux release 7.5.1804 (Core)
When I login as postgres and run:
bash-4.2$ /usr/pgsql-11/bin/initdb -D /var/lib/pgsql/11/data
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /var/lib/pgsql/11/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/pgsql-11/bin/pg_ctl -D /var/lib/pgsql/11/data -l logfile start
bash-4.2$
then I run
bash-4.2$ /usr/pgsql-11/bin/pg_ctl start -l logfile -D /var/lib/pgsql/11/data
waiting for server to start..../bin/sh: logfile: Permission denied
stopped waiting
pg_ctl: could not start server
Examine the log output.
bash-4.2$ date
Wed Oct 24 01:50:44 -05 2018
bash-4.2$
I search in GOOGLE for "waiting for server to start..../bin/sh: logfile: Permission denied" but this error only happened in MAC and no solutions is displayed...
Also I run
bash-4.2$ postgres --version;postmaster --version;
postgres (PostgreSQL) 11.0
postgres (PostgreSQL) 11.0
bash-4.2$
then I believe PostgreSQL 11 is fine installed, but I can't start server.
I install with this line:
yum install postgresql-jdbc.noarch postgresql-jdbc-javadoc.noarch postgresql-unit11.x86_64 postgresql-unit11-debuginfo.x86_64 postgresql11.x86_64 postgresql11-contrib.x86_64 postgresql11-debuginfo.x86_64 postgresql11-devel.x86_64 postgresql11-docs.x86_64 postgresql11-libs.x86_64 postgresql11-odbc.x86_64 postgresql11-plperl.x86_64 postgresql11-plpython.x86_64 postgresql11-pltcl.x86_64 postgresql11-server.x86_64 postgresql11-tcl.x86_64 postgresql11-test.x86_64
I didn't add [postgresql11-llvmjit.x86_64] because this requires many dependences.
CentOS Linux release 7.5.1804 + PostgreSQL 11 ?
Do I need install aditional software?
The logfile is not to be taken literally, it should be replaced with a file where the PostgreSQL server process has write permission.
You don't do it that way normally, rather you edit postgresql.conf and configure log_destination, logging_collector, log_directory and log_filename. Then you start PostgreSQL without the -l option.
Your attempt to start PostgreSQL with systemctl probably did work; you can check with
systemctl status postgresql-11
So, I faced the same issue.
First I checked for if and what all servers were active on my machine using :
brew services list
Then I had some servers active so I stopped them as you need to stop them for posgresql server to work.
brew services stop < name of the server/s active >
Then I created a new database
initdb /usr/local/var/postgres2
and started the server using
pg_ctl -D /usr/local/var/postgres2 -l logfile start
Worked fine for me !!
enter image description here
I think there is a background postgres process running and firstly you need to kill this service. I had same problem and fixed by this command:
sudo /etc/init.d/postgresql stop
or you can try to kill all postgres named processes in your system monitor and delete postmaster.pid file in your database directory.
Then you can start the server as:
sudo -u <your-username> pg_ctl -D <your-database-file-location> start
An example for this:
sudo -u shc /home/shc/postgres-built/bin/pg_ctl -D /home/shc/mydatabase start
After you are done, to not face same problems again, stop the server as:
sudo -u <your-username> pg_ctl -D <your-database-file-location> stop