Trying to install cstore_fdw on a docker postgres container just to test the columnar functionality, and I'm stuck at the point of build/make the cstore repo.
The os is debian 9, postgres 11, i've installed make and a few tools I need.
The step where you build and run the make cmd, is failing bcz of postgres. Found in "building" section here: https://github.com/citusdata/cstore_fdw
Directories
cstore dir: /var/lib/postgresql/cstore_fdw/
pg config dir: /var/lib/postgresql/data/postgresql.conf
at the cstore dir I run the following and get the following error, and I don't know why.
PATH=/var/lib/postgresql/data/:$PATH make
Makefile:38: /usr/lib/postgresql/11/lib/pgxs/src/makefiles/pgxs.mk: No such file or directory
Makefile:45: *** PostgreSQL 9.3 or 9.4 or 9.5 or 9.6 or 10 or 11 is required to compile this extension. Stop.
I've also tried to use the pg bin directory in the make cmd, which is /usr/lib/postgresql/11/bin/, I think, and that gives same error.
Thanks for any help
Your docker image does not contain the PostgreSQL extension building infrastructure PGXS.
You will need to install that and the PostgreSQL header files.
Related
I'm using RHEL 8.4 and I followed the installation instruction at Set up a single-node Citus cluster on your own Linux machine from RPM packages..
Step 1 and 2 (Install PostgreSQL 14 and the Citus extension and Initialize the Cluster) went through without any issues.
But When I tried (Step 3: Start the database server)
pg_ctl -D citus -o "-p 9700" -l citus_logfile start
I got the following error in terminal
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
The log output
FATAL: could not access file "citus": No such file or directory
LOG: database system is shut down
I do have a directory 'citus' in postgres user's home directory with all required files in it along with postgresql.conf
Kindly help.
It appears you ran into one of 2 issues:
Possible issue #1:
You did not successfully install citus package and the citus.so file is missing.
Can you see the file /usr/pgsql-14/lib/citus.so that should be installed via the package citus102_14?
If that is not the case, you should figure out why the package is not installed.
Possible issue #2
You do not run pg_ctl command in the home directory of postgres user.
I was getting the same error. I am using the Postgres compiled from the source file. Thus, instead of
sudo apt-get -y install postgresql-14-citus-10.2
I compiled the source of the Citus from Github and specified the path to location of my Postgres while configuring as:
git clone https://github.com/citusdata/citus.git
cd citus
./configure --prefix=/usr/local/pgsql/
make
sudo make install
I do not get an error during pg_ctl start and I can load extension as CREATE EXTENSION citus;.
I have downloaded Portainer onto my server and created a PostgreSQL database in a container there.
Today I could no longer get access to the database. The log shows a message that there is a version problem.
I already read into some similar issues like Postgres container crashes with `database files are incompatible with server` after container's image has been updated to the latest one and Postgres container crashes with `database files are incompatible with server` after container's image has been updated to the latest one
and the solutions brew postgresql-upgrade-database did not work.
What can I do?
LOG
2021-10-03 [1] FATAL: database files are incompatible with server
2021-10-03 [1] DETAIL: The data directory was initialized by PostgreSQL version 13, which is not compatible with this version 14.0 (Debian 14.0-1.pgdg110+1).
PostgreSQL Database directory appears to contain a database; Skipping initialization
I also found this https://www.postgresql.org/docs/14/upgrading.html but the commands didn't work. Do I need to do this in the container somehow, or what commands will work to keep it running in the container?
You need to update the data file to the new format with this command:
$ brew postgresql-upgrade-database
I resolved it by
Removing the postgres image
Remove the volume
Pull the image again
Assuming you know the docker commands for above step.
On top of the answer suggesting removing all volumes/images/containers - if you have a shared volume for the DB in docker-compose.yml, like:
db:
image: postgres:14.1-alpine
volumes:
- ./tmp/db:/var/lib/postgresql/data
You will also need to remove the postgres mapped volume data which lives in ./tmp/db. Otherwise those conflicting files would still be there.
If you don't care about the data - you use the database for development purposes and you'll be able to re-create the db easily, just run rm -r ./tmp/db. Than you can just docker-compose up and re-create your database.
In case you care about the data, use pg_dumpall to dump you data before removing the files and restore after you run docker-compose up and your postgres service is ready again.
had a problem after running
brew update
brew upgrade
Brew upgraded postgresql to 14 which gave me
psql: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I have also seen the same error you have by inspecting
tail /usr/local/var/postgres/server.log
I have downgraded to version 13 running
brew uninstall postgresql
brew install postgresql#13
echo 'export PATH="/usr/local/opt/postgresql#13/bin:$PATH"' >> ~/.zshrc
and the command I've used to start the DB again is
pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
I think because of the difference postgresql13 with 14 in initialization.
You can make a backup of your database before the new version, delete it and migrate database to the new version
for example in django:
dump: ./manage.py dumpdata -o mydata.json
load: ./manage.py loaddata mydata.json
django: Of course, if you have important information, do this separately for each model and pay attention to the dependencies when loading.
In docker-compose.yml you could change
db-data:/var/lib/postgresql/data:rw
to this
db-data:/var/lib/postgresql#14/data:rw
Delete image and run
docker-compose up -d
one more time
I was also facing the same issue with postgres in keycloak.
Downgrading the version to 13 resolved my issue.
for me the database directory was in /tmp/db and since no important data was there I remove all and everything worked well ...
but if your data is important then read these:
https://www.postgresql.org/docs/13/upgrading.html
https://www.postgresql.org/docs/13/pgupgrade.html
https://www.kostolansky.sk/posts/upgrading-to-postgresql-14/
I resolved it by
Removing the postgres container
docker rm "container name"
Removing the postgres image
docker rmi "image name"
docker-compose stop
docker-compose down
list all volumes
docker volume ls
remove volume
docker volume rm "volume name"
Pull the image again
Assuming you know the docker commands for above step.
I had the same issue after home brew update on my mac - Install #14 of postgresql
This is what help to me:
Install old binaries v13
brew install postgresql#13
Backup Your old DB folder
Initdb with new name
now use pg_upgrade like this:
pg_upgrade --old-datadir [Your old DB folder] --new-datadir [Your new
DB folder] --old-bindir [link to old bin (v13)]
for example:
pg_upgrade --old-datadir PSQ-data --new-datadir PSQ-data_new
--old-bindir /usr/local/Cellar/postgresql#13/13.9/bin
Source for old bin You will have after install #13, just look what you receive on terminal after install "Summary".
After this operation just start sever with new DB
I use tableplus for my general admin.
Currently using the docker postgres image at 10.3 for both production and localhost development.
Because tableplus upgraded their postgres 10 drivers to 10.5, I can no longer use pg_restore to restore the backup files which are dumped using 10.5 --format=custom
See image for how I backup using tableplus. And how it uses 10.5 driver
The error message I get is pg_restore: [archiver] unsupported version (1.14) in file header
What i tried
I tried in localhost to simply change the tag for postgres in my dockerfile from 10.3 to 10.5 and it didn't work
original dockerfile
FROM postgres:10.3
COPY ./maintenance /usr/local/bin/maintenance
RUN chmod +x /usr/local/bin/maintenance/*
RUN mv /usr/local/bin/maintenance/* /usr/local/bin \
&& rmdir /usr/local/bin/maintenance
to
FROM postgres:10.5
COPY ./maintenance /usr/local/bin/maintenance
RUN chmod +x /usr/local/bin/maintenance/*
RUN mv /usr/local/bin/maintenance/* /usr/local/bin \
&& rmdir /usr/local/bin/maintenance
My host system for development is macOS.
I have many existing databases and schemas in my development docker postgres. So I am currently stumped as to how to upgrade safely without destroying old data.
Can advise?
Also I think a long term is to figure out how to have data files outside the docker (i.e. inside my host system) so that everytime I want to upgrade my docker image for postgres I can do so safely without fear.
I like to ask about how to switch to such a setup as well.
If I understand you correctly, you want to restore a custom format dump taken with 10.5 into a 10.3 database.
That won't be possible if the archive format has changed between 10.3 and 10.5.
As a workaround, you could use a “plain format” dump (option --format=plain) which does not have an “archive version”. But any problems during restore are yours to deal with, since downgrading PostgreSQL isn't supported.
You should always use the same version for development and production, and you should always use the latest minor release (currently 10.13). Everything else is asking for trouble.
backup as plain text like this: warning! the file will be huge. Around 17x more than regular custom format. My typical 90mb is now 1.75Gb
copy the backup file into the postgres container docker cp ~/path/to/dump/in-host-system/2020-07-08-1.dump <name_of_postgres_container>:/backups
go to the bash of your postgres container docker exec -it <name_of_postgres_container> bash
inside the bash of postgres container: psql -U username -d dbname < backups/2020-07-08-1.dump
That will work
So,
My admin installed (don't know how) postgresql 9.2 (64 bit version) on a centos7 machine. The thing is I need to initalize the database so I should run something like:
service postgresql-9.2 initdb
but I get error:
The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, force-reload, status). For other actions, please try to use systemctl.
Did some search and people said that you should initialize via direct call to some script, but every given location does not exists in this CentOS 7 machine.
Like for example this location (yeah, it is for 9.3 and I use 9.2, but similar location does not exist anyway):
/usr/lib/pgsql-9.3/bin/postgresql93-setup initdb
or this
/usr/pgsql-9.3/bin/postgresql93-setup initdb
I found out
/usr/lib64/pgsql
but this directory has only a bunch of '*.so' files.
How the hell do I run initdb for postgresql 9.2 on centos7?
You can run it directly from postgres account
#su - postgres -c pg_ctl initdb
Just installed the Postgresql 9.3 on Centos.
#cd /etc/init.d
#ls postgres*
and there is a posgresql-9.3 script
#./postgresql-9.3
gives you the options to run with and one of them is initdb
I followed this Almost idiot's guide to install postgresql and postgis on Centos 7 and got stuck starting the database itself with the following command:
[root#localhost cmaps]# service postgresql-9.3 initdb
The service command supports only basic LSB actions (start, stop, restart, try-restart, reload, force-reload, status). For other actions, please try to use systemctl.
If I install the postgresql package from Centos repository (version 9.2) the command works, but then I they don't have postgis, and there is no official posgresql-92 repository for Centos 7.
Any idea on what's happening?
Thank you!
I had the same problem and solve it by changing the path
/usr/pgsql-9.3/bin/postgresql93-setup initdb
Note that the path is different
CentOS 7 is based on systemd. PostgreSQL packages use a different command for initdb there:
/usr/lib/pgsql-9.3/bin/postgresql93-setup initdb
On older versions, call the initscript directly.
/etc/init.d/postgresql-9.3 initdb
For Centos 7.3 installed on MS Azure platform the script is located at
/usr/bin/postgresql-setup initdb
the version number is absent from the name of these as well:
db data & config location: /var/lib/pgsql/data/
service name /sbin/service postgresql status or systemctl status postgresql.service