I am trying to deploy PostgresDatabase on azure container instance.
To deploy on docker using bind mount(since Azure container Instance only support bind mount) i am using the below command, and it is deployed on docker.
docker run -d -p 5434:5432 --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -e PGDATA=/var/lib/postgresql/data/pgdata -v /home/ubuntu/volum:/var/lib/postgresql/data postgres
If i do something similar for deploying on Azure container Instance
az container create \
--resource-group $ACI_PERS_RESOURCE_GROUP \
--name postgreariesdb25-1 \
--location eastus \
--image postgres \
--dns-name-label $ACI_DNS_LABEL \
--environment-variables POSTGRES_PASSWORD=mysecretpassword PGDATA=/var/lib/postgresql/data/pgdata \
--ports 5432 \
--azure-file-volume-account-name $ACI_PERS_STORAGE_ACCOUNT_NAME \
--azure-file-volume-account-key $STORAGE_KEY \
--azure-file-volume-share-name $ACI_PERS_SHARE_NAME \
--azure-file-volume-mount-path /var/lib/postgresql/data
I am getting the below message inside logs of Azure Container
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data/pgdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 20
selecting default shared_buffers ... 400kB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
2020-11-24 05:23:39.218 UTC [85] FATAL: data directory "/var/lib/postgresql/data/pgdata" has wrong ownership
2020-11-24 05:23:39.218 UTC [85] HINT: The server must be started by the user that owns the data directory.
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/data/pgdata"
running bootstrap script ...
Volume Mount is required to have data in case of container restart.
This is a known error for mounting Azure File Share to Azure Container Instance. Currently, it does not support to change the ownership of the mount point. If you do not want to use other services, then you need to create a script to move the data to the mount point and the mount point should be a new folder that does not exist in the image. For you, the mount point /var/lib/postgresql/data exists in the image and contains the files that Postgresql depends on, then this point cannot be the mount point.
Related
tell me how can I store PostgreSQL database data in an Azure Storage account. The PostgreSQL deploy to Azure Container Instance. When I restart the Azure Container instance all data disappears.
Dockerfile
FROM timescale/timescaledb:latest-pg12
ENV POSTGRES_USER=admin
POSTGRES_DB=dev-timescaledb
POSTGRES_PASSWORD=password
PGDATA=/var/lib/postgresql/data/pgdata
CMD ["postgres", "-c", "max_connections=500"]
Command for creating a Container Instance and mounting a Storage Account
az container create --resource-group test-env --name test-env --image
test-env.azurecr.io/timescale:latest --registry-username test-env
--registry-password "registry-password" --dns-name-label test-env --ports 5432 --cpu 2 --memory 5 --azure-file-volume-account-name testenv --azure-file-volume-account-key
'account-key'
--azure-file-volume-share-name 'postgres-data' --azure-file-volume-mount-path '/var/lib/postgresql/data'
but i got an error
data directory “/var/lib/postgresql/data/pgdata” has wrong ownership
The server must be started by the user that owns the data directory.
It caused by an existing issue that you cannot change the ownership of the mount point when you mount the Azure File Share to the Container Instance. And it cannot be solved currently. You can find the same issue in SO. I recommend you use the AKS with the disk volume and it will solve the problem for Postgres on persisting data.
I'm following instructions in this article to pull a Postgres Docker image and run it.
$ mkdir -p $HOME/docker/volumes/postgres
$
$ docker pull postgres:9.5
$
$ docker run --name pg-docker -e POSTGRES_PASSWORD=postgres -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres:9.5
I see Postgres files and data getting created in the path I mounted with -v above, but the container exits soon after starting, and the files vanish.
My questions:
Why does the container exit at all?
Why does it delete all the files? The whole purpose of -v is to
preserve data outside the container.
Update 1:
Here are the logs from the container that I was able to get when the container was briefly up (after the container exited, I only got an error message Error: No such container: postgres:9.5 but no logs):
$ docker logs pg-docker
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default timezone ... Etc/UTC
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
Update 2:
Running docker without -d produced more output, which has the reason for this situation:
creating template1 database in /var/lib/postgresql/data/base/1 ...
LOG: could not link file "pg_xlog/xlogtemp.36" to "pg_xlog/000000010000000000000001": Operation not supported
FATAL: could not open file "pg_xlog/000000010000000000000001": No such file or directory
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/data"
Does it matter that I'm on a Windows system inside a Git Bash shell?
I'm using this Dockerfile to deploy it on openshift. - https://github.com/sclorg/postgresql-container/tree/master/9.5
It works fine, until I enabled ssl=on and injected the server.crt and server.key file into the postgres pod via volume mount option.
Secret is created like
$ oc secret new postgres-secrets \
server.key=postgres/server.key \
server.crt=postgres/server.crt \
root-ca.crt=ca-cert
The volume is created as bellow and attached to the given BuidlConfig of postgres.
$ oc volume dc/postgres \
--add --type=secret \
--secret-name=postgres-secrets \
--default-mode=0600 \
-m /var/lib/pgdata/data/secrets/secrets/
Problem is the mounted files of secret.crt and secret.key files is owned by root user, but postgres expect it should be owned by the postgres user. Because of that the postgres server won't come up and says this error.
waiting for server to start....FATAL: could not load server
certificate file "/var/lib/pgdata/data/secrets/secrets/server.crt":
Permission denied stopped waiting pg_ctl: could not start server
How we can insert a volume and update the uid:guid of the files in it ?
It looks like this is not trivial, as it requires to set Volume Security Context so all the containers in the pod are run as a certain user https://docs.openshift.com/enterprise/3.1/install_config/persistent_storage/pod_security_context.html
In the Kubernetes projects, this is something that is still under discussion https://github.com/kubernetes/kubernetes/issues/2630, but seems that you may have to use Security Contexts and PodSecurityPolicies in order to make it work.
I think the easiest option (without using the above) would be to use a container entrypoint that, before actually executing PostgreSQL, it chowns the files to the proper user (postgres in this case).
I've been running my docker images upon my Vagrant machine (the box is ubuntu 14.04) without any big issues. But the following error is racking my brains. I wish you people can help me.
When I run this:
$ docker run -it -v /vagrant/postgresql/data:/var/lib/postgresql/data postgres:9.2
I get this error
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
fixing permissions on existing directory /var/lib/postgresql/data ... ok
initdb: could not create directory "/var/lib/postgresql/data/pg_xlog": Permission denied
initdb: removing contents of data directory "/var/lib/postgresql/data"
vagrant#legionat:/vagrant/sonarqube$ docker run -it -v /vagrant/postgresql/data:/var/lib/postgresql/data postgres:9.2
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
fixing permissions on existing directory /var/lib/postgresql/data ... ok
initdb: could not create directory "/var/lib/postgresql/data/pg_xlog": Permission denied
initdb: removing contents of data directory "/var/lib/postgresql/data"
I've tried open all the permissions of /vagrant/postgresql without success. Maybe this is a problem of the official docker image.
EDIT:
I've just noticed that that is a lot of people facing the same problem as me: https://github.com/docker-library/postgres/issues/26
And, as someone asked about this in the comments, here it goes:
$ ls -l /vagrant/postgresql/data
total 0
If you are just concerned about persisting the data, I would recommend using a data volume instead of a host volume.
Run docker volume create --name pgdata
Then connect it to your container with:
docker run --rm --name pg -v pgdata:/var/lib/PostgreSQL/data postgres:9.2
Even after that container is gone, you can start a new one connected to the volume and your data will be there.
Just make sure your Vagrant user has the permission to access this directory:
ls -ld /vagrant/postgresql/data
And as deinspanjer said. you can use named volume for persisting the data
I'm using the stock official Postgres image from Docker Hub. docker pull postgres. I wanted to map the data directory in the Postgres container to my OS X host. So, I tried this.
docker run --rm -p 5432:5432 -e POSTGRES_PASSWORD=mypass -v `pwd`/data:/var/lib/postgresql/data postgres
This resulted in the Postgres container failing to launch correctly.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... initdb: could not create directory "/var/lib/postgresql/data/global": Permission denied
initdb: removing contents of data directory "/var/lib/postgresql/data"
The goal I'm trying to achieve is to have my database data stored on the host machine, so that I can start a postgres container and have it read (or load) the database from a previous instance. Am I on the right track or is this a stupid way to achieve database persistence?
According to official documentation you should use boot2docker to resolve the issue. However, without it, you won't be able to mount container.