Actually learning docker,
i manipulate postgres containers and asking myself the
following questions :
I launch a first postgres container like this :
docker run -e POSTGRES_PASSWORD=secret -p 5464:5432 -v postgres-data:/var/lib/postgresql/data -d postgres
and then a second container, using this command, and by consequence EXACTLY THE SAME VOLUME.
docker run -p 5465:5432 -v postgres-data:/var/lib/postgresql/data -d postgres
Is it a problem ?
And my most essential question is :
do i have to consider i have two postgres servers sharing the same configurations files,
or do i have to conside i have two postgres containers sharing the same postgres server ?
It's not really clear for me.
Thanks in advance.
Yes, that's a problem. I think PostgreSQL is clever enough that one of the databases just won't start up. In the worst case, this is a recipe for data corruption. This isn't specific to Docker; just in general, you can't run two databases against the same physical storage.
A typical container-oriented setup is to have two separate databases with two separate volumes, one for each service that requires a database.
Related
I have two Postgres databases and I want to sync data between themes.
So far I have these two containers, exactly the same with different posts and different names.
docker container run --name='p1' -d -p 5435:5432 -v /tmp/dbs/test/:/var/lib/postgresql/data postgres
docker container run --name='p2' -d -p 5436:5432 -v /tmp/dbs/test/:/var/lib/postgresql/data postgres
The problem happens when something changed.
If I change something in p1 like insert a row, then I can't see it in p2.
But if I kill, and run containers again, then I can see the inserted data in both of themes.
Why this is happening?
Is there a way to sync data between themes?
Running two postmaster processes on the same files is a sure road to data corruption. Don't do that.
You cannot have multi-master replication with standard PostgreSQL, but you can have a read-only standby server.
I have the following docker-compose file:
version: '3'
services:
web:
build:
context: ./django_httpd_mod_wsgi
ports:
- "8000:80"
db:
build:
context: ./postgresql
volumes:
- db-data:/var/lib/postgres/data
volumes:
db-data:
I am building psotgresql image using archlinux:
The following is my postgresql Dockerfile:
FROM archlinux/base
RUN yes | pacman -S postgresql
RUN mkdir /run/postgresql/
RUN chown -R postgres:postgres /run/postgresql/
USER postgres
RUN initdb -D /var/lib/postgres/data
RUN psql -c 'CREATE DATABASE btgapp;'
RUN psql -c "CREATE USER simha WITH PASSWORD 'krishna';"
RUN psql -c 'GRANT ALL PRIVILEGES ON DATABASE btgapp TO simha;'
CMD ["/usr/bin/postgres","-D","/var/lib/postgres/data"]
When i try to do:
docker-compose up
I get the error:
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/run/postgresql/.s.PGSQL.5432"?
ERROR: Service 'db' failed to build: The command '/bin/sh -c psql -c 'CREATE DATABASE dbname;'' returned a non-zero code: 2
I understood that i have to run the psql -c CREATE DATABSE "dbname" after starting the postgresql server by /usr/bin/postgres -D /var/lib/postgres/data
But i cannot start multiple commands in a Dockerfile. So how to do this.
The option is start a script. But then it will be difficult to see postgres running as a single process.
Based on the comments, I will try to answer here.
I believe that you should go with the postgres 11-alpine image. And I will try to explain why here.
Official docker images come with a number of benefits that you should always consider before starting your own.
Upgrade path is easy - when a new revision of the application wrapped in the image is released, the official docker image will in most cases be updated along with it. And ususally the changes respect the configuration conventions that the image has established. Such as environment variables, startup specifics. So that users can simple change the tag in their stacks, and upgrade. There may of course be breaking changes - always check this.
Large user base - when images like postgres have been downloaded more than 10 milliion times (2019), this does not only mean that it is popular, but inherently works like a guarantee that the image has been tested thoroughly. Any elementary bugs have been weeded out already, and you will have an easy time with the image.
Optimized for size and performance - you can be sure that attention has been paid to a lot of details, minimizing the size of the image and maximizing performance. Many projects publish their applications on a few different linux distros. Like postgres - they publish debian and a alpine based images. The alpine image is the smaller one, while the debian is slightly larger, but gives you access to the vast debian package repositories if you need extra packages installed.
Easy configuration - maintainers of the official images usually understand that usecases of their userbase very well. And they try to make our lives as developers and admins easier (god bless them). Official images usually have some pretty good documentation sitting right on their docker hub landing page, or a link to a github repo where the README.md will cover common usecases. I find that these instructions are worth a good read from top to bottom.
I understand that you want to keep the image small, but what do you know - the postgres project has got your usecase covered.
The latest alpine postgres image tagged 11-alpine has a compressed footprint of 28 MB and decompressed of 70MB. While the archlinux/base image that you want to start off with has compressed base footprint of 153MB and a decompressed size of 445MB. And that's before you introduce postgres itself.
Add to that, that the database and user that you want created on startup - can be handled in the environment variables alone for the official postgres image. Like this:
docker run -d --name some-postgres \
-e POSTGRES_PASSWORD=mysecretpassword \
-e POSTGRES_USER=simha \
-e POSTGRES_DB=btgapp \
postgres:11-alpine
If that does not cover the initialization that you need for your database, then you can copy .sql scripts (and .sh scripts) into a special location in the image - and they will be executed on startup. For this you can extend their image like this:
init-user-db.sh
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER simha;
CREATE DATABASE btgapp;
GRANT ALL PRIVILEGES ON DATABASE btgapp TO simha;
EOSQL
And then with a Dockerfile like this:
Dockerfile
FROM postgres:11-alpine
COPY ./init-user-db.sh /docker-entrypoint-initdb.d/init-user-db.sh
(This is taken from the postgres description on docker hub)
In closing - I would recommend that you do not prioritize the distro that an image is based on over the usability and maintainability. Docker enables us to run applications in containers without really caring too much about what distro is inside the container. It's all linux anyway. At the end of the day, I expect that you want a stable postgres database container like me. This is what I get with the official postgres image.
I hope I helped you evaluate your options on this.
I'm new to docker. You can take a look at my last questions here and see that I've been asking questions down this line. I read the docs carefully, and also read several articles on the web (which is pretty difficult given the rapid versioning in docker), but I still can't get a clear picture of how am I supposed to use containers and its impact on persistance.
The official postgres image creates a volume in its Dockerfile using this command
VOLUME /var/lib/postgresql/data
And the readme.md file shows only one example of how to run the image
docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
When I try that, I can see (with "docker inspect some-postgres") that the volume created lives in a random directory in my host, and it seems to "belong" to that particular container.
So here are some questions that may help my understanding:
It looks (from the official postgres image docs) that expected usage is to use "docker run" to create the container, and "docker start" afterwards (this last bit I inferred from the fact that -d and --name are used). This makes sense to me, but conflicts with a lot of information I've seen regarding containers should be ephemeral. If spin a new container every time, then the default VOLUME config in the Dockerfile doesn't work for persistance. What's the right way of doing things?
Given the above is correct (that I can run once and start many times), the only reason I see for the VOLUME command in the Dockerfile is I/O performance because of the CoW filesystem bypass. Is this right?
Could you please clearly explain what's wrong with using this approach over the (I think unofficially) recommended way of using a data container? I'd like to know the pros/cons to my specific situation, which is a node js intranet application.
Thanks,
Awer
You're correct that you can start the container using 'docker run' and start it again in the future using 'docker start' assuming you haven't removed the container. You're also correct that docker containers are supposed to be ephemeral and you shouldn't be in bad shape if the container disappears. What you can do is mount a volume into the docker container to the storage location of the database.
docker run -v /postgres/storage:/container/postgres --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d postgres
If you know the location of where the database writes to inside the container you can mount it correctly and then even if you remove the postgres container, when you start back up all your data will persist. You may need to mount some other areas that control configurations as well unless you modify and save the container.
I want to make two minor improvements on the official MongoDB docker so that it starts with the --auth enabled and uses a separate data container to store the data. What's the best way to do this?
If all are set, how should I start the shell? Will it be possible for someone without a username and password to access any of the databases available? Which directory should I backup?
EDIT
Apparently, this is not enough:
docker run --name mymongoname1 -v /my/local/dir:/data/db -d -P mongo:latest
OK, so partial answer, because I haven't messed around with docker auth.
Containerising storage is done with a storage container. That's basically a container created off a token instance, with some volumes assigned.
So for elasticsearch (which I know isn't mongo, but it is at least a NoSQL db) I've been using:
docker create -v /es_data:/es_data --name elasticsearch_data es-base /bin/true
Then:
docker run -d -p 9200:9200 --vols-from elasticsearch_data elasticsearch-2.1.0
This connects the container volume to my es container - in this example it passes through a host volume, but you don't actually need to any more, because the container can hold the data in the docker filesystem. (And then I think you can push the data container around too, but I've not got that far!)
If you run ps -a you will see the data container in Created state. Just watch if you're doing a cleanup script that you don't delete it, because unlike running containers, you can freely delete it...
Today I was researching and trying docker, and with the most of things I was impressed. There are still some questions for me about docker.
Can anyone more experienced than me with Docker tell me what is the best way to login to postgres container (run bash), in order to view some postgres configuration files, view postgres logs, log into postgres shell, execute pg_dump for example, etc. etc..., and everything this while postgres process is running.
I see that people usually run one process per container, and with this approach I am not sure what is the best way to do mentioned actions on container which runs postgres?
Any advices?
Thanks!
You can usually get a shell like this:
docker exec -it some-node bash
The canonical docker way would be not to log in to the running db container, but instead do docker logs or link other containers to do maintenance tasks (e.g. docker run -it --rm --link <my-pg-container>:pg <my-pg-image> pgsql --host pg etc..