I have a standard Python docker image that needs to start after postgers is properly started in its standard image.
I understand that I can add this Bash command in the docker-compose file:
command: bash -c 'while !</dev/tcp/db/5432; do sleep 1; done; npm start'
depends_on:
- mypostgres
But I don't have bash installed in the standard python docker image, and I'm trying to keep the installation minimal.
Is there a way to wait for postgres without having bash installed in my image?
I have a standard Python docker image that needs to start after postgres is properly started in its standard image.
You mentioned "Python docker image", but you appear to be calling npm start, which is a node.js application, not a Python application.
The standard Python images do have bash installed (as do the official Node images):
$ docker run -it --rm python:3.10 bash
root#c9bdac2e23f9:/#
However, just checking for the port to be available may be insufficient in any case, so really what you want is to execute a query against the database and only continue once the query is successful.
A common solution is to install the postgres cli and run psql in a loop, like this:
until psql -h $HOST -U $USER -d $DB_NAME -c 'select 1' >/dev/null 2>&1; do
echo 'Waiting for database...'
sleep 1
done
You can use environment variables or a .pgpass file to provide the appropriate password.
If you are building a custom image, it may be better to place this logic in your ENTRYPOINT script rather than embedding it in the command field of your docker-compose.yaml.
If you don't want to psql, you can write the same logic in Python or Node utilizing whatever Postgres bindings are available (e.g., something like psycopg2 for Python).
A better solution is to make your application robust in the face of database failures, because this allows your application to continue running if the database is briefly unavailable during a restart.
Related
i try to understand why,
by executing this command :
docker run --rm -it postgres bash
container starts well, gives me a bash prompt, without intializing
a postgres server.
In fact, when i only execute this :
docker run --rm -it postgres
container tries to intialize a postgres server and failed
because a non provided '-e POSTGRES_PASSWORD' sequence
which is absolutly normal.
But the question is :
what is the mechanism in 'docker' or 'in the official postgres image'
that tell container to :
not initialize a postgres server when an argument is provided
at the end of 'docker run --rm -it postgres' sequence
(like bash or psql..)
DO initialize a postgres server when NO argument is provided
(docker run --rm -it postgres)
Thanks by advance.
The postgres image Dockerfile is set up as
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["postgres"]
When an image has both an ENTRYPOINT and a CMD, the command part is passed as additional parameters to the entrypoint. If you docker run postgres bash, the bash command overrides the command part but leaves the entrypoint intact.
This entrypoint wrapper script setup is a common and useful technique. The script can do anything it needs to do to make the container ready to use, and then end with the shell command exec "$#" to run the command it got passed as arguments. Typical uses for this include dynamically setting environment variables, populating mounted volumes, and (for application containers more than database containers) waiting for a database or other container dependency to be ready.
In the particular case of the postgres image, its entrypoint script does (simplified):
if [ "$1" = 'postgres' ] && ! _pg_want_help "$#"; then
docker_setup_env
docker_create_db_directories
if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
docker_init_database_dir
docker_setup_db
docker_process_init_files /docker-entrypoint-initdb.d/*
fi
fi
exec "$#"
Since the entrypoint script is just a shell script and it does have the command as positional parameters, it can make decisions based on what the command actually is. In this case, if [ "$1" = 'postgres' ] – if the main container command is to run the PostgreSQL server – then do the first-time initialization, otherwise don't.
I am trying to run netbox based on their standard guide on Docker Hub with a slight difference that I need our existing postgres dump to be restored when the postgres container starts.
I have tried a few approaches like defining a command option in docker-compose file like (and a few more combinations):
sleep 2 && psql -U netbox -f netbox.sql
sleep is required to prevent psql command running before the postgres service is started.
Or defining a bash script that does the database restore but all these approaches cause the container to exit after that command/script is run.
My last resort was to utilize bash forking and this is what the postgres snippet of docker-compose looks like:
postgres:
image: postgres:13-alpine
env_file: env/postgres.env
command:
- sh
- -c
- (sleep 3 && cd /home && psql -U netbox -f netbox.sql) & su -c postgres postgres
volumes:
- ./my_db:/home/
- netbox-postgres-data:/var/lib/postgresql/data
Sadly this throws results in:
postgres: could not access the server configuration file
"/var/lib/postgresql/data/postgresql.conf": No such file or directory
If I omit the command section of docker-compose, the container starts up fine and I can navigate and ls the directory in the error message but it is not what I really need because this container will go on to be part of a much larger jungle of an ecosystem with little to no control over it afterwards.
Could it be my bash forking or the problem lies somewhere else?
Thanks in advance
I was able to find a solution by going through the thread that David Maze shared in the comments.
In my case, placing the *.sql file inside /docker-entrypoint-initdb.d did not work but I wrote a bash script, placed it in /docker-entrypoint-initdb.d directory and it got triggered.
The bash script was a very simple one, it would cd to the directory containing the sql dump and then restore it by running psql:
psql -U netbox -f netbox.sql
I've got a custom image based on the official postgres image and I want to extend the entrypoint of the parent image so that it would create new users and databases if they don't exist yet every time a container starts up. Is it possible? Like my image would execute all the commands from the standard entrypoint plus my own shell script.
I know about putting my own scripts into the /docker-entrypoint-initdb.d directory, but it seems that they get executed only when the volume is created the first time.
What you need to do is something like below
setup_user.sh
sleep 10
echo "execute commands to setup user"
setup.sh
sh setup_user.sh &
./docker-entrypoint.sh postgres
And your image should use the ENTRYPOINT as
ENTRYPOINT ["/setup.sh"]
You need to start your setup script in background and let the origin entryscript do its works to start the database
In addition to the accepted answer Docker - extend the parent's ENTRYPOINT, instead of sleeping a specific time, you may want to consider executing your script similar to how ''docker-entrypoint.sh'' of the postgres docker image does it (docker-entrypoint.sh; to init the DB, they start the server, execute initialization commands, and shut it down again). Thus:
setup_user.sh
su - "$YOUR_PG_USER" -c '/usr/local/bin/pg_ctl -D /var/lib/postgresql/data -o "-c listen_addresses='localhost'" -w start'
psql -U "$YOUR_PG_USER" "$YOUR_PG_DATABASE" < "$YOUR_SQL_COMMANDS"
su - "$YOUR_PG_USER" -c '/usr/local/bin/pg_ctl -D /var/lib/postgresql/data -m fast -w stop'
setup.sh
./setup_user.sh && ./docker-entrypoint.sh postgres
I don't really understand how I can fill in my mongodb image with a simple script like demoInstall.js.
I have my image and I can launch a mongodb instance. But I can not access to the docker image "mongo shell" to fill this one with the script.
I tried this command :
sudo docker run --entrypoint=/bin/cat mongo/ubuntu /tmp/devInstall.js | mongo --host IPAdress
But it's using the local mongo and not the image :/
Finally my aim is simple, I need to pull my image on a virgin server and launch a basic bash script who fill some informations in the Docker db.
The command you use does pipe on the output of the docker locally. You might call with explicit bash -c instead:
sudo docker run -it --rm mongo/ubuntu /bin/bash -c '/bin/cat /tmp/devInstall.js | mongo --host IPAdress'
I am not sure the IPAdress will be available though. You might want to define it via environmental parameter or container linking.
I would mount a volume with this argument:
-v /local_init_script_folder:/bootstrap
And then with a similar commandline like you proposed, you cann access the contents of the folder as /bootstrap from within the container.
All the tutorials point out to running postgres in the format of
docker run -d -p 5432 \
-t <your username>/postgresql \
/bin/su postgres -c '/usr/lib/postgresql/9.2/bin/postgres \
-D /var/lib/postgresql/9.2/main \
-c config_file=/etc/postgresql/9.2/main/postgresql.conf'
Why can't we in our Docker file have:
ENTRYPOINT ["/etc/init.d/postgresql-9.2", "start"]
And simply start the container by
docker run -d psql
Is that not the purpose of Entrypoint or am I missing something?
the difference is that the init script provided in /etc/init.d is not an entry point. Its purpose is quite different; to get the entry point started, in the background, and then report on the success or failure to the caller. that script causes a postgres process, usually indirectly via pg_ctl, to be started, detached from the controlling terminal.
for docker to work best, it needs to run the application directly, attached to the docker process. that way it can usefully and generically terminate it when the user asks for it, or quickly discover and respond to the process crashing.
Exemplify that IfLoop said.
Using CMD into Dockerfiles:
USE postgres
CMD ["/usr/lib/postgresql/9.2/bin/postgres", "-D", "/var/lib/postgresql/9.2/main", "-c", "config_file=/etc/postgresql/9.2/main/postgresql.conf"]
To run:
$docker run -d -p 5432:5432 psql
Watching PostgeSQL logs:
$docker logs -f POSTGRES_CONTAINER_ID