I want to setup a cron script that automatically creates dumps from a specific Postgres database running in a Docker container. I know how to execute commands in a container from outside and also am familiar with pg_dump.
Somehow, for my container and database , I can't seem to make it work:
docker exec <container> pg_dump -U postgres <mydb> > /pg-snaps/<mydb>_$(date).sql
I get the following error:
zsh: no such file or directory: /pg-snaps/<mydb>_<date>.sql
The directory /pg-snaps exists. I can execute the same command inside the container, and it works. I have no idea why it doesn't work with docker exec. I looked up the methodology on how to do this, and it suggests the same as I want to do. Adding " " around the command to be executed also results in a 'no such file or directory'.
try this example:
docker exec -it <container> sh -c 'pg_dump -U postgres <mydb> > /pg-snaps/<mydb>_$(date).sql'
Related
i try to understand why,
by executing this command :
docker run --rm -it postgres bash
container starts well, gives me a bash prompt, without intializing
a postgres server.
In fact, when i only execute this :
docker run --rm -it postgres
container tries to intialize a postgres server and failed
because a non provided '-e POSTGRES_PASSWORD' sequence
which is absolutly normal.
But the question is :
what is the mechanism in 'docker' or 'in the official postgres image'
that tell container to :
not initialize a postgres server when an argument is provided
at the end of 'docker run --rm -it postgres' sequence
(like bash or psql..)
DO initialize a postgres server when NO argument is provided
(docker run --rm -it postgres)
Thanks by advance.
The postgres image Dockerfile is set up as
ENTRYPOINT ["docker-entrypoint.sh"]
CMD ["postgres"]
When an image has both an ENTRYPOINT and a CMD, the command part is passed as additional parameters to the entrypoint. If you docker run postgres bash, the bash command overrides the command part but leaves the entrypoint intact.
This entrypoint wrapper script setup is a common and useful technique. The script can do anything it needs to do to make the container ready to use, and then end with the shell command exec "$#" to run the command it got passed as arguments. Typical uses for this include dynamically setting environment variables, populating mounted volumes, and (for application containers more than database containers) waiting for a database or other container dependency to be ready.
In the particular case of the postgres image, its entrypoint script does (simplified):
if [ "$1" = 'postgres' ] && ! _pg_want_help "$#"; then
docker_setup_env
docker_create_db_directories
if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
docker_init_database_dir
docker_setup_db
docker_process_init_files /docker-entrypoint-initdb.d/*
fi
fi
exec "$#"
Since the entrypoint script is just a shell script and it does have the command as positional parameters, it can make decisions based on what the command actually is. In this case, if [ "$1" = 'postgres' ] – if the main container command is to run the PostgreSQL server – then do the first-time initialization, otherwise don't.
I am trying to run netbox based on their standard guide on Docker Hub with a slight difference that I need our existing postgres dump to be restored when the postgres container starts.
I have tried a few approaches like defining a command option in docker-compose file like (and a few more combinations):
sleep 2 && psql -U netbox -f netbox.sql
sleep is required to prevent psql command running before the postgres service is started.
Or defining a bash script that does the database restore but all these approaches cause the container to exit after that command/script is run.
My last resort was to utilize bash forking and this is what the postgres snippet of docker-compose looks like:
postgres:
image: postgres:13-alpine
env_file: env/postgres.env
command:
- sh
- -c
- (sleep 3 && cd /home && psql -U netbox -f netbox.sql) & su -c postgres postgres
volumes:
- ./my_db:/home/
- netbox-postgres-data:/var/lib/postgresql/data
Sadly this throws results in:
postgres: could not access the server configuration file
"/var/lib/postgresql/data/postgresql.conf": No such file or directory
If I omit the command section of docker-compose, the container starts up fine and I can navigate and ls the directory in the error message but it is not what I really need because this container will go on to be part of a much larger jungle of an ecosystem with little to no control over it afterwards.
Could it be my bash forking or the problem lies somewhere else?
Thanks in advance
I was able to find a solution by going through the thread that David Maze shared in the comments.
In my case, placing the *.sql file inside /docker-entrypoint-initdb.d did not work but I wrote a bash script, placed it in /docker-entrypoint-initdb.d directory and it got triggered.
The bash script was a very simple one, it would cd to the directory containing the sql dump and then restore it by running psql:
psql -U netbox -f netbox.sql
I am trying to use the following approach in order to backup PostgreSQL database that is in a Docker container:
cmd /c 'docker exec -t postgres-dev pg_dump dvdrental -U postgres -c -v >
C:\\postgresql\\backup\\dvdrental_%date:~-4,4%.%date:~-7,2%.%date:~-10,2%_%time:~0,2%.%time:~3,2%.sql'
However, it gives the following file that has the size ok 0 KB and with the extension of %tme:
dvdrental_2021.05.24_%tme
I tried some different combinations, but it still the same and I am not able to get backup with the following format:
dvdrental_2021.05.24_15.30
So, how can I get the expected result as mentioned above? I use Windows, but I think it is not important as the command is executed on Docker container.
I am working with a codebase that has a docker run command as follows (real name and password removed):
docker run -it --rm --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=password postgres:11.6 -d2
I know that -d flag is to --detach the container, but what is -d2? I can't figure out the purpose of this flag at the end of the command. I'm also confused why it's at the end of the command and not before the IMAGE name like the other flags.
The docker command line is order sensitive. Once docker sees an option or flag it cannot parse, it treats that as the image name. And everything after the image name is the command to run instead of the default command. In other words:
docker run ${options_to_run} ${image_name} ${command_override}
In the postgres image, the entrypoint is docker-entrypoint.sh and the default command is postgres. That means docker will run this container by default as docker-entrypoint.sh postgres (it concatenates the entrypoint and command together into one command with args to run). With the -d2 command override, that becomes docker-entrypoint.sh -d2 and the entrypoint script may interpret that as an option to change how it will run. The entrypoint has special handling for flags:
if [ "${1:0:1}" = '-' ]; then
set -- postgres "$#"
fi
....
exec "$#"
Which means the entrypoint arguments are modified from -d2 to postgres -d2 and then the shell in pid 1 is replaced by the command line arguments, postgres running with the -d2 argument.
I found the answer. -d2 is a postgres CLI option for specifying the debugging level. We are executing the postgres container with that postgres CLI option.
From postgres --help:
-d 1-5 debugging level
We typically have to pg_dump from multiple different versions of databases. I want to run the command inside a Docker container with the right Postgres version and have the dump output to my files rather than the container.
I thought I'd achieve it like so;
docker run -it postgres:9.6.6-alpine pg_dump --file backupFile.bak --dbname=CONNECTIONSTRING --verbose --format=c --blobs > backupFile.bak
however this just outputs the terminal output of the pg_dump command to a file, not the actual dump. I end up with a local file that's just the verbose log of the command.
What am I missing?
I can think of two options here:
Mount a volume to a local folder and dump the file there. When the container exits, the file will still be there on the host. You wouldn't need to run the container interactively. The command might look something like this (not tested):
docker run --rm -v <host_folder>:<container_folder> postgres:9.6.6-alpine pg_dump --file backupFile.bak --dbname=CONNECTIONSTRING --verbose --format=c --blobs
The backup file will still remain in <host_folder> after the container stops.
Start the container as-is, run docker cp to pull the file out of the container to the local filesystem, then stop the container. Probably not as easy or efficient as option 1.