How to escape quotes in a docker postgres command - postgresql

I have the following command that I'm running (renamed some variables):
docker exec docker_name sh -c 'psql dbname -U joeadmin -c "update table set field='really_longstringwithabunf3493829#########=';"'
When I run this, it will throw the following error:
ERROR: syntax error at or near ";"
LINE 1: ...longstringwithabunf3493829#########=;
^
How can I escape or retain the single quotes so this will work. Also, if this is just a horrible way of approaching this, I'm open to other suggestions. The use case is making changes to a db on a docker container before exporting data out of it.

Working with strings can be a bit ugly at times with Docker. In your case, I would suggest using double-quotes first, then escape inner double-quotes with \:
docker exec docker_name sh -c "psql dbname -U joeadmin -c \"update table set field='really_longstringwithabunf3493829#########=';\""
A cleaner option would be to create an environment variable:
STR="really_longstringwithabunf3493829#########="
docker exec docker_name sh -c "psql dbname -U joeadmin -c \"update table set field='${STR}';\""
Bear in mind that string interpolation happens in the host OS, not in the container (unless you escape the dollar-sign with \${STR}
Still cleaner would just be to create a file and then copy it into the container:
echo "update table set field='really_longstringwithabunf3493829#########=';" > ~/myfile.txt
docker cp ~/myfile.txt docker_name:/tmp
docker exec docker_name sh -c "psql dbname -U joeadmin -f /tmp/myfile.txt"

Related

escape single quotes in the string templates when env variables are passed to psql

I'm passing env variables from .env file to my bash script and I want to run psql command. Unfortunatelly I'm struggling with a correct string template so my psql always fails.
#!/bin/bash
set -o allexport
source .env
if [[ $SHOULD_UPDATE_CONTENT==1 ]]
then
# 1. Truncate all the tables before inserting data
r="select $DB_SCHEMA.truncate_tables('user','$DB_SCHEMA');"
docker exec $CONTAINER_NAME psql -U dev -d data -v ON_ERROR_STOP=1 -c "${r}"
fi
set +o allexport
.env
DB_HOST=db
DB_PORT=5432
DB_DATABASE=dbTest
DB_SCHEMA=schema
SHOULD_UPDATE_CONTENT=1

How to combine exec commands in shell script

I am running postgreSQL on docker.
In order to access the container I run
winpty docker exec -it postgres_db bash
Now that I am inside the docker container, I run this to access as user 'postgres' in postgres server
psql -U postgres
Then what I want to do is create a database
create database test;
I can I do this sequentially in a script?
I want something like this:
#!/bin/bash
winpty docker run -p 8005:5432 --name postgres_db -e POSTGRES_PASSWORD=password -d postgres
sleep 5
winpty docker exec -it postgres_db bash -c "psql -U postgres" [-c "create database test;"]
create database cannot be executed if im not inside "psql -U postgres". Obviously the last line -c "create database test;" is wrong, just to make you understand what I want to do.
winpty docker exec -it postgres_db bash -c "psql -U postgres -c 'CREATE DATABASE test;'"
Both bash -c and psql -c take one shell word as input. A single or double quoted string is one word. You need to nest the quotes, just like the -c commands are nested.
Alternative:
winpty docker exec -it postgres_db bash -c "psql -U postgres -c \"CREATE DATABASE test;\""

Docker exec - cannot call postgres with environment variables

I have multiple Environment Variables defined on my Postgres container, such as POSTGRES_USER. The container is running and I want to connect to Postgres from the command line using exec.
I'm unable to connect with the following:
docker exec -it <CONTAINER-ID> psql -U $POSTGRES_USER -d <DB NAME>
I understand that the variable is defined on the container and the following does work:
docker exec -it <CONTAINER-ID> bash -c 'psql -U $POSTGRES_USER -d <DB NAME>'
Is there a way for me to execute the psql command directly from docker exec and call the environment variable on the container?
docker exec -it <CONTAINER-ID> psql -U ????? -d <DB NAME>
Depending on your use case, what you could do, instead of passing a user to the psql command is to define the environment variable PGUSER to the container at boot time.
This way, it will be the default user for PostgreSQL, if you do not specify any, so you won't even have to specify it in order to connect:
$ docker run --name postgres -e POSTGRES_PASSWORD=bar -e POSTGRES_USER=foo -e PGUSER=foo -d postgres
e250f0821613a5e2021e94772a732f299874fc7a16b340ada4233afe73744423
$ docker exec -ti postgres psql -d postgres
psql (12.4 (Debian 12.4-1.pgdg100+1))
Type "help" for help.
postgres=#
The reason this isn't working for you is because when you run the command
docker exec -it <CONTAINER-ID> psql -U $POSTGRES_USER -d <DB NAME>
You're running it on your host. So, $POSTGRES_USER refers to the environment variable on your host, not your container. That variable isn't set on your host.
The second command
docker exec -it <CONTAINER-ID> bash -c 'psql -U $POSTGRES_USER -d <DB NAME>'
works because you're passing the command in the quotes to the shell in the container, where that variable actually exists.
The method in the second command is the way to do what you're trying to do, unless you set the variable on your host somehow and make sure it has the same value as it does in your container.
The easiest way to do this would be to reference your host variable at image build time.
So, in your Dockerfile, if you write ENV POSTGRES_USER=${POSTGRES_USER} it will look in the host environment for that value, and use it.
If you set the variables this way, then your command will work.

Problem with nested quotes in bash: mongodump with query in a docker container via ssh

I'm backing up my database that's in a docker container, and since the total filesize is too large to fit onto the remaining disk space I execute it via SSH and dump it onto my local pc with this command (I'm using Ubuntu default bash):
docker-machine ssh my-machine-image "docker exec container-id /bin/sh -c 'mongodump --archive -u=admin --authenticationDatabase=admin -p=mongo-pwd --db=my-db --collection=my-collection --gzip'" > myfile.dump
This works pretty great, however I'm having trouble getting it to work with the --query command. Mongodump requires it to be in strict JSON and I'm having trouble with getting the nested quotes in bash to work. My most successful attempt (aka it actually successfuly executed the command instead of returning a syntax/JSON error) was with a string literal like this, however that seems to parse the JSON wrong, since it always returns 0 documents, no matter the query:
docker-machine ssh my-machine-image "docker exec container-id /bin/sh -c $'mongodump --archive -u=admin --authenticationDatabase=admin -p=mongo-pwd --db=my-db --collection=my-collection --query=\'{ \"_id\": { \"$oid\": \"some_random_object_id\" } }\' --gzip'" > myfile.dump
What is the correct way to pass strict JSON to the --query parameter with this amount of nested quotes?
Since you have multiple layers of quoting it would be easiest to assign each layer to a variable. Then use bash's printf %q to automatically quote any string for use in a shell.
#! /usr/bin/env bash
json='{"_id": { "'"$oid"'": "some_random_object_id" } }'
cmd="mongodump --archive -u=admin --authenticationDatabase=admin -p=mongo-pwd --db=my-db --collection=my-collection --query=$(printf %q "$json") --gzip"
sshCmd="docker exec container-id /bin/sh -c $(printf %q "$cmd")"
docker-machine ssh my-machine-image "$sshCmd"

Postgres dump specific table with a capital letter

I am trying to perform a postgres dump of a specific table using -t. However, the table has a capital letter in it and I get a "No matching tables were found." I tried using quotations and double quotations around the table name but they did not work. How can I get pg to recognize the capitals? Thanks!
pg_dump -h hostname dbname -t tableName > pgdump.sql
Here is the complete command to dump your table in plain mode:
pg_dump --host localhost --port 5432 --username "postgres" --role "postgres" --format plain --file "complete_path_file" --table "schema_name.\"table_name\"" "database_name"
OR you can just do:
pg_dump -t '"tablename"' database_name > data_base.sql
Look to the last page here: Documentation
The above solutions do not work for me under Windows 7 x64. PostgreSQL 9.4.5. But this does, at last (sigh):
-t "cms.\"FooContents\""
either...
pg_dump.exe -p 8888 --username=user -t "cms.\"FooContents\"" basdb
...or...
pg_dump.exe -p 8888 --username=user -table="cms.\"FooContents\"" basdb
Inside a cmd window, I had to put three (!) double quotes around the table name if it countains upper case letters.
Example
pg_dump -t """Colors""" database > database.colors.psql
This worked for me:
pg_dump -f file.sql -U user -t 'schema.\"Table\"' database
As part of a node script I had to surround with single and double quotes, e.g.
` ... --table 'public."IndexedData"'`
The accepted solution worked in a bash console, but not as part of a node script, only the single quote approach.
Thanks to #Dirk Zabel suggestion, the following worked for me:
Windows 10 CMD
pg_dump -d "MyDatabase" -h localhost -p 5432 -U postgres --schema=public -t """TableName""" > TableName.sql
Bash
pg_dump -d "MyDatabase" -h localhost -p 5432 -U postgres --schema=public -t "\"TableName\"" > TableName.sql
Powershell
the good (shortest)
& 'C:\Program Files\PostgreSQL\12\bin\pg_dump.exe' -d db_name -t '\"CasedTableName\"'
the bad (requires --%)
& 'C:\Program Files\PostgreSQL\12\bin\pg_dump.exe' --% -d db_name -t "\"CasedTableName\""
the ugly (requires `")
& 'C:\Program Files\PostgreSQL\12\bin\pg_dump.exe' -d db_name -t "\`"CasedTableName\`""
The main point of confusion for me was the absolute necessity of having \" in there. I assumed that maybe there was a weird bug in the way powershell or psql was parsing the arguments, but it turns out it's explained in the docs:
Some native commands expect arguments that contain quote characters. Normally, PowerShell's command line parsing removes the quote character you provided. The parsed arguments are then joined into a single string with each parameter separated by a space. This string is then assigned to the Arguments property of a ProcessStartInfo object. Quotes within the string must be escaped using extra quotes or backslash (\) characters.
And of course ProcessStartInfo.Arguments Remarks tells us:
To include quotation marks in the final parsed argument, triple-escape each mark.