This question already has answers here:
How to escape single quotes within single quoted strings
(25 answers)
Closed 1 year ago.
I'm trying to insert data into some tables of a Postgres database with a bash script. I got it working like this
psql -c '\copy raw.ap1_1 from file.csv with csv header;'
The problem is I wan't to give the copy command the Quote option, but I can't get it to work because of the quotes outside. I've tried wrapping single quotes in double quotes, double quotes in single quotes, using a backslash to escape quotes, etc.
What is want is something like this:
psql -c '\copy raw.ap1_1 from file.csv with csv header quote as '"';'
But of course I can't do that because the quotes are taken as part of the bash script. I either get a postgres syntax error, or I get a parsing EOF error (because the quotes override themselves).
This should work:
psql -c "\\copy raw.ap1_1 from file.csv (format 'csv', header, quote '\"')"
PostgreSQL's SQL offers an alternative quoting method called dollar quoting. That is probably the easiest way to do it.
psql -c '\copy raw.ap1_1 from file.csv with csv header quote as $$"$$'
Note that neither dollar signs nor double quotes are special inside shell single quotes. Also note that \copy doesn't end with a ;. Further note that specifying a quote character which is the same as the default seems a bit silly.
If you don't want to use dollar quoting, you could escape correctly at the shell level. You can't do that from within the shell's single quote so first you have to close the quotes, then print '"' each one escaped with a backslash. (If this were not the end of the line, you would then re-open the shell's single quote to finish the line.)
psql -c '\copy raw.ap1_1 from file.csv with csv header quote as '\'\"\'
First of all ,you will login . Then
Copy table_name from 'C:\\\\expdata_202111122321.csv' with delimiter ';' csv header encoding 'windows-1254';
Related
My problem is that I have this command:
sed -i -E 's/((^|\s)PATH=)([^\$]*)$/\1${PATH:-\3}/g' /foo
and as far as I use it with single quotes it works fine but now I need to use it with double-quotes (because Dockerfile doesn't like single ones) and when I try to translate it to:
"s/((^|\s)PATH=)([^\$]*)$/\1${PATH:-\3}/g"
I get this error: unknown option to `s'. So I read that I can try to replace the separators and I tried with #:
"s#((^|\s)PATH=)([^\$]*)$#\1${PATH:-\3}#g"
But with that I get: unterminated `s' command. I also tried to replace with ;:
"s;((^|\s)PATH=)([^\$]*)$;\1${PATH:-\3};g"
and that "succeeded" but the end result was different to what I expected. Any ideas on how can I translate this properly to use it with doubled-quotes?
Don't! You can easily translate it to a string with backslashes and no quotes:
$ printf "%q\n" 's/((^|\s)PATH=)([^\$]*)$/\1${PATH:-\3}/g'
s/\(\(\^\|\\s\)PATH=\)\(\[\^\\\$\]\*\)\$/\\1\$\{PATH:-\\3\}/g
$ sed -E s/\(\(\^\|\\s\)PATH=\)\(\[\^\\\$\]\*\)\$/\\1\$\{PATH:-\\3\}/g
As mentioned by #glenn jackman escaping backslashes and dollar signs does the work, this worked for me:
"s/((^|\\s)PATH=)([^\\\$]*)\$/\\1\${PATH:-\\3}/g"
Thanks!
I have a file like this test.sql:
'here is' and \' other for 'you'
and would like to replace the \' (escaped single quote) with '' (2 singles quotes) for postgres and leave other single quotes alone. How would I do this. I have tried:
Mon Mar 16$ sed -i.bak s/\'/''/g test.sql
but this takes out all the single quotes.
Your enemy in this is shell quoting. The string
s/\'/''/g
is mangled by the shell before it is given to sed. For the shell, '' is an empty string, and \' suppresses this special meaning of single quotes (so that the quote is an actual single quote character). What sed sees after processing is
s/'//g
...wich just removes all single quotes.
There are several ways to work around the problem; one of them is
sed -i.bak "s/\\\\'/''/g" test.sql
Inside the doubly-quoted shell string, backslashes need to be escaped (exceptions exist). This means that "s/\\\\'/''/g" in the shell command translates to s/\\'/''/g as argument to sed. In sed regexes, backslashes also need escaping, so this is, in fact, what we wanted to happen: All instances of \' will be replaced with ''.
sed "s/[\\]'/''/g" test.sql
# also work but may depend on shell
sed "s/[\]'/''/g" test.sql
same idea as Wintermute but using class to avoig multi escaping for shell than sed in double quote
The context of the problem is that i am trying to pass passphrases that have single quotes in them to the input_password parameter of an openssl configuration file.
Taken from openssl configuration file manual:
It is possible to escape certain characters by using any kind of quote
or the \ character. By making the last character of a line a \ a value
string can be spread across multiple lines. In addition the sequences
\n, \r, \b and \t are recognized
I assumed that single quotes can be escaped. The problem is i am having difficulty producing escaped single quotes with sed.
With this, i mean that the following expressions do not yield the same results if ran from a script or from the command line. Take the following commands:
cut -d: -f2 "$EMPLOYEE_CREDENTIALS_FILE"| sed -e "s|'|\\\'|g"
Where "$EMPLOYEE_CREDENTIALS_FILE" is a file with formated like
username:password
When i run this command directly in bash it produces the output i assumed openssl.conf tolerates:
tentacle caravan sovereignty appraisal sass\'s
And when i run it in the script it just produces the normal unescaped passphrase:
tentacle caravan sovereignty appraisal sass's
I would like to know what is it that i am doing wrong.
I have finally solved the problem.
I am now in a position to confirm that openssl configuration files allow escaped single quotes to appear in the input_password field. Not only that but the actual password used by openssl is unescaped as intended. An end user would never know that the password was escaped in the configuration file.
I had quite some trouble making sed work inside a script because double quotes, from what i understood, consume the backslashes but not the single quotes. On the other hand, single quotes don't consume the backslashes and because of that can't interpret escaped characters. This means i can't escape single quotes inside single quotes.
This took me to a solution
cut -d: -f2 "$EMPLOYEE_CREDENTIALS_FILE"| sed -e "s|'|"'\\\'"'|g"
This is similar to the answer found in this stackoverflow question but it doesn't work for my case.
The reason 3 backslashes are need are also found in the answer:
By using double quotes around the sed command, you remove the need to
worry about embedded single quotes. You do have to then worry about
escaping since the shell will absorb one level of escapes.
I want to dump a select query to a tab-delimited text file using psql -F. However, this doesn't work:
psql -Umyuser mydb -F '\t' --no-align -c "select * from mytable" -o /tmp/dumpfile.txt
That makes the delimiter a literal \t. How do I get it to use real tabs instead?
I think you just need to use a literal tab. How this works depends on your shell. Have you seen this post?
In the bash shell you can do this with $'\t'.
Using the example in your question:
psql -Umyuser mydb -AF $'\t' --no-align -c "select * from mytable" -o /tmp/dumpfile.txt
From man bash:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard. [...] The expanded result is single-quoted, as if the dollar sign had not been present.
In Unix, you can also type
ctrl-V tab
ctrl-V tells the terminal not to interpret the next key.
This also works with carriage returns (^M) and many other special keys like arrow keys
In case if somebody looked for how to do it in the interactive shell:
\f '\t'
I want to replace the double quotes in the sed command in the following example with single quotes.
set new_string to do shell script "echo " & quoted form of list_string & " | sed -e 's/$/\"/' -e 's/^/\"/' -e 's/^/+/'"
However if I replace the double quotes with single quotes I get an error, is there a way to escape single quotes?
I'm no sed ninja, so any hints on how to go about this is highly appreciated.
if you want to replace " with ' using sed:
sed 's/"/\x27/g' yourFile
\x27 - single
\x22 - double
it could make code looks cleaner, and with less escape.
see the test:
kent$ cat quote.tmp
""""""
kent$ sed 's/"/\x27/g' quote.tmp
''''''
fYou had a quotation fault. Just to replace double quotes for single quotes, this is enough
set list_string to "This program said: \"Hello World!\""
set new_string to do shell script "/bin/echo -n " & quoted form of list_string & " | sed -e 's/\"/'\\''/g'"
Explaining 's/\"/'\''/g'
The \\ and \" is needed in the applescript environment and will be in the shell just \ and ". So what's entering the shell is 's/"/'\''/g'. Then what's with all the quotes? A very common mistake is thinking that quotations on the command line works the same as in programming. A single quote turns substitution on or off. So the first single quote turns substitution off which mean the next characters will be interpreted as text and has no special meanings (including the escape character). So to escape a single quote we'll need to turn the substitution on, then we can escape a single quote and turn the substitution off again.
You need to be careful about which quotes are being parsed by sed and which are being parsed by the environment invoking sed. Normal invocations of sed come from shell scripts, but (based on your tag) it appears that you're calling it from an AppleScript.
From a shell script you would say
| sed -e 's/$/'\''/' -e 's/^/'\''/' -e 's/^/+/'
But I don't know if sh-style escaping rules are in effect for you or whether you need to additionally escape the \