I was replacing one line as at httpd.conf , facing problem withescaping backslash when am using sed ,
source line
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{True-Client-IP}i\"" combined
replaced with
LogFormat "%h %l %u %t %D \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{True-Client-IP}i\"" combined
any tip to escape all these special characters
Try this:
sed -e 's/%t \\"/%t %D \\"/'
Through sed,
$ sed -i 's/^\(.*%t\)\(.*\)$/\1 %D\2/g' file
LogFormat "%h %l %u %t %D \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{True-Client-IP}i\"" combined
OR
$ sed -i 's/%t/%t %D/' file
LogFormat "%h %l %u %t %D \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" \"%{True-Client-IP}i\"" combined
Related
How to convert a character from and to its decimal, binary, octal, or hexadecimal representations in BASH / Shell ?
Convert a character from and to its decimal, binary, octal, or hexadecimal representations in BASH with printf and od
Some relevant documentation and Q&A:
od man pages:
https://www.gnu.org/software/coreutils/manual/html_node/od-invocation.html
printf man pages:
https://www.gnu.org/software/coreutils/manual/html_node/printf-invocation.html
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html
What is the difference between UTF-8 and Unicode?
How do I print an ASCII character by different code points in Bash?
How to print an octal value's corresponding UTF-8 character in bash?
Unicode char representations in BASH / shell: printf vs od
Convert binary, octal, decimal and hexadecimal values between each other in BASH / Shell
Convert a character from and to its decimal representation
single_ascii_char="A"
echo -n $single_ascii_char | od -A n -t d1
65
printf %d "'$single_ascii_char"
65
code=65
printf "\u$(printf %04x $code)\n" # use \u for up to 4 hexadecimal digits
A
printf "\U$(printf %08x $code)\n" # use \U for up to 8 hexadecimal digits
A
single_unicode_char="😈"
printf %d "'$single_unicode_char"
128520
echo -n $single_unicode_char | iconv -t UTF-32LE | od -A n -t d # d or u, d4, u4, dI, dL
128520 # or UTF-32BE, depending on system's endianness
code=128520
printf "\u$(printf %04x $code)\n" # use \u for up to 4 hexadecimal digits
á½ 8
printf "\U$(printf %08x $code)\n" # use \U for up to 8 hexadecimal digits
😈
Convert a character from and to its binary representation
single_ascii_char="A"
echo "obase=2; $(printf %d "'$single_ascii_char")" | bc
1000001
code="1000001"
printf "\u$(printf %04x $((2#$code)) )\n" # use \u for up to 4 hexadecimal digits
A
printf "\U$(printf %08x $((2#$code)) )\n" # use \U for up to 8 hexadecimal digits
A
single_unicode_char="😈"
echo "obase=2; $(printf %d "'$single_unicode_char")" | bc
11111011000001000
code="11111011000001000" # with or without leading 0s
printf "\u$(printf %04x $((2#$code)) )\n" # use \u for up to 4 hexadecimal digits
á½ 8
printf "\U$(printf %08x $((2#$code)) )\n" # use \U for up to 8 hexadecimal digits
😈
Convert a character from and to its octal representation
single_ascii_char="A"
printf %o "'$single_ascii_char"
101
echo -n $single_ascii_char | od -A n -t o1
101
code="\101"
printf %b "$code\n"
A
printf "$code\n"
A
single_unicode_char="😈"
printf %o "'$single_unicode_char"
373010
echo -n $single_unicode_char | iconv -t UTF-32LE | od -A n -t o # or o4
00000373010 # or UTF-32BE, depending on system's endianness
code="00000373010" # insert at least one leading 0 for printf to understand it's an octal
printf "\U$(printf %08x "$code")\n"
😈
echo -n "$single_unicode_char" | od -A n -t c # c or o1
360 237 230 210
code="\360\237\230\210"
printf %b "$code\n"
😈
printf "$code\n"
😈
Convert a character from and to its hexadecimal representation
single_ascii_char="A"
printf %x "'$single_ascii_char"
41
echo -n "$single_ascii_char" | od -A n -t x1
41
code="41"
printf "\u$code\n" # use \u for up to 4 hexadecimal digits
A
printf "\U$code\n" # use \U for up to 8 hexadecimal digits
A
single_unicode_char="😈"
printf %x "'$single_unicode_char"
1f608
printf %X "'$single_unicode_char"
1F608
echo -n $single_unicode_char | iconv -t UTF-32LE | od -A n -t x
0001f608 # or UTF-32BE, depending on system's endianness
code="1f608"
printf "\u$code\n" # use \u for up to 4 hexadecimal digits
á½ 8
printf "\U$code\n" # use \U for up to 8 hexadecimal digits
😈
printf %#x "'$single_unicode_char"
0x1f608
printf %#X "'$single_unicode_char"
0X1F608
code="0x1f608"
printf "\u$(printf %04x $code)\n" # use \u for up to 4 hexadecimal digits
á½ 8
printf "\U$(printf %08x $code)\n" # use \U for up to 8 hexadecimal digits
😈
echo -n "$single_unicode_char" | od -A n -t x1
f0 9f 98 88
code="\xf0\x9f\x98\x88"
printf %b "$code\n"
😈
printf "$code\n"
😈
I tried quite a lot of thing and did some search without finding a proper way to do this :
I would like to modify the Apache LogFormat definition on several servers (a lot).
So : within httpd.conf,
replace this:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
with this:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %T" combined
LogFormat "%h %l %u %t \"%r\" %>s %b %T" common
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O %T" combinedio
without modifying that :
CustomLog "logs/access_log" common
CustomLog "logs/access_log" combined
So :
I'm looking to
s\" com\ %T" com\g', but only for lines containing LogFormat
And I'm starving to do such a thing.
You would be better served using a dedicated solution such as Ansible for this but using sed, you could do the following:
sed -i '/^LogFormat/s/" combined.*$/ %T&/' httpd.conf
Search for all lines starting with "LogFormat" and then substitute double quotes, space and "combined" for a space, "%T" and the pattern identified.
Can anyone please explain this perl behavior that i came across?
printf("%what_the\n");
printf("\%what_the\n");
Prints:
%what_the
%what_the
WHILE...
printf("%tomorrow\n");
printf("\%tomorrow\n");
Prints:
0morrow
0morrow
...EVEN WITH warnings and strict:
use strict;
use warnings;
printf("\%tomorrow\n");
Prints:
Missing argument in printf at - line 3.
0morrow
printf is different from regular print. You might be thinking it is the same, it is not. printf takes a pattern, which includes %. For example:
printf "%s\n", "tomorrow"; # prints "tomorrow\n"
%s is a placeholder for a string, which should be the second argument to printf.
The warning you get shows you the problem
Missing argument in printf at - line 3.
printf expects a second argument, because you have supplied a placeholder.
Not all letters following a percent sign is a valid combination, here's a few from the documentation from sprintf
%% a percent sign
%c a character with the given number
%s a string
%d a signed integer, in decimal
%u an unsigned integer, in decimal
%o an unsigned integer, in octal
%x an unsigned integer, in hexadecimal
%e a floating-point number, in scientific notation
%f a floating-point number, in fixed decimal notation
%g a floating-point number, in %e or %f notation
.... more
I do not see %to in there, but it seems to be what is being triggered. It prints a 0 because it casts the empty string (missing argument) to 0.
Documentation here.
The way to escape a % sign is to double it, not by a backslash. %o is the format for printing an octal number. Try doing printf "%tomorrow", 255;. The t is a modifier flag on %o to set the integer type.
https://perldoc.perl.org/functions/sprintf#size
HTH
I'm trying to find and replace specific string with sed command but when I run the command nothing seems to be happening.
FIND:
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
REPLACE WITH:
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
FILE CONTENT
hello
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
bye
COMMAND
$ sed -i 's/LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined/LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined/g' /etc/apache2/apache2.conf
I looked into How do I escape double and single quotes in SED? (bash) but cannot workout what the problem is.
Your file contains backslashes before quotes. Your sed regexp does likewise but in the sed case those are escape chars when you need literals. You need to escape the backslashes to make them literal, e.g.:
...\\"%r\\"..
See:
$ sed 's/LogFormat "%h %l %u %t \\"%r\\" %>s %O \\"%{Referer}i\\" \\"%{User-Agent}i\\"" combined/LogFormat "%{X-Forwarded-For}i %l %u %t \\"%r\\" %>s %b \\"%{Referer}i\\" \\"%{User-Agent}i\\"" combined/g' file
hello
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
bye
but consider using capture groups instead of duplicating all the text in the substitution:
$ sed 's/\(LogFormat "\)%h\( %l %u %t \\"%r\\" %>s %\)O\( \\"%{Referer}i\\" \\"%{User-Agent}i\\"" combined\)/\1%{X-Forwarded-For}i\2b\3/g' file
hello
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
bye
For some reason I am having a lot of trouble parsing date strings using Time::Piece.
So this works:
my $t = Time::Piece->strptime( 'Sunday, November 4, 2012 10:25:15 PM -0000' , "%A, %B %d, %Y %I:%M:%S %p %z" );
print $t;
But this does not:
my $temp_time = Time::Piece->strptime('7/23/2014 5:24:22 PM', "%-m/%-d/%Y %l:%M:%S %p");
print $temp_time;
I have also used '%D %r' as the format string but that also does not work. Do any of you have insight as to why this might be? For reference , the hour is 1-12 (not 01-12) and the month is 1-12 (not 0-12).
Thanks!
Change
"%-m/%-d/%Y %l:%M:%S %p"
to
"%m/%d/%Y %l:%M:%S %p"