I have the following content in a file
dhcp_option_domain:
- test.domain
And what I need to do is this:
whenever the value 'dhcp_option_domain:' followed by a newline and then ANY string, replace it with 'dhcp_option_domain:' followed by a newline and a variable.
ie if I set a variable of dhcp_domain="different.com" then then string above would convert to:
dhcp_option_domain:
- different.com
Note that both lines have and need to maintain leading 2 spaces.
I do not want to just do a search and replace on 'test.domain' as I have a few cases to use this and the values could be different each time the sed command is run.
I have tried a few methods such as:
dhcp_domain="something.com"
sed -i 's|dhcp_option_domain:\n.*|dhcp_option_domain:\n - $dhcp_domain|g' filename
however cannot get it to work.
Thanks.
As the manual explains:
sed operates by performing the following cycle on each line of input: first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space. Then commands are executed
Your regex (dhcp_option_domain:\n.*) does not match because there is no \n in the pattern space in the first place.
A possible solution:
sed '/dhcp_option_domain:$/{n;c\
- '"$dhcp_domain"'
}'
The /dhcp_option_domain:$/ part is an address. The following command is only executed on lines matching that pattern.
The { } command groups multiple commands into a single block.
The n command prints out the current pattern space and replaces it by the next line of input.
The c\ command replaces the current pattern space by whatever follows in the script. Here it gets a bit tricky. We have:
a literal newline in the sed program (required after c\), then
- (placing those characters in the pattern space literally, then
' (part of shell syntax, terminating the single-quoted part started by sed '...), then
" (starting a double-quoted part), then
$dhcp_domain (which, because it's in a double-quoted part, interpolates the contents of the dhcp_domain shell variable), then
" (terminating the double-quoted part), then
' (starting another single-quoted part), then
a literal newline again (terminating the text after c\), then
} (closing the block started by {).
By default, sed works line by line (using newline character to distinguish newlines)
$ cat ip.txt
foo baz
dhcp_option_domain:
- test.domain
123
dhcp_option_domain:
$ dhcp_domain='something.com'
$ sed '/^ dhcp_option_domain:/{n; s/.*/ - '"$dhcp_domain"'/}' ip.txt
foo baz
dhcp_option_domain:
- something.com
123
dhcp_option_domain:
/^ dhcp_option_domain:/ condition to match
{} to group more than one command to be executed when this condition is satisfied
n get next line
s/.*/ - '"$dhcp_domain"'/ replace it as required - note that shell variables won't be expanded inside single quotes, see sed substitution with bash variables
for details
note that last line in the file didn't trigger the change as there was no further line
tested on GNU sed, syntax might vary for other implementations
From GNU sed manual
n
If auto-print is not disabled, print the pattern space, then,
regardless, replace the pattern space with the next line of input. If
there is no more input then sed exits without processing any more
commands.
This might work for you (GNU sed):
sed '/dhcp_option_domain:$/{p;s// - '"${var}"'/;n;d}' file
Match on dhcp_option_domain:, print it, substitute the new domain name (maintaining indent), print the current line and fetch the next (n) and delete it.
Related
I have a file and I want to append a specific text, \0A, to the end of each of its lines.
I used this command,
sed -i s/$/\0A/ file.txt
but that didn't work with backslash \0A.
In its default operations, sed cyclically appends a line from input, less it's terminating <newline>-character, into the pattern space of sed.
The OP wants to use sed to append the character \0A at the end of a line. This is the hexadecimal representation of the <newline>-character (cfr. http://www.asciitable.com/). So from this perspective, the OP attempts to double space a files. This can be easilly done using:
sed G file
The G command, appends a newline followed by the content of the hold space to the pattern space. Since the hold space is always empty, it just appends a newline character to the pattern space. The default action of sed is to print the line. So this just double-spaces a file.
Your command should be fixed by simply enclosing s/$/\0A/ in single quotes (') and escaping the backslash (with another backslash):
sed -i 's/$/\\0A/' file.txt
Notice that the surrounding 's protect that string from being processed by the shell, but the bashslash still needed escape in order to protect it from SED itself.
Obviously, it's still possible to avoid the single quotes if you escape enough:
sed -i s/$/\\\\0A/ file.txt
In this case there are no single quotes to protect the string, so we need write \\ in the shell to get SED fed with \, but we need two of those \\, i.e. \\\\, so that SED is fed with \\, which is an escaped \.
Move obviously, I'd never ever suggest the second alternative.
I am new to sed and its functioning. I need to selectively replace space with "," in a file where the content of the file is as follows. I do not want replace space inside "" but all the other spaces needs to be replaced.
File Content
my data "this is my very first encounter with sed" "valuable" - - "c l e a r"
Used Pattern
using sed to replace space with "," - Patten is 's/ /,/g'
Actual Output
my,data,"this,is,my,very,first,encounter,with,sed",,"valuable",-,-,"c,l,e,a,r"
Expected Output
my,data,"this is my very first encounter with sed",,"valuable",-,-,"c l e a r"
The following sed script with comments with input from bash here string:
<<<'my data "this is my very first encounter with sed" "valuable" - - "c l e a r"' sed -E '
# Split input with each character on its own line
s/./&\n/g;
# Add a newline on the end to separate output from input
s/$/\n/;
# Each line has one character
# Add a leading character that stores "state"
# There are two states available - in quoting or not in quoting
# The state character is space when we are not in quotes
# The state character is double quote when we are in quotes
s/^/ /;
# For each character in input
:again; {
# Substitute a space that is not in quotes for a comma
s/^ / ,/
# When quotes is encountered and we are not in quotes
/^ "/{
# Change state to quotes
s//""/
b removed_quotes
} ; {
# When quotes is encountered and we are in quotes
# then we are no longer in quotes
s/^""/ "/
} ; : removed_quotes
# Preserve state as the first character
# Add the parsed character to the output on the end
# Preserve the rest
s/^(.)(.)\n(.*)/\1\3\2/;
# If end of input was not reached, then parse another character.
/^.\n/!b again;
};
# Remove the leading state character with the newline
s///;
'
outputs:
my,data,"this is my very first encounter with sed",,"valuable",-,-,"c l e a r"
And a oneliner, because who reads these comments:
sed -E 's/./&\n/g;s/$/\n/;s/^/ /;:a;s/^ / ,/;/^ "/{s//""/;bq;};s/^""/ "/;:q;s/^(.)(.)\n(.*)/\1\3\2/;/^.\n/!ba;s///'
I think a newline \n in s command replacement string is an extension not required by posix. Another unique character may be used instead of a newline to separate input while parsing. Anyway I tested that with GNU sed.
As mentioned in the comments, this is something better suited for an actual CSV parser instead of trying to kludge up something using regular expressions - especially sed's rather basic regular expressions.
A one-liner in perl using the useful Text::AutoCSV module (Install through your OS package manager or favorite CPAN client):
$ perl -MText::AutoCSV -e 'Text::AutoCSV->new(sep_char=>" ", out_sep_char=>",")->write' < input.txt
my,data,"this is my very first encounter with sed",,valuable,-,-,"c l e a r"
With GNU awk for FPAT:
$ awk -v FPAT='[^ ]*|"[^"]+"' -v OFS=',' '{$1=$1} 1' file
my,data,"this is my very first encounter with sed",,"valuable",-,-,"c l e a r"
Your input is a CSV where C in this case means "Character" instead of the traditional "Comma" and where the Character in question is a blank and you're just trying to convert it to a Comma-separated CSV. See What's the most robust way to efficiently parse CSV using awk? for more information on what the above does and on parsing CSVs with awk in general.
awk 'BEGIN {RS=ORS="\""} NR%2 {gsub(" ",",")} {print}' file
At the beginning, set the double quote as the record separator.
For odd records, i.e. outside quotes, replace globally any space with comma.
print every record.
This might work for you (GNU sed):
sed -E ':a;s/^((("[^"]*")*[^" ]*)*) /\1,/;ta' file
Replace, the group of zero or more double quoted strings followed by zero or more non-space characters zero or more time followed by a space with the group followed by a comma, repeated until failure.
I am trying to add multiple lines to a file, all with a leading a tab. The lines should be inserted on the first line after matching a string.
Assume a file with only one line, called "my-file.txt" as follows:
foo
I have tried the following sed command:
sed "/^foo\$/a \tinsert1\n\tinsert2" my-file.txt
This produces the following output:
foo
tinsert1
insert2
Notice how the the tab that should be on the first (inserted) line is omitted. Instead it prints an extra leading 't'.
Why? And how can I change my command to print the tab on the first line, as expected?
With GNU sed:
sed '/^foo$/a \\tinsert1\n\tinsert2' file
<---- single quotes! --->
Produces:
foo
insert1
insert2
From the manual:
a \
text Append text, which has each embedded newline preceded by a backslash.
Since the text to be append itself has to to be preceded by a backslash, it needs to be \\t at the beginning.
PS: If you need to use double quotes around the sed command because you want to inject shell variables, you need to escape the \ which precedes the text to be appended:
ins1="foo"
ins2="bar"
sed "/^foo\$/a \\\t${ins1}\n\t${ins2}" file
sed is for doing s/old/new on individual strings, that is all. Just use awk:
$ awk '{print} $0=="foo"{print "\tinsert1\n\tinsert2"}' file
foo
insert1
insert2
The above will work using any awk in any shell on every UNIX box and is trivial to modify to do anything else you might want to do in future.
I know there is a similar question in SO How can I replace mutliple empty lines with a single empty line in bash?. But my question is can this be implemented by just using the sed command?
Thanks
Give this a try:
sed '/^$/N;/^\n$/D' inputfile
Explanation:
/^$/N - match an empty line and append it to pattern space.
; - command delimiter, allows multiple commands on one line, can be used instead of separating commands into multiple -e clauses for versions of sed that support it.
/^\n$/D - if the pattern space contains only a newline in addition to the one at the end of the pattern space, in other words a sequence of more than one newline, then delete the first newline (more generally, the beginning of pattern space up to and including the first included newline)
You can do this by removing empty lines first and appending line space with G command:
sed '/^$/d;G' text.txt
Edit2: the above command will add empty lines between each paragraph, if this is not desired, you could do:
sed -n '1{/^$/p};{/./,/^$/p}'
Or, if you don't mind that all leading empty lines will be stripped, it may be written as:
sed -n '/./,/^$/p'
since the first expression just evaluates the first line, and prints it if it is blank.
Here: -n option suppresses pattern space auto-printing, /./,/^$/ defines the range between at least one character and none character (i.e. empty space between newlines) and p tells to print this range.
I want multi-line strings in java, so I seek a simple preprocessor to convert C-style multi-lines into single lines with a literal '\n'.
Before:
System.out.println("convert trailing backslashes\
this is on another line\
\
\
above are two blank lines\
But don't convert non-trailing backslashes, like: \"\t\" and \'\\\'");
After:
System.out.println("convert trailing backslashes\nthis is on another line\n\n\nabove are two blank lines\nBut don't convert non-trailing backslashes, like: \"\t\" and \'\\\'");
I thought sed would do it well, but sed is line-based, so replacing the '\' and the newline that follows it (effectively joining the two lines) is not very natural in sed. I adapted sredden79's oneliner to the following - it works, it's clever, but it's not clear:
sed ':a { $!N; s/\\\n/\\n/; ta }'
The substitute is of escaped literal backslash, newline with escaped literal backslash, n. :a is a label and ta is goto label if the substitute found a match; $ means the last line, and $! is the opposite (i.e. all lines but the last). N means to append the next line to the pattern space (thus making the \n character visible.)
EDIT here's a variation to keep compiler error line numbers etc accurate: it turns each extended line into "..."+\n (and handles the first and last lines of the String correctly):
sed ':a { $!N; s/\\\n/\\n"+\n"/; ta }'
giving:
System.out.println("convert trailing backslashes\n"+
"this is on another line\n"+
"\n"+
"\n"+
"above are two blank lines\n"+
"But don't convert non-trailing backslashes, like: \"\t\" and \'\\\'");
EDIT Actually, it would be better have Perl/Python style multi-line, where it starts and ends with a special code on one line (""" for python, I think).
Is there a simpler, saner, clearer way (maybe not using sed)?
Is there a simpler, saner, clearer way.
Forget the pre-processor, live with the limitation, complain about it (so that it will maybe be fixed in Java 7 or 8), and use an IDE to ease the pain.
Other alternatives (too troublesome I suppose, but still better than messing with the compilation process):
use a JVM-based language that does support here-docs
externalize the string into a resource file
A perl one-liner:
perl -0777 -pe 's/\\\n/\\n/g'
This will read either stdin or the file(s) named after it on the command line and write the output to stdout.
If you're using an editor that supports filtering, like vi or emacs, just filter your text through the above command and you're done:
If you're using Windows and have to worry about \r :
C:\> perl -0777 -pe "s/\\\r?\n/\\n/g"
although I think win32 Perl handles \r itself so this may be unnecessary.
The -0777 option is a special case of the -0 (that's a zero) option that defines the line or record separator. In this case, it means that we don't want any separator so read the entire file in as a single string.
The -pe option is a combination of -p (process line-by-line and print the result) and -e (next argument is (a line of) the program to execute)
A perl script to what you asked for.
while (<>) {
chomp;
print $_;
if (/\\$/) {
print "n";
} else {
print "\n";
}
}
sed 's/\x5c\x5c$/\x22\x5c\x5cn\x22/'
Hex for backslash and double quote is \x5c and \x22 respectively - it needs to be escaped so \x5c is doubled and the $ anchors to the end of the line.
Updated again per OP comment:
sed "{:a;N;\$!b a};s/\x5c\x5c\n/\x5c\x5cn/g"
The :a creates a label and the N appends a line to the pattern space, the b a branches back to the label :a except when its the last line $!;
After its all loaded - a single line substitution replaces all occurrences of a newline \n with a literal '\n' using the hex ascii code \x5c for the backslash.