sed pattern to insert a character after first m characters and then after every n characters - sed

If I have a string and I want to use sed such that a colon is inserted after the first 8 characters and then after every 2 characters after it, what would the sed pattern look like and what would the replacement pattern look like?


awk has substr() function, it can solve your problem eaiser:
awk -v m="8" -v n="2"
'{a=substr($0,1,m);
b=substr($0,m+1);
gsub(".{"n"}","&:",b)}$0=a":"b'
You can change the m and n to desired value (error handling was not done in my one-liner). If we do a test:
$ awk -v m="8" -v n="2" '{a=substr($0,1,m);b=substr($0,m+1);gsub(".{"n"}","&:",b)}$0=a":"b'<<<"aaaaaaaaaaaaaaa"
aaaaaaaa:aa:aa:aa:a


This might work for you (GNU sed):
sed 's/../&:/4g' file
Insert a : after the 4th occurrence of 2 characters and then globally throughout the line.

Related

Replace variable num in double quotes - SED

I have a line like this:
"abc/x-y-z": "^1.4"
I need to replace ^1.4with * in the same file such that the output is "abc/x-y-z": "*"
The num inside the double quotes could be any variable number.
I tried this but it is highly specific to ^1.4 number:
sed -i '21s/^1.4/*/' abc.json

With your shown samples, please try following. You need to escape ^ here to make it literal character and you need to escape . dot as well to make it treat as literal character.
sed 's/\^1\.4/*/' Input_file
OR as per OP's comment to make it dynamic try:
sed 's/\^[0-9]+\.[0-9]+/*/' Input_file
Also if you are performing it on 21st line of your file then use 21s like you tried in your attempt. This code will substitute only very 1st occurrence of ^1.4 here in case you want to substitute all occurrences then use g(globally substitution) option for above code.
I have not used -i option(to do inplace update into Input_file itself) once you are happy with results then use sed -i option in above code.

When making changes to files, I prefer the file editor ed to the stream editor sed (ed is standard, sed -i isn't, and different versions have different quirks that bite people here on a regular basis).
ed -s input.txt <<EOF
21s/"^[[:digit:]]\{1,\}\(\.[[:digit:]]\{1,\}\)\{0,1\}"/"*"/
w
EOF
On line 21, matches a quote followed by a carat followed by 1 or more digits, optionally followed by a period and another sequence of 1 or more digits and finally the trailing quote character. All that is replaced by "*", and finally the changed file is written back to disk.
Posix BREs are a pain, no? GNU ed 1.17 and newer, and NetBSD ed can take EREs instead:
ed -Es input.txt <<EOF
21s/"\^[[:digit:]]+(\.[[:digit:]]+)?"/"*"/
w
EOF
which is a lot easier to read.

Replace multiple lines using sed with other multiple lines

I am trying to find these two lines:
<paramsToUseForLimit></paramsToUseForLimit>
</hudson.plugins.throttleconcurrents.ThrottleJobProperty>
and replace them with:
<paramsToUseForLimit/>
</hudson.plugins.throttleconcurrents.ThrottleJobProperty>
<jenkins.branch.RateLimitBranchProperty_-JobPropertyImpl plugin="branch-api#2.6.3">
<durationName>hour</durationName>
<count>2</count>
<userBoost>false</userBoost>
</jenkins.branch.RateLimitBranchProperty_-JobPropertyImpl>
from my file config.json
Can someone help me to do this with sed ?
sed -ie "s/Those two lines/Replaced with those 7 lines/g" /config.json

sed may not be the best tool for this task.
sed -ie '/^ <\(paramsToUseForLimit>\)<\/\1/{
N
/\n<\/hudson.plugins.throttleconcurrents.ThrottleJobProperty>/{
i \
<paramsToUseForLimit/>\
</hudson.plugins.throttleconcurrents.ThrottleJobProperty>\
<jenkins.branch.RateLimitBranchProperty_-JobPropertyImpl plugin="branch-api#2.6.3">\
<durationName>hour</durationName>\
<count>2</count>\
<userBoost>false</userBoost>\
</jenkins.branch.RateLimitBranchProperty_-JobPropertyImpl>
d
}
}' /config.json
/^.../{ - if pattern space matches first match line:
N - append next line of input to pattern space
/\n.../{ - if pattern space matches second match line:
i \ - insert new text (\ before embedded newlines)
d- delete original text, implicit print, and start next cycle
otherwise, implicit print (two lines if N ran, else one line)
start next cycle
This is quite fragile. For example, watch out for shell metacharacters in the inserted text.

With GNU sed with -z option and inspired with Escape a string for a sed replace pattern question and answer - you can properly escape the patterns and replace newlines with \ n characters and then pass to sed. Then it's simple:
KEYWORD=' <paramsToUseForLimit></paramsToUseForLimit>
</hudson.plugins.throttleconcurrents.ThrottleJobProperty>'
REPLACE=' <paramsToUseForLimit/>
</hudson.plugins.throttleconcurrents.ThrottleJobProperty>
<jenkins.branch.RateLimitBranchProperty_-JobPropertyImpl plugin="branch-api#2.6.3">
<durationName>hour</durationName>
<count>2</count>
<userBoost>false</userBoost>
</jenkins.branch.RateLimitBranchProperty_-JobPropertyImpl>'
ESCAPED_KEYWORD=$(printf '%s\n' "$KEYWORD" | sed -z 's/[]\/$*.^[]/\\&/g; s/\n/\\n/g');
ESCAPED_REPLACE=$(printf '%s\n' "$REPLACE" | sed -z 's/[\/&]/\\&/g; s/\n/\\n/g')
sed -z "s/$ESCAPED_KEYWORD/$ESCAPED_REPLACE/" input_file.txt

Substring file name in Unix using sed command

I want to substring the File name in unix using sed command.
File name : Test_Test1_Test2_10082019_030013.csv.20191008-075740
I want the characters after the 3rd underscore or (all the characters after Test2 ) i need to be printed .
Can this be done using sed command?
I have tried this command
sed 's/^.*_\([^_]*\)$/\1/' <<< 'Test_Test1_Test2_10082019_030013.csv.20191008-075740'
but this is giving result as 030013.csv.20191008-075740
I need it from 10082019_030013.csv.20191008-075740
Thanks
Neha

To remove from the beginning up to including the 3rd underscore you can use
sed 's/^\([^_]*_\)\{3\}//' <<< 'Test_Test1_Test2_10082019_030013.csv.20191008-075740'
This removes the initial part that consists of 3 groups of (any number of non-underscore characters followed by an underscore). The result is
10082019_030013.csv.20191008-075740
If you use GNU sed you can switch it to extended regular expressions and omit the backslashes.
sed -r 's/^([^_]*_){3}//' <<< 'Test_Test1_Test2_10082019_030013.csv.20191008-075740'

Could you please try following.
sed 's/\([^_]*\)_\([^_]*\)_\([^_]*\)_\(.*\)/\4/' Input_file
Or as per Bodo's nice suggestion:
sed 's/[^_]*_[^_]*_[^_]_\(.*\)/\1/' Input_file

This might work for you (GNU sed):
sed 's/_/\n/3;s/.*\n//;t;s/Test2/\n/;s/.*\n//;t;d' file
Replace the third _ by a newline and then remove everything upto and including the first newline. If this succeeds, bail out and print the result. Otherwise, try the same method with Test2 and if this fails delete the entire line.

sed command to substitute any content after equals on line 3 with a string

I am trying to substitute this line
<data_item name="any_text">
which is on line 3 with
<data_item name="my_text">
So I tried something like sed '3s/=*/=my_text/' input_file > output_file
But this is printing my text at the beginning of the line. Tried braces around (=*) and (=my_text) but that doesn't do anything.

Try this
sed '3s/any_text/my_text/' file
Example:
$ echo '<data_item name="any_text">' | sed '1s/any_text/my_text/'
<data_item name="my_text">
Your code:
sed '3s/=*/=my_text/'
Your code will replace 0 or more equals on the third line with =my_text.

sed '3s/=".*"/="mytext"/'
should work. You need the . because it matches any character and the * says you want 0 or more of the preceding symbol .. In regex terms the * doesn't mean match anything. It is a quantifier which means it specifies and amount and is generally used to specify the number of the preceding terms you want to match.

sed '3 c\
<data_item name="my_text">' YourFile
Assuming that the content of my_text does not contain unescape \ & and '
avoid < and > on same file in samle instruction, could have unexpected result. Use a temporary file on -i on GNU sed

Using sed to swap columns X and X+1 inline in delimited file

I have a file with multiple lines and for line 2 to the end of the file I want to swap fields 8 and 9. The file is comma separated and I'd like to do the swap inline so I can run it on a batch of files using * wildcard. If this can be accomplished similarly with awk then that works for me too.
example:
header1,header2,header3,...,header8,header9,...,headerN
field1.1,...,field1.9,field1.8,...,field1.N
field2.1,...,field2.9,field2.8,...,field2.N
field3.1,...,field3.9,field3.8,...,field3.N
...
I think the command would look similar to sed -r -i '2,$s/^(([^,]*,){8})([^,]*,)([^,]*,)(.*)/\1\3\2\4/' temp*.log,
but \2 is not what I expect, it is the 7th field. I know that \2 will not be the 8th field because I have double parentheses there, but I'm not sure how to fix it. Could somebody please explain what this equation is doing and specifically what [^,] is doing and how the {8} is applied?
Thanks in advance.

In awk, you might use:
awk -F',' 'BEGIN {OFS=","} {t = $8; $8 = $9; $9 = t; print}'
In sed, the command is more convoluted, but it could be done.
sed -e 's/^\(\([^,]*,\)\{7\}\)\([^,]*,\)\([^,]*,\)/\1\4\3/'
Add the -i .bak option if your version of sed (e.g. GNU or BSD) supports it.
This uses the universally available sed regexes (it would work on even archaic versions of sed). You could lose most of the backslashes if you used 'extended regular expressions' instead:
sed -r -i 's/^(([^,]*,){7})([^,]*,)([^,]*,)/\1\4\3\5/'
Note the nested remembered (captured) patterns. The outer set is \1, the inner set would be \2 but that gets repeated 7 times, so you'd have the seventh field as \2. Anyway, that's why the eighth and ninth columns are switched with \4 and \3. \5 are the remaining columns.
(I note in passing that it would have been helpful to have some sample data in sufficiently the correct format to test with. It was a nuisance having to edit what is shown in the question to be able to test the code.)
If you need to do much CSV work, then either use Perl and its CSV modules (Text::CSV and Text::CSV_XS) or Python and its CSV module, or get CSVfix.

$2 is the second part in the RE
Denumbered by first occurence of (.
So in
'2,$s/^(([^,]*,){8})([^,]*,)([^,]*,)(.*)/\1\3\2\4/'
You could see (followind alignment):
$1 = (([^,]*,){8})
$2 = ([^,]*,)
$3 = ([^,]*,)
$4 = ([^,]*,)
and finaly $5 = (.*)
In this specific case, $2 must hold the last match of the height ({8}).

it seems that awk is the right tool:
awk -F',' -v OFS=',' '{t=$8;$8=$9;$9=t}7' file

This might work for you (GNU sed):
sed -ri '1!s/(,[^,]*)(,[^,]*)/\2\1/4' file
This swaps the 9th field with the 8th i.e. 8 / 2 = 4, if you wanted the 7th with the 8th:
sed -ri '1!{s/^/,/;s/(,[^,]*)(,[^,]*)/\2\1/4;s/^,//}' file