I have a file with multiple lines of
When i use the connection and request TEXT1 using mystring1.service
When i use the connection and request TEXT2 using mystring2.service
When i use the connection and request TEXT3 using mystring3.service
and I would like to replace it with
When i send a systemctl command through ssh and request TEXT1 using mystring1.service
When i send a systemctl command through ssh and request TEXT2 using mystring2.service
When i send a systemctl command through ssh and request TEXT3 using mystring3.service
where i pass in the 2 words after request and using into the new string.
I am trying to use sed and replace them with groups, it is however not working:
sed "s/When i use the connection and request ([\S]+) using ([\S]+)/When I send a systemctl command through ssh and request \1 using \2/g" input.txt > output.txt
I would do the minimal thing first (as suggested by #NateT):
sed 's/use the connection/send a systemctl command through ssh/' input.txt > output.txt
There are a couple of problems with your sed command. First, you need to use the -r option, or you need to escape the parenthesis and the + characters. Also, if you're going to use the \S token with sed, you don't need to square brackets. Here is a command equivalent to the one that you presented, but without the errors:
sed -r 's/When i use the connection and request (\S+) using (\S+)/When i send a systemctl command through ssh and request \1 using \2/' input.txt > output.txt
Or escaping the characters without the -r option:
sed 's/When i use the connection and request \(\S\+\) using \(\S\+\)/When i send a systemctl command through ssh and request \1 using \2/' input.txt > output.txt
Related
I'm trying to replace a line of text with a single quote using /x27 in my router device to re-configure wifi settings.
The command that I use
sed -i 's/option ssid.*/option ssid \x27test\x27/g' /some/file
The output after I execute above command on my computer(ubuntu 18)
option ssid 'test123'
The output after I execute above command on my router
option ssid x27test123x27
Expected output should be
option ssid 'test123'
So my code is not working properly on the router. How do I achieve this using /x27?
Note: I'm executing this script in a shell script and \x27test\x27 is actually \x27$OPTARG\x27
You can find my full code here
You can double quote Your sed command and then use single quotes inside:
sed -i "s/option ssid.*/option ssid 'test'/g" /some/file
In a comment you said your real code is:
sed -i 's/option ssid.*/option ssid '"'"'"'$OPTARG'"'"'"'/g'
That should be written more correctly and concisely as:
sed -i 's/\(option ssid\).*/\1 '"'$OPTARG'"'/g'
Look:
$ echo 'hi option ssid question' | sed 's/\(option ssid\).*/\1 '"'$RANDOM'"'/'
hi option ssid '5203'
Given your feedback to a different answer it sounds like the version of sed you're running requires a backup file name between the -i and the s/.... Otherwise there's a bug somewhere earlier in your shell script or $OPTARG contains forward slashes or backreferences.
I need to SFTP a file to a server. The password has a dollar sign $ and I need to escape it.
I tried with Perl and sed commands I am able to replace but the string following $ is not getting added.
Example:
echo "Np4$g" | perl -pe 's/$/\\\\\$/g'
output
Np4\\$
It supposed to be Np4\\$g, but g is not getting appended.
Code:
/usr/bin/expect <<EOF
set timeout -1
spawn sftp -C -oPort=$port $sftp_username#$host_name
expect "password:"
send "$password\r"
expect "sftp>"
cd $remote_dir
send "mput *.txt\r"
expect "sftp>"
send
Your command
echo "Np4$g" | perl -pe 's/$/\\\\\$/g'
is failing for two reasons
In "Np4$g", the shell is interpolating the variable g into the double-quoted string. It probably isn't defined so it is replaced with nothing, and you are passing just Np4 to perl. You need to use single quotes to prevent the interpolation
In the Perl substitution s/$/\\\\\$/g the $ in the pattern matches the end of the string, not a literal dollar. That means Np4 is changed to Np4\\$. You need to escape the dollar sign in the pattern to get it to match a literal $
This will work correctly
echo 'Np4$g' | perl -pe 's/\$/\\\$/g'
output
Np4\$g
I suggest to not escape and replace
"Np4$g"
by
'Np4$g'
I am trying to execute sed command inside TCL script . Basically i wanted to remove all empty lines from the input file before reading the file using TCL. so i tried following in my script
exec sed -i '/^\s*$/d' .tmp.PG_Ring
set fid [open ".tmp.PG_Ring" r]
But the script is dumping following Error .
sed: -e expression #1, char 1: unknown command: `''
while executing
"exec sed -i '/^\s*$/d' .tmp.PG_Ring"
(file "pg_ring.tcl" line 1)
could you please provide me work around for this & help me with best way to do this?
That won't work, as single quotes have no special meaning to Tcl at all. Tcl uses braces to mean the same sort of thing (except they nest nicely), so instead you can use this:.
exec sed -i {/^\s*$/d} .tmp.PG_Ring
I have tried to scan through the other posts in stack overflow for this, but couldn't get my code work, hence I am posting a new question.
Below is the content of file temp.
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/<env:Body><dp:response xmlns:dp="http://www.datapower.com/schemas/management"><dp:timestamp>2015-01-
22T13:38:04Z</dp:timestamp><dp:file name="temporary://test.txt">XJzLXJlc3VsdHMtYWN0aW9uX18i</dp:file><dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:file></dp:response></env:Body></env:Envelope>
This file contains the base64 encoded contents of two files names test.txt and test1.txt. I want to extract the base64 encoded content of each file to seperate files test.txt and text1.txt respectively.
To achieve this, I have to remove the xml tags around the base64 contents. I am trying below commands to achieve this. However, it is not working as expected.
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g' > test.txt
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g' > test1.txt
Below command:
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g'
produces output:
XJzLXJlc3VsdHMtYWN0aW9uX18i
<dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:response> </env:Body></env:Envelope>`
Howeveer, in the output I am expecting only first line XJzLXJlc3VsdHMtYWN0aW9uX18i. Where I am commiting mistake?
When i run below command, I am getting expected output:
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g'
It produces below string
lc3VsdHMtYWN0aW9uX18i
I can then easily route this to test1.txt file.
UPDATE
I have edited the question by updating the source file content. The source file doesn't contain any newline character. The current solution will not work in that case, I have tried it and failed. wc -l temp must output to 1.
OS: solaris 10
Shell: bash
sed -n 's_<dp:file name="\([^"]*\)">\([^<]*\).*_\1 -> \2_p' temp
I add \1 -> to show link from file name to content but for content only, just remove this part
posix version so on GNU sed use --posix
assuming that base64 encoded contents is on the same line as the tag around (and not spread on several lines, that need some modification in this case)
Thanks to JID for full explaination below
How it works
sed -n
The -n means no printing so unless explicitly told to print, then there will be no output from sed
's_
This is to substitute the following regex using _ to separate regex from the replacement.
<dp:file name=
Regular text
"\([^"]*\)"
The brackets are a capture group and must be escaped unless the -r option is used( -r is not available on posix). Everything inside the brackets is captured. [^"]* means 0 or more occurrences of any character that is not a quote. So really this just captures anything between the two quotes.
>\([^<]*\)<
Again uses the capture group this time to capture everything between the > and <
.*
Everything else on the line
_\1 -> \2
This is the replacement, so replace everything in the regex before with the first capture group then a -> and then the second capture group.
_p
Means print the line
Resources
http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
http://www.grymoire.com/Unix/Sed.html
/usr/xpg4/bin/sed works well here.
/usr/bin/sed is not working as expected in case if the file contains just 1 line.
below command works for a file containing only single line.
/usr/xpg4/bin/sed -n 's_<env:Envelope\(.*\)<dp:file name="temporary://BackUpDir/backupmanifest.xml">\([^>]*\)</dp:file>\(.*\)_\2_p' securebackup.xml 2>/dev/null
Without 2>/dev/null this sed command outputs the warning sed: Missing newline at end of file.
This because of the below reason:
Solaris default sed ignores the last line not to break existing scripts because a line was required to be terminated by a new line in the original Unix implementation.
GNU sed has a more relaxed behavior and the POSIX implementation accept the fact but outputs a warning.
I need to replace several URLs in a text file with some content dependent on the URL itself. Let's say for simplicity it's the first line of the document at the URL.
What I'm trying is this:
sed "s/^URL=\(.*\)/TITLE=$(curl -s \1 | head -n 1)/" file.txt
This doesn't work, since \1 is not set. However, the shell is getting called. Can I somehow push the sed match variables to that subprocess?
The accept answer is just plain wrong. Proof:
Make an executable script foo.sh:
#! /bin/bash
echo $* 1>&2
Now run it:
$ echo foo | sed -e "s/\\(foo\\)/$(./foo.sh \\1)/"
\1
$
The $(...) is expanded before sed is run.
So you are trying to call an external command from inside the replacement pattern of a sed substitution. I dont' think it can be done, the $... inside a pattern just allows you to use an already existent (constant) shell variable.
I'd go with Perl, see the /e option in the search-replace operator (s/.../.../e).
UPDATE: I was wrong, sed plays nicely with the shell, and it allows you do to that. But, then, the backlash in \1 should be escaped. Try instead:
sed "s/^URL=\(.*\)/TITLE=$(curl -s \\1 | head -n 1)/" file.txt
Try this:
sed "s/^URL=\(.*\)/\1/" file.txt | while read url; do sed "s#URL=\($url\)#TITLE=$(curl -s $url | head -n 1)#" file.txt; done
If there are duplicate URLs in the original file, then there will be n^2 of them in the output. The # as a delimiter depends on the URLs not including that character.
Late reply, but making sure people don't get thrown off by the answers here -- this can be done in gnu sed using the e command. The following, for example, decrements a number at the beginning of a line:
echo "444 foo" | sed "s/\([0-9]*\)\(.*\)/expr \1 - 1 | tr -d '\n'; echo \"\2\";/e"
will produce:
443 foo