I need to run one input file after another (same folder) and have previously used something along the lines of:
sed -i -e 's/file1/file1/g'
sh run file 1
sed -i -e 's/file1/file2/g'
sh run file 2
However, I recently tried to use this and it wouldn't work. The run file works so I assume I'm just using the sed -i -e command incorrectly?
Related
I want to overwrite the output of the shell command in a single file on the dockenginx container.
perl -pe 's/^server_names_hash_bucket_size 128;/#$&/' default.conf > default.conf
I ran the command, but the contents of the default.conf file are all erased.
perl -pe 's/^server_names_hash_bucket_size 128;/#$&/' default.conf >> default.conf
When I execute this command, the contents are added after that.
What should I do if I want to comment on this phrase "server_names_hash_bucket_size 128" in default.conf?
*External package is not available because it is a dockerginx.
Flag -i of the perl command allows you to edit the file inplace:
-i[extension] edit <> files in place (makes backup if extension supplied)
Now you can use:
perl -i -pe 's/^.*server_names_hash_bucket_size 128;/#$&/' default.conf
Additionally, you can also use sed -i instead of perl -i -pe
sed -i 's/^.*server_names_hash_bucket_size 128;/#&/' default.conf
sed and perl are both available inside nginx docker container.
There're multiple perl scripts that is ran from CYGWIN terminal. An example is,
$ perl IdGeneratorTool.pl JSmith -i userInfo.adb -o JSmith.txt
The above is an example. Were based on input parameter JSmith, it reads a db file, generate an ID and output that to a text file.
Now these perl scripts running on the CYGWIN keeps growing and it's added to a text file like shown below,
$ perl IdGeneratorTool.pl JSmith -i userInfo.adb -o JSmith.txt
$ perl IdGeneratorTool.pl PTesk -i userInfo.adb -o PTesk.txt
$ perl IdGeneratorTool.pl CMorris -i userInfo.adb -o CMorris.txt
$ perl IdGeneratorTool.pl JLawrence -i userInfo.adb -o JLawrence.txt
$ perl IdGeneratorTool.pl TCruise -i userInfo.adb -o TCruise.txt
...
....
......
.......
.........
And the list keeps growing.
I would like to know whether there's a way to execute all these perl scripts which are in a text file in one go.
I'm new to perl and doesn't have much idea as to what are the options.
An ideal scenario might be, a tool where i can open this text file and click a execute button and then it executes all the scripts and output multiple *.txt files into the same directory.
Or maybe a simple perl script that can do it.
Put them into a file makeall (or whatever you want to call it.
Put as a first line #!/bin/bash into the file
In cygwin enter chmod +x makeall
in cygwin enter ./makeall
With this you've created a bash script which'll do all your calls of the perl script.
Another option would to just put all the user information into a csv file and read that one in order to call your script.
WAIT! Even easier!
Put into the makeall script this:
#!/bin/bash
for user in \
JSmith \
PTesk \
CMorris \
JLawrence \
TCruise \
; do
perl IdGeneratorTool.pl "$user" -i userInfo.adb -o "$user".txt
done
Now you just need to add any additional user the same way I did for your examples.
Without seeing the source for IdGeneratorTool.pl it's hard to give any specific advice; but it is generally not hard to turn something like
do_stuff($ARGV[0], $opt_i, $opt_o);
into
while (<>) {
chomp;
$user, $adb, $outputfile = split('\t');
do_stuff($user, $adb, $outputfile);
}
to read the input from a tab-delimited file instead of from command-line arguments.
You can create text file with list of users (one per line) for example user_list.txt
JSmith
PTesk
CMorris
JLawrence
TCruise
Then create bash script process_list.sh with following content in same directory
#!/bin/bash
for user in `cat user_list.txt`
do
perl IdGeneratorTool.pl $user -i userInfo.adb -o ${user}.txt
done
Now make bash script executable chmod +x process_list.sh and it is ready for execution.
Once you need to add new user edit user_list.txt to add one more line into the file.
Polar Bear
I have tried to scan through the other posts in stack overflow for this, but couldn't get my code work, hence I am posting a new question.
Below is the content of file temp.
<?xml version="1.0" encoding="UTF-8"?>
<env:Envelope xmlns:env="http://schemas.xmlsoap.org/soap/envelope/<env:Body><dp:response xmlns:dp="http://www.datapower.com/schemas/management"><dp:timestamp>2015-01-
22T13:38:04Z</dp:timestamp><dp:file name="temporary://test.txt">XJzLXJlc3VsdHMtYWN0aW9uX18i</dp:file><dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:file></dp:response></env:Body></env:Envelope>
This file contains the base64 encoded contents of two files names test.txt and test1.txt. I want to extract the base64 encoded content of each file to seperate files test.txt and text1.txt respectively.
To achieve this, I have to remove the xml tags around the base64 contents. I am trying below commands to achieve this. However, it is not working as expected.
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g' > test.txt
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g' > test1.txt
Below command:
sed -n '/test.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test.txt">##g'|perl -p -e 's#</dp:file>##g'
produces output:
XJzLXJlc3VsdHMtYWN0aW9uX18i
<dp:file name="temporary://test1.txt">lc3VsdHMtYWN0aW9uX18i</dp:response> </env:Body></env:Envelope>`
Howeveer, in the output I am expecting only first line XJzLXJlc3VsdHMtYWN0aW9uX18i. Where I am commiting mistake?
When i run below command, I am getting expected output:
sed -n '/test1.txt"\>/,/\<\/dp:file\>/p' temp | perl -p -e 's#<dp:file name="temporary://test1.txt">##g'|perl -p -e 's#</dp:file></dp:response></env:Body></env:Envelope>##g'
It produces below string
lc3VsdHMtYWN0aW9uX18i
I can then easily route this to test1.txt file.
UPDATE
I have edited the question by updating the source file content. The source file doesn't contain any newline character. The current solution will not work in that case, I have tried it and failed. wc -l temp must output to 1.
OS: solaris 10
Shell: bash
sed -n 's_<dp:file name="\([^"]*\)">\([^<]*\).*_\1 -> \2_p' temp
I add \1 -> to show link from file name to content but for content only, just remove this part
posix version so on GNU sed use --posix
assuming that base64 encoded contents is on the same line as the tag around (and not spread on several lines, that need some modification in this case)
Thanks to JID for full explaination below
How it works
sed -n
The -n means no printing so unless explicitly told to print, then there will be no output from sed
's_
This is to substitute the following regex using _ to separate regex from the replacement.
<dp:file name=
Regular text
"\([^"]*\)"
The brackets are a capture group and must be escaped unless the -r option is used( -r is not available on posix). Everything inside the brackets is captured. [^"]* means 0 or more occurrences of any character that is not a quote. So really this just captures anything between the two quotes.
>\([^<]*\)<
Again uses the capture group this time to capture everything between the > and <
.*
Everything else on the line
_\1 -> \2
This is the replacement, so replace everything in the regex before with the first capture group then a -> and then the second capture group.
_p
Means print the line
Resources
http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
http://www.grymoire.com/Unix/Sed.html
/usr/xpg4/bin/sed works well here.
/usr/bin/sed is not working as expected in case if the file contains just 1 line.
below command works for a file containing only single line.
/usr/xpg4/bin/sed -n 's_<env:Envelope\(.*\)<dp:file name="temporary://BackUpDir/backupmanifest.xml">\([^>]*\)</dp:file>\(.*\)_\2_p' securebackup.xml 2>/dev/null
Without 2>/dev/null this sed command outputs the warning sed: Missing newline at end of file.
This because of the below reason:
Solaris default sed ignores the last line not to break existing scripts because a line was required to be terminated by a new line in the original Unix implementation.
GNU sed has a more relaxed behavior and the POSIX implementation accept the fact but outputs a warning.
I'm using Sed on a Mac. I am trying to do a simple string replace on a file that is not in the directory. I do:
sed -i 's/old/new/' /Users/A/file
and it says invalid command code A.
What do I need to do?
The -i option in OSX/BSD sed is a little different than the GNU/Linux version in that it requires a backup extension to be given, even if it's an empty string (which means that no backup will be made). The "invalid command code" error message occurs because s/old/new is taken as the backup extension and /Users/A/file is taken as the script (where A is seen as an invalid command name). So it needs to be something like:
sed -i '' 's/old/new/' /Users/A/file
if you have perl:
perl -p -i -e 's/old/new/' /Users/A/file
The -i in sed is not a standard across OS's (not a POSIX standard). This should work every time:
cp /Users/A/file /Users/A/file.sed
cat /Users/A/file.sed | sed 's/old/new' > /Users/A/file
I have the following extract from a script that fetches weather information from accuweather:
wget -O ./weather_raw $address
if [[ -s ./weather_raw ]]; then
egrep 'Currently|Forecast<\/title>|_31x31.gif' ./weather_raw > ./weather
sed -i '/AccuWeather\|Currently in/d' ./weather
sed -i -e 's/^[ \t]*//g' -e 's/<title>\|<\/title>\|<description>\|<\/description>//g' ./weather
sed -i -e 's/<img src="/\n/g' ./weather
sed -i '/^$/d' ./weather
sed -i -e 's/_31x31.*$//g' -e 's/^.*\/icons\///g' ./weather
sed -i -e '1s/.$//' -e '3s/.$//' -e '6s/.$//' ./weather
for (( i=2; i<=8; i+=3 ))
do
im=$(sed -n ${i}p ./weather)
sed -i $i"s/^.*$/$(test_image $im)/" ./weather
done
fi
The command that triggers the code above is in a conkyrc file and its ~/.conkyblue/accu_weather/rss/acc_rss. When I run the conkyrc script from the prompt, I get an error
sed: can't read /home/me/weather: No such file or directory
And indeed when I check, the "weather" file is not created. However if run the command ~/.conkyblue/accu_weather/rss/acc_rss directly from the prompt, it works as expected and create and puts content into the /home/me/weather file.
I don't know anything about the sed command although I'm trying to learn it as a result of this bother.
What could be the problem with the code. I don't think its a permission issue since the folder its writing into is my home folder and I of-course own it.
Thanks
It should have been created by egrep.
When you run your script, the weather directory will be created in the pwd of the script process.
Check and see why egrep does not create the file, or in which directory it created it.