I have a file that contains:
$conf['minified_version'] = 100;
I want to increment that 100 with sed, so I have this:
sed -r 's/(.*minified_version.*)([0-9]+)(.*)/echo "\1$((\2+1))\3"/ge'
The problem is that this strips the $conf from the original, along with any indentation spacing. What I have been able to figure out is that it's because it's trying to run:
echo " $conf['minified_version'] = $((100+1));"
so of course it's trying to replace the $conf with a variable which has no value.
Here is an awk version:
$ awk '/minified_version/{$3+=1} 1' file
$conf['minified_version'] = 101
This looks for lines that contain minified_version. Anytime such a line is found the third field, $3, is incremented by.
My suggested approach to this would be to have a file on-disk that contained nothing but the minified_version number. Then, incrementing that number would be as simple as:
minified_version=$(< minified_version)
printf '%s\n' "$(( minified_version + 1 ))" >minified_version
...and you could just put a sigil in your source file where that needs to be replaced. Let's say you have a file named foo.conf.in that contains:
$conf['minified_version'] = #MINIFIED_VERSION#
...then you could simply run, in your build process:
sed -e "s/#MINIFIED_VERSION#/$(<minified_version)/g" <foo.conf.in >foo.conf
This has the advantage that you never have code changing foo.conf.in, so you don't need to worry about bugs overwriting the file's contents. It also means that if you're checking your files into source control, so long as you only check in foo.conf.in and not foo.conf you avoid potential merge conflicts due to context near the version number changing.
Now, if you did want to do the native operation in-place, here's a somewhat overdesigned approach written in pure native bash (reading from infile and writing to outfile; just rename outfile back over infile when successful to make this an in-place replacement):
target='$conf['"'"'minified_version'"'"'] = '
suffix=';'
while IFS= read -r line; do
if [[ $line = "$target"* ]]; then
value=${line##*=}
value=${value%$suffix}
new_value=$(( value + 1 ))
printf '%s\n' "${target}${new_value}${suffix}"
else
printf '%s\n' "$line"
fi
done <infile >outfile
Related
Trying to do this sort of thing in perl:
sed '1 a<!-- $Header: $\n Purpose: system generated file -->' -i test.xml
Add the header block and purpose to line #2 in the file for xml, shell scripts, etc...
Don't want to do this either:
`sed '1 a<!-- \$Header: \$\n Purpose: system generated file -->' -i test.xml`
But realize it's an option if absolutely necessary.
If you only pass one file, you can use the following:
perl -i -pe'
$_ .= "<!-- \$Header: \$\n Purpose: system generated file -->\n" if $. == 1;
' test.xml
If you might pass multiple files, you'll need to add a line so that $. is reset at the end of each file.
perl -i -pe'
$_ .= "<!-- \$Header: \$\n Purpose: system generated file -->\n" if $. == 1;
close(ARGV) if eof;
' test*.xml
(Note: eof() means something different than just eof. how awful is that!)
I added line breaks for readability. The commands will work as is, but you can remove the line breaks if you so desire.
Try this way:
perl -ple '++$i == 2 and $_ = "changed" # change $_ as you want' in.txt > out.txt
This question already has answers here:
How do I add a line of text to the middle of a file using bash?
(6 answers)
Closed 10 years ago.
How do insert lines of text into file after a particular line in unix ?
Background: The file is an autogenerated textfile but I manually have to edit it every time it is regenerated to add in 4 additional lines after a particular line. I can gurantee that this line will always be in the file but I cannot guarantee excalty what line it will be on so I want the additional lines to be added on the basis of the position of this line rather than adding to a fixed rownumber. I want to automate this process as it is part of my build process.
I'm using Mac OSX so I can make use of unix comand line tools, but Im not very familiar with such tools and cannot work out how to do this.
EDIT
Thanks for the solution, although I havent managed to get them working yet:
I tried Sed solution
sed -i '/<string>1.0</string>/ a <key>CFBundleHelpBookFolder</key>\
<string>SongKongHelp</string>\
<key>CFBundleHelpBookName</key>\
<string>com.jthink.songkong.Help</string>
' /Applications/SongKong.app/Contents/Info.plist
but get error
sed: 1: "/Applications/SongKong. ...": invalid command code S
and I tried the bash solution
#!/bin/bash
while read line; do
echo "$line"
if [[ "$line" = "<string>1.0</string>"]]; then
cat mergefile.txt # or echo or printf your extra lines
fi
done < /Applications/SongKong.app/Contents/Info.plist
but got error
./buildosx4.sh: line 5: syntax error in conditional expression: unexpected token `;'
./buildosx4.sh: line 5: syntax error near `;'
./buildosx4.sh: line 5: ` if [[ "$line" = "<string>1.0</string>"]]; then'
EDIT 2
Now working, i was missing a space
#!/bin/bash
while read line; do
echo "$line"
if [[ "$line" = "<string>1.0</string>" ]]; then
cat mergefile.txt # or echo or printf your extra lines
fi
done < /Applications/SongKong.app/Contents/Info.plist
Assuming the marker line contains fnord and nothing else;
awk '1;/^fnord$/{print "foo"; print "bar";
print "baz"; print "quux"}' input >output
Another way to look at this is that you want to merge two files at some point in one of the files. If your extra four lines were in a separate file, you could make a more generic tool like this:
#!/usr/bin/awk
BEGIN {
SEARCH=ARGV[1]; # Get the search string from the command line
delete ARGV[1]; # Delete the argument, so subsequent arguments are files
}
# Collect the contents of the first file into a variable
NR==FNR {
ins=ins $0 "\n";
next;
}
1 # print every line in the second (or rather the non-first) file
# Once we're in the second file, if we see the match, print stuff...
$0==SEARCH {
printf("%s", ins); # Using printf instead of print to avoid the extra newline
next;
}
I've spelled this out for ease of documentation; you could obviously shorten it to something that looked more like triplee's answer. You'd invoke this like:
$ scriptname "Text to match" mergefile.txt origfile.txt > outputfile.txt
Done this way, you'd have a tool that could be used to achieve this kind of merge on different files and with different text.
Alternately, you could of course do this in pure bash.
#!/bin/bash
while read line; do
echo "$line"
if [[ "$line" = "matchtext" ]]; then
cat mergefile.txt # or echo or printf your extra lines
fi
done < origfile.txt
The problem can be solved efficiently for any filesize by this algorithm:
Read each line from the original file and print it to a tempfile.
If the last line was the marker line, print your insertion lines to the tempfile
Print the remaining lines
Rename the tempfile to the original filename.
As a Perl script:
#!perl
use strict; use warnings;
$^I = ".bak"; # create a backup file
while (<>) {
print;
last if /regex to determine if this is the line/;
}
print <<'END';
Yourstuff
END
print while <>; # print remaining lines
# renaming automatically done.
Testfile:
foo
bar
baz
qux
quux
Regex is /^ba/.
Usage: $ perl this_script.pl source-file
The testfile after processing:
foo
bar
Yourstuff
baz
qux
quux
use the sed 'a command with a regex for the line you need to match
sed -i '/the target line looks like this/ a this is line 1\
this is line 2\
this is line 3\
this is line 4
' FILENAME
I have really a lot of files named like:
1_x_0_a.jpg, 1_x_0_b.jpg, 1_x_5_a.jpg ... 15_x_160_a.jpg, 15_x_160_b.jpg, 15_x_165_a.jpg
I would like to change the file names as follows:
01_x_000_a.jpg, 01_x_000_b.jpg, 01_x_005_a.jpg
So, before x should be a number with 2 dig and after x with 3 digits.
The following code should work on relatively newer versions of MATLAB.
fileStruct = dir;
files = {fileStruct.name};
for oldFile = files
oldFile = oldFile{1}; //Takes string out of cell
// Embedding the sprintf in a regexprep only works in certain versions
newFile = regexprep(oldFile, '^(\d*)', '${sprintf(''%02d'', str2num($1))}');
newFile = regexprep(newFile, '(?<=_)(\d*)(?=_)', '${sprintf(''%03d'', str2num($1))}');
movefile(oldFile, newFile);
end
Here are some steps that should help you:
Use dir to get a list of file names.
Use regexprep to replace starting numbers by starting numbers with leading zeros
Use regexprep to replace middle numbers by starting numbers with up to two zeros
Use rename to change the file names
Note that I have not tried it and the documentation of rename is a bit strange as it refers to ftp sites, but it might just work. If it does not work, I guess you can just copy all the files and then remove the old ones.
If you are on a Unix or Linux machine you can try this small shell script:
In a terminal go to the directory where you have your files.
You can first try it without really renaming your files, by replacing the mv with echo to see if it works as expected.
for file in *; do
mv $file $(echo $file | awk -F '_' '{ printf "%02d_%s_%003d_%s\n", $1, $2, $3, $4 }')
done
or as a one liner
for file in *; do mv $file $(echo $file | awk -F '_' '{ printf "%02d_%s_%003d_%s\n", $1, $2, $3, $4 }'); done
For files
1_x_0_a.jpg
1_x_0_b.jpg
1_x_5_a.jpg
15_x_160_a.jpg
15_x_160_b.jpg
15_x_165_a.jpg
I get the result
01_x_000_a.jpg
01_x_000_b.jpg
01_x_005_a.jpg
15_x_160_a.jpg
15_x_160_b.jpg
15_x_165_a.jpg
I have a file of environment variables that I source in shell scripts, for example:
# This is a comment
ONE=1
TWO=2
THREE=THREE
# End
In my scripts, I source this file (assume it's called './vars') into the current environment, and change (some of) the variables based on user input. For example:
#!/bin/sh
# Read variables
source ./vars
# Change a variable
THREE=3
# Write variables back to the file??
awk 'BEGIN{FS="="}{print $1=$$1}' <./vars >./vars
As you can see, I've been experimenting with awk for writing the variables back, sed too. Without success. The last line of the script fails. Is there a way to do this with awk or sed (preferably preserving comments, even comments with the '=' character)? Or should I combine 'read' with string cutting in a while loop or some other magic? If possible, I'd like to avoid perl/python and just use the tools available in Busybox. Many thanks.
Edit: perhaps a use case might make clear what my problem is. I keep a configuration file consisting of shell environment variable declarations:
# File: network.config
NETWORK_TYPE=wired
NETWORK_ADDRESS_RESOLUTION=dhcp
NETWORK_ADDRESS=
NETWORK_ADDRESS_MASK=
I also have a script called 'setup-network.sh':
#!/bin/sh
# File: setup-network.sh
# Read configuration
source network.config
# Setup network
NETWORK_DEVICE=none
if [ "$NETWORK_TYPE" == "wired" ]; then
NETWORK_DEVICE=eth0
fi
if [ "$NETWORK_TYPE" == "wireless" ]; then
NETWORK_DEVICE=wlan0
fi
ifconfig -i $NETWORK_DEVICE ...etc
I also have a script called 'configure-network.sh':
#!/bin/sh
# File: configure-network.sh
# Read configuration
source network.config
echo "Enter the network connection type:"
echo " 1. Wired network"
echo " 2. Wireless network"
read -p "Type:" -n1 TYPE
if [ "$TYPE" == "1" ]; then
# Update environment variable
NETWORK_TYPE=wired
elif [ "$TYPE" == "2" ]; then
# Update environment variable
NETWORK_TYPE=wireless
fi
# Rewrite configuration file, substituting the updated value
# of NETWORK_TYPE (and any other updated variables already existing
# in the network.config file), so that later invocations of
# 'setup-network.sh' read the updated configuration.
# TODO
How do I rewrite the configuration file, updating only the variables already existing in the configuration file, preferably leaving comments and empty lines intact? Hope this clears things up a little. Thanks again.
You can't use awk and read and write from the same file (is part of your problem).
I prefer to rename the file before I rewrite (but you can save to a tmp and then rename too).
/bin/mv file file.tmp
awk '.... code ...' file.tmp > file
If your env file gets bigger, you'll see that is is getting truncated at the buffer size of your OS.
Also, don't forget that gawk (the std on most Linux installations) has a built in array ENVIRON. You can create what you want from that
awk 'END {
for (key in ENVIRON) {
print key "=" ENVIRON[key]
}
}' /dev/null
Of course you get everything in your environment, so maybe more than you want. But probably a better place to start with what you are trying to accomplish.
Edit
Most specifically
awk -F"=" '{
if ($1 in ENVIRON) {
printf("%s=%s\n", $1, ENVIRON[$1])
}
# else line not printed or add code to meet your situation
}' file > file.tmp
/bin/mv file.tmp file
Edit 2
I think your var=values might need to be export -ed so they are visible to the awk ENVIRON array.
AND
echo PATH=xxx| awk -F= '{print ENVIRON[$1]}'
prints the existing value of PATH.
I hope this helps.
P.S. as you appear to be a new user, if you get an answer that helps you please remember to mark it as accepted, and/or give it a + (or -) as a useful answer.
I don't exactly know what you are trying to do, but if you are trying to change the value of variable THREE ,
awk -F"=" -vt="$THREE" '$1=="THREE" {$2=t}{print $0>FILENAME}' OFS="=" vars
You can do this in just with bash:
rewrite_config() {
local filename="$1"
local tmp=$(mktemp)
# if you want the header
echo "# File: $filename" >> "$tmp"
while IFS='=' read var value; do
declare -p $var | cut -d ' ' -f 3-
done < "$filename" >> "$tmp"
mv "$tmp" "$filename"
}
Use it like
source network.config
# manipulate the variables
rewrite_config network.config
I use a temp file to maintain the existance of the config file for as long as possible.
I need to get a particular version string from a file (call it version.lst) and use it to compare another in a shell script. For example sake, the file contains lines that look like this:
V1.000 -- build date and other info here -- APP1
V1.000 -- build date and other info here -- APP2
V1.500 -- build date and other info here -- APP3
.. and so on. Let's say I am trying to grab the first version (in this case, V1.000) from APP1. Obviously, the versions can change and I want this to be dynamic. What I have right now works:
var = `cat version.lst | grep " -- APP1" | grep -Eo V[0-9].[0-9]{3}`
Pipe to grep will get the line containing APP1 and the second pipe to grep will get the version string. However, I hear grep is not the way to do this so I'd like to learn the best way using awk or sed. Any ideas? I am new to both and haven't found a tutorial easy enough to learn the syntax of it. Do they support egrep? Thanks!
Try this to get the complete version:
#!/bin/sh
app=APP1
var=$(awk -v "app=$app" '$NF == app {print $1}' version.lst)
or to get only the major version number, the last line could be:
var=$(awk -v "app=$app" '$NF == app {split($1,a,"."); print a[1]}' version.lst)
Using sed to get the complete version:
var=$(sed -n "/ $app\$/s/^\([^ ]*\).*/\1/p" version.lst)
or this to get only the major version number:
var=$(sed -n "/ $app\$/s/^\([^.]*\).*/\1/p" version.lst)
Explanations:
The second AWK command:
-v "app=$app" - set an AWK variable equal to a shell variable
$NF == app - if the last field is equal to the contents of the variable (NF is the number of field, so $NF is the contents of the NFth field)
{split($1,a,".") - then split the first field at the dot
print a[1] - and print the first part of the result of the split
The sed commands:
-n - don't print any output unless directed to
"/ $app\$/ - for any line that ends with (\$) the contents of the shell variable $app (not that double quotes are used to allow the variable to be expanded and it's a good idea to escape the second dollar sign)
s/^\([^ ]*\).*/\1/p" - starting at the beginning of the line (^), capture \(\) the sequence of characters that consists of non-spaces ([^ ]) (or non-dots in the second version) of any number (zero or more *) and match but don't capture all the rest of the characters on the line (.*), replace the matched text (the whole line in this case) with the string that was captured (the version number) (\1 refers to the first (only, in this case) capture group, and print it (p)
If I understood correctly: egrep "APP1$" version.lst | awk '{print $1}'
$ awk '/^V1\.00.* APP1$/{print $NF}' version.lst
APP1
That regular expression matches lines that start with "V1.00", followed by any number of any other characters, ending with " APP1". The backslash in the middle there might be really important--it matches only ".", and so it excludes (probably corrupt) lines that might begin with, say, "V1a00". The space before "APP1" excludes things like "APP2_APP1".
"NF" is an automatically generated variable that contains the number of field in the input line. It's also the number of the last field, which happens to be the one you're interested in.
There are a couple of ways to prune off the "V1". Here's one way, although you and I might not be talking about quite the same thing.
$ awk '/^V1\.00.* APP1$/{print substr($1, 1, index($1, ".") - 1), $NF}' version.lst
V1 APP1