I need a help in the shell scripting processing the file. The script should read each file in the path and replace the string in each row.
It should read each line and replace the 7th column with XXXX mentioned in the sample output. Any help in appreciated.
Input file data
"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"2455525445552000"|"1111-11-11"|75.00|"XXE11111"|"224425"
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"2455525445552000"|"1111-11-11"|95.00|"VVE11111"|"224425"
output file
"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"24555XXXXXXXXXX"|"1111-11-11"|75.00|"XXE11111"|"224425"
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"24555XXXXXXXXXX"|"1111-11-11"|95.00|"VVE11111"|"224425"
Script I used to run but it is not editing the input file
FILES=/home/auto/*.txt
for f in $FILES
do
echo "Processing $f file..."
cat $f | awk 'BEGIN {FS="|"; OFS="|"} {$7=substr($7, 1, 6)"XXXXXXXXXX\"";print}'
done
but I can't edit the exiting file in the directory. I need to use the sed -i option but it's not working.
I tried using the script in below server but I am getting the following error.
SunOS 5.10 Generic January 2005
echo "hello"
FILES=/export/home/*.txt
for f in $FILES
do
echo "Processing $f file..."
sed -i -r 's/"([^"]{6})[^"]*"/"\1XXXXXXXXXX"/6' "$f"
done
I get
sed: illegal option -- i
Using GNU sed with -i optoin
sed -i -r 's/"([^"]{5})[^"]*"/"\1XXXXXXXXXX"/5' file
"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"24555XXXXXXXXXX"|"1111-11-11"|75.00|"XXE11111"|"224425"
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"24555XXXXXXXXXX"|"1111-11-11"|95.00|"VVE11111"|"224425"
if your awk is gnu awk 4.1.0, there is in-place option, read man/info page.
otherwise, you could do:
awk '..code..' inputfile > tmpfile && mv tmpfile inputfile
note, the cat is not necessary, could (should) be removed.
A little ugly but you can try something like this with sed
sed -i 's/\(\([^|]*|\)\{6\}\)\(.\{6\}\).\{11\}\(.*\)/\1\3XXXXXXXXXXX\4/' file
So with your existing script, it will be -
FILES=/home/auto/*.txt
for f in $FILES
do
echo "Processing $f file..."
sed -i 's/\(\([^|]*|\)\{6\}\)\(.\{6\}\).\{11\}\(.*\)/\1\3XXXXXXXXXXX\4/' "$f"
done
Related
I have a folder of 500 *.INI files that I need to manually edit. Within each INI file, I have the line Source =. I would like that line to become Source = C:\software\{filename}.
For instance, a dx4.ini file would need to be fixed to become: Source = C:\software\dx4
Is there a quick way to do this with Find, Grep, or Sed functions?
You can try with sed
For example
Input file contents:
file.txt
Source =
some lines..
script:
newstring='Source = C:\software\dx4'
oldstring='Source ='
echo `sed "s/$oldstring/$newstring/g" file.txt` > file.txt
After running the above commands
output:
Source = C:\software\dx4
some lines..
If you want to edit a file in a script, I think ed is the way to go. Combined with a shell for loop:
for file in *.INI; do
base=$(basename "$file" .INI)
ed -s "$file" <<EOF
/^Source =/s/=/= C:\\\\software\\\\$base/
w
EOF
done
(This does assume that filenames will not have newlines or ampersands in their names)
With GNU awk for the 3rd arg to match(), gensub(), and "inplace" editing:
awk -i inplace '
match($0,/(.*Source = C:\\software\\){filename}(.*)/,a) {
fname = gensub(/\..*/,"",1,FILENAME)
$0 = a[1] fname a[2]
}
1' *.INI
The above assumes you're running in a UNIX environment though your use of the term folder instead of directory and that path starting with C: and containing backslashes makes me suspicious. If you're on Windows then save the part between the 2 's (exclusive) in a file named foo.awk and execute it as awk -i inplace foo.awk *.INI or however it is you normally execute commands like this in Windows.
find *.ini -type -f > stack
while read line
do
sed -i s"#Source =#Source = C:\\software\\dx4#" "${line}"
done < stack
Assuming that a} You have sed with "-i" (the insert flag, which AFAIK is not always portable) and b} sed doesn't crap itself about a double escape sequence, I think that will work.
I'm trying to add the filename of a text file into the first line of a the same text file. for example if the file name is called test1.txt, then the first line when you open the file should be test1.
below is what I've done so for, the only problem i have is that the word "$file" is being written to the file not the file name. any help is appreciated.
for file in *.txt; do
sed -i '1 i\$file' $file;
awk 'sub("$", "\r")' "$file" > "$file"1;
mv "$file"1 "$file";
done
Without concise, testable sample input and expected output it's an untested guess but it SOUNDS like all you need is:
awk -i inplace -v ORS='\r\n' 'FNR==1{print FILENAME}1' *
No shell loop or multiple commands required.
The above uses GNU awk for inplace editing and I'm assuming the sub() in your code was intended to add a \r at the end of every line.
I've just started learning more about sed and awk and put this into a file called insert.sed and sourced it and passed it a file name:
sed -i '1s/^./'$1'\'$'\n/g' $1
In trying it, it seems to work okay:
rent$ cat x.txt
<<< Who are you?
rent$ source insert.sed x.txt
rent$ cat x.txt
x.txt
<< Who are you?
It is cutting off the first character of the first line so I'd have to fix that otherwise it does add the file name to first line.
I'm sure there's better ways of doing it.
If you want test1 on first line, with gnu sed
sed -i '1{x;s/.*/fich=$(ps -p $PPID -o args=);fich=${fich##*\\} };echo ${fich%%.*}/e;G}' test1.txt
I'm trying to adapt an script to make it work in an aix server. The scripts must replace a line which contains a pattern with other line with the same pattern but adding more info.
Following my previous question here:
I want to replace the line which contain the pattern CASIOPEA_STORE_BDD_PWD, here the code which still doesn't work :
sed -i 's/^.*\bCASIOPEA_STORE_BDD_PWD\b.*$/CASIOPEA_STORE_BDD_PWD='MyCasioPass2014#'/g' casiopeia.conf
Now i'm trying the script on OS X and this command throws me the following error message:
sed: 1: "File's route ...": invalid command code m
Q2: Is it possible to add this line to the file casiopeia.conf if it isn't exists in the file ?
I don't think in place editing (-i) flag of sed works on AIX:
You could try:
sed "s/^.*\bCASIOPEA_STORE_BDD_PWD\b.*$/CASIOPEA_STORE_BDD_PWD=\'MyCasioPass2014\'/g" casiopeia.conf >casiopeia.edit
Or install the GNU version of sed on AIX
For Q2:
#!/bin/ksh
search="CASIOPEA_STORE_BDD_PWD"
replace="CASIOPEA_STORE_BDD_PWD=\'MyCasioPass2014\'"
filename="$1"
#Grep for search pattern
grep q "$search" "$filename"
if [ $? -ne -0 ]; then
#Match not found, append line
echo "$search" >> "$filename"
else
#Match found, create tmp file for inplace editing
tmpfile=$(mktemp "/tmp/$filename.XXXXX")
#Copy orignal to tmpfile
cat $filename >$tmpfile
#Sed line non-case sensitive
sed "s/^.*$search.*$/$replace/g" "$tmpfile" >"$filename"
#Remove temp file
rm "$tmpfile"
fi
How it is possible to make a dry run with sed?
I have this command:
find ./ -type f | xargs sed -i 's/string1/string2/g'
But before I really substitute in all the files, i want to check what it WOULD substitute. Copying the whole directory structure to check is no option!
Remove the -i and pipe it to less to paginate though the results. Alternatively, you can redirect the whole thing to one large file by removing the -i and appending > dryrun.out
I should note that this script of yours will fail miserably with files that contain spaces in their name or other nefarious characters like newlines or whatnot. A better way to do it would be:
while IFS= read -r -d $'\0' file; do
sed -i 's/string1/string2/g' "$file"
done < <(find ./ -type f -print0)
I would prefer to use the p-option:
find ./ -type f | xargs sed 's/string1/string2/gp'
Could be combined with the --quiet parameter for less verbose output:
find ./ -type f | xargs sed --quiet 's/string1/string2/gp'
From man sed:
p:
Print the current pattern space.
--quiet:
suppress automatic printing of pattern space
I know this is a very old thread and the OP doesn't really need this answer, but I came here looking for a dry run mode myself, so thought of adding the below piece of advice for anyone coming here in future. What I wanted to do was to avoid stomping the backup file unless there is something really changing. If you blindly run sed using the -i option with backup suffix, existing backup file gets overwritten, even when there is nothing substituted.
The way I ended up doing is to pipe sed output to diff and see if anything changed and then rerun sed with in-place update option, something like this:
if ! sed -e 's/string1/string2/g' $fpath | diff -q $fpath - > /dev/null 2>&1; then
sed -i.bak -e 's/string1/string2/g' $fpath
fi
As per OP's question, if the requirement is to just see what would change, then instead of running the in-pace sed, you could do the diff again with some informative messages:
if ! sed -e 's/string1/string2/g' $fpath | diff -q $fpath - > /dev/null 2>&1; then
echo "File $fpath will change with the below diff:"
sed -e 's/string1/string2/g' $fpath | diff $fpath -
fi
You could also capture the output in a variable to avoid doing it twice:
diff=$(sed -e 's/string1/string2/g' $fpath | diff $fpath -)
if [[ $? -ne 0 ]]; then
echo "File $fpath will change with the below diff:"
echo "$diff"
fi
I have a file that contains this kind of paths:
C:\bad\foo.c
C:\good\foo.c
C:\good\bar\foo.c
C:\good\bar\[variable subdir count]\foo.c
And I would like to get the following file:
C:\bad\foo.c
C:/good/foo.c
C:/good/bar/foo.c
C:/good/bar/[variable subdir count]/foo.c
Note that the non matching path should not be modified.
I know how to do this with sed for a fixed number of subdir, but a variable number is giving me trouble. Actually, I would have to use many s/x/y/ expressions (as many as the max depth... not very elegant).
May be with awk, but this kind of magic is beyond my skills.
FYI, I need this trick to correct some gcov binary files on a cygwin platform.
I am dealing with binary files; therefore, I might have the following kind of data:
bindata\bindata%bindataC:\good\foo.c
which should be translated as:
bindata\bindata%bindataC:/good/foo.c
The first \ must not be translated, despite that it is on the same line.
However, I have just checked my .gcno files while editing this text and it looks like all the paths are flanked with zeros, so most of the answers below should fit.
sed -e '/^C:\\good/ s/\\/\//g' input_file.txt
I would recommend you look into the cygpath utility, which converts path names from one format to another. For instance on my machine:
$ cygpath `pwd`
/home/jericson
$ cygpath -w `pwd`
D:\root\home\jericson
$ cygpath -m `pwd`
D:/root/home/jericson
Here's a Perl implementation of what you asked for:
$ echo 'C:\bad\foo.c
C:\good\foo.c
C:\good\bar\foo.c
C:\good\bar\[variable subdir count]\foo.c' | perl -pe 's|\\|/|g if /good/'
C:\bad\foo.c
C:/good/foo.c
C:/good/bar/foo.c
C:/good/bar/[variable subdir count]/foo.c
It works directly with the string, so it will work anywhere. You could combine it with cygpath, but it only works on machines that have that path:
perl -pe '$_ = `cygpath -m $_` if /good/'
(Since I don't have C:\good on my machine, I get output like C:goodfoo.c. If you use a real path on your machine, it ought to work correctly.)
You want to substitute '/' for all '\' but only on the lines that match the good directory path. Both sed and awk will let you do this by having a LHS (matching) expression that only picks the lines with the right path.
A trivial sed script to do this would look like:
/[Cc]:\\good/ s/\\/\//g
For a file:
c:\bad\foo
c:\bad\foo\bar
c:\good\foo
c:\good\foo\bar
You will get the output below:
c:\bad\foo
c:\bad\foo\bar
c:/good/foo
c:/good/foo/bar
Here's how I would do it in awk:
# fixpaths.awk
/C:\\good/ {
gsub(/\\/,"/",$1);
print $1 >> outfile;
}
Then run it using the command:
awk -f fixpaths.awk paths.txt; mv outfile paths.txt
Or with some help from good ol' Bash:
#!/bin/bash
cat file | while read LINE
do
if <bad_condition>
then
echo "$LINE" >> newfile
else
echo "$LINE" | sed -e "s/\\/\//g" >> newfile
fi
done
try this
sed -re '/\\good\\/ s/\\/\//g' temp.txt
or this
awk -F"\\" '{if($2=="good"){OFS="\/"; $1=$1;} print $0}' temp.txt