I'm having trouble in getting sed to remove just the specific line I want. Let's say I have a file that looks like this:
testfile
testfile.txt
testfile2
Currently I'm using this to remove the line I want:
sed -i "/$1/d" file
The issue is that with this if I were to give testfile as input it would delete all three lines but I want it to only remove the first line. How do I do this?
With grep
grep -x -F -v -- "$1" file
# or
grep -xFv -- "$1" file
-F is for "fixed strings" -- turns off regex engine.
-x is to match entire line.
-v is for "everything but" the matched line(s).
-- to signal the end of options, in case $1 starts with a hyphen.
To save the file
grep -xFv -- "$1" file | sponge file # `moreutils` package
# or
tmp=$(mktemp)
grep -xFv -- "$1" file > "$tmp" && mv "$tmp" file
So match the whole line.
var=testfile
sed -i '/^'"$var"'$/d' file
# or with " quoting
sed -i "/^$var\$/d" file
You can learn regex with fun online with regex crosswords.
Related
I need a help in the shell scripting processing the file. The script should read each file in the path and replace the string in each row.
It should read each line and replace the 7th column with XXXX mentioned in the sample output. Any help in appreciated.
Input file data
"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"2455525445552000"|"1111-11-11"|75.00|"XXE11111"|"224425"
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"2455525445552000"|"1111-11-11"|95.00|"VVE11111"|"224425"
output file
"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"24555XXXXXXXXXX"|"1111-11-11"|75.00|"XXE11111"|"224425"
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"24555XXXXXXXXXX"|"1111-11-11"|95.00|"VVE11111"|"224425"
Script I used to run but it is not editing the input file
FILES=/home/auto/*.txt
for f in $FILES
do
echo "Processing $f file..."
cat $f | awk 'BEGIN {FS="|"; OFS="|"} {$7=substr($7, 1, 6)"XXXXXXXXXX\"";print}'
done
but I can't edit the exiting file in the directory. I need to use the sed -i option but it's not working.
I tried using the script in below server but I am getting the following error.
SunOS 5.10 Generic January 2005
echo "hello"
FILES=/export/home/*.txt
for f in $FILES
do
echo "Processing $f file..."
sed -i -r 's/"([^"]{6})[^"]*"/"\1XXXXXXXXXX"/6' "$f"
done
I get
sed: illegal option -- i
Using GNU sed with -i optoin
sed -i -r 's/"([^"]{5})[^"]*"/"\1XXXXXXXXXX"/5' file
"2013-04-30"|"X"|"0000628"|"15000231"|"1999-12-05"|"ST"|"24555XXXXXXXXXX"|"1111-11-11"|75.00|"XXE11111"|"224425"
"2013-04-30"|"Y"|"0000928"|"95000232"|"1999-12-05"|"VT"|"24555XXXXXXXXXX"|"1111-11-11"|95.00|"VVE11111"|"224425"
if your awk is gnu awk 4.1.0, there is in-place option, read man/info page.
otherwise, you could do:
awk '..code..' inputfile > tmpfile && mv tmpfile inputfile
note, the cat is not necessary, could (should) be removed.
A little ugly but you can try something like this with sed
sed -i 's/\(\([^|]*|\)\{6\}\)\(.\{6\}\).\{11\}\(.*\)/\1\3XXXXXXXXXXX\4/' file
So with your existing script, it will be -
FILES=/home/auto/*.txt
for f in $FILES
do
echo "Processing $f file..."
sed -i 's/\(\([^|]*|\)\{6\}\)\(.\{6\}\).\{11\}\(.*\)/\1\3XXXXXXXXXXX\4/' "$f"
done
I'm trying to adapt an script to make it work in an aix server. The scripts must replace a line which contains a pattern with other line with the same pattern but adding more info.
Following my previous question here:
I want to replace the line which contain the pattern CASIOPEA_STORE_BDD_PWD, here the code which still doesn't work :
sed -i 's/^.*\bCASIOPEA_STORE_BDD_PWD\b.*$/CASIOPEA_STORE_BDD_PWD='MyCasioPass2014#'/g' casiopeia.conf
Now i'm trying the script on OS X and this command throws me the following error message:
sed: 1: "File's route ...": invalid command code m
Q2: Is it possible to add this line to the file casiopeia.conf if it isn't exists in the file ?
I don't think in place editing (-i) flag of sed works on AIX:
You could try:
sed "s/^.*\bCASIOPEA_STORE_BDD_PWD\b.*$/CASIOPEA_STORE_BDD_PWD=\'MyCasioPass2014\'/g" casiopeia.conf >casiopeia.edit
Or install the GNU version of sed on AIX
For Q2:
#!/bin/ksh
search="CASIOPEA_STORE_BDD_PWD"
replace="CASIOPEA_STORE_BDD_PWD=\'MyCasioPass2014\'"
filename="$1"
#Grep for search pattern
grep q "$search" "$filename"
if [ $? -ne -0 ]; then
#Match not found, append line
echo "$search" >> "$filename"
else
#Match found, create tmp file for inplace editing
tmpfile=$(mktemp "/tmp/$filename.XXXXX")
#Copy orignal to tmpfile
cat $filename >$tmpfile
#Sed line non-case sensitive
sed "s/^.*$search.*$/$replace/g" "$tmpfile" >"$filename"
#Remove temp file
rm "$tmpfile"
fi
Hi I use the following code to remove the trailing lines from a file. But is there anyway I can run this on 2000 files inside a folder with out renaming them ? thanx in advance
sed -e :a -e '/^\n*$/{$d;N;};/\n$/ba'
sed -i "" -e :a -e '/^\n*$/{$d;N;};/\n$/ba' YourFile
if it is in same folder with a pattern to select id like *.txt, replace YourFile by the shell pattern, if other selection pass each file name via a pre selection like find or a while read from a stream/file input
You can use find to list the files that you need, and then run sed on each on them. This version uses the -i option to modify the files in place:
find . -name "*.txt" | xargs -I % sed -i -e :a -e '/^\n*$/{$d;N;};/\n$/ba' %
I want TO extract list of names from other bigger file (input), having that name and some additional information associated with that name. My problem is with grep -f option, as it is not matching the exact entries in input file but some other entries that contain similar name.
I tried:
$ grep -f list.txt -A 1 input >output
Following are the format of files;
list.txt
TE_final_35005
TE_final_1040
Input file
>TE_final_10401
ACGTACGTACGTACGT
>TE_final_35005
ACGTACGATCAGT
>TE_final_1040
ACGTACGTACGT
Required output:
>TE_final_35005
ACGTACGATCAGT
>TE_final_1040
ACGTACGTACGT
output I am getting:
>TE_final_10401
ACGTACGTACGTACGT
>TE_final_35005
ACGTACGATCAGT
>TE_final_1040
ACGTACGTACGT
Although TE_final_10401 is not in the list.txt
How I can use ^ in list?
Please help to match the exact value or suggest other ways to do this.
Add the whole word switch (-w):
grep -w -A1 -f list.txt infile
Output:
>TE_final_35005
ACGTACGATCAGT
>TE_final_1040
ACGTACGTACGT
A couple of things, remove the blanks lines from the files first:
sed -i '/^\s*$/d' file list
Then -w is used to match whole words only and -A1 will print the next line after the match:
$ grep -w -A1 -f list file > new_file
$ cat new_file
>TE_final_35005
ACGTACGATCAGT
>TE_final_1040
ACGTACGTACGT
as others have mentioned, adding the -w flag is the cleanest and easiest approach based on your sample data. but since you explicitly asked how you could use ^ in list.txt, here's another option.
to add ^ and/or $ anchors to each line in list.txt:
$ cat list.txt
^>TE_final_35005[ ]*$
^>TE_final_1040[ ]*$
this searches for your patterns at the start of the line, preceded by a > character, and ignores any trailing spaces. then your previous command will work (assuming you either remove those blank lines or change your argument to -A 2).
if you'd like to add these anchors to the list file automatically (and delete any blank lines at the same time), use this awk construct:
awk '{if($0 != ""){print "^>"$0"[ ]*$"}}' list.txt >newlist.txt
or if you prefer sed inplace editing:
sed -i '/^[ ]*$/d;s/\(.*\)/^>\1[ ]*$/g' list.txt
I want to delete all the rows/lines in a file that has a specific character, '?' in my case. I hope there is a single line command in Bash or AWK or Perl. Thanks
You can use sed to modify the file "in-place":
sed -i "/?/d" file
Alternatively, use grep:
grep -v "?" file > newfile.txt
Even better, just a single line using sed
sed '/?/d' input
use -i to edit file in place.
perl -i -ne'/\?/ or print' file
or
perl -i -pe's/^.*?\?.*//s' file
Here are already grep, sed and perl solutions - only for fun, pure bash one:
pattern='?'
while read line
do
[[ "$line" =~ "$pattern" ]] || echo "$line"
done
translated
for every line on the STDIN
match it for the pattern =~
and if the match is not successful || - print out the line
awk '!($0~/?/){print $0}' file_name