I'm having trouble in getting sed to remove just the specific line I want. Let's say I have a file that looks like this:
testfile
testfile.txt
testfile2
Currently I'm using this to remove the line I want:
sed -i "/$1/d" file
The issue is that with this if I were to give testfile as input it would delete all three lines but I want it to only remove the first line. How do I do this?
With grep
grep -x -F -v -- "$1" file
# or
grep -xFv -- "$1" file
-F is for "fixed strings" -- turns off regex engine.
-x is to match entire line.
-v is for "everything but" the matched line(s).
-- to signal the end of options, in case $1 starts with a hyphen.
To save the file
grep -xFv -- "$1" file | sponge file # `moreutils` package
# or
tmp=$(mktemp)
grep -xFv -- "$1" file > "$tmp" && mv "$tmp" file
So match the whole line.
var=testfile
sed -i '/^'"$var"'$/d' file
# or with " quoting
sed -i "/^$var\$/d" file
You can learn regex with fun online with regex crosswords.
sed does not print line 1 to line 545.The result of the following command is empty resultfile.txt. Can someone say what should be the correct command?
RESULT=545;sed -n '1, $RESULT p' Configuration.txt > resultfile.txt
The above is not a sed issue, but rather a shell issue: you used single quotes, which inhibit variable expansion.
$ echo '$PWD'
$PWD
$ echo $PWD
/tmp
$ echo "$PWD"
/tmp
Either no quotes at all, or double quotes, allows the shell to expand variables. (No quotes at all means the shell expands everything; double quotes inhibit globbing, redirections, and such.)
Use double quotes so that $RESULT is expanded:
RESULT=545;sed -n "1, $RESULT p" Configuration.txt > resultfile.txt
By the way, the following would be simpler:
head -$RESULT Configuration.txt > resultfile.txt
And, if your file is big, this will quit as soon as you reach line 545 and be more efficient:
sed ${RESULT}q Configuration.txt > resultfile.txt
I can use sed from tcsh like this:
set a = `echo $a | sed -e 's_old_new_'`
Everything is fine, but when I want to do this:
set a = `echo $a | sed -e 's_old_*new_'`
I can see "set: No match." . How I can escape this star?
I don't know much about tcsh but few experiments suggest the set assigning a variable attempts to expand right side *. Here is something that may help:
set a="`echo '2e2' | sed -e 's_e_*_'`"
echo "$a"
2*2
echo $a
echo: No match.
So double quote around back quotes and it will work.
set a = "`echo $a | sed -e 's_old_new_'`"
A command substitution (` or $(..)) that is not enclosed in double quotes is subject to filename expansion (aka 'globbing') and word splitting.
Normally, a variable assignment would suppress filename expansion and word splitting of the RHS (even without double quotes), but apparently not filename expansion in the case of command substitution.
Here's a test I ran for reference purposes:
$ touch randomfile
$ a="*file"
$ var_expand=$a
$ echo "$var_expand"
*file
$
$ cmd_subst=$(echo '*file')
$ echo "$cmd_subst"
randomfile
So I guess it's good practice to always double quote the command substitution when you are assigning to variable.
safe="$(cmd)"
Note: This is tested in Bash but I think tcsh exhibits a similar behavior in this respect.
In a bash script, files with spaces show up as "File\ with\ spaces.txt" and I want to substitute those slashed-spaces with either _ or +.
How can I tell sed to do that? I had no success using;
$1=~/File\ with\ spaces.txt
ext=$1
web=$(echo "$ext" | sed 's/\ /+/')
I'm open to suggestions if there's a better way than through sed.
[EDIT]: Foo Bah's solution works well, but it substitutes only the first space because the text following it is treated as arguments rather than part of the $1. Any way around this?
sed 's/\\\\ /+/';
\\\\ evaluates to a \\ at the shell level, and then into a literal \ within sed.
Sed recognises \ as space just fine:
bee#i20 ~ $ echo file\ 123 | sed 's/\ /+/'
file+123
Your bash script syntax is all wrong, though.
Not sure what you were trying to do with the script, but here is an example of replacing spaces with +:
ext=~/File\ with\ spaces.txt
web=`echo "$ext" | sed 's/\ /+/g'`
echo $web
Upd:
Oh, and you need the g flag to replace all occurences of space, not only the first one. Fixed above.
you want to escape the slash:
web=$(echo "$ext" | sed 's/\\ /_/g')
single quotes are your friend
the following should be used with single quoted args for $1 and $2
#!/bin/bash
ESCAPE='\\'
if [ $# -ne 2 ];then
echo "$0 <TO_ESCAPE> <IN_STRING>"
echo args should be in single quotes!!
exit 1
fi
TO_ESCAPE="${1}"
IN_STRING="${2}"
if [ ${TO_ESCAPE} = '\' ];then
TO_ESCAPE='\\'
fi
echo "${IN_STRING}" | sed "s/${TO_ESCAPE}/${ESCAPE}${TO_ESCAPE}/g"
I have a simple shell script that removes trailing whitespace from a file. Is there any way to make this script more compact (without creating a temporary file)?
sed 's/[ \t]*$//' $1 > $1__.tmp
cat $1__.tmp > $1
rm $1__.tmp
You can use the in place option -i of sed for Linux and Unix:
sed -i 's/[ \t]*$//' "$1"
Be aware the expression will delete trailing t's on OSX (you can use gsed to avoid this problem). It may delete them on BSD too.
If you don't have gsed, here is the correct (but hard-to-read) sed syntax on OSX:
sed -i '' -E 's/[ '$'\t'']+$//' "$1"
Three single-quoted strings ultimately become concatenated into a single argument/expression. There is no concatenation operator in bash, you just place strings one after the other with no space in between.
The $'\t' resolves as a literal tab-character in bash (using ANSI-C quoting), so the tab is correctly concatenated into the expression.
At least on Mountain Lion, Viktor's answer will also remove the character 't' when it is at the end of a line. The following fixes that issue:
sed -i '' -e's/[[:space:]]*$//' "$1"
Thanks to codaddict for suggesting the -i option.
The following command solves the problem on Snow Leopard
sed -i '' -e's/[ \t]*$//' "$1"
It is best to also quote $1:
sed -i.bak 's/[[:blank:]]*$//' "$1"
var1="\t\t Test String trimming "
echo $var1
Var2=$(echo "${var1}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
echo $Var2
I have a script in my .bashrc that works under OSX and Linux (bash only !)
function trim_trailing_space() {
if [[ $# -eq 0 ]]; then
echo "$FUNCNAME will trim (in place) trailing spaces in the given file (remove unwanted spaces at end of lines)"
echo "Usage :"
echo "$FUNCNAME file"
return
fi
local file=$1
unamestr=$(uname)
if [[ $unamestr == 'Darwin' ]]; then
#specific case for Mac OSX
sed -E -i '' 's/[[:space:]]*$//' $file
else
sed -i 's/[[:space:]]*$//' $file
fi
}
to which I add:
SRC_FILES_EXTENSIONS="js|ts|cpp|c|h|hpp|php|py|sh|cs|sql|json|ini|xml|conf"
function find_source_files() {
if [[ $# -eq 0 ]]; then
echo "$FUNCNAME will list sources files (having extensions $SRC_FILES_EXTENSIONS)"
echo "Usage :"
echo "$FUNCNAME folder"
return
fi
local folder=$1
unamestr=$(uname)
if [[ $unamestr == 'Darwin' ]]; then
#specific case for Mac OSX
find -E $folder -iregex '.*\.('$SRC_FILES_EXTENSIONS')'
else
#Rhahhh, lovely
local extensions_escaped=$(echo $SRC_FILES_EXTENSIONS | sed s/\|/\\\\\|/g)
#echo "extensions_escaped:$extensions_escaped"
find $folder -iregex '.*\.\('$extensions_escaped'\)$'
fi
}
function trim_trailing_space_all_source_files() {
for f in $(find_source_files .); do trim_trailing_space $f;done
}
For those who look for efficiency (many files to process, or huge files), using the + repetition operator instead of * makes the command more than twice faster.
With GNU sed:
sed -Ei 's/[ \t]+$//' "$1"
sed -i 's/[ \t]\+$//' "$1" # The same without extended regex
I also quickly benchmarked something else: using [ \t] instead of [[:space:]] also significantly speeds up the process (GNU sed v4.4):
sed -Ei 's/[ \t]+$//' "$1"
real 0m0,335s
user 0m0,133s
sys 0m0,193s
sed -Ei 's/[[:space:]]+$//' "$1"
real 0m0,838s
user 0m0,630s
sys 0m0,207s
sed -Ei 's/[ \t]*$//' "$1"
real 0m0,882s
user 0m0,657s
sys 0m0,227s
sed -Ei 's/[[:space:]]*$//' "$1"
real 0m1,711s
user 0m1,423s
sys 0m0,283s
Just for fun:
#!/bin/bash
FILE=$1
if [[ -z $FILE ]]; then
echo "You must pass a filename -- exiting" >&2
exit 1
fi
if [[ ! -f $FILE ]]; then
echo "There is not file '$FILE' here -- exiting" >&2
exit 1
fi
BEFORE=`wc -c "$FILE" | cut --delimiter=' ' --fields=1`
# >>>>>>>>>>
sed -i.bak -e's/[ \t]*$//' "$FILE"
# <<<<<<<<<<
AFTER=`wc -c "$FILE" | cut --delimiter=' ' --fields=1`
if [[ $? != 0 ]]; then
echo "Some error occurred" >&2
else
echo "Filtered '$FILE' from $BEFORE characters to $AFTER characters"
fi
In the specific case of sed, the -i option that others have already mentioned is far and away the simplest and sanest one.
In the more general case, sponge, from the moreutils collection, does exactly what you want: it lets you replace a file with the result of processing it, in a way specifically designed to keep the processing step from tripping over itself by overwriting the very file it's working on. To quote the sponge man page:
sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This allows constructing pipelines that read from and write to the same file.
https://joeyh.name/code/moreutils/
To remove trailing whitespace for all files in the current directory, I use
ls | xargs sed -i 's/[ \t]*$//'
These answers confused me. Both of these sed commands worked for me on a Java source file:
sed 's/\s\+$/ filename
sed 's/[[:space:]]\+$// filename
for test purposes, I used:
$ echo " abc " | sed 's/\s\+$/-xx/'
abc-xx
$ echo -e " abc \t\t " | sed 's/\s\+$/-xx/'
abc-xx
Replacing all trailing whitespace with "-xx".
#Viktor wishes to avoid a temporay file, personally I would only use the -i => in-place with a back-up suffix. At least until I know the command works.
Sorry, I just found the existing responses a little oblique. sed is straightforward tool. It is easier to approach it in a straightforward way 90% of the time. Or perhaps I missed something, happy to corrected there.
To only strip whitespaces (in my case spaces and tabs) from lines with at least one non-whitespace character (this way empty indented lines are not touched):
sed -i -r 's/([^ \t]+)[ \t]+$/\1/' "$file"