Fairly certain I am missing something obvious!
$ cat test.txt
00 30 * * * /opt/timebomb.sh >> timebomb.log
01 30 * * * /opt/reincarnate.sh >> reincarnation.log
$ sed ':timebomb:d' test.txt
00 30 * * * /opt/timebomb.sh >> timebomb.log
01 30 * * * /opt/reincarnate.sh >> reincarnation.log
whereas
$ sed '/timebomb/d' test.txt
01 30 * * * /opt/reincarnate.sh >> reincarnation.log
Why is it the case? Aren't a different set of delimiters supported for the d command?
The delimiters // that you're using are not for the d command, they're for the addressing. I think you're comparing them to the slashes in the s/// command... However although both relate to regular expressions, they are different contexts.
The address (which appears before the command) filters which lines the command is applied to... The options (which appear after the command) are used to control the replacement applied.
If you want to use different delimiters for a regular expression in the address, you need to start the address with a backslash:
$ sed '\:timebomb:d' test.txt
01 30 * * * /opt/reincarnate.sh >> reincarnation.log
(To understand the difference between the address and the options, consider the output of the command:
$ sed '/timebomb/s/log/txt/' test.txt
The address chooses only the line(s) containing the regular expression timebomb, but the options tell the s command to replace the regular expression log with txt.)
The colon preceeds a label in sed, so your sed program looks like a couple of labels with nothing happening. The parser can see colons as delimiters if preceded by the s command, so that's a special case.
use gawk
gawk '!/timebomb/' file > newfile
you can just use the shell, no need external tools
while read -r line
do
case "$line" in
*timebomb* ) continue;;
*) echo "$line";;
esac
done <"file"
Related
I have read sed manual for the s/// command. There it says:
e
This command allows one to pipe input from a shell command into pattern
space. If a substitution was made, the command that is found in
pattern space is executed and pattern space is replaced with its output.
A trailing newline is suppressed; results are undefined if the command
to be executed contains a nul character. This is a GNU sed extension.
I don't know what is useful:
echo "1234" | sed 's/1/echo ss/e'
echo "1234" | sed 's/1/ss/'
These two commands result in the same, so what is the e modifier about?
$ printf "%s\n" 1234 2345 3456 |
> sed -e 's/\(..\)\(..\)/echo $((\1 * \2))/e'
408
1035
1904
$
This printf command echoes three 4-digit numbers on three lines. The sed script splits each line into a pair of 2-digit numbers, creates a command echo $((12 * 34)), for example, runs it, and the output (408 for the given values) is included in (as) the pattern space — which is then printed. So, for this script, the pairs of 2-digit numbers are multiplied and the result is shown.
You can get fancier if you wish:
$ printf "%s\n" 1234 2345 3456 |
> sed -e 's/\(..\)\(..\)/echo \1 \\* \2 = $((\1 * \2))/e'
12 * 34 = 408
23 * 45 = 1035
34 * 56 = 1904
$
Note the double backslash — that's rather important. You could avoid the need for that using double quotes:
printf "%s\n" 1234 2345 3456 |
sed -e 's/\(..\)\(..\)/echo "\1 * \2 = $((\1 * \2))"/e'
Beware: the notation will run the external command every time the s/// command actually makes a substitution. If you have millions of lines of data in your files, that could mean millions of commands executed.
The /e option is a GNU sed extension. It causes the result of the replacement to be passed to the shell for evaluation as a command. Observe:
vnix$ sed 's/a/pwd/' <<<a
pwd
vnix$ sed 's/a/pwd/e' <<<a
/home/tripleee
Your example caused identical behavior (echoing back exactly the replaced text) so it was a poorly chosen example.
Out of the box, the pattern space refers to the current input line, but there are ways to put something else in the pattern space. Substitution modifies the pattern space, but there are other sed commands which modify the pattern space in other ways. (For a trivial example, x swaps the pattern space with the hold space, which is basically another built-in variable which you can use for whatever you want.)
I have function that prints a header that needs to be applied across several files, but if I utilize a sed process substitution the lines prior to the last have a backslash \ on them.
E.g.
function print_header() {
cat << EOF
-------------------------------------------------------------------
$(date '+%B %d, %Y # ~ %r') ID:$(echo $RANDOM)
EOF
}
If I then take a file such as test.txt:
line 1
line 2
line 3
line 4
line 5
sed "1 i $(print_header | sed 's/$/\\/g')" test.txt
I get:
-------------------------------------------------------------------\
November 24, 2015 # ~ 11:18:28 AM ID:13187
line 1
line 2
line 3
line 4
line 5
Notice the troublesome backslash at the end of the first line, I'd like to not have that backslash appear. Any ideas?
I would use cat for that:
cat <(print_header) file > file_with_header
This behavior depends on the sed dialect. Unfortunately, it's one of the things which depends on which version you have.
To simplify debugging, try specifying verbatim text. Here's one from a Debian system.
vnix$ sed '1i\
> foo\
> bar' <<':'
> hello
> goodbye
> :
foo
bar
hello
goodbye
Your diagnostics appear to indicate that your sed dialect does not in fact require the backslash after the first i.
Since you are generating the contents of the header programmatically anyway, my recommended solution would be to refactor the code so that you can avoid this conundrum. If you don't want cat <<EOF test.txt then maybe experiment with sed 1r/dev/stdin' <<EOF test.txt (I could not get 1r- to work, but /dev/stdin should be portable to any Linux.)
Here is my kludgy fix, if you can find something more elegant I'll gladly credit you:
sed "1 i $(print_header | sed 's/$/\\/g;$s/$/\x01/')" test.txt | tr -d '\001'
This puts an unprintable SOH (\x01) ascii Start Of Header character after the inserted text, that precludes the backslashes and then I run it over tr to delete the SOH chars.
I am new to bash and having a tough time figuring this out.
Using sed, could anyone help me in finding only even numbers in a given file?
I figured out how to find all numbers starting from [0,2,4,6,8] using this:
sed -n 's/^[0-9]*[02468] /&/w even' <file
But this doesn't guarantee that the number is even for sure.
I am having trouble in finding if the matched number ends with either [0,2,4,6,8] for it to be even for sure.
So can any one help me out with this?
Your regex looks a bit weird and I am not sure what you want to do, but this should help:
sed -r -n 's/^[0-9]*?[02468] /even/g'
-r to enable extended regex, *? to make it non-greedy, and /g to perform replacement globally for all lines in file.
Your command should work fine assuming that there is a space after all even numbers and that they are all at the beginning of the lines:
$ echo 'foo
1231
2220
1254 ' | sed -n '/[0-9]*[02468] /p'
2220
1254
Also note that, as you don't actually do a substitution, you don't need the s command. Use an address (pattern) specifier and w command (like I did above with the p command).
To make sure that the even digit is the last, but is not necessarily followed by a space, you can do something like
$ echo 'foo
1231
2220
1254 ' | sed -n '/[0-9]*[02468]\($\|[^0-9]\)/p'
2220
1254
Actually, your case looks more like a use case for grep, not sed, because you do filtering rather than editing. Everything becomes easier with GNU grep, as you can do
$ echo 'foo
1231
2220
1254 ' | grep -P '\d*[02468](?!\d)'
2220
1254
Just append > even to the command to make it write to the file even.
$ cat file
1
2
3
498
57
12345678
$ awk '$0%2' file
1
3
57
$ awk '!($0%2)' file
2
498
12345678
Why don't you find the numbers ending with [02468] ?
We have a process which can use a file containing sed commands to alter piped input.
I need to replace a placeholder in the input with a variable value, e.g. in a single -e type of command I can run;
$ echo "Today is XX" | sed -e "s/XX/$(date +%F)/"
Today is 2012-10-11
However I can only specify the sed aspects in a file (and then point the process at the file), E.g. a file called replacements.sed might contain;
s/XX/Thursday/
So obviously;
$ echo "Today is XX" | sed -f replacements.sed
Today is Thursday
If I want to use an environment variable or shell value, though, I can't find a way to make it expand, e.g. if replacements.txt contains;
s/XX/$(date +%F)/
Then;
$ echo "Today is XX" | sed -f replacements.sed
Today is $(date +%F)
Including double quotes in the text of the file just prints the double quotes.
Does anyone know a way to be able to use variables in a sed file?
This might work for you (GNU sed):
cat <<\! > replacements.sed
/XX/{s//'"$(date +%F)"'/;s/.*/echo '&'/e}
!
echo "Today is XX" | sed -f replacements.sed
If you don't have GNU sed, try:
cat <<\! > replacements.sed
/XX/{
s//'"$(date +%F)"'/
s/.*/echo '&'/
}
!
echo "Today is XX" | sed -f replacements.sed | sh
AFAIK, it's not possible. Your best bet will be :
INPUT FILE
aaa
bbb
ccc
SH SCRIPT
#!/bin/sh
STRING="${1//\//\\/}" # using parameter expansion to prevent / collisions
shift
sed "
s/aaa/$STRING/
" "$#"
COMMAND LINE
./sed.sh "fo/obar" <file path>
OUTPUT
fo/obar
bbb
ccc
As others have said, you can't use variables in a sed script, but you might be able to "fake" it using extra leading input that gets added to your hold buffer. For example:
[ghoti#pc ~/tmp]$ cat scr.sed
1{;h;d;};/^--$/g
[ghoti#pc ~/tmp]$ sed -f scr.sed <(date '+%Y-%m-%d'; printf 'foo\n--\nbar\n')
foo
2012-10-10
bar
[ghoti#pc ~/tmp]$
In this example, I'm using process redirection to get input into sed. The "important" data is generated by printf. You could cat a file instead, or run some other program. The "variable" is produced by the date command, and becomes the first line of input to the script.
The sed script takes the first line, puts it in sed's hold buffer, then deletes the line. Then for any subsequent line, if it matches a double dash (our "macro replacement"), it substitutes the contents of the hold buffer. And prints, because that's sed's default action.
Hold buffers (g, G, h, H and x commands) represent "advanced" sed programming. But once you understand how they work, they open up new dimensions of sed fu.
Note: This solution only helps you replace entire lines. Replacing substrings within lines may be possible using the hold buffer, but I can't imagine a way to do it.
(Another note: I'm doing this in FreeBSD, which uses a different sed from what you'll find in Linux. This may work in GNU sed, or it may not; I haven't tested.)
I am in agreement with sputnick. I don't believe that sed would be able to complete that task.
However, you could generate that file on the fly.
You could change the date to a fixed string, like
__DAYOFWEEK__.
Create a temp file, use sed to replace __DAYOFWEEK__ with $(date +%Y).
Then parse your file with sed -f $TEMPFILE.
sed is great, but it might be time to use something like perl that can generate the date on the fly.
To add a newline in the replacement expression using a sed file, what finally worked for me is escaping a literal newline. Example: to append a newline after the string NewLineHere, then this worked for me:
#! /usr/bin/sed -f
s/NewLineHere/NewLineHere\
/g
Not sure it matters but I am on Solaris unix, so not GNU sed for sure.
I am trying to use unix sed command to find line numbers that match a particular regular expression. The pattern of my file is below
A<20 spaces>
<something>
<something>
..
..
A<20 spaces>
<soemthing>
<something>
I need all the line numbers of A<20 spaces>
I used sed -n '/A[ ]{20}/'= <file_name> but it does not work. If I manually type in twenty spaces it does work.
Can some one please tweak the above command to make it work.
The braces in the expression need to be escaped with backslashes:
% sed -n '/A[ ]\{20\}/=' test.txt
1
6
An alternative would be to use -E to interpret regular expressions as extended (modern) regular expressions:
% sed -nE '/A[ ]{20}/=' test.txt
1
6
Or potentially use grep instead, which takes fewer characters to specify the same search:
% grep -n 'A[ ]\{20\}' test.txt
The correct syntax would be /A \{20\}/ (and I'm failing to understand where you got your syntax from).
edit: repeat a space, not an A. not my day
use the -E or the -r switch for extended regexp
just to be sur of the content request is answered because it literraly mean "20 spaces" and not Twenty " " char that everyone understand due to the sed line sample failing (i guess this the good one so other reply are fine in this case)
sed -n "/<20 spaces>/ =" file_name