I have the following log line:
2017-01-01 Client:abc ID:144
I need to process it and insert it in MySQL, but sometimes the data might come wrong:
2017-01-01 Client:a, b .c ID:144
How can I perform a sed replace only inside the "Client:xxx" group so I can remove the unwanted characters?
Using backreferences and t command, to capture and output only wanted characters between Client: and ID:
$ sed -r ':a;s/(Client:[^ ,.]*)[ ,.](.* ID)/\1\2/;ta;' <<< "2017-01-01 Client:a, b .c ID:144"
2017-01-01 Client:abc ID:144
All characters up to and excluding next unwanted character([ ,.]) are captured and output. The t(for test) command loops to the beginning of the script(to the :a label) if the substitution succeeds.
Related
I am looking for non printable characters into a file, and I found this web page.
It shows the following command:
sed "l" file
If I am not mistaken, according to man, this option is:
List out the current line in a ''visually unambiguous'' form.
Moreover, when I run this command on a fake file with one line, the output is as follow:
The line is displayed twice, but each displayed line (in the output) contains at most 69 bytes of the input line. The rest of the line is displayed at the next line.
The second time the line is displayed, it is in its full length.
fake file
toto, titi, tatafdsfdsfdgfgfdsgrgdfgzfdgzgffgerssssssssssssssssssssssssss
Command
sed "l" fake_file
output
$ sed "l" fake_file
toto, titi, tatafdsfdsfdgfgfdsgrgdfgzfdgzgffgerssssssssssssssssssssss\
ssss$
toto, titi, tatafdsfdsfdgfgfdsgrgdfgzfdgzgffgerssssssssssssssssssssssssss
Questions
What does ''visually unambiguous'' exactly mean ?
Why is the output like this ? I was expecting only one line with the $ sign at the end. I was also not expecting output to be displayed on 69 bytes max.
Environment
Tested with same output on:
sed (GNU sed) 4.7
sed (GNU sed) 4.2.2
By default, sed outputs the result after processing a line. If you handle the output yourself, tell sed not to output the line by the -n switch.
sed -i '/#if UTS_UBUNTU_RELEASE_ABI > 255/c\/*#if UTS_UBUNTU_RELEASE_ABI > 255' /usr/src/ixgbevf-2.16.4/src/kcompat.h
I am trying to understand the above command but couldn't figure out what c\ is doing here?
This sed command says to test lines for the regular expression #if UTS_UBUNTU_RELEASE_ABI > 255 (because it starts with forward slash), if so, use the change command to replace the whole line with whatever follows. (the -i means in place.)
In this case, it will change the matching line to be the beginning of a block comment (inserts /*) per my local testing.
How to remove line if characters do not exist after a symbol (e.g. #)?
E.g.
hello#lawyer
B#b
smith#
Nac#gyo
treat#
Lines smith# and treat# will be removed as there are no characters after #.
I would post sample of my experimentation -- but have been so far off the mark that would be unhelpful.
using the delete command d and the $ anchor that matches the end of the line:
sed '/#$/d' file
/#$/: when this pattern succeeds, the d command is executed.
You could use grep -v:
grep -v '#$'
to exclude all lines that match the pattern "line ends with #".
I am trying to use a sed command to replace specials characters in my file.
The characters are %> to replace by ].
I'am using sed -r s/\%>\/\]\/g but i have this error bash: /]/g: No such file or directory, looks like sed doesn't like it.
Put your sed code inside quotes and also add the file-path you want to work with and finally don't escape the sed delimiters.
$ echo '%>' | sed 's/%>/]/g'
]
ie,
sed 's/%>/]/g' file
To complement Avinash Raj's correct and helpful answer:
Since you were using an overall unquoted string (neither single- nor double-quoted), you were on the right track by \-escaping individual characters in your sed command.
However, you neglected to \-quote >, which is what caused your problem:
> is one of the shell's so-called metacharacters
Metacharacters have special meaning and separate words
Thus, s/\%>\/\]\/g is mistakenly split into 2 arguments by >:
s/\% is passed to sed - as s/%, because the shell removes the \ instances (a process called quote removal).
As you can see, this is not a valid sed command, but that doesn't even come into play - see below.
>\/\]\/g is interpreted by the shell (bash), because it starts with output-redirection operator >; after quote removal, the shell sees >/]/g, tries to open file /]/g for writing, and fails, because your system doesn't have a subdirectory named ] in its root directory.
bash tries to open an output file specified by a redirection before running the command and, if it fails to open the file, does not run the command - which is what happened here:
bash complained about the nonexistent target directory and aborted processing of the command - sed was never even invoked.
Upshot:
In a string that is neither enclosed in single nor in double-quotes, you must \-quote:
all metacharacters: | & ; ( ) < > space tab
additionally, to prevent accidental pathname expansion (globbing): * ? [
Also note that if you need to quote (escape) characters for sed,you need to add an extra layer of quoting; for instance to instruct sed to use a literal . in the regex, you must pass \\. - two backslashes - so that sed sees the properly escaped \..
Given the above, it is much simpler to (habitually) use single quotes around your sed command, because it ensures that the string is passed as is to sed.
Let's compare a working version of your command to the one from Avinash Raj's answer (leaving out the -r for brevity):
sed s/\%\>\/\]\/g # ok - all metachars. \-quoted, others are, but needn't be quoted
sed s/%\>/]/g # ok - minimum \-quoting
sed 's/%>/]/g' # simplest: single-quoted command
I'm not sure whether I got the question correctly. If you want to replace either % or > by ] then sed is not required here. Use tr in this case:
tr '%>' ']' < input.txt
If you want to replace the sequence %> by ] then the sed command as shown by #AvinashRaj is the way to go.
I couldn't find an answer for this exact problem, so I'll ask it.
I'm working in Cygwin and want to reference previous commands using !n notation, e.g., if command 5 was which ls, then !5 runs the same command.
The problem is when trying to do substitution, so running:
!5:s/which \([a-z]\)/\1/
should just run ls, or whatever the argument was for which for command number 5.
I've tried several ways of doing this kind of substitution and get the same error:
bash: :s/which \([a-z]*\)/\1/: substitution failed
As far as I can tell the s/old/new/ history substitution syntax only does simple string substitution; it does not support full regexes. Here's what man bash has to say:
s/old/new/
Substitute new for the first occurrence of old in the event line. Any delimiter can be used in place of /. The final delimiter is optional if it is the last character of the event line. The delimiter may be quoted in old and new with a single backslash. If & appears in new, it is replaced by old. A single backslash will quote the &. If old is null, it is set to the last old substituted, or, if no previous history substitutions took place, the last string in a !?string[?] search.
Never fear, though. There are in fact easier ways to accomplish what you are trying to do:
!$ evaluates to the last argument of the previous command:
# ls /etc/passwd
/etc/passwd
# vim !$
vim /etc/passwd
!5:$ evaluates to the last argument of command #5:
# history
...
5: which ls
...
# !5:$
ls
You can also use Alt+. to perform an immediate substitution equivalent to !$. Alt+. is one of the best bash tricks I know.
This worked for me using Bash in Cygwin (note that my which ls command was number 501 in my history list; not 5 like yours):
$(!501 | sed 's/which \([a-z]\)/\1/')
You could also do it this way (which is shorter/cleaner):
$(!501 | sed 's/which //')