I'm trying to set up an exclusion list to pass to "grep -v" via a shell var, made up of mount points with the initial "/" removed and all separated by spaces. I'm building it like this:
EXCLUDE_LIST=$(mount | grep -v '/dev/' | awk '{ print $3 }' | sed 's,^[^/]*/,,' )
This yields the string 'dev proc sys tmp var/run'. This is the string I want to replace spaces with the '\|' so "grep -v" can use it.
Ive tried appending
sed 's/ /\|/g'
to the previous pipeline, then
sed 's/ /\\|/g'
and
sed 's/ /\\\|/g'
What I want is "dev\|proc\|sys\|tmp\|var/run", but I'm always getting "dev proc sys tmp var/run".
EDIT
The output of mount is
/dev/sda1 on / type ext4 (rw,reltime,nobarrier,commit=10)
devtmpfs on /dev type devtmpfs (rw,nosuid,relatime,sync,size=1024k,nr_inodes=502793,mode=755)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /dev/shm type tmpfs (rw,relatime,sync,size=1024k)
procfs on /proc type procfs (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,relatime,sync,size=524288k)
tmpfs on /var/run type tmpfs (rw,relatime,sync,size=131072k)
The output of "mount | grep -v '/dev/'" is:
devtmpfs on /dev type devtmpfs (rw,nosuid,relatime,sync,size=1024k,nr_inodes=502793,mode=755)
procfs on /proc type procfs (rw,nosuid,nodev,noexec,relatime)
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /tmp type tmpfs (rw,relatime,sync,size=524288k)
tmpfs on /var/run type tmpfs (rw,relatime,sync,size=131072k)
The output of "mount | grep -v '/dev/' | awk { print $3 }" is:
/dev
/proc
/sys
/tmp
/var/run
And finally, the output of "mount | grep -v '/dev/' | awk { print $3 } | sed 's,^[^/]*/,,'" is
dev proc sys tmp var/run
as previously stated.
You can use a single awk command:
mount | awk '!/\/dev\//{sub("/",d,$3);print $3;d="\\|"}' ORS=''
With the given input of mount it will print
dev\|proc\|sys\|tmp\|var/run
Explanation:
# Applies to all lines with does not match /dev
!/\/dev/{
# Remove leading / with delimiter d
# d will be empty upon first use
sub("/","",$3)
# Print the column
print $3
# Set delimiter after first usage
d="\\|"
}
ORS='' will silence awk's output record delimiter.
Try append this:
awk -v RS="" '{gsub(/[\r\n]+/,"\\|")}1'
That is:
mount | grep -v '/dev/' | awk '{ print $3 }' | sed 's,^[^/]*/,,' | awk -v RS="" '{gsub(/[\r\n]+/,"\\|")}1'
#=> sys\|proc\|dev\|sys/kernel/security\|run\|sys/fs/cgroup\|sys/fs/cgroup/systemd\|...
Btw, what system are you in? -- Seems it treats mount's output \n newlines as spaces.
-- Or is it just your typing error?
Since normally your series of commands will yield:
sys
proc
dev
sys/kernel/security
run
...
But you said it's a spaces separated string, so my best guess is your system/editor shows \n as space, which misled you believe they are spaces.
Correct me if I guess wrong.
Made a mistake, just fixed.
Also I don't believe sed will just change line-endings to spaces by itself...
Well, for fun, tried awk way too:
mount | awk -v ORS="\\\|" '$0!~"/dev/"{print gensub("^/","",1,$3)}'
#=> sys\|proc\|dev\|sys/kernel/security\|run\|sys/fs/cgroup\|sys/fs/cgroup/systemd\|...
For fun again:
mount| awk '!(FNR%2) && !/dev\/|^$/' RS="on /| type " ORS="\\\|"
Both awk will leave trailing \|.
Related
I am trying to run the command in kdb but it does not work. I knew it may be associated with some special character and I am trying to put the [] to escape but still does not work.
system "awk '/^Mem/ {print $2}' <(free -m)"
I tried
system "awk '/[^]Mem/ {print $2}' <(free -m)" - not working
Another solution just requires a rearrangement of your expression:
q)system"free -m | awk '/^Mem/ {print $2}'"
"25408"
EDIT:
The reason why your expression is failing from q is because of the shell being used. This answer explains the difference between shebang and dash shells. I've added a little test to showcase the difference.
coneill5#LPTP1893: [~] $ cat test.sh
#!/bin/bash
awk '/^Mem/ {print $2}' <(free -m)
coneill5#LPTP1893: [~] $ cat test1.sh
#!/bin/sh
awk '/^Mem/ {print $2}' <(free -m)
coneill5#LPTP1893: [~] $ ./test.sh
25408
coneill5#LPTP1893: [~] $ ./test1.sh
./test1.sh: 2: Syntax error: "(" unexpected
You can avoid a system call as kdb+ can return the physical memory available using .Q.w[]
q)floor (.Q.w[]`mphy)%1024 xexp 2
28072
\\
$ awk '/^Mem/ {print $2}' <(free -m)
28072
I'm trying to make a replace with sed:
cat myfile | grep router | sed -e 's/Custom devices \(DiY\) \[CC2530 router\]\(http:\/\/ptvo\.info\/cc2530-based-zigbee-coordinator-and-router-112\/\) \(CC2530\.ROUTER\)/CC2530 router/g'
The output of the piped grep is:
<text text-anchor="middle" x="455.5" y="-31.3" font-family="Times,serif" font-size="14.00" fill="#ffffff">Custom devices (DiY) [CC2530 router](http://ptvo.info/cc2530-based-zigbee-coordinator-and-router-112/) (CC2530.ROUTER)</text>
This works well:
cat myfile | grep Xiaomi | sed -e 's/Xiaomi Aqara temperature, humidity and pressure sensor/AqaraTHP/g'
What am I missing here?
The man page says that
-e command
Append the editing commands specified by the command argument to the
list of commands.
You need to use -E or -r option (whichever is supported by your sed):
cat myfile | grep router | sed -E 's/Custom devices \(DiY\) \[CC2530 router\]\(http:\/\/ptvo\.info\/cc2530-based-zigbee-coordinator-and-router-112\/\) \(CC2530\.ROUTER\)/CC2530 router/g'
With -E, \( denotes a plain ( symbol and ( is the indicator of capturing group.
I want to view for the files in /var/www the following data:
fullpath size mtime ctime md5
I ran the following command:
find /var/www/ -maxdepth 1 ! -type d -printf '%p\t%s\t%t\t%c\t' -exec md5sum {} \;
which gives me:
(fullpath size mtime ctime md5 fullpath)
/var/www/intranet/admin/tpl/view.tpl.php 1448 Wed Dec 16 18:51:06.0000000000 2015 Fri Sep 15 09:08:36.0805775786 2017 e0b7dacaf7c90fb0fbe7a69c331e36aa /var/www/intranet/admin/tpl/view.tpl.php
How can I filter the last fullpath?????? I do not want to show it. All fields are TAB separated.
I tried:
find /var/www/ -maxdepth 1 ! -type d -printf '%p\t%s\t%t\t%c\t'
-exec md5sum {} | awk '{print $1}'\;
for which I received the error: "find: missing argument to `-exec'"
find /var/www/ -maxdepth 1 ! -type d -printf '%p\t%s\t%t\t%c\t' -exec md5sum {} + | awk '{print $1}'
for which I got only the md5sum.
Thanks in advance!
Pipelines (|) are a shell-feature. To get shell features, one needs to invoke a shell:
find /var/www -maxdepth 1 ! -type d -printf '%p\t%s\t%t\t%c\t' -exec sh -c 'md5sum "$1" | awk '\''{print $1}'\' MD5 {} \;
Or, if you prefer commands spread over multiple lines:
find /var/www \
-maxdepth 1 \
! -type d \
-printf '%p\t%s\t%t\t%c\t' \
-exec sh -c 'md5sum "$1" | awk '\''{print $1}'\' MD5 {} \;
Notes
sh -c somestring invokes a shell and instructs it to execute whatever commands are in somestring.
sh -c somestring MD5 {} invokes the shell and executes somestring and assigns $0 to MD5 and $1 to whatever find substitutes for {}.
$0 is only used by the shell when it creates error messages and otherwise unimportant.
A complication is that our command, somestring, must contain both single quotes and double-quotes which is why we need escaped single-quotes.
In our case, we want somestring to be:
md5sum "$1" | awk '{print $1}'
To prevent the main shell from substituting in for $1, we need to put this inside single-quotes. However, we can't put single-quotes inside single-quotes. The workaround is to use this for our single-quoted string:
'md5sum "$1" | awk '\''{print $1}'\'
I am trying to grep ip addresses from one file and remove the contents of the grep from a second file with a single command.
The masterfile has
header x.x.x.x
header x.x.x.x
header2 y.y.y.y
header2 y.y.y.y
header3 z.z.z.z
header3 z.z.z.z
The tempfile has
x.x.x.x
x.x.x.x
y.y.y.y
y.y.y.y
z.z.z.z
z.z.z.z
Attempt
grep "header" masterfile | sed 's/^.* //' | sed -i "" '/$/d' tempfile
Would also like to remove the empty line after the sed command removes an entry.
If I understand it correct, you need the following:
grep -v -F -f <(grep '\<header\>' masterfile | cut -d' ' -f2) tempfile
or
grep -v -F -f <(sed 's/^header *//' masterfile) tempfile
For your input, it'd produce:
y.y.y.y
y.y.y.y
z.z.z.z
z.z.z.z
In order to save the changes to tempfile, you could redirect the command output to another file and move it to the desired file:
grep -v -F -f <(sed 's/^header *//' masterfile) tempfile > tmp && mv tmp tempfile
EDIT: You seem to be sh which doesn't support Process Substitution. In that case, you can use the following:
grep '\<header\>' master | cut -d' ' -f2 | grep -v -F -f - tempfile
Produce a sed script from the first file and feed it to a second sed instance.
sed 's%^header \(.*\)%/^\1\$/d%' masterfile |
sed -i "" -f - tempfile
On Linux, -f - says to read the script from standard input. On some other platforms, this does not work; then, you'll have to save the script to a temporary file (or maybe you have /dev/fd/1 or /dev/stdin).
"ls" behaves differently when its output is being piped:
> ls ???
bar foo
> ls ??? | cat
bar
foo
How does it know, and how would I do this in Perl?
In Perl, the -t file test operator indicates whether a filehandle
(including STDIN) is connected to a terminal.
There is also the -p test operator to indicate whether a filehandle
is attached to a pipe.
$ perl -e 'printf "term:%d, pipe:%d\n", -t STDIN, -p STDIN'
term:1, pipe:0
$ perl -e 'printf "term:%d, pipe:%d\n", -t STDIN, -p STDIN' < /tmp/foo
term:0, pipe:0
$ echo foo | perl -e 'printf "term:%d, pipe:%d\n", -t STDIN, -p STDIN'
term:0, pipe:1
File test operator documentation at perldoc -f -X.
use IO::Interactive qw(is_interactive);
is_interactive() or warn "Being piped\n";