I am trying with the below options
1.copied the data i need in text from out put of
ps ax -o rss,command | sort -nr | head -n 10
2.But this output contains extract like below
856232 /usr/java/jdk1.7.0/bin/java -Djava.util.logging.config.file=/data/vmware/server/xxxxx/conf/logging.properties -XX:MaxPermSize=512m -Xmx1024m -Xms1024m -XX:PermSize=256m -Xss256k -
but i need the string only after /data/vmware/server ie xxxxx.
i tried sed and awk but not getting intended ones.
You could Use grep.
ps ax -o rss,command | sort -nr | head -n 10 | grep -oP 'data/vmware/server/\K[^/]*'
Example:
$ echo '856232 /usr/java/jdk1.7.0/bin/java -Djava.util.logging.config.file=/data/vmware/server/xxxxx/conf/logging.properties -XX:MaxPermSize=512m -Xmx1024m -Xms1024m -XX:PermSize=256m -Xss256k -' | grep -oP 'data/vmware/server/\K[^/]*'
xxxxx
OR
sed.
$ echo '856232 /usr/java/jdk1.7.0/bin/java -Djava.util.logging.config.file=/data/vmware/server/xxxxx/conf/logging.properties -XX:MaxPermSize=512m -Xmx1024m -Xms1024m -XX:PermSize=256m -Xss256k -' | sed 's~.*data/vmware/server/\([^/]*\).*~\1~'
xxxxx
Related
Just a basic question about grep command line. I found the way how to know that the service is running in process by using this command line:
ps -ef |grep -v grep | grep mongodb
I'm confused about the second grep:
|grep -v grep |
Why I need to use the "grep" after " -v " ???
What is the different between that command and this command ?
ps -ef |grep mongodb
Thank you!
When you grep "mongodb" through command line, your command also contains the word "mongodb" . So you will get two results. Which is flawed result. grep -v is to remove the grep command typed by user.
sh-4.1$ ps -ef |grep -v grep | grep mongodb
ps 17308 30074 0 06:05 pts/300 00:00:00 sh mongodb
vs
sh-4.1$ ps -ef |grep mongodb
ps 17308 30074 0 06:05 pts/300 00:00:00 sh mongodb
ps 17456 30074 0 06:05 pts/300 00:00:00 grep mongodb #<<<This also contains mongodb word. Hence result is flawed.
The -v option tells grep not to let through anything matching the pattern, in this case lines that contain the string "grep".
So if you omit the grep -v grep
your grep process itself would also be displayed in the output after the second command in the pipe (and also after the third, as the grep process itself contains the word "mongodb").
I am trying to grep ip addresses from one file and remove the contents of the grep from a second file with a single command.
The masterfile has
header x.x.x.x
header x.x.x.x
header2 y.y.y.y
header2 y.y.y.y
header3 z.z.z.z
header3 z.z.z.z
The tempfile has
x.x.x.x
x.x.x.x
y.y.y.y
y.y.y.y
z.z.z.z
z.z.z.z
Attempt
grep "header" masterfile | sed 's/^.* //' | sed -i "" '/$/d' tempfile
Would also like to remove the empty line after the sed command removes an entry.
If I understand it correct, you need the following:
grep -v -F -f <(grep '\<header\>' masterfile | cut -d' ' -f2) tempfile
or
grep -v -F -f <(sed 's/^header *//' masterfile) tempfile
For your input, it'd produce:
y.y.y.y
y.y.y.y
z.z.z.z
z.z.z.z
In order to save the changes to tempfile, you could redirect the command output to another file and move it to the desired file:
grep -v -F -f <(sed 's/^header *//' masterfile) tempfile > tmp && mv tmp tempfile
EDIT: You seem to be sh which doesn't support Process Substitution. In that case, you can use the following:
grep '\<header\>' master | cut -d' ' -f2 | grep -v -F -f - tempfile
Produce a sed script from the first file and feed it to a second sed instance.
sed 's%^header \(.*\)%/^\1\$/d%' masterfile |
sed -i "" -f - tempfile
On Linux, -f - says to read the script from standard input. On some other platforms, this does not work; then, you'll have to save the script to a temporary file (or maybe you have /dev/fd/1 or /dev/stdin).
I have a script2:
# This is script2 that is called by script1.
CURRENT_TOMCAT_PROCESS=`ps -ef | grep java | grep $TOMCAT_USER | grep -v grep | awk '{print $2}'`
echo "---> $CURRENT_TOMCAT_PROCESS"
and I call script2 in script1:
ssh $user#$server 'bash -s' < script2
It works fine. But I'm having trouble make the backtick work in a HERE document:
ssh $user#$server 'bash -s' <<EOF
CURRENT_TOMCAT_PROCESS=`ps -ef | grep java | grep $TOMCAT_USER | grep -v grep | awk '{print \$2}'`
echo "---> $CURRENT_TOMCAT_PROCESS"
EOF
(If I don't assign it to a variable and just print it out it works fine, but when I try to assign it to CURRENT_TOMCAT_PROCESS variable using backticks, it doesn't work.)
How can I make this work?
Thanks,
===============================================================================
I could make it work the following way. There are lots of escaping involved:
ssh $user#$server 'bash -s' <<EOF
CURRENT_TOMCAT_PROCESS="\`ps -ef | grep java | grep $TOMCAT_USER | grep -v grep | awk '{print \$2}'\`"
echo "---> \$CURRENT_TOMCAT_PROCESS"
EFO
I think it is reasonable to escape, because you want to transfer the '$' to remote site. You seems make a typo on your last result. I tried to type here again
TOMCATE_USER=foo
ssh $user#$server 'bash -s' <<EOF
CURRENT_TOMCAT_PROCESS="\`ps -ef | grep java | grep $TOMCAT_USER | grep -v grep | awk '{print \$2}'\`"
echo "---> \$CURRENT_TOMCAT_PROCESS"
EOF
Hi I have lots of logfiles with ^[[1m (as vim displays it) in them. I want to watch a logfile life via
tail -n 1000 -f logfile.log | grep <expression-for-escape-sequence>
and only get lines that have bold in them.
I am not sure which grep options I should use and have tried the following already:
tail -n 1000 -f logfile.log | grep "\033\0133\061\0155"
tail -n 1000 -f logfile.log | grep "\033\01331m"
tail -n 1000 -f logfile.log | grep "\033\[1m"
It does not work though... And yes there are bold lines in the last 1000 lines of logfile.log, testing with
echo -e "\033\01331mTest\033\01330m" | grep ...
same results... ;)
Appreciate any help!
Use single quotes with a dollar sign in front—as in $'...'—to have the shell convert the \033 escape sequence into an ESC character:
tail -n 1000 -f logfile.log | grep $'\033\[1m'
From man bash:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard.
This works (in a POSIX shell, not necessarily bash):
echo -e "\033\01331mTest\033\01330m" | grep "$(printf "\x1b\\[1m")"
say i have a directory with hi.txt and blah.txt and i execute the following command on a linux-ish command line
ls *.* | xargs -t -i{} echo {}
the output you will see is
echo blah.txt
blah.txt
echo hi.txt
hi.txt
i'd like to redirect the stderr output (say 'echo blah.txt' fails...), leaving only the output from the xargs -t command written to std out, but it looks as if it's stderr as well.
ls *.* | xargs -t -i{} echo {} 2> /dev/null
Is there a way to control it, to make it output to stdout?
Use:
ls | xargs -t -i{} echo {} 2>&1 >/dev/null
The 2>&1 sends the standard error from xargs to where standard output is currently going; the >/dev/null sends the original standard output to /dev/null. So, the net result is that standard output contains the echo commands, and /dev/null contains the file names. We can debate about spaces in file names and whether it would be easier to use a sed script to put 'echo' at the front of each line (with no -t option), or whether you could use:
ls | xargs -i{} echo echo {}
(Tested: Solaris 10, Korn Shell ; should work on other shells and Unix platforms.)
If you don't mind seeing the inner workings of the commands, I did manage to segregate the error output from xargs and the error output of the command executed.
al * zzz | xargs -t 2>/tmp/xargs.stderr -i{} ksh -c "ls -dl {} 2>&1"
The (non-standard) command al lists its arguments one per line:
for arg in "$#"; do echo "$arg"; done
The first redirection (2>/tmp/xargs.stderr) sends the error output from xargs to the file /tmp/xargs.stderr. The command executed is 'ksh -c "ls -dl {} 2>&1"', which uses the Korn shell to run ls -ld on the file name with any error output going to standard output.
The output in /tmp/xargs.stderr looks like:
ksh -c ls -dl x1 2>&1
ksh -c ls -dl x2 2>&1
ksh -c ls -dl xxx 2>&1
ksh -c ls -dl zzz 2>&1
I used 'ls -ld' in place of echo to ensure I was testing errors - the files x1, x2, and xxx existed, but zzz does not.
The output on standard output looked like:
-rw-r--r-- 1 jleffler rd 1020 May 9 13:05 x1
-rw-r--r-- 1 jleffler rd 1069 May 9 13:07 x2
-rw-r--r-- 1 jleffler rd 87 May 9 20:42 xxx
zzz: No such file or directory
When run without the command wrapped in 'ksh -c "..."', the I/O redirection was passed as an argument to the command ('ls -ld'), and it therefore reported that it could not find the file '2>&1'. That is, xargs did not itself use the shell to do the I/O redirection.
It would be possible to arrange for various other redirections, but the basic problem is that xargs makes no provision for separating its own error output from that of the commands it executes, so it is hard to do.
The other rather obvious option is to use xargs to write a shell script, and then have the shell execute it. This is the option I showed before:
ls | xargs -i{} echo echo {} >/tmp/new.script
You can then see the commands with:
cat /tmp/new.script
You can run the commands to discard the errors with:
sh /tmp/new.script 2>/dev/null
And, if you don't want to see the standard output from the commands either, append 1>&2 to the end of the command.
So I believe what you want is to have as stdout is
the stdout from the utility that xargs executes
the listing of commands generated by xargs -t
You want to ignore the stderr stream generated by the
executed utility.
Please correct me if I'm wrong.
First, let's create a better testing utility:
% cat myecho
#!/bin/sh
echo STDOUT $#
echo STDERR $# 1>&2
% chmod +x myecho
% ./myecho hello world
STDOUT hello world
STDERR hello world
% ./myecho hello world >/dev/null
STDERR hello world
% ./myecho hello world 2>/dev/null
STDOUT hello world
%
So now we have something that actually outputs to both stdout and stderr, so we
can be sure we're only getting what we want.
A tangential way to do this is not to use xargs, but rather, make. Echoing a command
and then doing it is kind of what make does. That's its bag.
% cat Makefile
all: $(shell ls *.*)
$(shell ls): .FORCE
./myecho $# 2>/dev/null
.FORCE:
% make
./myecho blah.txt 2>/dev/null
STDOUT blah.txt
./myecho hi.txt 2>/dev/null
STDOUT hi.txt
% make >/dev/null
%
If you're tied to using xargs, then you need to modify your utility that
xargs uses so it surpresses stderr. Then you can use the 2>&1 trick others
have mentioned to move the command listing generated by xargs -t from stderr
to stdout.
% cat myecho2
#!/bin/sh
./myecho $# 2>/dev/null
% chmod +x myecho2
% ./myecho2 hello world
STDOUT hello world
% ls *.* | xargs -t -i{} ./myecho2 {} 2>&1
./myecho blah.txt 2>/dev/null
STDOUT blah.txt
./myecho hi.txt 2>/dev/null
STDOUT hi.txt
% ls *.* | xargs -t -i{} ./myecho2 {} 2>&1 | tee >/dev/null
%
So this approach works, and collapses everything you want to stdout (leaving out what you don't want).
If you find yourself doing this a lot, you can write a general utility to surpress stderr:
% cat surpress_stderr
#!/bin/sh
$# 2>/dev/null
% ./surpress_stderr ./myecho hello world
STDOUT hello world
% ls *.* | xargs -t -i{} ./surpress_stderr ./myecho {} 2>&1
./surpress_stderr ./myecho blah.txt 2>/dev/null
STDOUT blah.txt
./surpress_stderr ./myecho hi.txt 2>/dev/null
STDOUT hi.txt
%
xargs -t echos the commands to be executed to stderr before executing them. If you want them to instead echo to stderr, you can pipe stderr to stdout with the 2>&1 construct:
ls *.* | xargs -t -i{} echo {} 2>&1
It looks like xargs -t goes to stderr, and there's not much you can do about it.
You could do:
ls | xargs -t -i{} echo "Foo: {}" >stderr.txt | tee stderr.txt
to display only the stderr data on your terminal as your command runs, and then grep through stderr.txt after to see if anything unexpected occurred, along the lines of grep -v Foo: stderr.txt
Also note that on Unix, ls *.* isn't how you display everything. If you want to see all the files, just run ls on its own.
As I understand your problem using GNU Parallel http://www.gnu.org/software/parallel/ would do the right thing:
ls *.* | parallel -v echo {} 2> /dev/null