I want to extract this data http://code.google.com/p/memcached-session-manager/wiki/JMXStatistics via jmx but using only command line.
This is because is the only way to enter to the server I got.
Any pointer would help
thanks in advance
I did up this quick-n-dirty solution after banging my head against the JMX protocol's inability to tunnel out over ssh.
(1) found some jmxterm thing, (2) stuff some simple command into it (runs an MBean operation) (after examining what JConsole did when attached to local tomcat..) and (3) format it into something readable.
#!/bin/bash
# wget http://downloads.sourceforge.net/cyclops-group/jmxterm-1.0-alpha-4-uber.jar
JARFILE="jmxterm-1.0-alpha-4-uber.jar"
JMXHOST=127.0.0.1
JMXPORT=9003
function die() { echo $1 ; exit 1 ; }
function checkexe() { [ -x `which $1` ] || die "cant find executable program $1" ; }
PROGS="java grep cat sed tr xsltproc xmllint"
for P in $PROGS ; do checkexe $P ; done
[ -f $JARFILE ] || die "can't see jar file $JARFILE"
## jmxterm commands to get thread stack dump
cat >/tmp/myscript.jmx<<EOF
domain java.lang
bean java.lang:type=Threading
run dumpAllThreads 1 1
quit
EOF
## get the stack traces
java -jar $JARFILE -l $JMXHOST:$JMXPORT -i /tmp/myscript.jmx > /tmp/allthreads.txt
grep "threadId" /tmp/allthreads.txt || die "stack trace get seemed to fail ?!"
## turn them into xml
cat /tmp/allthreads.txt | \
sed \
-e "s|\[ |<array>|g" -e "s| \]|</array>|g" \
-e "s|{|<obj>|g" -e "s|}|</obj>|g" \
-e "s|\([^ <>]*\) = \([^ <>]*\);|<\1>\2</\1>|g" \
-e "s|\([^ ]*\) = \([^;]*\) *;|<\1>\2</\1>|g" | \
tr '\r\n\t' ' ' | tr -s ' ' | \
sed \
-e "s|\([^ =]*\) = \([^;=]*\);|<\1>\2</\1>|g" \
-e "s|\([^ ]*\) = \([^;]*\);|<\1>\2</\1>|g"\
-e "s| *, *||g" > /tmp/allthreads.xml
## xsl to convert xml-ified stack traces to simpler format
cat >/tmp/jmxterm_threads.xslt<<EOF
<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="/">
<jmx>
<xsl:for-each select="/array/obj">
<xsl:sort select="threadId" data-type="number"/>
<xsl:apply-templates select="." mode="thread"/>
</xsl:for-each>
</jmx>
</xsl:template>
<xsl:template match="obj" mode="thread">
<thread id="{threadId/text()}">
<xsl:copy-of select="threadName"/>
<xsl:copy-of select="threadState"/>
<xsl:copy-of select="suspended"/>
<xsl:copy-of select="inNative"/>
<xsl:copy-of select="waitedCount"/>
<xsl:copy-of select="blockedCount"/>
<xsl:apply-templates select="stackTrace"/>
</thread>
</xsl:template>
<xsl:template match="stackTrace">
<stackTrace>
<xsl:for-each select="./array/obj">
<stackfn class="{className}" method="{methodName}" file="{fileName}" line="{lineNumber}"/>
</xsl:for-each>
</stackTrace>
</xsl:template>
</xsl:stylesheet>
EOF
## simplify xml
xsltproc /tmp/jmxterm_threads.xslt /tmp/allthreads.xml | xmllint --format - > /tmp/allthreads_simple.xml
cat /tmp/allthreads_simple.xml
For CLI access I recommend jmx4perl and the readline based JMX shell j4psh. They require a Jolokia agent on the the server side, though. These agents are available in several flavours (war, osgi, mule, jdk6). Since memcached-session-manager runs within a Tomcat, you probably should use the WAR agent and simply drop it into the webapps/ directory.
Related
i have a file.txt with contents
2021-12-03;12.20.31;13;00000.00;00000.00;NO LINK
2021-12-03;12.33.31;15;00199.94;00000.00;Status OK
2021-12-03;12.35.33; 2;01962.33;00015.48;;Status OK
2021-12-03;13.05.31;13;00000.00;00000.00;NO LINK
so what command to output like below
2021-12-03;12:20:31;13;00000.00;00000.00;NO LINK
2021-12-03;12:33:31;15;00199.94;00000.00;Status OK
2021-12-03;12:35:33; 2;01962.33;00015.48;Status OK
2021-12-03;13:05:31;13;00000.00;00000.00;NO LINK
note.
cut -b 12-19 file.txt (is time)
Thanks for your help.
Rido
I assumed that the lines you want to modify are contained in a file (which I called filea.txt). The script should solve your problem.
Contents of the file 'filea.txt':
$> cat filea.txt
2021-12-03;12.20.31;13;00000.00;00000.00;NO LINK
2021-12-03;12.33.31;15;00199.94;00000.00;Status OK
2021-12-03;12.35.33; 2;01962.33;00015.48;;Status OK
2021-12-03;13.05.31;13;00000.00;00000.00;NO LINK
Script File:
$> cat refrm
#!/usr/bin/bash
in_file="filea.txt"
while read -r line || [ -n "$line" ];
do
line=$(echo "${line}" | sed -E 's/;{2,}/;/g')
IFS=$'\n'
line=$(echo ${line} | sed 's/;/\n/g')
arr=($(IFS='\n' ; echo "${line}"))
for ((n=0; n < ${#arr[*]}; n++))
do
if [[ ${arr[n]} =~ ^[0-9]{2}\.[0-9]{2}\.[0-9]{2} ]];
then
arr[n]=`echo ${arr[n]} | sed 's/\./:/g'`
fi
done
nline=$(IFS=";" ; echo "${arr[*]}")
echo "${nline}"
done < ${in_file}
Output:
$> refrm
2021-12-03;12:20:31;13;00000.00;00000.00;NO LINK
2021-12-03;12:33:31;15;00199.94;00000.00;Status OK
2021-12-03;12:35:33; 2;01962.33;00015.48;Status OK
2021-12-03;13:05:31;13;00000.00;00000.00;NO LINK
I have some federal pdf forms with filled data init. Lets say for example i765 and I have the data of this form available in a text format, with duly filled in details. How can I extract the data from this form with minimum parsing. Lets say how can write a script that identifies "difference" , which in itself is nothing but the filled information.
For eg: if a line contains..
SSN: (Whitespace) and the actual filled in form has SSN: ABC!##456
so the filled in information is nothing but ABC!##456 which just a difference between the strings . Is there a known approach that i can follow. Any pointers are much appreciated.
If we are talking about Linux Tools then you could try various solutions , like:
$ join -t"=" -a1 -o 0,2.2 <(sort emptyform) <(sort filledform) # "=" is used as delimiter
Or even awk without sorting requirements:
$ awk 'BEGIN{FS=OFS="="}NR==FNR{a[$1]=$2;next}{if ($1 in a) {print;delete a[$1]}} \
END{print "\n Missing fields:";for (i in a) print i,a[i]}' empty filled
Testing:
cat <<EOF >empty
Name=""
Surname=""
Age=""
Address=""
Kids=""
Married=""
EOF
cat <<EOF >filled
Name="George"
Surname="Vasiliou"
Age="42"
Address="Europe"
EOF
join -t"=" -a1 -o 0,2.2 <(sort empty) <(sort filled)
#Output:
Address="Europe"
Age="42"
Kids=
Married=
Name="George"
Surname="Vasiliou"
awk output
awk 'BEGIN{FS=OFS="="}NR==FNR{a[$1]=$2;next}{if ($1 in a) {print;delete a[$1]}} \
END{print "\nnot completed fields:";for (i in a) print i,a[i]}' empty filled
Name="George"
Surname="Vasiliou"
Age="42"
Address="Europe"
not completed fields:
Married=""
Kids=""
Especially in awk if you remove the print from {if ($1 in a) {print;delete a[$1]}} the END section will print out for you only the missing fields.
Another alternative with a nice visual interface is with diff utility:
$ diff -y <(sort empty) <(sort filled)
Address="" | Address="Europe"
Age="" | Age="42"
Kids="" | Name="George"
Married="" | Surname="Vasiliou"
Name="" <
Surname="" <
Often I need to analyze large logs in console.
I use the following command to colorize important keywords:
echo "string1\nerror\nsuccess\nstring2\nfail" | perl -p -e 's/(success)/\e[1;32;10m$&\e[0m/g;' -e 's/(error|fail)/\e[0;31;10m$&\e[0m/g'
It will colorize "success" with green, and error messages with red and keeps others lines unchanged (as they contain some useful info).
But in some cases I need to colorize values after some marker, but not marker itself, i.e. in these lines
Marker1: value1
Marker2: value2
need to highlight only value1 and value2 by known markers.
I'm looking for a way to modify my current oneliner to add this function
Also I tried the following solution, which I like less
#!/bin/bash
default=$(tput op)
red=$(tput setaf 1 || tput AF 1)
green=$(tput setaf 2 || tput AF 2)
sed -u -r "s/(Marker1: )(.+)$/\1${red}\2${default}/
s/(Marker2: )(.+)$/\1${green}\2${default}/" "${#}"`
But it has some problem with buffering, so it's ok for some constant file, but log which is continuosly running is not displayed at all
UPDATE:
Found a solution with help of some perl guru.
echo -e "string1\nerror\nsuccess\nstring2\nfail\nMaker1: value1\nMaker2: value2" | \
perl -p \
-e 's/(success)/\e[32m$&\e[0m/g;' \
-e 's/(error|fail)/\e[31m$&\e[0m/g;' \
-e 's/(Maker1:) (.*)/$1 \e[36m$2\e[0m/m;' \
-e 's/(Maker2:) (.*)/$1 \e[01;34m$2\e[0m/m;'
echo -e "string1\nerror\nsuccess\nstring2\nfail\nMaker1: value1\nMaker2: value2" | \
perl -p \
-e 's/(success)/\e[32m$&\e[0m/g;' \
-e 's/(error|fail)/\e[31m$&\e[0m/g;' \
-e 's/(Maker1:) (.*)/$1 \e[36m$2\e[0m/m;' \
-e 's/(Maker2:) (.*)/$1 \e[01;34m$2\e[0m/m;'
#!/bin/bash
default=$(tput op)
red=$(tput setaf 1 || tput AF 1)
green=$(tput setaf 2 || tput AF 2)
#default='e[0m'
#red='e[0;31;10m'
#green='e[1;32;10m'
# automaticaly use passed argument file if any or stdin if not
sed -u -r \
"/success/ s//${green}&${default}/
/error|fail/ s//${red}&${default}/
/^Marker1:/ {s//\1${red}/;s/$/${default}/;}
/^Marker2:/ {s//\1${green}/;s/$/${default}/;}" \
$( [ ${##} -gt 0 ] && echo ${#} )
For a one line:
remove other line thans sed one
replace newline in sed by ;
use directly the terminal code in place of variable
remove the last line if you pipe or use specific file instead
I have two web pages
Page 1:
<data>
<item>
<name>Item 1</name>
<url>http://someUrl.html</url>
</item>
</data>
Page 2: http://someUrl.html
<data>
<info>Info 1</info>
<info>Info 2</info>
<info>Info 3</info>
</data>
I want to crawl page 1 and follow all the links there and generate the following output
Item 1, Info 1
Item 1, Info 2
Item 1, Info 3
...
How can i achieve this using Xidel?
I recently found Xidel, so I'm no expert, but in my opinion it's an extremely powerful swiss-knife commandline scrape tool, that should be known by many more people.
Now, to answer your question I think the following (using html-templates) does exactly what you want:
xidel -q page1.html --extract-exclude=name -e "<name>{name:=text()}</name>*" -f "<url>{link:=text()}</url>*" -e "<info>{string-join(($name, text()), ', ')}</info>*" --hide-variable-names
Or, even shorter with CSS selectors:
xidel -q page1.html --extract-exclude=name -e "name:=css('name')" -f "link:=css('url')" -e "css('info')/string-join(($name,.),', ')" --hide-variable-names
Or, shortest with XPath:
xidel -q page1.html --extract-exclude=name -e name:=//name -f link:=//url -e "//info/string-join(($name,.),', ')" --hide-variable-names
The shortest line possible (but not in CSV format) would be:
xidel -q page1.html -e //name,//info -f //url
The above commands are for Windows, so make sure to swap the quotes <-> double quotes when on mac/ux!
If you need explanation for the different parts in the lines, just ask... :-) Cheers!
You're talking about "all the links there", so instead of what you posted I'm going to assume as input:
<data>
<item>
<name>Item 1</name>
<url>http://someUrl1.html</url>
</item>
<item>
<name>Item 2</name>
<url>http://someUrl2.html</url>
</item>
<item>
<name>Item 3</name>
<url>http://someUrl3.html</url>
</item>
</data>
Linux:
xidel -s input.html -e 'for $item in //item for $info in doc($item/url)//info return $item/name||", "||$info'
#or
xidel -s input.html -e '
for $item in //item
for $info in doc($item/url)//info
return
$item/name||", "||$info
'
Windows:
xidel -s input.html -e "for $item in //item for $info in doc($item/url)//info return $item/name||', '||$info"
#or
xidel -s input.html -e ^"^
for $item in //item^
for $info in doc($item/url)//info^
return^
$item/name^|^|', '^|^|$info^
"
The 1st for-loop iterates over every <item>-node. The 2nd for-loop opens the url and iterates over every <info>-node. And the return clause is a simple string concatenation.
The output in this case:
Item 1, Info 1
Item 1, Info 2
Item 1, Info 3
Item 2, Info 4
Item 2, Info 5
Item 2, Info 6
Item 3, Info 7
Item 3, Info 8
Item 3, Info 9
I'm trying to port a GNU sed command to BSD sed (in OSX). The command is:
cat -- "$1" | sed -n -e "\${/^#/H;x;/${tapPrintTapOutputSedPattern}/p;}" \
-e "/${tapPrintTapOutputSedPattern}/{x;/${tapPrintTapOutputSedPattern}/p;b;}" \
-e "/^#/{H;b;}" \
-e "x;/${tapPrintTapOutputSedPattern}/p" \
-e "/^Bail out!/q"
It works on GNU sed, but BSD sed gives this error:
sed: 2: "/^Bail out!/q
": unexpected EOF (pending }'s)
This is the command after the variable expansions, in case it's relevant:
cat -- "test021.tap" | sed -n \
-e "\${/^#/H;x;/^not ok\|^ok \([0-9]\+ \)\?# [tT][oO][dD][oO]\|^Bail out!/p;}" \
-e "/^not ok\|^ok \([0-9]\+ \)\?# [tT][oO][dD][oO]\|^Bail out!/{x;/^not ok\|^ok \([0-9]\+ \)\?# [tT][oO][dD][oO]\|^Bail out!/p;b;}" \
-e "/^#/{H;b;}" \
-e "x;/^not ok\|^ok \([0-9]\+ \)\?# [tT][oO][dD][oO]\|^Bail out!/p" \
-e "/^Bail out!/q"
Any ideas about why/how to fix it?
Cheers!
Try using newlines instead of a semicolons, at least before the branch commands (b) in the statements. See if this works:
sed -n "
\${
/^#/H
x
/${tapPrintTapOutputSedPattern}/p
}
/${tapPrintTapOutputSedPattern}/{
x
/${tapPrintTapOutputSedPattern}/p
b
}
/^#/{
H
b
}
x
/${tapPrintTapOutputSedPattern}/p
/^Bail out!/q
" "$1"