I have a dataset like this:-
1. Sun Jul 5 00:04:01 EDT 2015
2. root 1 0 0.0 0.0 640 10372 Apr20 init [3]
3. root 2 1 0.0 0.0 0 0 Apr20 [migration/0]
And I need to filter out the timestamp from first line and add it with each message from second line.
Please help me to do this using logstash grok plugin.
Related
I am trying to save partition value from hdfs path to a file for different tables
Tried using sed to pull last 8 digits but since partition value changes (sometimes it may be YYYYMMDD sometimes YYYYMM) trying to see if we can grep data_dt from hdfs path instead of using sed.
Code used
hadoop fs -ls <hdfs_path> | sort -k6,7 | tail -2 > partition_info.txt
partitions=$(sed -e 's,.*\(.\{8\}\)$,\1,' partition_info.txt)
echo $partitions > partition_tables.txt
Desired Output example
20200531
202005
202004
20200601
Hadoop fs -ls output looks like this
drwxr-xr-x - kmedgel kmedgego 0 2020-05-30 09:33 /km/gold/edge_gold/otsd_cmpl/data_dt=20200530
drwxr-xr-x - kmedgel kmedgego 0 2020-05-31 09:33 /km/gold/edge_gold/otsd_cmpl/data_dt=20200531
drwxr-xr-x - kmedgel kmedgego 0 2020-06-01 09:34 /km/gold/edge_gold/otsd_cmpl/data_dt=20200601
drwxr-xr-x - kmedgel kmedgego 0 2020-06-02 09:34 /km/gold/edge_gold/otsd_cmpl/data_dt=20200602
drwxr-xr-x - kmedgel kmedgego 0 2020-06-03 09:55 /km/gold/edge_gold/otsd_cmpl/data_dt=20200603
Used while loop saying notNeed for the whole statement except for the field we are looking for i.e data_dt
Answer
while IFS="=" read -r notNeed data_dt
do
{
echo $data_dt
}
done
I am trying to check which process is taking up most disk i/o on my solaris server as it is behaving very much slow. Need help.
iostat -xtc
extended device statistics tty cpu
device r/s w/s kr/s kw/s wait actv svc_t %w %b tin tout us sy wt id
sd1 75.9 979.9 113.3 3524.9 0.0 5.4 5.1 0 69 0 53 1 2 0 97
nfs1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
It is quite hard, iotop script from Brendan Gregg can help you.
Here is the script:
http://www.brendangregg.com/DTrace/iotop
Here is the explanatory paper:
http://www.brendangregg.com/Solaris/paper_diskubyp1.pdf
If I look at the installation directory I see:
mhoggan#1USDMHOGGAN /cygdrive/c/Program Files (x86)/SoftErg/WinGDB
$ ls -l
total 19620
drwxrwx---+ 1 SYSTEM SYSTEM 0 Jan 26 11:00 1033
drwxrwx---+ 1 SYSTEM SYSTEM 0 Jan 26 11:00 Generators
-rwxr-x---+ 1 SYSTEM SYSTEM 583168 Jan 10 13:51 softergCrashReporter.exe
-rwxr-x---+ 1 SYSTEM SYSTEM 59392 Jan 10 13:52 softergProcessBreaker64.exe
-rwxr-x---+ 1 SYSTEM SYSTEM 546816 Jan 10 13:51 softergShell.exe
drwxrwx---+ 1 SYSTEM SYSTEM 0 Jan 26 11:00 Templates
drwxrwx---+ 1 SYSTEM SYSTEM 0 Jan 26 11:00 Visualizers
-rwxr-x---+ 1 SYSTEM SYSTEM 3116544 Jan 10 13:29 wingdb.chm
-rwxr-x---+ 1 SYSTEM SYSTEM 1363456 Jan 10 13:46 wingdbAdaLanguagePack.dll
-rwxr-x---+ 1 SYSTEM SYSTEM 30208 Jan 10 13:53 wingdbAutomationAPI.dll
-rwxr-x---+ 1 SYSTEM SYSTEM 11030016 Jan 10 13:50 wingdbEngine.dll
-rwxr-x---+ 1 SYSTEM SYSTEM 1495040 Jan 10 13:54 wingdbMakefilePack.dll
-rwxr-x---+ 1 SYSTEM SYSTEM 1845760 Jan 10 13:51 wingdbRemoteTerminalPack.dll
If I go to the start menu and type WinGDB I don't see any executable. However, http://www.wingdb.com/docs/pages/wg_launching.htm instructs me to use the UI I don't know how to launch. How do you launch WinGDB GUI from start menu or command line?
What is the difference between two commands, pls help to explain it.
ps -ef|sort +6|tail
oracle 55676 1 0 03:06:16 - 0:36 oracleprod (LOCAL=NO)
oracle 24876 1 0 02:52:56 - 0:40 oracleprod (LOCAL=NO)
oracle 41616 1 0 07:00:59 - 0:44 oracleprod (LOCAL=NO)
oracle 43460 1 0 02:45:05 - 0:53 oracleprod (LOCAL=NO)
oracle 25754 1 0 08:10:03 - 1:01 oracleprod (LOCAL=NO)
ps -ef|sort +5|tail
root 5440 2094 0 Nov 21 - 0:47 /usr/sbin/syslogd
root 9244 1 0 Nov 21 - 3:26 ./pcimapsvr.ip -D0
root 10782 1 0 Nov 21 - 4:41 ./pciconsvr.ip -D0
Why do both commands show different processes ? And if I keep on changing the value of 'sort +3' or reduce, the processes keeps on changing. What exactly is command all about ? Please help to explain.
You are sorting the wrong columns using both an obsolete syntax and a wrong method. No surprise random processes show up.
You'll get the top consumers that way:
ps -ef | sort -n -k8 | tail
-n means sort numerically
-k8 means sort the the eight column (cumulative execution time)
i created a GWT project in eclipse, and it's time to put some code back into source control. i'm not sure at this point which files are generated and can be left out of source control,
a. under war/myapp/gwt/... i see many, many files related to the standard GWT themes.
b. under war/myapp,
-rw-r--r-- 1 10102022 1602597546 1876 Jan 24 16:41 0182DE3CC529E42DA72BBD969A44841E.gwt.rpc
-rw-r--r-- 1 10102022 1602597546 1456 Jan 24 14:09 4F701266A6E52E1E409583EA9AEC39E2.gwt.rpc
-rw-r--r-- 1 10102022 1602597546 1876 Jan 25 08:38 D98FD8FE56B70659E9608109BCF8B3C1.gwt.rpc
-rw-r--r-- 1 10102022 1602597546 43 Dec 16 16:01 clear.cache.gif
drwxr-xr-x 6 10102022 1602597546 204 Jan 25 08:26 gwt
-rw-r--r-- 1 10102022 1602597546 11289 Dec 17 01:33 hosted.html
-rw-r--r-- 1 10102022 1602597546 5232 Jan 25 08:31 photodrop_web_gwt.nocache.js
normally i'd just rely on eclipse build > clean to get rid of the build time artifacts. however, i did that, and i still see WEB-INF/classes full of class files, so i know that clean isn't working.
"war/myapp" is by default GWT's output directory. So as long as you haven't saved any files there manually (you shouldn't), you can delete that directory completely.
As always, make a backup first...
I'm using source control for GWT + GAE, and this ignore file has been working great:
syntax: regexp
^war/myapp$
syntax: regexp
^war/WEB-INF/appengine-generated/datastore-indexes-auto\.xml$
syntax: regexp
^war/WEB-INF/appengine-generated/local_db\.bin