I'm trying to manipulate a string and the first step is to truncate the variable after a combination, so first I create the var:
SERVER_CONFIGURATION_FILE=$(curl some remove url)
And now my $SERVER_CONFIGURATION_FILE is something like that
client
dev tun
proto udp
remote 192.145.127.237 1194
resolv-retry infinite
remote-random
nobind
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
persist-key
persist-tun
ping 15
ping-restart 0
ping-timer-rem
reneg-sec 0
comp-lzo no
remote-cert-tls server
auth-user-pass
verb 3
pull
fast-io
cipher AES-256-CBC
auth SHA512
<ca>
-----BEGIN CERTIFICATE-----
MIIFCjCCAvKgAwIBAgIBATANBgkqhkiG9w0BAQ0FADA5MQswCQYDVQQGEwJQQTEQ
MA4GA1UEChMHTm9yZFZQTjEYMBYGA1UEAxMPTm9yZFZQTiBSb290IENBMB4XDTE2
MDEwMTAwMDAwMFoXDTM1MTIzMTIzNTk1OVowOTELMAkGA1UEBhMCUEExEDAOBgNV
BAoTB05vcmRWUE4xGDAWBgNVBAMTD05vcmRWUE4gUm9vdCBDQTCCAiIwDQYJKoZI
hvcNAQEBBQADggIPADCCAgoCggIBAMkr/BYhyo0F2upsIMXwC6QvkZps3NN2/eQF
kfQIS1gql0aejsKsEnmY0Kaon8uZCTXPsRH1gQNgg5D2gixdd1mJUvV3dE3y9FJr
XMoDkXdCGBodvKJyU6lcfEVF6/UxHcbBguZK9UtRHS9eJYm3rpL/5huQMCppX7kU
eQ8dpCwd3iKITqwd1ZudDqsWaU0vqzC2H55IyaZ/5/TnCk31Q1UP6BksbbuRcwOV
skEDsm6YoWDnn/IIzGOYnFJRzQH5jTz3j1QBvRIuQuBuvUkfhx1FEwhwZigrcxXu
MP+QgM54kezgziJUaZcOM2zF3lvrwMvXDMfNeIoJABv9ljw969xQ8czQCU5lMVmA
37ltv5Ec9U5hZuwk/9QO1Z+d/r6Jx0mlurS8gnCAKJgwa3kyZw6e4FZ8mYL4vpRR
hPdvRTWCMJkeB4yBHyhxUmTRgJHm6YR3D6hcFAc9cQcTEl/I60tMdz33G6m0O42s
Qt/+AR3YCY/RusWVBJB/qNS94EtNtj8iaebCQW1jHAhvGmFILVR9lzD0EzWKHkvy
WEjmUVRgCDd6Ne3eFRNS73gdv/C3l5boYySeu4exkEYVxVRn8DhCxs0MnkMHWFK6
MyzXCCn+JnWFDYPfDKHvpff/kLDobtPBf+Lbch5wQy9quY27xaj0XwLyjOltpiST
LWae/Q4vAgMBAAGjHTAbMAwGA1UdEwQFMAMBAf8wCwYDVR0PBAQDAgEGMA0GCSqG
SIb3DQEBDQUAA4ICAQC9fUL2sZPxIN2mD32VeNySTgZlCEdVmlq471o/bDMP4B8g
nQesFRtXY2ZCjs50Jm73B2LViL9qlREmI6vE5IC8IsRBJSV4ce1WYxyXro5rmVg/
k6a10rlsbK/eg//GHoJxDdXDOokLUSnxt7gk3QKpX6eCdh67p0PuWm/7WUJQxH2S
DxsT9vB/iZriTIEe/ILoOQF0Aqp7AgNCcLcLAmbxXQkXYCCSB35Vp06u+eTWjG0/
pyS5V14stGtw+fA0DJp5ZJV4eqJ5LqxMlYvEZ/qKTEdoCeaXv2QEmN6dVqjDoTAo
k0t5u4YRXzEVCfXAC3ocplNdtCA72wjFJcSbfif4BSC8bDACTXtnPC7nD0VndZLp
+RiNLeiENhk0oTC+UVdSc+n2nJOzkCK0vYu0Ads4JGIB7g8IB3z2t9ICmsWrgnhd
NdcOe15BincrGA8avQ1cWXsfIKEjbrnEuEk9b5jel6NfHtPKoHc9mDpRdNPISeVa
wDBM1mJChneHt59Nh8Gah74+TM1jBsw4fhJPvoc7Atcg740JErb904mZfkIEmojC
VPhBHVQ9LHBAdM8qFI2kRK0IynOmAZhexlP/aT/kpEsEPyaZQlnBn3An1CRz8h0S
PApL8PytggYKeQmRhl499+6jLxcZ2IegLfqq41dzIjwHwTMplg+1pKIOVojpWA==
-----END CERTIFICATE-----
</ca>
key-direction 1
<tls-auth>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----
e685bdaf659a25a200e2b9e39e51ff03
0fc72cf1ce07232bd8b2be5e6c670143
f51e937e670eee09d4f2ea5a6e4e6996
5db852c275351b86fc4ca892d78ae002
d6f70d029bd79c4d1c26cf14e9588033
cf639f8a74809f29f72b9d58f9b8f5fe
fc7938eade40e9fed6cb92184abb2cc1
0eb1a296df243b251df0643d53724cdb
5a92a1d6cb817804c4a9319b57d53be5
80815bcfcb2df55018cc83fc43bc7ff8
2d51f9b88364776ee9d12fc85cc7ea5b
9741c4f598c485316db066d52db4540e
212e1518a9bd4828219e24b20d88f598
a196c9de96012090e333519ae18d3509
9427e7b372d348d352dc4c85e18cd4b9
3f8a56ddb2e64eb67adfc9b337157ff4
-----END OpenVPN Static key V1-----
</tls-auth>
From here I want to delete all after <ca> tag
I tried various way, but every time the result is the same, I don't know what I do wrong
The last I tried is
echo "$SERVER_CONFIGURATION_FILE" | sed 's/<c[.\s\S\d\D\w\W]*//'
but remove only <ca and nothing more and the funny is that under regex101 this regex works
I feel pretty stupid but I can't find what I miss!
Any idea?
Please note that I'm under sh with busybox 1.30
Is this what you're trying to do?
$ awk '/<ca>/{exit} 1' file
client
dev tun
proto udp
remote 192.145.127.237 1194
resolv-retry infinite
remote-random
nobind
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
persist-key
persist-tun
ping 15
ping-restart 0
ping-timer-rem
reneg-sec 0
comp-lzo no
remote-cert-tls server
auth-user-pass
verb 3
pull
fast-io
cipher AES-256-CBC
auth SHA512
if your text is in a variable or coming from a pipe otherwise then just call it like this:
echo "$SERVER_CONFIGURATION_FILE" | awk '/<ca>/{exit} 1'
You don't need sed or awk for that. You can do it directly in the shell like this:
SERVER_CONFIGURATION_FILE="${SERVER_CONFIGURATION_FILE%%<ca>*}"
(Tested in dash; I assume busybox sh works the same way.)
${var%%pattern} expands to the contents of var, but with the longest suffix matching the glob pattern pattern stripped away.
The main problem with your sed attempt is that sed reads input one line at a time, so matches from one line wouldn't affect the other lines. (Also, your regex is slightly crazy and doesn't work like that in sed anyway.)
A working sed solution:
sed '/<ca>/{x;q}'
Or, perhaps more directly:
sed -n '/<ca>/q; p'
I.e. do not print by default; if line matches <ca>, quit; otherwise print and read the next line.
Related
Right now a lot of symbols are not readable via memcat/memccat or telnet (get) commands.
Is there anyway to convert the telnet/memcat/memccat output to UTF8 readable (except native clients)?
aim-server[~/www/next/src]$ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
get 20_dev_cache:c9bf76a9fa1e19ad92ab7195c64e47f8
VALUE 20_dev_cache:c9bf76a9fa1e19ad92ab7195c64e47f8 84 7914
�a:9:{s:14:"Licenses";a:2`�8:"QSL/TestModule"; 49:"JJJC-UNKK-MXZL-DAGW��� 42#20:"NNHJ2�6312-1 ";} R2��Tok �#�1#�#jC/TaxJar#E1:"�1
coreVersion#03#05:"majo`,3:"5.4#6#in� 8 B`build#
~35�� q#Wa ceKeyValue#$ �#!ad MURL#3 ehttps://xlocal/next/src/`'.php#(!jwav IN |1 L public_key#73 ssh-rsa A
ADAQAB CAQDWq4/EAVRilQslmKeA9A6y8f5i+oJSg0dfwaXUnbP6f4YwuJq8TOzr/q05HoQGS/biMExeu/YF2nu/Vo2RoBCV9rW5j+wPeIicgUQHarO+zoLTFM2+xdR7aG2MMEW4NO+4fdXgmRiqm9z6LJW4wpISZWiBqbJxsjaxeCvCVSsAtxlmP8Fg19lDB9OiMsll+GkMVAprH4xxhUiPz2hs2c6f9kf5kG2lf2lNS2bovobNFT2etds9so7HdbJ8GPVJf2wC0xUaRR7QNm0+HZ2SzfybYR11WQqmBrWVhZBVxhN343Knjh3jjW3jx/eOWWVeHezG8apD7YC58/ZVUhW3KdFbB3huUXNYmY8FWtYUkC+QEIjLkste57FvIMGb2Opwm4+inVf4hsVBs/a14bJgJGIn+7mLwvgsoAePunpRyv3tk0Wz+yM2KFy7xjHxt4wIIRUVRrp9D4ZHJbB9H7f0YcJRfCv/guIXywOJ1e1TDsjGfImbk77M+gV/gSHdnrlR2QRQYar7PcNWicVRKbDmbz9FERxhoSyd0Bhq+Jgp31oc2qxQ89#"6# 6:"auth�� KhgMQ+CBK9tAHrv00P4VIMj6tmBIENviEVAcMLbN+JbtnqXDrRpm RIbwxMurQeTqUAqgfva/nHoucBFAZETCX+LnsrIG2KvoVjP3XcZsrGQ== aim#example.comB�1#� installedAdC�224C�D)
�2#��G#�$A�� w��Y��0#)��i � �Ec��7:"enab �";b:0�;�/$C$�$�4#~utomatedShippingRefunds71LBS/SeentyOnePo#�
CDev/�`S3Imag�"� � �#+#Z
I've installed fail2ban 0.10.5-2.el7 from EPEL on CentOS 7.8. I'm trying to get it to work with systemd for processing a Tomcat log (also systemd).
In jail.local I added:
[guacamole]
enabled = true
port = http,https
backend = systemd
In filter.d/guacamole.conf:
[Definition]
failregex = Authentication attempt from <HOST> for user "[^"]*" failed\.$
ignoreregex =
journalmatch = _SYSTEMD_UNIT=tomcat.service + _COMM=java
If I run journalctl -u tomcat.service I see all the log lines. The ones I am interested in look like this:
May 18 13:58:26 myhost catalina.sh[42065]: 13:58:26.485 [http-nio-8080-exec-6] WARN o.a.g.r.auth.AuthenticationService - Authentication attempt from 1.2.3.4 for user "test" failed.
If I redirect journalctl -u tomcat.service to a log file, and process it with fail2ban-regex then it works exactly the way I want it to work, finding all the lines it needs.
% fail2ban-regex /tmp/j9 /etc/fail2ban/filter.d/guacamole.conf
Running tests
=============
Use failregex filter file : guacamole, basedir: /etc/fail2ban
Use log file : /tmp/j9
Use encoding : UTF-8
Results
=======
Failregex: 47 total
|- #) [# of hits] regular expression
| 1) [47] Authentication attempt from <HOST> for user "[^"]*" failed\.$
`-
Ignoreregex: 0 total
Date template hits:
|- [# of hits] date format
| [1] ExYear(?P<_sep>[-/.])Month(?P=_sep)Day(?:T| ?)24hour:Minute:Second(?:[.,]Microseconds)?(?:\s*Zone offset)?
| [570] {^LN-BEG}(?:DAY )?MON Day %k:Minute:Second(?:\.Microseconds)?(?: ExYear)?
`-
Lines: 571 lines, 0 ignored, 47 matched, 524 missed
[processed in 0.12 sec]
However, if fail2ban reads the journal directly then it does not work:
fail2ban-regex systemd-journal /etc/fail2ban/filter.d/guacamole.conf
It comes back right away, and processes 0 lines!
Running tests
=============
Use failregex filter file : guacamole, basedir: /etc/fail2ban
Use systemd journal
Use encoding : UTF-8
Use journal match : _SYSTEMD_UNIT=tomcat.service + _COMM=java
Results
=======
Failregex: 0 total
Ignoreregex: 0 total
Lines: 0 lines, 0 ignored, 0 matched, 0 missed
[processed in 0.00 sec]
I've tried to remove _COMM=java. It doesn't make a difference.
If I leave out the journal match line altogether, it at least processes all the lines from the journal, but does not find any matches (even though, as I mentioned, it processes a dump of the log file fine):
Running tests
=============
Use failregex filter file : guacamole, basedir: /etc/fail2ban
Use systemd journal
Use encoding : UTF-8
Results
=======
Failregex: 0 total
Ignoreregex: 0 total
Lines: 202271 lines, 0 ignored, 0 matched, 202271 missed
[processed in 34.54 sec]
Missed line(s): too many to print. Use --print-all-missed to print all 202271 lines
Either this is a bug, or I'm missing a small detail.
Thanks for any help you can provide.
To make sure the filter definition is properly initialised, it would be good to include the common definition. Your filter definition (/etc/fail2ban/filter.d/guacamole.conf) would therefore look like:
[INCLUDES]
before = common.conf
[Definition]
journalmatch = _SYSTEMD_UNIT='tomcat.service'
failregex = Authentication attempt from <HOST> for user "[^"]*" failed\.$
ignoreregex =
A small note given that your issue only occurs with systemd but not flat files, could you try the same pattern without $ at the end? Maybe there is an issue with the end of line when printed to the journal?
In your jail definition (/etc/fail2ban/jail.d/guacamole.conf), remember to define the ban time/find time/retries if they haven't already been defined in the default configuration:
[guacamole]
enabled = true
port = http,https
maxretry = 3
findtime = 1h
bantime = 1d
# "backend" specifies the backend used to get files modification.
# systemd: uses systemd python library to access the systemd journal.
# Specifying "logpath" is not valid for this backend.
# See "journalmatch" in the jails associated filter config
backend = systemd
Remember to restart the fail2ban service after doing such changes.
Is it possible to detect with a fail2ban regex, a MAC address ?
What I can do: Detect the source ip address
From the log file
Jan 18 11:15:14 server kernel: [DROP]IN=eth0 OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:00:00 SRC=1.2.3.4 DST=4.5.6.7 LEN=40 TOS=0x00 PREC=0x00 TTL=239
Fail2ban failregex
failregex = \[DROP\]IN=.* OUT= MAC=.* SRC=<HOST>*
Give 1.2.3.4 as a result. Then, any fail2ban action can use the <ip> variable.
What I would like to do: Detect the MAC address
Is it possible to do something like that?
failregex = \[DROP\]IN=.* OUT= MAC=<MAC> *
Shoud give 00:00:00:00:00:00:00:00:00:00:00:00:00:00 as a result. Then, any fail2ban action could use a<mac> variable for exemple.
Is someone have an answer, or any documentation? I didn't find informations about fail2ban regex or detection possibilities.
Not sure how to do it on Fail2Ban config and I don't have one to test right now. I did a little bash script to test this and maybe you can extract from here the regex.
#!/bin/bash
str="Jan 18 11:15:14 server kernel: [DROP]IN=eth0 OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:00:00 SRC=1.2.3.4 DST=4.5.6.7 LEN=40 TOS=0x00 PREC=0x00 TTL=239"
[[ ${str} =~ \[DROP\].*MAC=([0-9:]+) ]] && test_output="${BASH_REMATCH[1]}"
echo $test_output
It gives as output: 00:00:00:00:00:00:00:00:00:00:00:00:00:00
I'm capturing the mac inside a group (between parenthesis) and asigning it to a var. Maybe could be like your <MAC>.
I can't assure this because as I said, I don't have fail2ban right now to test, so is a "wild thing"... but I can venture to suggest something like this:
failregex = \[DROP\].*MAC=<MAC>
I hope it helps.
From fail2ban manual website:
As a convenience, you can use the predefined entity in your regexes. is an alias for (?:::f{4,6}:)?(?P\S+), which matches either a hostname or an IPv4 address (possibly embedded in an IPv6 address).
I understand that it is possible use custom regex, but how? There isn't lot of articles about that..
I am trying to gather statistics via kstat which I currently use dtrace to gather.
It is not count based information but new data every single time.
The minimum interval on kstat print is 1 second.However ,the data that I need changes several times within a second.Is there a way(API) to get data from kstat whenever the kstat is updated that doesn't use dtrace ?
Outside with dtrace, there is no way to get the statistics when they are updated however, the C libkstat API allows retrieving kstat statistics with an arbitrary sub second sampling rate.
There is also a perl api should you want to do it with scripting.
A very simple way to use it would be to create a customized kstat command (which happen to already be a perl script leveraging the kstat perl api) and modify it to use high resolution timers instead of the default one, e.g.:
$ sed '
s/sleep($interval);/Time::HiRes::usleep($interval*1000.);/
/use Sun::Solaris::Kstat/a\
use Time::HiRes;
' /usr/bin/kstat > /var/tmp/kstat_ms
$ chmod +x /var/tmp/kstat_ms
$ /var/tmp/kstat_ms -n lo0 500 3
module: lo instance: 0
name: lo0 class: net
crtime 19.559031813
ipackets 532
opackets 532
snaptime 4309.506435597
module: lo instance: 0
name: lo0 class: net
crtime 19.559031813
ipackets 534
opackets 534
snaptime 4310.008578348
module: lo instance: 0
name: lo0 class: net
crtime 19.559031813
ipackets 536
opackets 536
snaptime 4310.511617682
I've borrowed and written the following code to output the disconnect time. All works well but I'm curious as to how I could tighten/ shorten the code. If anyone feels like having some fun then I'd love to see what can be done. Be a learning lesson for me.
Cheers in advance.
Input:
ftp> !:--- FTP commands below here ---
ftp> lcd C:\Utilities\Performance_Testing\
\Utilities\Performance_Testing\: File not found
Verbose mode On .
ftp> verbose
binary
200 Switching to Binary mode.
ftp> put "test_file_5M.bin"
200 PORT command successful.
150 Ok to send data.
226 File receive OK.
ftp: 5242880 bytes sent in Seconds Kbytes/sec.
ftp> 44.81117.00disconnect
221 Goodbye.
ftp> bye
Code:
#Obtain UT external put value.
ut1intput=$(awk '
NR==70 {
for(i=1;i<=NF;i++) {
if($i=="ftp>") {
sub(/disconnect/, "", $(i+1));
print $(i+1)
}
}
}' filename.txt)
utintputvalue=`echo $ut1intput | awk -F. '{print $2"."$3}'| sed 's/^..//'`
Output:
UT external put value is 1220.98
Given your posted sample input and the desired output you said you want in your comment:
$ sed -n 's/^.*\(......\)disconnect/UT external put value is \1/p' file
UT external put value is 117.00