How can I cleanly list all currently banned IPs on fail2ban? with one IP per line?
Below is the list I get when I execute :
fail2ban-client status sshd
on my Ubunutu 18.04 server. I know the bare minimum when it comes to linux and servers. I don't even know how to get the version of fail2ban I am using, and yes, I googled it a lot.
Is there a way to get one IP per line?
fail2ban-client status sshd Status for the jail: sshd |- Filter | |-
Currently failed: 3 | |- Total failed: 1266 | - File list: /var/log/auth.log <br>- Actions |- Currently banned: 118 |-
Total banned: 345 `- Banned IP list: 61.177.173.10 49.234.214.215
152.231.140.150 180.76.247.65 43.129.26.69 196.206.231.249 43.153.27.174 43.157.1.29 180.167.207.234 43.252.62.60 43.154.88.243 200.7.168.217 64.227.187.235 186.226.37.45 183.98.146.157 182.93.7.194 143.244.163.108 122.194.229.62 112.85.42.74 61.177.173.36 177.91.52.133 103.124.94.169 122.194.229.54 61.177.172.59 61.177.173.16 61.177.173.40 141.98.11.23 61.177.172.108 61.177.173.37 112.85.42.53 122.194.229.40 189.202.214.250 112.85.42.87 49.248.153.6 143.110.243.129 43.129.24.85 112.85.42.151 134.19.146.45 61.177.172.76 112.85.42.229 61.177.172.89 61.177.172.91 61.177.172.61 195.29.51.135 45.67.34.253 20.205.39.78 194.165.16.5 61.177.172.124 160.16.209.119 61.177.173.35 177.19.138.138 103.63.108.25 61.177.172.60 43.154.205.162 138.219.192.207 222.82.211.78 61.177.172.160 112.85.42.15 165.232.189.7 61.177.173.39 147.182.179.237 207.154.211.157 120.92.11.9 209.97.162.0 45.234.188.11 167.71.220.220 104.248.140.201 90.189.182.30 68.183.236.92 103.86.49.28 61.177.172.98 43.154.137.134 207.154.228.201 61.177.173.42 43.154.2.84 45.135.232.155 139.59.64.41 43.154.58.123 218.92.0.221 88.215.177.224 193.169.255.38 51.140.185.84 46.101.137.28 122.194.229.92 139.59.187.229 5.180.31.119 112.85.42.73 185.59.139.99 122.194.229.65 1.15.251.60 46.19.139.42 165.22.198.10 61.177.173.44 193.168.195.23 61.177.172.174 89.232.192.40 61.177.173.41 82.196.4.168 61.177.172.87 64.227.108.47 159.89.55.150 117.122.212.78 159.223.148.195 206.217.131.233 138.197.222.211 121.225.234.182 164.92.106.112 185.220.102.251 36.110.228.254 45.153.160.132 171.25.193.20 113.31.117.79 51.143.96.123 159.89.29.240 172.247.15.76 159.223.229.50 14.161.50.104 68.183.125.190
p.s. I don't really mind listing the IPs here. My server is not a public server, so anyone being banned is 99% a bot, or something else up to no good.
Use a regex:
fail2ban-client status sshd | grep -o '[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}'
Related
I have deployed redis for two different services (redis HA cluster). At the moment there is no load on these services. The services and therefore redis are isolated from each other - they are in different namespaces. But for some reason, RAM consumption in one of the services has increased 3.5 times and continues to grow. The settings of both services and redis are absolutely indetectable.
In redis, in which RAM consumption is growing, I checked the availability of the redis port with an ordinary tcp scan. It's been few days since then but my RAM consumption keeps going up.
How can this be avoided?
1 image
2 image
NAME CPU(cores) MEMORY(bytes)
gateway-dp-69c7cdb95f-l9lx5 2m 71Mi
redis-haproxy-deployment-7669cfc845-82d66 2m 5Mi
redis-haproxy-deployment-7669cfc845-n8fdv 1m 4Mi
redis-sentinel-node-0 23m 20Mi
redis-sentinel-node-1 24m 15Mi
redis-sentinel-node-2 23m 15Mi
# Server
redis_version:7.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:4178210212378ca8
redis_mode:standalone
os:Linux 5.4.0-124-generic x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:1
process_supervised:no
run_id:b559cd4c9089ef8ca2eb3f358be010442e406df0
tcp_port:6379
server_time_usec:1669742636280140
uptime_in_seconds:272790
uptime_in_days:3
hz:10
configured_hz:10
lru_clock:8798252
executable:/redis-server
config_file:
io_threads_active:0
# Clients
connected_clients:12
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:20567
client_recent_max_output_buffer:20504
blocked_clients:1
tracking_clients:0
clients_in_timeout_table:1
# Memory
used_memory:2909400
used_memory_human:2.77M
used_memory_rss:10985472
used_memory_rss_human:10.48M
used_memory_peak:3087040
used_memory_peak_human:2.94M
used_memory_peak_perc:94.25%
used_memory_overhead:2115983
used_memory_startup:863144
used_memory_dataset:793417
used_memory_dataset_perc:38.77%
allocator_allocated:3194064
allocator_active:3956736
allocator_resident:6594560
total_system_memory:4122685440
total_system_memory_human:3.84G
used_memory_lua:39936
used_memory_vm_eval:39936
used_memory_lua_human:39.00K
used_memory_scripts_eval:312
number_of_cached_scripts:1
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:72704
used_memory_vm_total_human:71.00K
used_memory_functions:184
used_memory_scripts:496
used_memory_scripts_human:496B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.24
allocator_frag_bytes:762672
allocator_rss_ratio:1.67
allocator_rss_bytes:2637824
rss_overhead_ratio:1.67
rss_overhead_bytes:4390912
mem_fragmentation_ratio:3.81
mem_fragmentation_bytes:8098368
mem_not_counted_for_evict:13728
mem_replication_backlog:1048592
mem_total_replication_buffers:1066208
mem_clients_slaves:17632
mem_clients_normal:185679
mem_cluster_links:0
mem_aof_buffer:256
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:3783
rdb_bgsave_in_progress:0
rdb_last_save_time:1669469846
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:380928
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
aof_current_size:256321
aof_base_size:0
aof_pending_rewrite:0
aof_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0
# Stats
total_connections_received:705629
total_commands_processed:4665151
instantaneous_ops_per_sec:19
total_net_input_bytes:263925380
total_net_output_bytes:2451972513
total_net_repl_input_bytes:0
total_net_repl_output_bytes:357887569
instantaneous_input_kbps:1.04
instantaneous_output_kbps:14.30
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:1.28
rejected_connections:0
sync_full:2
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:4126
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:1883
keyspace_misses:1943
pubsub_channels:1
pubsub_patterns:2
pubsubshard_channels:0
latest_fork_usec:460
total_forks:2
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:4
dump_payload_sanitizations:0
total_reads_processed:4765884
total_writes_processed:6409329
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:81525
reply_buffer_expands:81478
# Replication
role:master
connected_slaves:2
slave0:ip=redis-sentinel-node-1.redis-sentinel-headless.gateway.svc.cluster.local,port=6379,state=online,offset=178946913,lag=1
slave1:ip=redis-sentinel-node-2.redis-sentinel-headless.gateway.svc.cluster.local,port=6379,state=online,offset=178946913,lag=1
master_failover_state:no-failover
master_replid:5b0ec844fea2a86eb46bf9aad6af8d4dda19959c
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:178948225
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:177889321
repl_backlog_histlen:1058905
# CPU
used_cpu_sys:464.200618
used_cpu_user:525.614928
used_cpu_sys_children:0.004708
used_cpu_user_children:0.018015
used_cpu_sys_main_thread:463.688291
used_cpu_user_main_thread:525.247753
# Modules
# Errorstats
errorstat_ERR:count=3
errorstat_NOSCRIPT:count=1
# Cluster
cluster_enabled:0
# Keyspace
db0:keys=3,expires=0,avg_ttl=0
I don't know what to do.
Since the documention of the cardano developer portal is outdated (old testnet) i researched and now know about the new testnets documented here https://book.world.dev.cardano.org/environments.html and on Github.
I followed the tutorial on the documentation https://developers.cardano.org/docs/get-started/running-cardano about running the node for the testnet, but instead of using the deprecated files i used the new enviroment files for the pre production testnet.
Now i cant sync the network.
I get the following info over and over again:
TrConnectError (Just 127.0.0.1:1337) 3.72.231.105:30000 Network.Socket.connect: <socket: 29>: invalid argument (Invalid argument)
TrConnectionManagerCounters (ConnectionManagerCounters {fullDuplexConns = 0, duplexConns = 0, unidirectionalConns = 0, inboundConns = 0, outboundConns = 0})
TracePromoteColdFailed 50 0 3.72.231.105:30000 160.633570297628s Network.Socket.connect: <socket: 29>: invalid argument (Invalid argument)
TraceGovernorWakeup
TracePublicRootsRequest 100 1
TracePublicRootRelayAccessPoint [RelayAccessDomain "preprod-node.world.dev.cardano.org" 30000]
TracePublicRootResult "preprod-node.world.dev.cardano.org" [(3.72.231.105,60)]
TracePublicRootsResults (fromList []) 9 512s
console info from the node, same es in the text
I can get the sync status which looks like the first time running like this:
{
"block": 0,
"epoch": 0,
"era": "Byron",
"hash": "9ad7ff320c9cf74e0f5ee78d22a85ce42bb0a487d0506bf60cfb5a91ea4497d2",
"slot": 0,
"syncProgress": "0.01"
}
I tried it with the devnet, and preview testnet too - didn't work either.
Cardano node version (currently the newest):
cardano-node 1.35.3 - linux-x86_64 - ghc-8.10
git rev ea6d78c775d0f70dde979b52de022db749a2cc32
Does anyone know why this happens and how to fix it?
Run the node with "--host-addr 0.0.0.0". For example:
cardano-node run --config $HOME/cardano/preprod/config.json
--database-path $HOME/cardano/db
--socket-path $HOME/cardano/db/node.socket
--host-addr 0.0.0.0
--port 1337
--topology $HOME/cardano/preprod/topology.json
The following RE matches "http://my.domain/video.mp4" successfully but cannot match "http://my.domain/abc/video.mp4".
location ~ "^.+(mp4|mkv|m4a)$" {
root /home/user/Videos;
}
The log of Nginx reads
[05/May/2021:12:29:25 +0800] "GET /video.mp4 HTTP/1.1" status: 206, body_bytes: 1729881
[05/May/2021:12:29:46 +0800] "GET /abc/video.mp4 HTTP/1.1" status: 404, body_bytes: 555
This is wired. Actually, I want URLs under "/service1/" to be mapped to user1's directory and URLs under "/service2" to be mapped to user2's. So I write
location ~ "^/service1/.+(mp4|mkv|m4a)$" {
root /home/user1/Videos;
}
location ~ "^/service2/.+(mp4|mkv|m4a)$" {
root /home/user2/Videos;
}
And as the first example, this config cannot match anything.
I searched a lot on google. No answer can explain this. I want to get it to work. Thanks!
I know where the problem is. In my second config, if I require "/service1/abcd.mp4", Nginx will try to locate it at "/home/user1/Videos/service1/abcd.mp4" but actually the file is at "/home/user1/Videos/abcd.mp4". Theoretically, I can bypass it by rewrite
location ~ "^/service1/.+(mp4|mkv|m4a)$" {
rewrite "^/service1/(.+(mp4|mkv|m4a))$" "$1";
root /home/user1/Videos;
}
location ~ "^/service2/.+(mp4|mkv|m4a)$" {
rewrite "^/service2/(.+(mp4|mkv|m4a))$" "$1";
root /home/user2/Videos;
}
But this is not working. I am getting crazy.
Thanks a lot for your time in reading this. I would really appreciate if you can show me some lights on how to achieve this.
the idea to build a PS script to revoke\release few license based on few conditions from an command line output
sample license status can be fetched through a command line below
--------------------------------------------------------------------
Trust Flags = FULLY TRUSTED
Fulfillment Type: TRIAL
Status: ENABLED
Fulfillment ID: LOCAL_TRIAL_FID_586
Entitlement ID: SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PCR8
Product ID: NAME=Tableau Desktop TS;VERSION=4.0
Suite ID: NONE
Expiration date: 23-oct-2020
Feature line(s):
INCREMENT TableauDesktop tableau 2021.1108 permanent 1 \
VENDOR_STRING=EntitlementID=;EDITION=Professional;CAP=REG:STANDARD,WARN:14,NOGRACE;DC_STD=default;DC_CAP=;TRIALVER=2019.1;FulfillmentID=;ActivationID=;OEMNAME=;GRACE=;MAP_STD=default;MAP_CAP=;OFFLINE= \
ISSUER="Tableau Software" ISSUED=9-nov-2018 START=8-nov-2018 \
TS_OK SIGN="042D 811B 5D78 81EA E6E7 28BD 607A F3D3 028E DC82 \
E310 A6BC C1D5 0913 5CBC 18B5 8671 7C7D C0B7 3C46 D1E7 A16C \
6C84 3694 BB4C DB73 4B59 C419 D820 58E0"
--------------------------------------------------------------------
Trust Flags = FULLY TRUSTED
Fulfillment Type: TRIAL
Status: ENABLED
Fulfillment ID: LOCAL_TRIAL_FID_590
Entitlement ID: SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PTR2
Product ID: NAME=Tableau Desktop TS;VERSION=4.0
Suite ID: NONE
Expiration date: 23-oct-2020
Feature line(s):
INCREMENT TableauDesktop tableau 2021.1108 permanent 1 \
VENDOR_STRING=EntitlementID=;EDITION=Professional;CAP=REG:STANDARD,WARN:14,NOGRACE;DC_STD=default;DC_CAP=;TRIALVER=2019.1;FulfillmentID=;ActivationID=;OEMNAME=;GRACE=;MAP_STD=default;MAP_CAP=;OFFLINE= \
ISSUER="Tableau Software" ISSUED=9-nov-2018 START=8-nov-2018 \
TS_OK SIGN="042D 811B 5D78 81EA E6E7 28BD 607A F3D3 028E DC82 \
E310 A6BC C1D5 0913 5CBC 18B5 8671 7C7D C0B7 3C46 D1E7 A16C \
6C84 3694 BB4C DB73 4B59 C419 D820 58E0"
--------------------------------------------------------------------
we need to parse the "Trust Flags", "status" and "Entitlement ID" from both the entries in to an hash-table so that we can perform logical operations.
your directions will be much helpful!! My sincere thanks again
You can use a switch statement with the -Regex switch to perform regular expression-based line-by-line processing:
# Initialize the (ordered) output hash table.
$hashTable = [ordered] #{}
# Process the input file line by line and populate the hash table.
switch -file input.txt -regex {
'^(Trust Flags|status|Entitlement ID):? +(?:= +)?(.*)' {
$hashTable[$Matches.1] = $Matches.2
}
}
# Output the resulting hash tabe.
$hashTable
The above yields:
Name Value
---- -----
Trust Flags FULLY TRUSTED
Status ENABLED
Entitlement ID SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PTR2
I would first check if the command utility offers you a way to control the output. Many command line utilities do provide options for creating structured output such as csv or xml. If you are indeed limited to just text, then this is a perfect scenario to utilize ConvertFrom-String
Now depending on how much the data varies, you may need to adjust the "sample" data used in the template. I've found the key is to provide just enough training data and not too much. See the example below.
First create a template. I'm not sure what other possible values you may face but I did change the second example in the template just to provide a wider net. You could adjust these to actual possible values for better results.
$template = #'
Trust Flags = {TrustFlags*:FULLY TRUSTED}
Fulfillment Type: TRIAL
Status: {Status:ENABLED}
Fulfillment ID: LOCAL_TRIAL_FID_586
Entitlement ID: {EntitlementID:SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PCR8}
Trust Flags = {TrustFlags*:not trusted}
Fulfillment Type: TRIAL
Status: {Status:Disabled}
Fulfillment ID: LOCAL_TRIAL_FID_590
Entitlement ID: {EntitlementID:AB_12345678ABCDEF}
'#
Now apply the template to the text
$text = #'
--------------------------------------------------------------------
Trust Flags = FULLY TRUSTED
Fulfillment Type: TRIAL
Status: ENABLED
Fulfillment ID: LOCAL_TRIAL_FID_586
Entitlement ID: SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PCR8
Product ID: NAME=Tableau Desktop TS;VERSION=4.0
Suite ID: NONE
Expiration date: 23-oct-2020
Feature line(s):
INCREMENT TableauDesktop tableau 2021.1108 permanent 1 \
VENDOR_STRING=EntitlementID=;EDITION=Professional;CAP=REG:STANDARD,WARN:14,NOGRACE;DC_STD=default;DC_CAP=;TRIALVER=2019.1;FulfillmentID=;ActivationID=;OEMNAME=;GRACE=;MAP_STD=default;MAP_CAP=;OFFLINE= \
ISSUER="Tableau Software" ISSUED=9-nov-2018 START=8-nov-2018 \
TS_OK SIGN="042D 811B 5D78 81EA E6E7 28BD 607A F3D3 028E DC82 \
E310 A6BC C1D5 0913 5CBC 18B5 8671 7C7D C0B7 3C46 D1E7 A16C \
6C84 3694 BB4C DB73 4B59 C419 D820 58E0"
--------------------------------------------------------------------
Trust Flags = FULLY TRUSTED
Fulfillment Type: TRIAL
Status: ENABLED
Fulfillment ID: LOCAL_TRIAL_FID_590
Entitlement ID: SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PTR2
Product ID: NAME=Tableau Desktop TS;VERSION=4.0
Suite ID: NONE
Expiration date: 23-oct-2020
Feature line(s):
INCREMENT TableauDesktop tableau 2021.1108 permanent 1 \
VENDOR_STRING=EntitlementID=;EDITION=Professional;CAP=REG:STANDARD,WARN:14,NOGRACE;DC_STD=default;DC_CAP=;TRIALVER=2019.1;FulfillmentID=;ActivationID=;OEMNAME=;GRACE=;MAP_STD=default;MAP_CAP=;OFFLINE= \
ISSUER="Tableau Software" ISSUED=9-nov-2018 START=8-nov-2018 \
TS_OK SIGN="042D 811B 5D78 81EA E6E7 28BD 607A F3D3 028E DC82 \
E310 A6BC C1D5 0913 5CBC 18B5 8671 7C7D C0B7 3C46 D1E7 A16C \
6C84 3694 BB4C DB73 4B59 C419 D820 58E0"
--------------------------------------------------------------------
'#
$text | ConvertFrom-String -TemplateContent $template -OutVariable results
TrustFlags Status EntitlementID
---------- ------ -------------
FULLY TRUSTED ENABLED SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PCR8
FULLY TRUSTED ENABLED SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PTR2
For the demonstration I used Out-Variable so we could see the output as well as capture to a variable. This obviously could be changed to just $variable = instead. The $results variable is a PSCustomObject which you can use like any other.
$results | where trustflags -eq 'Fully Trusted'
TrustFlags Status EntitlementID
---------- ------ -------------
FULLY TRUSTED ENABLED SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PCR8
FULLY TRUSTED ENABLED SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PTR2
$results.entitlementid
SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PCR8
SC_LVJ1BYNH8ZF6H57OSCBZTFWPVR7PTR2
To use it against a file it's probably best to use Get-Content -Raw depending on just how large those files are.
Get-Content $textfile -Raw | ConvertFrom-String -TemplateContent $template -OutVariable results
I am using google container engine and getting tons of dns errors in the logs.
Like:
10:33:11.000 I0720 17:33:11.547023 1 dns.go:439] Received DNS Request:kubernetes.default.svc.cluster.local., exact:false
And:
10:46:11.000 I0720 17:46:11.546237 1 dns.go:539] records:[0xc8203153b0], retval:[{10.71.240.1 0 10 10 false 30 0 /skydns/local/cluster/svc/default/kubernetes/3465623435313164}], path:[local cluster svc default kubernetes]
This is the payload.
{
metadata: {
severity: "ERROR"
serviceName: "container.googleapis.com"
zone: "us-central1-f"
labels: {
container.googleapis.com/cluster_name: "some-name"
compute.googleapis.com/resource_type: "instance"
compute.googleapis.com/resource_name: "fluentd-cloud-logging-gke-master-cluster-default-pool-f5547509-"
container.googleapis.com/instance_id: "instanceid"
container.googleapis.com/pod_name: "fdsa"
compute.googleapis.com/resource_id: "someid"
container.googleapis.com/stream: "stderr"
container.googleapis.com/namespace_name: "kube-system"
container.googleapis.com/container_name: "kubedns"
}
timestamp: "2016-07-20T17:33:11.000Z"
projectNumber: ""
}
textPayload: "I0720 17:33:11.547023 1 dns.go:439] Received DNS Request:kubernetes.default.svc.cluster.local., exact:false"
log: "kubedns"
}
Everything is working just the logs are polluted with errors. Any ideas on why this is happening or if I should be concerned?
Thanks for the question, Aaron. Those error messages are actually just tracing/debugging output from the container and don't indicate that anything is wrong. The fact that they get written out as error messages has been fixed in Kubernetes at head and will be better in the next release of Kubernetes.