Confuse about fail2ban behavior with firewallD in Centos 7 - fail2ban

Question

I was using fail2ban/iptables in a Centos 6 server.
I moved to Centos 7 and now I am using fail2ban/firewallD (installed by Webmin/Virtualmin with their defaults)
These are cat /var/log/maillog | grep "disconnect from unknown" screen shots
cat /var/log/fail2ban.log | grep Ban only displays
2019-10-27 16:52:22,975 fail2ban.actions [8792]: NOTICE [proftpd] Ban 111.225.204.32
Furthermore tailf /var/log/fail2ban.log displays several "already banned" of the same IP. In this case fail2ban, after maxretry is reached it tries to ban the IP.
Here are my configurations (partial), I left them as they were by defaults but changed bantimes.
jail.local
[postfix]
enabled = true
port = smtp,465,submission
bantime = -1
[postfix-sasl]
enabled = true
port = smtp,465,submission,imap3,imaps,pop3,pop3s
bantime = -1
[dovecot]
enabled = true
port = pop3,pop3s,imap,imaps,submission,465,sieve
bantime = -1
jail.conf
[DEFAULT]
findtime = 600
maxretry = 5
backend = auto
filter = %(__name__)s
port = 0:65535
banaction = iptables-multiport
banaction_allports = iptables-allports
action_ = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="% > (port)s", protocol="%(protocol)s", chain="%(chain)s"]
action = %(action_)s
jail.d/00-firewalld.conf
[DEFAULT]
banaction = firewallcmd-ipset
These files exist: action.d/firewallcmd-ipset.conf and filter.d/postfix.conf
firewall-cmd --direct --get-all-rules
ipv4 filter INPUT_direct 0 -p tcp -m multiport --dports ssh -m set
--match-set fail2ban-default src -j REJECT --reject-with icmp-port-unreachable
ipv4 filter INPUT 0 -p tcp -m multiport --dports ssh -m set --match-set fail2ban-sshd src -j REJECT --reject-with icmp-port-unreachable
ipv4 filter INPUT 0 -p tcp -m multiport --dports 10000 -m set --match-set fail2ban-webmin-auth src -j REJECT --reject-with icmp-port-unreachable
ipv4 filter INPUT 0 -p tcp -m multiport --dports ssh,sftp -m set --match-set fail2ban-ssh-ddos src -j REJECT --reject-with icmp-port-unreachable
After manually running
firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='193.56.28.0/24' reject"
and
firewall-cmd --reload this output of tailf /var/log/fail2ban.log
stopped.
How can I get all those IPs banned after they reach maxretry value?
Would they be banned forever despite service restart or reload?
Edit 1:
From fail2ban.log with action=firewalld-cmd ipset
From fail2ban.log with action=iptables-allports
Edit 2:
It seems (I guess) something is flushing configurations (I guess it would be Webmin) because after a while I start getting error logs like failed to execute ban jail 'dovecot' action iptables-allports so I am trying this:
in actions.d created banning.conf
[Definition]
actionban = /usr/bin/firewall-cmd --permanent --add-rich-rule="rule family='ipv4' source address='<IP>' reject"; ; /usr/bin/firewall-cmd --reload
and at jail.local
[DEFAULT]
banaction = iptables-multiport
banning
But I get Error in action definition banning
I know this is not a solution.
Before moving the server I was using fail2ban/iptables (not firewalld) for years not having to pay attention beyond default settings.

Answer 1

How can I get all those IPs banned after they reach maxretry value?
Your issue has probably nothing with maxretry etc.
If you see [jail] Ban 192.0.2.1 and several [jail] 192.0.2.1 already banned messages hereafter (especially after some minutes after the "Ban" message for same Jail/IP), this means only that your banning action (firewalld) does not work at all (after ban, the intruder-IP is still able to repeat its attempts).
In the last time we had certain issues with that (especially with combination firewalld + CentOS) - see for example https://github.com/fail2ban/fail2ban/issues/1609 as well as related firewalld issue - https://github.com/firewalld/firewalld/issues/515.
So check your native net-filter (iptables, etc), if you see some (white-listing established traffic) rules before fail2ban chains, it looks like your configuration is not fail2ban (or whatever banning-system) capable... here may be the answer for you - https://github.com/fail2ban/fail2ban/issues/2503#issuecomment-533105500.
Here is another similar issue with an example excerpt illustrating "wrong iptables rule that bypass fail2ban" - https://github.com/fail2ban/fail2ban/issues/2545#issuecomment-543347684
In this case:
either switch the backend of firewalld (as suggested above);
or switch the banaction of fail2ban to something native (iptables/ipset/etc).
or even add still one action dropping or killing active established connection of the banned IP (using something like tcpkill, killcx, ss etc).
UPDATE 1
jail.local example:
[DEFAULT]
banaction = iptables-multiport
banaction_allports = iptables-allports
[postfix-sasl]
enabled = true
[dovecot]
enabled = true
...
If after fail2ban reload you'd still see some IP making attempts after ban and already banned in fail2ban.log, provide log-excerpt of fail2ban by the first ban or else some possible errors around (because already banned is too late and does not help at all).
If no errors are there, provide output of iptables -nL.