Artemis 2.18.0 replica set doesn't work properly - activemq-artemis

Question

I have running Artemis version 2.17.0 replica set with master and two slaves. It works fine when I check "Broker diagram" view in the web console I see connection between master and slave (the other slave as a backup) as on the picture
I upgraded now Artemis to 2.18.0 version and after restarting all artemis brokers when I check "Broker diagram" I see only master node and there is no link to the slave like on the picture above. The other two nodes are running as a slaves so there is only one master.
As I said on Artemis-2.17.0 it works.
Someone knows why is that ? here is for example the broker.xml for master node
<?xml version='1.0'?>
<configuration xmlns="urn:activemq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xi="http://www.w3.org/2001/XInclude"
xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
<core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:activemq:core ">
<name>0.0.0.0</name>
<persistence-enabled>true</persistence-enabled>
<journal-type>ASYNCIO</journal-type>
<paging-directory>data/paging</paging-directory>
<bindings-directory>data/bindings</bindings-directory>
<journal-directory>data/journal</journal-directory>
<large-messages-directory>data/large-messages</large-messages-directory>
<journal-datasync>true</journal-datasync>
<journal-min-files>2</journal-min-files>
<journal-pool-files>10</journal-pool-files>
<journal-device-block-size>4096</journal-device-block-size>
<journal-file-size>10M</journal-file-size>
<journal-buffer-timeout>28000</journal-buffer-timeout>
<journal-max-io>4096</journal-max-io>
<!-- how often we are looking for how many bytes are being used on the disk in ms -->
<disk-scan-period>5000</disk-scan-period>
<!-- once the disk hits this limit the system will block, or close the connection in certain protocols
that won't support flow control. -->
<max-disk-usage>100</max-disk-usage>
<!-- should the broker detect dead locks and other issues -->
<critical-analyzer>true</critical-analyzer>
<critical-analyzer-timeout>150000</critical-analyzer-timeout>
<critical-analyzer-check-period>60000</critical-analyzer-check-period>
<critical-analyzer-policy>HALT</critical-analyzer-policy>
<page-sync-timeout>1628000</page-sync-timeout>
<global-max-size>204Mb</global-max-size>
<!-- Connectors -->
<connectors>
<connector name="netty-connector">tcp://artemis01:61616?sslEnabled=true;trustStorePath=/client_ts.p12;trustStorePassword=12345</connector>
</connectors>
<acceptors>
<acceptor name="netty-acceptor">tcp://artemis01:61616?sslEnabled=true;keyStorePath=/broker_ks.p12;keyStorePassword=123456</acceptor>
</acceptors>
<cluster-connections>
<cluster-connection name="my-cluster">
<connector-ref>netty-connector</connector-ref>
<retry-interval>1000</retry-interval>
<retry-interval-multiplier>3</retry-interval-multiplier>
<use-duplicate-detection>true</use-duplicate-detection>
<message-load-balancing>STRICT</message-load-balancing>
<discovery-group-ref discovery-group-name="my-discovery-group"/>
</cluster-connection>
</cluster-connections>
<broadcast-groups>
<broadcast-group name="my-broadcast-group">
<local-bind-address>artemis01</local-bind-address>
<local-bind-port>9876</local-bind-port>
<group-address>231.7.7.7</group-address>
<group-port>9876</group-port>
<broadcast-period>2000</broadcast-period>
<connector-ref>netty-connector</connector-ref>
</broadcast-group>
</broadcast-groups>
<discovery-groups>
<discovery-group name="my-discovery-group">
<local-bind-address>artemis01</local-bind-address>
<local-bind-port>9876</local-bind-port>
<group-address>231.7.7.7</group-address>
<group-port>9876</group-port>
<refresh-timeout>10000</refresh-timeout>
</discovery-group>
</discovery-groups>
<network-check-list>artemis02,artemis03</network-check-list>
<network-check-period>5000</network-check-period>
<network-check-timeout>2000</network-check-timeout>
<network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command>
<network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command>
<!-- Other config -->
<ha-policy>
<replication>
<master>
<check-for-live-server>true</check-for-live-server>
</master>
</replication>
</ha-policy>
<security-settings>
<security-setting match="#">
<permission type="createNonDurableQueue" roles="amq"/>
<permission type="deleteNonDurableQueue" roles="amq"/>
<permission type="createDurableQueue" roles="amq"/>
<permission type="deleteDurableQueue" roles="amq"/>
<permission type="createAddress" roles="amq"/>
<permission type="deleteAddress" roles="amq"/>
<permission type="consume" roles="amq"/>
<permission type="browse" roles="amq"/>
<permission type="send" roles="amq"/>
<!-- we need this otherwise ./artemis data imp wouldn't work -->
<permission type="manage" roles="amq"/>
</security-setting>
</security-settings>
<addresses>
<address name="exampleQueue">
<anycast>
<queue name="exampleQueue"/>
</anycast>
</address>
<address name="DLQ">
</address>
<address name="ExpiryQueue">
<anycast>
<queue name="ExpiryQueue" />
</anycast>
</address>
</addresses>
<address-settings>
<!-- if you define auto-create on certain queues, management has to be auto-create -->
<address-setting match="activemq.management#">
<dead-letter-address>DLQ</dead-letter-address>
<expiry-address>ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<!-- with -1 only the global-max-size is in use for limiting -->
<max-size-bytes>-1</max-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>PAGE</address-full-policy>
<auto-create-queues>true</auto-create-queues>
<auto-create-addresses>true</auto-create-addresses>
<auto-create-jms-queues>true</auto-create-jms-queues>
<auto-create-jms-topics>true</auto-create-jms-topics>
</address-setting>
<!--default for catch all-->
<address-setting match="#">
<dead-letter-address>DLQ</dead-letter-address>
<expiry-address>ExpiryQueue</expiry-address>
<redelivery-delay>0</redelivery-delay>
<auto-create-dead-letter-resources>true</auto-create-dead-letter-resources>
<!-- with -1 only the global-max-size is in use for limiting -->
<max-size-bytes>-1</max-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>PAGE</address-full-policy>
<auto-create-queues>true</auto-create-queues>
<auto-create-addresses>true</auto-create-addresses>
<auto-create-jms-queues>true</auto-create-jms-queues>
<auto-create-jms-topics>true</auto-create-jms-topics>
</address-setting>
<address-setting match="exampleQueue">
<dead-letter-address>DLQ</dead-letter-address>
<redelivery-delay>1000</redelivery-delay>
<max-delivery-attempts>3</max-delivery-attempts>
<max-size-bytes>-1</max-size-bytes>
<page-size-bytes>1048576</page-size-bytes>
<message-counter-history-day-limit>10</message-counter-history-day-limit>
<address-full-policy>PAGE</address-full-policy>
</address-setting>
</address-settings>
<!-- Uncomment the following if you want to use the Standard LoggingActiveMQServerPlugin pluging to log in events
<broker-plugins>
<broker-plugin class-name="org.apache.activemq.artemis.core.server.plugin.impl.LoggingActiveMQServerPlugin">
<property key="LOG_ALL_EVENTS" value="true"/>
<property key="LOG_CONNECTION_EVENTS" value="true"/>
<property key="LOG_SESSION_EVENTS" value="true"/>
<property key="LOG_CONSUMER_EVENTS" value="true"/>
<property key="LOG_DELIVERING_EVENTS" value="true"/>
<property key="LOG_SENDING_EVENTS" value="true"/>
<property key="LOG_INTERNAL_EVENTS" value="true"/>
</broker-plugin>
</broker-plugins>
-->
<diverts>
</diverts>
</core>
</configuration>
Log from Master - artemis01
AMQ222208: SSL handshake failed for client from /195.10.125.225:58790: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown.
Logs from Slave - artemis02
2021-08-24 00:07:06,544 ERROR [org.apache.activemq.artemis.core.client] AMQ214016: Failed to create netty connection: javax.net.ssl.SSLHandshakeException: No name matching artemis01.mydomain.com found
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) [java.base:]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:325) [java.base:]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:268) [java.base:]
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263) [java.base:]
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1340) [java.base:]
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1215) [java.base:]
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1158) [java.base:]
at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) [java.base:]
at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:445) [java.base:]
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1260) [java.base:]
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1247) [java.base:]
at java.base/java.security.AccessController.doPrivileged(AccessController.java:691) [java.base:]
at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1192) [java.base:]
at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1550) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1396) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1237) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1286) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:795) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.66.Final.jar:4.1.66.Final]
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.18.0.jar:2.18.0]
Caused by: java.security.cert.CertificateException: No name matching artemis01.mydomain.com found
at java.base/sun.security.util.HostnameChecker.matchDNS(HostnameChecker.java:229) [java.base:]
at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:102) [java.base:]
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:452) [java.base:]
at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:412) [java.base:]
at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:292) [java.base:]
at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:144) [java.base:]
at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1318) [java.base:]

Answer 1

I set up a simple pair of brokers using replication and the "Broker Diagram" displays both the master and the slave as expected:
Ensure you have the "Show Backup Brokers" checkbox selected.
Regarding the SSL issue, see the upgrade instructions for 2.18.0 in the documentation:
...core clients will now expect the CN or Subject Alternative Name values of the broker's SSL certificate to match the hostname in the client's URL.
...
To deal with this you can do one of the following:
Update your SSL certificates to use a hostname which matches the hostname in the client's URL. This is the recommended option with regard to security.
Update any connector using sslEnabled=true to also use verifyHost=false. Using this option means that you won't get the extra security of hostname verification, but no certificates will need to change. This essentially restores the previous default behavior.