What is a preferred solution in case when EAP runs in 2-3 nodes and we want to use shared jms queues there?
Is standalone HA mode with shared queue-s enough, or better use domain mode there?
The preferred way would be to run Artemis by itself and connect to it. You can also define connectors to a specific node that would host the JMS broker.
HA is there to do what it says : HA, not for sharing destinations.
Related
We have setup a Kafka Cluster for High Availability and distributed data load. The current consumers and producers specify all the broker IP addresses to connect to the cluster. In the future, there will be the need to continuosly monitor the cluster and add a new broker based on collected metrics and overall system performance. In case a broker crashes, as soon as possible we have to add a new broker with a different IP.
In these scenarios, we have to change all client configurations, a time consuming and stressful operation.
I think we can setup a Config Server (e.g. Spring Cloud Config Server) to specify all the broker IP addresses in a centralized manner, so we have to change all in one place, without touching all the clients, but I don't know if it is the best approach. Obviously, the clients must be programmed to get broker list from config server.
There's a better approach?
Worth pointing out that the "bootstrap" process doesn't require giving every single broker address to the clients, really only the first available address in the list is used for the initial connection, then the advertised.listeners on all the broker configs in the cluster, is what the clients actually use
The answer to your question is to use service discovery, yes. That could be Spring Could Config, but the more general option would be Hashicorp Consul or other service that uses DNS (Kubernetes uses CoreDNS, by default, for example, or AWS Route53).
Then you edit the /etc/resolv.conf of each machine (assuming Linux) the client is running on to include the DNS servers, and you can simply refer to kafka.your.domain:9092 rather than using IP addresses
You could use a load balancer (with a friendly dns like kafka.domain.com), which points to all of your brokers. We do this in our environment. Your clients then connect to kafka.domain.com:9092.
As soon as you add new brokers, you only change the load balancer endpoints and not the client configuration.
Additionally please note that you only need to connect to one bootstrap broker and don't have to list all of them in the client configuration.
On-prem--->HAProxy(AWS)--->Kafka(AWS). We can allow the external communication using advertised.listerers property and we can use the listeners for internal communication. If we are enabling both the settings,the communication is not happening properly. We are using 0.10.2 as Kafka version.
I believe we have some setting to do through zookeeper to control the broker communication. How we can do it using cloudera?
See https://rmoff.net/2018/08/02/kafka-listeners-explained/. You need to set up two listeners, one for the external IP through which your clients connect, one for the internal AWS network through which your brokers will communicate with each other.
I deployed confluent on a Single cloud on a Linux VM and I'm planning to configure interceptors for the Consumers.
But how do I configure Control Center in order to access its Web-Interface?
Deployed Confluent Control Center Enterprise Edition.
Not very clear what issues you're having.
Edit the server.properties file. Add the bootstrap servers, Zookeeper quorum, and optionally the connect cluster.
With that, you should at least be able to see the topics in the cluster
Run the control-center binary with the server property file as a parameter.
See additional setup steps required for the clients to get monitoring to work at the documentation
I am new to Jboss fuse karaf(jboss-fuse-6.3) and activeMQ.
Currently I am accessing activeMq on localhost:61616, my question is what is more suitable way of using activeMQ - on same server(localhost:port) or can I use activeMQ hosted on remote server, which method is more fast and safe ?
Thanks.
If you are using it for dev purpose, then the most suitable way is to use the embedded one that comes with Fuse out of the box because it requires less maintenance.
However if you are asking about AMQ in production environment then most likely it should be highly available and able to survive crashes without data losses. In this case you need to think about some failover machanism, so the safer option is to use an external ActiveMQ broker or even better - a cluster of AMQ brokers. In this case the speed of message delivery and consumption depends on the network connectivity as well.
Also if we talk about speed you should consider where AMQ should store it's messages especially the undelivered ones. By default it comes with a file based DB KahaDB which can be changed. In both cases it's fast enough.
Wish you luck with the ESB!
I have gone through (not fully) ActiveMQ and tried to figure out the deployment model for my application.
I am bit confused on that.
I want to make the system High Availability and decided to use the following. Please correct me if anything is wrong or disadvantage of the model.
Deployment Modle:
Will deploy Brokers in M1 and M2 respectivley.
Use Hardware load balancer (Either F5 or Zeus) to connect either one of the broker (M1 or M2) based on the load.
Want to publish a message using Load balancer URL.
I have gone through network of brokers and we need to mintain some topology. I fell which makes the system more complicated if system grows horizontally. So it is better to have one load balancer to distribute the load.
Questions
Is this above model will send message to any one of the Broker?
Consumer Will be deployed in Tomcat (Think i need to use embeded brokers to configure either M1 or M2). Is it possible to use Load balancer URL instaed of M1 or M2?
Is it possible to have single Web Console Admin to monitor both M1 and M2.
Do we have any performance issue using Spring's feature to consume message.
Sorry to shoot out so many questions. Please help me to correct the deployment model.
I think the best way to get some load balancing with some activemq servers is having a : network of brokers and your consumers/producers (in your webapps) should use some failover
So if a producer p1 send a message on a queue on broker 1, the consumer c1 can read the message on broker 2.
[Edit] I have never tried to add some hardware balancer instead of the activemq protocole failover. It should work : just try it and tell us.
3- I do not think it is possible to have only one Web Console to monitor both of your brokers.
4- As far as I am concerned I do not have any performance issue with my Spring configuration.
There are a lot of questions there.
The first thing is to do is start simple. If your application's load is being handled with just one broker, consider setting up high availability through a master-slave setup. For this you do not need a load balancer - the ActiveMQ client library has a failover mechanism where you can define the URLs to a set of brokers that the client should attempt to connect to.
If you are looking at setting up an infrastructure where one broker will not be able to deal with the message load (you can test the maximum throughput of your broker using the performance module), I would advise you to read up on how networks of brokers work. If you do go down this path, you really need to understand ActiveMQ.
On monitoring, a web console can only show you the internals of a single broker. To get insight around what is going on around a set of brokers you will need a monitoring tool such as FuseHQ/Hyperic that is able to aggregate JMX information from a number of boxes.
Performance with Spring is not a problem as long as you configure it correctly (see the section on PooledConnectionFactory).
I see that you are a new user, so if this answers your question, please tick it.