I am trying to use a queue with multiple subscribers (each with a unique selector) along with setting the destination.consumer.exclusive flag to true. But when I post a message to this Queue, I see that the message is available with in the queue but none of the subscribers have picked it up in spite of it meeting one of the consumer's selector criteria.
I see the following details on the AMQ UI console:
Number of pending messages - 1
Number of consumers - 6
Messages enqueued - 1
Messages dequeued - 0
Although the number of messages pending on the queue is 1, none of the consumers have any "enqueues" on them in spite of the pnding mesage meeting the selection criteria.
Exclusive consumers would override any selector in terms of Queue load balancing so use either one or the other. Exclusive consumer is named that way for a reason, namely the consumer is the only one that can consume from the Queue until it goes offline. It really doesn't make any sense to mix selectors and exclusive options in the first place.
Related
We have a kubernetes deployment having 2 pods. We want to send messages to both these pods in 1:n way (each message should be received by both) and each message should be delivered at least once(QoS1).
To achieve the same, we have a topic and 4 queues created beforehand. All four queues are subscribed to the topic and hence receives messages from the same. We want to now startup the 4 pods and ensure that each of them consumes from 4 different queues. However, pods do not have any index and hence it is tricky to decide on which queue one has to subscribe from. Is there a way to check if a queue has active consumer before joining to ensure that it does not join an already consumed queue ?
Not sure I understand your question precisely, but if you want to know if a flow is able to receive messages from a queue, maybe this Active Consumer Indication could help you.
I have a single node ActiveMQ instance with two competing consumers connected to a topic. The topic subscription is shared as per JMS 2.0 specification. Shared subscription does guarantee that only either of the subscribers (using same subscription name) gets the message. But what I noticed is that it does not guarantee that the second message is delivered only if the first one is acknowledged. In case if the first consumer takes time to acknowledge the message, the second message is delivered to the free consumer even before the acknowledgement of the first one is sent by the consumer to the broker. Is this a standard behaviour? And is there a way to stop the broker from delivering the second message before the acknowledgement of the first one?
ActiveMQ Artemis allows the exclusive queues. They are special queues which route all messages to only one consumer at a time.
Obviously exclusive queues have a draw back that you cannot scale out the consumers to improve consumption as only one consumer would technically be active.
However I would suggest to take a look at the message grouping to scale out your solution. Message groups are useful when you want all messages for a certain value of the property to be processed serially by the same consumer, without stopping the delivery of messages with different value of the property to other consumers.
The picture below depicts my basic use-case using message groups and Spring-based JMS consumers.
Please note, here the concurrency refers to the config set as shown below:
defaultJmsListenerContainerFactory.setConcurrency("3-10");
Would the G1 and G2 listener receive messages concurrently for the respective groups?
Would in any case the message from one group wait for the dispatch of any message in another group?
Generally speaking, multiple consumers receiving grouped messages can receive them concurrently. However, there are caveats...
The core JMS client implementations actually consumes messages from a local data structure that is filled with messages asynchronously based on the consumerWindowSize which is 1 MiB (1024 * 1024 bytes) by default. If a consumer is receiving messages from a large, contiguous group and its "window" fills up then the broker will not be able to dispatch any more messages to it and will have to wait for the consumer to acknowledge messages in order to dispatch more. Once that block of grouped messages is dispatched then the broker will be able to dispatch messages from other groups to other consumers.
This is also explained in the documentation (although in a bit less detail).
If ActiveMQ Artemis is configured with a redelivery-delay > 0 and a JMS listener uses ctx.rollback() or ctx.recover() then the broker will redeliver the message as expected. But if a producer pushes a message to the queue during a redelivery then the receiver gets unordered messages.
For example:
Queue: 1 -> message 1 is redelivered as expected
Push during the redelivery phase
Queue: 2,3 -> the receiver gets 2,3,1
With a redelivery-delay of 0 everything is ok, but the frequency of redeliveries on consumer side is too high. My expectation is that every delivery to the consumer should be stopped until the unacknowledged message is purged from the queue or acknowledged. We are using a queue for connection with single devices. Every device has it's own I/O queue with a single consumer. The word queue suggest strict ordering to me. It could be nice to make this behavior configurable like "strict_redelivery_order".
What you're seeing is the expected behavior. If you use a redelivery-delay > 0 then delivery order will be broken. If you use a redelivery-delay of 0 then delivery order will not be broken. Therefore, if you want to maintain strict order then use a redelivery-delay of 0.
If the broker blocked delivery of all other messages on the queue during a redelivery delay that would completely destroy message throughput performance. What if the redelivery delay were 60 seconds or 10 minutes? The queue would be blocked that entire time. This would not be tenable for an enterprise message broker serving hundreds or perhaps thousands of clients each of whom may regularly be triggering redeliveries on shared queues. This behavior is not configurable.
If you absolutely must maintain message order even for messages that cannot be immediately consumed and a redelivery-delay of 0 causes redeliveries that are too fast then I see a few potential options (in no particular order):
Configure a dead-letter address and set a max-delivery-attempts to a suitable value so after a few redeliveries the problematic message can be cleared from the queue.
Implement a delay of your own in your client. This could be as simple as catching any exception and using a Thread.sleep() before calling ctx.rollback().
I'm new to messaging and a little unclear as to whether it is possible for MSMQ to deliver out-of-order messages for transactional queues. I suppose it must be because if a message is not processed correctly (and since we will be using multiple "competing consumers"), then other consumers could continue to process messages while the failed message is placed back on queue. Just can't seem to find a black-and-white answer anywhere on this.
Negative black-and-white answer are hard to find( they don't often exist).
You are confusing two terms here( I think). delivery is from the sender to the queue. consuming is from the queue to the consumer. Those two action can't be put in the same transaction. They are totally separate action ( this is one of the points of queuing )
More to the point: from "Microsoft Message Queuing Services (MSMQ) Tips"
That these messages will either be sent together, in the order they were sent, or not at all. In addition, consecutive transactions initiated from the same machine to the same queue will arrive in the order they were committed relative to each other.
This is the only case of order in msmq.
Sadly you won't find anything about ordered consuming because its not relevant. You can consume messages from msmq any way you want.
Update: If you must have ordered processing, than I don't see the reason to use many consumers. You will have to implement the order in your code.
Do your messages need to be processed in order because:
1) They are different steps of a workflow? If so, you should create different queues to handle the different steps. Process 1 reads Queue 1, does its thing, then writes to Queue 2, and so forth.
2) They have different priorities? If the priority levels are fairly coarse (and the order of messages within priorities doesn't matter), you should create high-priority and low-priority queues. Consumers read from the higher priority queues first.
3) A business rule specifies it. For example, "customer orders must be processed in the order they are received." Message queues are not appropriate for this kind of sequencing since they only convey the order in which messages are received. A process that periodically polls a database for an ordered list of tasks would be more suitable.