Is there any way to get history from a queue?
I'm able to get the current count of messages that are in the queue, but is it possible to get a number of all messages that already passed on this queue?
You can use the following metrics:
MessagesAdded - number of messages added to this queue since it was created
MessagesAcknowledged - number of messages acknowledged from this queue since it was created
MessagesExpired - number of messages expired from this queue since it was created
MessagesKilled - number of messages removed from this queue since it was created due to exceeding the max delivery attempts
Related
I have scenario where i want to send message to a alert service that would process the message and would send it to hipchat.
But I want the message to be active only for a minute. If hipchat is down (hypothetical) then the message should not be sent to hipchat.
I am using kafka so one of the service sends the message to kafka then the message is consumed by alert service(it polls the service) which processes the message (kafka consumer) while processing it checks that the time now and the time of the message is not greater than one minute. If not, it sends the message to hipchat aynchronously.
Enhancement:
I want a way to construct a self destruction message so that i automatically disappears after one minute. Is there a way to do it with kafka ? OR is there a better alternate than kafka (flink/sqs). If yes, how?
You can make use of the Kafka topic configurations retention.ms and delete.retention.ms as described in the Topic Level Configs.
The retention.ms should be set to 1 minute (60000 ms) and the delete.retention.ms should be set to 0 in your case. That way, the messages will stay in the Kafka Topic for one minute before they get deleted. However, that also means that you might loose messages if your consumer takes more then one minute to consume all messages (especially when reading a topic from beginning).
Details on those configurations are:
delete.retention.ms: The amount of time to retain delete tombstone markers for log compacted topics. This setting also gives a bound on the time in which a consumer must complete a read if they begin from offset 0 to ensure that they get a valid snapshot of the final stage (otherwise delete tombstones may be collected before they complete their scan).
retention.ms: This configuration controls the maximum time we will retain a log before we will discard old log segments to free up space if we are using the "delete" retention policy. This represents an SLA on how soon consumers must read their data. If set to -1, no time limit is applied.
If ActiveMQ Artemis is configured with a redelivery-delay > 0 and a JMS listener uses ctx.rollback() or ctx.recover() then the broker will redeliver the message as expected. But if a producer pushes a message to the queue during a redelivery then the receiver gets unordered messages.
For example:
Queue: 1 -> message 1 is redelivered as expected
Push during the redelivery phase
Queue: 2,3 -> the receiver gets 2,3,1
With a redelivery-delay of 0 everything is ok, but the frequency of redeliveries on consumer side is too high. My expectation is that every delivery to the consumer should be stopped until the unacknowledged message is purged from the queue or acknowledged. We are using a queue for connection with single devices. Every device has it's own I/O queue with a single consumer. The word queue suggest strict ordering to me. It could be nice to make this behavior configurable like "strict_redelivery_order".
What you're seeing is the expected behavior. If you use a redelivery-delay > 0 then delivery order will be broken. If you use a redelivery-delay of 0 then delivery order will not be broken. Therefore, if you want to maintain strict order then use a redelivery-delay of 0.
If the broker blocked delivery of all other messages on the queue during a redelivery delay that would completely destroy message throughput performance. What if the redelivery delay were 60 seconds or 10 minutes? The queue would be blocked that entire time. This would not be tenable for an enterprise message broker serving hundreds or perhaps thousands of clients each of whom may regularly be triggering redeliveries on shared queues. This behavior is not configurable.
If you absolutely must maintain message order even for messages that cannot be immediately consumed and a redelivery-delay of 0 causes redeliveries that are too fast then I see a few potential options (in no particular order):
Configure a dead-letter address and set a max-delivery-attempts to a suitable value so after a few redeliveries the problematic message can be cleared from the queue.
Implement a delay of your own in your client. This could be as simple as catching any exception and using a Thread.sleep() before calling ctx.rollback().
I am trying to use a queue with multiple subscribers (each with a unique selector) along with setting the destination.consumer.exclusive flag to true. But when I post a message to this Queue, I see that the message is available with in the queue but none of the subscribers have picked it up in spite of it meeting one of the consumer's selector criteria.
I see the following details on the AMQ UI console:
Number of pending messages - 1
Number of consumers - 6
Messages enqueued - 1
Messages dequeued - 0
Although the number of messages pending on the queue is 1, none of the consumers have any "enqueues" on them in spite of the pnding mesage meeting the selection criteria.
Exclusive consumers would override any selector in terms of Queue load balancing so use either one or the other. Exclusive consumer is named that way for a reason, namely the consumer is the only one that can consume from the Queue until it goes offline. It really doesn't make any sense to mix selectors and exclusive options in the first place.
I am trying to deliver a JMS message after some time passes, my initial idea was to use expiry queue and to put the messages in a queue that doesn't have any consumers. So I have 3 default queues:
WaitQueue - (expiry queue for this one is set to SendQueue)
SendQueue - this one has consumers that process the messages(by default this one has expiryQueue as its timeout queue)
ExpiryQueue - default jboss queue for all messages that really expired(not intentionally)
In insert a message into the WaitQueue with my intended delay as TimeToLive, after the time expires I expect to see the messages in SendQueue(and the consumers to process them), however it stays empty and the messages directly go to ExpiryQueue, any ideas what is wrong?
The statistics for SendQueue shows that "Received messages" increase, but current messages stays at 0, so they arrive but get forwarded immediately to the last ExpiryQueue.
Instead of using expiry queue approach which is more resource intensive; you could consider using delivery delay at the Message level.
In case of HornetQ, you can set the property _HQ_SCHED_DELIVERY.
https://docs.jboss.org/hornetq/2.3.0.Final/docs/user-manual/html/scheduled-messages.html
TextMessage message = session.createTextMessage("This is a scheduled message message which will be delivered in 5 sec.");
message.setLongProperty("_HQ_SCHED_DELIVERY", System.currentTimeMillis() + 5000);
producer.send(message);
Since JMS2.0 (JavaEE7) this property can also be set on MessageProducer. See https://github.com/jboss/jboss-jms-api_spec/blob/master/src/main/java/javax/jms/MessageProducer.java#L285
Use ActiveMQ :
Senario:
Server will send many messages to client through Queue.
However ,i nedd to drop the message in the queue if there is no consumer(client)
Thanks in advance!
You can use non persistent messaging and the message is dropped if there is no active consumers.
Another alternative could be to use message expiry, so the message expires after X period, if they are not consumed from the queue.
Set a JMSExpiration on each message for some duration (30 seconds? 5 minutes?), and then any message that's not consumed after that amount of time (whether because there's no consumer or because the consumer's running behind) will be sent to the DLQ. Or if you don't want it in the DLQ, then configure the dead letter strategy to set processExpired=false or use a Discarding DLQ Plugin, both documented at http://activemq.apache.org/message-redelivery-and-dlq-handling.html.