Is there a best practice on setting a retention policy on ActiveMQ Artemis dead-letter queues?
I was looking through the documentation, but I cannot find anything related. My best approach would be calling removeMessages(string) with a filter AMQTimestamp > TIMESTAMP.
There's no real best practice here as it's really dependent on use-case and use-cases vary widely in their needs this regard.
Using removeMessages(string) with a filter AMQTimestamp > TIMESTAMP is certainly fine when you want to remove messages administratively (or even potentially with a script). However, if you want to set up something more automated you can just use the expiry-delay address setting, e.g.:
<address-setting match="myAddress">
<expiry-delay>300000</expiry-delay> <!-- 300 seconds (5 minutes) -->
</address-setting>
If there's no expiry address defined then the messages will simply be removed after the expiry-delay elapses. If there is an expiry address defined (e.g. in a parent's address-setting) then those messages will be routed to any queues bound to that address according to the configured routing type(s). However, if you want to remove the expiry address so that the messages are just dropped then you can, e.g.:
<address-setting match="myAddress">
<expiry-address/>
<expiry-delay>300000</expiry-delay> <!-- 300 seconds (5 minutes) -->
</address-setting>
Related
We operate an ActiveMQ Artemis message broker to distribute incoming measurements (think weather stations) to downstream systems. There often are multiple consumers. So for example given the address measurements.africa, there would be multicast queues consumer1.africa and consumer2.africa attached.
Now some downstream systems are not reliable, they sometimes fail to attend to their queues. This requires manual intervention on our part to prevent the message broker from filling up. Since these unreliable systems should get messages on a best effort basis, I'd like to automate the culling of messages, when those queues start to fill up. At the same time, there are other queues that should reliably keep messages until downstream systems are available again.
So I'd thought to give the unreliable queues a common prefix, say unreliable. and then in broker.xml configure those queues to drop messages:
<address-setting match="reliable.#">
<expiry-delay>-1</expiry-delay>
</address-setting>
<address-setting match="unreliable.#">
<expiry-delay>300000</expiry-delay>
<expiry-address></expiry-address>
</address-setting>
However, I found that I can only configure settings on the address itself, not on the queues on this address. So the queues reliable.consumer1.africa and unreliable.consumer2.africa cannot have different settings configured in broker.xml. The queues are created automatically by the Apache Camel JMS component, using the subscriptionName= parameter.
I've read through the Artemis manual on address settings, but it doesn't mention a way. Is there a way to create queues with different settings on the same address?
There is no way to apply address settings to a specific queue. In almost all circumstances if one setting is viable for one queue then it is viable for all the queues bound to that address.
However, in circumstances where you really need different address settings then you can split the message flow using a non-exclusive divert and another address, e.g.:
<diverts>
<divert name="measurements.africa-divert">
<address>measurements.africa</address>
<forwarding-address>unreliable.measurements.africa</forwarding-address>
<exclusive>false</exclusive>
</divert>
</diverts>
<addresses>
...
<address name="measurements.africa">
<multicast/>
</address>
<address name="unreliable.measurements.africa">
<multicast/>
</address>
</addresses>
With this configuration any message sent to measurements.africa will also be sent to unreliable.measurements.africa which means that unreliable consumers can subscribe to unreliable.measurements.africa and reliable consumers can continue to use measurements.africa. Then you can apply your address settings like so:
<address-setting match="measurements.africa">
<expiry-delay>-1</expiry-delay>
</address-setting>
<address-setting match="unreliable.#">
<expiry-delay>300000</expiry-delay>
<expiry-address></expiry-address>
</address-setting>
As far as I understand paging will be carried out on adresses if they exceed the defined size. Currently we experience a paging, but not on known addresses (queues). It seems like it is an internal queue from ActiveMQ? Is it possible to understand what kind of address ActiveMQ is paging here?
WARN [org.apache.activemq.artemis.core.server] AMQ222038: Starting paging on address '$.artemis.internal.my-cluster.fec50662-55c7-11eb-91d1-005056903119'; size is currently: 25,238,532 bytes; max-size-bytes: -1; global-size-bytes: 524,357,417
This is important for us, because we have analyzed that this paging causing the inability for the messages in our queues to be consumed.
The address named $.artemis.internal.my-cluster.fec50662-55c7-11eb-91d1-005056903119, and the related queue, are used for intra-cluster communication. When messages need to be moved from one node to another they are sent to this address and then forwarded to another broker by the internal cluster bridge.
Given the log message I would surmise that you've reached the global-size-bytes which is calculated by adding up the bytes from all addresses. You might consider increasing your global-max-size in broker.xml.
You say that this paging is preventing your consumers from consuming messages. However, it's also worth noting that paging is typically caused by consumers not consuming messages, not the other way around. When consumers slow down or stop then messages build up in the broker and it has no choice but to begin paging. Therefore you would likely see both of these things simultaneously which could lead to misattribution.
I created a queue with a name that does not match the address name
Then I send a message to the queue with the name QUEUE
The code completed without errors, but the message did not appear in the queue with the name QUEUE and also in DLQ, there are also no errors in the Artemis logs. Where did the messages go?
apache-artemis v. 2.11.0, default config after creat broker
Also the second question, is it possible to send a message directly to such a queue, the name of which is different from the name of the address?
This is the basic addressing model used by ActiveMQ Artemis - send messages to an address, consume them from a queue. How messages are routed from the address to the queue depends on the configured routing-type. You can read all about this in the address model documentation.
If you send a message to an address that doesn't exist then it will be routed nowhere. In other words it will just be discarded. If you want to catch these kinds of messages in a dead-letter resource then you can set:
<address-settings>
<address-setting match="#">
<dead-letter-address>DLA</dead-letter-address>
<send-to-dla-on-no-route>true</send-to-dla-on-no-route>
</address-setting>
</address-settings>
<addresses>
<address name="DLA">
<anycast>
<queue name="DLQ" />
</anycast>
</address>
</addresses>
This is discussed in the documentation.
If for some reason you want to send a message directly to a queue then you can use the fully qualified queue name (i.e. FQQN) which follows the pattern <address>::<queue>. So in your case you'd use:
final Destination dest = session.createQueue("ADD::QUEUE");
This is also discussed in the documentation.
You'd probably also benefit from reading the documentation about how JMS concepts are mapped onto the core API.
We would like to reject the messages being placed in hornetq greater than 50 MB.
Could we restrict it in the configuration at queue/connection factory level.
Placing large messages in HornetQ is causing heap issue and the server is getting crashed.
Any help appreciated.
Edit your .xml configuration like:
<address-setting match="your/queue/address">
<max-size-bytes>104857600</max-size-bytes>
<page-size-bytes>104857600</page-size-bytes>
<address-full-policy>DROP</address-full-policy>
</address-setting>
From the docs:
Messages are stored per address on the file system.
Instead of paging messages when the max size is reached, an address can also be configured to just drop messages when the address is full.
..to do so, set address-full-policy to DROP (messages will be silently dropped).
The above settings are documented at:
https://docs.jboss.org/hornetq/2.2.5.Final/user-manual/en/html/queue-attributes.html
While, especially concerning the message size elements: https://docs.jboss.org/hornetq/2.2.5.Final/user-manual/en/html/paging.html
I need to create a proxy service scheduler that receive messages of the queue after 5 minutes. like queue produce message either a single or multiple but proxy receieve that messages after interval of every 5 minutes. how can i achieve this only using oracle service bus ...
Kindly help me for this
OSB do not provide Scheduler capabilities out of the box. You can do either of the following:
For JMS Queue put infinite retries by not setting retry limit and set retry interval as 5 minutes.
Create a scheduler. Check this post for the same: http://blogs.oracle.com/jamesbayer/entry/weblogic_scheduling_a_polling
Answer left for reference only, messages shouldn't be a subject to complex computed selections in this way, some value comparison and pattern matching only.
To fetch only old enough messages from queue,
not modifying queue or messages
not introducing any new brokers between queue and consumer
not prematurely consuming messages
, use Message Selector field of OSB Proxy on JMS Transport tab to set boolean expression (SQL 92) that checks that message's JMSTimestamp header is at least 5 minutes older than current time.
... and I wasn't successful to quickly produce valid message selector neither from timestamp nor JMSMessageID (it contains time in milis - 'ID:<465788.1372152510324.0>').
I guess somebody could still use it in some specific case.
You can use Quartz scheduler APIs to create schedulers across domains.
Regards,
Sajeev
I don't know whether this works for you, but its working good for me. May be you can use this to do your needful.
Goto Transport Details of your Proxy Service, under Advanced Options tab, set the following fields.
Polling Frequency (Mention your frequency 300 sec(5 min))
Physical Directory (may be here you need to give your Queue path)