We operate an ActiveMQ Artemis message broker to distribute incoming measurements (think weather stations) to downstream systems. There often are multiple consumers. So for example given the address measurements.africa, there would be multicast queues consumer1.africa and consumer2.africa attached.
Now some downstream systems are not reliable, they sometimes fail to attend to their queues. This requires manual intervention on our part to prevent the message broker from filling up. Since these unreliable systems should get messages on a best effort basis, I'd like to automate the culling of messages, when those queues start to fill up. At the same time, there are other queues that should reliably keep messages until downstream systems are available again.
So I'd thought to give the unreliable queues a common prefix, say unreliable. and then in broker.xml configure those queues to drop messages:
<address-setting match="reliable.#">
<expiry-delay>-1</expiry-delay>
</address-setting>
<address-setting match="unreliable.#">
<expiry-delay>300000</expiry-delay>
<expiry-address></expiry-address>
</address-setting>
However, I found that I can only configure settings on the address itself, not on the queues on this address. So the queues reliable.consumer1.africa and unreliable.consumer2.africa cannot have different settings configured in broker.xml. The queues are created automatically by the Apache Camel JMS component, using the subscriptionName= parameter.
I've read through the Artemis manual on address settings, but it doesn't mention a way. Is there a way to create queues with different settings on the same address?
There is no way to apply address settings to a specific queue. In almost all circumstances if one setting is viable for one queue then it is viable for all the queues bound to that address.
However, in circumstances where you really need different address settings then you can split the message flow using a non-exclusive divert and another address, e.g.:
<diverts>
<divert name="measurements.africa-divert">
<address>measurements.africa</address>
<forwarding-address>unreliable.measurements.africa</forwarding-address>
<exclusive>false</exclusive>
</divert>
</diverts>
<addresses>
...
<address name="measurements.africa">
<multicast/>
</address>
<address name="unreliable.measurements.africa">
<multicast/>
</address>
</addresses>
With this configuration any message sent to measurements.africa will also be sent to unreliable.measurements.africa which means that unreliable consumers can subscribe to unreliable.measurements.africa and reliable consumers can continue to use measurements.africa. Then you can apply your address settings like so:
<address-setting match="measurements.africa">
<expiry-delay>-1</expiry-delay>
</address-setting>
<address-setting match="unreliable.#">
<expiry-delay>300000</expiry-delay>
<expiry-address></expiry-address>
</address-setting>
Related
Is there a best practice on setting a retention policy on ActiveMQ Artemis dead-letter queues?
I was looking through the documentation, but I cannot find anything related. My best approach would be calling removeMessages(string) with a filter AMQTimestamp > TIMESTAMP.
There's no real best practice here as it's really dependent on use-case and use-cases vary widely in their needs this regard.
Using removeMessages(string) with a filter AMQTimestamp > TIMESTAMP is certainly fine when you want to remove messages administratively (or even potentially with a script). However, if you want to set up something more automated you can just use the expiry-delay address setting, e.g.:
<address-setting match="myAddress">
<expiry-delay>300000</expiry-delay> <!-- 300 seconds (5 minutes) -->
</address-setting>
If there's no expiry address defined then the messages will simply be removed after the expiry-delay elapses. If there is an expiry address defined (e.g. in a parent's address-setting) then those messages will be routed to any queues bound to that address according to the configured routing type(s). However, if you want to remove the expiry address so that the messages are just dropped then you can, e.g.:
<address-setting match="myAddress">
<expiry-address/>
<expiry-delay>300000</expiry-delay> <!-- 300 seconds (5 minutes) -->
</address-setting>
As far as I understand paging will be carried out on adresses if they exceed the defined size. Currently we experience a paging, but not on known addresses (queues). It seems like it is an internal queue from ActiveMQ? Is it possible to understand what kind of address ActiveMQ is paging here?
WARN [org.apache.activemq.artemis.core.server] AMQ222038: Starting paging on address '$.artemis.internal.my-cluster.fec50662-55c7-11eb-91d1-005056903119'; size is currently: 25,238,532 bytes; max-size-bytes: -1; global-size-bytes: 524,357,417
This is important for us, because we have analyzed that this paging causing the inability for the messages in our queues to be consumed.
The address named $.artemis.internal.my-cluster.fec50662-55c7-11eb-91d1-005056903119, and the related queue, are used for intra-cluster communication. When messages need to be moved from one node to another they are sent to this address and then forwarded to another broker by the internal cluster bridge.
Given the log message I would surmise that you've reached the global-size-bytes which is calculated by adding up the bytes from all addresses. You might consider increasing your global-max-size in broker.xml.
You say that this paging is preventing your consumers from consuming messages. However, it's also worth noting that paging is typically caused by consumers not consuming messages, not the other way around. When consumers slow down or stop then messages build up in the broker and it has no choice but to begin paging. Therefore you would likely see both of these things simultaneously which could lead to misattribution.
I created a queue with a name that does not match the address name
Then I send a message to the queue with the name QUEUE
The code completed without errors, but the message did not appear in the queue with the name QUEUE and also in DLQ, there are also no errors in the Artemis logs. Where did the messages go?
apache-artemis v. 2.11.0, default config after creat broker
Also the second question, is it possible to send a message directly to such a queue, the name of which is different from the name of the address?
This is the basic addressing model used by ActiveMQ Artemis - send messages to an address, consume them from a queue. How messages are routed from the address to the queue depends on the configured routing-type. You can read all about this in the address model documentation.
If you send a message to an address that doesn't exist then it will be routed nowhere. In other words it will just be discarded. If you want to catch these kinds of messages in a dead-letter resource then you can set:
<address-settings>
<address-setting match="#">
<dead-letter-address>DLA</dead-letter-address>
<send-to-dla-on-no-route>true</send-to-dla-on-no-route>
</address-setting>
</address-settings>
<addresses>
<address name="DLA">
<anycast>
<queue name="DLQ" />
</anycast>
</address>
</addresses>
This is discussed in the documentation.
If for some reason you want to send a message directly to a queue then you can use the fully qualified queue name (i.e. FQQN) which follows the pattern <address>::<queue>. So in your case you'd use:
final Destination dest = session.createQueue("ADD::QUEUE");
This is also discussed in the documentation.
You'd probably also benefit from reading the documentation about how JMS concepts are mapped onto the core API.
We would like to reject the messages being placed in hornetq greater than 50 MB.
Could we restrict it in the configuration at queue/connection factory level.
Placing large messages in HornetQ is causing heap issue and the server is getting crashed.
Any help appreciated.
Edit your .xml configuration like:
<address-setting match="your/queue/address">
<max-size-bytes>104857600</max-size-bytes>
<page-size-bytes>104857600</page-size-bytes>
<address-full-policy>DROP</address-full-policy>
</address-setting>
From the docs:
Messages are stored per address on the file system.
Instead of paging messages when the max size is reached, an address can also be configured to just drop messages when the address is full.
..to do so, set address-full-policy to DROP (messages will be silently dropped).
The above settings are documented at:
https://docs.jboss.org/hornetq/2.2.5.Final/user-manual/en/html/queue-attributes.html
While, especially concerning the message size elements: https://docs.jboss.org/hornetq/2.2.5.Final/user-manual/en/html/paging.html
What are architectural patterns/solutions that make distributed queues tick?
Please share for both ordered and non-ordered types.
You can think of the backend of a queue as a replicated database. (I am assuming the queues you are talking about consider themselves as durable: when they accept a message, they guarantee at least once delivery.)
As a replicated database, the message queue backend uses a replication protocol to make sure the message is on at least N hosts before acknowledging receipt to the sender. Common replication protocols are 2PC, 3PC, and consensus protocols like Raft, Multi-Paxos, and Chain Replication.
To send a message to a receiver, you have to do almost the same replication with a message lease. The queue server reserves the message for a certain period of time; it sends the message to the receiver, and if/when the receiver ackowledges receipt of the message the server deletes the message. Otherwise, the servers will resend the message to the next available receiver.
Some message queues stop there, others add lots of bells and whistles. SQS is one queue implementation that doesn't add many bells and whistles so that it can scale more. It allows them, for example, to shard the queue so that one SQS queue is actually made of many—even thousands—of these queues as described above. As an aside, I once heard one SQS developer ask another "What does 'ordering' mean when you are accepting millions of messages per second?"
That being said, some queues do provide strong ordering guarantees. (I have implemented a couple of these types of systems.) The cost of this is less ability to scale. To maintain ordering the queue's complexity goes way up. The queue has to maintain an ordered log of all the messages, and have the same ordering replicated across its servers. This is much much harder than unordered replication. Ordered queue systems typically elect a master to maintain the ordering and all messages are routed to the master. They also tend to use the more complex protocols for replication.