I have many queues, i.e
my.queue.no.1
my.queue.no.2
my.queue.no.3
my.queue.no.4
And I want to redirect unsuccessful messages to DLQ, but I don't want to mix messages from all queues to a one DLQ.
Is it possible to have multiple DLQs?
i.e
my.queue.no.1
my.queue.no.dlq.1
my.queue.no.2
my.queue.no.dlq.2
my.queue.no.3
my.queue.no.dlq.3
my.queue.no.4
my.queue.no.dlq.4
P.S I'm using Artemis 2.16.0
ActiveMQ Artemis can automatically create the defined dead-letter-address and a corresponding dead-letter queue when a message is undeliverable enabling the auto-create-dead-letter-resources address setting. The dead-letter-queue-prefix address setting can be used to define a prefix used for automatically created dead-letter queues, i.e. to create DLQ.my.queue.no.NNN queues under the DLA address:
<address-settings>
<address-setting match="my.queue.no.#">
<dead-letter-address>DLA</dead-letter-address>
<auto-create-dead-letter-resources>true</auto-create-dead-letter-resources>
<dead-letter-queue-prefix>DLQ.</dead-letter-queue-prefix>
...
Related
I use live/backup with shared-storage, and I use a non-transacted JMS session. I always send one message, and I always receive one message then acknowledge and receive second message only after successful first acknowledge.
I got this exception in my non-transacted session:
Execution of JMS message listener failed. Caused by: [javax.jms.TransactionRolledBackException - AMQ219030: The transaction was rolled back on failover to a backup server]
javax.jms.TransactionRolledBackException: AMQ219030: The transaction was rolled back on failover to a backup server
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollbackOnFailover(ClientSessionImpl.java:904)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:927)
at org.apache.activemq.artemis.jms.client.ActiveMQMessage.acknowledge(ActiveMQMessage.java:719)
It happens because the session was marked as "rollbackOnly". I got this state after the following steps:
I use Spring-JMS. Consumer session works 24/7 (infinite loop session.receive())
The Master Node crashed, then the Master node was restarted
After recovery (After a couple of hours), I sent a message to the queue. The consumer read the message and throw Exception on acknowledge(because was marked as rollback-only)
I read message again (this is not very bad for my task) but Redelivery Count has not been increased
My consumer code:
onMessage(Message message) {
if (redeliveryCount(message) > 0){
processAsDublicate(message); // It's not invoked - it is error in my business logic.
}
}
I migrated from another broker and and I thought not to change the client logic
Question:
How to avoid TransactionRolledBackException for Non-Transactional session? If this is not possible i should change consumer code?
Thank you in Advance
UPDATE AFTER ANSWER:
https://github.com/apache/activemq-artemis/tree/2.14.0/examples/features/ha/replicated-failback
This example is not suitable for my case - I don't have non-acknowledged messages. I got this state after the following steps: 1) Restart server 2) consume message 3) acknoledge message
We use a broker for ~30 applications (24/7) ~ 200 consumers in total
For example, on the weekend we restart the JMS Broker
Will all consumers start getting this exception after consume new messages
(They don't have non-acknowledged messages)
The TransactionRolledBackException is expected as you can see in the replicated-failback example.
To prevent a consumer from receiving the same message more times, an idempotent consumer must be implemented, ie Apache Camel provides an Idempotent consumer component that would work with any JMS provider, see: http://camel.apache.org/idempotent-consumer.html
I'm configuring an Apache Artemis message-broker. The broker will accept big files and downstream consumers access the topic to process the latest files. Now I'm wondering how to make the latest files available for dev-runs. Because the messages only arrive a few times a day, the test runs would need to access the last few sent messages and can't wait for the next.
For production and staging-systems, I found that durable subscriptions work fine. I've adapted an Apache Camel config to serve as an illustration. Here are two consumers that receive messages, each using a durable subscription:
<route id="inbox">
<from uri="file:inbox"/>
<to uri="activemq:topic:testing"/>
</route>
<route id="outbox-staging">
<from uri="activemq:topic:testing?clientId=staging&durableSubscriptionName=staging"/>
<to uri="file:outbox-staging"/>
</route>
<route id="outbox-production">
<from uri="activemq:topic:testing?clientId=production&durableSubscriptionName=production"/>
<to uri="file:outbox-production"/>
</route>
This is fine. If a consumer is offline it will pick up messages when it comes back online. Now if another consumer joins for testing;
<route id="outbox-testing" streamCache="true">
<from uri="activemq:topic:testing?clientId=my-local-consumer&durableSubscriptionName=my-local-consumer"/>
<to uri="file:outbox-local"/>
</route>
because the subscription didn't exist before, the consumer will have to wait for new messages. What I'm looking for is new subscribers to be immediately primed with the available messages. I found different names for the concept such as prefetchPolicy, consumerWindowSize, or "retroactive consumer". But it's unclear to me which terms apply to Apache Artemis and how to set them up because the examples mostly refer to Apache ActiveMQ.
How can a configure Artemis so that a consumer joining on a new subscription gets past messages?
The prefetchPolicy doesn't apply to ActiveMQ Artemis. It's for ActiveMQ 5.x.
The consumerWindowSize does apply to ActiveMQ Artemis.
However, neither prefetchPolicy nor consumerWindowSize apply to this situation as they're both related to "flow control" and have nothing to do with putting "missed" messages onto a JMS topic subscription.
The "retroactive consumer" feature is for ActiveMQ 5.x. A similar feature (called "retroactive address") will be available in ActiveMQ Artemis 2.11. It was implemented as part of ARTEMIS-2504.
Therefore you have a few options:
Wait for ActiveMQ Artemis 2.11 to be released (should be released in January).
Build your own version of ActiveMQ Artemis based on the master branch which includes the retroactive address feature.
Modify your test environment so that new subscribers don't have to wait so long for messages (e.g. send them more frequently).
I am using pykafka, I can get topic names but I cant send message. My code is shown below
client = KafkaClient(hosts='xx.xx.xx.xx:9092')
topic = client.topics['test']
producer = topic.get_sync_producer()
producer.produce(b"message")
And I get this error message
raise ProduceFailureError("Delivery report not received after timeout")
pykafka.exceptions.ProduceFailureError: Delivery report not received after timeout
broker.id=1
listeners=PLAINTEXT://localhost:9092
I am sending to an external IP message
If you are setting hosts=some.external.IP:9092, then you need to edit Kafka properties with advertised.listeners=PLAINTEXT://some.external.IP:9092 and make listeners=PLAINTEXT://:9092 to listen on external interfaces.
Listing the topics uses a different protocol, which is why it works fine.
I'm using the Virtual Topics concepts of Activemq 5.14.5 (https://activemq.apache.org/virtual-destinations) with MQTT protocol.
ActiveMQ will pick up the messages written to the topic and write them to a queue (or multiple queues or topics). See the ${ACTIVEMQ_HOME}/conf/activemq.xml configuration below:
<beans>
<broker>
...
<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<compositeQueue name="MY.QUEUE">
<forwardTo>
<queue physicalName="FOO" />
<topic physicalName="BAR" />
</forwardTo>
</compositeQueue>
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>
...
</broker>
</beans>
Using Mqtt.fx software (https://mqttfx.jensd.de/) I'm only able to dequeue from a Topic (BAR). How can I dequeue from a Queue (FOO) to see the messages arrived on it?
I am new tot this and learning about the protocol MQTT and broker Activemq.
I would like to enable message redistribution on my 2-nodes cluster with static hosts. But it does not seem to work.
1) I have 10 producers that write to the queue "MyTestQueue" on node 1 (but no consumers).
2) I have 1 consumer on node 2 (but no producers) that consumes messages from node 2.
I expect that node 1 will redistribute the messages to node 2 where the consumer exists, but it does not. The message count on node 1 is still equal the amount of messages that was sent to node 1.
I have the following configuration in my broker.xml that sets forward-when-no-consumers to false.
I also have set redistribution-delay to a value of zero.
<jms xmlns="urn:activemq:jms">
<queue name="MyTestQueue"/>
</jms>
...
<cluster-connections>
<cluster-connection name="my-test-cluster">
<address>jms</address>
<connector-ref>server0-connector</connector-ref>
<retry-interval>500</retry-interval>
<use-duplicate-detection>true</use-duplicate-detection>
<forward-when-no-consumers>false</forward-when-no-consumers>
<message-load-balancing>ON_DEMAND</message-load-balancing>
<max-hops>1</max-hops>
<confirmation-window-size>1024</confirmation-window-size>
<static-connectors>
<connector-ref>server1-connector</connector-ref>
</static-connectors>
</cluster-connection>
</cluster-connections>
...
<address-settings>
<address-setting match="#">
<redelivery-delay>5000</redelivery-delay>
<redelivery-delay-multiplier>3</redelivery-delay-multiplier>
<max-redelivery-delay>10000</max-redelivery-delay>
<max-delivery-attempts>10</max-delivery-attempts>
<max-size-bytes>104857600</max-size-bytes>
<page-size-bytes>10485760</page-size-bytes>
<address-full-policy>PAGE</address-full-policy>
<redistribution-delay>0</redistribution-delay>
</address-setting>
</address-settings>
How can I get the message redistribution to work?
This might be related to a known issue. There is a situation on which the broker fails to load balance the messages if they don't contain the application properties field.
Could you, please, try with that?
Couple of things...
What version of Artemis are you using? Have you tried reproducing this with version 2.2.0?
What kind of client are you using (e.g. JMS, AMQP, STOMP, etc.)?
Do you have a reproducible test-case (e.g. a modified version of one of the examples shipped with Artemis)?
The configuration element <forward-when-no-consumers> is not valid in Artemis (although it was in older versions of HornetQ).
Remove the <address>jms</address> from cluster connection configuration - each cluster connection only applies to addresses that match the specified address. And make sure that you're using compatible client because messages from 1.x clients to 2.x cluster are lost when the are load balanced to nodes with matching consumers.
Here's official, working example of ActiveMQ Artemis configuration with symmetric cluster, on demand load balancing and message redistribution