Please, Can you help me?
There are two services which send messages to a remote queue using NServiceBus.
The first service is located in the first domain (server1).
The second service is located in the second domain (server2).
The remote queue is located at the third server (server3) in the second domain.
I can send messages from server2 to server3.
But when I try to send messages from server1 to server3 I always get this error:
Common.Logging.ConfigurationException: The destination queue 'queue#server3' could not be found. You may have misconfigured the destination for this kind of message (Messages.Message) in the MessageEndpointMappings of the UnicastBusConfig section in your configuration file. It may also be the case that the given queue just hasn't been created yet, or has been deleted. ---> NServiceBus.Unicast.Queuing.QueueNotFoundException ---> System.Messaging.MessageQueueException: Insufficient resources to perform operation.
at System.Messaging.MessageQueue.SendInternal(Object obj, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
at NServiceBus.Unicast.Queuing.Msmq.MsmqMessageSender.NServiceBus.Unicast.Queuing.ISendMessages.Send(TransportMessage message, Address address)
--- End of inner exception stack trace ---
at NServiceBus.Unicast.Queuing.Msmq.MsmqMessageSender.NServiceBus.Unicast.Queuing.ISendMessages.Send(TransportMessage message, Address address)
at NServiceBus.Unicast.UnicastBus.SendMessage(IEnumerable`1 addresses, String correlationId, MessageIntentEnum messageIntent, Object[] messages)
--- End of inner exception stack trace ---
at NServiceBus.Unicast.UnicastBus.SendMessage(IEnumerable`1 addresses, String correlationId, MessageIntentEnum messageIntent, Object[] messages)
at NServiceBus.Unicast.UnicastBus.SendMessage(Address address, String correlationId, MessageIntentEnum messageIntent, Object[] messages)
at NServiceBus.Unicast.UnicastBus.NServiceBus.IBus.Send(Object[] messages)
Both domains are located in the same network. I can ping server3 from server1, so I don't understand why NServiceBus can't find the destination queue.
I've tried to use MSMQ direct format to specify the destination queue address. But NServiceBus doesn't support it.
User has answered the question him self with:
don't know what the reason of this issue was. I've tried everything.
All settings were valid. It started to work only after I've rebooted
server1
Related
I use ActiveMQ Artemis 2.10.1 and getting message listener thread hanging issue.
Thread is going into TIMED_WAITING and recover only after client JVM restart. This is an indeterminate issue and not able to reproduce easily. Client library version is 2.16.0.
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at org.apache.activemq.artemis.core.client.impl.LargeMessageControllerImpl.waitCompletion(LargeMessageControllerImpl.java:301)
- locked <0x000000050cd4e4f0> (a org.apache.activemq.artemis.core.client.impl.LargeMessageControllerImpl)
at org.apache.activemq.artemis.core.client.impl.LargeMessageControllerImpl.saveBuffer(LargeMessageControllerImpl.java:275)
- locked <0x000000050cd4e4f0> (a org.apache.activemq.artemis.core.client.impl.LargeMessageControllerImpl)
at org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.checkBuffer(ClientLargeMessageImpl.java:159)
at org.apache.activemq.artemis.core.client.impl.ClientLargeMessageImpl.getBodyBuffer(ClientLargeMessageImpl.java:91)
at org.apache.activemq.artemis.jms.client.ActiveMQBytesMessage.readBytes(ActiveMQBytesMessage.java:220)
at com.eu.jms.JMSEventBus.onMessage(JMSEventBus.java:385)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:746)
at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:684)
at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:651)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:317)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:255)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1166)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1158)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1055)
at java.lang.Thread.run(Thread.java:748) ```
The client is waiting in LargeMessageControllerImpl.waitCompletion. This wait will not block forever. The code waits in a loop for packets of a large message. As long as packets of the large message are still arriving the client will continue to wait until all the packets have arrived or if a packet doesn't arrive within the given timeout it will throw an error. The timeout is based on the callTimeout which is configured on the client's URL. The default callTimeout is 30000 (i.e. 30 seconds).
My guess is that your client is receiving a very large message or the network has slown down or perhaps a combination of these things. You can turn on TRACE logging for org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl to see the individual large message packets arriving at the client if you want more insight into what's happening.
To be clear, it's not surprising that thread dumps show your client waiting here as this is the most likely place for the code to be waiting while it receives a large message. It doesn't mean the client is stuck.
Keep in mind that if there is an actual network error or loss of connection the client will throw an error. Also, the client maintains an independent thread which sends & receives "ping" packets to & from the broker respectively. If the client doesn't get the expected ping response then it will throw an error as well. The fact that none of this happened with your client indicates the connection is valid.
I would recommend checking the size of the message at the head of the queue. The broker supports arbitrarily large messages so it could potentially be many gigs which the client will happily sit and receive as long as the connection is valid.
I use ActiveMQ Artemis 2.17.0, and I'm looking to avoid message loss in a producer during fail-over.
Message publish loss handled during Artemis active to passive switch by catching ActiveMQUnBlockedException and sending the message again.
The brokers are configured as active/passive HA shared-store. Active node configured in host1 and passive node configured in host2.
Url is:
(tcp://host1:61616,tcp://host2:61616)?ha=true&reconnectAttempts=-1&blockOnDurableSend=false
blockOnDurableSend set to false for high throughput.
During active to passive switch publishing code throws ActiveMQUnBlockedException but not during passive to active switching.
We're using Spring 4.2.5 and CachingConnectionFactory for connection factory.
I'm using the following code to send messages:
private void sendMessageInternal(final ConnectionFactory connectionFactory, final Destination queue, final String message)
throws JMSException {
try (final Connection connection = connectionFactory.createConnection();) {
connection.start();
try (final Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
final MessageProducer producer = session.createProducer(queue);) {
final TextMessage textMessage = session.createTextMessage(message);
producer.send(textMessage);
}
} catch (JMSException thr) {
if (thr.getCause() instanceof ActiveMQUnBlockedException) {
// consider as fail-over disconnection, send message again.
} else {
throw thr;
}
}
}
In host1 machine, Artemis deployed as master - node1.
In host2 machine, Artemis deployed as slave - node2.
following steps I did to simulate fail-over
node1 and node2 started
node1 started as live server and node2 started as backup server
killed node1, node2 become live server
client publish code threw ActiveMQUnBlockedException and handled to send message again
started node1 again. node1 become live server and node2 become backup again
client publish code did not throw ActiveMQUnBlockedException and loss in message.
Getting following error stack during step #3. ( Killed node1 and node2 become Live server).
javax.jms.JMSException: AMQ219016: Connection failure detected. Unblocking a blocking call that will never get a response
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:540)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:434)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sessionStop(ActiveMQSessionContext.java:470)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.stop(ClientSessionImpl.java:1121)
at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.stop(ClientSessionImpl.java:1110)
at org.apache.activemq.artemis.jms.client.ActiveMQSession.stop(ActiveMQSession.java:1244)
at org.apache.activemq.artemis.jms.client.ActiveMQConnection.stop(ActiveMQConnection.java:339)
at org.springframework.jms.connection.SingleConnectionFactory$SharedConnectionInvocationHandler.localStop(SingleConnectionFactory.java:644)
at org.springframework.jms.connection.SingleConnectionFactory$SharedConnectionInvocationHandler.invoke(SingleConnectionFactory.java:577)
at com.sun.proxy.$Proxy5.close(Unknown Source)
at com.eu.amq.failover.test.ProducerNodeTest.sendMessageInternal(ProducerNodeTest.java:133)
at com.eu.amq.failover.test.ProducerNodeTest.sendMessage(ProducerNodeTest.java:110)
at com.eu.amq.failover.test.ProducerNodeTest.main(ProducerNodeTest.java:90)
The ActiveMQUnBlockedException you're getting is coming from Spring's invocation of javax.jms.Connection#stop. It's not related to sending a message. Re-sending a message when you get this specific exception could result in a duplicate message.
Ultimately your problem is directly related to setting blockOnDurableSend=false. This tells the client to "fire and forget." In other words the client won't wait for a response from the broker to ensure the message actually made it successfully. This lack of waiting increases throughput but decreases reliability.
If you really want to mitigate potential message loss you have two main options.
Set blockOnDurableSend=true. This will reduce message throughput, but it's the simplest way to guarantee the message arrived at the broker successfully.
Use a CompletionListener. This will allow you to keep blockOnDurableSend=false, but the application will still be informed if there are problems sending the message although the information will be provided asynchronously. This feature was added in JMS 2 specifically for this kind of scenario. See the JavaDoc for more details.
I have enabled message delivery logs on our Artemis instances using broker plugins according to this page. To draw some analytics by mapping end to end message delivery and receipt timings between publisher -> artemis server -> subscriber, I'm trying to see if the contents of the message that are being logged to artemis log file (To be specific Message ID) can be accessed by the publishing and subscribing .NET applications we have. Below are logs from the artemis.log file for a message with MessageId indicating various events.
20:50:24,552 INFO
[org.apache.activemq.artemis.core.server.plugin.impl] AMQ841010:
routed message with ID: 2231685496, result: OK
20:50:24,552 INFO
[org.apache.activemq.artemis.core.server.plugin.impl] AMQ841009: sent
message with ID: 2231685496, session name:
9d9c035b-176e-11ea-ab75-020ff9805db8, session connectionID: 68a7ec34,
result: OK
20:50:24,553 INFO
[org.apache.activemq.artemis.core.server.plugin.impl] AMQ841012:
delivered message with message ID: 2231685496, to consumer on address:
News.Source.T, queue: f0586137-5ad3-4c77-b2c7-5b68daad672c, consumer
sessionID: fcbcd194-3295-11ea-a2c0-0a89c5c4c02a, consumerID: 0
20:50:24,554 INFO
[org.apache.activemq.artemis.core.server.plugin.impl] AMQ841014:
acknowledged message ID: 2231685496, messageRef sessionID:
fcbcd194-3295-11ea-a2c0-0a89c5c4c02a, with messageRef consumerID: 0,
messageRef QueueName: f0586137-5ad3-4c77-b2c7-5b68daad672c, with
ackReason: NORMAL
We are using AMQPNetLite for this and haven't found anything that can help us tie these messages sent and received to the logs that are being written to the artemis.log file. I've been looking to understand if there is a way to get a hold of the MessageId from these logs on the publisher application. Any pointers on this topic are much appreciated.
Messaging clients can't get data from the broker's log files since that data is just in a text-based log and not actually in the message broker itself. However, you could use something like the NotificationActiveMQServerPlugin which, instead of logging this information, will actually send messages with this information to the management notification address. Clients can create subscriptions on the management notification address and receive the messages and then take action based on that information. The notification messages may not contain all the information you need, but you can easily extend this class to create your own plugin which includes all the information you need.
I implemented a imap idle mail receiver with spring integration mail 4.2.6.
When, in the configuration of the receiver in the tag
<mail:imap-idle-channel-adapter mail-filter-expression="something" ... >
i use the attribute "mail-filter-expression" this causes continuous ( about every second) polling of the remote mail service with this repeated message:
10:29:21 INFO o.s.i.mail.ImapMailReceiver - attempting to receive mail from folder [INBOX]
If i remove the attribute "mail-filter-expression" this continuos polling disappears.
I don't know if this is the desired behavior of the framework.
My idea is to limit this comtinuos polling of the remote service, in fact i'm using the imap-idle-channel-adapter
this is my configuration of the receiver:
<mail:imap-idle-channel-adapter
id="mailAdapterMailInAccettazione"
store-uri="imaps://...the url of the remote service
channel="emailsInAccettazioneDaLeggere"
auto-startup="true"
should-delete-messages="false"
should-mark-messages-as-read="false"
java-mail-properties="javaMailProperties"
mail-filter-expression="subject matches '(?i).*somenthing: .*'" >
<mail:transactional synchronization-factory="syncFactoryAccettazione"/>
</mail:imap-idle-channel-adapter>
I want to cofigure the receiver in idle mode but using the mail-filter-expression attribute because i want to read only some emails.
UPDATE : after further investigation i discovered that the remote mail service doesn't support RECENT and USER flag:
This email server does not support RECENT or USER flags
So this is what was happening:
mail-filter-expression caused messages to get filtered, but no message was flagged as RECENT = false. This caused
continuous attempts to read the messages as all messages always appeared "
new"
.
I re implemented the flow not using the mail-filter-expression but filtering messages in subsequent channels
When MSMQManagementClass.Init(Object& Machine, Object& PathName, Object& FormatName) method is called it works fine when the client and msmq service are both on the same box.
When the client and msmq service are in different domain i get the following error
System.Runtime.InteropServices.COMException (0xC00E000B): The Message
Queuing service is not available at
MSMQ.MSMQManagementClass.Init(Object& Machine, Object& PathName,
Object& FormatName)
on windows 2008 server
Below code gives the error
var msmq = new MSMQManagement();
var machineName = MachineName
var pathName = null
var formatName = DIRECT=TCP:ipaddress\private$\testq
msmq.Init(ref machineName, ref pathName, ref formatName);
but
this works fine
var queue = new MessageQueue(FormatName:DIRECT=TCP:ipaddress\private$\testq)
Do you mean cross-domain or cross-forest?
Cross-forest MSMQ? You need to be trusting
You are performing an RPC operation to a remote machine so the following should help:
Understanding how MSMQ security blocks RPC traffic
Cheers
John
To use MSMQ it requires that MSMQ is installed on both the sender and receiver.