I'm trying to replace some bespoke message queues with ActiveMQ, and I need to talk to them (a lot) from Perl. ActiveMQ provides a Stomp interface and Perl has Net::Stomp, so this seems like it should be fine, but it's not.
Even if I send a BEGIN frame over Stomp the messages sent with SEND are immediately published, and if I ABORT the transaction nothing happens.
I can't find any clear answers suggesting that it's not possible, that it is possible, or that there's a relevant bit of configuration. Also, Stomp doesn't seem to be a great protocol for checking for error responses from the server.
Am I out of luck?
BTW the best place to ask Perl/ActiveMQ/Stomp questions is the ActiveMQ user forum as lots of Perl-Stomp folks hang out there.
The trick with STOMP transactions is to make sure each message you send or each acknowledgement you make includes the transaction ID header. See the transaction handling section of the STOMP protocol.
The reason for this is that with STOMP you could have many transactions taking place at the same time if your client is multi threaded - along with some non-transacted operations.
Have a look at Net::Stomp::Receipt. It's a subclass of Net::Stomp that implements "return receipts" from the Stomp protocol, and allow you to make sure the correct reception of your message, and abort the transaction otherwise.
You have to wrap the acknowledgements inside a transaction.
In pseudocode (or pseudo STOMP) this would be:
BEGIN [TRANSACTION-ID] -> send to server
MESSAGE [MESSAGE-ID] (received) <- received from server
ACK [MESSAGE-ID] [TRANSACTION-ID] -> send to server
COMMIT [TRANSACTION-ID] -> send to server
I have already gotten this working with the PHP driver (patching the abort call to use the transaction ID when I pass in a frame object to acknowledge).
Unfortunately, after redelivering four messages the client stops. At least this happens to me.
Related
I am using InfiniBand/RDMA for this. My client machine is sending a block to the server (via RDMA Send), and I want the client to get back a special 8 byte acknowledgement. I know the server could send it along separately, but that would duplicate work since my understanding for RDMA Send, the NIC is already sending some form of ACK via the completion queue. However, I was looking at InfiniBand docs, the completion queue doesn't seem to have an immediate along with it, just an id (which is itself less than 8 bytes, so there aren't any funny tricks I could play). I was wondering if I could get suggestions on how to do this?
Thank you!
I'm not sure I fully understand the question, but on the client side, a completion queue entry will be generated when the send work request is fully executed. "Fully executed" means that the RDMA-level ACK has been received from the server in your case. The work request ID in that completion entry is indeed 64 bits, and it is common to use it for a pointer to an auxiliary structure holding whatever info you want for the work request.
If you want the server to actively send back some data that it chooses after it receives the send from the client, then I don't think there's any way around having the server post a work request to do that.
I'm currently developing a bot with POE::Component::IRC whose job, amongst other things, is to post a notice to a list of channels on a schedule, for one week.
I can't seem to find a way to check that the message has been successfully sent to a channel though. The old Net::IRC package would fire a message received event for every message sent to a channel, including ones it itself had sent. POE seems not to do this - at least, the irc_public event is not firing when the bot's own message is published on the channel.
Is there a flag I can pass to the event handler to say "I'd really like to receive all messages please, even my own"? Or is there a way to do this with some sort of RAW event handler?
The IRC protocol does not echo your PRIVMSGs back to you, so you just have to trust that the server received your message and handled it the way it should.
If you just want to receive POE events for messages you send, there's a plug-in for that: POE::Component::IRC::Plugin::BotTraffic. It doesn't actually do anything to verify that the messages ever reach the server, though.
Fortunately, IRC runs on top of TCP, which provides guaranteed in-order delivery. Thus, as long as the connection doesn't drop or hang indefinitely, you can pretty safely assume that your commands will reach the server.
If you wanted to be absolutely sure, you could always follow your PRIVMSG with some command, such as TIME or PING, that you know the server will reply to; if it does, you'll know that it received your PRIVMSG too. Of course, even then there's still no guarantee that the server actually passed the message on to the intended recipient(s); things like netsplits do occur from time to time, and can cause messages to be dropped.
It's not clear how to use jain SIP stack in mutli-thread environment. I need to create multiple SIP sessions from different threads, e.g each client should be proceeded in its own transaction. Below is few options:
Use single SipProvider for receiving and sending SIP requests and do multiplexing on application side. SipProvider is not thread-safe, hence sending requests requires proper locking.
Create new SipProvider and new ListeningPoint for each client. This is how it works for me now. However, I don't really like it. And it's not clear, whther SipStack threadsafe or not
Create new instance of SipStack for every client
Its been a long time since I thought about JAIN-SIP (or even SIP for that matter or even Java) but here goes:
Set the re-entrant listener flag when you create the stack. (look up the javadocs). Specify a thread pool size. When a sip request or response comes along, the stack may potentially create a new thread for you and invoke your listener.
Your critical section is the SipListener implementation. You should not block for ever in it - otherwise new inbound requests and responses will not be routed to the sip listener for the transaction that is being processed at the time you blocked.
Hope that answers your question. Happy hacking.
Thats it.
why don't you ue SIP Servlets, it lets you focus on your application logic and handles those details for you ?
See http://code.google.com/p/sipservlets/
This is a rather general question about TCP sockets. I got a client/server application setup where messages are sent over the wire via TCP. The implementation is done via C++ POCO, however the question is not related to a certain technology.
A message can be a request (initiated by the client) or a response (initiated by the server).
A request has the structure:
Message Header
Request Header
Parameters
A response has the structure
Message Header
Response Header
Parameters
I know TCP guarantees that sent packages will be delivered in the order they have been sent. However, nothing can be assumed about the timespan a delivery might need.
On both sides I have a read/send timeout configured. Now I wonder how to have a clean set up on the transmitted data after a timeout. Don't know how to express this in the right terms, so let me describe an example:
Server S sends a response to the client (Message Header, Response Header, Parameters are put into the stream)
Client C receives the message header partially (e.g. the first 4 bytes of 12)
After these 4 bytes have been received, the reception timeout occurs
On client-side, an appropriate exception is thrown, the reception will be stopped.
The client considers the package as invalid.
Now the problem is, when the client tries to receive another package, he might receive the lasting part of the "old" response message header. From the point of view of the currently processed transaction (send request/get response), the client receives garbage.
So it seems that after a timeout has occured (no matter whether it has been on client or server-side), the communication should continue with a "clean setup", meaning that none of the communication partners will try to send some old package data and that no old package data is stored within the stream buffer of the respective socket.
So how are such situations commonly handled? Is there some kind of design pattern / idiomatic way to solve this?
How are such situations handled within other TCP-based protocols, e.g. HTTP?
In all the TCP samples around the net I've never seen an implementation that deals with those kind of problems...
Thank you in advance
when the client tries to receive another package, he might receive the lasting part of the "old" response message header
He will receive the rest of the failed message, if he receives anything at all. He can't receive anything else, and specifically data that was sent later can't be received before or instead of data that was sent earlier. It is a reliable byte-stream. You can code accordingly.
the communication should continue with a "clean setup", meaning that none of the communication partners will try to send some old package data
You can't control that. If a following message has been written to the TCP socket send buffer, which is all that send() actually does, it will be sent, and there is no way of preventing it short of resetting the connection.
So you either need to code your client to cope with the entire bytestream as it arrives or possibly close the connection on a timeout and start again.
I need to work with MSMQ (Microsoft Message Queuing). What is it, what is it for, how does it work? How is it different from web services?
With all due respect to #Juan's answer, both are ways of exchanging data between two disconnected processes, i.e. interprocess communication channels (IPC). Message queues are asynchronous, while webservices are synchronous. They use different protocols and back-end services to do this so they are completely different in implementation, but similar in purpose.
You would want to use message queues when there is a possibility that the other communicating process may not be available, yet you still want to have the message sent at the time of the client's choosing. Delivery will occur the when process on the other end wakes up and receives notification of the message's arrival.
As its name states, it's just a queue manager.
You can Send objects (serialized) to the queue where they will stay until you Receive them.
It's normally used to send messages or objects between applications in a decoupled way
It has nothing to do with webservices, they are two different things
Info on MSMQ:
https://msdn.microsoft.com/en-us/library/ms711472(v=vs.85).aspx
Info on WebServices:
http://msdn.microsoft.com/en-us/library/ms972326.aspx
Transactional Queue Management 101
A transactional queue is a middleware system that asynchronously routes messages of one sort of another between hosts that may or may not be connected at any given time. This means that it must also be capable of persisting the message somewhere. Examples of such systems are MSMQ and IBM MQ
A Transactional Queue can also participate in a distributed transaction, and a rollback can trigger the disposal of messages. This means that a message is guaranteed to be delivered with at-most-once semantics or guaranteed delivery if not rolled back. The message won't be delivered if:
Host A posts the message but Host B
is not connected
Something (possibly but not
necessarily initiated from Host A)
rolls back the transaction
B connects after the transaction is
rolled back
In this case B will never be aware the message even existed unless informed through some other medium. If the transaction was rolled back, this probably doesn't matter. If B connects and collects the message before the transaction is rolled back, the rollback will also reverse the effects of the message on B.
Note that A can post the message to the queue with the guarantee of at-most-once delivery. If the transaction is committed Host A can assume that the message has been delivered by the reliable transport medium. If the transaction is rolled back, Host A can assume that any effects of the message have been reversed.
Web Services
A web service is remote procedure call or other service (e.g. RESTFul API's) published by a (typically) HTTP Server. It is a synchronous request/response protocol and has no guarantee of delivery built into the protocol. It is up to the client to validate that the service has been correctly run. Typically this will be through a reply to the request or timeout of the call.
In the latter case, web services do not guarantee at-most-once semantics. The server can complete the service and fail to deliver a response (possibly through something outside the server going wrong). The application must be able to deal with this situation.
IIRC, RESTFul services should be idempotent (the same state is achieved after any number of invocations of the same service), which is a strategy for dealing with this lack of guaranteed notification of success/failure in web service architectures. The idea is that conceptually one writes state rather than invoking a service, so one can write any number of times. This means that a lack of feedback about success can be tolerated by the application as it can re-try the posting until it gets a 'success' message from the server.
Note that you can use Windows Communication Foundation (WCF) as an abstraction layer above MSMQ. This gives you the feel of working with a service - with only one-way operations.
For more information, see:
http://msdn.microsoft.com/en-us/library/ms789048.aspx
Actually there is no relation between MSMQ and WebService.
Using MSMQ for interprocess communication (you can use also sockets, windows messaging, mapped memory).
it is a windows service that responsible for keeping messages till someone dequeue them.
you can say it is more reliable than sockets as messages are stored on a harddisk but it is slower than other IPC techniques.
You can use MSMQ in dotnet with small lines of code, Just Declare your MessageQueue object and call Receive and Send methods.
The Message itself can be normal string or binary data.
As everyone has explained MSMQ is used as a queue for messages. Messages can be wrapper for actual data, object and anything that you can serialize and send across the wire. MSMQ has it's own limitations. MSMQ 1.0 and MSMQ 2.0 had a 4MB message limit. This restriction was lifted off with MSMQ 3.0. Message oriented Middleware (MOM) is a concept that heavily depends on Messaging. Enterprise Service Bus foundation is built on Messaging. All these new technologies, depend on Messaging for asynchronous data delivery with reliability.
MSMQ stands for Microsoft Messaging Queue.
It is simply a queue that stores messages formatted so that it can pass to DB (may on same machine or on Server). There are different types of queues over there which categorizes the messages among themselves.
If there is some problem/error inside message or invalid message is passed, it automatically goes to Dead queue which denotes that it is not to be processed further. But before passing a message to dead queue it will retry until a max count and till it is not processed. Then it will be sent to the Dead queue.
It is generally used for sending log message from client machine to server or DB so that if there is any issue happens on client machine then developer or support team can go through log to solve problem.
MSMQ is also a service provided by Microsoft to Get records of Log files.
You get Better Idea from this blog http://msdn.microsoft.com/en-us/library/ms711472(v=vs.85).aspx.