I have a quickfixj initiator connecting to vendor's acceptor and receiving messages. I keep the fix messages in a buffer which is processed by a thread. To avoid loosing the message in case crash with message in the buffer, I have the last seqnum processed, and plan to send resend message for that next seqnum on my side when I reconnect.
I know the better solution would be that I save the messages before I receives them, but the design is to avoid doing any db access in the onMessage call.
I didn't find any example how this could be done, resending request for a specific seqnum. Should I simply overload the logon message and send the seqnum?
Anyone has an example?
I guess you are already in synch as per the last thread if quickfixj crash in onmessage, will I lose my current message?.
QuickFixJ manages 2 sequence numbers:
SenderSequenceNum: Sequence number used in sending messages.
TargetSequenceNum: Sequence number expected to receive.
So you have two options:
Option 1: Process the receive messages on the QuickfixJ onMessage() callback thread. So that in case of an exception the sequence number does not increment. And QuickFixJ automatically sends the resend request on receiving next fix message as it will detect the sequence gap.
Option 2: Persist the sequence number that you have successfully processed. In case of crash, on restart you can set the expected receive sequence number using:
Session.lookupSession(session_).setNextTargetMsgSeqNum();
So if you receive a sequence number higher than that, QuickfixJ automatically sends resend the request.
Note: Do not change the sender sequence number then another party will receive a sequence number lower than expected and can cause disconnection.
Related
For a guaranteed message receiver, in an ACK-based protocol like Apache Kafka, TIBCO EMS/RVCM, IBM MQ and JMS there is a way to explicitly acknowledge the receipt of a message. Explicit Acks are not just automatically sent when you return from a dispatcher's callback but an extra method on the session or message to say "I've processed this message". The reason for the existence of this explicit ack is that you can safely queue received messages to be processed by another thread at a later time and then only call this explicit-ack method once your are really done processing this message (safely storing to DB, forwarding to another MOM, etc.) Having this explicit method ensures that you are not losing messages even when you crash after receiving messages but didn't process them yet.
Now with QuickFIX/J (of FIX in general) I know it's not ACK-based but instead persists the last received SeqNum in a file and instead of sendings Acks, message guarantee is achieved by sending ResendRequests for missed SeqNums. But still, is there a way to tell the QuickFIX/J API "I don't automatically want you to persist this last SeqNum once I exit this onMessage() callback but hold off until I tell you so". In other words is there a Session variation which doesn't persist SeqNums automatically and then I can call something on the FIX message to persist this last Seqnum once I've really processed/saved that message ?
(If this feature doesn't exist I think it would be a good addition to the API)
quickfixj initiator getting Disconnecting: Encountered END_OF_STREAM while trying to logon to the acceptor. We are using vendor's fix engine as acceptor. and feedback from acceptor is that logon request for xxxx was not accepted, incoming too small, expect 305, received 27.
I read the quickfix documentation but didn't get it exactly what's the proper solution for the sequence number mismatch. I understand that if I am disconnected, my initiator will send an 35=4 for resend with initiator side seqnum asking acceptor to resend the messages and fill up the gap.
But in what case, if initiator is sending a lower seqnum will be rejected by acceptor and refuse the connection?
And what's the proper procedure to handle this kind of rejection and reconnect? In order to not loose any message, how should both side do the reset and fill the gap?
In case there is a break between the initiator and acceptor, what's the recommended solution to keep the messages in sync and not loosing any?
Due to the first sentence of your question I would like to show you an answer to the same error message Disconnecting: Encountered END_OF_STREAM. There is a blog post by bhageera quoted.
In the end the reason was pretty silly… the counterparty I was connecting to allows only 1 connection per user/password (i.e. session with those credentials) at a time. As it turns out there was another application using the same credentials against the same TargetCompID. As soon as that application was killed off, the current one logged in fine.
I searched for the cause of the bug for a while, until I realized that I had two initiators with the same credentials running on two different test environments.
According to default logic in QuickfixJ:
QuickfixJ manages 2 sequence number, expectedSeqNum to receive(targetSeqNum) and nextSeqNumber to sent.
Check the next expected target SeqNum against the received SeqNum.If a mismatch is detected, apply the following logic:
if lower than expected SeqNum, logout
if higher, send a resend request
In your case received was lower than expected so it gets disconnected.
Reason for receiving higher than expected SeqNum:
Receiver misses some message so it could be a normal scenario.
Reason for lower than expected SeqNum(Your case):
One of the counterparties resets its sequence number, which is not expected it should be agreed by both the counterparties.
In a normal scenario, whenever you miss the message you will receive a higher number and it would be managed by QuickFixJ.
I am trying to implement Fix.4.2 protocol, but It is difficult to understand the message log I attached below. Here Logon(35=A) request was sent with MsgSeqNum(34=1) from client. Then for testing ResendRequest and SequenceReset session level messages I sent a NewOrderSingle request with MsgSeqNum=7 (instead of MsgSeqNum=2, as subsequent messages should have incremeted msgseqnum after logon request). As expected MsgSeqNum is too high than recieved one Fiximulator responded with a ResendRequest(35=2) to send from 2 to 0 (i.e., from 2 to 7). Here why the Fiximulator is not waiting for client's reply ? instead it is sending an heartbeat message. Why the client is sending ResendRequest in response to ResendRequest of Fiximulator instead of sending SequenceReset message ?.
Also explain remaining cases if possible.
Thanks in advance.
What is your status of ResetOnLogon in your config file for the acceptor ? Default value is N so it isn't being reset. Always check your config file or try debugging to figure out issues.
ResetOnLogon Determines if sequence numbers should be reset when recieving a logon request. Acceptors only
I have a QuickFIX/J application running as acceptor. ResetOnLogon is N in the configuration.
When the initiator is logged on, since the seq nums are different the initiator app sends the messages and I see those messages in the FIX log file. The first one of those message is passed to the application layer but the others are not, all are discarded.
What can be the reason that the messages are received but not passed to the application level?
The most likely reason for this is that the messages contain the PossDupFlag <43> with a 'Y' value, and a MsgSeqNum <34> that is infact recognized as a dupe by the engine. In that case you won't receive these as application level messages.
I have a strange problem on one of my clients workstation. I have a simple application that exchanges some data over network between two endpoints.
Basically the transaction goes like this:
Client A listens for incomming connection
Client B connects to A and sends some data
Client A read this data for further processing
Now the strange part is that client A does not receive whole data (sometimes it a part of buffer sometimes it is empty).
The A client uses WSAEventSelect function and waits for FD_READ to read data sent by B and for FD_CLOSE to detect disconnection.
Usually ( everytime except this one particular client) the FD_READ is signaled, data is processed and after that FD_CLOSE is signaled and all is fine, but here instead FD_READ i receive FD_CLOSE.
Can someone tell me how this is possible? Another thing is that program was working fine for about a year and suddenly it crashed.
Now the strange part is that client A does not receive whole data (sometimes it a part of buffer sometimes it is empty).
There's nothing strange about that, that's how TCP works, except that you will never receive zero bytes in blocking mode.
Usually ( everytime except this one particular client) the FD_READ is signaled, data is processed and after that FD_CLOSE is signaled and all is fine, but here instead FD_READ i receive FD_CLOSE.
Note that FD_READ can be signalled any number of times, not just once. You're not guaranteed to receive an entire message in a single read.
Can someone tell me how this is possible?
The client has closed the connection.
Quoting http://msdn.microsoft.com/en-us/library/windows/desktop/ms741576%28v=vs.85%29.aspx
"An application should check for remaining data upon receipt of FD_CLOSE to avoid any possibility of losing data."
So if the error code associated with the FD_CLOSE notification is 0, you should check to see if you still have data to read, that might be where your missing data is.
If the error code is NOT 0, then there was an error and the missing data is probably lost.