I am writing an acceptor application and using a persistent FIX session. I am trying to write a recovery mode, such that if I go offline or my program restarts, when I reconnect I want to reprocess all the messages sent to me during the day to get back to the current state.
To do this, when I start up I send a resend request for all messages to the server. They fire me back all the relevant messages, and they are marked possdupflag=Y and possresend=Y. Before each message, they send a sequence reset for the repeated message they are about to send.
The problem is though, these messages do not seem to be processed by my message cracker. Both fromAdmin and fromApp do not get these messages. I assume they are being ignored because of the dup flag and/or resend. So is there a way for me to tell QuickFIX that I want to see these messages?
On that note- if anyone has any recommendations on better recovery processes I would be open to them.
Thanks.
There's at least a couple of potential problems with this recovery strategy. The first is that it's not very friendly to your trading counterparty. If you only receive a small number of messages during your session then it may not be an issue, but if you receive hundreds of thousands of messages then your counterparty might complain about the massive resends.
The other issue is that message resend is intended for error recovery and is managed by the session protocol layer. In QuickFIX/J (and other FIX engines) the session maintains recovery state in addition to sending the ResendRequest automatically when it detects a sequence number gap. Your approach might work if you reset the next expected incoming sequence number to 1. When the session receives the next message with a higher sequence number it will detect the gap and request the missing messages. If the messages are validated, they will be forwarded to application layer with the PossDup flag set. If you send the ResendRequest message yourself the behavior is undefined since the session state will not have been set up properly.
I recommend using a MessageLog implementation to store your incoming messages in a form you can use for recovery when your application starts. You can look at the implementation of the existing message logs (FileLog, JdbcLog) to get some ideas.
The behaviour occurs because the engine's persistance system tells it that the recieved messages are resent messages and so (per the FIX protocol specification) are discarded. Here we save FIXml strings into our database to provide a similar recovery ability to that which you describe(they are also written to xml files on disk for other reasons). I don't believe that there is any way to tell quickfix that you want to see duplicate messages but it is probably better to use a different form or persistance to save on connection overheads. Quickfix does provide a way of outputting messages to file as they come in if that helps.
I too have the same issue and What Frank Says is absolutely correct ,
Just use the below method to set the target sequence number to the begin seq number of the desired resend req .
getSession()->setNextTargetMsgSeqNum(atoi(seq.c_str()));
The engine internally identifies that the target number is way too large and automatically sends resend request , and all messages will be captured in onMessage call back itself as usual
Related
We are using googles steaming pull subscriber the design is as follows
We are doing
sending file from FE (frontend) to BE (backend)
BE converting that file to ByteArray and publishing to pubsub topic as message (so ByteArray going as message)
Topic sending that message to subscriber, subscriber converting the ByteArray to file again
that converted file subscriber sending to that tool
tool doing some cool stuff with file and notify the status to subscriber
that status going to BE and BE update the DB and sending that status to FE
Now in our subscriber when we receive message we are immediately acknowledge it and removing the listener of subscriber so that we don't get message any more
and when that tool done that stuff, it sending status to subscriber (we have express server running on subscriber) and
after receiving status we are re-creating listener of subscriber to receive message
Note
that tool may take 1hr or more to do stuff
we are using ordering key to properly distribute message to VM's
this code is working fine but my question is
is there any flaw in this (bcz we r removing listener then again re creating it or anything like that)
or any better option or GCP services to best fit this design
or any improvement in code
EDIT :
Removed code sample
I would say that there are several parts of this design that are sub-optimal. First of all, acking a message before you have finished processing it means you risk message loss. What happens if your tool or subscriber crashes after acking the message, but before processing has completed? This means when the processes start back up, they will not receive the message again. Are you okay with requests from the frontend possibly never being processed? If not, you'll want to ack after processing is completed, or--given that your processing takes so long--persist the request to a database or to some storage and then acknowledge the message. If you are going to have to persist the file somewhere else anyway, you might want to consider taking Pub/Sub out of the picture and just writing the file to storage like GCS and then having your subscribers instead read out of GCS directly.
Secondly, stopping the subscriber upon each message being received is an anti-pattern. Your subscriber should be receiving and processing each message as it arrives. If you need to limit the number of messages being processed in parallel, use message flow control.
Also ordering keys isn't really a way to "properly distribute message to VM's." Ordering keys is only a means by which to ensure ordered delivery. There are no guarantees that the messages for the same ordering key will continually go to the same subscriber client. In fact, if you shut down the subscriber client after receiving each message, then another subscriber could receiving the next message for the ordering key since you've acked the earlier message. If all you mean by "properly distribute message" is that you want the messages delivered in order, then this is the correct way to use ordering keys.
You say you have a subscription per client, then whether or not that is the right thing to do depends on what you mean by "client." If client means "user of the front end," then I imagine you plan to have a different topic per user as well. If so, then you need to keep in mind the 10,000 topic-per-project limit. If you mean that each VM has its own subscription, then note that each VM is going to receive every message published to the topic. If you only want one VM to receive each message, then you need to use the same subscription across all VMs.
In general, also keep in mind that Cloud Pub/Sub has at-least-once delivery semantics. That means that even an acknowledged message could be redelivered, so you do need to be prepared to handle duplicate message delivery.
Is there a way to log (in a file or database table) the messages sent to and/or received from a Service Broker service without altering the activation procedure or message sending code?
I know I can alter the activation procedure to write the received message to a table before (or after) processing it. And I can change the sending stored procedure to do the same but I am looking for something that:
I won't have to build the infrastructure on my own and code and debug from scratch.
Has been extensively tested and known to not interfere with normal processing (transparent).
I can enable or disable at will.
Note: I need to log the actual message content and associated metadata.
TIA
Short answer: No.
You can monitor the Broker:Conversation Event Class via a server trace or as a event notification but will only give you metadata, not the message content.
You can enable queue retention but the retained messages are deleted anyway when the conversation ends and is problematic to turn on/off.
The best option, by far, is to do it from the activated stored procedure itself.
Our system generates some messages (unsolicited cancel for example) it needs to send to the other party after a disconnect/connection lost, as soon as the connection recovers.
The problem is that we trigger sending those in onLogon(), but if there's a Resend Request that's too early and we had problems (maybe just because of how is implemented on the other end) when we had too many messages to send (hundreds).
I'm aware that ResendRequest may not come and it is impossible to figure that out without simply waiting, but what would be the best approach for us using QuickFIX/J to send our messages as soon as possible but after sequence numbers are synchronized?
EDIT: I'm trying to solve this using FIX 4.2. FIX 4.4 actually introduced http://www.onixs.biz/fix-dictionary/4.4/tagNum_789.html which would solve my problem (as long as the other party sends this optional tag too).
Thanks
My 10 cents is it sounds like you're trying to treat 2 scenarios in 1 go, and that's difficult. Do 1 thing at a time. For example, if it's your network that causes you to disconnect, before your client knows you've disconnected, your clients will send resend requests, right? Meanwhile, if a client disconnects but you don't then when they reconnect you gap fill. You've got to look carefully at the scenarios. Yes, a resend request may not come at all, it all depends how the client configures things their side. Maybe, per this question you want to send sequence resets because actually, the messages you're trying to send are quotes, right? I mean, what kind of messages are you trying to resend after a disco?
A Quickfix client validates incoming messages using XML spec files. If a message fails validation, quickfix automatically sends a rejection response. AFAIK in this case quickfix does not call the standard callback for incoming messages fromApp(), so up till now I was unable to programatically capture these erroneous incoming messages and handle them.
Is there a way to capture incoming FIX messages which fail quickfix validation?
Of course they may appear in the default quickfix log files, but I would rather capture them in my code in realtime.
There is not.
QuickFIX simply does not consider this a useful feature. If a message is invalid, QF performs the protocol-specified behavior and there is nothing that the application could or should do to recover. Any fix will require developer analysis and xml and/or code fixes, thus log files are sufficient to record the issue.
If you would like an automated alert when such errors occur, I suggest perhaps some kind of external log monitoring app that could watch your logs for occurrences of 35=3 or 35=j. (On the cheap side, a composition of cron/grep actions could do this very easily.)
Validation via XML spec file is in session level processing.
So, there is not suitable hook for this.
On the other hand, there are some configuration parameters;
UseDataDictionary : eliminates validation
ValidateUserDefinedFields : eliminates user defined field's validation
look for detailed descriptions
edit:
If your real problem is monitoring rejections, capturing Reject(3) and BusinessReject(j) messages at toAdmin() hook is sufficient.
What is the purpose of the message store structure in quickfix? I understand that you can log all incoming and outgoing fix messages via the message store interface and quickfix provides multiple implementations like file store etc.
My question is why do you even care about the message store other than logging your fix messages for the record?
You are confusing the MessageStore and the Log, which are two different things.
The MessageStore is for internal engine use. It tracks the current incoming and outgoing message sequence numbers, session start time, and other stuff. If your app goes down for whatever reason, when it restarts, it uses the MessageStore to resume where it left off with regards to sequence number and whether to reset the session.
The Log, however, is just a log. The engine doesn't really care about it. It's for the developers.