Is it possible for QuickFIX/J to remove some tags of the incoming FIX message? If yes, why would it be like that?
The problem is when receiving 35=X (MarketDataIncrementalRefresh) message. The message comes from the server, and has the corresponding tangs, including the tags 48 SecurityID and 22 SecurityIDSource and the checksum is the correct one, calculated including those 2 fields. When processing the message in the client, the message does no longer contain the 48 and 22 tags and the checksum is also different. Because of those 2 missing tags, that are required, the client cannot process the message. I couldn't trace the cause of this. (The data dictionary contains the tags 22 and 48 and are marked as required).
The session config:
[session]
BeginString=FIXT.1.1
SocketConnectPort=9122
UseDataDictionary=Y
AllowUnknownMsgFields=Y
ValidateFieldsOutOfOrder=N
ValidateUnorderedGroupFields=N
ValidateIncomingMessage=N
TransportDataDictionary=transport_dict_FIXT11.xml
AppDataDictionary=data_dict_FIX44.xml
SocketUseSSL=Y
EnabledProtocols=TLSv1.2
SocketKeyStore=file.jks
SocketKeyStorePassword=password
Example of message from log:
Failed to process 8=FIXT.1.1|9=145|35=X|34=226|49=SenderCompID|52=20201202-13:42:34.345|56=TargetCompID|262=EU000A1U99281083|268=1|279=0|264=1|269=0|278=EU000A1U9928_BID_1|110=1000|10=217:
quickfix.FieldNotFound: Field [48] was not found in message.
Related
When I send the order request on the FIX server as :
8=FIXT.1.1|9=179|35=D|34=34|49=135|52=20200206-04:52:04.406|56=PSE|1=11101401117577|11=1580964724079|38=1000|40=2|44=10.5|54=1|55=AGI|59=0|60=20200206-12:52:04.406|453=1|448=13501100|447=C|452=12|10=009|)
The response that I receive is:
8=FIXT.1.1|9=000362|35=8|49=PSE|56=135|34=34|57=13501100|52=20200206-04:52:04.533|37=NONE|11=1580964724079|453=4|448=13501999|447=C|452=11|448=13501999|447=C|452=36|448=13501100|447=C|452=12|448=135|447=C|452=1|17=1904|150=8|39=8|103=99|1=11101401117577|55=AGI|54=1|38=1000|44=10.5|59=0|528=F|151=0|14=0|60=20200206-04:52:04.356|58=(293): **Orders are not being accepted at this time|10=020|)**
With the PSE error as:
<20200206-04:52:04, FIXT.1.1:135->PSE, error> (Rejecting invalid message: quickfix.IncorrectTagValue: Value is incorrect (out of range) for this tag, field=528 field=528: 8=FIXT.1.1|9=362|35=8|34=34|49=PSE|52=20200206-04:52:04.533|56=135|57=13501100|1=11101401117577|11=1580964724079|14=0|17=1904|37=NONE|38=1000|39=8|44=10.5|54=1|55=AGI|58=(293): Orders are not being accepted at this time|59=0|60=20200206-04:52:04.356|103=99|150=8|151=0|528=F|453=4|448=13501999|447=C|452=11|448=13501999|447=C|452=36|448=13501100|447=C|452=12|448=135|447=C|452=1|10=132|)
<20200206-04:52:04, FIXT.1.1:135->PSE, error> **(Reject sent for Message 34: Value is incorrect (out of range) for this tag**, field=528:528)
According to the documentary the value for 1 should be 14 digit trading account(1=xxxxxxxxxxxxxx) but when I give that, then I had receive the above error.So what value should be given for 1 so the response should be receive.
So what is the solution for this error?
The flow is as follows:
you send a NewOrderSingle
they reply with an ExecutionReport that they don't accept orders at that time
you (and not PSE) reject that ExecutionReport because tag 528 on the ExecutionReport that PSE sent contains a value that is not contained in your data dictionary.
What you need to do is either of
Read PSE's rules of engagement (i.e. their FIX spec) and add all needed tags and values to your data dictionary.
Configure your session to not validate incoming messages (setting ValidateIncomingMessage=N) but this means you would need to do validation by yourself if required. E.g. if you wanted to retrieve values from tag 150/ExecType then you can be sure that only values which are allowed in the data dictionary reach your application. With disabled validation you would need to do these checks by yourself.
Alternatively you could set RejectInvalidMessage=N which also will not reject the message but only log the warning with the validation error.
enter image description hereCan someone please help me to solve the init abort happening from our server side.
Verification tag is always coming as 0 from the transmitter end. But receiver is giving some random value of verification tag.
Please help.
enter image description here
In general, it might happen that endpoint is not able to establish association and send ABORT chunk as response on INIT. This might happen in various situation (such as misconfiguration, when you haven't created endpoint on server side for that association; or lack of resources).
INIT chunk is a bit special (packet that carries it always has verification tag set to zero in common header). INIT chunk also has so called initiate tag - it is basically a verification tag that sender of the INIT chunk expect to see in all the packet it will receive in the scope of this association.
When ABORT is sent as a response on INIT chunk, it will be sent with verification tag set to the initiate tag from INIT. That is exactly what you can see in you wireshark log.
What seems to be strange in you log file is that the sender of ABORT chunk does not follow RFC 4960 in respect to using t-bit.
RFC 4960, chapter 8.4, bullet 3 says:
If, for whatever
reason, the INIT cannot be processed normally and an ABORT has to
be sent in response, the Verification Tag of the packet
containing the ABORT chunk MUST be the Initiate Tag of the
received INIT chunk, and the T bit of the ABORT chunk has to be
set to 0, indicating that the Verification Tag is NOT reflected.
In your case the sender of ABORT chunk is using initiate tag as verification tag (as defined in RFC), but it also sets t-bit to 1 - it is a violation of RFC. Since t-bit is set incorrectly in the packet that carries ABORT chunk, it stop the sender of INIT from handing it correctly. Basically the sender of INIT chunk cannot handle that ABORT.
The sender of ABORT could have also include cause code in ABORT chunk to provide more details about reason of ABORT. However for whatever reasons it hasn't been done and the actual reason of ABORT will remain mystery.
In conclusion:
It is hard to say why the ABORT has been sent based on information
provided. If you can - contact you vendor and suggest to include
cause code in the ABORT chunk as it defined in RFC in order to make
debugging easier.
I would guess that something has not been initialized/configured on you server side, that is why the association is not setup up. I
would start with checking sockets/endpoint configuration on server
side.
The server side of your association (receiver of INIT/sender of ABORT) violates RFC 4960 and set t-bit incorrectly. Because of that
the sender of INIT chunk cannot handle ABORT correctly. I would
suggest to submit a bug report for your server side
implementation/vendor in order get it fixed.
I am using QuickFix/J 1.14.3 with fix4.2 as both Initiator and Acceptor.
On Acceptor I tried to disable all Validation checks in the config file as seen below. The fromAdmin() is called with the violating message. I am not sure I want to mess with the data dictionary but I thought by disabling all checks I could actually get the message. The acceptor sets a field as follows.
newOrderSingle.setField(new IntField(6401, 79 ));
Any help would be appreciated.
UseDataDictionary=N
DefaultMarketPrice=12.30
ValidateUserDefinedFields=N
ValidateFieldsHaveValues=N
ValidateFieldsOutOfOrder=N
ValidateUserDefinedMessages=N
UseDataDictionary=N
# ACCEPTOR SIMULATOR
[session]
SenderCompID=baml
TargetCompID=mjt
BeginString=FIX.4.2
SocketAcceptPort=9879
CheckCompID=N
First of all add property
AllowUnknownMsgFields=Y
And use data dictionary it allows to process groups.
UseDataDictionary=Y
I need to disable quickfix validation.
Setting UseDataDictionary=N does not work.
Removing paths to dictionary files in the config file does not work.
Renaming xml files in the directory to some other extension does not work.
Still rejecting messages based on failed validation.
The directory is correct. Please help! I'm sure I am missing something trivial.
Thank you!
PS - in reply to some questions:
The error I am getting is: 58=Tag specified out of required order 371=143 372=8
Indeed, the client is sending tag 143 in the message body, not the header. The client will not change the code. I tried adding tag 143 to the ExecutionReport body, but that did not help. I am weary of taking tag 143 out of the header just for these guys.
PS - config file:
[DEFAULT]
ConnectionType=initiator
ReconnectInterval=60
FileStorePath=store
FileLogPath=log
StartTime=00:00:00
EndTime=00:00:00
UseDataDictionary=N
DataDictionary=/xxxxxx
TransportDataDictionary=/xxxxx
ResetOnLogon=N
DefaultApplVerID=9
[SESSION]
BeginString=FIX.4.4
SenderCompID=xxxxxx
TargetCompID=xxxxxx
SocketConnectHost=xxxxxxxxx
SocketConnectPort=52400
HeartBtInt=30
You can disable this specific check by setting the ValidateFieldsOutOfOrder parameter to N in your configuration file. See Configuring QuickFIX.
You have to have UseDataDictionary=Y if you have repeating groups, it tells QuickFIX how to handle items in repeated groups. "Tag specified out of required order 371=143 372=8" usually means that a header tag is in the message body, a body tag is in the header, a repeating group tag is in the body, a body tag is in a repeated group etc.. Since the client is sending tag 143 in the message body rather than the header you will need to edit the DataDictionary so that that tag appears in the message body instead of the header. Since you can set the DataDictionary on a per connection basis I would strongly recommend that you create a copy of the original dictionary, give it a client specific name and edit that file and then connect it to that connection keeping the original clean.
I was hoping someone can shed some light on how the Quickfixn engine handles outgoing FIX messages... I have an outgoing connection set up, and I'm getting heartbeats. When I generate an outgoing message however, it gets rejected because it says that tag 58 is invalid for this message type... (35=AE) ... Normally, if this was an inbound connection, I could just modify the Data Dictionary and everything would be fine... but seeing as how this is an outgoing connection, plus I have my
UseDataDictionary property set to 'N' ... what does the quickfix engine use to validate the outgoing message? Can something be changed to allow the engine to pass the message ? Or is the only resolution not to include this tag in my outgoing message?
Any help on this matter would be greatly appreciated.
Edit-
The message is getting rejected by the quickfix engine. The message that I'm constructing and the respective reject message are:
8=FIX.4.4 9=400 35=AE 34=38 49=XXX 52=20130528-23:11:04.040 56=YYY 31=1.3022 32=1000000.00 39=0 55=EUR/USD 58=ABCD 60=20130528-22:34:52.000 64=20130531 75=20130529 570=N 571=ABCD 5495=0 5971=1302200.00 552=1 54=2 37=ABCD 453=3 448=LP1-DBAB 447=D 452=17 448=XXX 447=D 452=1 448=XXX 447=D 452=19 15=EUR 120=USD 10=082
8=FIX.4.4 9=130 35=3 34=38 49=YYY 52=20130528-23:11:04.283 56=XXX 45=38 58=Tag not defined for this message type 371=58 372=AE 373=2 10=033
I've seen incoming messages get rejected by the quickfix engine because the data dictionary didn't have the correct specs for the message... I thought this might be the same thing but the outgoing connection doesn't seem to use the data dictionary.
Your FIX library does not reject a message. The message is sent to the counter-party instead, which then rejects your message as invalid upon receiving and validating it. And the reason for that is because tag 58, if present, must be a part of “NoSides repeating group (tag 552), which in your case it is not, making the message ill-formed. What you have to do is send a "logically" correct message. I recommend you refer to the appropriate FIX protocol specification for a reference on how to construct a correct message.
Vlad's answer is correct, but I want to alert you to one other danger in your question.
I have my UseDataDictionary property set to 'N'
I am 90% sure you don't want to do this. Whatever you think you're gaining by using =N is probably based on a misunderstanding of something.
Without a DD, you can't read messages with repeating groups, because the engine won't know what fields go in what group.
In practice, every venue uses repeating groups. Therefore, you'll need to set UseDataDictionary=Y and you need to specify an xml file with DataDictionary=<file>.
The only reason we allow =N in QF/n is to be consistent with QF/C++ and QF/j.