I have an application that could potentitally throw an error on receiving a ExecutionReport (35=8) message.
This error is thrown at the application level and not at the fix engine level.
The fix engine records the message as seen and therefore will not send a ResendRequest (35=2). However the application has not processed it and I would like to manually trigger a re-processing of the missed ExecutionReport.
Forcing a ResendRequest (35=2) does not work as it requires modifying the expected next sequence number.
I was wonderin if FIX supports replaying of messages but without requiring a sequence number reset?
When processing an execution report, you should not throw any exceptions and expect FIX library to handle it. You either process the report or you have a system failure (i.e. call abort()). Therefore, if your code that handles execution report throws an exception and you know how to handle it, then catch it in that very same function, eliminate the cause of the problem and try processing again. For example (pseudo-code):
// This function is called by FIX library. No exceptions must be thrown because
// FIX library has no idea what to do with them.
void on_exec_report(const fix::msg &msg)
{
for (;;) {
try {
// Handle the execution report however you want.
handle_exec_report(msg);
} catch(const try_again_exception &) {
// Oh, some resource was temporarily unavailable? Try again!
continue;
} catch(const std::exception &) {
// This should never happen, but it did. Call 911.
abort();
}
}
}
Of course, it is possible to make FIX library do a re-transmission request and pass you that message again if exception was thrown. However, it does not make any sense at all because what is the point of asking the sender (over the network, using TCP/IP) to re-send a message that you already have (up your stack :)) and just need to process. Even if it did, what's the guarantee it won't happen again? Re-transmission in this case is not only doesn't sound right logically, the other side (i.e. exchange) may call you up and ask to stop doing this crap because you put too much load on their server with unnecessary re-transmit (because IRL TCP/IP does not lose messages and FIX sequence sync process happens only when connecting, unless of course some non-reliable transport is used, which is theoretically possible but doesn't happen in practice).
When aborting, however, it is FIX library`s responsibility not to increment RX sequence unless it knows for sure that user has processed the message. So that next time application starts, it actually performs synchronization and receives missing messages. If QuickFIX is not doing it, then you need to either fix this, take care of this manually (i.e. go screw with the file where it stores RX/TX sequence numbers), or use some other library that handles this correctly.
This is the wrong thing to do.
A ResendRequest tells the other side that there was some transmission error. In your case, there wasn't, so you shouldn't do that. You're misusing the protocol to cover your app's mistakes. This is wrong. (Also, as Vlad Lazarenko points out in his answer, if they did resend it, what's to say you won't have the error again?)
If an error occurs in your message handler, then that's your problem, and you need to find it and fix it, or alternately you need to catch your own exception and handle it accordingly.
(Based on past questions, I bet you are storing ExecutionReports to a DB store or something, and you want to use Resends to compensate for DB storage exceptions. Bad idea. You need to come up with your own redundancy solution.)
Related
The title is one way of doing what I ultimately want; others may be acceptable.
So I have a class that already exists. It controls some other stuff, and has a start and stop method. We are now making the list of "other stuff" depend on some settings saved in shared preferences, but shared preferences requires an asynchronous SharedPreferences.getInstance() call. I don't want to have to propagate asynchronicity all the way up the call chain. How do I avoid that? (Alternative 1: is there a library that you can persist data with, without messing directly with asynchronicity?)
Part of the problem is that the start and stop methods can be called at any point after the constructor (and indeed, when I just did SharedPreferences.getInstance().then(INITIALIZE LATE FINAL FIELDS), without await, start got called before the initialization happened and the code threw an exception.
(Alternative 2: is it acceptable to save the result of a then, and later call then on it? Like, do this._future = SharedPreferences.getInstance().then((prefs) {return LIST_OF_STUFF;}), and in start do this._future.then((listOfStuff) {START LIST OF STUFF; return listOfStuff;}), and similarly in stop? Seems...suspicious, but possibly functional.) I'd be ok with making the start and stop methods asynchronous, I guess, as long as I don't have to actually handle that in the code that calls them, but that doesn't really solve the problem that I need the SharedPreferences.getInstance() to have completed so I can have the list of stuff, first.
My default mechanism for handling concurrency is Communicating Sequential Processes, so I took a leaf from that book and thought, "What if I started an isolate that accepts start/stop messages? It can await the SharedPreferences, then loop over incoming messages and start/stop the Stuff as requested. I wrote an isolate to do that...and apparently you can't pass a ReceivePort in to an isolate (at least not by declaring it in the outer scope and using the reference in the isolate - it gives me an error Invalid argument(s): Illegal argument in isolate message: (object is aReceivePort)). You only get to directly hand the isolate a SendPort, so IT can send THE CALLER messages. The idiom I see in most places is to have the isolate send back a new SendPort, whose ReceivePort the isolate keeps. However, this returns me to my original problem - creating the isolate (and receiving its SendPort) is asynchronous, so it may not be done by the time start or stop get called. Since you pretty clearly CAN send a ReceivePort between...wait, I forgot it's technically not sending a ReceivePort, just another SendPort. Can you really only send SendPorts, specifically? That's...really obnoxious, if so. It seems pointlessly pedantic and obstructive. ...Ok, yeah, ReceivePort is explicitly restricted from being sent. Argh. I even thought I had a workaround to the "can't pass via scope" problem: make a new ReceivePort in the calling code, use its SendPort to send start and stop messages, and jerryrig some kind of handover of the ReceivePort to the isolate once that comes online. But if ReceivePort is entirely forbidden from being sent to the isolate, I guess that's out. So...how do I do what I want to do??? How do I pull a setting from disk, potentially queuing up start and stop messages while I wait for the persistence library to become available, without exposing to the calling code that we're making asynchronous calls under the hood?? Am I gonna have to make some kind of awful kludge of dangerous asynchronously-modified flags? I'll be rather disappointed in Flutter if that's what it comes down to.
Quick question, because I feel like I must be missing something.
I'm using rxjs here because it's what I've got in-front of me, this is a general reactiveX question, I believe.
Let's say I have a set of Observables like so:
network_request = some_thing // An observable that produces the result of a network call
event_stream = network_request.flatMapLatest(function(v) {
return connectToThing(v) // This is another observable that needs v
}) // This uses the result of the network call to form a long-term event-based connection
So, this works ok.
Problem, though.
Sometimes the connection thing fails.
So, if I do event_stream.retry() it works great. When it fails, it redoes the network call and gets a new v to use to make a new connection.
Problem
What happens if I want two things chained off of my network_request?
Maybe I want the UI to do something every time the network call completes, like show something about v in the UI?
I can do:
shared = network_request.share() // Other implementations call this refCount
event_stream = shared.flatMapLatest(...) // same as above
ui_stream = shared.flatMapLatest(...) // Other transformation on network response
If I didn't do share then it would have made two requests, which isn't what I want, but with share, when event_stream later has an error, it doesn't retry the network request because the refcount is still at 1 (due to ui_stream), so it immediately returns completed.
What I want
This is obviously a small example I've made up to explain my confusion.
What I want is that every time the result of event_stream (that long term connection) has an error all of the following happens:
the network request is made again
the new response of that request is used to build a new connection and event_stream goes on with new events like nothing happened
that same response is also emitted in ui_stream to lead to further processing
This doesn't feel like a complicated thing, so I must just be misunderstanding something fundamental when it comes to splitting / fanning out RX things.
Workarounds I think I could do but would like to avoid
I'm looking to export these observables, so I can't just build them again and then say "Hey, here's the new thing". I want event_stream and all the downstream processing to not know there's been a disconnection.
Same for ui_stream. It just got a new value.
I could probably work something out using a Subject as a generation counter that I ping every time I want everything to restart, and put the network_request into a flatMap based on that, so that I can break the share...
But that feels like a really hacky solution, so I feel there has to be a better way than that.
What have I fundamentally misunderstood?
As I've been thinking about this more I've come to the same realization as ionoy, which is that retry just disconnects and reconnects, and upstream doesn't know it was due to an error.
When I thought about what I wanted, I realized I really wanted something like a chain, and also a spectator, so I've got this for now:
network_request = some_thing
network_shadow = new Rx.Subject()
event_stream = network_request.do(network_shadow).flatMapLatest(...)
ui_stream = network_shadow.whatever
This has the property where an retry in event_stream or downstream will cause the whole thing to restart, whereas ui_stream is its own thing.
Any errors over there don't do anything, since network_shadow isn't actually a subscriber to event_stream, but it does peel the values off so long as the main event chain is running.
I feel like this isn't ideal, but it is better than what I was concerned I would have to do, which is have a restartEverything.onNext() in an doOnError, which would have been gross.
I'm going to work with this for now, and we'll see where it bites me...
You need to make your cold observable hot by using Publish. Read http://www.introtorx.com/Content/v1.0.10621.0/14_HotAndColdObservables.html#HotAndCold for a great explanation.
I am using pseudo-synchronous sockets in a Windows Phone 7 application. My socket code is based on the sample from http://msdn.microsoft.com/en-us/library/hh202858(v=vs.92).aspx.
The server's sending pattern is somewhat unpredictable. It starts with a fixed-size header that contains the length of the rest of the message. I first read in this header, and then I read the specified number of bytes from the socket.
Since I need to send messages to the server as well, and my attempts at duplexing the socket with a thread for receiving and another thread for sending caused lots of problems, I have a loop like this in my code:
while (KeepConnectionGoing)
{
byte[] Rcvd;
Rcvd = Socket.Receive();//Returns null if no message received in 50 ms
if (Rcvd != null)
{
ParseMessage(Rcvd);
}
if (HasMessageThatNeedsToBeSent())
{
byte[] Message = GetMessageToSend();
Socket.Send(Message);
}
}
This works fine for the majority of the time, but strange things happen when the message is null.
Because the timeout in the Receive method (see the linked sample) uses a ManualResetEvent, the receive request on the socket is never actually cancels. Even though the method returns, that request waits around somewhere, and when data is available on the socket, chomps up the header. The event handler has nothing to do with the data it received (since the method has returned and the variables in the method will never be used again), the data basically disappears. The read request I expect to return the header skips reads the bytes after the header, and I have no idea how long the message is.
I'd like to be able to cancel all outstanding requests if the socket times out. I am using anonymous methods like in the sample since it simplifies everything and prevents me from having to write all the state transfer code myself. Thus, I cannot unhook the event handler. I think though, that even if I were using a method as the event handler, but unhooking before the asynchronous operation is done, the callback method would still be called. (I haven't tested this, it's just my understanding)
Right now, the only solution I can see is hacking together some static byte arrays (ie. having a static byte[] Header and if it is null, I read the header, otherwise I read the message), but that seems like a really inelegant solution and very prone to race conditions.
Is there a better way?
Thanks
It appears there really is no good way to do this. A poll method would be nice, but Silverlight doesn't have it. I hacked together a solution using static flags to tell me what state I am in (Has the header been requested, has the message been requested), a static int for the length and a static buffer.
At the beginning of the method, either the header or the body can be requested. If the header has already been requested, the thread waits until a valid body length is available. If this wait times out, that means that the header receive operation is still pending, but there really is no message available. Otherwise, it reads in that length of a message.
If the header has not been requested, receive the header. In the event handler, after completion, check to see if the control flow has already continued (i.e. the receive operation took too long, so the function returned already, but is now actually done). Update the length, then request the body unless it timed out.
I have a few doubts regarding exception handling in iPhone. Here are they:
Suppose I have a chain of methods which are being called one after the other, that is, method A calls method B, which in turn calls method C, which calls method D. Which is the best place to put my try-catch block (is it method A or B or C or D or all of them). Also, I need to display an alert to user that an exception has occured and then, I want to log this exception to my server. So, if I am writing my try - catch block in all of this methods and if an exception occurs in method D; then I think the alert will be displayed 4 times and the web service for logging will also be called 4 times (till control reaches to catch block of method A). So, should I just use #throw; in catch block of method B, C and D and write my logic in catch block of method A (top level method) or should I avoid writing try - catch at all in methods B,C and D.
I need some sort of error code from the exception (because my web service needs parameters error-code and description). Is it possible to convert an exception to error or will I need to hard-code this code?
I had read somewhere about NSSetUncaughtExceptionHandler. And I think, if I can set this handler (in appDidFinishLaunching method of app delegate) and in the handler method, if I show to user some alert and call the web service; then I need not write try - catch block in each of my methods, in each of my classes. Am I right??
If an exception has occured and I have written either a try - catch block or NSSetUncaughtExceptionHandler, then will my app continue running or it will not respond to any of the user events. (I am sure it will handle the crash. What I want to know is whether it will hang)
Someone please enlighten me on this EXCEPTION TOPIC.
0) Avoid exceptions in Cocoa. They are generally non-recoverable. You might catch them for your own error reporting, but it's generally unsafe to assume you can recover from them.
1) If you need to catch, catch it immediately. Don't write your own throws -- instead, convert it to something like an NSError and pass that around. NSError can contain all the information you need to display or send an error code as well as a localized message.
2) You cannot convert an NSException into an NSError (directly) because an NSException does not have all the properties an NSError has - it is a different data representation. For one, an error code is not available. Two, the description is not localized. The best you can do is to create an error code and domain, then use the properties you need from the NSException and store that in an NSError. This could look something like the following:
// error checking omitted
extern NSString* const MONExceptionHandlerDomain;
extern const int MONNSExceptionEncounteredErrorCode;
NSError * NewNSErrorFromException(NSException * exc) {
NSMutableDictionary * info = [NSMutableDictionary dictionary];
[info setValue:exc.name forKey:#"MONExceptionName"];
[info setValue:exc.reason forKey:#"MONExceptionReason"];
[info setValue:exc.callStackReturnAddresses forKey:#"MONExceptionCallStackReturnAddresses"];
[info setValue:exc.callStackSymbols forKey:#"MONExceptionCallStackSymbols"];
[info setValue:exc.userInfo forKey:#"MONExceptionUserInfo"];
return [[NSError alloc] initWithDomain:MONExceptionHandlerDomain code:MONNSExceptionEncounteredErrorCode userInfo:info];
}
#catch (NSException * exc) {
NSError * err = NewNSErrorFromException(exc);
...
}
If the APIs you use throw exceptions you are expected to catch and recover from (e.g. not truly exceptional cases), then yes, you could catch and attempt to continue. Unfortunately, anybody that writes exceptions in Cocoa with the intent that you will catch them probably does not understand the issues well enough to implement a solid unwind implementation (e.g. even if it produces leaks, it's not solid).
3) That's really not the time or place to display an alert. If you install a top level exception handler (via NSSetUncaughtExceptionHandler) - You should simply log a message -- then the exception handler will abort. Your app is in an unstable state -- continuing is worse than aborting. You may want to send these custom messages home, it's best to do so at the next launch of your app.
4) In the majority of cases, your app is in an unstable state and you should not continue. But, to actually answer it for those corner cases: "Yes, you can recover and continue when you catch, but you should only attempt to recover and continue when the throwing API states that recovery is supported. If the problem is beyond your control, and the problem is not exceptional (e.g. file not found), and the vendor really expects you to continue, then I would have to assume that they expect you to continue, even though it really isn't (100% safe).". Do not attempt to recover/continue from within your top level exception handler (the program will abort after it returns). If you want to be very fancy and present that immediately on OSX, another process would be best. If you are calling through a pure C++ interface, then the unwinding is well defined, and the need to catch is necessary - do continue if it is recoverable. Exceptions in C++ can be recoverable and well defined - they are also used quite extensively (including less than exceptional conditions).
(IMO...) Exceptions in ObjC should not have been introduced, and any method that throws from system or third party libraries should be deprecated. They don't unwind well, or in a well defined manner. As well, unwinding flows against normal Cocoa program flow. That means that touching any objc object's memory/relations that was in mutation at the time of the throw and which lies between the throw and the catch is as good as undefined behaviour. Problem is - you have no idea what that memory is (in most cases, and within reasonable maintenance time). C++ exceptions are well defined, and they unwind correctly (e.g. destructors are called) - but trying to continue in an ObjC context is ignoring any consequences of undefined behavior. IMO, they should only exist for ObjC++ (because C++ requires them).
In an ideal world, your ObjC programs and the libraries you use would not use exceptions (at all). Since you use libraries that do throw (including Cocoa), install a top level exception handler only if you need some special information about the error. Where the API mandates that you could expect an exception thrown due to circumstances beyond your control and you are expected to recover, then write a catch but immediately convert that logic to normal program flow (e.g. NSError) - you never need to write your own throw. -[NSArray objectAtIndex: and "object does not respond to selector" are examples of programmer errors - they should not be caught, but the program should be corrected.
Many of the Caché API methods return a %Status object which indicates if this is an error. The thing is, when it's an unknown error I don't know how to handle (like a network failure) what I really want to do is "throw" the error so my code stops what it's doing and the error gets caught by some higher level error handler (and/or the built-in %ETN error log).
I could use ztrap like:
s status = someObject.someMethod()
ztrap:$$$ISERR(status)
But that doesn't report much detail (unlike, say, .NET where I can throw an exception all the way to to the top of the stack) and I'm wondering if there are any better ways to do this.
Take a look at the Class Reference for %Exception.StatusException. You can create an exception from your status and throw it to whatever error trap is active at the time (so the flow of control would be the same as your ZTRAP example), like this
set sc = someobj.MethodReturningStatus()
if $$$ISERR(sc) {
set exception = ##class(%Exception.StatusException).CreateFromStatus(sc)
throw exception
}
However, in order to recover the exception information inside the error trap code that catches this exception, the error trap must have been established with try/catch. The older error handlers, $ztrap and $etrap, do not provide you with the exception object and you will only see that you have a <NOCATCH> error as the $ZERROR value. Even in that case, the flow of control will work as you want it to, but without try/catch, you would be no better off than you are with ZTRAP
These are two different error mechanisms and can't be combined in this way. ztrap and %ETN are for Cache level errors (the angle bracket errors like <UNDEFINED>). %Status objects are for application level errors (including errors that occurred through the use of the Cache Class Library) and you can choose how you want to handle them yourself. It's not really meaningful to handle a bad %Status through the Cache error mechanism because no Cache error has occurred.
Generally what most people do is something akin to:
d:$$$ISERR(status) $$$SomeMacroRelevantToMyAppThatWillHandleThisStatus(status)
It is possible to create your own domain with your own whole host of %Status codes with attendant %msg values for your application. Your app might have tried to connect to an FTP server and had a bad password, but that doesn't throw a <DISCONNECT> and there is no reason to investigate the stack, just an application level error that needs to be handled, possibly by asking the user to enter a new password.
It might seem odd that there are these two parallel error mechanisms, but they are describing two different types of errors. Think of one of them being "platform" level errors, and the other as "application level errors"
Edit: One thing I forgot, try DecomposeStatus^%apiOBJ(status) or ##class(%Status).LogicalToOdbc(status) to convert the status object to a human readable string. Also, if you're doing command line debugging or just want to print the readable form to the principal device, you can use $system.OBJ.DisplayError(status).