For some client side procedures, I implement remote logging to log the calling of the procedure. The log is printed several times with different thread id, even though the procedure is only called once. Some rpc requests are sent to the sever a few times which causes some database session problem. Is it normal? Is there anyway to avoid it?
Thanks
This is not normal, and suggests there is a bug on your client causing it to send the same call more than once. Try adding logging on the client where you invoke the RPC call, and possibly add breakpoints to confirm why it is being called twice.
My best guess with no other information would be that you have more than one event handler wired up to the same button, or something like that.
--
More specifically, your servlet container starts multiple threads to handle incoming requests - if two requests come in close succession, they might be handled by different threads.
As you noted, this can cause problems with a database, where two simultaneous calls could be made to change the same data, especially if you have some checks to ensure that a servlet call cannot accidentally overwrite some newer data. This is almost certainly a bug in your client code, and debugging it should start there.
Related
We use Retrofit/OkHttp3 for all network traffic from our Android application. So far everything seems to run quite smoothly.
However, we have now occasionally had our app/process run out of file handles.
Android allows for a max of 1024 file handles per process
OkHttp will create a new thread for each async call
Each thread created this way will (from our observation) be responsible for 3 new file handles (2 pipes and one socket).
We were able to debug this exactly, where each dispached async call using .enqueue() will lead to an increase of open file handles of 3.
The problem is, that the ConnectionPool in OkHttp seems to be keeping the connection threads around for much longer than they are actually needed. (This post talks of five minutes, though I haven't seen this specified anywhere.)
That means if you are quickly dispatching request, the connection pool will grow in size, and so will the number of file handles - until you reach 1024 where the app crashes.
I've seen that it is possible to limit the number of parallel calls with Dispatcher.setMaxRequests()(although it seems unclear whether this actually works, see here) - but that still doesn't quite address the issue with the open threads and file handles piling up.
How could we prevent OkHttp from creating too many file handles?
I am answering my own question here to document this issue we had. It took us a while to figure this out and I think others might encounter this too and might be glad for this answer.
Our problem was that we created one OkHttpClient per request, as we used it's builder/interceptor API to configure some per-request parameters like HTTP headers or timeouts.
By default each OkHttpClient comes with its own connection pool, which of course blows up the number of connections/threads/file handles and prevents proper reuse in the pool.
Our solution
We solved the problem by manually creating a global ConnectionPool in a singleton, and then passing that to the OkHttpClient.Builder object which builds the actual OkHttpClient.
This still allows for per-request configuration using the OkHttpClient.Builder
Makes sure all OkHttpClient instances are still using a common connection pool.
We were then able to properly size the global connection pool.
I have implemented a Notify/Listen mechanism, so when a special request is sent to the web server, using notify I can notify the workers (in Python) that there's a pending request waiting to be processed.
The implementation works fine, but the problem is that if the workers server is restarting, the notification gets lost, since at that particular time there's no listener.
I can implement a service like MQRabbit or similar, but my needs are so simple that implement such a monster is too much.
Is there any way, a configuration variable perhaps, that can give some persistence to the notification mechanism?
Thanks in advance
I don't think there is a way to persist notification channels, but you can simply store the pending requests to a table, and have the worker check for any missed work on startup.
Either a timestamp or a pending/completed flag would work, depending on what kind of work it's doing.
For consistency, you can have the NOTIFY fire from an INSERT trigger on the queue table, and have the worker always check for any remaining work (not just a specific request) when notified.
I've got a Backbone web application that talks to a RESTful PHP server. For PUT and POST it matters in which order the requests arrive at the server and for GET it matters in which order the responses arrive at the client.
The web application does not need to be used concurrently by multiple users, but what might happen is that the user changes its name twice really fast. Then the order in which the server processes PUT /name/Ann and PUT /name/Bea determines whether the name is set to Ann or Bea.
Backbone.Safesync and Backbone.Sync.AjaxQueue are two libraries that try to solve this problem. Doesn't Safesync only solve the problem with GET? Sync.AjaxQueue is outdated, but might serve as inspiration to implement a custom queued sync function. Making sync synchronous would solve the problem. If a request is only sent after the previous response is received, then only one request is processed at a time.
Any advice on how to proceed?
BTW: I don't think using PATCH requests would solve anything, because in my example the same attribute is changed twice.
There's a few ways to solve this, here's two:
add a timestamp to all requests, store it in the DB as "modified" and let the server check whether the timestamp of the new request is later than the one in the DB in order to be valid
use Promises to delay the second request from being made before the first one is responded on, there's a promise/deferred mechanism built into jquery, but you can also use a 3rd party one, for instance Q or when
If you can afford the delay, an easy approach is to set the async option to false when you call whatever method you're calling that results in the Backbone.sync. For example, in the appropriate model(s) simply override the default sync method to include the additional option.
I just got asked to reduce the traffic made by my GWT app. There is one method that checks for status.
This method is an asynchronous call wrapped in a Timer. I know web apps are stateless and all that, but I do wonder if there is some other way to do this, or if everyone has a Timer wrapped around a call when they need this kind of behaviour.
You can check out gwteventservice. It claims to have a way to push server events and notify the client.
I have a feeling they might be implemented as long running (hanging) client to server RPC calls which time out after an interval (say 20sec), and then are re-made. The server returns the callback if an event happens in the meanwhile.
I haven't used it personally but know of people using it to push events to the client. Have a look at the docs . If my assumption is correct, the idea is to send an RPC call to the server which does not return (hangs). If an event happens on the server, the server responds and that RPC call returns with the event. If there is no event, the call will time out in 20 seconds. Then a new call is made to the server which hangs in the same way until there is an event.
What this achieves is that it reduces the number of calls to the server to either one for each event (if there is one), or a call once every 20 seconds (if there isn't one). It looks like the 20 sec interval can be configured.
I imagine that if there is no event the amount of data sent back will be minimal - it might be possible to cancel the callback entirely, or have it fail without data transfer, but I don't really know.
Here is another resource on ServerPush - which is likely what's implemented by gwteventservice.
Running on Google app engine you could use they Channel technology
http://code.google.com/intl/en-US/appengine/docs/java/channel/overview.html
If you need the client to get the status from the server, then you pretty much have to make a call to the server to get it's status.
You could look at reducing the size of some of your messages
You could wind back the timer so the status call goes out less often
You could "optimise" so that the timer gets reset when some other client/server IO happens (i.e. if the server replies you know it is ok, so you don't need to send the next status request).
My iPhone application supports a proprietary network protocol using the CocoaAsyncSocket library. I need to be able to send a network message out when my iPhone application is closed. The code that sends the message is getting called from the app delegate, but the application shuts down before the message actually goes out. Is there a way to keep the application alive long enough for the message to go out?
Bruce
The docs from Apple don't specifically state this, but the sense I get from looking around the Web and from personal experience is that you have about 4 to 5 seconds after the user hits the Home button to shut your app before your application actually terminates. The iPhone OS is controlling this so you can't block the termination to allow your program to finish first. Basically when your time is up, your program is killed.
There may be another solution, though. First I'd confirm that your code is really taking more than 5 seconds to run. Perhaps you can have it run in response to a button tap, and time how long it runs. If it is more than 5 seconds, you probably are running into this time out issue.
You might then find a way to trigger a message to be sent from a server that is always running. You should have enough time to trigger a remote action, which in turn could then take as long as it needs to run.
Or perhaps you could save the vital information to the iPhone file system on exit, and send that message the next time someone starts the application, which should theoretically give you enough time.
Hope this helps!
I assume you're already calling it from your AppDelegate's:
- (void)applicationWillTerminate:(UIApplication *)application
But as you've discovered there's no guarantee it'll be called or will be allowed to finish. There are a few options that may or may not work depending on what you're trying to do:
If you need the server to perform some sort of cleaning operation triggered by when the client app is gone then you could try watching for TCP socket closure on the server and treating that as the triggering event. But if you explicitly need to send data back with the closure this may not work.
If the data you're sending back is not time-sensitive then you can do like most of the analytics libraries do and cache the data (along with a uuid) on the client then try to send it on app closure. If it goes through, you can clear the cache (or do it the next time the app is run). If it doesn't, it's saved and you can send out when the app is run next. On the server, you would use the uuid to avoid duplicate requests.
If the material is time-sensitive then your best bet is to implement heartbeat and send periodic updated values to the server. Then when the client app dies the server times out the heartbeat and can use the last received value as the final closing point of data.
In either case, if an explicit closure event is required by your custom protocol then you may want to reconsider using it in a real-life mobile environment where things have to be much more fluid and tolerant of failure.
As others have noted, there's no way to be absolutely certain that you'll be able to send this, but there are approaches to help.
As Ken notes, you do in practice get a few seconds between "willTerminate" and forced termination, so there generally is time to do what you need.
A problem you're almost certainly running into is with CocoaAsyncSocket. When you get the "willTerminate" message, you're on the last run loop of the main thread. So if you block the main thread, and CocoaAsyncSocket is running on the main thread, it'll never get processed. As I recall, CocoaAsyncSocket won't actually send all the data until the next event loop.
One approach, therefore, is to keep pumping the event loop yourself:
- (void)applicationWillTerminate:(UIApplication *)application
{
// ...Send your message with CocoaAsyncSocket...
while (! ...test to see if it sent...)
{
[[NSRunLoop currentRunLoop] runMode:NSDefaultRunLoopMode beforeDate:[NSDate distantFuture]];
}
}
I've also looked at putting this work onto a background thread and letting the main thread terminate, in theory letting us go back to Springboard while continuing to run for a few seconds. It's not immediately clear to me whether this will work properly using NSThread (which are detached). Using POSIX threads (which are joinable by default) may work, but probably circumvents any advantages of the background thread. Anyway, it's something to look at if useful. In my apps, we've used the "post next time we launch" approach, since that always works (even if you crash).