I am using a RMI/JRMP client/server employing spring-remoting. There are thousands of clients trying to connect and send requests. I have coded the server to refuse accepting a request when its plate is full than client can resend after waiting for some time. It works fine for most part, but for some strange reason it starts throwing below error intermittently for some clients and than starts working again for them when they keep retrying.
nested exception is org.springframework.remoting.RemoteLookupFailureException: Lookup of RMI stub failed;
nested exception is java.rmi.UnknownHostException: Unknown host: a.b.c.d;
I would appreciate if someone could throw some light on this, on causes and if there are ways to overcome this.
Thanks
Related
I have a Vert.x app that gets HTTP requests as a Server and later down the road sends the data (HTTP request as well) as a Client to several other servers (more than one Client exits).
I see in the logs that sometimes I get io.vertx.core.VertxException: Connection was closed
but with no other info.
How can I know which connection was the one that was actually closed? I have more than one connections active.
I tried to add exceptionHandler to HttpServer and to HttpClientRequest, but they both were never called.
The io.vertx.core.VertxException: Connection was closed can be triggered for both client connections and server connections.
You'd get these errors on your HTTP client connection if the remote server closed the connection before completing the response, and you'd be able to capture them by setting the appropriate handlers on the client request (With Vert.x 4, you'd do something like ...send().onFailure(err -> /* handle the failure */) ), which I believe you already do.
You'd get these errors for the HTTP server connection if the remote client disconnects, either before your server completed the response, or - if the connection has keep-alive enabled (which is the default for HTTP/1.1 connections) - even after the response was sent.
In case the client closed the server connection before the response was fully sent, you should be able to capture and handle there errors in the HttpServer.exceptionHandler(), as I'm sure you already do.
In case the client closed the server connection after the response was fully sent, while it is in a keep-alive state, then there is no HttpServerRequest (or RoutingContext if you are using vertx-web, as you should) where the exception happens in, so Vert.x would just disregard the error (see here for the code).
So why do you still see those errors in the log? It could be various things because that exception is also used to handle EventBus connections and all kinds of internal network streams managed by Vert.x servers - and all without a stack trace (the actual exception instance being thrown is created statically here), so Vert.x kinds of sucks in that way.
My recommendation: make sure you attach error handlers to everything (pay attention to websocket connections or HTTP response streams, if you use them) and if you still get those errors in the logs - maybe you can just ignroe them as the commenter suggested.
I am using the wrk HTTP benchmarking tool to test a server. And I am getting READ, WRITE as well as CONNECTION and TIMEOUT errors.
What I understand is:
CONNECTION errors, are caused by the refusal of a TCP connection.
Which could involve every element in the connection chain (Client,
ISP and Server).
TIMEOUT errors, are caused by the host failing to respond to a
request within a certain time.
But what about READ and WRITE errors?
I would really appreciate, if someone could point me in the direction of a good resource?
So what I understood from this code from the WRK repository is that.
WRITE ERROR’s happen when attempting to write on a connection, but it fails because of a closed socket on the server.
READ ERROR’s happen when attempting to read on a connection, but it fails because of a closed socket on the server.
Happy if anybody can confirm or refute that.
I'm studying the following scenario: while get/set operations to Couchbase I shutting down node(power off on virtual machine). After that, I power on the machine and waiting for Couchbase node recovery. When node's status changing to "healthy" I expect that client reconnect and get/set operations continues. But sometimes reconnection of client occurs immediately, sometime doesn't occur within a few minutes.
So my question is:
Are there some configuration on server side, or on client side that guarantee a wholly reconnection of client?
I use JavaSDK.
A small addition:
Couchbase client is based on spymemcached client. If someone knows any hints with memcached, that could solve problem, I'll be very glad to see them.
Another addition:
Client stops trying to establish connection after this exception:
Exception in thread "Thread-122" java.lang.IllegalStateException: Got empty SASL auth mech list. 11:59:25,731 ERROR [stderr] (Thread-122) at net.spy.memcached.auth.AuthThread.listSupportedSASLMechanisms(AuthThread.java:99) 11:59:25,731 ERROR [stderr] (Thread-122) at net.spy.memcached.auth.AuthThread.run(AuthThread.java:112)
But I can't understand, why this exception happens so irregularly.
It's a small bug with response of CouchbaseServer. It's my discussion of this problem with developer of spymemcached client. Problem was solved when I hard-coded some string at this client.
Currently when an error occurs in a NodeJS server (eg: Buffer "out of bounds"), the entire server grinds to a halt. Is there a way to configure it such that only the socket on which the error occured gets disconnected leaving the server online and the other connections active? I could use the "uncaughtException" error but then there would be no way of knowing which socket caused the error/crashed. How is this handled typically?
You can use the try .. catch(err) construct to catch any exceptions occurring within any callback of the the socket, and subsequently socket.end(), to disconnect the client if an error occurs. This prevents the entire server from crashing. Obviously for the other "uncaught" errors you'll have to resort to the "uncaughtException" callback, which does not end the server. Why don't you read this and this for more answers.
My client app gets intermittent winsock errors (10060, 10053) against one particular server we interface with. I have it re-trying the request that failed, but sometimes it fails repeatedly, and I give up after 5 re-tries. Would it be likely to help at all if I closed the socket and created a new one? (I know nothing about the server-side.)
Ok, so the errors that you're getting are:
10060 - WSAETIMEDOUT
10053 - WSAECONNABORTED
When do you get them? What are you doing at the time?
You get a WSAECONNABORTED when the remote end of the connection, or possibly an intermediary router, resets the connection and sends an RST. This could simply be the remote end issuing a non lingering close or it could be the remote end aborting or crashing.
You can't continue doing anything with a connection that has had a WSAECONNABORTED on it as the connection has been aborted and is no more; it is a dead connection, it has passed on...
Context matters immensely as to why you might get a WSAETIMEDOUT exception and the context will dictate if retrying is sensible or not.
One thing I would try is- do tracert to your server.
Often when someone is connected through VPN; you may see this error because your local and remote ip addresses overlap.
e.g. if your local ipaddress range is 192.168.1.xxx and vpn remote range is also 192.168.1.xxx you will also see this error.
sharrajesh