Can we recognise inaccessible url using apache CloseableHttpAsyncClient - httpclient

I am using CloseableHttpAsyncClient for downloading image with socketreadtimemout of 10 sec , connect timeout 5sec and doing a retry if it fails with sockettimeout and Timeout exception.
Now when the url is not accessible(404) then instead of resource not found exception it is failing with socket timeout and doing a retry again.So in my case for this invalid url also we are trying again and it adds up to the latency(~10+10=20 sec).
Below is the code snippet
Future<HttpResponse> httpResponseFuture = asyncCloseableHttpClient.execute(httpUriRequest, null);
try {
return httpResponseFuture.get(10000, TimeUnit.MILLISECONDS);
} catch (ExecutionException e) {
Throwable cause = e.getCause() != null ? e.getCause() : e;
if (cause instanceof ConnectException) {
throw new DownloadConnectionException("ConnectionException " + cause, DOWNLOAD_FAILED, cause);
}
if (cause instanceof SocketTimeoutException) {
throw new DownloadTimeoutException(DOWNLOAD_TIMEOUT_EXCEPTION);
}
if (cause instanceof ConnectionClosedException) {
throw new DownloadConnectionClosedException("ConnectionClosedException " + DOWNLOAD_FAILED, cause);
}
if(cause instanceof UnsupportedCharsetException) {
throw new BadRequestException("Image download failed with UnsupportedCharsetException " + cause,
INVALID_CONTENT_TYPE_EXCEPTION, cause);
}
} catch (TimeoutException e) {
throw new DownloadTimeoutException(DOWNLOAD_TIMEOUT_EXCEPTION);
}
This the config values for CloseableHttpAsyncClient
RequestConfig config = RequestConfig.custom()
.setSocketTimeout(10000)
.setConnectTimeout(5000)
.setRedirectsEnabled(true)
.setMaxRedirects(3)
.setStaleConnectionCheckEnabled(false) // never set this to true, 30 ms extra per request
.setProxyPreferredAuthSchemes(Arrays.asList(AuthSchemes.BASIC))
.build();
This is retry config
RetryConfig.custom().maxAttempts(downloadAttempts).retryOnException(e -> ( e instanceof DownloadTimeoutException) ) .waitDuration(Duration.ofMillis(30)).build();
To give more context why I am setting the socketreadtimemout of 10 sec because let's say for some bigger image download it may fail 1st time,so in that case retry is valid scenario but not in the resource not found case.
Can we do anything to make resource not found/invalid url fail fast so that it wont fail with sockettimeout exception.

Related

Karate ApacheHttpClient Response is failing with error com.intuit.karate - Runtimejava.io.IOException: Attempted read from closed stream

Reference: java.io.IOException: Attempted read from closed stream
Reference: https://github.com/karatelabs/karate/blob/master/karate-core/src/main/java/com/intuit/karate/http/ApacheHttpClient.java
KarateCore - class file: ApacheHttpClient.java is unable to process the Response its failing at the code line
CloseableHttpClient client = clientBuilder.build();
CloseableHttpResponse httpResponse;
byte [] bytes;
try {
httpResponse = client.execute(requestBuilder.build());
HttpEntity responseEntity = httpResponse.getEntity();
if (responseEntity == null || responseEntity.getContent() == null) {
bytes = Constants.ZERO_BYTES;
} else {
**InputStream is = responseEntity.getContent();**
bytes = FileUtils.toBytes(is);
}
request.setEndTimeMillis(System.currentTimeMillis());
} catch (Exception e) {
if (e instanceof ClientProtocolException && e.getCause() != null) { // better error message
throw new RuntimeException(e.getCause());
} else {
throw new RuntimeException(e);
}
}
The Code is failing at line InputStream is = responseEntity.getContent(); when trying to read from a closed stream. The exception message displayed
Error com.intuit.karate - Runtimejava.io.IOException: Attempted read from closed stream.
May be the InputStream need to updated.
I am able read the Http Response content using below code
BufferedReader br = new BufferedReader(
new InputStreamReader((httpResponse.getEntity().getContent())));
String output;
while ((output = br.readLine()) != null){
output = br,readLine();
System.out.println(output);
}
Also the able to read the response using EntityUtils as a string content
String content = EntityUtils.toString(responseEntity);
System.out.println(content);
Not sure if i am missing something in the feature:scenario file response or the ApacheHttpClient.java file need to updated to read InputStream and then convert to bytes.
Feature: Hello
Scenario: Rest API Post
Given url 'some url path'
And header Content-Type = 'application/json'
And request { username: 'abc', password: 'pwd' }
When method POST
Then status 200
And print 'Response is:', response
The expected Response is a JSON format as:
{
"accessToken": "akjdoioikf",
"expires":"2020-01-29T01:09:48Z"
}
Any suggestions, appreciated!

ErrorDecoder not catching SocketTimeoutException

With feign builder I am making call to external URL. I have set readTimeout as 2seconds, I am getting the SocketTimeoutException but it's not going to ErrorDecoder.
Feign Builder Configuration:
Options options = new Options(10000, TimeUnit.MILLISECONDS, 2000,
TimeUnit.MILLISECONDS, false);
return Feign.builder()
.logLevel(level)
.client(client)
.retryer(Retryer.NEVER_RETRY)
.options(options)
.errorDecoder(feignErrorDecoder())
.exceptionPropagationPolicy(UNWRAP);
private ErrorDecoder feignErrorDecoder() {
return (methodKey, response) -> {
return new MyCustomException("ERROR_TIMEOUT",
"Timeout Occurred: " + response.status());
};
}
When I call service exception is not going to ErrorDecoder. java.lang.reflect.UndeclaredThrowableException coming.
ErrorDecoder is not called when an IOException (SocketTimeoutException) is thrown. See SynchronousMethodHandler#executeAndDecode(...). In general decoders are only called when a response is returend by the api-call. The ErrorCoder is only called when the http error code is not 2xx and 4xx.

How to catch/Capture "javax.net.ssl.SSLHandshakeException: Failed to create SSL connection" while sending message over java vert.x Eventbus

I am trying to use SSL over eventbus. To test the failure case I tried sending message to the eventbus from another verticle in same cluster by passing some different keystore.
I am getting below exception on console but it is not failing the replyHandler hence my code is not able to detect the SSL exception.
my code:
eb.request("ping-address", "ping!", new DeliveryOptions(), reply -> {
try {
if (reply.succeeded()) {
System.out.println("Received reply " + reply.result().body());
} else {
System.out.println("An exception " + reply.cause().getMessage());
}
} catch (Exception e) {
System.out.println("An error occured" + e.getCause());
}
});
Exception on console:
**javax.net.ssl.SSLHandshakeException: Failed to create SSL connection**
at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:109)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:341)
at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:327)
at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:319)
at io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1249)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1230)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1271)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:813)
Caused by: javax.net.ssl.SSLException: Received fatal alert: bad_certificate
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1647)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1615)
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1781)
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1070)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:896)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1329)
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1224)
... 20 more
But handler is failing for timeout after 30 sec.
Timed out after waiting 30000(ms) for a reply. address: __vertx.reply.8419a431-d633-4ba8-a12e-c41fd5a4f37a, repliedAddress: ping-address
I want to capture the SSL exception immediately and handle it. Please guide me how can I Capture/catch this exception.
I tried with below code. Below one is able to handle the exception and I am not getting reply result from called event-bus. Reply result is always null. (value is always null)
MessageProducer<Object> ms = eb.sender("ping-address");
ms.write("ping!", reply -> {
if (reply.succeeded()) {
reply.map(value -> {
System.out.println("Received reply " + value);
return reply;
});
} else {
System.out.println("No reply");
System.out.println("An exception : " + reply.cause().getMessage());
}
});
You can't catch this exception because the Vert.x clustered EventBus implementation buffers messages when the nodes are not connected together. The message could be sent later if the problem is only temporary.
If you want to be notified earlier, you could set a lower timeout in DeliveryOptions.

How to handle UndeliverableException error thrown via blockingFirst() in Micronaut?

Below is a snippet from my micronaut web service:
try {
val result = hClient.exchange(GET<String>("$readEndpoint/$token")).blockingFirst()
logger.error("result")
} catch (e: Exception) {
logger.error(e.message)
}
hClient is a reactive http client injected as #Inject val hClient: RxHttpClient
The endpoint is throwing "Connection reset by peer" exception.
Issue I am facing
Even though I have wrapped code in try and catch, An exception io.reactivex.exceptions.UndeliverableException is thrown and not caught.
I basically get two exceptions thrown, one is caught by catch block with message Error occurred reading HTTP response: Connection reset by peer, another one is flowing up to service with message io.reactivex.exceptions.UndeliverableException: The exception could not be delivered to the consumer because it has already canceled/disposed the flow or the exception has nowhere to go to begin with. Further reading: https://github.com/ReactiveX/RxJava/wiki/What's-different-in-2.0#error-handling | java.nio.channels.ClosedChannelException
Reproducible via below code
Keep timeout too less to receive a timeout error.
micronaut:
http:
client:
read-timeout: 1s
#Controller("/")
class TokenController(
#Client("https://hello123456789.com/dummy") #Inject val hClient: RxHttpClient
) {
#Get("/test")
fun refresh(): String {
try {
val result = hClient.exchange(HttpRequest.GET<String>("/token/1234")).blockingFirst()
println("result")
} catch (e: Exception) {
println(e.message)
}
return ""
}
}
Googling told me that I need to add global onError to rxjava but couldn't find how to do that in Micronaut.
Any help is appreciated.

Paypal-IPN Simulator ends up in HTTP 404 error after successfully completion of the function

have spent lot of hours trying to figure this out with Paypal Simulator, Sandbox but the result is same. My handler function(handleIpn) gets called and processed, with "Verified" "Complete" status but the IPN history as well as the simulator ends up in the HTTP 404 error. On IPN Simulator page the error is - "We're sorry, but there's an HTTP error. Please try again." My set up is Java-Spring MVC.
#RequestMapping(value = "/ipnHandler.html")
public void handleIpn (HttpServletRequest request) throws IpnException {
logger.info("inside ipn");
IpnInfo ipnInfo = new IpnInfo();
Enumeration reqParamNames = request.getParameterNames();
StringBuilder cmd1 = new StringBuilder();
String pName;
String pValue;
cmd1.append("cmd=_notify-validate");
while (reqParamNames.hasMoreElements()) {
pName = (String) reqParamNames.nextElement();
pValue = request.getParameter(pName);
try{
cmd1.append("&").append(pName).append("=").append(pValue);
}
catch(Exception e){
e.printStackTrace();
}
}
try
{
URL u = new URL("https://www.sandbox.paypal.com/cgi-bin/webscr");
HttpsURLConnection con = (HttpsURLConnection) u.openConnection();
con.setRequestMethod("POST");
con.setRequestProperty("Host", "www.sandbox.paypal.com/cgi-bin/webscr");
con.setRequestProperty("Content-length", String.valueOf(cmd1.length()));
con.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
con.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 5.0;Windows98;DigExt)");
con.setDoOutput(true);
con.setDoInput(true);
DataOutputStream output = new DataOutputStream(con.getOutputStream());
output.writeBytes(cmd1.toString());
output.flush();
output.close();
//4. Read response from Paypal
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String res = in.readLine();
in.close();
//5. Capture Paypal IPN information
ipnInfo.setLogTime(System.currentTimeMillis());
ipnInfo.setItemName(request.getParameter("item_name"));
ipnInfo.setItemNumber(request.getParameter("item_number"));
ipnInfo.setPaymentStatus(request.getParameter("payment_status"));
ipnInfo.setPaymentAmount(request.getParameter("mc_gross"));
ipnInfo.setPaymentCurrency(request.getParameter("mc_currency"));
ipnInfo.setTxnId(request.getParameter("txn_id"));
ipnInfo.setReceiverEmail(request.getParameter("receiver_email"));
ipnInfo.setPayerEmail(request.getParameter("payer_email"));
ipnInfo.setResponse(res);
// ipnInfo.setRequestParams(reqParamNames);
//6. Validate captured Paypal IPN Information
if (res.equals("VERIFIED")) {
//6.1. Check that paymentStatus=Completed
if(ipnInfo.getPaymentStatus() == null || !ipnInfo.getPaymentStatus().equalsIgnoreCase("COMPLETED"))
ipnInfo.setError("payment_status IS NOT COMPLETED {" + ipnInfo.getPaymentStatus() + "}");
//6.2. Check that txnId has not been previously processed
IpnInfo oldIpnInfo = this.getIpnInfoService().getIpnInfo(ipnInfo.getTxnId());
if(oldIpnInfo != null)
ipnInfo.setError("txn_id is already processed {old ipn_info " + oldIpnInfo);
//6.3. Check that receiverEmail matches with configured {#link IpnConfig#receiverEmail}
if(!ipnInfo.getReceiverEmail().equalsIgnoreCase(this.getIpnConfig().getReceiverEmail()))
ipnInfo.setError("receiver_email " + ipnInfo.getReceiverEmail()
+ " does not match with configured ipn email " + this.getIpnConfig().getReceiverEmail());
//6.4. Check that paymentAmount matches with configured {#link IpnConfig#paymentAmount}
if(Double.parseDouble(ipnInfo.getPaymentAmount()) != Double.parseDouble(this.getIpnConfig().getPaymentAmount()))
ipnInfo.setError("payment amount mc_gross " + ipnInfo.getPaymentAmount()
+ " does not match with configured ipn amount " + this.getIpnConfig().getPaymentAmount());
//6.5. Check that paymentCurrency matches with configured {#link IpnConfig#paymentCurrency}
if(!ipnInfo.getPaymentCurrency().equalsIgnoreCase(this.getIpnConfig().getPaymentCurrency()))
ipnInfo.setError("payment currency mc_currency " + ipnInfo.getPaymentCurrency()
+ " does not match with configured ipn currency " + this.getIpnConfig().getPaymentCurrency());
}
else
ipnInfo.setError("Inavlid response {" + res + "} expecting {VERIFIED}");
logger.info("ipnInfo = " + ipnInfo);
this.getIpnInfoService().log(ipnInfo);
//7. In case of any failed validation checks, throw {#link IpnException}
if(ipnInfo.getError() != null)
throw new IpnException(ipnInfo.getError());
}
catch(Exception e)
{
if(e instanceof IpnException)
throw (IpnException) e;
logger.log(Level.FATAL, e.toString(), e);
throw new IpnException(e.toString());
}
//8. If all is well, return {#link IpnInfo} to the caller for further business logic execution
paymentController.processSuccessfulPayment(ipnInfo);
}
Any help /pointers would greatly appreciate.
thanks.
Finally, got it working! Didn't realize that my issue of redirection in Spring MVC could have impact on Paypal - IPN status. May be my lack of good understanding of HTTP redirections! In above method instead of void return am now returning a jsp page, so "void" is changed to "String" with returning value the jsp file name.
Hope it does help someone!