HTTPClient 4.x Auth cache not set in the context - httpclient

I am using HttpClient 4.5.x to call a SOAP web service with NTLM authentication.
Authentication happens successfully. It is a 3 way handshake.
For example, if i do post-request including images or other data content, then for each handshake request, the data is sent.
One recommendation from the HttpClients material online is to do a cheap request first, and use the same client context object for the subsequent big size request.
It also says this in the documentation - As of version 4.1, HttpClient automatically caches information about hosts it has successfully authenticated.
I tried the same. I have both these requests subsequently happening in the same method, and the same thread. The (default) caching does not happen. Both times the 3-way handshake was happening.
In the log I see the following statement
[org.apache.http.client.protocol.RequestAuthCache] Auth cache not set in the context
May be this default does not work for NTLM.
Is there any flag to turn the caching on?
Or should I create AuthCache myself and maintain? It looks like only for preemptive authentication, one creates auth cache. So, I am doubtful if it applies to my case.
private void callServiceWithAuthentication (ByteArrayEntity entity)
{
try {
/*
AuthScope authScope = new AuthScope("WEBSERVICE-HOST", 443, AuthScope.ANY_REALM, "ntlm");
*/
CloseableHttpClient httpclient = HttpClients.createDefault();
NTCredentials ntCredentials = new NTCredentials(this.userName, this.password, null, "DOMAIN");
CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, ntCredentials);
httpClientContext = HttpClientContext.create();
httpClientContext.setCredentialsProvider(credentialsProvider);
HttpHost targetHost = new HttpHost ("WEBSERVICE_HOST", 443);
// First cheap-cost request
HttpGet httpGet = new HttpGet ("WEBSERVICE_URL");
CloseableHttpResponse httpResponse = httpclient.execute(httpGet, httpClientContext);
HttpEntity getResponseEntity = httpResponse.getEntity();
// Following real costly post request
HttpPost post = new HttpPost("WEBSERVICE_URL");
post.setEntity(entity);
// Execute request
try {
CloseableHttpResponse response = httpclient.execute(post, httpClientContext);
StatusLine statusLine = response.getStatusLine();
HttpEntity rentity = response.getEntity();
System.out.println ("HTTP Response Code: " + statusLine.getStatusCode());
if (statusLine.getStatusCode() >= 300) {
throw new HttpResponseException(
statusLine.getStatusCode(),
statusLine.getReasonPhrase());
}
if (entity == null) {
throw new ClientProtocolException("Response contains no content");
}
System.out.println (rentity.getContent());
}
catch (Exception e) {
e.printStackTrace();
}
finally {
// Release current connection to the connection pool once you are done
post.releaseConnection();
}
}
catch (Exception e) {e.printStackTrace();}
finally {
}
}

I had to have the first response entity consume before the second request. Then the second request did not do the (3-way handshake) authentication again.
EntityUtils.consume(getResponseEntity);
I have more cleanup to do

Related

Vertx - threads are stuck while sending response back to client

I'm using vertx-4.2.6 to build a proxy service which takes requests from clients (for ex: browser, standalone apps etc), invoke a single thirdparty server, gets the response and send the same response back to client who initiated the request.
In this process, I'm using shared Webclient across multiple requests, i'm getting response from thirdparty quickly (mostly in milli seconds) but sometimes the response is not returned back to client and stucks at ctx.end(response).
Whenever i restart my proxy server, it serves requests sometimes without any issues but time goes on, lets say by EOD, for new requests client seeing 503 error -service unavailable I'm using one MainVerticle with 10 instances. I'm not using any worker threads.
Below is the pseudo code:
MainVerticle
DeploymentOptions depOptions = new DeploymentOptions();
depOptions.setConfig(config);
depOptions.setInstances(10);
vertx.deployVerticle(MainVerticle.class.getName(), depOptions);
.....
router.route("/api/v1/*")
.handler(new HttpRequestHandler(vertx));
HttpRequestHandler
public class HttpRequestHandler implements Handler<RoutingContext> {
private final Logger LOGGER = LogManager.getLogger( HttpRequestHandler.class );
private WebClient webClient;
public HttpRequestHandler(Vertx vertx) {
super(vertx);
this.webClient=createWebClient(vertx);
}
private WebClient createWebClient(Vertx vertx) {
WebClientOptions options=new WebClientOptions();
options.setConnectTimeout(30000);
WebClient webClient = WebClient.create(vertx,options);
return webClient;
}
#Override
public void handle(RoutingContext ctx) {
ctx.request().bodyHandler(bh -> {
ctx.request().headers().remove("Host");
StopWatch sw=StopWatch.createStarted();
LOGGER.info("invoking CL end point with the given request details...");
/*
* Invoking actual target
*/
webClient.request(ctx.request().method(),target_port,target_host, "someURL")
.timeout(5000)
.sendBuffer(bh)
.onSuccess(clResponse -> {
LOGGER.info("CL response statuscode: {}, headers: {}",clResponse.statusCode(),clResponse.headers());
LOGGER.trace("response body from CL: {}",clResponse.body());
sw.stop();
LOGGER.info("Timetaken: {}ms",sw.getTime()); //prints in milliseconds
LOGGER.info("sending response back to client...."); //stuck here
/*
* prepare the final response and return to client..
*/
ctx.response().setStatusCode(clResponse.statusCode());
ctx.response().headers().addAll(clResponse.headers());
if(clResponse.body()!=null) {
ctx.response().end(clResponse.body());
}else {
ctx.response().end();
}
LOGGER.info("response SENT back to client...!!"); //not getting this log for certain requests and gives 503 - service unavailable to clients after 5 seconds..
}).onFailure(err -> {
LOGGER.error("Failed while invoking CL server:",err);
sw.stop();
if(err.getCause() instanceof java.net.ConnectException) {
connectionRefused(ctx);
}else {
invalidResponse(ctx);
}
});
});
Im suspecting issue might be due to shared webclient. But i'm not sure. I'm new to Vertx and i'm not getting any clue what's going wrong. Please suggest if there are any options to be set on WebClientOptions to avoid this issue.

MSF4J POST method receiving partial data

I'm new to MSF4J and I need to write a REST API that accepts a large XML data through POST. I am using
request.getMessegeBody()
method to get the data. I discovered that it's now deprecated but I couldn't find the newer version of it so I decided to use it anyway.
The problem is, when I send data to the microservice for the first time, it doesn't get the whole data. All the subsequent requests will get the full message body except the first.
When I try passing the request through ESB, ESB receives the whole body but when it reaches the endpoint it will be truncated.
I have also tried sending requests from different rest clients but for the first time it always gets the incomplete message body
#POST
#Consumes({ "application/xml", "application/json", "text/xml" })
#Path("test/")
public Response getReqNotification(#Context Request request) throws Exception {
Response.ResponseBuilder respBuilder =
Response.status(Response.Status.OK).entity(request);
ByteBuf b = request.getMessageBody();
byte[] bb = new byte[b.readableBytes()];
b.duplicate().readBytes(bb);
System.out.println(new String(bb));
return respBuilder.build();
}
I expect it to print the full message(which is about 2000 bytes long) every time when I send a request, but I'm only getting around 800 bytes when I first run the microservice.
I hope ill get assistance here. I have tried elsewhere but wso2 doesn't have much documentation (⌣_⌣”)
I still don't really understand what I was doing wrong but with the help of this link I have managed to come up with the following code and it works fine.
The major cha is that I now use request.getMessageContentStream() instead of the depricated request.getMessageBody()
#Consumes({ "application/xml", "application/json", "text/xml" })
#Path("test/")
public Response getReqNotification(#Context Request request) throws Exception {
Response.ResponseBuilder respBuilder =
Response.status(Response.Status.OK).entity(request);
String data = "";
BufferedInputStream bis = new BufferedInputStream(request.getMessageContentStream());
ByteArrayOutputStream bos = new ByteArrayOutputStream();
try {
int d;
while ((d = bis.read()) != -1) {
bos.write(d);
}
data = bos.toString();
} catch (IOException ex) {
ex.printStackTrace();
} finally {
try {
bos.close();
} catch (IOException e) {
}
}
System.out.println(data);
//////do stuff
return respBuilder.build();
}

Is RESTEasy RegisterBuiltin.register necessary when using ClientResponse<T>

I am developing a REST client using JBOSS app server and RESTEasy 2.3.6. I've included the following line at the beginning of my code:
RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
Here's the rest of the snippet:
RegisterBuiltin.register(ResteasyProviderFactory.getInstance());
DefaultHttpClient httpclient = new DefaultHttpClient();
httpclient.getCredentialsProvider().setCredentials(
new AuthScope(host, port, AuthScope.ANY_REALM), new UsernamePasswordCredentials(userid,password));
ClientExecutor executor = createAuthenticatingExecutor(httpclient, host, port);
String uriTemplate = "http://myhost:8080/webapp/rest/MySearch";
ClientRequest request = new ClientRequest(uriTemplate, executor);
request.accept("application/json").queryParameter("query", searchArg);
ClientResponse<SearchResponse> response = null;
List<MyClass> values = null;
try
{
response = request.get(SearchResponse.class);
if (response.getResponseStatus().getStatusCode() != 200)
{
throw new Exception("REST GET failed");
}
SearchResponse searchResp = response.getEntity();
values = searchResp.getValue();
}
catch (ClientResponseFailure e)
{
log.error("REST call failed", e);
}
finally
{
response.releaseConnection();
}
private ClientExecutor createAuthenticatingExecutor(DefaultHttpClient client, String server, int port)
{
// Create AuthCache instance
AuthCache authCache = new BasicAuthCache();
// Generate BASIC scheme object and add it to the local auth cache
BasicScheme basicAuth = new BasicScheme();
HttpHost targetHost = new HttpHost(server, port);
authCache.put(targetHost, basicAuth);
// Add AuthCache to the execution context
BasicHttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.AUTH_CACHE, authCache);
// Create ClientExecutor.
ApacheHttpClient4Executor executor = new ApacheHttpClient4Executor(client, localContext);
return executor;
}
The above is a fairly simple client that employs the ClientRequest/ClientResponse<T> technique. This is documented here. The above code does work (only left out some trivial variable declarations like host and port). It is unclear to me from the JBOSS documentation as to whether I need to run RegisterBuiltin.register first. If I remove the line completely - my code still functions. Do I really need to include the register method call given the approach I have taken? The Docs say I need to run this once per VM. Secondly, if I am required to call it, is it safe to call more than one time in the same VM?
NOTE: I do understand there are newer versions of RESTEasy for JBOSS, we are not there yet.

httpunit PutMethodWebRequest throws IOException; bad file descriptor

Could someone explain why this httpunit test case keeps failing in wc.getResponse with "bad file descriptor". I added the is.close() as a guess and moved it before and after the failure but that had no effect. This tests put requests to a Dropwizard app.
public class TestCircuitRequests
{
static WebConversation wc = new WebConversation();
static String url = "http://localhost:8888/funl/circuit/test.circuit1";
#Test
public void testPut() throws Exception
{
InputStream is = new FileInputStream("src/test/resources/TestCircuit.json");
WebRequest rq = new PutMethodWebRequest(url, is, "application/json");
wc.setAuthentication("FUNL", "foo", "bar");
WebResponse response = wc.getResponse(rq);
is.close();
}
No responses? So I'll try myself based on what I learned fighting this.
Httpunit is an old familiar tool that I'd use if I could. But it hasn't been updated in more than two years, so I gather its support for #PUT requests isn't right.
So I converted to Jersey-client instead. After a bunch of struggling I wound up with this code which does seem to work:
#Test
public void testPut() throws Exception
{
InputStream is = new FileInputStream("src/test/resources/TestCircuit.json");
String circuit = StreamUtil.readFully(is);
is.close();
Authenticator.setDefault(new MyAuthenticator());
ClientConfig config = new DefaultClientConfig();
Client client = Client.create(config);
com.sun.jersey.api.client.WebResource service = client.resource(url);
Builder builder = service.accept(MediaType.APPLICATION_JSON);
builder.entity(circuit, MediaType.APPLICATION_JSON);
builder.put(String.class, circuit);
return;
}
This intentionally avoids JAX-RS automatic construction of beans from JSON strings.

Errorstream in HttpUrlConnection

I want to do a POST request to an HTTP Servlet I wrote myself. Good case (HTTP response Code 200) always works fine by using URL.openConnection() method. But when I receive a desired error response code (e.g. 400) then I thought I have to use HttpUrlConnection.getErrorStream(). But the ErrorStream object is null though I am sending data back from the servlet in error case (I want to evaluate this data to generate error messages).
This is what my code looks like:
HttpURLConnection con = null;
try {
//Generating request String
String request = "request="+URLEncoder.encode(xmlGenerator.getStringFromDocument(xmlGenerator.generateConnectRequest(1234)),"UTF-8");
//Receiving HttpUrlConnection (DoOutput = true; RequestMethod is set to "POST")
con = openConnection();
if (con != null){
PrintWriter pw = new PrintWriter(con.getOutputStream());
pw.println(request);
pw.flush();
pw.close();
InputStream errorstream = con.getErrorStream();
BufferedReader br = null;
if (errorstream == null){
InputStream inputstream = con.getInputStream();
br = new BufferedReader(new InputStreamReader(inputstream));
}else{
br = new BufferedReader(new InputStreamReader(errorstream));
}
String response = "";
String nachricht;
while ((nachricht = br.readLine()) != null){
response += nachricht;
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
So my question is, why returns getErrorStream() null though status code is 400 (I can see it in the IOException that is thrown when it calls con.getInputStream())
Thanks
From the java documentation on getErrorStream():
Returns the error stream if the connection failed but the server sent useful data nonetheless. The typical example is when an HTTP server responds with a 404, which will cause a FileNotFoundException to be thrown in connect, but the server sent an HTML help page with suggestions as to what to do.
This method will not cause a connection to be initiated. If the connection was not connected, or if the server did not have an error while connecting or if the server had an error but no error data was sent, this method will return null. This is the default.
So if you didn't get to the server (bad url for example) or the server didn't send anything in the response, getErrorStream() will return null.
InputStream inputStream = null;
try {
inputStream = connection.getInputStream();
} catch(IOException exception) {
inputStream = connection.getErrorStream();
}
It is like when you set response header status code as anything beyond 200, the connection object is reset. it will generate SocketTimeoutException while getting the inputstream but when it comes in the catch it gives you the inputstream anyway, what you are expecting.
Digging a little bit into JDK code, I finally find the reason. HttpURLConnection#getErrorStream() returns null when receiving a 401 or 407, not because the noop implementation in the abstract class, but because HttpURLConnection closes/clears the connection immediately if it sees a 401/407 when in streaming mode(i.e., POST). See the source of the concrete implementation of HttpURLConnection: http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/sun/net/www/protocol/http/HttpURLConnection.java#1079
That said, when you catch an IOException when calling getInputStream(), the connection to server is already closed and the underlining socket is cleared, so you would always get null when calling getErrorStream().
The other options many have suggested is to check the status code before calling getInputStream or getErrorStream. This won't for 401 and 407 either because the internal errorStream is only set when you call getInputStream, i.e., it's a basically a copy of the inputStream when status code != 200. But again when you call getInputStream, the connection will be closed.
Put the statement conn.setAllowUserInteraction(true); before execute the request and the connection will not be closed, even receiving 401 status.
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setAllowUserInteraction(true); // <<<--- ### HERE
//do something
conn.connect();
boolean isError = (conn.getResponseCode() >= 400);
InputSteam is = isError ? con.getErrorStream() : con.getInputStream();
As suggested in the Android documentation:
String responseString;
try {
responseString = readInputStream(con.getInputStream());
} catch (final IOException e) {
// This means that an error occurred, read the error from the ErrorStream
try {
responseString = readInputStream(con.getErrorStream());
} catch (IOException e1) {
throw new IllegalStateException("Unable to read error body.", e);
}
}
private String readInputStream(final InputStream inputStream) throws IOException {
final BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
final StringBuilder responseString = new StringBuilder();
String line;
while ((line = bufferedReader.readLine()) != null) {
responseString.append(line);
}
bufferedReader.close();
return responseString.toString();
}