Caching Digest Authentication Responses - httpclient

I am using Apache's HttpClient to talk to a server that requires Digest Authentication.
I set up an HttpClientContext with credentials like so:
#Bean
public HttpClientContext getCredentialsHttpClientContext() {
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(createCredentialsProvider());
context.setAuthCache(new BasicAuthCache());
return context;
}
Later on I use this context to stream data from my server:
public InputStream getInputStreamFromUrl(String url) throws IOException {
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpResponse httpResponse = httpClient.execute(new HttpGet(url), clientContext);
return httpResponse.getEntity().getContent();
}
Later on in processing this input stream, I need to make calls to the same server using the HttpClientContext, and every subsequent call requires authentication.
Is it possible to cache the results of the digest auth, so that subsequent requests don't need to be authenticated again?

#oleg's comments above pointed me in the right direction.
I actually create a singleton HttoClientContext bean:
#Bean
public HttpClientContext getCredentialsHttpClientContext() {
HttpClientContext context = HttpClientContext.create();
context.setCredentialsProvider(createCredentialsProvider());
context.setAuthCache(new BasicAuthCache());
return context;
}
I inject this into a service that I use in the getInputStreamFromUrl method above.
In other cases, I am using the RestOperations template. In order for this to work, I created the following Spring bean, creates a RestTemplate using the same HttpClientContext singelton:
#Bean
public RestOperations getIcsCredentialedRestOperations() {
HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactoryWithContext(httpClient, httpClientContext);
return new RestTemplate(requestFactory);
}
In the above example, HttpComponentsClientHttpRequestFactory extends HttpComponentsClientHttpRequestFactory overriding the createHttpContext method returning the context that I want.

Related

HTTP request doesn't work with a paricular rest

I'm making an application filled with various rest services, so I create a one-for-all HTTP class in order to allow a client application to keep asking information, via rest, to a server application
public HttpURLConnection HTTPSENDJSON(String urlAPI,String out,String requestmethod) throws IOException {
URL url = new URL(urlAPI);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setConnectTimeout(5000);
conn.setRequestProperty("Content-Type", "application/json");
conn.setRequestProperty("Accept", "application/json");
conn.setRequestMethod(requestmethod);
conn.setDoOutput(true);
conn.setDoInput(true);
OutputStream os = conn.getOutputStream();
System.out.println(out);
os.write(out.getBytes());
os.flush();
os.close();
return conn;
urlAPI is the desired URL, a string is the JSON string (I'm using GSON) and the requestmethod is a string in order to switch from PUT\POST\GET\PATCH.
So, as I wrote, it's all ok if I need to retrieve information from DB\insert a new record
ATM my Client application makes a call to the server who calls an EJB in order to CRUD the information.
this is the Client method who call the upper method (the HTTPSENDJSON )
public String modifica() throws IOException {
Universal_HTTPREQUEST httprequest = new Universal_HTTPREQUEST();
String url= "http://localhost:8080/ModuloWebClientNuovo/rest/clientela/modifica/account/"+ac.getId()+"";
Gson g = new Gson();
String out=g.toJson(ac, Account.class);
httprequest.HTTPSENDJSON(url, out,"PUT");
and this is the working (at least with POSTMAN) services
#PUT
#Path("modifica/account/{id}")
#Produces(MediaType.APPLICATION_JSON)
#Consumes(MediaType.APPLICATION_JSON)
public Response modificaaccount(#PathParam("id") int id,Account a) {
System.out.println("i'm inside the api and i wrote: "+ a.toString());
ac.updateAccount(a);
return Response.status(200).entity(a).build() ;
}
The Client doesn't even make the call to the server, BUT the only with this specific rest, other works fine.
update account EJB is:
#Stateless
public class AccountEJB implements IAccountCrud {
#EJB
Iconnessioni x;
#Override
public void updateAccount(Account account) {
EntityManager entityManager=x.apriconnessione();
entityManager.merge(account);
x.chiudiconnessione(entityManager);
}
}
Fixed whit a new from scratch wildfly

Is it possible to apply dictionaries for Citrus static response adapter response template?

I'm using Citrus static response adapter to mock services, and I need to change values in its payload for every test case. Ideally I think about usage of dictionaries for each test case. There is sample of my current scenario:
#Autowired
#Qualifier("checkRegistrationEndpointAdapter")
public StaticResponseEndpointAdapter checkRegistrationEndpointAdapter;
protected void setAdapterResponse(StaticResponseEndpointAdapter adapter, String filenamepath){
URL url = this.getClass().getResource(filenamepath);
String payload = null;
try {
payload = Resources.toString(url, Charsets.UTF_8);
} catch (IOException e) {
e.printStackTrace();
}
adapter.setMessagePayload(payload);
}
#CitrusTest
public void TestCase02() throws IOException {
http()
.client(CLIENT)
.post()
.payload(new ClassPathResource("templates/thisStartRequestMsg.xml", getClass()))
.dictionary("TC02");
http()
.client(CLIENT)
.response()
.messageType("xml")
.payload(new ClassPathResource("templates/thisStartResponseMsg.xml", getClass()));
action(new AbstractTestAction() {
#Override
public void doExecute(TestContext testContext) {
setAdapterResponse(checkRegistrationEndpointAdapter, "templates/check-registration-v1CheckRegistrationResponseMsg.xml");
}
});
http()
.client(CLIENT)
.response()
.messageType("xml")
.payload(new ClassPathResource("templates/check-registration-v1CheckRegistrationRequestMsg.xml", getClass()))
.dictionary("TC02");
}
How can I apply dictionary to the payload set in my setAdapterResponse method?
Note: this question relates to Can I use Citrus variable in Citrus static response adapter payload?
Static response adapter has currently no support for data dictionaries. I wonder why you put so much effort into static response adapters? Why not using the full Citrus http server power with receiving the request and providing a response inside the test case?

Spring Cloud - Getting Retry Working In RestTemplate?

I have been migrating an existing application over to Spring Cloud's service discovery, Ribbon load balancing, and circuit breakers. The application already makes extensive use of the RestTemplate and I have been able to successfully use the load balanced version of the template. However, I have been testing the situation where there are two instances of a service and I drop one of those instances out of operation. I would like the RestTemplate to failover to the next server. From the research I have done, it appears that the fail-over logic exists in the Feign client and when using Zuul. It appears that the LoadBalancedRest template does not have logic for fail-over. In diving into the code, it looks like the RibbonClientHttpRequestFactory is using the netflix RestClient (which appears to have logic for doing retries).
So where do I go from here to get this working?
I would prefer to not use the Feign client because I would have to sweep A LOT of code.
I had found this link that suggested using the #Retryable annotation along with #HystrixCommand but this seems like something that should be a part of the load balanced rest template.
I did some digging into the code for RibbonClientHttpRequestFactory.RibbonHttpRequest:
protected ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException {
try {
addHeaders(headers);
if (outputStream != null) {
outputStream.close();
builder.entity(outputStream.toByteArray());
}
HttpRequest request = builder.build();
HttpResponse response = client.execute(request, config);
return new RibbonHttpResponse(response);
}
catch (Exception e) {
throw new IOException(e);
}
}
It appears that if I override this method and change it to use "client.executeWithLoadBalancer()" that I might be able to leverage the retry logic that is built into the RestClient? I guess I could create my own version of the RibbonClientHttpRequestFactory to do this?
Just looking for guidance on the best approach.
Thanks
To answer my own question:
Before I get into the details, a cautionary tale:
Eureka's self preservation mode sent me down a rabbit hole while testing the fail-over on my local machine. I recommend turning self preservation mode off while doing your testing. Because I was dropping nodes at a regular rate and then restarting (with a different instance ID using a random value), I tripped Eureka's self preservation mode. I ended up with many instances in Eureka that pointed to the same machine, same port. The fail-over was actually working but the next node that was chosen happened to be another dead instance. Very confusing at first!
I was able to get fail-over working with a modified version of RibbonClientHttpRequestFactory. Because RibbonAutoConfiguration creates a load balanced RestTemplate with this factory, rather then injecting this rest template, I create a new one with my modified version of the request factory:
protected RestTemplate restTemplate;
#Autowired
public void customizeRestTemplate(SpringClientFactory springClientFactory, LoadBalancerClient loadBalancerClient) {
restTemplate = new RestTemplate();
// Use a modified version of the http request factory that leverages the load balacing in netflix's RestClient.
RibbonRetryHttpRequestFactory lFactory = new RibbonRetryHttpRequestFactory(springClientFactory, loadBalancerClient);
restTemplate.setRequestFactory(lFactory);
}
The modified Request Factory is just a copy of RibbonClientHttpRequestFactory with two minor changes:
1) In createRequest, I removed the code that was selecting a server from the load balancer because the RestClient will do that for us.
2) In the inner class, RibbonHttpRequest, I changed executeInternal to call "executeWithLoadBalancer".
The full class:
#SuppressWarnings("deprecation")
public class RibbonRetryHttpRequestFactory implements ClientHttpRequestFactory {
private final SpringClientFactory clientFactory;
private LoadBalancerClient loadBalancer;
public RibbonRetryHttpRequestFactory(SpringClientFactory clientFactory, LoadBalancerClient loadBalancer) {
this.clientFactory = clientFactory;
this.loadBalancer = loadBalancer;
}
#Override
public ClientHttpRequest createRequest(URI originalUri, HttpMethod httpMethod) throws IOException {
String serviceId = originalUri.getHost();
IClientConfig clientConfig = clientFactory.getClientConfig(serviceId);
RestClient client = clientFactory.getClient(serviceId, RestClient.class);
HttpRequest.Verb verb = HttpRequest.Verb.valueOf(httpMethod.name());
return new RibbonHttpRequest(originalUri, verb, client, clientConfig);
}
public class RibbonHttpRequest extends AbstractClientHttpRequest {
private HttpRequest.Builder builder;
private URI uri;
private HttpRequest.Verb verb;
private RestClient client;
private IClientConfig config;
private ByteArrayOutputStream outputStream = null;
public RibbonHttpRequest(URI uri, HttpRequest.Verb verb, RestClient client, IClientConfig config) {
this.uri = uri;
this.verb = verb;
this.client = client;
this.config = config;
this.builder = HttpRequest.newBuilder().uri(uri).verb(verb);
}
#Override
public HttpMethod getMethod() {
return HttpMethod.valueOf(verb.name());
}
#Override
public URI getURI() {
return uri;
}
#Override
protected OutputStream getBodyInternal(HttpHeaders headers) throws IOException {
if (outputStream == null) {
outputStream = new ByteArrayOutputStream();
}
return outputStream;
}
#Override
protected ClientHttpResponse executeInternal(HttpHeaders headers) throws IOException {
try {
addHeaders(headers);
if (outputStream != null) {
outputStream.close();
builder.entity(outputStream.toByteArray());
}
HttpRequest request = builder.build();
HttpResponse response = client.executeWithLoadBalancer(request, config);
return new RibbonHttpResponse(response);
}
catch (Exception e) {
throw new IOException(e);
}
//TODO: fix stats, now that execute is not called
// use execute here so stats are collected
/*
return loadBalancer.execute(this.config.getClientName(), new LoadBalancerRequest<ClientHttpResponse>() {
#Override
public ClientHttpResponse apply(ServiceInstance instance) throws Exception {}
});
*/
}
private void addHeaders(HttpHeaders headers) {
for (String name : headers.keySet()) {
// apache http RequestContent pukes if there is a body and
// the dynamic headers are already present
if (!isDynamic(name) || outputStream == null) {
List<String> values = headers.get(name);
for (String value : values) {
builder.header(name, value);
}
}
}
}
private boolean isDynamic(String name) {
return name.equals("Content-Length") || name.equals("Transfer-Encoding");
}
}
public class RibbonHttpResponse extends AbstractClientHttpResponse {
private HttpResponse response;
private HttpHeaders httpHeaders;
public RibbonHttpResponse(HttpResponse response) {
this.response = response;
this.httpHeaders = new HttpHeaders();
List<Map.Entry<String, String>> headers = response.getHttpHeaders().getAllHeaders();
for (Map.Entry<String, String> header : headers) {
this.httpHeaders.add(header.getKey(), header.getValue());
}
}
#Override
public InputStream getBody() throws IOException {
return response.getInputStream();
}
#Override
public HttpHeaders getHeaders() {
return this.httpHeaders;
}
#Override
public int getRawStatusCode() throws IOException {
return response.getStatus();
}
#Override
public String getStatusText() throws IOException {
return HttpStatus.valueOf(response.getStatus()).name();
}
#Override
public void close() {
response.close();
}
}
}
I had the same problem but then, out of the box, everything was working (using a #LoadBalanced RestTemplate). I am using Finchley version of Spring Cloud, and I think my problem was that I was not explicity adding spring-retry in my pom configuration. I'll leave here my spring-retry related yml configuration (remember this only works with #LoadBalanced RestTemplate, Zuul of Feign):
spring:
# Ribbon retries on
cloud:
loadbalancer:
retry:
enabled: true
# Ribbon service config
my-service:
ribbon:
MaxAutoRetries: 3
MaxAutoRetriesNextServer: 1
OkToRetryOnAllOperations: true
retryableStatusCodes: 500, 502

Why HandshakeRequest doesn't return HttpSession in my ServerEndpointConfig.Configurator?

There is a good example for sharing HttpSession between Websocket and Rest service. (Spring DispatchServlet cannot find resource within Jetty) But it doesn't work for me. I'm not sure is there any thing I'm missing?
I'm using Jetty as websocket server and also I created a WebApp as well which injected by SpringConfig.
private void init() throws Exception
{
Server server = new Server();
// Create SSL Connector
ServerConnector serverConnector = getSSLConnector(server);
// Bundle to server
server.setConnectors(new Connector[] { serverConnector });
// Create request handler collection
HandlerCollection handlers = new HandlerCollection();
// Add WebSocket handler
final ServletContextHandler servletContextHandler = getWebSocketContextHandler();
handlers.addHandler(servletContextHandler);
// Add Servlet handler
handlers.addHandler(getWebAppServletContextHandler());
server.setHandler(handlers);
// Initial WebSocket
WebSocketServerContainerInitializer.configureContext(servletContextHandler);
// Start Jetty
server.start();
server.join();
}
Both WebSocket and Rest are working under same port perfectly, of course, with different context paths.
Now, I created a Rest service:
#RequestMapping(value = "/login", method = RequestMethod.POST)
#Consumes({ MediaType.APPLICATION_JSON_VALUE })
#Produces({ MediaType.APPLICATION_JSON_VALUE })
public #ResponseBody Message login(#RequestBody Credential credential, #Context HttpServletRequest servlerRequest)
{
...
HttpSession session = servlerRequest.getSession(true);
session.setAttribute("userName", credential.getUserName());
...
Message message = new Message();
...
return message;
}
In this service I created a HttpSession and stored something in. As I said, it works, and so does the session.
Rest client:
public void login() throws KeyManagementException, NoSuchAlgorithmException
{
final String loginServiceUri = HTTP_SERVICE_BASE_URI + "/login";
ClientConfig clientConfig = new DefaultClientConfig();
...
Client client = Client.create(clientConfig);
WebResource webResource = client.resource(loginServiceUri);
ClientResponse response = webResource
.type("application/json")
.post(ClientResponse.class, new Credential("user","pass"));
if (response.getStatus() != 200) {
throw new RuntimeException("Failed : HTTP error code : " + response.getStatus());
}
List<NewCookie>cookies = response.getCookies();
ClientEndpointConfigurator.setCookies(cookies); <== Store cookies as well as session to ClientEndpointConfigrator class
Message message = response.getEntity(Message.class);
...
}
ClientEndpointConfigrator class has a static list for all cookies which like this:
public class ClientEndpointConfigurator extends ClientEndpointConfig.Configurator {
private static List<NewCookie> cookies = null;
public static void setCookies(List<NewCookie> cookies) {
ClientEndpointConfigurator.cookies = cookies;
}
...
#Override
public void beforeRequest(Map<String, List<String>> headers) {
...
if(null != cookies)
{
List<String> cookieList = new ArrayList<String>();
for(NewCookie cookie: cookies)
{
cookieList.add(cookie.toString());
}
headers.put("Cookie", cookieList);
}
...
}
}
beforeRequest() method will put all cookies to request header. If you inspect the cookieList, you will see:
[JSESSIONID=tvum36z6j2bc1p9uf2gumxguh;Version=1;Path=/rs;Secure]
Things looks prefect.
Finally, create a server end ServerEndpointConfigurator class, and override the modifyHandshake() method to retrieve the session and cookies
public class SpringServerEndpointConfigurator extends ServerEndpointConfig.Configurator {
#Override
public void modifyHandshake(ServerEndpointConfig sec, HandshakeRequest request, HandshakeResponse response) {
super.modifyHandshake(sec, request, response);
httpSession = (HttpSession)request.getHttpSession(); <== **It returns null here!**
...
}
}
}
I can't get my HttpSession back! and if you print headers out, you will see the cookie has been changed:
Cookie: JSESSIONID="tvum36z6j2bc1p9uf2gumxguh";$Path="/rs"
Any one knows what's the reason?
All right, I figured it out, it's because I put WebSocket and Rest to different context handler. Jetty keeps handlers isolate to each other. To share session information, you have to put them together.
But if someone does want to separate them, it is still possible done by sharing SessionManager or SessionHandler. There are many ways to achieve this, you can inject SessionHandler to each ServletContext or just define it as a static variable and put it on somewhere every one can reach, each way works.

HTTPClient unable to establish route between https and http

I am testing HttpClient 4.2 by hitting a mixture of http and https links.
HttpClient seems to stick with the protocol from the first call. If the first call is http, then all following https calls fail but http calls are fine. And vice versa.
Here is the test code I used.
#Test
public void testNoRedirectMixed() throws ClientProtocolException, IOException {
HttpClient httpclient = new DefaultHttpClient();
httpclient=WebClientDevWrapper.wrapClient(httpclient);
HttpClientParams.setRedirecting(httpclient.getParams(), false);
{
HttpGet httpget = new HttpGet("http://www.hotmail.com");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
assertTrue(EntityUtils.toString(entity).indexOf("com")>0);
}
try {
HttpGet httpget = new HttpGet("https://www.hotmail.com");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
}catch (Exception e) {
e.printStackTrace();
}
{
HttpGet httpget = new HttpGet("http://www.baidu.com");
HttpResponse response = httpclient.execute(httpget);
HttpEntity entity = response.getEntity();
assertTrue(EntityUtils.toString(entity).indexOf("com")>0);
}
}
The second request (https) will fail, but the baidu request is fine.
Caused by: org.apache.http.HttpException: Unable to establish route: planned = {s}->https://www.hotmail.com; current = {s}->http://www.hotmail.com
at org.apache.http.impl.client.DefaultRequestDirector.establishRoute(DefaultRequestDirector.java:842)
I also have to disable redirection because hotmail redirects request: http://www.hotmail.com -> https://www.hotmail.com or https://www.hotmail.com -> https://www.live.com. A similar error is thrown in either cases.
The wrapper is shown below. It is used to accept all certificates.
public class WebClientDevWrapper {
public static HttpClient wrapClient(HttpClient base) {
try {
SSLContext ctx = SSLContext.getInstance("TLS");
X509TrustManager tm = new X509TrustManager() {
public void checkClientTrusted(X509Certificate[] xcs,
String string) throws CertificateException {
}
public void checkServerTrusted(X509Certificate[] xcs,
String string) throws CertificateException {
}
public X509Certificate[] getAcceptedIssuers() {
return new X509Certificate[]{};
}
};
ctx.init(null, new TrustManager[] { tm }, null);
SSLSocketFactory ssf = new SSLSocketFactory(ctx);
ssf.setHostnameVerifier(SSLSocketFactory.ALLOW_ALL_HOSTNAME_VERIFIER);
ClientConnectionManager ccm = base.getConnectionManager();
SchemeRegistry sr = ccm.getSchemeRegistry();
sr.register(new Scheme("https", ssf, 443));
DefaultHttpClient client= new DefaultHttpClient(ccm, base.getParams());
return client;
} catch (Exception ex) {
ex.printStackTrace();
return null;
}
}
}
HttpClient should be able to manage connections absolutely transparently to the user. This problem is likely to be caused by a regression introduced in the 4.2 release (see HTTPCLIENT-1193).
Use either PoolingConnectionManager or SingleConnectionManager instead of the default one until 4.2.1 version is released.
You are trying to use one connection to communicate to a number of different sites. AFAIR You have to create new connection (== new client) for every unique site.