Setting the host header in java httpClient - httpclient

I am trying to set the host header when using java httpClient. I have found that it can be done in apache httpClient using setVirtualHost. How can I do the same with the java httpClient.

The Host-Header is automatically set if you send a request. The following program sends a GET request to httpbin.org/headers and displays the headers which arrived on server side.
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
public class Test {
public static void main(String[] args) throws Exception {
HttpClient client = HttpClient.newHttpClient();
HttpRequest request = HttpRequest.newBuilder()
.uri(new URI("http://httpbin.org/headers"))
.GET()
.build();
HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());
System.out.println(response.body());
}
}
The output of this program is
{
"headers": {
"Content-Length": "0",
"Host": "httpbin.org",
"Http2-Settings": "AAEAAEAAAAIAAAABAAMAAABkAAQBAAAAAAUAAEAA",
"Upgrade": "h2c",
"User-Agent": "Java-http-client/11.0.2",
"X-Amzn-Trace-Id": "Root=1-5e89b28a-7028ca70c49af58d120493b1"
}
}
You can change the value of the Host field by defining another URI, e.g. if the URI http://www.httpbin.org/headers is used, then the value of the Host header field is www.httpbin.org.
The Host header however cannot be set individually. A set of disallowed headers is defined in class jdk.internal.net.http.common.Utils.
private static final Set<String> DISALLOWED_HEADERS_SET;
static {
// A case insensitive TreeSet of strings.
TreeSet<String> treeSet = new TreeSet<>(String.CASE_INSENSITIVE_ORDER);
treeSet.addAll(Set.of("connection", "content-length",
"date", "expect", "from", "host", "upgrade", "via", "warning"));
DISALLOWED_HEADERS_SET = Collections.unmodifiableSet(treeSet);
}

Related

Token mismatch exception when deploying in any VM

Here is my csrf and cors handler of my vertx application
#Log4j2
public class CsrfVerticle extends AbstractVerticle {
private final Set<HttpMethod> httpMethodSet =
new HashSet<>(Arrays.asList(GET, POST, OPTIONS, PUT, DELETE, HEAD));
private final Set<String> headerSet = new HashSet<>(
Arrays.asList("Content-Type", "Authorization", "Origin", "Accept", "X-Requested-With",
"Cookie", "X-XSRF-TOKEN"));
private Connection dbConnection;
private WebClient webClient;
private Vertx vertx;
public void start() throws Exception {
super.start();
HttpServer httpServer = TestService.vertx.createHttpServer();
Router router = Router.router(TestService.vertx);
SessionStore store = LocalSessionStore.create(vertx);
SessionHandler sessionHandler = SessionHandler.create(store)
.setCookieSameSite(CookieSameSite.STRICT)
.setCookieHttpOnlyFlag(false);
router.route().handler(LoggerHandler.create());
if (TestService.serviceConfiguration.isEnableCSRF()) {
router.route()
.handler(CorsHandler.create("*").allowedMethods(httpMethodSet).allowedHeaders(headerSet)
.allowCredentials(true).addOrigin(TestService.serviceConfiguration.getFrontendUrl()));
router.route().handler(
CSRFHandler.create(vertx, csrfSecret()).setCookieHttpOnly(false))
.handler(sessionHandler);
} else {
router.route()
.handler(CorsHandler.create("*").allowedMethods(httpMethodSet).allowedHeaders(headerSet)
.allowCredentials(true)).handler(sessionHandler);
}
dbConnection = createConnection(TestService.serviceConfiguration.getJdbcConfig());
TestAuth testAuth = new TestAuth(TestService.serviceConfiguration.getUsername(),
TestService.serviceConfiguration.getPassword());
AuthenticationHandler basicAuthHandler = BasicAuthHandler.create(testAuth);
router.route("/student/*").handler(basicAuthHandler);
router.route("/student/add").method(HttpMethod.POST).handler(this::handleAddUser);
router.route("/student/get").method(HttpMethod.GET).handler(this::handleGetUser);
router.route("/student/delete").method(HttpMethod.DELETE)
.handler(this::handleDeleteUser);
router.route("/student/update").method(HttpMethod.PUT).handler(this::handleUpdateUser);
httpServer.requestHandler(router).listen(TestService.serviceConfiguration.getPort());
log.info("Console Server Verticle Started Successfully. Listening to {} port",
TestService.serviceConfiguration.getPort());
}
I am able to receive cookies in browser and send it back along with updated X-XSRF-TOKEN attached to the header
Everything works fine in my local but when deploying in VM I get the below error for all post requests
ctx.fail(403, new IllegalArgumentException("Token signature does not match"));
from csrf handler of vertx.
Here are the frontend code to add x-xsrf-token when sending requests to backend
createXsrfHeader(headers: HttpHeaders) {
let xsrfToken = Cookies.get('XSRF-TOKEN')
let sessionToken = Cookies.get('vertx-web.session')
if(xsrfToken)
headers = headers.append('X-XSRF-TOKEN', xsrfToken);
// if(xsrfToken && sessionToken)
// headers = headers.append('Cookie', `XSRF-TOKEN=${xsrfToken}; vertx-web.session=${sessionToken}`);
return headers;
}
[Adding header to post request]
callPostRequest(subUrl: string,reqData: any) {
let headers = new HttpHeaders();
headers = this.createAuthorizationHeader(headers);
headers = this.createXsrfHeader(headers);
return this.http.post<any>(this.basicApiUrl+subUrl, reqData, {
headers: headers,
withCredentials : true
}).pipe(map(resData => {
// console.log(resData);
return resData;
}));
}
[Adding header to put request]
callPutRequest(subUrl: string,reqData: any) {
let headers = new HttpHeaders();
headers = this.createAuthorizationHeader(headers);
headers = this.createXsrfHeader(headers);
return this.http.put<any>(this.basicApiUrl+subUrl, reqData,{
headers: headers,
withCredentials : true
}).pipe(map(resData => {
// console.log(resData);
return resData;
}));
}
[Adding header to delete request]
callDeleteRequest(subUrl: string,reqData?: any) {
let headers = new HttpHeaders();
headers = this.createAuthorizationHeader(headers);
headers = this.createXsrfHeader(headers);
return this.http.delete<any>(this.basicApiUrl+subUrl, {
headers: headers,
withCredentials : true
}).pipe(map(resData => {
// console.log(resData);
return resData;
}));
}
Is there any ways to solve it.
I believe your problem is here:
router.route() // <--- HERE
.handler(
CSRFHandler.create(vertx, csrfSecret())
.setCookieHttpOnly(false))
.handler(sessionHandler);
You are telling the application to create a new CSRF token for each request that is happening, instead of being specific of which end points are really form-based endpoints.
Imagine the following, your form is on /student/form your browser may request:
/student/form (new CSRF token: OK)
/images/some-image-in-the-html.png (new CSRF token: probably Wrong)
/css/styles.css (new CSRF token: probably Wrong)
...
Now the issue is that the 1st call did correctly generated a token, but the following 2+ will generate new tokens too and these won't match the 1st so your tokens are always misaligned.
You probably need to be more specific with the resources you want to protect, from your code I am assuming that you probably want something like:
router.route(""/student/*") // <--- Froms are always here under
.handler(
CSRFHandler.create(vertx, csrfSecret())
.setCookieHttpOnly(false))
.handler(sessionHandler);
Be careful if calling other endpoints would affect the forms too. Note that you can add multiple handlers per route, so you can be more explicit with:
router.route(""/student/add")
// 1st always CSRF checks
.handler(
CSRFHandler.create(vertx, csrfSecret())
.setCookieHttpOnly(false))
// and now we the handler that will handle the form data
.handler(this::handleAddUser)

ReactiveFeignClient - how to propagate headers from controller to client without auth

I received X-Forwarded-Host and X-Forwarded-Proto in my controller endpoints, and the endpoint has a reactive pipeline to call a ReactiveFeignClient class.
These headers should be propagated to my client requests, but as I see it, it has not. I have no Principal in this pipeline, because the endpoints needs no auth, so I cannot use ReactiveSecurityContextHolder.withAuthentication(user)
I already added a WebFilter to read headers from request:
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
return chain.filter(exchange).subscriberContext((context) -> {
ServerHttpRequest request = exchange.getRequest();
Map<String, String> headers = (Map)request.getHeaders().toSingleValueMap().entrySet().stream().filter((entry) -> {
return ((String)entry.getKey()).equalsIgnoreCase(this.authLibConfig.getXForwardedHostHeader()) || ((String)entry.getKey()).equalsIgnoreCase(this.authLibConfig.getXForwardedProtoHeader());
}).collect(Collectors.toMap(Entry::getKey, Entry::getValue));
System.out.println("Adding all headers now: ");
context.put("headers_to_propagate", headers);
return context;
});
}
But I don't know where in the config of client can I retrieve them from the Context and put into requests in client.
Now I do this:(
#Bean
public ReactiveHttpRequestInterceptor forwardingHeadersInterceptor(ReactiveFeignUtils reactiveFeignUtils) {
return reactiveFeignUtils::mutateRequestHeadersForNoAuthRequests;
}
And:
public Mono<ReactiveHttpRequest> mutateRequestHeadersForNoAuthRequests(ReactiveHttpRequest reactiveHttpRequest) {
return Mono.subscriberContext().doOnNext((context) -> {
System.out.println("Current context: " + context.toString());
if (context.hasKey("headers_to_propagate")) {
System.out.println("Getting all host headers: ");
reactiveHttpRequest.headers().putAll((Map)context.get("headers_to_propagate"));
}
}).thenReturn(reactiveHttpRequest);
}
But no headers are forwarded.
I ended up creating a customized class implementing Authentication and add these fields as metadata property to it; because even though this endpoint requires no auth, headers related to member id and other auth info are received, so I can construct an Authentication principal.
Actually as I see, working with this object is the only way.

Encoded special characters in parameters causing spring boot contract testing query does not match problem

While writing the spring boot contract testing on consumer side, I having problem when request parameters contains special characters. They'll automatically encoding causing the test failed due to the spring consider that the "Query does not match"
"自动制动" has been encoded as "%E8%87%AA%E5%8A%A8%E5%88%B6%E5%8A%A8"
Check the log, i could see:
Query: word = 自动制动 | word: %E8%87%AA%E5%8A%A8%E5%88%B6%E5%8A%A8 <<<<< Query does not match
Here's my groovy file on producer side:
Contract.make {
description "Returns \"Auto hold\"'s canonical value_Mandarin"
name "getSynonym_AutoHold_canonical_Mandarin"
request {
urlPath( "/synonyms"){
headers {"accept: application/json;charset=UTF-8"}
queryParameters {
parameter("filter","canonical")
parameter("lang", "cmn-CHN")
parameter("word","自动制动")
}
}
method GET()
}
response {
status OK()
headers {
contentType applicationJson()
}
body '''
{
"canonical": "autohold",
"word": "自动制动"
}'''
}
}
And here's what I have in consumer side:
#Test
public void testSynonyms_Cmn(){
RestTemplate restTemplate = new RestTemplate();
String url = "http://localhost:8080/synonyms";
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.APPLICATION_JSON_UTF8);
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url)
.queryParam("filter","canonical")
.queryParam("lang","cmn-CHN")
.queryParam("word","自动制动");
HttpEntity<?> entity = new HttpEntity<>(httpHeaders);
CentralizedSynonyms centralizedSynonyms = restTemplate.exchange(builder.toUriString(), HttpMethod.GET, entity
, CentralizedSynonyms.class).getBody();
assertThat(centralizedSynonyms.getWord()).isEqualTo("自动制动");
assertThat(centralizedSynonyms.getCanonical()).isEqualTo("autohold");
}
I had something similar and fixed it with this:
url value(consumer("/path1/path2/something%3Dsomethingelse"), producer("/path1/path2/something=somethingelse"))

GET img/png with grails.plugins.rest.client.RestResponse

I am using grails.plugins.rest.client.RestResponse to GET a binary resource of content-type png from a rest endpoint.
import grails.plugins.rest.client.RestBuilder
import grails.plugins.rest.client.RestResponse
String url = "http://localhost:8080/rest-endpoint/image.png"
RestBuilder rest = new RestBuilder()
RestResponse restResponse = rest.get(url){
auth username, password
accept "image/png"
contentType "image/png"
}
byte[] png_image = restResponse.responseEntity.body
println "length " + png_image.length
I'm not sure why the length that is returned is ~500bytes less than expected and I have tried with a different image and a different url and the length returned is lower everytime. Any idea why?
I had the same problem and ended up using HTTPBuilder instead.
import groovyx.net.http.ContentType
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.Method
class RestUtil {
static byte[] getBytes(String url) {
new HTTPBuilder(url).request(Method.GET, ContentType.BINARY) {
requestContentType = ContentType.BINARY
response.success = { resp, binary ->
return binary.bytes
}
}
}
}
Hope that's helpful.

How can I get authorization token in rest API using rest assured? Is it possible?

Currently using Postman I have to do post request to API_URL/login and I pass username and password and in return i get token see below:
Example Request:
/login
POST
Body
{
"username": "admin",
"password": "admin"
}
Return
{
"token": "1234-56789-..."
}
Can you tell me how would I do .. i tried using .parameters but it says deprecated...
You'd have to know what the schema is of the response. For instance, if the output is like thus:
{
"success": true,
"authorization_token": "eyJ0eXAiOiJCZWFy...",
"refresh_token": "eyJ0eXAiOiJCZWFyZ...",
"type": "Bearer",
...
}
Knowing the schema, you can extract the entire output and parse through it, like thus:
import io.restassured.response.Response;
Response response =
RestAssured.given().
header("Content-Type", "application/json").
body(loginPayload).
when().
post("/login").
then().
log().ifError().
statusCode(200).
contentType("application/vnd.api+json").
body("$", hasKey("authorization_token")). //authorization_token is present in the response
body("any { it.key == 'authorization_token' }", is(notNullValue())). //authorization_token value is not null - has a value
extract().response();
The auth token could then be set to a string variable
String auth_token = response.path("authorization_token").toString();
extract().response(); returns the entire reponse, but you could change this to extract().path("authorization_token") for just single string
REST Assured supports mapping Java objects to/from JSON. You just need a JSON parser such as Jackson or Gson on the classpath.
First define a class to represent the user credentials:
class Credentials {
private String username;
private String password;
// Getters and setters
}
Then define class to represent the authentication token:
class AuthenticationToken {
private String token;
// Getters and setters
}
And finally perform the request to exchange the credentials for the token:
#Test
public void authenticate() {
Credentials credentials = new Credentials();
credentials.setUsername("admin");
credentials.setPassword("password");
AuthenticationToken authenticationToken =
given()
.accept("application/json")
.contentType("application/json")
.body(credentials)
.expect()
.statusCode(200)
.when()
.post("/login")
.then()
.log().all()
.extract()
.body().as(AuthenticationToken.class);
assertNotNull(authenticationToken.getToken());
}