While writing the spring boot contract testing on consumer side, I having problem when request parameters contains special characters. They'll automatically encoding causing the test failed due to the spring consider that the "Query does not match"
"自动制动" has been encoded as "%E8%87%AA%E5%8A%A8%E5%88%B6%E5%8A%A8"
Check the log, i could see:
Query: word = 自动制动 | word: %E8%87%AA%E5%8A%A8%E5%88%B6%E5%8A%A8 <<<<< Query does not match
Here's my groovy file on producer side:
Contract.make {
description "Returns \"Auto hold\"'s canonical value_Mandarin"
name "getSynonym_AutoHold_canonical_Mandarin"
request {
urlPath( "/synonyms"){
headers {"accept: application/json;charset=UTF-8"}
queryParameters {
parameter("filter","canonical")
parameter("lang", "cmn-CHN")
parameter("word","自动制动")
}
}
method GET()
}
response {
status OK()
headers {
contentType applicationJson()
}
body '''
{
"canonical": "autohold",
"word": "自动制动"
}'''
}
}
And here's what I have in consumer side:
#Test
public void testSynonyms_Cmn(){
RestTemplate restTemplate = new RestTemplate();
String url = "http://localhost:8080/synonyms";
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.APPLICATION_JSON_UTF8);
UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url)
.queryParam("filter","canonical")
.queryParam("lang","cmn-CHN")
.queryParam("word","自动制动");
HttpEntity<?> entity = new HttpEntity<>(httpHeaders);
CentralizedSynonyms centralizedSynonyms = restTemplate.exchange(builder.toUriString(), HttpMethod.GET, entity
, CentralizedSynonyms.class).getBody();
assertThat(centralizedSynonyms.getWord()).isEqualTo("自动制动");
assertThat(centralizedSynonyms.getCanonical()).isEqualTo("autohold");
}
I had something similar and fixed it with this:
url value(consumer("/path1/path2/something%3Dsomethingelse"), producer("/path1/path2/something=somethingelse"))
Related
I'm using Spring RestTemplate to perform POST request sending a PDF file. The filename contains some UTF-8 characters (e.g. é, è, à, ê, ë).
The problem is that after sending the request, on the other side where the request is received, the filename doesn't have the expected UTF-8 characters, and I have something like ?popi?.pdf instead.
I've tried to explicitly set UTF-8 charset in RestTemplate, but it still doesn't work.
Here is my code,
public SomeThing storeFile(InputStream content, String fileName) {
Charset utf8 = Charset.forName("UTF-8");
RestTemplate restTemplate = new RestTemplate();
HttpHeaders headersFile = new HttpHeaders();
headersFile.setContentType(MediaType.APPLICATION_OCTET_STREAM);
headersFile.setContentDispositionFormData("file", fileName);
List<Charset> listCharSet = new ArrayList<Charset>();
listCharSet.add(utf8);
headersFile.setAcceptCharset(listCharSet);
InputStreamResource inputStreamResource = new InputStreamResource(content);
HttpEntity<InputStreamResource> requestEntityFile = new HttpEntity<>(inputStreamResource, headersFile);
MultiValueMap<String, Object> multipartRequest = new LinkedMultiValueMap<>();
multipartRequest.add("file", requestEntityFile);
RestTemplate newRestTemplate = new RestTemplate();
MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter = new MappingJackson2HttpMessageConverter();
HttpMessageConverter stringHttpMessageConverter = new StringHttpMessageConverter(Charset.forName("UTF-8"));
newRestTemplate.getMessageConverters().add(0, stringHttpMessageConverter);
newRestTemplate.getMessageConverters().add(mappingJackson2HttpMessageConverter);
FormHttpMessageConverter convForm = new FormHttpMessageConverter();
convForm.setCharset(utf8);
newRestTemplate.getMessageConverters().add(convForm);
HttpHeaders header = new HttpHeaders();
header.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(multipartRequest, header);
ResponseEntity<String> result = newRestTemplate.postForEntity(env.getProperty("core.endpoint") + "/documents", requestEntity, String.class);
}
according to rfc7578 when you POST a file with multipart/form-data you should use "percent-encoding" instead of filename*
NOTE: The encoding method described in [RFC5987], which would add a
"filename*" parameter to the Content-Disposition header field, MUST NOT be used.
it could be easyly realesed:
RestTemplate restTemplate = new RestTemplate();
restTemplate.getMessageConverters().add(0, new FormHttpMessageConverter() {
#Override
protected String getFilename(Object part) {
if (part instanceof Resource) {
Resource resource = (Resource) part;
try {
return URLEncoder.encode(resource.getFilename(), "UTF-8");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
return null;
}
} else {
return null;
}
}
});
Most likely the file encoding's system property of the used JVM hasn't been explicitly set, meanwhile the operating system where JVM runs is not using UTF-8 as the default charset. For instance, if JVM runs on Windows, and we don't specify the charset, the default value will be Windows-1252 instead.
Could you double check the JVM arguments of both applications that send and receive the file? Please ensure that it has -Dfile.encoding=UTF-8 argument before specifying the main class name.
Please also ensure that the application/service which receives the file has been configured to accept UTF-8 charset.
Feel free to also check the other possible related answers, if adding the file.encoding argument on JVMs doesn't help solving the problem,
How to get UTF-8 working in Java webapps?
Spring MVC UTF-8
Encoding
Using Grails 3.0.9, and grabbing the freshest REST API with this snippet in gradle.build:
compile 'org.grails:grails-datastore-rest-client:4.0.7.RELEASE', {
['commons-codec', 'grails-async', 'grails-core',
'grails-plugin-converters', 'grails-web', 'groovy'].each {
exclude module: it
}
}
I am trying to make the following POST request:
def rest = new RestBuilder(headers:["X-LSS-Env":"devmo"], connectTimeout:10000, readTimeout:20000)
response = rest.post("http://..../..") {
accept "application/json"
contentType "application/json"
json jsonBuilder
}
Now, the POST receiver gets the json okay, give back a response okay, but this is the problem: it receives the headers as an empty map or as null!
So, what is the correct way of passing header data to the POST receiver? This is needed because the environment key X-LSS-Env could have different values, which instructs the receiver to do further routing based on it. Same with the GET request of course.
* UPDATE *
The consumer of my POST requests is actually a Java application, running on Apache Tomcat/8.0.26. The is how the service looks on the other side:
private javax.servlet.http.HttpServletRequest hsr;
#POST
#Path("/na")
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response postSomething(Ggfp ggfp ){
try {
Enumeration<String> hnames = hsr.getHeaderNames();
int i = 0;
while (hnames.hasMoreElements()) {
String headerName = hnames.nextElement();
System.out.println(++i+ " headerName: " + headerName);
String val = hsr.getHeader(headerName);
System.out.println(" val: " + val);
}
String hval = hsr.getHeader("X-LSS-Env");
return Response.status(Status.OK).entity("X-LSS-Env is " + hval).build();
} catch (Exception e) {
}
}
Calling this service from Postman works, headers are identified. Calling it from the Grails app results into an empty map - like I am sending no headers!
The RestBuilder constructor never liked the way I used (or abused) it. Here is a clean way of achieving what I set out to do, with tryCatch logic if a timeout transpires.
def makePostWsr(serviceUrl, jsonBuilder) {
try {
def rest = new RestBuilder(connectTimeout:connectTimeout, readTimeout:readTimeout)
def response = rest.post("$wsUrl/$serviceUrl") {
header 'X-LSS-Env', 'devmo'
accept "application/json"
contentType "application/json"
json jsonBuilder
}
response
} catch (Exception e) {
println "== problem makePostWsr on $serviceUrl"
null
}
}
I need help with spring handling an error.
a client service is sending a request accepting two different content types - binary and json. when everything works fine I prefer communicating to my server with binary encoding to save bandwidth. but on error I would like serialise ResponseEntity to json as my binary serialiser do not know how to serialise it to binary format, plus it is better for logging, etc.
I configured instance of ResponseEntityExceptionHandler and I am handling different exceptions from that implementation. but spring always choses binary format as it is first on the accept (or produces) list.
all I get is (because spring do not know how to serialise ResponseEntity to my custom binary format. see AbstractMessageConverterMethodProcessor#writeWithMessageConverters)
org.springframework.web.HttpMediaTypeNotAcceptableException: Could not find acceptable representation
client sends
headers {Accept: [application/custom-binary, application/json]
server's controller is configured to
// pseudo code
#RequestMapping(method = GET, produces = {"application/custom-binary", APPLICATION_JSON_VALUE})
public BannerMetaCollection get(#RequestParam(value = "q") UUID[] q) {
if (q != null) {
return service.getAllDataWith(q);
} else {
throw new IllegalArgumentException("invalid data");
}
}
// pseudo code
public class RestExceptionResolverSupport extends ResponseEntityExceptionHandler {
#ExceptionHandler
public ResponseEntity<Object> illegalArgumentException(IllegalArgumentException ex, WebRequest request {
Object body = errorResponse()
.withCode(BAD_REQUEST)
.withDescription("Request sent is invalid")
.withMessage(ex.getMessage())
.build());
return new ResponseEntity<Object>(body, new HttpHeaders(), HttpStatus.BAD_REQUEST);
}
}
any hints?
What I do to get this to work is that a let my endpoint method return a ResponseEntity and I don't declare what content is produced in the #RequestMapping annotation. I then set the Content-type header myself before returning the response, e.g.
// pseudo code
#RequestMapping(method = GET)
public ResponseEntity<BannerMetaCollection> get(#RequestParam(value = "q") UUID[] q) {
if (q != null) {
HttpHeaders headers = new HttpHeaders();
headers.add(HttpHeaders.CONTENT_TYPE, "application/custom-binary");
return new ResponseEntity<>(service.getAllDataWith(q),
headers,
HttpStatus.OK);
} else {
throw new IllegalArgumentException("invalid data");
}
}
I am using Grails with RESTful to develop my web application. Everything works fine, till I upgrade my application to Grails 2.3. Here is my UrlMappings:
I still send request, submit or do some other things normally, but in POST, PUT requests, the parameters are missing. Server just recognize only the parameters I put on the URL directly, but the remain I enclose in form or model when submit cannot be found in the "params" variable. He is my UrlMappings:
class UrlMappings {
static mappings = {
"/$controller/$action?/$id?"{ constraints {} }
name apiSingle: "/api/$controller/$id"(parseRequest:true){
action = [GET: "show", PUT: "update", DELETE: "delete"]
constraints { id(matches:/\d+/) }
}
name apiCollection: "/api/$controller"(parseRequest:true){
action = [GET: "list", POST: "save"]
}
name api2: "/api/$controller/$action"(parseRequest:true)
name api3: "/api/$controller/$action/$id"(parseRequest:true)
"/"(view:"/welcome")
"500"(view:'/error')
}
}
I have read the latest document of Grails 2.3, at http://grails.org/doc/latest/guide/theWebLayer.html#restfulMappings
but I think it is not clear. I have tried it follow the documentation but have no result. And there are no any sample about using Grails 2.3 with RESTful for me to refer.
How can I make it work normally as before, and can access all parameter values in REST request? Thank you so much!
According to this http://grails.1312388.n4.nabble.com/Grails-2-3-and-parsing-json-td4649119.html parseRequest has no effect since Grails 2.3
If you use JSON as request body you can accees request params as request.JSON.paramName
As a workaround you can add a filter that will populate data from JSON to params:
class ParseRequestFilters {
def filters = {
remoteCalls(uri: "/remote/**") {
before = {
if (request.JSON) {
log.debug("Populating parsed json to params")
params << request.JSON
}
}
}
}
}
Adding on to Kipriz's answer and cdeszaq's comment, you can write a recursive method to inject nested params. Something along these lines:
public void processNestedKeys(Map requestMap, String key) {
if (getParameterValue(requestMap, key) instanceof JSONObject) {
String nestedPrefix = key + ".";
Map nestedMap = getParameterValue(requestMap, key)
for (Map.Entry<String, Object> entry : nestedMap.entrySet()) {
String newKey = nestedPrefix + entry.key;
requestMap.put(newKey, getParameterValue(nestedMap, entry.key))
processNestedKeys(requestMap, "${nestedPrefix + entry.key}");
}
}
}
public static Map populateParamsFromRequestJSON(def json) {
Map requestParameters = json as ConcurrentHashMap
for (Map.Entry<String, Object> entry : requestParameters.entrySet()) {
processNestedKeys(requestParameters, entry.key)
}
return requestParameters
}
A typical SOAP client request using JAX-WS might be
FooService service = new FooService();
FooPort port = service.getFooPort();
FooPayload payload = new FooPayload();
payload.setHatSize(3);
payload.setAlias("The Hat");
...
port.processRequest(payload);
This generates an HTTP request content something like
<?xml ... ?>
<S:Envelope xmlns:S="http://...soap-envelope">
<S:Body>
<!-- payload -->
</S:Body>
</S:Envelope>
By manipulating the arguments to the port.processRequest() call you can only affect the "payload" part. You can't affect the outer part of the XML message.
I want to insert a SOAP header just before the SOAP Body
<S:Header>
<X:Security xmlns:X="http://...wsssecurity...>
<X:BinarySecurityToken>kjh...897=</X:BinarySecurityToken>
</X:Security>
</S:Header>
How do I do that?
Thanks Nuno,
Just as soon as I work out how to log in properly to stackoverflow.com I'll do the right thing with your reply.
In the mean time here's the code I ended up with:
FooService service = new FooService();
service.setHandlerResolver(new HandlerResolver() {
public List<Handler> getHandlerChain(PortInfo portInfo) {
List<Handler> handlerList = new ArrayList<Handler>();
handlerList.add(new RGBSOAPHandler());
return handlerList;
}
});
FooPort port = service.getFooPort();
FooPayload payload = new FooPayload();
payload.setHatSize(3);
payload.setAlias("The Hat");
...
port.processRequest(payload);
and
class RGBSOAPHandler implements SOAPHandler<SOAPMessageContext> {
public Set<QName> getHeaders() {
return new TreeSet();
}
public boolean handleMessage(SOAPMessageContext context) {
Boolean outboundProperty =
(Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
if (outboundProperty.booleanValue()) {
SOAPMessage message = context.getMessage();
try {
SOAPEnvelope envelope = context.getMessage()
.getSOAPPart().getEnvelope();
SOAPFactory factory = SOAPFactory.newInstance();
String prefix = "X";
String uri = "http://...wsssecurity...";
SOAPElement securityElem =
factory.createElement("Security",prefix,uri);
SOAPElement tokenElem =
factory.createElement("BinarySecurityToken",prefix,uri);
tokenElem.addTextNode("kjh...897=");
securityElem.addChildElement(tokenElem);
SOAPHeader header = envelope.addHeader();
header.addChildElement(securityElem);
} catch (Exception e) {
System.out.println("Exception in handler: " + e);
}
} else {
// inbound
}
return true;
}
public boolean handleFault(SOAPMessageContext context) {
throw new UnsupportedOperationException("Not supported yet.");
}
public void close(MessageContext context) {
//
}
}
you might want to look at handlers and handler chains.- I recently had to add a cookie to a given Webservice call and that was how i did it, just created a handler that intercepted the initial call and injected the cookie, you can also manipulate the call headers with a Pivot Handler
for add Soap header, if you implement the WS on the web application server, the Was will add security part at header , after you have configure as per WS-SECURITY standard , such as web-policy etc. I don't understand why need add yourself except the encrypted content part , such as encrypted password etc