Jersey web service non-ascii characters - rest

I am new to REST and jersey. I wrote a simple RESTful Web Service using Jersey 1.17 API. The Web service accepts data through POST method. When I pass data having non-ascii characters, it does not read them correctly.
#POST
#Path("hello")
#Consumes(MediaType.APPLICATION_FORM_URLENCODED + ";charset=UTF-8")
public Response hello(#FormParam("message") String message) {
System.out.println(message);
return Response.status(200).entity("hello" + message).build();
}
When I pass data having non-ascii characters in parameter 'message' it does not print it correctly.
curl --data "message=A função, Ãugent" http://localhost:8080/search/hello/
POST method prints "A fun??o, ?ugent"

I do not think Jersey is caring about the charset that is defined at #Consumes. I guess Jersey simply uses the request.getParameter method that uses the encoding of the request to resolve parameters.
You have many options to set the encoding:
In case the servlet container supports, set the default encoding of the connector
Set the default encoding of the jvm to UTF8
Create a Servlet Filter that catches this call and call request.setCharacterEncoding("UTF8"); In this case you must ensure that setCharacterEncoding is called before any other getter function (like getParameter) as the character encoding is set during the first get call on the request.
Do a transform on the parameter value by hand. You can get the ServletRequest and query the encoding. After that you can say:
new String(message.getBytes(currentEncoding), "UTF8");
In your case I would prefer the third one.

Related

Springboot doesn't let through percentage and backslash in URL

We got a get request that sends string characters in url, so we use path variables to receive them. Apparently there is no way that the calling service would change its method of calling backend so we need to be able to accept a url with the following unencoded characters:
When percentage sign % is sent a http 400 is returned. It does go through if the two characters following % make up an UTF-encoded character
Backslash is converted into a forward slash. I need it to stay backslash.
I'm guessing these might be Tomcat or servlet configuration issues.
(spring boot version 1.5.14.RELEASE)
Percent signs (%) should be no problem if you properly URL encode them (%25). However, slashes and backslashes will not work with Tomcat, even if you encode them (%2F and %5C).
You could set the following properties when running the application:
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
-Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true
However, this won't fix the issue, because in this case, those encoded slashes will be recognized as real ones. So, let's say you have the following controller:
#ResponseBody
#RequestMapping("/api/{foo}")
public String getFoo(#PathVariable String foo) {
return foo;
}
Well, then if you call /api/test%5Ctest, it won't be able to find the correct path. A solution to this problem is to use wildcard matchers and to parse the URL itself from the incoming HttpServletRequest:
#RequestMapping("/api/**")
public String getFoo(HttpServletRequest request) {
// ...
}
Another solution is to use a completely different web container. For example, when using Jetty, this isn't a problem at all, and URL encoded slashes and backslashes will both work.
Spring 5 now blocks encoded percent signs by default. To enable them, create a new Bean that calls setAllowUrlEncodedPercent()
#Bean
public HttpFirewall allowEncodedParamsFirewall() {
StrictHttpFirewall firewall = new StrictHttpFirewall();
firewall.setAllowUrlEncodedPercent(true);
return firewall;
}
There are similar method-calls for forward- and backwards-slash
What you are experiencing is not specific to Spring Boot. Instead, it's a restriction of HTTP.
The HTTP standard requires that any URL containing the percent characters must be decoded by the web server (cf page 36):
If the Request-URI is encoded using the "% HEX HEX" encoding [42], the
origin server MUST decode the Request-URI in order to properly
interpret the request.
As a result, it's not possible to escape the slash character reliably.
Therefore, when the slash is used in a URL – with or without encoding – it will be treated as a path separator. So it cannot be used in a Spring Boot path variable. Similar problem exist for the percent sign and backslash.
Your best options are to use query parameters or a POST request.
In the following URL, the value test_with_/and_% is transmitted:
https://host/abc/def?text=test_with_%2F_and%25
final String path =
request.getAttribute(HandlerMapping.PATH_WITHIN_HANDLER_MAPPING_ATTRIBUTE).toString();
final String bestMatchingPattern =
request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE).toString();
String arguments = new AntPathMatcher().extractPathWithinPattern(bestMatchingPattern, path);
if (null != arguments && !arguments.isEmpty()) {
pattern = pattern + '/' + arguments;
}
I also faced similar problem and I have used this so hope this might help

Rest service that accepts a URL as a parameter

I'm trying to pass a complete URL as a parameter to a java-based REST service (GET), but I'm not sure how to format it in order to avoid a "HTTP 400 Bad Request". I've tried Base64 encoding, but still get the 400 error. I think part of the problem is that the url contains a question mark, "?", since it seems to be fine if I remove that and pass the url as-is. I'm not sure what is the problem when its encoded.
example url - http://my.site.com/testing-service?some+parms
method annotations:
#GET
#Path("/{fullurl}")
#Produces("application/json")
public Response findByUrl(#PathParam("fullurl") String fullurl)
...
(I've updated the description with a little more detail per the first couple of comments)
Apparently the encoding approach was close, but Base64 (java or commons-code) didn't work for whatever reason (length perhaps?). I found switching to Base32 (commons-code) works for my situation.

Grails rest plugin not encoding ampersand

I'm using the Grails rest plugin, and having issues with parameters containing an ampersand. Here is an example of my query:
def query = [
method: 'artist.getinfo',
artist: 'Matt & Kim',
format: 'json'
]
withRest(uri:'http://ws.audioscrobbler.com/') {
def resp = get(path: '/2.0/', query: query)
}
I think that the get method should automatically URL encode the parameters in query - it correctly converts spaces to '+'. However, it leaves the ampersand as is, which is incorrect (it should be encoded to %26).
I tried manually encoding the artist name before calling get, but then the rest plugin encodes the percent sign!
I turned on logging for the rest client, so I can see what URLs it's requesting.
Originally: http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=Matt+&+Kim&format=json
If I manually encode the name: http://ws.audioscrobbler.com/2.0/?method=artist.getinfo&artist=Matt+%2526+Kim&format=json
Do I need to set an encoding type? (the last.fm API specifies UTF-8) Is this a bug?
As of version 0.7, the rest plugin is using a version of HTTPBuilder which has issues encoding (and decoding) the ampersand character.
There is a JIRA Issue about this with a suggested workaround (upgrading HTTPBuilder to >= 0.5.2)

How to produce both xml and json for a rest based service?

I am trying to produce both xml and json from my rest service.
#Produces({"application/xml", "application/json"})
However, when I try to use the service using curl/SOAPUI, I get back either xml or json depending on which is mentioned first. In simple words, only the first method is considered. Is there a workaround?
You should check this link out - oracle docs for #Produces
The spec says that it does indeed default to the first one if that is acceptable as specified by the media type on the request. You should check your soapUI tool and see what headers you are sending. If they are both being sent you will get a response with the first one listed in your #Produces annotation.

Parsing response from the WSDL

I've generated the web service client in eclipse for the OpenCalais WSDL using the "develop" client type. Actually I was following this post so not really going in detail. Now when I get the results this way: new CalaisLocator().getcalaisSoap().enlighten(key, content, requestParams);, I get the String object, containing the response XML. Sure it's possible to parse that XML, but I think there must be some way to do it automatically, e.g. getting the response object in the form of some list whatsoever?
The response from the SOAP interface is already parsed. The englighten() method returns an XML string. When you call it with SOAP, this response is wrapped within even more XML. The SOAP library already parses the outer SOAP XML and returns the result of the enlighten() method, which is also XML.