Tomcat, JAX-RS, Jersey, #PathParam: how to pass dots and slashes? - rest

Having a method like this:
#GET #Path("/name/{name}")
#Produces(MediaType.TEXT_PLAIN)
public String getProperty(#PathParam("name") String name) {
System.out.println(name);
}
How do I pass a value like "test./test"?
/name/test./test gives HTTP 404
/name/test.%2Ftest gives HTTP 400
/name/test.%252Ftest prints test%2Ftest
But if I do name = URLDecoder.decode(name); it prints /test and the first part of test. disappears.
There is one or two questions like this already but they are old and there was no good solution found, I thought I'll ask again.

The pattern in the #Path annotation is internally turned into a regular expression, with the template parts matching only selected characters by default. In particular, they normally don't match / characters; that's almost always the right thing to do (as it lets you put templates part way through a path) but in this case it isn't as you're wanting to consume the whole subsequent path. To get everything, we have to override the regular expression fragment for that particular template; this is actually pretty easy, since we just put in the template fragment a : followed by the RE that we want to use:
#GET #Produces(MediaType.TEXT_PLAIN)
#Path("/name/{name:.+}")
public String getProperty(#PathParam("name") String name) {
return name;
}
This will match all characters after the /name/ (up to but not including any ? query part) but will only match if there's something there at all. Be aware that if you have any other #Path("/name/...") things about, things can get really confusing! So don't do that.

If you using tomcat, and want pass / in pathparam. besides the #Path("/name/{name:.+}") stuff as 'Donal Fellows' said, you should add -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true to your jvm arguments, see also tomcat security-howto.

Try specifying the encoding type, the following works for me with /name/test.%252Ftest:
System.out.println(URLDecoder.decode(name, "UTF-8"));
return URLDecoder.decode(name, "UTF-8");

Glassfish v4 accept encoded scape for slash %2f. Then we can pass the encoded String test.%2Ftest and get the result test./test using URLDecoder.decode(name, "UTF-8"). I think this is a better solution especially when you have many params in one request. Using the path #Path("/name/{name:.+}") is great solution when we have few parameters in a request.
Using %252f complicates the client request becouse are needed to contruct the encoding request String manually. With glassfish v4 it's easy to use percent encoding with URLEncoder.encode in client and URLDecoder.decode in server to wished Strings. The most programing languages has percent encoding and decoding, therefore it's perfect solution.
I tried enable encoded slash in glassfish v3 but no success, here is the sintaxe I tried used
bin\asadmin set configs.config.server-config.network-config.protocols.protocol.http-listener-1.http.encoded-slash-enabled=true configs.config.server-config.network-config.protocols.protocol.http-listener-1.http.encoded-slash-enabled=true
Command set executed successfully.
Regards
Cassio Seffrin

Related

Springboot doesn't let through percentage and backslash in URL

We got a get request that sends string characters in url, so we use path variables to receive them. Apparently there is no way that the calling service would change its method of calling backend so we need to be able to accept a url with the following unencoded characters:
When percentage sign % is sent a http 400 is returned. It does go through if the two characters following % make up an UTF-encoded character
Backslash is converted into a forward slash. I need it to stay backslash.
I'm guessing these might be Tomcat or servlet configuration issues.
(spring boot version 1.5.14.RELEASE)
Percent signs (%) should be no problem if you properly URL encode them (%25). However, slashes and backslashes will not work with Tomcat, even if you encode them (%2F and %5C).
You could set the following properties when running the application:
-Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
-Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true
However, this won't fix the issue, because in this case, those encoded slashes will be recognized as real ones. So, let's say you have the following controller:
#ResponseBody
#RequestMapping("/api/{foo}")
public String getFoo(#PathVariable String foo) {
return foo;
}
Well, then if you call /api/test%5Ctest, it won't be able to find the correct path. A solution to this problem is to use wildcard matchers and to parse the URL itself from the incoming HttpServletRequest:
#RequestMapping("/api/**")
public String getFoo(HttpServletRequest request) {
// ...
}
Another solution is to use a completely different web container. For example, when using Jetty, this isn't a problem at all, and URL encoded slashes and backslashes will both work.
Spring 5 now blocks encoded percent signs by default. To enable them, create a new Bean that calls setAllowUrlEncodedPercent()
#Bean
public HttpFirewall allowEncodedParamsFirewall() {
StrictHttpFirewall firewall = new StrictHttpFirewall();
firewall.setAllowUrlEncodedPercent(true);
return firewall;
}
There are similar method-calls for forward- and backwards-slash
What you are experiencing is not specific to Spring Boot. Instead, it's a restriction of HTTP.
The HTTP standard requires that any URL containing the percent characters must be decoded by the web server (cf page 36):
If the Request-URI is encoded using the "% HEX HEX" encoding [42], the
origin server MUST decode the Request-URI in order to properly
interpret the request.
As a result, it's not possible to escape the slash character reliably.
Therefore, when the slash is used in a URL – with or without encoding – it will be treated as a path separator. So it cannot be used in a Spring Boot path variable. Similar problem exist for the percent sign and backslash.
Your best options are to use query parameters or a POST request.
In the following URL, the value test_with_/and_% is transmitted:
https://host/abc/def?text=test_with_%2F_and%25
final String path =
request.getAttribute(HandlerMapping.PATH_WITHIN_HANDLER_MAPPING_ATTRIBUTE).toString();
final String bestMatchingPattern =
request.getAttribute(HandlerMapping.BEST_MATCHING_PATTERN_ATTRIBUTE).toString();
String arguments = new AntPathMatcher().extractPathWithinPattern(bestMatchingPattern, path);
if (null != arguments && !arguments.isEmpty()) {
pattern = pattern + '/' + arguments;
}
I also faced similar problem and I have used this so hope this might help

Wrong NSURLQueryItem percentage encoding for Google CSE

I'm writing app using Google custom search engine.
I received my search engine ID XXXXXXXX219143826571:7h9XXXXXXX (most interesting part bold).
Now I'm trying to use NSURLQueryItem to embed my ID into URL by using:
let params = ["cx" : engineID,...]
...
components.queryItems = parameters.map {
NSURLQueryItem(name: String($0), value: String($1))
}
It should percentage escape item to XXXXXXXX219143826571%3A7h9XXXXXXX (This value I'm getting when using Google APIs explorer while testing, it shows url dress that was used). It is not doing it. I'm getting url without escaping, no changes. If I use escaped string as engine ID in this mapping, I'm getting escaped string XXXXXXXX219143826571%253A7h9XXXXXXX (additional '25' is added to query).
Can someone tell me how to fix it? I don't want to use String and then convert it to URL by NSURL(string: str)!. It is not elegant.
Edit:
I'm using app Info.plist to save ID and I retrieve it by calling:
String(NSBundle.mainBundle().objectForInfoDictionaryKey("ApiKey")!)
Colons are allows in the query part of a URL string. There should be no need to escape them.
Strictly speaking, the only things that absolutely have to be encoded in that part of a URL are ampersands, hash marks (#), and (assuming you're doing a GET query with form encoding) equals signs. However, question marks in theory may cause problems, slashes are technically not allowed (but work just fine), and semicolons are technically allowed (but again, work in practice).
Colons, AFAIK, only have special meaning in the context of paths (if the OS treats it as a path separator) and in that it separates the scheme (protocol) from the rest of the URL.
So don't worry about the colon being unencoded unless the Google API barfs for some reason.

Rest service that accepts a URL as a parameter

I'm trying to pass a complete URL as a parameter to a java-based REST service (GET), but I'm not sure how to format it in order to avoid a "HTTP 400 Bad Request". I've tried Base64 encoding, but still get the 400 error. I think part of the problem is that the url contains a question mark, "?", since it seems to be fine if I remove that and pass the url as-is. I'm not sure what is the problem when its encoded.
example url - http://my.site.com/testing-service?some+parms
method annotations:
#GET
#Path("/{fullurl}")
#Produces("application/json")
public Response findByUrl(#PathParam("fullurl") String fullurl)
...
(I've updated the description with a little more detail per the first couple of comments)
Apparently the encoding approach was close, but Base64 (java or commons-code) didn't work for whatever reason (length perhaps?). I found switching to Base32 (commons-code) works for my situation.

How to build a Uri in Spray?

I would like to make a simple GET request via Spray with a few query parameters
Get("http://localhost/user?email=abc+a#abc.com")
However + means a space in application/x-www-form-urlencoded content resulting the call to http://localhost/user?email=abc a#abc.com (with a space instead of plus sign).
I could use a non-Spray java.net.URLEncoder to encode the URL before passing it to the GET request however I doing this every time seems like a hack.
Is there a Spray way of applying query parameters and encoding them?
Uri("http://localhost/").withQuery(Map("email"->"abc+a#abc.com")) is a nice way to construct a Uri but it doesn't encode the params as well...
Actually Uri("http://localhost/").withQuery(Map("email"->"abc+a#abc.com")) works fine as it encodes the special symbols.
However, Uri("http://localhost/").withQuery("email=abc+a#abc.com") doesn't.
I use java.net.URLEncoder. I believe that is the accepted method.
It would be nice if that happened automatically!

float in REST-API call

how can one send float values to a RESTful API?
/api/set/integername/49
/api/set/charname/B
/api/set/floatname/49.33
The third one doesn't work.
404: Not Found
Can float values be encoded somehow to make this possible or do I have to wrap it in a json object?
I would just do something simple like convert the decimal point to an underscore :) It's less messy than URL encoding.
Do you have to support scientific notation (ex 1.5e+20)? In that case you'll have to deal with + and -.
(though - doesn't need encoding)
This is a duplicate question to Passing double/float through URL to Web Api query string
The answer provided there was to add a slash to the end of the querystring. I've just tested that with a blank Microsoft WebApi2 project and it works a treat.