I'm trying to pass a complete URL as a parameter to a java-based REST service (GET), but I'm not sure how to format it in order to avoid a "HTTP 400 Bad Request". I've tried Base64 encoding, but still get the 400 error. I think part of the problem is that the url contains a question mark, "?", since it seems to be fine if I remove that and pass the url as-is. I'm not sure what is the problem when its encoded.
example url - http://my.site.com/testing-service?some+parms
method annotations:
#GET
#Path("/{fullurl}")
#Produces("application/json")
public Response findByUrl(#PathParam("fullurl") String fullurl)
...
(I've updated the description with a little more detail per the first couple of comments)
Apparently the encoding approach was close, but Base64 (java or commons-code) didn't work for whatever reason (length perhaps?). I found switching to Base32 (commons-code) works for my situation.
Related
I'm making requests to Twitter, using the OAuth1.0 signing process to set the Authorization header. They explain it step-by-step here, which I've followed. It all works, most of the time.
Authorization fails whenever special characters are sent without percent encoding in the query component of the request. For example, ?status=hello%20world! fails, but ?status=hello%20world%21 succeeds. But the change from ! to the percent encoded form %21 is only made in the URL, after the signature is generated.
So I'm confused as to why this fails, because AFAIK that's a legally encoded query string. Only the raw strings ("status", "hello world!") are used for signature generation, and I'd assume the server would remove any percent encoding from the query params and generate its own signature for comparison.
When it comes to building the URL, I let URLComponents do the work, so I don't add percent encoding manually, ex.
var urlComps = URLComponents()
urlComps.scheme = "https"
urlComps.host = host
urlComps.path = path
urlComps.queryItems = [URLQueryItem(key: "status", value: "hello world!")]
urlComps.percentEncodedQuery // "status=hello%20world!"
I wanted to see how Postman handled the same request. I selected OAuth1.0 as the Auth type and plugged in the same credentials. The request succeeded. I checked the Postman console and saw ?status=hello%20world%21; it was percent encoding the !. I updated Postman, because a nice little prompt asked me to. Then I tried the same request; now it was getting an authorization failure, and I saw ?status=hello%20world! in the console; the ! was no longer being percent encoded.
I'm wondering who is at fault here. Perhaps Postman and I are making the same mistake. Perhaps it's with Twitter. Or perhaps there's some proxy along the way that idk, double encodes my !.
The OAuth1.0 spec says this, which I believe is in the context of both client (taking a request that's ready to go and signing it before it's sent), and server (for generating another signature to compare against the one received):
The parameters from the following sources are collected into a
single list of name/value pairs:
The query component of the HTTP request URI as defined by
[RFC3986], Section 3.4. The query component is parsed into a list
of name/value pairs by treating it as an
"application/x-www-form-urlencoded" string, separating the names
and values and decoding them as defined by
[W3C.REC-html40-19980424], Section 17.13.4.
That last reference, here, outlines the encoding for application/x-www-form-urlencoded, and says that space characters should be replaced with +, non-alphanumeric characters should be percent encoded, name separated from value by =, and pairs separated by &.
So, the OAuth1.0 spec says that the query string of the URL needs to be decoded as defined by application/x-www-form-urlencoded. Does that mean that our query string needs to be encoded this way too?
It seems to me, if a request is to be signed using OAuth1.0, the query component of the URL that gets sent must be encoded in a way that is different to what it would normally be encoded in? That's a pretty significant detail if you ask me. And I haven't seen it explicitly mentioned, even in Twitter's documentation. And evidently the folks at Postman overlooked it too? Unless I'm not supposed to be using URLComponents to build a URL, but that's what it's for, no? Have I understood this correctly?
Note: ?status=hello+world%21 succeeds; it tweets "hello world!"
I ran into a similar issue.
put the status in post body, not query string.
Percent-encoding:
private encode(str: string) {
// encodeURIComponent() escapes all characters except: A-Z a-z 0-9 - _ . ! ~ * " ( )
// RFC 3986 section 2.3 Unreserved Characters (January 2005): A-Z a-z 0-9 - _ . ~
return encodeURIComponent(str)
.replace(/[!'()*]/g, c => "%" + c.charCodeAt(0).toString(16).toUpperCase());
}
I would like to make a simple GET request via Spray with a few query parameters
Get("http://localhost/user?email=abc+a#abc.com")
However + means a space in application/x-www-form-urlencoded content resulting the call to http://localhost/user?email=abc a#abc.com (with a space instead of plus sign).
I could use a non-Spray java.net.URLEncoder to encode the URL before passing it to the GET request however I doing this every time seems like a hack.
Is there a Spray way of applying query parameters and encoding them?
Uri("http://localhost/").withQuery(Map("email"->"abc+a#abc.com")) is a nice way to construct a Uri but it doesn't encode the params as well...
Actually Uri("http://localhost/").withQuery(Map("email"->"abc+a#abc.com")) works fine as it encodes the special symbols.
However, Uri("http://localhost/").withQuery("email=abc+a#abc.com") doesn't.
I use java.net.URLEncoder. I believe that is the accepted method.
It would be nice if that happened automatically!
I need to update an order which is done via PUT method passing the order id as part of the https url string and a single parameter, the status_id.
https://mystore.mybigcommerce.com/orders/12345.json
I have tried several methods to pass the status_id value but no matter what I try "status_id=12" or formatted as JSON "{"status_id": 12,}" I always get the same response:
[{"status":415,"message":"The specified input content type is not valid."}]
I have also tried as a POST request passing the JSON or XML code as raw data but that method is not supported.
How am I supposed to pass that field=value pair? can I embed it in the url string?
I also tried it but it wouldn't work for me.
Any ideas?
In case you are wondering I am doing it within FileMaker with TROIUrl plugIn, not a very popular technology, but the GET method retrieving orders works like a charm
TURL_Put( ""; $url ;"status_id=12") (I have also tried other FM plugIns to no avail)
Don't get too caught up in the Filemaker part, I don't expect many people out there to be familiar with BigCommerce and Filemaker. I just need a generic answer.
Thanks
Commandline tool curl is worth a try. It supports put and https.
Mac OS X: curl already installed, call from FileMaker via AppleScript do shell script.
Windows: must be installed, call via Powershell.
It works for me using { "status_id": "3" } which means you probably need to put quotes around the actual number.
Also, it is a PUT operation and application/json which is part of the request content.
The error message received by the OP:
[{"status":415,"message":"The specified input content type is not valid."}]
Is saying that he did not supply the 'Content-Type' header in his request or that the header supplied is for a content type that is not allowed. For the OP's case using JSON he would need to include the header:
Content-Type: application/json
in his HTTPS request. This description can be found along with those of the other status codes you may see here:
https://developer.bigcommerce.com/api/status-codes
Having a method like this:
#GET #Path("/name/{name}")
#Produces(MediaType.TEXT_PLAIN)
public String getProperty(#PathParam("name") String name) {
System.out.println(name);
}
How do I pass a value like "test./test"?
/name/test./test gives HTTP 404
/name/test.%2Ftest gives HTTP 400
/name/test.%252Ftest prints test%2Ftest
But if I do name = URLDecoder.decode(name); it prints /test and the first part of test. disappears.
There is one or two questions like this already but they are old and there was no good solution found, I thought I'll ask again.
The pattern in the #Path annotation is internally turned into a regular expression, with the template parts matching only selected characters by default. In particular, they normally don't match / characters; that's almost always the right thing to do (as it lets you put templates part way through a path) but in this case it isn't as you're wanting to consume the whole subsequent path. To get everything, we have to override the regular expression fragment for that particular template; this is actually pretty easy, since we just put in the template fragment a : followed by the RE that we want to use:
#GET #Produces(MediaType.TEXT_PLAIN)
#Path("/name/{name:.+}")
public String getProperty(#PathParam("name") String name) {
return name;
}
This will match all characters after the /name/ (up to but not including any ? query part) but will only match if there's something there at all. Be aware that if you have any other #Path("/name/...") things about, things can get really confusing! So don't do that.
If you using tomcat, and want pass / in pathparam. besides the #Path("/name/{name:.+}") stuff as 'Donal Fellows' said, you should add -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true to your jvm arguments, see also tomcat security-howto.
Try specifying the encoding type, the following works for me with /name/test.%252Ftest:
System.out.println(URLDecoder.decode(name, "UTF-8"));
return URLDecoder.decode(name, "UTF-8");
Glassfish v4 accept encoded scape for slash %2f. Then we can pass the encoded String test.%2Ftest and get the result test./test using URLDecoder.decode(name, "UTF-8"). I think this is a better solution especially when you have many params in one request. Using the path #Path("/name/{name:.+}") is great solution when we have few parameters in a request.
Using %252f complicates the client request becouse are needed to contruct the encoding request String manually. With glassfish v4 it's easy to use percent encoding with URLEncoder.encode in client and URLDecoder.decode in server to wished Strings. The most programing languages has percent encoding and decoding, therefore it's perfect solution.
I tried enable encoded slash in glassfish v3 but no success, here is the sintaxe I tried used
bin\asadmin set configs.config.server-config.network-config.protocols.protocol.http-listener-1.http.encoded-slash-enabled=true configs.config.server-config.network-config.protocols.protocol.http-listener-1.http.encoded-slash-enabled=true
Command set executed successfully.
Regards
Cassio Seffrin
I'm using Jon Crosby's open source Objective-C OAuth library http://code.google.com/p/oauthconsumer/ for some basic http authentication that does not deal with tokens, only consumer key and consumer secret. My code works great for GET, GET with parameters in the URL, and POST. When I issue a POST request that has parameters in the URL, though, the request fails authorization. I'm trying to figure out why.
The server is using Apache Commons OAuth, so I'd like to compare my base string with that library. Here's a contrived example and the base string and signature produced by my library. Can anyone see what the problem is?
consumer key: abcdef
consumer secret: ghijkl
POST request: http://emptyrandomhost.com/a/uriwith/params?interesting=foo&prolific=bar
my base string: POST&http%3A%2F%2Femptyrandomhost.com%2Fa%2Furiwith%2Fparams&interesting%3Dfoo%26oauth_consumer_key%3Dabcdef%26oauth_nonce%3D1%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D2%26oauth_version%3D1.0%26prolific%3Dbar
This data produces the following OAuth header authorization:
Authorization: OAuth oauth_consumer_key="abcdef",
oauth_version="1.0",
oauth_signature_method="HMAC-SHA1",
oauth_timestamp="2",
oauth_nonce="1",
oauth_signature="Z0PVIz5Lo4eB7aZFT8FE3%2FFlbz0%3D"
And apparently my signature is wrong. The problem has to either be in the construction of the base string, in the way that the HMAC-SHA1 function is implemented (using Apple's CCHmac from CommonHMAC.h, so hopefully this isn't it), or with my Base64Transcoder, which is open source c. 2003 by Jonathan Wight/Toxic Software. I primarily suspect the base string, since the requests work for GET and POST and only fail with POST with URL parameters as above.
Can someone with lots of OAuth experience spot the problem above? Something else that would be very useful is the base string that is produced by Apache Commons OAuth in their authentication. Thanks.
As per RFC 5849 section 3.4.1.2, the OAuth base string URI does not include the query string or fragment. If either the client or the server does not remove the query parameters from the base string URI and add them to the normalized OAuth parameter list, the signatures won't match. Unfortunately, it's hard to tell which side is making this mistake. But it's easy to determine this is the problem: If it always works without query parameters but always fails with query parameters, you can be pretty sure that one side or the other is generating the wrong base string. (Be sure that it always happens though... intermittent errors would be something else. Similarly, if it never works with or without a query string, that would also be something else.) The other possibility is that normalization was done incorrectly — the parameter list must be sorted and percent encoded sequences must be upper-cased. If it's not normalized correctly on both sides, that will also cause a base string mismatch, and thus a signature mismatch.
you can build and check visually your request at this URL:
http://hueniverse.com/2008/10/beginners-guide-to-oauth-part-iv-signing-requests/
Open the boxes denoted by [+] signs and fill in your values, that way you may be able to see if the problem is at your code, or at the provider side.