When requesting a list of a users threads through https://graph.facebook.com/me/threads some of the threads are coming through with a forward slash character in the ID.
For example:
"id": "t_Cb0/atPMZaJw/cUuNtLW8B",
However trying to request this thread through https://graph.facebook.com/t_Cb0/atPMZaJw/cUuNtLW8B obviously fails due to the forward slashes
The error specifically returned is:
"message": "Unknown path components: /atPMZaJw/cUuNtLW8B"
I tried escaping the forward slashes with %2F but returns the same error, how can these be dealt with?
There are bug reports already about this, to
workaround: https://graph.facebook.com/?ids=atPMZaJw%2FcUuNtLW8B should work (i.e pass the escaped string in the 'id' or 'ids' parameter instead of as part of the request path)
Related
I'm making requests to Twitter, using the OAuth1.0 signing process to set the Authorization header. They explain it step-by-step here, which I've followed. It all works, most of the time.
Authorization fails whenever special characters are sent without percent encoding in the query component of the request. For example, ?status=hello%20world! fails, but ?status=hello%20world%21 succeeds. But the change from ! to the percent encoded form %21 is only made in the URL, after the signature is generated.
So I'm confused as to why this fails, because AFAIK that's a legally encoded query string. Only the raw strings ("status", "hello world!") are used for signature generation, and I'd assume the server would remove any percent encoding from the query params and generate its own signature for comparison.
When it comes to building the URL, I let URLComponents do the work, so I don't add percent encoding manually, ex.
var urlComps = URLComponents()
urlComps.scheme = "https"
urlComps.host = host
urlComps.path = path
urlComps.queryItems = [URLQueryItem(key: "status", value: "hello world!")]
urlComps.percentEncodedQuery // "status=hello%20world!"
I wanted to see how Postman handled the same request. I selected OAuth1.0 as the Auth type and plugged in the same credentials. The request succeeded. I checked the Postman console and saw ?status=hello%20world%21; it was percent encoding the !. I updated Postman, because a nice little prompt asked me to. Then I tried the same request; now it was getting an authorization failure, and I saw ?status=hello%20world! in the console; the ! was no longer being percent encoded.
I'm wondering who is at fault here. Perhaps Postman and I are making the same mistake. Perhaps it's with Twitter. Or perhaps there's some proxy along the way that idk, double encodes my !.
The OAuth1.0 spec says this, which I believe is in the context of both client (taking a request that's ready to go and signing it before it's sent), and server (for generating another signature to compare against the one received):
The parameters from the following sources are collected into a
single list of name/value pairs:
The query component of the HTTP request URI as defined by
[RFC3986], Section 3.4. The query component is parsed into a list
of name/value pairs by treating it as an
"application/x-www-form-urlencoded" string, separating the names
and values and decoding them as defined by
[W3C.REC-html40-19980424], Section 17.13.4.
That last reference, here, outlines the encoding for application/x-www-form-urlencoded, and says that space characters should be replaced with +, non-alphanumeric characters should be percent encoded, name separated from value by =, and pairs separated by &.
So, the OAuth1.0 spec says that the query string of the URL needs to be decoded as defined by application/x-www-form-urlencoded. Does that mean that our query string needs to be encoded this way too?
It seems to me, if a request is to be signed using OAuth1.0, the query component of the URL that gets sent must be encoded in a way that is different to what it would normally be encoded in? That's a pretty significant detail if you ask me. And I haven't seen it explicitly mentioned, even in Twitter's documentation. And evidently the folks at Postman overlooked it too? Unless I'm not supposed to be using URLComponents to build a URL, but that's what it's for, no? Have I understood this correctly?
Note: ?status=hello+world%21 succeeds; it tweets "hello world!"
I ran into a similar issue.
put the status in post body, not query string.
Percent-encoding:
private encode(str: string) {
// encodeURIComponent() escapes all characters except: A-Z a-z 0-9 - _ . ! ~ * " ( )
// RFC 3986 section 2.3 Unreserved Characters (January 2005): A-Z a-z 0-9 - _ . ~
return encodeURIComponent(str)
.replace(/[!'()*]/g, c => "%" + c.charCodeAt(0).toString(16).toUpperCase());
}
I am getting my objects by calling
https://<bucket>.storage.googleapis.com/?prefix=folder%2F<object name>%2F&delimiter=/&max-keys=1000
I have tried with other special characters like !, #, #, $, ^, &, *, (, ), etc.
For the other special characters I just encode them in the , and I get the response just fine.
For example, with object "!#" under folder, the url is:
https://<bucket>.storage.googleapis.com/?prefix=folder%2F%21%22%2F&delimiter=/&max-keys=1000
However, when I try with object names with "%" and encode the percent sign to "%25", I get the following error:
<?xml version='1.0' encoding='UTF-8'?><Error><Code>InvalidSecurity</Code> <Message>The provided security credentials are not valid.</Message><Details>Request was not signed or contained a malformed signature</Details></Error>
What could be causing this issue ?
Edit
So I have tried double encoding the percent sign such that '%' character becomes "%2525" in the request. However, in the response, the prefix is strangely "%25". After testing with more cases, it turns out a request is successful only when "%25" is followed by 2 characters both within the range of '0' and 'f', however, the response prefix would be wrong. For example, "%25ab" in the request would result in "%ab" in the response prefix.
I believe this is a service side bug: see https://issuetracker.google.com/issues/117932947
I think a workaround is to encode the percent twice. But this may start failing in the future when the bug is fixed.
The error message you're seeing is because you don't have enough permissions to access to your object.
If you're using an authentication method (APIkey, bearer, etc) make sure that they have the needed Roles for GCS.
However, I can see that you're calling the objects just as a GET request. Try to Make your objects public and try it again with that encoding (%25). It should work.
Hope this is helpful!
In my akka http app I receive a lot of errors like this one:
Illegal request-target: Invalid input 'v', expected HEXDIG (line 1, column 72)
It happens when I simply add a '%' character to the URL. In this case akka exoects a hex value.
Is it possible to somehow 'clean up' or encode/ignore such characters?
For users of an API who got this error message and don't know how to solve it:
I had the same problem, with a URL that queries something from a third party database, one part of the URL was simply not correct and that caused this error message.
I think this error message is misleading since it sounds like you should encode the URL differently which is not the case at all, just check if the structure of the URL is correct.
I'm writing app using Google custom search engine.
I received my search engine ID XXXXXXXX219143826571:7h9XXXXXXX (most interesting part bold).
Now I'm trying to use NSURLQueryItem to embed my ID into URL by using:
let params = ["cx" : engineID,...]
...
components.queryItems = parameters.map {
NSURLQueryItem(name: String($0), value: String($1))
}
It should percentage escape item to XXXXXXXX219143826571%3A7h9XXXXXXX (This value I'm getting when using Google APIs explorer while testing, it shows url dress that was used). It is not doing it. I'm getting url without escaping, no changes. If I use escaped string as engine ID in this mapping, I'm getting escaped string XXXXXXXX219143826571%253A7h9XXXXXXX (additional '25' is added to query).
Can someone tell me how to fix it? I don't want to use String and then convert it to URL by NSURL(string: str)!. It is not elegant.
Edit:
I'm using app Info.plist to save ID and I retrieve it by calling:
String(NSBundle.mainBundle().objectForInfoDictionaryKey("ApiKey")!)
Colons are allows in the query part of a URL string. There should be no need to escape them.
Strictly speaking, the only things that absolutely have to be encoded in that part of a URL are ampersands, hash marks (#), and (assuming you're doing a GET query with form encoding) equals signs. However, question marks in theory may cause problems, slashes are technically not allowed (but work just fine), and semicolons are technically allowed (but again, work in practice).
Colons, AFAIK, only have special meaning in the context of paths (if the OS treats it as a path separator) and in that it separates the scheme (protocol) from the rest of the URL.
So don't worry about the colon being unencoded unless the Google API barfs for some reason.
One of my routes in an MVC project serves documents that have been uploaded.
For SEO and user friendliness purposes I want the document title to be included in the URL, the route will take the ID from the incoming URL, match it with the document then redirect to a URL with the filename appended to the ID. As document titles can have a wide variety of characters in the title including ones that are used to break up parameters the filename is a catchall parameter.
This works fine for almost all characters in the title including reserved ones such as "/" but when the title include the combination "/ " routing breaks. Not just in terms of not match this route but apparently bypassing the entirety of the application and returning a 404, I tried to use Phil Haack's RouteDebugger but that was also giving a 404 rather than catching the request.
My web.config has requestvalidation turned off and I can't seem to find anyway to get the application to catch the request.
I would recommend you filtering those character or you will have many headaches. Here's a function you might find interesting.