I have seen few articles about Best Practices with REST API and they are suggesting belo for multi column sort.
GET /users?sort_by=-last_modified,+email
https://www.moesif.com/blog/technical/api-design/REST-API-Design-Filtering-Sorting-and-Pagination/
When I am using this approach, I see that - works fine but + gets replaced by a space.
A quick google indicates that + is a special character after ? in URL. What am I missing out here?
> The following characters have special meaning in the path component of
> your URL (the path component is everything before the '?'): ";" |
> "/" | "?"
>
> In addition to those, the following characters have special meaning in
> the query part of your URL (everything after '?'). Therefore, if they
> are after the '?' you need to escape them: ":" | "#" | "&" | "=" |
> "+" | "$" | ","
>
> For a more in-depth explanation, see the RFC.
What am I missing out here?
History, mostly.
U+002B (+) is a sub-delim, in the context of a URI, and can be used freely in the query part; see RFC 3986 Appendix A.
But on the web, a common source of query data is HTML form submissions; when we submit a form, the processing engine collects the key value pairs from the form and creates an application/x-www-form-urlencoded character sequence, which becomes the query of the URI.
Because this is such a common case, the query parsers in web server frameworks often default to reversing the encoding before giving your bespoke code access to the data.
Which means that in your web logs, you would see:
/users?sort_by=-last_modified,+email
because that's the URI that you received, but in your parameter mapping you would see
"sort_by" = "-last_modified, email"
Because the "form data" is being decoded before you get to look at it.
Form urlencoding has an explicit step in it that replaces any spaces (U+0020) with U+002B, and U+002B is instead percent-encoded.
To check if this is what is going on, try instead the following request:
GET /users?sort_by=-last_modified,%2Bemail
What I expect you will find is that the plus you are looking for now appears in your form parameters:
"sort_by" = "-last_modified,+email"
Related
I came across this and was wondering what this means and how it works?
What's the significance of using #regex here and how does it expand?
https://github.com/microsoft/azure-pipelines-tasks/blob/master/Tasks/DownloadPackageV0/task.json
"endpointUrl": "{{endpoint.url}}/{{ **#regex ([a-fA-F0-9\\-]+/)[a-fA-F0-9\\-]+ feed }}_apis**/Packaging/Feeds/{{ **#regex [a-fA-F0-9\\-]*/([a-fA-F0-9\\-]+) feed** }}{{#if view}}#{{{view}}}{{/if}}/Packages?includeUrls=false"
Also I would like to know how many packages will it return and display in the Task input UI dropdown if there are thousands of packages in the feed. Is there a known limit like first 100 or something?
#regex doesn't appear to actually be documented anywhere, but it takes two space-delimited arguments. The first is a regular expression and the second is a "path expression" identifying what value to match against, in this case the value of the feed input parameter. If the regex matches the value, it returns the first capturing subexpression, otherwise it returns the empty string.
In this particular context, the feed parameter is formatted as 'projectId/feedId', where projectId and feedId are GUIDs, and projectId and the / are eliminated for organization-scoped feeds (i.e. feeds that are not inside a project). The first regex therefore extracts the project ID and inserts it into the URL, and the second regex extracts the feed ID and inserts it into the URL.
As of this writing, the default limit on the API it's calling is 1000.
Regex stands for regular expression, which allows you to match any pattern rather than an exact string. You can find more info on how to use it in Azure Devops here
This regex is very specific. In this case, the regex ([a-fA-F0-9\\-]+/)[a-fA-F0-9\\-]+\ matches one or more of the following 1) letters a-f (small or capital) Or 2) \ Or 3) - followed by / and then again one or more of those characters.
You can copy the regex [a-fA-F0-9\\-]+/)[a-fA-F0-9\\-]+ into https://regexr.com/ to play around with it, to see what does and doesn't match the pattern.
Examples:
it matches: a/a a/b abcdef-\/dcba
but doesn't match: /a, abcdef, this-doesn't-match
Note that the full endpoint consists of concatenations of both regular expression and hardcoded strings!
I want to find the exact strings, throw ' and throw " in javascript files in a given repo using the Github API.
It says:
You can't use the following wildcard characters as part of your search
query: . , : ; / \ ` ' " = * ! ? # $ & + ^ | ~ < > ( ) { } [ ]. The
search will simply ignore these symbols.
I'm supposing there is no way to find these exact strings using the API?
I have tried various searches with no luck. Trying to escape the " with \ doesn't work either.
https://api.github.com/search/code?q=%22throw+%27%22+in:file+language:js+repo:angular/angular.js
All of the queries I try return, for instance, https://github.com/angular/angular.js/blob/master/docs/config/tag-defs/tutorial-step.js which just finds the throw and disregards the '.
An alternative to this strategy is to find where there is NOT Error on the line, so that the search is throw NOT Error to try to find where someone is throwing a string and not throw new Error(. This strategy doesn't work for some reason. For instance,
https://api.github.com/search/code?q=%22throw%20NOT%20Error%22+in:file+language:js+repo:driverdan/node-XMLHttpRequest
There are many times that a plain string is thrown in https://github.com/driverdan/node-XMLHttpRequest/blob/master/lib/XMLHttpRequest.js but, the file does contain the string Error; so, I guess this is enough to make the result empty. I'm seeing a way to "search" in my client program as a workaround; but, I would rather get the results from the API.
I'm supposing there is no way to find these exact strings using the API?
Correct, that's not possible currently.
I'm creating a PowerShell script that will assemble an HTTP path from user input. The output has to convert any spaces in the user input to the product specific codes, "%2F".
Here's a sample of the source and the output:
The site URL can be a constant, though a variable would be a better approach for reuse, as used in the program is: /http:%2F%2SPServer/Projects/"
$Company="Company"
$Product="Product"
$Project="The new project"
$SitePath="$SiteUrl/$Company/$Product/$Project"
As output I need:
'/http:%2F%2FSPServer%2FProjects%2FCompany%2FProductF2FThe%2Fnew%2Fproject'
To replace " " with %20 and / with %2F and so on, do the following:
[uri]::EscapeDataString($SitePath)
The solution of #manojlds converts all odd characters in the supplied string.
If you want to do escaping for URLs only, use
[uri]::EscapeUriString($SitePath)
This will leave, e.g., slashes (/) or equal signs (=) as they are.
Example:
# Returns http%3A%2F%2Ftest.com%3Ftest%3Dmy%20value
[uri]::EscapeDataString("http://test.com?test=my value")
# Returns http://test.com?test=my%20value
[uri]::EscapeUriString("http://test.com?test=my value")
For newer operating systems, the command is changed. I had problems with this in Server 2012 R2 and Windows 10.
[System.Net.WebUtility] is what you should use if you get errors that [System.Web.HttpUtility] is not there.
$Escaped = [System.Net.WebUtility]::UrlEncode($SitePath)
The output transformation you need (spaces to %20, forward slashes to %2F) is called URL encoding. It replaces (escapes) characters that have a special meaning when part of a URL with their hex equivalent preceded by a % sign.
You can use .NET framework classes from within Powershell.
[System.Web.HttpUtility]::UrlEncode($SitePath)
Encodes a URL string. These method overloads can be used to encode the entire URL, including query-string values.
http://msdn.microsoft.com/en-us/library/system.web.httputility.urlencode.aspx
I'm passing a URL parameter to a form via a get request. I need to URL encode the parameter when the parameter contains a '#' . Otherwise the request fails. Why is this required ? Why do I need to URL encode the '#' parameter but not other text ?
'#' is used in URLs to indicate where a fragment identifier
(bookmarks/anchors in HTML) begins.
The part following the # is never seen by the server. It is generally used for navigation at the client-end.
The following characters need to be encoded in order to be used literally.
When using GET, anything after # (and the # itself) will not be seen by the server.
The padding char for the official base64 is '=', which might need to be percent-encoded when used in a URL. I'm trying to find the best padding char so that my encoded string can be both url safe (I'll be using the encoded string as parameter value, such as id=encodedString) AND filename safe (I'll be using the encoded string directly as filename).
Dot ('.') is a popular candidate, it's url safe but it's not exactly filename safe: Windows won't allow a file name which ends with a trailing dot.
'!' seems to be a viable choice, although I googled and I've never seen anybody using it as the padding char. Any ideas? Thanks!
Update: I replaced "+" with "-" (minus) and replaced "/" with "_" (underscore) in my customized base64 encoding already, so '-' or '_' is not available for the padding char any more.
The best solution (I've spent last month working on this problem with an email sending website) is to not use padding character (=) at all
The only reason why padding character is there is because of "lazy" decoders. You can extremely easy add missing = -> just do %4 on text and subtract the number you get from 4 and that is how many = you need to add in string end. Here is C# code:
var pad = 4 - (text.Length % 4);
if (pad < 4)
text = text.PadRight(text.Length + pad, '=');
Also, most people who do this are interested in replacing + and / with other URL safe character... I propose:
replace with -
/ replace with _
DO NOT USE . as it can produce crazy results on different systems / web servers (for example on IIS Base64 encoded string can't end with . or IIS will search for the file)
The RFC 2396 unreserved characters in URIs are:
"-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"
It's worth pointing out, though, that the Microsoft article also says "Do not assume case sensitivity." Perhaps you should just stick with base 16 or 32?
The Wikipedia article states;
a modified Base64 for URL variant
exists, where no padding '=' will be
used
I would go with '-' or '_'
They're URL and file safe, and they looks more or less like padding