I'm passing a URL parameter to a form via a get request. I need to URL encode the parameter when the parameter contains a '#' . Otherwise the request fails. Why is this required ? Why do I need to URL encode the '#' parameter but not other text ?
'#' is used in URLs to indicate where a fragment identifier
(bookmarks/anchors in HTML) begins.
The part following the # is never seen by the server. It is generally used for navigation at the client-end.
The following characters need to be encoded in order to be used literally.
When using GET, anything after # (and the # itself) will not be seen by the server.
Related
My gitweb installation works so far, but all generated links that include a querystring, e.g.
host/gitweb?p=somepath&a=summary
are somehow malformed.
First, the ampersand is replaced by a semicolon. When inspecting the html, the link looks like
host/gitweb?p=somepath;a=summary
When klicking the link, the browser escapes the ';' to '%3b', so the url sent to the server looks like
host/gitweb?p=somepath%3ba=summary
The gitweb.cgi does not parse this and displays a 404 error page. When I replace the '%3b' with a ';' or a '&', everything works fine.
How can this be fixed on the server-side?
So far, I have tried to find the line producing the ';' in the urls, which is line 1457
$href .= "?" . join(';', #result) if scalar #result;
replacing the ';' by '&' gives me a malformed xhtml in the browser. Replacing it by '&' forces the browser again to escape the ';' which produces broken urls again.
The issue is kind of hidden (I can view the repositories), if I set the option
$feature{'pathinfo'}{'default'} = [1];
in the gitweb.conf file, but unfortunately, folders containing multiple repositories cannot be displayed, since the respective link uses some query-parameters.
You do not encoded or decoded the query string as a hole.
The param name and value must be encoded and decoded individually and the delimiters of the query string i.e..
= ; &
never get URL encoded or decoded.
You can check the server environment variable $ENV{REQUEST_URI} to see if the server did receive the information encoded like you said. If the browser is sending it then that is the problem and there is nothing you can do in your Perl code to fix that. Because it will just cause more problems down the road in your Perl code.
I somehow solved the problem using a rewrite rule in apache:
RewriteEngine on
RewriteCond %{QUERY_STRING} ^a=project_list%3bpf=(.*)$
RewriteRule (.*) $1?a=project_list;pf=%1 [QSA]
It only works for the project list with a specified path with the pathinfo feature enabled. A holistic approach would be to search for all possible query parameters and to replace them all.
However, this solution is not satisfying, since it does not fix the actual problem of the weird url escaping of gitweb.
I'm creating a PowerShell script that will assemble an HTTP path from user input. The output has to convert any spaces in the user input to the product specific codes, "%2F".
Here's a sample of the source and the output:
The site URL can be a constant, though a variable would be a better approach for reuse, as used in the program is: /http:%2F%2SPServer/Projects/"
$Company="Company"
$Product="Product"
$Project="The new project"
$SitePath="$SiteUrl/$Company/$Product/$Project"
As output I need:
'/http:%2F%2FSPServer%2FProjects%2FCompany%2FProductF2FThe%2Fnew%2Fproject'
To replace " " with %20 and / with %2F and so on, do the following:
[uri]::EscapeDataString($SitePath)
The solution of #manojlds converts all odd characters in the supplied string.
If you want to do escaping for URLs only, use
[uri]::EscapeUriString($SitePath)
This will leave, e.g., slashes (/) or equal signs (=) as they are.
Example:
# Returns http%3A%2F%2Ftest.com%3Ftest%3Dmy%20value
[uri]::EscapeDataString("http://test.com?test=my value")
# Returns http://test.com?test=my%20value
[uri]::EscapeUriString("http://test.com?test=my value")
For newer operating systems, the command is changed. I had problems with this in Server 2012 R2 and Windows 10.
[System.Net.WebUtility] is what you should use if you get errors that [System.Web.HttpUtility] is not there.
$Escaped = [System.Net.WebUtility]::UrlEncode($SitePath)
The output transformation you need (spaces to %20, forward slashes to %2F) is called URL encoding. It replaces (escapes) characters that have a special meaning when part of a URL with their hex equivalent preceded by a % sign.
You can use .NET framework classes from within Powershell.
[System.Web.HttpUtility]::UrlEncode($SitePath)
Encodes a URL string. These method overloads can be used to encode the entire URL, including query-string values.
http://msdn.microsoft.com/en-us/library/system.web.httputility.urlencode.aspx
I am passing my variables as follows to url using GET method
http://www.mysite.com/demo.php?sid=123121&email_id=stevemartin144%40gmail.com
& when i print $_GET on demo.php it displays parameters as follows:
email_id stevemartin144#gmail.com
sid 123121
instead of above output i want parameters as i passed
email_id stevemartin144%40gmail.com
sid 123121
I don't want to convert %40 into #
please suggest me solution on this
Thanks in advance
"%40" in a URL means "#". If you want a "%" to mean "%", you need to URL encode it to "%25".
URL encoding is just a transport encoding. If you feed in "#", its transport encoded version is "%40", but the recipient will get "#" again. If you want to feed in "%40" and have the recipient receive "%40", you need to URL encode it to "%2540".
If the recipient correctly receives "#" but you want to use the URL encoded version for whatever reason, you can also have the recipient urlencode it again.
Notes:
Online Converter:
Replace special characters with its equivalent hexadecimal unicode.
For a list of unicodes refer the website https://unicode-table.com (or) http://unicodelookup.com
Local Converter using Python:
Reference: conversion of password "p#s#w:E" to unicode will be as follows,
# = %40
$ = %24
# = %23
: = %3A
p#s#w:E = p%40s%23w%3AE
Input:
[root#localhost ~]# python -c "import sys, urllib as enc; print enc.quote_plus(sys.argv[1])" "p#s#w:E"
Output:
p%40s%23w%3AE
I have the following code in a Mako template:
<a href="#" onclick='getCompanyHTML("${fund.investments[inv_name].name | u}"); return false;'>${inv_name}</a>
This applies url escaping to the name string of an object representing a company. The resulting escaped string is then used in a url. The Mako documentation states that url encoding is provided using urllib.quote_plus(string.encode('utf-8')).
On the server I receive the company name part into the argument investment_name:
def Investment(client, fund_name, investment_name, **kwargs):
client = urllib.unquote_plus(client)
fund_name = urllib.unquote_plus(fund_name)
investment_name = urllib.unquote_plus(investment_name)
I then use investment_name as a key back in to the same dictionary from which it was extracted in the template.
This works fine for all the standard cases, such as spaces, slashes, and single quotes in the company name. However, it fails if the company name contains unicode characters outside of the ascii character set.
For instance, the url for company name "Eptisa Servicios de Ingeniería S.L." is rendered as "Eptisa+Servicios+de+Ingenier%C3%ADa+S.L." When this value arrives back at the server, I'm reversing the url escaping but clearly failing to decode the unicode properly because my attempt to use the result as a dictionary key generates a key error.
I've tried adding unicode decoding in these two forms, without luck:
investment_name = urllib.unquote_plus(investment_name.decode('utf-8'))
investment_name = urllib.unquote_plus(investment_name.encode('raw_unicode_escape').decode('utf-8'))
Can anyone suggest what I must do to "Eptisa+Servicios+de+Ingenier%C3%ADa+S.L." to turn it back into "Eptisa Servicios de Ingeniería S.L."?
Do it in the reverse order: first unquote then .decode('utf-8')
Do not mix bytes and Unicode strings.
Example
import urllib
q = "Eptisa+Servicios+de+Ingenier%C3%ADa+S.L."
b = urllib.unquote_plus(q)
u = b.decode("utf-8")
print u
Note: print u might produce UnicodeEncodeError. To fix it:
print u.encode(character_encoding_your_console_understands)
Or set PYTHONIOENCODING environment variable.
On Unix you could try locale.getpreferredencoding() as character encoding, on Windows see output of chcp
I am trying to use a web API of a service written in Perl (OTRS).
The data is sent in JSON format.
One of the string values inside the JSON structure contains a pound sign, which in apparently is used as a comment character in JSON.
This results in a parsing error:
unexpected end of string while parsing
JSON string
I couldn't find how to escape the character in order to get the string parsed successfully.
The obvious slash escaping results in:
illegal backslash escape sequence in
string
Any ideas how to escape it?
Update:
The URL I am trying to use looks something like that (simplified but still causes the error):
http://otrs.server.url/otrs/json.pl?User=username&Password=password&Object=TicketObject&Method=ArticleSend&Data={"Subject":"[Ticket#100000] Test Ticket from OTRS"}
Use Uri::escape:
use URI::Escape;
my $safe = uri_escape($url);
See rfc1738 for the list of characters which can be unsafe.
The hash symbol, #, has a special meaning in URLs, not in JSON. Your URL is probably getting truncated at the hash before the remove server even sees it:
http://otrs.server.url/otrs/json.pl?User=username&Password=password&Object=TicketObject&Method=ArticleSend&Data={"Subject":"[Ticket
And that means that the remote server gets mangled JSON in Data. The solution is to URL encode your parameters before pasting them together to form your URL; eugene y tells you how to do this.