I wrote a simple script to serve custom HTTP error 403 page and I use the following code:
use CGI qw/:standard/;
print header(
'-Status' => 403,
'-Type' => 'text/html; charset=utf-8',
'-Cache-Control' => 'private, no-cache, no-store, must-revalidate, max-age=0',
'-Pragma' => 'no-cache');
...
print $html;
I expected system to return Forbidden status text automatically in HTTP header.
Unfortunately it returns 403 OK instead of 403 Forbidden. Text phrase is more likely added by browser.
Sure, I can explicitly add the status text using '-Status' => '403 Forbidden', but I would still like to know why isn't this done automatically, and why I am getting OK status instead...
Is there a way to force Perl to add default (English) status text for selected response code?
Chrome is the culprit here. You can verify by running your snippet on the command line, which outputs the following headers:
Status: 403
Pragma: no-cache
Cache-control: private, no-cache, no-store, must-revalidate, max-age=0
Content-Type: text/html; charset=utf-8
Notice the status is plain-old 403.
CGI.pm does not know about the reason phrases recommended by the HTTP spec. Nor should it: they are merely recommendations (not defaults), and you can change them to whatever you want without affecting the protocol (403 Go away anyone?). According to the standard, clients are not required to even look at the reason phrase.
So no, unless you modify CGI.pm, there is no way to force Perl to add a reason phrase. Even if you do provide a reason phrase, browsers can do what they wish with them (although most browsers will probably behave sanely).
I've been looking long time for this, and basically it's very simple:
see
https://serverfault.com/questions/121735/how-to-return-404-from-apache2-cgi-program
In sort:
change
use CGI qw/:standard/;
print header(
'-Status' => 403,
'-Type' => 'text/html; charset=utf-8',
...
to
print "Status: 403 Forbidden\r\n";
print "Content-Type: text/html\r\n\r\n";
print "<h1>403 Forbidden!</h1>";
Related
So I was testing my website and I tried connecting with the TRACE http method. In response I got a massive string. I don't know what it is. Does anybody know what could it be and if it's some sort of vulnerability?
This is the string I'm talking about:
VFJBQ0UgLy5odHBhc3N3ZCBIVFRQLzEuMQ0KSG9zdDogd3d3LnNzZmt6LnNpDQpVc2VyLUFnZW50OiBNb3ppbGxhLzUuMCAoWDExOyBMaW51eCB4ODZfNjQ7IHJ2OjkxLjApIEdlY2tvLzIwMTAwMTAxIEZpcmVmb3gvOTEuMA0KQWNjZXB0OiB0ZXh0L2h0bWwsYXBwbGljYXRpb24veGh0bWwreG1sLGFwcGxpY2F0aW9uL3htbDtxPTAuOSxpbWFnZS93ZWJwLCovKjtxPTAuOA0KQWNjZXB0LUxhbmd1YWdlOiBlbi1VUyxlbjtxPTAuNQ0KQWNjZXB0LUVuY29kaW5nOiBnemlwLCBkZWZsYXRlDQpDb25uZWN0aW9uOiBrZWVwLWFsaXZlDQpDb29raWU6IGpzQ29va2llV2FybmluZ0NoZWNrPWRlY2xpbmVkDQpVcGdyYWRlLUluc2VjdXJlLVJlcXVlc3RzOiAxDQpDYWNoZS1Db250cm9sOiBtYXgtYWdlPTAsIG5vLWNhY2hlDQpPcmlnaW46IGh0dHA6Ly93d3cuc3Nma3ouc2kNClByYWdtYTogbm8tY2FjaGUNCg0K
It's a Base64 encoded string. Decoded it looks like this:
TRACE /.htpasswd HTTP/1.1
Host: www.ssfkz.si
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: jsCookieWarningCheck=declined
Upgrade-Insecure-Requests: 1
Cache-Control: max-age=0, no-cache
Origin: http://www.ssfkz.si
Pragma: no-cache
Which per se does not really look like a security flaw and much rather like a basic implementation of the TRACE http method which states that the contents of the request shall be reflected in their entirety in the response body.
Interesting note though, looking at the specification:
A client MUST NOT generate header fields in a TRACE request containing sensitive data that might be disclosed by the response. For example, it would be foolish for a user agent to send stored user credentials [RFC7235] or cookies [RFC6265] in a TRACE request. The final recipient of the request SHOULD exclude any request header fields that are likely to contain sensitive data when that recipient generates the response body.
So ideally the response should not have contained the Cookie header (to fully comply with the specification by my understanding the client you used to send the requests should not have included them in the first place however).
My web service uses JWT-based authorization bearer token authentication:
HTTP clients send a valid POST to /v1/auth/signIn with a valid JSON request entity (includes username + password info)
If they authenticate successfully, that endpoint sends back an auth bearer token as an HTTP response header that (from curl) looks like:
Response from curl:
HTTP/1.1 200 OK
Date: Tue, 04 Sep 2018 01:18:28 GMT
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Access-Control-Expose-Headers: Authorization
Authorization: Bearer <big_huge_string>
Content-Length: 0
Subsequent service calls to authenticated endpoints just need to include the token as an HTTP request header whose key/name is Authorization and whose value is "Bearer <xyz>" (where <xyz>) is the <big_huge_string> that came back on the sign in call above. Pretty basic standard JWT stuff.
I'm trying to write a Postman collection that starts with a "Sign In Request" that successfully signs in and gets a JWT token from the service, and then adds the appropriate HTTP request header in each subsequent call. Any ideas as to how I can:
Extract the <big_huge_string> off the HTTP response header that I'll get back from my Sign In Request?; and then
How to save that <big_huge_string> as a variable and inject that as an HTTP request header for all subsequent calls?
Thanks in advance!
Update
Tried the suggestion:
Getting closer, but console.log(...) isn't printing anything to Postman (or at least I don't know where to look for it). I should mention I'm not using the Chrome Application version of Postman, but the standalone app/executable (Version 6.1.4):
Any ideas how/where I can get console.log(...) working? I'm concerned about just changing the test to:
pm.test("Can Extract JWT", function() {
var authHeader = pm.response.headers.toObject().Authorization;
pm.expect(authHeader).to.not.be.equal(null);
pm.globals.set('token', authHeader)
});
Without first seeing what that authHeader even is. Any ideas?!
Once you have that Token value you can reference it in each of the request headers using the {{token}} syntax. It's getting the sign in Auth header that's the harder part.
You could use pm.response.headers to get a list of the Headers and then extract out the value that you need.
This is returned as a list so maybe using something like Lodash or converting this to an object can help get the value you need. It would be something like pm.response.headers.toObject().Authorization - I haven't tried it so my syntax might be slightly wrong.
You can log the Headers out to the Postman console and narrow it down that way to - just wrap it in a Console.log() statement.
When you get that value, it's just a basic pm.globals.set('token, pm.response.headers.toObject().Authorization) to save this globally.
I heard that wolkenkit also offers a REST API but didn't find any documentation for that. I sifted through the sources and found some indications on how to do this.
I am using HTTPie for doing requests from the cli:
$ http post https://local.wolkenkit.io:3500/v1/read/lists/labels
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Cache-Control: no-store, no-cache, must-revalidate, proxy-revalidate
Connection: keep-alive
Date: Wed, 30 Aug 2017 21:08:07 GMT
Expires: 0
Pragma: no-cache
Surrogate-Control: no-store
X-FRAME-OPTIONS: DENY
X-Powered-By: Express
X-XSS-Protection: 1; mode=block
content-type: application/json
transfer-encoding: chunked
{"name":"heartbeat"}
{"label":"first","id":"baa1b2b6-ab85-4929-a488-0cae622de20a","isAuthorized":{"owner":"anonymous","forAuthenticated":false,"forPublic":true}}
{"label":"second","id":"7fc6c3c9-3285-4292-b3db-6d88ca90a347","isAuthorized":{"owner":"anonymous","forAuthenticated":false,"forPublic":true}}
I have two entries in my label readModel, but there appears to be a third one {"name":"heartbeat"}. Where does that come from and what does it mean?
Is this a bug or may i have created that entry accidentally?
Disclaimer: I am one of the developers of wolkenkit.
This is actually neither a bug, nor did you create the entry accidentally ;-)
Under the hood we stream JSON over HTTP, and we had the experience that some proxy servers (and similar things) caused issues when there were long pauses between two data packets.
In the past we changed the way how the read model is being delivered a few times, and I don't think that this is really still required, so this is a holdover from the past. (If we were talking about the events route, the story would be different, here it is definitely still needed.)
In the library that we use under the hood, json-lines-client, we filter out the heartbeat events:
const isNotHeartbeat = function (data) {
const keys = Object.keys(data);
return !(
(keys.length === 1) &&
(keys[0] === 'name') &&
(data.name === 'heartbeat')
);
};
(Taken from the source code of json-lines-client 0.7.9)
For the moment, I'd suggest to introduce a similar logic to your code, so that you simply ignore these events (there may be more than one over time, and they do not need to be the first one).
It is either not being sent, or not being received correctly. Using curl direct from the command line (using the -d option) or from PHP (using CURLOPT_POSTFIELDS) does work.
I start with a PSR-7 request:
$request = GuzzleHttp\Psr7\Request('POST', $url);
I add authentication header, which authenticates against the API correctly:
$request = $request->withHeader('Authorization', 'Bearer ' . $accessToken);
Then I add the request body:
// The parameter for the API function
$body = \GuzzleHttp\Psr7\stream_for('args=dot');
$request = $request->withBody($body);
I can send the message to the API:
$client = new \GuzzleHttp\Client();
$response = $client->send($request, ['timeout' => 2]);
The response I get back indicates that the "args" parameter was simply not seen by the API. I have tried moving the authentication token to the args:
'args=dot&access_token=123456789'
This should work, and does work with curl from the command line (-d access_token=123456789) but the API fails to see that parameter also when sending cia curl (6.x) as above.
I can see the message does contain the body:
var_dump((string)$request->getBody());
// string(8) "args=dot"
// The "=" is NOT URL-encoded in any way.
So what could be going wrong here? Are the parameters not being sent, or are they being sent in the wrong format (maybe '=' is being encoded?), or is perhaps the wrong content-type being used? It is difficult to see what is being sent "on the wire" when using Guzzle, since the HTTP message is formatted and sent many layer deep.
Edit: Calling up a local test script instead of the remote API, I get this raw message detail:
POST
CONNECTION: close
CONTENT-LENGTH: 62
HOST: acadweb.co.uk
USER-AGENT: GuzzleHttp/6.1.1 curl/7.19.7 PHP/5.5.9
args=dot&access_token=5e09d638965288937dfa0ca36366c9f8a44d4f3e
So it looks like the body is being sent, so I guess something else is missing to tell the remote API how to interpret that body.
Edit: the command-line curl that does work, sent to the same test script, gives me two additional header fields in the request:
CONTENT-TYPE: application/x-www-form-urlencoded
ACCEPT: */*
I'm going to guess it is the content-type header which is missing from the Guzzle request which is the source of the problem. So is this a Guzzle bug? Should it not always sent a Content-Type, based on the assumptions it makes that are listed in the documentation?
The Content-Type header was the issue. Normally, Guzzle will hold your hand and insert headers it deems necessary, and makes a good guess at the Content-Type based on what you have given it, and how you have given it.
With Guzzle's PSR-7 messages, none of that hand-holding is done. It strictly leaves all the headers for you to handle. So when adding POST parameters to a PSR-7 Request, you must explicitly set the Content-Type:
$params = ['Foo' => 'Bar'];
$body = \GuzzleHttp\Psr7\stream_for(http_build_query($params));
$request = $request->withBody($body);
$request = $request->withHeader('Content-Type', 'application/x-www-form-urlencoded');
The ability to pass in the params as an array and to leave Guzzle to work out the rest, does not apply to Guzzle's PSR-7 implementation. It's a bit clumsy, as you need to serialise the POST parameters into a HTTP query string, and then stick that into a stream, but there you have it. There may be an easier way to handle this (e.g. a wrapper class I'm not aware of), and I'll wait and see if any come up before accepting this answer.
Be aware also that if constructing a multipart/form-data Request message, you need to add the boundary string to the Content-Type:
$request = $request->withHeader('Content-Type', 'multipart/form-data; boundary=' . $boundary);
Where $boundary can be something like uniq() and is used in construction the multipart body.
The GuzzleHttp\Client provides all necessary wrapping.
$response = $client->post(
$uri,
[
'auth' => [null, 'Bearer ' . $token],
'form_params' => $parameters,
]);
Documentation available Guzzle Request Options
Edit: However, if your requests are being used within GuzzleHttp\Pool then, you can simply everything into the following:
$request = new GuzzleHttp\Psr7\Request(
'POST',
$uri,
[
'Authorization' => 'Bearer ' . $token,
'Content-Type' => 'application/x-www-form-urlencoded'
],
http_build_query($form_params, null, '&')
);
I building a module for compressing HTTP output. Reading the spec, I haven't found a clear distinction on a couple of things:
Accept-Encoding:
Should this be treated the same as a Accept-Encoding: * or as if no header is present?
Or what if I don't support gzip, but I get a header like this:
Accept-Encoding: gzip
Should I return a 406 error or just return the data unencoded?
EDIT:
I've read over the spec a few times. It mentions my first case, but it doesn't define what the behavior of the server should be.
Should I treat this case as if the header is not present? Or should I return a 406 error because there's no way to encode something given the field value ('' isn't a valid encoding).
There is written everything in the Spec: 14.3 Accept-Encoding:
The special "*" symbol in an Accept-Encoding field matches any
available content-coding not explicitly listed in the header
field.
If an Accept-Encoding field is present in a request, and if the server cannot send a response which is acceptable according to the Accept-Encoding header, then the server SHOULD send an error response with the 406 (Not Acceptable) status code.
edit:
If the Accept-Encoding field-value is empty, then only the "identity"
encoding is acceptable.
In this case, if "identity" is one of the available content-codings, then the server SHOULD use the "identity" content-coding, unless it has additional information that a different content-coding is meaningful to the client.
What is "identity"
identity
The default (identity) encoding; the use of no transformation whatsoever. This content-coding is used only in the Accept- Encoding header, and SHOULD NOT be used in the Content-Encoding header.