How can I extract non-standard HTTP headers using Perl's LWP? - perl

I'm working with a web application that sends some non-standard HTTP headers in its response to a login request. The header in question is:
SSO_STATUS: LoginFailed
I tried to extract it with LWP::Response as $response->header('SSO_STATUS') but it does not work. It does work for standard headers such as Set-Cookie, Expires etc.
Is there a way to work with the raw headers?

if you see the documentation of HTTP::Headers, it states that
The header field name spelling is normally canonicalized
including the '_' to '-' translation. There are some application where
this is not appropriate. Prefixing field names with ':' allow you to
force a specific spelling. For example if you really want a header field
name to show up as foo_bar instead of "Foo-Bar", you might set it like
this:
$h->header(":foo_bar" => 1);
These field names are returned with the ':' intact for
$h->header_field_names and the $h->scan callback, but the colons do
not show in $h->as_string.

See this thread on Perlmonks.
You need to access the value of the header field as $response->header('SSO-STATUS').
The syntax for setting fields with underscores in names:
$response->header(':SSO_STATUS' => 'foo');

Related

TYPO3 7.6: 404 error page: HTML wrapped in numbers

I created my own “404 Page not found” error page on a TYPO3 website and implemented it via the /typo3conf/LocalConfiguration.php as follows, using the page’s Speaking URL path:
return [
...
'FE' => [
...
'pageNotFound_handling' => '/page-not-found/',
]
]
Now when I call a non-existing page, the error page gets displayed but there is a 4-digit alphanumeric number (hexadecimal as far as I’ve seen by now) BEFORE the HTML source code and a “0” AFTER it. Example (the number in the beginning is different after most of the reloads):
37b3
<!DOCTYPE html>
...
</html>
0
When calling the error page URL itself the page is returned correctly without those numbers.
Having the RealURL extension activated or deactivated does not make a difference.
Thanks a lot in advance!
I added the full description from the install tool and I guess we might find the solution there.
How TYPO3 should handle requests for non-existing/accessible pages.
empty (default)
The next visible page upwards in the page tree is shown.
'true' or '1'
An error message is shown.
String
Static HTML file to show (reads content and outputs with correct headers), e.g. notfound.html or http://www.example.org/errors/notfound.html.
Prefix "REDIRECT:"
If prefixed with "REDIRECT:" it will redirect to the URL/script after the prefix.
Prefix "READFILE:"
If prefixed with "READFILE" then it will expect the remaining string to be a HTML file which will be read and outputted directly after having the marker "###CURRENT_URL###" substituted with REQUEST_URI and ###REASON### with reason text, for example: READFILE:fileadmin/notfound.html.
Prefix "USER_FUNCTION:"
If prefixed with "USER_FUNCTION:" a user function is called, e.g. USER_FUNCTION:fileadmin/class.user_notfound.php:user_notFound->pageNotFound where the file must contain a class user_notFound with a method pageNotFound() inside with two parameters $param and $ref.
What you configured:
You're passing a string, thus TYPO3 expects to find a file - which you don't have, because it's more like an URL.
From what you try to achieve I'd go with REDIRECT:/page-not-found/.
Thanks for pointing this one out btw, I will remove the string configuration from the core since it does not make sense to have more people trip into this pitfall.
In short: change the following line in the FE section of your LocalConfiguration.php:
'pageNotFound_handling' => '/your404page.html',
to
'pageNotFound_handling' => 'REDIRECT:/your404page.html',
Cause
The actual cause is a combination of chunked Content-Encoding and the TYPO3 not being able to decode that in some cases. In your case the page not found handler eventually uses GeneralUtility::getUrl() to retrieve the error page.
If you have [SYS][curlUse] enabled it will use cUrl to retrieve the page and there is no problem.
If you don't have [SYS][curlUse] enabled it will open a socket, read the headers and then read the rest of the body. If the webserver uses "chunked" Content-Encoding the body will contain blocks of data and each block starts with a line with the length in hexadecimal format. The content ends with an empty block (with of course a line with the length "0").
cUrl apparently knows how to decode chunked data.
getUrl() itself does not know how to handle chunked data and uses the content as is as the page content.
In TYPO3 8 LTS the guzzle library is used to handle HTTP requests. In the guzzle code I can't find anything about handling chunked data. Guzzle will check if the cUrl PHP extension is present and use that as preferred transport. In most installations cUrl is present and since this decodes chunked data automagically no problem is visible. I have to test guzzle with PHP that has cUrl disabled to see if the issue is also present in v8/master.
Workaround/solution
If the PHP extension cUrl is enabled in your installation you can simply set [SYS][curlUse] in the Install Tool. The numbers around the 404 page content will disappear.

Parsing Just SIP Custom Header Values

I want to parse JUST SIP Custom Headers [ or call attached data ] not
other header fields.
As I see some guys use "X-" to distinguish those headers from standard headers.
But that is just convention.
In some systems , attached-data [ custom-headers ] are put into just after "Content-Length: " header but some put it after "Contact: " header.
I really can not find a generic AND elegant way, just to parse custom headers.
The only bad solution is create a Look-Up Table which contains all standard sip headers names and if header name is not in that list, parse that header which is ugly...
Any suggestion for more elegant solution ?
It depends on what you want to accomplish, but since servers/clients/proxy can inject any header they want, your only solution really is to have a white-list of valid header names. The major down-side is you have to take care of any new RFCs that define new "official" headers.
Depending on the use-case, you might want to just go for headers starting with X-. As you said, it's just a convention, but it's one that in wide use, IIRC.

How to specify Header attributes in apiary documentation

Starting with Apiary, I'm currently specifying the APIs for our project.
I was able to define the definitions for the API and defining the parameters works great.
Now I would like to add also the values passed in the HTTP Headers into my documentation (like pagination, version number of the API,...)
When browsing through the documentation I found that headers could be adden within the payload block or the request block, but I want them to be displayed in the documentation.
Is this possible and what's the best way to achieve this?
You can now use the Headers section.
Parameters are actually not required to be present in the URL template.
Thus, what you can do is to just have
+ Parameters
+ id (required, number, `1`) ... Numeric `id` of the Note to perform action with. Has example value.
+ X-My-Header (required, number, `5469`) ... My header does something
and this is going to be rendered in the table you mentioned as well.
You are going to have a warning from the parser, but it should work as expected.

BigCommerce API Update Order with PUT

I need to update an order which is done via PUT method passing the order id as part of the https url string and a single parameter, the status_id.
https://mystore.mybigcommerce.com/orders/12345.json
I have tried several methods to pass the status_id value but no matter what I try "status_id=12" or formatted as JSON "{"status_id": 12,}" I always get the same response:
[{"status":415,"message":"The specified input content type is not valid."}]
I have also tried as a POST request passing the JSON or XML code as raw data but that method is not supported.
How am I supposed to pass that field=value pair? can I embed it in the url string?
I also tried it but it wouldn't work for me.
Any ideas?
In case you are wondering I am doing it within FileMaker with TROIUrl plugIn, not a very popular technology, but the GET method retrieving orders works like a charm
TURL_Put( ""; $url ;"status_id=12") (I have also tried other FM plugIns to no avail)
Don't get too caught up in the Filemaker part, I don't expect many people out there to be familiar with BigCommerce and Filemaker. I just need a generic answer.
Thanks
Commandline tool curl is worth a try. It supports put and https.
Mac OS X: curl already installed, call from FileMaker via AppleScript do shell script.
Windows: must be installed, call via Powershell.
It works for me using { "status_id": "3" } which means you probably need to put quotes around the actual number.
Also, it is a PUT operation and application/json which is part of the request content.
The error message received by the OP:
[{"status":415,"message":"The specified input content type is not valid."}]
Is saying that he did not supply the 'Content-Type' header in his request or that the header supplied is for a content type that is not allowed. For the OP's case using JSON he would need to include the header:
Content-Type: application/json
in his HTTPS request. This description can be found along with those of the other status codes you may see here:
https://developer.bigcommerce.com/api/status-codes

Perl::Dancer how to include a file path as a parameter in the URI

I'm new to the Dancer framework and web apps in general. I have a Dancer project in which I have a route that accepts multiple parameters. So far, no sweat. However, if one of the parameters has a file path as its value then the route is not found.
I have tried encoding the parameter string as follows to eliminate the forward slashes:
$paramString =~ s/\//%2F/g;
and this does encode the slashes as expected (I print it out in the log to make sure).
However, after the parameter string is appended to the base URI for the route I'm interested in, the URI shows up in the browser in unencoded form, a 404 error is raised and the log says that the unencoded route can't be found.
I looked into the Request.pm module and found that in the init method a private method called _url_decode is called which removes the encoding. Is there a way to disable this when it is not desired?
I also tried using the uri_for method to create the URI. In this case, the encoded URI does show up in the browser, however, the route is still not found and the log indicates that the unencoded form (with the forward slashes) is being used to search for the route
Trying to match 'GET /exome_proj_config/project_type=exome&project_root=/usr/local/projects/users/pdagosto/projects&analysis_project_name=Test' against /^\/exome_proj_config\/([^\/]+)$/ (generated from '/exome_proj_config/:project_type:project_root:analysis_project_name') in /home/pdagosto/perl5/lib/perl5/Dancer/Route.pm l. 84 here
Since the regex used for the match is clearly looking for a string without any forward slashes following the one at the end of the base URI it's clear that the route will never be found.
Is there a way to have a URI parameter that contains a path or must some other approach be used?
It is possible to have a URI with a file path or slashes in the parameter provided that the parameter is part of the query string rather than the path. (See http://en.wikipedia.org/wiki/Uniform_resource_locator.)
For example see this Dancer script:
use strict;
use warnings;
use Dancer;
get '/file/action/:action' => sub {
my $filename = param('filename');
my $action = param('action');
return "Filename is $filename and action is $action";
};
dance;
If you put this string into the browser
http://localhost:3000/file/action/delete?filename=/folder/filename.txt
then you should expect to see this:
Filename is /folder/filename.txt and action is delete
In your question the URL you show uses the & character to separate parameters but it looks like you need a ? character first to separate the query string from the path. It is unclear from your question how you are creating your requests - but provided you can put the filename in the query string part of the URL then the approach above should work.