A quick question bothering me for months now..
When one is sending a SOAP request (using SoapUI or not) and signing it with private key. WHAT exactly is digested for the signature?
Whichever part of the referenced object in the request I take for sha1 digestion I get a different value than the one SoapUI generates. Tried googling arround but couldn't find a decent article clearly explaining what part of the ref'ed object is digested for signature.
pc-04 ~ $ nc -l 0.0.0.0 5588
POST / HTTP/1.1
Accept-Encoding: gzip,deflate
Content-Type: text/xml;charset=UTF-8
SOAPAction: "urn:[_renamed_]:v1/sendMessage"
Content-Length: 2418
Host: localhost:5588
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:[_renamed_]:schema:v1">
<soapenv:Header><wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"><ds:Signature Id="SIG-927771137712DBCACC150572539981324" xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"><ec:InclusiveNamespaces PrefixList="soapenv urn" xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"/></ds:CanonicalizationMethod><ds:SignatureMethod Algorithm="http://www.w3.org/2000/09/xmldsig#rsa-sha1"/><ds:Reference URI="#id-927771137712DBCACC150572539981023"><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"><ec:InclusiveNamespaces PrefixList="urn" xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#"/></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1"/><ds:DigestValue>40g3y73kx2NLAk+RtE7A6XYQfq0=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>RNusKNwPvnzII/DZo2nkK8i/xDkq5bSjoZ+vW+BBv7jwoatosim7wjm5JiZGHjfGo1wnJNTA1nTn
/U6sjQZZvtNSuDKHcaY5FphluMYkn5fKSt8JvDsqH+dgNr3w6GMFshDimoIjG3/Opt0rUw7Mgi/x
cUYx95VqYPX20/SQ5MY=</ds:SignatureValue><ds:KeyInfo Id="KI-927771137712DBCACC150572539981021"><wsse:SecurityTokenReference wsu:Id="STR-927771137712DBCACC150572539981022"><ds:X509Data><ds:X509IssuerSerial><ds:X509IssuerName>1.2.840.113549.1.9.1=#161c72696361726461732e62756369756e6173406e6f7274616c2e636f6d,CN=[_renamed_],O=[_renamed_] UAB,L=[_renamed_],ST=[_renamed_],C=LT</ds:X509IssuerName><ds:X509SerialNumber>10598525849934796037</ds:X509SerialNumber></ds:X509IssuerSerial></ds:X509Data></wsse:SecurityTokenReference></ds:KeyInfo></ds:Signature></wsse:Security>
<urn:requestHeader>
<!--Optional:-->
<headtag1>
<inbox>?</inbox>
</headtag1>
</urn:requestHeader>
</soapenv:Header>
<soapenv:Body wsu:Id="id-927771137712DBCACC150572539981023" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<urn:message>
<name>Andrew</name>
<lname>Hopkins</lname>
<username>andhop335</username>
</urn:message>
</soapenv:Body>
</soapenv:Envelope>pc-04 ~ $
pc-04 ~ $
How does SoapUI get that digest?
pc-045 ~ $ echo 40g3y73kx2NLAk+RtE7A6XYQfq0= | base64 -d | xxd -p
e34837cbbde4c7634b024f91b44ec0e976107ead
The question is unanswered for some days but I am sticking at some other problems with that.
So, the digest is calculated over the part (object) in your xml which is referenced by attribute in the reference object:
<ds:Reference URI="#id-927771137712DBCACC150572539981023">
The object to sign is so referenced by "id-927771137712DBCACC150572539981023" (without the #):
<soapenv:Body wsu:Id="id-927771137712DBCACC150572539981023" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
In this case, the content of soapenv:Body has to be digested AFTER canonilization, it has been processed (transformed) by c14n, which is stated in:
<ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#">
For further Infos take a look here:
https://www.w3.org/TR/xmldsig-core/#sec-ReferenceProcessingModel
https://www.w3.org/TR/2001/NOTE-SOAP-dsig-20010206/#XML-Signature
https://www.w3.org/TR/xmldsig-core/#sec-TransformAlg
Related
I am actually trying to get a list of files(or their internalId's) inside a folder using soap webservice from netsuite file cabinet
I am using the following request to make a search request
<soapenv:Header>
<urn:tokenPassport>
<urn1:account xmlns:urn1="urn:core_2020_2.platform.webservices.netsuite.com">*****</urn1:account>
<urn1:consumerKey xmlns:urn1="urn:core_2020_2.platform.webservices.netsuite.com">*****</urn1:consumerKey>
<urn1:token xmlns:urn1="urn:core_2020_2.platform.webservices.netsuite.com">*****</urn1:token>
<urn1:nonce xmlns:urn1="urn:core_2020_2.platform.webservices.netsuite.com">*****</urn1:nonce>
<urn1:timestamp xmlns:urn1="urn:core_2020_2.platform.webservices.netsuite.com">*****</urn1:timestamp>
<urn1:signature algorithm="HMAC-SHA256" xmlns:urn1="urn:core_2020_2.platform.webservices.netsuite.com">*****</urn1:signature>
</urn:tokenPassport>
</soapenv:Header>
<soapenv:Body>
<search xmlns="urn:messages_2020_2.platform.webservices.netsuite.com">
<searchRecord type="FileSearchBasic" xmlns="urn:common_2020_2.platform.webservices.netsuite.com">
<folder type="SearchMultiSelectField" internalId="2760"/>
</searchRecord>
</search>
</soapenv:Body>
However its giving error following error
<faultcode>soapenv:Server.userException</faultcode>
<faultstring>org.xml.sax.SAXException: {urn:core_2020_2.platform.webservices.netsuite.com}SearchRecord is an abstract type and cannot be instantiated</faultstring>
after a lot of research I was able to make a successful soap request to get list of files inside a folder, following is the request body
<soap:Body>
<platformMsgs:search>
<platformMsgs:searchRecord xmlns:s0="urn:filecabinet_2020_2.documents.webservices.netsuite.com" xsi:type="s0:FolderSearchAdvanced">
<s0:criteria>
<s0:basic>
<internalId operator="anyOf">
<searchValue internalId="2760" type="folder" />
</internalId>
</s0:basic>
</s0:criteria>
<s0:columns>
<s0:basic>
<internalId/>
<name/>
</s0:basic>
<fileJoin>
<internalId/>
<name/>
<modified/>
<documentSize/>
</fileJoin>
</s0:columns>
</platformMsgs:searchRecord>
</platformMsgs:search>
</soap:Body>
I have Ubuntu server(16.04) + Nagios, also I have created a script that makes a screenshot(Nagios status) every night and sends this screenshot to two recipients. But occurs problem, I receive mail with images(embed in a body, not attachment) - is OK, but my friend receives the same mail with broken images(blank files in attachment).
Any suggestion, how to solve this problem?
Script code:
#!/bin/bash
cat <<EOT | /usr/sbin/sendmail -t
TO: #email1, #email2
SUBJECT: Report: Nagios Event Log $(date +%F --date=yesterday)
MIME-Version: 1.0
Content-Type: multipart/related;boundary="XYZ"
--XYZ
Content-Type: text/html; charset=ISO-8859-15
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-15">
</head>
<body bgcolor="#ffffff" text="#000000">
Hello Team,<br>Daily Nagios report of $(date +%F --date=yesterday) is generated.
<img src="cid:part1.06090408.01060107" alt="">
<br>Best Regards, Nagios Admin
</body>
</html>
--XYZ
Content-Type: image/png;name="Nagios-EventLog-`date +%F --date="yesterday"`.png"
Content-Transfer-Encoding: base64
Content-ID: <part1.06090408.01060107>
Content-Disposition: inline; filename="Nagios-EventLog-`date +%F --date="yesterday"`.png"
$(base64 /some_path/NagiosReport/Nagios-EventLog-`date +%F --date="yesterday"`.png)
--XYZ--
EOT
You lack a newline between the MIME headers and the base64 image data.
Running base64 in a command substitution inside a here document will probably produce overlong lines in the output. Try (rough pseudocode)
( cat <<EOF
From: blah blah ...
Subject: blah blah ...
:
--XYZ
Content-description: image/png; name=etc etc
EOF
base64 file
printf "\n--XYZ--\n" ) | sendmail -oi -t
(I'm assuming you did PATH=/usr/sbin:$PATH near the top of the script so you don't have to hard-code the path to sendmail.)
If improved knowledge of MIME isn't a personal development goal, probably use a program which knows how to do this properly. Many people use mutt to send mail on their behalf without having to worry about how exactly to do it right.
As a stylistic aside, running $(date +%F) multiple times seems clunky. Just run it once and capture the output in a variable. (In the pathological case, the script runs around midnight, and you get different dates in different parts of the message!)
I currently expand my test suite to increase the test coverage. I want to test my controller and the html output that it renders, but I found a problem in using delete methods. Let me explain it in an example.
I have a route:
$r->delete('/backups/:id')
->to('backup#delete_backup')
->name('backup_delete');
that points to the following function in the backup controller:
sub delete_backup {
my $self = shift;
my $id = $self->param('id');
if ( something ) {
$self->flash( msg => "Backup id $id deleted!" );
}
else{
$self->flash( msg => "Cannot delete, backup id $id not found!" );
}
$self->redirect_to($self->url_for('backup_index'));
}
where the method that handles the route backup_index just displays the $msg and shows few other irrelevant data.
Next, I want to test this method, so I write a test:
$t_logged_in->ua->max_redirects(3);
my $page = $t_logged_in->app->url_for( 'backup_delete', id => $backup_id );
$t_logged_in->delete_ok($page)
->status_isnt( 404, "Checking: 404 $page" )
->status_isnt( 500, "Checking: 500 $page" );
The test is passed. But now, I want to check if the text is correct on the web page that is shown after redirecting. So I do the following:
$t_logged_in->ua->max_redirects(3);
my $page = $t_logged_in->app->url_for( 'backup_delete', id => $backup_id );
$t_logged_in->delete_ok($page)
->status_isnt( 404, "Checking: 404 $page" )
->status_isnt( 500, "Checking: 500 $page" )
->content_unlike(qr/Cannot delete,/i)
->content_like(qr/deleted/i);
The test fails. It fails because the content is empty, so the matching is done as there were:
'' =~ /deleted/i;
'' !~ /Cannot delete,/i;
and this is of course false in both cases. Of course, in the browser, the redirects work perfectly and I see everything as designed in the test. I can change the method to POST or GET but I wanted to make the routing properly in the way an API would be designed.
Question: how to design the test such that the content can be matched after the redirect?
For those who want to dig deeper, I give links to Github.
Routing definition line 303
Controller Backup function delete_backup line 135
Functional Test of Backup Controller line 53
Sorry no one answered this yet. I used to try to keep a closer eye on stack overflow but I got lazy :-P. This is a very interesting question.
Standard redirects (301/302) use the same verb as the original request UNLESS the original request was a POST Interestingly, this behavior was not really intended. A redirect is supposed to use the same request method as the original, but the POST->GET redirect is so commonly intended that most browsers added this behavior even though it was against the original definition. However some people really wanted a POST to remain a POST and now it depended on the implementation. In fact this got so bad that HTTP added response statuses 307/308 which always redirects as the same http verb (ie POST -> POST), even thought that was already the official behavior of the old ones but that ship had sailed. This means that POST->GET, but DELETE->DELETE is the defacto behavior of the 301/302.
This can be demonstrated in the following one liner ($_ in these one-liners is the controller):
perl -Mojo -E 'del "/delete" => sub { $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
Host: 127.0.0.1:62690
Accept-Encoding: gzip
User-Agent: Mojolicious (Perl)
Content-Length: 0
HTTP/1.1 302 Found
Date: Sun, 18 Sep 2016 03:25:58 GMT
Server: Mojolicious (Perl)
Location: /
Content-Length: 0
DELETE / HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:62690
Accept-Encoding: gzip
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Content-Length: 5
Date: Sun, 18 Sep 2016 03:25:58 GMT
Server: Mojolicious (Perl)
I got a DELETE
You can see that since my / route handles all methods we get a response, however the request was a DELETE. I see that in your app, you redirect to another named route whose url also handles DELETE. This is a source of your confusion as you are actually ending up there in your tests. Your redirect is effectively DELETE /backups/<<id>> -> DELETE /backups, if I'm reading your code correctly.
Now as opposed to the people who wanted to keep POST as POST, what you want here is the opposite, you want to redirect from DELETE to GET because you are trying to show a response that is not the original resource. This is the defined behavior of the little-known 303 response in which any request method is redirected to a GET. Indeed this is what early browsers should have insisted on rather than breaking the 302.
In the following example I modify the response code to 303 explicitly.
$ perl -Mojo -E 'del "/delete" => sub { $_->res->code(303); $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:62716
Accept-Encoding: gzip
Content-Length: 0
HTTP/1.1 303 See Other
Location: /
Content-Length: 0
Server: Mojolicious (Perl)
Date: Sun, 18 Sep 2016 03:27:19 GMT
DELETE / HTTP/1.1
User-Agent: Mojolicious (Perl)
Accept-Encoding: gzip
Content-Length: 0
Host: 127.0.0.1:62716
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Server: Mojolicious (Perl)
Content-Length: 5
Date: Sun, 18 Sep 2016 03:27:19 GMT
I got a DELETE
But whoops, that's still a DELETE! That's because there was a bug in Mojo::UserAgent so that it didn't handle 303s correctly. I'd suggest you still make the change to your code since browsers seem to handle 303s correctly. However since Mojo::UserAgent powers Test::Mojo, your tests can't test that behavior yet. Once fixed you will see it work correctly, as my example does in my local branch:
$ perl -Ilib -Mojo -E 'del "/delete" => sub { $_->res->code(303); $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:64924
Accept-Encoding: gzip
HTTP/1.1 303 See Other
Date: Sun, 18 Sep 2016 04:59:37 GMT
Location: /
Server: Mojolicious (Perl)
Content-Length: 0
GET / HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:64924
Accept-Encoding: gzip
HTTP/1.1 200 OK
Date: Sun, 18 Sep 2016 04:59:37 GMT
Content-Type: text/html;charset=UTF-8
Server: Mojolicious (Perl)
Content-Length: 12
I got a GET
To see more about redirect response types see https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection
I am working on a series of Powershell scripts for my company to use to transfer data to and from Box.com. The one thing I just can't figure out is uploading files. The Box API requires a multipart POST operation for uploads, and I've seen a few answers here on SO indicating that I should be able to do that in Powershell (such as this one). But I can't seem to get it working.
Here's the code I have right now:
Function Post-File {
Param(
[Parameter(Mandatory=$True,Position=1)]
[string]$SourcePath,
[Parameter(Mandatory=$False,Position=2)]
[string]$FolderId = ############
)
#Variables for building URIs
$baseUrl = "https://upload.box.com/api/2.0/files/content"
#Set Authorization for API requests
$headers = #{}
$AccessToken = Refresh-Tokens #A reference to another function that definitely works
$headers.Add("Authorization", "Bearer $AccessToken")
#Set POST content
$body = #{}
$body.Add("filename", [IO.File]::ReadAllBytes($SourcePath))
$body.Add("parent_id", $FolderId)
#Upload the file
Invoke-RestMethod -Uri $baseUrl -Method Post -Headers $headers -ContentType "multipart/form-data" -Body $body
}
Here's the response I get back:
{
"type":"error",
"status":400,
"code":"invalid_request_parameters",
"help_url":"http://developers.box.com/docs/#errors",
"message":"Invalid input parameters in request",
"request_id":"1764475572534bcddfe04b7"
}
I've also tried a couple of other permutations that aren't working. I've tried using the -InFile switch in Invoke-RestMethod instead of -Body. I've also tried using Get-Content -Raw in place of [IO.File]::ReadAllBytes. Both of those return a more generic error: The server encountered an internal error or misconfiguration and was unable to complete your request..
I'm pretty sure this has something to do with my filename parameter not being correct, but I'm not sure how to fix it. According to the Box API, here's what it should look like in curl. Can someone help me properly translate this for Powershell?
https://upload.box.com/api/2.0/files/content \
-H "Authorization: Bearer ACCESS_TOKEN" \
-F filename=#FILE_NAME \
-F parent_id=PARENT_FOLDER_ID
There are a couple things that make this a little tricky in PowerShell:
The filename parameter specifies the name of the file, not the contents.
A request body is required in order to specify the file name and destination.
PowerShell treats -InFile and -Body arguments as mutually exclusive.
PowerShell does not appear to natively support multipart/form-data POSTs, per the question that you referenced.
Since the request body is required (1,2) -- and thus -InFile can't be used (3) -- I think you might need to roll your own multipart/form-data body (4) that contains the necessary metadata and file content. This blog post describes a method for doing so. The content there is a short string (a tweet) but the principle is the same.
Below is a Fiddler trace of an upload I just performed using the Box Windows SDK. This shows how the request should look as it goes across the wire. The $BOUNDARY is an arbitrary, unique string -- a GUID works great.
POST https://upload.box.com/api/2.0/files/content HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: multipart/form-data; boundary="$BOUNDARY"
Host: upload.box.com
Content-Length: 2118
Accept-Encoding: gzip, deflate
--$BOUNDARY
Content-Disposition: form-data; name="file"; filename="$FILENAME"
<THE CONTENT OF $FILENAME>
--$BOUNDARY
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name="metadata"
{"parent":{"id":"$PARENT_FOLDER_ID"},"name":"$FILENAME"}
--$BOUNDARY--
Here is the response I received:
HTTP/1.1 201 Created
Date: Mon, 14 Apr 2014 12:52:33 GMT
Server: Apache
Cache-Control: no-cache, no-store
Content-Length: 364
Connection: close
Content-Type: application/json
{"total_count":1,"entries":[{"type":"file","id":"$ID","name":"$FILENAME", ... }]}
I may sound dumb, but what happens when you do this?
$body.Add("filename", $([IO.File]::ReadAllBytes($SourcePath)))
I am sending the following curl command from terminal in Mac OSX:
curl -d "OPERATION_NAME=ADD_REQUEST&TECHNICIAN_KEY=1AD….4&INPUT_DATA=TestData.xml" http://xxx.xx.xx.xx/sdpapi/request/
I am getting back the response:
FailedError when performing - ADD_REQUEST - Content is not allowed in prolog.
Here is my xml file:
<?xml version="1.0" encoding="UTF-8"?>
<Operation>
<Details>
<requester>Me</requester>
<subject>Test</subject>
<description>Testing curl input</description>
</Details>
</Operation>
I have checked and my xml file is indeed a UTF-8 file. I can tell from google searches that this most likely has to do with my encoding, however, I can't find how to fix it.
I've also tried saving the xml file as ANSI on my pc, xml file looks like this:
<?xml version="1.0"?>
<Operation>
<Details>
<requester>Me</requester>
<subject>Test</subject>
<description>Testing curl input</description>
</Details>
</Operation>
I downloaded Notepad++ and checked the encoding, which is UTF8 with no BOM.
I am still getting the same error. Can anyone see what I am doing wrong?
Updated 9/19 to add:
In addition to everything I've tried in the comments below, I've also tried this:
curl -d "OPERATION_NAME=ADD_REQUEST&TECHNICIAN_KEY=xxxxxxxxxxxxxxxxx&INPUT_DATA=<?xml version="1.0" encoding="utf-8"?><Operation><Details><requester>Me</requester><subject>Test</subject><description>Testing curl input</description></Details></Operation>" http://xxx.xx.xx.xx/sdpapi/request/
The error I'm getting now is: "Error when performing - ADD_REQUEST - The value following "version" in the XML declaration must be a quoted string."
Anyone have any thoughts?
I replaced <?xml version="1.0" encoding="utf-8"?> with <?xml version=%221.0%22 encoding=%22utf-8%22?>
It is now working.