Upload a file to Box.com using Powershell - rest

I am working on a series of Powershell scripts for my company to use to transfer data to and from Box.com. The one thing I just can't figure out is uploading files. The Box API requires a multipart POST operation for uploads, and I've seen a few answers here on SO indicating that I should be able to do that in Powershell (such as this one). But I can't seem to get it working.
Here's the code I have right now:
Function Post-File {
Param(
[Parameter(Mandatory=$True,Position=1)]
[string]$SourcePath,
[Parameter(Mandatory=$False,Position=2)]
[string]$FolderId = ############
)
#Variables for building URIs
$baseUrl = "https://upload.box.com/api/2.0/files/content"
#Set Authorization for API requests
$headers = #{}
$AccessToken = Refresh-Tokens #A reference to another function that definitely works
$headers.Add("Authorization", "Bearer $AccessToken")
#Set POST content
$body = #{}
$body.Add("filename", [IO.File]::ReadAllBytes($SourcePath))
$body.Add("parent_id", $FolderId)
#Upload the file
Invoke-RestMethod -Uri $baseUrl -Method Post -Headers $headers -ContentType "multipart/form-data" -Body $body
}
Here's the response I get back:
{
"type":"error",
"status":400,
"code":"invalid_request_parameters",
"help_url":"http://developers.box.com/docs/#errors",
"message":"Invalid input parameters in request",
"request_id":"1764475572534bcddfe04b7"
}
I've also tried a couple of other permutations that aren't working. I've tried using the -InFile switch in Invoke-RestMethod instead of -Body. I've also tried using Get-Content -Raw in place of [IO.File]::ReadAllBytes. Both of those return a more generic error: The server encountered an internal error or misconfiguration and was unable to complete your request..
I'm pretty sure this has something to do with my filename parameter not being correct, but I'm not sure how to fix it. According to the Box API, here's what it should look like in curl. Can someone help me properly translate this for Powershell?
https://upload.box.com/api/2.0/files/content \
-H "Authorization: Bearer ACCESS_TOKEN" \
-F filename=#FILE_NAME \
-F parent_id=PARENT_FOLDER_ID

There are a couple things that make this a little tricky in PowerShell:
The filename parameter specifies the name of the file, not the contents.
A request body is required in order to specify the file name and destination.
PowerShell treats -InFile and -Body arguments as mutually exclusive.
PowerShell does not appear to natively support multipart/form-data POSTs, per the question that you referenced.
Since the request body is required (1,2) -- and thus -InFile can't be used (3) -- I think you might need to roll your own multipart/form-data body (4) that contains the necessary metadata and file content. This blog post describes a method for doing so. The content there is a short string (a tweet) but the principle is the same.
Below is a Fiddler trace of an upload I just performed using the Box Windows SDK. This shows how the request should look as it goes across the wire. The $BOUNDARY is an arbitrary, unique string -- a GUID works great.
POST https://upload.box.com/api/2.0/files/content HTTP/1.1
Authorization: Bearer $TOKEN
Content-Type: multipart/form-data; boundary="$BOUNDARY"
Host: upload.box.com
Content-Length: 2118
Accept-Encoding: gzip, deflate
--$BOUNDARY
Content-Disposition: form-data; name="file"; filename="$FILENAME"
<THE CONTENT OF $FILENAME>
--$BOUNDARY
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name="metadata"
{"parent":{"id":"$PARENT_FOLDER_ID"},"name":"$FILENAME"}
--$BOUNDARY--
Here is the response I received:
HTTP/1.1 201 Created
Date: Mon, 14 Apr 2014 12:52:33 GMT
Server: Apache
Cache-Control: no-cache, no-store
Content-Length: 364
Connection: close
Content-Type: application/json
{"total_count":1,"entries":[{"type":"file","id":"$ID","name":"$FILENAME", ... }]}

I may sound dumb, but what happens when you do this?
$body.Add("filename", $([IO.File]::ReadAllBytes($SourcePath)))

Related

PowerAutomate use REST API turn on reporting site collection feature

We can turn this feature on via PowerShell PnP, but need to do it via PowerAutomate REST API call.
I have the following for our FLOW REST API call.
Site Address >> https://MYCO.sharepoint.com/sites/Sandbox
Method >> POST
URI >> /_api/web/features
Headers:
-accept : application/json;odata.metadata=none
-content-type : application/json
BODY:
{
"url": "https://MYCO.sharepoint.com/sites/Sandbox/_api/web/features/add('7094bd89-2cfe-490a-8c7e-fbace37b4a34')"
}
When I run the FLOW I get:
Error
Action 'Send_an_HTTP_request_to_SharePoint' failed
Invalid request.
clientRequestId: 690efeb4-ddce-49a2-b5d7-58e5e06db54a
serviceRequestId: 94ea649f-80ce-b000-973f-9686227e5f12
Try giving entire thing in URI and leave body blank
Site Address >> https://MYCO.sharepoint.com/sites/Sandbox
Method >> POST
URI >> /_api/web/features/add('7094bd89-2cfe-490a-8c7e-fbace37b4a34')
Headers:
-accept : application/json;odata.metadata=none
-content-type : application/json
BODY:

Powershell-command Invoke-webrequest login form hangs

Using Powershell 5.0.10586.117 I want to request a specific web-site (on IIS 8.5) containing a form to log-in. Here is the specific code:
$Url = Invoke-Webrequest -Uri "https://not-work/hello.aspx" -SessionVariable SesVar
$UrlForm = $Url.Forms[0]
When doing so Powershell hangs and never returns anything as long as I do not interrupt manually.
Using
$Url = Invoke-Webrequest -Verbose -Uri "https://not-work/hello.aspx" -SessionVariable SesVar
I can see the following error:
VERBOSE: Get https://not-work/hello.aspx with 0-byte payload.
Running the same website in Google Chrome 55.0.2883.87 it works. Using an older version of the application code on a different (likely older IIS) it works too in Powershell.
So I am wondering, if this issue has to do with this bug in Powershell?
Note: Even when using
$Url = Invoke-Webrequest -UseBasicParsing -Verbose -Uri "https://not-work/hello.aspx" -SessionVariable SesVar
it fails.
For the sake of completeness, curl also gives an HTTP status code 200 (like in Chrome), but there are some differences in comparison to the working version:
Non-working website
C:\Tools\curl>curl -k https://not-work/hello.aspx -I HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 12671
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/8.5 X-AspNet-Version: 4.0.30319
Set-Cookie: __AntiXsrfToken=d438e79fe7004ad09d950a3b8b4e507d; path=/; HttpOnly
Set-Cookie: .ASPXAUTH=; expires=Mon, 11-Oct-1999 22:00:00 GMT; path=/; HttpOnly
X-Powered-By: ASP.NET
Date: Thu, 12 Jan 2017 13:57:33 GMT
Working website
C:\Tools\curl>curl -k https://works-well/hello.aspx -I HTTP/1.1 200 OK
Cache-Control: private Content-Length: 12306 Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
Server: WWW Server/1.1 X-XSS-Protection: 1; mode=block
Set-Cookie: __AntiXsrfToken=6142de1c5d474da780530da39fea033f; path=/;HttpOnly; Secure
Date: Thu, 12 Jan 2017 15:56:47 GMT
By the way, in Internet Explorer (IE) 11.0.9600.18537 objects like images are not loaded (as I believe Powershell works "under the hood" using IE DLLs) and the "hourglass" is running permanently too next to the tab of the requested URI.
So again my question: Has this issue to do with this bug in Powershell or could it be something else?
I'm troubleshooting my own situation of the same nature. My website seems to have changed a few things around. One of which is the FORMS.
Can you confirm that your website has a FORMS option?
EX.
#GET WEBSESSION
$url = "https://myaccount.nytimes.com/"
$r = Invoke-WebRequest -Uri $url -SessionVariable ws -UseBasicParsing
return $r
Notice that I use -UseBasicParsing. For my nytimes example I have to. This is because nytimes is using JS and that doesn't seem to return well for PS. Removing -UseBasicParsing will result in my WebRequest hanging.
You will notice that with the above return the 'FORMS : ' is null.
Right now I do not have a solution. Other than, my website clearly requires credentials and I clearly need to provide. So I'm now writing a muttery of different attempts using -Body $bodyParam #{} and -Headers $headerParams #{}. So far it's to no avail.
Simply put, PS doesn't seem to behave well with JS type websites and FORMS seems to be used for only a specific set of website, not for all type sites.
Hopefully someone more skilled can assist cause this is racking my brain.

Fail to test the rendered output after a redirect from a DELETE route - Mojolicious

I currently expand my test suite to increase the test coverage. I want to test my controller and the html output that it renders, but I found a problem in using delete methods. Let me explain it in an example.
I have a route:
$r->delete('/backups/:id')
->to('backup#delete_backup')
->name('backup_delete');
that points to the following function in the backup controller:
sub delete_backup {
my $self = shift;
my $id = $self->param('id');
if ( something ) {
$self->flash( msg => "Backup id $id deleted!" );
}
else{
$self->flash( msg => "Cannot delete, backup id $id not found!" );
}
$self->redirect_to($self->url_for('backup_index'));
}
where the method that handles the route backup_index just displays the $msg and shows few other irrelevant data.
Next, I want to test this method, so I write a test:
$t_logged_in->ua->max_redirects(3);
my $page = $t_logged_in->app->url_for( 'backup_delete', id => $backup_id );
$t_logged_in->delete_ok($page)
->status_isnt( 404, "Checking: 404 $page" )
->status_isnt( 500, "Checking: 500 $page" );
The test is passed. But now, I want to check if the text is correct on the web page that is shown after redirecting. So I do the following:
$t_logged_in->ua->max_redirects(3);
my $page = $t_logged_in->app->url_for( 'backup_delete', id => $backup_id );
$t_logged_in->delete_ok($page)
->status_isnt( 404, "Checking: 404 $page" )
->status_isnt( 500, "Checking: 500 $page" )
->content_unlike(qr/Cannot delete,/i)
->content_like(qr/deleted/i);
The test fails. It fails because the content is empty, so the matching is done as there were:
'' =~ /deleted/i;
'' !~ /Cannot delete,/i;
and this is of course false in both cases. Of course, in the browser, the redirects work perfectly and I see everything as designed in the test. I can change the method to POST or GET but I wanted to make the routing properly in the way an API would be designed.
Question: how to design the test such that the content can be matched after the redirect?
For those who want to dig deeper, I give links to Github.
Routing definition line 303
Controller Backup function delete_backup line 135
Functional Test of Backup Controller line 53
Sorry no one answered this yet. I used to try to keep a closer eye on stack overflow but I got lazy :-P. This is a very interesting question.
Standard redirects (301/302) use the same verb as the original request UNLESS the original request was a POST Interestingly, this behavior was not really intended. A redirect is supposed to use the same request method as the original, but the POST->GET redirect is so commonly intended that most browsers added this behavior even though it was against the original definition. However some people really wanted a POST to remain a POST and now it depended on the implementation. In fact this got so bad that HTTP added response statuses 307/308 which always redirects as the same http verb (ie POST -> POST), even thought that was already the official behavior of the old ones but that ship had sailed. This means that POST->GET, but DELETE->DELETE is the defacto behavior of the 301/302.
This can be demonstrated in the following one liner ($_ in these one-liners is the controller):
perl -Mojo -E 'del "/delete" => sub { $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
Host: 127.0.0.1:62690
Accept-Encoding: gzip
User-Agent: Mojolicious (Perl)
Content-Length: 0
HTTP/1.1 302 Found
Date: Sun, 18 Sep 2016 03:25:58 GMT
Server: Mojolicious (Perl)
Location: /
Content-Length: 0
DELETE / HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:62690
Accept-Encoding: gzip
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Content-Length: 5
Date: Sun, 18 Sep 2016 03:25:58 GMT
Server: Mojolicious (Perl)
I got a DELETE
You can see that since my / route handles all methods we get a response, however the request was a DELETE. I see that in your app, you redirect to another named route whose url also handles DELETE. This is a source of your confusion as you are actually ending up there in your tests. Your redirect is effectively DELETE /backups/<<id>> -> DELETE /backups, if I'm reading your code correctly.
Now as opposed to the people who wanted to keep POST as POST, what you want here is the opposite, you want to redirect from DELETE to GET because you are trying to show a response that is not the original resource. This is the defined behavior of the little-known 303 response in which any request method is redirected to a GET. Indeed this is what early browsers should have insisted on rather than breaking the 302.
In the following example I modify the response code to 303 explicitly.
$ perl -Mojo -E 'del "/delete" => sub { $_->res->code(303); $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:62716
Accept-Encoding: gzip
Content-Length: 0
HTTP/1.1 303 See Other
Location: /
Content-Length: 0
Server: Mojolicious (Perl)
Date: Sun, 18 Sep 2016 03:27:19 GMT
DELETE / HTTP/1.1
User-Agent: Mojolicious (Perl)
Accept-Encoding: gzip
Content-Length: 0
Host: 127.0.0.1:62716
HTTP/1.1 200 OK
Content-Type: text/html;charset=UTF-8
Server: Mojolicious (Perl)
Content-Length: 5
Date: Sun, 18 Sep 2016 03:27:19 GMT
I got a DELETE
But whoops, that's still a DELETE! That's because there was a bug in Mojo::UserAgent so that it didn't handle 303s correctly. I'd suggest you still make the change to your code since browsers seem to handle 303s correctly. However since Mojo::UserAgent powers Test::Mojo, your tests can't test that behavior yet. Once fixed you will see it work correctly, as my example does in my local branch:
$ perl -Ilib -Mojo -E 'del "/delete" => sub { $_->res->code(303); $_->redirect_to("/") }; any "/" => { inline => q[I got a <%= $c->req->method %>] }; app->start' get -r -v -M DELETE /delete
DELETE /delete HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:64924
Accept-Encoding: gzip
HTTP/1.1 303 See Other
Date: Sun, 18 Sep 2016 04:59:37 GMT
Location: /
Server: Mojolicious (Perl)
Content-Length: 0
GET / HTTP/1.1
Content-Length: 0
User-Agent: Mojolicious (Perl)
Host: 127.0.0.1:64924
Accept-Encoding: gzip
HTTP/1.1 200 OK
Date: Sun, 18 Sep 2016 04:59:37 GMT
Content-Type: text/html;charset=UTF-8
Server: Mojolicious (Perl)
Content-Length: 12
I got a GET
To see more about redirect response types see https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection

Setting all headers when using Powershell Invoke-WebRequest

I would like to download PDF file from online form. I used the fiddler to find out the header and post fields. However, I cannot get the header from Firefox to match with Powershell using Invoke-webrequest.
Original request
POST http://somesight/SPLeam/EAMDisplayReportWithParamValues.asp HTTP/1.1
Host: someserver
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:27.0) Gecko/20100101 Firefox/27.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://someserver/SPLeam/EAMPickParamValues.asp?plant=01&reportuser=reports&reportuserpw=pw&ReportName=SPLEAM%5CMNT+%2D+MM+Tracking+Rpt%2Erpt
Cookie: ASPSESDFSDIDSSQAQBQC=APHPHMNCHIECSFSSPOIKBDCAM`
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 132
Text0=12%2F1%2F2013&Text1=12%2F31%2F2013&Text2=MESGI&Drop2=MESCI&Text3=01&Text4=&Drop4=&Text5=R&Text6=N&cmdSaveParameters=Run+Report
POWERSHELL
POST http://someserver/SPLeam/EAMPickParamValues.asp?plant=01&reportuser=reports&reportuserpw=pw&ReportName=SPLEAM%5CMNT+%2D+MM+Tracking+Rpt%2Erpt HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT; Windows NT 6.1; en-US) WindowsPowerShell/3.0
Content-Type: application/x-www-form-urlencoded
Host: someserver
Content-Length: 129
Connection: Keep-Alive
cmdSaveParameters=Run%2bReport&Text3=01&Text5=R&Text1=12%252F31%252F2013&Drop2=MI&Text2=MI&Text4=&Text6=N&Text0=12%252F1%252F2013
I run it as
$f = invoke-webrequest -uri $url -UserAgent $useragent -method Post -Body $param
How can I make Powershell's request match what Firefox sent?
.NET's HTTPWebRequest object has separate properties for the Accept and Referer headers; you must manually set these properties instead of trying to add the headers directly.
(These header restrictions exist partly for security purposes; partial trust code cannot set the properties).

Uploading zip file works in curl, but not in Powershell - why?

I'm switching from regular files to zip files doing an upload, and was told I'd need to use a header in this format - Content-Type: application/zip.
So, I can get my file to upload properly via curl with the following:
curl --verbose --header "Content-Type: application/zip" --data-binary #C:\Junk\test.zip "http://client.xyz.com/submit?username=test#test.com&password=testpassword&job=test"
However, when I write a simple powershell script to do the same thing, I run into problems - the data isn't loaded. I don't know how to get a good error message returned, so I don't know the details, but bottom line the data isn't getting in.
$FullFileName = "C:\Junk\test.zip"
$wc = new-object System.Net.WebClient -Verbose
$wc.Headers.Add("Content-Type: application/zip")
$URL = "http://client.xyz.com/submit?username=test#test.com&password=testpassword&job=test"
$wc.UploadFile( $URL, $FullFileName )
# $wc.UploadData( $URL, $FullFileName )
I've tried using UploadData instead of UploadFile, but that doesn't appear to work either.
Thanks,
Sylvia
I don't necessarily have a solution but I think the issue is that you are trying to upload a binary file using the WebClient object. You most likely will need the UploadData method but I think you are going to have to run the zip file into an array of bytes to upload. That I'm not sure of off the top of my head.
If you haven't, be sure to look at the MSDS docs for this class and methods: http://msdn.microsoft.com/en-us/library/system.net.webclient_methods.aspx
Now that I look at it again I think you need: $wc.Headers.Add("Content-Type", "application/zip") because the collection is key/value paired. Check out this SO question:
WebClient set headers
Also if your still having issues you might need to add a user agent header. I think curl has it's own.
$userAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2;)"
$wc.Headers.Add("user-agent", $userAgent)